Matches in SemOpenAlex for { <https://semopenalex.org/work/W4307652408> ?p ?o ?g. }
Showing items 1 to 62 of
62
with 100 items per page.
- W4307652408 abstract "Data-free knowledge distillation (DFKD) conducts knowledge distillation via eliminating the dependence of original training data, and has recently achieved impressive results in accelerating pre-trained language models. At the heart of DFKD is to reconstruct a synthetic dataset by inverting the parameters of the uncompressed model. Prior DFKD approaches, however, have largely relied on hand-crafted priors of the target data distribution for the reconstruction, which can be inevitably biased and often incompetent to capture the intrinsic distributions. To address this problem, we propose a prompt-based method, termed as PromptDFD, that allows us to take advantage of learned language priors, which effectively harmonizes the synthetic sentences to be semantically and grammatically correct. Specifically, PromptDFD leverages a pre-trained generative model to provide language priors and introduces a reinforced topic prompter to control data synthesis, making the generated samples thematically relevant and semantically plausible, and thus friendly to downstream tasks. As shown in our experiments, the proposed method substantially improves the synthesis quality and achieves considerable improvements on distillation performance. In some cases, PromptDFD even gives rise to results on par with those from the data-driven knowledge distillation with access to the original training data." @default.
- W4307652408 created "2022-11-04" @default.
- W4307652408 creator A5004615610 @default.
- W4307652408 creator A5015574447 @default.
- W4307652408 creator A5026310569 @default.
- W4307652408 creator A5041805674 @default.
- W4307652408 creator A5055740842 @default.
- W4307652408 date "2022-05-16" @default.
- W4307652408 modified "2023-09-26" @default.
- W4307652408 title "Prompting to Distill: Boosting Data-Free Knowledge Distillation via Reinforced Prompt" @default.
- W4307652408 doi "https://doi.org/10.48550/arxiv.2205.07523" @default.
- W4307652408 hasPublicationYear "2022" @default.
- W4307652408 type Work @default.
- W4307652408 citedByCount "0" @default.
- W4307652408 crossrefType "posted-content" @default.
- W4307652408 hasAuthorship W4307652408A5004615610 @default.
- W4307652408 hasAuthorship W4307652408A5015574447 @default.
- W4307652408 hasAuthorship W4307652408A5026310569 @default.
- W4307652408 hasAuthorship W4307652408A5041805674 @default.
- W4307652408 hasAuthorship W4307652408A5055740842 @default.
- W4307652408 hasBestOaLocation W43076524081 @default.
- W4307652408 hasConcept C107673813 @default.
- W4307652408 hasConcept C119857082 @default.
- W4307652408 hasConcept C137293760 @default.
- W4307652408 hasConcept C154945302 @default.
- W4307652408 hasConcept C160920958 @default.
- W4307652408 hasConcept C177769412 @default.
- W4307652408 hasConcept C178790620 @default.
- W4307652408 hasConcept C185592680 @default.
- W4307652408 hasConcept C204030448 @default.
- W4307652408 hasConcept C2780440489 @default.
- W4307652408 hasConcept C41008148 @default.
- W4307652408 hasConcept C46686674 @default.
- W4307652408 hasConceptScore W4307652408C107673813 @default.
- W4307652408 hasConceptScore W4307652408C119857082 @default.
- W4307652408 hasConceptScore W4307652408C137293760 @default.
- W4307652408 hasConceptScore W4307652408C154945302 @default.
- W4307652408 hasConceptScore W4307652408C160920958 @default.
- W4307652408 hasConceptScore W4307652408C177769412 @default.
- W4307652408 hasConceptScore W4307652408C178790620 @default.
- W4307652408 hasConceptScore W4307652408C185592680 @default.
- W4307652408 hasConceptScore W4307652408C204030448 @default.
- W4307652408 hasConceptScore W4307652408C2780440489 @default.
- W4307652408 hasConceptScore W4307652408C41008148 @default.
- W4307652408 hasConceptScore W4307652408C46686674 @default.
- W4307652408 hasLocation W43076524081 @default.
- W4307652408 hasLocation W43076524082 @default.
- W4307652408 hasOpenAccess W4307652408 @default.
- W4307652408 hasPrimaryLocation W43076524081 @default.
- W4307652408 hasRelatedWork W1987859285 @default.
- W4307652408 hasRelatedWork W1996541855 @default.
- W4307652408 hasRelatedWork W3151529617 @default.
- W4307652408 hasRelatedWork W3159988495 @default.
- W4307652408 hasRelatedWork W3195168932 @default.
- W4307652408 hasRelatedWork W3200719183 @default.
- W4307652408 hasRelatedWork W4280581589 @default.
- W4307652408 hasRelatedWork W4292969247 @default.
- W4307652408 hasRelatedWork W4307652408 @default.
- W4307652408 hasRelatedWork W4313488044 @default.
- W4307652408 isParatext "false" @default.
- W4307652408 isRetracted "false" @default.
- W4307652408 workType "article" @default.