Matches in SemOpenAlex for { <https://semopenalex.org/work/W4318719005> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4318719005 abstract "Contrary to its original interpretation as a facilitator of knowledge transfer from one model to another, some recent studies have suggested that knowledge distillation (KD) is instead a form of regularization. Perhaps the strongest support of all for this claim is found in its apparent similarities with label smoothing (LS). This paper investigates the stated equivalence of these two methods by examining the predictive uncertainties of the models they train. Experiments on four text classification tasks involving teachers and students of different capacities show that: (a) In most settings, KD and LS drive model uncertainty (entropy) in completely opposite directions, and (b) In KD, the student's predictive uncertainty is a direct function of that of its teacher, reinforcing the knowledge transfer view." @default.
- W4318719005 created "2023-02-01" @default.
- W4318719005 creator A5055236060 @default.
- W4318719005 date "2023-01-29" @default.
- W4318719005 modified "2023-09-27" @default.
- W4318719005 title "Knowledge Distillation $approx$ Label Smoothing: Fact or Fallacy?" @default.
- W4318719005 doi "https://doi.org/10.48550/arxiv.2301.12609" @default.
- W4318719005 hasPublicationYear "2023" @default.
- W4318719005 type Work @default.
- W4318719005 citedByCount "0" @default.
- W4318719005 crossrefType "posted-content" @default.
- W4318719005 hasAuthorship W4318719005A5055236060 @default.
- W4318719005 hasBestOaLocation W43187190051 @default.
- W4318719005 hasConcept C105795698 @default.
- W4318719005 hasConcept C106301342 @default.
- W4318719005 hasConcept C111919701 @default.
- W4318719005 hasConcept C118615104 @default.
- W4318719005 hasConcept C121332964 @default.
- W4318719005 hasConcept C144237770 @default.
- W4318719005 hasConcept C149782125 @default.
- W4318719005 hasConcept C154945302 @default.
- W4318719005 hasConcept C15744967 @default.
- W4318719005 hasConcept C2776135515 @default.
- W4318719005 hasConcept C2776831358 @default.
- W4318719005 hasConcept C2777894999 @default.
- W4318719005 hasConcept C2780069185 @default.
- W4318719005 hasConcept C33923547 @default.
- W4318719005 hasConcept C3770464 @default.
- W4318719005 hasConcept C41008148 @default.
- W4318719005 hasConcept C77805123 @default.
- W4318719005 hasConcept C97355855 @default.
- W4318719005 hasConceptScore W4318719005C105795698 @default.
- W4318719005 hasConceptScore W4318719005C106301342 @default.
- W4318719005 hasConceptScore W4318719005C111919701 @default.
- W4318719005 hasConceptScore W4318719005C118615104 @default.
- W4318719005 hasConceptScore W4318719005C121332964 @default.
- W4318719005 hasConceptScore W4318719005C144237770 @default.
- W4318719005 hasConceptScore W4318719005C149782125 @default.
- W4318719005 hasConceptScore W4318719005C154945302 @default.
- W4318719005 hasConceptScore W4318719005C15744967 @default.
- W4318719005 hasConceptScore W4318719005C2776135515 @default.
- W4318719005 hasConceptScore W4318719005C2776831358 @default.
- W4318719005 hasConceptScore W4318719005C2777894999 @default.
- W4318719005 hasConceptScore W4318719005C2780069185 @default.
- W4318719005 hasConceptScore W4318719005C33923547 @default.
- W4318719005 hasConceptScore W4318719005C3770464 @default.
- W4318719005 hasConceptScore W4318719005C41008148 @default.
- W4318719005 hasConceptScore W4318719005C77805123 @default.
- W4318719005 hasConceptScore W4318719005C97355855 @default.
- W4318719005 hasLocation W43187190051 @default.
- W4318719005 hasOpenAccess W4318719005 @default.
- W4318719005 hasPrimaryLocation W43187190051 @default.
- W4318719005 hasRelatedWork W2051752773 @default.
- W4318719005 hasRelatedWork W2144186360 @default.
- W4318719005 hasRelatedWork W2382706727 @default.
- W4318719005 hasRelatedWork W2393075133 @default.
- W4318719005 hasRelatedWork W2617784773 @default.
- W4318719005 hasRelatedWork W2748952813 @default.
- W4318719005 hasRelatedWork W2899084033 @default.
- W4318719005 hasRelatedWork W3135029621 @default.
- W4318719005 hasRelatedWork W62305045 @default.
- W4318719005 hasRelatedWork W1566585206 @default.
- W4318719005 isParatext "false" @default.
- W4318719005 isRetracted "false" @default.
- W4318719005 workType "article" @default.