Matches in SemOpenAlex for { <https://semopenalex.org/work/W4317207175> ?p ?o ?g. }
Showing items 1 to 96 of
96
with 100 items per page.
- W4317207175 endingPage "109338" @default.
- W4317207175 startingPage "109338" @default.
- W4317207175 abstract "Knowledge distillation (KD) has emerged as an essential technique not only for model compression, but also other learning tasks such as continual learning. Given the richer application spectrum and potential online usage of KD, knowledge distillation efficiency becomes a pivotal component. In this work, we study this little-explored but important topic. Unlike previous works that focus solely on the accuracy of student network, we attempt to achieve a harder goal – to obtain a performance comparable to conventional KD with a lower computation cost during the transfer. To this end, we present UNcertainty-aware mIXup (UNIX), an effective approach that can reduce transfer cost by 20% to 30% and yet maintain comparable or achieve even better student performance than conventional KD. This is made possible via effective uncertainty sampling and a novel adaptive mixup approach that select informative samples dynamically over ample data and compact knowledge in these samples. We show that our approach inherently performs hard sample mining. We demonstrate the applicability of our approach to improve various existing KD approaches by reducing their queries to a teacher network. Extensive experiments are performed on CIFAR100 and ImageNet. Code and model are available at https://github.com/xuguodong03/UNIXKD." @default.
- W4317207175 created "2023-01-18" @default.
- W4317207175 creator A5005626854 @default.
- W4317207175 creator A5009437145 @default.
- W4317207175 creator A5055183049 @default.
- W4317207175 date "2023-06-01" @default.
- W4317207175 modified "2023-10-18" @default.
- W4317207175 title "Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup" @default.
- W4317207175 cites W2008989859 @default.
- W4317207175 cites W2963975998 @default.
- W4317207175 cites W2990952853 @default.
- W4317207175 cites W3084361264 @default.
- W4317207175 cites W3094247396 @default.
- W4317207175 cites W3199621691 @default.
- W4317207175 cites W4205474609 @default.
- W4317207175 cites W4225131199 @default.
- W4317207175 cites W4281710448 @default.
- W4317207175 doi "https://doi.org/10.1016/j.patcog.2023.109338" @default.
- W4317207175 hasPublicationYear "2023" @default.
- W4317207175 type Work @default.
- W4317207175 citedByCount "3" @default.
- W4317207175 countsByYear W43172071752023 @default.
- W4317207175 crossrefType "journal-article" @default.
- W4317207175 hasAuthorship W4317207175A5005626854 @default.
- W4317207175 hasAuthorship W4317207175A5009437145 @default.
- W4317207175 hasAuthorship W4317207175A5055183049 @default.
- W4317207175 hasBestOaLocation W43172071752 @default.
- W4317207175 hasConcept C106131492 @default.
- W4317207175 hasConcept C111919701 @default.
- W4317207175 hasConcept C112968700 @default.
- W4317207175 hasConcept C11413529 @default.
- W4317207175 hasConcept C119857082 @default.
- W4317207175 hasConcept C120665830 @default.
- W4317207175 hasConcept C121332964 @default.
- W4317207175 hasConcept C124101348 @default.
- W4317207175 hasConcept C140779682 @default.
- W4317207175 hasConcept C154945302 @default.
- W4317207175 hasConcept C168167062 @default.
- W4317207175 hasConcept C177264268 @default.
- W4317207175 hasConcept C178790620 @default.
- W4317207175 hasConcept C185592680 @default.
- W4317207175 hasConcept C192209626 @default.
- W4317207175 hasConcept C199360897 @default.
- W4317207175 hasConcept C204030448 @default.
- W4317207175 hasConcept C2776760102 @default.
- W4317207175 hasConcept C2777904410 @default.
- W4317207175 hasConcept C31972630 @default.
- W4317207175 hasConcept C41008148 @default.
- W4317207175 hasConcept C43126263 @default.
- W4317207175 hasConcept C45374587 @default.
- W4317207175 hasConcept C97355855 @default.
- W4317207175 hasConceptScore W4317207175C106131492 @default.
- W4317207175 hasConceptScore W4317207175C111919701 @default.
- W4317207175 hasConceptScore W4317207175C112968700 @default.
- W4317207175 hasConceptScore W4317207175C11413529 @default.
- W4317207175 hasConceptScore W4317207175C119857082 @default.
- W4317207175 hasConceptScore W4317207175C120665830 @default.
- W4317207175 hasConceptScore W4317207175C121332964 @default.
- W4317207175 hasConceptScore W4317207175C124101348 @default.
- W4317207175 hasConceptScore W4317207175C140779682 @default.
- W4317207175 hasConceptScore W4317207175C154945302 @default.
- W4317207175 hasConceptScore W4317207175C168167062 @default.
- W4317207175 hasConceptScore W4317207175C177264268 @default.
- W4317207175 hasConceptScore W4317207175C178790620 @default.
- W4317207175 hasConceptScore W4317207175C185592680 @default.
- W4317207175 hasConceptScore W4317207175C192209626 @default.
- W4317207175 hasConceptScore W4317207175C199360897 @default.
- W4317207175 hasConceptScore W4317207175C204030448 @default.
- W4317207175 hasConceptScore W4317207175C2776760102 @default.
- W4317207175 hasConceptScore W4317207175C2777904410 @default.
- W4317207175 hasConceptScore W4317207175C31972630 @default.
- W4317207175 hasConceptScore W4317207175C41008148 @default.
- W4317207175 hasConceptScore W4317207175C43126263 @default.
- W4317207175 hasConceptScore W4317207175C45374587 @default.
- W4317207175 hasConceptScore W4317207175C97355855 @default.
- W4317207175 hasFunder F4320320766 @default.
- W4317207175 hasLocation W43172071751 @default.
- W4317207175 hasLocation W43172071752 @default.
- W4317207175 hasOpenAccess W4317207175 @default.
- W4317207175 hasPrimaryLocation W43172071751 @default.
- W4317207175 hasRelatedWork W1601353299 @default.
- W4317207175 hasRelatedWork W1965942136 @default.
- W4317207175 hasRelatedWork W2025583494 @default.
- W4317207175 hasRelatedWork W2321656188 @default.
- W4317207175 hasRelatedWork W2379533788 @default.
- W4317207175 hasRelatedWork W2961085424 @default.
- W4317207175 hasRelatedWork W4233194547 @default.
- W4317207175 hasRelatedWork W4246329793 @default.
- W4317207175 hasRelatedWork W4306674287 @default.
- W4317207175 hasRelatedWork W4224009465 @default.
- W4317207175 hasVolume "138" @default.
- W4317207175 isParatext "false" @default.
- W4317207175 isRetracted "false" @default.
- W4317207175 workType "article" @default.