Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378473762> ?p ?o ?g. }
Showing items 1 to 57 of
57
with 100 items per page.
- W4378473762 abstract "emph{Consistent teaching} is an effective paradigm for implementing knowledge distillation (KD), where both student and teacher models receive identical inputs, and KD is treated as a function matching task (FunMatch). However, one limitation of FunMatch is that it does not account for the transfer of adversarial robustness, a model's resistance to adversarial attacks. To tackle this problem, we propose a simple but effective strategy called Adversarial Function Matching (AdvFunMatch), which aims to match distributions for all data points within the $ell_p$-norm ball of the training data, in accordance with consistent teaching. Formulated as a min-max optimization problem, AdvFunMatch identifies the worst-case instances that maximizes the KL-divergence between teacher and student model outputs, which we refer to as mismatched examples, and then matches the outputs on these mismatched examples. Our experimental results show that AdvFunMatch effectively produces student models with both high clean accuracy and robustness. Furthermore, we reveal that strong data augmentations (emph{e.g.}, AutoAugment) are beneficial in AdvFunMatch, whereas prior works have found them less effective in adversarial training. Code is available at url{https://gitee.com/zihui998/adv-fun-match}." @default.
- W4378473762 created "2023-05-27" @default.
- W4378473762 creator A5043745622 @default.
- W4378473762 creator A5086029723 @default.
- W4378473762 creator A5087354951 @default.
- W4378473762 creator A5087486355 @default.
- W4378473762 date "2023-05-24" @default.
- W4378473762 modified "2023-09-25" @default.
- W4378473762 title "AdvFunMatch: When Consistent Teaching Meets Adversarial Robustness" @default.
- W4378473762 doi "https://doi.org/10.48550/arxiv.2305.14700" @default.
- W4378473762 hasPublicationYear "2023" @default.
- W4378473762 type Work @default.
- W4378473762 citedByCount "0" @default.
- W4378473762 crossrefType "posted-content" @default.
- W4378473762 hasAuthorship W4378473762A5043745622 @default.
- W4378473762 hasAuthorship W4378473762A5086029723 @default.
- W4378473762 hasAuthorship W4378473762A5087354951 @default.
- W4378473762 hasAuthorship W4378473762A5087486355 @default.
- W4378473762 hasBestOaLocation W43784737621 @default.
- W4378473762 hasConcept C104317684 @default.
- W4378473762 hasConcept C11413529 @default.
- W4378473762 hasConcept C119857082 @default.
- W4378473762 hasConcept C126255220 @default.
- W4378473762 hasConcept C154945302 @default.
- W4378473762 hasConcept C185592680 @default.
- W4378473762 hasConcept C33923547 @default.
- W4378473762 hasConcept C37736160 @default.
- W4378473762 hasConcept C41008148 @default.
- W4378473762 hasConcept C55493867 @default.
- W4378473762 hasConcept C63479239 @default.
- W4378473762 hasConceptScore W4378473762C104317684 @default.
- W4378473762 hasConceptScore W4378473762C11413529 @default.
- W4378473762 hasConceptScore W4378473762C119857082 @default.
- W4378473762 hasConceptScore W4378473762C126255220 @default.
- W4378473762 hasConceptScore W4378473762C154945302 @default.
- W4378473762 hasConceptScore W4378473762C185592680 @default.
- W4378473762 hasConceptScore W4378473762C33923547 @default.
- W4378473762 hasConceptScore W4378473762C37736160 @default.
- W4378473762 hasConceptScore W4378473762C41008148 @default.
- W4378473762 hasConceptScore W4378473762C55493867 @default.
- W4378473762 hasConceptScore W4378473762C63479239 @default.
- W4378473762 hasLocation W43784737621 @default.
- W4378473762 hasOpenAccess W4378473762 @default.
- W4378473762 hasPrimaryLocation W43784737621 @default.
- W4378473762 hasRelatedWork W2952919291 @default.
- W4378473762 hasRelatedWork W3035729345 @default.
- W4378473762 hasRelatedWork W3114774860 @default.
- W4378473762 hasRelatedWork W3193857078 @default.
- W4378473762 hasRelatedWork W3208304128 @default.
- W4378473762 hasRelatedWork W4225307033 @default.
- W4378473762 hasRelatedWork W4287549955 @default.
- W4378473762 hasRelatedWork W4306177400 @default.
- W4378473762 hasRelatedWork W4307654699 @default.
- W4378473762 hasRelatedWork W4311734044 @default.
- W4378473762 isParatext "false" @default.
- W4378473762 isRetracted "false" @default.
- W4378473762 workType "article" @default.