Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204279503> ?p ?o ?g. }
Showing items 1 to 90 of
90
with 100 items per page.
- W3204279503 abstract "Adversarial training is one effective approach for training robust deep neural networks against adversarial attacks. While being able to bring reliable robustness, adversarial training (AT) methods in general favor high capacity models, i.e., the larger the model the better the robustness. This tends to limit their effectiveness on small models, which are more preferable in scenarios where storage or computing resources are very limited (e.g., mobile devices). In this paper, we leverage the concept of knowledge distillation to improve the robustness of small models by distilling from adversarially trained large models. We first revisit several state-of-the-art AT methods from a distillation perspective and identify one common technique that can lead to improved robustness: the use of robust soft labels – predictions of a robust model. Following this observation, we propose a novel adversarial robustness distillation method called Robust Soft Label Adversarial Distillation (RSLAD) to train robust small student models. RSLAD fully exploits the robust soft labels produced by a robust (adversarially-trained) large teacher model to guide the student’s learning on both natural and adversarial examples in all loss terms. We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack. We also provide a set of understandings on our RSLAD and the importance of robust soft labels for adversarial robustness distillation. Code: https://github.com/zibojia/RSLAD." @default.
- W3204279503 created "2021-10-11" @default.
- W3204279503 creator A5003572809 @default.
- W3204279503 creator A5046787369 @default.
- W3204279503 creator A5047962986 @default.
- W3204279503 creator A5078711649 @default.
- W3204279503 date "2021-10-01" @default.
- W3204279503 modified "2023-10-17" @default.
- W3204279503 title "Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better" @default.
- W3204279503 cites W2183341477 @default.
- W3204279503 cites W2194775991 @default.
- W3204279503 cites W2759378924 @default.
- W3204279503 cites W2798302089 @default.
- W3204279503 cites W2904170036 @default.
- W3204279503 cites W2962858109 @default.
- W3204279503 cites W2962872506 @default.
- W3204279503 cites W2963163009 @default.
- W3204279503 cites W2963771536 @default.
- W3204279503 cites W2963857521 @default.
- W3204279503 cites W2964137095 @default.
- W3204279503 cites W2987861506 @default.
- W3204279503 cites W2998239226 @default.
- W3204279503 cites W3021182036 @default.
- W3204279503 cites W3034457371 @default.
- W3204279503 cites W3035160371 @default.
- W3204279503 cites W3035447895 @default.
- W3204279503 cites W3035467354 @default.
- W3204279503 cites W3099502074 @default.
- W3204279503 cites W3116564952 @default.
- W3204279503 doi "https://doi.org/10.1109/iccv48922.2021.01613" @default.
- W3204279503 hasPublicationYear "2021" @default.
- W3204279503 type Work @default.
- W3204279503 sameAs 3204279503 @default.
- W3204279503 citedByCount "13" @default.
- W3204279503 countsByYear W32042795032021 @default.
- W3204279503 countsByYear W32042795032022 @default.
- W3204279503 countsByYear W32042795032023 @default.
- W3204279503 crossrefType "proceedings-article" @default.
- W3204279503 hasAuthorship W3204279503A5003572809 @default.
- W3204279503 hasAuthorship W3204279503A5046787369 @default.
- W3204279503 hasAuthorship W3204279503A5047962986 @default.
- W3204279503 hasAuthorship W3204279503A5078711649 @default.
- W3204279503 hasBestOaLocation W32042795032 @default.
- W3204279503 hasConcept C104317684 @default.
- W3204279503 hasConcept C119857082 @default.
- W3204279503 hasConcept C153083717 @default.
- W3204279503 hasConcept C154945302 @default.
- W3204279503 hasConcept C165696696 @default.
- W3204279503 hasConcept C178790620 @default.
- W3204279503 hasConcept C185592680 @default.
- W3204279503 hasConcept C204030448 @default.
- W3204279503 hasConcept C37736160 @default.
- W3204279503 hasConcept C38652104 @default.
- W3204279503 hasConcept C41008148 @default.
- W3204279503 hasConcept C51632099 @default.
- W3204279503 hasConcept C55493867 @default.
- W3204279503 hasConcept C63479239 @default.
- W3204279503 hasConceptScore W3204279503C104317684 @default.
- W3204279503 hasConceptScore W3204279503C119857082 @default.
- W3204279503 hasConceptScore W3204279503C153083717 @default.
- W3204279503 hasConceptScore W3204279503C154945302 @default.
- W3204279503 hasConceptScore W3204279503C165696696 @default.
- W3204279503 hasConceptScore W3204279503C178790620 @default.
- W3204279503 hasConceptScore W3204279503C185592680 @default.
- W3204279503 hasConceptScore W3204279503C204030448 @default.
- W3204279503 hasConceptScore W3204279503C37736160 @default.
- W3204279503 hasConceptScore W3204279503C38652104 @default.
- W3204279503 hasConceptScore W3204279503C41008148 @default.
- W3204279503 hasConceptScore W3204279503C51632099 @default.
- W3204279503 hasConceptScore W3204279503C55493867 @default.
- W3204279503 hasConceptScore W3204279503C63479239 @default.
- W3204279503 hasFunder F4320321001 @default.
- W3204279503 hasLocation W32042795031 @default.
- W3204279503 hasLocation W32042795032 @default.
- W3204279503 hasOpenAccess W3204279503 @default.
- W3204279503 hasPrimaryLocation W32042795031 @default.
- W3204279503 hasRelatedWork W3046843850 @default.
- W3204279503 hasRelatedWork W3124408655 @default.
- W3204279503 hasRelatedWork W3193660423 @default.
- W3204279503 hasRelatedWork W3204279503 @default.
- W3204279503 hasRelatedWork W4221152960 @default.
- W3204279503 hasRelatedWork W4287023716 @default.
- W3204279503 hasRelatedWork W4297785512 @default.
- W3204279503 hasRelatedWork W4311734044 @default.
- W3204279503 hasRelatedWork W4312641744 @default.
- W3204279503 hasRelatedWork W4378942430 @default.
- W3204279503 isParatext "false" @default.
- W3204279503 isRetracted "false" @default.
- W3204279503 magId "3204279503" @default.
- W3204279503 workType "article" @default.