Matches in SemOpenAlex for { <https://semopenalex.org/work/W4319301163> ?p ?o ?g. }
Showing items 1 to 79 of
79
with 100 items per page.
- W4319301163 abstract "Knowledge distillation which learns a lightweight student model by distilling knowledge from a cumbersome teacher model is an attractive approach for learning compact deep neural networks (DNNs). Recent works further improve student network performance by leveraging multiple teacher networks. However, most of the existing knowledge distillation-based multi-teacher methods use separately pretrained teachers. This limits the collaborative learning between teachers and the mutual learning between teachers and student. Network quantization is another attractive approach for learning compact DNNs. However, most existing network quantization methods are developed and evaluated without considering multi-teacher support to enhance the performance of quantized student model. In this paper, we propose a novel framework that leverages both multi-teacher knowledge distillation and network quantization for learning low bit-width DNNs. The proposed method encourages both collaborative learning between quantized teachers and mutual learning between quantized teachers and quantized student. During learning process, at corresponding layers, knowledge from teachers will form an importance-aware shared knowledge which will be used as input for teachers at subsequent layers and also be used to guide student. Our experimental results on CIFAR-100 and ImageNet datasets show that the compact quantized student models trained with our method achieve competitive results compared to other state-of-the-art methods, and in some cases, surpass the full precision models." @default.
- W4319301163 created "2023-02-07" @default.
- W4319301163 creator A5025723803 @default.
- W4319301163 creator A5048219836 @default.
- W4319301163 creator A5080253938 @default.
- W4319301163 date "2023-01-01" @default.
- W4319301163 modified "2023-09-30" @default.
- W4319301163 title "Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks" @default.
- W4319301163 cites W2117539524 @default.
- W4319301163 cites W2194775991 @default.
- W4319301163 cites W2586654419 @default.
- W4319301163 cites W2604700561 @default.
- W4319301163 cites W2620998106 @default.
- W4319301163 cites W2743289088 @default.
- W4319301163 cites W2940288022 @default.
- W4319301163 cites W2963480671 @default.
- W4319301163 cites W2963521187 @default.
- W4319301163 cites W2982157312 @default.
- W4319301163 cites W3034719990 @default.
- W4319301163 cites W3034887213 @default.
- W4319301163 cites W3035282660 @default.
- W4319301163 cites W3035321581 @default.
- W4319301163 cites W3045672834 @default.
- W4319301163 cites W3128299679 @default.
- W4319301163 doi "https://doi.org/10.1109/wacv56688.2023.00637" @default.
- W4319301163 hasPublicationYear "2023" @default.
- W4319301163 type Work @default.
- W4319301163 citedByCount "1" @default.
- W4319301163 countsByYear W43193011632023 @default.
- W4319301163 crossrefType "proceedings-article" @default.
- W4319301163 hasAuthorship W4319301163A5025723803 @default.
- W4319301163 hasAuthorship W4319301163A5048219836 @default.
- W4319301163 hasAuthorship W4319301163A5080253938 @default.
- W4319301163 hasBestOaLocation W43193011632 @default.
- W4319301163 hasConcept C111919701 @default.
- W4319301163 hasConcept C11413529 @default.
- W4319301163 hasConcept C119857082 @default.
- W4319301163 hasConcept C138020889 @default.
- W4319301163 hasConcept C154945302 @default.
- W4319301163 hasConcept C178790620 @default.
- W4319301163 hasConcept C185592680 @default.
- W4319301163 hasConcept C204030448 @default.
- W4319301163 hasConcept C28855332 @default.
- W4319301163 hasConcept C40567965 @default.
- W4319301163 hasConcept C41008148 @default.
- W4319301163 hasConcept C50644808 @default.
- W4319301163 hasConcept C56739046 @default.
- W4319301163 hasConcept C98045186 @default.
- W4319301163 hasConceptScore W4319301163C111919701 @default.
- W4319301163 hasConceptScore W4319301163C11413529 @default.
- W4319301163 hasConceptScore W4319301163C119857082 @default.
- W4319301163 hasConceptScore W4319301163C138020889 @default.
- W4319301163 hasConceptScore W4319301163C154945302 @default.
- W4319301163 hasConceptScore W4319301163C178790620 @default.
- W4319301163 hasConceptScore W4319301163C185592680 @default.
- W4319301163 hasConceptScore W4319301163C204030448 @default.
- W4319301163 hasConceptScore W4319301163C28855332 @default.
- W4319301163 hasConceptScore W4319301163C40567965 @default.
- W4319301163 hasConceptScore W4319301163C41008148 @default.
- W4319301163 hasConceptScore W4319301163C50644808 @default.
- W4319301163 hasConceptScore W4319301163C56739046 @default.
- W4319301163 hasConceptScore W4319301163C98045186 @default.
- W4319301163 hasLocation W43193011631 @default.
- W4319301163 hasLocation W43193011632 @default.
- W4319301163 hasOpenAccess W4319301163 @default.
- W4319301163 hasPrimaryLocation W43193011631 @default.
- W4319301163 hasRelatedWork W2025883075 @default.
- W4319301163 hasRelatedWork W2026249744 @default.
- W4319301163 hasRelatedWork W2692154213 @default.
- W4319301163 hasRelatedWork W2743872637 @default.
- W4319301163 hasRelatedWork W2893268702 @default.
- W4319301163 hasRelatedWork W2961085424 @default.
- W4319301163 hasRelatedWork W2996077742 @default.
- W4319301163 hasRelatedWork W4306674287 @default.
- W4319301163 hasRelatedWork W4319301163 @default.
- W4319301163 hasRelatedWork W4383960674 @default.
- W4319301163 isParatext "false" @default.
- W4319301163 isRetracted "false" @default.
- W4319301163 workType "article" @default.