Matches in SemOpenAlex for { <https://semopenalex.org/work/W3035163969> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W3035163969 abstract "Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually limited to transferring the knowledge only between the last layers of the networks, while latter approaches were capable of performing multi-layer KD, further increasing the accuracy of the student. However, despite their improved performance, these methods still suffer from several limitations that restrict both their efficiency and flexibility. First, existing KD methods typically ignore that neural networks undergo through different learning phases during the training process, which often requires different types of supervision for each one. Furthermore, existing multi-layer KD methods are usually unable to effectively handle networks with significantly different architectures (heterogeneous KD). In this paper we propose a novel KD method that works by modeling the information flow through the various layers of the teacher model and then train a student model to mimic this information flow. The proposed method is capable of overcoming the aforementioned limitations by using an appropriate supervision scheme during the different phases of the training process, as well as by designing and training an appropriate auxiliary teacher model that acts as a proxy model capable of “explaining” the way the teacher works to the student. The effectiveness of the proposed method is demonstrated using four image datasets and several different evaluation setups." @default.
- W3035163969 created "2020-06-19" @default.
- W3035163969 creator A5035280141 @default.
- W3035163969 creator A5041054091 @default.
- W3035163969 creator A5061050264 @default.
- W3035163969 date "2020-06-01" @default.
- W3035163969 modified "2023-10-12" @default.
- W3035163969 title "Heterogeneous Knowledge Distillation Using Information Flow Modeling" @default.
- W3035163969 cites W1602122992 @default.
- W3035163969 cites W2070148066 @default.
- W3035163969 cites W2138621090 @default.
- W3035163969 cites W2161969291 @default.
- W3035163969 cites W2214409633 @default.
- W3035163969 cites W2294370754 @default.
- W3035163969 cites W2587218622 @default.
- W3035163969 cites W2620998106 @default.
- W3035163969 cites W2739879705 @default.
- W3035163969 cites W2781486669 @default.
- W3035163969 cites W2883332448 @default.
- W3035163969 cites W2919115771 @default.
- W3035163969 cites W2936864631 @default.
- W3035163969 cites W2964005298 @default.
- W3035163969 cites W2969355370 @default.
- W3035163969 cites W2996320484 @default.
- W3035163969 doi "https://doi.org/10.1109/cvpr42600.2020.00241" @default.
- W3035163969 hasPublicationYear "2020" @default.
- W3035163969 type Work @default.
- W3035163969 sameAs 3035163969 @default.
- W3035163969 citedByCount "67" @default.
- W3035163969 countsByYear W30351639692019 @default.
- W3035163969 countsByYear W30351639692020 @default.
- W3035163969 countsByYear W30351639692021 @default.
- W3035163969 countsByYear W30351639692022 @default.
- W3035163969 countsByYear W30351639692023 @default.
- W3035163969 crossrefType "proceedings-article" @default.
- W3035163969 hasAuthorship W3035163969A5035280141 @default.
- W3035163969 hasAuthorship W3035163969A5041054091 @default.
- W3035163969 hasAuthorship W3035163969A5061050264 @default.
- W3035163969 hasBestOaLocation W30351639692 @default.
- W3035163969 hasConcept C105795698 @default.
- W3035163969 hasConcept C111919701 @default.
- W3035163969 hasConcept C119857082 @default.
- W3035163969 hasConcept C124101348 @default.
- W3035163969 hasConcept C134306372 @default.
- W3035163969 hasConcept C154945302 @default.
- W3035163969 hasConcept C178790620 @default.
- W3035163969 hasConcept C185592680 @default.
- W3035163969 hasConcept C204030448 @default.
- W3035163969 hasConcept C2779227376 @default.
- W3035163969 hasConcept C2780598303 @default.
- W3035163969 hasConcept C33923547 @default.
- W3035163969 hasConcept C41008148 @default.
- W3035163969 hasConcept C50644808 @default.
- W3035163969 hasConcept C77618280 @default.
- W3035163969 hasConcept C98045186 @default.
- W3035163969 hasConceptScore W3035163969C105795698 @default.
- W3035163969 hasConceptScore W3035163969C111919701 @default.
- W3035163969 hasConceptScore W3035163969C119857082 @default.
- W3035163969 hasConceptScore W3035163969C124101348 @default.
- W3035163969 hasConceptScore W3035163969C134306372 @default.
- W3035163969 hasConceptScore W3035163969C154945302 @default.
- W3035163969 hasConceptScore W3035163969C178790620 @default.
- W3035163969 hasConceptScore W3035163969C185592680 @default.
- W3035163969 hasConceptScore W3035163969C204030448 @default.
- W3035163969 hasConceptScore W3035163969C2779227376 @default.
- W3035163969 hasConceptScore W3035163969C2780598303 @default.
- W3035163969 hasConceptScore W3035163969C33923547 @default.
- W3035163969 hasConceptScore W3035163969C41008148 @default.
- W3035163969 hasConceptScore W3035163969C50644808 @default.
- W3035163969 hasConceptScore W3035163969C77618280 @default.
- W3035163969 hasConceptScore W3035163969C98045186 @default.
- W3035163969 hasLocation W30351639691 @default.
- W3035163969 hasLocation W30351639692 @default.
- W3035163969 hasOpenAccess W3035163969 @default.
- W3035163969 hasPrimaryLocation W30351639691 @default.
- W3035163969 hasRelatedWork W1131059 @default.
- W3035163969 hasRelatedWork W2151812 @default.
- W3035163969 hasRelatedWork W388674 @default.
- W3035163969 hasRelatedWork W4174498 @default.
- W3035163969 hasRelatedWork W4518066 @default.
- W3035163969 hasRelatedWork W5041667 @default.
- W3035163969 hasRelatedWork W5606278 @default.
- W3035163969 hasRelatedWork W614962 @default.
- W3035163969 hasRelatedWork W6521790 @default.
- W3035163969 hasRelatedWork W8762874 @default.
- W3035163969 isParatext "false" @default.
- W3035163969 isRetracted "false" @default.
- W3035163969 magId "3035163969" @default.
- W3035163969 workType "article" @default.