Matches in SemOpenAlex for { <https://semopenalex.org/work/W3108989436> ?p ?o ?g. }
Showing items 1 to 91 of
91
with 100 items per page.
- W3108989436 abstract "Knowledge distillation has become an important technique for model compression and acceleration. The conventional knowledge distillation approaches aim to transfer knowledge from teacher to student networks by minimizing the KL-divergence between their probabilistic outputs, which only consider the mutual relationship between individual representations of teacher and student networks. Recently, the contrastive loss-based knowledge distillation is proposed to enable a student to learn the instance discriminative knowledge of a teacher by mapping the same image close and different images far away in the representation space. However, all of these methods ignore that the teacher's knowledge is multi-level, e.g., individual, relational and categorical level. These different levels of knowledge cannot be effectively captured by only one kind of supervisory signal. Here, we introduce Multi-level Knowledge Distillation (MLKD) to transfer richer representational knowledge from teacher to student networks. MLKD employs three novel teacher-student similarities: individual similarity, relational similarity, and categorical similarity, to encourage the student network to learn sample-wise, structure-wise and category-wise knowledge in the teacher network. Experiments demonstrate that MLKD outperforms other state-of-the-art methods on both similar-architecture and cross-architecture tasks. We further show that MLKD can improve the transferability of learned representations in the student network." @default.
- W3108989436 created "2020-12-07" @default.
- W3108989436 creator A5040998948 @default.
- W3108989436 creator A5056657952 @default.
- W3108989436 creator A5067760326 @default.
- W3108989436 creator A5089100714 @default.
- W3108989436 date "2020-12-01" @default.
- W3108989436 modified "2023-09-27" @default.
- W3108989436 title "Multi-level Knowledge Distillation." @default.
- W3108989436 cites W1686810756 @default.
- W3108989436 cites W2108598243 @default.
- W3108989436 cites W2138621090 @default.
- W3108989436 cites W2194775991 @default.
- W3108989436 cites W2612445135 @default.
- W3108989436 cites W2739879705 @default.
- W3108989436 cites W2798991696 @default.
- W3108989436 cites W2936864631 @default.
- W3108989436 cites W2949117887 @default.
- W3108989436 cites W2949558627 @default.
- W3108989436 cites W2963125010 @default.
- W3108989436 cites W2963140444 @default.
- W3108989436 cites W2963468606 @default.
- W3108989436 cites W2963759070 @default.
- W3108989436 cites W2964111476 @default.
- W3108989436 cites W2964118293 @default.
- W3108989436 cites W2982242214 @default.
- W3108989436 cites W3005680577 @default.
- W3108989436 cites W3035524453 @default.
- W3108989436 cites W3118608800 @default.
- W3108989436 hasPublicationYear "2020" @default.
- W3108989436 type Work @default.
- W3108989436 sameAs 3108989436 @default.
- W3108989436 citedByCount "0" @default.
- W3108989436 crossrefType "posted-content" @default.
- W3108989436 hasAuthorship W3108989436A5040998948 @default.
- W3108989436 hasAuthorship W3108989436A5056657952 @default.
- W3108989436 hasAuthorship W3108989436A5067760326 @default.
- W3108989436 hasAuthorship W3108989436A5089100714 @default.
- W3108989436 hasConcept C103278499 @default.
- W3108989436 hasConcept C115961682 @default.
- W3108989436 hasConcept C119857082 @default.
- W3108989436 hasConcept C154945302 @default.
- W3108989436 hasConcept C178790620 @default.
- W3108989436 hasConcept C185592680 @default.
- W3108989436 hasConcept C204030448 @default.
- W3108989436 hasConcept C2776960227 @default.
- W3108989436 hasConcept C41008148 @default.
- W3108989436 hasConcept C49937458 @default.
- W3108989436 hasConcept C5274069 @default.
- W3108989436 hasConcept C56739046 @default.
- W3108989436 hasConcept C97931131 @default.
- W3108989436 hasConceptScore W3108989436C103278499 @default.
- W3108989436 hasConceptScore W3108989436C115961682 @default.
- W3108989436 hasConceptScore W3108989436C119857082 @default.
- W3108989436 hasConceptScore W3108989436C154945302 @default.
- W3108989436 hasConceptScore W3108989436C178790620 @default.
- W3108989436 hasConceptScore W3108989436C185592680 @default.
- W3108989436 hasConceptScore W3108989436C204030448 @default.
- W3108989436 hasConceptScore W3108989436C2776960227 @default.
- W3108989436 hasConceptScore W3108989436C41008148 @default.
- W3108989436 hasConceptScore W3108989436C49937458 @default.
- W3108989436 hasConceptScore W3108989436C5274069 @default.
- W3108989436 hasConceptScore W3108989436C56739046 @default.
- W3108989436 hasConceptScore W3108989436C97931131 @default.
- W3108989436 hasLocation W31089894361 @default.
- W3108989436 hasOpenAccess W3108989436 @default.
- W3108989436 hasPrimaryLocation W31089894361 @default.
- W3108989436 hasRelatedWork W2956027310 @default.
- W3108989436 hasRelatedWork W2982242214 @default.
- W3108989436 hasRelatedWork W2997941347 @default.
- W3108989436 hasRelatedWork W3004809259 @default.
- W3108989436 hasRelatedWork W3015320289 @default.
- W3108989436 hasRelatedWork W3045672834 @default.
- W3108989436 hasRelatedWork W3092748482 @default.
- W3108989436 hasRelatedWork W3099057664 @default.
- W3108989436 hasRelatedWork W3101547357 @default.
- W3108989436 hasRelatedWork W3110124923 @default.
- W3108989436 hasRelatedWork W3125024256 @default.
- W3108989436 hasRelatedWork W3130859587 @default.
- W3108989436 hasRelatedWork W3131861520 @default.
- W3108989436 hasRelatedWork W3154971402 @default.
- W3108989436 hasRelatedWork W3173799170 @default.
- W3108989436 hasRelatedWork W3181648049 @default.
- W3108989436 hasRelatedWork W3199986538 @default.
- W3108989436 hasRelatedWork W3200103931 @default.
- W3108989436 hasRelatedWork W3211717326 @default.
- W3108989436 hasRelatedWork W3211782300 @default.
- W3108989436 isParatext "false" @default.
- W3108989436 isRetracted "false" @default.
- W3108989436 magId "3108989436" @default.
- W3108989436 workType "article" @default.