FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4285133995> ?p ?o ?g. }

• W4285133995 endingPage "1" @default.
• W4285133995 startingPage "1" @default.
• W4285133995 abstract "Knowledge distillation is a technique to enhance the generalization ability of a student model by exploiting outputs from a teacher model. Recently, feature-map based variants explore knowledge transfer between manually assigned teacher-student pairs in intermediate layers for further improvement. However, layer semantics may vary in different neural networks, resulting in performance degeneration due to negative regularization from semantic mismatch in manual layer associations. To address this issue, we propose semantic calibration for cross-layer knowledge distillation (SemCKD), which automatically assigns proper target layers of the teacher model for each student layer with an attention mechanism. With a learned attention distribution, each student layer distills knowledge contained in multiple teacher layers rather than a specific intermediate layer for appropriate cross-layer supervision. We further provide theoretical analysis of the association weights and conduct extensive experiments to demonstrate the effectiveness of our approach. On average, SemCKD improves the student Top-1 classification accuracy by 4.27% across twelve different teacher-student model combinations on CIFAR-100. Code is available at <uri xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>https://github.com/DefangChen/SemCKD</uri> ." @default.
• W4285133995 created "2022-07-14" @default.
• W4285133995 creator A5031767581 @default.
• W4285133995 creator A5034363096 @default.
• W4285133995 creator A5046093778 @default.
• W4285133995 creator A5056678232 @default.
• W4285133995 creator A5061883190 @default.
• W4285133995 creator A5068325469 @default.
• W4285133995 date "2022-01-01" @default.
• W4285133995 modified "2023-10-16" @default.
• W4285133995 title "SemCKD: Semantic Calibration for Cross-Layer Knowledge Distillation" @default.
• W4285133995 cites W1677182931 @default.
• W4285133995 cites W1965555277 @default.
• W4285133995 cites W2022508996 @default.
• W4285133995 cites W2035996732 @default.
• W4285133995 cites W2057069782 @default.
• W4285133995 cites W2117539524 @default.
• W4285133995 cites W2145607950 @default.
• W4285133995 cites W2163922914 @default.
• W4285133995 cites W2194775991 @default.
• W4285133995 cites W2294370754 @default.
• W4285133995 cites W2766447205 @default.
• W4285133995 cites W2883780447 @default.
• W4285133995 cites W2887783173 @default.
• W4285133995 cites W2936864631 @default.
• W4285133995 cites W2955192706 @default.
• W4285133995 cites W2962858109 @default.
• W4285133995 cites W2963091558 @default.
• W4285133995 cites W2963125010 @default.
• W4285133995 cites W2963140444 @default.
• W4285133995 cites W2963163009 @default.
• W4285133995 cites W2964137095 @default.
• W4285133995 cites W2982157312 @default.
• W4285133995 cites W2982242214 @default.
• W4285133995 cites W2986015886 @default.
• W4285133995 cites W2996970889 @default.
• W4285133995 cites W3034756453 @default.
• W4285133995 cites W3035163969 @default.
• W4285133995 cites W3108075360 @default.
• W4285133995 cites W3108491103 @default.
• W4285133995 cites W3109689953 @default.
• W4285133995 cites W3174102142 @default.
• W4285133995 cites W4214535608 @default.
• W4285133995 cites W4224927203 @default.
• W4285133995 cites W4312258136 @default.
• W4285133995 doi "https://doi.org/10.1109/tkde.2022.3171571" @default.
• W4285133995 hasPublicationYear "2022" @default.
• W4285133995 type Work @default.
• W4285133995 citedByCount "1" @default.
• W4285133995 countsByYear W42851339952022 @default.
• W4285133995 crossrefType "journal-article" @default.
• W4285133995 hasAuthorship W4285133995A5031767581 @default.
• W4285133995 hasAuthorship W4285133995A5034363096 @default.
• W4285133995 hasAuthorship W4285133995A5046093778 @default.
• W4285133995 hasAuthorship W4285133995A5056678232 @default.
• W4285133995 hasAuthorship W4285133995A5061883190 @default.
• W4285133995 hasAuthorship W4285133995A5068325469 @default.
• W4285133995 hasConcept C119857082 @default.
• W4285133995 hasConcept C124101348 @default.
• W4285133995 hasConcept C134306372 @default.
• W4285133995 hasConcept C154945302 @default.
• W4285133995 hasConcept C177148314 @default.
• W4285133995 hasConcept C177264268 @default.
• W4285133995 hasConcept C178790620 @default.
• W4285133995 hasConcept C184337299 @default.
• W4285133995 hasConcept C185592680 @default.
• W4285133995 hasConcept C199360897 @default.
• W4285133995 hasConcept C204030448 @default.
• W4285133995 hasConcept C204321447 @default.
• W4285133995 hasConcept C23123220 @default.
• W4285133995 hasConcept C2776135515 @default.
• W4285133995 hasConcept C2776760102 @default.
• W4285133995 hasConcept C2779227376 @default.
• W4285133995 hasConcept C33923547 @default.
• W4285133995 hasConcept C41008148 @default.
• W4285133995 hasConceptScore W4285133995C119857082 @default.
• W4285133995 hasConceptScore W4285133995C124101348 @default.
• W4285133995 hasConceptScore W4285133995C134306372 @default.
• W4285133995 hasConceptScore W4285133995C154945302 @default.
• W4285133995 hasConceptScore W4285133995C177148314 @default.
• W4285133995 hasConceptScore W4285133995C177264268 @default.
• W4285133995 hasConceptScore W4285133995C178790620 @default.
• W4285133995 hasConceptScore W4285133995C184337299 @default.
• W4285133995 hasConceptScore W4285133995C185592680 @default.
• W4285133995 hasConceptScore W4285133995C199360897 @default.
• W4285133995 hasConceptScore W4285133995C204030448 @default.
• W4285133995 hasConceptScore W4285133995C204321447 @default.
• W4285133995 hasConceptScore W4285133995C23123220 @default.
• W4285133995 hasConceptScore W4285133995C2776135515 @default.
• W4285133995 hasConceptScore W4285133995C2776760102 @default.
• W4285133995 hasConceptScore W4285133995C2779227376 @default.
• W4285133995 hasConceptScore W4285133995C33923547 @default.
• W4285133995 hasConceptScore W4285133995C41008148 @default.
• W4285133995 hasFunder F4320321001 @default.
• W4285133995 hasFunder F4320338464 @default.
• W4285133995 hasLocation W42851339951 @default.
• W4285133995 hasOpenAccess W4285133995 @default.
• W4285133995 hasPrimaryLocation W42851339951 @default.

• next