Matches in SemOpenAlex for { <https://semopenalex.org/work/W4310266072> ?p ?o ?g. }
Showing items 1 to 77 of
77
with 100 items per page.
- W4310266072 abstract "Recent advances have indicated the strengths of self-supervised pre-training for improving representation learning on downstream tasks. Existing works often utilize self-supervised pre-trained models by fine-tuning on downstream tasks. However, fine-tuning does not generalize to the case when one needs to build a customized model architecture different from the self-supervised model. In this work, we formulate a new knowledge distillation framework to transfer the knowledge from self-supervised pre-trained models to any other student network by a novel approach named Embedding Graph Alignment. Specifically, inspired by the spirit of instance discrimination in self-supervised learning, we model the instance-instance relations by a graph formulation in the feature embedding space and distill the self-supervised teacher knowledge to a student network by aligning the teacher graph and the student graph. Our distillation scheme can be flexibly applied to transfer the self-supervised knowledge to enhance representation learning on various student networks. We demonstrate that our model outperforms multiple representative knowledge distillation methods on three benchmark datasets, including CIFAR100, STL10, and TinyImageNet. Code is here: https://github.com/yccm/EGA." @default.
- W4310266072 created "2022-11-30" @default.
- W4310266072 creator A5033035704 @default.
- W4310266072 creator A5040372929 @default.
- W4310266072 creator A5086043611 @default.
- W4310266072 date "2022-11-23" @default.
- W4310266072 modified "2023-09-25" @default.
- W4310266072 title "Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment" @default.
- W4310266072 doi "https://doi.org/10.48550/arxiv.2211.13264" @default.
- W4310266072 hasPublicationYear "2022" @default.
- W4310266072 type Work @default.
- W4310266072 citedByCount "0" @default.
- W4310266072 crossrefType "posted-content" @default.
- W4310266072 hasAuthorship W4310266072A5033035704 @default.
- W4310266072 hasAuthorship W4310266072A5040372929 @default.
- W4310266072 hasAuthorship W4310266072A5086043611 @default.
- W4310266072 hasBestOaLocation W43102660721 @default.
- W4310266072 hasConcept C119857082 @default.
- W4310266072 hasConcept C132525143 @default.
- W4310266072 hasConcept C13280743 @default.
- W4310266072 hasConcept C134306372 @default.
- W4310266072 hasConcept C136389625 @default.
- W4310266072 hasConcept C154945302 @default.
- W4310266072 hasConcept C17744445 @default.
- W4310266072 hasConcept C178790620 @default.
- W4310266072 hasConcept C185592680 @default.
- W4310266072 hasConcept C185798385 @default.
- W4310266072 hasConcept C199539241 @default.
- W4310266072 hasConcept C204030448 @default.
- W4310266072 hasConcept C205649164 @default.
- W4310266072 hasConcept C2776359362 @default.
- W4310266072 hasConcept C33923547 @default.
- W4310266072 hasConcept C41008148 @default.
- W4310266072 hasConcept C41608201 @default.
- W4310266072 hasConcept C50644808 @default.
- W4310266072 hasConcept C59404180 @default.
- W4310266072 hasConcept C77618280 @default.
- W4310266072 hasConcept C80444323 @default.
- W4310266072 hasConcept C94625758 @default.
- W4310266072 hasConceptScore W4310266072C119857082 @default.
- W4310266072 hasConceptScore W4310266072C132525143 @default.
- W4310266072 hasConceptScore W4310266072C13280743 @default.
- W4310266072 hasConceptScore W4310266072C134306372 @default.
- W4310266072 hasConceptScore W4310266072C136389625 @default.
- W4310266072 hasConceptScore W4310266072C154945302 @default.
- W4310266072 hasConceptScore W4310266072C17744445 @default.
- W4310266072 hasConceptScore W4310266072C178790620 @default.
- W4310266072 hasConceptScore W4310266072C185592680 @default.
- W4310266072 hasConceptScore W4310266072C185798385 @default.
- W4310266072 hasConceptScore W4310266072C199539241 @default.
- W4310266072 hasConceptScore W4310266072C204030448 @default.
- W4310266072 hasConceptScore W4310266072C205649164 @default.
- W4310266072 hasConceptScore W4310266072C2776359362 @default.
- W4310266072 hasConceptScore W4310266072C33923547 @default.
- W4310266072 hasConceptScore W4310266072C41008148 @default.
- W4310266072 hasConceptScore W4310266072C41608201 @default.
- W4310266072 hasConceptScore W4310266072C50644808 @default.
- W4310266072 hasConceptScore W4310266072C59404180 @default.
- W4310266072 hasConceptScore W4310266072C77618280 @default.
- W4310266072 hasConceptScore W4310266072C80444323 @default.
- W4310266072 hasConceptScore W4310266072C94625758 @default.
- W4310266072 hasLocation W43102660721 @default.
- W4310266072 hasOpenAccess W4310266072 @default.
- W4310266072 hasPrimaryLocation W43102660721 @default.
- W4310266072 hasRelatedWork W2335364074 @default.
- W4310266072 hasRelatedWork W2912933387 @default.
- W4310266072 hasRelatedWork W2988416191 @default.
- W4310266072 hasRelatedWork W3013644914 @default.
- W4310266072 hasRelatedWork W3094076422 @default.
- W4310266072 hasRelatedWork W3126201013 @default.
- W4310266072 hasRelatedWork W3162567751 @default.
- W4310266072 hasRelatedWork W4221088574 @default.
- W4310266072 hasRelatedWork W4285260836 @default.
- W4310266072 hasRelatedWork W4300480195 @default.
- W4310266072 isParatext "false" @default.
- W4310266072 isRetracted "false" @default.
- W4310266072 workType "article" @default.