FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3137609883> ?p ?o ?g. }

• W3137609883 endingPage "1" @default.
• W3137609883 startingPage "1" @default.
• W3137609883 abstract "Remarkable achievements have been obtained by deep neural networks in the last several years. However, the breakthrough in neural networks accuracy is always accompanied by explosive growth of computation and parameters, which leads to a severe limitation of model deployment. In this paper, we propose a novel knowledge distillation technique named self-distillation to address this problem. Self-distillation attaches several attention modules and shallow classifiers at different depths of neural networks and distills knowledge from the deepest classifier to the shallower classifiers. Different from the conventional knowledge distillation methods where the knowledge of the teacher model is transferred to another student model, self-distillation can be considered as knowledge transfer in the same model - from the deeper layers to the shallow layers. Moreover, the additional classifiers in self-distillation allow the neural network to work in a dynamic manner, which leads to a much higher acceleration. Experiments demonstrate that self-distillation has consistent and significant effectiveness on various neural networks and datasets. On average, 3.49 and 2.32 percent accuracy boost are observed on CIFAR100 and ImageNet. Besides, experiments show that self-distillation can be combined with other model compression methods, including knowledge distillation, pruning and lightweight model design." @default.
• W3137609883 created "2021-03-29" @default.
• W3137609883 creator A5006570986 @default.
• W3137609883 creator A5043676563 @default.
• W3137609883 creator A5054041500 @default.
• W3137609883 date "2021-01-01" @default.
• W3137609883 modified "2023-10-09" @default.
• W3137609883 title "Self-Distillation: Towards Efficient and Compact Neural Networks" @default.
• W3137609883 cites W1996901117 @default.
• W3137609883 cites W2108598243 @default.
• W3137609883 cites W2194775991 @default.
• W3137609883 cites W2294370754 @default.
• W3137609883 cites W2300242332 @default.
• W3137609883 cites W2562731582 @default.
• W3137609883 cites W2604790786 @default.
• W3137609883 cites W2620998106 @default.
• W3137609883 cites W2739879705 @default.
• W3137609883 cites W2752782242 @default.
• W3137609883 cites W2798170643 @default.
• W3137609883 cites W2883780447 @default.
• W3137609883 cites W2884561390 @default.
• W3137609883 cites W2886851211 @default.
• W3137609883 cites W2901505625 @default.
• W3137609883 cites W2952787292 @default.
• W3137609883 cites W2962944050 @default.
• W3137609883 cites W2963037989 @default.
• W3137609883 cites W2963140444 @default.
• W3137609883 cites W2963150697 @default.
• W3137609883 cites W2963163009 @default.
• W3137609883 cites W2963393494 @default.
• W3137609883 cites W2963446712 @default.
• W3137609883 cites W2964016283 @default.
• W3137609883 cites W2964137095 @default.
• W3137609883 cites W2981698279 @default.
• W3137609883 cites W2981751377 @default.
• W3137609883 cites W2981819252 @default.
• W3137609883 cites W2982083293 @default.
• W3137609883 cites W2982157312 @default.
• W3137609883 cites W2982220706 @default.
• W3137609883 cites W2982242214 @default.
• W3137609883 cites W2984618279 @default.
• W3137609883 cites W2987861506 @default.
• W3137609883 cites W2990045899 @default.
• W3137609883 cites W2990231018 @default.
• W3137609883 cites W2990792015 @default.
• W3137609883 cites W2996999409 @default.
• W3137609883 cites W2997006708 @default.
• W3137609883 cites W3004127093 @default.
• W3137609883 cites W3009751875 @default.
• W3137609883 cites W3034406766 @default.
• W3137609883 cites W3091093673 @default.
• W3137609883 cites W753847829 @default.
• W3137609883 doi "https://doi.org/10.1109/tpami.2021.3067100" @default.
• W3137609883 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/33735074" @default.
• W3137609883 hasPublicationYear "2021" @default.
• W3137609883 type Work @default.
• W3137609883 sameAs 3137609883 @default.
• W3137609883 citedByCount "27" @default.
• W3137609883 countsByYear W31376098832021 @default.
• W3137609883 countsByYear W31376098832022 @default.
• W3137609883 countsByYear W31376098832023 @default.
• W3137609883 crossrefType "journal-article" @default.
• W3137609883 hasAuthorship W3137609883A5006570986 @default.
• W3137609883 hasAuthorship W3137609883A5043676563 @default.
• W3137609883 hasAuthorship W3137609883A5054041500 @default.
• W3137609883 hasBestOaLocation W31376098831 @default.
• W3137609883 hasConcept C108010975 @default.
• W3137609883 hasConcept C11413529 @default.
• W3137609883 hasConcept C119857082 @default.
• W3137609883 hasConcept C154945302 @default.
• W3137609883 hasConcept C178790620 @default.
• W3137609883 hasConcept C185592680 @default.
• W3137609883 hasConcept C204030448 @default.
• W3137609883 hasConcept C41008148 @default.
• W3137609883 hasConcept C45374587 @default.
• W3137609883 hasConcept C50644808 @default.
• W3137609883 hasConcept C6557445 @default.
• W3137609883 hasConcept C86803240 @default.
• W3137609883 hasConceptScore W3137609883C108010975 @default.
• W3137609883 hasConceptScore W3137609883C11413529 @default.
• W3137609883 hasConceptScore W3137609883C119857082 @default.
• W3137609883 hasConceptScore W3137609883C154945302 @default.
• W3137609883 hasConceptScore W3137609883C178790620 @default.
• W3137609883 hasConceptScore W3137609883C185592680 @default.
• W3137609883 hasConceptScore W3137609883C204030448 @default.
• W3137609883 hasConceptScore W3137609883C41008148 @default.
• W3137609883 hasConceptScore W3137609883C45374587 @default.
• W3137609883 hasConceptScore W3137609883C50644808 @default.
• W3137609883 hasConceptScore W3137609883C6557445 @default.
• W3137609883 hasConceptScore W3137609883C86803240 @default.
• W3137609883 hasFunder F4320321001 @default.
• W3137609883 hasLocation W31376098831 @default.
• W3137609883 hasLocation W31376098832 @default.
• W3137609883 hasOpenAccess W3137609883 @default.
• W3137609883 hasPrimaryLocation W31376098831 @default.
• W3137609883 hasRelatedWork W2961085424 @default.
• W3137609883 hasRelatedWork W2981133353 @default.
• W3137609883 hasRelatedWork W3046775127 @default.

• next