FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4309478457> ?p ?o ?g. }

• W4309478457 endingPage "481" @default.
• W4309478457 startingPage "466" @default.
• W4309478457 abstract "Knowledge distillation as a broad class of methods has led to the development of lightweight and memory efficient models, using a pre-trained model with a large capacity (teacher network) to train a smaller model (student network). Recently, additional variations for knowledge distillation, utilizing activation maps of intermediate layers as the source of knowledge, have been studied. Generally, in computer vision applications, it is seen that the feature activation learned by a higher-capacity model contains richer knowledge, highlighting complete objects while focusing less on the background. Based on this observation, we leverage the teacher’s dual ability to accurately distinguish between positive (relevant to the target object) and negative (irrelevant) areas. We propose a new loss function for distillation, called angular margin-based distillation (AMD) loss. AMD loss uses the angular distance between positive and negative features by projecting them onto a hypersphere, motivated by the near angular distributions seen in many feature extractors. Then, we create a more attentive feature that is angularly distributed on the hypersphere by introducing an angular margin to the positive feature. Transferring such knowledge from the teacher network enables the student model to harness the teacher’s higher discrimination of positive and negative features, thus distilling superior student models. The proposed method is evaluated for various student–teacher network pairs on four public datasets. Furthermore, we show that the proposed method has advantages in compatibility with other learning techniques, such as using fine-grained features, augmentation, and other distillation methods." @default.
• W4309478457 created "2022-11-28" @default.
• W4309478457 creator A5003097277 @default.
• W4309478457 creator A5023797933 @default.
• W4309478457 creator A5044832576 @default.
• W4309478457 creator A5062945520 @default.
• W4309478457 date "2023-01-01" @default.
• W4309478457 modified "2023-10-01" @default.
• W4309478457 title "Leveraging angular distributions for improved knowledge distillation" @default.
• W4309478457 cites W2194775991 @default.
• W4309478457 cites W2233116163 @default.
• W4309478457 cites W2294370754 @default.
• W4309478457 cites W2520774990 @default.
• W4309478457 cites W2739879705 @default.
• W4309478457 cites W2786070938 @default.
• W4309478457 cites W2889292885 @default.
• W4309478457 cites W2936864631 @default.
• W4309478457 cites W2959289524 @default.
• W4309478457 cites W2962858109 @default.
• W4309478457 cites W2962898354 @default.
• W4309478457 cites W2963140444 @default.
• W4309478457 cites W2963163009 @default.
• W4309478457 cites W2963466847 @default.
• W4309478457 cites W2964137095 @default.
• W4309478457 cites W2969985801 @default.
• W4309478457 cites W2982242214 @default.
• W4309478457 cites W2986015886 @default.
• W4309478457 cites W2992308087 @default.
• W4309478457 cites W3004127093 @default.
• W4309478457 cites W3018349615 @default.
• W4309478457 cites W3034368386 @default.
• W4309478457 cites W3103152812 @default.
• W4309478457 cites W3111522705 @default.
• W4309478457 cites W3113395635 @default.
• W4309478457 cites W3115306706 @default.
• W4309478457 cites W3127215335 @default.
• W4309478457 cites W3159704894 @default.
• W4309478457 cites W3178139507 @default.
• W4309478457 cites W3185864054 @default.
• W4309478457 cites W4200516889 @default.
• W4309478457 cites W4229333471 @default.
• W4309478457 doi "https://doi.org/10.1016/j.neucom.2022.11.029" @default.
• W4309478457 hasPublicationYear "2023" @default.
• W4309478457 type Work @default.
• W4309478457 citedByCount "2" @default.
• W4309478457 countsByYear W43094784572023 @default.
• W4309478457 crossrefType "journal-article" @default.
• W4309478457 hasAuthorship W4309478457A5003097277 @default.
• W4309478457 hasAuthorship W4309478457A5023797933 @default.
• W4309478457 hasAuthorship W4309478457A5044832576 @default.
• W4309478457 hasAuthorship W4309478457A5062945520 @default.
• W4309478457 hasBestOaLocation W43094784572 @default.
• W4309478457 hasConcept C119857082 @default.
• W4309478457 hasConcept C138885662 @default.
• W4309478457 hasConcept C153083717 @default.
• W4309478457 hasConcept C153180895 @default.
• W4309478457 hasConcept C154945302 @default.
• W4309478457 hasConcept C178790620 @default.
• W4309478457 hasConcept C185592680 @default.
• W4309478457 hasConcept C204030448 @default.
• W4309478457 hasConcept C2776401178 @default.
• W4309478457 hasConcept C2776562905 @default.
• W4309478457 hasConcept C41008148 @default.
• W4309478457 hasConcept C41895202 @default.
• W4309478457 hasConcept C52622490 @default.
• W4309478457 hasConcept C774472 @default.
• W4309478457 hasConcept C83665646 @default.
• W4309478457 hasConceptScore W4309478457C119857082 @default.
• W4309478457 hasConceptScore W4309478457C138885662 @default.
• W4309478457 hasConceptScore W4309478457C153083717 @default.
• W4309478457 hasConceptScore W4309478457C153180895 @default.
• W4309478457 hasConceptScore W4309478457C154945302 @default.
• W4309478457 hasConceptScore W4309478457C178790620 @default.
• W4309478457 hasConceptScore W4309478457C185592680 @default.
• W4309478457 hasConceptScore W4309478457C204030448 @default.
• W4309478457 hasConceptScore W4309478457C2776401178 @default.
• W4309478457 hasConceptScore W4309478457C2776562905 @default.
• W4309478457 hasConceptScore W4309478457C41008148 @default.
• W4309478457 hasConceptScore W4309478457C41895202 @default.
• W4309478457 hasConceptScore W4309478457C52622490 @default.
• W4309478457 hasConceptScore W4309478457C774472 @default.
• W4309478457 hasConceptScore W4309478457C83665646 @default.
• W4309478457 hasFunder F4320332180 @default.
• W4309478457 hasLocation W43094784571 @default.
• W4309478457 hasLocation W43094784572 @default.
• W4309478457 hasLocation W43094784573 @default.
• W4309478457 hasOpenAccess W4309478457 @default.
• W4309478457 hasPrimaryLocation W43094784571 @default.
• W4309478457 hasRelatedWork W1977222486 @default.
• W4309478457 hasRelatedWork W2049538278 @default.
• W4309478457 hasRelatedWork W2115092356 @default.
• W4309478457 hasRelatedWork W2144059113 @default.
• W4309478457 hasRelatedWork W2146076056 @default.
• W4309478457 hasRelatedWork W2368520799 @default.
• W4309478457 hasRelatedWork W2546942002 @default.
• W4309478457 hasRelatedWork W2772780115 @default.
• W4309478457 hasRelatedWork W2811390910 @default.
• W4309478457 hasRelatedWork W3003836766 @default.

• next