FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3158686502> ?p ?o ?g. }

• W3158686502 abstract "The recent studies of knowledge distillation have discovered that ensembling the dark knowledge from multiple teachers or students contributes to creating better soft targets for training, but at the cost of significantly more computations and/or parameters. In this work, we present BAtch Knowledge Ensembling (BAKE) to produce refined soft targets for anchor images by propagating and ensembling the knowledge of the other samples in the same mini-batch. Specifically, for each sample of interest, the propagation of knowledge is weighted in accordance with the inter-sample affinities, which are estimated on-the-fly with the current network. The propagated knowledge can then be ensembled to form a better soft target for distillation. In this way, our BAKE framework achieves online knowledge ensembling across multiple samples with only a single network. It requires minimal computational and memory overhead compared to existing knowledge ensembling methods. Extensive experiments demonstrate that the lightweight yet effective BAKE consistently boosts the classification performance of various architectures on multiple datasets, e.g., a significant +0.7% gain of Swin-T on ImageNet with only +1.5% computational overhead and zero additional parameters. BAKE does not only improve the vanilla baselines, but also surpasses the single-network state-of-the-arts on all the benchmarks." @default.
• W3158686502 created "2021-05-10" @default.
• W3158686502 creator A5024522346 @default.
• W3158686502 creator A5033792593 @default.
• W3158686502 creator A5047065510 @default.
• W3158686502 creator A5047802386 @default.
• W3158686502 creator A5065073978 @default.
• W3158686502 creator A5066727108 @default.
• W3158686502 creator A5073531889 @default.
• W3158686502 date "2021-04-27" @default.
• W3158686502 modified "2023-10-16" @default.
• W3158686502 title "Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification" @default.
• W3158686502 cites W1797268635 @default.
• W3158686502 cites W1821462560 @default.
• W3158686502 cites W1861492603 @default.
• W3158686502 cites W2097117768 @default.
• W3158686502 cites W2108598243 @default.
• W3158686502 cites W2152161678 @default.
• W3158686502 cites W2154455818 @default.
• W3158686502 cites W2183341477 @default.
• W3158686502 cites W2194775991 @default.
• W3158686502 cites W2302255633 @default.
• W3158686502 cites W2549139847 @default.
• W3158686502 cites W2560023338 @default.
• W3158686502 cites W2565639579 @default.
• W3158686502 cites W2592691248 @default.
• W3158686502 cites W2613718673 @default.
• W3158686502 cites W2620998106 @default.
• W3158686502 cites W2802198257 @default.
• W3158686502 cites W2890159224 @default.
• W3158686502 cites W2904170036 @default.
• W3158686502 cites W2955425717 @default.
• W3158686502 cites W2962793481 @default.
• W3158686502 cites W2962843773 @default.
• W3158686502 cites W2963060032 @default.
• W3158686502 cites W2963073614 @default.
• W3158686502 cites W2963140444 @default.
• W3158686502 cites W2963150697 @default.
• W3158686502 cites W2963163009 @default.
• W3158686502 cites W2963207607 @default.
• W3158686502 cites W2963342403 @default.
• W3158686502 cites W2963397674 @default.
• W3158686502 cites W2963446712 @default.
• W3158686502 cites W2963703197 @default.
• W3158686502 cites W2964220233 @default.
• W3158686502 cites W2982242214 @default.
• W3158686502 cites W2987861506 @default.
• W3158686502 cites W2992308087 @default.
• W3158686502 cites W2995607862 @default.
• W3158686502 cites W2995852213 @default.
• W3158686502 cites W3005680577 @default.
• W3158686502 cites W3016719260 @default.
• W3158686502 cites W3033469067 @default.
• W3158686502 cites W3034363135 @default.
• W3158686502 cites W3034429256 @default.
• W3158686502 cites W3034695001 @default.
• W3158686502 cites W3034756453 @default.
• W3158686502 cites W3035160371 @default.
• W3158686502 cites W3035314311 @default.
• W3158686502 cites W3035321581 @default.
• W3158686502 cites W3035524453 @default.
• W3158686502 cites W3095367607 @default.
• W3158686502 cites W3101821705 @default.
• W3158686502 cites W3103165805 @default.
• W3158686502 cites W3118608800 @default.
• W3158686502 cites W3174265363 @default.
• W3158686502 cites W3177096435 @default.
• W3158686502 cites W3202406646 @default.
• W3158686502 cites W3203532272 @default.
• W3158686502 cites W3204610735 @default.
• W3158686502 doi "https://doi.org/10.48550/arxiv.2104.13298" @default.
• W3158686502 hasPublicationYear "2021" @default.
• W3158686502 type Work @default.
• W3158686502 sameAs 3158686502 @default.
• W3158686502 citedByCount "1" @default.
• W3158686502 countsByYear W31586865022021 @default.
• W3158686502 crossrefType "posted-content" @default.
• W3158686502 hasAuthorship W3158686502A5024522346 @default.
• W3158686502 hasAuthorship W3158686502A5033792593 @default.
• W3158686502 hasAuthorship W3158686502A5047065510 @default.
• W3158686502 hasAuthorship W3158686502A5047802386 @default.
• W3158686502 hasAuthorship W3158686502A5065073978 @default.
• W3158686502 hasAuthorship W3158686502A5066727108 @default.
• W3158686502 hasAuthorship W3158686502A5073531889 @default.
• W3158686502 hasBestOaLocation W31586865021 @default.
• W3158686502 hasConcept C111919701 @default.
• W3158686502 hasConcept C11413529 @default.
• W3158686502 hasConcept C119857082 @default.
• W3158686502 hasConcept C120567893 @default.
• W3158686502 hasConcept C124101348 @default.
• W3158686502 hasConcept C134306372 @default.
• W3158686502 hasConcept C154945302 @default.
• W3158686502 hasConcept C178790620 @default.
• W3158686502 hasConcept C185592680 @default.
• W3158686502 hasConcept C198531522 @default.
• W3158686502 hasConcept C204030448 @default.
• W3158686502 hasConcept C2779960059 @default.
• W3158686502 hasConcept C33923547 @default.
• W3158686502 hasConcept C41008148 @default.
• W3158686502 hasConcept C43617362 @default.

• next