Matches in SemOpenAlex for { <https://semopenalex.org/work/W4280496504> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W4280496504 endingPage "160" @default.
- W4280496504 startingPage "160" @default.
- W4280496504 abstract "Large-scale automatic speech recognition model has achieved impressive performance. However, huge computational resources and massive amount of data are required to train an ASR model. Knowledge distillation is a prevalent model compression method which transfers the knowledge from large model to small model. To improve the efficiency of knowledge distillation for end-to-end speech recognition especially in the low-resource setting, a Mixup-based Knowledge Distillation (MKD) method is proposed which combines Mixup, a data-agnostic data augmentation method, with softmax-level knowledge distillation. A loss-level mixture is presented to address the problem caused by the non-linearity of label in the KL-divergence when adopting Mixup to the teacher–student framework. It is mathematically shown that optimizing the mixture of loss function is equivalent to optimize an upper bound of the original knowledge distillation loss. The proposed MKD takes the advantage of Mixup and brings robustness to the model even with a small amount of training data. The experiments on Aishell-1 show that MKD obtains a 15.6% and 3.3% relative improvement on two student models with different parameter scales compared with the existing methods. Experiments on data efficiency demonstrate MKD achieves similar results with only half of the original dataset." @default.
- W4280496504 created "2022-05-22" @default.
- W4280496504 creator A5015584080 @default.
- W4280496504 creator A5045081171 @default.
- W4280496504 creator A5055189079 @default.
- W4280496504 creator A5083539852 @default.
- W4280496504 creator A5084189341 @default.
- W4280496504 date "2022-05-11" @default.
- W4280496504 modified "2023-09-26" @default.
- W4280496504 title "MKD: Mixup-Based Knowledge Distillation for Mandarin End-to-End Speech Recognition" @default.
- W4280496504 cites W2160815625 @default.
- W4280496504 cites W2963738441 @default.
- W4280496504 cites W2970370903 @default.
- W4280496504 cites W3035574064 @default.
- W4280496504 cites W3118536434 @default.
- W4280496504 cites W3135495209 @default.
- W4280496504 cites W3156828761 @default.
- W4280496504 doi "https://doi.org/10.3390/a15050160" @default.
- W4280496504 hasPublicationYear "2022" @default.
- W4280496504 type Work @default.
- W4280496504 citedByCount "3" @default.
- W4280496504 countsByYear W42804965042023 @default.
- W4280496504 crossrefType "journal-article" @default.
- W4280496504 hasAuthorship W4280496504A5015584080 @default.
- W4280496504 hasAuthorship W4280496504A5045081171 @default.
- W4280496504 hasAuthorship W4280496504A5055189079 @default.
- W4280496504 hasAuthorship W4280496504A5083539852 @default.
- W4280496504 hasAuthorship W4280496504A5084189341 @default.
- W4280496504 hasBestOaLocation W42804965041 @default.
- W4280496504 hasConcept C104317684 @default.
- W4280496504 hasConcept C119857082 @default.
- W4280496504 hasConcept C153180895 @default.
- W4280496504 hasConcept C154945302 @default.
- W4280496504 hasConcept C178790620 @default.
- W4280496504 hasConcept C185592680 @default.
- W4280496504 hasConcept C188441871 @default.
- W4280496504 hasConcept C204030448 @default.
- W4280496504 hasConcept C41008148 @default.
- W4280496504 hasConcept C50644808 @default.
- W4280496504 hasConcept C55493867 @default.
- W4280496504 hasConcept C63479239 @default.
- W4280496504 hasConcept C74296488 @default.
- W4280496504 hasConceptScore W4280496504C104317684 @default.
- W4280496504 hasConceptScore W4280496504C119857082 @default.
- W4280496504 hasConceptScore W4280496504C153180895 @default.
- W4280496504 hasConceptScore W4280496504C154945302 @default.
- W4280496504 hasConceptScore W4280496504C178790620 @default.
- W4280496504 hasConceptScore W4280496504C185592680 @default.
- W4280496504 hasConceptScore W4280496504C188441871 @default.
- W4280496504 hasConceptScore W4280496504C204030448 @default.
- W4280496504 hasConceptScore W4280496504C41008148 @default.
- W4280496504 hasConceptScore W4280496504C50644808 @default.
- W4280496504 hasConceptScore W4280496504C55493867 @default.
- W4280496504 hasConceptScore W4280496504C63479239 @default.
- W4280496504 hasConceptScore W4280496504C74296488 @default.
- W4280496504 hasFunder F4320309612 @default.
- W4280496504 hasFunder F4320321001 @default.
- W4280496504 hasFunder F4320335777 @default.
- W4280496504 hasIssue "5" @default.
- W4280496504 hasLocation W42804965041 @default.
- W4280496504 hasOpenAccess W4280496504 @default.
- W4280496504 hasPrimaryLocation W42804965041 @default.
- W4280496504 hasRelatedWork W2610906757 @default.
- W4280496504 hasRelatedWork W2743258233 @default.
- W4280496504 hasRelatedWork W2921182884 @default.
- W4280496504 hasRelatedWork W2938746851 @default.
- W4280496504 hasRelatedWork W2997969508 @default.
- W4280496504 hasRelatedWork W3208883981 @default.
- W4280496504 hasRelatedWork W4211165872 @default.
- W4280496504 hasRelatedWork W4307834408 @default.
- W4280496504 hasRelatedWork W4320925816 @default.
- W4280496504 hasRelatedWork W4321091470 @default.
- W4280496504 hasVolume "15" @default.
- W4280496504 isParatext "false" @default.
- W4280496504 isRetracted "false" @default.
- W4280496504 workType "article" @default.