Matches in SemOpenAlex for { <https://semopenalex.org/work/W3048208370> ?p ?o ?g. }
- W3048208370 abstract "Attention-based sequence-to-sequence (seq2seq) models have achieved promising results in automatic speech recognition (ASR). However, as these models decode in a left-to-right way, they do not have access to context on the right. We leverage both left and right context by applying BERT as an external language model to seq2seq ASR through knowledge distillation. In our proposed method, BERT generates soft labels to guide the training of seq2seq ASR. Furthermore, we leverage context beyond the current utterance as input to BERT. Experimental evaluations show that our method significantly improves the ASR performance from the seq2seq baseline on the Corpus of Spontaneous Japanese (CSJ). Knowledge distillation from BERT outperforms that from a transformer LM that only looks at left context. We also show the effectiveness of leveraging context beyond the current utterance. Our method outperforms other LM application approaches such as n-best rescoring and shallow fusion, while it does not require extra inference cost." @default.
- W3048208370 created "2020-08-13" @default.
- W3048208370 creator A5023133164 @default.
- W3048208370 creator A5034216052 @default.
- W3048208370 creator A5038044080 @default.
- W3048208370 creator A5039280307 @default.
- W3048208370 creator A5040282669 @default.
- W3048208370 creator A5072037248 @default.
- W3048208370 date "2020-08-09" @default.
- W3048208370 modified "2023-09-27" @default.
- W3048208370 title "Distilling the Knowledge of BERT for Sequence-to-Sequence ASR" @default.
- W3048208370 cites W1821462560 @default.
- W3048208370 cites W1828163288 @default.
- W3048208370 cites W1915251500 @default.
- W3048208370 cites W2062988874 @default.
- W3048208370 cites W2127141656 @default.
- W3048208370 cites W2183341477 @default.
- W3048208370 cites W2327501763 @default.
- W3048208370 cites W2577366047 @default.
- W3048208370 cites W2752047430 @default.
- W3048208370 cites W2886025712 @default.
- W3048208370 cites W2888779557 @default.
- W3048208370 cites W2889504751 @default.
- W3048208370 cites W2905933322 @default.
- W3048208370 cites W2935811960 @default.
- W3048208370 cites W2936774411 @default.
- W3048208370 cites W2937649809 @default.
- W3048208370 cites W2949640357 @default.
- W3048208370 cites W2962739339 @default.
- W3048208370 cites W2962784628 @default.
- W3048208370 cites W2963341956 @default.
- W3048208370 cites W2963362078 @default.
- W3048208370 cites W2963403868 @default.
- W3048208370 cites W2963827914 @default.
- W3048208370 cites W2964121744 @default.
- W3048208370 cites W2965373594 @default.
- W3048208370 cites W2970597249 @default.
- W3048208370 cites W2972991710 @default.
- W3048208370 cites W2990391581 @default.
- W3048208370 cites W2993326934 @default.
- W3048208370 cites W3034729383 @default.
- W3048208370 cites W3034775979 @default.
- W3048208370 cites W3035317912 @default.
- W3048208370 cites W648786980 @default.
- W3048208370 cites W854541894 @default.
- W3048208370 doi "https://doi.org/10.48550/arxiv.2008.03822" @default.
- W3048208370 hasPublicationYear "2020" @default.
- W3048208370 type Work @default.
- W3048208370 sameAs 3048208370 @default.
- W3048208370 citedByCount "1" @default.
- W3048208370 countsByYear W30482083702021 @default.
- W3048208370 crossrefType "posted-content" @default.
- W3048208370 hasAuthorship W3048208370A5023133164 @default.
- W3048208370 hasAuthorship W3048208370A5034216052 @default.
- W3048208370 hasAuthorship W3048208370A5038044080 @default.
- W3048208370 hasAuthorship W3048208370A5039280307 @default.
- W3048208370 hasAuthorship W3048208370A5040282669 @default.
- W3048208370 hasAuthorship W3048208370A5072037248 @default.
- W3048208370 hasBestOaLocation W30482083701 @default.
- W3048208370 hasConcept C119599485 @default.
- W3048208370 hasConcept C127413603 @default.
- W3048208370 hasConcept C137293760 @default.
- W3048208370 hasConcept C153083717 @default.
- W3048208370 hasConcept C154945302 @default.
- W3048208370 hasConcept C165801399 @default.
- W3048208370 hasConcept C166957645 @default.
- W3048208370 hasConcept C204321447 @default.
- W3048208370 hasConcept C2775852435 @default.
- W3048208370 hasConcept C2776214188 @default.
- W3048208370 hasConcept C2778112365 @default.
- W3048208370 hasConcept C2779343474 @default.
- W3048208370 hasConcept C28490314 @default.
- W3048208370 hasConcept C41008148 @default.
- W3048208370 hasConcept C54355233 @default.
- W3048208370 hasConcept C66322947 @default.
- W3048208370 hasConcept C86803240 @default.
- W3048208370 hasConcept C95457728 @default.
- W3048208370 hasConceptScore W3048208370C119599485 @default.
- W3048208370 hasConceptScore W3048208370C127413603 @default.
- W3048208370 hasConceptScore W3048208370C137293760 @default.
- W3048208370 hasConceptScore W3048208370C153083717 @default.
- W3048208370 hasConceptScore W3048208370C154945302 @default.
- W3048208370 hasConceptScore W3048208370C165801399 @default.
- W3048208370 hasConceptScore W3048208370C166957645 @default.
- W3048208370 hasConceptScore W3048208370C204321447 @default.
- W3048208370 hasConceptScore W3048208370C2775852435 @default.
- W3048208370 hasConceptScore W3048208370C2776214188 @default.
- W3048208370 hasConceptScore W3048208370C2778112365 @default.
- W3048208370 hasConceptScore W3048208370C2779343474 @default.
- W3048208370 hasConceptScore W3048208370C28490314 @default.
- W3048208370 hasConceptScore W3048208370C41008148 @default.
- W3048208370 hasConceptScore W3048208370C54355233 @default.
- W3048208370 hasConceptScore W3048208370C66322947 @default.
- W3048208370 hasConceptScore W3048208370C86803240 @default.
- W3048208370 hasConceptScore W3048208370C95457728 @default.
- W3048208370 hasLocation W30482083701 @default.
- W3048208370 hasOpenAccess W3048208370 @default.
- W3048208370 hasPrimaryLocation W30482083701 @default.
- W3048208370 hasRelatedWork W2046811668 @default.
- W3048208370 hasRelatedWork W204968311 @default.