Matches in SemOpenAlex for { <https://semopenalex.org/work/W3027687786> ?p ?o ?g. }
Showing items 1 to 90 of
90
with 100 items per page.
- W3027687786 abstract "While recurrent neural networks still largely define state-of-the-art speech recognition systems, the Transformer network has been proven to be a competitive alternative, especially in the offline condition. Most studies with Transformers have been constrained in a relatively small scale setting, and some forms of data argumentation approaches are usually applied to combat the data sparsity issue. In this paper, we aim at understanding the behaviors of Transformers in the large-scale speech recognition setting, where we have used around 65,000 hours of training data. We investigated various aspects on scaling up Transformers, including model initialization, warmup training as well as different Layer Normalization strategies. In the streaming condition, we compared the widely used attention mask based future context lookahead approach to the Transformer-XL network. From our experiments, we show that Transformers can achieve around 6% relative word error rate (WER) reduction compared to the BLSTM baseline in the offline fashion, while in the streaming fashion, Transformer-XL is comparable to LC-BLSTM with 800 millisecond latency constraint." @default.
- W3027687786 created "2020-05-29" @default.
- W3027687786 creator A5022890229 @default.
- W3027687786 creator A5029670581 @default.
- W3027687786 creator A5030625154 @default.
- W3027687786 creator A5077401426 @default.
- W3027687786 date "2020-05-19" @default.
- W3027687786 modified "2023-10-18" @default.
- W3027687786 title "Exploring Transformers for Large-Scale Speech Recognition" @default.
- W3027687786 cites W1522301498 @default.
- W3027687786 cites W2005708641 @default.
- W3027687786 cites W2064675550 @default.
- W3027687786 cites W2150355110 @default.
- W3027687786 cites W2396384435 @default.
- W3027687786 cites W2795138957 @default.
- W3027687786 cites W2802023636 @default.
- W3027687786 cites W2892009249 @default.
- W3027687786 cites W2906625520 @default.
- W3027687786 cites W2911291251 @default.
- W3027687786 cites W2962835968 @default.
- W3027687786 cites W2963403868 @default.
- W3027687786 cites W2963542740 @default.
- W3027687786 cites W2964084166 @default.
- W3027687786 cites W2964308564 @default.
- W3027687786 cites W2970290486 @default.
- W3027687786 cites W2972818416 @default.
- W3027687786 cites W2976556660 @default.
- W3027687786 cites W2977728428 @default.
- W3027687786 cites W2981661615 @default.
- W3027687786 cites W2986922898 @default.
- W3027687786 cites W2992632249 @default.
- W3027687786 cites W2998814410 @default.
- W3027687786 cites W3008191852 @default.
- W3027687786 cites W3016010032 @default.
- W3027687786 doi "https://doi.org/10.48550/arxiv.2005.09684" @default.
- W3027687786 hasPublicationYear "2020" @default.
- W3027687786 type Work @default.
- W3027687786 sameAs 3027687786 @default.
- W3027687786 citedByCount "8" @default.
- W3027687786 countsByYear W30276877862020 @default.
- W3027687786 countsByYear W30276877862021 @default.
- W3027687786 crossrefType "posted-content" @default.
- W3027687786 hasAuthorship W3027687786A5022890229 @default.
- W3027687786 hasAuthorship W3027687786A5029670581 @default.
- W3027687786 hasAuthorship W3027687786A5030625154 @default.
- W3027687786 hasAuthorship W3027687786A5077401426 @default.
- W3027687786 hasBestOaLocation W30276877861 @default.
- W3027687786 hasConcept C114466953 @default.
- W3027687786 hasConcept C119599485 @default.
- W3027687786 hasConcept C127413603 @default.
- W3027687786 hasConcept C136886441 @default.
- W3027687786 hasConcept C144024400 @default.
- W3027687786 hasConcept C154945302 @default.
- W3027687786 hasConcept C165801399 @default.
- W3027687786 hasConcept C19165224 @default.
- W3027687786 hasConcept C199360897 @default.
- W3027687786 hasConcept C28490314 @default.
- W3027687786 hasConcept C41008148 @default.
- W3027687786 hasConcept C50644808 @default.
- W3027687786 hasConcept C66322947 @default.
- W3027687786 hasConceptScore W3027687786C114466953 @default.
- W3027687786 hasConceptScore W3027687786C119599485 @default.
- W3027687786 hasConceptScore W3027687786C127413603 @default.
- W3027687786 hasConceptScore W3027687786C136886441 @default.
- W3027687786 hasConceptScore W3027687786C144024400 @default.
- W3027687786 hasConceptScore W3027687786C154945302 @default.
- W3027687786 hasConceptScore W3027687786C165801399 @default.
- W3027687786 hasConceptScore W3027687786C19165224 @default.
- W3027687786 hasConceptScore W3027687786C199360897 @default.
- W3027687786 hasConceptScore W3027687786C28490314 @default.
- W3027687786 hasConceptScore W3027687786C41008148 @default.
- W3027687786 hasConceptScore W3027687786C50644808 @default.
- W3027687786 hasConceptScore W3027687786C66322947 @default.
- W3027687786 hasLocation W30276877861 @default.
- W3027687786 hasOpenAccess W3027687786 @default.
- W3027687786 hasPrimaryLocation W30276877861 @default.
- W3027687786 hasRelatedWork W1502265686 @default.
- W3027687786 hasRelatedWork W2971191050 @default.
- W3027687786 hasRelatedWork W2977257638 @default.
- W3027687786 hasRelatedWork W3034772996 @default.
- W3027687786 hasRelatedWork W3180497743 @default.
- W3027687786 hasRelatedWork W3210541621 @default.
- W3027687786 hasRelatedWork W4226102207 @default.
- W3027687786 hasRelatedWork W4226466875 @default.
- W3027687786 hasRelatedWork W4288095186 @default.
- W3027687786 hasRelatedWork W4313065238 @default.
- W3027687786 isParatext "false" @default.
- W3027687786 isRetracted "false" @default.
- W3027687786 magId "3027687786" @default.
- W3027687786 workType "article" @default.