Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204696009> ?p ?o ?g. }
- W3204696009 endingPage "1532" @default.
- W3204696009 startingPage "1519" @default.
- W3204696009 abstract "We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio. We find that the combination of pre-training, self-training and scaling up model size greatly increases data efficiency, even for extremely large tasks with tens of thousands of hours of labeled data. In particular, on an ASR task with 34k hours of labeled data, by fine-tuning an 8 billion parameter pre-trained Conformer model we can match state-of-the-art (SoTA) performance with only 3% of the training data and significantly improve SoTA with the full training set. We also report on the universal benefits gained from using big pre-trained and self-trained models for a large set of downstream tasks that cover a wide range of speech domains and span multiple orders of magnitudes of dataset sizes, including obtaining SoTA performance on many public benchmarks. In addition, we utilize the learned representation of pre-trained networks to achieve SoTA results on non-ASR tasks." @default.
- W3204696009 created "2021-10-11" @default.
- W3204696009 creator A5001205723 @default.
- W3204696009 creator A5002729731 @default.
- W3204696009 creator A5006451413 @default.
- W3204696009 creator A5007069562 @default.
- W3204696009 creator A5010253402 @default.
- W3204696009 creator A5014592284 @default.
- W3204696009 creator A5016918393 @default.
- W3204696009 creator A5027763497 @default.
- W3204696009 creator A5030491381 @default.
- W3204696009 creator A5032798707 @default.
- W3204696009 creator A5034995105 @default.
- W3204696009 creator A5045147540 @default.
- W3204696009 creator A5048771433 @default.
- W3204696009 creator A5056830118 @default.
- W3204696009 creator A5067837926 @default.
- W3204696009 creator A5070108948 @default.
- W3204696009 creator A5070513394 @default.
- W3204696009 creator A5071715737 @default.
- W3204696009 creator A5071773009 @default.
- W3204696009 creator A5074220692 @default.
- W3204696009 creator A5076426606 @default.
- W3204696009 creator A5076659859 @default.
- W3204696009 creator A5080192908 @default.
- W3204696009 creator A5081428852 @default.
- W3204696009 creator A5084104975 @default.
- W3204696009 creator A5088551093 @default.
- W3204696009 date "2022-10-01" @default.
- W3204696009 modified "2023-10-16" @default.
- W3204696009 title "BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition" @default.
- W3204696009 cites W1494198834 @default.
- W3204696009 cites W1524333225 @default.
- W3204696009 cites W165878654 @default.
- W3204696009 cites W1828163288 @default.
- W3204696009 cites W1915251500 @default.
- W3204696009 cites W1993660824 @default.
- W3204696009 cites W2030931454 @default.
- W3204696009 cites W2033256038 @default.
- W3204696009 cites W2052666245 @default.
- W3204696009 cites W2088622183 @default.
- W3204696009 cites W2101210369 @default.
- W3204696009 cites W2101234009 @default.
- W3204696009 cites W2111316763 @default.
- W3204696009 cites W2121879602 @default.
- W3204696009 cites W2125336414 @default.
- W3204696009 cites W2127141656 @default.
- W3204696009 cites W2250357346 @default.
- W3204696009 cites W2396589722 @default.
- W3204696009 cites W2513345070 @default.
- W3204696009 cites W2593116425 @default.
- W3204696009 cites W2726515241 @default.
- W3204696009 cites W2767487732 @default.
- W3204696009 cites W2767754137 @default.
- W3204696009 cites W2773070064 @default.
- W3204696009 cites W2783831488 @default.
- W3204696009 cites W2794753807 @default.
- W3204696009 cites W2796487725 @default.
- W3204696009 cites W2797583228 @default.
- W3204696009 cites W2842511635 @default.
- W3204696009 cites W2892008152 @default.
- W3204696009 cites W2900096133 @default.
- W3204696009 cites W2900212944 @default.
- W3204696009 cites W2911109671 @default.
- W3204696009 cites W2913178639 @default.
- W3204696009 cites W2926827382 @default.
- W3204696009 cites W2936774411 @default.
- W3204696009 cites W2940322076 @default.
- W3204696009 cites W2962760690 @default.
- W3204696009 cites W2962907457 @default.
- W3204696009 cites W2962911098 @default.
- W3204696009 cites W2963122170 @default.
- W3204696009 cites W2963341956 @default.
- W3204696009 cites W2963403868 @default.
- W3204696009 cites W2963920996 @default.
- W3204696009 cites W2964245029 @default.
- W3204696009 cites W2973049979 @default.
- W3204696009 cites W2973727699 @default.
- W3204696009 cites W2977259558 @default.
- W3204696009 cites W2980077696 @default.
- W3204696009 cites W2988736778 @default.
- W3204696009 cites W2991213871 @default.
- W3204696009 cites W2995181338 @default.
- W3204696009 cites W2995929068 @default.
- W3204696009 cites W2996383576 @default.
- W3204696009 cites W2998532468 @default.
- W3204696009 cites W3003382064 @default.
- W3204696009 cites W3006926732 @default.
- W3204696009 cites W3015265920 @default.
- W3204696009 cites W3015522062 @default.
- W3204696009 cites W3015995734 @default.
- W3204696009 cites W3016010032 @default.
- W3204696009 cites W3016181583 @default.
- W3204696009 cites W3016400019 @default.
- W3204696009 cites W3021234081 @default.
- W3204696009 cites W3025165719 @default.
- W3204696009 cites W3026041220 @default.
- W3204696009 cites W3027083471 @default.