Matches in SemOpenAlex for { <https://semopenalex.org/work/W4372260141> ?p ?o ?g. }
Showing items 1 to 76 of
76
with 100 items per page.
- W4372260141 abstract "Representation models pre-trained on unlabeled data show competitive performance in speech recognition, even when fine-tuned on small amounts of labeled data. The continual representation learning (CTRL) framework combines pre-training and continual learning methods to obtain powerful representation. CTRL relies on two neural networks, online and offline models, where the fixed latter model transfers information to the former model with continual learning loss. In this paper, we present momentum continual representation learning (M-CTRL), a framework that slowly updates the offline model with an exponential moving average of the online model. Our framework aims to capture information from the offline model improved on past and new domains. To evaluate our framework, we continually pre-train wav2vec 2.0 with M-CTRL in the following order: Librispeech, Wall Street Journal, and TED-LIUM V3. Our experiments demonstrate that M-CTRL improves the performance in the new domain and reduces information loss in the past domain compared to CTRL." @default.
- W4372260141 created "2023-05-07" @default.
- W4372260141 creator A5002418613 @default.
- W4372260141 creator A5012684064 @default.
- W4372260141 creator A5037747694 @default.
- W4372260141 creator A5051680529 @default.
- W4372260141 date "2023-06-04" @default.
- W4372260141 modified "2023-09-27" @default.
- W4372260141 title "M-CTRL: A Continual Representation Learning Framework with Slowly Improving Past Pre-Trained Model" @default.
- W4372260141 cites W2024490156 @default.
- W4372260141 cites W2127141656 @default.
- W4372260141 cites W2137742200 @default.
- W4372260141 cites W2143104527 @default.
- W4372260141 cites W2473930607 @default.
- W4372260141 cites W2933138175 @default.
- W4372260141 cites W2953190524 @default.
- W4372260141 cites W2973049979 @default.
- W4372260141 cites W3209059054 @default.
- W4372260141 doi "https://doi.org/10.1109/icassp49357.2023.10096793" @default.
- W4372260141 hasPublicationYear "2023" @default.
- W4372260141 type Work @default.
- W4372260141 citedByCount "0" @default.
- W4372260141 crossrefType "proceedings-article" @default.
- W4372260141 hasAuthorship W4372260141A5002418613 @default.
- W4372260141 hasAuthorship W4372260141A5012684064 @default.
- W4372260141 hasAuthorship W4372260141A5037747694 @default.
- W4372260141 hasAuthorship W4372260141A5051680529 @default.
- W4372260141 hasBestOaLocation W43722601411 @default.
- W4372260141 hasConcept C119857082 @default.
- W4372260141 hasConcept C120822770 @default.
- W4372260141 hasConcept C134306372 @default.
- W4372260141 hasConcept C154945302 @default.
- W4372260141 hasConcept C17744445 @default.
- W4372260141 hasConcept C199539241 @default.
- W4372260141 hasConcept C2776359362 @default.
- W4372260141 hasConcept C33923547 @default.
- W4372260141 hasConcept C36503486 @default.
- W4372260141 hasConcept C41008148 @default.
- W4372260141 hasConcept C50644808 @default.
- W4372260141 hasConcept C51632099 @default.
- W4372260141 hasConcept C59404180 @default.
- W4372260141 hasConcept C67186912 @default.
- W4372260141 hasConcept C77088390 @default.
- W4372260141 hasConcept C94625758 @default.
- W4372260141 hasConceptScore W4372260141C119857082 @default.
- W4372260141 hasConceptScore W4372260141C120822770 @default.
- W4372260141 hasConceptScore W4372260141C134306372 @default.
- W4372260141 hasConceptScore W4372260141C154945302 @default.
- W4372260141 hasConceptScore W4372260141C17744445 @default.
- W4372260141 hasConceptScore W4372260141C199539241 @default.
- W4372260141 hasConceptScore W4372260141C2776359362 @default.
- W4372260141 hasConceptScore W4372260141C33923547 @default.
- W4372260141 hasConceptScore W4372260141C36503486 @default.
- W4372260141 hasConceptScore W4372260141C41008148 @default.
- W4372260141 hasConceptScore W4372260141C50644808 @default.
- W4372260141 hasConceptScore W4372260141C51632099 @default.
- W4372260141 hasConceptScore W4372260141C59404180 @default.
- W4372260141 hasConceptScore W4372260141C67186912 @default.
- W4372260141 hasConceptScore W4372260141C77088390 @default.
- W4372260141 hasConceptScore W4372260141C94625758 @default.
- W4372260141 hasLocation W43722601411 @default.
- W4372260141 hasOpenAccess W4372260141 @default.
- W4372260141 hasPrimaryLocation W43722601411 @default.
- W4372260141 hasRelatedWork W2122552724 @default.
- W4372260141 hasRelatedWork W2403788517 @default.
- W4372260141 hasRelatedWork W2954428433 @default.
- W4372260141 hasRelatedWork W2984205432 @default.
- W4372260141 hasRelatedWork W3087493185 @default.
- W4372260141 hasRelatedWork W3196155444 @default.
- W4372260141 hasRelatedWork W4206762304 @default.
- W4372260141 hasRelatedWork W4237551280 @default.
- W4372260141 hasRelatedWork W4296932909 @default.
- W4372260141 hasRelatedWork W4320892944 @default.
- W4372260141 isParatext "false" @default.
- W4372260141 isRetracted "false" @default.
- W4372260141 workType "article" @default.