Matches in SemOpenAlex for { <https://semopenalex.org/work/W4380994056> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4380994056 abstract "The excellent generalization ability of self-supervised learning (SSL) for speech foundation models has garnered significant attention. HuBERT is a successful example that utilizes offline clustering to convert speech features into discrete units for a masked language modeling pretext task. However, simply clustering features as targets by k-means does not fully inspire the model's performance. In this work, we present an unsupervised method to improve SSL targets. Two models are proposed, MonoBERT and PolyBERT, which leverage context-independent and context-dependent phoneme-based units for pre-training. Our models outperform other SSL models significantly on the LibriSpeech benchmark without the need for iterative re-clustering and re-training. Furthermore, our models equipped with context-dependent units even outperform target-improvement models that use labeled data during pre-training. How we progressively improve the unit discovery process is demonstrated through experiments." @default.
- W4380994056 created "2023-06-17" @default.
- W4380994056 creator A5007473150 @default.
- W4380994056 creator A5068357345 @default.
- W4380994056 creator A5074750047 @default.
- W4380994056 creator A5074951662 @default.
- W4380994056 creator A5079594267 @default.
- W4380994056 creator A5089966579 @default.
- W4380994056 date "2023-06-15" @default.
- W4380994056 modified "2023-09-24" @default.
- W4380994056 title "Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representation" @default.
- W4380994056 doi "https://doi.org/10.48550/arxiv.2306.08920" @default.
- W4380994056 hasPublicationYear "2023" @default.
- W4380994056 type Work @default.
- W4380994056 citedByCount "0" @default.
- W4380994056 crossrefType "posted-content" @default.
- W4380994056 hasAuthorship W4380994056A5007473150 @default.
- W4380994056 hasAuthorship W4380994056A5068357345 @default.
- W4380994056 hasAuthorship W4380994056A5074750047 @default.
- W4380994056 hasAuthorship W4380994056A5074951662 @default.
- W4380994056 hasAuthorship W4380994056A5079594267 @default.
- W4380994056 hasAuthorship W4380994056A5089966579 @default.
- W4380994056 hasBestOaLocation W43809940561 @default.
- W4380994056 hasConcept C119857082 @default.
- W4380994056 hasConcept C13280743 @default.
- W4380994056 hasConcept C134306372 @default.
- W4380994056 hasConcept C137293760 @default.
- W4380994056 hasConcept C151730666 @default.
- W4380994056 hasConcept C153083717 @default.
- W4380994056 hasConcept C154945302 @default.
- W4380994056 hasConcept C177148314 @default.
- W4380994056 hasConcept C185798385 @default.
- W4380994056 hasConcept C205649164 @default.
- W4380994056 hasConcept C2779343474 @default.
- W4380994056 hasConcept C33923547 @default.
- W4380994056 hasConcept C41008148 @default.
- W4380994056 hasConcept C73555534 @default.
- W4380994056 hasConcept C8038995 @default.
- W4380994056 hasConcept C86803240 @default.
- W4380994056 hasConceptScore W4380994056C119857082 @default.
- W4380994056 hasConceptScore W4380994056C13280743 @default.
- W4380994056 hasConceptScore W4380994056C134306372 @default.
- W4380994056 hasConceptScore W4380994056C137293760 @default.
- W4380994056 hasConceptScore W4380994056C151730666 @default.
- W4380994056 hasConceptScore W4380994056C153083717 @default.
- W4380994056 hasConceptScore W4380994056C154945302 @default.
- W4380994056 hasConceptScore W4380994056C177148314 @default.
- W4380994056 hasConceptScore W4380994056C185798385 @default.
- W4380994056 hasConceptScore W4380994056C205649164 @default.
- W4380994056 hasConceptScore W4380994056C2779343474 @default.
- W4380994056 hasConceptScore W4380994056C33923547 @default.
- W4380994056 hasConceptScore W4380994056C41008148 @default.
- W4380994056 hasConceptScore W4380994056C73555534 @default.
- W4380994056 hasConceptScore W4380994056C8038995 @default.
- W4380994056 hasConceptScore W4380994056C86803240 @default.
- W4380994056 hasLocation W43809940561 @default.
- W4380994056 hasOpenAccess W4380994056 @default.
- W4380994056 hasPrimaryLocation W43809940561 @default.
- W4380994056 hasRelatedWork W3046775127 @default.
- W4380994056 hasRelatedWork W3087576162 @default.
- W4380994056 hasRelatedWork W3196155444 @default.
- W4380994056 hasRelatedWork W3209574120 @default.
- W4380994056 hasRelatedWork W3210156800 @default.
- W4380994056 hasRelatedWork W4221150964 @default.
- W4380994056 hasRelatedWork W4285260836 @default.
- W4380994056 hasRelatedWork W4287665842 @default.
- W4380994056 hasRelatedWork W4312365371 @default.
- W4380994056 hasRelatedWork W4312903001 @default.
- W4380994056 isParatext "false" @default.
- W4380994056 isRetracted "false" @default.
- W4380994056 workType "article" @default.