Matches in SemOpenAlex for { <https://semopenalex.org/work/W3025429027> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W3025429027 abstract "Autoregressive Predictive Coding (APC), as a self-supervised objective, has enjoyed success in learning representations from large amounts of unlabeled data, and the learned representations are rich for many downstream tasks. However, the connection between low self-supervised loss and strong performance in downstream tasks remains unclear. In this work, we propose Vector-Quantized Autoregressive Predictive Coding (VQ-APC), a novel model that produces quantized representations, allowing us to explicitly control the amount of information encoded in the representations. By studying a sequence of increasingly limited models, we reveal the constituents of the learned representations. In particular, we confirm the presence of information with probing tasks, while showing the absence of information with mutual information, uncovering the model's preference in preserving speech information as its capacity becomes constrained. We find that there exists a point where phonetic and speaker information are amplified to maximize a self-supervised objective. As a byproduct, the learned codes for a particular model capacity correspond well to English phones." @default.
- W3025429027 created "2020-05-21" @default.
- W3025429027 creator A5008107321 @default.
- W3025429027 creator A5030898980 @default.
- W3025429027 creator A5039081803 @default.
- W3025429027 date "2020-05-17" @default.
- W3025429027 modified "2023-10-14" @default.
- W3025429027 title "Vector-Quantized Autoregressive Predictive Coding" @default.
- W3025429027 cites W1494198834 @default.
- W3025429027 cites W2024490156 @default.
- W3025429027 cites W2842511635 @default.
- W3025429027 cites W2885318751 @default.
- W3025429027 cites W2962739339 @default.
- W3025429027 cites W2963045354 @default.
- W3025429027 cites W2963341956 @default.
- W3025429027 cites W2963403868 @default.
- W3025429027 cites W2963799213 @default.
- W3025429027 cites W2964121744 @default.
- W3025429027 cites W2972943112 @default.
- W3025429027 cites W2973049979 @default.
- W3025429027 cites W2973157397 @default.
- W3025429027 cites W2982223350 @default.
- W3025429027 cites W2988736778 @default.
- W3025429027 cites W2995680346 @default.
- W3025429027 cites W2996383576 @default.
- W3025429027 cites W3003875258 @default.
- W3025429027 cites W3015213852 @default.
- W3025429027 cites W3015265920 @default.
- W3025429027 cites W3015419784 @default.
- W3025429027 cites W3016011332 @default.
- W3025429027 cites W3035202887 @default.
- W3025429027 cites W3125709657 @default.
- W3025429027 doi "https://doi.org/10.48550/arxiv.2005.08392" @default.
- W3025429027 hasPublicationYear "2020" @default.
- W3025429027 type Work @default.
- W3025429027 sameAs 3025429027 @default.
- W3025429027 citedByCount "3" @default.
- W3025429027 countsByYear W30254290272020 @default.
- W3025429027 countsByYear W30254290272021 @default.
- W3025429027 crossrefType "posted-content" @default.
- W3025429027 hasAuthorship W3025429027A5008107321 @default.
- W3025429027 hasAuthorship W3025429027A5030898980 @default.
- W3025429027 hasAuthorship W3025429027A5039081803 @default.
- W3025429027 hasBestOaLocation W30254290271 @default.
- W3025429027 hasConcept C105795698 @default.
- W3025429027 hasConcept C119857082 @default.
- W3025429027 hasConcept C152139883 @default.
- W3025429027 hasConcept C154945302 @default.
- W3025429027 hasConcept C159877910 @default.
- W3025429027 hasConcept C162324750 @default.
- W3025429027 hasConcept C179518139 @default.
- W3025429027 hasConcept C21547014 @default.
- W3025429027 hasConcept C2776207758 @default.
- W3025429027 hasConcept C2778061373 @default.
- W3025429027 hasConcept C28490314 @default.
- W3025429027 hasConcept C33923547 @default.
- W3025429027 hasConcept C41008148 @default.
- W3025429027 hasConcept C52622258 @default.
- W3025429027 hasConceptScore W3025429027C105795698 @default.
- W3025429027 hasConceptScore W3025429027C119857082 @default.
- W3025429027 hasConceptScore W3025429027C152139883 @default.
- W3025429027 hasConceptScore W3025429027C154945302 @default.
- W3025429027 hasConceptScore W3025429027C159877910 @default.
- W3025429027 hasConceptScore W3025429027C162324750 @default.
- W3025429027 hasConceptScore W3025429027C179518139 @default.
- W3025429027 hasConceptScore W3025429027C21547014 @default.
- W3025429027 hasConceptScore W3025429027C2776207758 @default.
- W3025429027 hasConceptScore W3025429027C2778061373 @default.
- W3025429027 hasConceptScore W3025429027C28490314 @default.
- W3025429027 hasConceptScore W3025429027C33923547 @default.
- W3025429027 hasConceptScore W3025429027C41008148 @default.
- W3025429027 hasConceptScore W3025429027C52622258 @default.
- W3025429027 hasLocation W30254290271 @default.
- W3025429027 hasOpenAccess W3025429027 @default.
- W3025429027 hasPrimaryLocation W30254290271 @default.
- W3025429027 hasRelatedWork W1601361876 @default.
- W3025429027 hasRelatedWork W1622610421 @default.
- W3025429027 hasRelatedWork W1998224536 @default.
- W3025429027 hasRelatedWork W2078003659 @default.
- W3025429027 hasRelatedWork W2139048253 @default.
- W3025429027 hasRelatedWork W2150119286 @default.
- W3025429027 hasRelatedWork W3025429027 @default.
- W3025429027 hasRelatedWork W3097286738 @default.
- W3025429027 hasRelatedWork W3198884251 @default.
- W3025429027 hasRelatedWork W93423440 @default.
- W3025429027 isParatext "false" @default.
- W3025429027 isRetracted "false" @default.
- W3025429027 magId "3025429027" @default.
- W3025429027 workType "article" @default.