Matches in SemOpenAlex for { <https://semopenalex.org/work/W4365460601> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W4365460601 abstract "Audiovisual representation learning typically relies on the correspondence between sight and sound. However, there are often multiple audio tracks that can correspond with a visual scene. Consider, for example, different conversations on the same crowded street. The effect of such counterfactual pairs on audiovisual representation learning has not been previously explored. To investigate this, we use dubbed versions of movies to augment cross-modal contrastive learning. Our approach learns to represent alternate audio tracks, differing only in speech content, similarly to the same video. Our results show that dub-augmented training improves performance on a range of auditory and audiovisual tasks, without significantly affecting linguistic task performance overall. We additionally compare this approach to a strong baseline where we remove speech before pretraining, and find that dub-augmented training is more effective, including for paralinguistic and audiovisual tasks where speech removal leads to worse performance. These findings highlight the importance of considering speech variation when learning scene-level audiovisual correspondences and suggest that dubbed audio can be a useful augmentation technique for training audiovisual models toward more robust performance." @default.
- W4365460601 created "2023-04-15" @default.
- W4365460601 creator A5016069022 @default.
- W4365460601 creator A5023992316 @default.
- W4365460601 creator A5047748210 @default.
- W4365460601 creator A5069969107 @default.
- W4365460601 date "2023-04-12" @default.
- W4365460601 modified "2023-09-28" @default.
- W4365460601 title "Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning" @default.
- W4365460601 doi "https://doi.org/10.48550/arxiv.2304.05600" @default.
- W4365460601 hasPublicationYear "2023" @default.
- W4365460601 type Work @default.
- W4365460601 citedByCount "0" @default.
- W4365460601 crossrefType "posted-content" @default.
- W4365460601 hasAuthorship W4365460601A5016069022 @default.
- W4365460601 hasAuthorship W4365460601A5023992316 @default.
- W4365460601 hasAuthorship W4365460601A5047748210 @default.
- W4365460601 hasAuthorship W4365460601A5069969107 @default.
- W4365460601 hasBestOaLocation W43654606011 @default.
- W4365460601 hasConcept C108650721 @default.
- W4365460601 hasConcept C121332964 @default.
- W4365460601 hasConcept C133378560 @default.
- W4365460601 hasConcept C154945302 @default.
- W4365460601 hasConcept C15744967 @default.
- W4365460601 hasConcept C162324750 @default.
- W4365460601 hasConcept C169760540 @default.
- W4365460601 hasConcept C17744445 @default.
- W4365460601 hasConcept C178253425 @default.
- W4365460601 hasConcept C185592680 @default.
- W4365460601 hasConcept C187736073 @default.
- W4365460601 hasConcept C188027245 @default.
- W4365460601 hasConcept C199539241 @default.
- W4365460601 hasConcept C26760741 @default.
- W4365460601 hasConcept C2776359362 @default.
- W4365460601 hasConcept C2776502983 @default.
- W4365460601 hasConcept C2778334786 @default.
- W4365460601 hasConcept C2780451532 @default.
- W4365460601 hasConcept C28490314 @default.
- W4365460601 hasConcept C41008148 @default.
- W4365460601 hasConcept C44870925 @default.
- W4365460601 hasConcept C46312422 @default.
- W4365460601 hasConcept C59404180 @default.
- W4365460601 hasConcept C60115397 @default.
- W4365460601 hasConcept C71139939 @default.
- W4365460601 hasConcept C77805123 @default.
- W4365460601 hasConcept C94625758 @default.
- W4365460601 hasConceptScore W4365460601C108650721 @default.
- W4365460601 hasConceptScore W4365460601C121332964 @default.
- W4365460601 hasConceptScore W4365460601C133378560 @default.
- W4365460601 hasConceptScore W4365460601C154945302 @default.
- W4365460601 hasConceptScore W4365460601C15744967 @default.
- W4365460601 hasConceptScore W4365460601C162324750 @default.
- W4365460601 hasConceptScore W4365460601C169760540 @default.
- W4365460601 hasConceptScore W4365460601C17744445 @default.
- W4365460601 hasConceptScore W4365460601C178253425 @default.
- W4365460601 hasConceptScore W4365460601C185592680 @default.
- W4365460601 hasConceptScore W4365460601C187736073 @default.
- W4365460601 hasConceptScore W4365460601C188027245 @default.
- W4365460601 hasConceptScore W4365460601C199539241 @default.
- W4365460601 hasConceptScore W4365460601C26760741 @default.
- W4365460601 hasConceptScore W4365460601C2776359362 @default.
- W4365460601 hasConceptScore W4365460601C2776502983 @default.
- W4365460601 hasConceptScore W4365460601C2778334786 @default.
- W4365460601 hasConceptScore W4365460601C2780451532 @default.
- W4365460601 hasConceptScore W4365460601C28490314 @default.
- W4365460601 hasConceptScore W4365460601C41008148 @default.
- W4365460601 hasConceptScore W4365460601C44870925 @default.
- W4365460601 hasConceptScore W4365460601C46312422 @default.
- W4365460601 hasConceptScore W4365460601C59404180 @default.
- W4365460601 hasConceptScore W4365460601C60115397 @default.
- W4365460601 hasConceptScore W4365460601C71139939 @default.
- W4365460601 hasConceptScore W4365460601C77805123 @default.
- W4365460601 hasConceptScore W4365460601C94625758 @default.
- W4365460601 hasLocation W43654606011 @default.
- W4365460601 hasOpenAccess W4365460601 @default.
- W4365460601 hasPrimaryLocation W43654606011 @default.
- W4365460601 hasRelatedWork W2346634343 @default.
- W4365460601 hasRelatedWork W2903184186 @default.
- W4365460601 hasRelatedWork W2952745240 @default.
- W4365460601 hasRelatedWork W2979626747 @default.
- W4365460601 hasRelatedWork W3114848507 @default.
- W4365460601 hasRelatedWork W3150003370 @default.
- W4365460601 hasRelatedWork W3196778703 @default.
- W4365460601 hasRelatedWork W4289147925 @default.
- W4365460601 hasRelatedWork W4301143707 @default.
- W4365460601 hasRelatedWork W4362598752 @default.
- W4365460601 isParatext "false" @default.
- W4365460601 isRetracted "false" @default.
- W4365460601 workType "article" @default.