Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378513185> ?p ?o ?g. }
Showing items 1 to 93 of
93
with 100 items per page.
- W4378513185 abstract "Multi-modal Contrastive Representation (MCR) learning aims to encode different modalities into a semantically aligned shared space. This paradigm shows remarkable generalization ability on numerous downstream tasks across various modalities. However, the reliance on massive high-quality data pairs limits its further development on more modalities. This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR). Specifically, given two existing MCRs pre-trained on (A, B) and (B, C) modality pairs, we project them to a new space and use the data from the overlapping modality B to aligning the two MCRs in the new space. Meanwhile, since the modality pairs (A, B) and (B, C) are already aligned within each MCR, the connection learned by overlapping modality can also be transferred to non-overlapping modality pair (A, C). To unleash the potential of C-MCR, we further introduce a semantic-enhanced inter- and intra-MCR connection method. We first enhance the semantic consistency and completion of embeddings across different modalities for more robust alignment. Then we utilize the inter-MCR alignment to establish the connection, and employ the intra-MCR alignment to better maintain the connection for inputs from non-overlapping modalities. We take the field of audio-visual contrastive learning as an example to demonstrate the effectiveness of C-MCR. We connect pre-trained CLIP and CLAP models via texts to derive audio-visual contrastive representations. Remarkably, without using any paired audio-visual data and further tuning, C-MCR achieves state-of-the-art performance on six datasets across three audio-visual downstream tasks." @default.
- W4378513185 created "2023-05-28" @default.
- W4378513185 creator A5004658590 @default.
- W4378513185 creator A5007505729 @default.
- W4378513185 creator A5009897266 @default.
- W4378513185 creator A5012728201 @default.
- W4378513185 creator A5016950354 @default.
- W4378513185 creator A5020551506 @default.
- W4378513185 creator A5028127027 @default.
- W4378513185 creator A5033355998 @default.
- W4378513185 creator A5065361552 @default.
- W4378513185 creator A5068548095 @default.
- W4378513185 creator A5077059859 @default.
- W4378513185 date "2023-05-22" @default.
- W4378513185 modified "2023-09-30" @default.
- W4378513185 title "Connecting Multi-modal Contrastive Representations" @default.
- W4378513185 doi "https://doi.org/10.48550/arxiv.2305.14381" @default.
- W4378513185 hasPublicationYear "2023" @default.
- W4378513185 type Work @default.
- W4378513185 citedByCount "0" @default.
- W4378513185 crossrefType "posted-content" @default.
- W4378513185 hasAuthorship W4378513185A5004658590 @default.
- W4378513185 hasAuthorship W4378513185A5007505729 @default.
- W4378513185 hasAuthorship W4378513185A5009897266 @default.
- W4378513185 hasAuthorship W4378513185A5012728201 @default.
- W4378513185 hasAuthorship W4378513185A5016950354 @default.
- W4378513185 hasAuthorship W4378513185A5020551506 @default.
- W4378513185 hasAuthorship W4378513185A5028127027 @default.
- W4378513185 hasAuthorship W4378513185A5033355998 @default.
- W4378513185 hasAuthorship W4378513185A5065361552 @default.
- W4378513185 hasAuthorship W4378513185A5068548095 @default.
- W4378513185 hasAuthorship W4378513185A5077059859 @default.
- W4378513185 hasBestOaLocation W43785131851 @default.
- W4378513185 hasConcept C111919701 @default.
- W4378513185 hasConcept C13355873 @default.
- W4378513185 hasConcept C134306372 @default.
- W4378513185 hasConcept C144024400 @default.
- W4378513185 hasConcept C154945302 @default.
- W4378513185 hasConcept C177148314 @default.
- W4378513185 hasConcept C17744445 @default.
- W4378513185 hasConcept C185592680 @default.
- W4378513185 hasConcept C188027245 @default.
- W4378513185 hasConcept C199539241 @default.
- W4378513185 hasConcept C204321447 @default.
- W4378513185 hasConcept C2524010 @default.
- W4378513185 hasConcept C2776359362 @default.
- W4378513185 hasConcept C2776436953 @default.
- W4378513185 hasConcept C2778572836 @default.
- W4378513185 hasConcept C2779903281 @default.
- W4378513185 hasConcept C2780226545 @default.
- W4378513185 hasConcept C33923547 @default.
- W4378513185 hasConcept C36289849 @default.
- W4378513185 hasConcept C41008148 @default.
- W4378513185 hasConcept C71139939 @default.
- W4378513185 hasConcept C94625758 @default.
- W4378513185 hasConceptScore W4378513185C111919701 @default.
- W4378513185 hasConceptScore W4378513185C13355873 @default.
- W4378513185 hasConceptScore W4378513185C134306372 @default.
- W4378513185 hasConceptScore W4378513185C144024400 @default.
- W4378513185 hasConceptScore W4378513185C154945302 @default.
- W4378513185 hasConceptScore W4378513185C177148314 @default.
- W4378513185 hasConceptScore W4378513185C17744445 @default.
- W4378513185 hasConceptScore W4378513185C185592680 @default.
- W4378513185 hasConceptScore W4378513185C188027245 @default.
- W4378513185 hasConceptScore W4378513185C199539241 @default.
- W4378513185 hasConceptScore W4378513185C204321447 @default.
- W4378513185 hasConceptScore W4378513185C2524010 @default.
- W4378513185 hasConceptScore W4378513185C2776359362 @default.
- W4378513185 hasConceptScore W4378513185C2776436953 @default.
- W4378513185 hasConceptScore W4378513185C2778572836 @default.
- W4378513185 hasConceptScore W4378513185C2779903281 @default.
- W4378513185 hasConceptScore W4378513185C2780226545 @default.
- W4378513185 hasConceptScore W4378513185C33923547 @default.
- W4378513185 hasConceptScore W4378513185C36289849 @default.
- W4378513185 hasConceptScore W4378513185C41008148 @default.
- W4378513185 hasConceptScore W4378513185C71139939 @default.
- W4378513185 hasConceptScore W4378513185C94625758 @default.
- W4378513185 hasLocation W43785131851 @default.
- W4378513185 hasOpenAccess W4378513185 @default.
- W4378513185 hasPrimaryLocation W43785131851 @default.
- W4378513185 hasRelatedWork W2096647984 @default.
- W4378513185 hasRelatedWork W2474574787 @default.
- W4378513185 hasRelatedWork W2546190447 @default.
- W4378513185 hasRelatedWork W2949074159 @default.
- W4378513185 hasRelatedWork W2952745240 @default.
- W4378513185 hasRelatedWork W2959445501 @default.
- W4378513185 hasRelatedWork W3211385060 @default.
- W4378513185 hasRelatedWork W4298715519 @default.
- W4378513185 hasRelatedWork W4301143707 @default.
- W4378513185 hasRelatedWork W4328007250 @default.
- W4378513185 isParatext "false" @default.
- W4378513185 isRetracted "false" @default.
- W4378513185 workType "article" @default.