Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571004> ?p ?o ?g. }
Showing items 1 to 73 of
73
with 100 items per page.
- W4385571004 abstract "How to achieve better end-to-end speech translation (ST) by leveraging (text) machine translation (MT) data? Among various existing techniques, multi-task learning is one of the effective ways to share knowledge between ST and MT in which additional MT data can help to learn source-to-target mapping. However, due to the differences between speech and text, there is always a gap between ST and MT. In this paper, we first aim to understand this modality gap from the target-side representation differences, and link the modality gap to another well-known problem in neural machine translation: exposure bias. We find that the modality gap is relatively small during training except for some difficult cases, but keeps increasing during inference due to the cascading effect. To address these problems, we propose the Cross-modal Regularization with Scheduled Sampling (Cress) method. Specifically, we regularize the output predictions of ST and MT, whose target-side contexts are derived by sampling between ground truth words and self-generated words with a varying probability. Furthermore, we introduce token-level adaptive training which assigns different training weights to target tokens to handle difficult cases with large modality gaps. Experiments and analysis show that our approach effectively bridges the modality gap, and achieves significant improvements over a strong baseline in all eight directions of the MuST-C dataset." @default.
- W4385571004 created "2023-08-05" @default.
- W4385571004 creator A5075806509 @default.
- W4385571004 creator A5078940234 @default.
- W4385571004 date "2023-01-01" @default.
- W4385571004 modified "2023-09-24" @default.
- W4385571004 title "Understanding and Bridging the Modality Gap for Speech Translation" @default.
- W4385571004 doi "https://doi.org/10.18653/v1/2023.acl-long.884" @default.
- W4385571004 hasPublicationYear "2023" @default.
- W4385571004 type Work @default.
- W4385571004 citedByCount "0" @default.
- W4385571004 crossrefType "proceedings-article" @default.
- W4385571004 hasAuthorship W4385571004A5075806509 @default.
- W4385571004 hasAuthorship W4385571004A5078940234 @default.
- W4385571004 hasBestOaLocation W43855710041 @default.
- W4385571004 hasConcept C104317684 @default.
- W4385571004 hasConcept C105580179 @default.
- W4385571004 hasConcept C119857082 @default.
- W4385571004 hasConcept C146849305 @default.
- W4385571004 hasConcept C149364088 @default.
- W4385571004 hasConcept C154945302 @default.
- W4385571004 hasConcept C174348530 @default.
- W4385571004 hasConcept C185592680 @default.
- W4385571004 hasConcept C203005215 @default.
- W4385571004 hasConcept C204321447 @default.
- W4385571004 hasConcept C2776135515 @default.
- W4385571004 hasConcept C2776214188 @default.
- W4385571004 hasConcept C2780226545 @default.
- W4385571004 hasConcept C28490314 @default.
- W4385571004 hasConcept C2984842247 @default.
- W4385571004 hasConcept C31258907 @default.
- W4385571004 hasConcept C38652104 @default.
- W4385571004 hasConcept C41008148 @default.
- W4385571004 hasConcept C48145219 @default.
- W4385571004 hasConcept C50644808 @default.
- W4385571004 hasConcept C55493867 @default.
- W4385571004 hasConceptScore W4385571004C104317684 @default.
- W4385571004 hasConceptScore W4385571004C105580179 @default.
- W4385571004 hasConceptScore W4385571004C119857082 @default.
- W4385571004 hasConceptScore W4385571004C146849305 @default.
- W4385571004 hasConceptScore W4385571004C149364088 @default.
- W4385571004 hasConceptScore W4385571004C154945302 @default.
- W4385571004 hasConceptScore W4385571004C174348530 @default.
- W4385571004 hasConceptScore W4385571004C185592680 @default.
- W4385571004 hasConceptScore W4385571004C203005215 @default.
- W4385571004 hasConceptScore W4385571004C204321447 @default.
- W4385571004 hasConceptScore W4385571004C2776135515 @default.
- W4385571004 hasConceptScore W4385571004C2776214188 @default.
- W4385571004 hasConceptScore W4385571004C2780226545 @default.
- W4385571004 hasConceptScore W4385571004C28490314 @default.
- W4385571004 hasConceptScore W4385571004C2984842247 @default.
- W4385571004 hasConceptScore W4385571004C31258907 @default.
- W4385571004 hasConceptScore W4385571004C38652104 @default.
- W4385571004 hasConceptScore W4385571004C41008148 @default.
- W4385571004 hasConceptScore W4385571004C48145219 @default.
- W4385571004 hasConceptScore W4385571004C50644808 @default.
- W4385571004 hasConceptScore W4385571004C55493867 @default.
- W4385571004 hasLocation W43855710041 @default.
- W4385571004 hasOpenAccess W4385571004 @default.
- W4385571004 hasPrimaryLocation W43855710041 @default.
- W4385571004 hasRelatedWork W1484029852 @default.
- W4385571004 hasRelatedWork W1512718085 @default.
- W4385571004 hasRelatedWork W2167662847 @default.
- W4385571004 hasRelatedWork W2917049097 @default.
- W4385571004 hasRelatedWork W2999635570 @default.
- W4385571004 hasRelatedWork W3195009982 @default.
- W4385571004 hasRelatedWork W4205476602 @default.
- W4385571004 hasRelatedWork W4287021360 @default.
- W4385571004 hasRelatedWork W4379525811 @default.
- W4385571004 hasRelatedWork W2610387714 @default.
- W4385571004 isParatext "false" @default.
- W4385571004 isRetracted "false" @default.
- W4385571004 workType "article" @default.