Matches in SemOpenAlex for { <https://semopenalex.org/work/W3168481568> ?p ?o ?g. }
- W3168481568 abstract "Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. However, much of this work only relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages, which is loose and implicit for aligning the contextual representations between languages. In this paper, we plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. More importantly, when fine-tuning on downstream tasks, the cross-attention module can be plugged in or out on-demand, thus naturally benefiting a wider range of cross-lingual tasks, from language understanding to generation. As a result, the proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark, covering text classification, sequence labeling, question answering, and sentence retrieval. For cross-lingual generation tasks, it also outperforms all existing cross-lingual models and state-of-the-art Transformer variants on WMT14 English-to-German and English-to-French translation datasets, with gains of up to 1~2 BLEU." @default.
- W3168481568 created "2021-06-22" @default.
- W3168481568 creator A5013292507 @default.
- W3168481568 creator A5019056615 @default.
- W3168481568 creator A5019525050 @default.
- W3168481568 creator A5046597133 @default.
- W3168481568 creator A5046872618 @default.
- W3168481568 creator A5047856952 @default.
- W3168481568 creator A5051080513 @default.
- W3168481568 creator A5081108106 @default.
- W3168481568 date "2020-10-29" @default.
- W3168481568 modified "2023-09-27" @default.
- W3168481568 title "VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation" @default.
- W3168481568 cites W2194775991 @default.
- W3168481568 cites W2626778328 @default.
- W3168481568 cites W2742155240 @default.
- W3168481568 cites W2792760996 @default.
- W3168481568 cites W2805003733 @default.
- W3168481568 cites W2891555348 @default.
- W3168481568 cites W2903810591 @default.
- W3168481568 cites W2914120296 @default.
- W3168481568 cites W2915977242 @default.
- W3168481568 cites W2945260553 @default.
- W3168481568 cites W2953109491 @default.
- W3168481568 cites W2963250244 @default.
- W3168481568 cites W2963341956 @default.
- W3168481568 cites W2963532001 @default.
- W3168481568 cites W2965373594 @default.
- W3168481568 cites W2971863715 @default.
- W3168481568 cites W2982399380 @default.
- W3168481568 cites W2983040767 @default.
- W3168481568 cites W2989539713 @default.
- W3168481568 cites W2996822578 @default.
- W3168481568 cites W2998653236 @default.
- W3168481568 cites W3001434439 @default.
- W3168481568 cites W3010768098 @default.
- W3168481568 cites W3013571468 @default.
- W3168481568 cites W3013840636 @default.
- W3168481568 cites W3014635508 @default.
- W3168481568 cites W3016512233 @default.
- W3168481568 cites W3034469191 @default.
- W3168481568 cites W3035497479 @default.
- W3168481568 cites W3042711927 @default.
- W3168481568 cites W3045462440 @default.
- W3168481568 cites W3066373881 @default.
- W3168481568 cites W3085479580 @default.
- W3168481568 cites W3092327118 @default.
- W3168481568 cites W3102483398 @default.
- W3168481568 cites W3107668149 @default.
- W3168481568 doi "https://doi.org/10.48550/arxiv.2010.16046" @default.
- W3168481568 hasPublicationYear "2020" @default.
- W3168481568 type Work @default.
- W3168481568 sameAs 3168481568 @default.
- W3168481568 citedByCount "12" @default.
- W3168481568 countsByYear W31684815682020 @default.
- W3168481568 countsByYear W31684815682021 @default.
- W3168481568 countsByYear W31684815682022 @default.
- W3168481568 crossrefType "posted-content" @default.
- W3168481568 hasAuthorship W3168481568A5013292507 @default.
- W3168481568 hasAuthorship W3168481568A5019056615 @default.
- W3168481568 hasAuthorship W3168481568A5019525050 @default.
- W3168481568 hasAuthorship W3168481568A5046597133 @default.
- W3168481568 hasAuthorship W3168481568A5046872618 @default.
- W3168481568 hasAuthorship W3168481568A5047856952 @default.
- W3168481568 hasAuthorship W3168481568A5051080513 @default.
- W3168481568 hasAuthorship W3168481568A5081108106 @default.
- W3168481568 hasBestOaLocation W31684815681 @default.
- W3168481568 hasConcept C111919701 @default.
- W3168481568 hasConcept C118505674 @default.
- W3168481568 hasConcept C121332964 @default.
- W3168481568 hasConcept C137293760 @default.
- W3168481568 hasConcept C138885662 @default.
- W3168481568 hasConcept C154945302 @default.
- W3168481568 hasConcept C165801399 @default.
- W3168481568 hasConcept C203005215 @default.
- W3168481568 hasConcept C204321447 @default.
- W3168481568 hasConcept C2777530160 @default.
- W3168481568 hasConcept C2777601683 @default.
- W3168481568 hasConcept C41008148 @default.
- W3168481568 hasConcept C41895202 @default.
- W3168481568 hasConcept C62520636 @default.
- W3168481568 hasConcept C66322947 @default.
- W3168481568 hasConceptScore W3168481568C111919701 @default.
- W3168481568 hasConceptScore W3168481568C118505674 @default.
- W3168481568 hasConceptScore W3168481568C121332964 @default.
- W3168481568 hasConceptScore W3168481568C137293760 @default.
- W3168481568 hasConceptScore W3168481568C138885662 @default.
- W3168481568 hasConceptScore W3168481568C154945302 @default.
- W3168481568 hasConceptScore W3168481568C165801399 @default.
- W3168481568 hasConceptScore W3168481568C203005215 @default.
- W3168481568 hasConceptScore W3168481568C204321447 @default.
- W3168481568 hasConceptScore W3168481568C2777530160 @default.
- W3168481568 hasConceptScore W3168481568C2777601683 @default.
- W3168481568 hasConceptScore W3168481568C41008148 @default.
- W3168481568 hasConceptScore W3168481568C41895202 @default.
- W3168481568 hasConceptScore W3168481568C62520636 @default.
- W3168481568 hasConceptScore W3168481568C66322947 @default.
- W3168481568 hasLocation W31684815681 @default.
- W3168481568 hasOpenAccess W3168481568 @default.
- W3168481568 hasPrimaryLocation W31684815681 @default.