Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571644> ?p ?o ?g. }
Showing items 1 to 77 of
77
with 100 items per page.
- W4385571644 abstract "Without any explicit cross-lingual training data, multilingual language models can achieve cross-lingual transfer. One common way to improve this transfer is to perform realignment steps before fine-tuning, i.e., to train the model to build similar representations for pairs of words from translated sentences. But such realignment methods were found to not always improve results across languages and tasks, which raises the question of whether aligned representations are truly beneficial for cross-lingual transfer. We provide evidence that alignment is actually significantly correlated with cross-lingual transfer across languages, models and random seeds. We show that fine-tuning can have a significant impact on alignment, depending mainly on the downstream task and the model. Finally, we show that realignment can, in some instances, improve cross-lingual transfer, and we identify conditions in which realignment methods provide significant improvements. Namely, we find that realignment works better on tasks for which alignment is correlated with cross-lingual transfer when generalizing to a distant language and with smaller models, as well as when using a bilingual dictionary rather than FastAlign to extract realignment pairs. For example, for POS-tagging, between English and Arabic, realignment can bring a +15.8 accuracy improvement on distilmBERT, even outperforming XLM-R Large by 1.7. We thus advocate for further research on realignment methods for smaller multilingual models as an alternative to scaling." @default.
- W4385571644 created "2023-08-05" @default.
- W4385571644 creator A5032555732 @default.
- W4385571644 creator A5063951580 @default.
- W4385571644 creator A5078015538 @default.
- W4385571644 creator A5086067014 @default.
- W4385571644 date "2023-01-01" @default.
- W4385571644 modified "2023-09-24" @default.
- W4385571644 title "Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers" @default.
- W4385571644 doi "https://doi.org/10.18653/v1/2023.findings-acl.189" @default.
- W4385571644 hasPublicationYear "2023" @default.
- W4385571644 type Work @default.
- W4385571644 citedByCount "0" @default.
- W4385571644 crossrefType "proceedings-article" @default.
- W4385571644 hasAuthorship W4385571644A5032555732 @default.
- W4385571644 hasAuthorship W4385571644A5063951580 @default.
- W4385571644 hasAuthorship W4385571644A5078015538 @default.
- W4385571644 hasAuthorship W4385571644A5086067014 @default.
- W4385571644 hasBestOaLocation W43855716441 @default.
- W4385571644 hasConcept C121332964 @default.
- W4385571644 hasConcept C137293760 @default.
- W4385571644 hasConcept C138885662 @default.
- W4385571644 hasConcept C150899416 @default.
- W4385571644 hasConcept C154945302 @default.
- W4385571644 hasConcept C162324750 @default.
- W4385571644 hasConcept C165801399 @default.
- W4385571644 hasConcept C173608175 @default.
- W4385571644 hasConcept C187736073 @default.
- W4385571644 hasConcept C203005215 @default.
- W4385571644 hasConcept C204321447 @default.
- W4385571644 hasConcept C2524010 @default.
- W4385571644 hasConcept C2776175482 @default.
- W4385571644 hasConcept C2780451532 @default.
- W4385571644 hasConcept C33923547 @default.
- W4385571644 hasConcept C41008148 @default.
- W4385571644 hasConcept C41895202 @default.
- W4385571644 hasConcept C62520636 @default.
- W4385571644 hasConcept C66322947 @default.
- W4385571644 hasConcept C96455323 @default.
- W4385571644 hasConcept C99844830 @default.
- W4385571644 hasConceptScore W4385571644C121332964 @default.
- W4385571644 hasConceptScore W4385571644C137293760 @default.
- W4385571644 hasConceptScore W4385571644C138885662 @default.
- W4385571644 hasConceptScore W4385571644C150899416 @default.
- W4385571644 hasConceptScore W4385571644C154945302 @default.
- W4385571644 hasConceptScore W4385571644C162324750 @default.
- W4385571644 hasConceptScore W4385571644C165801399 @default.
- W4385571644 hasConceptScore W4385571644C173608175 @default.
- W4385571644 hasConceptScore W4385571644C187736073 @default.
- W4385571644 hasConceptScore W4385571644C203005215 @default.
- W4385571644 hasConceptScore W4385571644C204321447 @default.
- W4385571644 hasConceptScore W4385571644C2524010 @default.
- W4385571644 hasConceptScore W4385571644C2776175482 @default.
- W4385571644 hasConceptScore W4385571644C2780451532 @default.
- W4385571644 hasConceptScore W4385571644C33923547 @default.
- W4385571644 hasConceptScore W4385571644C41008148 @default.
- W4385571644 hasConceptScore W4385571644C41895202 @default.
- W4385571644 hasConceptScore W4385571644C62520636 @default.
- W4385571644 hasConceptScore W4385571644C66322947 @default.
- W4385571644 hasConceptScore W4385571644C96455323 @default.
- W4385571644 hasConceptScore W4385571644C99844830 @default.
- W4385571644 hasLocation W43855716441 @default.
- W4385571644 hasOpenAccess W4385571644 @default.
- W4385571644 hasPrimaryLocation W43855716441 @default.
- W4385571644 hasRelatedWork W1512718085 @default.
- W4385571644 hasRelatedWork W2166245164 @default.
- W4385571644 hasRelatedWork W3025271737 @default.
- W4385571644 hasRelatedWork W3033862527 @default.
- W4385571644 hasRelatedWork W3033942572 @default.
- W4385571644 hasRelatedWork W3097571385 @default.
- W4385571644 hasRelatedWork W4287761227 @default.
- W4385571644 hasRelatedWork W4323355575 @default.
- W4385571644 hasRelatedWork W4381786178 @default.
- W4385571644 hasRelatedWork W2610387714 @default.
- W4385571644 isParatext "false" @default.
- W4385571644 isRetracted "false" @default.
- W4385571644 workType "article" @default.