Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286977254> ?p ?o ?g. }
Showing items 1 to 86 of
86
with 100 items per page.
- W4286977254 abstract "Recent work has shown evidence that the knowledge acquired by multilingual BERT (mBERT) has two components: a language-specific and a language-neutral one. This paper analyses the relationship between them, in the context of fine-tuning on two tasks -- POS tagging and natural language inference -- which require the model to bring to bear different degrees of language-specific knowledge. Visualisations reveal that mBERT loses the ability to cluster representations by language after fine-tuning, a result that is supported by evidence from language identification experiments. However, further experiments on 'unlearning' language-specific representations using gradient reversal and iterative adversarial learning are shown not to add further improvement to the language-independent component over and above the effect of fine-tuning. The results presented here suggest that the process of fine-tuning causes a reorganisation of the model's limited representational capacity, enhancing language-independent representations at the expense of language-specific ones." @default.
- W4286977254 created "2022-07-25" @default.
- W4286977254 creator A5022407073 @default.
- W4286977254 creator A5027041248 @default.
- W4286977254 creator A5048549686 @default.
- W4286977254 creator A5086259576 @default.
- W4286977254 date "2021-09-14" @default.
- W4286977254 modified "2023-09-30" @default.
- W4286977254 title "On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning" @default.
- W4286977254 doi "https://doi.org/10.48550/arxiv.2109.06935" @default.
- W4286977254 hasPublicationYear "2021" @default.
- W4286977254 type Work @default.
- W4286977254 citedByCount "0" @default.
- W4286977254 crossrefType "posted-content" @default.
- W4286977254 hasAuthorship W4286977254A5022407073 @default.
- W4286977254 hasAuthorship W4286977254A5027041248 @default.
- W4286977254 hasAuthorship W4286977254A5048549686 @default.
- W4286977254 hasAuthorship W4286977254A5086259576 @default.
- W4286977254 hasBestOaLocation W42869772541 @default.
- W4286977254 hasConcept C116834253 @default.
- W4286977254 hasConcept C121332964 @default.
- W4286977254 hasConcept C129353971 @default.
- W4286977254 hasConcept C129792486 @default.
- W4286977254 hasConcept C137293760 @default.
- W4286977254 hasConcept C138885662 @default.
- W4286977254 hasConcept C151730666 @default.
- W4286977254 hasConcept C154945302 @default.
- W4286977254 hasConcept C157524613 @default.
- W4286977254 hasConcept C168167062 @default.
- W4286977254 hasConcept C195324797 @default.
- W4286977254 hasConcept C199360897 @default.
- W4286977254 hasConcept C204321447 @default.
- W4286977254 hasConcept C2776214188 @default.
- W4286977254 hasConcept C2779343474 @default.
- W4286977254 hasConcept C2779439875 @default.
- W4286977254 hasConcept C41008148 @default.
- W4286977254 hasConcept C41895202 @default.
- W4286977254 hasConcept C59822182 @default.
- W4286977254 hasConcept C62520636 @default.
- W4286977254 hasConcept C74672266 @default.
- W4286977254 hasConcept C83479923 @default.
- W4286977254 hasConcept C86803240 @default.
- W4286977254 hasConcept C97355855 @default.
- W4286977254 hasConcept C98045186 @default.
- W4286977254 hasConceptScore W4286977254C116834253 @default.
- W4286977254 hasConceptScore W4286977254C121332964 @default.
- W4286977254 hasConceptScore W4286977254C129353971 @default.
- W4286977254 hasConceptScore W4286977254C129792486 @default.
- W4286977254 hasConceptScore W4286977254C137293760 @default.
- W4286977254 hasConceptScore W4286977254C138885662 @default.
- W4286977254 hasConceptScore W4286977254C151730666 @default.
- W4286977254 hasConceptScore W4286977254C154945302 @default.
- W4286977254 hasConceptScore W4286977254C157524613 @default.
- W4286977254 hasConceptScore W4286977254C168167062 @default.
- W4286977254 hasConceptScore W4286977254C195324797 @default.
- W4286977254 hasConceptScore W4286977254C199360897 @default.
- W4286977254 hasConceptScore W4286977254C204321447 @default.
- W4286977254 hasConceptScore W4286977254C2776214188 @default.
- W4286977254 hasConceptScore W4286977254C2779343474 @default.
- W4286977254 hasConceptScore W4286977254C2779439875 @default.
- W4286977254 hasConceptScore W4286977254C41008148 @default.
- W4286977254 hasConceptScore W4286977254C41895202 @default.
- W4286977254 hasConceptScore W4286977254C59822182 @default.
- W4286977254 hasConceptScore W4286977254C62520636 @default.
- W4286977254 hasConceptScore W4286977254C74672266 @default.
- W4286977254 hasConceptScore W4286977254C83479923 @default.
- W4286977254 hasConceptScore W4286977254C86803240 @default.
- W4286977254 hasConceptScore W4286977254C97355855 @default.
- W4286977254 hasConceptScore W4286977254C98045186 @default.
- W4286977254 hasLocation W42869772541 @default.
- W4286977254 hasLocation W42869772542 @default.
- W4286977254 hasOpenAccess W4286977254 @default.
- W4286977254 hasPrimaryLocation W42869772541 @default.
- W4286977254 hasRelatedWork W124635070 @default.
- W4286977254 hasRelatedWork W138710363 @default.
- W4286977254 hasRelatedWork W226586525 @default.
- W4286977254 hasRelatedWork W2761417937 @default.
- W4286977254 hasRelatedWork W2963097991 @default.
- W4286977254 hasRelatedWork W3088333221 @default.
- W4286977254 hasRelatedWork W3200898860 @default.
- W4286977254 hasRelatedWork W3211796083 @default.
- W4286977254 hasRelatedWork W4286977254 @default.
- W4286977254 hasRelatedWork W3045475294 @default.
- W4286977254 isParatext "false" @default.
- W4286977254 isRetracted "false" @default.
- W4286977254 workType "article" @default.