Matches in SemOpenAlex for { <https://semopenalex.org/work/W3203136703> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W3203136703 abstract "Large multilingual models, such as mBERT, have shown promise in crosslingual transfer. In this work, we employ pruning to quantify the robustness and interpret layer-wise importance of mBERT. On four GLUE tasks, the relative drops in accuracy due to pruning have almost identical results on mBERT and BERT suggesting that the reduced attention capacity of the multilingual models does not affect robustness to pruning. For the crosslingual task XNLI, we report higher drops in accuracy with pruning indicating lower robustness in crosslingual transfer. Also, the importance of the encoder layers sensitively depends on the language family and the pre-training corpus size. The top layers, which are relatively more influenced by fine-tuning, encode important information for languages similar to English (SVO) while the bottom layers, which are relatively less influenced by fine-tuning, are particularly important for agglutinative and low-resource languages." @default.
- W3203136703 created "2021-10-11" @default.
- W3203136703 creator A5031718651 @default.
- W3203136703 creator A5043923735 @default.
- W3203136703 creator A5050036814 @default.
- W3203136703 creator A5059720765 @default.
- W3203136703 date "2021-09-26" @default.
- W3203136703 modified "2023-09-27" @default.
- W3203136703 title "On the Prunability of Attention Heads in Multilingual BERT." @default.
- W3203136703 cites W2963310665 @default.
- W3203136703 cites W2963341956 @default.
- W3203136703 cites W2964303116 @default.
- W3203136703 cites W2970120757 @default.
- W3203136703 cites W2970820321 @default.
- W3203136703 cites W2970854433 @default.
- W3203136703 cites W2972324944 @default.
- W3203136703 cites W2980965328 @default.
- W3203136703 cites W2986300872 @default.
- W3203136703 cites W2996428491 @default.
- W3203136703 cites W2997710335 @default.
- W3203136703 cites W3015233032 @default.
- W3203136703 cites W3035390927 @default.
- W3203136703 cites W3102519438 @default.
- W3203136703 cites W3118485687 @default.
- W3203136703 cites W3123624731 @default.
- W3203136703 hasPublicationYear "2021" @default.
- W3203136703 type Work @default.
- W3203136703 sameAs 3203136703 @default.
- W3203136703 citedByCount "0" @default.
- W3203136703 crossrefType "posted-content" @default.
- W3203136703 hasAuthorship W3203136703A5031718651 @default.
- W3203136703 hasAuthorship W3203136703A5043923735 @default.
- W3203136703 hasAuthorship W3203136703A5050036814 @default.
- W3203136703 hasAuthorship W3203136703A5059720765 @default.
- W3203136703 hasConcept C104317684 @default.
- W3203136703 hasConcept C111919701 @default.
- W3203136703 hasConcept C118505674 @default.
- W3203136703 hasConcept C119857082 @default.
- W3203136703 hasConcept C154945302 @default.
- W3203136703 hasConcept C185592680 @default.
- W3203136703 hasConcept C204321447 @default.
- W3203136703 hasConcept C41008148 @default.
- W3203136703 hasConcept C55493867 @default.
- W3203136703 hasConcept C63479239 @default.
- W3203136703 hasConceptScore W3203136703C104317684 @default.
- W3203136703 hasConceptScore W3203136703C111919701 @default.
- W3203136703 hasConceptScore W3203136703C118505674 @default.
- W3203136703 hasConceptScore W3203136703C119857082 @default.
- W3203136703 hasConceptScore W3203136703C154945302 @default.
- W3203136703 hasConceptScore W3203136703C185592680 @default.
- W3203136703 hasConceptScore W3203136703C204321447 @default.
- W3203136703 hasConceptScore W3203136703C41008148 @default.
- W3203136703 hasConceptScore W3203136703C55493867 @default.
- W3203136703 hasConceptScore W3203136703C63479239 @default.
- W3203136703 hasLocation W32031367031 @default.
- W3203136703 hasOpenAccess W3203136703 @default.
- W3203136703 hasPrimaryLocation W32031367031 @default.
- W3203136703 hasRelatedWork W1794363059 @default.
- W3203136703 hasRelatedWork W2250542250 @default.
- W3203136703 hasRelatedWork W2250853822 @default.
- W3203136703 hasRelatedWork W2742122839 @default.
- W3203136703 hasRelatedWork W2769107230 @default.
- W3203136703 hasRelatedWork W2908475389 @default.
- W3203136703 hasRelatedWork W2956023193 @default.
- W3203136703 hasRelatedWork W2972688845 @default.
- W3203136703 hasRelatedWork W2986195527 @default.
- W3203136703 hasRelatedWork W3020918146 @default.
- W3203136703 hasRelatedWork W3092736191 @default.
- W3203136703 hasRelatedWork W3100198908 @default.
- W3203136703 hasRelatedWork W3103727211 @default.
- W3203136703 hasRelatedWork W3106169611 @default.
- W3203136703 hasRelatedWork W3108950052 @default.
- W3203136703 hasRelatedWork W3157444148 @default.
- W3203136703 hasRelatedWork W3185293939 @default.
- W3203136703 hasRelatedWork W3211621949 @default.
- W3203136703 hasRelatedWork W3212748587 @default.
- W3203136703 hasRelatedWork W339896394 @default.
- W3203136703 isParatext "false" @default.
- W3203136703 isRetracted "false" @default.
- W3203136703 magId "3203136703" @default.
- W3203136703 workType "article" @default.