Matches in SemOpenAlex for { <https://semopenalex.org/work/W4308244210> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W4308244210 abstract "Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. We apply MTF to the pretrained multilingual BLOOM and mT5 model families to produce finetuned variants called BLOOMZ and mT0. We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus. Finetuning on multilingual tasks with English prompts further improves performance on English and non-English tasks leading to various state-of-the-art zero-shot results. We also investigate finetuning on multilingual tasks with prompts that have been machine-translated from English to match the language of each dataset. We find training on these machine-translated prompts leads to better performance on human-written prompts in the respective languages. Surprisingly, we find models are capable of zero-shot generalization to tasks in languages they have never intentionally seen. We conjecture that the models are learning higher-level capabilities that are both task- and language-agnostic. In addition, we introduce xP3, a composite of supervised datasets in 46 languages with English and machine-translated prompts. Our code, datasets and models are freely available at https://github.com/bigscience-workshop/xmtf." @default.
- W4308244210 created "2022-11-09" @default.
- W4308244210 creator A5000043237 @default.
- W4308244210 creator A5002839718 @default.
- W4308244210 creator A5017606722 @default.
- W4308244210 creator A5037408928 @default.
- W4308244210 creator A5045077843 @default.
- W4308244210 creator A5049133108 @default.
- W4308244210 creator A5052454696 @default.
- W4308244210 creator A5055124539 @default.
- W4308244210 creator A5056219662 @default.
- W4308244210 creator A5068036546 @default.
- W4308244210 creator A5073811679 @default.
- W4308244210 creator A5076733732 @default.
- W4308244210 creator A5081787254 @default.
- W4308244210 creator A5082096652 @default.
- W4308244210 creator A5084957527 @default.
- W4308244210 creator A5085451281 @default.
- W4308244210 creator A5090298428 @default.
- W4308244210 creator A5090528235 @default.
- W4308244210 creator A5091003789 @default.
- W4308244210 date "2022-11-03" @default.
- W4308244210 modified "2023-09-24" @default.
- W4308244210 title "Crosslingual Generalization through Multitask Finetuning" @default.
- W4308244210 doi "https://doi.org/10.48550/arxiv.2211.01786" @default.
- W4308244210 hasPublicationYear "2022" @default.
- W4308244210 type Work @default.
- W4308244210 citedByCount "0" @default.
- W4308244210 crossrefType "posted-content" @default.
- W4308244210 hasAuthorship W4308244210A5000043237 @default.
- W4308244210 hasAuthorship W4308244210A5002839718 @default.
- W4308244210 hasAuthorship W4308244210A5017606722 @default.
- W4308244210 hasAuthorship W4308244210A5037408928 @default.
- W4308244210 hasAuthorship W4308244210A5045077843 @default.
- W4308244210 hasAuthorship W4308244210A5049133108 @default.
- W4308244210 hasAuthorship W4308244210A5052454696 @default.
- W4308244210 hasAuthorship W4308244210A5055124539 @default.
- W4308244210 hasAuthorship W4308244210A5056219662 @default.
- W4308244210 hasAuthorship W4308244210A5068036546 @default.
- W4308244210 hasAuthorship W4308244210A5073811679 @default.
- W4308244210 hasAuthorship W4308244210A5076733732 @default.
- W4308244210 hasAuthorship W4308244210A5081787254 @default.
- W4308244210 hasAuthorship W4308244210A5082096652 @default.
- W4308244210 hasAuthorship W4308244210A5084957527 @default.
- W4308244210 hasAuthorship W4308244210A5085451281 @default.
- W4308244210 hasAuthorship W4308244210A5090298428 @default.
- W4308244210 hasAuthorship W4308244210A5090528235 @default.
- W4308244210 hasAuthorship W4308244210A5091003789 @default.
- W4308244210 hasBestOaLocation W43082442101 @default.
- W4308244210 hasConcept C134306372 @default.
- W4308244210 hasConcept C137293760 @default.
- W4308244210 hasConcept C154945302 @default.
- W4308244210 hasConcept C162324750 @default.
- W4308244210 hasConcept C177148314 @default.
- W4308244210 hasConcept C187736073 @default.
- W4308244210 hasConcept C204321447 @default.
- W4308244210 hasConcept C2780451532 @default.
- W4308244210 hasConcept C33923547 @default.
- W4308244210 hasConcept C41008148 @default.
- W4308244210 hasConceptScore W4308244210C134306372 @default.
- W4308244210 hasConceptScore W4308244210C137293760 @default.
- W4308244210 hasConceptScore W4308244210C154945302 @default.
- W4308244210 hasConceptScore W4308244210C162324750 @default.
- W4308244210 hasConceptScore W4308244210C177148314 @default.
- W4308244210 hasConceptScore W4308244210C187736073 @default.
- W4308244210 hasConceptScore W4308244210C204321447 @default.
- W4308244210 hasConceptScore W4308244210C2780451532 @default.
- W4308244210 hasConceptScore W4308244210C33923547 @default.
- W4308244210 hasConceptScore W4308244210C41008148 @default.
- W4308244210 hasLocation W43082442101 @default.
- W4308244210 hasOpenAccess W4308244210 @default.
- W4308244210 hasPrimaryLocation W43082442101 @default.
- W4308244210 hasRelatedWork W142374489 @default.
- W4308244210 hasRelatedWork W1803932089 @default.
- W4308244210 hasRelatedWork W1985007624 @default.
- W4308244210 hasRelatedWork W2081647779 @default.
- W4308244210 hasRelatedWork W2176369193 @default.
- W4308244210 hasRelatedWork W2359001871 @default.
- W4308244210 hasRelatedWork W3107474891 @default.
- W4308244210 hasRelatedWork W3185852197 @default.
- W4308244210 hasRelatedWork W4205820553 @default.
- W4308244210 hasRelatedWork W2584532118 @default.
- W4308244210 isParatext "false" @default.
- W4308244210 isRetracted "false" @default.
- W4308244210 workType "article" @default.