Matches in SemOpenAlex for { <https://semopenalex.org/work/W3194309076> ?p ?o ?g. }
- W3194309076 abstract "Large pre-trained language models (LMs) such as GPT-3 have acquired a surprising ability to perform zero-shot learning. For example, to classify sentiment without any training examples, we can “prompt” the LM with the review and the label description “Does the user like this movie?”, and ask whether the next word is “yes” or “no”. However, the next word prediction training objective is still misaligned with the target zero-shot learning objective. To address this weakness, we propose meta-tuning, which directly optimizes the zero-shot learning objective by fine-tuning pre-trained language models on a collection of datasets. We focus on classification tasks, and construct the meta-dataset by aggregating 43 existing datasets and annotating 441 label descriptions in a question-answering (QA) format. When evaluated on unseen tasks, meta-tuned models outperform a same-sized QA model and the previous SOTA zero-shot learning system based on natural language inference. Additionally, increasing parameter count from 220M to 770M improves AUC-ROC scores by 6.3%, and we forecast that even larger models would perform better. Therefore, measuring zero-shot learning performance on language models out-of-the-box might underestimate their true potential, and community-wide efforts on aggregating datasets and unifying their formats can help build models that answer prompts better." @default.
- W3194309076 created "2021-08-30" @default.
- W3194309076 creator A5004921249 @default.
- W3194309076 creator A5043123384 @default.
- W3194309076 creator A5046116522 @default.
- W3194309076 creator A5088888083 @default.
- W3194309076 date "2021-01-01" @default.
- W3194309076 modified "2023-10-18" @default.
- W3194309076 title "Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections" @default.
- W3194309076 cites W1527313647 @default.
- W3194309076 cites W2070246124 @default.
- W3194309076 cites W2113459411 @default.
- W3194309076 cites W2114524997 @default.
- W3194309076 cites W2251939518 @default.
- W3194309076 cites W2460159515 @default.
- W3194309076 cites W2483215953 @default.
- W3194309076 cites W2753160622 @default.
- W3194309076 cites W2777746208 @default.
- W3194309076 cites W2805744755 @default.
- W3194309076 cites W2806120502 @default.
- W3194309076 cites W2807333695 @default.
- W3194309076 cites W2809324505 @default.
- W3194309076 cites W2893160345 @default.
- W3194309076 cites W2916132663 @default.
- W3194309076 cites W2922580172 @default.
- W3194309076 cites W2946659172 @default.
- W3194309076 cites W2949433733 @default.
- W3194309076 cites W2962910668 @default.
- W3194309076 cites W2963012544 @default.
- W3194309076 cites W2963341956 @default.
- W3194309076 cites W2963709474 @default.
- W3194309076 cites W2963777589 @default.
- W3194309076 cites W2963846996 @default.
- W3194309076 cites W2964756555 @default.
- W3194309076 cites W2969513720 @default.
- W3194309076 cites W2970200208 @default.
- W3194309076 cites W2970862333 @default.
- W3194309076 cites W2975429091 @default.
- W3194309076 cites W2981852735 @default.
- W3194309076 cites W3002104146 @default.
- W3194309076 cites W3030163527 @default.
- W3194309076 cites W3043372854 @default.
- W3194309076 cites W3085177480 @default.
- W3194309076 cites W3094300879 @default.
- W3194309076 cites W3098267758 @default.
- W3194309076 cites W3098824823 @default.
- W3194309076 cites W3099215402 @default.
- W3194309076 cites W3099655892 @default.
- W3194309076 cites W3100452485 @default.
- W3194309076 cites W3102743123 @default.
- W3194309076 cites W3114114934 @default.
- W3194309076 cites W3114796327 @default.
- W3194309076 cites W3118905363 @default.
- W3194309076 cites W3119438769 @default.
- W3194309076 cites W3121904249 @default.
- W3194309076 cites W3124687886 @default.
- W3194309076 cites W3126259453 @default.
- W3194309076 cites W3132736064 @default.
- W3194309076 cites W3152515526 @default.
- W3194309076 cites W3152956381 @default.
- W3194309076 cites W3154200459 @default.
- W3194309076 cites W3157005959 @default.
- W3194309076 cites W3158635868 @default.
- W3194309076 cites W3167525829 @default.
- W3194309076 cites W3167602185 @default.
- W3194309076 cites W3173777717 @default.
- W3194309076 cites W3178814223 @default.
- W3194309076 cites W3190860428 @default.
- W3194309076 doi "https://doi.org/10.18653/v1/2021.findings-emnlp.244" @default.
- W3194309076 hasPublicationYear "2021" @default.
- W3194309076 type Work @default.
- W3194309076 sameAs 3194309076 @default.
- W3194309076 citedByCount "21" @default.
- W3194309076 countsByYear W31943090762021 @default.
- W3194309076 countsByYear W31943090762022 @default.
- W3194309076 countsByYear W31943090762023 @default.
- W3194309076 crossrefType "proceedings-article" @default.
- W3194309076 hasAuthorship W3194309076A5004921249 @default.
- W3194309076 hasAuthorship W3194309076A5043123384 @default.
- W3194309076 hasAuthorship W3194309076A5046116522 @default.
- W3194309076 hasAuthorship W3194309076A5088888083 @default.
- W3194309076 hasBestOaLocation W31943090761 @default.
- W3194309076 hasConcept C119857082 @default.
- W3194309076 hasConcept C120665830 @default.
- W3194309076 hasConcept C121332964 @default.
- W3194309076 hasConcept C137293760 @default.
- W3194309076 hasConcept C138885662 @default.
- W3194309076 hasConcept C154945302 @default.
- W3194309076 hasConcept C162324750 @default.
- W3194309076 hasConcept C178790620 @default.
- W3194309076 hasConcept C185592680 @default.
- W3194309076 hasConcept C187736073 @default.
- W3194309076 hasConcept C192209626 @default.
- W3194309076 hasConcept C199360897 @default.
- W3194309076 hasConcept C204321447 @default.
- W3194309076 hasConcept C2776214188 @default.
- W3194309076 hasConcept C2778344882 @default.
- W3194309076 hasConcept C2780451532 @default.
- W3194309076 hasConcept C2780801425 @default.
- W3194309076 hasConcept C2780813799 @default.