Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571674> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4385571674 abstract "Recent works on instruction tuning (IT) have achieved great performance with zero-shot generalizability to unseen tasks. With additional context (e.g., task definition, examples) provided to models for fine-tuning, they achieved much higher performance than untuned models. Despite impressive performance gains, what models learn from IT remains understudied. In this work, we analyze how models utilize instructions during IT by comparing model training with altered vs. original instructions. Specifically, we create simplified task definitions by removing all semantic components and only leaving the output space information, and delusive examples that contain incorrect input-output mapping. Our experiments show that models trained on simplified task definition or delusive examples can achieve comparable performance to the ones trained on the original instructions and examples. Furthermore, we introduce a random baseline to perform zeroshot classification tasks, and find it achieves similar performance (42.6% exact-match) as IT does (43% exact-match) in low resource setting, while both methods outperform naive T5 significantly (30% per exact-match). Our analysis provides evidence that the impressive performance gain of current IT models can come from picking up superficial patterns, such as learning the output format and guessing. Our study highlights the urgent need for more reliable IT methods and evaluation." @default.
- W4385571674 created "2023-08-05" @default.
- W4385571674 creator A5030248499 @default.
- W4385571674 creator A5081617623 @default.
- W4385571674 date "2023-01-01" @default.
- W4385571674 modified "2023-09-24" @default.
- W4385571674 title "Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning" @default.
- W4385571674 doi "https://doi.org/10.18653/v1/2023.acl-short.113" @default.
- W4385571674 hasPublicationYear "2023" @default.
- W4385571674 type Work @default.
- W4385571674 citedByCount "0" @default.
- W4385571674 crossrefType "proceedings-article" @default.
- W4385571674 hasAuthorship W4385571674A5030248499 @default.
- W4385571674 hasAuthorship W4385571674A5081617623 @default.
- W4385571674 hasBestOaLocation W43855716741 @default.
- W4385571674 hasConcept C105795698 @default.
- W4385571674 hasConcept C111368507 @default.
- W4385571674 hasConcept C119857082 @default.
- W4385571674 hasConcept C12725497 @default.
- W4385571674 hasConcept C127313418 @default.
- W4385571674 hasConcept C151730666 @default.
- W4385571674 hasConcept C154945302 @default.
- W4385571674 hasConcept C162324750 @default.
- W4385571674 hasConcept C187736073 @default.
- W4385571674 hasConcept C204321447 @default.
- W4385571674 hasConcept C206345919 @default.
- W4385571674 hasConcept C27158222 @default.
- W4385571674 hasConcept C2779343474 @default.
- W4385571674 hasConcept C2780451532 @default.
- W4385571674 hasConcept C31258907 @default.
- W4385571674 hasConcept C33923547 @default.
- W4385571674 hasConcept C41008148 @default.
- W4385571674 hasConcept C86803240 @default.
- W4385571674 hasConceptScore W4385571674C105795698 @default.
- W4385571674 hasConceptScore W4385571674C111368507 @default.
- W4385571674 hasConceptScore W4385571674C119857082 @default.
- W4385571674 hasConceptScore W4385571674C12725497 @default.
- W4385571674 hasConceptScore W4385571674C127313418 @default.
- W4385571674 hasConceptScore W4385571674C151730666 @default.
- W4385571674 hasConceptScore W4385571674C154945302 @default.
- W4385571674 hasConceptScore W4385571674C162324750 @default.
- W4385571674 hasConceptScore W4385571674C187736073 @default.
- W4385571674 hasConceptScore W4385571674C204321447 @default.
- W4385571674 hasConceptScore W4385571674C206345919 @default.
- W4385571674 hasConceptScore W4385571674C27158222 @default.
- W4385571674 hasConceptScore W4385571674C2779343474 @default.
- W4385571674 hasConceptScore W4385571674C2780451532 @default.
- W4385571674 hasConceptScore W4385571674C31258907 @default.
- W4385571674 hasConceptScore W4385571674C33923547 @default.
- W4385571674 hasConceptScore W4385571674C41008148 @default.
- W4385571674 hasConceptScore W4385571674C86803240 @default.
- W4385571674 hasLocation W43855716741 @default.
- W4385571674 hasOpenAccess W4385571674 @default.
- W4385571674 hasPrimaryLocation W43855716741 @default.
- W4385571674 hasRelatedWork W2084164722 @default.
- W4385571674 hasRelatedWork W2110230818 @default.
- W4385571674 hasRelatedWork W2395078704 @default.
- W4385571674 hasRelatedWork W2961085424 @default.
- W4385571674 hasRelatedWork W3037322406 @default.
- W4385571674 hasRelatedWork W4200511449 @default.
- W4385571674 hasRelatedWork W4206344445 @default.
- W4385571674 hasRelatedWork W4306674287 @default.
- W4385571674 hasRelatedWork W4319453497 @default.
- W4385571674 hasRelatedWork W4224009465 @default.
- W4385571674 isParatext "false" @default.
- W4385571674 isRetracted "false" @default.
- W4385571674 workType "article" @default.