Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287888538> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4287888538 abstract "Commonsense reasoning tasks follow a standard paradigm of finetuning pretrained language models on the target task data, where samples are introduced to the model in a random order during training. However, recent research suggests that data order can have a significant impact on the performance of finetuned models for natural language understanding. Hence, we examine the effect of a human-like easy-to-difficult curriculum during finetuning of language models for commonsense reasoning tasks. We use paced curriculum learning to rank data and sample training mini-batches with increasing levels of difficulty from the ranked dataset during finetuning. Further, we investigate the effect of an adaptive curriculum, i.e., the data ranking is dynamically updated during training based on the current state of the learner model. We use a teacher model to measure difficulty of each sample and experiment with three measures based on question answering probability, variability and out-of-distribution. To understand the effectiveness of curriculum learning in various scenarios, we apply it on full model fine-tuning as well as parameter-efficient prompt-tuning settings. Our results show that fixed as well as adaptive curriculum learning significantly improve performance for five commonsense reasoning tasks, i.e., SocialIQA, CosmosQA, CODAH, HellaSwag, WinoGrande in both tuning settings. Further, we find that prioritizing the difficult samples in the tail end of training improves generalization to unseen in-domain data as well as out-of-domain data. Our work provides evidence and encourages research into curriculum learning for commonsense reasoning." @default.
- W4287888538 created "2022-07-26" @default.
- W4287888538 creator A5001987532 @default.
- W4287888538 creator A5048025332 @default.
- W4287888538 date "2022-01-01" @default.
- W4287888538 modified "2023-09-30" @default.
- W4287888538 title "On Curriculum Learning for Commonsense Reasoning" @default.
- W4287888538 doi "https://doi.org/10.18653/v1/2022.naacl-main.72" @default.
- W4287888538 hasPublicationYear "2022" @default.
- W4287888538 type Work @default.
- W4287888538 citedByCount "0" @default.
- W4287888538 crossrefType "proceedings-article" @default.
- W4287888538 hasAuthorship W4287888538A5001987532 @default.
- W4287888538 hasAuthorship W4287888538A5048025332 @default.
- W4287888538 hasBestOaLocation W42878885381 @default.
- W4287888538 hasConcept C114614502 @default.
- W4287888538 hasConcept C119857082 @default.
- W4287888538 hasConcept C134306372 @default.
- W4287888538 hasConcept C137293760 @default.
- W4287888538 hasConcept C154945302 @default.
- W4287888538 hasConcept C15744967 @default.
- W4287888538 hasConcept C162324750 @default.
- W4287888538 hasConcept C164226766 @default.
- W4287888538 hasConcept C177148314 @default.
- W4287888538 hasConcept C187736073 @default.
- W4287888538 hasConcept C189430467 @default.
- W4287888538 hasConcept C193221554 @default.
- W4287888538 hasConcept C19417346 @default.
- W4287888538 hasConcept C204321447 @default.
- W4287888538 hasConcept C207685749 @default.
- W4287888538 hasConcept C2780451532 @default.
- W4287888538 hasConcept C30542707 @default.
- W4287888538 hasConcept C33923547 @default.
- W4287888538 hasConcept C36503486 @default.
- W4287888538 hasConcept C41008148 @default.
- W4287888538 hasConcept C47177190 @default.
- W4287888538 hasConcept C86037889 @default.
- W4287888538 hasConceptScore W4287888538C114614502 @default.
- W4287888538 hasConceptScore W4287888538C119857082 @default.
- W4287888538 hasConceptScore W4287888538C134306372 @default.
- W4287888538 hasConceptScore W4287888538C137293760 @default.
- W4287888538 hasConceptScore W4287888538C154945302 @default.
- W4287888538 hasConceptScore W4287888538C15744967 @default.
- W4287888538 hasConceptScore W4287888538C162324750 @default.
- W4287888538 hasConceptScore W4287888538C164226766 @default.
- W4287888538 hasConceptScore W4287888538C177148314 @default.
- W4287888538 hasConceptScore W4287888538C187736073 @default.
- W4287888538 hasConceptScore W4287888538C189430467 @default.
- W4287888538 hasConceptScore W4287888538C193221554 @default.
- W4287888538 hasConceptScore W4287888538C19417346 @default.
- W4287888538 hasConceptScore W4287888538C204321447 @default.
- W4287888538 hasConceptScore W4287888538C207685749 @default.
- W4287888538 hasConceptScore W4287888538C2780451532 @default.
- W4287888538 hasConceptScore W4287888538C30542707 @default.
- W4287888538 hasConceptScore W4287888538C33923547 @default.
- W4287888538 hasConceptScore W4287888538C36503486 @default.
- W4287888538 hasConceptScore W4287888538C41008148 @default.
- W4287888538 hasConceptScore W4287888538C47177190 @default.
- W4287888538 hasConceptScore W4287888538C86037889 @default.
- W4287888538 hasLocation W42878885381 @default.
- W4287888538 hasOpenAccess W4287888538 @default.
- W4287888538 hasPrimaryLocation W42878885381 @default.
- W4287888538 hasRelatedWork W1786507113 @default.
- W4287888538 hasRelatedWork W1934555896 @default.
- W4287888538 hasRelatedWork W2047778511 @default.
- W4287888538 hasRelatedWork W2293317945 @default.
- W4287888538 hasRelatedWork W2499321295 @default.
- W4287888538 hasRelatedWork W2753828065 @default.
- W4287888538 hasRelatedWork W3035387406 @default.
- W4287888538 hasRelatedWork W3120117209 @default.
- W4287888538 hasRelatedWork W4318960487 @default.
- W4287888538 hasRelatedWork W4323349240 @default.
- W4287888538 isParatext "false" @default.
- W4287888538 isRetracted "false" @default.
- W4287888538 workType "article" @default.