Matches in SemOpenAlex for { <https://semopenalex.org/work/W4384648625> ?p ?o ?g. }
Showing items 1 to 79 of
79
with 100 items per page.
- W4384648625 abstract "Large language models~(LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data. However, widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many low-quality instances with incorrect or irrelevant responses, which are misleading and detrimental to IFT. In this paper, we propose a simple and effective data selection strategy that automatically identifies and filters out low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce AlpaGasus, which is finetuned on only 9k high-quality data filtered from the 52k Alpaca data. AlpaGasus significantly outperforms the original Alpaca as evaluated by GPT-4 on multiple test sets and the controlled human evaluation. Its 13B variant matches $>90%$ performance of its teacher LLM (i.e., Text-Davinci-003 generating the 52k data) on test tasks. It also provides 5.7x faster training, reducing the training time for a 7B variant from 80 minutes (for Alpaca) to 14 minutes. Moreover, the experiments prove the efficacy of our method across diverse datasets, base models, and LLM filters. Overall, AlpaGasus demonstrates a novel data-centric IFT paradigm that can be generally applied to instruction-tuning data, leading to faster training and better instruction-following models. Our project page is available at: url{https://lichang-chen.github.io/AlpaGasus/}" @default.
- W4384648625 created "2023-07-19" @default.
- W4384648625 creator A5015050986 @default.
- W4384648625 creator A5019731648 @default.
- W4384648625 creator A5030330619 @default.
- W4384648625 creator A5039076312 @default.
- W4384648625 creator A5039714067 @default.
- W4384648625 creator A5041666153 @default.
- W4384648625 creator A5044205212 @default.
- W4384648625 creator A5046887194 @default.
- W4384648625 creator A5051027971 @default.
- W4384648625 creator A5060016795 @default.
- W4384648625 creator A5066326081 @default.
- W4384648625 date "2023-07-17" @default.
- W4384648625 modified "2023-10-14" @default.
- W4384648625 title "AlpaGasus: Training A Better Alpaca with Fewer Data" @default.
- W4384648625 doi "https://doi.org/10.48550/arxiv.2307.08701" @default.
- W4384648625 hasPublicationYear "2023" @default.
- W4384648625 type Work @default.
- W4384648625 citedByCount "0" @default.
- W4384648625 crossrefType "posted-content" @default.
- W4384648625 hasAuthorship W4384648625A5015050986 @default.
- W4384648625 hasAuthorship W4384648625A5019731648 @default.
- W4384648625 hasAuthorship W4384648625A5030330619 @default.
- W4384648625 hasAuthorship W4384648625A5039076312 @default.
- W4384648625 hasAuthorship W4384648625A5039714067 @default.
- W4384648625 hasAuthorship W4384648625A5041666153 @default.
- W4384648625 hasAuthorship W4384648625A5044205212 @default.
- W4384648625 hasAuthorship W4384648625A5046887194 @default.
- W4384648625 hasAuthorship W4384648625A5051027971 @default.
- W4384648625 hasAuthorship W4384648625A5060016795 @default.
- W4384648625 hasAuthorship W4384648625A5066326081 @default.
- W4384648625 hasBestOaLocation W43846486251 @default.
- W4384648625 hasConcept C111472728 @default.
- W4384648625 hasConcept C119857082 @default.
- W4384648625 hasConcept C121332964 @default.
- W4384648625 hasConcept C138885662 @default.
- W4384648625 hasConcept C151730666 @default.
- W4384648625 hasConcept C153294291 @default.
- W4384648625 hasConcept C154945302 @default.
- W4384648625 hasConcept C2776145971 @default.
- W4384648625 hasConcept C2777211547 @default.
- W4384648625 hasConcept C2777267654 @default.
- W4384648625 hasConcept C2779530757 @default.
- W4384648625 hasConcept C2780586882 @default.
- W4384648625 hasConcept C41008148 @default.
- W4384648625 hasConcept C51632099 @default.
- W4384648625 hasConcept C86803240 @default.
- W4384648625 hasConceptScore W4384648625C111472728 @default.
- W4384648625 hasConceptScore W4384648625C119857082 @default.
- W4384648625 hasConceptScore W4384648625C121332964 @default.
- W4384648625 hasConceptScore W4384648625C138885662 @default.
- W4384648625 hasConceptScore W4384648625C151730666 @default.
- W4384648625 hasConceptScore W4384648625C153294291 @default.
- W4384648625 hasConceptScore W4384648625C154945302 @default.
- W4384648625 hasConceptScore W4384648625C2776145971 @default.
- W4384648625 hasConceptScore W4384648625C2777211547 @default.
- W4384648625 hasConceptScore W4384648625C2777267654 @default.
- W4384648625 hasConceptScore W4384648625C2779530757 @default.
- W4384648625 hasConceptScore W4384648625C2780586882 @default.
- W4384648625 hasConceptScore W4384648625C41008148 @default.
- W4384648625 hasConceptScore W4384648625C51632099 @default.
- W4384648625 hasConceptScore W4384648625C86803240 @default.
- W4384648625 hasLocation W43846486251 @default.
- W4384648625 hasOpenAccess W4384648625 @default.
- W4384648625 hasPrimaryLocation W43846486251 @default.
- W4384648625 hasRelatedWork W1459710595 @default.
- W4384648625 hasRelatedWork W2033364610 @default.
- W4384648625 hasRelatedWork W2130553454 @default.
- W4384648625 hasRelatedWork W2153927146 @default.
- W4384648625 hasRelatedWork W2551249631 @default.
- W4384648625 hasRelatedWork W2797776314 @default.
- W4384648625 hasRelatedWork W2949671220 @default.
- W4384648625 hasRelatedWork W3022007134 @default.
- W4384648625 hasRelatedWork W3163689946 @default.
- W4384648625 hasRelatedWork W4317548404 @default.
- W4384648625 isParatext "false" @default.
- W4384648625 isRetracted "false" @default.
- W4384648625 workType "article" @default.