Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378718184> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4378718184 abstract "Methods for adapting language models (LMs) to new tasks and domains have traditionally assumed white-box access to the model, and work by modifying its parameters. However, this is incompatible with a recent trend in the field, where the highest quality models are only available as black-boxes through inference APIs. Even when the model weights are available, the computational cost of fine-tuning large LMs can be prohibitive for most practitioners. In this work, we present a lightweight method for adapting large LMs to new domains and tasks, assuming no access to their weights or intermediate activations. Our approach fine-tunes a small white-box LM and combines it with the large black-box LM at the probability level through a small network, learned on a small validation set. We validate our approach by adapting a large LM (OPT-30B) to several domains and a downstream task (machine translation), observing improved performance in all cases, of up to 9%, while using a domain expert 23x smaller." @default.
- W4378718184 created "2023-05-30" @default.
- W4378718184 creator A5023341622 @default.
- W4378718184 creator A5047151336 @default.
- W4378718184 creator A5083161796 @default.
- W4378718184 date "2023-05-23" @default.
- W4378718184 modified "2023-09-27" @default.
- W4378718184 title "CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models" @default.
- W4378718184 doi "https://doi.org/10.48550/arxiv.2305.16876" @default.
- W4378718184 hasPublicationYear "2023" @default.
- W4378718184 type Work @default.
- W4378718184 citedByCount "0" @default.
- W4378718184 crossrefType "posted-content" @default.
- W4378718184 hasAuthorship W4378718184A5023341622 @default.
- W4378718184 hasAuthorship W4378718184A5047151336 @default.
- W4378718184 hasAuthorship W4378718184A5083161796 @default.
- W4378718184 hasBestOaLocation W43787181841 @default.
- W4378718184 hasConcept C111472728 @default.
- W4378718184 hasConcept C119857082 @default.
- W4378718184 hasConcept C127413603 @default.
- W4378718184 hasConcept C134306372 @default.
- W4378718184 hasConcept C137293760 @default.
- W4378718184 hasConcept C138885662 @default.
- W4378718184 hasConcept C154945302 @default.
- W4378718184 hasConcept C177264268 @default.
- W4378718184 hasConcept C180932941 @default.
- W4378718184 hasConcept C199360897 @default.
- W4378718184 hasConcept C201995342 @default.
- W4378718184 hasConcept C202444582 @default.
- W4378718184 hasConcept C203005215 @default.
- W4378718184 hasConcept C2776214188 @default.
- W4378718184 hasConcept C2779530757 @default.
- W4378718184 hasConcept C2780451532 @default.
- W4378718184 hasConcept C33923547 @default.
- W4378718184 hasConcept C36503486 @default.
- W4378718184 hasConcept C41008148 @default.
- W4378718184 hasConcept C94966114 @default.
- W4378718184 hasConcept C9652623 @default.
- W4378718184 hasConceptScore W4378718184C111472728 @default.
- W4378718184 hasConceptScore W4378718184C119857082 @default.
- W4378718184 hasConceptScore W4378718184C127413603 @default.
- W4378718184 hasConceptScore W4378718184C134306372 @default.
- W4378718184 hasConceptScore W4378718184C137293760 @default.
- W4378718184 hasConceptScore W4378718184C138885662 @default.
- W4378718184 hasConceptScore W4378718184C154945302 @default.
- W4378718184 hasConceptScore W4378718184C177264268 @default.
- W4378718184 hasConceptScore W4378718184C180932941 @default.
- W4378718184 hasConceptScore W4378718184C199360897 @default.
- W4378718184 hasConceptScore W4378718184C201995342 @default.
- W4378718184 hasConceptScore W4378718184C202444582 @default.
- W4378718184 hasConceptScore W4378718184C203005215 @default.
- W4378718184 hasConceptScore W4378718184C2776214188 @default.
- W4378718184 hasConceptScore W4378718184C2779530757 @default.
- W4378718184 hasConceptScore W4378718184C2780451532 @default.
- W4378718184 hasConceptScore W4378718184C33923547 @default.
- W4378718184 hasConceptScore W4378718184C36503486 @default.
- W4378718184 hasConceptScore W4378718184C41008148 @default.
- W4378718184 hasConceptScore W4378718184C94966114 @default.
- W4378718184 hasConceptScore W4378718184C9652623 @default.
- W4378718184 hasLocation W43787181841 @default.
- W4378718184 hasOpenAccess W4378718184 @default.
- W4378718184 hasPrimaryLocation W43787181841 @default.
- W4378718184 hasRelatedWork W1974878518 @default.
- W4378718184 hasRelatedWork W2037831268 @default.
- W4378718184 hasRelatedWork W2896411932 @default.
- W4378718184 hasRelatedWork W2903389359 @default.
- W4378718184 hasRelatedWork W2912887033 @default.
- W4378718184 hasRelatedWork W2930926105 @default.
- W4378718184 hasRelatedWork W2998015774 @default.
- W4378718184 hasRelatedWork W3107474891 @default.
- W4378718184 hasRelatedWork W4288754364 @default.
- W4378718184 hasRelatedWork W4296608056 @default.
- W4378718184 isParatext "false" @default.
- W4378718184 isRetracted "false" @default.
- W4378718184 workType "article" @default.