Matches in SemOpenAlex for { <https://semopenalex.org/work/W4382334310> ?p ?o ?g. }
Showing items 1 to 47 of
47
with 100 items per page.
- W4382334310 abstract "As an efficient alternative to conventional full finetuning, parameter-efficient finetuning (PEFT) is becoming the prevailing method to adapt pretrained language models. In PEFT, a lightweight module is learned on each dataset while the underlying pretrained language model remains unchanged, resulting in multiple compact modules representing diverse skills when applied to various domains and tasks. In this paper, we propose to compose these parameter-efficient modules through linear arithmetic operations in the weight space, thereby integrating different module capabilities. Specifically, we first define addition and negation operators for the module, and then further compose these two basic operators to perform flexible arithmetic. Our approach requires emph{no additional training} and enables highly flexible module composition. We apply different arithmetic operations to compose the parameter-efficient modules for (1) distribution generalization, (2) multi-tasking, (3) unlearning, and (4) domain transfer. Additionally, we extend our approach to detoxify Alpaca-LoRA, the latest instruction-tuned large language model based on LLaMA. Empirical results demonstrate that our approach produces new and effective parameter-efficient modules that significantly outperform existing ones across all settings." @default.
- W4382334310 created "2023-06-28" @default.
- W4382334310 creator A5014829857 @default.
- W4382334310 creator A5019672830 @default.
- W4382334310 creator A5068902955 @default.
- W4382334310 creator A5087431853 @default.
- W4382334310 date "2023-06-26" @default.
- W4382334310 modified "2023-09-25" @default.
- W4382334310 title "Composing Parameter-Efficient Modules with Arithmetic Operations" @default.
- W4382334310 doi "https://doi.org/10.48550/arxiv.2306.14870" @default.
- W4382334310 hasPublicationYear "2023" @default.
- W4382334310 type Work @default.
- W4382334310 citedByCount "0" @default.
- W4382334310 crossrefType "posted-content" @default.
- W4382334310 hasAuthorship W4382334310A5014829857 @default.
- W4382334310 hasAuthorship W4382334310A5019672830 @default.
- W4382334310 hasAuthorship W4382334310A5068902955 @default.
- W4382334310 hasAuthorship W4382334310A5087431853 @default.
- W4382334310 hasBestOaLocation W43823343101 @default.
- W4382334310 hasConcept C134306372 @default.
- W4382334310 hasConcept C177148314 @default.
- W4382334310 hasConcept C33923547 @default.
- W4382334310 hasConcept C41008148 @default.
- W4382334310 hasConcept C80444323 @default.
- W4382334310 hasConcept C94375191 @default.
- W4382334310 hasConceptScore W4382334310C134306372 @default.
- W4382334310 hasConceptScore W4382334310C177148314 @default.
- W4382334310 hasConceptScore W4382334310C33923547 @default.
- W4382334310 hasConceptScore W4382334310C41008148 @default.
- W4382334310 hasConceptScore W4382334310C80444323 @default.
- W4382334310 hasConceptScore W4382334310C94375191 @default.
- W4382334310 hasLocation W43823343101 @default.
- W4382334310 hasOpenAccess W4382334310 @default.
- W4382334310 hasPrimaryLocation W43823343101 @default.
- W4382334310 hasRelatedWork W1507084099 @default.
- W4382334310 hasRelatedWork W1994274153 @default.
- W4382334310 hasRelatedWork W2012014189 @default.
- W4382334310 hasRelatedWork W2104044130 @default.
- W4382334310 hasRelatedWork W2249271273 @default.
- W4382334310 hasRelatedWork W2463562490 @default.
- W4382334310 hasRelatedWork W2885094885 @default.
- W4382334310 hasRelatedWork W4295850094 @default.
- W4382334310 hasRelatedWork W4298165655 @default.
- W4382334310 hasRelatedWork W589109686 @default.
- W4382334310 isParatext "false" @default.
- W4382334310 isRetracted "false" @default.
- W4382334310 workType "article" @default.