Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385570815> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4385570815 abstract "Parameter-efficient fine-tuning (PEFT) of pre-trained language models has recently demonstrated remarkable achievements, effectively matching the performance of full fine-tuning while utilizing significantly fewer trainable parameters, and consequently addressing the storage and communication constraints. Nonetheless, various PEFT methods are limited by their inherent characteristics. In the case of sparse fine-tuning, which involves modifying only a small subset of the existing parameters, the selection of fine-tuned parameters is task- and domain-specific, making it unsuitable for federated learning. On the other hand, PEFT methods with adding new parameters typically introduce additional inference latency. In this paper, we demonstrate the feasibility of generating a sparse mask in a task-agnostic manner, wherein all downstream tasks share a common mask. Our approach, which relies solely on the magnitude information of pre-trained parameters, surpasses existing methodologies by a significant margin when evaluated on the GLUE benchmark. Additionally, we introduce a novel adapter technique that directly applies the adapter to pre-trained parameters instead of the hidden representation, thereby achieving identical inference speed to that of full fine-tuning. Through extensive experiments, our proposed method attains a new state-of-the-art outcome in terms of both performance and storage efficiency, storing only 0.03% parameters of full fine-tuning." @default.
- W4385570815 created "2023-08-05" @default.
- W4385570815 creator A5021240812 @default.
- W4385570815 creator A5021578107 @default.
- W4385570815 creator A5076808820 @default.
- W4385570815 date "2023-01-01" @default.
- W4385570815 modified "2023-09-24" @default.
- W4385570815 title "Parameter-Efficient Fine-Tuning without Introducing New Latency" @default.
- W4385570815 doi "https://doi.org/10.18653/v1/2023.acl-long.233" @default.
- W4385570815 hasPublicationYear "2023" @default.
- W4385570815 type Work @default.
- W4385570815 citedByCount "0" @default.
- W4385570815 crossrefType "proceedings-article" @default.
- W4385570815 hasAuthorship W4385570815A5021240812 @default.
- W4385570815 hasAuthorship W4385570815A5021578107 @default.
- W4385570815 hasAuthorship W4385570815A5076808820 @default.
- W4385570815 hasBestOaLocation W43855708151 @default.
- W4385570815 hasConcept C113775141 @default.
- W4385570815 hasConcept C119857082 @default.
- W4385570815 hasConcept C121332964 @default.
- W4385570815 hasConcept C13280743 @default.
- W4385570815 hasConcept C137293760 @default.
- W4385570815 hasConcept C154945302 @default.
- W4385570815 hasConcept C157524613 @default.
- W4385570815 hasConcept C162324750 @default.
- W4385570815 hasConcept C177284502 @default.
- W4385570815 hasConcept C185798385 @default.
- W4385570815 hasConcept C187736073 @default.
- W4385570815 hasConcept C205649164 @default.
- W4385570815 hasConcept C2776214188 @default.
- W4385570815 hasConcept C2780451532 @default.
- W4385570815 hasConcept C41008148 @default.
- W4385570815 hasConcept C62520636 @default.
- W4385570815 hasConcept C76155785 @default.
- W4385570815 hasConcept C82876162 @default.
- W4385570815 hasConcept C9390403 @default.
- W4385570815 hasConceptScore W4385570815C113775141 @default.
- W4385570815 hasConceptScore W4385570815C119857082 @default.
- W4385570815 hasConceptScore W4385570815C121332964 @default.
- W4385570815 hasConceptScore W4385570815C13280743 @default.
- W4385570815 hasConceptScore W4385570815C137293760 @default.
- W4385570815 hasConceptScore W4385570815C154945302 @default.
- W4385570815 hasConceptScore W4385570815C157524613 @default.
- W4385570815 hasConceptScore W4385570815C162324750 @default.
- W4385570815 hasConceptScore W4385570815C177284502 @default.
- W4385570815 hasConceptScore W4385570815C185798385 @default.
- W4385570815 hasConceptScore W4385570815C187736073 @default.
- W4385570815 hasConceptScore W4385570815C205649164 @default.
- W4385570815 hasConceptScore W4385570815C2776214188 @default.
- W4385570815 hasConceptScore W4385570815C2780451532 @default.
- W4385570815 hasConceptScore W4385570815C41008148 @default.
- W4385570815 hasConceptScore W4385570815C62520636 @default.
- W4385570815 hasConceptScore W4385570815C76155785 @default.
- W4385570815 hasConceptScore W4385570815C82876162 @default.
- W4385570815 hasConceptScore W4385570815C9390403 @default.
- W4385570815 hasLocation W43855708151 @default.
- W4385570815 hasOpenAccess W4385570815 @default.
- W4385570815 hasPrimaryLocation W43855708151 @default.
- W4385570815 hasRelatedWork W2972987451 @default.
- W4385570815 hasRelatedWork W2983785000 @default.
- W4385570815 hasRelatedWork W2988839259 @default.
- W4385570815 hasRelatedWork W3138953784 @default.
- W4385570815 hasRelatedWork W3209304705 @default.
- W4385570815 hasRelatedWork W4221160826 @default.
- W4385570815 hasRelatedWork W4361231003 @default.
- W4385570815 hasRelatedWork W4378473991 @default.
- W4385570815 hasRelatedWork W4384345669 @default.
- W4385570815 hasRelatedWork W4385567149 @default.
- W4385570815 isParatext "false" @default.
- W4385570815 isRetracted "false" @default.
- W4385570815 workType "article" @default.