Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385474674> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4385474674 abstract "Current state-of-the-art results in computer vision depend in part on fine-tuning large pre-trained vision models. However, with the exponential growth of model sizes, the conventional full fine-tuning, which needs to store a individual network copy for each tasks, leads to increasingly huge storage and transmission overhead. Adapter-based Parameter-Efficient Tuning (PET) methods address this challenge by tuning lightweight adapters inserted into the frozen pre-trained models. In this paper, we investigate how to make adapters even more efficient, reaching a new minimum size required to store a task-specific fine-tuned network. Inspired by the observation that the parameters of adapters converge at flat local minima, we find that adapters are resistant to noise in parameter space, which means they are also resistant to low numerical precision. To train low-precision adapters, we propose a computational-efficient quantization method which minimizes the quantization error. Through extensive experiments, we find that low-precision adapters exhibit minimal performance degradation, and even 1-bit precision is sufficient for adapters. The experimental results demonstrate that 1-bit adapters outperform all other PET methods on both the VTAB-1K benchmark and few-shot FGVC tasks, while requiring the smallest storage size. Our findings show, for the first time, the significant potential of quantization techniques in PET, providing a general solution to enhance the parameter efficiency of adapter-based PET methods. Code: https://github.com/JieShibo/PETL-ViT" @default.
- W4385474674 created "2023-08-02" @default.
- W4385474674 creator A5041072876 @default.
- W4385474674 creator A5060171279 @default.
- W4385474674 creator A5091046771 @default.
- W4385474674 date "2023-07-31" @default.
- W4385474674 modified "2023-09-23" @default.
- W4385474674 title "Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy" @default.
- W4385474674 doi "https://doi.org/10.48550/arxiv.2307.16867" @default.
- W4385474674 hasPublicationYear "2023" @default.
- W4385474674 type Work @default.
- W4385474674 citedByCount "0" @default.
- W4385474674 crossrefType "posted-content" @default.
- W4385474674 hasAuthorship W4385474674A5041072876 @default.
- W4385474674 hasAuthorship W4385474674A5060171279 @default.
- W4385474674 hasAuthorship W4385474674A5091046771 @default.
- W4385474674 hasBestOaLocation W43854746741 @default.
- W4385474674 hasConcept C111919701 @default.
- W4385474674 hasConcept C113775141 @default.
- W4385474674 hasConcept C11413529 @default.
- W4385474674 hasConcept C134306372 @default.
- W4385474674 hasConcept C152124472 @default.
- W4385474674 hasConcept C154945302 @default.
- W4385474674 hasConcept C177284502 @default.
- W4385474674 hasConcept C186633575 @default.
- W4385474674 hasConcept C28855332 @default.
- W4385474674 hasConcept C33923547 @default.
- W4385474674 hasConcept C41008148 @default.
- W4385474674 hasConcept C57273362 @default.
- W4385474674 hasConcept C9390403 @default.
- W4385474674 hasConceptScore W4385474674C111919701 @default.
- W4385474674 hasConceptScore W4385474674C113775141 @default.
- W4385474674 hasConceptScore W4385474674C11413529 @default.
- W4385474674 hasConceptScore W4385474674C134306372 @default.
- W4385474674 hasConceptScore W4385474674C152124472 @default.
- W4385474674 hasConceptScore W4385474674C154945302 @default.
- W4385474674 hasConceptScore W4385474674C177284502 @default.
- W4385474674 hasConceptScore W4385474674C186633575 @default.
- W4385474674 hasConceptScore W4385474674C28855332 @default.
- W4385474674 hasConceptScore W4385474674C33923547 @default.
- W4385474674 hasConceptScore W4385474674C41008148 @default.
- W4385474674 hasConceptScore W4385474674C57273362 @default.
- W4385474674 hasConceptScore W4385474674C9390403 @default.
- W4385474674 hasLocation W43854746741 @default.
- W4385474674 hasOpenAccess W4385474674 @default.
- W4385474674 hasPrimaryLocation W43854746741 @default.
- W4385474674 hasRelatedWork W2030492936 @default.
- W4385474674 hasRelatedWork W2090601786 @default.
- W4385474674 hasRelatedWork W2351992004 @default.
- W4385474674 hasRelatedWork W2352632909 @default.
- W4385474674 hasRelatedWork W2358034992 @default.
- W4385474674 hasRelatedWork W2372547243 @default.
- W4385474674 hasRelatedWork W2378079574 @default.
- W4385474674 hasRelatedWork W2380207131 @default.
- W4385474674 hasRelatedWork W2383319832 @default.
- W4385474674 hasRelatedWork W2541477304 @default.
- W4385474674 isParatext "false" @default.
- W4385474674 isRetracted "false" @default.
- W4385474674 workType "article" @default.