Matches in SemOpenAlex for { <https://semopenalex.org/work/W4381248054> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4381248054 abstract "Pre-training & fine-tuning is a prevalent paradigm in computer vision (CV). Recently, parameter-efficient transfer learning (PETL) methods have shown promising performance in transferring knowledge from pre-trained models with only a few trainable parameters. Despite their success, the existing PETL methods in CV can be computationally expensive and require large amounts of memory and time cost during training, which limits low-resource users from conducting research and applications on large models. In this work, we propose Parameter, Memory, and Time Efficient Visual Adapter ($mathrm{E^3VA}$) tuning to address this issue. We provide a gradient backpropagation highway for low-rank adapters which removes large gradient computations for the frozen pre-trained parameters, resulting in substantial savings of training memory and training time. Furthermore, we optimise the $mathrm{E^3VA}$ structure for dense predictions tasks to promote model performance. Extensive experiments on COCO, ADE20K, and Pascal VOC benchmarks show that $mathrm{E^3VA}$ can save up to 62.2% training memory and 26.2% training time on average, while achieving comparable performance to full fine-tuning and better performance than most PETL methods. Note that we can even train the Swin-Large-based Cascade Mask RCNN on GTX 1080Ti GPUs with less than 1.5% trainable parameters." @default.
- W4381248054 created "2023-06-20" @default.
- W4381248054 creator A5023485080 @default.
- W4381248054 creator A5029695887 @default.
- W4381248054 creator A5058687102 @default.
- W4381248054 creator A5072548035 @default.
- W4381248054 creator A5076877539 @default.
- W4381248054 date "2023-06-16" @default.
- W4381248054 modified "2023-09-25" @default.
- W4381248054 title "Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions" @default.
- W4381248054 doi "https://doi.org/10.48550/arxiv.2306.09729" @default.
- W4381248054 hasPublicationYear "2023" @default.
- W4381248054 type Work @default.
- W4381248054 citedByCount "0" @default.
- W4381248054 crossrefType "posted-content" @default.
- W4381248054 hasAuthorship W4381248054A5023485080 @default.
- W4381248054 hasAuthorship W4381248054A5029695887 @default.
- W4381248054 hasAuthorship W4381248054A5058687102 @default.
- W4381248054 hasAuthorship W4381248054A5072548035 @default.
- W4381248054 hasAuthorship W4381248054A5076877539 @default.
- W4381248054 hasBestOaLocation W43812480541 @default.
- W4381248054 hasConcept C113775141 @default.
- W4381248054 hasConcept C11413529 @default.
- W4381248054 hasConcept C119857082 @default.
- W4381248054 hasConcept C154945302 @default.
- W4381248054 hasConcept C177284502 @default.
- W4381248054 hasConcept C185592680 @default.
- W4381248054 hasConcept C199360897 @default.
- W4381248054 hasConcept C34146451 @default.
- W4381248054 hasConcept C41008148 @default.
- W4381248054 hasConcept C43617362 @default.
- W4381248054 hasConcept C45374587 @default.
- W4381248054 hasConcept C75608658 @default.
- W4381248054 hasConcept C9390403 @default.
- W4381248054 hasConceptScore W4381248054C113775141 @default.
- W4381248054 hasConceptScore W4381248054C11413529 @default.
- W4381248054 hasConceptScore W4381248054C119857082 @default.
- W4381248054 hasConceptScore W4381248054C154945302 @default.
- W4381248054 hasConceptScore W4381248054C177284502 @default.
- W4381248054 hasConceptScore W4381248054C185592680 @default.
- W4381248054 hasConceptScore W4381248054C199360897 @default.
- W4381248054 hasConceptScore W4381248054C34146451 @default.
- W4381248054 hasConceptScore W4381248054C41008148 @default.
- W4381248054 hasConceptScore W4381248054C43617362 @default.
- W4381248054 hasConceptScore W4381248054C45374587 @default.
- W4381248054 hasConceptScore W4381248054C75608658 @default.
- W4381248054 hasConceptScore W4381248054C9390403 @default.
- W4381248054 hasLocation W43812480541 @default.
- W4381248054 hasOpenAccess W4381248054 @default.
- W4381248054 hasPrimaryLocation W43812480541 @default.
- W4381248054 hasRelatedWork W1965681098 @default.
- W4381248054 hasRelatedWork W2066229105 @default.
- W4381248054 hasRelatedWork W2084485267 @default.
- W4381248054 hasRelatedWork W2140468882 @default.
- W4381248054 hasRelatedWork W2352648941 @default.
- W4381248054 hasRelatedWork W2961085424 @default.
- W4381248054 hasRelatedWork W4225307033 @default.
- W4381248054 hasRelatedWork W4231350079 @default.
- W4381248054 hasRelatedWork W4244612519 @default.
- W4381248054 hasRelatedWork W4306674287 @default.
- W4381248054 isParatext "false" @default.
- W4381248054 isRetracted "false" @default.
- W4381248054 workType "article" @default.