Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571241> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4385571241 abstract "Efficient finetuning of pretrained language transformers is becoming increasingly prevalent for solving natural language processing tasks. While effective, it can still require a large number of tunable parameters. This can be a drawback for low-resource applications and training with differential-privacy constraints, where excessive noise may be introduced during finetuning. To this end, we propose a novel language transformer finetuning strategy that introduces task-specific parameters in multiple transformer layers. These parameters are derived from fixed random projections of a single trainable vector, enabling finetuning with significantly fewer parameters while maintaining performance. We achieve within 5% of full finetuning performance on GLUE tasks with as few as 4,100 parameters per task, outperforming other parameter-efficient finetuning approaches that use a similar number of per-task parameters. Besides, the random projections can be precomputed at inference, avoiding additional computational latency. All these make our method particularly appealing for low-resource applications. Finally, our method achieves the best or comparable utility compared to several recent finetuning methods when training with the same privacy constraints, underscoring its effectiveness and potential real-world impact." @default.
- W4385571241 created "2023-08-05" @default.
- W4385571241 creator A5046444281 @default.
- W4385571241 creator A5075920466 @default.
- W4385571241 creator A5081316458 @default.
- W4385571241 date "2023-01-01" @default.
- W4385571241 modified "2023-09-24" @default.
- W4385571241 title "Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning" @default.
- W4385571241 doi "https://doi.org/10.18653/v1/2023.findings-acl.799" @default.
- W4385571241 hasPublicationYear "2023" @default.
- W4385571241 type Work @default.
- W4385571241 citedByCount "0" @default.
- W4385571241 crossrefType "proceedings-article" @default.
- W4385571241 hasAuthorship W4385571241A5046444281 @default.
- W4385571241 hasAuthorship W4385571241A5075920466 @default.
- W4385571241 hasAuthorship W4385571241A5081316458 @default.
- W4385571241 hasBestOaLocation W43855712411 @default.
- W4385571241 hasConcept C119857082 @default.
- W4385571241 hasConcept C121332964 @default.
- W4385571241 hasConcept C154945302 @default.
- W4385571241 hasConcept C162324750 @default.
- W4385571241 hasConcept C165801399 @default.
- W4385571241 hasConcept C187736073 @default.
- W4385571241 hasConcept C2776214188 @default.
- W4385571241 hasConcept C2780451532 @default.
- W4385571241 hasConcept C41008148 @default.
- W4385571241 hasConcept C62520636 @default.
- W4385571241 hasConcept C66322947 @default.
- W4385571241 hasConcept C76155785 @default.
- W4385571241 hasConcept C82876162 @default.
- W4385571241 hasConceptScore W4385571241C119857082 @default.
- W4385571241 hasConceptScore W4385571241C121332964 @default.
- W4385571241 hasConceptScore W4385571241C154945302 @default.
- W4385571241 hasConceptScore W4385571241C162324750 @default.
- W4385571241 hasConceptScore W4385571241C165801399 @default.
- W4385571241 hasConceptScore W4385571241C187736073 @default.
- W4385571241 hasConceptScore W4385571241C2776214188 @default.
- W4385571241 hasConceptScore W4385571241C2780451532 @default.
- W4385571241 hasConceptScore W4385571241C41008148 @default.
- W4385571241 hasConceptScore W4385571241C62520636 @default.
- W4385571241 hasConceptScore W4385571241C66322947 @default.
- W4385571241 hasConceptScore W4385571241C76155785 @default.
- W4385571241 hasConceptScore W4385571241C82876162 @default.
- W4385571241 hasLocation W43855712411 @default.
- W4385571241 hasOpenAccess W4385571241 @default.
- W4385571241 hasPrimaryLocation W43855712411 @default.
- W4385571241 hasRelatedWork W1987753576 @default.
- W4385571241 hasRelatedWork W2961085424 @default.
- W4385571241 hasRelatedWork W3008625068 @default.
- W4385571241 hasRelatedWork W3046775127 @default.
- W4385571241 hasRelatedWork W4285260836 @default.
- W4385571241 hasRelatedWork W4286629047 @default.
- W4385571241 hasRelatedWork W4306321456 @default.
- W4385571241 hasRelatedWork W4306674287 @default.
- W4385571241 hasRelatedWork W2779562428 @default.
- W4385571241 hasRelatedWork W4224009465 @default.
- W4385571241 isParatext "false" @default.
- W4385571241 isRetracted "false" @default.
- W4385571241 workType "article" @default.