Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385570529> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W4385570529 abstract "Prompt tuning is one of the successful approaches for parameter-efficient tuning of pre-trained language models. Despite being arguably the most parameter-efficient (tuned soft prompts constitute <0.1% of total parameters), it typically performs worse than other efficient tuning methods and is quite sensitive to hyper-parameters. In this work, we introduce Residual Prompt Tuning - a simple and efficient method that significantly improves the performance and stability of prompt tuning. We propose to reparameterize soft prompt embeddings using a shallow network with a residual connection. Our experiments show that Residual Prompt Tuning significantly outperforms prompt tuning across T5-Large, T5-Base and BERT-Base models. Notably, our method reaches +7 points improvement over prompt tuning on SuperGLUE benchmark with T5-Base model and allows to reduce the prompt length by 10 times without hurting performance. In addition, we show that our approach is robust to the choice of learning rate and prompt initialization, and is effective in few-shot settings." @default.
- W4385570529 created "2023-08-05" @default.
- W4385570529 creator A5010636698 @default.
- W4385570529 creator A5012276327 @default.
- W4385570529 creator A5012370787 @default.
- W4385570529 creator A5054253075 @default.
- W4385570529 creator A5057420103 @default.
- W4385570529 creator A5081831578 @default.
- W4385570529 creator A5090519936 @default.
- W4385570529 date "2023-01-01" @default.
- W4385570529 modified "2023-09-24" @default.
- W4385570529 title "Residual Prompt Tuning: improving prompt tuning with residual reparameterization" @default.
- W4385570529 doi "https://doi.org/10.18653/v1/2023.findings-acl.421" @default.
- W4385570529 hasPublicationYear "2023" @default.
- W4385570529 type Work @default.
- W4385570529 citedByCount "0" @default.
- W4385570529 crossrefType "proceedings-article" @default.
- W4385570529 hasAuthorship W4385570529A5010636698 @default.
- W4385570529 hasAuthorship W4385570529A5012276327 @default.
- W4385570529 hasAuthorship W4385570529A5012370787 @default.
- W4385570529 hasAuthorship W4385570529A5054253075 @default.
- W4385570529 hasAuthorship W4385570529A5057420103 @default.
- W4385570529 hasAuthorship W4385570529A5081831578 @default.
- W4385570529 hasAuthorship W4385570529A5090519936 @default.
- W4385570529 hasBestOaLocation W43855705291 @default.
- W4385570529 hasConcept C11413529 @default.
- W4385570529 hasConcept C114466953 @default.
- W4385570529 hasConcept C121332964 @default.
- W4385570529 hasConcept C13280743 @default.
- W4385570529 hasConcept C134306372 @default.
- W4385570529 hasConcept C155512373 @default.
- W4385570529 hasConcept C157524613 @default.
- W4385570529 hasConcept C185798385 @default.
- W4385570529 hasConcept C199360897 @default.
- W4385570529 hasConcept C205649164 @default.
- W4385570529 hasConcept C33923547 @default.
- W4385570529 hasConcept C41008148 @default.
- W4385570529 hasConcept C42058472 @default.
- W4385570529 hasConcept C62520636 @default.
- W4385570529 hasConceptScore W4385570529C11413529 @default.
- W4385570529 hasConceptScore W4385570529C114466953 @default.
- W4385570529 hasConceptScore W4385570529C121332964 @default.
- W4385570529 hasConceptScore W4385570529C13280743 @default.
- W4385570529 hasConceptScore W4385570529C134306372 @default.
- W4385570529 hasConceptScore W4385570529C155512373 @default.
- W4385570529 hasConceptScore W4385570529C157524613 @default.
- W4385570529 hasConceptScore W4385570529C185798385 @default.
- W4385570529 hasConceptScore W4385570529C199360897 @default.
- W4385570529 hasConceptScore W4385570529C205649164 @default.
- W4385570529 hasConceptScore W4385570529C33923547 @default.
- W4385570529 hasConceptScore W4385570529C41008148 @default.
- W4385570529 hasConceptScore W4385570529C42058472 @default.
- W4385570529 hasConceptScore W4385570529C62520636 @default.
- W4385570529 hasLocation W43855705291 @default.
- W4385570529 hasOpenAccess W4385570529 @default.
- W4385570529 hasPrimaryLocation W43855705291 @default.
- W4385570529 hasRelatedWork W1485630101 @default.
- W4385570529 hasRelatedWork W2374512474 @default.
- W4385570529 hasRelatedWork W2418863753 @default.
- W4385570529 hasRelatedWork W2498017833 @default.
- W4385570529 hasRelatedWork W2745121476 @default.
- W4385570529 hasRelatedWork W2755231872 @default.
- W4385570529 hasRelatedWork W3007689282 @default.
- W4385570529 hasRelatedWork W3035794324 @default.
- W4385570529 hasRelatedWork W4323927473 @default.
- W4385570529 hasRelatedWork W4375957494 @default.
- W4385570529 isParatext "false" @default.
- W4385570529 isRetracted "false" @default.
- W4385570529 workType "article" @default.