Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385570165> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4385570165 abstract "Prompting has gained tremendous attention as an efficient method for the adaptation of large-scale language models.However, prompts often act against human intuition and report unstable performances, which has motivated methods that automatically find effective prompts.One popular approach is gradient-based search, which iteratively updates a (randomly) initialized prompt towards the optimal one with the guide of gradients.We propose a novel regularization method, CoRe, for gradient-based prompt tuning techniques, which guides a prompt to produce a task context properly.CoRe realizes two regularization effects — context attuning and context filtering — that improve prediction performance in a zero-shot in-context learning setting where a model makes inferences only with the prompt tuned by CoRe, without any demonstration examples for in-context learning.Context attuning guides the context generated by the input and the tuned prompt toward embedding the appropriate context for the task.In our theoretical analysis, regularizing the context extends to improving zero-shot in-context learning performance.Context filtering steers the prompt to select only the task-related context so that context attuning solely focuses on creating and sending the right task context.We evaluate CoRe on natural language understanding datasets and two large language models, GPT2-XL and GPT-J.Our training scheme shows performance improvements up to 11.9% on GPT2-XL, and up to 6.3% on GPT-J in zero-shot settings." @default.
- W4385570165 created "2023-08-05" @default.
- W4385570165 creator A5024873601 @default.
- W4385570165 creator A5055053843 @default.
- W4385570165 creator A5070230072 @default.
- W4385570165 creator A5072880323 @default.
- W4385570165 creator A5083084972 @default.
- W4385570165 creator A5087565126 @default.
- W4385570165 date "2023-01-01" @default.
- W4385570165 modified "2023-09-24" @default.
- W4385570165 title "Two Examples are Better than One: Context Regularization for Gradient-based Prompt Tuning" @default.
- W4385570165 doi "https://doi.org/10.18653/v1/2023.findings-acl.206" @default.
- W4385570165 hasPublicationYear "2023" @default.
- W4385570165 type Work @default.
- W4385570165 citedByCount "0" @default.
- W4385570165 crossrefType "proceedings-article" @default.
- W4385570165 hasAuthorship W4385570165A5024873601 @default.
- W4385570165 hasAuthorship W4385570165A5055053843 @default.
- W4385570165 hasAuthorship W4385570165A5070230072 @default.
- W4385570165 hasAuthorship W4385570165A5072880323 @default.
- W4385570165 hasAuthorship W4385570165A5083084972 @default.
- W4385570165 hasAuthorship W4385570165A5087565126 @default.
- W4385570165 hasBestOaLocation W43855701651 @default.
- W4385570165 hasConcept C119857082 @default.
- W4385570165 hasConcept C137293760 @default.
- W4385570165 hasConcept C151730666 @default.
- W4385570165 hasConcept C154945302 @default.
- W4385570165 hasConcept C183322885 @default.
- W4385570165 hasConcept C2776135515 @default.
- W4385570165 hasConcept C2779343474 @default.
- W4385570165 hasConcept C2781238097 @default.
- W4385570165 hasConcept C41008148 @default.
- W4385570165 hasConcept C86803240 @default.
- W4385570165 hasConceptScore W4385570165C119857082 @default.
- W4385570165 hasConceptScore W4385570165C137293760 @default.
- W4385570165 hasConceptScore W4385570165C151730666 @default.
- W4385570165 hasConceptScore W4385570165C154945302 @default.
- W4385570165 hasConceptScore W4385570165C183322885 @default.
- W4385570165 hasConceptScore W4385570165C2776135515 @default.
- W4385570165 hasConceptScore W4385570165C2779343474 @default.
- W4385570165 hasConceptScore W4385570165C2781238097 @default.
- W4385570165 hasConceptScore W4385570165C41008148 @default.
- W4385570165 hasConceptScore W4385570165C86803240 @default.
- W4385570165 hasLocation W43855701651 @default.
- W4385570165 hasOpenAccess W4385570165 @default.
- W4385570165 hasPrimaryLocation W43855701651 @default.
- W4385570165 hasRelatedWork W1989705153 @default.
- W4385570165 hasRelatedWork W2961085424 @default.
- W4385570165 hasRelatedWork W3046775127 @default.
- W4385570165 hasRelatedWork W4285260836 @default.
- W4385570165 hasRelatedWork W4286629047 @default.
- W4385570165 hasRelatedWork W4306321456 @default.
- W4385570165 hasRelatedWork W4306674287 @default.
- W4385570165 hasRelatedWork W4383605243 @default.
- W4385570165 hasRelatedWork W4384392961 @default.
- W4385570165 hasRelatedWork W4224009465 @default.
- W4385570165 isParatext "false" @default.
- W4385570165 isRetracted "false" @default.
- W4385570165 workType "article" @default.