Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387559078> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W4387559078 abstract "Prompt engineering is effective and important in the deployment of LLMs but is poorly understood mathematically. Here, we formalize prompt engineering as an optimal control problem on LLMs -- where the prompt is considered a control variable for modulating the output distribution of the LLM. Within this framework, we ask a simple question: given a sequence of tokens, does there always exist a prompt we can prepend that will steer the LLM toward accurately predicting the final token? We call such an optimal prompt the magic word since prepending the prompt causes the LLM to output the correct answer. If magic words exist, can we find them? If so, what are their properties? We offer analytic analysis on the controllability of the self-attention head where we prove a bound on controllability as a function of the singular values of its weight matrices. We take inspiration from control theory to propose a metric called $k-epsilon$ controllability to characterize LLM steerability. We compute the $k-epsilon$ controllability of a panel of large language models, including Falcon-7b, Llama-7b, and Falcon-40b on 5000 WikiText causal language modeling tasks. Remarkably, we find that magic words of 10 tokens or less exist for over 97% of WikiText instances surveyed for each model." @default.
- W4387559078 created "2023-10-12" @default.
- W4387559078 creator A5014506392 @default.
- W4387559078 creator A5031358004 @default.
- W4387559078 creator A5064578796 @default.
- W4387559078 creator A5084491666 @default.
- W4387559078 date "2023-10-02" @default.
- W4387559078 modified "2023-10-13" @default.
- W4387559078 title "What's the Magic Word? A Control Theory of LLM Prompting" @default.
- W4387559078 doi "https://doi.org/10.48550/arxiv.2310.04444" @default.
- W4387559078 hasPublicationYear "2023" @default.
- W4387559078 type Work @default.
- W4387559078 citedByCount "0" @default.
- W4387559078 crossrefType "posted-content" @default.
- W4387559078 hasAuthorship W4387559078A5014506392 @default.
- W4387559078 hasAuthorship W4387559078A5031358004 @default.
- W4387559078 hasAuthorship W4387559078A5064578796 @default.
- W4387559078 hasAuthorship W4387559078A5084491666 @default.
- W4387559078 hasBestOaLocation W43875590781 @default.
- W4387559078 hasConcept C11413529 @default.
- W4387559078 hasConcept C121332964 @default.
- W4387559078 hasConcept C138885662 @default.
- W4387559078 hasConcept C162324750 @default.
- W4387559078 hasConcept C176217482 @default.
- W4387559078 hasConcept C21547014 @default.
- W4387559078 hasConcept C2777704519 @default.
- W4387559078 hasConcept C28826006 @default.
- W4387559078 hasConcept C33923547 @default.
- W4387559078 hasConcept C38652104 @default.
- W4387559078 hasConcept C41008148 @default.
- W4387559078 hasConcept C41895202 @default.
- W4387559078 hasConcept C48145219 @default.
- W4387559078 hasConcept C48209547 @default.
- W4387559078 hasConcept C62520636 @default.
- W4387559078 hasConcept C80444323 @default.
- W4387559078 hasConcept C90805587 @default.
- W4387559078 hasConceptScore W4387559078C11413529 @default.
- W4387559078 hasConceptScore W4387559078C121332964 @default.
- W4387559078 hasConceptScore W4387559078C138885662 @default.
- W4387559078 hasConceptScore W4387559078C162324750 @default.
- W4387559078 hasConceptScore W4387559078C176217482 @default.
- W4387559078 hasConceptScore W4387559078C21547014 @default.
- W4387559078 hasConceptScore W4387559078C2777704519 @default.
- W4387559078 hasConceptScore W4387559078C28826006 @default.
- W4387559078 hasConceptScore W4387559078C33923547 @default.
- W4387559078 hasConceptScore W4387559078C38652104 @default.
- W4387559078 hasConceptScore W4387559078C41008148 @default.
- W4387559078 hasConceptScore W4387559078C41895202 @default.
- W4387559078 hasConceptScore W4387559078C48145219 @default.
- W4387559078 hasConceptScore W4387559078C48209547 @default.
- W4387559078 hasConceptScore W4387559078C62520636 @default.
- W4387559078 hasConceptScore W4387559078C80444323 @default.
- W4387559078 hasConceptScore W4387559078C90805587 @default.
- W4387559078 hasLocation W43875590781 @default.
- W4387559078 hasOpenAccess W4387559078 @default.
- W4387559078 hasPrimaryLocation W43875590781 @default.
- W4387559078 hasRelatedWork W1496421768 @default.
- W4387559078 hasRelatedWork W2003779889 @default.
- W4387559078 hasRelatedWork W2036697162 @default.
- W4387559078 hasRelatedWork W2332386680 @default.
- W4387559078 hasRelatedWork W2542825942 @default.
- W4387559078 hasRelatedWork W2561315646 @default.
- W4387559078 hasRelatedWork W3201620972 @default.
- W4387559078 hasRelatedWork W4205698120 @default.
- W4387559078 hasRelatedWork W4239246781 @default.
- W4387559078 hasRelatedWork W2248621902 @default.
- W4387559078 isParatext "false" @default.
- W4387559078 isRetracted "false" @default.
- W4387559078 workType "article" @default.