Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387323413> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4387323413 abstract "Large language models (LLMs) exhibit remarkable performance improvement through in-context learning (ICL) by leveraging task-specific examples in the input. However, the mechanisms behind this improvement remain elusive. In this work, we investigate how LLM embeddings and attention representations change following in-context-learning, and how these changes mediate improvement in behavior. We employ neuroscience-inspired techniques such as representational similarity analysis (RSA) and propose novel methods for parameterized probing and measuring ratio of attention to relevant vs. irrelevant information in Llama-2 70B and Vicuna 13B. We designed three tasks with a priori relationships among their conditions: reading comprehension, linear regression, and adversarial prompt injection. We formed hypotheses about expected similarities in task representations to investigate latent changes in embeddings and attention. Our analyses revealed a meaningful correlation between changes in both embeddings and attention representations with improvements in behavioral performance after ICL. This empirical framework empowers a nuanced understanding of how latent representations affect LLM behavior with and without ICL, offering valuable tools and insights for future research and practical applications." @default.
- W4387323413 created "2023-10-04" @default.
- W4387323413 creator A5011960878 @default.
- W4387323413 creator A5033668003 @default.
- W4387323413 creator A5054629140 @default.
- W4387323413 creator A5055894819 @default.
- W4387323413 creator A5059293348 @default.
- W4387323413 creator A5061429872 @default.
- W4387323413 date "2023-09-30" @default.
- W4387323413 modified "2023-10-05" @default.
- W4387323413 title "In-Context Learning in Large Language Models: A Neuroscience-inspired Analysis of Representations" @default.
- W4387323413 doi "https://doi.org/10.48550/arxiv.2310.00313" @default.
- W4387323413 hasPublicationYear "2023" @default.
- W4387323413 type Work @default.
- W4387323413 citedByCount "0" @default.
- W4387323413 crossrefType "posted-content" @default.
- W4387323413 hasAuthorship W4387323413A5011960878 @default.
- W4387323413 hasAuthorship W4387323413A5033668003 @default.
- W4387323413 hasAuthorship W4387323413A5054629140 @default.
- W4387323413 hasAuthorship W4387323413A5055894819 @default.
- W4387323413 hasAuthorship W4387323413A5059293348 @default.
- W4387323413 hasAuthorship W4387323413A5061429872 @default.
- W4387323413 hasBestOaLocation W43873234131 @default.
- W4387323413 hasConcept C103278499 @default.
- W4387323413 hasConcept C115961682 @default.
- W4387323413 hasConcept C138885662 @default.
- W4387323413 hasConcept C151730666 @default.
- W4387323413 hasConcept C154945302 @default.
- W4387323413 hasConcept C15744967 @default.
- W4387323413 hasConcept C162324750 @default.
- W4387323413 hasConcept C180747234 @default.
- W4387323413 hasConcept C187736073 @default.
- W4387323413 hasConcept C188147891 @default.
- W4387323413 hasConcept C199360897 @default.
- W4387323413 hasConcept C2776035688 @default.
- W4387323413 hasConcept C2779343474 @default.
- W4387323413 hasConcept C2780451532 @default.
- W4387323413 hasConcept C37736160 @default.
- W4387323413 hasConcept C41008148 @default.
- W4387323413 hasConcept C41895202 @default.
- W4387323413 hasConcept C46312422 @default.
- W4387323413 hasConcept C511192102 @default.
- W4387323413 hasConcept C554936623 @default.
- W4387323413 hasConcept C86803240 @default.
- W4387323413 hasConceptScore W4387323413C103278499 @default.
- W4387323413 hasConceptScore W4387323413C115961682 @default.
- W4387323413 hasConceptScore W4387323413C138885662 @default.
- W4387323413 hasConceptScore W4387323413C151730666 @default.
- W4387323413 hasConceptScore W4387323413C154945302 @default.
- W4387323413 hasConceptScore W4387323413C15744967 @default.
- W4387323413 hasConceptScore W4387323413C162324750 @default.
- W4387323413 hasConceptScore W4387323413C180747234 @default.
- W4387323413 hasConceptScore W4387323413C187736073 @default.
- W4387323413 hasConceptScore W4387323413C188147891 @default.
- W4387323413 hasConceptScore W4387323413C199360897 @default.
- W4387323413 hasConceptScore W4387323413C2776035688 @default.
- W4387323413 hasConceptScore W4387323413C2779343474 @default.
- W4387323413 hasConceptScore W4387323413C2780451532 @default.
- W4387323413 hasConceptScore W4387323413C37736160 @default.
- W4387323413 hasConceptScore W4387323413C41008148 @default.
- W4387323413 hasConceptScore W4387323413C41895202 @default.
- W4387323413 hasConceptScore W4387323413C46312422 @default.
- W4387323413 hasConceptScore W4387323413C511192102 @default.
- W4387323413 hasConceptScore W4387323413C554936623 @default.
- W4387323413 hasConceptScore W4387323413C86803240 @default.
- W4387323413 hasLocation W43873234131 @default.
- W4387323413 hasOpenAccess W4387323413 @default.
- W4387323413 hasPrimaryLocation W43873234131 @default.
- W4387323413 hasRelatedWork W1561927205 @default.
- W4387323413 hasRelatedWork W2482350142 @default.
- W4387323413 hasRelatedWork W2502115930 @default.
- W4387323413 hasRelatedWork W3126451824 @default.
- W4387323413 hasRelatedWork W3176240006 @default.
- W4387323413 hasRelatedWork W3191453585 @default.
- W4387323413 hasRelatedWork W4246396837 @default.
- W4387323413 hasRelatedWork W4285226279 @default.
- W4387323413 hasRelatedWork W4297672492 @default.
- W4387323413 hasRelatedWork W4310988119 @default.
- W4387323413 isParatext "false" @default.
- W4387323413 isRetracted "false" @default.
- W4387323413 workType "article" @default.