Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288368497> ?p ?o ?g. }
- W4288368497 abstract "Abstract Humans use complex linguistic structures to transmit ideas to one another. The brain is thought to deploy specialized computations to process these structures. Recently, a new class of artificial neural networks based on the Transformer architecture has revolutionized the field of language modeling, attracting attention from neuroscientists seeking to understand the neurobiology of language in silico . Transformers integrate information across words via multiple layers of structured circuit computations, forming increasingly contextualized representations of linguistic content. Prior work has focused on the internal representations (the “embeddings”) generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into functionally-specialized “transformations” to provide a complementary window onto linguistic computations in the human brain. Using functional MRI data acquired while participants listened to naturalistic spoken stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent syntactic computations performed by individual, functionally-specialized “attention heads” differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers, contextual distances, and syntactic dependencies in a low-dimensional cortical space. Our findings indicate that large language models and the cortical language network may converge on similar trends of functional specialization for processing natural language." @default.
- W4288368497 created "2022-07-29" @default.
- W4288368497 creator A5001928585 @default.
- W4288368497 creator A5047918106 @default.
- W4288368497 creator A5056266198 @default.
- W4288368497 creator A5065697534 @default.
- W4288368497 creator A5067264638 @default.
- W4288368497 creator A5070059815 @default.
- W4288368497 creator A5074948482 @default.
- W4288368497 creator A5077079119 @default.
- W4288368497 creator A5078519295 @default.
- W4288368497 date "2022-06-09" @default.
- W4288368497 modified "2023-10-16" @default.
- W4288368497 title "Shared functional specialization in transformer-based language models and the human brain" @default.
- W4288368497 cites W1507893557 @default.
- W4288368497 cites W1715013381 @default.
- W4288368497 cites W1920463315 @default.
- W4288368497 cites W1965248225 @default.
- W4288368497 cites W1982352919 @default.
- W4288368497 cites W1982972718 @default.
- W4288368497 cites W1985086416 @default.
- W4288368497 cites W1985466488 @default.
- W4288368497 cites W1988812422 @default.
- W4288368497 cites W1990689692 @default.
- W4288368497 cites W1991825895 @default.
- W4288368497 cites W1992570774 @default.
- W4288368497 cites W2011589116 @default.
- W4288368497 cites W2016738977 @default.
- W4288368497 cites W2024270062 @default.
- W4288368497 cites W2028041302 @default.
- W4288368497 cites W2036447011 @default.
- W4288368497 cites W2036531193 @default.
- W4288368497 cites W2038201838 @default.
- W4288368497 cites W2049980174 @default.
- W4288368497 cites W2051220130 @default.
- W4288368497 cites W2057307785 @default.
- W4288368497 cites W2063951486 @default.
- W4288368497 cites W2064304610 @default.
- W4288368497 cites W2064675550 @default.
- W4288368497 cites W2072923434 @default.
- W4288368497 cites W2096462008 @default.
- W4288368497 cites W2098782906 @default.
- W4288368497 cites W2101807845 @default.
- W4288368497 cites W2102506398 @default.
- W4288368497 cites W2110485445 @default.
- W4288368497 cites W2112796928 @default.
- W4288368497 cites W2112891119 @default.
- W4288368497 cites W2115442027 @default.
- W4288368497 cites W2117140276 @default.
- W4288368497 cites W2118992292 @default.
- W4288368497 cites W2121319176 @default.
- W4288368497 cites W2124799830 @default.
- W4288368497 cites W2128672892 @default.
- W4288368497 cites W2130010412 @default.
- W4288368497 cites W2130074181 @default.
- W4288368497 cites W2132473597 @default.
- W4288368497 cites W2135335872 @default.
- W4288368497 cites W2139297841 @default.
- W4288368497 cites W2155034734 @default.
- W4288368497 cites W2158431045 @default.
- W4288368497 cites W2162593248 @default.
- W4288368497 cites W2168217710 @default.
- W4288368497 cites W2169281351 @default.
- W4288368497 cites W2171045029 @default.
- W4288368497 cites W2176287621 @default.
- W4288368497 cites W2186059026 @default.
- W4288368497 cites W2194775991 @default.
- W4288368497 cites W2195506630 @default.
- W4288368497 cites W2250539671 @default.
- W4288368497 cites W2274405424 @default.
- W4288368497 cites W2344975321 @default.
- W4288368497 cites W2480148714 @default.
- W4288368497 cites W2592971502 @default.
- W4288368497 cites W2618530766 @default.
- W4288368497 cites W2622627557 @default.
- W4288368497 cites W2755473124 @default.
- W4288368497 cites W2755954527 @default.
- W4288368497 cites W2782213998 @default.
- W4288368497 cites W2883992483 @default.
- W4288368497 cites W2908124316 @default.
- W4288368497 cites W2946417913 @default.
- W4288368497 cites W2950457889 @default.
- W4288368497 cites W2951103577 @default.
- W4288368497 cites W2951583631 @default.
- W4288368497 cites W2952101405 @default.
- W4288368497 cites W2952124519 @default.
- W4288368497 cites W2952662516 @default.
- W4288368497 cites W2952856751 @default.
- W4288368497 cites W2963341956 @default.
- W4288368497 cites W2964303116 @default.
- W4288368497 cites W2972324944 @default.
- W4288368497 cites W2972498556 @default.
- W4288368497 cites W2978368159 @default.
- W4288368497 cites W2978450937 @default.
- W4288368497 cites W2979826702 @default.
- W4288368497 cites W2983769105 @default.
- W4288368497 cites W3004619146 @default.
- W4288368497 cites W3006343517 @default.
- W4288368497 cites W3007869734 @default.
- W4288368497 cites W3009009806 @default.