Matches in SemOpenAlex for { <https://semopenalex.org/work/W4310631979> ?p ?o ?g. }
Showing items 1 to 58 of
58
with 100 items per page.
- W4310631979 abstract "Pretrained language models that have been trained to predict the next word over billions of text documents have been shown to also significantly predict brain recordings of people comprehending language. Understanding the reasons behind the observed similarities between language in machines and language in the brain can lead to more insight into both systems. Recent works suggest that the prediction of the next word is a key mechanism that contributes to the alignment between the two. What is not yet understood is whether prediction of the next word is necessary for this observed alignment or simply sufficient, and whether there are other shared mechanisms or information that is similarly important. In this work, we take a first step towards a better understanding via two simple perturbations in a popular pretrained language model. The first perturbation is to improve the model's ability to predict the next word in the specific naturalistic stimulus text that the brain recordings correspond to. We show that this indeed improves the alignment with the brain recordings. However, this improved alignment may also be due to any improved word-level or multi-word level semantics for the specific world that is described by the stimulus narrative. We aim to disentangle the contribution of next word prediction and semantic knowledge via our second perturbation: scrambling the word order at inference time, which reduces the ability to predict the next word, but maintains any newly learned word-level semantics. By comparing the alignment with brain recordings of these differently perturbed models, we show that improvements in alignment with brain recordings are due to more than improvements in next word prediction and word-level semantics." @default.
- W4310631979 created "2022-12-13" @default.
- W4310631979 creator A5019637022 @default.
- W4310631979 creator A5072973927 @default.
- W4310631979 date "2022-12-01" @default.
- W4310631979 modified "2023-10-16" @default.
- W4310631979 title "Language models and brain alignment: beyond word-level semantics and prediction" @default.
- W4310631979 doi "https://doi.org/10.48550/arxiv.2212.00596" @default.
- W4310631979 hasPublicationYear "2022" @default.
- W4310631979 type Work @default.
- W4310631979 citedByCount "0" @default.
- W4310631979 crossrefType "posted-content" @default.
- W4310631979 hasAuthorship W4310631979A5019637022 @default.
- W4310631979 hasAuthorship W4310631979A5072973927 @default.
- W4310631979 hasBestOaLocation W43106319791 @default.
- W4310631979 hasConcept C11413529 @default.
- W4310631979 hasConcept C137293760 @default.
- W4310631979 hasConcept C138885662 @default.
- W4310631979 hasConcept C154945302 @default.
- W4310631979 hasConcept C182548165 @default.
- W4310631979 hasConcept C184337299 @default.
- W4310631979 hasConcept C199360897 @default.
- W4310631979 hasConcept C204321447 @default.
- W4310631979 hasConcept C2776214188 @default.
- W4310631979 hasConcept C41008148 @default.
- W4310631979 hasConcept C41895202 @default.
- W4310631979 hasConcept C70777604 @default.
- W4310631979 hasConcept C90805587 @default.
- W4310631979 hasConceptScore W4310631979C11413529 @default.
- W4310631979 hasConceptScore W4310631979C137293760 @default.
- W4310631979 hasConceptScore W4310631979C138885662 @default.
- W4310631979 hasConceptScore W4310631979C154945302 @default.
- W4310631979 hasConceptScore W4310631979C182548165 @default.
- W4310631979 hasConceptScore W4310631979C184337299 @default.
- W4310631979 hasConceptScore W4310631979C199360897 @default.
- W4310631979 hasConceptScore W4310631979C204321447 @default.
- W4310631979 hasConceptScore W4310631979C2776214188 @default.
- W4310631979 hasConceptScore W4310631979C41008148 @default.
- W4310631979 hasConceptScore W4310631979C41895202 @default.
- W4310631979 hasConceptScore W4310631979C70777604 @default.
- W4310631979 hasConceptScore W4310631979C90805587 @default.
- W4310631979 hasLocation W43106319791 @default.
- W4310631979 hasLocation W43106319792 @default.
- W4310631979 hasOpenAccess W4310631979 @default.
- W4310631979 hasPrimaryLocation W43106319791 @default.
- W4310631979 hasRelatedWork W2021222228 @default.
- W4310631979 hasRelatedWork W2252084125 @default.
- W4310631979 hasRelatedWork W2359001871 @default.
- W4310631979 hasRelatedWork W2724648934 @default.
- W4310631979 hasRelatedWork W2774843407 @default.
- W4310631979 hasRelatedWork W3010761119 @default.
- W4310631979 hasRelatedWork W4320486724 @default.
- W4310631979 hasRelatedWork W4385452111 @default.
- W4310631979 hasRelatedWork W4385571594 @default.
- W4310631979 hasRelatedWork W2121178211 @default.
- W4310631979 isParatext "false" @default.
- W4310631979 isRetracted "false" @default.
- W4310631979 workType "article" @default.