Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571791> ?p ?o ?g. }
Showing items 1 to 61 of
61
with 100 items per page.
- W4385571791 abstract "Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent work has shown that a zero-pass approach, where parameters are interpreted directly without a forward/backward pass is feasible for some Transformer parameters, and for two-layer attention networks. In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on. We derive a simple theoretical framework to support our arguments and provide ample evidence for its validity. First, an empirical analysis showing that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Second, we present two applications of our framework: (a) aligning the parameters of different models that share a vocabulary, and (b) constructing a classifier without training by “translating” the parameters of a fine-tuned classifier to parameters of a different model that was only pretrained. Overall, our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only." @default.
- W4385571791 created "2023-08-05" @default.
- W4385571791 creator A5007128951 @default.
- W4385571791 creator A5022734062 @default.
- W4385571791 creator A5045872048 @default.
- W4385571791 creator A5065717258 @default.
- W4385571791 date "2023-01-01" @default.
- W4385571791 modified "2023-09-24" @default.
- W4385571791 title "Analyzing Transformers in Embedding Space" @default.
- W4385571791 doi "https://doi.org/10.18653/v1/2023.acl-long.893" @default.
- W4385571791 hasPublicationYear "2023" @default.
- W4385571791 type Work @default.
- W4385571791 citedByCount "0" @default.
- W4385571791 crossrefType "proceedings-article" @default.
- W4385571791 hasAuthorship W4385571791A5007128951 @default.
- W4385571791 hasAuthorship W4385571791A5022734062 @default.
- W4385571791 hasAuthorship W4385571791A5045872048 @default.
- W4385571791 hasAuthorship W4385571791A5065717258 @default.
- W4385571791 hasBestOaLocation W43855717911 @default.
- W4385571791 hasConcept C119599485 @default.
- W4385571791 hasConcept C119857082 @default.
- W4385571791 hasConcept C127413603 @default.
- W4385571791 hasConcept C138885662 @default.
- W4385571791 hasConcept C154945302 @default.
- W4385571791 hasConcept C165801399 @default.
- W4385571791 hasConcept C2777601683 @default.
- W4385571791 hasConcept C2781067378 @default.
- W4385571791 hasConcept C41008148 @default.
- W4385571791 hasConcept C41608201 @default.
- W4385571791 hasConcept C41895202 @default.
- W4385571791 hasConcept C66322947 @default.
- W4385571791 hasConcept C95623464 @default.
- W4385571791 hasConceptScore W4385571791C119599485 @default.
- W4385571791 hasConceptScore W4385571791C119857082 @default.
- W4385571791 hasConceptScore W4385571791C127413603 @default.
- W4385571791 hasConceptScore W4385571791C138885662 @default.
- W4385571791 hasConceptScore W4385571791C154945302 @default.
- W4385571791 hasConceptScore W4385571791C165801399 @default.
- W4385571791 hasConceptScore W4385571791C2777601683 @default.
- W4385571791 hasConceptScore W4385571791C2781067378 @default.
- W4385571791 hasConceptScore W4385571791C41008148 @default.
- W4385571791 hasConceptScore W4385571791C41608201 @default.
- W4385571791 hasConceptScore W4385571791C41895202 @default.
- W4385571791 hasConceptScore W4385571791C66322947 @default.
- W4385571791 hasConceptScore W4385571791C95623464 @default.
- W4385571791 hasLocation W43855717911 @default.
- W4385571791 hasOpenAccess W4385571791 @default.
- W4385571791 hasPrimaryLocation W43855717911 @default.
- W4385571791 hasRelatedWork W1986582023 @default.
- W4385571791 hasRelatedWork W2556319748 @default.
- W4385571791 hasRelatedWork W2968260065 @default.
- W4385571791 hasRelatedWork W3006943036 @default.
- W4385571791 hasRelatedWork W3200179079 @default.
- W4385571791 hasRelatedWork W4200511449 @default.
- W4385571791 hasRelatedWork W4206534706 @default.
- W4385571791 hasRelatedWork W4229079080 @default.
- W4385571791 hasRelatedWork W4297645476 @default.
- W4385571791 hasRelatedWork W4299487748 @default.
- W4385571791 isParatext "false" @default.
- W4385571791 isRetracted "false" @default.
- W4385571791 workType "article" @default.