Matches in SemOpenAlex for { <https://semopenalex.org/work/W4221154557> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4221154557 abstract "The Transformer architecture aggregates input information through the self-attention mechanism, but there is no clear understanding of how this information is mixed across the entire model. Additionally, recent works have demonstrated that attention weights alone are not enough to describe the flow of information. In this paper, we consider the whole attention block -- multi-head attention, residual connection, and layer normalization -- and define a metric to measure token-to-token interactions within each layer. Then, we aggregate layer-wise interpretations to provide input attribution scores for model predictions. Experimentally, we show that our method, ALTI (Aggregation of Layer-wise Token-to-token Interactions), provides more faithful explanations and increased robustness than gradient-based methods." @default.
- W4221154557 created "2022-04-03" @default.
- W4221154557 creator A5012821419 @default.
- W4221154557 creator A5014771820 @default.
- W4221154557 creator A5074210163 @default.
- W4221154557 date "2022-03-08" @default.
- W4221154557 modified "2023-09-23" @default.
- W4221154557 title "Measuring the Mixing of Contextual Information in the Transformer" @default.
- W4221154557 doi "https://doi.org/10.48550/arxiv.2203.04212" @default.
- W4221154557 hasPublicationYear "2022" @default.
- W4221154557 type Work @default.
- W4221154557 citedByCount "0" @default.
- W4221154557 crossrefType "posted-content" @default.
- W4221154557 hasAuthorship W4221154557A5012821419 @default.
- W4221154557 hasAuthorship W4221154557A5014771820 @default.
- W4221154557 hasAuthorship W4221154557A5074210163 @default.
- W4221154557 hasBestOaLocation W42211545571 @default.
- W4221154557 hasConcept C104317684 @default.
- W4221154557 hasConcept C11413529 @default.
- W4221154557 hasConcept C119599485 @default.
- W4221154557 hasConcept C127413603 @default.
- W4221154557 hasConcept C136886441 @default.
- W4221154557 hasConcept C144024400 @default.
- W4221154557 hasConcept C154945302 @default.
- W4221154557 hasConcept C155512373 @default.
- W4221154557 hasConcept C165801399 @default.
- W4221154557 hasConcept C185592680 @default.
- W4221154557 hasConcept C19165224 @default.
- W4221154557 hasConcept C31258907 @default.
- W4221154557 hasConcept C41008148 @default.
- W4221154557 hasConcept C48145219 @default.
- W4221154557 hasConcept C55493867 @default.
- W4221154557 hasConcept C63479239 @default.
- W4221154557 hasConcept C66322947 @default.
- W4221154557 hasConceptScore W4221154557C104317684 @default.
- W4221154557 hasConceptScore W4221154557C11413529 @default.
- W4221154557 hasConceptScore W4221154557C119599485 @default.
- W4221154557 hasConceptScore W4221154557C127413603 @default.
- W4221154557 hasConceptScore W4221154557C136886441 @default.
- W4221154557 hasConceptScore W4221154557C144024400 @default.
- W4221154557 hasConceptScore W4221154557C154945302 @default.
- W4221154557 hasConceptScore W4221154557C155512373 @default.
- W4221154557 hasConceptScore W4221154557C165801399 @default.
- W4221154557 hasConceptScore W4221154557C185592680 @default.
- W4221154557 hasConceptScore W4221154557C19165224 @default.
- W4221154557 hasConceptScore W4221154557C31258907 @default.
- W4221154557 hasConceptScore W4221154557C41008148 @default.
- W4221154557 hasConceptScore W4221154557C48145219 @default.
- W4221154557 hasConceptScore W4221154557C55493867 @default.
- W4221154557 hasConceptScore W4221154557C63479239 @default.
- W4221154557 hasConceptScore W4221154557C66322947 @default.
- W4221154557 hasLocation W42211545571 @default.
- W4221154557 hasOpenAccess W4221154557 @default.
- W4221154557 hasPrimaryLocation W42211545571 @default.
- W4221154557 hasRelatedWork W2375389409 @default.
- W4221154557 hasRelatedWork W2787684247 @default.
- W4221154557 hasRelatedWork W2971191050 @default.
- W4221154557 hasRelatedWork W3007689282 @default.
- W4221154557 hasRelatedWork W3035794324 @default.
- W4221154557 hasRelatedWork W3100971012 @default.
- W4221154557 hasRelatedWork W3101785758 @default.
- W4221154557 hasRelatedWork W4205427372 @default.
- W4221154557 hasRelatedWork W4287178710 @default.
- W4221154557 hasRelatedWork W4289148071 @default.
- W4221154557 isParatext "false" @default.
- W4221154557 isRetracted "false" @default.
- W4221154557 workType "article" @default.