Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287759118> ?p ?o ?g. }
Showing items 1 to 68 of
68
with 100 items per page.
- W4287759118 abstract "Attention-based methods have played important roles in model interpretations, where the calculated attention weights are expected to highlight the critical parts of inputs~(e.g., keywords in sentences). However, recent research found that attention-as-importance interpretations often do not work as we expected. For example, learned attention weights sometimes highlight less meaningful tokens like [SEP], ,, and ., and are frequently uncorrelated with other feature importance indicators like gradient-based measures. A recent debate over whether attention is an explanation or not has drawn considerable interest. In this paper, we demonstrate that one root cause of this phenomenon is the combinatorial shortcuts, which means that, in addition to the highlighted parts, the attention weights themselves may carry extra information that could be utilized by downstream models after attention layers. As a result, the attention weights are no longer pure importance indicators. We theoretically analyze combinatorial shortcuts, design one intuitive experiment to show their existence, and propose two methods to mitigate this issue. We conduct empirical studies on attention-based interpretation models. The results show that the proposed methods can effectively improve the interpretability of attention mechanisms." @default.
- W4287759118 created "2022-07-26" @default.
- W4287759118 creator A5019560977 @default.
- W4287759118 creator A5031863894 @default.
- W4287759118 creator A5056582866 @default.
- W4287759118 creator A5074743331 @default.
- W4287759118 creator A5083415156 @default.
- W4287759118 creator A5086529957 @default.
- W4287759118 date "2020-06-10" @default.
- W4287759118 modified "2023-10-17" @default.
- W4287759118 title "Why Attentions May Not Be Interpretable?" @default.
- W4287759118 hasPublicationYear "2020" @default.
- W4287759118 type Work @default.
- W4287759118 citedByCount "0" @default.
- W4287759118 crossrefType "posted-content" @default.
- W4287759118 hasAuthorship W4287759118A5019560977 @default.
- W4287759118 hasAuthorship W4287759118A5031863894 @default.
- W4287759118 hasAuthorship W4287759118A5056582866 @default.
- W4287759118 hasAuthorship W4287759118A5074743331 @default.
- W4287759118 hasAuthorship W4287759118A5083415156 @default.
- W4287759118 hasAuthorship W4287759118A5086529957 @default.
- W4287759118 hasBestOaLocation W42877591181 @default.
- W4287759118 hasConcept C105795698 @default.
- W4287759118 hasConcept C111472728 @default.
- W4287759118 hasConcept C119857082 @default.
- W4287759118 hasConcept C138885662 @default.
- W4287759118 hasConcept C154945302 @default.
- W4287759118 hasConcept C169345407 @default.
- W4287759118 hasConcept C171078966 @default.
- W4287759118 hasConcept C199360897 @default.
- W4287759118 hasConcept C2776401178 @default.
- W4287759118 hasConcept C2781067378 @default.
- W4287759118 hasConcept C33923547 @default.
- W4287759118 hasConcept C41008148 @default.
- W4287759118 hasConcept C41895202 @default.
- W4287759118 hasConcept C50335755 @default.
- W4287759118 hasConcept C527412718 @default.
- W4287759118 hasConceptScore W4287759118C105795698 @default.
- W4287759118 hasConceptScore W4287759118C111472728 @default.
- W4287759118 hasConceptScore W4287759118C119857082 @default.
- W4287759118 hasConceptScore W4287759118C138885662 @default.
- W4287759118 hasConceptScore W4287759118C154945302 @default.
- W4287759118 hasConceptScore W4287759118C169345407 @default.
- W4287759118 hasConceptScore W4287759118C171078966 @default.
- W4287759118 hasConceptScore W4287759118C199360897 @default.
- W4287759118 hasConceptScore W4287759118C2776401178 @default.
- W4287759118 hasConceptScore W4287759118C2781067378 @default.
- W4287759118 hasConceptScore W4287759118C33923547 @default.
- W4287759118 hasConceptScore W4287759118C41008148 @default.
- W4287759118 hasConceptScore W4287759118C41895202 @default.
- W4287759118 hasConceptScore W4287759118C50335755 @default.
- W4287759118 hasConceptScore W4287759118C527412718 @default.
- W4287759118 hasLocation W42877591181 @default.
- W4287759118 hasOpenAccess W4287759118 @default.
- W4287759118 hasPrimaryLocation W42877591181 @default.
- W4287759118 hasRelatedWork W11356396 @default.
- W4287759118 hasRelatedWork W13954494 @default.
- W4287759118 hasRelatedWork W14789944 @default.
- W4287759118 hasRelatedWork W2076915 @default.
- W4287759118 hasRelatedWork W265079 @default.
- W4287759118 hasRelatedWork W354571 @default.
- W4287759118 hasRelatedWork W4085024 @default.
- W4287759118 hasRelatedWork W7002624 @default.
- W4287759118 hasRelatedWork W728297 @default.
- W4287759118 hasRelatedWork W9402503 @default.
- W4287759118 isParatext "false" @default.
- W4287759118 isRetracted "false" @default.
- W4287759118 workType "article" @default.