Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288086191> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W4288086191 abstract "Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory. In order to better understand BERT and other Transformer-based models, we present a layer-wise analysis of BERT's hidden states. Unlike previous research, which mainly focuses on explaining Transformer models by their attention weights, we argue that hidden states contain equally valuable information. Specifically, our analysis focuses on models fine-tuned on the task of Question Answering (QA) as an example of a complex downstream task. We inspect how QA models transform token vectors in order to find the correct answer. To this end, we apply a set of general and QA-specific probing tasks that reveal the information stored in each representation layer. Our qualitative analysis of hidden state visualizations provides additional insights into BERT's reasoning process. Our results show that the transformations within BERT go through phases that are related to traditional pipeline tasks. The system can therefore implicitly incorporate task-specific information into its token representations. Furthermore, our analysis reveals that fine-tuning has little impact on the models' semantic abilities and that prediction errors can be recognized in the vector representations of even early layers." @default.
- W4288086191 created "2022-07-28" @default.
- W4288086191 creator A5027602672 @default.
- W4288086191 creator A5044994184 @default.
- W4288086191 creator A5077285800 @default.
- W4288086191 creator A5082325279 @default.
- W4288086191 date "2019-11-03" @default.
- W4288086191 modified "2023-10-06" @default.
- W4288086191 title "How Does BERT Answer Questions?" @default.
- W4288086191 cites W2070246124 @default.
- W4288086191 cites W2150593711 @default.
- W4288086191 cites W2295676751 @default.
- W4288086191 cites W2563574619 @default.
- W4288086191 cites W2962739339 @default.
- W4288086191 cites W2962772482 @default.
- W4288086191 cites W2963323070 @default.
- W4288086191 doi "https://doi.org/10.1145/3357384.3358028" @default.
- W4288086191 hasPublicationYear "2019" @default.
- W4288086191 type Work @default.
- W4288086191 citedByCount "32" @default.
- W4288086191 countsByYear W42880861912020 @default.
- W4288086191 countsByYear W42880861912021 @default.
- W4288086191 countsByYear W42880861912022 @default.
- W4288086191 countsByYear W42880861912023 @default.
- W4288086191 crossrefType "proceedings-article" @default.
- W4288086191 hasAuthorship W4288086191A5027602672 @default.
- W4288086191 hasAuthorship W4288086191A5044994184 @default.
- W4288086191 hasAuthorship W4288086191A5077285800 @default.
- W4288086191 hasAuthorship W4288086191A5082325279 @default.
- W4288086191 hasBestOaLocation W42880861912 @default.
- W4288086191 hasConcept C111919701 @default.
- W4288086191 hasConcept C118505674 @default.
- W4288086191 hasConcept C119857082 @default.
- W4288086191 hasConcept C121332964 @default.
- W4288086191 hasConcept C137293760 @default.
- W4288086191 hasConcept C154945302 @default.
- W4288086191 hasConcept C162324750 @default.
- W4288086191 hasConcept C165801399 @default.
- W4288086191 hasConcept C17744445 @default.
- W4288086191 hasConcept C187736073 @default.
- W4288086191 hasConcept C199360897 @default.
- W4288086191 hasConcept C199539241 @default.
- W4288086191 hasConcept C204321447 @default.
- W4288086191 hasConcept C2776359362 @default.
- W4288086191 hasConcept C2780451532 @default.
- W4288086191 hasConcept C38652104 @default.
- W4288086191 hasConcept C41008148 @default.
- W4288086191 hasConcept C43521106 @default.
- W4288086191 hasConcept C44291984 @default.
- W4288086191 hasConcept C48145219 @default.
- W4288086191 hasConcept C62520636 @default.
- W4288086191 hasConcept C66322947 @default.
- W4288086191 hasConcept C94625758 @default.
- W4288086191 hasConcept C98045186 @default.
- W4288086191 hasConceptScore W4288086191C111919701 @default.
- W4288086191 hasConceptScore W4288086191C118505674 @default.
- W4288086191 hasConceptScore W4288086191C119857082 @default.
- W4288086191 hasConceptScore W4288086191C121332964 @default.
- W4288086191 hasConceptScore W4288086191C137293760 @default.
- W4288086191 hasConceptScore W4288086191C154945302 @default.
- W4288086191 hasConceptScore W4288086191C162324750 @default.
- W4288086191 hasConceptScore W4288086191C165801399 @default.
- W4288086191 hasConceptScore W4288086191C17744445 @default.
- W4288086191 hasConceptScore W4288086191C187736073 @default.
- W4288086191 hasConceptScore W4288086191C199360897 @default.
- W4288086191 hasConceptScore W4288086191C199539241 @default.
- W4288086191 hasConceptScore W4288086191C204321447 @default.
- W4288086191 hasConceptScore W4288086191C2776359362 @default.
- W4288086191 hasConceptScore W4288086191C2780451532 @default.
- W4288086191 hasConceptScore W4288086191C38652104 @default.
- W4288086191 hasConceptScore W4288086191C41008148 @default.
- W4288086191 hasConceptScore W4288086191C43521106 @default.
- W4288086191 hasConceptScore W4288086191C44291984 @default.
- W4288086191 hasConceptScore W4288086191C48145219 @default.
- W4288086191 hasConceptScore W4288086191C62520636 @default.
- W4288086191 hasConceptScore W4288086191C66322947 @default.
- W4288086191 hasConceptScore W4288086191C94625758 @default.
- W4288086191 hasConceptScore W4288086191C98045186 @default.
- W4288086191 hasFunder F4320335254 @default.
- W4288086191 hasLocation W42880861911 @default.
- W4288086191 hasLocation W42880861912 @default.
- W4288086191 hasLocation W42880861913 @default.
- W4288086191 hasOpenAccess W4288086191 @default.
- W4288086191 hasPrimaryLocation W42880861911 @default.
- W4288086191 hasRelatedWork W2972312591 @default.
- W4288086191 hasRelatedWork W3016124757 @default.
- W4288086191 hasRelatedWork W3034520363 @default.
- W4288086191 hasRelatedWork W3092323224 @default.
- W4288086191 hasRelatedWork W3098382480 @default.
- W4288086191 hasRelatedWork W3102568136 @default.
- W4288086191 hasRelatedWork W3104417388 @default.
- W4288086191 hasRelatedWork W4221139500 @default.
- W4288086191 hasRelatedWork W4287598411 @default.
- W4288086191 hasRelatedWork W4385572674 @default.
- W4288086191 isParatext "false" @default.
- W4288086191 isRetracted "false" @default.
- W4288086191 workType "article" @default.