Matches in SemOpenAlex for { <https://semopenalex.org/work/W3097861677> ?p ?o ?g. }
- W3097861677 endingPage "1234" @default.
- W3097861677 startingPage "1219" @default.
- W3097861677 abstract "Pretrained language models achieve state-of-the-art results on many NLP tasks, but there are still many open questions about how and why they work so well. We investigate the contextualization of words in BERT. We quantify the amount of contextualization, i.e., how well words are interpreted in context, by studying the extent to which semantic classes of a word can be inferred from its contextualized embedding. Quantifying contextualization helps in understanding and utilizing pretrained language models. We show that the top layer representations support highly accurate inference of semantic classes; that the strongest contextualization effects occur in the lower layers; that local context is mostly sufficient for contextualizing words; and that top layer representations are more task-specific after finetuning while lower layer representations are more transferable. Finetuning uncovers task-related features, but pretrained knowledge about contextualization is still well preserved." @default.
- W3097861677 created "2020-11-09" @default.
- W3097861677 creator A5012225484 @default.
- W3097861677 creator A5031030600 @default.
- W3097861677 creator A5049682051 @default.
- W3097861677 creator A5071144367 @default.
- W3097861677 date "2020-01-01" @default.
- W3097861677 modified "2023-09-26" @default.
- W3097861677 title "Quantifying the Contextualization of Word Representations with Semantic Class Probing" @default.
- W3097861677 cites W131533222 @default.
- W3097861677 cites W135437175 @default.
- W3097861677 cites W1579321212 @default.
- W3097861677 cites W1632114991 @default.
- W3097861677 cites W2008652694 @default.
- W3097861677 cites W2026185168 @default.
- W3097861677 cites W2035408139 @default.
- W3097861677 cites W2038721957 @default.
- W3097861677 cites W2040004971 @default.
- W3097861677 cites W2064675550 @default.
- W3097861677 cites W2065030658 @default.
- W3097861677 cites W2066444522 @default.
- W3097861677 cites W2116715664 @default.
- W3097861677 cites W2144578941 @default.
- W3097861677 cites W2153579005 @default.
- W3097861677 cites W2159882563 @default.
- W3097861677 cites W2250539671 @default.
- W3097861677 cites W2251939518 @default.
- W3097861677 cites W22702538 @default.
- W3097861677 cites W2493916176 @default.
- W3097861677 cites W2549835527 @default.
- W3097861677 cites W2740782137 @default.
- W3097861677 cites W2888329843 @default.
- W3097861677 cites W2906152891 @default.
- W3097861677 cites W2907849599 @default.
- W3097861677 cites W2910243263 @default.
- W3097861677 cites W2915128308 @default.
- W3097861677 cites W2951941824 @default.
- W3097861677 cites W2953307569 @default.
- W3097861677 cites W2962739339 @default.
- W3097861677 cites W2962776659 @default.
- W3097861677 cites W2962777840 @default.
- W3097861677 cites W2963026768 @default.
- W3097861677 cites W2963310665 @default.
- W3097861677 cites W2963341956 @default.
- W3097861677 cites W2963403868 @default.
- W3097861677 cites W2963421945 @default.
- W3097861677 cites W2963558636 @default.
- W3097861677 cites W2963756346 @default.
- W3097861677 cites W2963809228 @default.
- W3097861677 cites W2964121744 @default.
- W3097861677 cites W2964204621 @default.
- W3097861677 cites W2964303116 @default.
- W3097861677 cites W2970120757 @default.
- W3097861677 cites W2970352191 @default.
- W3097861677 cites W2970597249 @default.
- W3097861677 cites W2970820321 @default.
- W3097861677 cites W2970862333 @default.
- W3097861677 cites W2970971581 @default.
- W3097861677 cites W2971033911 @default.
- W3097861677 cites W2971569798 @default.
- W3097861677 cites W2972324944 @default.
- W3097861677 cites W2973154008 @default.
- W3097861677 cites W2989195139 @default.
- W3097861677 cites W2990704537 @default.
- W3097861677 cites W2995446988 @default.
- W3097861677 cites W2998554035 @default.
- W3097861677 cites W3022717752 @default.
- W3097861677 cites W3035153870 @default.
- W3097861677 cites W3037282604 @default.
- W3097861677 cites W7542544 @default.
- W3097861677 doi "https://doi.org/10.5282/ubm/epub.74039" @default.
- W3097861677 hasPublicationYear "2020" @default.
- W3097861677 type Work @default.
- W3097861677 sameAs 3097861677 @default.
- W3097861677 citedByCount "6" @default.
- W3097861677 countsByYear W30978616772020 @default.
- W3097861677 countsByYear W30978616772021 @default.
- W3097861677 crossrefType "proceedings-article" @default.
- W3097861677 hasAuthorship W3097861677A5012225484 @default.
- W3097861677 hasAuthorship W3097861677A5031030600 @default.
- W3097861677 hasAuthorship W3097861677A5049682051 @default.
- W3097861677 hasAuthorship W3097861677A5071144367 @default.
- W3097861677 hasConcept C137293760 @default.
- W3097861677 hasConcept C138885662 @default.
- W3097861677 hasConcept C151730666 @default.
- W3097861677 hasConcept C154945302 @default.
- W3097861677 hasConcept C162324750 @default.
- W3097861677 hasConcept C178790620 @default.
- W3097861677 hasConcept C185592680 @default.
- W3097861677 hasConcept C187736073 @default.
- W3097861677 hasConcept C199360897 @default.
- W3097861677 hasConcept C204321447 @default.
- W3097861677 hasConcept C2776214188 @default.
- W3097861677 hasConcept C2779227376 @default.
- W3097861677 hasConcept C2779343474 @default.
- W3097861677 hasConcept C2780451532 @default.
- W3097861677 hasConcept C2780712339 @default.
- W3097861677 hasConcept C41008148 @default.