Matches in SemOpenAlex for { <https://semopenalex.org/work/W3199755307> ?p ?o ?g. }
- W3199755307 abstract "Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models. In this work, we extend the probing studies to two other models in the family, namely ELECTRA and XLNet, showing that variations in the pre-training objectives or architectural choices can result in different behaviors in encoding linguistic information in the representations. Most notably, we observe that ELECTRA tends to encode linguistic knowledge in the deeper layers, whereas XLNet instead concentrates that in the earlier layers. Also, the former model undergoes a slight change during fine-tuning, whereas the latter experiences significant adjustments. Moreover, we show that drawing conclusions based on the weight mixing evaluation strategy -- which is widely used in the context of layer-wise probing -- can be misleading given the norm disparity of the representations across different layers. Instead, we adopt an alternative information-theoretic probing with minimum description length, which has recently been proven to provide more reliable and informative results." @default.
- W3199755307 created "2021-09-27" @default.
- W3199755307 creator A5004874606 @default.
- W3199755307 creator A5028223983 @default.
- W3199755307 creator A5031986521 @default.
- W3199755307 creator A5077490282 @default.
- W3199755307 creator A5091017313 @default.
- W3199755307 date "2021-09-13" @default.
- W3199755307 modified "2023-09-27" @default.
- W3199755307 title "Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations" @default.
- W3199755307 cites W2081580037 @default.
- W3199755307 cites W2160654481 @default.
- W3199755307 cites W2250263931 @default.
- W3199755307 cites W2251939518 @default.
- W3199755307 cites W2914924671 @default.
- W3199755307 cites W2946359678 @default.
- W3199755307 cites W2946417913 @default.
- W3199755307 cites W2948947170 @default.
- W3199755307 cites W2953369973 @default.
- W3199755307 cites W2962739339 @default.
- W3199755307 cites W2963341956 @default.
- W3199755307 cites W2963403868 @default.
- W3199755307 cites W2963846996 @default.
- W3199755307 cites W2964110616 @default.
- W3199755307 cites W2964204621 @default.
- W3199755307 cites W2964303116 @default.
- W3199755307 cites W2970597249 @default.
- W3199755307 cites W2970820321 @default.
- W3199755307 cites W2970862333 @default.
- W3199755307 cites W2978670439 @default.
- W3199755307 cites W3035750922 @default.
- W3199755307 cites W3037530970 @default.
- W3199755307 cites W3087873698 @default.
- W3199755307 cites W3098300729 @default.
- W3199755307 cites W3098824823 @default.
- W3199755307 cites W3100308117 @default.
- W3199755307 cites W3103368673 @default.
- W3199755307 cites W3104350794 @default.
- W3199755307 cites W3106290101 @default.
- W3199755307 cites W3111372685 @default.
- W3199755307 cites W3118485687 @default.
- W3199755307 cites W3123806455 @default.
- W3199755307 cites W3152409010 @default.
- W3199755307 cites W3174082608 @default.
- W3199755307 cites W3176807265 @default.
- W3199755307 cites W630532510 @default.
- W3199755307 hasPublicationYear "2021" @default.
- W3199755307 type Work @default.
- W3199755307 sameAs 3199755307 @default.
- W3199755307 citedByCount "0" @default.
- W3199755307 crossrefType "posted-content" @default.
- W3199755307 hasAuthorship W3199755307A5004874606 @default.
- W3199755307 hasAuthorship W3199755307A5028223983 @default.
- W3199755307 hasAuthorship W3199755307A5031986521 @default.
- W3199755307 hasAuthorship W3199755307A5077490282 @default.
- W3199755307 hasAuthorship W3199755307A5091017313 @default.
- W3199755307 hasConcept C104317684 @default.
- W3199755307 hasConcept C111472728 @default.
- W3199755307 hasConcept C125411270 @default.
- W3199755307 hasConcept C138885662 @default.
- W3199755307 hasConcept C154945302 @default.
- W3199755307 hasConcept C166957645 @default.
- W3199755307 hasConcept C17744445 @default.
- W3199755307 hasConcept C178790620 @default.
- W3199755307 hasConcept C185592680 @default.
- W3199755307 hasConcept C191795146 @default.
- W3199755307 hasConcept C199539241 @default.
- W3199755307 hasConcept C204321447 @default.
- W3199755307 hasConcept C2779227376 @default.
- W3199755307 hasConcept C2779343474 @default.
- W3199755307 hasConcept C2780253743 @default.
- W3199755307 hasConcept C41008148 @default.
- W3199755307 hasConcept C41895202 @default.
- W3199755307 hasConcept C55493867 @default.
- W3199755307 hasConcept C66746571 @default.
- W3199755307 hasConcept C95457728 @default.
- W3199755307 hasConceptScore W3199755307C104317684 @default.
- W3199755307 hasConceptScore W3199755307C111472728 @default.
- W3199755307 hasConceptScore W3199755307C125411270 @default.
- W3199755307 hasConceptScore W3199755307C138885662 @default.
- W3199755307 hasConceptScore W3199755307C154945302 @default.
- W3199755307 hasConceptScore W3199755307C166957645 @default.
- W3199755307 hasConceptScore W3199755307C17744445 @default.
- W3199755307 hasConceptScore W3199755307C178790620 @default.
- W3199755307 hasConceptScore W3199755307C185592680 @default.
- W3199755307 hasConceptScore W3199755307C191795146 @default.
- W3199755307 hasConceptScore W3199755307C199539241 @default.
- W3199755307 hasConceptScore W3199755307C204321447 @default.
- W3199755307 hasConceptScore W3199755307C2779227376 @default.
- W3199755307 hasConceptScore W3199755307C2779343474 @default.
- W3199755307 hasConceptScore W3199755307C2780253743 @default.
- W3199755307 hasConceptScore W3199755307C41008148 @default.
- W3199755307 hasConceptScore W3199755307C41895202 @default.
- W3199755307 hasConceptScore W3199755307C55493867 @default.
- W3199755307 hasConceptScore W3199755307C66746571 @default.
- W3199755307 hasConceptScore W3199755307C95457728 @default.
- W3199755307 hasLocation W31997553071 @default.
- W3199755307 hasOpenAccess W3199755307 @default.
- W3199755307 hasPrimaryLocation W31997553071 @default.
- W3199755307 hasRelatedWork W1522294902 @default.