Matches in SemOpenAlex for { <https://semopenalex.org/work/W4306352814> ?p ?o ?g. }
Showing items 1 to 66 of
66
with 100 items per page.
- W4306352814 abstract "State-of-the-art pretrained contextualized models (PCM) eg. BERT use tasks such as WiC and WSD to evaluate their word-in-context representations. This inherently assumes that performance in these tasks reflect how well a model represents the coupled word and context semantics. We question this assumption by presenting the first quantitative analysis on the context-word interaction being tested in major contextual lexical semantic tasks. To achieve this, we run probing baselines on masked input, and propose measures to calculate and visualize the degree of context or word biases in existing datasets. The analysis was performed on both models and humans. Our findings demonstrate that models are usually not being tested for word-in-context semantics in the same way as humans are in these tasks, which helps us better understand the model-human gap. Specifically, to PCMs, most existing datasets fall into the extreme ends (the retrieval-based tasks exhibit strong target word bias while WiC-style tasks and WSD show strong context bias); In comparison, humans are less biased and achieve much better performance when both word and context are available than with masked input. We recommend our framework for understanding and controlling these biases for model interpretation and future task design." @default.
- W4306352814 created "2022-10-16" @default.
- W4306352814 creator A5014017147 @default.
- W4306352814 creator A5081393566 @default.
- W4306352814 creator A5063848163 @default.
- W4306352814 date "2021-12-13" @default.
- W4306352814 modified "2023-10-17" @default.
- W4306352814 title "Measuring Context-Word Biases in Lexical Semantic Datasets" @default.
- W4306352814 doi "https://doi.org/10.48550/arxiv.2112.06733" @default.
- W4306352814 hasPublicationYear "2021" @default.
- W4306352814 type Work @default.
- W4306352814 citedByCount "0" @default.
- W4306352814 crossrefType "posted-content" @default.
- W4306352814 hasAuthorship W4306352814A5014017147 @default.
- W4306352814 hasAuthorship W4306352814A5063848163 @default.
- W4306352814 hasAuthorship W4306352814A5081393566 @default.
- W4306352814 hasBestOaLocation W43063528141 @default.
- W4306352814 hasConcept C130318100 @default.
- W4306352814 hasConcept C138885662 @default.
- W4306352814 hasConcept C151730666 @default.
- W4306352814 hasConcept C154945302 @default.
- W4306352814 hasConcept C162324750 @default.
- W4306352814 hasConcept C184337299 @default.
- W4306352814 hasConcept C187736073 @default.
- W4306352814 hasConcept C199360897 @default.
- W4306352814 hasConcept C204321447 @default.
- W4306352814 hasConcept C2778828372 @default.
- W4306352814 hasConcept C2779343474 @default.
- W4306352814 hasConcept C2780451532 @default.
- W4306352814 hasConcept C41008148 @default.
- W4306352814 hasConcept C41895202 @default.
- W4306352814 hasConcept C86803240 @default.
- W4306352814 hasConcept C90805587 @default.
- W4306352814 hasConceptScore W4306352814C130318100 @default.
- W4306352814 hasConceptScore W4306352814C138885662 @default.
- W4306352814 hasConceptScore W4306352814C151730666 @default.
- W4306352814 hasConceptScore W4306352814C154945302 @default.
- W4306352814 hasConceptScore W4306352814C162324750 @default.
- W4306352814 hasConceptScore W4306352814C184337299 @default.
- W4306352814 hasConceptScore W4306352814C187736073 @default.
- W4306352814 hasConceptScore W4306352814C199360897 @default.
- W4306352814 hasConceptScore W4306352814C204321447 @default.
- W4306352814 hasConceptScore W4306352814C2778828372 @default.
- W4306352814 hasConceptScore W4306352814C2779343474 @default.
- W4306352814 hasConceptScore W4306352814C2780451532 @default.
- W4306352814 hasConceptScore W4306352814C41008148 @default.
- W4306352814 hasConceptScore W4306352814C41895202 @default.
- W4306352814 hasConceptScore W4306352814C86803240 @default.
- W4306352814 hasConceptScore W4306352814C90805587 @default.
- W4306352814 hasLocation W43063528141 @default.
- W4306352814 hasLocation W43063528142 @default.
- W4306352814 hasOpenAccess W4306352814 @default.
- W4306352814 hasPrimaryLocation W43063528141 @default.
- W4306352814 hasRelatedWork W2061150891 @default.
- W4306352814 hasRelatedWork W2112408611 @default.
- W4306352814 hasRelatedWork W2118772635 @default.
- W4306352814 hasRelatedWork W2808113865 @default.
- W4306352814 hasRelatedWork W2963364736 @default.
- W4306352814 hasRelatedWork W2970166416 @default.
- W4306352814 hasRelatedWork W3046869600 @default.
- W4306352814 hasRelatedWork W3112765659 @default.
- W4306352814 hasRelatedWork W3125649664 @default.
- W4306352814 hasRelatedWork W4232784337 @default.
- W4306352814 isParatext "false" @default.
- W4306352814 isRetracted "false" @default.
- W4306352814 workType "article" @default.