Matches in SemOpenAlex for { <https://semopenalex.org/work/W4377866534> ?p ?o ?g. }
Showing items 1 to 57 of
57
with 100 items per page.
- W4377866534 abstract "We study the relationship between the entropy of intermediate representations and a model's robustness to distributional shift. We train models consisting of two feed-forward networks end-to-end separated by a discrete $n$-bit channel on an unsupervised contrastive learning task. Different masking strategies are applied after training that remove a proportion of low-entropy bits, high-entropy bits, or randomly selected bits, and the effects on performance are compared to the baseline accuracy with no mask. We hypothesize that the entropy of a bit serves as a guide to its usefulness out-of-distribution (OOD). Through experiment on three OOD datasets we demonstrate that the removal of low-entropy bits can notably benefit OOD performance. Conversely, we find that top-entropy masking disproportionately harms performance both in-distribution (InD) and OOD." @default.
- W4377866534 created "2023-05-24" @default.
- W4377866534 creator A5062598891 @default.
- W4377866534 creator A5065134355 @default.
- W4377866534 date "2023-05-20" @default.
- W4377866534 modified "2023-09-26" @default.
- W4377866534 title "Low-Entropy Latent Variables Hurt Out-of-Distribution Performance" @default.
- W4377866534 doi "https://doi.org/10.48550/arxiv.2305.12238" @default.
- W4377866534 hasPublicationYear "2023" @default.
- W4377866534 type Work @default.
- W4377866534 citedByCount "0" @default.
- W4377866534 crossrefType "posted-content" @default.
- W4377866534 hasAuthorship W4377866534A5062598891 @default.
- W4377866534 hasAuthorship W4377866534A5065134355 @default.
- W4377866534 hasBestOaLocation W43778665341 @default.
- W4377866534 hasConcept C101721835 @default.
- W4377866534 hasConcept C104317684 @default.
- W4377866534 hasConcept C106301342 @default.
- W4377866534 hasConcept C121332964 @default.
- W4377866534 hasConcept C154945302 @default.
- W4377866534 hasConcept C167981619 @default.
- W4377866534 hasConcept C185592680 @default.
- W4377866534 hasConcept C33923547 @default.
- W4377866534 hasConcept C41008148 @default.
- W4377866534 hasConcept C55493867 @default.
- W4377866534 hasConcept C62520636 @default.
- W4377866534 hasConcept C63479239 @default.
- W4377866534 hasConcept C9679016 @default.
- W4377866534 hasConceptScore W4377866534C101721835 @default.
- W4377866534 hasConceptScore W4377866534C104317684 @default.
- W4377866534 hasConceptScore W4377866534C106301342 @default.
- W4377866534 hasConceptScore W4377866534C121332964 @default.
- W4377866534 hasConceptScore W4377866534C154945302 @default.
- W4377866534 hasConceptScore W4377866534C167981619 @default.
- W4377866534 hasConceptScore W4377866534C185592680 @default.
- W4377866534 hasConceptScore W4377866534C33923547 @default.
- W4377866534 hasConceptScore W4377866534C41008148 @default.
- W4377866534 hasConceptScore W4377866534C55493867 @default.
- W4377866534 hasConceptScore W4377866534C62520636 @default.
- W4377866534 hasConceptScore W4377866534C63479239 @default.
- W4377866534 hasConceptScore W4377866534C9679016 @default.
- W4377866534 hasLocation W43778665341 @default.
- W4377866534 hasOpenAccess W4377866534 @default.
- W4377866534 hasPrimaryLocation W43778665341 @default.
- W4377866534 hasRelatedWork W1593940414 @default.
- W4377866534 hasRelatedWork W2032064063 @default.
- W4377866534 hasRelatedWork W2363475415 @default.
- W4377866534 hasRelatedWork W2383844702 @default.
- W4377866534 hasRelatedWork W3009468862 @default.
- W4377866534 hasRelatedWork W3015162217 @default.
- W4377866534 hasRelatedWork W3023313292 @default.
- W4377866534 hasRelatedWork W3034892578 @default.
- W4377866534 hasRelatedWork W4297899318 @default.
- W4377866534 hasRelatedWork W4302012992 @default.
- W4377866534 isParatext "false" @default.
- W4377866534 isRetracted "false" @default.
- W4377866534 workType "article" @default.