Matches in SemOpenAlex for { <https://semopenalex.org/work/W3119259767> ?p ?o ?g. }
- W3119259767 abstract "Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a range of downstream tasks. In this paper we explore whether it is possible to learn disentangled representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspects. Concretely, we learn binary masks over transformer weights or hidden units to uncover subsets of features that correlate with a specific factor of variation; this eliminates the need to train a disentangled model from scratch for a particular task. We evaluate this method with respect to its ability to disentangle representations of sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. By combining masking with magnitude pruning we find that we can identify sparse subnetworks within BERT that strongly encode particular aspects (e.g., semantics) while only weakly encoding others (e.g., syntax). Moreover, despite only learning masks, disentanglement-via-masking performs as well as — and often better than —previously proposed methods based on variational autoencoders and adversarial training." @default.
- W3119259767 created "2021-01-18" @default.
- W3119259767 creator A5036790226 @default.
- W3119259767 creator A5070182106 @default.
- W3119259767 creator A5073129092 @default.
- W3119259767 date "2021-01-01" @default.
- W3119259767 modified "2023-09-26" @default.
- W3119259767 title "Disentangling Representations of Text by Masking Transformers" @default.
- W3119259767 cites W1691728462 @default.
- W3119259767 cites W2113459411 @default.
- W3119259767 cites W2187089797 @default.
- W3119259767 cites W2242818861 @default.
- W3119259767 cites W2460474657 @default.
- W3119259767 cites W2570431255 @default.
- W3119259767 cites W2613332842 @default.
- W3119259767 cites W2753738274 @default.
- W3119259767 cites W2775844704 @default.
- W3119259767 cites W2785961484 @default.
- W3119259767 cites W2888161220 @default.
- W3119259767 cites W2902476877 @default.
- W3119259767 cites W2903538854 @default.
- W3119259767 cites W2904405987 @default.
- W3119259767 cites W2927085091 @default.
- W3119259767 cites W2931212643 @default.
- W3119259767 cites W2945445411 @default.
- W3119259767 cites W2946417913 @default.
- W3119259767 cites W2949678053 @default.
- W3119259767 cites W2962739339 @default.
- W3119259767 cites W2962917899 @default.
- W3119259767 cites W2963104724 @default.
- W3119259767 cites W2963226019 @default.
- W3119259767 cites W2963341956 @default.
- W3119259767 cites W2963674932 @default.
- W3119259767 cites W2963813662 @default.
- W3119259767 cites W2963826681 @default.
- W3119259767 cites W2964127395 @default.
- W3119259767 cites W2964204621 @default.
- W3119259767 cites W2964212550 @default.
- W3119259767 cites W2964238855 @default.
- W3119259767 cites W2964299589 @default.
- W3119259767 cites W2965373594 @default.
- W3119259767 cites W2970211217 @default.
- W3119259767 cites W2970597249 @default.
- W3119259767 cites W2982111970 @default.
- W3119259767 cites W2989701728 @default.
- W3119259767 cites W2994934025 @default.
- W3119259767 cites W3104100020 @default.
- W3119259767 cites W3104223418 @default.
- W3119259767 cites W3118485687 @default.
- W3119259767 doi "https://doi.org/10.18653/v1/2021.emnlp-main.60" @default.
- W3119259767 hasPublicationYear "2021" @default.
- W3119259767 type Work @default.
- W3119259767 sameAs 3119259767 @default.
- W3119259767 citedByCount "2" @default.
- W3119259767 countsByYear W31192597672021 @default.
- W3119259767 countsByYear W31192597672023 @default.
- W3119259767 crossrefType "proceedings-article" @default.
- W3119259767 hasAuthorship W3119259767A5036790226 @default.
- W3119259767 hasAuthorship W3119259767A5070182106 @default.
- W3119259767 hasAuthorship W3119259767A5073129092 @default.
- W3119259767 hasBestOaLocation W31192597671 @default.
- W3119259767 hasConcept C104317684 @default.
- W3119259767 hasConcept C119857082 @default.
- W3119259767 hasConcept C121332964 @default.
- W3119259767 hasConcept C142362112 @default.
- W3119259767 hasConcept C153349607 @default.
- W3119259767 hasConcept C154945302 @default.
- W3119259767 hasConcept C165801399 @default.
- W3119259767 hasConcept C184337299 @default.
- W3119259767 hasConcept C185592680 @default.
- W3119259767 hasConcept C199360897 @default.
- W3119259767 hasConcept C204321447 @default.
- W3119259767 hasConcept C2777402240 @default.
- W3119259767 hasConcept C41008148 @default.
- W3119259767 hasConcept C55493867 @default.
- W3119259767 hasConcept C60048249 @default.
- W3119259767 hasConcept C62520636 @default.
- W3119259767 hasConcept C66322947 @default.
- W3119259767 hasConcept C66746571 @default.
- W3119259767 hasConcept C97931131 @default.
- W3119259767 hasConceptScore W3119259767C104317684 @default.
- W3119259767 hasConceptScore W3119259767C119857082 @default.
- W3119259767 hasConceptScore W3119259767C121332964 @default.
- W3119259767 hasConceptScore W3119259767C142362112 @default.
- W3119259767 hasConceptScore W3119259767C153349607 @default.
- W3119259767 hasConceptScore W3119259767C154945302 @default.
- W3119259767 hasConceptScore W3119259767C165801399 @default.
- W3119259767 hasConceptScore W3119259767C184337299 @default.
- W3119259767 hasConceptScore W3119259767C185592680 @default.
- W3119259767 hasConceptScore W3119259767C199360897 @default.
- W3119259767 hasConceptScore W3119259767C204321447 @default.
- W3119259767 hasConceptScore W3119259767C2777402240 @default.
- W3119259767 hasConceptScore W3119259767C41008148 @default.
- W3119259767 hasConceptScore W3119259767C55493867 @default.
- W3119259767 hasConceptScore W3119259767C60048249 @default.
- W3119259767 hasConceptScore W3119259767C62520636 @default.
- W3119259767 hasConceptScore W3119259767C66322947 @default.
- W3119259767 hasConceptScore W3119259767C66746571 @default.
- W3119259767 hasConceptScore W3119259767C97931131 @default.
- W3119259767 hasLocation W31192597671 @default.