Matches in SemOpenAlex for { <https://semopenalex.org/work/W4226366933> ?p ?o ?g. }
Showing items 1 to 64 of
64
with 100 items per page.
- W4226366933 abstract "Self-supervised learning approach like contrastive learning is attached great attention in natural language processing. It uses pairs of training data augmentations to build a classification task for an encoder with well representation ability. However, the construction of learning pairs over contrastive learning is much harder in NLP tasks. Previous works generate word-level changes to form pairs, but small transforms may cause notable changes on the meaning of sentences as the discrete and sparse nature of natural language. In this paper, adversarial training is performed to generate challenging and harder learning adversarial examples over the embedding space of NLP as learning pairs. Using contrastive learning improves the generalization ability of adversarial training because contrastive loss can uniform the sample distribution. And at the same time, adversarial training also enhances the robustness of contrastive learning. Two novel frameworks, supervised contrastive adversarial learning (SCAL) and unsupervised SCAL (USCAL), are proposed, which yields learning pairs by utilizing the adversarial training for contrastive learning. The label-based loss of supervised tasks is exploited to generate adversarial examples while unsupervised tasks bring contrastive loss. To validate the effectiveness of the proposed framework, we employ it to Transformer-based models for natural language understanding, sentence semantic textual similarity and adversarial learning tasks. Experimental results on GLUE benchmark tasks show that our fine-tuned supervised method outperforms BERT$_{base}$ over 1.75%. We also evaluate our unsupervised method on semantic textual similarity (STS) tasks, and our method gets 77.29% with BERT$_{base}$. The robustness of our approach conducts state-of-the-art results under multiple adversarial datasets on NLI tasks." @default.
- W4226366933 created "2022-05-05" @default.
- W4226366933 creator A5019376939 @default.
- W4226366933 creator A5020285133 @default.
- W4226366933 creator A5022526821 @default.
- W4226366933 creator A5038553293 @default.
- W4226366933 creator A5039085563 @default.
- W4226366933 creator A5053352714 @default.
- W4226366933 creator A5085059762 @default.
- W4226366933 date "2021-11-25" @default.
- W4226366933 modified "2023-09-26" @default.
- W4226366933 title "Simple Contrastive Representation Adversarial Learning for NLP Tasks" @default.
- W4226366933 hasPublicationYear "2021" @default.
- W4226366933 type Work @default.
- W4226366933 citedByCount "0" @default.
- W4226366933 crossrefType "posted-content" @default.
- W4226366933 hasAuthorship W4226366933A5019376939 @default.
- W4226366933 hasAuthorship W4226366933A5020285133 @default.
- W4226366933 hasAuthorship W4226366933A5022526821 @default.
- W4226366933 hasAuthorship W4226366933A5038553293 @default.
- W4226366933 hasAuthorship W4226366933A5039085563 @default.
- W4226366933 hasAuthorship W4226366933A5053352714 @default.
- W4226366933 hasAuthorship W4226366933A5085059762 @default.
- W4226366933 hasBestOaLocation W42263669331 @default.
- W4226366933 hasConcept C104317684 @default.
- W4226366933 hasConcept C119857082 @default.
- W4226366933 hasConcept C154945302 @default.
- W4226366933 hasConcept C185592680 @default.
- W4226366933 hasConcept C204321447 @default.
- W4226366933 hasConcept C2777530160 @default.
- W4226366933 hasConcept C37736160 @default.
- W4226366933 hasConcept C41008148 @default.
- W4226366933 hasConcept C55493867 @default.
- W4226366933 hasConcept C59404180 @default.
- W4226366933 hasConcept C63479239 @default.
- W4226366933 hasConcept C8038995 @default.
- W4226366933 hasConceptScore W4226366933C104317684 @default.
- W4226366933 hasConceptScore W4226366933C119857082 @default.
- W4226366933 hasConceptScore W4226366933C154945302 @default.
- W4226366933 hasConceptScore W4226366933C185592680 @default.
- W4226366933 hasConceptScore W4226366933C204321447 @default.
- W4226366933 hasConceptScore W4226366933C2777530160 @default.
- W4226366933 hasConceptScore W4226366933C37736160 @default.
- W4226366933 hasConceptScore W4226366933C41008148 @default.
- W4226366933 hasConceptScore W4226366933C55493867 @default.
- W4226366933 hasConceptScore W4226366933C59404180 @default.
- W4226366933 hasConceptScore W4226366933C63479239 @default.
- W4226366933 hasConceptScore W4226366933C8038995 @default.
- W4226366933 hasLocation W42263669331 @default.
- W4226366933 hasOpenAccess W4226366933 @default.
- W4226366933 hasPrimaryLocation W42263669331 @default.
- W4226366933 hasRelatedWork W10553295 @default.
- W4226366933 hasRelatedWork W11007999 @default.
- W4226366933 hasRelatedWork W11360949 @default.
- W4226366933 hasRelatedWork W3036585 @default.
- W4226366933 hasRelatedWork W5236451 @default.
- W4226366933 hasRelatedWork W628720 @default.
- W4226366933 hasRelatedWork W8671328 @default.
- W4226366933 hasRelatedWork W8788258 @default.
- W4226366933 hasRelatedWork W90294 @default.
- W4226366933 hasRelatedWork W9531778 @default.
- W4226366933 isParatext "false" @default.
- W4226366933 isRetracted "false" @default.
- W4226366933 workType "article" @default.