Matches in SemOpenAlex for { <https://semopenalex.org/work/W3213881810> ?p ?o ?g. }
- W3213881810 abstract "The fine-tuning of pre-trained language models has a great success in many NLP fields. Yet, it is strikingly vulnerable to adversarial examples, e.g., word substitution attacks using only synonyms can easily fool a BERT-based sentiment analysis model. In this paper, we demonstrate that adversarial training, the prevalent defense technique, does not directly fit a conventional fine-tuning scenario, because it suffers severely from catastrophic forgetting: failing to retain the generic and robust linguistic features that have already been captured by the pre-trained model. In this light, we propose Robust Informative Fine-Tuning (RIFT), a novel adversarial fine-tuning method from an information-theoretical perspective. In particular, RIFT encourages an objective model to retain the features learned from the pre-trained model throughout the entire fine-tuning process, whereas a conventional one only uses the pre-trained weights for initialization. Experimental results show that RIFT consistently outperforms the state-of-the-arts on two popular NLP tasks: sentiment analysis and natural language inference, under different attacks across various pre-trained language models." @default.
- W3213881810 created "2021-11-22" @default.
- W3213881810 creator A5039765869 @default.
- W3213881810 creator A5042324027 @default.
- W3213881810 creator A5047929313 @default.
- W3213881810 creator A5050386762 @default.
- W3213881810 creator A5079198353 @default.
- W3213881810 date "2021-12-22" @default.
- W3213881810 modified "2023-09-23" @default.
- W3213881810 title "How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?" @default.
- W3213881810 cites W1682403713 @default.
- W3213881810 cites W1840435438 @default.
- W3213881810 cites W2014703436 @default.
- W3213881810 cites W2113459411 @default.
- W3213881810 cites W2152790380 @default.
- W3213881810 cites W2153579005 @default.
- W3213881810 cites W2187089797 @default.
- W3213881810 cites W2250539671 @default.
- W3213881810 cites W2555897561 @default.
- W3213881810 cites W2766108848 @default.
- W3213881810 cites W2787708942 @default.
- W3213881810 cites W2799007037 @default.
- W3213881810 cites W2804613085 @default.
- W3213881810 cites W2908510526 @default.
- W3213881810 cites W2913266441 @default.
- W3213881810 cites W2947415936 @default.
- W3213881810 cites W2949128310 @default.
- W3213881810 cites W2949759968 @default.
- W3213881810 cites W2950687935 @default.
- W3213881810 cites W2962739339 @default.
- W3213881810 cites W2962818281 @default.
- W3213881810 cites W2963126845 @default.
- W3213881810 cites W2963207607 @default.
- W3213881810 cites W2963341956 @default.
- W3213881810 cites W2963389226 @default.
- W3213881810 cites W2963403868 @default.
- W3213881810 cites W2963488798 @default.
- W3213881810 cites W2963501948 @default.
- W3213881810 cites W2963857521 @default.
- W3213881810 cites W2963969878 @default.
- W3213881810 cites W2964153729 @default.
- W3213881810 cites W2964420626 @default.
- W3213881810 cites W2970727289 @default.
- W3213881810 cites W2995607862 @default.
- W3213881810 cites W3005680577 @default.
- W3213881810 cites W3103340107 @default.
- W3213881810 cites W3106428938 @default.
- W3213881810 cites W3118062200 @default.
- W3213881810 doi "https://doi.org/10.48550/arxiv.2112.11668" @default.
- W3213881810 hasPublicationYear "2021" @default.
- W3213881810 type Work @default.
- W3213881810 sameAs 3213881810 @default.
- W3213881810 citedByCount "0" @default.
- W3213881810 crossrefType "posted-content" @default.
- W3213881810 hasAuthorship W3213881810A5039765869 @default.
- W3213881810 hasAuthorship W3213881810A5042324027 @default.
- W3213881810 hasAuthorship W3213881810A5047929313 @default.
- W3213881810 hasAuthorship W3213881810A5050386762 @default.
- W3213881810 hasAuthorship W3213881810A5079198353 @default.
- W3213881810 hasBestOaLocation W32138818101 @default.
- W3213881810 hasConcept C104317684 @default.
- W3213881810 hasConcept C114466953 @default.
- W3213881810 hasConcept C119857082 @default.
- W3213881810 hasConcept C121332964 @default.
- W3213881810 hasConcept C137293760 @default.
- W3213881810 hasConcept C138885662 @default.
- W3213881810 hasConcept C154945302 @default.
- W3213881810 hasConcept C157524613 @default.
- W3213881810 hasConcept C185592680 @default.
- W3213881810 hasConcept C199360897 @default.
- W3213881810 hasConcept C204321447 @default.
- W3213881810 hasConcept C2776214188 @default.
- W3213881810 hasConcept C2984842247 @default.
- W3213881810 hasConcept C37736160 @default.
- W3213881810 hasConcept C41008148 @default.
- W3213881810 hasConcept C41895202 @default.
- W3213881810 hasConcept C50644808 @default.
- W3213881810 hasConcept C55493867 @default.
- W3213881810 hasConcept C62520636 @default.
- W3213881810 hasConcept C63479239 @default.
- W3213881810 hasConcept C66402592 @default.
- W3213881810 hasConcept C7149132 @default.
- W3213881810 hasConceptScore W3213881810C104317684 @default.
- W3213881810 hasConceptScore W3213881810C114466953 @default.
- W3213881810 hasConceptScore W3213881810C119857082 @default.
- W3213881810 hasConceptScore W3213881810C121332964 @default.
- W3213881810 hasConceptScore W3213881810C137293760 @default.
- W3213881810 hasConceptScore W3213881810C138885662 @default.
- W3213881810 hasConceptScore W3213881810C154945302 @default.
- W3213881810 hasConceptScore W3213881810C157524613 @default.
- W3213881810 hasConceptScore W3213881810C185592680 @default.
- W3213881810 hasConceptScore W3213881810C199360897 @default.
- W3213881810 hasConceptScore W3213881810C204321447 @default.
- W3213881810 hasConceptScore W3213881810C2776214188 @default.
- W3213881810 hasConceptScore W3213881810C2984842247 @default.
- W3213881810 hasConceptScore W3213881810C37736160 @default.
- W3213881810 hasConceptScore W3213881810C41008148 @default.
- W3213881810 hasConceptScore W3213881810C41895202 @default.
- W3213881810 hasConceptScore W3213881810C50644808 @default.
- W3213881810 hasConceptScore W3213881810C55493867 @default.