Matches in SemOpenAlex for { <https://semopenalex.org/work/W3107969673> ?p ?o ?g. }
- W3107969673 abstract "Large Transformer models have achieved impressive performance in many natural language tasks. In particular, Transformer based language models have been shown to have great capabilities in encoding factual knowledge in their vast amount of parameters. While the tasks of improving the memorization and generalization of Transformers have been widely studied, it is not well known how to make transformers forget specific old facts and memorize new ones. In this paper, we propose a new task of explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts. This task is useful in many scenarios, such as updating stale knowledge, protecting privacy, and eliminating unintended biases stored in the models. We benchmarked several approaches that provide natural baseline performances on this task. This leads to the discovery of key components of a Transformer model that are especially effective for knowledge modifications. The work also provides insights into the role that different training phases (such as pretraining and fine-tuning) play towards memorization and knowledge modification." @default.
- W3107969673 created "2020-12-07" @default.
- W3107969673 creator A5017466259 @default.
- W3107969673 creator A5018301052 @default.
- W3107969673 creator A5018931712 @default.
- W3107969673 creator A5057451920 @default.
- W3107969673 creator A5065795593 @default.
- W3107969673 creator A5079621556 @default.
- W3107969673 creator A5082943001 @default.
- W3107969673 date "2021-05-04" @default.
- W3107969673 modified "2023-09-23" @default.
- W3107969673 title "Modifying Memories in Transformer Models" @default.
- W3107969673 cites W1496189301 @default.
- W3107969673 cites W1503259811 @default.
- W3107969673 cites W1566289585 @default.
- W3107969673 cites W1793121960 @default.
- W3107969673 cites W2030408698 @default.
- W3107969673 cites W2059933135 @default.
- W3107969673 cites W2163274265 @default.
- W3107969673 cites W2483215953 @default.
- W3107969673 cites W2560647685 @default.
- W3107969673 cites W2785611959 @default.
- W3107969673 cites W2938830017 @default.
- W3107969673 cites W2946930197 @default.
- W3107969673 cites W2953356739 @default.
- W3107969673 cites W2962724315 @default.
- W3107969673 cites W2962783425 @default.
- W3107969673 cites W2962881743 @default.
- W3107969673 cites W2963341956 @default.
- W3107969673 cites W2963403868 @default.
- W3107969673 cites W2963612262 @default.
- W3107969673 cites W2964303773 @default.
- W3107969673 cites W2970986510 @default.
- W3107969673 cites W2971227267 @default.
- W3107969673 cites W2971869958 @default.
- W3107969673 cites W2975059944 @default.
- W3107969673 cites W2995154514 @default.
- W3107969673 cites W2995636882 @default.
- W3107969673 cites W3002800333 @default.
- W3107969673 cites W3003265726 @default.
- W3107969673 cites W3007672467 @default.
- W3107969673 cites W3016309009 @default.
- W3107969673 cites W3027879771 @default.
- W3107969673 cites W3030163527 @default.
- W3107969673 cites W3035261884 @default.
- W3107969673 cites W3037831233 @default.
- W3107969673 cites W3039578880 @default.
- W3107969673 cites W3044438666 @default.
- W3107969673 cites W3082274269 @default.
- W3107969673 cites W3089477688 @default.
- W3107969673 cites W3100152912 @default.
- W3107969673 cites W3100352836 @default.
- W3107969673 cites W3102568136 @default.
- W3107969673 cites W3102659883 @default.
- W3107969673 cites W3102839769 @default.
- W3107969673 cites W3106325613 @default.
- W3107969673 cites W3169283738 @default.
- W3107969673 hasPublicationYear "2021" @default.
- W3107969673 type Work @default.
- W3107969673 sameAs 3107969673 @default.
- W3107969673 citedByCount "8" @default.
- W3107969673 countsByYear W31079696732021 @default.
- W3107969673 crossrefType "journal-article" @default.
- W3107969673 hasAuthorship W3107969673A5017466259 @default.
- W3107969673 hasAuthorship W3107969673A5018301052 @default.
- W3107969673 hasAuthorship W3107969673A5018931712 @default.
- W3107969673 hasAuthorship W3107969673A5057451920 @default.
- W3107969673 hasAuthorship W3107969673A5065795593 @default.
- W3107969673 hasAuthorship W3107969673A5079621556 @default.
- W3107969673 hasAuthorship W3107969673A5082943001 @default.
- W3107969673 hasConcept C119599485 @default.
- W3107969673 hasConcept C119857082 @default.
- W3107969673 hasConcept C127413603 @default.
- W3107969673 hasConcept C154945302 @default.
- W3107969673 hasConcept C15744967 @default.
- W3107969673 hasConcept C165801399 @default.
- W3107969673 hasConcept C180747234 @default.
- W3107969673 hasConcept C30038468 @default.
- W3107969673 hasConcept C41008148 @default.
- W3107969673 hasConcept C66322947 @default.
- W3107969673 hasConceptScore W3107969673C119599485 @default.
- W3107969673 hasConceptScore W3107969673C119857082 @default.
- W3107969673 hasConceptScore W3107969673C127413603 @default.
- W3107969673 hasConceptScore W3107969673C154945302 @default.
- W3107969673 hasConceptScore W3107969673C15744967 @default.
- W3107969673 hasConceptScore W3107969673C165801399 @default.
- W3107969673 hasConceptScore W3107969673C180747234 @default.
- W3107969673 hasConceptScore W3107969673C30038468 @default.
- W3107969673 hasConceptScore W3107969673C41008148 @default.
- W3107969673 hasConceptScore W3107969673C66322947 @default.
- W3107969673 hasOpenAccess W3107969673 @default.
- W3107969673 hasRelatedWork W119746310 @default.
- W3107969673 hasRelatedWork W1485938889 @default.
- W3107969673 hasRelatedWork W1971690354 @default.
- W3107969673 hasRelatedWork W2070775927 @default.
- W3107969673 hasRelatedWork W2090568355 @default.
- W3107969673 hasRelatedWork W2169223738 @default.
- W3107969673 hasRelatedWork W2272952952 @default.
- W3107969673 hasRelatedWork W2397017496 @default.
- W3107969673 hasRelatedWork W2560186857 @default.