Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571512> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4385571512 abstract "Pre-trained language models (PLM) have made impressive results in a wide range of NLP tasks and it has been revealed that one of the key factors to their success is the parameters of these models implicitly learn various types of knowledge in the pre-training corpus.However, encoding knowledge implicitly in the model parameters has two fundamental drawbacks. First, the knowledge is neither editable nor scalable once the model is trained, which is especially problematic in that knowledge is consistently evolving. Second, it lacks interpretability and prevents us from understanding what kind of knowledge PLM needs to solve a certain task. In this paper, we introduce {pasted macro ‘MODEL’}, a pre-training model with differentiable plug-in memory (DPM). The key intuition behind is to decouple the knowledge storage from model parameters with an editable and scalable key-value memory and leverage knowledge in an explainable manner by knowledge retrieval in the {pasted macro ‘MEMORY’}. We conduct extensive experiments under various settings to justify this design choice. In domain adaptation setting, {pasted macro ‘MODEL’} could be easily adapted to different domains with pluggable in-domain memory—obtaining 3.95 F1 improvements across four domains, without any in-domain training. {pasted macro ‘MODEL’} could also keep absorbing new knowledge after pre-training is done by knowledge updating operation in the {pasted macro ‘MEMORY’} without re-training. Finally, we show that by incorporating training samples into {pasted macro ‘MEMORY’} with knowledge prompting, {pasted macro ‘MODEL’} could further be improved by the instruction of in-task knowledge." @default.
- W4385571512 created "2023-08-05" @default.
- W4385571512 creator A5024933588 @default.
- W4385571512 creator A5037132097 @default.
- W4385571512 creator A5043098453 @default.
- W4385571512 creator A5045361090 @default.
- W4385571512 creator A5057256093 @default.
- W4385571512 date "2023-01-01" @default.
- W4385571512 modified "2023-09-24" @default.
- W4385571512 title "Decouple knowledge from paramters for plug-and-play language modeling" @default.
- W4385571512 doi "https://doi.org/10.18653/v1/2023.findings-acl.901" @default.
- W4385571512 hasPublicationYear "2023" @default.
- W4385571512 type Work @default.
- W4385571512 citedByCount "0" @default.
- W4385571512 crossrefType "proceedings-article" @default.
- W4385571512 hasAuthorship W4385571512A5024933588 @default.
- W4385571512 hasAuthorship W4385571512A5037132097 @default.
- W4385571512 hasAuthorship W4385571512A5043098453 @default.
- W4385571512 hasAuthorship W4385571512A5045361090 @default.
- W4385571512 hasAuthorship W4385571512A5057256093 @default.
- W4385571512 hasBestOaLocation W43855715121 @default.
- W4385571512 hasConcept C107457646 @default.
- W4385571512 hasConcept C124469403 @default.
- W4385571512 hasConcept C137293760 @default.
- W4385571512 hasConcept C153083717 @default.
- W4385571512 hasConcept C154945302 @default.
- W4385571512 hasConcept C166955791 @default.
- W4385571512 hasConcept C199360897 @default.
- W4385571512 hasConcept C204321447 @default.
- W4385571512 hasConcept C207685749 @default.
- W4385571512 hasConcept C26517878 @default.
- W4385571512 hasConcept C2781067378 @default.
- W4385571512 hasConcept C38652104 @default.
- W4385571512 hasConcept C41008148 @default.
- W4385571512 hasConcept C48044578 @default.
- W4385571512 hasConcept C77088390 @default.
- W4385571512 hasConceptScore W4385571512C107457646 @default.
- W4385571512 hasConceptScore W4385571512C124469403 @default.
- W4385571512 hasConceptScore W4385571512C137293760 @default.
- W4385571512 hasConceptScore W4385571512C153083717 @default.
- W4385571512 hasConceptScore W4385571512C154945302 @default.
- W4385571512 hasConceptScore W4385571512C166955791 @default.
- W4385571512 hasConceptScore W4385571512C199360897 @default.
- W4385571512 hasConceptScore W4385571512C204321447 @default.
- W4385571512 hasConceptScore W4385571512C207685749 @default.
- W4385571512 hasConceptScore W4385571512C26517878 @default.
- W4385571512 hasConceptScore W4385571512C2781067378 @default.
- W4385571512 hasConceptScore W4385571512C38652104 @default.
- W4385571512 hasConceptScore W4385571512C41008148 @default.
- W4385571512 hasConceptScore W4385571512C48044578 @default.
- W4385571512 hasConceptScore W4385571512C77088390 @default.
- W4385571512 hasLocation W43855715121 @default.
- W4385571512 hasOpenAccess W4385571512 @default.
- W4385571512 hasPrimaryLocation W43855715121 @default.
- W4385571512 hasRelatedWork W2170640846 @default.
- W4385571512 hasRelatedWork W2359001871 @default.
- W4385571512 hasRelatedWork W2364921833 @default.
- W4385571512 hasRelatedWork W2619688110 @default.
- W4385571512 hasRelatedWork W2909124124 @default.
- W4385571512 hasRelatedWork W3034085606 @default.
- W4385571512 hasRelatedWork W3045683288 @default.
- W4385571512 hasRelatedWork W3048924380 @default.
- W4385571512 hasRelatedWork W4287761100 @default.
- W4385571512 hasRelatedWork W4293093780 @default.
- W4385571512 isParatext "false" @default.
- W4385571512 isRetracted "false" @default.
- W4385571512 workType "article" @default.