Matches in SemOpenAlex for { <https://semopenalex.org/work/W4303202235> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W4303202235 abstract "Knowledge-Enhanced Pre-trained Language Models (KEPLMs) are pre-trained models with relation triples injecting from knowledge graphs to improve language understanding abilities. To guarantee effective knowledge injection, previous studies integrate models with knowledge encoders for representing knowledge retrieved from knowledge graphs. The operations for knowledge retrieval and encoding bring significant computational burdens, restricting the usage of such models in real-world applications that require high inference speed. In this paper, we propose a novel KEPLM named DKPLM that Decomposes Knowledge injection process of the Pre-trained Language Models in pre-training, fine-tuning and inference stages, which facilitates the applications of KEPLMs in real-world scenarios. Specifically, we first detect knowledge-aware long-tail entities as the target for knowledge injection, enhancing the KEPLMs' semantic understanding abilities and avoiding injecting redundant information. The embeddings of long-tail entities are replaced by pseudo token representations formed by relevant knowledge triples. We further design the relational knowledge decoding task for pre-training to force the models to truly understand the injected knowledge by relation triple reconstruction. Experiments show that our model outperforms other KEPLMs significantly over zero-shot knowledge probing tasks and multiple knowledge-aware language understanding tasks. We further show that DKPLM has a higher inference speed than other competing models due to the decomposing mechanism." @default.
- W4303202235 created "2022-10-07" @default.
- W4303202235 creator A5018499475 @default.
- W4303202235 creator A5030086195 @default.
- W4303202235 creator A5034474990 @default.
- W4303202235 creator A5041581408 @default.
- W4303202235 creator A5059743923 @default.
- W4303202235 creator A5066159151 @default.
- W4303202235 creator A5089187298 @default.
- W4303202235 date "2021-12-02" @default.
- W4303202235 modified "2023-09-28" @default.
- W4303202235 title "DKPLM: Decomposable Knowledge-enhanced Pre-trained Language Model for Natural Language Understanding" @default.
- W4303202235 doi "https://doi.org/10.48550/arxiv.2112.01047" @default.
- W4303202235 hasPublicationYear "2021" @default.
- W4303202235 type Work @default.
- W4303202235 citedByCount "0" @default.
- W4303202235 crossrefType "posted-content" @default.
- W4303202235 hasAuthorship W4303202235A5018499475 @default.
- W4303202235 hasAuthorship W4303202235A5030086195 @default.
- W4303202235 hasAuthorship W4303202235A5034474990 @default.
- W4303202235 hasAuthorship W4303202235A5041581408 @default.
- W4303202235 hasAuthorship W4303202235A5059743923 @default.
- W4303202235 hasAuthorship W4303202235A5066159151 @default.
- W4303202235 hasAuthorship W4303202235A5089187298 @default.
- W4303202235 hasBestOaLocation W43032022351 @default.
- W4303202235 hasConcept C11413529 @default.
- W4303202235 hasConcept C119857082 @default.
- W4303202235 hasConcept C120567893 @default.
- W4303202235 hasConcept C124101348 @default.
- W4303202235 hasConcept C125411270 @default.
- W4303202235 hasConcept C137293760 @default.
- W4303202235 hasConcept C154945302 @default.
- W4303202235 hasConcept C162324750 @default.
- W4303202235 hasConcept C187736073 @default.
- W4303202235 hasConcept C195324797 @default.
- W4303202235 hasConcept C199360897 @default.
- W4303202235 hasConcept C204321447 @default.
- W4303202235 hasConcept C207685749 @default.
- W4303202235 hasConcept C25343380 @default.
- W4303202235 hasConcept C2776214188 @default.
- W4303202235 hasConcept C2780451532 @default.
- W4303202235 hasConcept C2987255567 @default.
- W4303202235 hasConcept C41008148 @default.
- W4303202235 hasConcept C44291984 @default.
- W4303202235 hasConcept C56289545 @default.
- W4303202235 hasConcept C57273362 @default.
- W4303202235 hasConcept C98045186 @default.
- W4303202235 hasConceptScore W4303202235C11413529 @default.
- W4303202235 hasConceptScore W4303202235C119857082 @default.
- W4303202235 hasConceptScore W4303202235C120567893 @default.
- W4303202235 hasConceptScore W4303202235C124101348 @default.
- W4303202235 hasConceptScore W4303202235C125411270 @default.
- W4303202235 hasConceptScore W4303202235C137293760 @default.
- W4303202235 hasConceptScore W4303202235C154945302 @default.
- W4303202235 hasConceptScore W4303202235C162324750 @default.
- W4303202235 hasConceptScore W4303202235C187736073 @default.
- W4303202235 hasConceptScore W4303202235C195324797 @default.
- W4303202235 hasConceptScore W4303202235C199360897 @default.
- W4303202235 hasConceptScore W4303202235C204321447 @default.
- W4303202235 hasConceptScore W4303202235C207685749 @default.
- W4303202235 hasConceptScore W4303202235C25343380 @default.
- W4303202235 hasConceptScore W4303202235C2776214188 @default.
- W4303202235 hasConceptScore W4303202235C2780451532 @default.
- W4303202235 hasConceptScore W4303202235C2987255567 @default.
- W4303202235 hasConceptScore W4303202235C41008148 @default.
- W4303202235 hasConceptScore W4303202235C44291984 @default.
- W4303202235 hasConceptScore W4303202235C56289545 @default.
- W4303202235 hasConceptScore W4303202235C57273362 @default.
- W4303202235 hasConceptScore W4303202235C98045186 @default.
- W4303202235 hasLocation W43032022351 @default.
- W4303202235 hasOpenAccess W4303202235 @default.
- W4303202235 hasPrimaryLocation W43032022351 @default.
- W4303202235 hasRelatedWork W3137014599 @default.
- W4303202235 hasRelatedWork W3201862016 @default.
- W4303202235 hasRelatedWork W3212129137 @default.
- W4303202235 hasRelatedWork W3217374296 @default.
- W4303202235 hasRelatedWork W4226133092 @default.
- W4303202235 hasRelatedWork W4287634190 @default.
- W4303202235 hasRelatedWork W4293261622 @default.
- W4303202235 hasRelatedWork W4303202235 @default.
- W4303202235 hasRelatedWork W4307665269 @default.
- W4303202235 hasRelatedWork W4312874963 @default.
- W4303202235 isParatext "false" @default.
- W4303202235 isRetracted "false" @default.
- W4303202235 workType "article" @default.