Matches in SemOpenAlex for { <https://semopenalex.org/work/W3211734609> ?p ?o ?g. }
- W3211734609 abstract "Pre-trained language models have been found to capture a surprisingly richamount of lexical knowledge, ranging from commonsense properties of everydayconcepts to detailed factual knowledge about named entities. Among others, thismakes it possible to distill high-quality word vectors from pre-trainedlanguage models. However, it is currently unclear to what extent it is possibleto distill relation embeddings, i.e. vectors that characterize the relationshipbetween two words. Such relation embeddings are appealing because they can, inprinciple, encode relational knowledge in a more fine-grained way than ispossible with knowledge graphs. To obtain relation embeddings from apre-trained language model, we encode word pairs using a (manually orautomatically generated) prompt, and we fine-tune the language model such thatrelationally similar word pairs yield similar output vectors. We find that theresulting relation embeddings are highly competitive on analogy (unsupervised)and relation classification (supervised) benchmarks, even without anytask-specific fine-tuning. Source code to reproduce our experimental resultsand the model checkpoints are available in the following repository:https://github.com/asahi417/relbert" @default.
- W3211734609 created "2021-11-22" @default.
- W3211734609 creator A5009887598 @default.
- W3211734609 creator A5060273715 @default.
- W3211734609 creator A5086289154 @default.
- W3211734609 date "2021-09-21" @default.
- W3211734609 modified "2023-09-27" @default.
- W3211734609 title "Distilling Relation Embeddings from Pre-trained Language Models" @default.
- W3211734609 cites W115166160 @default.
- W3211734609 cites W1535654961 @default.
- W3211734609 cites W1593045043 @default.
- W3211734609 cites W1654905138 @default.
- W3211734609 cites W1660519191 @default.
- W3211734609 cites W1852412531 @default.
- W3211734609 cites W1880262756 @default.
- W3211734609 cites W1965605789 @default.
- W3211734609 cites W2005603639 @default.
- W3211734609 cites W2068737686 @default.
- W3211734609 cites W2098801107 @default.
- W3211734609 cites W2101234009 @default.
- W3211734609 cites W2123143128 @default.
- W3211734609 cites W2135964261 @default.
- W3211734609 cites W2141599568 @default.
- W3211734609 cites W2153579005 @default.
- W3211734609 cites W2155157567 @default.
- W3211734609 cites W2250533418 @default.
- W3211734609 cites W2250539671 @default.
- W3211734609 cites W2460442863 @default.
- W3211734609 cites W2493916176 @default.
- W3211734609 cites W2608204223 @default.
- W3211734609 cites W2610751324 @default.
- W3211734609 cites W2739638526 @default.
- W3211734609 cites W2751143192 @default.
- W3211734609 cites W2799027102 @default.
- W3211734609 cites W2803986565 @default.
- W3211734609 cites W2903760398 @default.
- W3211734609 cites W2910243263 @default.
- W3211734609 cites W2946359678 @default.
- W3211734609 cites W2950339735 @default.
- W3211734609 cites W2950401678 @default.
- W3211734609 cites W2952391587 @default.
- W3211734609 cites W2962724755 @default.
- W3211734609 cites W2962769333 @default.
- W3211734609 cites W2963176474 @default.
- W3211734609 cites W2963201227 @default.
- W3211734609 cites W2963219906 @default.
- W3211734609 cites W2963239938 @default.
- W3211734609 cites W2963341956 @default.
- W3211734609 cites W2963768750 @default.
- W3211734609 cites W2963958374 @default.
- W3211734609 cites W2964022985 @default.
- W3211734609 cites W2964121744 @default.
- W3211734609 cites W2964207259 @default.
- W3211734609 cites W2964242916 @default.
- W3211734609 cites W2964350953 @default.
- W3211734609 cites W2965058444 @default.
- W3211734609 cites W2965373594 @default.
- W3211734609 cites W2970161131 @default.
- W3211734609 cites W2970476646 @default.
- W3211734609 cites W2971016963 @default.
- W3211734609 cites W2978491132 @default.
- W3211734609 cites W2987249037 @default.
- W3211734609 cites W2988217457 @default.
- W3211734609 cites W2998557616 @default.
- W3211734609 cites W3035102548 @default.
- W3211734609 cites W3098267758 @default.
- W3211734609 cites W3098613713 @default.
- W3211734609 cites W3098824823 @default.
- W3211734609 cites W3098903812 @default.
- W3211734609 cites W3099178230 @default.
- W3211734609 cites W3099700870 @default.
- W3211734609 cites W3102226577 @default.
- W3211734609 cites W3102659883 @default.
- W3211734609 cites W3103136066 @default.
- W3211734609 cites W3111372685 @default.
- W3211734609 cites W3117576675 @default.
- W3211734609 cites W3125506016 @default.
- W3211734609 cites W3137573489 @default.
- W3211734609 cites W3139080614 @default.
- W3211734609 cites W3152884768 @default.
- W3211734609 cites W3153427360 @default.
- W3211734609 cites W3154903254 @default.
- W3211734609 cites W3173777717 @default.
- W3211734609 cites W3174082608 @default.
- W3211734609 hasPublicationYear "2021" @default.
- W3211734609 type Work @default.
- W3211734609 sameAs 3211734609 @default.
- W3211734609 citedByCount "0" @default.
- W3211734609 crossrefType "posted-content" @default.
- W3211734609 hasAuthorship W3211734609A5009887598 @default.
- W3211734609 hasAuthorship W3211734609A5060273715 @default.
- W3211734609 hasAuthorship W3211734609A5086289154 @default.
- W3211734609 hasConcept C104317684 @default.
- W3211734609 hasConcept C124101348 @default.
- W3211734609 hasConcept C137293760 @default.
- W3211734609 hasConcept C138885662 @default.
- W3211734609 hasConcept C154945302 @default.
- W3211734609 hasConcept C162324750 @default.
- W3211734609 hasConcept C177264268 @default.
- W3211734609 hasConcept C185592680 @default.