FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3213074177> ?p ?o ?g. }

• W3213074177 abstract "Masked language models (MLMs) such as BERT have revolutionized the field of Natural Language Understanding in the past few years. However, existing pre-trained MLMs often output an anisotropic distribution of token representations that occupies a narrow subset of the entire representation space. Such token representations are not ideal, especially for tasks that demand discriminative semantic meanings of distinct tokens. In this work, we propose TaCL (Token-aware Contrastive Learning), a novel continual pre-training approach that encourages BERT to learn an isotropic and discriminative distribution of token representations. TaCL is fully unsupervised and requires no additional data. We extensively test our approach on a wide range of English and Chinese benchmarks. The results show that TaCL brings consistent and notable improvements over the original BERT model. Furthermore, we conduct detailed analysis to reveal the merits and inner-workings of our approach." @default.
• W3213074177 created "2021-11-22" @default.
• W3213074177 creator A5026154387 @default.
• W3213074177 creator A5044702929 @default.
• W3213074177 creator A5073342482 @default.
• W3213074177 creator A5073413742 @default.
• W3213074177 creator A5080253330 @default.
• W3213074177 creator A5086032589 @default.
• W3213074177 date "2022-01-01" @default.
• W3213074177 modified "2023-09-26" @default.
• W3213074177 title "TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning" @default.
• W3213074177 cites W2157364932 @default.
• W3213074177 cites W219040644 @default.
• W3213074177 cites W2252066972 @default.
• W3213074177 cites W25062297 @default.
• W3213074177 cites W2567657016 @default.
• W3213074177 cites W2769112066 @default.
• W3213074177 cites W2925618549 @default.
• W3213074177 cites W2962904552 @default.
• W3213074177 cites W2963310665 @default.
• W3213074177 cites W2963323070 @default.
• W3213074177 cites W2963341956 @default.
• W3213074177 cites W2963748441 @default.
• W3213074177 cites W2964121744 @default.
• W3213074177 cites W2965373594 @default.
• W3213074177 cites W2970597249 @default.
• W3213074177 cites W2970681607 @default.
• W3213074177 cites W2982399380 @default.
• W3213074177 cites W2988217457 @default.
• W3213074177 cites W3033406728 @default.
• W3213074177 cites W3034238904 @default.
• W3213074177 cites W3034379414 @default.
• W3213074177 cites W3034978746 @default.
• W3213074177 cites W3082274269 @default.
• W3213074177 cites W3115295967 @default.
• W3213074177 cites W3122838366 @default.
• W3213074177 cites W3131870090 @default.
• W3213074177 cites W3132164968 @default.
• W3213074177 cites W3135367836 @default.
• W3213074177 cites W3156636935 @default.
• W3213074177 cites W3173783447 @default.
• W3213074177 cites W3175362188 @default.
• W3213074177 cites W3176047188 @default.
• W3213074177 cites W3198147814 @default.
• W3213074177 cites W3203711169 @default.
• W3213074177 cites W3204670646 @default.
• W3213074177 cites W3206719829 @default.
• W3213074177 cites W3211872137 @default.
• W3213074177 cites W3212618200 @default.
• W3213074177 cites W3213730158 @default.
• W3213074177 cites W3170611326 @default.
• W3213074177 doi "https://doi.org/10.18653/v1/2022.findings-naacl.191" @default.
• W3213074177 hasPublicationYear "2022" @default.
• W3213074177 type Work @default.
• W3213074177 sameAs 3213074177 @default.
• W3213074177 citedByCount "2" @default.
• W3213074177 countsByYear W32130741772021 @default.
• W3213074177 countsByYear W32130741772023 @default.
• W3213074177 crossrefType "proceedings-article" @default.
• W3213074177 hasAuthorship W3213074177A5026154387 @default.
• W3213074177 hasAuthorship W3213074177A5044702929 @default.
• W3213074177 hasAuthorship W3213074177A5073342482 @default.
• W3213074177 hasAuthorship W3213074177A5073413742 @default.
• W3213074177 hasAuthorship W3213074177A5080253330 @default.
• W3213074177 hasAuthorship W3213074177A5086032589 @default.
• W3213074177 hasBestOaLocation W32130741771 @default.
• W3213074177 hasConcept C119857082 @default.
• W3213074177 hasConcept C154945302 @default.
• W3213074177 hasConcept C17744445 @default.
• W3213074177 hasConcept C199539241 @default.
• W3213074177 hasConcept C204321447 @default.
• W3213074177 hasConcept C2776359362 @default.
• W3213074177 hasConcept C38652104 @default.
• W3213074177 hasConcept C41008148 @default.
• W3213074177 hasConcept C48145219 @default.
• W3213074177 hasConcept C94625758 @default.
• W3213074177 hasConcept C97931131 @default.
• W3213074177 hasConceptScore W3213074177C119857082 @default.
• W3213074177 hasConceptScore W3213074177C154945302 @default.
• W3213074177 hasConceptScore W3213074177C17744445 @default.
• W3213074177 hasConceptScore W3213074177C199539241 @default.
• W3213074177 hasConceptScore W3213074177C204321447 @default.
• W3213074177 hasConceptScore W3213074177C2776359362 @default.
• W3213074177 hasConceptScore W3213074177C38652104 @default.
• W3213074177 hasConceptScore W3213074177C41008148 @default.
• W3213074177 hasConceptScore W3213074177C48145219 @default.
• W3213074177 hasConceptScore W3213074177C94625758 @default.
• W3213074177 hasConceptScore W3213074177C97931131 @default.
• W3213074177 hasLocation W32130741771 @default.
• W3213074177 hasLocation W32130741772 @default.
• W3213074177 hasOpenAccess W3213074177 @default.
• W3213074177 hasPrimaryLocation W32130741771 @default.
• W3213074177 hasRelatedWork W2026121273 @default.
• W3213074177 hasRelatedWork W2102106825 @default.
• W3213074177 hasRelatedWork W2375389409 @default.
• W3213074177 hasRelatedWork W2752271443 @default.
• W3213074177 hasRelatedWork W2801772698 @default.
• W3213074177 hasRelatedWork W2961085424 @default.
• W3213074177 hasRelatedWork W2983744209 @default.
• W3213074177 hasRelatedWork W4306674287 @default.

• next