Matches in SemOpenAlex for { <https://semopenalex.org/work/W3165567187> ?p ?o ?g. }
Showing items 1 to 100 of
100
with 100 items per page.
- W3165567187 abstract "Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on downstream tasks. However, further increases and modifications based on such large-scale models are usually impractical due to memory limitations. In this work, we introduce a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations. We explore different training tasks and observe that current cross-lingual training tasks leave a lot to be desired for this shallow architecture. To ameliorate this, we propose a novel cross-lingual language model, which combines the existing single-word masked language model with the newly proposed cross-lingual token-level reconstruction task. We further augment the training task by the introduction of two computationally-lite sentence-level contrastive learning tasks to enhance the alignment of cross-lingual sentence representation space, which compensates for the learning bottleneck of the lightweight transformer for generative tasks. Our comparisons with competing models on cross-lingual sentence retrieval and multilingual document classification confirm the effectiveness of the newly proposed training tasks for a shallow model." @default.
- W3165567187 created "2021-06-07" @default.
- W3165567187 creator A5008316265 @default.
- W3165567187 creator A5017982536 @default.
- W3165567187 creator A5028836340 @default.
- W3165567187 creator A5073756389 @default.
- W3165567187 creator A5076240722 @default.
- W3165567187 date "2021-05-28" @default.
- W3165567187 modified "2023-09-27" @default.
- W3165567187 title "Lightweight Cross-Lingual Sentence Representation Learning" @default.
- W3165567187 cites W2153579005 @default.
- W3165567187 cites W2251765408 @default.
- W3165567187 cites W2283196293 @default.
- W3165567187 cites W2609278920 @default.
- W3165567187 cites W2885323099 @default.
- W3165567187 cites W2886198413 @default.
- W3165567187 cites W2948972665 @default.
- W3165567187 cites W2962735107 @default.
- W3165567187 cites W2963341956 @default.
- W3165567187 cites W2963403868 @default.
- W3165567187 cites W2963672008 @default.
- W3165567187 cites W2963721344 @default.
- W3165567187 cites W2963915291 @default.
- W3165567187 cites W2963979492 @default.
- W3165567187 cites W2970193165 @default.
- W3165567187 cites W2970618241 @default.
- W3165567187 cites W2971031524 @default.
- W3165567187 cites W2978017171 @default.
- W3165567187 cites W2996428491 @default.
- W3165567187 cites W3033406728 @default.
- W3165567187 cites W3034457371 @default.
- W3165567187 cites W3034978746 @default.
- W3165567187 cites W3035016936 @default.
- W3165567187 cites W3035390927 @default.
- W3165567187 cites W3035547806 @default.
- W3165567187 cites W3077436702 @default.
- W3165567187 cites W3098824823 @default.
- W3165567187 cites W3100806282 @default.
- W3165567187 cites W3105966348 @default.
- W3165567187 cites W3118942129 @default.
- W3165567187 doi "https://doi.org/10.48550/arxiv.2105.13856" @default.
- W3165567187 hasPublicationYear "2021" @default.
- W3165567187 type Work @default.
- W3165567187 sameAs 3165567187 @default.
- W3165567187 citedByCount "0" @default.
- W3165567187 crossrefType "posted-content" @default.
- W3165567187 hasAuthorship W3165567187A5008316265 @default.
- W3165567187 hasAuthorship W3165567187A5017982536 @default.
- W3165567187 hasAuthorship W3165567187A5028836340 @default.
- W3165567187 hasAuthorship W3165567187A5073756389 @default.
- W3165567187 hasAuthorship W3165567187A5076240722 @default.
- W3165567187 hasBestOaLocation W31655671871 @default.
- W3165567187 hasConcept C121332964 @default.
- W3165567187 hasConcept C137293760 @default.
- W3165567187 hasConcept C149635348 @default.
- W3165567187 hasConcept C154945302 @default.
- W3165567187 hasConcept C162324750 @default.
- W3165567187 hasConcept C165801399 @default.
- W3165567187 hasConcept C187736073 @default.
- W3165567187 hasConcept C204321447 @default.
- W3165567187 hasConcept C2777530160 @default.
- W3165567187 hasConcept C2780451532 @default.
- W3165567187 hasConcept C2780513914 @default.
- W3165567187 hasConcept C41008148 @default.
- W3165567187 hasConcept C59404180 @default.
- W3165567187 hasConcept C62520636 @default.
- W3165567187 hasConcept C66322947 @default.
- W3165567187 hasConceptScore W3165567187C121332964 @default.
- W3165567187 hasConceptScore W3165567187C137293760 @default.
- W3165567187 hasConceptScore W3165567187C149635348 @default.
- W3165567187 hasConceptScore W3165567187C154945302 @default.
- W3165567187 hasConceptScore W3165567187C162324750 @default.
- W3165567187 hasConceptScore W3165567187C165801399 @default.
- W3165567187 hasConceptScore W3165567187C187736073 @default.
- W3165567187 hasConceptScore W3165567187C204321447 @default.
- W3165567187 hasConceptScore W3165567187C2777530160 @default.
- W3165567187 hasConceptScore W3165567187C2780451532 @default.
- W3165567187 hasConceptScore W3165567187C2780513914 @default.
- W3165567187 hasConceptScore W3165567187C41008148 @default.
- W3165567187 hasConceptScore W3165567187C59404180 @default.
- W3165567187 hasConceptScore W3165567187C62520636 @default.
- W3165567187 hasConceptScore W3165567187C66322947 @default.
- W3165567187 hasLocation W31655671871 @default.
- W3165567187 hasLocation W31655671872 @default.
- W3165567187 hasOpenAccess W3165567187 @default.
- W3165567187 hasPrimaryLocation W31655671871 @default.
- W3165567187 hasRelatedWork W1573537589 @default.
- W3165567187 hasRelatedWork W159132833 @default.
- W3165567187 hasRelatedWork W20999564 @default.
- W3165567187 hasRelatedWork W2359001871 @default.
- W3165567187 hasRelatedWork W3033862527 @default.
- W3165567187 hasRelatedWork W3082447286 @default.
- W3165567187 hasRelatedWork W3097571385 @default.
- W3165567187 hasRelatedWork W3196747313 @default.
- W3165567187 hasRelatedWork W4205820553 @default.
- W3165567187 hasRelatedWork W4318978824 @default.
- W3165567187 isParatext "false" @default.
- W3165567187 isRetracted "false" @default.
- W3165567187 magId "3165567187" @default.
- W3165567187 workType "article" @default.