Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571469> ?p ?o ?g. }
Showing items 1 to 55 of
55
with 100 items per page.
- W4385571469 abstract "Global co-occurrence information is the primary source of structural information on multilingual corpora, and we find that analogical/parallel compound words across languages have similar co-occurrence counts/frequencies (normalized) giving weak but stable self-supervision for cross-lingual transfer. Following the observation, we aim at associating contextualized representations with relevant (contextualized) representations across languages with the help of co-occurrence counts. The result is MLM-GC (MLM with Global Co-occurrence) pre-training that the model learns local bidirectional information from MLM and global co-occurrence information from a log-bilinear regression. Experiments show that MLM-GC pre-training substantially outperforms MLM pre-training for 4 downstream cross-lingual tasks and 1 additional monolingual task, showing the advantages of forming isomorphic spaces across languages." @default.
- W4385571469 created "2023-08-05" @default.
- W4385571469 creator A5046520540 @default.
- W4385571469 creator A5081756234 @default.
- W4385571469 date "2023-01-01" @default.
- W4385571469 modified "2023-09-24" @default.
- W4385571469 title "Multilingual Pre-training with Self-supervision from Global Co-occurrence Information" @default.
- W4385571469 doi "https://doi.org/10.18653/v1/2023.findings-acl.475" @default.
- W4385571469 hasPublicationYear "2023" @default.
- W4385571469 type Work @default.
- W4385571469 citedByCount "0" @default.
- W4385571469 crossrefType "proceedings-article" @default.
- W4385571469 hasAuthorship W4385571469A5046520540 @default.
- W4385571469 hasAuthorship W4385571469A5081756234 @default.
- W4385571469 hasBestOaLocation W43855714691 @default.
- W4385571469 hasConcept C153294291 @default.
- W4385571469 hasConcept C154290570 @default.
- W4385571469 hasConcept C154945302 @default.
- W4385571469 hasConcept C162324750 @default.
- W4385571469 hasConcept C187736073 @default.
- W4385571469 hasConcept C204321447 @default.
- W4385571469 hasConcept C205203396 @default.
- W4385571469 hasConcept C205649164 @default.
- W4385571469 hasConcept C2777211547 @default.
- W4385571469 hasConcept C2780451532 @default.
- W4385571469 hasConcept C31972630 @default.
- W4385571469 hasConcept C41008148 @default.
- W4385571469 hasConceptScore W4385571469C153294291 @default.
- W4385571469 hasConceptScore W4385571469C154290570 @default.
- W4385571469 hasConceptScore W4385571469C154945302 @default.
- W4385571469 hasConceptScore W4385571469C162324750 @default.
- W4385571469 hasConceptScore W4385571469C187736073 @default.
- W4385571469 hasConceptScore W4385571469C204321447 @default.
- W4385571469 hasConceptScore W4385571469C205203396 @default.
- W4385571469 hasConceptScore W4385571469C205649164 @default.
- W4385571469 hasConceptScore W4385571469C2777211547 @default.
- W4385571469 hasConceptScore W4385571469C2780451532 @default.
- W4385571469 hasConceptScore W4385571469C31972630 @default.
- W4385571469 hasConceptScore W4385571469C41008148 @default.
- W4385571469 hasLocation W43855714691 @default.
- W4385571469 hasOpenAccess W4385571469 @default.
- W4385571469 hasPrimaryLocation W43855714691 @default.
- W4385571469 hasRelatedWork W1512718085 @default.
- W4385571469 hasRelatedWork W1514108082 @default.
- W4385571469 hasRelatedWork W1569841287 @default.
- W4385571469 hasRelatedWork W2081647779 @default.
- W4385571469 hasRelatedWork W2293457016 @default.
- W4385571469 hasRelatedWork W2369308426 @default.
- W4385571469 hasRelatedWork W2789919619 @default.
- W4385571469 hasRelatedWork W4237750775 @default.
- W4385571469 hasRelatedWork W1551406738 @default.
- W4385571469 hasRelatedWork W2610387714 @default.
- W4385571469 isParatext "false" @default.
- W4385571469 isRetracted "false" @default.
- W4385571469 workType "article" @default.