Matches in SemOpenAlex for { <https://semopenalex.org/work/W4309132338> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4309132338 abstract "Large pre-trained models have revolutionized natural language processing (NLP) research and applications, but high training costs and limited data resources have prevented their benefits from being shared equally amongst speakers of all the world's languages. To address issues of cross-linguistic access to such models and reduce energy consumption for sustainability during large-scale model training, this study proposes an effective and energy-efficient framework called GreenPLM that uses bilingual lexicons to directly translate pre-trained language models of one language into another at almost no additional cost. We validate this approach in 18 languages' BERT models and show that this framework is comparable to, if not better than, other heuristics with high training costs. In addition, given lightweight continued pre-training on limited data where available, this framework outperforms the original monolingual language models in six out of seven tested languages with up to 200x less pre-training efforts. Aiming at the Leave No One Behind Principle (LNOB), our approach manages to reduce inequalities between languages and energy consumption greatly. We make our codes and models publicly available here: url{https://github.com/qcznlp/GreenPLMs}" @default.
- W4309132338 created "2022-11-23" @default.
- W4309132338 creator A5032723793 @default.
- W4309132338 creator A5051633211 @default.
- W4309132338 creator A5052723398 @default.
- W4309132338 creator A5054429473 @default.
- W4309132338 creator A5073606278 @default.
- W4309132338 creator A5079280336 @default.
- W4309132338 creator A5081953757 @default.
- W4309132338 creator A5083041690 @default.
- W4309132338 creator A5090886028 @default.
- W4309132338 date "2022-11-13" @default.
- W4309132338 modified "2023-10-16" @default.
- W4309132338 title "GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost" @default.
- W4309132338 doi "https://doi.org/10.48550/arxiv.2211.06993" @default.
- W4309132338 hasPublicationYear "2022" @default.
- W4309132338 type Work @default.
- W4309132338 citedByCount "0" @default.
- W4309132338 crossrefType "posted-content" @default.
- W4309132338 hasAuthorship W4309132338A5032723793 @default.
- W4309132338 hasAuthorship W4309132338A5051633211 @default.
- W4309132338 hasAuthorship W4309132338A5052723398 @default.
- W4309132338 hasAuthorship W4309132338A5054429473 @default.
- W4309132338 hasAuthorship W4309132338A5073606278 @default.
- W4309132338 hasAuthorship W4309132338A5079280336 @default.
- W4309132338 hasAuthorship W4309132338A5081953757 @default.
- W4309132338 hasAuthorship W4309132338A5083041690 @default.
- W4309132338 hasAuthorship W4309132338A5090886028 @default.
- W4309132338 hasBestOaLocation W43091323381 @default.
- W4309132338 hasConcept C111919701 @default.
- W4309132338 hasConcept C119599485 @default.
- W4309132338 hasConcept C121332964 @default.
- W4309132338 hasConcept C127413603 @default.
- W4309132338 hasConcept C127705205 @default.
- W4309132338 hasConcept C137293760 @default.
- W4309132338 hasConcept C154945302 @default.
- W4309132338 hasConcept C173608175 @default.
- W4309132338 hasConcept C204321447 @default.
- W4309132338 hasConcept C2776175482 @default.
- W4309132338 hasConcept C2778755073 @default.
- W4309132338 hasConcept C2780165032 @default.
- W4309132338 hasConcept C41008148 @default.
- W4309132338 hasConcept C51632099 @default.
- W4309132338 hasConcept C62520636 @default.
- W4309132338 hasConceptScore W4309132338C111919701 @default.
- W4309132338 hasConceptScore W4309132338C119599485 @default.
- W4309132338 hasConceptScore W4309132338C121332964 @default.
- W4309132338 hasConceptScore W4309132338C127413603 @default.
- W4309132338 hasConceptScore W4309132338C127705205 @default.
- W4309132338 hasConceptScore W4309132338C137293760 @default.
- W4309132338 hasConceptScore W4309132338C154945302 @default.
- W4309132338 hasConceptScore W4309132338C173608175 @default.
- W4309132338 hasConceptScore W4309132338C204321447 @default.
- W4309132338 hasConceptScore W4309132338C2776175482 @default.
- W4309132338 hasConceptScore W4309132338C2778755073 @default.
- W4309132338 hasConceptScore W4309132338C2780165032 @default.
- W4309132338 hasConceptScore W4309132338C41008148 @default.
- W4309132338 hasConceptScore W4309132338C51632099 @default.
- W4309132338 hasConceptScore W4309132338C62520636 @default.
- W4309132338 hasLocation W43091323381 @default.
- W4309132338 hasOpenAccess W4309132338 @default.
- W4309132338 hasPrimaryLocation W43091323381 @default.
- W4309132338 hasRelatedWork W142374489 @default.
- W4309132338 hasRelatedWork W1803932089 @default.
- W4309132338 hasRelatedWork W1985007624 @default.
- W4309132338 hasRelatedWork W2176369193 @default.
- W4309132338 hasRelatedWork W2351428524 @default.
- W4309132338 hasRelatedWork W2359001871 @default.
- W4309132338 hasRelatedWork W3107474891 @default.
- W4309132338 hasRelatedWork W68817833 @default.
- W4309132338 hasRelatedWork W88325386 @default.
- W4309132338 hasRelatedWork W2584532118 @default.
- W4309132338 isParatext "false" @default.
- W4309132338 isRetracted "false" @default.
- W4309132338 workType "article" @default.