Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385764025> ?p ?o ?g. }
Showing items 1 to 74 of
74
with 100 items per page.
- W4385764025 abstract "Large pre-trained models have revolutionized natural language processing (NLP) research and applications, but high training costs and limited data resources have prevented their benefits from being shared equally amongst speakers of all the world's languages. To address issues of cross-linguistic access to such models and reduce energy consumption for sustainability during large-scale model training, this study proposes an effective and energy-efficient framework called GreenPLM that uses bilingual lexicons to directly ``translate'' pre-trained language models of one language into another at almost no additional cost. We validate this approach in 18 languages' BERT models and show that this framework is comparable to, if not better than, other heuristics with high training costs. In addition, given lightweight continued pre-training on limited data where available, this framework outperforms the original monolingual language models in six out of seven tested languages with up to 200x less pre-training efforts. Aiming at the Leave No One Behind Principle (LNOB), our approach manages to reduce inequalities between languages and energy consumption greatly. We make our codes and models publicly available at https://github.com/qcznlp/GreenPLMs." @default.
- W4385764025 created "2023-08-12" @default.
- W4385764025 creator A5032723793 @default.
- W4385764025 creator A5051633211 @default.
- W4385764025 creator A5052723398 @default.
- W4385764025 creator A5054429473 @default.
- W4385764025 creator A5070966744 @default.
- W4385764025 creator A5073606278 @default.
- W4385764025 creator A5079280336 @default.
- W4385764025 creator A5081953757 @default.
- W4385764025 creator A5083041690 @default.
- W4385764025 creator A5090886028 @default.
- W4385764025 date "2023-08-01" @default.
- W4385764025 modified "2023-10-03" @default.
- W4385764025 title "GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost" @default.
- W4385764025 doi "https://doi.org/10.24963/ijcai.2023/698" @default.
- W4385764025 hasPublicationYear "2023" @default.
- W4385764025 type Work @default.
- W4385764025 citedByCount "0" @default.
- W4385764025 crossrefType "proceedings-article" @default.
- W4385764025 hasAuthorship W4385764025A5032723793 @default.
- W4385764025 hasAuthorship W4385764025A5051633211 @default.
- W4385764025 hasAuthorship W4385764025A5052723398 @default.
- W4385764025 hasAuthorship W4385764025A5054429473 @default.
- W4385764025 hasAuthorship W4385764025A5070966744 @default.
- W4385764025 hasAuthorship W4385764025A5073606278 @default.
- W4385764025 hasAuthorship W4385764025A5079280336 @default.
- W4385764025 hasAuthorship W4385764025A5081953757 @default.
- W4385764025 hasAuthorship W4385764025A5083041690 @default.
- W4385764025 hasAuthorship W4385764025A5090886028 @default.
- W4385764025 hasBestOaLocation W43857640251 @default.
- W4385764025 hasConcept C111919701 @default.
- W4385764025 hasConcept C119599485 @default.
- W4385764025 hasConcept C121332964 @default.
- W4385764025 hasConcept C127413603 @default.
- W4385764025 hasConcept C127705205 @default.
- W4385764025 hasConcept C137293760 @default.
- W4385764025 hasConcept C154945302 @default.
- W4385764025 hasConcept C204321447 @default.
- W4385764025 hasConcept C2778755073 @default.
- W4385764025 hasConcept C2780165032 @default.
- W4385764025 hasConcept C41008148 @default.
- W4385764025 hasConcept C51632099 @default.
- W4385764025 hasConcept C62520636 @default.
- W4385764025 hasConceptScore W4385764025C111919701 @default.
- W4385764025 hasConceptScore W4385764025C119599485 @default.
- W4385764025 hasConceptScore W4385764025C121332964 @default.
- W4385764025 hasConceptScore W4385764025C127413603 @default.
- W4385764025 hasConceptScore W4385764025C127705205 @default.
- W4385764025 hasConceptScore W4385764025C137293760 @default.
- W4385764025 hasConceptScore W4385764025C154945302 @default.
- W4385764025 hasConceptScore W4385764025C204321447 @default.
- W4385764025 hasConceptScore W4385764025C2778755073 @default.
- W4385764025 hasConceptScore W4385764025C2780165032 @default.
- W4385764025 hasConceptScore W4385764025C41008148 @default.
- W4385764025 hasConceptScore W4385764025C51632099 @default.
- W4385764025 hasConceptScore W4385764025C62520636 @default.
- W4385764025 hasLocation W43857640251 @default.
- W4385764025 hasLocation W43857640252 @default.
- W4385764025 hasOpenAccess W4385764025 @default.
- W4385764025 hasPrimaryLocation W43857640251 @default.
- W4385764025 hasRelatedWork W1756885467 @default.
- W4385764025 hasRelatedWork W1989705153 @default.
- W4385764025 hasRelatedWork W2274408985 @default.
- W4385764025 hasRelatedWork W2359001871 @default.
- W4385764025 hasRelatedWork W2510503866 @default.
- W4385764025 hasRelatedWork W2884576438 @default.
- W4385764025 hasRelatedWork W2905364337 @default.
- W4385764025 hasRelatedWork W3046207468 @default.
- W4385764025 hasRelatedWork W3120880016 @default.
- W4385764025 hasRelatedWork W3204941900 @default.
- W4385764025 isParatext "false" @default.
- W4385764025 isRetracted "false" @default.
- W4385764025 workType "article" @default.