FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3035124264> ?p ?o ?g. }

• W3035124264 abstract "Pre-trained contextual representations (e.g., BERT) have become the foundation to achieve state-of-the-art results on many NLP tasks. However, large-scale pre-training is computationally expensive. ELECTRA, an early attempt to accelerate pre-training, trains a discriminative model that predicts whether each input token was replaced by a generator. Our studies reveal that ELECTRA's success is mainly due to its reduced complexity of the pre-training task: the binary classification (replaced token detection) is more efficient to learn than the generation task (masked language modeling). However, such a simplified task is less semantically informative. To achieve better efficiency and effectiveness, we propose a novel meta-learning framework, MC-BERT. The pre-training task is a multi-choice cloze test with a reject option, where a meta controller network provides training input and candidates. Results over GLUE natural language understanding benchmark demonstrate that our proposed method is both efficient and effective: it outperforms baselines on GLUE semantic tasks given the same computational budget." @default.
• W3035124264 created "2020-06-19" @default.
• W3035124264 creator A5021438219 @default.
• W3035124264 creator A5051093347 @default.
• W3035124264 creator A5055723755 @default.
• W3035124264 creator A5061200287 @default.
• W3035124264 creator A5068285627 @default.
• W3035124264 creator A5070990160 @default.
• W3035124264 creator A5074453054 @default.
• W3035124264 creator A5084566053 @default.
• W3035124264 date "2020-06-10" @default.
• W3035124264 modified "2023-09-24" @default.
• W3035124264 title "MC-BERT: Efficient Language Pre-Training via a Meta Controller" @default.
• W3035124264 cites W1816313093 @default.
• W3035124264 cites W1980816265 @default.
• W3035124264 cites W2124807415 @default.
• W3035124264 cites W2270070752 @default.
• W3035124264 cites W2296073425 @default.
• W3035124264 cites W2626778328 @default.
• W3035124264 cites W2787560479 @default.
• W3035124264 cites W2799054028 @default.
• W3035124264 cites W2945785363 @default.
• W3035124264 cites W2948223045 @default.
• W3035124264 cites W2949433733 @default.
• W3035124264 cites W2950813464 @default.
• W3035124264 cites W2963341956 @default.
• W3035124264 cites W2965373594 @default.
• W3035124264 cites W2975059944 @default.
• W3035124264 cites W2981852735 @default.
• W3035124264 cites W3013571468 @default.
• W3035124264 cites W3025557979 @default.
• W3035124264 doi "https://doi.org/10.48550/arxiv.2006.05744" @default.
• W3035124264 hasPublicationYear "2020" @default.
• W3035124264 type Work @default.
• W3035124264 sameAs 3035124264 @default.
• W3035124264 citedByCount "6" @default.
• W3035124264 countsByYear W30351242642020 @default.
• W3035124264 countsByYear W30351242642021 @default.
• W3035124264 crossrefType "posted-content" @default.
• W3035124264 hasAuthorship W3035124264A5021438219 @default.
• W3035124264 hasAuthorship W3035124264A5051093347 @default.
• W3035124264 hasAuthorship W3035124264A5055723755 @default.
• W3035124264 hasAuthorship W3035124264A5061200287 @default.
• W3035124264 hasAuthorship W3035124264A5068285627 @default.
• W3035124264 hasAuthorship W3035124264A5070990160 @default.
• W3035124264 hasAuthorship W3035124264A5074453054 @default.
• W3035124264 hasAuthorship W3035124264A5084566053 @default.
• W3035124264 hasBestOaLocation W30351242641 @default.
• W3035124264 hasConcept C119857082 @default.
• W3035124264 hasConcept C121332964 @default.
• W3035124264 hasConcept C12267149 @default.
• W3035124264 hasConcept C13280743 @default.
• W3035124264 hasConcept C137293760 @default.
• W3035124264 hasConcept C154945302 @default.
• W3035124264 hasConcept C162324750 @default.
• W3035124264 hasConcept C163258240 @default.
• W3035124264 hasConcept C185798385 @default.
• W3035124264 hasConcept C187736073 @default.
• W3035124264 hasConcept C195324797 @default.
• W3035124264 hasConcept C203479927 @default.
• W3035124264 hasConcept C204321447 @default.
• W3035124264 hasConcept C205649164 @default.
• W3035124264 hasConcept C2779439875 @default.
• W3035124264 hasConcept C2780451532 @default.
• W3035124264 hasConcept C2780992000 @default.
• W3035124264 hasConcept C38652104 @default.
• W3035124264 hasConcept C41008148 @default.
• W3035124264 hasConcept C48145219 @default.
• W3035124264 hasConcept C62520636 @default.
• W3035124264 hasConcept C6557445 @default.
• W3035124264 hasConcept C66905080 @default.
• W3035124264 hasConcept C86803240 @default.
• W3035124264 hasConcept C97931131 @default.
• W3035124264 hasConceptScore W3035124264C119857082 @default.
• W3035124264 hasConceptScore W3035124264C121332964 @default.
• W3035124264 hasConceptScore W3035124264C12267149 @default.
• W3035124264 hasConceptScore W3035124264C13280743 @default.
• W3035124264 hasConceptScore W3035124264C137293760 @default.
• W3035124264 hasConceptScore W3035124264C154945302 @default.
• W3035124264 hasConceptScore W3035124264C162324750 @default.
• W3035124264 hasConceptScore W3035124264C163258240 @default.
• W3035124264 hasConceptScore W3035124264C185798385 @default.
• W3035124264 hasConceptScore W3035124264C187736073 @default.
• W3035124264 hasConceptScore W3035124264C195324797 @default.
• W3035124264 hasConceptScore W3035124264C203479927 @default.
• W3035124264 hasConceptScore W3035124264C204321447 @default.
• W3035124264 hasConceptScore W3035124264C205649164 @default.
• W3035124264 hasConceptScore W3035124264C2779439875 @default.
• W3035124264 hasConceptScore W3035124264C2780451532 @default.
• W3035124264 hasConceptScore W3035124264C2780992000 @default.
• W3035124264 hasConceptScore W3035124264C38652104 @default.
• W3035124264 hasConceptScore W3035124264C41008148 @default.
• W3035124264 hasConceptScore W3035124264C48145219 @default.
• W3035124264 hasConceptScore W3035124264C62520636 @default.
• W3035124264 hasConceptScore W3035124264C6557445 @default.
• W3035124264 hasConceptScore W3035124264C66905080 @default.
• W3035124264 hasConceptScore W3035124264C86803240 @default.
• W3035124264 hasConceptScore W3035124264C97931131 @default.
• W3035124264 hasLocation W30351242641 @default.
• W3035124264 hasOpenAccess W3035124264 @default.

• next