Matches in SemOpenAlex for { <https://semopenalex.org/work/W3199464963> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W3199464963 abstract "Recent progress in the Natural Language Processing domain has given us several State-of-the-Art (SOTA) pretrained models which can be finetuned for specific tasks. These large models with billions of parameters trained on numerous GPUs/TPUs over weeks are leading in the benchmark leaderboards. In this paper, we discuss the need for a benchmark for cost and time effective smaller models trained on a single GPU. This will enable researchers with resource constraints experiment with novel and innovative ideas on tokenization, pretraining tasks, architecture, fine tuning methods etc. We set up Small-Bench NLP, a benchmark for small efficient neural language models trained on a single GPU. Small-Bench NLP benchmark comprises of eight NLP tasks on the publicly available GLUE datasets and a leaderboard to track the progress of the community. Our ELECTRA-DeBERTa (15M parameters) small model architecture achieves an average score of 81.53 which is comparable to that of BERT-Base's 82.20 (110M parameters). Our models, code and leaderboard are available at this https URL" @default.
- W3199464963 created "2021-09-27" @default.
- W3199464963 creator A5025657503 @default.
- W3199464963 creator A5038819414 @default.
- W3199464963 creator A5088200625 @default.
- W3199464963 date "2021-09-22" @default.
- W3199464963 modified "2023-09-27" @default.
- W3199464963 title "Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing" @default.
- W3199464963 cites W2799054028 @default.
- W3199464963 cites W2945260553 @default.
- W3199464963 cites W2950813464 @default.
- W3199464963 cites W2963341956 @default.
- W3199464963 cites W2963403868 @default.
- W3199464963 cites W2964121744 @default.
- W3199464963 cites W2965373594 @default.
- W3199464963 cites W2990704537 @default.
- W3199464963 cites W2996428491 @default.
- W3199464963 cites W3013571468 @default.
- W3199464963 cites W3082274269 @default.
- W3199464963 cites W3105163367 @default.
- W3199464963 cites W3122890974 @default.
- W3199464963 cites W3133702157 @default.
- W3199464963 hasPublicationYear "2021" @default.
- W3199464963 type Work @default.
- W3199464963 sameAs 3199464963 @default.
- W3199464963 citedByCount "0" @default.
- W3199464963 crossrefType "posted-content" @default.
- W3199464963 hasAuthorship W3199464963A5025657503 @default.
- W3199464963 hasAuthorship W3199464963A5038819414 @default.
- W3199464963 hasAuthorship W3199464963A5088200625 @default.
- W3199464963 hasConcept C119857082 @default.
- W3199464963 hasConcept C13280743 @default.
- W3199464963 hasConcept C137293760 @default.
- W3199464963 hasConcept C138885662 @default.
- W3199464963 hasConcept C154945302 @default.
- W3199464963 hasConcept C177264268 @default.
- W3199464963 hasConcept C185798385 @default.
- W3199464963 hasConcept C199360897 @default.
- W3199464963 hasConcept C204321447 @default.
- W3199464963 hasConcept C205649164 @default.
- W3199464963 hasConcept C2776760102 @default.
- W3199464963 hasConcept C41008148 @default.
- W3199464963 hasConcept C41895202 @default.
- W3199464963 hasConcept C43126263 @default.
- W3199464963 hasConcept C90805587 @default.
- W3199464963 hasConceptScore W3199464963C119857082 @default.
- W3199464963 hasConceptScore W3199464963C13280743 @default.
- W3199464963 hasConceptScore W3199464963C137293760 @default.
- W3199464963 hasConceptScore W3199464963C138885662 @default.
- W3199464963 hasConceptScore W3199464963C154945302 @default.
- W3199464963 hasConceptScore W3199464963C177264268 @default.
- W3199464963 hasConceptScore W3199464963C185798385 @default.
- W3199464963 hasConceptScore W3199464963C199360897 @default.
- W3199464963 hasConceptScore W3199464963C204321447 @default.
- W3199464963 hasConceptScore W3199464963C205649164 @default.
- W3199464963 hasConceptScore W3199464963C2776760102 @default.
- W3199464963 hasConceptScore W3199464963C41008148 @default.
- W3199464963 hasConceptScore W3199464963C41895202 @default.
- W3199464963 hasConceptScore W3199464963C43126263 @default.
- W3199464963 hasConceptScore W3199464963C90805587 @default.
- W3199464963 hasLocation W31994649631 @default.
- W3199464963 hasOpenAccess W3199464963 @default.
- W3199464963 hasPrimaryLocation W31994649631 @default.
- W3199464963 hasRelatedWork W1819293553 @default.
- W3199464963 hasRelatedWork W2084660366 @default.
- W3199464963 hasRelatedWork W2239144794 @default.
- W3199464963 hasRelatedWork W2250379827 @default.
- W3199464963 hasRelatedWork W2587914027 @default.
- W3199464963 hasRelatedWork W2594990650 @default.
- W3199464963 hasRelatedWork W2805493160 @default.
- W3199464963 hasRelatedWork W2885723791 @default.
- W3199464963 hasRelatedWork W2886958107 @default.
- W3199464963 hasRelatedWork W2937249639 @default.
- W3199464963 hasRelatedWork W2954767663 @default.
- W3199464963 hasRelatedWork W2963240192 @default.
- W3199464963 hasRelatedWork W2963674387 @default.
- W3199464963 hasRelatedWork W2964324519 @default.
- W3199464963 hasRelatedWork W2987776948 @default.
- W3199464963 hasRelatedWork W3037896803 @default.
- W3199464963 hasRelatedWork W3104613320 @default.
- W3199464963 hasRelatedWork W3120129399 @default.
- W3199464963 hasRelatedWork W3128874339 @default.
- W3199464963 hasRelatedWork W3129020024 @default.
- W3199464963 isParatext "false" @default.
- W3199464963 isRetracted "false" @default.
- W3199464963 magId "3199464963" @default.
- W3199464963 workType "article" @default.