Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385569873> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4385569873 abstract "This paper aims to benchmark recent progress in language understanding models that output contextualised representations at the character level. Many such modelling architectures and methods to train those architectures have been proposed, but it is currently unclear what the relative contributions of the architecture vs. the pretraining objective are to final model performance. We explore the design space of such models, comparing architectural innovations (Clark et al., 2022, Jaegle et al., 2022, Tay et al., 2021) and a variety of different pretraining objectives on a suite of evaluation tasks with a fixed training procedure in order to find the currently optimal way to build and train character-level BERT-like models. We find that our best performing character-level model exceeds the performance of a token-based model trained with the same settings on the same data, suggesting that character-level models are ready for more widespread adoption. Unfortunately, the best method to train character-level models still relies on a subword-level tokeniser during pretraining, and final model performance is highly dependent on tokeniser quality. We believe our results demonstrate the readiness of character-level models for multilingual language representation, and encourage NLP practitioners to try them as drop-in replacements for token-based models." @default.
- W4385569873 created "2023-08-05" @default.
- W4385569873 creator A5029820246 @default.
- W4385569873 date "2023-01-01" @default.
- W4385569873 modified "2023-09-24" @default.
- W4385569873 title "What is the best recipe for character-level encoder-only modelling?" @default.
- W4385569873 doi "https://doi.org/10.18653/v1/2023.acl-long.326" @default.
- W4385569873 hasPublicationYear "2023" @default.
- W4385569873 type Work @default.
- W4385569873 citedByCount "0" @default.
- W4385569873 crossrefType "proceedings-article" @default.
- W4385569873 hasAuthorship W4385569873A5029820246 @default.
- W4385569873 hasBestOaLocation W43855698731 @default.
- W4385569873 hasConcept C13280743 @default.
- W4385569873 hasConcept C136197465 @default.
- W4385569873 hasConcept C137293760 @default.
- W4385569873 hasConcept C154945302 @default.
- W4385569873 hasConcept C166957645 @default.
- W4385569873 hasConcept C17744445 @default.
- W4385569873 hasConcept C185592680 @default.
- W4385569873 hasConcept C185798385 @default.
- W4385569873 hasConcept C199539241 @default.
- W4385569873 hasConcept C204321447 @default.
- W4385569873 hasConcept C205649164 @default.
- W4385569873 hasConcept C2524010 @default.
- W4385569873 hasConcept C2776359362 @default.
- W4385569873 hasConcept C2778671685 @default.
- W4385569873 hasConcept C2780861071 @default.
- W4385569873 hasConcept C31903555 @default.
- W4385569873 hasConcept C33923547 @default.
- W4385569873 hasConcept C38652104 @default.
- W4385569873 hasConcept C41008148 @default.
- W4385569873 hasConcept C48145219 @default.
- W4385569873 hasConcept C79581498 @default.
- W4385569873 hasConcept C94625758 @default.
- W4385569873 hasConcept C95457728 @default.
- W4385569873 hasConceptScore W4385569873C13280743 @default.
- W4385569873 hasConceptScore W4385569873C136197465 @default.
- W4385569873 hasConceptScore W4385569873C137293760 @default.
- W4385569873 hasConceptScore W4385569873C154945302 @default.
- W4385569873 hasConceptScore W4385569873C166957645 @default.
- W4385569873 hasConceptScore W4385569873C17744445 @default.
- W4385569873 hasConceptScore W4385569873C185592680 @default.
- W4385569873 hasConceptScore W4385569873C185798385 @default.
- W4385569873 hasConceptScore W4385569873C199539241 @default.
- W4385569873 hasConceptScore W4385569873C204321447 @default.
- W4385569873 hasConceptScore W4385569873C205649164 @default.
- W4385569873 hasConceptScore W4385569873C2524010 @default.
- W4385569873 hasConceptScore W4385569873C2776359362 @default.
- W4385569873 hasConceptScore W4385569873C2778671685 @default.
- W4385569873 hasConceptScore W4385569873C2780861071 @default.
- W4385569873 hasConceptScore W4385569873C31903555 @default.
- W4385569873 hasConceptScore W4385569873C33923547 @default.
- W4385569873 hasConceptScore W4385569873C38652104 @default.
- W4385569873 hasConceptScore W4385569873C41008148 @default.
- W4385569873 hasConceptScore W4385569873C48145219 @default.
- W4385569873 hasConceptScore W4385569873C79581498 @default.
- W4385569873 hasConceptScore W4385569873C94625758 @default.
- W4385569873 hasConceptScore W4385569873C95457728 @default.
- W4385569873 hasLocation W43855698731 @default.
- W4385569873 hasOpenAccess W4385569873 @default.
- W4385569873 hasPrimaryLocation W43855698731 @default.
- W4385569873 hasRelatedWork W1485630101 @default.
- W4385569873 hasRelatedWork W2081245617 @default.
- W4385569873 hasRelatedWork W2176796957 @default.
- W4385569873 hasRelatedWork W2359001871 @default.
- W4385569873 hasRelatedWork W3040652864 @default.
- W4385569873 hasRelatedWork W3166508187 @default.
- W4385569873 hasRelatedWork W4287601440 @default.
- W4385569873 hasRelatedWork W4287629333 @default.
- W4385569873 hasRelatedWork W4307415490 @default.
- W4385569873 hasRelatedWork W4377161536 @default.
- W4385569873 isParatext "false" @default.
- W4385569873 isRetracted "false" @default.
- W4385569873 workType "article" @default.