Matches in SemOpenAlex for { <https://semopenalex.org/work/W3213014097> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W3213014097 abstract "Transformer-based language models have taken the NLP world by storm. However, their potential for addressing important questions in language acquisition research has been largely ignored. In this work, we examined the grammatical knowledge of RoBERTa (Liu et al., 2019) when trained on a 5M word corpus of language acquisition data to simulate the input available to children between the ages 1 and 6. Using the behavioral probing paradigm, we found that a smaller version of RoBERTa-base that never predicts unmasked tokens, which we term BabyBERTa, acquires grammatical knowledge comparable to that of pre-trained RoBERTa-base - and does so with approximately 15X fewer parameters and 6,000X fewer words. We discuss implications for building more efficient models and the learnability of grammar from input available to children. Lastly, to support research on this front, we release our novel grammar test suite that is compatible with the small vocabulary of child-directed input." @default.
- W3213014097 created "2021-11-22" @default.
- W3213014097 creator A5008108776 @default.
- W3213014097 creator A5023802054 @default.
- W3213014097 creator A5026220115 @default.
- W3213014097 creator A5030075195 @default.
- W3213014097 date "2021-01-01" @default.
- W3213014097 modified "2023-10-13" @default.
- W3213014097 title "BabyBERTa: Learning More Grammar With Small-Scale Child-Directed Language" @default.
- W3213014097 cites W1599016936 @default.
- W3213014097 cites W1980096805 @default.
- W3213014097 cites W2001223827 @default.
- W3213014097 cites W2011364639 @default.
- W3213014097 cites W2048631778 @default.
- W3213014097 cites W2100175650 @default.
- W3213014097 cites W2101252293 @default.
- W3213014097 cites W2127438782 @default.
- W3213014097 cites W2135891083 @default.
- W3213014097 cites W2143992713 @default.
- W3213014097 cites W2167543949 @default.
- W3213014097 cites W2592920763 @default.
- W3213014097 cites W2787076719 @default.
- W3213014097 cites W2798665661 @default.
- W3213014097 cites W2882319491 @default.
- W3213014097 cites W2962801832 @default.
- W3213014097 cites W2962911926 @default.
- W3213014097 cites W2963341956 @default.
- W3213014097 cites W2963751529 @default.
- W3213014097 cites W2996728628 @default.
- W3213014097 cites W3004346089 @default.
- W3213014097 cites W3035267217 @default.
- W3213014097 cites W3041594829 @default.
- W3213014097 cites W3098613713 @default.
- W3213014097 cites W3103536442 @default.
- W3213014097 cites W3134522972 @default.
- W3213014097 cites W3168987555 @default.
- W3213014097 cites W3211156350 @default.
- W3213014097 doi "https://doi.org/10.18653/v1/2021.conll-1.49" @default.
- W3213014097 hasPublicationYear "2021" @default.
- W3213014097 type Work @default.
- W3213014097 sameAs 3213014097 @default.
- W3213014097 citedByCount "5" @default.
- W3213014097 countsByYear W32130140972022 @default.
- W3213014097 countsByYear W32130140972023 @default.
- W3213014097 crossrefType "proceedings-article" @default.
- W3213014097 hasAuthorship W3213014097A5008108776 @default.
- W3213014097 hasAuthorship W3213014097A5023802054 @default.
- W3213014097 hasAuthorship W3213014097A5026220115 @default.
- W3213014097 hasAuthorship W3213014097A5030075195 @default.
- W3213014097 hasBestOaLocation W32130140971 @default.
- W3213014097 hasConcept C138885662 @default.
- W3213014097 hasConcept C148934300 @default.
- W3213014097 hasConcept C154945302 @default.
- W3213014097 hasConcept C166957645 @default.
- W3213014097 hasConcept C204321447 @default.
- W3213014097 hasConcept C26022165 @default.
- W3213014097 hasConcept C2777601683 @default.
- W3213014097 hasConcept C2777723229 @default.
- W3213014097 hasConcept C41008148 @default.
- W3213014097 hasConcept C41895202 @default.
- W3213014097 hasConcept C4554734 @default.
- W3213014097 hasConcept C4768521 @default.
- W3213014097 hasConcept C74672266 @default.
- W3213014097 hasConcept C79581498 @default.
- W3213014097 hasConcept C95457728 @default.
- W3213014097 hasConceptScore W3213014097C138885662 @default.
- W3213014097 hasConceptScore W3213014097C148934300 @default.
- W3213014097 hasConceptScore W3213014097C154945302 @default.
- W3213014097 hasConceptScore W3213014097C166957645 @default.
- W3213014097 hasConceptScore W3213014097C204321447 @default.
- W3213014097 hasConceptScore W3213014097C26022165 @default.
- W3213014097 hasConceptScore W3213014097C2777601683 @default.
- W3213014097 hasConceptScore W3213014097C2777723229 @default.
- W3213014097 hasConceptScore W3213014097C41008148 @default.
- W3213014097 hasConceptScore W3213014097C41895202 @default.
- W3213014097 hasConceptScore W3213014097C4554734 @default.
- W3213014097 hasConceptScore W3213014097C4768521 @default.
- W3213014097 hasConceptScore W3213014097C74672266 @default.
- W3213014097 hasConceptScore W3213014097C79581498 @default.
- W3213014097 hasConceptScore W3213014097C95457728 @default.
- W3213014097 hasLocation W32130140971 @default.
- W3213014097 hasOpenAccess W3213014097 @default.
- W3213014097 hasPrimaryLocation W32130140971 @default.
- W3213014097 hasRelatedWork W1503204927 @default.
- W3213014097 hasRelatedWork W1987665455 @default.
- W3213014097 hasRelatedWork W2069128200 @default.
- W3213014097 hasRelatedWork W2150540909 @default.
- W3213014097 hasRelatedWork W2162731409 @default.
- W3213014097 hasRelatedWork W2164342658 @default.
- W3213014097 hasRelatedWork W3012550294 @default.
- W3213014097 hasRelatedWork W3169831614 @default.
- W3213014097 hasRelatedWork W4245991804 @default.
- W3213014097 hasRelatedWork W4302791369 @default.
- W3213014097 isParatext "false" @default.
- W3213014097 isRetracted "false" @default.
- W3213014097 magId "3213014097" @default.
- W3213014097 workType "article" @default.