Séminaire de Recherche en Linguistique

Children’s remarkable ability to learn language has been an object of fascination in science for millennia. But in just the last few years, neural language models (LMs) have also proven to be incredibly adept at learning human language. I argue that—under the right circumstances—we can use LMs to test key hypotheses about language acquisition that have been difficult or impossible to evaluate with human subjects. As a proof-of-concept, I obtain experimental evidence for the controversial claim that language models can acquire structural rules such as subject-auxiliary inversion in English without the aid of direct evidence in the input. Nonetheless, obstacles remain in the way of using LMs as models of human language acquisition. LMs are extremely inefficient learners compared to humans, requiring hundreds or thousands of times more data to reach human-like grammatical generalizations. I present results from the first BabyLM Challenge, a shared task challenging researchers to train LMs with constrained data inspired by the input to children. From over 30 submissions, including my own work, we uncover promising architectural and data preprocessing advances which improve data efficiency, and we explore multimodality and interaction as methods to increase developmental plausibility of language models.