While language models (LMs) grow larger and gain new capabilities, their performance in non-English languages increasingly lags behind. This is due to the curse of multilinguality, where each individual language's performance suffers when models are trained on more languages. In this talk, I first examine how current language models do and don't capture different languages and uncover how the curse of multilinguality develops during multilingual model training. Building on these insights, I then present a new method, Multilingual Expert Language Models (X-ELM), that breaks this curse of multilinguality by facilitating more equitable multilingual language modeling. We show that X-ELMs provide many performance and efficiency benefits over exisiting multilingual modeling approaches, indicating their potential to democratize multilingual NLP.
*** The talk will be delivered in person (MFF UK, Malostranské nám. 25, 4th floor, room S1) and will be streamed via Zoom. For details how to join the Zoom meeting, please write to sevcikova et ufal.mff.cuni.cz ***
Terra Blevins is a postdoctoral researcher at the University of Vienna and an incoming assistant professor at Northeastern University. She holds a Ph.D. in Computer Science from the University of Washington, where she was advised by Luke Zettlemoyer and worked as a visiting researcher at Facebook AI Research (FAIR). Her research focuses on multilingual NLP and analyzing the linguistic knowledge of language models, with the overarching aim of using analysis insights to build better-performing and more equitable multilingual systems.