Combining electrophysiology and deep neural network models in research on language comprehension

• Applicant: Professorin Dr. Milena Rabovsky
• Subject Area: Biological Psychology and Cognitive Neuroscience
• Term: since 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 400565621

Language processing is one of the most intriguing and arguably uniquely human capacities. Even though much research from different disciplines and perspectives revolves around better understanding the neurobiological and cognitive basis of language processing, the answers to even basic questions remain elusive. For instance, is language processing best viewed as an abstract rule-based system that is too complex to be learned and essential aspects of which therefore must be innate? Or can the language capacity emerge from domain-general learning mechanisms which resemble those supporting more basic aspects of cognition such as perception? What does it mean to understand a sentence? Does the process of sentence comprehension consist of retrieving word meanings from memory and subsequently assigning them to roles or slots in a compositional representation of sentence meaning as often assumed? The proposed research program addresses these issues based on the following guiding principles. First, to understand human language comprehension, we need to take the evidence provided by neuroscientific data - such as event-related brain potentials (ERPs) providing direct online indicators of electrical brain activity during comprehension - serious, even at the cost of trading long held ideas about how language comprehension should work in principle. Second, it is crucial to precisely understand what these brain signals mean in terms of underlying processes. A principled way to understand a process is to rebuild it, and in that sense, it seems essential to understand language-related brain signals by simulating them with computationally explicit and theoretically precise implemented models to get a deep and thorough understanding of their functional and mechanistic basis, and thereby the functional and mechanistic basis of the underlying language comprehension process. Lastly, it seems of key importance to find ways to link language related brain signals and computational models as closely and systematically as possible to test and adjust or corroborate the ideas implemented in the models. The proposed research program focuses primarily on the N400 component of the ERP which is the most widely used ERP component in research on language and meaning processing. The N400 has been used as a dependent variable in more than a thousand empirical studies, yet remains difficult to capture within traditional approaches to language comprehension. A computationally explicit and precise understanding of the cognitive process underlying this component will take us a long way towards understanding human language comprehension. This is the primary goal of the proposed research program.

• DFG Programme: Independent Junior Research Groups
• International Connection: USA
• Cooperation Partner: Professor Dr. James McClelland