Recurrent neural network models for exploration in dynamic environments.

• Applicant: Professor Dr. Jan Peters
• Subject Area: Human Cognitive and Systems Neuroscience ; Experimental and Theoretical Network Neuroscience
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 496990750

Recurrent neural networks models have received increasing interest in cognitive and systems neuroscience. These models have recently been successfully trained on tasks from the human and animal neuroscience literature, and might yield insights into potential computational mechanisms underlying higher cognitive functions. Here we will use recurrent neural network models to shed light on a fundamental problem in reinforcement learning – the exploration/exploitation trade-off. Agents are regularly faced with the problem of deciding whether to select well-known options for reward maximization, or whether to explore novel options for information gain. Human exploration is supported by at least two strategies – choice randomization (random exploration) and directed exploration of uncertain options for information gain. For the first objective, we will use computational modeling of recurrent neural network behavior to test the prediction that these network models exhibit computational strategies similar to those observed in humans to solve the exploration/exploitation trade-off. For the second objective, the computations and representations underlying network performance will be examined via a detailed analysis of the dynamics embedded in their hidden unit time courses. Together, the project will shed light on computational mechanisms that might support learning and decision-making in dynamic environments.

• DFG Programme: Research Grants