Localized Mean-Field Games and Collective Intelligence

• Applicant: Professor Dr. Heinz Koeppl
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Mathematics ; Statistical Physics, Nonlinear Dynamics, Complex Systems, Soft and Fluid Matter, Biological Physics
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 517777863

This project focuses on understanding agent-based models with large populations of agents which are difficult to analyze through traditional methods due to the large number of agents. In particular, recent years have shown the success of automated sequential decision-making via reinforcement learning (RL) and multi-agent reinforcement learning (MARL) with prominent application in various applications ranging from robotics, autonomous cars and stratospheric balloons over teams of unmanned aerial vehicles to strategic games. Meanwhile, large-population systems remain of considerable interest to a number of areas in science and engineering such as epidemics, robotic systems, bacteria colonies, schools of fish or flocks of birds. More generally, these scenarios fall under the area of collective swarm intelligence, a study of decentralized systems where agents perform highly localized interaction and are still capable of achieving global behavior without any single agent being aware of the overall state of the swarm. Such systems will be the key motivation of our project. However, while MARL techniques are empirically effective in particular for few-agent systems, theoretical analysis and practical design of algorithms for many agents remains difficult. A recent solution proposed for this issue of scalability is a combination of RL with the idea of so-called mean-field games (MFGs). Competitive MFGs and cooperative mean-field control formally assume an infinitude of agents, which gives access to modeling in statistical terms, i.e. considering only the distribution of each of infinitely many identical agents. Here, it is often possible to obtain both theoretically rigorous results for large systems and practically tractable algorithms for otherwise intractable problems. While such mean-field frameworks have been successfully applied already in a variety of applications such as power networks, smart heating and more in engineering and finance, the theory of MFGs as well as its intersection with modern deep RL and collective intelligence still remain to be developed. Here, in order for MFGs to find further application in large-population systems, it is necessary to develop a better understanding of MFGs with high locality, both in information structure and in interaction. In other words, agents shall have only partial knowledge of the whole system, and interact only with a subset of agents such as neighbors on a graph. Such systems and algorithmic or learning-based solutions have yet to be explored in full generality. The development of suitable theories and algorithms for aforementioned localized mean-field systems will thus constitute the primary goal of our project.

• DFG Programme: Research Grants