SFB 1233:  Robust Vision - Inference Principles and Neural Mechanisms

• Subject Area: Medicine ; Computer Science, Systems and Electrical Engineering ; Social and Behavioural Sciences
• Term: since 2017
• Website: Homepage
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 276693517

The Collaborative Research Centre (CRC) "Robust Vision" brings together leading researchers in machine learning, computer vision, and systems neuroscience to uncover the computational principles underlying robust visual processing in biological and artificial systems. The past decade has seen unprecedented advances in computer vision and machine learning, with the emergence of systems that can flexibly solve highly sophisticated tasks, such as object segmentation in open world conditions. In parallel, neuroscience has been rapidly advancing towards studying distributed neural computations underlying natural behavior, and the field of NeuroAI has used machine learning methods to advance our understanding of brain function. The CRC has been at the forefront of these exciting developments. The projects productively collaborated in leveraging the fast paced progress in machine vision for studying the neural basis of robust vision and task flexibility in the brain, and to reveal commonalities and differences between artificial and biological vision systems. CRC members have extensively collaborated and produced more than 230 CRC-funded publications, which have already been cited more than 8,500 times. Building on the extensive collaborations and scientific successes in the first two funding periods, we will now focus on integrative approaches to unravel the principles and mechanisms enabling agent-centric robust vision. Biological vision remains unparalleled in its efficiency: Animals can robustly make visually-guided decisions with limited computational resources and limited access to data. To understand this agent-centric robustness, we will advance modelling- and analysis-approaches for characterizing biological vision systems, and compare intelligent visual behavior and neural representations in brains and machines. We will work with select animal models and build, based on experimental data, computational models of biological vision at multiple levels of detail, ranging from "digital twin’" models of early visual processing, to active sampling strategies for gaze control, and to synthetic agents in virtual environments. The CRC is organized into four research themes that collaboratively target central aspects of agent-centric robust vision: Object-centric vision (Theme A), High- level neural representations (B), Active visual inference ©, and Early information selection (D). All projects will develop methods and concepts for evaluating representations in biological and artificial networks to investigate which inductive biases of brains and machines enable efficient visual performance. Supported by a cross-sectional project, we will build open model and evaluation platforms that support collaborative usage by the entire research community to generate a long-lasting benefit of the CRC beyond ist funding period. Together with the development and employment of AI vision systems, this will make the CRC a unique research center in Germany.

• DFG Programme: Collaborative Research Centres

Current projects

• A01 - Robust material inference (Project Heads Gehler, Peter ;  Geiger, Andreas ;  Koepke, Almut Sophia ;  Lensch, Hendrik ;  Schölkopf, Bernhard ;  Zhang, Dan )
• A02 - Evaluation of autonomous vision for agent-centric vision (Project Heads Akata, Zeynep ;  Brendel, Wieland )
• B01 - High-level visual and multi-modal representations in the human brain and ANNs (Project Heads Liebe, Ph.D., Stefanie ;  Macke, Jakob ;  Oganian, Yulia )
• B02 - Large-scale neuronal interactions during natural vision (Project Heads Bartels, Ph.D., Andreas ;  Siegel, Markus )
• B03 - Probing and modelling object-centric dynamic scene processing in the human brain (Project Heads Bartels, Ph.D., Andreas ;  Bethge, Matthias ;  Black, Ph.D., Michael )
• C01 - Prediction & model-building in uncertain environments (Project Heads Berens, Philipp ;  Dayan, Peter ;  Franz, Volker ;  von Luxburg, Ulrike )
• C02 - Early visual processing in the presence of eye movements - Benchmarking of active early vision (Project Heads Franke, Katrin ;  Hafed, Ph.D., Ziad ;  Kümmerer, Matthias ;  Schaeffel, Frank ;  Schwarz, Ph.D., Christina ;  Wichmann, Felix A. )
• D01 - Exciting stimuli for mice and how they are encoded by the early visual system (Project Heads Busse, Laura ;  Euler, Thomas ;  Franke, Katrin ;  Schaeffel, Frank )
• D02 - Information selection in mouse dLGN and SC across internal and external contexts (Project Heads Bethge, Matthias ;  Euler, Thomas ;  Macke, Jakob )
• D03 - Software, methods and computational tools to evaluate computational models of vision (Project Heads Berens, Philipp ;  Busse, Laura ;  Franke, Katrin ;  Sinz, Fabian )
• INF - Software, methods and computational tools to evaluate computational models of vision (Project Heads Berens, Philipp ;  Bethge, Matthias ;  Eggensperger, Katharina ;  Macke, Jakob ;  Sinz, Fabian ;  Wichmann, Felix A. )
• Zehem15 - Administration (Project Head Bethge, Matthias )

Completed projects

• 01 - Physics-based scene understanding (Project Heads Gehler, Peter ;  Lensch, Hendrik )
• 03 - Comparing humans and machines on robust visual inference (Project Heads Bethge, Matthias ;  Wallis, Ph.D., Thomas )
• 04 - Causal inference strategies in human vision (Project Heads Bethge, Matthias ;  Schölkopf, Bernhard ;  Wichmann, Felix A. )
• 06 - Top-down control of visual inference in sensory representations in early visual cortex (Project Heads Macke, Jakob ;  Nienborg, Hendrikje ;  Sinz, Fabian ;  Wichmann, Felix A. )
• 08 - Integration of bottom-up and top-down processing in sleep-dependent (Project Heads Nienborg, Hendrikje ;  Rauss, Karsten )
• 14 - Retinal Disease Models as a Tool for Understanding Robust Vision (Project Heads Macke, Jakob ;  Schwarz, Ph.D., Christina ;  Stingl, Katarina ;  Zeck, Günther ;  Zrenner, Eberhart )
• 17 - Learning explainable policies for self-driving cars from little data (Project Heads Akata, Zeynep ;  Geiger, Andreas )
• TRAT01 - Physiologically inspired robust electro-optical autofocals (Project Head Wahl, Siegfried )

• Applicant Institution: Eberhard Karls Universität Tübingen
• Participating University: Georg-August-Universität Göttingen; Ludwig-Maximilians-Universität München; Technische Universität München (TUM)
• Participating Institution: Max-Planck-Institut für biologische Kybernetik; Abteilung Computational Neuroscience; Max-Planck-Institut für Intelligente Systeme; Max-Planck-Institut für Intelligente Systeme (MPI); Standort Tübingen
• Spokesperson: Professor Dr. Matthias Bethge