Cognitive Knowledge Structures in Scene Perception - Behavioral and Neural Correlates of Semantic and Syntactic Scene Processing

• Applicant: Professorin Dr. Melissa Le-Hoa Vo
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: from 2014 to 2018
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 242736900

While our visual world is complex, its composition conforms to certain rules. For instance, most objects require a horizontal surface to rest upon and two objects seldom coexist in the very same location. Even beyond such basic knowledge about physical properties of objects, we seem to have detailed knowledge regarding what objects tend to be found where within certain scenes. The aim of this research program is to gain further insight into how cognitive knowledge structures and expectations regarding objects in scenes help us to efficiently guide our attention and action when viewing and interacting within naturalistic scenes.The proposed research has three major objectives: First, the program will be dedicated to identifying and then utilizing the rules that govern the composition of scenes. For this purpose, we will perform large-scale analyses of image statistics to create the “Real-World Lexicon” — an easily accessible database containing information regarding object frequencies and typical locations within and across scenes. In addition, we will create a corpus of highly controlled image material for experimental use. The second aim of the program concerns the development of scene knowledge. Similar to the grammar in language processing that allows us to understand an unlimited number of novel sentences, we seem to possess a “scene grammar” that guides us even in unfamiliar places. By studying both artificial rule learning in adults and the development of scene understanding in infants and toddlers, we hope to find out how such scene knowledge develops over time. The emergence of rule-governed expectations will be tracked by a combination of psychophysics, eye tracking, and the measurement of brain responses. The third objective of the research program concerns the use of scene knowledge during active exploration of real-world, 3D scenarios. Participants will search for and interact with objects, while wearing a mobile, headmounted eye tracker. This setup will allow us to test what sources of guidance become dominant once search takes place beyond 2D displays, and actually involves active body movements in the real world.Moreover, an essential part of the program will include the evaluation, further development, and use of emerging techniques, such as the co-registration of eye movements and brain responses to measure so-called fixation-related potentials (FRPs). Particularly during scene viewing, the online measurement of brain potentials in response to what an observer is currently looking at will allow us to better relate cognitive processes to the actual visual input sampled by each individual observer. The outcome of this program has great potential for many applications within and outside of science ranging from more cognitively inspired technical systems to early diagnosis of developmental disorders in infants and toddlers.

• DFG Programme: Independent Junior Research Groups