Action-based updating of visual short-term memory

• Applicants: Dr. Sven Ohl; Professor Dr. Martin Rolfs
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: from 2015 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 283703834

The versatility of human behavior depends on coordinated, goal-directed movements. They are the basis for our interaction with dynamic environments, and the mechanisms that control them are entwined with perceptual and cognitive functions. While the influences of actions on visual perception have been studied amply, the influences of action on visual memory are not understood. The proposed research program examines the influence of goal-directed actions on the transfer of visual information into short-term memory and its maintenance therein. A main focus will be on saccades as a model system of goal-directed movements. Preliminary data demonstrates that saccade planning and execution have a strong influence on visual information held in memory, perhaps because of the general necessity to store visual information across retinal image shifts every time the eyes move. In four subprojects, involving a total of twelve experiments, we strive to meet four main objectives. First, we will delineate the conditions under which saccadic eye movements influence the transfer and maintenance of content in visual memory. Do these influences occur in a sensitive period, do they depend on memory load, and what is the representational format on which they operate? Second, we will study the spatial specificity of the saccadic influence on memory maintenance. Are priorities in memory performance restricted to the saccade target or do they spread to locations nearby? Do visual memory and goal-directed actions work on similar spatial scales? Third, we will assess the relation of saccadic influences on memory maintenance to similar effects previously observed for covert attention shifts. Specifically, do the mechanisms underlying these two types of influences overlap or are they independent? Is the saccadic influence on visual memory obligatory? Fourth, we will evaluate whether saccadic influences on visual memory generalize to other types of goal directed movements, such as manual reaches. We will determine the temporal profile and conditions of memory demand under which these movements influence the stabilization and maintenance of visual memory, and find out whether reaches and saccades affect these processes independently. By investigating whether and how actions affect visual information maintained in memory, the proposed project will uncover how movements shape aspects of cognition beyond the encoding of incoming sensory signals. This is an important goal, because actions may provide a rapid and crucial mechanism to prioritize behaviorally relevant memories in a capacity-limited system. In turn, prioritized visual memories might play a key role in maintaining perceptual continuity and supporting sequences of actions across saccadic eye movements. Finally, current models of visual memory need to incorporate the impact of action on memory performance to provide a meaningful account of memory in the acting observer.

• DFG Programme: Research Grants