The influence of visual movement on auditory rhythm perception

• Applicant: Dr. Yi-Huang Su
• Subject Area: General, Cognitive and Mathematical Psychology ; Cognitive, Systems and Behavioural Neurobiology
• Term: from 2012 to 2018
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 230798057

The proposed project constitutes a continuation of the ongoing project that deals with multisensory (audiovisual) rhythm perception, in which the visual rhythm is derived from observed biological motion. Rhythm perception is imminently linked to human movements. The ongoing project has established the effect of a visually evoked beat on auditory rhythm perception, using a simple human movement. In the new research, the cross-modal effects will be investigated in more complex rhythmic movements derived from human dance. Coordinated movements amongst different body parts, as in dance, are hypothesized to communicate hierarchical modules of visual rhythm, such as pulse, beat, and meter. These observed movements are expected to interact with auditory rhythms in beat and meter perception. The project comprises three parts. Various perceptual and sensorimotor paradigms will be employed, complemented by an EEG experiment to probe the neural correlates. Part A will establish visual perception of pulse and beat in certain whole-body dance movements. Part B will investigate the cross-modal effects of such visual rhythms on auditory beat and meter perception, and the mechanisms underlying audiovisual rhythm integration. Part C will test embodied rhythm perception in the context of visual action recognition, in which the cross-modal effects will be examined as a function of the movement agency (self vs. other). As a whole, the proposed research aims to elucidate multisensory rhythm perception in an integrated framework of action-perception coupling and embodied music and dance cognition. Results may yield implications for developing multimodal rhythmic stimuli to guide movements or to train other sensorimotor processes, such as speech.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Joachim Hermsdörfer