Auditory-motor control analysis of voice production in hearing impaired speakers by means of Machine Learning

• Applicants: Professor Dr.-Ing. Michael Döllinger; Professor Dr.-Ing. Ulrich Hoppe
• Subject Area: Otolaryngology, Phoniatrics and Audiology ; Acoustics
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 511281818

Speech production is a highly complex process involving the coordination of the respiratory, laryngeal, and oral motor systems as well as a large network of brain regions being involved in motor, somatosensory, and auditory tasks. The auditory feedback plays an important role in tuning the speech motor control (SMC) system. It is well known that auditory deprivation, due to hearing loss, may result in significant deteriorations of articulatory processes. However, little is known about the involved neuro-feedback networks and the underlying mechanisms are not fully understood yet.The central objective in this study on Speech Motor Control mechanisms in hearing impaired patients is the identification of the impact of disturbed auditory input on audio-kinesthetic processes. By applying and analyzing multi-sensor based data including laryngeal high-speed imaging, electroencephalography (EEG), and the acoustic voice signal the project aims to delineate the interaction between perception and motoric. The Pitch-Shift-Reflex (PSR) known as pitch changes in response to modified auditory feedback will be used as a paradigm for investigating SMC related processes.Data for both hearing impaired patients and normal hearing subjects will be recorded. Analysis of the multi-sensor based data will be performed using machine learning techniques to identify physiological conclusive features that reflect SMC processes. For differentiating between groups, parameter-driven and data-driven machine learning approaches will be investigated. This yields clinical relevant parameters which represent SMC deterioration. The major goal of the study will be pursued by correlating these parameters with patient specific audiological characteristics as degree, duration and frequency range of hearing loss, age of patients, frequency difference limen, and speech perception. By comparing PSR changes with these audiological characteristics deeper insight in kind and extent of SMC deterioration due to auditory decline is expected. Innovative scientific aspects of the project are (1) the use of high-speed-video endoscopy allowing for direct observation of the laryngeal dynamics during the PSR. (2) Machine learning approaches will be applied to reveal differences in underlying SMC parameters in high-speed videos, EEG, and acoustic data between normal hearing subjects and hearing impaired. (3) Multi-regression analysis for identified SMC parameters and patient specific audiological characteristics.

• DFG Programme: Research Grants