Evaluating auditory cognition in classroom environments across age groups from preschool children to adults using audiovisual virtual reality - EArAge-VR

• Applicants: Professorin Dr.-Ing. Janina Fels; Professorin Dr. Maria Klatte; Professor Dr.-Ing. Alexander Raake
• Subject Area: Acoustics ; General, Cognitive and Mathematical Psychology ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 444697733

In the renewal proposal EArAge-VR, we will investigate audiovisual immersive virtual environments for the evaluation of auditory cognition in classroom scenarios, gradually increasing the realism of the research methodology. The existing knowledge on the effects of classroom environments on cognitive performance is predominantly based on auditory or, in parts, visual examinations. However, this does not represent real everyday situations. Thus, the demand for combined, close-to real-life audiovisual investigations is high. In EArAge-VR, we aim (1) to create audiovisual classroom scenarios and validate auditory cognition test paradigms and evaluation tools (performance measures and questionnaires concerning Quality of Experience), and make them available for research across laboratories, including less technology-focused ones. In parallel, we aim to (2) get further insight into the processes and mechanisms underlying auditory cognitive performance in complex classroom environments, and (3) investigate a respective developmental change with age. Based on the findings of the EcoClass-VR project from the first phase of AUDICTIVE, we will use a selection of the paradigms created and evaluated. This will enable us to analyze auditory cognition in terms of speech perception, listening comprehension, and listening-related scene analysis in virtual classroom environments in children and adults. The auditory and visual scenes created in the first funding period will be combined and expanded, in a stepwise manner, towards even more realistic, daily-life classroom scenarios. This incorporates dynamic acoustic rendering, audiovisual integration, and increased interaction with the virtual scenes. In addition, in view of the disproportionately strong effects of classroom noise scenarios on listening performance in second-graders in comparison to 3rd-graders and adults found in the first funding period, the sample will be expanded to preschool and first-grade children, and non-native listeners (L2-learners). To be able to test young children in an immersive environment, who cannot wear head-mounted displays, we will create alternative setups using surrounding screens. This allows for a comparison of the developmental changes regarding auditory cognition as well as the collected indirect Quality of Experience indicators across age groups. By revisiting the collected data from both project phases, EcoCLass-VR and EArAge-VR, we will also be able to draw joint conclusions and give recommendations for further research, with the aim for future standardization of the test paradigms found to be most pertinent to identify effects of scene complexity and developmental changes across age groups in auditory cognition.

• DFG Programme: Priority Programmes
• Subproject of: SPP 2236: Auditory Cognition in Interactive Virtual Environments – AUDICTIVE
• Co-Investigator: Professor Dr. Thomas Lachmann