Immersive Binaural Communication | Models und Algorithms

• Applicant: Professor Dr.-Ing. Peter Jax
• Subject Area: Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 509806277

The binaural communication uses principles of head-oriented sound recording, e.g. via an artificial head, and of the synthesis of head-related signals. Via binaural technologies, real acoustic scenes can be reproduced authentically, or virtual acoustic situations can be auralized. Through accurate reproduction and playback at the receiver, the user gets a realistic spatial listening impression and can benefit from natural cognitive listening capabilities in an optimal way. In contrast to the single-channel playback of classical telephony, the realistic playback of complex binaural scenes delivers information on positions of persons in the room and on the spatiality of acoustic surroundings. The listener is given the impression of being in the middle of the acoustic scene (immersion). The cocktail party effect leads to a significant improvement in the perception of the scene.Preliminary investigations have revealed two fundamental challenges. Firstly, all participants on the far end must use headphones in order to get the desired spatial listening impression. Carrying headphones, however, is often considered as unpleasant and it considerably impairs the communication among each other. Secondly, head movements are not considered, e.g. turning the head takes the full acoustic scene along – an irritating effect that does not correspond to expectations in a natural environment.Accordingly, the main targets for this project on immersive binaural communication are the- binaural reproduction without headphones and without conventional head tracker, and the- spatially stable playback of the binaural signal in case of head movements of the listener despite the fixed position of the artificial head microphone system.For the control of the loudspeaker playback with adaptive crosstalk cancellation (CTC), two microphones shall be positioned at the ears in order to facilitate fast system identification of transfer paths. In parallel, the microphone signals in conjunction with information from the adaptive CTC system are used for acoustic head tracking.In order to obtain a spatially stable playback, the binaural signal, traditionally regarded as a stiff signal configuration, must be counter-rotated in case of a head rotation of the listener. This novel time- and frequency-selective signal modification is a particular challenge in case of several simultaneously active sound sources within the recorded audio scene.The novel scientific methods developed in this project shall enable a better immersion of the listener in the binaural acoustic scene, as well as interaction at the receiver. This will allow for significantly more natural and intuitive audio communication. Furthermore, the expected research results have considerable relevance for future spatial audiovisual formats and applications.

• DFG Programme: Research Grants