Learning Tonal Representations of Music Signals Using Deep Neural Networks

• Applicant: Professor Dr.-Ing. Christof Weiß
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2020 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 443992185

With the growing impact of technology, musicological research is subject to a fundamental transformation. Digitized data and specialized algorithms enable systematic analyses of large music corpora. Recently, such corpus studies were performed based on audio recordings involving methods from digital signal processing and machine learning. In this context, the tonal analysis of the music signals regarding chords, scales, or keys plays a significant role. Traditional analysis methods rely on signal processing techniques to extract tonal feature representations that indicate the presence of musical pitch classes over time, thus allowing for an explicit semantic interpretation. The objective of this project is to use deep neural networks for learning tonal representations, which are interpretable, robust, and invariant regarding timbre, instrumentation, and acoustic conditions. The project builds on complex scenarios of classical music where time-aligned scores and multiple performances of the pieces can be used for training, validating, and testing the algorithms. From a technical perspective, this project investigates approaches for learning pitch-class, multi-pitch, and salience representations. Among others, sequence learning techniques that can handle weakly-aligned annotations and U-net architectures that are inspired by hierarchical musical structures will be explored. Applying the learned representations to complex music scenarios aims for developing robust tonal analysis methods by exploiting the potential of novel deep-learning algorithms, thus paving the way towards a new level of computational music research.

• DFG Programme: Research Fellowships
• International Connection: France
• Host: Professor Geoffroy Peeters