Automated Methods and Tools for Analyzing and Structuring Choral Music
Final Report Abstract
The increasing digitization results in extensive music collections, which, in addition to audio and video recordings, also contain symbolically or graphically encoded sheet music. One such example is the multimedia music catalog offered by the Carus publishing house, a leading international music publisher for religious and secular choral music. In addition to its extensive catalog of carefully edited sheet music comprising more than 30,000 choir pieces, Carus also produces reference recordings and video material for teaching and practice purposes. Generally speaking, the main goal of Music Information Retrieval (MIR) is to develop efficient search and navigation systems that allow users to analyze and search through complex music collections concerning different music-related aspects. In the DFG-funded transfer project “Automated Methods and Tools for Analyzing and Structuring Choral Music” (AnChor), we adapted and improved MIR methods while testing them in practice. As a first central contribution, we developed web-based user interfaces for interactive and synchronous access to different music representations and analysis results. Not only are these interfaces of interest for possible commercial applications, but they also serve to communicate research results across different disciplines. One of the AnChor project’s important research activities was to develop and implement a content-based retrieval system, where a user can make a search request in the form of a short audio excerpt or a YouTube link (e.g., a choir recording sung by an amateur choir). The goal is then to automatically identify, based purely on the acoustic waveform, all other performances or versions of the piece of music in a given collection (e.g., a high-quality recording by a professional choir in the Carus catalog). For this task, we could achieve considerable improvements in terms of the retrieval system’s runtime and storage requirements by combining embedding techniques based on neural networks with graph-based indexing techniques. Another research focus was on automated methods for measuring, analyzing, and adjusting intonation fluctuations in unaccompanied, polyphonic vocal music. Finally, based on the experience gained with our project partner, we developed music datasets along with annotations, which we made freely available for research purposes. These datasets are suitable for developing and evaluating algorithmic approaches for various MIR tasks, including music transcription, music synchronization, chord recognition, fundamental frequency estimation, and intonation analysis. Through the collaboration with the Carus publishing house, it was also possible to establish cross-connections to music education and musicology and initiate interdisciplinary collaborations.
Publications
- (2018). A web-based interface for score following and track switching in choral music. In Demos and Late Breaking News of the International Society for Music Information Retrieval Conference (ISMIR), Paris, France
Zalkow, F., Rosenzweig, S., Graulich, J., Dietz, L., Lemnaouar, E. M., and Müller, M.
- (2018). Bridging the Gap: Enriching YouTube videos with jazz music annotations. Frontiers in Digital Humanities, 5:1–11
Balke, S., Dittmar, C., Abeßer, J., Frieler, K., Pfleiderer, M., and Müller, M.
(See online at https://doi.org/10.3389/fdigh.2018.00001) - (2019). Tools for semiautomatic bounding box annotation of musical measures in sheet music. In Demos and Late Breaking News of the International Society for Music Information Retrieval Conference (ISMIR), Delft, The Netherlands
Zalkow, F., Corrales, A. V., Tsai, T., Arifi-Müller, V., and Müller, M.
- (2019). Towards measuring intonation quality of choir recordings: A case study on Bruckner’s Locus Iste. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 276–283, Delft, The Netherlands
Weiß, C., Schlecht, S. J., Rosenzweig, S., and Müller, M.
- (2020). Dagstuhl ChoirSet: A multitrack dataset for MIR research on choral singing. Transactions of the International Society for Music Information Retrieval (TISMIR), 3(1):98–110
Rosenzweig, S., Cuesta, H., Weiß, C., Scherbaum, F., Gómez, E., and Müller, M.
(See online at https://doi.org/10.5334/tismir.48) - (2020). Learning low-dimensional embeddings of audio shingles for crossversion retrieval of classical music. Applied Sciences, 10(1)
Zalkow, F. and Müller, M.
(See online at https://doi.org/10.3390/app10010019) - (2020). TuneIn: A web-based interface for practicing choral parts. In Demos and Late Breaking News of the International Society for Music Information Retrieval Conference (ISMIR), Montreal, Canada
Rosenzweig, S., Dietz, L., Graulich, J., and Müller, M.
- (2021). Efficient retrieval of music recordings using graph-based index structures. Signals, 2(2):336–352
Zalkow, F. and Müller, M.
(See online at https://doi.org/10.3390/signals2020021) - (2021). Schubert Winterreise dataset: A multimodal scenario for music analysis. ACM Journal on Computing and Cultural Heritage (JOCCH), 15(2):25:1–18
Weiß, C., Zalkow, F., Arifi-Müller, V., Müller, M., Koops, H. V., Volk, A., and Grohganz, H.
(See online at https://doi.org/10.1145/3429743)
