Individualized feedback in computer-assisted spoken language learning

• Applicant: Professor Dr. Bernd Möbius
• Subject Area: General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
• Term: from 2013 to 2016
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 230875573

The overall aim of the project is to gain an order of magnitude in the usability and efficiency of computer-assisted language learning software by adapting content, feedback and exercises to individual learners in the speech dimension of a foreign language. This will be achieved by integrating the production and perception of the learner's own speech and focusing on the French-German language pair, in both directions.Difficulties experienced by learners depend on the phonology of their native language and are thus non-symmetric. A particular strength of the project is to involve speech scientists who have worked extensively on the phonetics and phonology of the two languages, and especially in cross-linguistic contexts.Providing learners with feedback is a central issue of foreign language learning. The first objective is to provide the learner with automated feedback which derives from an analysis of the learner's utterance and targets specifically the acoustic features to be improved. The second objective is to offer feedback that relies on phonetic knowledge incorporated in the learning system and interacts relevantly with automatic speech recognition (ASR) and signal processing. This means that confidence in the results returned by ASR will be taken into account to derive diagnoses and feedback: the higher the confidence the more explicit the level of feedback.The third objective is to reach a high level of individualization of the user interface, type of exercises, nature and level of feedback (textual, acoustic, visual) and to develop a learner profile, taking into account the phonetic difficulties experienced by the individual learner on the levels of production and perception. A key aspect consists of using the learner's own voice.During a training session, the learner will be exposed to modified versions of his/her own production. This will enable the learner to measure the learning progress by a comparison of his/her own realizations with the feedback proposed by the system. This objective requires a precise definition of the phonetic difficulties predicted for the L1-L2 pair by speech scientists of both languages. These difficulties concern the segmental level (e.g., stop aspiration, final devoicing, vowel quantity) and the suprasegmental level (e.g., lexical stress, contrastive accent). The target features will be selected by considering the phonological and phonetic systems of both languages and ranked by observed recurring errors made by learners, to ensure a relevant learning progress. This procedure will be driven by a careful analysis of a corpus comprising French and German L2 speech data recorded by learners at different levels of proficiency. This corpus will be made available to the scientific community.The project will rely on a tight interaction between phonetics/speech science and signal processing/speech recognition, which is both necessary and expected to be efficient for the project proposed here.

• DFG Programme: Research Grants
• International Connection: France
• Participating Person: Dr. Yves Laprie