Non-smooth Bi-level Optimization for Computer Vision and Machine Learning

• Applicant: Professor Dr. Peter Ochs
• Subject Area: Mathematics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 442790210

Most of the models for solving practical problems in Computer Vision, Machine Learning and related Natural Sciences depend on the choice of parameters. Often, the automatic data based estimation of parameters is a tremendous challenge. This holds, in particular, for Computer Vision and Machine Learning applications with their distinct characteristics of high dimensional data, high dimensional parameter spaces and the fact that problems are naturally modeled with non-smooth functions. Formally, the parameter optimization problem belongs to the class of bi-level optimization problems. Such problems are difficult to solve and require numerical solutions even for low dimensional problems.The goal of this project is the development of a theoretical and practical framework for efficiently solving bi-level optimization problems with the characteristics pointed out above. This will improve the solutions in practical problems, allow us to solve new problems, and will provide theoretical convergence guarantees. Moreover, we expect theoretical insights into heuristic solution strategies in related areas, for example, “backpropagation strategy” (chain rule) for non-differentiable ReLU activation functions in the training of neural networks.

• DFG Programme: Research Grants