Efficient Non-alternating Optimization for Matrix and Tensor Factorization

• Applicant: Professor Dr. Peter Ochs
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 438478660

Structured Matrix Factorization has a vast number of applications in many disciplines such as, for example, Machine Learning, Computer Vision, Signal Processing, Information Retrieval, Language Processing. The corresponding optimization problem is usually approached by alternating minimization techniques, which have several drawbacks: (i) often the solution is biased towards one of the optimization variables, (ii) they have poor convergence guarantees (e.g., if non-smooth regularization such as sparsity penalty terms are used), (iii) they cannot be used for symmetric Matrix Factorization, and (iv) they are hard to accelerate or adapt to stochastic optimization setting. Therefore, in this project, we develop a direct (non-alternating) optimization approach for Matrix Factorization, which overcomes these limitations. Key is an adaptation of the geometry of the algorithm to that of the optimization problem. This so-called concept of relative smoothness comes with several favorable properties for Matrix Factorization. While the first approaches already show a promising performance, in this project, we seek to establish state-of-the-art algorithms for the class of matrix and tensor factorization problems and beyond with strong theoretical convergence guarantees.

• DFG Programme: Research Grants