Transfer and Representation Learning for Physical Simulations

• Applicant: Professor Dr. Nils Thuerey
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 413611294

This research project targets the analysis and development of new algorithms for learning with simulated and real data in the context physical systems. Our goal is to develop methods for the analysis of real objects with the help of physical simulations, and as starting points, we will focus on two specific classes of objects and materials: fluids and cloth, as representatives of Eulerian and Lagrangian physics models, respectively. While physics solvers are typically very difficult to invert with traditional numerical methods, trained models based on deep neural networks are very efficient to evaluate, and readily provide gradients for optimization problems. Hence, they are particularly amenable to inverse problems, which are a particular focus area of this proposal. More specifically, we will inherent address shortcomings of current algorithms for physical transfer learning applications for volumetric flow effects. We will target learning an efficient reduced representation with the help of constrained neural networks, and the latent space learned with this approach will be employed for transfer learning by targeting a mapping from visual data into the latent space, and the prediction of future states of the system. We will additionally target Lagrangian descriptions, which are particularly suitable for interfaces, such as those of liquids and materials with clear surface geometries, such as fabrics. We plan to learn a representation of the physical dynamics, in order to provide efficient means for simulations with a pre-trained model, and inverse problems in the form of video analysis. As a third part of the project, we will target a broader scope of physical learning problems. Computing reduced PDE-based descriptions for complex data sets is a highly challenging problem, and we will address system identification problems within this context to arrive at improved descriptions of physical behavior observed in real-world measurements. This work is part of the research unit proposal “Learning and Simulation in Visual Computing”, and establishes many areas for synergies between the different directions of work within the unit. Among others, the learned physical models will provide powerful priors for visual analysis in computer vision related directions, and we will leverage the powerful means for differentiable visualization from the other proposals in order to improve the accuracy of inferred solutions for predicting physical dynamics.

• DFG Programme: Research Units
• Subproject of: FOR 2987: Learning and Simulation in Visual Computing