Deep Shape Representation for Shape Analysis, Modeling, and Reconstruction

• Applicant: Professor Dr. Leif Kobbelt
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 449823330

Digital 3D models are essential in a wide spectrum of diverse applications ranging from industrial design, digital media and entertainment to virtual reality, and 3D printing. For traditional 3D model generation, users have to invest a lot of time on professional CAD software and/or need to acquire expensive equipment such as laser scanners. With the increasing availability of large 3D model repositories on the internet in recent years, a paradigm shift becomes feasible from geometric design to data driven approaches where an intelligent modeling system supports the user by leveraging (statistical) knowledge that has been derived from pre-existing designs. Recent advances in deep learning are quite promising and nourish the hope that similar breakthroughs can be expected for geometric tasks. However, 3D models are very different from 2D images or videos in a number of aspects. 3D models (volumetric or B-rep) can have a complex topology and structure, details and features across several orders of magnitude, and auxiliary attributes like textures associated. Established geometry representations for deep learning applications do not support all of these aspects simultaneously. Therefore, ICT-CAS and RWTH intend to thoroughly address this issue in an international collaboration. Specifically:(1) We will investigate a novel representation of 3D geometry that combines hierarchical composition (structure) with geometry deformation (shape) and attribute mapping (appearance). The representation will be suitable for efficient and effective processing with deep neural networks.(2) For this representation we will develop a number of fundamental low-level operations like segmentation and classification, shape abstraction and matching as well as symmetry analysis.(3) The fundamental operations will allow us to solve challenging high level tasks like single view reconstruction and sophisticated (interactive) data driven 3D modeling tools which support the user by interpreting the user's intention and creating plausible 3D models in a data driven manner on the basis of (statistical) knowledge derived from large repositories of objects.

• DFG Programme: Research Grants
• International Connection: China
• Partner Organisation: National Natural Science Foundation of China
• Cooperation Partner: Professor Lin Gao, Ph.D.