Deep learning based computational imaging and optical metrology

• Applicant: Dr.-Ing. Karsten Frenner
• Subject Area: Measurement Systems
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 449502487

This Sino-German joint research project (SG-JRP) aims at exploring deep learning for computational imaging and optical metrology (CIOM). From mathematical point of view, CIOM can be fundamentally categorized as methodologies to solve inverse problems in imaging and metrology, which infer the objects or scenes from imperfect physical measurement. Typical examples in imaging and metrology include ghost imaging, digital holography, light-field imaging, and imaging through random medium. Under the framework of Sino-German Research Cooperation Group (GZ 1391) the research partners in both German and Chinese sides have conducted systematic investigations on aforementioned research topics and produced a series of valuable research outputs in past a few years. Meanwhile they have made a number of personal interactions, including mutual visits of research staff and Group workshops. Through those activities both German and Chinese colleagues have recognized the trend of development in this discipline and identified some critical scientific issues to be further addressed, leading to current joint research proposal (JRP).Recently it has become evident that deep learning (DL) techniques are capable of achieving competitive performance in solving many inverse problems. However, DL has yet to provide a radical improvement over analytical methods for solving inverse problems in CIOM. In this JRP, we select three modalities for imaging and metrology to study by incorporating DL strategies: (1) German group in Stuttgart proposes an approach that will train neural networks (NN) in order to solve the inverse problem of scatterometry; (2) Chinese group in Shenzhen proposes an approach that will incorporate the DL into structured-light-field (SLF) to identify the fringe order for phase reconstruction as well as the aberration correction; (3) the group in Shanghai will focus on the development of novel deep learning methods by incorporating a physical (analytical) model into a neural network for noninvasive real-time imaging through the optically thick and dynamic scattering media.The overall objective of this JRP is to offer some insight as to how deep-learning (DL) strategies can solve the inverse problems in imaging and metrology. Furthermore, we address some fundamental questions, such as how deep learning and analytical methods or physical models can be combined to provide better solutions to the inverse problems in CIOM. Beyond the research goals specified in this proposal, the participating young researchers of both sides will benefit from the implementation of the overall project through the intended interactions and created synergies.

• DFG Programme: Research Grants
• International Connection: China
• Partner Organisation: National Natural Science Foundation of China
• Cooperation Partner: Professor Dr. Xiang Peng