Generation of 3D object and environment models with an imaging sonar

• Applicant: Professor Dr. Andreas Birk
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 535678995

When employing underwater robots, 3D object and environment information are useful mission results and they also form an important basis for machine intelligence and agency, i.e. the use of (semi-)autonomous systems. Various optical methods are already very successfully used for the generation of underwater 3D data. However, the necessary visibility conditions are hardly or even not at all given in many environments. For sonar as an alternative, the generation of 3D data is in contrast still a major scientific challenge. In this project, the generation of 3D information from the 2D data of an imaging sonar, also known as an acoustic camera, is explored. In contrast to what the name suggests, the projection function of this type of sonar is very different from that of an optical camera. The data is arranged on one of the two image axes by distance, rather than by angle as in the pinhole camera model. Multiple points that can be placed very differently in the scene can hence be projected onto one point in the image. Thus, in 3D environments or with objects, there is for example considerable distortion compared to what is expected according to human vision. As further drawbacks, these sonar images have quite low resolution and they are heavily affected by noise. The resulting challenges are addressed as follows. First, a reformulation of the well-known optical method of bundle adjustment for the special projective geometry of an imaging sonar is developed. In particular, the ambiguities that arise are represented and taken into account in the optimization. Furthermore, the additional fourth degree of freedom (scale) of spectral registration methods is used as an additional constraint. Second, new methods are explored that allow robust registration with four degrees of freedom (translation, rotation, scale) of very noisy 2D data. In particular, the use of these frequency-based methods as image features, i.e., for finding correspondences between points in sonar images, is also explored. The objective is to achieve a quality in the 3D data that goes far beyond the state of the art in research, thus laying the foundations that will allow our approach to be used in relevant scenarios. All developed methods are therefore also intensively evaluated in real systems in realistic environments.

• DFG Programme: Research Grants