Holistic Scene Understanding

• Applicant: Professor Dr.-Ing. Bodo Rosenhahn, since 6/2016
• Subject Area: Geodesy, Photogrammetry, Remote Sensing, Geoinformatics, Cartography ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2014 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 249306183

One of the common goals of photogrammetry and computer vision is to fully understand a scene, so called holistic scene understanding, which is the ability to infer context-aware scene description from images or videos. The goal of the envisaged project is the holistic scene understanding for interpreting 3D image sequences obtained from dynamic scenes. Scene understanding involves many sub-tasks, such as (a) scene labeling, (b) object detection and tracking, object reasoning, and (c) object (human, vehicle, etc.) behavior analysis. Each of these sub-tasks explains some aspect of a particular scene, and in order to fully understand a scene, one would need to solve all these sub-tasks. Several independent efforts have resulted in good performance for some of these sub-tasks. In real scenarios such as dynamic urban scenes, the sub-tasks are naturally coupled, for example, if we know the 3D structure of the scene, we could make a better hypothesis of the location of a vehicle. Therefore, the potential of an integrated system will be explored, which consists of a low-level scene labeling model, a mid-level object reasoning model, and a high-level behavior analysis model for building a context-aware scene description. This integrated model will take advantage of dynamic information as well as static information coming from semantic labels and geometry. The novelty is threefold: (1) The three levels, each having its own appearance, scale, geometric, and semantic model of the scene, are meant to smoothly interact using statistical inference between all levels to arrive at a consistent dynamic scene representation and labeling. (2) Detection and learning of object dynamics as well as their behavior context for context-aware behavior modeling are investigated for image sequences. (3) The build-up of the scene description provides a smart integration of bottom-up and top-down reasoning and allows to incorporate prior knowledge and boost performance.

• DFG Programme: Research Grants
• Ehemaliger Antragsteller: Professor Dr.-Ing. Michael Ying Yang, until 5/2016