Activity Map Completion with Dynamic Objects (P1)

• Applicant: Professor Dr. Jürgen Gall
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2017
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 313421352

Activity maps describe in a scene where activities can occur and strongly depend on the context of a scene. For instance, the activity washing dishes is an activity we expect close to a sink in a kitchen but it is less likely to happen at the sink in the bathroom. Such knowledge about the scene is essential when we want to predict the trajectory of a person if we know the intention of the person. But it also helps to predict the intention, if a person approaches an area where certain activities have been previously observed. The activities, however, can vary depending of the time of day. For instance, it is more likely that a person makes coffee in the morning than in the evening. Our goal is therefore to learn time-dependent activity maps that change over time. Furthermore, we relax the assumption that the complete 3d geometry is given. While this assumption is reasonable for indoor scenarios where the rooms can be completely reconstructed in advance, it does not apply if the robot needs to operate in a new environment or the environment changes over time. Our goal is therefore to anticipate in addition to the activity map the 3d geometry that has not been observed. This includes the geometry that is occluded, for instance the occluded handle of a mug, but also the completion of sparse depth measurements for objects that are distant and geometry that has not been observed, but will become visible while the robot or vehicle is moving. This allows a robot to anticipate the geometry around the corner or of a room before the robot enters it in order to plan the motion accordingly. In addition, we also want to anticipate the geometry together with activity maps. Since moving objects like humans, bicyclists, or cars change their positions, we also need to anticipate where the objects will be at a given time point. We therefore aim to anticipate the semantic scene geometry of the static part of the scene as well as the motion of the moving objects in the scene.

• DFG Programme: Research Units
• Subproject of: FOR 2535: Anticipating Human Behavior