Motion and Depth Estimation From Spatio-temporally Coded Video

• Applicant: Professorin Dr.-Ing. Anita Sellent
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2011 to 2014
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 207123795

Estimating depth and motion from coded aperture video, we often encounter motion blur due to fast motion and slow exposure times. Motion blur destroys high frequency image content that is necessary for reconstruction of a sharp all-in-focus image. But motion blur is also a valuable hint for motion estimation algorithms. In the extension of the project, we therefore propose to encode the motion blur temporally. This prevents the loss of high frequencies while adding a further motion cue to each frame of the video. We obtain thus a spatio-temporally coded video that considers the most frequently encountered image degradations: motion blur and defocus blur. With the coding we can on the one hand exploit the blur to gain information on the scene. On the other hand we can also faithfully remove the degradations.In the evaluation of coded aperture video, we find that not all spatial codes are equally suitable for depth and correspondence estimation. In spatio-temporal coded video, the combination of spatial and temporal codes effects the results additionally. In the project we propose to investigate the effect of spatio-temporal codes analytically. Geometric considerations in multidimensional space allow for quadratic formulations with additional physical feasibility constraints. Solutions for this kind of problem have been characterized in recent optimization literature and can be applied directly to find an optimal spatio-temporal code.

• DFG Programme: Research Fellowships
• International Connection: Switzerland
• Host: Professor Dr. Paolo Favaro