Project Details
Projekt
Efficient Reconstruction and Rendering of Dynamic, Wide-Range Lightfields
Subject Area
Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Measurement Systems
Measurement Systems
Term
from 2014 to 2020
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 255647877
Image based rendering (IBR) is an important research theme in both computer vision and graphics. A scene is rendered from a set of previously captures images as samples of the scene light field. Depending on the approach, not only color images but also depth images may guide the novel view rendering. Previously, IBR research has focused mainly on static scenes or used expensive 2D camera fields for capture, or have applied constraints regarding the observed scenes, like models of human characters, which require expensive global optimization. The requested research investigates the large-scale acquisition, the dynamic representation, and the efficient rendering of time-varying light fields of scenes with consideration of dynamic scene geometry, reflectance and illumination conditions. Since real scenes exhibit a large degree of coherence in space and time, we will investigate to capture not the complete dense light field, but to acquire sparse space-time sections of the light field. The space-time light field is captured by a horizontal multi-camera rig, consisting of 25 cameras with total width of 2.5m, acquiring 30 fps per camera. The rig is motor- controlled and shiftable over a 2D range of 2.5m vertical and horizontal, hence capable of capturing 1D-slices of the light field over time. Simultaneously, a scene point is viewed by 25 angular samples which allows to estimate reflectance and illumination properties. In addition, several depth cameras (Kinect/ToF) capture scene depth. The central assumption of this proposal is that sparse sampling of the light field in combination with depth-compensated multi-view interpolation allows for high-quality reconstruction of the dense light field. This reconstruction enables a series of important visual applications, like content-generation for free-viewpoint television and auto-stereoscopic parallax displays. To reach the research goals, we have centered the proposal around two central approaches. Firstly we will investigate local cost functions for efficient and robust optimisation to accumulate the dense dynamic light field from a sparse data acquisition, by evaluating scene geometry and motion under consideration of reflectivity and illumination. Secondly, we will employ a suitable hybrid representation of geometry and imagery for reconstructing and storing the dynamic dense light field. All approaches are tuned towards high computational efficiency in order to capture, store, and render novel views in minimal time.
DFG Programme
Research Grants
