1-Bit 3-Dimensional Imaging

• Applicants: Professor Dr. Bhaskar Choubey; Privatdozent Dr.-Ing. Miguel Heredia Conde
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 514479946

Time-of-Flight (ToF) cameras are 3D imaging systems able to capture the geometry of a scene. Typically, little attention is paid to the quantization scheme and uniform quantization of the measurements with enough number of bits is assumed. In this project we aim to follow up on recent results showing that reliable multiple-path 3D imaging can be obtained from one-bit-quantized data. The first one-bit ToF pixel will be designed, fabricated and evaluated. The ability to obtain reliable 3D images from one-bit data enables a massive reduction of the data to be captured and transmitted by the camera. This reduces the hardware requirements upstream in terms of memory and bandwidth. Furthermore, one-bit ToF imaging deals with a new form of inverse problem, in which the measurements have undergone a non-linear transformation. The reduction of the data flow per pixel also allows for larger ToF pixel arrays, paving the way for Megapixel Tof imaging. The binary nature of the measurements also avoids the need for ADC banks, freeing chip space and further helping on this front. Despite to date all existing ToF cameras rely on uniform quantization, preliminary work of Dr. Heredia Conde has demonstrated the feasibility of the proposed concept by means of realistic simulations. These first results suggest that such a one-bit ToF pixel would exhibit superior performance to alternative sensing schemes relying on different tradeoffs between number of measurements and bit depth for the same bit budget. In this project we will design and construct the first one-bit ToF camera. This hardware novelty will be coupled to an algorithmic counterpart, able to estimate the geometry of the observed scene from the measurements. The result will be an unprecedented time-resolved computational imaging system, in which the information is first encoded into lightweight streams of one-bit data and then decoded into one or several depth images. The necessary tasks for the development of a functional one-bit 3D imaging system can be summarized as follows. First, the architecture of the one-bit pixel will be designed and simulated. Then a single pixel or a small array will be fabricated for testing. The fabricated device will be tested for operability. Building upon the insights gained from this first device, a complete array of ToF pixels will be fabricated and integrated with a modulated NIR illumination system and control circuitry and the system will be tested. The necessary algorithms for reconstructing the 3D geometry of the scene from the obtained one-bit data will be developed. This includes the disentanglement of multiple return paths per pixel. Finally, the overall performance of the system as 3D imaging sensor is to be systematically evaluated and compared to the state of the art and to the results obtained from simulations.

• DFG Programme: Research Grants