Automatic spatiotemporal alignment of large-scale 3D+t point clouds

• Applicant: Professor Dr.-Ing. Johannes Stegmaier
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Developmental Biology
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 432051322

Multidimensional fluorescence microscopy has become a common technique in biology labs all over the world as it represents an invaluable tool to study early embryonic development in space and time (3D+t) at cellular resolution. Automatic segmentation and tracking algorithms are used to extract thousands of cell movement trajectories from potentially terabyte-scale 3D+t image data sets that offer the possibility for a detailed analysis of inter-individual differences. A fundamental problem that remains after having obtained such tracked point clouds, however, is the comparison of individual experiments to confirm biological hypotheses in multiple repeats. The lack of fully automated solutions to this 3D+t alignment problem currently limits whole-embryo analyses to simple specimens, early time points or manual analyses.The aim of the proposed project is the development of new methods for automated spatiotemporal alignment of large 3D+t point clouds. As complex organisms usually lack one-to-one cell correspondences that could be used for registration, a fundamental part of the project will be the development of generic descriptors to identify various anatomical regions at different developmental stages using both classical and machine learning-based approaches. We plan to generate synthetic training data for the machine learning approaches using a comprehensive simulation platform that allows mimicking embryonic development of different specimens at multiple levels of detail. We explicitly want to use memory-efficient point cloud representations to be able to align even data sets that would cover multiple terabytes using a dense data structure like images. The identified anatomical landmarks will enable precise alignment of 3D+t data sets and we will develop a modular similarity metric that can combine information from a heterogeneous set of such spatiotemporal descriptors. We envision a coarse-to-fine strategy, which first temporally aligns the data sets using methods like dynamic time warping, then continues with a rigid spatial registration of the individual frames and finally uses adapted elastic registration techniques to warp the point cloud surfaces onto each other to form a 3D+t atlas of embryonic development. The results will be general methods for spatiotemporal alignment of 3D+t point cloud data sets, open-source implementations and the application to large-scale light-sheet microscopy experiments of zebrafish and fruit fly embryos. Combined with existing automated segmentation and tracking methods, the aspired alignment framework will represent a large step further towards the ultimate goal of analyzing large-scale imaging experiments in a fully automated fashion, which is essential to keep pace with the continuously advancing 3D+t fluorescence microscopy techniques.

• DFG Programme: Research Grants