End-to-End Optimization for Energy-Driven Scientific Visualization

• Applicant: Professor Dr.-Ing. Tobias Günther
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 517157369

Scientific visualization is a research discipline that supports scientists in science, technology, and engineering in the analysis and visualization of large and complex spatial data sets, for example in meteorology, geophysics, or in fluid dynamics. The creation of scientific visualizations is modeled by the so-called visualization pipeline, which consists of a sequence of data transformations (comprising pre-processing and feature extraction), a visual mapping (which visually encodes data attributes using color and other visual channels), and a rendering process that synthesizes the final image. In recent years, more and more optimization methods have been researched that describe the feature extraction and the visual mapping as energy minimization problems, where the optimal parameters are searched that best fulfill a desirable objective function. This includes the adjustment of transparency, the selection of colors, the optimal placement of particle trajectories, as well as the selection of a suitable reference frame prior to a conventional feature extraction. So far, optimizations have only been performed in individual pipeline steps. With this proposal, we formulate energy minimizations across the entire visualization pipeline, enabling the concurrent optimization of geometry and visibility to improve quality metrics measured on the final image. In this research project, we investigate the opportunities and limitations of such non-linear parameter optimizations across the full visualization pipeline to lay the foundations for the next generation of energy-driven visualization systems. The research project is divided into three steps. First, we research novel non-linear quality metrics that judge multi-object visibility in direct volume rendering and build a visual analysis tool to explore the convergence behavior of the non-linear solvers. Second, we extend the optimization across the full visualization pipeline by optimizing the placement of line geometry at the concurrent optimization of its visibility. And third, we integrate additional user constraints into the optimization and assist in the navigation and exploration of time-dependent data. With the end-to-end optimization across the entire visualization pipeline, numerous existing optimization problems can be revisited in the future in conjunction as part of a comprehensive energy-driven visualization pipeline.

• DFG Programme: Research Grants