FOR 1306:  Stratosphere - Information Management above the Clouds

Science, industry and users of the internet produce larger and larger data sets. Due to their sheer size, this data cannot be analysed efficiently on single computers anymore. In this Research Unit, several universities in the Berlin and Potsdam area are investigating how large-scale information management can be enabled by a virtualised cloud computing environment, to address the computing challenges of the coming decade.
Main focus of the research are the creation of a programming model and a massively parallel, adaptive and robust data processing engine, and its application to three use cases (analysing climate simulation data, cleansing of linked open data, and the analysis of large text data sets in the life sciences). The resulting system will be made available to the research community as open source.

DFG Programme Research Units

• A Library for Scalable Analytics and Mining in Stratosphere (Applicant Naumann, Felix )
• A Scalable, Massively-Parallel Runtime System with Predictable Performance (Applicant Kao, Odej )
• Compiling and Optimizing Iterative Data Analysis Programs with Shared State on Evolving Datasets (Applicant Markl, Volker )
• Continuous Query Re-Optimization in Stratosphere (Applicant Freytag, Ph.D., Johann-Christoph )
• Coordination (Applicant Markl, Volker )
• Data Profiling and Data Cleansing on Stratosphere (Applicant Naumann, Felix )
• Massively Parallel, Adaptive and Fault-Tolerant Execution of Data Flow Programs on Dynamic Clouds (Applicant Kao, Odej )
• Scalable Information Extraction in Stratosphere (Applicant Leser, Ulf )
• Stratosphere Data and Processing Model, its Optimization and Parallelization (Applicant Markl, Volker )
• Uncertainty and Data Cleansing in the Stratosphere Cloud Data Management System (Applicant Naumann, Felix )
• Web Data Analytics and Scientific Workflows (Applicant Leser, Ulf )

Spokesperson Professor Dr. Volker Markl