Rearchitecting Cloud Database Engines Using Modern Storage Techniques for OLTP

• Applicant: Professor Dr. Viktor Leis
• Subject Area: Data Management, Data-Intensive Systems, Computer Science Methods in Business Informatics
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 551650645

In today's AI-driven era, rising data demands significantly burden data center storage, leading to increased operational costs and energy consumption, which adversely affect the environment. Our collaborative research seeks to redesign database system architectures to optimize the use of expensive storage devices in single-node servers and cloud environments, focusing on OLTP workloads characterized by frequent small random writes. Such write patterns amplify internal flash writes, thus degrading the performance and lifespan of SSDs, which increases electronic waste. This phenomenon is further exacerbated in the cloud when storing replicas across distributed storage nodes. To minimize write amplification and storage needs when using flash SSDs as a storage device, we aim to develop a new cloud database system using LeanStore's out-of-place writes. Additionally, we plan to explore cutting-edge storage technologies like ZNS (Zoned Namespace), FDP (Flexible Data Placement), and CMM-H (CXL Memory Module - Hybrid) within LeanStore. These technologies, largely unexplored in relational DBMS (Database Management System) and cloud systems, position us as potential pioneers in this field, aiming to improve performance and operational cloud costs. The project involves three parts: (1) Eliminating write amplification with ZNS, FDP, and RAID: ZNS and FDP allow to achieve no write amplification on the SSD, by moving the management of data placement to the application. LeanStore's out-of-place capability makes it ideally suited for these technologies. Plans are in place to adapt LeanStore to be compatible with Samsung's ZNS and FDP, and thus enhance transaction throughput while maintaining almost zero write amplification. Moreover, we plan to implement RAID at the DBMS level to reduce I/O amplification when using directly attached NVMe arrays. (2) Expanding LeanStore as a Cloud Page Server: Current cloud-native OLTP systems often require multiple replicas of database pages and logs, which significantly increases storage demands. We aim to transform LeanStore into a disaggregated storage engine using a RAID-like design. This adaptation will reduce the storage requirements while still preserving recovery capabilities. (3) LeanStore as a Cloud Control Plane with CMM-H: A control plane engine is crucial for managing metadata to support multi-tenancy. LeanStore's architecture will also be utilized to develop this control plane engine, and we plan to use CMM-H for storing the metadata of the data planes by using CXL.io and CXL.mem protocols for better scalability and reliability. The expected output of the project is a cloud-native OLTP database system that is based on new innovative storage technologies. We will share our findings by publishing papers in top-tier conferences or formalizing them into patents. Moreover, we plan to test the prototype's effectiveness in data centers.

• DFG Programme: Research Grants
• International Connection: South Korea
• Cooperation Partner: Professor Sangwon Lee, Ph.D.