Management of a hydrogen-based decentralized energy supply system using multi-agent reinforcement learning

• Applicant: Professor Dr.-Ing. Jan C. Aurich
• Subject Area: Production Systems, Operations Management, Quality Management and Factory Planning
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 547425292

For reasons of sustainability, energy cost reduction, and resilience to energy shortages companies are increasingly deploying decentralized energy supply systems (DESS). This involves the systematic control and improvement of energy use in a manufacturing system intending to reduce energy consumption and the associated environmental impact by continuously improving energy efficiency through the storage and use of energy at the most opportune time. The conversion of electrical energy into hydrogen is particularly suitable, as it enables medium to long-term storage and has advantages over the sole use of battery technologies for this period. Furthermore, fluctuations in the availability of renewable energies can be better compensated. Technological advances have made electrolyzers, storage systems, and fuel cells commercially viable for use by private companies. However, existing approaches for the management of DESS do not account for dynamic changes within boundary conditions because they consider only static and comprehensively described systems. Furthermore, they are limited concerning the considerable complexity resulting from multiple subsystems, decision variables, and constraints. Therefore, a novel approach is needed that considers the expected share of renewable energy, energy costs, and energy demand to decide when to draw energy from which source, store it in the form of hydrogen or electrical energy, or convert it back to electrical energy to power the production system. The collaborative project outlined in this proposal aims to investigate how the management of a hydrogen-based DESS can be realized using a multi-agent reinforcement learning (MARL) approach. The use of MARL allows the decomposition of the complex optimization problem into smaller subproblems and is therefore expected to provide good solutions for energy management. For this purpose, it will be investigated how the MARL approach is to be designed. Key contributions are the investigation of how the number of agents influences the overall performance and the explainability of this approach. The following sub-goals are defined as a basis for the investigations: 1) Utilization of digital twins and development of a framework to enable the management of a DESS with MARL, 2) Investigating the performance of different MARL approaches for the management of a DESS, 3) Assuring the transferability of the project results.

• DFG Programme: Research Grants
• International Connection: Brazil
• Partner Organisation: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior; (CAPES); Setor bancario Norte
• Cooperation Partner: Professor Dr. Thércio Henrique de Carvalho Costa