Bayesian Optimization with Partial and Dynamic Observations (ParDyBO)

• Applicant: Professor Dr. Sebastian Trimpe
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 556142469

Studies report that up to 32% of PID controllers in industrial settings perform only fairly or poorly. Modern model-based methods, like robust or predictive control, also involve tuning parameters (e.g., uncertainty or costs). And despite machine learning advances in control systems, tuning remains a challenge due to the introduction of its own tuning parameters. Thus, advanced control in practice always relies on configuring parameters based on hardware experiments. In the last decade, Bayesian optimization (BO) has emerged as a potent and flexible approach for controller tuning from limited experimental trials. BO methods rely on three components: a surrogate model, an observation model, and an acquisition function. The surrogate model maps controller parameters to performance and is updated with new observations. This model guides decisions on the next parameter configuration to assess, formalized as optimization of the acquisition function. Different acquisition functions result in various BO algorithms capturing the user's intent, such as rapid learning through information maximization or ensuring safe exploration. This versatility underlies BO's success in many applications. While much focus has been on surrogate models and acquisition functions, the observation model has been largely overlooked. Almost all works use a simple observation model: the evaluation of the black-box function plus noise. This assumes observations are (i) single scalar values (monolithic) and (ii) measured at a single point in time (static). Yet, these assumptions are oversimplifications. The black-box function usually consists of multiple components (e.g., sum of cost terms), and acquiring an observation is a dynamic process over time. Ignoring these aspects limits optimal tuning. The ParDyBO project moves beyond the monolithic and static observation model in BO. We introduce observation models that (i) capture the partial nature of most performance evaluations and (ii) characterize observation as a dynamic process. From this new formulation, we develop two innovative BO classes. First, using a surrogate model for partial observations, we create BO algorithms that actively decide which components to evaluate. Second, by identifying models of observation dynamics, we exploit predictions for improved planning and early stopping of experiments. Focusing on the observation model, ParDyBO represents a new BO and controller tuning paradigm with multiple innovations: leveraging partial and dynamical observations for better experiment design, integrating evaluation length as a novel decision dimension, and making online decisions about current experiments. We will demonstrate faster tuning, improved data efficiency, and resource savings through synthetic experiments, controller-tuning benchmarks, and hardware demonstrators. Open-sourcing all components is key to establishing "BO with partial and dynamic observations" (ParDyBO) as a new direction within the community.

• DFG Programme: Research Grants