Public Health Effects of a Novel Transport Policy: A Case Study for Advancing Systems Thinking in Natural Experiment Evaluation

• Applicant: Dr. Kerstin Sell
• Subject Area: Public Health, Healthcare Research, Social and Occupational Medicine
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 552256253

Policy evaluation is a powerful tool in social sciences to study the effects of policies, including their indirect, unintended and inequitable effects, as well as underlying mechanisms of impact. Public health researchers are concerned with evaluating polices from various sectors, such as food production, housing, or transport, which have substantial – albeit often indirect – effects on health. This complexity is addressed in the ‘complex systems’ evaluation tradition, which has gained traction among proponents of other evaluation traditions, including natural experiment evaluation, which was historically concerned with answering narrower questions of effectiveness. In this fellowship, I will bridge these two evaluation traditions and leverage the opportunity provided by an innovative, large-scale transport policy as a case study; in 2023, Germany implemented an unprecedented policy encouraging public transport use, the 'Deutschlandticket’, which allows for unlimited use of the country’s public transport and regional trains. I aim to ultimately advance the methodological toolset for retrospective evaluation of policies implemented in ‘complex systems’. I will develop a systems-informed theory of change of the policy (objective 1), utilise this to design a systems-informed natural experiment evaluation (objective 2), and develop methodological guidance for these approaches (objective 3). After an initial stakeholder mapping exercise, I will develop a systems map for 'public transport use and health' in a workshop with researchers and decision-makers and subsequently advance the map into a theory of change. This approach has not been tested in public health but has been strongly recommended for its enhanced capacity to capture complexity. The second methodological advancement lies in the subsequent utilisation of the systems-informed theory of change as the basis for designing a natural experiment evaluation. The theory of change will delineate the evaluation questions. Subsequently, I will undertake a thorough evaluability assessment to ascertain whether and how the Deutschlandticket policy can be evaluated retrospectively, in a systems-informed manner and drawing on cutting-edge methods for natural experiment evaluation. The proposed research will substantially advance the evaluation of complex, real-world health and social impacts of policies by providing evaluators in public health and other social sciences with methodological guidance to comprehensively consider ‘complex systems’ in both theory of change development and natural experiment evaluation design. It will further result in a protocol and corresponding grant proposal to conduct a systems-informed natural experiment evaluation of the policy. The systems-informed theory of change for the Deutschlandticket policy will be of high interest to transport policymakers and advocates, in Germany, the UK, and beyond.

• DFG Programme: WBP Fellowship
• International Connection: United Kingdom
• Host: Professor Dr. Frank de Vocht