Better evaluations of European health care systems through a standardised methodology
The PECUNIA project
Why do different European countries use such different methods to make economic evaluations? Health economists from various European countries are working closely together on three-year EU project PECUNIA to arrive at a standardised method for the economic assessment of health and health care interventions. The project focuses on the costs and especially the benefits of these kinds of interventions in other sectors of society. 'Methodological improvements can provide insight into cost savings in other sectors.'
Ask researchers in different European countries to assess the value of a health or health care intervention and you will get different results. Professor of Public Health Technology Assessment Silvia Evers wonders if there are real differences or if these differences result from the use of different methodological approaches. 'You can’t compare results obtained from different methods.' The goal of EU project PECUNIA was to develop a standardised, harmonised method for the assessment of costs and benefits in European health care systems. 'Standardising the research framework and costing methods will facilitate fairer comparisons between countries.' What makes PECUNIA special within health economics is the fact that it takes a multi-sectoral perspective. This means that all relevant costs and benefits of economic evaluations, both within and outside the health care sector, are taken into account. The study which took place between 2018-2022 is part of a longstanding, innovative research line on intersectoral costs and benefits that has been ongoing since 2011 within the research unit on health economics and HTA and now as part of Maastricht Health Economics and Technology Assessment” Center (MaastrichtHETA). The Pecunia study focuses on mental health. Mental health disorders particularly often have financial consequences in other sectors than health care. Aggie Paulus, professor of economics of Education and Healthcare, provides several examples. 'Alcohol or drug addictions, for instance, can contribute to costs in the criminal justice sector. Depression in children can lead to school absenteeism and to costs in the educational sector.' At the same time, 'preventive health care interventions can produce benefits in other sectors of society as well, such as employment, education, safety, family life and leisure. 'However, so far, we do not yet know much about these intersectoral costs and benefits.
PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions)
- Start project: 2018-2022
- This project has received funding from: European Union’s Horizon 2020 research and innovation programme under grant agreement No 779292
- Project website
To standardise the methodology, consensus must be reached about the research questions, measurement instruments and costing methods. 'The methodological approach to economic evaluations tends to vary from country to country', says Ruben Drost, assistant professor in the department of Health Services Research. 'What is the influence of a health intervention on the use of health care, education and criminal justice? What are the costs and benefits and how are they calculated?' The researchers will study which methods are used in these sectors and what their advantages and disadvantages are. Evers likens the study to a Rubik’s Cube. 'We want to compare and standardise different methods and different sectors across different countries.' PECUNIA has been developing tools to align international methods, says Irina Pokhilenko. Pokhilenko has researched the costs and benefits that treatments have in the educational sector. Luca Janssen has been doing the same for criminal justice. 'The goal is to present health care providers and patients in each country with the same questions, about contact with the police, for example, or following specific types of education', says Pokhilenko. 'This way, the data obtained are comparable and transferable.' Janssen adds.
The PECUNIA consortium consists of ten partners from six European countries, all exchanging methodological expertise. 'Our colleagues at the University of Bristol are experts in measuring economic evaluations', says Pokhilenko. 'Erasmus University Rotterdam knows a lot about productivity loss as a result of health care. Our team provides knowledge about multi-sectoral costs and benefits, especially in little-researched sectors like education and criminal justice.' The researchers also use each other’s results so as to arrive at a uniform approach. 'And our working together increases the chances of the new method being applicable in other countries as well', says Janssen. The research team expects that a standardised assessment of health interventions will lead to efficient, evidence-based collaborative care models, both internationally and within the Netherlands. This will make it easier to determine the effectiveness and feasibility of a treatment. As Paulus explains, 'A treatment may be too expensive from a health care point of view, but still be interesting from a broader, multi-sectoral point of view because it has financial benefits in another sector of society. In other words, taking a broader perspective doesn’t just have scientific and methodological value for health economics and value-based healthcare, but also practical and social advantages.' This may ultimately pave the way for multi-sectoral policy.
The PECUNIA project also aims to show that a harmonised methodology will benefit health economics, both internationally and within the Netherlands. As Evers says: ‘it’s important to calculate health costs correctly. The costs and benefits of health interventions in other sectors of society will be ignored if you don’t attach any value to them or don’t take them into account financially speaking. This may lead to wrong decisions about interventions, based on incomplete economic information. Methodological improvements can provide insight into cost savings in other sectors.'
Text: Updated and adapted from Hans van Vinkeveen
ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions: PECUNIA tackles the healthcare challenges of an evergrowing and rapidly ageing population in the EU by developing new standardised, harmonised and validated methods and tools for the assessment of costs and outcomes in European healthcare systems. Comparing and exploiting data across different countries and sectors, PECUNIA provides direct comparable solutions to improve chronic and mental healthcare in all EU health systems.
Team members Maastricht University:
Involved research line
Our most important scientific output
- Pokhilenko, I. (2022). Costs and Benefits in Education: Addressing Methodological Challenges and Broadening the Societal Perspective in Mental Health Economics. Maastricht University. https://doi.org/10.26481/dis.20220630ip
- Janssen, L. (2022). Identification and Measurement of Criminal Justice Costs in Mental Health Economics Crossing Lines. https://doi.org/10.26481/dis.20221118lj
- Janssen LMM, Pokhilenko I, Drost RMWA, Paulus ATG, Thorn J, Hollingworth W, Noble S, Berger M, Simon J, Evers SMAA; PECUNIA Group. Methods for think-aloud interviews in health-related resource-use research: the PECUNIA RUM instrument. Expert Rev Pharmacoecon Outcomes Res. 2023 Apr;23(4):383-389. doi:10.1080/14737167.2023.2187379.
- Pokhilenko I, Janssen LMM, Paulus ATG, Drost RMWA, Hollingworth W, Thorn JC, Noble S, Simon J, Fischer C, Mayer S, Salvador-Carulla L, Konnopka A, Hakkaart van Roijen L, Brodszky V, Park AL, Evers SMAA; PECUNIA Group. Development of an Instrument for the Assessment of Health-Related Multi-sectoral Resource Use in Europe: The PECUNIA RUM. Appl Health Econ Health Policy. 2023 Mar;21(2):155-166. doi: 10.1007/s40258-022-00780-7.
- Pokhilenko I, Kast T, Janssen LMM, Evers SMAA, Paulus ATG, Simon J, Mayer S, Berger M, Konnopka A, Muntendorf L, Brodszky V, García-Pérez L, Park A, Salvador-Carulla L, Drost RMWA; PECUNIA-group. International comparability of reference unit costs of education services: when harmonizing methodology is not enough (PECUNIA project). Expert Rev Pharmacoecon Outcomes Res. 2023 Jan;23(1):135-141. doi: 10.1080/14737167.2023.2152331. Epub 2022 Dec 15. PMID:36472303.
- Muntendorf, L. K., Konig, H. H., Janssen, L., Pokhilenko, I., Drost, R., Simon, J., ... & Konnopka, A. (2022). Calculation of Standardized Unit Costs of Mental Health in the Criminal justice Sector-Results of the PECUNIA Project for Germany. Gesundheitsokonomie und Qualitatsmanagement. DOI: 10.1055/a-1867-1458
- Janssen LMM, Drost RMWA, Paulus ATG, Garfield K, Hollingworth W, Noble S,Thorn JC, Pokhilenko I, Evers SMAA. Aspects and Challenges of Resource Use Measurement in Health Economics: Towards a Comprehensive Measurement Framework. Pharmacoeconomics. 2021 Sep;39(9):983-993. doi: 10.1007/s40273-021-01048-z. Epub 2021 Jun 25. PMID: 34169466; PMCID: PMC8352823.
- Pokhilenko I, Janssen LMM, Evers SMAA, Drost RMWA, Schnitzler L, Paulus ATG. Do Costs in the Education Sector Matter? A Systematic Literature Review of the Economic Impact of Psychosocial Problems on the Education Sector. Pharmacoeconomics. 2021 Aug;39(8):889-900. doi: 10.1007/s40273-021-01049-y. Epub 2021 Jun 14. PMID: 34121169; PMCID: PMC8298334.
- Bremmers LGM, Evers SMAA, Drost RMWA, Janssen LMM, Pokhilenko I, Paulus ATG, Norton EC, Yoon J, Cuddeback GS, Morrissey JP. Intersectoral Costs and Benefits of Mental and Behavioural Disorders in the Education Sector: an Exploration of Costing Methods. J Ment Health Policy Econ. 2020 Sep 1;23(3):115-137. PMID: 33411675.
- Pokhilenko I, Janssen LMM, Hiligsmann M, Evers SMAA, Drost RMWA, Paulus ATG, Bremmers LGM. The Relative Importance of Education and Criminal Justice Costs and Benefits in Economic Evaluations: A Best-Worst Scaling Experiment. Pharmacoeconomics. 2021 Jan;39(1):99-108. doi: 10.1007/s40273-020-00966-8. Epub 2020 Oct 7. PMID: 33026633; PMCID: PMC7790798.
- Janssen LMM, Pokhilenko I, Evers SMAA, Paulus ATG, Simon J, König HH, Brodszky V, Salvador-Carulla L, Park AL, Hollingworth W, Drost RMWA; PECUNIA group. Exploring the identification, validation, and categorization of the cost and benefits of criminal justice in mental health: the PECUNIA project. Int J Technol Assess Health Care. 2020 Jul 27:1-8. doi: 10.1017/S0266462320000471.
- Drost RMWA, Paulus ATG, Evers SMAA. Five pillars for societal perspective. Int J Technol Assess Health Care. 2020 Apr;36(2):72-74. doi: 10.1017/S026646232000001X. Epub 2020 Jan 31. PMID: 32000868.
- Drost RMWA, van der Putten IM, Ruwaard D, Evers SMAA, Paulus ATG. Conceptualizations of the societal perspective within economic evaluations: a systematic review. Int J Technol Assess Health Care. 2017 Jan;33(2):251-260.doi: 10.1017/S0266462317000526. Epub 2017 Jun 23. PMID: 28641592.
- Mayer S, Paulus ATG, Łaszewska A, Simon J, Drost RMWA, Ruwaard D, Evers SMAA. Health-Related Resource-Use Measurement Instruments for Intersectoral Costs and Benefits in the Education and Criminal Justice Sectors. Pharmacoeconomics. 2017 Sep;35(9):895-908. doi: 10.1007/s40273-017-0522-4. PMID: 28597368; PMCID: PMC5563348.
For an overview of the many conference contributions and presentations, see: https://www.pecunia-project.eu/results
- See https://zenodo.org/record/4455364#.ZCQp83ZBxRY for a standardized costing template for selected costing approaches: criminal justice and education.
- See https://www.pecunia-project.eu/tools for the resource use measurement instruments, tools and compendia that have been developed as part of this project. They include the following:
The PECUNIA RUM Instrument. This instrument is a comprehensive validated, generic, self-reported, multi-sectoral, multi-national harmonised instrument available in multiple languages that measures resource use in all relevant sectors for costing from a societal perspective in the adult population: health and social care, education, (criminal) justice, productivity losses, and informal care.
The Pecunia Costing Concept. Cross-country comparability of the PECUNIA RUM is achieved by defining the measurement unit for resource use in all domains via the harmonised PECUNIA Costing Concept which accounts for the specific requirements of each resource use type. All relevant resource use items are unambiguously linked to the other PECUNIA Costing Tools via the PECUNIA Coding System, allowing valid comparisons and costing across countries and sectors by referring to activity-based rather than linguistic equivalence of services.
The PECUNIA Reference Unit Cost Templates. These templates (PECUNIA RUC Templates) are a Microsoft Excel©-based set of tools that enable researchers to calculate unit costs comparable across countries and sectors (so called Reference Unit Costs (RUCs)) using standardized and scientifically validated methods. In total, there are six PECUNIA RUC Templates designed for developing RUCs in different sectors in any country affected by the (spill-over) effects of health interventions. The RUCs were developed using standardized and validated PECUNIA RUC Templates to introduce a higher degree of methodological harmonization and transparency in unit costs within and across sectors and countries.
The PECUNIA Reference Unit Costs Compendium. This compendium (PECUNIA RUC Compendium) is a Microsoft Excel®-based multi-sectoral, multi-country database of unit costs comparable across countries and sectors and developed using standardized and scientifically validated methods (so called Reference Unit Costs (RUCs)). In its current form the PECUNIA RUC Compendium contains 95 RUC estimates for 16 different service and resource use items for six countries (Austria, Germany, Hungary, the Netherlands, the UK (specifically England), and Spain.
The PECUNIA Patient-Reported-Outcome-Measure-Mental Health Compendium. This compendium (PECUNIA PROM-MH Compendium) is a Microsoft Excel©-based electronic library of 204 non-disease specific PROMs used in mental health research. It provides meta-data information of existing, publicly available generic PROMs suitable for use in (mental) health research and their applicability in multi-sectoral, multi-national and multi-person (economic) evaluations.
Our most important output for societal target groups
Output for policy-makers
The results of the Pecunia project provide important insights into social challenges and can be of interest to policymakers, particularly those in the health and education sectors and the criminal justice sector. By highlighting the cross-sectoral economic impact of mental problems, the results of the Pecunia project are aligned with broader approaches to policymaking, such as Health in All Policies and systems thinking that stress the importance of collaboration across societal sectors to improve population health.
Output in terms of guidelines and manuals
Building on a previously developed guideline by Maastricht University on intersectoral costs and benefits (see here), the results of the Pecunia project were incorporated into the existing and updated guideline of the National Care Institute (Zorginstituut Nederland) on economic evaluation (see cost manual in the guideline).
- What is not measured is not counted: ECUNIA Resource Use Measurement (RUM) Instrument (2021, June 10).
- Introduction to PECUNIA Workshop by Judit Simon (2021a, June 10).
- Flash statements of all PECUNIA Principal Investigators (2021a, June 10).