DORA

The San Francisco Declaration on Research Assessment (DORA) is an international initiative that aims to improve how research and researchers are assessed. DORA was developed in 2012 in response to concerns about the use of journal-based metrics, such as the Journal Impact Factor, as proxies for the quality of individual research articles or researchers.

DORA promotes a more responsible approach to research assessment. This means assessing research on its own merits and recognising the diversity of research outputs, contributions, practices and impacts. For example, DORA calls for consideration of outputs such as datasets and software, alongside publications, and for qualitative evidence of research impact, such as contributions to policy and practice. This also includes recognising Open Science practices that contribute to the quality, transparency, accessibility and societal value of research. DORA has developed specific guidance on recognising Open Science practices in the assessment of researchers.

DORA has since developed into a broader movement for responsible research assessment, supporting approaches that recognise the different characteristics and contributions that make research high quality.

This commitment is closely connected to both Open Science and Recognition & Rewards. Open Science practices can make important contributions to the quality, transparency, reproducibility and societal value of research, but these contributions are not always captured by traditional research assessment. Recognising and rewarding such practices as part of research assessment can help create an environment in which UM researchers are encouraged and supported to work openly. DORA explicitly supports the recognition and reward of Open Science and open scholarship practices in the assessment of researchers, alongside other contributions to research, education, outreach and leadership.

At UM, this is captured in its Recognition & Rewards policy, which aims to broaden the way academic contributions are valued by recognising different roles, activities and contributions rather than focusing predominantly on traditional research outputs and metrics. In this way, responsible research assessment can help create an environment in which quality, contribution, collaboration, openness, impact and other forms of value can be recognised alongside traditional research outputs.

The Maastricht University Library's Research Intelligence team supports this approach by helping faculties, research groups, support staff and individual researchers responsibly assess the academic and societal impact of research. Research intelligence is intended to support, not replace, qualitative evaluation. Analyses are therefore question-driven: the aim is to measure what you want to know, rather than simply what can be measured. This recognises that research quality and impact are multidimensional and that metrics can capture only some aspects of them.

In line with DORA, the Research Intelligence team does not use the Journal Impact Factor (JIF) or other journal-based metrics to assess individual researchers or individual publications. Where quantitative indicators are used, they are selected and interpreted in relation to the purpose and context of the assessment. The team also considers a broad range of research outputs and forms of impact, including outputs such as datasets and software and qualitative evidence of impact, such as contributions to policy and practice.

Through these approaches, DORA, Open Science and Recognition & Rewards reinforce a shared principle: research assessment should recognise the diverse ways in which researchers contribute to high-quality research and to society, rather than relying on a narrow set of quantitative indicators.

Research should be assessed on its own merits

Journal-based metrics, such as the Journal Impact Factor (JIF), should not be used as a proxy for the quality of an individual research article or researcher. The journal in which a paper is published does not, by itself, demonstrate the quality or significance of the research.

At UM, research assessment should therefore focus on the quality, significance and impact of the research itself, including the context in which it was conducted, rather than on the perceived prestige of the journal in which it is published.

Different research outputs can demonstrate your contribution

DORA calls on institutions to consider the qualitative value and impact of different research outputs and contributions in research assessment and not simply focus on quantitative metrics. This provides scope to recognise contributions that may otherwise remain invisible in traditional assessment approaches.

Research is more than journal publications. Depending on the discipline and research context, valuable contributions can include datasets, software, methods, protocols, publications, educational resources, policy contributions and other research outputs.

In line with UM's ambition to foster a research culture in which Open Science is the norm, Open Science practices also form part of a researcher's contribution. For example, sharing research data and software, publishing openly, preregistering research, making research and education materials available, or engaging citizens and other stakeholders in research can contribute to the transparency, reproducibility, accessibility and societal impact of research. These practices may sometimes constitute a contribution in their own right, while in other cases they are integrated into a research output or activity that is being assessed, for example, when a publication is accompanied by openly available data or software.

Metrics should be used responsibly

DORA does not mean that all metrics are inappropriate. Rather, quantitative indicators should be used carefully and in ways that are appropriate to the purpose and context of an assessment.

DORA's current guidance recommends that indicators should be clear, transparent, specific, contextual and fair. No single indicator can capture the complexity of research quality.

Learn more

• Read the San Francisco Declaration on Research Assessment (DORA)
• DORA: Guidance on the responsible use of quantitative indicators in research assessment
• Maastricht University’s signing of the DORA declaration
• Learn more about Recognition & Rewards at Maastricht University
• Learn more about responsible metrics at Maastricht University