Scientists world-wide join forces to investigate reproducibility in social and behavioural sciences.

Reproducibility means obtaining the same results by re-running the same analyses on the same dataset. It differs from replicability (testing the same question with new data) and robustness (testing the same question using alternative analyses on the same data).

The study found that data and code sharing remain limited. Only 24% of papers made their data available, and just 20% shared both data and code. Without access to these materials, independent verification of results is impossible.

When reproduction was possible, results were often—but not always—successfully reproduced. About 72% of the papers were reproduced approximately, while 53% matched the original results exactly. Availability of both data and code strongly improved reproducibility: 88% of such studies were reproduced approximately and 75% precisely. In contrast, when analysts had to reconstruct datasets from external sources, precise reproducibility dropped to about 11%.

Reproducibility rates also varied by discipline. Political science and economics performed better than other fields, possibly because journals in these areas more often require data sharing, code sharing, and reproducibility checks. “It’s too bad that our discipline, psychology, lags behind,” prof. Jonas adds “we’ve had wake up calls before and we’ve definitely made procedural improvements, pre-registration for example. But we don’t share our data and code enough. I hope this publication will encourage us to keep developing as a discipline, we can do better!”.

Prof. Jonas also emphasises the importance of participating in these kinds of large scale projects. “It offers you the opportunity to learn as a researcher by improving your methods and procedures for example, while making a valuable contribution to your field”.

Visit Nature to read the full publication.