Synopsis

Keywords: Transferable skills: Reproducible research

The talk will present the importance of reproducibility in scientific research and how it has become a crucial part of scientific practices. The talk will also discuss the role of open data and source methods in promoting reproducibility and creating a more accessible and equitable scientific landscape. Finally, the talk will explain how implementing reproducible practices can improve the scientific output of labs in terms of both quality and efficiency.

Open Source Code, Open Data, preregistration, meta-analyses: they are all aspects of reproducible research. But what do we mean by reproducibility, and why do we want it?When we make a scientific discovery, our assumption is that what we have found is “true”, that is, our finding corresponds to reality. If this is the case, most of other studies with the same hypothesis would come to the same conclusion. That is, the study would replicate.
Until the early 2000s, the community was not particularly concerned with the replicability of findings. The traditional yet somewhat arbitrary threshold of 0.05 for the p-value of “significant” findings had become a seemingly acceptable level of error. Indeed, if a finding was not true, in theory, the overwhelming majority of published papers investigating it should report its falsehood. However, Ioannidis in 2005 (1) dropped a metaphorical bombshell onto the scientific world: due to systemic and personal biases, it could be statistically demonstrated that a vast majority of reported findings in the literature were very likely to be false. The scientific community was made aware of the pressures to only publish significant findings resulted in a significant distortion of reality, because type one errors were overrepresented in the literature. In a word, the findings were said to be not reproducible.
Alongside pure selection bias, by which studies with novel, significant, and surprising findings were published in higher-ranking journals, whereas nonsignificant or less striking studies were relegated to second-tier publications, when published at all, the community had also drifted towards questionable practices of artificially inflating the significance and the effect sizes of the published findings, which were collectively termed p-hacking.
So, we are coming back to the original question: what is reproducible science? In a nutshell, reproducible science is a paradigm shift in what we value in science, so that most published findings are better representing reality.
Reproducible science is thus founded on multiple pillars (2), broadly grouped into: rigorous statistics, open methods and data, and innovation in publication (preregistration, registered reports, open reviews, etc.). Open data and open source methods, in particular, are a crucial element of reproducible science, because they allow independent validation of findings and a more collaborative view of science.
Open science has led to a culture of transparency, inclusivity, and collaboration in the scientific community. It encourages researchers to share their findings, data, and code openly, thereby fostering a more accessible and egalitarian scientific landscape. Open science promotes the idea that science should be a public good, and that knowledge should be freely accessible to everyone. The culture of open science is about more than just transparency; it is about promoting a more ethical and responsible scientific process. It is about holding ourselves accountable for the work we produce and striving towards a more equitable scientific community. By embracing open science, we can create a culture of reproducible science that benefits everyone.
But reproducible science is not just an ideological stance focused on a big, utopian picture. It is also a set of practices that can help every scientist and every lab increase their productivity and maximize their impact (3). In this talk, we will briefly explore how implementing a set of reproducible practices can improve the scientific output of your lab both in quality and efficiency.

Acknowledgements

I acknowledge the Reproducible Research Study Group for the help and support in putting this talk together.