Summary
The United Kingdom is experiencing the largest-ever increase in public investment in research and development, with the Government R&D budget set to reach £20 billion a year by 2024/5. The creation of the new Department for Science, Innovation and Technology has been advanced by the Government as heralding an increased focus on research and innovation—seen to be among Britain’s main strengths.
At the same time, there have been increasing concerns raised that the integrity of some scientific research is questionable because of failures to be able to reproduce the claimed findings of some experiments or analyses of data and therefore confirm that the original researcher’s conclusions were justified. Some people have described this as a ‘reproducibility crisis’.
In 2018, our predecessor committee published a report ‘Research Integrity’. Some of the recommendations of that report were implemented—such as the establishment of a national research integrity committee.
This report looks in particular at the issue of the reproducibility of research.
We found that while there are many reports of problems of non-reproducibility, there has been no comprehensive and rigorous assessment of the scale of the problem in the UK, nor which disciplines are most affected and therefore the extent to which this is indeed a ‘crisis’.
While we welcome the establishment of the new Committee on Research Integrity and note that one of its so-called strategic pillars is to “define the evidence base”, we are concerned about the absence of reproducibility as a priority in the new organisation’s strategy. We recommend that a sub-committee focussed solely on questions of reproducibility in research should be established.
Evidence to our inquiry raised important concerns about the academic publishing industry in—however unintentionally—giving rise to pressures that can undermine research integrity. To be successful, academics need to establish a strong list of publications in highly-rated learned journals. We heard a widespread view that journals favoured for publication original—rather than repeated—research and work which had striking or new outcomes. This meant that the value of conducting repeated or confirmatory studies was much reduced, and that there were strong incentives to obtain striking research findings. We call upon publishers to commit to publishing without prejudice confirmatory studies and those whose findings turn out not to be novel or striking.
We also recommend that publishers should make a commitment to more timely publication of retractions of published articles and the correction of errors as soon as they become known.
Some witnesses described how the short-term tenure of early career academic contracts and research grants provide insufficient time for researchers to ensure that their work is reproducible by others. We call upon funders, including UK Research and Innovation, to consider whether its grants provide the resources necessary to ensure that work that it funds is reproducible, and we recommend that it requires reproducibility as a condition of grants awarded.
Training researchers in research integrity and the need to ensure reproducibility is inconsistent and often absent. We recommend mandating the provision of such training at undergraduate, postgraduate and early career researcher stages.
We welcome UKRI’s policy of requiring open access to research that it funds, but we recommend that this should go further in requiring the recipients of research grants to share data and code alongside the publications arising from the funded research.
We believe that a wider set of measures of academic success than lists of publications should be encouraged. The Future Research Assessment Programme, being carried out by UKRI, should address this and funders should also consider wider use of the ‘resume for researchers’ format in funding calls.
