NSF Science of Science Proposal: Randomized Trial of Registered Reports, 2022, Nosek et al

NSF Science of Science Proposal: Randomized Trial of Registered Reports, MetaArXiv Preprints, 2022, Brian Nosek et al.

Center for Open Science NSF Science of Science: Discovery, Communication, and Impact (SoS.DCI) Randomized Trial of Registered Reports - Project Summary

Overview. The Registered Reports (RR) publishing model could be transformative to how a significant portion of research is conducted, peer reviewed, and published.

In a RR, authors submit a paper detailing the research question and proposed methodology to test the question prior to observing the outcomes of the study or studies. In the first stage of peer review, reviewers assess the importance of the question and the quality of the methodology proposed to investigate it.

If the paper passes this stage of peer review, the authors receive “in-principle acceptance” meaning that the journal commits to publishing the paper regardless of outcomes as long as the authors follow through with competent execution and reporting of the research.

The second stage of peer review, after the results are known and added to the paper, assesses adherence to the original commitments, clarity of the distinction between planned and unplanned analyses, and accuracy of interpretation of findings, and not whether the results are positive, interesting, or consistent with hypotheses.

RRs are most applicable to research that is testing a hypothesis. In theory, RRs: 1. Eliminate publication bias against negative results because publication decisions are made without knowledge of the results. 2. Increase clarity between planned (hypothesis testing; confirmatory) and unplanned (hypothesis generating; exploratory) analyses, thereby increasing the diagnosticity of statistical inferences. 3. Leverage peer review expertise more effectively to foster better research methodology and more informative outcomes--regardless of whether they are positive or negative results, or are consistent or inconsistent with hypothesized outcomes.

full article and pdf link at: https://osf.io/preprints/metaarxiv/jvxab/

 

@Caroline Struthers

Reminds me of the talks I went to at Guy's pre Covid, where Hilda Bastian had been due to speak.