Registration and primary outcome reporting in behavioral health trials, 2022, Taylor & Gorman

Registration and primary outcome reporting in behavioral health trials | BMC Medical Research Methodology | Full Text (biomedcentral.com)

Abstract
Background
Registration of research studies is designed to lock investigators into a data collection and analysis plan before a study starts and thereby limit their ability to engage in flexible data analysis and selective outcome reporting. Studies of registered clinical trials show that one- to two-thirds are registered after the study has started and that non-adherence to important design and analytic features, such as reporting data pertaining to all primary outcomes, remains high. Less is known about the effects of registration on research transparency and integrity outside of clinical trials. To address this gap in knowledge, the current study examined the effects of registration on the reporting of research findings in a sample of behavioral health trials published in BMC Public Health.

Methods
Registered trials published in the BMC Public Health section “Health Behavior, Health Promotion and Society” between 2011 and 2015 were included in the study. For each trial, we reviewed associated online submissions from 13 different registration sites. For those determined to have been prospectively registered, we used the trial registry, MEDLINE (Pubmed), PsychINFO, Web of Science and e-mails to investigators to identify subsequent publications from the study that reported results pertaining to primary outcomes. The two investigators then independently reviewed the outcome publication(s) and compared the primary outcomes reported in these to the registered primary outcomes.

Results
The final analytic sample comprised 136 locatable, registered trials with an identifiable start date. Sixty-eight of the 136 were prospectively registered. Among these prospectively registered trials, only 16 published manuscripts reported outcomes and methods that were concordant with their registrations.

Conclusions
Retrospective submission of protocols for publication and retrospective registration remain common in public health research, and adherence to prespecified outcomes is rare. In its current form, registration of behavioral and health promotion trials is likely to have minimal effect on preventing selective outcome reporting in publications, and the pervasiveness of vague and incomplete registry entries means that registries will have limited utility in terms of facilitating replication studies.





The conclusion in the text is also worth the read.

 

From the paper

"Registration documents are also used by the Cochrane Systematic Reviews in their assessment of selective outcome reporting bias [28]. Specifically, Cochrane strongly encourages “review authors to attempt to retrieve the pre-specified analysis intentions for each trial” and states that the best sources of such information are trial registry entries and trial protocol or design papers published in journals.” Only documents that are date-stamped “confirming the analysis intentions were finalized before unblinded outcome data were available to trial investigators” should be used by reviewers in assessing risk of bias resulting from selective reporting. Recent evidence shows that Cochrane reviewers perform selective outcome reporting bias assessments very poorly and appear to make little use of registries and published protocols [29]. The reasons for this are not entirely clear, but the current research suggests that there is a large proportion of registries (about half) and published protocols (~90%) that do not meet the requirements set by Cochrane for assessing selective outcome reporting. Indeed, it would be a problem if these retrospectively produced registries and protocols were used to assess selective outcome reporting bias, since low bias might simply reflect the fact that the outcomes reported in these were determined after data were collected and analyzed."

I've created this thread, Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews, 2019, Saric et al, for the paper referenced as 29 in the above quote.

 

Wasn't there some thing a few years ago forcing exactly this as a new standard? I must have imagined it. Or maybe it was only for pharma studies, leaving behavioral studies to... do whatever they feel like doing to get better results, I guess.

It would be kind of funny considering the unhinged rant in the Lancet that whined about Cochrane not doing things the Cochrane way, when in fact not doing things the Cochrane way is actually common what Cochrane does. It would be if it weren't serious anyway.