1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 11th September 2023 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Registration and primary outcome reporting in behavioral health trials, 2022, Taylor & Gorman

Discussion in 'Research methodology news and research' started by Arvo, Feb 11, 2022.

  1. Arvo

    Arvo Senior Member (Voting Rights)

    Registration and primary outcome reporting in behavioral health trials | BMC Medical Research Methodology | Full Text (biomedcentral.com)

    Registration of research studies is designed to lock investigators into a data collection and analysis plan before a study starts and thereby limit their ability to engage in flexible data analysis and selective outcome reporting. Studies of registered clinical trials show that one- to two-thirds are registered after the study has started and that non-adherence to important design and analytic features, such as reporting data pertaining to all primary outcomes, remains high. Less is known about the effects of registration on research transparency and integrity outside of clinical trials. To address this gap in knowledge, the current study examined the effects of registration on the reporting of research findings in a sample of behavioral health trials published in BMC Public Health.

    Registered trials published in the BMC Public Health section “Health Behavior, Health Promotion and Society” between 2011 and 2015 were included in the study. For each trial, we reviewed associated online submissions from 13 different registration sites. For those determined to have been prospectively registered, we used the trial registry, MEDLINE (Pubmed), PsychINFO, Web of Science and e-mails to investigators to identify subsequent publications from the study that reported results pertaining to primary outcomes. The two investigators then independently reviewed the outcome publication(s) and compared the primary outcomes reported in these to the registered primary outcomes.

    The final analytic sample comprised 136 locatable, registered trials with an identifiable start date. Sixty-eight of the 136 were prospectively registered. Among these prospectively registered trials, only 16 published manuscripts reported outcomes and methods that were concordant with their registrations.

    Retrospective submission of protocols for publication and retrospective registration remain common in public health research, and adherence to prespecified outcomes is rare. In its current form, registration of behavioral and health promotion trials is likely to have minimal effect on preventing selective outcome reporting in publications, and the pervasiveness of vague and incomplete registry entries means that registries will have limited utility in terms of facilitating replication studies.

    The conclusion in the text is also worth the read.
    bobbler, Sean, Peter Trewhitt and 2 others like this.
  2. Andy

    Andy Committee Member

    Hampshire, UK
    From the paper

    "Registration documents are also used by the Cochrane Systematic Reviews in their assessment of selective outcome reporting bias [28]. Specifically, Cochrane strongly encourages “review authors to attempt to retrieve the pre-specified analysis intentions for each trial” and states that the best sources of such information are trial registry entries and trial protocol or design papers published in journals.” Only documents that are date-stamped “confirming the analysis intentions were finalized before unblinded outcome data were available to trial investigators” should be used by reviewers in assessing risk of bias resulting from selective reporting. Recent evidence shows that Cochrane reviewers perform selective outcome reporting bias assessments very poorly and appear to make little use of registries and published protocols [29]. The reasons for this are not entirely clear, but the current research suggests that there is a large proportion of registries (about half) and published protocols (~90%) that do not meet the requirements set by Cochrane for assessing selective outcome reporting. Indeed, it would be a problem if these retrospectively produced registries and protocols were used to assess selective outcome reporting bias, since low bias might simply reflect the fact that the outcomes reported in these were determined after data were collected and analyzed."

    I've created this thread, Risk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews, 2019, Saric et al, for the paper referenced as 29 in the above quote.
    bobbler, Mithriel, Trish and 6 others like this.
  3. rvallee

    rvallee Senior Member (Voting Rights)

    Wasn't there some thing a few years ago forcing exactly this as a new standard? I must have imagined it. Or maybe it was only for pharma studies, leaving behavioral studies to... do whatever they feel like doing to get better results, I guess.

    It would be kind of funny considering the unhinged rant in the Lancet that whined about Cochrane not doing things the Cochrane way, when in fact not doing things the Cochrane way is actually common what Cochrane does. It would be if it weren't serious anyway.
    bobbler and Peter Trewhitt like this.

Share This Page