The Lancet: "Offline: The gravy train of systematic reviews" Richard Horton Nov 2019

Discussion in 'Research methodology news and research' started by Sly Saint, Nov 15, 2019.

  1. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    9,958
    Location:
    UK
    https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(19)32766-7/fulltext
     
    Michelle, Sean, Lidia and 9 others like this.
  2. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    15,238
    Location:
    London, UK
    It is likely to require a bit more than Dr Roberts suggests.
     
    Michelle, Sean, alktipping and 10 others like this.
  3. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,420
    "Roberts argues that trust could be restored if reviewers included only prospectively registered trials and checked that trial data were real and accurate."
    Agreed restoration of trust would need more than that, but it would probably be a good starting point for establishing a fuller set of requirements.
     
    Last edited: Nov 15, 2019
    alktipping, EzzieD, rvallee and 2 others like this.
  4. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    13,757
    Location:
    Canada
    It's weird that this could have been written specifically about us and yet most of the people involved would insist that our case is different, despite fitting every single point. Which actually points to the root issue, that much of those reviews are the product of people who want the conclusions they present to be true, which is the common fault behind all BPS ME research. The PACE researchers were personally involved in grading their own work in the Cochrane review, FFS.

    It particularly mentions RCTs (nevermind the non-controlled ones like PACE, and in fact every BPS trial ever) and how many small ones are sometimes laundered into being more reliable than they are. Which is a critical issue surrounding PACE, that even after begrudgingly admitting all its limitations, its defenders simply point to a small catalogues of unreliable trials, which PACE authors actually admitted they aligned their own results to match the expectations they had out of those, that the conclusions of the definitive trial still stand, somehow.

    And no doubt Horton will continue to defend PACE despite hitting every single point he makes here, precisely because of the root cause at play in this crisis of reliability: he has a personal stake in the matter, his reputation depends on PACE holding until his retirement. Unsure how it happened but his promotion of PACE, especially the absurd claim that it was a "neutral" experiment by unbiased researchers, completely voids every point he makes here. And yet he is right. But he will still defend having done the same thing.

    About a week ago I noticed a tweet by him saying that he had hit upon what he considers the biggest scandal in medical research, possibly in decades. The timeline here fits that this is what he referred to.

    So how can this continue? How can a body of evidence detached from reality that fits every single one of the criteria exposed here as generically corrupting and amounting to an uninterpretable mass of biased conclusions still be defended by someone who recognizes those flaws are disqualifying but whose very reputation is threatened for having committed the same mistake, not once but twice over?

    This was the inevitable endgame, as pushing through the ideological BPS model of ME was not the validation of this model but guaranteeing its failure. Cherry-picked research with fraudulent claims and conclusions can somehow withstand being carried over the line behind academic ivory towers, but it will always fail in practice and create enormous pressure on those who mistook the starting line for the finish.

    Here Horton is essentially arguing for his own demise. He is guilty of all the critical problems he points to here. But he does not seem to realize it precisely for the same reason that the crisis exists in the first place: he just wants those conclusions to be true and now has committed himself in an impossible situation, of decrying the very thing he has committed, pointing the finger everywhere but at his own self.

    Yet another thing where WE TOLD YOU SO. For years and years WE TOLD YOU SO and you didn't listen. So what now? Where do we go from here with some of the most egregious examples of this crisis of reliability that have been proven again and again to be completely detached from reality and causing enormous harm to millions?
     
    MEMarge, Michelle, TiredSam and 9 others like this.
  5. Kalliope

    Kalliope Senior Member (Voting Rights)

    Messages:
    6,593
    Location:
    Norway
  6. NelliePledge

    NelliePledge Moderator Staff Member

    Messages:
    14,975
    Location:
    UK West Midlands
    Spot on Trevor Butterworth. Classic person in glass house throwing stones stuff from Horton.
     
    MEMarge, Andy, rvallee and 2 others like this.

Share This Page