1. Read our 'News in Brief' post for w/c 18th March, Guest, by clicking here.
    Dismiss Notice

David Tuller: Trial By Error: Cochrane's Report on Courtney's Complaint

Discussion in 'General ME/CFS News' started by Kalliope, Mar 12, 2019.

  1. Kalliope

    Kalliope Senior Member (Voting Rights)

    Messages:
    1,765
    Likes Received:
    18,858
    Location:
    Norway
    Cochrane's Report on Courtney's Complaint

    I now have a copy of Cochrane’s report about Courtney’s complaint. It is clear from the analysis that Cochrane’s concerns involved the scientific accuracy and integrity of the exercise review. Contrary to the claims of the exercise review’s defenders, Cochrane was not responding to a hysterical mob of patients but to the cogent and well-reasoned argumentation of one patient in particular.
     
    shak8, Inara, pteropus and 37 others like this.
  2. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    497
    Likes Received:
    4,428
    Location:
    Belgium
    Do we know anything about the date on this document and by whom it was written?
     
    Barry, Binkie4, MSEsperanza and 3 others like this.
  3. dave30th

    dave30th Senior Member (Voting Rights)

    Messages:
    523
    Likes Received:
    8,750
    The date is at the end: April 10, 2018.

    The author is the person at Cochrane who was given responsibility to assess Bob's complaint.
     
    Inara, Barry, MeSci and 11 others like this.
  4. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    965
    Likes Received:
    8,683
    Location:
    Canada
    What are Cochrane's reasons to reject and bury this report? It makes for a completely different review.

    Where does it come from? Or is it sources and methods?

    How does it respect Cochrane's policies to bury evidence of possible harm and deterioration and the true nature of bias and strength of evidence? This is basically complete distortion of the quality of evidence, the difference between high and low overall quality.

    How likely is it we are going to find the same from the Lancet about PACE and PLOSone, especially about the release of the data on the latter?

    Are there any steps we can take collectively to accelerate something, anything? There are cascading consequences that can come as a result of this move and the time for denial is over. Someone needs to answer all the questions and fast.
     
    shak8, Barry, Atle and 9 others like this.
  5. deleder2k

    deleder2k Established Member

    Messages:
    9
    Likes Received:
    63
    I'm not sure what "sources and methods" mean, but it is from a reliable source. Why Cochrane isn't open with this kind of stuff is a question, but I guess they would call it "internal work" or something like that. Cochrane doesn't like a PR scandal.

    Someone did leak e-mails to Reuters last fall. I don't think Cochrane wasn't very happy with that. The troubling part is that only one side of the story got out. Seems like the Reuters journalist didn't read this complaint. That is a problem.
     
    shak8, Inara, Barry and 7 others like this.
  6. dave30th

    dave30th Senior Member (Voting Rights)

    Messages:
    523
    Likes Received:
    8,750
    I've just added an update to the post. Norway has a website where agencies list their documents and correspondence and people--anyone--can ask to see them.
     
  7. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    2,797
    Likes Received:
    23,241
    To me, it looks like Cochrane is trying to avoid embarrassing Larun and acknowledging that they got things wrong and that their processes failed. Courtney's complaint was so solid that they can't pretend he was wrong, but they certainly don't seem overly concerned that Larun was wrong in the first-place and then responded to Courtney pointing out her failings with dismissive "we must agree to disagree" nonsense. I don't think that they've 'buried' the report recognising Courtney was right (a report that was really unnecessary imo - Courtney clearly explained himself so that anyone could see for themselves that he was right) but they don't seem overly concerned by the fact that Larun has shown herself to be, at best, incompetent, or that her Cochrane review is continuing to cause patients problems.

    That is a level of transparency that amazed me!

    I just searched for 'lightning process'... 90 results: https://www.einnsyn.no/sok?offset=25&searchTerm=lightning process
     
    Last edited: Mar 13, 2019
    Barry, rvallee, Atle and 9 others like this.
  8. dave30th

    dave30th Senior Member (Voting Rights)

    Messages:
    523
    Likes Received:
    8,750
    yeah, I was surprised also.
     
    Barry, rvallee, MSEsperanza and 6 others like this.
  9. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    612
    Likes Received:
    4,140
    Location:
    betwixt and between
    Thank you, @dave30th, for bringing this report to light!
    I wish Cochrane added this report at a prominent place to the review.

    Have I understood properly that the person in charge of assessing Bob's criticism agreed with all of the key points Bob made?

    P.S.: Typos in the added update: "That is also the month when Dr Tovey leaves his position as Cochrane’s as editor-in-chiefl it..."

    Edit: I think I don't understand this paragraph:

    Does this mean Bob's formal complaint included the omission of objective outcomes, but this point was not recognized in the Cochrane report as a key issue because Larun's justification had been accepted? Or was this issue not addressed in the report because all other comments, regardless how substantial they were, were not assessed with the same thoroughness as the formal complaint was?
     
    Last edited: Mar 13, 2019
    Barry, rvallee, MeSci and 4 others like this.
  10. Alvin

    Alvin Senior Member (Voting Rights)

    Messages:
    2,834
    Likes Received:
    13,919
    So lets see Alem made himself much worse getting the PACE data, Bob dies from bad healthcare and perhaps his imprisonment and forced exercise.
    Funny how both cases are about pushing beyond our envelope, what the PACErs claim should be curing us. Both these people should be alive and healthy by their energy expenditures :emoji_face_palm:
    These two paid the price while the perpetrators claim they are the real victims

    @dave30th A story idea?
     
    Last edited: Mar 13, 2019
    Inara, pteropus, Barry and 14 others like this.
  11. fivetowns

    fivetowns Established Member (Voting Rights)

    Messages:
    37
    Likes Received:
    328
    Does anyone think it would be worth sending this new evidence with a request for a correction to Reuters and the Guardian?
     
    shak8, Barry, rvallee and 11 others like this.
  12. Alvin

    Alvin Senior Member (Voting Rights)

    Messages:
    2,834
    Likes Received:
    13,919
    Yes
     
  13. Skycloud

    Skycloud Senior Member (Voting Rights)

    Messages:
    1,454
    Likes Received:
    10,997
    Location:
    UK
    I agree about requesting corrections, but I also know that any corrections made will be tucked away where no one much will see them except those who are looking hard.
     
  14. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    2,680
    Likes Received:
    22,422
    "Authors must avoid giving direct recommendations, or referring to any intervention as ‘safe’
    or ‘effective’. Specifically, the following statement must be removed;

    We think the evidence suggests that exercise therapy might be an effective and safe
    intervention for patients able to attend clinics as outpatients
    .” "

    If only this were implemented (not just on the Cochrane review).
     
    Snow Leopard, shak8, pteropus and 9 others like this.
  15. chrisb

    chrisb Senior Member (Voting Rights)

    Messages:
    1,476
    Likes Received:
    10,579
    I see we are going to have to keep an eye on @dave30th to ensure that he does not sneak any "literary" allusions past us.
     
    MSEsperanza, Trish, Nellie and 2 others like this.
  16. Skycloud

    Skycloud Senior Member (Voting Rights)

    Messages:
    1,454
    Likes Received:
    10,997
    Location:
    UK
    I misread the title every time.
     
    Barry, Trish and chrisb like this.
  17. Cheshire

    Cheshire Moderator Staff Member

    Messages:
    1,368
    Likes Received:
    11,568
    If you have a look at the amended abstract they propose (last page), it's not revolutionary (nothing like "CBT and GET are ineffective", but the accumulation of "mays" seriously undermines the claims of success). That Larun et al. couldn't sign this is telling.
    upload_2019-3-13_10-50-21.png
     
    Last edited: Mar 13, 2019
  18. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    802
    Likes Received:
    8,403
    Location:
    Mid-Wales
    For comparison, this is what the Authors' conclusion of the abstract *still* says:
    Have they provided any reason at all why this hasn't been changed?
     
  19. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    497
    Likes Received:
    4,428
    Location:
    Belgium
    So in summary, it seems that Cochrane agreed with Robert Courtney on using the FINE trial unpublished data and the PACE trial selective reporting bias. But it's not clear if this will significantly alter the results. It seems mostly an illustration of Larun et al.'s bias in conducting this review.

    The most important critique - leaving out all objectives outcomes - is not mentioned. Vinks analysis showed that most of the GET-trials had objective outcomes. Some found improvements, but most of them didn't. So the conclusion of that would probably be that GET fails to show improvement on objective outcomes.

    As I see it the main critique addressed in this document, is the fact that at follow-up, GET no longer showed a significant improvement on all outcomes except for sleep. So normally you would expect the review to write something like: "GET showed significant improvements post-treatment but no longer at follow-up". But instead of simply saying that, the Cochrane commentator suggests it is best to state that no reliable conclusion can be made of the follow-up data. So he/she wants the review to state:
    So the outcomes at post-treatment are deemed reliable to mention, but the data at follow-up isn't. What the differences between the two?
    • If we take fatigue as the main outcome, 453 patients in the intervention group and 387 patients in the control group gave data post-intervention. Data came from 7 studies.
    • At follow-up there was data for 373 patients in the intervention group and 297 patients in the control group. Data came from 4 studies.

    That doesn't seem like a large difference to me. It's confusing to only state the post-treatment outcomes and act like the follow-up doesn't provide reliable outcomes. There ain't much of a difference between the two. In both cases, the majority of patients came from three studies: Powell et al. 2001, the FINE trial (Wearden et al. 2010) and the PACE trial (White et al. 2011). The interesting thing is that those three trial reported the follow-up data as their main outcome! So I think it's misleading, as the Cochrane commentator suggests, to state the post-treatment data and consider the follow-up data too unreliable to mention. That's contrary to how these trials were conducted and reported. And for obvious reasons: the response bias is probably much larger shortly after treatment. That's why well-conducted trials measure their main outcome some weeks/months after the end of treatment.
     
    Last edited: Mar 13, 2019
    Inara, pteropus, JohnM and 6 others like this.
  20. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    497
    Likes Received:
    4,428
    Location:
    Belgium
    It seems that the whole methodology of Cochrane fails to give an accurate view of what actually happened in these trials. Anyone looking at the data (not the propaganda of the authors) of the GET-trials neutrally would come to the following conclusion: "there seemed to be an improvement on subjective outcomes post-treatment but that disappears at follow-up and objective outcomes showed no improvement. The initial improvement was probably due to lack of blinding and response bias. So GET doesn't work." The Cochrane method of doing a review seems to avoid that conclusion by taking each part of the outcomes separately. The lack of blinding, for example, should be used to interpret the results. Instead, Cochrane simply addresses it by downgrading the certainty of their conclusions.

    If they were to take each objective outcome (VO2 max, walking test, fitness test, employment, disability payments) separately, they would probably conclude that for each outcome there isn't enough reliable data to reach a conclusion. But that would miss the obvious conclusion that these trial do not show objective improvement if you look at all these objective outcomes together. I hope this will not be the case, if they decide to include objective outcomes.
     
    Last edited: Mar 13, 2019
    shak8, pteropus, inox and 8 others like this.

Share This Page