1. Guest, the 'News in Brief' for the week beginning 3rd August 2020 is here.
    Dismiss Notice
  2. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019

Discussion in 'PsychoSocial ME/CFS Research' started by MEMarge, Oct 2, 2019.

  1. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    4,310
    Likes Received:
    33,484
    Location:
    Canada
    I don't know about the old review but the new review specifies the following selection criteria:
    PACE was not a RCT. It also plainly concluded no difference in outcome between treatment arms. As the biggest piece of evidence, its own conclusions do not match the review's conclusions.

    It seems the material is locked for now so I can't check the other trials but from Vink's review most of the other trials were not controlled as well.

    Not the first time I have seen this, using the RCT label then talking only of randomized trials. So it seems to be a new thing, I guess, that RCT also applies to randomized trials. Drop a word here. Add a word there. Change another word's meaning. Ridiculous.
     
  2. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    1,157
    Likes Received:
    8,322
    Location:
    betwixt and between
    #MEAction are right that the review authors still claim they "included eight RCTs with data from 1518 participants."

    Yet the biggest trial, the PACE trial, wasn't a Randomised Controlled Trial, and even the PIs called it only a "Randomised Trial".
    Didn't ceck the other trials.

    Edited to add: @rvallee beat me to it.
     
    LadyBirb, JaneL, WillowJ and 18 others like this.
  3. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,169
    Likes Received:
    12,191
    Location:
    Mid-Wales
    Here's the What's New panel:
    cochrane_exercise_review2019.png
    I'm going to do a side-by-side comparison with the abstracts in due course. There are some "interesting" changes in there.
     
    alktipping, JaneL, ukxmrv and 30 others like this.
  4. Adrian

    Adrian Administrator Staff Member

    Messages:
    4,717
    Likes Received:
    23,839
    Location:
    UK
    I went to the pdf version and selected the full version. Characteristics of the studies starts at page 44. Pace starts on page 55.

    They go through each trial and assess the risks. They don't discuss issues with subjective outcomes with the PACE risks and also give a low risk for "complete outcome data(attrition bias) All outcomes " Low risk "None found"

    Although I seem to remember a quite high GET drop out rate I haven't checked

    https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub8/epdf/full
     
    MEMarge, JaneL, WillowJ and 10 others like this.
  5. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    4,310
    Likes Received:
    33,484
    Location:
    Canada
    I'll add: I'm not impressed by no conflicts declared from the authors. Laughable. Either Cochrane's entire process is irreparably broken, or it allows arbitrary exemptions that make having a process irrelevant.

    Then again, the protocol deviation in PACE is "addressed" by stating it was noted in private steering notes and approved. Approving of a conflict of interest and recording protocol changes does not excuse either, it merely describes what happened, and even at that very reluctantly.

    The massive conflicts of interest from PACE authors were not declared either, so this is second degree conflict laundering. What a mess. The promise of a new review in 2020 with this in place is a mockery of clinical evidence and the scientific method.
     
  6. NelliePledge

    NelliePledge Moderator Staff Member

    Messages:
    5,424
    Likes Received:
    35,615
    Location:
    UK West Midlands
    New review starting in 2020 goodness knows how long it will take before it is published.
     
    LadyBirb, JaneL, MEMarge and 9 others like this.
  7. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    5,884
    Likes Received:
    38,531
    Page 56:

    upload_2019-10-2_21-17-45.png

    We know this is not a low risk of bias for fully unblinded studies, where the date of first seeing the data is by no means the first time trending of data will be evident.


    upload_2019-10-2_21-25-56.png

    Must be something else.
     
    MEMarge, LadyBirb, JaneL and 13 others like this.
  8. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    3,657
    Likes Received:
    29,883
    Their section on selective reporting bias is BS. Truly shameful that Cochrane let them publish this.
     
    MEMarge, JaneL, WillowJ and 17 others like this.
  9. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,169
    Likes Received:
    12,191
    Location:
    Mid-Wales
    The main thing that strikes me about the differences in this version is the number of times they state "We are uncertain..." (lots and lots).

    In the 2017 version, the only time "uncertainty" is mentioned is here:
    The "interesting" part for me was the substantial change to the Background section of the abstract. I'm not sure it helps though, because it gives the impression that the review adequately considered some things that it clearly didn't. Compare 2017 with 2019:
    To me, this indicates that they equate "persistent, medically unexplained fatigue" with "persistent postexertional fatigue" (whatever that is), that the included studies adequately screened for ME (when they didn't).

    To try to be positive, they have made a big deal of the chasm in evidence between findings at the end of therapy and long-term, where uncertainty abounds, and the uncertainty over the risk of serious side-effects.

    I think they've tried, but all it does is show up that they really don't know what they are doing.
     
  10. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    1,825
    Likes Received:
    17,297
    Location:
    Belgium
    Haven't read the new review yet, but I'm rather optimistic by this statement from the new editor in Chief:
    This might be more important than what's in the updated review.

    It seems like they have decided to start all over again with this review. It's not made explicit but I suspect that this will involve new authors (would be weird otherwise).

    So going back to where the Cochrane email correspondence stops. It seems that the new Editor in chief Soares-Weiser decided that Larun et al. should go ahead and publish the updates they had already worked on, but that this doesn't address all the problems and that Cochrane should simply start over.

    I personally would have preferred to have the review retracted as it could be misleading therapists and policymakers. I suspect the current option was the preferred compromise from a political perspective. But it's good news that Cochrane accepts that this updated review doesn't address all the issues and that they want a full update of the review, starting with the protocol (I suspect that means that objective outcomes need to be included).
     
    MEMarge, LadyBirb, Channa and 42 others like this.
  11. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    5,884
    Likes Received:
    38,531
    You could be right. Just so long as they don't sit on it too long.
     
  12. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,151
    Likes Received:
    73,007
    I do not see how an editor could have approved ‘moderate certainty evidence that probably...’ This is meaningless. Moderate certainty cannot then be qualified by probably.

    The text appears to be an appalling case of confused English. I had thought that Cochrane’s GRADE system gauged evidence of ‘moderate quality’ not moderate certainty. Moderate quality evidence of probably would make sense but moderate certainty of probably simply indicates that the authors have no understanding of what they are about.

    I am in Peru at present but likely will Communicate with Cochrane about this when I am back.

    How can we have a published review with a key conclusion making no sense at all?
     
    MEMarge, LadyBirb, Channa and 42 others like this.
  13. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    3,657
    Likes Received:
    29,883
    From their correspondence, it looks like 'probably' is standard for moderate quality evidence in Cochrane reviews:

    "Some practical issues will arise, e.g. which term shall we use to reflect the grading? Usually
    “probably” is used for moderate level evidence, while “may” is used for low level. We are willing to
    accept “may”."

    https://www.s4me.info/threads/cochr...evelopments-2018-19.10030/page-16#post-205744
     
    MEMarge, JaneL, Annamaria and 7 others like this.
  14. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    4,310
    Likes Received:
    33,484
    Location:
    Canada
    The statement frankly hits most of the notes and is committal. But the publication of this revised version of the review completely undercuts it. It's very hard to reconcile the two other than this future update being more of the same whitewash. Meanwhile the same recommendations stand as far as health professionals are concerned, despite lacking any credible evidence. After a whole year, there is still nothing but confusion and misleading claims. It still reeks of politics and conflicts.
     
    MEMarge, Chezboo, Channa and 15 others like this.
  15. Dolphin

    Dolphin Senior Member (Voting Rights)

    Messages:
    2,369
    Likes Received:
    12,131
    I hope some people submit comments. The authors are expected to reply. Also the comments and their reply(s) are available indefinitely in the review document.
     
  16. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,151
    Likes Received:
    73,007
    I get that probably is what moderate quality is supposed to mean but you cannot use the same probabilistic statement twice in a sentence. It is a bit like a double negative, it should combine to produce something else.

    I think there is a Freudian slip at work here. They wanted to say there was moderate quality evidence... But that would be an outright lie. So they side step using the official moderate quality and use moderate certainty and because that would still be a lie they qualify it with probably. So they end up with a sentence that if it means anything means ‘God knows whether’ but which sounds as if it says moderate quality evidence.
     
    MEMarge, Chezboo, LadyBirb and 37 others like this.
  17. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,151
    Likes Received:
    73,007
    I think to be scientific a word just needs to have an adequately defined meaning. Probably has a very precise meaning in ordinary English: a more than 50% chance. So it is fine - if based on adequate evidence.
     
    MEMarge, LadyBirb, JaneL and 17 others like this.
  18. Adrian

    Adrian Administrator Staff Member

    Messages:
    4,717
    Likes Received:
    23,839
    Location:
    UK
    Yet there is no admission of the serious failures with the review. Or apologies for misleading people.


    This basically shows a very weak editorial process and that they will publish things they know to be wrong. In effect they are admitting their brand is not trustworthy.
     
    MEMarge, LadyBirb, JaneL and 26 others like this.
  19. Adrian

    Adrian Administrator Staff Member

    Messages:
    4,717
    Likes Received:
    23,839
    Location:
    UK
    Because Cochrane are admitting they will not apply editorial control over things they publish.
     
  20. Adrian

    Adrian Administrator Staff Member

    Messages:
    4,717
    Likes Received:
    23,839
    Location:
    UK
    I assume the combination of two moderate values comes to a low value. (Like combining to .5 probabilities to give 0.25)
     

Share This Page