1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th March 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Caroline Struthers' letter to Cochrane Governing Board

Discussion in 'Psychosomatic news - ME/CFS and Long Covid' started by Peter Trewhitt, Nov 29, 2018.

  1. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    9,574
    Location:
    UK
  2. Trish

    Trish Moderator Staff Member

    Messages:
    51,890
    Location:
    UK
  3. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,855
    Location:
    betwixt and between
    Just wanted to drop an older not-yet-posted comment I found in my drafts folder (on David Tovey's answer regarding objective outcomes in his response to Caroline Struthers' complaint) :

    @Caroline Struthers:
    "Using only subjective outcomes in unblindable trials is terrible methodology. This is even more inexplicable in a review. And avoidable. Some of the included trials, including PACE, did measure objective outcomes, such as employment levels. They didn’t support the positive findings from subjective measures."

    David Tovey:
    "The review was conducted in a manner that was consistent with the review protocol. In fact, objective measures such as resource use were reported when they were identified. To describe this as ‘terrible methodology’ is simply an opinion and it is not shared by independent methodologists who we consulted, or the Cochrane Handbook.

    "It is common for Cochrane Reviews to use patient-reported outcomes as primary outcomes for chronic conditions. When the primary complaint is a private experience (e.g., pain, fatigue, anxiety) the most appropriate outcome is frequently the direct observation and report of patient experience by the patient, for whom it matters most. Obviously, there are limitations and risk of bias"


    Others have pointed out and criticized the gist of this answer -- the idea that using only subjective outcomes in unblindable trials is OK and to describe this as terrible methodology is just an opinion. I think Jonathan's comment is most illustrative:
    I realized only now that, in addition, Tovey rejected Caroline's criticism by saying objective measures actually were reported. However, the only objective measure he mentions is resource use.

    Apologies if that also has already been mentioned -- I now wonder, on the one hand, if this is true: Which objective outcomes did the review take into account?

    On the other hand, I wonder about resource use as an adequate indicator of treatment effect, especially if one of the treatment effects is that patients learnt that they won't get any help from health services but instead get worse.

    Reduced resource use might even correlate with giving positive feedback about one's health -- just say that you're better so the therapist or doctor won't keep misunderstanding and insisting you need more of their treatment.

    (Put his here since I am still not able to read much and don't want to jam the active threads where people keep repeating their and others' substantial criticism as well as adding new valuable comments that might help improve things with regard to the Cochrane saga.)
     
    Last edited: May 24, 2020
  4. Sean

    Sean Moderator Staff Member

    Messages:
    7,046
    Location:
    Australia
    Using and reporting objective outcomes is only half the story. They have to be taken fully into account, not ignored, downplayed, misrepresented, and then finally claimed to be irrelevant anyway.
     
  5. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    821
    Location:
    Oxford UK
    These are the measures used in the review. Nothing objectively measured except potentially adverse events - which weren't reported properly in the trials, dropouts, also not reported properly I don't think, sleep if measured by polysomnography, and resource use. I was a "consumer" referee on a Cochrane review of a drug for chronic pain once. I mentioned that they hadn't included outcomes I considered important - it might have been work/study status or something like that, and maybe quality of life... ? Can't remember. I was told there was no point including these outcomes because the studies didn't measure them. That's the whole point isn't it?! Cochrane really don't get that they could and should be highlighting these sorts of omissions to help improve primary research so future studies do measure meaningful outcomes. In this case, some of the studies did measure objective outcomes, but the Cochrane reviewers chose not to include them in the review.
    upload_2020-5-25_10-52-36.png
     
  6. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,385
    That's like a car MOT tester not testing the brakes because the car doesn't have any!

    Cochrane seem content to be driven by the scientists and not by the science.
     
    Last edited: May 25, 2020
  7. Trish

    Trish Moderator Staff Member

    Messages:
    51,890
    Location:
    UK
    Cochrane seem content to be driven by the scientists quacks and not by the science.
     
  8. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,385
    Yes, but my meaning is that even where trials have been run by good scientists genuinely striving for good science, there are still going to be trials where they have messed up. Cochrane is supposed to take a wholly independent approach, and not be influenced by the scientists themselves, else Cochrane cease to be independent and frankly lose their right to claim any sort of credibility.

    So when reviewing any and all trials, no matter who the scientists are, Cochrane must always guard against blindly transferring bias resident in trials under review, out into their own review. @Caroline Struthers example of Cochrane deliberately ignoring the absence of something crucial - because it was not there! - is a pantomime-absurd example of what a strangely unscientific collective mindset Cochrane seem to have.
     
  9. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    12,299
    Location:
    Canada
    Is it really? Giving veto to the authors of a review over retraction certainly doesn't support that, especially when it's clear that the review is completely unreliable and does not meet any acceptable standards, especially a veto over a review that was essentially rating their own grading of their own homework on a course they invented themselves with zero supervision over any part of that process. The authors basically own the review.

    Cochrane seems content with "don't ask me I just work here" and behaving as if the authors are threatening their puppies or something. Same thing with most BPS papers frankly, editors are content with pretending that they have no control over what they publish and because the concerns were dismissed by the authors it's out of their hands. Horton basically did exactly that with the PACE papers, taking no editorial position as if the authors were in charge. Authority without responsibility.

    What seems apparent is that there simply is no accountability or quality control for BPS "research". Everything goes. No one takes any position other than leaving everything to the authors and if they choose to dismiss the concerns then that's all there is to it. The substance of the papers isn't even evaluated, it's entirely a style review. If there is a review at all beyond reading the abstract or being a like-minded pal of the authors. Or when the authors themselves rate their own grading of their own work of their own invented subject matter.

    It's all a mockery of the scientific process.
     
  10. Barry

    Barry Senior Member (Voting Rights)

    Messages:
    8,385
    That's where the "supposed to" bit of my comment came in ...
     

Share This Page