Cochrane Review: 'Exercise therapy for chronic fatigue syndrome' 2017, Larun et al. - Recent developments, 2018-19

The paper may have been withdrawn by Cochrane rather than retracted, but when you read these emails it is clear the paper is effectively stuffed. Can't really see the authors hawking their review round other publishers with any success now, nor wanting to humiliate themselves further by trying. In effect it's dead I would have thought, withdrawn, retracted, whatever. Provided it really does get pulled. Presumably it can no longer be referenced by others as a Cochrane review, because Cochrane has withdrawn it; that is quite potent. Does this also mean that papers already citing it will need to clarify that Cochrane review they cite is no longer upheld by Cochrane? Seems like this does create quite a lot of waves.

And to be fair to David Tovey as far as is reasonable, he did stand by what he knew he had to, possibly because he knew that if he did not then the new incumbent would walk into a complete sh*t storm ... and maybe because he knew he was walking out of it. There are still so many who are not standing by what they know they have to.

I imagine @dave30th is already well aware of this.
 
Last edited:
Jonathan Edwards said:
I do not actually see why anyone other than the authors is involved in this. Bringing in muscle from administrators is not what science is about. The phoniness of the Cochrane procedure is transparent.
Click to expand...
Yes, it seems to show how the normal Cochrane process is to be bullied by authors, and having the "Cochrane" brand attached to your work is not so much about merit but about influence.
 
Barry said:
Yes, it seems to show how the normal Cochrane process is to be bullied by authors, and having the "Cochrane" brand attached to your work is not so much about merit but about influence.
Click to expand...

I was thinking about this today. Unblinded clinical trials with self-reported outcomes would probably not be accepted as evidence by Cochrane if the treatment in question was a fringe treatment like homeopathy, reiki, etc. It would probably be considered embarassing to even take any of such studies seriously.

Why is CBT/GET accepted then? I can think of two reasons: the associated narrative aligns with the popular prejudice and views rather than contradicting it, and the people behind it have influence and eminence.
 
Now would be a good time for a new properly executed review of exercise treatments for ME/CFS to be submitted to Cochrane. That would give Cochrane an easy way to move forward.

Does anyone know what David Tovey is doing now?

(Oh, and thanks @Michiel Tack for explaining the heterogeneity issue - I've added an edit to my earlier post.)
 
Barry said:
In effect it's dead I would have thought, withdrawn, retracted, whatever. Provided it really does get pulled. Presumably it can no longer be referenced by others as a Cochrane review, because Cochrane has withdrawn it; that is quite potent. Does this also mean that papers already citing it will need to clarify that Cochrane review they cite is no longer upheld by Cochrane? Seems like this does create quite a lot of waves.
Click to expand...
Hopefully at least a small tsunami.
 
Barry said:
[my bold]

So science is about ignoring issues that are further discovered to need addressing? "We agreed that no additional issues would be brought into the process" ... are they scientists or babies?
Click to expand...
Petulant is definitely a proper term to apply here.

Not a good look. It shows a level of unprofessionalism that is simply unacceptable in medicine. They are clearly incapable of the responsibility that is required and should never work unsupervised, or ideally at all.
 
strategist said:
I was thinking about this today. Unblinded clinical trials with self-reported outcomes would probably not be accepted as evidence by Cochrane if the treatment in question was a fringe treatment like homeopathy, reiki, etc. It would probably be considered embarassing to even take any of such studies seriously.

Why is CBT/GET accepted then? I can think of two reasons: the associated narrative aligns with the popular prejudice and views rather than contradicting it, and the people behind it have influence and eminence.
Click to expand...
This is what will hurt them in the long term.

Arbitrary decisions are exactly what science, even the evidence-based kind, is exactly all about not being. Something we've also seen in the recent paper about the "effectiveness" of CBT/GET in "fatigue clinics", that there is no standard, quality control is arbitrary and all over the place. This is what people who aren't serious about their responsibility do: things that may or may not work, or whatever.

In any professional field this is unacceptable. In medicine, it should be criminally actionable and lead to serious systemic reform. Too many people working in medicine are showing complete disregard for the consequences of what they do, to vulnerable people no less, and it's clear that the problem is not a fringe issue but pervasive and found throughout with no one in charge and no oversight to speak of.
 
Michiel Tack said:
The problem that the results of the GET-studies might not apply to patients who satisfy more recent diagnostic criteria was briefly mentioned. It seems that the authors will have to add a statement like "patients diagnosed using other criteria may experience different effects.”
Click to expand...

IMO, this is also a critical issue. As we saw in the 2016 Addendum to the 2014 evidence review by the US Agency for Healthcare Research and Quality (AHRQ), evidence of effectiveness disappeared once Oxford studies were excluded from the analysis. So all this discussion about low versus moderate of evidence and "probably" versus "may" is predicated on studies that used criteria that the NIH and AHRQ said should be retired because it includes patients with other conditions and thus can not be assumed to be generalizable to patients who actually have ME. The 2016 AHRQ Addendum pointedly concluded that studies using definitions that required the hallmark PEM were "blatantly missing" from the evidence base

The March 2019 note stated the changes requested of the authors would also address concerns raised in feedback since the Robert Courtney complaint. Does anyone know if they were asked to address this issue? Unless I missed it, the correspondence attached in this thread does not seem to discuss this.

Edited to add that even aside from the criteria issue, I agree with all the points made about how rotten this process is
 
Last edited:
The MCID thing often seems misused to me. If patients give their views on MCID on the assumption that the questionnaire scoring will not be affected by potential biases due to rigorous research methods (double-blind, etc), and then this MCID is used to hype the results of trivial differences between groups in a non-blinded trial, then that just seems like BS.

Also, while I've not looked at these particular studies, some MCID work seems designed to give researchers an artificially low MCID. For example, asking patients to select question answers showing a MCID on questionnaires that include very different items that are scored equally. That generates a MCID that is only really valid for changes to the particular items on the questionnaire selected but seems to be used for changes to other items that may not b so valued by patients.

It sounds like we're probably going to have to start digging in to all these things soon in response to Cochrane.
 
strategist said:
The people involved in this at Cochrane seem to have difficulty understanding that modest self-reported positive effects is exactly what you would expect if the treatment did not work but was tested without adequately controlling for bias while introducing positive expectations.
Click to expand...
It is arguably the biggest weakness in their case. The type and magnitude of their results are exactly what you would expect from known methodological biases and confounders.

All they have done is figure out how to maximise those biases and confounders, and then simply arbitrarily relabeled them as 'therapeutic'.

Whether gross incompetence, or deliberate deception, is an open question. But either way they are clearly unfit for task and should be removed from power immediately.

rvallee said:
Transparency is a beautiful thing. There should be more of it. In fact, there should be only it. Secrecy is a pillar of mediocrity.
Click to expand...
And tyranny.

Barry said:
Hopefully also act as a test case and trigger for further reassessments of other Cochrane poor reviews.
Click to expand...
Important point.

The first critical win (or handful of wins) is usually the hardest to achieve. But once that is done it opens up space and provides cover for other individuals/journals/institutions etc to start acting too.

Just because it is not obvious or immediate, doesn't mean that big shifts in the power relationships are not happening, or have not happened already. It just typically takes time for the consequences to flow through.

Cold comfort to those of us simply trying to survive for now, let alone prosper. But unfortunately that is how politics works, and this disaster is as much a political as a scientific problem.
 
NelliePledge said:
Well I imagine like RCGP their publishing process is somewhat long winded so possibly we can’t read anything into the time lag???
Click to expand...

Trish said:
The latest amendment:

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub7/full

17 June 2019

Amended

Addition of new published note 'Cochrane’s Editor in Chief has received the revised version of the review from the author team with changes made in response to the complaint by Robert Courtney. The process has taken longer than hoped; the amended review is being finalised and it will be published during the next 2 months.'
Click to expand...

Presumably they reached an agreement; either the authors or Cochrane gave some ground...
 
Michiel Tack said:
Thanks to @Marit @memhj for providing these documents. Are these all the emails of that exchange that are publicly available? I got the impression we only got snapshots. The website linked, eInnsyn, also has documents on this from 2018 but you have to file a request to see them, apparently. Does anyone have them?
Click to expand...

David Tuller got access to some, and wrote about it.
https://www.s4me.info/threads/david...cochranes-report-on-courtneys-complaint.8555/

I asked for, and received, the rest. But the PDF document I got is too big to upload here.

I'm happy to send the document as email to anyone who knows how to upload it here. Just send me a DM with an email address.
 
“When a collection of experienced and dispassionate colleagues are all making, in effect, the same criticisms, it is hard to ignore this. You are right that I said that I didn’t want to fall out over one increment on a GRADE rating, but the rating in question jars with me every time I see it. I can see three possible reasons for a downgrade: lack of blinding/subjective outcomes, imprecision, and inconsistency, so the conclusion that this is moderate certainty evidence seems indefensible to me, and as we know, I am not alone in this. As you know, there are others that would add selective outcome reporting to this list. It seems absolutely clear to me that this is low certainty evidence”

Wow, never thought I’d see an acknowledgement, both that these criticisms are coming from many sources, including “dispassionate” ones, and that concluding that this is moderate certainty evidence is indefensible.
 
You must log in or register to reply here.
Back
Top Bottom