Trial By Error: Some Good News on Cochrane, David Tuller

Cheshire

Senior Member (Voting Rights)
New post by @dave30th

In what can only be characterized as a welcome surprise, Cochrane has rejected the revision of a 2014 review of exercise treatments for chronic fatigue syndrome, stating that the work does not meet the organization’s “quality standards.” Cochrane revealed the decision late Friday in a statement appended to the review, which itself was a revision of a review first published years earlier.

That review reported that exercise therapy, most often graded exercise therapy, was effective. Yet the review was a methodological mess. Two patients–Tom Kindlon and the late Robert Courtney—wrote extensive and cogent critiques after it was published. When the lead author of the review provided inadequate responses, Courtney filed a formal complaint with Cochrane.
 
@dave30th says in the Virology Blog post:

The Cochrane statement does leave some questions [un]answered. Given that a revision was deemed necessary in the first place, the current published version is obviously unsatisfactory as is. Will this published but unsatisfactory version be withdrawn? If not, why not? If so, when?

Moreover, the Cochrane statement seems conveniently ambiguous on a key point. Will the authors of the current review be asked to develop the “full update” themselves or will they just be participating in discussions of what steps should be taken going forward? Until given reason to think otherwise, I will interpret that ambiguity as deliberate and therefore as a positive sign.

These questions in turn raise the issue do people/groups need to take any action at present, both in terms of supporting Cochrane for the action they have taken so far, reinforcing the need to move ME/CFS out of the mental health context into a more appropriate group, and in terms of pressure for the current exercise and CBT reviews to be withdrawn given Cochrane have implicitly accepted they are problematic/inadequate.
 
Sounds encouraging but the review should be retracted. The studies are all junk, there is nothing to review except how bad they are.
I can't disagree with this reasoning. Although I could see a review in which it was explained that all the studies have a high risk of bias and therefore the entire review, whatever the findings, has a high risk of bias. I mean, you'd have a review with null results, essentially.
 
Cochrane could do a good thing by publishing an article that points out all the problems with these studies. Cochrane would then at least contribute to setting higher standards, discouraging other researchers from following the same flawed recipe.
 
I can't disagree with this reasoning. Although I could see a review in which it was explained that all the studies have a high risk of bias and therefore the entire review, whatever the findings, has a high risk of bias. I mean, you'd have a review with null results, essentially.
Yes. An objective review would show that the evidence for efficacy and safety of exercise therapy for PwME is extremely poor. Reviews should not only be done to prove how wonderful an intervention might be; let's have some reviews that show it how it really is.
 
Last edited:
Will this published but unsatisfactory version be withdrawn? If not, why not? If so, when?

Their failure to withdraw the review after Blakemore and others pressured them always looked weak. This new statement makes it even more inexplicable. One reason I thought they would let Larun get away with minor changes is that refusing to publish her work and releasing a statement like this would leave Cochrane looking ridiculous.

Is it a bit impressive that they're willing to look ridiculous rather than publish whatever Larun submitted? My standards may now be so low that 'ridiculous but not entirely corrupt' counts as impressive for UK medical research institutions.

The most useful thing Cochrane could do for patients is to remove their reviews and let us get on with trying to sort out the mess that has been imposed upon us.
 
Cochrane could do a good thing by publishing an article that points out all the problems with these studies. Cochrane would then at least contribute to setting higher standards, discouraging other researchers from following the same flawed recipe.

That would be better than just withdrawing, yes. Have it on public record once more that those studies are a pile of crap.
 
Their failure to withdraw the review after Blakemore and others pressured them always looked weak. This new statement makes it even more inexplicable. One reason I thought they would let Larun get away with minor changes is that refusing to publish her work and releasing a statement like this would leave Cochrane looking ridiculous.

Is it a bit impressive that they're willing to look ridiculous rather than publish whatever Larun submitted? My standards may now be so low that 'ridiculous but not entirely corrupt' counts as impressive for UK medical research institutions.

The most useful thing Cochrane could do for patients is to remove their reviews and let us get on with trying to sort out the mess that has been imposed upon us.

And quite similar to PLOS One: yeah, we know they violated our terms and agreements but, nope, remains published.

So it's not just Cochrane. It's also Lancet. PACE is indefensible and Horton loves to yack about how much of published research is wrong and researchers sometimes get high off their own supply and it's also ruining millions of people's lives but, nope, remains published with the highest of thumbs up, recipient of awards for courage in research for courageously standing up on the neck of the sick and helpless. And BMJ with the absurdly unethical research they are allowing to be pushed on children, in addition to the pseudoscientific quackery of LP.

Big questions need to be asked about the political pressure and, possibly, threats made to those rocking the boat. We have seen over the years many medical professionals saying they have to keep shut about the topic or risk... something. It's not clear what, but there is definitely abuse of authority in pushing a political agenda.

The scientific argument has long been settled. So what is keeping the disbelief suspended? And how are authorities fine with this? Surely they understand that if it can be done this visibly and egregiously here, it can be done everywhere. This puts doubt into basically all research published in the UK, as it is clearly influenced by political agendas, to the point where an entire body of research can be suppressed in favor of delusions with barely any protest (and in fact loud protests when the delusion is challenged in the slightest).

It's very low quality research that cannot stand scrutiny. Cochrane know this, no doubt. It will inevitably be falsified and they will have to explain how they gave the triple thumbs up through an extra careful validation process to certify that this 100% subjective research was free of bias and conflict of interest and of the highest level of clinical certainty despite being 100% wrong.

That's very hard to justify and the window is closing to act before the damage to their reputation is fatal. Individual researchers may come and go, but organisations outlive them and have to live with reputational damage long after the fraudsters have abandoned ship. This seems to largely rest on who is political in charge, so possibly as long as Tories continue to push their modest proposal, this fiction will get political backing. But those organisations will outlive any political agenda. What comes after?
 
Their failure to withdraw the review after Blakemore and others pressured them always looked weak. This new statement makes it even more inexplicable. One reason I thought they would let Larun get away with minor changes is that refusing to publish her work and releasing a statement like this would leave Cochrane looking ridiculous.

Is it a bit impressive that they're willing to look ridiculous rather than publish whatever Larun submitted? My standards may now be so low that 'ridiculous but not entirely corrupt' counts as impressive for UK medical research institutions.

The most useful thing Cochrane could do for patients is to remove their reviews and let us get on with trying to sort out the mess that has been imposed upon us.

I think a number of authorities have been willing to look ridiculous rather than challenge PACE. There are obviously certain power structures in place that people don't want to challenge.
 
This suggests to me that QMUL and others ought to start bearing some blame and having their reputations on the line rather than the researchers.

Might be a good tactic.

I think QMUL as an institution have acted very badly and that needs to reflect on the vice chancellor and the whole university council who failed to act. Bad researchers happen but institutions are responsible for maintaining governance. QMUL continue on their coverup and I think senior staff there should be fired due to the failures of governance. Same with the vice chancellor of Bristol who has failed to act when they are doing research without ethical approval.
 
what do you think needs to happen for Cochrane to actually retract the review?
my guess is really no better than anyone else's at this point. If the authors would agree to the withdrawal, that would obviously make it easier. perhaps that is part of the ongoing discussions mentioned in Cochrane's statement. we'll find out this month, it seems.
 
my guess is really no better than anyone else's at this point. If the authors would agree to the withdrawal, that would obviously make it easier. perhaps that is part of the ongoing discussions mentioned in Cochrane's statement. we'll find out this month, it seems.

No doubt it would be easier if the authors would agree to the withdrawal but that shouldn't be necessary to remove a faulty review.

I'd like to know what the problem is (which would give us a better idea of the solution).
 
Back
Top Bottom