Cochrane Review: 'Exercise therapy for chronic fatigue syndrome' 2017, Larun et al. - Recent developments, 2018-19

[chrisb]

To win at tactical leaking timing is everything.

 

[Binkie4]

Thanks @Sly Saint .

 

[Binkie4]

New Editor- in - Chief has been in place since June 1st? Am I right? What is her view? ( Hope I’ve remembered correctly that it is a ‘she’. Don’t remember reading anything about her).

Somewhere in this correspondence ( scanned quickly), Tovey refers to discussing this issue with ‘ Carla’, the new editor in chief. Since nothing has happened, what conclusions can be drawn from this?
Agree with @chrisb that it is very good to have this correspondence from Tovey’s last week in office in the public domain.

From a very quick scan, it read to me that Tovey was happy to disagree with the authors, withdraw the review and leave them free to publish elsewhere, while they were fighting tooth and nail to have this retained as the review, and defending the word ‘ moderate’ for quality of the evidence.

With the Nice guideline review in progress, this is hugely important.

 

[Barry]

From a very quick scan, it read to me that Tovey was happy to disagree with the authors, withdraw the review and leave them free to publish elsewhere, while they were fighting tooth and nail to have this retained as the review, and defending the word ‘ moderate’ for quality of the evidence.

Maybe in his last week in the job he was finally happy to drop a big pile of poo in his office, knowing it would not be him coming in to face it next week.

 

[Esther12]

To me this looks like Cochrane head office are fighting to get the authors to do the bare minimum, just to avoid embarrassing Cochrane. But letting the authors get away with what is remotely 'defensible' is still going to be really harmful. This whole process looks completely broken to me.

 

[feeb]

David Tovey recommended:
"I also think that in any public comments it may be important and useful to stress that this is not equivalent to a retraction of the review."

There's a nuance here that I don't understand. What's the difference between a withdrawal and a retraction? Other than the former being a polite way of allowing the authors to save some face, I mean.

 

[Snow Leopard]

There's a nuance here that I don't understand. What's the difference between a withdrawal and a retraction? Other than the former being a polite way of allowing the authors to save some face, I mean.

That is pretty much the difference as far as I am aware.

 

[large donner]

There's a nuance here that I don't understand. What's the difference between a withdrawal and a retraction?

Cowardice.

 

[Trish]

I think he was suggesting that if it's withdrawn as a Cochrane review the authors are free to seek publication elsewhere. Whereas if it's retracted, it's dead.

 

[ME/CFS Skeptic]

Thanks to @Marit @memhj for providing these documents. Are these all the emails of that exchange that are publicly available? I got the impression we only got snapshots. The website linked, eInnsyn, also has documents on this from 2018 but you have to file a request to see them, apparently. Does anyone have them?

To me this looks like Cochrane head office are fighting to get the authors to do the bare minimum, just to avoid embarrassing Cochrane.

I agree. Tovey constantly argues that he wants to come to something that's defensible, instead of something that is right and accurate.

It's distressing to see how Atle repeatedly uses the argument that they only want to focus on the issues Tovey originally raised because that's what they agreed on. That came across as rather childish behavior to me, given the stakes involved. If there are additional issues with the review, then surely these should be addressed by the authors' regardless of whether this was part of an agreement or not. it's their responsibility as clinicians all over the world use these reviews to treat patients.

The main issue, the lack of blinding + the focus on subjective instead of objective outcomes, is only mentioned briefly and Tovey seems to argue that simply downgrading the reliability of the evidence would do to address this issue. [EDIT: If patients report having recovered from taking homeopathy, we don't say that 'homeopathy may reduce fatigue spectacularly but with a low level of evidence'. That would be misleading.]

Most of the discussion revolved around the Powell et al. 2001 study that reported much bigger improvements than the other studies and so created heterogeneity. Excluding it halved the SMD for fatigue at follow-up but apparently, the authors had only mentioned this on page 134 of 142.

Another issue discussed is that the improvement on the Chalder Fatigue Scale is really small. Tovey argues that the evidence that this points towards a clinically meaningful effect is low. He wrote:

A drop of -3.4 on a 33 point scale really is pretty small, not moderate. And even if one uses the declared MCID of 2.3, which I think is arguable, surely the range of the absolute effects demonstrated includes levels that would not exceed the MICD, or would sit at the very bottom end in terms of magnitude. So, either one should downgrade the certainty of evidence further, or it should be acknowledged that the magnitude of the effect is likely small.

The problem that the results of the GET-studies might not apply to patients who satisfy more recent diagnostic criteria was briefly mentioned. It seems that the authors will have to add a statement like "patients diagnosed using other criteria may experience different effects.”

Another important point is that GET-trials used many different outcomes and that only fatigue pointed towards significant improvement. Tovey seemed to be aware of this problem, as he wrote:

it seems to me that the striking message to me is the absence of compelling evidence of benefit with the marginal exception of fatigue. I just don’t think this comes across.

 

[ME/CFS Skeptic]

@Caroline Struthers, these emails may be of interest to you.

 

[large donner]

I think he was suggesting that if it's withdrawn as a Cochrane review the authors are free to seek publication elsewhere. Whereas if it's retracted, it's dead.

I'm still trying to work out if Tovey has withdrawn or retracted testicles in this case.

 

[Hoopoe]

The people involved in this at Cochrane seem to have difficulty understanding that modest self-reported positive effects is exactly what you would expect if the treatment did not work but was tested without adequately controlling for bias while introducing positive expectations.

They might even be more than modest in some cases, while still not being real.

Such positive effect should not be seen as little bits of evidence of efficacy as they are no more consistent with this than a lack of efficacy.

 

[Jonathan Edwards]

But letting the authors get away with what is remotely 'defensible' is still going to be really harmful. This whole process looks completely broken to me.

The flip side to that is that with the correspondence out in the open I can ask anyone on the NICE committee who wishes to defend these trials to explain how any rational human being can conclude other than that the whole process is completely broken.

 

[Jonathan Edwards]

difficulty understanding that modest self-reported positive effects is exactly what you would expect if the treatment did not work

Actually the effect is so small it looks as if the real effect of the treatment is that it makes people worse!

 

[rvallee]

2017/10566 - F08 Exercise Therapy for chronic fatigue syndrome - Cochrane review

https://einnsyn.no/saksmappe?id=http://data.einnsyn.no/noark4/Saksmappe--983744516--10566--2017

View attachment 7687

Transparency is a beautiful thing. There should be more of it. In fact, there should be only it. Secrecy is a pillar of mediocrity.

 

[rvallee]

Maybe in his last week in the job he was finally happy to drop a big pile of poo in his office, knowing it would not be him coming in to face it next week.

Too bad he maybe did the very least in the circumstances and won't be deserving of any credit while being subjected to harsh criticism and howling by the ideological brigade.

It's pretty obvious he would be vindicated with time if he took meaningful steps. Or maybe he did and we'll know later but meanwhile people keep getting hurt as millions did during his tenure while the review stood for years so zero credit due.

 

[rvallee]

There's a nuance here that I don't understand. What's the difference between a withdrawal and a retraction? Other than the former being a polite way of allowing the authors to save some face, I mean.

I guess that's what the "collaboration" part is. Cochrane makes editorial decisions but not unilaterally.

Unfortunately the execution is completely broken.

 

[rvallee]

Thanks to @Marit @memhj for providing these documents. Are these all the emails of that exchange that are publicly available? I got the impression we only got snapshots. The website linked, eInnsyn, also has documents on this from 2018 but you have to file a request to see them, apparently. Does anyone have them?


I agree. Tovey constantly argues that he wants to come to something that's defensible, instead of something that is right and accurate.

It's distressing to see how Atle repeatedly uses the argument that they only want to focus on the issues Tovey originally raised because that's what they agreed on. That came across as rather childish behavior to me, given the stakes involved. If there are additional issues with the review, then surely these should be addressed by the authors' regardless of whether this was part of an agreement or not. it's their responsibility as clinicians all over the world use these reviews to treat patients.

The main issue, the lack of blinding + the focus on subjective instead of objective outcomes, is only mentioned briefly and Tovey seems to argue that simply downgrading the reliability of the evidence would do to address this issue. [EDIT: If patients report having recovered from taking homeopathy, we don't say that 'homeopathy may reduce fatigue spectacularly but with a low level of evidence'. That would be misleading.]

Most of the discussion revolved around the Powell et al. 2001 study that reported much bigger improvements than the other studies and so created heterogeneity. Excluding it halved the SMD for fatigue at follow-up but apparently, the authors had only mentioned this on page 134 of 142.

Another issue discussed is that the improvement on the Chalder Fatigue Scale is really small. Tovey argues that the evidence that this points towards a clinically meaningful effect is low. He wrote:


The problem that the results of the GET-studies might not apply to patients who satisfy more recent diagnostic criteria was briefly mentioned. It seems that the authors will have to add a statement like "patients diagnosed using other criteria may experience different effects.”

Another important point is that GET-trials used many different outcomes and that only fatigue pointed towards significant improvement. Tovey seemed to be aware of this problem, as he wrote:

That last sentence by Tovey is clear-headed and shows he understands that not only this is indefensible but that it always was and should never have been published, that the recommendations made in the review do not stem from reliable evidence but rather what its authors want to be true despite evidence to the contrary.

And yet it was and remained published through legitimate criticism that is so precise and thorough that it warrants immediate action. So what gives? Cochrane publishes reviews it knows to be junk and simply does not have a process in place to deal with this? This is literally their entire purpose and mission.

 

[Barry]

David,

This is confusing. We agreed on a list of issues that we were to address. We have addressed all of those. We agreed that no additional issues would be brought into the process, but you seem to have done that, despite our agreement. So, please point specifically to the issues that we agreed to address, that we have not satisfactorily addressed. We will then urgently seek to address those with the aim of meeting the requirements we have agreed upon.

Thank you,

Atle

[my bold]

So science is about ignoring issues that are further discovered to need addressing? "We agreed that no additional issues would be brought into the process" ... are they scientists or babies?