2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles

[Hutan]

Oh, and thanks very much @Jonathan Edwards for the planned letter! Excellent.

 

[Peter T]

Actually, didn't the decision to replace the review only get made under K SW's watch?

My understanding is

1. Tovey recognised the flaws and promised to withdraw it
2. Though this decision was not announced publicly the review authors and others started a campaign for it to be kept
3. Tovey backed down and agreed to allow the authors to submit an edited version
4. The authors submitted a first edit which Tovey rejected leading to disagreement and an independent arbitrator being called in
5. Soare-Weiser then in post agreed to publish an edited version that failed to take on board all the arbitrator’s recommendations
6. Soare-Weiser recognised the problems with the now 2019/2024 version so promised a completely new review with radically a new stakeholder/patient consultation process beging the process in early 2020

 

[Jonathan Edwards]

Actually, didn't the decision to replace the review only get made under K SW's watch?

Yes, but Tovey had already identified it as substandard.

 

[Kitty]

Thanks, Jonathan, that looks great.

Couple of minor suggestions sharpening up—disregard if they change any of the meaning, they're not intended to.

What it may not convey is why people within Cochrane and outside should be so deeply ‘shocked’. The international healthcare community needs to be aware of the implications of the statement made by Hilda Bastian (2025) on her blog, Absolutely Maybe, given as the title of this letter.

What it may not convey is why people inside and outside Cochrane should be so deeply shocked. The international healthcare community needs to be aware of the implications of Hilda Bastian's (2025) statement on her blog, Absolutely Maybe, given as the title of this letter.

re-writing ... re-write ... sub-standard

rewriting ... rewrite ... substandard

Having authored a review

As a review author

Chalmers dismissed my concerns at the time, but it is now very clear that they were well-founded.

Chalmers dismissed my concerns, but it is now very clear they were well-founded.

Whoever was responsible for the decision to block the project from early on needs to be named and shamed.

Whoever was responsible for the decision to block the project needs to be named and shamed.

 

[Caroline Struthers]

and the subsequent Editor in Chief, Karla Soare-Weiser.

No, she didn't think it was substandard when she allowed it to be published in 2019. One of the IAG and I met with her in July 2019 and he begged her not to publish it. She told us it was too late and the deal was done. As we found out from correspondence obtained through FOI, the deal was done in June 2019 https://acrobat.adobe.com/id/urn:aaid:sc:EU:314a94ec-b56c-48fa-b8f8-890b87a960c6
The same month, days after we met with her, Cochrane changed its withdrawal policy so no Cochrane reviews have ever been withdrawn since.
She was even weaker than David Tovey on this, and that is saying something. She totally caved to pressure and has refused to engage with anyone on it. She needs to resign.

 

[MSEsperanza]

I had sight of another version of the review, never published,

I think adding a year to that sentence could help with regard to clarity?

"In [year] I had sight of another version..."

 

[Sean]

Whoever was responsible for the decision to block the project needs to be named and shamed. Otherwise, Cochrane’s reputation is worthless.

I think this language is too intense for something you want BMJ to publish.

It might be, but at least half the point of submitting this is to expose the BMJ editorial office to the arguments. I am not in favour of pussyfooting. Last time I sent in something about as critical it was published. Thee might be another wording. I will have a think.

I think it might be better to say that those who blocked it should be required to publicly put their case for it, with their names attached.

This still names and shames them, but somewhat indirectly, and puts the onus on them to overtly justify their strident opposition, via the normal scientific public debate process.

 

[Trish]

She was even weaker than David Tovey on this, and that is saying something. She totally caved to pressure and has refused to engage with anyone on it. She needs to resign.

Have you been able to identify where the pressure was coming from? Was it just Larun and her boss at that stage, or had the Wessely, White, Glasziou, and other BPS crew piled in too?

 

[Hutan]

No, she didn't think it was substandard when she allowed it to be published in 2019. One of the IAG and I met with her in July 2019 and he begged her not to publish it. She told us it was too late and the deal was done. As we found out from correspondence obtained through FOI, the deal was done in June 2019

Do you know when the decision to make the new review was made (which K S-W announced as being necessary because the methodology of the Larun review was outdated, etc)?

Was it at the same time as the Larun review was published, or did it come later?

 

[Hutan]

Have you been able to identify where the pressure was coming from? Was it just Larun and her boss at that stage, or had the Wessely, White, Glasziou, and other BPS crew piled in too?

Yes, do we know anything about Karla S-W that might indicate why she is so supportive of the Larun et al review? Does she have connections with any BPS proponents or insurance companies? Does she have any experience of ME/CFS?

 

[Caroline Struthers]

Yes, do we know anything about Karla S-W that might indicate why she is so supportive of the Larun et al review? Does she have personal connections with any BPS proponents? Does she have any experience of ME/CFS?

She's a psychiatrist. She will know Paul Garner very well. She's co-authored stuff with him eg https://www.bmj.com/content/354/bmj.i3507

 

[Caroline Struthers]

Have you been able to identify where the pressure was coming from? Was it just Larun and her boss at that stage, or had the Wessely, White, Glasziou, and other BPS crew piled in too?

There's a guy called Andy Oxman, well known in Cochrane, who was at the NIPH at the time, who is also very close with Iain Chalmers. Chalmers is famously hostile to people with ME. He turned very frosty with me in 2019 when he realised I was an ally! Anyway, Oxman was the one that suggested Gordon Guyatt as an arbitrator allowing the ridiculous torturing of the text to be able to say that exercise probably helps with fatigue. Although GG said they should clarify it doesn't have an important effect, the authors chose to ignore him

 

[ME/CFS Skeptic]

Although GG said they should clarify it doesn't have an important effect, the authors chose to ignore him

anyone has the text of guyatts reply?

 

[Nightsong]

anyone has the text of guyatts reply?

Think the important-effect comment was from this PDF of FOI'd correspondence:

Just to clarify, the judgment of moderate is contingent on rating certainty on a non-zero effect.

This implies that the authors do not claim an important effect and should avoid any language that claims an important effect. It would be reasonable to require in limitations that they highlight that they have not established an important effect.

 

[rvallee]

“Advocates of the intervention launched a full-on bid to try to stop the project. “

Details should remain on point, but would it be worth mentioning that a similar thing happened with the NICE guideline? As a pattern of behavior, one that is especially appalling given that the same ideologues have for years alleged, implied and even stated outright that we were not only doing that, but very successful at it.

Personally I think that patterns of bad behavior are very important. Especially one coming from people who accuse others, who obviously aren't doing the same, which is basically textbook DARVO.

 

[Peter T]

There's a guy called Andy Oxman, well known in Cochrane, who was at the NIPH at the time, who is also very close with Iain Chalmers

Would Andy Oxman and Sir Iain Chalmers’ employers offer potential channels for FOI requests on this issue, via the Norwegian Institute of Public Health and via Centre for Evidence-Based Medicine in the UK respectively? (alternatively for Sir Iain Chalmers there is the Swiss funded James Lind Library).

Interestingly the names cropping up now in relation to Cochrane go beyond the usual suspects more directly involved with ME/CFS, perhaps suggesting that those involved are seeking to protect the methodology of unblinded trials relying on subjective outcome measures as much as wanting to use exercise based intervention with people with ME/CFS and Long Covid.

[added to second paragraph]

 

[Hutan]

She's a psychiatrist. She will know Paul Garner very well. She's co-authored stuff with him eg https://www.bmj.com/content/354/bmj.i3507

The linked paper is
When and how to update systematic reviews: consensus and checklist
Hilarious

Newly identified studies can change the conclusion of a review. If they have not been included, this threatens the validity of the review, and, at worst, means the review could mislead. For patients and other healthcare consumers, this means that care and policy development might not be fully informed by the latest research; furthermore, researchers could be misled and carry out research in areas where no further research is actually needed.1 Thus, there are clear benefits to updating reviews, rather than duplicating the entire process as new evidence emerges or new methods develop. Indeed, there is probably added value to updating a review, because this will include taking into account comments and criticisms, and adoption of new methods in an iterative process.2 3 4 5 6

The decision needs to take into account whether the review addresses a current question, uses valid methods, and is well conducted; and whether there are new relevant methods, new studies, or new information on existing included studies. Given this information, the agency, editors, or authors need to judge whether the update will influence the review findings or credibility sufficiently to justify the effort in updating it.

I guess we can say that in the case of the relabelling of the 2019 review as 2024 without changing a word, the expectation was that the update would indeed influence the credibility of it sufficiently to justify the effort. Whether it does or not remains to be seen.

 

[Hutan]

That paper about when and how to update systematic reviews says:

The PUGs panel defined an update of a systematic review as a new edition of a published systematic review with changes that can include new data, new methods, or new analyses to the previous edition. This expands on a previous definition of a systematic review update.10 An update asks a similar question with regard to the participants, intervention, comparisons, and outcomes (PICO) and has similar objectives; thus it has similar inclusion criteria. These inclusion criteria can be modified in the light of developments within the topic area with new interventions, new standards, and new approaches. Updates will include a new search for potentially relevant studies and incorporate any eligible studies or data; and adjust the findings and conclusions as appropriate. Box 1 provides some examples.

So, according to a consensus paper that Karla S-W signed up to and that had Paul Garner as corresponding author, an update should incorporate relevant studies. It did not, despite there having been some e.g. Magenta.

That seems to me to be a point of weakness - it is good practice for any update to include any relevant new studies and the 2024 Larun et al update did not.

Did the review use valid methods and was it well conducted?
If the question is current and clearly defined, the systematic review needs to have used valid methods and be well conducted. If the review has vague inclusion criteria, poorly articulated outcomes, or inappropriate methods, then updating should not proceed. If the question is current, and the review has been cited or used, then it might be appropriate to simply start with a new protocol. The appraisal should take into account the methods in use when the review was done.

They say here that if a review has vague inclusion criteria (yes), poorly articulated outcomes (probably, ignored objective outcomes), inappropriate methods (absolutely, included subjective evidence from unblinded trials unsupported by objective evidence), then updating should not proceed, and it is better to start again.

 

[Hutan]

If anyone is making a complaint to Cochrane about the relabelling of a review and calling it an update, that paper with Garner and Soares-Weiser among the authors would be an excellent reference to cite when suggesting the update did not follow good practice.

 

[bobbler]

My concern is that when people read websites or magazines, they skim, and if your title doesn't summarise what you say in your letter, a huge proportion of the readership is unlikely to ever get the message - they'll simply skip over your letter and keep going. It's the nature of how people read this kind of thing that it will be a real win if they read the first paragraph, let alone the rest.

“Advocates of the intervention launched a full-on bid to try to stop the project" is opaque, because it doesn't say what project you're talking about. I'd go for the single thing you'd shout out of your car window about the situation if you passed a medical friend walking along the pavement. Something like, 'Cochrane's failures over its ME/CFS review severely damage its reputation' (there will be better phrases, but this is a quick illustration).

And then I think the first sentence needs to say what has actually happened, for those who didn't read the article. At the moment you start with, 'The News article by Jacqui Wise (2025) today gives a fair account of the disgraceful behaviour at Cochrane over the planned re-writing of the Systematic Review on Exercise Therapy for ME/CFS.' But how about, 'The News article by Jacqui Wise (2025) today gives a fair account of the disgraceful behaviour at Cochrane in abandoning the planned re-writing of its poor-quality Systematic Review on Exercise Therapy for ME/CFS, which risks harm to patients.

Even if that was all that anyone read, you would have successfully stuck the boot into Cochrane, as it deserves.

Loving the car window point. I’m being childish now in admitting it gave me fond memories of The Inbetweeners famous habit of shouting a certain nick name out of car windows at people waiting for a bus. But variations on that wouldn’t get published

agree strongly on the point that it needs to be made very clear who these people are and what they are supporting

I find it grammatically interesting that such people given what they’ve clearly been up to solidly over the space of time since 2017 ish onwards got called ‘advocates of an intervention’ when advocacy is normally ‘for’ the animate

and yet the same journals have been happy to publish terms like ‘activist’ for people who pointed out glaring errors that caused harm , time after time as each of these got proven. These are the very definition of lobbyists are they not if going by the behaviour /actions that define that term?

the least we can do is make the conflict of interest clear ie what are they ‘pro’ (whether it shells or harms patients or not)? As that seems at least part of the driver (I’m sadly starting to worry some see it ‘upsetting patients’ at least, potentially the other harm aspects, not as a disbenefit).

at the end of a day this is business for these people not what others would expect in medicine. These same people aren’t there for the long haul and good if treating either the illness or the patient properly whatever that may be, but are supply-led

so she’s right on the term interventions - but they sure don’t have any broad taste on what they mean by that . It’s an old and tired variation on the same narrow theme they’ve used to block all the normal angles of enquiry for treatment or management advice etc

so yes the name could be more ‘on the nose’ to what it is they are pushing and the word advocate for treatment that is harming patients seems very at odds