Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019 and new date December 2024

[MSEsperanza]

From that list, it looks like basically one account posted and around 20 others retweeted it from them. The original account is Physio Meets Science @PhysioMeScience

Might be worth telling them and anyone else who posts it what the deal is to see if they'll delete. Very likely they don't know it's not a new review.

Yes, probably they didn't realize. Which underscores how misleading it is.

I don't have a Twitter account anymore, but I used the contact form on their website.

Thank you and also to those who reacted on Xitter.

Physio Meets Science seems to be a German commercial outfit providing physio 'educational' materials.

Yes, and they have more than 90.000 followers on X. Many German-speaking physiotherapists, including those teaching at colleges, seem to like what they have to offer.

 

[Caroline Struthers]

From that list, it looks like basically one account posted and around 20 others retweeted it from them. The original account is Physio Meets Science @PhysioMeScience

https://www.physiomeetsscience.net/

Might be worth telling them and anyone else who posts it what the deal is to see if they'll delete. Very likely they don't know it's not a new review.

Looks like at least one person has replied to their tweet to tell them.

I don't have a Twitter account anymore, but I used the contact form on their website.

Have replied to their Tweet

 

[ME/CFS Skeptic]

Not sure -- I remember vaguely that adding an editorial note/ a link to the EiC's announcement was an idea proposed by forum people? I think the version history is right that it was not added before February 2020.

I also have a 2019 version that does not include the editorial note with a link to the update process, even though it includes all the rest of the version history, all the way back to 2004. The latest update mentioned is this:

 

[ME/CFS Skeptic]

Wasn't there some rule that if a review is substantially out of date, the editors can put a warning sign to it? Like they did with the 2008 review of CBT for ME/CFS. It says:

This 2008 review predates the mandatory use of GRADE methodology to assess the strength of evidence, and the review is no longer current. It should not be used for clinical decision-making. The author team is no longer available to maintain the review.

https://www.cochrane.org/CD001027/DEPRESSN_cognitive-behaviour-therapy-chronic-fatigue-syndrome

The Larun et al. review is now also no longer maintained, with the latest search done 10 years ago.

Maybe @Caroline Struthers knows more about this procedure?

 

[Nightsong]

There is a Cochrane policy guide for when editorial notes can be used:

https://documentation.cochrane.org/egr/editorial-notes-318472364.html#Editorialnotes-Policy

I think the most relevant one might be Purpose 1 ("to add a statement by the Editor in Chief about the status or context of a Cochrane review or protocol" - "usually initiated by the Editor in Chief")?

 

[Peter T]

A selection from the Cochrane policy guide for when editorial notes can be used (sorry formatting was the best I could manage):

1. To add a statement by the Editor in Chief about the status or context of a Cochrane review or protocol. - Usually initiated by the Editor in Chief.
   
2. To publish an Expression of Concern about a review or protocol. - May be used -as part of the process when studies included in a Cochrane review are identified as potentially problematic or as part of an investigation relating to concerns about the conduct or reporting of a Cochrane review.
3. To direct readers to a more recent review or protocol that has been published independently of the current review (not as an update or amendment). - May be used on an older review when the review question has been addressed in a newer review with a broader scope or different methodology; or when a new protocol has been published; or in any other case where a review has not been updated but has been superseded by another review.

It seems all are a possibility given

• the Editor in Chief could act if she wished
• there are problems with the review in terms of failure to address bias and failure to report harms, is out of date based on evidence prior to 2011, etc
• there is a more recent fuller review of the evidence that supersedes this review, namely that contained in the new NICE guidelines in the ME/CFS 2021 guidelines

However I doubt Cochrane would consider any grounds to add a further note that did not suit their BPS favourites.

 

[Trish]

There is a Cochrane policy guide for when editorial notes can be used:

https://documentation.cochrane.org/egr/editorial-notes-318472364.html#Editorialnotes-Policy

I think the most relevant one might be Purpose 1 ("to add a statement by the Editor in Chief about the status or context of a Cochrane review or protocol" - "usually initiated by the Editor in Chief")?

Quoting from this guide to adding editorial notes:
____________________

Decisions and communication
Before requesting publication of an Editorial note, take the following steps:

• consult the Editorial notes policy above;
• determine the type of Editorial note required; and
• contact Cochrane Support (support@cochrane.org) to seek prior approval from the Research Integrity team.

After publishing an Editorial note, Cochrane Support will notify David Hives at Wiley (dhives@wiley.com) with a link to the review or protocol.

Following notification, David will

• • manually update the PubMed version of the article (the Editorial note will display witihin 24 hours of the update), and
  • if the full text of the review is on PubMed Central (PMC), send the Editorial note to PMC Support, who will update the PMC version of the article (this can take up to two weeks).

Specifications for publishing and display
Editorial notes are created and managed in RevMan using the Published notes section. For details on how to add or edit an Editorial note, see the RevMan knowledge base.

Editorial notes are published as part of a standard publication workflow. To add an Editorial note to a published review – making no other changes to the review – use the 'Amended' What's new event (no new citation) (see Dates and events in RevMan Knowledge Base for implementation instructions). During the publication process, the Editorial note is extracted from the published notes and displayed separately.

When published, the Editorial note will display in the Cochrane Library as a text box above the abstract online and in the PDF version, and above the Plain Language Summary on https://www.cochrane.org/evidence. As noted above, Cochrane Support will contact Wiley to initiate manual changes to the PubMed and PubMed Central version(s) of the article.
____________
End of quote, more at link.


That seems to make it clear that if all that is changed is adding a new editorial note, it should be added to the existing review with no new citation. I think that means you shouldn't pretend a review is new just because it has the new note added, nor should you change its publication date.

Cochrane lists the 2024 version as version 9, the 2019 version as version 8, in other words, they have given it a new citation.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub9/information#history

It also says the Research Integrity Unit has to approve it. I wonder whether they approved changing the date and calling it a new version of the review.

 

[Caroline Struthers]

Wasn't there some rule that if a review is substantially out of date, the editors can put a warning sign to it? Like they did with the 2008 review of CBT for ME/CFS. It says:

https://www.cochrane.org/CD001027/DEPRESSN_cognitive-behaviour-therapy-chronic-fatigue-syndrome

The Larun et al. review is now also no longer maintained, with the latest search done 10 years ago.

Maybe @Caroline Struthers knows more about this procedure?

That warning on the CBT review took me a lot of to-ing and fro-ing to secure. It was I think a last ditch attempt to get me off their back rather than something that is routinely done when reviews are badly out of date. I did ask them to put a similar note on all out of date reviews but, funnily enough, they said no. It was shortly after this note was put on the CBT review that the CEO Mark Wilson left Cochrane abruptly, but I think he stepped down mainly because the NIHR stopped funding UK Cochrane Groups on his watch. I will dig out the correspondence as I don't think I posted it on my blog. But this is the letter from MW sent to me days before he left Cochrane which mentions the note
https://acrobat.adobe.com/id/urn:aaid:sc:EU:03ca6e6b-ffe8-4e28-aa7b-74ec15f19459

 

[Medfeb]

I have an October 2019 version and am seeing the same thing as @MCFS

I also have a 2019 version that does not include the editorial note with a link to the update process, even though it includes all the rest of the version history, all the way back to 2004. The latest update mentioned is this:

View attachment 24631

I have an Oct 3, 2019 PDF of the 2019 review and am seeing the same thing as @ME/CFS Skeptic - no editorial note and the most recent amendment is dated August 8, 2019.

The citation in the Oct 2019 PDF is listed as follows, which is the same as in the current PDF of the 2019 review (here).

"Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2019, Issue 10. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub8."​

But the publication dates in these two PDFs are not the same:
My 2019 version - "Publication status and date: Edited (conclusions changed), published in Issue 10, 2019."
The current version - "Publication status and date: Edited (no change to conclusions), published in Issue 3, 2021."

And the "What's New" sections are different. For the Oct 3, 2019 PFD, there are 2 entries dated August 8, 2019 while in the current version, there's just one entry dated March 18, 2021. From August 8, 2019 to March 18, 2021 inclusive, there are 8 distinct dates of amendments. So does this suggest they are releasing a different version of the 2019 review for at least some of these different dates?

As an aside, the online version of the publication history (here) does not include the August 8, 2019 amendments. An error in maintaining the revision history for online purposes? the format is not the same as in the PDF so it looks like something being maintained separately

 

[ME/CFS Skeptic]

As an aside, the online version of the publication history (here) does not include the August 8, 2019 amendments. An error in maintaining the revision history for online purposes? the format is not the same as in the PDF so it looks like something being maintained separately

Yes I found this confusing.

For the 2017 Apr 25 update they mention revisions from 2017 May 05, all the way up to 2019 Jun 17. This seems like an error because all these changes came after the April 2017 update?

And for the 2019 Oct 02 update the revision dates go from 2020 Feb 06 to 2021 Mar 18. It doesn't mention the 8 August 2019 changes that explain why an amendment was made. That part seems to left out of the version history on the Cochrane website.

 

[ME/CFS Skeptic]

So the 2019 version that I had, that was published shortly after the amendment says (my bolding):

Publication status and date: Edited (conclusions changed), published in Issue 10, 2019.

Citation: Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2019, Issue 10. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub8

While the 2019 version that is now on the Cochrane website says (my bolding):

Publication status and date: Edited (no change to conclusions), published in Issue 3, 2021.

Citation: Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2019, Issue 10. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub8.

So It seems that they have the exact same DOI and citation but that there are (at least) two versions: one from 2019 without the editorial note and one from 2021 that has the editorial note.

In other words, they did not create a new citation and publication for the first editorial note that announced an update of the review. But they did so for the editorial note that announced this update is cancelled?

 

[Trish]

In other words, they did not create a new citation and publication for the first editorial note that announced an update of the review. But they did so for the editorial note that announced this update is cancelled?

I have tried to clarify this with Hilda Bastian on her talkpage:

trishrhymes
December 22, 2024 at 10:02 am
Hi again, Hilda,

I have a question about Cochrane’s policy that I don’t know who else to ask. For sure if I ask Cochrane they won’t reply.

This document, is Cochrane’s policy on adding editorial notes:

https://documentation.cochrane.org/egr/editorial-notes-318472364.html#Editorialnotes-Policy

It includes:

“Editorial notes are published as part of a standard publication workflow. To add an Editorial note to a published review – making no other changes to the review – use the ‘Amended’ What’s new event (no new citation)”

The ‘new’ 2024 version with the editorial note is listed in the version history as version 9 and it’s made clear on the 2019 version (version 8) that there is a new version, with a redirect to the 2024 version.

Isn’t that contrary to Cochrane policy? Surely it says that if all you’re doing is adding an editorial note, it should be added to the existing version, not lead to creation of a new version. That is important because Cochrane have published the review as a new review, which it isn’t, and it’s being cited as a new review.

The policy also says that before adding a note, prior approval has to come from the Research integrity team. I’m shocked that an integrity team would approve publication of an old review unchanged as if it were an updated review. Especially in a case like this where there are outstanding published Comments that editors have said will be addressed in a new review and outstanding complaints that have not been resolved.

Thanks,

Trish

She replied:

Hilda Bastian
December 22, 2024 at 11:25 am
Hello, Trish!

I’m replying to your specific question on that policy, without commenting on my thoughts on the policy or this recent action yet – beyond that I don’t support that action. I didn’t want to keep you waiting though.

That wording in the policy explains how to add a note without creating a new citation (version), but it does not seem to expressly rule it out, either. Arguably, the policy neither supports nor proscribes doing this. (Usual practice is another question, which I know you’re also waiting for me to respond on – my apologies again for taking longer on this.)

Hilda

 

[rvallee]

Ah, yes, the Air Bud excuse: Ain't no rule that says a dog can't play basketball. Very serious stuff. Bastian still shilling for Cochrane pretty much says everything.

 

[Trish]

I think Hilda is just pointing out the policy seems to be ambiguous.

 

[Sly Saint]

"Updating Cochrane reviews
An update of a Cochrane review must involve a search for new studies."

If any new studies are found, these must be added to the relevant section of the Cochrane review and classified as included, excluded, or ongoing studies (or ‘Studies awaiting classification’ if all reasonable efforts to classify it in one of these ways have failed), before labelling the revised Cochrane review as an update. Any other change to a Cochrane review, and any change to a protocol for a Cochrane review, is classified as an amendment.

Editorial policies | Cochrane Library

eta: this clearly didn't happen

 

[Trish]

Editorial policies | Cochrane Library

More from this link:

Post-publication peer review
There is a Comments feature present on all Cochrane reviews. All comments submitted via this channel receive a response and, if appropriate, the comment and the response from the Cochrane review author will be published. When necessary, the review will be revised and updated in response to post-publication peer review.

This has not happened with the Larun review, for example Michiel Tack's 2020 comment which detailed multiple problems with the review and which was reponded to with an editors letter saying it would be addressed as a priority in the new review - hasn't happened because the new review process has been cancelled.

Updating Cochrane reviews
An update of a Cochrane review must involve a search for new studies. If any new studies are found, these must be added to the relevant section of the Cochrane review and classified as included, excluded, or ongoing studies (or ‘Studies awaiting classification’ if all reasonable efforts to classify it in one of these ways have failed), before labelling the revised Cochrane review as an update. Any other change to a Cochrane review, and any change to a protocol for a Cochrane review, is classified as an amendment.

The 2024 and earlier listed items in the history are described as 'version', not as updates. It's very confusing.

A Cochrane review should be updated based on need. Aspects to consider are the currency of the question, the impact and usage of the current version, the availability of additional studies (or additional data for studies already included), and an assessment of the likely change of any newly identified studies or additional data on the current review version; in addition to methodological enhancements that may be required.
Authors need to complete a new licence for publication and declare relevant conflicts of interest for each update.

Nope, me neither. The total reversal of announced need for a new review to rejection of any changes and publication as a new 'version' is incomprehensible to me. It seems to be completely contrary to Cochrane's own policies.

Withdrawal of published articles
Cochrane reviews
Cochrane reviews are withdrawn when serious issues with conduct or reporting are identified.

Reasons for withdrawing a Cochrane review or protocol:

• Serious error in a Cochrane review. Following the conclusions of the published review could result in harm to patients or populations of interest (other than known adverse effects); and/or there are factual errors in describing one or more included studies that risk misinforming implementation or investment decisions about an intervention; and/or the reported treatment effect is inconsistent with the real effect shown in the reported data.

and 3 more reasons.

Withdrawing a Cochrane review generates a new version of the review with a new DOI. The withdrawn content remains accessible via the version history.

Prior to July 2019, Cochrane reviews were withdrawn for other reasons (e.g. the review no longer represented the best current evidence or it was superseded by another systematic review). Prior to March 2022, protocols were also withdrawn to inform readers that a protocol would not be progressed to the review stage. This is now indicated by the addition of an Editorial note to the protocol.
Withdrawal followed by republication
The Editors may accept for publication a revised version of a Cochrane review that addresses the issues raised in the withdrawn notice. Any such version will be re-evaluated using standard Cochrane editorial process, and may then be approved for publication (as a separate, subsequent version) by the Editor in Chief in consultation with the Network Senior Editor.

I have highlighted the basis on which we argued that the review should be withdrawn.
I don't understand the bit about withdrawal generating a new review that I've bolded.

 

[forestglip]

From that list, it looks like basically one account posted and around 20 others retweeted it from them. The original account is Physio Meets Science @PhysioMeScience

https://www.physiomeetsscience.net/

Might be worth telling them and anyone else who posts it what the deal is to see if they'll delete. Very likely they don't know it's not a new review.

Looks like at least one person has replied to their tweet to tell them.

I don't have a Twitter account anymore, but I used the contact form on their website.

They responded and basically told me to take it up with Cochrane. And it looks like their tweet is still up.

please contact Chochrane directly

https://www.cochranelibrary.com/cds....pub9/full?highlightAbstract=exercise|exercis

 

[Hutan]

I don't understand the bit about withdrawal generating a new review that I've bolded.

Withdrawing a Cochrane review generates a new version of the review with a new DOI. The withdrawn content remains accessible via the version history.

I think there's an issue with Cochrane being both a journal publisher and a 'provider of 'gold standard' medical advice'. If a review is published and then deemed to be not only not gold standard medical advice but a potential source of harm etc, it can be withdrawn as a Cochrane review, but it still exists. It's quite a mess, and I don't understand how a new DOI can be created either. Presumably the review could have been cited in other papers, so changing the DOI would stuff up those citations?

 

[Peter T]

I think there's an issue with Cochrane being both a journal publisher and a 'provider of 'gold standard' medical advice'. If a review is published and then deemed to be not only not gold standard medical advice but a potential source of harm etc, it can be withdrawn as a Cochrane review, but it still exists. It's quite a mess, and I don't understand how a new DOI can be created either. Presumably the review could have been cited in other papers, so changing the DOI would stuff up those citations?

If you go to the old citation Larun et al (2019) is still there but at the top in red (if my memory serves me) is a note to say this is not the most recent version and a link to the identical Larun et al (2024).

 

[Trish]

I think there's an issue with Cochrane being both a journal publisher and a 'provider of 'gold standard' medical advice'. If a review is published and then deemed to be not only not gold standard medical advice but a potential source of harm etc, it can be withdrawn as a Cochrane review, but it still exists. It's quite a mess, and I don't understand how a new DOI can be created either. Presumably the review could have been cited in other papers, so changing the DOI would stuff up those citations?

I wonder whether all this relatively new stuff that restricts how reviews can be withdrawn was done specifically to scupper withdrawal of the Larun review, and the bit about being replaced by a new review was to tie withdrawal to a new review being approved. Bingo, drag out a new review process, then cancel it for fake reasons, and Larun can keep on being republished every 5 years unchanged for ever.