The S4ME committee has today submitted the following set of complaints to the Cochrane official complaints process.
For discussion, go to this thread:
2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles
It's long so has been split over two posts.
___________________________
Complaints about republishing an outdated and flawed review as if it were a new review, and the failure to withdraw it.
These complaints refer to the review Exercise Therapy for CFS by Larun et al, 2019 and 2024 versions (the Review). [1,2]
In each case, we set out our contention that Cochrane has acted wrongly, demonstrating failure to follow its own processes, guidance and precedents, and has acted with severe misjudgement.
Complaint 2025-1: Creation of a new citation and publication date as a result of attaching an editorial note to an unchanged review
Complaint 2025-2: Publication of a review as a new version without critical comments being addressed
Complaint 2025-3: Misleading labelling of the unaddressed critical comments as applying to 'a previous version'
Complaint 2025-4: The appearance but not the substance of an update
Complaint 2025-5: Questioning Cochrane's decision making. What was the real reason behind Cochrane's actions on this review?
____________
We quote in full the editorial note attached to the 2024 (version 9) version of the Review, the Review being identical in all respects, other than the editorial note, to the 2019 (version 8) Review.
"Editorial note (19 December 2024; amended 31 January 2025):
Larun L, Brurberg KG, Odgaard‐Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2019, Issue 10. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub8. Accessed 18 December 2024.
This Editorial Note is for the above article, published online on 2 October 2019 on the Cochrane Library (
https://www.cochranelibrary.com/), and has been issued by the Publisher, John Wiley & Sons Ltd, in agreement with the Cochrane Collaboration. The Editorial note has been agreed to inform readers that Cochrane is ceasing the production of a full update of this Cochrane review. A pilot project for engaging interest holders in the development of this Cochrane review was initiated on 2 October 2019 (see Editorial Note below) and has now been disbanded. Cochrane maintains its decision to publish this Cochrane review in 2019, which includes studies from searches up to 9 May 2014." [2]
____________
Complaint 2025-1: Creation of a new citation and publication date as a result of attaching an editorial note to an unchanged review
The first part of the editorial note says:
"This Editorial Note is for the above article, published online on 2 October 2019 on the Cochrane Library (
https://www.cochranelibrary.com/), and has been issued by the Publisher, John Wiley & Sons Ltd, in agreement with the Cochrane Collaboration." [2]
We understand this to mean that the publisher agreed that the note be attached to the 2019 version of the Review, that is version 8. They have not done so. There is no indication in the editorial note that the publisher agreed to republish the Review as a new version with a new publication date and version number, and with the only change being to add the editorial note.
Quoting from Cochrane's policy on editorial notes:
"Specifications for publishing and display
Editorial notes are published as part of a standard publication workflow.
To add an Editorial note to a published review – making no other changes to the review – use the 'Amended' What's new event (no new citation)." [3]
Therefore Cochrane has failed to follow its own policy by republishing the 2019 version 8 of the Review as a new 2024 version 9, creating a new citation, instead of attaching the editorial note to version 8 and leaving the citation unchanged. Further, it has failed to follow the agreement with the publisher that the note is to be attached to the 2019 version.
The only reasons we could find in Cochrane's editorial policy for creating a new version with a new date and a new citation, were the creation of an updated review, with all the requirements of an up to date literature search and up to date methodology, as we detail below in complaint 4A, the withdrawal of a review, or the creation of an update following a withdrawal. Clearly none of these applies in this case. [4]
The effect of the 2024 date in the new citation is to suggest to readers that the Review represents the state of knowledge in 2024, when in fact the search for studies to be included was carried out in 2014 and Cochrane has acknowledged that review methodology has improved since the work on the review was done.
________________
Complaint 2025-2: Publication as a new version without the critical comments being addressed
Previous versions of the Review by Larun et al, since the original 2015 version, have all been the result of substantive amendments following feedback and required editorial scrutiny and approval before the new version was published. Indeed the 2019 version required multiple editorial interventions, and the previous editor in chief, David Tovey, indicated that it should be withdrawn.
Following publication in 2019, several substantive critical comments raised further issues with the 2019 version, notably that by Michiel Tack in 2020. [5]
A senior Cochrane editor, John Hilton, responded to Tack's comment:
" Note on the status of this review:
Response from the editorial team at the Cochrane Editorial and Methods Department:
"We thank Michiel Tack for the feedback on this review, and we appreciate the thorough and well-referenced comment. This review and its protocol are in the process of being updated as a priority... We will ensure that the comments you have provided will be reviewed during the updating process by the independent advisory group and the authors of the review." [6]
Since the 2024 version has not addressed any of the issues raised by Tack and others, the normal Cochrane process of editorial scrutiny and amendment has not happened. It is clear from Hilton's comment that publication of a new version of the review was intended to take into account the critical comments of Tack and others.
We note that Tack has drawn attention to this problem in his recent comment: "Comments remained unaddressed for years" [7]
The effect of publication without consideration of the comments is that the review continues to have possible identified flaws. This and the long delays in responding to comments also indicates that Cochrane has problems with the functioning of its quality control system, suggesting that there may be a wider problem with the accuracy of the information Cochrane endorses.
____________
Complaint 2025-3: Misleading labelling of the unaddressed critical comments as applying to 'a previous version'
We note that all the comments that had been submitted addressing issues in the 2019 version are now attached to the 2024 version and labelled with the note:
"Note: This comment relates to a previous version of this review: https://doi.org/10.1002/14651858.CD003200.pub8"
Given that the Review is unchanged and there is no evidence that the authors or editors have addressed any of the comments, they clearly apply equally to the 2024 version, as Michiel Tack explains in his comment [7]. The comments apply until the issues they raise have been considered, any necessary action taken and the commenter informed.
The effect of Cochrane noting that these comments do not apply to the latest (but unchanged) version is that Cochrane appears to be dishonestly suggesting to readers of the 2024 version that the comments were considered and any necessary action has already been taken.
____________________
Complaint 2025-4: The appearance but not the substance of an update
We quote from Cochrane's editorial policies:
"Updating Cochrane reviews
An update of a Cochrane review must involve a search for new studies. If any new studies are found, these must be added to the relevant section of the Cochrane review and classified as included, excluded, or ongoing studies (or ‘Studies awaiting classification’ if all reasonable efforts to classify it in one of these ways have failed), before labelling the revised Cochrane review as an update. Any other change to a Cochrane review, and any change to a protocol for a Cochrane review, is classified as an amendment.
"A Cochrane review should be updated based on need. Aspects to consider are the currency of the question, the impact and usage of the current version, the availability of additional studies (or additional data for studies already included), and an assessment of the likely change of any newly identified studies or additional data on the current review version; in addition to methodological enhancements that may be required.
"Updated Cochrane reviews acknowledge and cite the previous versions of the same Cochrane review.
"Authors need to complete a new licence for publication and declare relevant conflicts of interest for each update." [4]
None of the above requirements for an update have been met in the 2024 version [2], yet the publication date is suggestive of a recent update. The editorial note says:
"The Editorial note has been agreed to inform readers that Cochrane is ceasing the production of a
full update of this Cochrane review". [2]
This leaves open the possibility that this version is instead a partial update. There is nothing to make clear that this is the same document that Cochrane itself recognised as inadequate and requiring replacement in 2019. [9]
The 2024 replicate of the 2019 version obviously has not considered new relevant research and evidence. It has not applied modern review methodology standards. It has not corrected the flaws. And yet it, to most readers, it will appear to be a recent synthesis of the evidence.
In the 2016 paper, "When and how to update systematic reviews: consensus and checklist", Cochrane experts led by Professor Paul Garner, and including the current editor-in-chief, Dr Soares-Weiser, state:
"Newly identified studies can change the conclusion of a review. If they have not been included, this threatens the validity of the review, and, at worst, means the review could mislead... Indeed, there is probably added value to updating a review, because this will include taking into account comments and criticisms, and adoption of new methods in an iterative process" [13]
A careful consideration of new studies, comments, criticisms and methods would have substantially improved the quality of the Review. It surely runs against Cochrane's stated values to have created the appearance of an update, while nothing at all has changed.
Part A: New research missed
Cochrane's unsigned statement dated 16th December 2024 suggested that there is insufficient new research in the field to warrant an update (or indeed a new review to replace it). [8]
We question the basis for this claim of no new research, since we understand that in cancelling the replacement review project, Cochrane also rejected the draft protocol. If there was no agreed protocol in place with specifications of the research to be included, how can Cochrane editors claim there is no relevant new research?
Actually, to the contrary, there has been new research since the Review searched for studies in 2014, as well as accumulating evidence of harm.
For example, a graded exercise therapy and graded activity therapy study was done in children and young people, the MAGENTA study [10]. It was completed in 2019 and finally published in 2024, after the NICE ME/CFS Guideline process was complete. This study collected and published data on objective outcomes including activity monitoring using wearable technology. The study concluded that there was 'very limited improvement in either study group evident by the 6-month or 12-month assessment points'. There was no evidence of a clinically important increase in physical function or improved school attendance.
Some people have dismissed the MAGENTA study, noting that the Review is only of adult studies and so studies of young people are irrelevant. There is no reason to think that exercise therapy would be any more effective for adults with ME/CFS than for the younger people in this study. In fact, it could be expected that young people would respond better to exercise therapy, given some evidence for better natural recovery rates and the fact that they have their parents to assist with compliance with the programme. Professor Crawley, probably the leading pediatric clinician promoting exercise therapy for children with ME/CFS at the time, ran the MAGENTA study. In a presentation in 2016, [14] she noted that "this is different to adult chronic fatigue syndrome because children are much more likely to recover than adults". At that time, she clearly believed that the therapy was helping her patients. And yet, in a study with objective outcomes, it was not. Professor Crawley retired from medical practice just before the paper on the study was published.
As well as not finding a significant improvement from baseline, the MAGENTA study documented objective and long term physical deterioration and a reduction in the numbers of participants reporting school attendance; 27% of the participants in the graded exercise therapy arm were assessed as having suffered a clinically significant deterioration. The hospital admission of one participant due to suicidal ideation was acknowledged to be possibly related to the exercise therapy. The MAGENTA study provides objective trial-quality corroboration of the reports of harm from other sources as documented in Michiel Tack's 2020 comment [5]
, by the Science for ME committee in March 2024 [11] and in the many testimonies of signatories to a petition started in 2023. [12].
The Review does not warn that its guidance is harmful if applied to children and young people. We know that the Review is being used to support the application of exercise therapy to this group. For example, the Australian organisation of family doctors (RACGP) promotes the use of Graded Exercise Therapy for 'people with CFS/ME', not only adults, citing the Review. [15]
Since 2014, there have been many studies of Chinese exercise therapies such as tai chi and other forms of qigong. For example there is a review of 13 trials of tai chi and qigong [15]. It is odd that Cochrane, which claims global expertise in the identification of relevant studies, missed these. These Chinese studies have the same serious trial design flaws as the European studies included in the Review.
An effect of suggesting that there is insufficient new evidence for an update, and then appearing to update the Review but not taking into account the new evidence that actually is available is that readers will not benefit from recent evidence. Young people, and people in general will be subjected to therapy that does not work and that can harm, and clinicians and governments will not be advised that these treatments do not work. Patients and clinicians will continue to be confused by contradictory conclusions, with later work done by organisations such as NICE [19] and the CDC [18] finding these treatments do not work. Cochrane's reputation as a provider of unbiased and up to date evidence will suffer.
Another effect flows from Cochrane's tacit acceptance of poor trial methodologies. A meaningful update was an opportunity to highlight the widespread flawed methodology used in these studies (notably subjective outcomes in unblinded trials) that make findings of benefits unreliable, and call for the quality of studies of exercise therapy to be improved. The opportunity was missed.
Part B: Outdated review methods
When Cochrane's Editor-in-Chief, Karla Soares-Weiser, committed to a replacement review in 2019, she noted
that the Review was "still based on a set of methods from 2002" and that
"a new approach to the publication of evidence in this area is needed" for this "globally important health topic". [9]
Five years later, years that include a pandemic that is leaving large numbers of people with post-Covid-19 ME/CFS, the topic that was globally important apparently now does not warrant any effort to ensure Cochrane's endorsed presentation of the evidence is sound.
The Review has adverse events as a second primary outcome, but interpreted this in the narrowest possible way. It failed to seek evidence beyond the eight included clinical trials, only one of which attempted to investigate adverse events, despite Cochrane's reviewer guidelines specifying wider search for harms data. [17]
Objective outcomes for which data was available in the included trials was ignored, and long term outcomes not included.
The Review, now dated 2024, includes these statements under the heading:
"Agreements and disagreements with other studies or reviews
...
The revised version offers major additions and changes. In line with recent updates provided in the
Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011c), we have implemented several methodological improvements,..." [1,2]
This edition of the handbook is out of date. We note that the version we accessed in March 2024:
"specifies a more detailed analysis of adverse effects in cases such as this where the potential for harm has a major influence on treatment and policy decisions. In such cases it is expected that a bespoke search process will be undertaken that includes a wider selection of sources than that required to identify data on beneficial outcomes". [11, 17]
Despite specifying adverse outcomes (harms) as one of only two primary outcomes, the Review failed to follow this guidance.
The effect of the various outdated methodologies is to produce a 2024 dated review that reflects poorly on Cochrane and results in inaccurate conclusions. It is a failure of procedure and a serious misjudgement.
________________
continued in the next post...
