The Cochrane correspondence
The Cochrane review on exercise therapy for patients suffering from chronic fatigue syndrome has been the subject of ongoing controversy. In October 2018, a Reuters news article suggested that Cochrane would temporarily withdraw the review due to pressure and complaints from online activists. Publically released correspondence between Cochrane’s former Editor in Chief and senior employees of the Norwegian Institute of Public Health offers a different perspective to the dispute.
A contested review
Cochrane is an international organization renowned for its systematic reviews of medical research, commonly regarded as the gold standard in evidence-based healthcare. [1] Its review on graded exercise therapy (GET) for patients suffering from chronic fatigue syndrome (CFS) however, is controversial and contested. [2]
GET instructs patients to gradually increase the duration and intensity of exercise beyond their perceived limit to reverse deconditioning. [3] CFS patients, however, report a marked deterioration in health when they exceed a certain activity level, a phenomenon commonly referred to as post-exertional malaise. [4] The Cochrane review by Larun et al. suggests that GET is a safe and effective treatment for CFS [5], although its methodology and conclusions have been widely criticized. [6]
Following a formal complaint about the review to the Editorial Board, Cochrane decided to conduct an internal audit to assess the validity of the critique. In October 2018, a Reuters news article reported that Cochrane would temporarily withdraw the contested review, a decision that was deemed “disproportionate and poorly justified” by the authors. [7] Publically released correspondence between Cochrane’s former Editor in Chief David Tovey and senior employees at the Norwegian Institute of Public Health (NIPH), provides a different perspective to the controversy. NIPH hosts Cochrane Norway and employs two of the authors of the Cochrane review in question.
Justified criticism
The email exchange confirms that the review suffered from multiple methodological faults. Cochrane has added a notification to the review, claiming it is “substantially out of date and in need of updating.” [8] In the correspondence, Tovey stressed that the authors failed to address the criticism of their review. An independent assessment of submitted comments indicated that these were cogent and that “there was a case to answer.” [9] The authors were given the opportunity to address the errors, but their resubmission was considered inadequate by multiple reviewers. As Tovey explained:
“I asked my internal team to review the submission, and also three senior colleagues who I believe have the expertise and detachment required to provide an objective evaluation. As the attached documents show, all of these assessors continue to have important concerns […].” [10]
Larun et al. previously faced similar problems with a related Cochrane review that intended to use individual patient data (IPD) instead of aggregate outcomes on GET for CFS. The IPD manuscript also received negative reviews and had to be resubmitted. Again, multiple experts indicated that Larun et al. failed to grasp the methodological issues raised. Tovey announced Cochrane’s decision to cancel the IPD review as follows:
“[…] we were disappointed when the peer reviewers of the re-submitted review all agreed that the authors had not adequately addressed the methodological issues. We therefore decided that we had no option but to reject the IPD review.” [9]
Bias and activism
In total, Larun et al. erred on 5 different occasions while writing these two Cochrane reviews on GET for CFS. The email correspondence suggests the authors might have a bias in working towards a favorable conclusion. In an email of 25 October, Tovey wrote that “the overall tone of the conclusions appears to us to be overly optimistic.” [11] In another, he described the authors’ intention as “stretching the truth beyond its limit.” [12] The main author of the review, Lillebeth Larun has researched and promoted the use of exercise therapy for CFS before working on the Cochrane review. [13, 14] Another author, Kjetil Gundro Brurberg has since its publication been an outspoken defender of the treatment, claiming that “in the absence of evidence of efficacy for other interventions, health professionals should offer patients with CFS/ME the opportunity to try […] GET.” [15] Brurgberg wrote that some of the criticism towards the evidence for GET may be “counterproductive” and “discourage further research.” [15] Similar sentiments were expressed by Larun. In the email exchange, Tovey noted “that Lillebeth, and the trialists of the included reviews, tend to refer to all criticism of the trials and the review as being the result of ‘activism’.“ [9] Cochrane disagreed and emphasized that the criticism was “thoughtful and reasonable.“ [9] In an interview with Reuters, however, Larun expressed concern that the activists who have plagued her for years have now got to her editors. [16]
Leaked emails
In an email dated November 2, 2018, the Acting Director General of NIPH, Trygve Ottersen, advised Cochrane not to give in to the criticism:
“[…] on a controversial topic as CFS/ME, a continuous stream of criticism is to be expected. We hope however that the Cochrane editorial office will stay committed to Cochrane’s standard procedures also in this case […].” [17]
Shortly before that, internal emails were leaked to the press. On October 17, Reuters reported that Cochrane would temporarily withdraw the review “amid fierce criticism and pressure from activists and patients.”[7] It is unclear who leaked the information but the tone of the article aligns with the authors’ suggestion that Cochrane is conceding to unfounded but fierce criticism from the CFS community. Despite Tovey’s caution that “this not about patient pressure”, the article strongly suggests otherwise. Colin Blakemore, a former chief executive of Britain’s Medical Research Council, is quoted saying that the decision “set a worrying precedent for scientific evidence being over-ridden by the opinions of activists.” The Reuters article was taken over by multiple media outlets [18] and provoked anger and concern on social media. Clare Gerada, former chairperson of the Council of the Royal College of General Practitioners, commented on Twitter that “Cochrane has lost its way” [19] and that “we are entering an era of bullying scientists when lobby groups disagree with the results.” [20]
A second resubmission
The publically released email correspondence indicates such criticism to be unfounded. Concerns about the review were raised by multiple experts. “When a collection of experienced and dispassionate colleagues are all making, in effect, the same criticisms”, Tovey wrote, “it is hard to ignore this.” [21] It is unclear if the leaked emails and public criticism impacted Cochrane’s handling of the matter. The review on GET for CFS, however, has not been withdrawn. On June 17, 2019, Cochrane announced that it has received the second resubmission of Larun et al. The amended review will be published during the next 2 months. [8]
[EDIT: This post has been updated to correct spelling mistakes.]
Great post, thanks, Michiel.
A common orthographic error made by many native speakers, including Michael Sharpe, is 'publically' rather than the correct 'publicly'.
And being former head of the RCGP explains a lot about how the lousy NHS internal training sourced from the "rousing reassurance" FINE trial happened. Also why, despite there being no evidence for it, the NHS qualifies ME as somatoform in practice and internal documentation, in contradiction with the NICE guidelines, which hint at it but at least refrain from making unevidenced statements.
Oh yes Aunty Clare’s pep talks drink fewer pints and get off the bus s stop earlier. The role play is so fake. You can tell it’s not a genuine ME patient they haven’t got a print out of any research they found online