I'm thinking about submitting the following rapid response - any suggestions before I do so?
-----------------------------------------
I would like to respond to the comment by Jason Busse and colleagues as it includes some remarkable statements. The authors criticize the NICE guidance committee on myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) for employing “a disastrous misapplication of GRADE methodology.” In a draft document, the committee rated the quality of evidence for GET as low to very low.
As an example of an “appropriate” application of GRADE, Busse and colleagues refer to a contested Cochrane review on GET for ME/CFS. This review rated the quality of evidence for GET in reducing fatigue as moderate. According to Busse and colleagues, this is “sufficient in GRADE methodology to justify strong favorable recommendations.” These statements are concerning for several reasons.
The Cochrane review cited by Busse and colleagues is currently being updated, following concerns about its methodology. Cochrane’s Editor-in-Chief, Dr. Karla Soares-Weiser explained that “this amended review is still based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS from the 1990s.” [1]
The review is far from an exemplary assessment of the evidence for GET. It highlights fatigue assessments made directly after treatment ended and downplays assessments that were made several months later (the latter formed the primary outcome for the trials that provide most of the data). Follow-up results do not support the recommendation that GET reduces fatigue. In addition, the review compares GET to a passive control condition where patients received less time and attention from healthcare providers. The review also focused on subjective outcomes and ignored negative results on objective measurements such as employment figures, activity levels, and fitness tests which all tend to show no significant improvement in the GET group.
There is reasonable concern that the reduction seen on fatigue questionnaires reflects response bias rather than a genuine reduction in fatigue. Patients in the GET-group, for example, received instructions to interpret their symptoms as less threatening and more benign. According to one therapist manual on GET “participants are encouraged to see symptoms as temporary and reversible, as a result of their current physical weakness, and not as signs of progressive pathology.” Treatment manuals also included strong assertions designed to strengthen patients’ expectations of GET. One patient booklet stated: “You will experience a snowballing effect as increasing fitness leads to increasing confidence in your ability. You will have conquered CFS by your own effort and you will be back in control of your body again.”
These instructions were not given to patients in the control group and result in a high risk of response bias. It was therefore reasonable for the NICE guideline committee to rate the quality of evidence for GET as low. Other reviews had previously come to a similar conclusion. [2, 3]
If the GRADE system was used as Busse and colleagues recommend, there would be a high risk that quack treatments and various forms of pseudo-science also provide reliable evidence of effectiveness in randomized trials. All that is needed is an intervention where therapists actively manipulate how patients interpret and report their symptoms. One example should suffice to clarify this point.
Suppose an intervention based on ‘neurolinguistic programming’ where therapists assume that saying one is fatigued, reinforces neural circuits that perpetuate fatigue. The intervention consists of breaking this vicious cycle by encouraging patients to no longer see or report themselves as fatigued. This example is not that far-fetched as there are already behavioral interventions for ME/CFS that are based on similar principles. [4] According to GRADE methodology, however, such attempts to manipulate how patients report their symptoms, form no reason to downgrade the quality of evidence of randomized trials, even if fatigue questionnaires are used as the primary outcome.
The first and foremost principle of rating quality of evidence should be to understand the specifics of what is being assessed. One has to understand the intervention and the way it impacts patients. By providing a standardized checklist and algorithm to assess quality of evidence, the GRADE methodology discourages researchers from studying the details of what happens in randomized trials. The rapid response by Busse and colleagues is an example of how this approach might result in questionable treatment recommendations.
References
1. Cochrane. Publication of Cochrane Review: ‘Exercise therapy for chronic fatigue syndrome.’
https://www.cochrane.org/news/publication-cochrane-review-exercise-therapy-chronic-fatigue-syndrome. Accessed 26 Nov 2019.
2. Vink M, Vink-Niese A. Graded exercise therapy for myalgic encephalomyelitis/chronic fatigue syndrome is not effective and unsafe. Re-analysis of a Cochrane review. Health Psychol Open. 2018;5:2055102918805187.
3. Tack M, Tuller DM, Struthers C. Bias caused by reliance on patient-reported outcome measures in non-blinded randomized trials: an in-depth look at exercise therapy for chronic fatigue syndrome. Fatigue: Biomedicine, Health & Behavior. 2020;8:181–92.
4. Reme SE, Archer N, Chalder T. Experiences of young people who have undergone the Lightning Process to treat chronic fatigue syndrome/myalgic encephalomyelitis--a qualitative study. Br J Health Psychol. 2013;18:508–25.
