Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al

Thought -
Interesting that the authors quoted Cochrane* i.e. in dealing with the issue of lack of objective outcome criteria in trials. I recall someone advising that it's often easier to accept the other person's evaluation criteria [Cochrane in this case] and just point out that they applied these criteria incorrectly. Here they've pointed out that Cochrane downgrades for subjective outcome criteria [bold, underlined, italic text].
Apologies for my lack of clarity!
[EDIT] - I like the fact that they've highlighted that the objective outcome data "indicated no clinically relevant improvements".
Thanks to the authors.

"
2. Blinding and subjective outcomes
White et al. write: “The NICE committee decided to downgrade all fatigue outcomes based on the premise that it is a subjective measure.” This is incorrect. Fatigue outcomes in studies with a low risk of bias such as the double-blind Rituximab trial were not downgraded.

NICE did downgrade quality of evidence when subjective outcomes (not just fatigue, but also pain, sleep, quality of life, etc.) were used in trials where participants and therapists were not blinded as this combination creates a high risk of bias. This grading is in accordance with the Cochrane Handbook which states that “the potential for bias cannot be ignored even if the outcome assessor cannot be blinded." (2) In the case of patient-reported outcome measures such as fatigue, the Cochrane Handbook considers patients to be the outcome assessor.

Moreover, trials of GET and CBT also included objective measurements such as fitness tests, actigraphy, and employment data and these indicated no clinically relevant improvements. (3)
 
Jonathan Edwards said:
I think it is a pity that NICE produced this comment:

“In developing our guideline, as well as bringing together the best available scientific evidence, we also listened to the real, lived experience and testimony of people with ME/CFS to produce a balanced guideline which has their wellbeing at its heart.”

The committee may have listened to people with ME but the guideline is based entirely on the evidence from the trials. It has no business to be "balanced". It is evidence-based.
Click to expand...

Not sure about the whole comment, but the 'balance' bit seems most problematic to me, too. Balance is not needed to have patients' wellbeing at one's heart.

Perhaps NICE was surprised that listening to well-chosen patient representatives actually helped assess the evidence, and this evidence didn't contradict much of patients' experience?

Those clinician researchers though who prefer to believe their own experience and in their own questionable research practices instead and did not listen to patients' and their allies' reasonable critique now have to learn that also by common standards their research isn't good enough to back up their beliefs.

Also, those who can't accept the rigorous assessment of their own research and beliefs have to learn that research in their area could have been so much better if they had listened to their patients' and others' critique.

If they had listened, they could have learned a while ago that only because you belief reliable objective outcome measures in your specialist field don't exist, there is no exception to the rule that using only subjective outcomes in unblinded trials is a combination that has a too high risk of bias to be useful.

They could have learned which treatment outcomes matter to people with ME and they could have tried to develop better outcome measures.


(Apologies for multiple edits.)
 
Last edited:
MSEsperanza said:
Not sure about the whole comment, but the 'balance' bit seems most problematic to me, too. Balance is not needed to have patients' wellbeing at one's heart.

Perhaps the surprise here was that listening to patients actually helped assess the evidence, and this assessment didn't contradict much of patients' experience.

Those clinician researchers though who prefer to believe their own experience and in their own questionable research practices instead and did not listen to patients' and their allies' reasonable critique now have to learn that also by common standards their research isn't good enough to back up their beliefs.

Also those who can't accept that assessment of their own research and beliefs have to learn that research in their area could have been so much better if they had listened to their patients' and others' critique.

If they had listened, they could have learned a while ago that only because you belief reliable objective outcome meausre don't exist, their isn't an exception to the rule that subjective outcomes in unblinded trials usually have a too high risk of bias to be useful.

They could have learned which outcomes matter to people with ME and they could have developed better outcome measures.
Click to expand...
The irony is that the whole BPS approach relies on believing patients if they say they feel a bit better following therapy, but dismissing us as unreliable witnesses to our own illness experiences in almost all other contexts.
 
Robert 1973 said:
The irony is that the whole BPS approach relies on believing patients if they say they feel a bit better following therapy, but dismissing us as unreliable witnesses to our own illness experiences in almost all other contexts.
Click to expand...
Even with the huge imbalance between both groups. They are choosing to harm most of us in order to feel like they helped a few, most of which would have likely improved on their own anyway. Usually the concern over first do no harm is that it's even unacceptable to harm a few in order to help most, or the balance has to be extreme.

And at the very best the "helping" part can be defined as 1/7 have some form of subjective benefit with no objective evidence. It's so grotesque that people simply don't believe it when we expose the simple facts. This has all been extremely immoral.
 
Trial By Error: Rapid Response to Anti-NICE Whine de Coeur; UK Government Seeks Input to Interim Delivery Plan for ME/CFS

"The Journal of Neurology, Neurosurgery, and Psychiatry (JNNP) has finally published a cogent rapid response to its recent whine de coeur from the PACE authors and their cronies. In the commentary, the co-authors criticized eight purported “anomalies” they believe occurred during the process of developing the 2021 ME/CFS guideline from Britain’s National Institute for Health and Care Excellence (NICE). Although a number of rapid responses are known to have been sent, this is the first one JNNP has decided to share

More accurately, it’s the first except for the rapid response from MEAction UK that was shared by JNNP–and then quickly un-shared. (I wrote about that here.) In a statement, the journal attributed the decision to disappear the rapid response to “inappropriate inflammatory language” that had apparently not been inflammatory enough to prevent the initial posting. At least TheBMJ.com ran a news item about the commentary–JNNP is one of the journals under the BMJ publishing umbrella—and then published a response to the news report from Dr Charles Shepherd, medical advisor to the ME Association."

https://virology.ws/2023/08/14/tria...ks-input-to-interim-delivery-plan-for-me-cfs/
 
Andy said:
I wonder if there is any value in submitting those rapid responses that weren't printed by the JNNP to the BMJ as a response to the news article?
Click to expand...
I have been considering doing so with mine. It would require adapting, and I've had too much else on my plate to do anything about it. I hope the organisations whose responses were rejected by the JNNP will submit them as responses to the BMJ article.
 
Tom Kindlon said:
New, non-discussion thread I have just set up:

(No discussion) E-letters submitted to JNNP replying to White et al. "Anomalies in the review process & interpretation of the evidence in the NICE..."
https://www.s4me.info/threads/no-di...rpretation-of-the-evidence-in-the-nice.34704/

FYI @rvallee @Joan Crawford
Click to expand...
@rvallee has now posted his e-letter here
https://www.s4me.info/threads/no-di...f-the-evidence-in-the-nice.34704/#post-489350 which disappointingly has not been posted by the JNNP on their website
 
It's an excellent letter, @rvallee. Thank you for trying. There must be a way readers of the article can be alerted to all the responses that haven't been published.
Maybe some tweeting by pwME with big followings linking the thread with the letters. Maybe @dave30th can do an article. Maybe someone with clout could do a letter to BMJ.
 
Trish said:
It's an excellent letter, @rvallee. Thank you for trying. There must be a way readers of the article can be alerted to all the responses that haven't been published.
Maybe some tweeting by pwME with big followings linking the thread with the letters. Maybe @dave30th can do an article. Maybe someone with clout could do a letter to BMJ.
Click to expand...
With 12,900 odd followers I’ve quite a big following and have shared:
https://x.com/tomkindlon/status/1689656010045263878?s=46&t=fB55_-omyDIDREBokX-E6Q
https://x.com/tomkindlon/status/1691576814815052135?s=46&t=fB55_-omyDIDREBokX-E6Q
Click to expand...
I saw last week the @mecfsnews account also linked to the thread. The more, the merrier
 
I just went to look for the PubPeer link where people can post post publication comments and found this one by Peter White (a different Peter White to the one who was lead author of the anomalies paper).

It starts:

The following points were submitted in a short Rapid Response but not published. The JNNP has stated that they took-down Rapid Reponses because of ‘inappropriate inflammatory language’. This reply includes more detail than the Rapid Response, but is similar in tone.

In Brief:
• White et al’s assertion that Post-Exertional Malaise is a new requirement in the 2021 NICE guideline is false. Post-Exertional Malaise was required in the previous 2007 NICE guideline.
• White et al’s assertion that, in trials, Graded Exercise Therapy was “only increased as the patient feels able, dependent on their symptomatic response”, is also false, see Moss-Morris (2005).
• There is a systemic under-reporting and dismissal of patient reports of harm across all medical fields. Dismissal of reports of harm repeats mistakes of other devastating medical scandals as evidenced by the UK Govt ‘Cumberledge Review’ published in 2020.


Continues at:

https://pubpeer.com/publications/95FD14B92656A4AD5498AD9928930C
 
PubMed Commons was a great resource for posting such comments. Also comments were prominent as they were on the PubMed website. I spent quite a bit of time over an extended period posting over 100 of my e-letters only for the project to be scrapped after a few years as not enough were posted. PubPeer did plan to copy across the responses (I discovered this after I had already copied across a lot of mine).
 
Joan Crawford said:
And here is my rapid response submission:

Dear Editor,

Contrary to the claims made in the published review article, ‘Anomalies in the review process and interpretation of the evidence in the NICE guideline for chronic fatigue syndrome and myalgic encephalomyelitis’ [1], NICE have rejected these in their entirety [2]. The article makes multiple allegations. This includes the opinion that the NICE process deviated from standard process and was, as a result, unexpected and/or unacceptable to the authors in its findings. The authors, whose extraordinarily long list of conflicts of interests, include being the developers, testers of their own clinical trials/therapies that have been deemed to be low quality by the NICE evidence review [3] dislike robust and thorough scrutiny. Sadly, the authors have failed to provide objective, robust evidence to back up their assertions. Neither did the authors demonstrate how they have counteracted their bias regarding their own theories, hypotheses, and resulting low quality evidence due to the use of well-known, flawed, poor-quality methods resulting in a high risk of expectation bias by relying largely on subjective outcome measures in clinical trials. [3] Due to the very nature of the phenomenon being assessed, such trials cannot be blinded. The complexity and severity of the condition required consideration by NICE to strengthen the scientific process, which is evidence of their determination to produce a robust guideline following the failure of the 2007 guideline.[4]

The article failed to inform the editor, reviewers, and readers about the pause in publication by NICE due to the challenge by a group of professionals, including some of the authors of this paper, prior to the proposed release of guidance in August 2021. During this pause, a round table was convened whereby evidence could have been presented to NICE which, if this passed a quality threshold, would have resulted in a reconsideration of findings. For the challenging group, who include co-authors of this paper, the outcome was that no evidence was presented, and this absence of evidence was agreed upon by attendees. There was nothing of substance on offer, as there is not in the article by White et al. now either. The NICE guideline was then published unaltered in October 2021, [5] after an unnecessary two-month delay.

The authors’ focus suggests that they believe NICE are exceptional in their assessment of the published evidence or are out-of-sync with other international institutions. This is troubling as the first reference that they use is the Institute of Medicine 2015 report [6]. After extensive review evaluating the evidence for the diagnostic criteria for ME/CFS, concluding that, “Patient reports and symptom surveys as well as scientific evidence consistently showed that impaired function, PEM (an exacerbation of some or all of an individual's ME/CFS symptoms after physical or cognitive exertion, or orthostatic stress that leads to a reduction in functional ability), and unrefreshing sleep are characteristic symptoms almost universally present in ME/CFS; thus, the committee considered them to be core symptoms.” Thus, it would be remiss of NICE to not incorporate this into their review process. Demonstrating that NICE did not arbitrarily redefine ME/CFS as suggested. It should be noted that PEM was also required in the 2007 NICE guidelines (CG53 2007, p14) for diagnosis, “characterised by post-exertional malaise and/or fatigue (typically delayed for example by at least 24 hours, with slow recovering over several days).” Also, p18, “The diagnosis of CFS/ME should, be reconsidered if none of the following key features are present: post exertional fatigue or malaise.” [5] The authors must have known this.

Word count 576 (excluding references).

References
1. White P, et al. J Neurol Neurosurg Psychiatry 2023;0:1–8. doi:10.1136/jnnp-2022-330463
2. ME/CFS guidance that discourages exercise is flawed, say researchers Available: https://www.theguardian.com/society...scouraging-exercise-is-flawed-say-researchers The Guardian newspaper.
3. Edwards, J. (2021). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: diagnosis and management. NICE guideline NG206. Appendix 3: Expert testimonies The difficulties of conducting intervention trials for the treatment of myalgic encephalomyelitis/chronic fatigue syndrome. Available: https://www.nice.org.uk/guidance/ng206/evidence/appendix-3-expert-testimonies-pdf-333546588760
4. National Institute for Health and Care Excellence. Myalgic Encephalomyelitis (or Encephalopathy)/Chronic fatigue syndrome: diagnosis and management. NICE guideline [NG206], 2021. Available: https://www.nice.org.uk/guidance/ng206
5. National Institute for Health and Care Excellence. Chronic fatigue syndrome/
myalgic encephalomyelitis (or encephalopathy) Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy): diagnosis and management of CFS/ME in adults and children NICE guideline [CG 53], 2007. Available: https://sacfs.asn.au/download/CG53NICEGuideline.pdf
6. Institute of Medicine. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: redefining an illness. Washington, DC: The National Academies Press, 2015.
Click to expand...
 
You must log in or register to reply here.
Back
Top Bottom