Anomalies in the review process and interpretation of the evidence in the NICE guideline for (CFS & ME), 2023, White et al

Last edited:
Sly Saint said:
they keep moving all the PACE stuff
Trying to keep the latest link up to date on this thread
https://www.s4me.info/threads/the-pace-trial.22088/
Click to expand...

A tangent to this thread, but did anyone ever manage to access the PACE data from the sharing platform where it was supposedly accessible.

[added - see our thread on the PACE trial data https://www.s4me.info/threads/pace-trial-data.2337/page-9#post-483591 though it seems no one has reported successfully accessing the data there either.]
 
Last edited:
Dolphin said:
I note that this journal takes e-letters:
https://jnnp.bmj.com/content/early/2023/07/09/jnnp-2022-330463.responses

Such e-letters don't have to be perfect but best to do them in scientific format with numbered references underneath. Even just one or two references underneath might be sufficient. One easy one is
1. White P, Abbey S, Angus B, et al. Anomalies in the review process and interpretation of the evidence in the NICE guideline for chronic fatigue syndrome and myalgic encephalomyelitisJournal of Neurology, Neurosurgery & Psychiatry Published Online First: 10 July 2023. doi: 10.1136/jnnp-2022-330463


Not sure whether it publishes any letters formally: there is no mention here of any guidance on this topic:
https://jnnp.bmj.com/pages/authors/#submission_guidelines
Click to expand...
I've seen someone say in a private conversation that e-letters to the Journal of Neurology, Neurosurgery & Psychiatry should be kept under 600 words because this is a BMJ Journal. That's not correct. See for example the second last e-letter published https://jnnp.bmj.com/content/early/...-revision-to-functional-neurological-disorder. The BMJ itself does have a 600-word limit. But different BMJ journals have their own rules as can be seen by submission guidelines in general.
 
White and colleagues said:
NICE’s own reviewers found only one study that tested the diagnostic utility of individual symptoms, which stated that PEM had a sensitivity of 0.50 and specificity of 0.57, that is, low.
Click to expand...

This seems to be another sentence that sounds like a logical argument is being made but then falls apart when examined more closely.

White cited this to argue against PEM being considered a required symptom for diagnosing ME/CFS.

One would assume that White cited it because it contains data on the diagnostic sensitivity and specificity of PEM. But the study examined whether certain symptoms in people with a CFS-like illness predicted a later diagnosis of CFS. This is what the sensitivity specificity values refer to (unless I've misunderstood). It says nothing at all about how important PEM is in people that have ME/CFS. All it seems to say is that having PEM and a CFS-like illness is a poor indicator that the person will meet CFS diagnostic criteria 10 years later.

One could dig deeper into this study to see how PEM was defined and assessed and whether that was good enough. The study author was L. Jason who I consider to be a person that favors somewhat unspecific definitions of PEM. But there's no need to dig deeper.

A description of the study (copied from NICE documents):

During ‘wave 1’ of the study, people who
screened positive for CFS-like illness
received a series of baseline ‘index tests’
via a structured psychiatric interview,
medical history interview and complete
medical examination.

During ‘wave 2’, 10 years later, they were
reassessed and categorised as CFS,
idiopathic chronic fatigue, exclusions or
controls by a team of physicians – this
diagnosis was the ‘reference standard’.
Click to expand...

Edit: should have said "it says nothing at all about whether PEM is core symptom of ME/CFS. People that don't yet have ME/CFS, unsurprisingly, don't yet have all of the symptoms. And people who had ME/CFS can improve over time and no longer meet diagnostic criteria."
 
Last edited:
Hoopoe said:
This seems to be another sentence that sounds like a logical argument is being made but then falls apart when examined more closely.

White cited this to argue against PEM being considered a required symptom for diagnosing ME/CFS.

One would assume that White cited it because it contains data on the diagnostic sensitivity and specificity of PEM. But the study examined whether certain symptoms in people with a CFS-like illness predicted a later diagnosis of CFS. This is what the sensitivity specificity values refer to (unless I've misunderstood). It says nothing at all about how important PEM is in people that have ME/CFS. All it seems to say is that having PEM and a CFS-like illness is a poor indicator that the person will meet CFS diagnostic criteria 10 years later.

One could dig deeper into this study to see how PEM was defined and assessed and whether that was good enough. The study author was L. Jason who I consider to be a person that favors somewhat unspecific definitions of PEM.

A description of the study (copied from NICE documents):
Click to expand...
Hopefully there will be lots of e-letters picking the commentary apart. Some of the authors might feel some embarrassment for trusting Peter White and being a co-author of the piece.

People could send in more than one as I have done with e-letters in the past.
 
White and colleagues said:
NICE’s own reviewers found only one study that tested the diagnostic utility of individual symptoms, which stated that PEM had a sensitivity of 0.50 and specificity of 0.57, that is, low.
As NICE concluded: ‘The (established) diagnostic criteria have not been evaluated in terms of their measurement validity and accuracy in diagnosing ME/CFS.’15 This equally applies to the newly proposed NICE diagnosis.
Click to expand...

Actual quote from NICE: https://www.nice.org.uk/guidance/ng206/evidence/d-identifying-and-diagnosing-mecfs-pdf-9265183025

The diagnostic criteria have not been evaluated in terms of their measurement validity and
accuracy in diagnosing ME/CFS. Without a biomarker it is not possible to definitively know if
a person has or does not have ME/CFS. Without such a reference standard (or ‘gold
standard’) it is not possible to assess the measurement validity of the different criteria
Click to expand...

White et al. agrees with NICE so far.

A bit later NICE says

There is no current gold standard for diagnosing ME/CFS so it is not possible to validate the
criteria used in different definitions. A pragmatic approach that bypasses the difficulties
concerning measurement validity is possible. If the criteria cannot, due to the lack of a
reference standard, be shown to be ‘correct’ or ‘not correct’, then the second best option is to
show that the criteria have been developed using optimal methods. This is because an
unbiased, clearly reported, evidence-based and consensus-driven process utilising the
expertise of patients, clinicians and researchers is most likely to lead to more clinically useful
criteria. This is the basis of the quality criteria used in this review. Quality was measured
using a set of quality criteria based on the AGREE II quality criteria, as described in
Appendix D
Click to expand...

So let's look at how NICE rated the quality of various diagnostic criteria. According to Google, AGREE II is widely used guideline appraisal tool.

The IOM and NICE 2007 criteria were rated as having "serious limitations". All other criteria had "very serious limitations".

Looking at bit closer, the IOM criteria "met" or partially met" slightly more quality criteria but this didn't result in a higher final grade than NICE 2007.

How the decision was made:

The committee made a consensus decision that the IOM 2015 criteria were a useful set of
criteria, having advantages over other criteria in terms of usability (see discussion in the
other factors the committee took into account) and an optimum balance of
inclusion/exclusion criterion. The committee agreed to use the IOM, 2015 criteria as a basis
for their recommendation of when to suspect someone may have ME/CFS. The criteria were
modified slightly and this is described below. The committee considered the modifications
and clarifications improved the usefulness and usability of the IOM 2015 criteria.
Click to expand...

So in summary White et al seem to agree with NICE that there is no gold standard for determining which diagnsotic criteria is most accurate, but it's presented as if it were an argument against NICE's decision. White et al seem to say nothing about the quality grading of the diagnostic criteria (presumably because they do not see a weak spot, but maybe they say something elsewhere in the article. The Fukuda criteria seem clearly outdated: the committee opinion agrees with the AGREE II guideline grading tool.
 
Last edited:
DokaGirl said:
You are right! There is no treatment for ME!
Click to expand...
The treatment for ME is consideration.

I've been bed bound since 2013. My severity is due to Consequential Progression / Deteriorative Progression, years of inappropriate housing, lack of care, and an inappropriate sectioning in 2014 where they starved me until I walked to the canteen. If I was treated differently and shown consideration I would only be house bound and still able to talk.
 
White et al said:
Most trials have used either the CDC or Oxford definitions
of CFS/ME, neither of which mandate PEM, although it is an
optional symptom of the CDC definition. 14 23 PEM is common in
populations of patients with CFS/ME. Some 85% of participants
reported PEM in eight CBT trials available for consideration, the
prevalence depending on its definition.24 An individual patient
data analysis found no moderating effect of PEM on the impact
of CBT on either fatigue or functional outcomes. 24 NICE did
undertake a sensitivity analysis to assess whether the presence
of PEM in trials affected outcomes. However, they arbitrarily
set the threshold for trials with the prevalence of PEM at >94%
of participants and did not include the data from the eight trials
mentioned above. 17
Furthermore, one trial of self-help based on the principles
of GET, which did use an illness definition that mandated
PEM, found the exercise intervention was effective in reducing
fatigue.25 Another large trial (PACE) used a sensitivity analysis
to show that using an ME definition that mandated postexer-
tional fatigue made no significant difference to the more positive
outcomes after both CBT and GET, when compared with adap-
tive pacing therapy and usual care.26 Indeed, PEM improved
more with CBT and GET compared with the comparison
treatments. 26
Click to expand...

A lot of this seems of little relevance. The important parts are: the "participants with PEM" treshold for clinical trials, and the question of whether clinical trial outcomes between participants with PEM and without PEM are different. If they are different this would support the decision to require PEM as symptom. If it makes no difference this would argue against it, but it would not be only factor to be considered.

NICE did undertake a sensitivity analysis to assess whether the presence
of PEM in trials affected outcomes. However, they arbitrarily
set the threshold for trials with the prevalence of PEM at >94%
of participants and did not include the data from the eight trials
mentioned above.
Click to expand...

NICE called this the "PEM reanalysis" and says that in the original analysis they excluded all clinical trials where PEM was not a required symptom. After stakeholder consultation, they performed another analysis with slightly less stringent rules that allowed clinical trials where ≥95% of participants met certain, slightly more flexible criteria intended to ensure they had PEM (I'm skipping the uninteresting details). This reanalysis didn't change the conclusions.

Why is White et al criticizing the less stringest PEM reanalysis but not the original plan where it was decided to exclude clinical trials where PEM was not required? I'm unable to find discussions that explain the precise rationale for that original plan by the NICE team.

Another large trial (PACE) used a sensitivity analysis
to show that using an ME definition that mandated postexer-
tional fatigue made no significant difference to the more positive
outcomes after both CBT and GET, when compared with adap-
tive pacing therapy and usual care.
Click to expand...

Sounds interesting but I'm exhausted now and can't look. This sensitivity analysis was probably discussed extensively already by forum members.
 
Lidia Thompson said:
"David Jarvis 13 July, 2023 3:07 pm
Any guideline with ME in the title is going straight in the bin anyway for lack of rigour in using a pseudoscience sounding made up name. With no proven, as yet, pathological process it is by definition a syndrome."
Click to expand...

Yes, of course the quote as a whole is totally non sequitur.
The last sentence is fine. But it in no way justifies the first, which is a bogus justification of criminal negligence.
 
Midnattsol said:
I'm pretty sure both Wyller and Flottorp also have said it will do no harm to exercise and that there is nothing wrong with patients.
Click to expand...
There's also the FINE trial that is getting a bit forgotten in this, especially as it has formed the basis for NHS training despite showing null results.

"You're fine, there's nothing wrong with you, this is the moment where you push", etc.
 
White and colleagues said:
The emphasis NICE placed on PEM is debatable.2 18 Its prev-
alence varies according to how the symptom is defined and it is
not specific to CFS/ME, being found in many conditions which
present with pathological fatigue.19–22
Click to expand...

That prevalence of PEM varies according to its definition could also mean that there is a need for more stringent diagnostic criteria and clearer definitions of PEM.

If PEM is seen in other conditions, that means it's not unique to ME/CFS. That does not contradict the idea that PEM is a hallmark symptom of ME/CFS.

Also, the 4 studies cited as evidence PEM is found in "many conditions" don't seem particularly convincing at first glance. They seem more like White defining PEM as equivalent to postexertional fatigue and then concluding that PEM is a common symptom in patients with fatigue. But I admit that I haven't looked at them in detail because I'm exhausted.
 
You must log in or register to reply here.
Back
Top Bottom