What's the difference between using symptoms for making a diagnosis in clinical care vs measuring a treatment effect in clinical trials? (Discussion)

[MSEsperanza]

As the title indicates.

Proponents of therapist delivered treatments like psychotherapy or exercise repeatedly rejected the argument that relying solely on subjective outcomes in open label trials to assess the effectiveness of said treatments was a bad idea.

In their criticism on the NICE guideline committee's evaluation of the evidence for benefits and harms of potential treatments for ME/CFS, Flottorp et al write [1]:

"It is uncontroversial that a diagnosis of CFS/ME rests upon subjective symptoms. But paradoxically NICE decided that evidence from clinical trials of CBT and GET showing improvement in subjective symptoms would be considered unreliable.

"Given the first premise, subjective symptoms are the most valid endpoints, and interventions improving these symptoms are treatments, not only 'symptom management'."

Since even experts in assessing evidence and methodologists responsible for developing tools to assess the risks of bias in clinical trials either use similar arguments or don't seem to see a problem with this kind of arguments, I get the impression our criticism needs to be worded more clearly.

Or perhaps their needs to be an additional discussion on the points that might be a bit challenging to deal with in clinical trials investigating treatments for symptoms that can't be objectively measured (yet)?


[1] Signe A. Flottorp, Kjetil Brurberg, Per Fink, Hans Knoop, Vegard B B Wyller (2022), New NICE guideline on chronic fatigue syndrome: more ideology than science? The Lancet, Volume 399, Issue 10325, 611 - 613, https://doi.org/10.1016/S0140-6736(22)00183-0 , Forum thread here.

 

[MSEsperanza]

My thoughts as a lay person:

1) The relevance of symptoms for making a diagnosis is different from the relevance of symptoms for measuring a treatment effect, especially in the context of clinical trials.

When patients inform their doctors about symptoms that are not only unpleasant but substantially reduce their physical and often also cognitive function, there sometimes might be reasons why patients' reports can be biased and should be doubted.

I assume that these reasons don't apply often enough to be relevant, but anyway, they are different from the reasons why patients' reports on a treatment effect can be biased.

Also, a doctor makes a diagnosis not only based on individual scores for particular symptoms, but a on a thorough clinical assessment. If there aren’t established biological markers, it's even more important that doctor and patient talk with each other to get a clear picture of each symptom and the specific pattern of symptoms.

It might be easier to have objective outcomes for assessing disability, but that’s a different topic.


2) Using specific subjective outcomes in a clinical trial is fine. If there aren't any objective measurements specific to the illness, you have to use subjective outcomes. However, in an unblinded trial, additional objective measures are needed. There are a couple of objective measures relevant to people with ME/CFS, especially actimetry and I think school/ work attendance.

Also, it should be easy to establish a cognitive test specific for the kind of cognitive fatigability and brain fog PWME experience.


3) It would be great if patients were asked which outcomes and measurements they find relevant. That applies for both subjective and objective outcomes and measurements.

For example, members of the S4ME forum wrote a thorough criticism of the Chalder Fatigue Scale. They also made suggestions for a review of the DePaul Symptoms Questionnaire that measures Post Exertional Malaise (PEM).]


Related discussion threads:

Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? -- Link

Questionnaires - design, validation and use in ME/CFS research -- Link

 

[MSEsperanza]

Apologies for being repetitive with the argument about the need for objective outcomes in unblinded trials.

Also, some of the points I'd like to discuss have been worded much better in ME/CFS Skeptic's rebuttal of Flottorp et al:

The authors argue that, since a diagnosis of ME/CFS is based on subjective symptoms, these are also the most valid endpoints in clinical trials. The NICE guideline downgraded trials on GET and CBT because subjective outcomes were used while patients nor therapists could be blinded to treatment allocation. Flottorp et al. disagree with this judgment. They argue that “absence of blinding and subjective outcomes is common in studies of non-pharmacological interventions for conditions without objective criteria, but does not imply that all such studies are biased and should be downgraded.”

There are several problems with this line of reasoning. Just because something is common, doesn’t mean it is reliable. Similarly, a lack of objective measurements of ME/CFS doesn’t mean that subjective questionnaires suddenly become immune to response bias. And as the Cochrane handbook explains, “the potential for bias cannot be ignored even if the outcome assessor cannot be blinded.”

The problem with non-blinded trials is that participants know if they are receiving the treatment that is being tested or not. Patients who realize they are getting an active intervention rather than the control might be more optimistic about their health or report symptoms according to what they think will please the researchers.

The intervention itself may also change how patients report their symptoms. Booklets on GET and CBT explained how these treatments would help them get better. They told patients how their symptoms might not result from an occult disease, but from more benign explanations such as deconditioning, stress or anxiety. Therapists were instructed to encourage optimism and clarify that impairments were reversible if patients committed to treatment. All these factors might have influenced how patients rated the severity of their fatigue. That’s why subjective outcomes are considered problematic as outcomes in non-blinded treatment trials, compared to subjective measurements taken outside the context of a clinical trial.

This is all common knowledge. That lack of blinding makes the use of subjective outcomes problematic, is explained in introductory textbooks. It’s acknowledged as a source of bias in guidelines on assessing evidence (such as the GRADE and Cochrane handbooks), and in the principles that both the FDA and EMA use for clinical trials (formulated at the International Conference on Harmonization, ICH). Some quotes and references are listed below for those interested.

Edit: wording.

 

[Mithriel]

I have said before that the idea that there can be no objective measure of subjective symptoms is wrong. If the patient says they are so fatigued they cannot do gardening any more, a diary of how many times they can weed the borders before and after the treatment is an objective measure.

If you feel that subjective symptoms are best served by subjective outcomes that is fine as long as the high risk of bias is noted. This is similar to a study on the metabolism of nicotine paid for by the tobacco industry. The results may be perfectly valid but the funding should be acknowledge so the reader can take it into account.

This is what was done by the NICE committee. They said the risk of bias made the trials of low quality. What Flottorp is arguing for is different; that trials like PACE are not held to the same standard as other scientific trials. That is unacceptable and indefensible for a medical professional to ask for such a thing and then complain when it is not done.

If you are colour blind you are still expected to stop at a red light. if you do not have the imagination to do a trial with an objective outcome you must accept that it makes the trial at high risk of bias so reduces its value.

Not even mentioning all the other problems with the trials!

 

[MSEsperanza]

If the patient says they are so fatigued they cannot do gardening any more, a diary of how many times they can weed the borders before and after the treatment is an objective measure.

Not sure whether diaries that the patient fills in could qualify as an actual objective measure?

I agree though that specific activity diaries would be better than most symptom questionnaires, both in a clinical care setting and for the use in clinical trials.

With regard to clinical trials, the usefulness of outcome measures are discussed here:

Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? -- Link

(Added links to that and other related related forum posts to post #2 .)