What's the difference between using symptoms for making a diagnosis in clinical care vs measuring a treatment effect in clinical trials? (Discussion)

As the title indicates.

Proponents of therapist delivered treatments like psychotherapy or exercise repeatedly rejected the argument that relying solely on subjective outcomes in open label trials to assess the effectiveness of said treatments was a bad idea.

In their criticism on the NICE guideline committee's evaluation of the evidence for benefits and harms of potential treatments for ME/CFS, Flottorp et al write [1]:

"It is uncontroversial that a diagnosis of CFS/ME rests upon subjective symptoms. But paradoxically NICE decided that evidence from clinical trials of CBT and GET showing improvement in subjective symptoms would be considered unreliable.

"Given the first premise, subjective symptoms are the most valid endpoints, and interventions improving these symptoms are treatments, not only 'symptom management'."

Since even experts in assessing evidence and methodologists responsible for developing tools to assess the risks of bias in clinical trials either use similar arguments or don't seem to see a problem with this kind of arguments, I get the impression our criticism needs to be worded more clearly.

Or perhaps their needs to be an additional discussion on the points that might be a bit challenging to deal with in clinical trials investigating treatments for symptoms that can't be objectively measured (yet)?


[1] Signe A. Flottorp, Kjetil Brurberg, Per Fink, Hans Knoop, Vegard B B Wyller (2022), New NICE guideline on chronic fatigue syndrome: more ideology than science? The Lancet, Volume 399, Issue 10325, 611 - 613, https://doi.org/10.1016/S0140-6736(22)00183-0 , Forum thread here.

 

My thoughts as a lay person:

1) The relevance of symptoms for making a diagnosis is different from the relevance of symptoms for measuring a treatment effect, especially in the context of clinical trials.

When patients inform their doctors about symptoms that are not only unpleasant but substantially reduce their physical and often also cognitive function, there sometimes might be reasons why patients' reports can be biased and should be doubted.

I assume that these reasons don't apply often enough to be relevant, but anyway, they are different from the reasons why patients' reports on a treatment effect can be biased.

Also, a doctor makes a diagnosis not only based on individual scores for particular symptoms, but a on a thorough clinical assessment. If there aren’t established biological markers, it's even more important that doctor and patient talk with each other to get a clear picture of each symptom and the specific pattern of symptoms.

It might be easier to have objective outcomes for assessing disability, but that’s a different topic.


2) Using specific subjective outcomes in a clinical trial is fine. If there aren't any objective measurements specific to the illness, you have to use subjective outcomes. However, in an unblinded trial, additional objective measures are needed. There are a couple of objective measures relevant to people with ME/CFS, especially actimetry and I think school/ work attendance.

Also, it should be easy to establish a cognitive test specific for the kind of cognitive fatigability and brain fog PWME experience.


3) It would be great if patients were asked which outcomes and measurements they find relevant. That applies for both subjective and objective outcomes and measurements.

For example, members of the S4ME forum wrote a thorough criticism of the Chalder Fatigue Scale. They also made suggestions for a review of the DePaul Symptoms Questionnaire that measures Post Exertional Malaise (PEM).]


Related discussion threads:

Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? -- Link

Questionnaires - design, validation and use in ME/CFS research -- Link

 

Apologies for being repetitive with the argument about the need for objective outcomes in unblinded trials.

Also, some of the points I'd like to discuss have been worded much better in@Michiel Tack's/ ME/CFS Skeptic's rebuttal of Flottorp et al:

Edit: wording.

 

I have said before that the idea that there can be no objective measure of subjective symptoms is wrong. If the patient says they are so fatigued they cannot do gardening any more, a diary of how many times they can weed the borders before and after the treatment is an objective measure.

If you feel that subjective symptoms are best served by subjective outcomes that is fine as long as the high risk of bias is noted. This is similar to a study on the metabolism of nicotine paid for by the tobacco industry. The results may be perfectly valid but the funding should be acknowledge so the reader can take it into account.

This is what was done by the NICE committee. They said the risk of bias made the trials of low quality. What Flottorp is arguing for is different; that trials like PACE are not held to the same standard as other scientific trials. That is unacceptable and indefensible for a medical professional to ask for such a thing and then complain when it is not done.

If you are colour blind you are still expected to stop at a red light. if you do not have the imagination to do a trial with an objective outcome you must accept that it makes the trial at high risk of bias so reduces its value.

Not even mentioning all the other problems with the trials!

 

Not sure whether diaries that the patient fills in could qualify as an actual objective measure?

I agree though that specific activity diaries would be better than most symptom questionnaires, both in a clinical care setting and for the use in clinical trials.

With regard to clinical trials, the usefulness of outcome measures are discussed here:

Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? -- Link

(Added links to that and other related related forum posts to post #2 .)