Measuring fatigue. Discussion of alternatives to questionnaires.

[Medfeb]

Not sure if this is helpful but just in case...

To improve consistency of data being collected across studies and thus the comparability of those studies, NIH's Common Data Elements initiative collated information on instruments available to assess the various domains of illness. This link contains summary documents for each domain along with descriptions of various instruments that were reviewed.

https://www.commondataelements.ninds.nih.gov/MECFS.aspx#tab=Data_Standards​

The instruments are ranked by whether they were considered core (for all studies), supplemental (for specific studies and already used effectively in ME research), or exploratory (essentially potentially useful but more study needed). There are very few core instruments as most would be specific to the type of study.

This all needs more work but it might be worth looking at what they came up with. A few challenges on the fatigue domain were lack of consistent definition, potential overlap with other domains (e.g. is post-exertional fatigue part of fatigue or PEM), and questions of whether the definition should be specific to ME or whether it should use a definition of fatigue common across all domains. Note that Chalder was not included on the list of fatigue instruments.

 

[Barry]

And the need is to measure a specific pattern of motor fatigue X, that is characteristic of ME, or maybe patterns X and Y and even Z that are patterns characteristic of subsets of ME.

I think this is valid. When I walk with my wife, and sometimes comment on how well she is doing, she reminds me that it is because she has her camera with her. When in photography mode, we have lots of brief stops whilst my wife takes photos, then we move on to the next photo break; in effect a series of short walks, with short breaks in between. It's as if my wife has a very small store of readily available energy which expires quickly, but with a few minutes respite partially charges back up again. She will eventually run out of steam and need to get back home again, but with this exercise profile she can get significantly further than with no pauses.

 

[Jonathan Edwards]

what if they're left-handed?

Well if they are left handed the left wrist would be the right wrist to put the accelerometer on, wouldn't it?

 

[Graham]

But in reality, for many of these studies the complexities of analysing data from fitness monitors etc., or the complexities of arranging objective tests is too much. An ideal route would be a basic questionnaire asking concrete yes/no questions, and a simple computer-controlled cognitive test - say watching reaction times deteriorate (rather like my eye test for peripheral vision). A basic questionnaire could simply list a number of "activities", such as making the bed, going out for a meal, and ask whether pre-ME the person always did it, and whether they still do post-ME. No use for my son who went down with it aged 8, and I haven't thought through it properly, but I would have thought such a list was possible.

 

[Trish]

A basic questionnaire could simply list a number of "activities", such as making the bed, going out for a meal, and ask whether pre-ME the person always did it, and whether they still do post-ME. No use for my son who went down with it aged 8, and I haven't thought through it properly, but I would have thought such a list was possible.

I agree that getting people to say what activities they can do is better than getting them to say how bad a symptom is, but it's still subjective and can vary wildly according to how you interpret the questions.

Isn't that what the SF-36 physical functioning questionnaire is supposed to capture. The problem is, I could say 'I can make the bed' But that would be on a very good day, and only once and I'd have to rest for the rest of the day, and probably have PEM the next day.

I think fatigue and pain and other 'subjective' things are very hard to capture in a questionnaire. And as you demonstrated in your video, even SF-36 can vary significantly depending how you interpret the questions.

I think the only time a questionnaire, or scoring system is useful for subjective symptoms is for a single patient to say whether something has improved or got worse within a short time span, like before and after taking a pain killer.

 

[Jonathan Edwards]

But in reality, for many of these studies the complexities of analysing data from fitness monitors etc., or the complexities of arranging objective tests is too much.

I am not so sure. If you work out what to extract from the software in advance there is no slog other than getting subjects to wear the accelerometers and download the data. No complexities at all. Working out what to extract might take a three year study of relevant patterns but at least you have objective data. I am not at all convinced that any questionnaire is going to be of great use.

The key problem is that treatment blinding may well not be feasible. That means that bias in form filling becomes crucially important. Yes-no questions like have you gone back to work may help, but in many cases may not yield anything useful.

 

[duncan]

I think the only time a questionnaire, or scoring system is useful for subjective symptoms is for a single patient to say whether something has improved or got worse within a short time span, like before and after taking a pain killer.

In the absence of legislation that gives patients legal recourse when any doctor dismisses or minimizes a patient's reported symptoms, patients will come out wanting. Placebo effect comes to mind, and that, of course, is the tip of the iceberg.

 

[Graham]

I appreciate what Jon and Trish say, but, stepping back, there is an enormous gulf between what I used to do and what I do now. I'm talking of a questionnaire that measures that gulf. I'm not a fan of questionnaires in general, and I used to try to get my students to be very restrictive and time-limited when asking questions, such as "How many hours of TV did you watch last night?" (The balance comes from asking lots of people, on a variety of days)

What would be ideal, or scientifically valid, is very different from what studies are likely to settle for. It must be rather like ending up with me as a husband: the reality is quite a departure from the theoretical ideal! But if we could come up with a more concrete measure, wouldn't it be an improvement? After all, we aren't looking for a treatment that ups our performance from 15% to 20%, but something significant that will properly change our lives, so the measure does not have to be that sensitive, but it does have to be robust.

There are many questions that would be misleading, such as how much TV you watched. I can imagine that for some the amount would increase, and for others it would vanish. But everyday tasks? Do you shave every day? Do you shower/bath every day? Do you play any sports? Do you go for long walks? Couldn't we come up with a list of perhaps 20 things that many of us once did that we do not do any more. The score would be the number that we no longer do, so it wouldn't matter if we never did many of the things on the list. The score itself wouldn't represent how badly affected we were, but the improvement in the score would represent an improvement.

I'm rambling as usual, but if my random thoughts act as a catalyst to those of you with brains that still work, that will be good.

 

[duncan]

A problem would be when anyone in the medical community takes that questionnaire data, and misinterprets it as either delusional or as reflective of everyday symptoms that are exaggerated or inappropriately attributed by the patient as being unusual.

So it wouldn't necessarily be the questionnaire vehicle, or the questions, or even the wording - it would be the interpretation, presumptions, or dogmatic intolerance by the non-patient medical community.

 

[Barry]

If you work out what to extract from the software in advance there is no slog other than getting subjects to wear the accelerometers and download the data. No complexities at all. Working out what to extract might take a three year study of relevant patterns but at least you have objective data. I am not at all convinced that any questionnaire is going to be of great use.

Key is to record sufficiently rich data in the first place. Exactly what information proves most useful may not become fully apparent until part way through the data collection phase, but so long as the necessary data has been recorded, then that information should be extractable. But of course there's an element of chicken and egg: sod's law says the information you later realise you need may not have been preceded with the insight of which data were needed.

My personal hunch would be to include recording of data which allows good profiling of energy consumption rates, sampled at sufficient frequency so that really short term trends can be observed as well as longer. I'm guessing direct measurement of energy consumption is probably not feasible, but maybe there are metrics that could be captured and post-processed to infer energy consumption? Such energy consumption profiles could then be compared to healthy controls. It may be that the profile characteristics show some kind of signature for ME?

Needless to say I'm not for one second suggesting this is all that is needed, or it is anything like as simple as that. Just that I think such data should be collected.

 

[Ravn]

@Hutan posted this online tool to assess reaction time in another thread: http://www.sleepdisordersflorida.com/pvt1.html#responseOut

Psychomotor Vigilance Test
Try this game to test your alertness and vigilance.
Instructions: Press start button. Single click on the box as soon as possible after red numbers appear in the box. The red numbers will appear at random times. The test lasts 2 minutes or until you press the Stop button.

Just tried it and don't know whether to laugh or to cry at the result. I tried very hard and achieved an average response time of 694 msec. This resulted in the following message in bold:

Your results show that your alertness may be suboptimal. Consider medical evaluation.

The test is quite tiring itself so won't be doing it on a regular basis but intend to repeat on a very good day and a very bad PEM day.

Thoughts about this type of test:

Advantage:
Doctors know this type of test and view it as an acceptable part of a diagnostic work-up in other conditions. I recall doing a very similar test in hospital back when the neuros were still trying to figure out whether I might have MS (I haven't).

Disadvantage:
Doctors have to trust patients to give their very best performance; cheating is possible.

Question:
What exactly does the test measure? Presumably increased fatigue would cause slower reaction time to a visual stimulus, at least that's what the traffic safety people are telling us. But there could be other factors playing in. For example I had the impression, when doing the test, that I registered the visual signal almost instantaneously but that there was a noticeable delay between my brain ordering “click!” and my finger executing the order.

 

[Marco]

Wouldn't the design of any instrument depend on the intended use?

For example is it to be used as a general purpose measure of 'fatigue'; to document problems with completing everyday tasks to demonstrate incapacity; to provide objective evidence that fatigue is not just perceptual; as a snapshot or for continuous measurement; or as a quality of life 'anchor' in the context of a clinical drug trial where there are objective physiological measures available in which case a subjective questionnaire is perfectly legitimate.

 

[ME/CFS Skeptic]

I could say 'I can make the bed' But that would be on a very good day, and only once and I'd have to rest for the rest of the day, and probably have PEM the next day.

I agree with Graham that a questionnaire could be useful if the questions are smart enough. You could for example ask: do simple tasks like making the bed or preparing a meal exceed your energy limit, do you have to include extra rest periods to recover from such tasks? For normal people that would be a weird question but it's very recognisable for ME/CFS patients.

 

[Jonathan Edwards]

do simple tasks like making the bed or preparing a meal exceed your energy limit, do you have to include extra rest periods to recover from such tasks?

Questions like this are all very well in theory but I doubt they are immune to role playing bias.

In my experience a majority of people taking part in trials want to 'help' the investigators by contributing to getting the answer the subjects think the investigator wants. Most people see the point of research as discovering what you want to discover, not proving yourself wrong (which is the real objective in a sense). So a lot of patients on the 'good treatment' will think to themselves.

'Well, I do take a few extra rests but I won't say so because that will upset the therapist lady. I don't really NEED to take rests so I can say no.'

The problem with trials is that any statistically significant difference is hailed as a positive result. Just a mild systematic bias will show up as significant and all that needs is for a minority of subjects to be 'trying to help'.

 

[Unable]

@Hutan posted this online tool to assess reaction time in another thread: http://www.sleepdisordersflorida.com/pvt1.html#responseOut

Just tried it and don't know whether to laugh or to cry at the result. I tried very hard and achieved an average response time of 694 msec.

Very interesting. I thought I was quite good at the test! Apparently not!




(Although, I do wonder if internet speed may affect results? Our internet speed is rubbish.)

 

[Unable]

(Although, I do wonder if internet speed may affect results? Our internet speed is rubbish.)

My healthy husband has just done the test and got similar results to me. We both thought the numbers continued to go up after the screen had been tapped so the delay was the iPad or internet, and not the subject’s slow responses!

 

[Graham]

The problem with trials is that any statistically significant difference is hailed as a positive result. Just a mild systematic bias will show up as significant and all that needs is for a minority of subjects to be 'trying to help'.

From my reading of the Chalder Fatigue Scale results from my own survey, the FINE data and the PACE data, I think that is a particular problem with questions that ask for degrees of difficulty. People can be convinced or feel obliged to regrade their difficulty with, say, walking, from very bad to just bad, but it seems to be less likely that they say the problem has vanished.

The other problem is caused by researchers looking for any improvement, however slight. Any proper treatment of my ME will create enormous changes: many of these studies hail an imperceptible improvement as miraculous just because it reaches the level of statistical significance.

In one way I wish that spreadsheets etc. had never been invented. It is far too easy to apply sophisticated statistical analyses to data without actually applying any trace of common sense. (Yes, PACE, I mean you!)

I am fully cynical about the merits of questionnaires, just as I am about a number of things that permeate society. But I recognize that they will continue to be used.

 

[Barry]

The other problem is caused by researchers looking for any improvement, however slight.

A fundamental problem surely is that with such questionnaires the signal-to-noise ratio is appallingly low. The noise level is going to be very high, whilst many of the signals being supposedly acquired will sit well within the noise. And the noise is nothing like random.

 

[Trish]

The other problem is caused by researchers looking for any improvement, however slight. Any proper treatment of my ME will create enormous changes: many of these studies hail an imperceptible improvement as miraculous just because it reaches the level of statistical significance.

I think this is a key point. If I look at my scores on the SF-36 physical functioning questionnaire (calculated retrospectively on what I remember of my ability/disability levels), and my guess at my daily step count:

Healthy before ME - SF-36PF score 100, (estimated daily steps 6000 to over 10,000)
Mild ME - Score fluctuated from 0 to 50, (estimated daily steps 0 to 4000)
Severe ME - Score fluctuates from 0 to 10, (estimated daily steps 0 to 500)

In other words, the differences are dramatic between severity levels in my case, but within the Mild stage, which is the stage where most people can actually attend ME clinics and CBT/GET treatments, the numbers can fluctuate over a wide range without telling you anything real about the person's state of health.

If clinical trials for ME would accept that improvement should only be claimed if the steps and SF-36 scores are maintained over a long period (a year or more) at a level dramatically different from the previous level averaged over a year.

Perhaps questionnaires should ask people to fill in their best and worst days over the last month or more, so someone with mild ME would say their SF-36 ranges from, say, 20 to 50 when not crashed and down to 0 to 10 when crashed.

Then 'improvement' for that person might require their new range to be higher, and not overlap with the old range, so they might have to score over 60 most of the time.

And recovery would only be registered if they return to their pre-illness score for at least a year.

 

[chrisb]

In some ways the discussion of fatigue reminds me of the discussion of hysteria by Slater. Words that appear useful as adjectives may be worthless as substantives. There is no such "thing", for want of a better word, as fatigue. It is merely an abstract noun used in various ways to convey various meanings. No amount of enquiry will get one any closer to discovering what fatigue "is". It can only help ascertain how people are using the word in any particular situation.

Reification is always problematic.