UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

[Trish]

I think we need to be careful in our criticisms of use of feedback, because we don't know what changes Sarah Tyson's team are making to their PROMs following feedback from the thousands who have completed the questionnaires, people writing to them, and comments from people here and on other social media. We simply don't know if our feedback is being listened to.

Yes, Sarah Tyson has decided to stop interacting here, and the MEA head of Trustees has dismissed our concerns, but he also said the team will continue to read comments here.

As I understand it, the 3 questionnaires sent out so far are, or at least should be, a first stage in a multistage process of development. Once analysed there can be radical changes, including shortening, changes in wording, dividing into sub questionnaires for different severity levels and more.

I am still hoping for the team to have the courage to make radical changes, like making the symptom questionnaire a simple one page list with tick boxes, abandoning the PASS questionnaire that wrongly conflated PEM with symptoms after exertion, and using the already established FUNCAP instead if their poorly designed function questionnaire.

I think they are still due to do one on mental health and one on service satisfaction. These also need to be short tick box lists with space for filling in individual concerns, in my opinion.

I hope, if they do a mood, mental health one, it is very short and asks direct questions such as do you experience anxiety and/or low mood that interferes with your ability to function, and do you want help with your mood or mental health?
They need to avoid the sort of long questionnaires on anxiety and depression that dig in to aspects of mood with prompt sentences most of which are inappropriate for people whose lives have been turned upside down by physical illness.

 

[Kitty]

I'm still hoping they come to understand that questionnaires should never be filled in whilst the patient is in a clinic.

If they're sitting there, at best they're already fatigued and stressed, and at worst they may be hanging on by their fingernails. Questionnaires need people to have the cognitive capacity to answer meaningfully (and as we know cognitive capacity is often severely impaired in brain fog and fatigue), and have time for reflection/mind changing about responses.

 

[MrMagoo]

I was reminded of an earlier post by Sarah Tyson that she said wearables aren’t actually possible to use in the NHS, none met the regs of the appropriate regulator or some such.
But a quick Google shows a lot of NHS trusts piloting wearables, and the NHS long term plan includes use of wearables
https://www.england.nhs.uk/long-read/medical-devices-and-digital-tools/

This pilot for wearables for patients with Parkinson’s, IBD etc on FATIGUE and how that is the main symptom which affects quality of life!
https://www.newcastle-hospitals.nhs...tigue-and-disturbed-sleep-in-chronic-disease/

 

[Trish]

The PACE trial used wearables 20 years ago. They used step monitors for a week at the start and were supposed to use them again at the end, until they chickened out. And the GET group used heart rate monitors during their daily timed exercise session and were told to stay below a certain level for a while and gradually increase duration of activity, then increase intensity of exercise to raise their heart rate to a higher level. And interestingly, the APT group were instucted that they were not allowed to use heart rate monitoring to help with pacing, they had to rely on symptoms and activity diaries.

If I remember correctly, the 2007 NICE guideline recommended heart rate monitoring for GET.

I was unimpressed by Sarah Tyson's confidence in PROMs as more reliable and valid that wearables data in her talk today, which I comment on here:
https://www.s4me.info/threads/onlin...me-cfs-4th-june-2024.38150/page-2#post-536220

 

[MrMagoo]

The PACE trial used wearables 20 years ago. They used step monitors for a week at the start and were supposed to use them again at the end, until they chickened out. And the GET group used heart rate monitors during their daily timed exercise session and were told to stay below a certain level for a while and gradually increase duration of activity, then increase intensity of exercise to raise their heart rate to a higher level. And interestingly, the APT group were instucted that they were not allowed to use heart rate monitoring to help with pacing, they had to rely on symptoms and activity diaries.

If I remember correctly, the 2007 NICE guideline recommended heart rate monitoring for GET.

I was unimpressed by Sarah Tyson's confidence in PROMs as more reliable and valid that wearables data in her talk today, which I comment on here:
https://www.s4me.info/threads/onlin...me-cfs-4th-june-2024.38150/page-2#post-536220

Off to read, thanks for the update

 

[Kitty]

I was reminded of an earlier post by Sarah Tyson that she said wearables aren’t actually possible to use in the NHS, none met the regs of the appropriate regulator or some such.

And unwieldy PROMs that don't actually capture anything meaningful do?

 

[MrMagoo]

And unwieldy PROMs that don't actually capture anything meaningful do?

It’s Sarah Tyson’s world, we just live in it.

 

[Evergreen]

The PACE trial used wearables 20 years ago. They used step monitors for a week at the start and were supposed to use them again at the end, until they chickened out.

Just a side note on this point, while it is often described this way - as something they planned to do and then "dropped" - the trial management group meeting minutes tell a different story. They planned for actigraphy as a predictor, and then considered making actigraphy not just a predictor but also an outcome, but decided not to add it as an outcome. So it wasn't something they were supposed to do that they reneged on, they just considered doing it and opted not to.

I know it's a detail, but I think it's important as it gets repeated a lot, and if they want to, the PACE crew can correctly argue against that detail, instead of addressing the problematic decision not to use any objective primary outcomes (despite considering a few), and their particularly questionable concern about actigraphy as an outcome (the Bleijenberg study not having positive actigraphy findings).

When I wrote about this on another thread (for anyone interested see here
https://s4me.info/threads/paul-garn...es-and-other-media.15629/page-105#post-530333), I was asked what the difference is between predictors and outcomes, so here it is in case it's helpful to someone:

Outcome measures are measured twice, before therapy and after therapy, to see whether someone improved.

Predictors are measured once, before therapy, so that you can see if any of them are useful for predicting whether someone improves or not. For example, you might want to know if people who are sick for less than 2 years are more likely to improve than people sick for longer, because it could help you direct services towards the people most likely to benefit. So their interest in actigraphy was whether a particular activity pattern - eg boom and bust - predicted whether someone improved or not in the trial.

Some things are both outcome measures and predictors, because you might want to know whether someone's depression improved with therapy AND whether being depressed at baseline made you more or less likely to improve with therapy.

A simpler example of a predictor would be age - you might want to know if younger people are more likely to improve than older people.

Edited to correct grammatical errors and add last line.

 

[Sean]

If the primary goal of therapy is to (genuinely and sustainably) increase patients' overall activity capacity, and it should be (even Wessely and Chalder think so*), then it must be measurable by objective means. That is simply not up for negotiation in my view.

I have yet to see a good argument for not using such means, and relying solely (or even primarily) on subjective measures, patient or clinician reported.

(*"...in the later stages of treatment patients are encouraged to increase their activity (which must ultimately be the aim of any treatment)"

Wessely, David, Butler, & Chalder – 1990

Wessely has also stated that his basic approach for ME/CFS patients is rehabilitative.)

 

[bobbler]

If the primary goal of therapy is to (genuinely and sustainably) increase patients' overall activity capacity, and it should be (even Wessely and Chalder think so*), then it must be measurable by objective means. That is simply not up for negotiation in my view.

I have yet to see a good argument for not using such means, and relying solely (or even primarily) on subjective measures, patient or clinician reported.

(*"...in the later stages of treatment patients are encouraged to increase their activity (which must ultimately be the aim of any treatment)"

Wessely, David, Butler, & Chalder – 1990

Wessely has also stated that his basic approach for ME/CFS patients is rehabilitative.)

It’s relying on the Heins eg al (2013) paper which includes Knoop and has the charts showing perceived activity increased in certain groups but objective didn’t

 

[Jonathan Edwards]

(*"...in the later stages of treatment patients are encouraged to increase their activity (which must ultimately be the aim of any treatment)"

This is of course wrong, where your version is right @Sean.

More pedantically, the aim of treatment is not to increase activity but to make the patient well enough to be able to normalise their activity if they want to.

On the reasonable assumption that most people will want to then the outcome measure should be dual: there should be both an improvement in subjective report of wellbeing AND objective evidence of increased activity. So, as we have discussed before, there is a need for a multiple threshold measure of the sort we have used in rheumatoid arthritis for thirty five years.

 

[Trish]

More pedantically, the aim of treatment is not to increase activity but to make the patient well enough to be able to normalise their activity if they want to.

On the reasonable assumption that most people will want to then the outcome measure should be dual: there should be both an improvement in subjective report of wellbeing AND objective evidence of increased activity. So, as we have discussed before, there is a need for a multiple threshold measure of the sort we have used in rheumatoid arthritis for thirty five years.

Yes.

I really don't understand why the research team on this project don't understand that. Or choose not to understand it.

I found Sarah Tyson's arguments againts wearables in her talk yesterday very weak, mostly based on ancedotes of wearables not recording accurate data, which i dispute, and her own preferences for what is a meaningful outcome.

And I would say it applies equally to service evaluation when there is no effective treatment. The aim is to do all that's possible to support people with symptomatic treatment and provision of aids and care, and to do no harm.

If service evaluation and/or patient care is to assess and track over time patients' disease severity, tnen it must also use both objective and subjective data, not rely solely on PROMs.

I would welcome a service that enabled me to track my disease severity over time, and that information in a clinically useable form be shared with my doctor. But only if it's tracking something that is a meaningful to me measure of my health and function. That would probably include a motion sensor that tracked 'steps' and time upright, HR and HRV possibly, a simple symptom tracker app, and a functional capacity questionnaire such as FUNCAP, preferably all recorded analysed via the same app. and wearable.

I think Visible is getting closest from what I've heard, though I'm disappointed its wearable is only a heart not a motion sensor (have I got that right?). I understand there is research being done on Visible's large data set.

 

[Adrian]

I was reminded of an earlier post by Sarah Tyson that she said wearables aren’t actually possible to use in the NHS, none met the regs of the appropriate regulator or some such.
But a quick Google shows a lot of NHS trusts piloting wearables, and the NHS long term plan includes use of wearables
https://www.england.nhs.uk/long-read/medical-devices-and-digital-tools/

This pilot for wearables for patients with Parkinson’s, IBD etc on FATIGUE and how that is the main symptom which affects quality of life!
https://www.newcastle-hospitals.nhs...tigue-and-disturbed-sleep-in-chronic-disease/

I was talking to someone recently who was commenting on various medical recording apps and the very poor security/privacy they seem to have. So this can be a real issue.

Generally, wearables won't have medical certifications but I think with using them along side PROMs we are talking about indicative information for anything clinical rather than as detailed accurate assured info. (but the security questions here worry me as that also raises quality issues).

If such mechanisms are used in research then a good data scientist will have an understanding of dealing with error ranges on incoming data and what can be accurately derived (although any statistician making claims from PROMs doesn't seem to have this type of appreciation).

The other issue is if an app is being used on say peoples smart phones (and this can do a lot of monitoring) then there are difficulties across different hardware and OS platforms which could lead to issues,

 

[Trish]

The other issue is if an app is being used on say peoples smart phones (and this can do a lot of monitoring) then there are difficulties across different hardware and OS platforms which could lead to issues,

Health apps I'm aware of all offer a cholce of android or apple. Don't most smartphones use one of other of these?

 

[Kitty]

More pedantically, the aim of treatment is not to increase activity but to make the patient well enough to be able to normalise their activity if they want to.

And where there is no treatment, it's moot anyway.

We have nothing that would allow a substantial proportion of patients to increase their activity levels, which is presumably why there's this insistence on only recording perceptions.

Health apps I'm aware of all offer a cholce of android or apple. Don't most smartphones use one of other of these?

Most do, but there are differences in the way the data is processed. Quite a lot of that goes on, and it involves algorithms that are never published. The so-called sleep monitors are worse than useless, and should carry a warning that there's no smart device available that can even tell whether a person is awake or asleep, far less what stage of sleep they're in.

But people who've used more than one system find them pretty consistent across the basic measures, the ones that need the least machine interpretation. Heart rate and steps are usually good enough for the purpose, and certainly an order of magnitude more accurate than my perception of how much I've done.

If people are worried about privacy, mechanical pedometers have been available for at least half a century, and there are also electronic heart rate monitors that don't connect to the internet. The trouble is that you need to record the results, and people are no longer used to having to do that.

 

[FMMM1]

Yes.

I really don't understand why the research team on this project don't understand that. Or choose not to understand it.

I found Sarah Tyson's arguments againts wearables in her talk yesterday very weak, mostly based on ancedotes of wearables not recording accurate data, which i dispute, and her own preferences for what is a meaningful outcome.

And I would say it applies equally to service evaluation when there is no effective treatment. The aim is to do all that's possible to support people with symptomatic treatment and provision of aids and care, and to do no harm.

If service evaluation and/or patient care is to assess and track over time patients' disease severity, tnen it must also use both objective and subjective data, not rely solely on PROMs.

I would welcome a service that enabled me to track my disease severity over time, and that information in a clinically useable form be shared with my doctor. But only if it's tracking something that is a meaningful to me measure of my health and function. That would probably include a motion sensor that tracked 'steps' and time upright, HR and HRV possibly, a simple symptom tracker app, and a functional capacity questionnaire such as FUNCAP, preferably all recorded analysed via the same app. and wearable.

I think Visible is getting closest from what I've heard, though I'm disappointed its wearable is only a heart not a motion sensor (have I got that right?). I understand there is research being done on Visible's large data set.

Ah me not keeping up/see the issue now - Sarah Tyson received £90K to develop a toolkit to assess clinical outcomes and it (solely) uses subjective outcome indicators - well blame lies primarily with ME Association. However, I'd also add that surely the "academic" (Sarah) should be objective --- know that these indicators are unreliable (re changes in activity pre/post intervention) and that there are now opportunities to measure reliably/objectively?
Finally I'd add that the Government shouldn't rely on Sarah's unreliable assessment of these clinical interventions --- you can see the problems in getting that message across - thanks (not) ME Association!

“I am delighted to have received this grant from the ME Association and look forward to working with the team, with people with ME/CFS, and with clinicians to co-produce a clinical assessment toolkit."
https://meassociation.org.uk/2023/0...nt-toolkit-in-nhs-me-cfs-specialist-services/

 

[FMMM1]

Health apps I'm aware of all offer a cholce of android or apple. Don't most smartphones use one of other of these?

Yea we need to be careful that perfection becomes our enemy --- the data needs to be good enough to say -- yes this persons activity has significantly improved (post intervention X) & that improvement is sustainable in the long term OR NO.

 

[MEMarge]

Some comments in the chat after Sarah talked re wearables were that other activities such as playing the piano or stroking the cat showed increases in steps.

For people with ME, the former certainly could be relevant activity.

Of course these types of wearables don't I assume measure cognitive energy consumption.

 

[Trish]

Of course these types of wearables don't I assume measure cognitive energy consumption.

No, we need other ways of taking that into account. There are quick tests that can show cognitive fatigue, I think, which could be useful eg how many of a sequence of digits such as a phone number you can recall after reading them.

 

[Trish]

Some comments in the chat after Sarah talked re wearables were that other activities such as playing the piano or stroking the cat showed increases in steps.

Of course, especially if worn on the dominant wrist. There are ways to get around this, either by wearing on the waist or thigh, or by taking into account that the wearable is recording arm movements including steps. I find it useful, for example that getting dressed and undressed records as steps, and for me is a physically demanding activity.
I agree, playing the piano is energetic activity, and it's fine that it's recorded.

These quibbles seem so minor compared with the subjectivity of PROMs, but Sarah didn't even mention the problems with PROM subjectivity, as far as I remember, and she has said on this thread, wrongly, that they provide objective data.