UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

[Kitty]

I'm not sure any other serious illness charity would, as a main measure it is being suggested most will the illness would have eg in a clinic, accept a survey asking about intimate relations and using the term 'avoid' for activities that relate to physical function.

Not sure any other illness charity would be asking what people do at all, would they? Why would a charity even think of doing that?

Nor would a clinician treating chronic illness. They ask how people are feeling, not what they're doing.

How can MEA (or anyone else) not see how utterly bizarre this is? How divorced it is from normal practice? How useless it is in a treatment setting, when asking people how they're doing would be just as good?

My mental voice seems to go up two octaves from sheer incredulity every time I think about it.

 

[bobbler]

Trish Davis
Perhaps it will help you understand where I'm coming from with PEM if you read the S4ME fact sheet on PEM. https://www.s4me.info/threads/science-for-me-fact-sheets.43310/post-606969

Trish Davis

https://www.s4me.info/docs/PEM_Factsheet.pdf

Sarah Tyson

Trish Davis The results of the PASS gives fascinating and detailed description of PEM in ME/CSF and I look forward to sharing and discussing the details in due course.



Sarah Tyson
Trish Davis If you don’t mind me saying, I think you have not grasped (or maybe are unwilling/unable to accept) that the point of the MEAQ is to ask about what people do, rather than the full detail of how they do it. There will be a multitude of different ways in which people perform tasks, make adaptation or other compromises to get through life. And that is fine. The issue is whether they do it, however they do it. Whether someone else may do it differently, or whether the person can do it in a certain way is irrelevant. We are not attempting to standardise the activities, we are asking if people do the activity, however they do it.

You ask about the scoring system and analysis approach. It is pretty hard to summarise as it is a large section of my life’s work! However, you really want to get your head around these issues I recommend the seminal texts by Anne Bowling. Luckily some of it appears to be available for free now-a-days. This is a good intro (Chapter 2) Measuring Health - Ann Bowling - Google Books
The analysis which focusses on a really thorough examination of the construct validity using Rasch analysis. The Wikipedia page is pretty good introduction. Rasch model - Wikipedia


Trish Davis
Sarah Tyson Thank you for the information about how you analyse the data. I'll take a look.
You say: "If you don’t mind me saying, I think you have not grasped (or maybe are unwilling/unable to accept) that the point of the MEAQ is to ask about what people do, rather than the full detail of how they do it."

My response to that is that I'm sorry to say I find this response patronising and missing my point. As I said earlier, I find FB particuarly difficult to use for in depth discussion, but I've tried to convey the central factors of cumulative effect and how often one can do an activity to gauging key factors of severity and need for care, which are presumably an important part of care planning. However good the mathemeatical/analytical instruments used, any summary statistic is too easily skewed by the sort of data this will provide. I think I'll have to leave it there. I have no energy left for arguing against someone so convinced their knowledge is superior to anything I might say.

So I'm starting to look into Ann Bowling. And her main focus, as a sociologist, has been on quality of life in aging/older people, and research methods. But I think the research methods interest has been intertwining with that focus on quality of life and aging/older, rather than about assessing clinical conditions ie medicine itself.

Ann Patricia Bowling - Wikipedia

en.wikipedia.org

The full title of the book referenced above seems to be: Measuring Health: A review of subjective health, wellbeing and quality of life measurement scales

It is interesting that Sarah is focusing in on construct validity rather than external validity (is it doing what it purports to do and can it be extrapolated)

If you were checking construct validity eg that it was measuring disability in ME/CFS or severity of ME/CFS then surely the obvious way to check this would be to compare to existing measures of those ... for example FUNCAP or SF-36 or just the severity of someone's diagnosis.

I can't remember what Sarah et al have described with regards to checking these.

Or whether if they have just used Rasch on their own data and don't have these hidden in the questions they are just doing internal consistency or are comparing it to something else with prior questions that for example focused on the Chalder Fatigue Questionnaire being used, or maybe they are measuring 'something else' as in PROM is about quality of life or subjective health

I know what we all think about AI but due to laziness and having had a quick check of it here is what the answer is on whether Rasch alone can confirm construct validity, rather than just 'internal consistency'

No, Rasch analysis alone is generally insufficient to fully establish construct validity; it provides strong statistical evidence for unidimensionality and item fit, but it is a part of a broader validation process that requires combining Rasch evidence with conceptual (theory-based) evidence, empirical evidence from item-person maps, and sometimes other psychometric methods like factor analysis. While a good fit to the Rasch model indicates that items are measuring a single, intended dimension, this doesn't automatically confirm that the dimension itself is the correct or relevant construct.

What Rasch Analysis Provides for Construct Validity

• Unidimensionality:
  Rasch analysis can test if the items on an instrument measure a single, underlying construct. If the data fits the Rasch model, it supports the idea that the instrument is focused on one dimension.
  
  
• Item Hierarchy:
  Rasch analysis creates a Wright map that shows the relative difficulty of items on a single logit scale. A logical hierarchy suggests the items are measuring the construct in an ordered way.
  
  
• Item Fit Statistics:
  Rasch analysis provides statistics (Infit and Outfit mean-squares) that indicate how well individual items and responses fit the model's expectations, flagging misfitting items for revision or removal.
  

Why Rasch Alone Is Not Enough

• Conceptual Link:
  A good Rasch fit doesn't confirm that the measured dimension is the theoretically relevant construct you intended to measure. You must link the statistical results to your underlying theory.
  
  
• Empirical Evidence:
  Other types of empirical evidence are needed. For example, factor analysis can provide different but complementary evidence about the structure of the items.
  
  
• Response Patterns:
  Rasch analysis helps evaluate how well respondents answer items in expected patterns, but a comprehensive validation also considers whether the responses align with the construct's theoretical meaning.
  
  
• Contextual Relevance:
  The ultimate goal of construct validity is to show that the instrument's scores truly reflect the construct of interest in a particular context. Rasch analysis is a powerful tool for this but needs to be integrated with other forms of validation.

 

[Trish]

I'm taking up Tyson's challenge to examine the methodology in the book and Wikipedia article she recommended to me (to shut me up) about the validity of her methods. I bought the book. It's so heavy I couldn't hold it to read so I took my scissors to it today and cut out the relevant pages. If i can make any sense of it I'll try to put together some conclusions and post them here.

 

[bobbler]

Not sure any other illness charity would be asking what people do at all, would they? Why would a charity even think of doing that?

Nor would a clinician treating chronic illness. They ask how people are feeling, not what they're doing.

How can MEA (or anyone else) not see how utterly bizarre this is? How divorced it is from normal practice? How useless it is in a treatment setting, when asking people how they're doing would be just as good?

My mental voice seems to go up two octaves from sheer incredulity every time I think about it.

I don't think I'm being 'hysterical' when I say some of these questions strike me as coercive guinea-pigging a population, just because someone thinks they can take advantage. Again.

when combined with the context under which they will be administered - which is far from optional, given the very specific situation pwme are put in compared to when they have for example MS or cancer with regards to options to 'say no' ie free will from coercion -

Can you imagine a 50yr old gentleman with a potential heart condition who is a surgeon or academic or senior executive being faced with this survey and thinking it was at all acceptable.

Their first question would be about how any of it was relevant to diagnosing or treating their medical problem.

The second thing that would happen is that if feeling like rising above it they would put those numerous pieces of paper to one side and politely say that they are not wasting their time and energy on that.

I am struggling to see how anyone who isn't being conned by the situation could imagine that true medical care could result from such surveys. Even in a 'long run, we are monitoring across lots of people to see what works' sense that would be the case if we did have proper medical clinics that might not have treatments yet but at least did clinical work-ups over the long-term for a significant enough sample of people across a spectrum or who are more severe, with strong histories to understand how people ended up that ill etc.

And if it weren't for the demographic that these individuals are assuming they are co-opting into this, the ones who won't have options to say no and end up with something written in a letter that puts their freedom or access to services at risk potentially, and the demographic of staff who want to do something like this ... could you imagine a busy 'ologist' being expected to read through all of this?

It feels like more than a liberty in the metaphorical sense and is a real concern with regards the Declaration of Helsinki - and Ethics is called that we must remember because of its origins. Someone claiming 'their own good reasons/intentions' without proof isn't enough to show that there isn't harm, and it is even more concerning in areas where there is encroachment on really, truly offering informed consent.

 

[Trish]

I am reminded of Sarah Tyson's response on this thread earlier in this project when we pointed out that pwME have been harmed by the use of questionnaires. Her response: 'Don't be silly'.

We are not safe on her hands. The clear purpose of these PROMS is the perpetuation of rehab clinics as the only provision in the NHS for ME/CFS. She even said on FB in response to one of my comments that clinics only see newly diagnosed, as if that justified all her rubbish. Shocking.

 

[Michelle]

Nor would a clinician treating chronic illness. They ask how people are feeling, not what they're doing.

To be fair, I wish clincians would focus more on what I'm doing rather than how I feel. I don't care what number I feel like on a 1-10 scale or what bubble I fill in on whatever questionnaire du jour we're using. What I care about is can I take a bath on my own? Can I clean my own home? Can I leave my home? Can I work?

And it doesn't take some complicated car crash of a questionnaire for a provider to sort that out. They lost me at "goal-setting."

I'm taking up Tyson's challenge to examine the methodology in the book and Wikipedia article she recommended to me (to shut me up) about the validity of her methods. I bought the book. It's so heavy I couldn't hold it to read so I took my scissors to it today and cut out the relevant pages. If i can make any sense of it I'll try to put together some conclusions and post them here.

::saluting:: Thank you for your service!

 

[Amw66]

I am reminded of Sarah Tyson's response on this thread earlier in this project when we pointed out that pwME have been harmed by the use of questionnaires. Her response: 'Don't be silly'.

We are not safe on her hands. The clear purpose of these PROMS is the perpetuation of rehab clinics as the only provision in the NHS for ME/CFS. She even said on FB in response to one of my comments that clinics only see newly diagnosed, as if that justified all her rubbish. Shocking.

It was
the conflation with fatigability and PEM
the significant lack of knowledge re PEM ( cumulative, rolling etc)
the simple cognitive dissonance

that has me very worried re what will be produced .

 

[Kitty]

I am struggling to see how anyone who isn't being conned by the situation could imagine that true medical care could result from such surveys.

I seriously doubt anyone in the medical world could imagine it either.

Even in a 'long run, we are monitoring across lots of people to see what works' sense

But that's a research study, not treatment practice. Studies inform practice, they're not done simultaneously with treatment consultations. (Or offered instead of treatment.)

The authors don't seem to have a grasp of this. They're so busy showing off their academic chops that they've lost all contact with common sense.

 

[Kitty]

To be fair, I wish clincians would focus more on what I'm doing rather than how I feel ... What I care about is can I take a bath on my own? Can I clean my own home? Can I leave my home? Can I work?

It's not really medical, maybe? After diagnosis, a doctor's main role is usually to find treatment and management approaches (assuming any exist) that maximise improvement and minimise side-effects and risk.

If you say you're feeling better or worse, it goes without saying that social and occupational factors will be improving or getting worse too. But it doesn't really change their role in it, which is to look for ways to help with symptom relief or damp down active disease, without unacceptable risk.

If they can do that, they know every facet of the person's life will improve. If they can't, there may not be much they can do about solving the problems the patient is left with.

 

[Midnattsol]

Does any other condition get asked to set goals for their condition?

Yes. At least in my country healthcare workers are taught how to help patients set (realistic*) goals. As information about ME is severely lacking, this knowledge is most likely used on other patient groups. For my profession it is all about behavior, with the idea that healthy living will improve symptoms in some way.

*What is realistic is up to the provider, obviously the patient doesn't necessarily know..

 

[bobbler]

I am reminded of Sarah Tyson's response on this thread earlier in this project when we pointed out that pwME have been harmed by the use of questionnaires. Her response: 'Don't be silly'.

We are not safe on her hands. The clear purpose of these PROMS is the perpetuation of rehab clinics as the only provision in the NHS for ME/CFS. She even said on FB in response to one of my comments that clinics only see newly diagnosed, as if that justified all her rubbish. Shocking.

I'd like to see her response if her eg boss told her that she had to fill in an equivalent survey in her own name, and in a situation where it would be put down as 'not helping herself' and 'being non-compliant' on her employment file if she did not for example. And it isn't like she can claim people fill in stuff like this online all the time and they are normal questions for people who aren't ill.

and was parsing words when asked who might see it and how it would be used and using different titles for it like 'the MEAQ (rather than PROM you filled out in clinic is only going to be used for and seen by' when you asked what the PROM could end up being used for.

Someone who for example was all too aware of insidious narrative making-up of 'multidisciplinary meetings' that could potentially suggest anti-psychological (harmful to someone's situation and mental health) stories if not correct that make them feel violated by suggesting stuff that is untrue will be being put in a no-win situation where by filling such in they provide material by having limited choices of inaccurately (purposely, as it has been flagged) options to choose or what... being seen as non-compliant .. oh we can't possibly offer any help or diagnosis because you wouldn't tell us about your intimate relations or whether you avoid going on a bus?

To say it is
1. not coercive and with huge data protection issues that explanations are evasive for. Not asking inappropriate and irrelevant questions that quite simply cannot be justified by any of the explanations of what it claims to either measure or help with. Book or not.

2. relevant to the medical condition and needed to either diagnose or help assess how a medical condition is changing

is silly

To say anyone flagging up barn door concerns that any normal, rational human being would ask simply because of the extraordinary nature of finding these on a questionnaire claiming to be about a medical condition is anything other than calm and correct and well within their rights and responsibilities

is silly

and seems to indicate something even more worrying about entitlement, power and belief in the basic rights of those people to you know ask normal questions. Which isn't normal or right surely?

What is pointing us to a favourite author whose books are on aging and old people's quality of life got to do as an explanation about something that is supposed to measure effectiveness of dealing with a specific medical condition, or to diagnose it - as the only measure of the condition itself? This is side-stepped into 'individualised scales' that don't focus on the condition from what I've read so far about what this seems to be and the description eg in this bit of one of Ann's books: https://journals.sagepub.com/doi/abs/10.1177/135910539600100408?download=true

This just seems to me like an even lower level of 'MUS'ing people and a survey literally of s**t-life syndrome as we all know it gets called by the professions and services who we are needing to know it is a specific ME/CFS medical condition and treat it with the proper medical curiosity it requires - PASS or not to add a bit of TATT

I'm left not really feeling like racking my brain is worth the effort to see what possible way this could accurately show a miracle remedy that gave someone 40% less illness actually did so. Which would make it pretty bad based on that finger in the wind test because at that level I'd think most of the existing ones would absolutely because even where they've fudged it or sidestepped it they still use ordinal scales.

The idea we don't need to measure 'how much' or 'ness' of an adjustment is just silly as even basic descriptions of Rasch talk of data needing to be ordinal or yes/no and somehow the 4 options have even broken that. Is avoid worse than adjust? Depends.. on so much. Would changing from adjust to avoid mean something had got worse or better.. depends because it could be because your kids now get the school bus.

 

[bicentennial]

It sounds like applied behavioural science doing a behavioural analysis with a view to behavioural modification. But they are in the wrong clinic.

However the Myalgic Encephalopathy Association is now renowned for insisting that even us animals must move. Definitely a Board decision.

Its a massive diversion, this obsession with behaviour in lives that are governed by rigorous disease management criteria, mild cases too.

Fastening on mild or moderate cases did not make the manipulation safer

 

[bicentennial]

I don't believe it was researched to develop a gold-standard profiling tool-kit. I think thats a slick but false impression, habitual windowdressing.

I believe it was researched to provide evidence for registration under the Medical Devices Regulation - Software Apps, with evidence & warrant of desperately grateful, approving, treasured-patient participation, thereto.

Evidentially aimed to get all BACME-staffed clinic-contracts (commercial and NHS) renewed, for a full complement of decorative CBT therapists to continue hogging budgets by displacing other fields doing core work.

But then I'm the "difficult" patient, profiled, so in anomaly the evidence of record is: "difficult patient ... would say that ... .. lets do another NDA"

 

[MrMagoo]

I'm taking up Tyson's challenge to examine the methodology in the book and Wikipedia article she recommended to me (to shut me up) about the validity of her methods. I bought the book. It's so heavy I couldn't hold it to read so I took my scissors to it today and cut out the relevant pages. If i can make any sense of it I'll try to put together some conclusions and post them here.

Well done but also take care!

 

[MrMagoo]

Yes. At least in my country healthcare workers are taught how to help patients set (realistic*) goals. As information about ME is severely lacking, this knowledge is most likely used on other patient groups. For my profession it is all about behavior, with the idea that healthy living will improve symptoms in some way.

*What is realistic is up to the provider, obviously the patient doesn't necessarily know..l

Yes I can’t remember what appointment it was, but I was “greeted” early on with a “what goals can we help with, getting out? Doing hobbies,day trips?” and I was like, well no my “goal” is to wash daily. Strangely, not something they had any help for.

 

[hotblack]

Yes I can’t remember what appointment it was, but I was “greeted” early on with a “what goals can we help with, getting out? Doing hobbies,day trips?” and I was like, well no my “goal” is to wash daily. Strangely, not something they had any help for.

My goal is for healthcare professionals to modify their behaviours so they don’t make me worse when I have to interact with them. Honestly, I’m perfectly fine at managing myself. I sometimes think it’s not me that needs help, it’s me that needs to help the people I have to work with to understand this.

 

[Jonathan Edwards]

I'm taking up Tyson's challenge to examine the methodology in the book and Wikipedia article she recommended to me (to shut me up) about the validity of her methods.

Interesting to hear your thoughts. From a brief scan through Rasch and Bowling this looks like typical sociological pseudomeasuring to me. This is exactly what I was complaining about way back when Tyson talked of 'gold standard' and then referred us to a list of studies that were more or less drivel.

 

[MrMagoo]

Not sure if this is a helpful line of thought, but are we objecting to
1. This iteration of a PROM for pwME
Or
2. Any PROM for pwME
3.would it be possible to write a PROM for ME/CFS without it being harmful?
4. Would it be possible to write a PROM for ME/CFS that is useful to pwME?

I think it’s useful to understand whether we are objecting to a PROM generally or just this one. I suspect a PROM might not be a great thing in our circumstances, but then again they do/will exist in the NHS.

I think the MEA at one point tried to argue that a PROM will happen anyway, so it is better they have input.

I think it’s useful just to hash out where the line is between the problem being PROM and/or Tyson et al’s complicated and “gold standard” noise.

 

[Trish]

Not sure if this is a helpful line of thought, but are we objecting to
1. This iteration of a PROM for pwME
Or
2. Any PROM for pwME
3.would it be possible to write a PROM for ME/CFS without it being harmful?
4. Would it be possible to write a PROM for ME/CFS that is useful to pwME?

I am objecting on 3 grounds:

• The PROMs designed so far show little undestanding of ME/CFS, PEM and the cumulative effect of exertions, so any data from them is likely to be highly misleading, and they are far too long and the options provided for each question often uninterpretable.
• The project is base on the premise that the PROMs will be used in old style rehab clinics run by physios, OT's or psych therapists, and the PROMs used as the basis for goal setting and writing care plans. They are also intended to provide data to judge the clinic's effectiveness, for which they are completely unsuitable.
• If the PROMs are used in this way, they will give MEA approval to the perpetuation of such clinics, instead of what is needed most, which is specialist physician led provision that caters for all severity levels, not just outpatient clinics for newly diagnosed mild to moderate cases.

FUNCAP is an example of a much better PROM, shorter, clear, easy to use, and fully takes into account the cumulative effect of exertion and how often we can do something, and it's already being used in some studies and apps such as Visible.

 

[MrMagoo]

I am objecting on 3 grounds:

• The PROMs designed so far show little undestanding of ME/CFS, PEM and the cumulative effect of exertions, so any data from them is likely to be highly misleading, and they are far too long and the options provided for each question often uninterpretable.
• The project is base on the premise that the PROMs will be used in old style rehab clinics run by physios, OT's or psych therapists, and the PROMs used as the basis for goal setting and writing care plans. They are also intended to provide data to judge the clinic's effectiveness, for which they are completely unsuitable.
• If the PROMs are used in this way, they will give MEA approval to the perpetuation of such clinics, instead of what is needed most, which is specialist physician led provision that caters for all severity levels, not just outpatient clinics for newly diagnosed mild to moderate cases.

FUNCAP is an example of a much better PROM, shorter, clear, easy to use, and fully takes into account the cumulative effect of exertion and how often we can do something, and it's already being used in some studies and apps such as Visible.

So would I be right to summarise as the problem being Tyson et al?