UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

Amw66 said:
Response received from Sarah Tyson .

How will this be used. Protocol for analysis / stats available ?
No the stats plans has not been publicised; this isn’t a trial. But we are using response rates, floor and ceiling effects, Cronbach’s alpha and inter-item correlation to assess construct validity including uni-dimensionality and redundancy; thematic content analysis for content validity; Spearmann rank corelations for criterion-related validity; analysis of variance for discriminant validity. We have not assessed ret-test reliability here as the CAN-Meis not intended to be used repeatedly. In the other assessments were we have assessed this, we have used independent t-tests and analysis of variance; the mean and standard deviations of the differences to calculate the minimum detectable difference; and analysis of variance to assess sensitivity to change.
None of the assessments in the toolkit are related to the professional background of the person who is receiving them. They would be used by a consultant/doctor as much as any other professional.
If you have a look at my replies to other comments, you will see that we have taken great care to make the the toolkit relevant and accessible to people who are more severely affected. This is reflected in the high proportion of participants who are more severely affected.
This assessment does not refer to PEM, there is a separate assessment for that. The paper regarding the development of the assessment will submitted for publication shortly.
In the analysis, we will be looking how people’s needs differ at different levels of severity.
Click to expand...

Really?
Amw66 said:
They would be used by a consultant/doctor as much as any other professional.
Click to expand...

Doesn't sound the type of healthcare anyone would want for any other illness? She's hardly forwarded the argument she has in mind when she says this well - how can these be used to help someone's health?


And no, really no proper scientific medicine would be using any of these types of stats tools layered on each other - particularly when such poor checking/regulation of the input data has occured, this hasn't operated like even an experiment where they've drawn x number of people meeting y criteria in and asked them to do z.

They've used MEA to distribute extremely long surveys many won't be able to complete without detriment to their accuracy, and with many ambiguously-written questions that the intention of how they will be used have been hidden ( a massive informed consent issue) to use so many stats it looks like a fishing expedition on an apparently endless and not either verified or confirmed from the list within the new guideline (to ensure they are 'on task') load of features.

SO we neither know the symptoms/aspects being measured, nor the weighting of each, nor who on the people they've recruited. Which makes the quoting of means and std deviations laughable - what do they think those mean? deviation from the middle of what 'group' on what 'factor'? You have to at least be very sure on one (either the group or the factor and how it operates) for that deviation to means something because it is a check of eg representativeness or range of the then dependent factor. ?



The problem is that even if we did finally get some decent, normal, clever independent physicians who saw a pwme and didn't have a bias so just looked into this with an open mind

Then somehow rather than this sort of thing reflect on those who designed it, and then empathising with us on the type of thing we've had to put up with and weather for all these years, hence asking for/stating a complete change of specialism being 'in charge' is necessary...

somehow this ends up reflecting on us as if we must have done something to deserve it, or that we had any power in asking for this.

SO it is important for our reputation as pwme and how we are ever to be seen and respected by other staff and scientists going forward that MEA reject this. And say it 'isn't ours [pwme] as an invention'

or as if our condition is this complicated - when it mightn't be the most straightforward illness but actually if you take out the having to pander to a political compromise/needs of staff it at least means you are just talking about the actual illness not having to include some 'it might also be a delusion' aspect etc.

what we need is for us to filter out other people's wishes and reframing based on their wanting to keep their jobs and solve their own cognitive dissonance, as they were or not wanting to have to set history straight.

So we can just say it is a condition of PEM/PESE (and deterioration if we overdo it) along with x symptoms that we know are pertinent. ANd therefore comes in different severities which have vastly different debility patterns - but that these link on a spectrum because that 'environment' and deterioration means 'within individuals' can move on that spectrum. Just like with other illnesses where you go through stages and get worse.

We need something looking to keep the nuance and the parts that are important by severity, but strip out the political compromise stuff. The worst of all is trying to hide that, make us 'mysterious' by their working and so even if it creates more work we can't strip out the stuff that's the old school 'but it might be how they think' additions because they've embedded it into all the other measures by using this weird method and strange 'psychometrics' instead of basic, straightforward data.

The most basic right we all have after everything we've been through as a group should be access to our own data - to be able to check its vericacy and know its source, see if it has been used in the way expected when that answer was given. Given how many errors and updates to the concepts there have been in the past. I can't see how this is useful or interrogable?
 
Last edited:
I don’t think it’s anything “worse” than setting up a framework which will generate positive outcomes for NHS commissioners. But it could be more sinister I suppose. It comes across as “the tail wagging the dog”

The fact that nobody can give a concise, relevant, meaningful and substantial answer about this project is a big red flag. As is Sarah Tyson’s conduct. We’ve had insults, obfuscation and the dismissal of wearables as an esoteric unreliable something which the NHS cannot comprehend - despite other parts of the NHS using them.
 
Haveyoutriedyoga said:
Sarah, if you’re still visiting the forum and reading this: you’re still talking down to people and it is rude. You could answer the question without pointing out that you have answered the question elsewhere (in other comments which the asker has not read) and without assuming a lack of knowledge (about the application of statistical tests) as demonstrated by your attempt to haze with a list of tests, presented without context that would make the list meaningful.
Click to expand...
To be fair, I'm not sure she gets the statistical tests she is citing - so I wouldn't assume that tone on that part is due to her assuming we know less than she does but that she perhaps is being passed on the info etc because there is a stats expert in her team/it's not her specific role.
 
bobbler said:
To be fair, I'm not sure she gets the statistical tests she is citing - so I wouldn't assume that tone on that part is due to her assuming we know less than she does but that she perhaps is being passed on the info etc because there is a stats expert in her team/it's not her specific role.
Click to expand...
It’s all part of a pattern of very defensive reactions though. Look at the tone without that part!
 
MrMagoo said:
I don’t think it’s anything “worse” than setting up a framework which will generate positive outcomes for NHS commissioners. But it could be more sinister I suppose. It comes across as “the tail wagging the dog”

The fact that nobody can give a concise, relevant, meaningful and substantial answer about this project is a big red flag. As is Sarah Tyson’s conduct. We’ve had insults, obfuscation and the dismissal of wearables as an esoteric unreliable something which the NHS cannot comprehend - despite other parts of the NHS using them.
Click to expand...
It is worse. Because the norm with these sorts of things is that you are coming up with 10 measures, and everyone who needs to be around a table then scrutinises these as measures and their weightings and methods - I've been part of how these things are done for other things.

There is a reason it is done transparently.

You then also get 'raw data' per quesitn, and you don't have to imagine (but not know) that because there are 50 other questions when people started not caring or not spotting the error in the question means they misread it etc. And can at least argue on x point if they then change something because 'there is no movement in question 9 from before and after treatment in 90% of attendees' because you have the raw data for each question and can see each question.

This strange methodology has made the data uninterrogable, unchangeable and no idea where the weightings are. For all we know on a black box algorithm it could be that only 3 questions have strong enough weighting and chance of variability due to ceiling/floor effects on their scales that the rest are just window dressing.

That's easily done - as we've seen with the critique of the CFQ scale, you can't get more exhausted than 5 and start on 5 because they've calibrated it wrongly for ME/CFS and so only those who don't have ME/CFS but have CF/something that won't get worse but might 'improve over 6months' will have changeable scores and therefore their 'movement' ends up having more impact that anyone else's as noone else's can move (due to the maximum being so low).
 
MrMagoo said:
Ok well we can all use technical jargon to describe something that isn’t very good.

Anybody able to translate?

she didn’t answer all of the questions did she?
Click to expand...

Nope

I think the two important bits of what she did say to focus on are:

Amw66 said:
This assessment does not refer to PEM, there is a separate assessment for that.

The paper regarding the development of the assessment will submitted for publication shortly.
Click to expand...

So some of us might want to be looking at whether we can get radical rest in.

The sad truth is that whilst the NIH at least did provide the raw data

I'm not sure the same standards apply here - which given the context (why isn't that a norm for this country, and then there is the fact it is a project funded by a charity for the illness, provided access to the participants and the 'trust' on which they filled it in despite the really worrying consent issues, and is about those with the illness) is a very important discussion issue in itself
 
All the more reason you’d expect Sarah to explain in succinctly in plain language. Rather than a list of expert jargon and a whole sentence explaining how they did something not here, but on a different project?

I think PEM is separated because it can’t be “fixed”
 
From Sarah Tyson .
I'll respond - probably tomorrow.
If anyone wants me to include anything specific , let me know

This is the stats plan and was outlined in the grant application. Yes, they are standard stats for psychometric analysis as recommended in the COSMIN guidelines (I’m conducting a study on measurement properties • COSMIN) A stats plan for psychometric testing wouldn’t get published as it is covered in the COSMIN guidelines. It is only really the stats plan for large Phase III trials that are published, and then rarely in a peer-reviewed journal.
I am sorry you feel the survey is inaccessible. Our approach is based on the advice from the 25% group, DecodeME team, PhysioforME and Caroline Kingdon, plus the severely affected people in our advisory group and I thought it was pretty much spot on, tbh. The completion rates are very high, which suggests that the survey is acceptable and feasible for all participants, and I haven’t had any comments (either as feedback or people contacting me directly) in this or previous surveys indicating that they were harmed, or they did not complete because it was too taxing. But I am always keen to learn how to improve things, so please tell me more. Which parts do you think are inaccessible? Why? What should we do to make them more accessible?
 
And more from Sarah

I would dearly love for there to be an equivalence of provision for ME/ CFS to say MS/ ALS, but this , in my opinion , will do little to provide this .my statement


ST -
I would dearly love that too, as would many others. But it isn’t going to happen by just demanding or saying it is in the NICE guidelines. In order to expand an existing service or develop a new one a service or trust has to prepare a detailed business plan, and this has to complete with all the conditions and issues that are trying to catch the eye of funders who do not have enough money or resources. One of the big barriers to an effective business plan for ME services is the lack of basic information about the demand in their area; the number of potential patients; the difficulties they face (ie symptoms PEM, disabilities); the care and services they need; and the extent to which current services met those needs. The toolkit, as well as providing information for individual patients, will provide information at a national and local level to support such business cases. It isnt a complete answer on its own, of course but it can make a contribution to move things forward.


Many of the questions are either not relevant , not relevant enough , or fail to hit the mark. My statement


ST
OK. I’m interested to know more. Which questions do you feel are irrelevant? Or not relevant enough? How? In what way do they fail to hit the mark?
One of challenges of producing these types of assessment tool is that there is such a wide range of ways in which people with ME are impacted. Which is why the protypes tend to be rather long, because we are aiming to be comprehensive. Part of the analysis involves working out which questions are not relevant or are only relevant to a sub-set of participants. And reducing them, or producing sub-scales for specific groups.


There are so many questions for data capture, but little nuance.myvsyatement

ST
As above, there are lot of questions because we are trying to be comprehensive. One of the aims of the analysis is to look at how the number of questions can be reduced in a rigorous and scientific way.
These types of assessment tools aren’t really for nuance. The aim isn't to capture everything there is to say or know on each topic, it is to highlight the issues which are important for the patient to open up further assessment and discussion with clinicians about management. That is where the nuance happens. Essentially the toolkit measures mop up the relatively straight forward, standardisable elements of an assessment, leaving more time and energy for the more complex, subjective, nuanced stuff.

How much pwME input was there to pull questionnaires together ? My query

ST-
People with ME are front and centre of the whole project. It is led by people with ME. The work is being done by people with ME. Four of the five research team members have ME/CFS (including me), the project is backed by a ME advisory group of 30 members who contribute to every stage of the project, plus we have had feedback from thousands of participants. The 25% group are part of the advisory group and have advised us throughout. There is also an advisory group of clinicians who work in ME services, most of whom also have personal experience of ME.
It is important to recognise that the aim of this project is to develop the toolkit. The version of each tool that we survey is a protype version It is drawn up after a scoping review of the relevant literature and review by the research team and advisory groups based on our lived experience. Then we devise the survey, and it is extensively piloted before being launched. We use the data from this prototype to make revisions based on the feedback and results of the statistics. Sometimes, we need to check out the revisions made with another round of data collection, as we did for the TIMES and PASS, but it is unlikely in the case of CNA-ME.
 
All that work by so many people, and still they don't produce anything that makes sense to me as useful either for helping individual patients or improving services.
It's all still based on therapist ĺed services as per BACME.
 
Amw66 said:
And more from Sarah

I would dearly love for there to be an equivalence of provision for ME/ CFS to say MS/ ALS, but this , in my opinion , will do little to provide this .my statement


ST -
I would dearly love that too, as would many others. But it isn’t going to happen by just demanding or saying it is in the NICE guidelines. In order to expand an existing service or develop a new one a service or trust has to prepare a detailed business plan, and this has to complete with all the conditions and issues that are trying to catch the eye of funders who do not have enough money or resources. One of the big barriers to an effective business plan for ME services is the lack of basic information about the demand in their area; the number of potential patients; the difficulties they face (ie symptoms PEM, disabilities); the care and services they need; and the extent to which current services met those needs. The toolkit, as well as providing information for individual patients, will provide information at a national and local level to support such business cases. It isnt a complete answer on its own, of course but it can make a contribution to move things forward.


Many of the questions are either not relevant , not relevant enough , or fail to hit the mark. My statement


ST
OK. I’m interested to know more. Which questions do you feel are irrelevant? Or not relevant enough? How? In what way do they fail to hit the mark?
One of challenges of producing these types of assessment tool is that there is such a wide range of ways in which people with ME are impacted. Which is why the protypes tend to be rather long, because we are aiming to be comprehensive. Part of the analysis involves working out which questions are not relevant or are only relevant to a sub-set of participants. And reducing them, or producing sub-scales for specific groups.


There are so many questions for data capture, but little nuance.myvsyatement

ST
As above, there are lot of questions because we are trying to be comprehensive. One of the aims of the analysis is to look at how the number of questions can be reduced in a rigorous and scientific way.
These types of assessment tools aren’t really for nuance. The aim isn't to capture everything there is to say or know on each topic, it is to highlight the issues which are important for the patient to open up further assessment and discussion with clinicians about management. That is where the nuance happens. Essentially the toolkit measures mop up the relatively straight forward, standardisable elements of an assessment, leaving more time and energy for the more complex, subjective, nuanced stuff.

How much pwME input was there to pull questionnaires together ? My query

ST-
People with ME are front and centre of the whole project. It is led by people with ME. The work is being done by people with ME. Four of the five research team members have ME/CFS (including me), the project is backed by a ME advisory group of 30 members who contribute to every stage of the project, plus we have had feedback from thousands of participants. The 25% group are part of the advisory group and have advised us throughout. There is also an advisory group of clinicians who work in ME services, most of whom also have personal experience of ME.
It is important to recognise that the aim of this project is to develop the toolkit. The version of each tool that we survey is a protype version It is drawn up after a scoping review of the relevant literature and review by the research team and advisory groups based on our lived experience. Then we devise the survey, and it is extensively piloted before being launched. We use the data from this prototype to make revisions based on the feedback and results of the statistics. Sometimes, we need to check out the revisions made with another round of data collection, as we did for the TIMES and PASS, but it is unlikely in the case of CNA-ME.
Click to expand...

  1. Re,'"...One of the big barriers to an effective business plan for ME services is the lack of basic information about the demand in their area; the number of potential patients; the difficulties they face (ie symptoms PEM, disabilities); the care and services they need; and the extent to which current services met those needs...."
  2. Re, "...People with ME are front and centre of the whole project. It is led by people with ME. The work is being done by people with ME...."

REALity check and 2 decades of dedicated lobbying etc........

  • Sarah Archer was right, only a radical change at the top of NHSE and DHSC via ministerial intervention will do the trick and no amount of assessment toolkits and data will deliver that change imho. The data is there but it's being buried alive!
  • This is because you can lead the real obstacle, NHS E to water, but you cannot make it (or the non execs we know so well) drink.........

FYI All the above, 1 & 2, applies has been assessed and has resulted in a business case in our ICS;
A Specification/Pathway (and potential for a robust, accountable Service Model - through rigourous Key Performance Indicators KPIs, Change Audits) is being finally worked through, ahead of ICS board endorsement and Procurement, which will also have ME patient/ Carer involvement.

Why has it it taken 2 decades?
Changes precipitated in 2010 political NHS upheaval.....and changes in governance/ decison making ... we were nearly there in 2009-2012...
*****2012, NHS England and all the other littel NHS 'arms length' bodies..., now all subsumed within****
  • 2024- Due to delays precipitated by NHS England's decision making on specific funding arrangments this year, time is too tight to secure agreement as was hoped, in March 2025.
  • 2022-2024 Due to delays in the DHSC Delivery report, ICSs are ham strung in their decision making. They have had to address Health Inequalities without the benefit of the Specialised Funding Stream mechaismsm for ME which NHS England has off loaded onto regional ICSs.
  • 'We" Suffolk & Norfolk, here have had to drive this beast ourselves (with the help of some truely wonderful, exceptional Suffolk Commissioners/Contracts and Transformation managers since 2014- year 10 years!....) Plus Health Overview and Scrutiny, Elected local members, Health watch ? LINk before from 2006/2007.....!2 decades!

Sarah Archer was right, only a radical change at the top of NHSE and DHSC via ministerial intervention will do the trick and no amount of assessment toolkits and data will deliver that change imho.

This is because you can lead the NHS E to water, but you cannot make it drink.........

I hope we can pull this off locally to prove it can be done, but we have been there before, so many times over 2 decades.....
 
Trish said:
All that work by so many people, and still they don't produce anything that makes sense to me as useful either for helping individual patients or improving services.
It's all still based on therapist ĺed services as per BACME.
Click to expand...
I sent the gist of a posting to one of our service coproducers who is on S4ME but didn't see it. He had sign up to the Toolkit stuff..
'
UK:ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services, 2023

'It occurs to me that use of a questionnaire like this by a therapist is a way of distancing themselves from the realities of the pwME's life. Especially if they are seeing pwME in groups, and not spending time in the group listening to individuals' situations and helping them resolve real life difficulties with living with ME/CFS....'

Do you identify A?
His reply was really alarming.
 
Last edited:
Trish said:
All that work by so many people, and still they don't produce anything that makes sense to me as useful either for helping individual patients or improving services.
It's all still based on therapist ĺed services as per BACME.
Click to expand...

From A Suffolk,

'is this the one Sarah Tyson is developing? I signed up to help and I found the process too difficult and tedious, incredibly long questionnaires. At the start they were using pwME’s precious little energy to create acronyms!

Then they leaked everyone’s email because they didn’t BCC, in breach of GDPR, and I pulled out of the process.

I haven’t read the most recent version of the questionnaires so I can’t really comment.'
 
You must log in or register to reply here.
Back
Top Bottom