Dear Sarah Tyson, Pete Gladwell and team, and MEA trustees,
We are writing to both the MEA Trustees as funders of the Clinical Toolkit project, and to the research team to ask for clarification of how this project is expected to achieve its stated aims. We start with a brief explanation of some of our concerns, and follow with a series of requests and questions.
We contend that it is incumbent on funders and researchers to explain fully the way in which the data collected from volunteer patients is intended to be interpreted and applied to patient care and service provision before asking them to participate in such data collection. Assurances that the toolkit and PROMs will be beneficial are not sufficient, particularly in the context of ME/CFS where PROMs have been used to validate harmful treatments and beliefs about pwME.
Having carefully read Neil Riley's letter (4 April 2024), the original MEA article about the project (8 May 2023) and Sarah Tyson's posts on the S4ME forum, we remain unsure about the aims of this project, reported initially as:
"The main outcome from this research will be a clinical toolkit and a greater understanding of patient difficulties, their needs, and satisfaction with service provision, along with a better appreciation of the assessment challenges that clinicians working in specialist services face and how they might be overcome.
Application of the toolkit will be of benefit to patients, to clinicians who are tasked with providing tailored care and support, and to healthcare commissioners who want to review and improve existing service provision – or create new services – that adopts the NICE Guideline recommendations."
This explanation suggests, and Sarah Tyson has stated in forum posts, that the project is not intended to produce outcome measures for use in clinical trials of new treatments. Rather that the focus is on producing materials that will facilitate implementation of the 2021 NICE guideline by improving patient care and services.
It seems that, with this project's focus on PROMs rather than other information gathering and monitoring tools, the aim is to produce more ME/CFS specific subjective outcome measures to directly replace the SF-36 PF and CFQ measures commonly used by the past GET/CBT clinics to claim effectiveness of short courses of rehabilitation style 'treatments'.
We are concerned that the new PROMs are intended to be used to provide
quantitative data to be used to support the continuation of rehabilitation style therapist led clinical services that only provide short term 'treatment' for mild and moderate pwME, rather than their replacement with NICE compliant doctor led diagnostic and care services with provision for all severity levels.
Given that there is no effective treatment for ME/CFS, and the NICE guideline recommends providing pwME with tools, knowledge and support for symptom contingent self management of exertion, any changes on subjective outcome measures (PROMs) relating to symptoms and activity levels will be a refection of natural fluctuations and lifestyle changes, not a measure of the usefulness of the clinic's input. They should therefore not be used as a gauge of service effectiveness.
The MEA article also states that:
"The toolkit will address the assessment needs and research recommendations (for a core outcomes database) identified in the
2021 NICE Clinical Guideline on ME/CFS. It will be produced following consultation with patients and with clinicians to ensure the toolkit can record accurate and reliable data. Then it will be made available to the network of services in England and in Northern Ireland, Scotland, and Wales, when new specialist services are commissioned."
It seems that the leaders of this project have interpreted the need for a core outcome database to mean that quantitative data derived from subjective PROMs will answer this need in future treatment trials and sevice evaluation. This assumption is patently false in the case of unblinded trials, where outcome measures need to be objective.
This has led to confusion about the aims of the project and how it is expected those aims can be fulfllled by the resources, including PROMs, being developed.
Requests and questions:
1.
Research protocol and ethics approval
Please could you provide a copy of the protocol and/or detailed funding application that spells out just how the materials being developed are expected to achieve the stated aims. We would like to understand how the MEA came to fund this project, and on what basis ethics approval was given.
2.
Research preparation
The MEA article says:
"The researchers undertook extensive work in preparation of the grant application. Using recent national guidance, they established the concepts to be measured and completed scoping reviews to identify any existing measurement tools. While these revealed that nothing suitable currently existed for ME/CFS several were found that could be developed."
Please could you provide the documentation that supports this.
The only information we found was from a very superficial study with a cohort of new patients able to attend a clinic, that is, at the highest functional level of ME/CFS.
Evaluating the ability of patient reported outcome measures to represent the functional limitation of people living with ME/CFS, 2023, Jones, Gladwell
Discussed on the forum
here
And another equally superficial study of the Chalder Fatigue Questionnaire:
Exploring the content validity of the Chalder Fatigue Scale using cognitive interviewing in an ME/CFS population
Discussed on the forum
here
This did not address the multiple problems with CFQ identified by the S4ME paper, some of which such as arbitrary weighting and ceiling effects are relevant for this project.
Submission to the public review on common data elements for ME/CFS: Problems with the Chalder Fatigue Questionnaire
We assume from the quoted statement that there is some other documentation seen by the MEA that provides a more in depth analysis of "concepts to be measured", and spelling out how new materiais are expected to achieve a way of "measuring concepts" in a clinically useful way for patients of all severity levels, that avoid the multiple failings of subjective measures, and moreover that will provide information for planning and evaluating NHS specialist services.
3.
Subjective versus objective monitoring tools and outcome measures
What is the rationale for opting to develop solely subjective questionnaire based outcome measures (PROMs) despite the well know problems of their use in unblinded trials and that have been used in service evaluation to justify use of harmful CBT/GET? What measures are being taken to ensure the PROMs being developed will provide valid evidence of clinically significant change that is not subject to the same subjective biases as all other PROMs (Calling PROMs gold standard and objective does not make them so).
Given the history of refusal to use objective outcome measures that do not support the short term spurious positive outcomes on subjective PROMs by CBT/GET researchers and clinics, it is particularly hard to accept the promotion of PROMs to the exclusion of objective measures in this project.
What is the rationale for rejection of objective measures such as continuous monitoring with wearables for activity, time upright, HR and HRV; cognitive testing; hand grip strength and fatiguability; 2 day CPET and other exercise challenges as monitoring tools for assisting with clinical care and as outcome measures for clinical trials?
What is the rationale for rejection of continuous monitoring using wearables and daily symptom and activity monitoring using apps. to assist pwME with activity management if they wish to use them, and to help with clinical reviews by providing a record of activity and symptoms patterns that does not rely on memory over long periods?
4. "Gold standard objective measures"
The MEA article says:
"The toolkit will address the assessment needs and research recommendations (for a core outcomes database) identified in the 2021 NICE Clinical Guideline on ME/CFS. It will be produced following consultation with patients and with clinicians to ensure the toolkit can record accurate and reliable data."
Sarah Tyson claimed in forum posts that the PROMs being developed with provide "gold standard objective data".
It is therefore clear that the PROMs are not intended solely as records of symptoms and activity levels as a basis for clinical review, but are also intended to provide numerical scores that can be used as outcome measures for service evaluation and clinical trials. We contend that in asking patients to give informed consent to participating in testing the PROMs, they should be informed of how the numbers derived will be calculated and interpreted, so they can give feedback on whether the scores have valid clinical meaning.
Please could you, for both the already developed PROMs and ones still in development, provide the intended algorithm for turning the answers to the questionnaires into numerical scores, and the rationale supporting the validity of such algoritms in each case. This will include what exactly is being 'measured', is it intended to be a linear scale, what change is deemed clinically significant, how will that be determined? If and algorithm is additive, how will it cope with possibly giving false impression of overall improvement, when minor improvements in some less troublesome aspects numerically outweight major detrioration in a single aspect, and when apparent improvement in symptom severity is a result of lifestyle changes, not of improvement in the underlying disease?
Can you provide an assurance that answers to questionnaires will not be spuriously consolidated to produce global measures of improvement, enabling multiple minor subjective improvements to outweigh real changes in function and so give a false impression of overall clinically significant improvement?
5. Problems with PASS and the replacement of PEM with 'symptoms after exertion'.
What is the rationale for removing the term PEM from the PROMs and replacing it with the more general and not disease specific 'symptoms after exertion'? This is a major concern given the high risk of ME/CFS being merged into generic FND, MUS etc. and resulting inappropriate treatment.
Following on from our detailed review of the PASS questionnaire, as spelled out in our
letter of 2nd April 2024, how are all of our concerns being addressed? Is the PASS PROM being redesigned from scratch as a much shorter questionnaire and using an accurate description of PEM as delayed and lasting more than 24 hours? Is it being considered that PASS be scrapped, and PEM be included instead, with a definition, in the symptom questionnaire, or that its occurence be assessed more accurately by longitudinal monitoring of symptoms and exertion using technology, as we have suggested?
6 The role of PROMs in clinical care
Please could you provide the rationale for using lengthy PROMs to assist with clinical care rather than a much simpler, shorter one page tick box list of symptoms and common activities of daily living, with space for the pwME to note particular concerns; the MEA severity scale; and, where the pwME chooses to use them, summary scores or graphs from continuous monitoring using wearables, apps or paper diaries.
Have pwME been asked which approach they would prefer? Have GP's or specialist doctors been asked which would be most useful to help them monitor pwME in their care? Are you aware that BACME already has a document for clinical use that collects information about symptoms and functional limitations? Have pwME been invited to critique that document? Is this research intended to replace that document or add to it?
We note that the self congratulatory
MEA update of 9th April, quotes only positive feedback from pwME, largely praising the PROMs on the basis of learning something new. This information would be better provide in documents the pwME can keep, and does not justify the use of PROMs. What is the rationale for providing patient eduction via PROMs rather than leaflets and web based resources that list symptoms, and describe PEM clearly, including possible triggers; a brief guide to pacing; and lists of sources of help and futher information?
5. Designing clinical services and service evaluation
Please could you respond to the concern that this approach using PROMs is designed to justify the continuation of therapist run clinics running short courses of 'treatment' with a behavioural and 'rehabilitation' focus. The NICE guideline does not support such a model. Rather it supports doctor led clinics with a focus on symptomatic treatments; provision of resources that enable supported self monitoring, rather than therapist led 'rehabilitation'; and more focus on the provision of accessible and appropriate medical and care support for the most severely affected, including appropriate home and hospital care.
Please could you also spell out exactly how it is envisaged the PROMs and toolkit will be used in clinics. Which specialism will be expected to administer them, what action is expected to be taken based on the results, at what stage in the diagnostic and monitoring of patients they will be repeated, how they will be expected to impact patient care, and how they will be used for service evaluation? How will the outcomes of the project be expected to contribute to changes and new provision of services?
7. Implementation by the NHS
In what way is it envisaged the PROMs and toolkit will fit in the as yet unpublished government delivery plan for ME/CFS, and the NHS training modules and other plans we have yet to see? We are concerned that it could be used to persuade funding bodies to fund a continuation of therapist led clinics providing short rehabilitation courses for the milder patients, instead of doctor led care with NICE compliant provision for all severity levels.
____________________
Comment
At present we cannot see how anything produced in this project will provide better information for patients, clinicians, or service providers than the already existing NICE guideline, a simple symtom, PEM and ADL list with tick boxes, the MEA severity scale, and the development of resources to help pwME manage their activity levels, including in the form of leaflets and web resources that explain symptoms, PEM, activity management, and provision and guidance on use of wearables, apps, diaries etc.
In the current situation where there is not effective treatment, service evaluation should not be based on numerical scores based on subjective PROMs. that will merely reflect natural fluctuations and therapist influence. It should be based on whether there is NICE compliant provision of diagnosis, patient education and support, and provision of appropriate services for all severity levels, and whether patients are able to access these services when needed.
We contend it is not ethical or clinically valid to ask pwME to test PROMs, as in this project, without providing the full picture of how the data will be used, including algorithms, clinically significant cut off points, which specialism will administer it, how they will interpret the data and for what purpose, and inviting pwME to express a view on alternative resources and outcome measures such as those we have suggested.
_________________
We assume the research team will be wiliing and able to provide the information we have requested and to make it public on both the MEA website and the S4ME forum as a sign of ongoing commitment to patient collaboration and better outcomes of the project. Until pwME fully understand how the aims of the project are intended to be realised, pwME are being asked to participate in the research without fully informed consent.
We hope this letter and our requests will be received and responded to in the spirit intended, of open and collaborative discussion with a view to improving the outcomes of this project for the benefit of pwME. We continue to keep the door open to Sarah Tyson, Pete Gladwell, MEA trustees and everyone working on and participating in this project, to join our members in constructive discussion, despite past slurs on our members. We look to you all to act professionally for the good of pwME.
Thank you.
Trish Davis....
