Open letters to the UK ME Association trustees about a research project developing PROMs, led by Sarah Tyson

In 2023 the UK ME Association announced it was funding a project, currently ongoing, which is developing Patient Reported Outcome Measures (PROMs), announced here:
ME Association funds research for a new clinical assessment toolkit in NHS ME/CFS specialist services.

The project is led by Professor Sarah Tyson, who joined our thread discussing it here:
https://www.s4me.info/threads/uk-me...in-nhs-me-cfs-specialist-services-2023.33221/

During that discussion, members expressed concerns about the project. As a result of these concerns, the Science for ME committee have today sent two letters to the Trustees of the MEA, copied to Sarah Tyson.

The letters are in the next two posts. We will post any replies on this thread.

To keep discussion of the project in one place, please post any comments on issues raised in the letters on the main public thread.

This thread will be locked so discussion is not split over several threads.

 

Letter 1

To the Chair and Trustees of the ME Association
Copy to Professor Sarah Tyson

Subject: Formal complaint about the leader of an MEA funded research project.

Dear Neil Riley and trustees,

We wish to lodge a formal complaint about Professor Sarah Tyson's behaviour towards members of the Science for ME forum while acting in her role as lead researcher on the MEA funded project developing a clinical assessment toolkit for use in UK NHS ME/CFS clinics.

As funders and overseers of the project, we are asking you to take urgent action, and to inform us what action has been taken. This letter will be made public on the Science for ME forum, along with any reply we receive from you, unless you specify a requirement for confidentiality of all or part of your reply.

Professor Tyson was welcomed by members of the Science for ME forum when she joined the thread where members are discussing the project and have shared their thoughts about the two PROMS that have been tested on people with ME/CFS so far.

Forum discussions are open for any member to comment freely so long as their posts are compliant with forum rules. Moderators do not tone police, but they do ensure members follow the rules, including not making personal attacks on other members. No posts on the thread commenting on the PROMS have been reported to moderators as making personal attacks. Professor Tyson did not report any posts to moderators as she is free to do, and we do not believe any members attacked her personally or behaved improperly towards her. Some members have been forthright in their criticisms of some aspects of the project, as they are about any research project we discuss. That is a large part of what the forum is for.

The forum's aims and values are available for all to see:

"We value four things especially:
· high quality scientific research, which will be necessary to produce effective treatments;
· open, critical discussion of claims and ideas;
· mutual support and respect;
· equality."
https://www.s4me.info/threads/welcome.25466/

Several members spent considerable time spelling out their concerns about the PASS questionniare, which is intended to 'measure' PEM.

Prof Tyson was willing, at first, to join our discussion of PASS. However, she suddenly turned on members with a completely unprofessional, unwarranted and shocking attack on everyone involved in the discussion.

In response to a suggestion that the project might be unsuccessful, and valid concern that badly designed PROMS can cause harm to pwME, Prof Tyson said:

"Don’t be silly. You haven’t actually presented any evidence that the project would be unsuccessful, or any rationale why completing a questionnaire would cause harm, which is clearly implausible."

After providing links to some of her own research work on stroke rehabilitation, Professor Tyson continued:

"I am not engaging with this thread any further now. I had joined it in anticipation of a constructive, critical discussion which could help progress the project and use of measurement tools in ME. Instead, I have been met with diatribes of, frankly, hysterical projection, catastrophising, conspiracy theories, overt hostility and insults. This is the sort of nonsense that given people with ME such a bad reputation, and I have no interest in being further involved."
https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-12#post-517050

While we understand that everyone has a bad day sometimes, we cannot understand why Prof Tyson has remained silent, making neither an apology nor asking for her attack to be removed. It is notable that members, though clearly shocked and hurt by this attack, have responded politely and constructively.

We are not asking you to tell Prof Tyson to apologise to our members, as any apology elicited in such a way, and a month after it the attack was made, is hardly likely to be sincere. What we are asking you to do is to explain to Prof Tyson why the words she used are so inappropriate, not only because the attack was unwarranted, but because the words chosen were so specifically inappropriate to be addressed to people with ME/CFS, many of whom have suffered appalling treatment from clinicians.

Several members have commented on the inappropriateness for this patient population of the specific accusations made by Professor Tyson.

https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-12#post-517120
https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-13#post-517343
https://www.s4me.info/threads/uk-me...alist-services-2023.33221/page-13#post-517347

Eariler in the discussion, Professor Tyson said she is not interested in history and only wants to move forward. Perhaps this is at the root of the problem, as Prof Tyson seems remarkably unaware of the flaws and pitfalls of PROMS when used in ME/CFS, and seems to be taking her lead from whatever BACME members are telling her about harms caused to pwME and about the veracity or otherwise of accusations by Sharpe et al of harassment when valid critiques are made of their research.
https://www.s4me.info/threads/uk-me...ialist-services-2023.33221/page-3#post-474276

It is with sadness that we send this letter. We do not lightly make such a serious complaint, and we hold no personal animosity towards Professor Tyson who we wish well. It is unfortunate that she appears to be out of her depth in understanding of ME/CFS, and unable to adjust to working with, and act professionally towards, people making valid criticisms of her and her team's work.

We will also be sending you a separate letter explaining our concerns about the whole project, and about the PASS PROM.

Yours sincerely,

Trish Davis and Maree Candish on behalf of the committee of the Science for ME forum.

 

Letter 2

To the Chair and Trustees of the ME Association
Copy to Professor Sarah Tyson

Subject: Concerns about an MEA funded research project.

Dear Neil Riley and trustees,

In response to concerns raised by many members of the Science for ME forum about the MEA funded project developing a clinical assessment toolkit for use by NHS ME/CFS clinics, and Dr Charles Shepherd's invitation to provide a summary of those concerns, we, the Science for ME committee, provide the following.

Given the MEA's role in funding this project, we are asking you to review the project. Our recommendation is that the project be halted and discussions held on a better way forward that avoids perpetuating the mistakes of the past, and improves clinical care, rather than the old model of therapist-led clinics that relied on Patient Reported Outcome Measures (PROMs) for patient records and for service evaluation.

We summarise the problems we see with the project, followed by a proposed alternative approach.

Problems with the project

1. Problems with the project brief
The assumption in the design of this project appears to be that the problem with past monitoring of patients and clinics was that the wrong PROMS were used, and that could be fixed by new ME/CFS specific PROMS designed by ME/CFS experts, PROMS experts and people with ME/CFS. While we agree that the past PROMS were flawed, we contend that the assumption that all that is needed is to create better PROMS, is false. It perpetuates the incorrect belief that subjective questionnaires, with all the attendant problems of undue influence, and faulty recall, are the only or best way to monitor the health and function of people with ME/CFS and the effectiveness of ME/CFS services. We are surprised that the MEA, which has been at the forefront of fighting against the false evidence created by subjective questionnaires, is so willing to support the development of a Clinical Toolkit consisting entirely of them.

It ignores the possibilities that new technology creates for easy to use objective measurement tools, despite these being commonly used by people with ME/CFS to help with pacing and despite their use in the management of other diseases.

The project is using 90,000 pounds of MEA members' contributions to develop tools for use in NHS clinics, clinics that are part of a model of service delivery that have not served people with ME/CFS well. Arguably, the NHS should be funding the development of tools its own service delivery and quality management tools. The project seems to be based in, and assume the ongoing existence, of a rehabilitation paradigm for ME/CFS, with baselines and goal setting, and the patient as the receiver of the clinician's wisdom. It assumes that clinician input makes a significant difference to disease course and that data, to be effective, has to be held and analysed by the clinician, with information fed back to the patient. However, in a disease where there are no treatments, we believe that the key is to empower the patient to manage their disease.


2. Problems with the project team, process and products
The conception of a clinical toolkit is limited by the project team's skills, beliefs and knowledge. For example, Professor Tyson has, on the forum, suggested the use of wearable technologies to gather objective data is not feasible.

The project team assumes it is not necessary to understand the history of past failings in ME/CFS clinical care, including problems with past use of subjective outcomes to support CBT/GET in service evaluation and research. Importantly, they are unwilling to contemplate the possibility that PROMs can cause harm, dismissing that possibility as 'silly', demonstrating breath-taking ignorance of the subject that they claim expertise in. PROMs have suggested that people have conditions that they do not have, such as when an 'inability to do things that people previously enjoyed' is counted as a sign of depression. This results in poorly targeted treatment, the overlooking of a real diagnosis, the bolstering psychosomatic ideology and stigmatisation. PROMS have also led to the belief that people do not have a health condition or disability, when in fact they do, resulting in missed treatment, care and access to support. PROMs with severity ceilings and ambiguous comparators and subjective bias have resulted in misleadingly positive outcomes, suggesting treatments such as GET are more effective than they are, resulting in a misallocation of funds, patient effort and harm. The summation of the presence or absence of diverse symptoms to produce an overall score can result in reported global improvements that do not reflect real improvements in quality of life.

Members of the project team have assumed that constructive criticisms from patients and clinicians with considerable knowledge and experience, are inferior to their views and have no value. They have disparaged people and critiques, and have dismissed concerns. (see our separate letter)

The project assumes experience developing and validating PROM's for short term treatment in other specific conditions is transferrable to ME/CFS, and will enable the creation of gold standard, objective and appropriate measures for long term monitoring of people with a complex, fluctuating, multi-symptom illness with no effective treatment. It assumes that numerical data derived from PROM's also measures the effectiveness of clinical care in ME/CFS, and is therefore useful for service evaluation, and potentially also as outcome measures for clinical trials, ignoring the substantial problems with changes in the frame of reference and bias.

As demonstrated in the evaluation of PASS below, there appears to be a tendency to produce PROMs that are too long and unclear. The project outcomes include unhelpful legacy terminology from an old model of care, for example 'baseline' which implies that management techniques are aiming to increase activity levels over time.

3. Problems with the PASS PROM
The PASS questionnaire, which we are told is intended to 'measure' PEM, has been made available for comment. Some members of the Science for ME forum attempted to fill in the questionnaire and found it full of ambiguities, contradictions, and questions we found impossible to answer. It was unclear what was being 'measured', as well as being far too long. The forum discussion of PASS starts here:
https://www.s4me.info/threads/uk-me...ialist-services-2023.33221/page-9#post-515595
We believe that the problems with this particular questionnaire are indicative some of the problems with the project as a whole.

a. Difficulty of quantifying PEM
Any attempt to 'measure' PEM with a numerical algorithm based on a questionnaire is likely to be either meaningless and unacceptably onerous. PEM varies in frequency, delay time, severity of symptoms, additional symptoms, duration, and loss of function. Variation in PEM depends to a large extent on the individual's life circumstances and activity management choices, not on how effective a clinic or clinician is. A PEM measure can therefore not contribute to an assessment of illness severity without also considering activity levels and patterns.

b. Replacement of PEM with 'symptoms after exertion'
Replacement of the specific phenomenon PEM with the more general 'symptoms after exertion' in the title and content of PASS means this is not a PEM questionnaire. Justifying this on the grounds that this is what some patients think PEM is, will cause confusion and hamper patient care and clinicians understanding of ME/CFS. There needs to be a clear understanding of the difference between PEM and other 'symptoms after exertion', which include fatiguablity, PEF, DOMS and exertion intolerance.

c. Conflation of daily fatiguabilty with PEM
Prof Tyson said in discussion that for people with mild ME, PEM is immediate and lasts just a few minutes, based, it seems, purely on her own experience. This conflates everyday fatiguablity with the distinct, usually delayed and longer lasting PEM. She said she hadn't heard of the concept of fatiguablity, and showed no understanding of the difference. The questionnaire reflects that confusion.

d. Assumption of single identifiable PEM triggers
There is an assumption throughout PASS that individual identifiable activities can be linked to episodes of PEM. There is no reference to the more commonly experienced cumulative effect of all activity over a day or more triggering PEM. This is compounded by the problem of relying on memory of how one felt and what one was doing in the past, up to a month ago, which may be highly unreliable.

e. Use of the term and concept baseline
The concept of 'baseline' is an idealised concept that many find fluctuates so much that it is unknown until hindsight tells us we have overstepped it. PASS assumes people know their baseline. Prof Tyson's does not appear to understand that many people's life circumstances or disease preclude finding a settled baseline.
The word 'baseline' has historical orgins as the first stage before increasing activity in GET and CBT clinics, and may not be an appropriate term when discussing PEM in clinics following the new NICE guideline. Prof Tyson's response that this is the term their advisory group prefer does not make it the best term, simply the most familiar from the past. The temptation for therapists with a 'rehabilitation' mindset may be to see a baseline as a starting point for increasing activity. Concepts such as exertion limit, energy envelope or PEM threshold may be more useful, as they focus on staying within limits rather than baselines as a starting point from which progress can be made.

f. Inappropriate use of 'rehabilitation' and 'goals'.
There is an assumption that PASS can be used to help with 'rehabilitation' and 'goal setting', particularly inappropriate terms for people living with ME/CFS, where there is no effective rehabilitation, and goal setting is fraught with assumptions that foward planning and goals are in any way appropriate, as they implicitly suggest increased activity levels, setting people up to fail. Prof Tyson showed no understanding of this problem.

g. No benchmarks for classifying activities
Questions ask for activities to be classified as very strenuous, strenuous, moderate slight or very slight. No benchmark is given of how specific activities that might be classified. When asked, Prof Tyson explained that it is about how the pwME finds the activity at the time. However, if PASS is to be used to 'measure' PEM, then someone who says they get PEM after strenuous activity, which at the time for them is walking 1000 metres, who gets much sicker and fills in PASS at their next assessment, ticks strenuous again, but this time that means shuffling 10 metres to the bathroom, how will PASS show that deterioration? This makes the questions framed that way completely meaningless.

h. Far too long and confusing
PASS is far too long, and tries to get fine detail about some aspects of symptoms after activity according to a particular view of PEM. It is more likely to cause confusion than clarity for clinicians, with the pwME needing to explain their interpretation and difficulties with PASS.

i. Not a valid monitoring measuring instrument
In attempting to turn this into a measuring instrument for monitoring individuals over time or for 'measuring' clinics in service evaluation, PASS will clearly not provide useful or objective data on a consistent linear scale, for the reasons set out above.
______________

We contend that all the above have led to a project that is repeating mistakes of the past, producing lengthy and unusable questionnaires that will not provide clinically useful data, or valid data for service evaluation, and in rejecting inclusion of objective outcome measures, deprives patients of useful tools to help with energy management such as wearables and apps, and other easily used and collated daily records such as time upright and severity scales.
_______________

Alternative approach.
We suggest the following alternative approach:

Skills, experience and approach needed in a project team for this approach:

In order to provide tools best suited to implementation of the NICE guideline, a fresh approach is needed, not just new PROMS. This requires a team with:

• In depth understanding of the flaws of the past PROMS based monitoring model, and flaws in the BPS hypothesis, Oxford criteria, and CBT/GET treatment model, so past mistakes are not repeated
• In depth understanding of ME/CFS, and the NICE guideline so mistakes are avoided such as renaming, redefining and misunderstanding PEM and use of terms from the GET model.
• Wide experience of the methods patients find most useful for monitoring their symptoms, PEM and activity levels now and for the future
• Knowledge of newer developments using technology such as wearables and apps that enable easily manageable daily symptom recording and ongoing tracking of activity over time,
• Open minds directed to find the most useful format of individual patients’ qualitative and quantitative data to efficiently and effectively inform and assist clinicians, including GPs who are most commonly the health professionals caring for people with ME/CFS, in implementing the new medical and care model, and enable long term monitoring of severity levels and care needs.

Suggested tools for development

For service evaluation
Since there is not effective treatment for ME/CFS, there is little need for complex PROMs. We suggest a simple short questionnaire that asks people using the service to report satisfaction or otherwise covering accessibility, respect, appropriate referrals for specific medical needs, symptomatic treatment, regular reviews, assistance with energy management including provision of wearables and apps if wanted, specialist hospital and home care for severely affected. Where courses are provided, numbers of participants and the percentage of people completing them should be recorded, along with simple user ratings of utility that are submitted anonymously. Service evaluation could also include a single figure for each patient based on their severity level on the MEA or Bell severity scale, age, employment status, (and, if available, objective measures such as steps tracked over time) to ensure provision of services for all severity levels, and spot unusual levels of improvement or deterioration in the clinic's patients.

For clinical care

Subjective, qualitative data
We suggest optional availability of, at most, two or 3 short questionnaires that take no more than five minutes in total to complete, and can be completed by the doctor during the consultation or by the patient in advance.

Questionnaire 1: A list of the most common ME/CFS symptoms, including physical and cognitive fatiguablity, and PEM, with tick boxes for never, yes and coping, yes and want help. Space for the pwME to write in additional troublesome symptoms
Questionnaire 2: A brief list of topics such as resting, sleeping, eating, standing, walking, lifting, bathing, dressing, reading, thinking, reacting quickly, socialising, coping with emotions, relationships, driving, working, studying, caring for children, housework, sport, financial issues, with space for the patient to note which they would like to discuss.
Questionnaire 3: The Bell or MEA severity scale, or equivalent 2 scales for physical and cognitive severity levels.

The above would be for use in focusing a clinical review, and could go in the pwME's notes, but the first 2 would not be turned into numerical data or used for service evaluation

Objective data and daily monitoring
Optional provision of wearable step, HR, HRV monitors with access to a suitable app such as Visible that provides for daily symptom and wearable data to be combined, for patients to use to assist with energy management and ideally to provide data for individual clinical monitoring records. Alternative provision of a tracking app or paper equivalent for patients to record symptom and severity scores and other data such as time upright each day.

Tracking daily data will provide pwME and their doctors with a clear and accurate picture of fluctuations, including episodes of PEM and overall trend in severity and changes in patterns of symptoms with far more accuracy than questionnaires that rely on memory.

Several of our members have suggested that monitoring would be much more effectively done using wearable activity and heart rate/HRV monitors and simple symptom and function apps. Prof Tyson was very dismissive of this approach, yet the use of technology for monitoring is very successfully already being used for other chronic conditions such as diabetes and asthma.

For clinical trials

Objective data for clinical trials (and for service evaluation if an effective treatment is established)
The lack of inclusion of objective measures in the reported outcomes of service evaluations and unblinded clinical trials has been disastrous for pwME, leading to decades of harmful treatments. Better PROMS will not solve this problem. Outcome measures must be in large part objective, and, because of the problem of the cumulative effect of exertions, single time testing such as the 6 minute walk test are not adequate objective tests. Valid outcome measures are likely to involve activity monitoring over weeks or months, time upright each day, ability to sustain employment or education, cognitive testing including cognitive fatigability, testing while in PEM, and trends in illness severity scale scores over time.
_______________

Conclusion and our request to the MEA:

Given the serious concerns we have about this whole project, we also raise concerns about the central role Professor Tyson and her colleagues are assuming for their PROMs as part of the implementation of the NICE guidelines across England. Professor Tyson said on the forum: "will also produce training materials as part of the project dissemination plans. It is possible that, in the future we will do further work to look at the process of and impact of implementation, but that's for another day." We also understand that Tyson has addressed one of the government implementation groups with her potentially disastrous idea of scrapping the term PEM in favour of 'symptoms after exertion.'

Given all of the above, and Professor Tyson's unacceptable behaviour towards pwME and others on the forum discussion, as raised in our separate letter, we request that the MEA close down this project. We suggest that the remaining funding be used to fund a new team, we suggest including Physios for ME if available, to work on producing more suitable clinical monitoring tools for use in specialist clinics and by GP's in the NHS.

This letter will be made public on the Science for ME forum, along with any reply we receive from you.

Yours sincerely,

Trish Davis and Maree Candish on behalf of the committee of the Science for ME forum.

 

Letter received from Neil Riley of the MEA today:

"Dear Trish and Maree.

Thank you for your email of the 2nd April 2024.

I note your formal complaint.

I have read carefully what you say. There are some points I agree with and others that I do not. . We come from our own experience of the illness, the information we read and our discussions with others. It is unsurprising that differences have emerged. Many of the comments on Science 4ME make that precise point

The ME Association has confidence in Professor Tyson. She has experience of the illness and knows the background to how patients with ME/CFS have been treated in the past. We are all of us in agreement that this must change. The toolkit project can contribute to that by clarifying patients’ main difficulties and priorities, what they want/need from the service, and what services are (and are not) doing well. The reality is that very little is known about what is currently offered for people with ME, whether by specialist services or elsewhere, how services have responded to the changed clinical guidance, nor how effective or helpful it may be. The information gained from this project and the toolkit will provide the essential bedrock for any effective change to clinical practice and service development.

Professor Tyson and ourselves are making sure that all the comments from your members will be looked at carefully. Many are constructive and will be taken into consideration as this project progresses.

Our trustees are meeting shortly and will be receiving a progress report from Professor Tyson.
.
Neil"