ME/CFS NICE Guideline - Safety testing of GET in PACE trial

The issue of GET and safety has of course been much discussed, especially regarding the the PACE trial authors' claims to have demonstrated its safety, versus the fact there exists good quality anecdotal evidence to the contrary.

There is however an aspect I'm not sure has been broached, and which I would like to be sure is considered. If it has already been addressed, then better it's done twice rather than not at all. The safety or otherwise of GET is of course a crucial consideration for the new NICE guideline, and is particularly why I'm posting this.

(If you prefer not to wade through all the words here, jump to the last two paragraphs! ).

Safety margins. Imagine a black box system, and varying one of its inputs through its range, looking at the effect on the system's output(s). Imagine the system is in a safe state, but that the input is capable of values that can put it into an unsafe state. Now vary the input progressively from its safe value towards an unsafe value; there will be some value at which the the system is deemed to be definitely in an unsafe state. Let's suppose for example that the input started at a value of 5, and the system became unsafe when it reached a value of 10. (All the numbers here are by way of the same arbitrary example).

This does not mean that the input can be reliably taken to 9.99 and presumed all is still safe, even for the single system it is being tested on. There will likely be a value significantly before 10, that will be deemed the maximum value at which the system might still be considered safe; let's say 9 for example, implying a safety margin of 1.0. Each type of system will be different, and each instance of a system within its type will vary.

In reality we will not be talking about a single input, but multiple inputs, with a corresponding geometric increase in complexity, especially if the system is nonlinear. Instead of a single value being changed, we will have the notion of a performance envelope, and the limits within that envelope before things become unsafe, and the safety margins needed within that envelope to remain safe.

The determination of such safety margins will be a combination of theoretical analysis and safety trials on real systems; such systems are invariably too complex to rely solely on theoretical analysis alone, which at best will always be an approximation.

At some point the safety trials have to be designed. I'm not claiming to be any sort of expert on this, but I think it highly likely that such safety trial design will be heavily reliant on the theoretical understanding that does exist, and just as importantly ... recognition of what sort of things are not fully understood, but nonetheless need to be tested.

The most dangerous mistake to make would be to think you understood the system behaviours well, when in fact you did not, and so designed your safety trials on that flawed understanding. You may then design you safety trials woefully inadequately, whilst fondly believing you have covered all the bases. Your system under test might successfully pass your badly designed safety trials, yet that same system might become very unsafe in the real world. Even if you run plenty of instances of your system type through these safety trials, the flawed design of the safety trial means they might all pass with flying colours, even though unsafe in the real world.

So if the system type in question is the human body, one of the most complex and nonlinear systems there is, your safety trial needs to be designed with high awareness of what is understood, and especially what is not fully understood but needs testing. If you have a hypothesis of what condition the intervention is operating on, then that can guide what testing might be needed in order to assert the intervention is safe. But if that hypothesis is wrong, woefully hopelessly wrong, then the resulting testing cannot possibly be used to assert it proves the intervention to be safe, because the safety testing will be as flawed as the hypothesis the whole thing is based on.

The PACE trial authors assert the safety of its interventions based on safety testing that is itself flawed, given it is based on their flawed hypothesis of what perpetuates ME/CFS - deconditioning. Once the real physiology of ME is understood, it will almost certainly show that the PACE trial missed testing all sorts of safety aspects of GET for PwME. The anecdotal evidence of harms tells us this. You simply cannot reliably design safety testing based on a seriously flawed hypothesis.

I'm sure this can be explained much more succinctly. I obviously don't have the medical expertise, and there will also be much better qualified engineers than myself who could do a better job. But I do think we need to get across to NICE why the PACE trial simply cannot be taken as any kind of reliable evidence for the safety of GET. And to me the flawed hypothesis is at the heart of that - how can safety be reliably assessed on that basis!

ETA: Minor edit for clarity.

 

Sorry @Barry , I had to skip straight to your final 2 paragraphs, as suggested.

Relapse (‘recovered from’ PVFS 2001) caused by Sepsis, itself caused by repeated uti’s, the last one which was resistant to the antibiotics. I was not deconditioned at the time I was hospitalised.
Part-time job (28 hours pw, with a minimum 30 min commute each way), busy mum, on an active Committee, member of Local History Group, Pilates, Balboa dancing (swing) with husband, 2 teenagers and a dog.

Within 4 months of Sepsis, I was referred to a ‘CFS/ME’ clinic (England). I had been walking the dog at least 5 times a week within a couple of weeks of being out of hospital (not far at all, but out every week day). I had not had time to get ‘deconditioned’.

I dealt with the Graded Exercise part of the therapy by measuring my week day walks with the dog. When one route seemed easy, I tried to increase the walk a little. However, this never worked. I only had so much energy available. If one day I had to prepare the dinner or do a school run or hung the washing out to dry, I couldn’t do the ‘extra’ amount of walking without severe payback. I just stayed within my energy levels to cope, as much as possible, with daily family life. I rested between activities, but found that my brainwork was the most severely affected.

I still don’t understand how the clinics insist that we learn how “not to boom and bust” and yet then want to encourage you to do make those incremental changes which, in themselves, are more than physically possible.

 

I haven't seen much discussion on this point yet but beyond safety, after 11 years of the current NICE guidelines, where is the data on its efficacy?

Safe or not (although it isn't), a full decade later the only cited evidence is, well, decade-old research. By this point the NHS, or whomever does that kind of monitoring, has, well, a full decade of data on what happens to patients with ME sent to these "treatments" and whether it actually improved their condition.

IIRC a question was asked of the NHS, which responded that they don't follow up on this and leave it to the individual "fatigue clinics", which don't do any follow-up either.

At this point Sharpe is kind of correct to point out that PACE is old by now, it's been a whole decade of guidelines built on the same assumptions. So where is the data? Where are the objective outcomes? It's on them to prove that it is efficacious, not on us to disprove it.

Especially considering the 2017 continuation, which surely must have been backed by data showing the guidelines are correct and useful. There has to be data, evidence, follow-up, studies. Not on a small number of participants but on the whole population. They claim that it has worked in practice without showing any evidence. This is an issue that needs to be raised IMO.

Since whether it's safe or not is based on assumptions about whether harm is even real, which psychosocial ideologues simply reject. But the point of whether it is efficacious is very much quantifiable and should not be restricted to decade-old research now that it has had plenty of time to be deployed and refined.

Where is the data, Lebowski?

 

It is very striking how the denial of possible harms underpins the BPS ideology and how neatly that dovetails with the DWP policies of pressure and sanctions.

Even the current phrasing that I've seen used such as "makes symptoms worse", or in our own words causes a "crash" is quite understated. It sounds transient and that isn't always the case.

Even if the initial crash is transient, isn't it possible that frequent challenges that trigger a crash may ultimately cause very long term, if not permanent, deterioration?

In my eyes, the BPS brigade were allowed to get away with the assumption that they would not cause harm. There is no evidence that assumption is correct and plenty of anecdotal reports to say it isn't.

They failed to ensure their trials were safe and thus broke the "First, do no harm rule." NHS England and the powers that be have colluded as they have never bothered to follow up with their patients and are happy when they just disappear.

This should bother NICE not just because of safety, but also from an efficacy and value for money point of view.

 

PACE is typically cited as evidence GET does not lead to harms. But PACE presumed the hypothesis of deconditioning, which will likely have influenced what safety testing was done, and much more to the point, what was not done. What data was gathered, and not gathered; analysed appropriately, and not; what outcomes were heeded, and what outcomes dismissed as irrelevant. Much of this is driven by what is designed into safety testing, and what is not ... and that in turn may be driven by what is believed to be understood, and what is considered unknown but nonetheless needs safety test coverage. The more it is believed is understood, and the less deemed unknown, the less coverage will be given to safety testing outside what is believed to be understood.

The less valid the safety testing is, the less valid it is to extrapolate its applicability to people outside of the trial's environment. I would have thought it feasible to make a strong case that PACE, and similar trials, based their "safety testing" on false assumptions, and so will have risked omitting much of the necessary test coverage. Given such a high risk of seriously inadequate safety testing, then no way should it be cited as remotely reliable evidence of non-harms. The anecdotal evidence of harms may in fact be much more reliable.

If GET were applied to people genuinely suffering from deconditioning, and genuinely feeling they had something more serious, then it could be a comfortably wide safety margin, between GET being safe for them and it becoming unsafe. For PwME, that safety margin could be much much narrower, with negligible margin between staying safe and tipping over the edge into unsafe - the point is that nobody knows, and nobody should be presuming to yet know. If the safety margin is believed to be much wider than it really is, practitioners will push people into the danger zone due to their ignorance, whilst sincerely believing they are helping. Bearing in mind that safety margin may have time delay as one of the contributory factors - seems OK today, and then with no more GET the PwME goes down hill a day or so later. I'd be amazed if PACE recorded all the necessary metrics, let alone analyse them; possibly even wilfully ignored some if they believed it just didn't match what they expected.

Safety has been touched on a lot of times in much of the advocacy, but I don't think has been singled out as worthy of its own focus. I'm pretty sure there is something worth digging into in what I've said here, but I don't have necessary wherewithal to do it justice. I would have thought a combination of someone with clinical trialling expertise (@Jonathan Edwards ), together with someone having considerable expertise in safety trials on safety critical engineering projects (not sure who best here), could collaborate to put a very convincing case to NICE. What do you think @dave30th?

 

I think that the criteria for a serious adverse event in the trial means that even giving doses of poison would have passed as safe if the dosage was carefully calculated. It would have been possible to kill everyone longer term, while claiming it was safe.

 

Yes, sorry @Tom Kindlon, I'd forgotten that you did a very good piece on reporting of harms. I do think still that with PACE etc (maybe common to other medical trials), explicit testing of safety is approached rather differently to the way it is in engineering. It feels to me as if nothing like the same degree of rigour (up front design, assessment of test coverage, implementation, etc) goes into safety testing of treatment, as does testing of efficacy. Which is an additional reason why statements of treatment safety based on PACE etc just should not hold up.

 

Safety testing in engineering would require precise, replicable objective measurements. Safety procedures e.g. in an operating theatre require a minimum of two independent observers verifying the facts e.g. number of swabs used during surgery.

 

I asked a similar question a while ago re the CQC (Care Quality Commission); I've been searching on their website but cannot find any assessments of these clinics.
https://www.s4me.info/threads/care-...ng-re-cbt-get-treatment-for-me-patients.2722/

eta: see also
https://www.s4me.info/threads/menta...east-london-foundation-trust.2784/#post-49931

https://www.s4me.info/threads/me-cfs-services-in-the-uk.5625/

https://www.s4me.info/threads/lack-...o-cut-services-for-young-people-with-me.5411/

 

Exactly! My wife was always very active, never gave herself the chance to decondition, and is still pretty active albeit within tighter limits. Which is why I know for absolute sure why the deconditioning theory is such a load of 'bullocks'.

 

Second thing I did after setting an appointment with my GP is join a gym.

Did not help, unsurprisingly. Made me much worse, in fact.

Then after my GP blew me off, refusing a 2nd appointment, I saw a psychologist. Massive waste of time once again.

I still tried to exercise within my limits. Only rest helped me long term. I relapsed multiple times over the years, always because of going over my energy limits. Everything about the psychosocial ideology is BS. All of it, it's lukewarm garbage.

 

Another apology needed! Despite not being the person with ME, I've always had a rubbish memory! I'd also forgotten Keith Geraghty did a great paper on harms from medical interventions .

https://www.s4me.info/threads/me-cf...e-medical-encounter-geraghty-et-al-2018.4707/

 

I think there may be another angle worth considering, though it may already have been.

The PACE authors typically counter claims of harms from GET, by saying that the delivery of GET was likely wrong, and that if done 'properly' it is OK. (I'll assume here that by 'properly' they mean as was done in PACE, implying as per the PACE GET therapists' manual). I know we are all convinced that GET is potentially harmful for at the very least a subset of pwME, but NICE as yet is not. So I'm addressing this here from NICE's standpoint, that for them the jury is still out regarding harms from GET.

I'm wondering if it is even necessary to convince them of the potential for harms from GET, though obviously if we can then that is all for the better. What if we can convince them the methods of delivery are so woefully inadequate, then even the best treatment would still be highly risky? The PACE authors have effectively conceded this possibility I think.

The PACE authors have themselves heavily implied that GET delivered in clinical settings may well be poorly controlled/managed, maybe even inadequately specified. Otherwise their statements of improperly administered GET would not hold water, even within their own line of reasoning. Moreover they have tacitly acknowledged that GET can be harmful if not delivered sufficiently in accordance with their own version of it - i.e. PACE.

So if the infrastructure does not exist to deliver a treatment reliably and consistently in normal clinical practice, then how can you possibly recommend it in a NICE guideline!

Given the PACE authors have effectively stated that GET delivery is broken in normal clinical usage, you then have to ask why. (Remember I'm looking at this from NICE's still-to-be-convinced perspective). What I'm wondering, is if the system is broken is there any sane way NICE could see it being mended in time for the new guideline? What if the problem is that the guidance to therapists, in a normal clinical environment, is just wrong or inadequate or never properly converted from that used for the clinical trial? If it could be demonstrated that the system is broken, and even if GET were wonderful, that the system cannot possibly be mended before the next guideline issue ... could that be game over for get so far as NICE are concerned. Do we even need to convince them of harms, if we can convince them the methods of delivery are woefully inadequate?

Sorry a bit of a ramble, but pushed for time.

And just in case anyone misunderstands me here, I am not in the slightest suggesting I think GET is OK!

 

As I've said before, in engineering where human safety is involved, you do not have to prove unequivocally something is unsafe before activating precautionary safety measures. And having done so, you will then have to provide evidence to a high level of confidence that safety has been (re)established, before then lifting those precautionary measures. The aviation industry is a prime example, as I've also said before. This is currently evident in the grounding of all Boeing 737 MAX 8 aircraft, following the tragic crashes ...

https://interestingengineering.com/boeing-737-max-8-software-fix-delayed-after-review

Strong circumstantial evidence sufficed for this precautionary grounding to be applied.

So why should medical safety issues be any different? Why should NICE require rock-solid evidence of harms from GET/CBT before 'grounding' them? The anecdotal evidence of harms to pwME from GET/CBT is probably every bit as convincing as the provisional evidence that convinced aviation authorities to ground the 737 MAXs.

I find it interesting that even if NICE merely 'grounded' GET/CBT pending further safety evaluations, the likelihood of such a restriction ever being lifted is pretty much zero, given high quality evidence of its safety does not, and almost certainly never will, exist.

I do truly believe this is a valid line of argument to put to NICE. Aviation safety and medical safety should not be poles apart.

 

But what would happen to the blue cardigans?

This therapy is now a huge animal that requires to be fed. Cue the multifarious conditions being reassigned cbt ( and exercise) .

 

The cardigans would also have to be grounded .

Boeing is also one of the biggest animals expecting to be fed, but they are still having to take this on the chin. And rightly so. Safety versus eminence.

 

https://www.bbc.co.uk/iplayer/episode/m00077cw/panorama-boeings-killer-planes. The Boeing 737 Max.

Just following up further on this, because I really do believe there are serious parallels here that might help NICE gain insights they should already have, but very likely don't have, and they may be less likely to gain purely from their normal comfort zone. I'm talking about GET of course, and that the burden of proof needed to "ground" it should be lower than the burden of proof needed to authorise it. And once then grounded, would then have to properly attain the higher burden of safety proof needed for re-authorisation.

I would ask anyone involved in the NICE guideline amendments to watch this sub-30 minute documentary on the Boeing 737 Max, because there are so many strong parallels that just jump out at you:-

1. Powerful conflicts of interests.
2. Human safety.
3. Egregiously flawed design.
4. Appalling lack of regulatory oversight.
5. Misrepresenting the nature of the design work so as to subvert the required degree of regulatory oversight.
6. Design pressures driven by cost savings.
7. Eminence and its influence.
8. Stating an outrageous primary-effect design flaw is not overly significant, because of other factors.
9. Denial by Boeing to this day that they are to blame for the harms caused.

The big difference to date, is that these aircraft have all been grounded well before full and final evidence has been obtained of the exact cause of the crashes. There are also a lot of legal actions being instigated against Boeing.

To me I see this akin to an on/off switch, with an asymmetric characteristic: considerable force needed to push it one way; much less force to trip the other way. This regulatory switch can be set to Unsafe significantly more easily than it can be set to Safe. Once in the Unsafe position, all the newly noted safety concerns require the much stronger burden of proof needed to set the switch back to Safe again.

If it helps, a brief insight into the fundamental 737 Max issues:-

• The Boeing 737 Max is based on long established design, but with bigger engines.
• The new flight characteristics needed a new software control system, Maneuvering Characteristics Augmentation System (MCAS).
• MCAS has a crucial feature: if it senses the aircraft is at risk of stalling, if forces the aircraft's nose down - the classic way of regaining airspeed and reducing angle of attack, and thereby avoid stalling.
• The fatal flaw: there was just one sensor to detect this condition. This is unheard of in modern aviation safety control systems, there is always dual redundancy so that a faulty sensor will almost certainly be detected.
• In the disaster scenario, the single sensor mistakenly indicates the aircraft is nose up when it is not, so MCAS insists on putting the aircraft into a dive, with trained crews typically at a loss what to do.

@MEMarge, @Keela Too, @Jonathan Edwards, @dave30th. Also @RDP as I believe you have a systems engineering background, and also maybe a safety engineering perspective as well.

 

Excellent thinking.