IiME letter to Mark Baker (NICE) re: CBT & GET as recommended treatments

[Alvin]

This is side-stepping the issue a bit, but to be honest the rot started with the introduction of the National Curriculum. Suddenly instead of just having targets (GCSEs, A-level etc.), every step of the way was specified, and it was a knowledge/skill based syllabus. I could ignore it to an extent, being an established teacher, and continue to encourage kids to challenge my statements, but for newer staff, it was all they had known. I gather Scotland and Wales have a less prescriptive curriculum where the aim is to produce useful thinkers.

I'm in north America so i don't understand what your referring to but i do know how our system works and critical thinking and wisdom are not traits that are taught its about rote learning of the syllabus to the exclusion of the bigger picture. Not complete exclusion but more then enough.

 

[Invisible Woman]

But the point is that the kids pretty soon homed in on the faults, and knew it was rubbish. Why is it that so few adults seem capable of understanding that? Why is it that beliefs become so entrenched that they cannot question the evidence?

I reckon there are two reasons:
1. Unless it affects them, or someone close to them, they have no interest. For Douglas Adams fans this would be the equivalent to enclosing it in an SEP field.

2. They see the problem quite clearly. However, it is in their own best interests to ignore the problem in case they upset someone with influence or do themselves out of an easy earner. In other words to pretend that it is enclosed in an SEP field.

For non Douglas Adams fans an SEP is Somebody Else's Problem and once something is placed in such a field it disappears.

 

[Indigophoton]

I think it's also about maintaining one's worldview in the face of evidence that would otherwise challenge it. Presented with a fact that would undermine their whole conceptual model of the world, and perhaps their sense of self along with it, people often resolve the cognitive dissonance by rejecting the fact in one way or another, rather than deal with the loss and upheaval that changing their minds would cause. The more invested in their view a person is, the more they are likely to resist changing their mind.

Sometimes people genuinely can't see or hear the evidence if it is too dissonant with their existing model; they don't perceive it correctly in the first place, but see it as something else from the start, and then maintain that position no matter the strength or clarity of the evidence to the contrary.

A trivial example would be, out for a walk in the woods, you see a slinky, curly thing amongst the ferns up ahead, and perceive it as a snake. Then your companion says, no, look closer, it's just old rope, and you suddenly see it as rope.

The only way I can understand Wessely's 'thing of beauty' comment, for instance, is to assume that he perceived snakes instead of rope, and is so personally invested in the BPS model that he cannot bring himself to acknowledge that it's just old rope after all.

 

[Snow Leopard]

When I was teaching, I would regularly put up some sort of claim as a poster in the classroom, then wait for the squeals of protest.

"9 out of 10 people here say that Maths is the best subject."

Maths is the best subject, I'll start a poll to prove it!!!

 

[Barry]

There are the BBC investigative journalists, but they have said their statistician thinks that PACE is not too bad.

Is it the case that the PACE stats per se were not an issue, but more that the stats were based on deeply flawed trial data? (genuine question, not an oblique statement). Cr*p in cr*p out no matter how rigorous the PACE statistician may have been? In which case could the BBC just be playing simple? Or are statisticians supposed to also fully understand the validity of the data they are given to work on?

I know there have now been lots of PACE critical papers written that include lots of the stats, but if my words above are right, is there room for a paper with a different slant. One focussing on how the stats were OK but the results nonetheless rubbish, because the stats was not the real problem? Something to just drive home that stats being OK does not let PACE, or the BBC, etc off the hook. How any one broken link in the chain can turn a trial into a dogpile? Then DT might like to pick up on such a paper? But of course if my premise is wrong, the rest of this post is not applicable.

 

[Jonathan Edwards]

Is it the case that the PACE stats per se were not an issue, but more that the stats were based on deeply flawed trial data? (genuine question, not an oblique statement). Cr*p in cr*p out no matter how rigorous the PACE statistician may have been? In which case could the BBC just be playing simple? Or are statisticians supposed to also fully understand the validity of the data they are given to work on?

Yes, this is what I said to our friends at the BBC. They should not have checked with a statistician but just with someone who understands clinical scientific method. Statisticians ought to understand bias but the way bias comes about it is not strictly a statistical issue. I don't think our friends were playing simply. I think they had found a courteous way of telling us they did not want to play our ball game.

I think we have already pointed out in print that there are problems much deeper than statistical fiddling. And the statistical filling is quite bad as well.

What I think may be important is that there were three of us approaching the BBC journalists and we were all saying something different. That I can see may be a reason why journalists might back off. I think this may be a lesson for NICE and other battles. If people see inconsistencies they will think they can ignore everything.

 

[Carolyn Wilshire]

Case studies could help a lot. They aren't RCTs by a long shot, but they're also a lot better than nothing. Researchers can also combine dozens of cases into a single study, which might be more persuasive as well. Perhaps collating those incidents and writing up a report to be published is something that @Tom Kindlon or @Carolyn Wilshire could do, if no NHS doctors are interested.

Its tricky if the cases are self-selected. But recently, I've been thinking a lot about narrative and narrative analysis, and the role it can play in identifying themes and concerns in patient's stories. Historically, I've avoided these approaches because they always seemed so vulnerable to investigator bias, but now with the mess in the CFS/FND/MUS literature, I can see the value in letting patients talk without any imposed structure. I wonder if this is an approach that can be applied to these case stories, to identify common themes, etc.

If nothing else, its a way to get those stories into the published literature.

Does anyone here have skills in narrative analysis - discourse analysis, conversation analysis or the like?

 

[Graham]

Personally, I think we have to refine and restrict our approach. I'm fully in favour of published studies/analyses showing up all the faults in PACE, but when we are presenting an argument to NICE, I think we have to keep it simple. There's little point in trying to convince the psychs, so we have to think what strategy to use on impartial assessors.

To my mind there are two stages in the argument.

The first is that using the Oxford criteria gives groups of patients that are unrepresentative of patients with ME/CFS. I think this is a difficult case to make, as reflected in discussions here. It's not that we can't see the problems ourselves, but that we are trying to convince a number of detached scientists. But the fact that the USA has dumped all such trials is important, and in doing so, has removed any claims that CBT is effective.

The second stage is to accept PACE as a significant, definitive trial with data that is informative, but an analysis that is deeply flawed, whatever group of patients they used. So when it shows that CBT produces no improvement in objective assessments, which is consistent with previous smaller studies, that is a key point. The improvements in subjective assessments are relatively small, especially considering that this was an unblinded trial where the comparison group was "neglected" and where CBT was actually designed to change the severity with which patients saw their symptoms. This shows that CBT wasn't even very effective in doing that. There is only one valid interpretation of these results: CBT did not actually improve symptoms, and only produced a small change in rating those symptoms. It is irrelevant whether this applies to patients with ME or not - it doesn't work.

I think we dilute our case when we attack PACE on too many fronts (even though there are so many fronts to choose from).

I've been thinking of an analogy, and have come up with this.

If we take a group of Seventh Day Adventists, who believe completely that God created the world in 6 days, and present them with the layers of rock and fossils in the Grand Canyon, they are forced to interpret the findings through their beliefs, and in doing so, have to ignore many scientific "truths". But of course, if you step away from their beliefs, a scientific analysis of the findings leads you down a very different path.

The psychs who fervently believe that ME/CFS is purely a psychological condition see the PACE trial as firm proof that CBT works. They "know" that all that is needed is for patients to change their perceptions, and they will, over time, be cured. So when the PACE trial shows a change in subjective reporting of symptoms, that is solid evidence that CBT works - as long as you only view the results through that belief. The failure to record objective improvements is ignored, in much the same way that the Seventh Day Adventists ignore inconvenient facts.

Again, NICE needs committee members who can step away from this unfounded belief and review the evidence as it is, not through some distorting lens of prior assumption.

 

[Hoopoe]

I can see the value in letting patients talk without any imposed structure. I wonder if this is an approach that can be applied to these case stories, to identify common themes, etc.

As child, when I first reported that something was wrong to my parents, immediate disbelief was the reaction. Disbelief for no good reason is probably a recurring theme.

 

[Inara]

Again, NICE needs committee members who can step away from this unfounded belief and review the evidence as it is, not through some distorting lens of prior assumption.

But maybe NICE isn't the "neutral observer" (or committee members), but also has its beliefs/agenda.

For instance, using your metaphor, Adventists believe the world was created in 6 days, and another Christian group (e.g. Charismatics) believe it was made by God, but not in 6 days, but in 6000 years (or 6 million years, it doesn't matter). Scientific findings aren't rejected, but interpreted within this belief system.

Maybe NICE and others welcome publications like PACE because they help solve a problem or achieve a goal. It would be useless to try to convince them, too, that this is unscientific.

 

[Graham]

But maybe NICE isn't the "neutral observer" (or committee members), but also has its beliefs/agenda.

Agreed: that's why we need a say on the selection of members.

 

[Barry]

Personally, I think we have to refine and restrict our approach. I'm fully in favour of published studies/analyses showing up all the faults in PACE, but when we are presenting an argument to NICE, I think we have to keep it simple. There's little point in trying to convince the psychs, so we have to think what strategy to use on impartial assessors.
...
I think we dilute our case when we attack PACE on too many fronts (even though there are so many fronts to choose from). [Shifted from further down Graham's post]

Fully agree. Just speaking hypothetically: If one were to rank all the PACE issues in order of "power to convince impartial assessors", and let's suppose there were a list of 10 issues. And again imagine one could "dry run" just the one highest priority, and see how effective it was; then try the first two; then the first three; etc. Some might think that by the time you get to 10 there would be an unassailable case, but I strongly suspect the effectiveness would peak early on, and very soon after be close to zero, long before 10. No matter how good the individual issues, the cumulative negative effect of "quantity overload" would rapidly undermine.

To my mind there are two stages in the argument.

The first is that using the Oxford criteria gives groups of patients that are unrepresentative of patients with ME/CFS. I think this is a difficult case to make, as reflected in discussions here. It's not that we can't see the problems ourselves, but that we are trying to convince a number of detached scientists. But the fact that the USA has dumped all such trials is important, and in doing so, has removed any claims that CBT is effective.

Agreed, but we have to be able to completely convince impartial assessors that we are not simply choosing our own favourable perspective on a circular argument - wanting to remove Oxford primarily to show CBT shown ineffective, whilst saying real PwME do not respond to CBT so Oxford not relevant. We would need a stronger argument than "the USA have dumped it", but more on the arguments underpinning that decision. Just being devil's advocate; I obviously know we do not argue this way, but doubters will need convincing.

The second stage is to accept PACE as a significant, definitive trial with data that is informative, but an analysis that is deeply flawed, whatever group of patients they used. So when it shows that CBT produces no improvement in objective assessments, which is consistent with previous smaller studies, that is a key point. The improvements in subjective assessments are relatively small, especially considering that this was an unblinded trial where the comparison group was "neglected" and where CBT was actually designed to change the severity with which patients saw their symptoms. This shows that CBT wasn't even very effective in doing that. There is only one valid interpretation of these results: CBT did not actually improve symptoms, and only produced a small change in rating those symptoms. It is irrelevant whether this applies to patients with ME or not - it doesn't work.

I think a key point here is that no matter what kind of therapy was being trialled, and no matter what presumptions there may have been about the underlying perpetuating reasons for ME/CFS, real physical function is the only main/sane primary measure of recovery, or even improvement. And because we are talking about real physical function, it is nonsensical to measure it any way other than objectively. If we were trying to measure only what people felt about their physical function, then asking them questions about it would be fine; but recognising that would then be affected by how they felt at the time they answered the questions, and what factors were influencing how they felt.

 

[large donner]

Maybe a technique would be to run a few workshops on random people to see which issues stick, like when comedians try out their jokes on small audiences before taking them on a big tour. Or when marketing companies test strategies out in workshop groups.

Not sure how we could arrange that but just an idea!

 

[Snow Leopard]

I think we dilute our case when we attack PACE on too many fronts (even though there are so many fronts to choose from).

Yes. Sadly some, like David Tuller, Malcolm Hooper don't seem to understand this. You don't throw everything you have in hope that something sticks, instead you choose the most fundamental issue and show that everything else falls like a house of cards.

 

[Barry]

Yes. Sadly some, like David Tuller, Malcolm Hooper don't seem to understand this. You don't throw everything you have in hope that something sticks, instead you choose the most fundamental issue and show that everything else falls like a house of cards.

It does also depend on who the target audience is, and what the objective is, which can vary according to the audience. For NICE, more than two or three issues would be pointless. But when trying to make clear all that is wrong with PACE, that is different - but not for NICE.

 

[Snow Leopard]

Maybe NICE and others welcome publications like PACE because they help solve a problem or achieve a goal. It would be useless to try to convince them, too, that this is unscientific.

We know this is true. This is why the PACE trial was funded by the DWP - to achieve a goal. Of course it backfired as the study showed no long term effect, no meaningful effect on objective outcomes and dammingly, no effect on work/study/welfare outcomes.

 

[Snow Leopard]

For NICE, more than two or three issues would be pointless. But when trying to make clear all that is wrong with PACE, that is different - but not for NICE.

Is it though? I mean almost no one actually reads what Malcom Hooper writes as it is way too long and nitpicky.

 

[Barry]

Is it though? I mean almost no one actually reads what Malcom Hooper writes as it is way too long and nitpicky.

I guess I was thinking more of DT when you replied.

 

[Barry]

We know this is true. This is why the PACE trial was funded by the DWP - to achieve a goal. Of course it backfired as the study showed no long term effect, no meaningful effect on objective outcomes and dammingly, no effect on work/study/welfare outcomes.

As the saying goes - there are none so blind as those who do not want to see.

 

[Graham]

I think what I am trying to say is that if I sat in front of the committee, I would start by saying that the Oxford criteria is very different from the criteria on the NICE guidelines: in fact it has very little in common with any of the sets of criteria for ME/CFS which mostly demand PEM. For that reason, the USA have decided that any studies using that criteria produce results that are inappropriate to patients with ME, and no longer take heed of them. That covers pretty much all the research carried out by British psychs, and all of the research supporting CBT.

But if you don't agree with that conclusion, and feel that data produced from patients selected by the Oxford criteria can be valid, then you have to look thoroughly at the data provided by the PACE control, and not take their conclusions at face value. If you deem the Oxford criteria to be acceptable, then the objective data from this and similar trials consistently and unequivocally show that CBT is ineffective. The subjective data is utterly irrelevant: the trial was non-blinded and the therapies were designed to change the way patients rated their symptoms.

There is no let-out clause here. Either the use of the Oxford criteria is misleading, or the objective data clearly show that CBT is ineffective.

I'd probably add a warning that the measures of harm were greatly restricted, and, given the substantial feedback from patients about their experiences with CBT and, in particular, GET, it would be dangerous to continue to recommend such therapies against such reports while they clearly fail to deliver any real, physical improvement.

I don't see that any scientist could argue with that (well, assuming it had been tidied up and presented properly).