The draft scope for the NICE guideline on ME/CFS is now out for consultation, June 2018

[Barry]

Whilst it is a strong argument that there was response bias which I have actually been writing about for years, it is insufficient. Considerable actual improvements occurred which are not explained by that ...

Good to see you hear Peter, and you have certainly sparked off an interesting debate! I also am interested how "considerable actual improvements" can be claimed from the subjective outcomes measured. The objective measures showed negligible benefit, and there was no control for participants exchanging some of their own non-trial activity in order for them to cope with trial activities; overall the objective measures may have been significantly worse in reality.

 

[Peter Kemp]

I thought most people on the butt end of your comments about 'deliberate strategies' etc. annoying to be honest, Peter.
I have given the arguments for not annoying people.
I am not sure what political sophistry has to do with it.

Dear Jonathan, this is just my take: 'political sophistry' is when I am insincere, self-censor and am selective or dishonest in order to sway or manipulate a group or situation for my own ends. Like I said, IMO there is nothing inherently wrong with it. I just don't care for it and try not to do it myself.

Do you know or have any ideas why NICE changed the topic of the new NICE guideline from 'CFS/ME' to 'ME/CFS? Could it be so that the PACE Trial and similar clinical trials would be excluded from their evidence search as inapplicable? White did remark that the trial was not for M.E. but for (from memory) 'people whose main symptom is fatigue', do you think NICE are aware of this? Other than NICE not knowing the difference and simply trying to please the patient community, I cannot think of any other reason for the change other than its being a stratagem.

 

[Peter Kemp]

Good to see you hear Peter, and you have certainly sparked off an interesting debate! I also am interested how "considerable actual improvements" can be claimed from the subjective outcomes measured. The objective measures showed negligible benefit, and there was no control for participants exchanging some of their own non-trial activity in order for them to cope with trial activities; overall the objective measures may have been significantly worse in reality.

Thank you Barry! When the PACE trial arms are assessed as groups, a tiny difference is found for GET and CBT. In groups, this difference is insignificant and far below the 'minimum detectable change' thresholds. An effect of moving the Primary Outcome Measure thresholds, was to isolate and magnify these differences, which made GET and CBT appear superior to APT and SMC.

However, there is another way of looking at the data which is what I have been working on. Around two-thirds of participants showed significant improvement, and nearly one half met established minimum detectable change thresholds with the primary outcome measures.

This change can be spun in different ways and as remarked earlier, one way that White et al exploited the change, was to blur the validity of the 'control group'. SMC applied properly as a control group completely (IMO) demolishes the 'treatments'. So they didn't! This move has the added implication that half of participants improved just by getting individual attention - Arrrrrgh! Therefore, whilst therapy bias and probably placebo had their own effects, something else was also influencing the outcomes of nearly half of participants. IMO that element was selection bias and this is something that we might need to use at some point.

Incidentally, just after the PACE Trial was published, I wrote an article on analysing 'CFS' research findings which included this snippet which anyone is welcome to use if the occasion arises:

In “Clients’ Deference in Psychotherapy”, Rennie (1994) identified 348 instances of client deference to the therapist occurring in just 16 therapy sessions. These most commonly related to the client’s, “Concern about the therapist’s approach”, followed closely by “Fear of criticising the therapist”. Rennie also noted that therapy provided free of charge (as in research) was a factor likely to engender deference. In the context of CFS psychosocial research, this suggests that participants (who may have been subjected to pressure to doubt their own experience and rationality) may rate subjective measures deferentially; i.e., in a way that they believe the therapist would approve of.

 

[Pen2]

@Trish thank you for the articles. The first one I have but I'll include the second one too. I have learned alot recently So I am hoping my PCP will want to update his knowledge so we can discuss what changes can be made etc.

I feel like I'm on too many meds with the only change being a very high blood pressure that still drops just as much as before.

Anyways, just wanted to thank you @Trish.

 

[Graham]

Because I am doing my own analysis of the PACE Trial, I have not read the doubtless, marvellous work of Matthees and Wilshire, but I have seen some summaries and charts. My sense is that whilst they raise serious issues about the conduct and analysis of the research, they have not demolished all the findings.

This bit bothers me, @Peter Kemp . Why would anyone choose not to look at other analyses of the PACE trial to see what actually has been covered? And why would someone assume that such studies are limited when they have not read the various papers (including those in the JoHP)? As someone who tries to take a scientific approach to these things, I believe that the first step is to find out what the situation is. Much of the problem with PACE and the similar studies was that medical professionals only looked at the summaries and a few charts: it's the detail that matters.

I also believe that it is important to focus on facts, on evidence, and to pin people down with those, rather than issue general disparaging comments.

Don't get me wrong. I know we are on the same side, and I am not wanting to give you the impression that I am dismissing your valuable efforts. But the only way we are going to make progress is to focus on analysis and fact, and not give our detractors every excuse to deride us. They love every chance they get to avoid answering the hard questions and sidestep into other issues. Every opportunity we give them to paint us as angry, negative irrationals allows them to get out of difficult situations.

Don't think I don't understand the frustrations and the deep anger though. It's bad enough having had ME for nearly 20 years, but my son has had it for nearly 30, from when he was 8 – that really drives deep.

My first priority is to remove from ME any trace of suggestion that the condition becomes permanent through psychological processes. Nailing that will, eventually, change the attitude to biomedical research, will change the public perception of the condition, and will, very slowly, percolate through the medical profession. Totally getting rid of CBT and GET in the NICE guidelines will be a massive step towards that goal.

Much of the present guidelines are, in reality, not too bad: they do emphasise the rights of the patient and the lack of knowledge. It's the fact that there are 120 lines pushing CBT and GET that totally distort the picture. There are no other sets of guidelines that begin to compare with this. The piece on CBT and GET actually start with the proviso that they have not been tested on the severely affected, and so are not advised for them, but then further down suggest the sort of GET that a severe patient can undertake. It is low-grade rubbish.

If interminable in-fighting between different factions of ME groups over definitions and other treatments allows this to slip through our fingers, we will only have ourselves to blame. We have to keep the message clear and keep the message simple. Evidence based.

 

[Sean]

Agree completely with Prof Edwards. The core aim for now is the removal of CBT and especially GET from treatment recommendations.

There are no other treatments with a sufficient evidence base for NICE to consider at this time.

 

[Jonathan Edwards]

Do you know or have any ideas why NICE changed the topic of the new NICE guideline from 'CFS/ME' to 'ME/CFS? Could it be so that the PACE Trial and similar clinical trials would be excluded from their evidence search as inapplicable? White did remark that the trial was not for M.E. but for (from memory) 'people whose main symptom is fatigue', do you think NICE are aware of this? Other than NICE not knowing the difference and simply trying to please the patient community, I cannot think of any other reason for the change other than its being a stratagem.

I presume it was done to be in tune with patients.

Peter Barry will be aware of the arguments about whether trials covered appropriate patient cohorts, as were people like ?Mark Baker who Charles Shepherd and others talked to. From what I have heard the re-assessment came about because NICE people did actually listen in detail to what ME advocates had to say.

My view as a scientist is that the wide selection in PACE was perfectly legitimate in itself. There are complicated reasons why it is likely to have defeated its own objective but all these trials should be assessed. That should not bother anyone because none of them even begin to be usable evidence. I think Cochrane has realised that now but it is too late to re-do the Cochrane reviews. Whether Peter Barry really understands the problem I don't know but he is no fool and I think was a good choice from the point of view of getting things right.

I would recommend you read all the published critiques. This is a very complex problem that has been taken apart on PR and here for years now. There are broad brush arguments and fine detail arguments.

 

[Jonathan Edwards]

Therefore, whilst therapy bias and probably placebo had their own effects, something else was also influencing the outcomes of nearly half of participants. IMO that element was selection bias and this is something that we might need to use at some point.

I don't understand this, Peter.

 

[Binkie4]

My reading on the forum has had to be curtailed recently but I have tried to read on this thread as often as I could.

I may still be missing things but has S4ME submitted a response to Nice as a response from the forum? Several individual people have written their own opinions but have I missed an overall response from here?

My memory is appalling lately so apologies if I have missed an important document.

 

[Andy]

My reading on the forum has had to be curtailed recently but I have tried to read on this thread as often as I could.

I may still be missing things but has S4ME submitted a response to Nice as a response from the forum? Several individual people have written their own opinions but have I missed an overall response from here?

My memory is appalling lately so apologies if I have missed an important document.

Our submission is being finalised at the moment.

 

[Binkie4]

Our submission is being finalised at the moment.

Thanks @Andy. Will this be available for members to read before submission to Nice?

 

[Peter Kemp]

Dear Graham and Barry and Rick and Jonathan, will try to answer all at once as rather tired.

The reason that I have kept away from other people's analyses of the PACE trial recently, is because I have done my own independent review and if I can get it published, do not want to be accused of plagiarism should my findings overlap. I have spent hundreds of hours analysing the data in every way imaginable.

I was writing analyses of the PACE trial within a week of its being published and I posted dozens of critical comments, charts and statistics on CoCure. I even broke down the data that was included in the Lancet report, and used it to prove that the claims of efficacy were false and inflated.

Actually, nobody seemed interested or to understand that the falsification of the reporting was proved, right there in the Lancet report. I wrote about the fact that Sharpe and Horton appeared on ABC radio Australia defending the PACE Trial, and that Sharpe said, (from memory) 'we have a number needed to treat (NNT), I think it's about 7'. Like he wasn't quite sure about such a significant statistic... The claims of Bleigenberg and Knoop (in the Lancet editorial) and Chalder (in national newspapers) that 30% 'recovered' or "got back to normal" produces an NNT of 4 (which would be very good treatment for a chronic condition). An NNT of '7' correlates with what I had already published - that only 15% showed any significant improvement over and above the control group. That critical statistic was not published in the Lancet, but the necessary figures were there for anyone prepared to do some calculations.

[NNT, number needed to treat: the number of patients that have to be treated in order to get a positive treatment effect in one of those patients. Calculated as 100 divided by the percentage of patients improved, rounded-up. E.g., 100/15(%)=6.7 NNT=7]

An NNT of 7, if valid, is a good result for treatment of a chronic condition in which if patients improve at all, it generally happens very, very slowly. Much higher NNTs can still be considered useful in a chronic, deteriorating or fatal conditions, especially if there is a good economic argument for the treatment.

I have seen that there is lots of attention on changing the Primary Outcome measures, and I wrote about that as well. Tampering with the Primary Outcomes is very dodgy practice and SHOULD have resulted in withdrawal of the PACE trial. But it has not yet done so and IMO it is far from certain that it will. This tampering and the numerous other key arguments that myself and others have made, do not demolish the trial data for GET or CBT. Anyone who has studied the data closely would know that.

What really undermines the PACE Trial IMO, but at the same time represents a problem for patients, is that there was far too much improvement in all the groups. This should not have occurred in a well defined M.E. or CFS participant population. The actual improvement which did occur (like it or not) can be misrepresented as a treatment effect. White et al implied that it was a result of getting Specialist Medical Care, which all participants received. Hence their finding that "CBT when added to SMC" and "GET when added to SMC" helps 'CFS/ME' patients etc., etc. They made SMC an integral part of the success of the treatments and reported that 60% "improved" and 30% "got back to normal" (Normal Range).

Therefore it is easy to undermine the claims made for GET and CBT in the PACE Trial, but it is not so easy to achieve two important goals. 1, to use the evidence to expunge GET and CBT from recommendations for these 'treatments'. 2, to use the evidence to show that what the PACE trial actually showed, is that GET and CBT do not treat M.E. or CFS, and therefore the theories on which these treatments are based are proved wrong. It can be achieved by showing that the broad improvements were spontaneous and unrelated to any aspect of participation but are as a result of selection bias: i.e., either deliberately or inadvertently choosing participants EXPECTED to benefit from the experimental treatments - like choosing 'fatigue' patients for GET. The whole purpose of randomisation is to avoid this confounding pit-fall but it did not completely work in the PACE Trial because the cohort from which randomisation was done, was already contaminated and resulted in a small treatment-effect.

Best Wishes,
Peter

This bit bothers me, @Peter Kemp . Why would anyone choose not to look at other analyses of the PACE trial to see what actually has been covered? And why would someone assume that such studies are limited when they have not read the various papers (including those in the JoHP)? As someone who tries to take a scientific approach to these things, I believe that the first step is to find out what the situation is. Much of the problem with PACE and the similar studies was that medical professionals only looked at the summaries and a few charts: it's the detail that matters.

I also believe that it is important to focus on facts, on evidence, and to pin people down with those, rather than issue general disparaging comments.

Don't get me wrong. I know we are on the same side, and I am not wanting to give you the impression that I am dismissing your valuable efforts. But the only way we are going to make progress is to focus on analysis and fact, and not give our detractors every excuse to deride us. They love every chance they get to avoid answering the hard questions and sidestep into other issues. Every opportunity we give them to paint us as angry, negative irrationals allows them to get out of difficult situations.

Don't think I don't understand the frustrations and the deep anger though. It's bad enough having had ME for nearly 20 years, but my son has had it for nearly 30, from when he was 8 – that really drives deep.

My first priority is to remove from ME any trace of suggestion that the condition becomes permanent through psychological processes. Nailing that will, eventually, change the attitude to biomedical research, will change the public perception of the condition, and will, very slowly, percolate through the medical profession. Totally getting rid of CBT and GET in the NICE guidelines will be a massive step towards that goal.

Much of the present guidelines are, in reality, not too bad: they do emphasise the rights of the patient and the lack of knowledge. It's the fact that there are 120 lines pushing CBT and GET that totally distort the picture. There are no other sets of guidelines that begin to compare with this. The piece on CBT and GET actually start with the proviso that they have not been tested on the severely affected, and so are not advised for them, but then further down suggest the sort of GET that a severe patient can undertake. It is low-grade rubbish.

If interminable in-fighting between different factions of ME groups over definitions and other treatments allows this to slip through our fingers, we will only have ourselves to blame. We have to keep the message clear and keep the message simple. Evidence based.

 

[Sasha]

The reason that I have kept away from other people's analyses of the PACE trial recently, is because I have done my own independent review and if I can get it published, do not want to be accused of plagiarism should my findings overlap.

Peter, if the idea is to appear to be independent and your analyses corroborate those of others, you have no way of proving that you didn't look at others' research. Sceptics will not be convinced by your simply saying that you didn't look. (I'm not questioning your word but am pointing out that you would be presenting an easy target to people who don't like your findings.)

If your review merely repeats what others have already found, it won't be published because it will add nothing to the literature.

Appearing to be ignorant of a literature to which you're trying to add will not impress journal reviewers and is likely to get your paper rejected. People submitting research papers are expected to know thoroughly the findings in the field that they're writing about.

If in your review you need to refer to a finding that someone else has made, you simply do so and give the reference to their paper.

Keeping oneself in deliberate ignorance of others' findings and arguments is no way to make progress. In academia, the norm is to study what is known, find the gaps in knowledge, and attempt to fill them. This isn't what you're doing. I hope you will think again about this.

 

[adambeyoncelowe]

Dear Graham and Barry and Rick and Jonathan, will try to answer all at once as rather tired.

The reason that I have kept away from other people's analyses of the PACE trial recently, is because I have done my own independent review and if I can get it published, do not want to be accused of plagiarism should my findings overlap. I have spent hundreds of hours analysing the data in every way imaginable.

I was writing analyses of the PACE trial within a week of its being published and I posted dozens of critical comments, charts and statistics on CoCure. I even broke down the data that was included in the Lancet report, and used it to prove that the claims of efficacy were false and inflated.

Actually, nobody seemed interested or to understand that the falsification of the reporting was proved, right there in the Lancet report. I wrote about the fact that Sharpe and Horton appeared on ABC radio Australia defending the PACE Trial, and that Sharpe said, (from memory) 'we have a number needed to treat (NNT), I think it's about 7'. Like he wasn't quite sure about such a significant statistic... The claims of Bleigenberg and Knoop (in the Lancet editorial) and Chalder (in national newspapers) that 30% 'recovered' or "got back to normal" produces an NNT of 4 (which would be very good treatment for a chronic condition). An NNT of '7' correlates with what I had already published - that only 15% showed any significant improvement over and above the control group. That critical statistic was not published in the Lancet, but the necessary figures were there for anyone prepared to do some calculations.

[NNT, number needed to treat: the number of patients that have to be treated in order to get a positive treatment effect in one of those patients. Calculated as 100 divided by the percentage of patients improved, rounded-up. E.g., 100/15(%)=6.7 NNT=7]

An NNT of 7, if valid, is a good result for treatment of a chronic condition in which if patients improve at all, it generally happens very, very slowly. Much higher NNTs can still be considered useful in a chronic, deteriorating or fatal conditions, especially if there is a good economic argument for the treatment.

I have seen that there is lots of attention on changing the Primary Outcome measures, and I wrote about that as well. Tampering with the Primary Outcomes is very dodgy practice and SHOULD have resulted in withdrawal of the PACE trial. But it has not yet done so and IMO it is far from certain that it will. This tampering and the numerous other key arguments that myself and others have made, do not demolish the trial data for GET or CBT. Anyone who has studied the data closely would know that.

What really undermines the PACE Trial IMO, but at the same time represents a problem for patients, is that there was far too much improvement in all the groups. This should not have occurred in a well defined M.E. or CFS participant population. The actual improvement which did occur (like it or not) can be misrepresented as a treatment effect. White et al implied that it was a result of getting Specialist Medical Care, which all participants received. Hence their finding that "CBT when added to SMC" and "GET when added to SMC" helps 'CFS/ME' patients etc., etc. They made SMC an integral part of the success of the treatments and reported that 60% "improved" and 30% "got back to normal" (Normal Range).

Therefore it is easy to undermine the claims made for GET and CBT in the PACE Trial, but it is not so easy to achieve two important goals. 1, to use the evidence to expunge GET and CBT from recommendations for these 'treatments'. 2, to use the evidence to show that what the PACE trial actually showed, is that GET and CBT do not treat M.E. or CFS, and therefore the theories on which these treatments are based are proved wrong. It can be achieved by showing that the broad improvements were spontaneous and unrelated to any aspect of participation but are as a result of selection bias: i.e., either deliberately or inadvertently choosing participants EXPECTED to benefit from the experimental treatments - like choosing 'fatigue' patients for GET. The whole purpose of randomisation is to avoid this confounding pit-fall but it did not completely work in the PACE Trial because the cohort from which randomisation was done, was already contaminated and resulted in a small treatment-effect.

Best Wishes,
Peter

This strikes me as a very odd way of going about things. Why wouldn't you read the other papers? Science isn't based on originality of thought or syntax (which is what creative writing is for*). Reading your contemporaries is pretty much a prerequisite for good science, otherwise, how can you be sure you're not repeating the same flaws?



*And even writers read widely to make sure they're not just rehashing something someone else has done better, and to keep in touch with what's going on in the writing world. I know because I am a writer, and the first thing I say to any student of mine is: read more!

 

[Adrian]

I may still be missing things but has S4ME submitted a response to Nice as a response from the forum? Several individual people have written their own opinions but have I missed an overall response from here?

My memory is appalling lately so apologies if I have missed an important document.

We have been preparing a submission. The deadline is 26th July. I have just started a thread with the points we are making here

https://www.s4me.info/threads/s4mes...-me-cfs-guidelines-draft-scope-document.5092/

 

[Peter Kemp]

To illustrate my previous points here are a couple of charts which show CHANGES to the primary outcome measures for every PACE participant with data. Please don't let this worry you, none of this data withstands critical scrutiny, but it does show what White et al have succeeded in defending for 7 years, because it looks favourable to their case. Changing the Primary Outcomes or any other thresholds does not alter the fact of this data and as I have remarked, I don't think that placebo or deference completely override it. IMO this is selection bias at its absolute worst.

Each participant is represented by a diamond whose colour denotes which trial-arm they were in. The number scale on the left is the points of CHANGE between baseline and outcome (one year).

 

[Adrian]

The reason that I have kept away from other people's analyses of the PACE trial recently, is because I have done my own independent review and if I can get it published, do not want to be accused of plagiarism should my findings overlap. I have spent hundreds of hours analysing the data in every way imaginable.

If you are aiming to publish a review then that will need to be set in the context of the current literature and hence you should be expected to have a related work or discussion section that does this. Hence you need to understand and comment on other peoples published work.

 

[Trish]

Keeping oneself in deliberate ignorance of others' findings and arguments is no way to make progress. In academia, the norm is to study what is known, find the gaps in knowledge, and attempt to fill them. This isn't what you're doing. I hope you will think again about this.

I agree, I find it very puzzling that anyone doing any scientific research would deliberately keep themselves in ignorance of the most up to date research in exactly the same area. In this case, on the same set of data. It makes no sense to me. Surely your aim should be to see what's already been done, and whether you can add a new perspective or not before you consider trying to seek publication.

 

[Adrian]

Please don't let this worry you, none of this data withstands critical scrutiny, but it does show what White et al have succeeded in defending for 7 years, because it looks favourable to their case. Changing the Primary Outcomes or any other thresholds does not alter the fact of this data and as I have remarked, I don't think that placebo or deference completely override it. IMO this is selection bias at its absolute worst.

The argument many here are making is that such results are due to reporting bias rather than selection bias. This is supported by comparisons when looking at objective vs subjective outcomes. Also if you look at the correlation between changes in the subjective scores vs the 6mwt it is different between the CBT/GET and APT/SMC groups.

I can believe that getting some medical help will lead to symptom improvement (for example, some decent pain medication).

I don't understand your claims for selection bias. At the heart of this claim there is an assumption that CBT/GET will help some with fatigue but not others and then you need to give evidence for this split. What was it in the selection process that supported bias, what are the possible different mechanisms. Why did the figures for Oxford+London, Oxford+CDC, Oxford show roughly the same improvements.

 

[Peter Kemp]

Dear Adrian, when I obtained the raw data from the PACE Trial I wanted to analyse it in my own way without external influences directing my attention one way or another, quite aside from the fact that it was impossible to find anyone with M.E. willing to help. So in keeping with my preferred approach of complete immersion into the data that is what I did. Since then I have had too much else to do whilst coping with chronic illness, but I will try to do even more.

This strikes me as a very odd way of going about things. Why wouldn't you read the other papers? Science isn't based on originality of thought or syntax (which is what creative writing is for*). Reading your contemporaries is pretty much a prerequisite for good science, otherwise, how can you be sure you're not repeating the same flaws?



*And even writers read widely to make sure they're not just rehashing something someone else has done better, and to keep in touch with what's going on in the writing world. I know because I am a writer, and the first thing I say to any student of mine is: read more!