Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[Barry]

Glasziou has said words to the effect that 'oh, but we know it works from clinical experience'.

If that were all that counted then why would anyone go to all the time and expense of running clinical trials? One of the main reasons for trials presumably is to guard against flawed conclusions being drawn from such things as clinical experience, which though valuable must be strewn with opportunities for all manner of biases.

 

[InfiniteRubix]

@Hilda Bastian

I'd be grateful if you can please confirm email addresses (also by direct message if need be) for

1) you/the independent group
2) the person who carries the responsibility within Cochrane for commissioning/monitoring the review
3) the person within Cochrane who is responsible for final sign off/chairing final sign off of the final review

There will be open correspondence regarding the pertinent legalities to Cochrane and those three parties will be copied. Your best intentions/approach are not in question and the letter is not regarding the independent group.

Many thanks.

 

[Hilda Bastian]

@Hilda Bastian

I'd be grateful if you can please confirm email addresses (also by direct message if need be) for

1) you/the independent group
2) the person who carries the responsibility within Cochrane for commissioning/monitoring the review
3) the person within Cochrane who is responsible for final sign off/chairing final sign off of the final review

There will be open correspondence regarding the pertinent legalities to Cochrane and those three parties will be copied. Your best intentions/approach are not in question and the letter is not regarding the independent group.

Many thanks.

Just two. The emails are on Cochrane's website
The email for me/IAG is: cochrane.IAG@gmail.com
The person responsible is the editor-in-chief: Karla Soares-Weiser, ksoares-weiser@cochrane.org

Under the circumstances, I don't think it would be wise for me to reply to further posts until I've gotten advice, and I don't know how long that will take. Sorry, everyone.

 

[Adrian]

Glasziou has said words to the effect that 'oh, but we know it works from clinical experience'. Yet as I pointed out to NICE you can see from PACE that nobody could know that.

The harms paper Graham and others wrote tried to look at what recording they do in clinics to see if they would know if patents improved or got worse and they didn't have strong enough recording for clinics to know if people are harmed. Therefore I assume that by 'clinical experience' what they mean is the impression given by patients who don't want to upset therapists rather than any real analysis and objective measures showing patients are much better (even without control groups to say they would be anyway!).

 

[Caroline Struthers]

Thanks for the links about the ACTIVE project.

Here's the text from March 2020 - the page is archived here:

The review’s revision and update will be conducted by systematic reviewers from Cochrane Response (Cochrane’s evidence consultancy unit), and additional new co-authors are being considered.

Great. Thanks very much. Is this the first time I sent a link to the ACTIVE project (which is where the idea for this pilot originated I believe)?

Is there a log of de-duplicated suggestions received via the IAG email and Twitter? I can look back over my own emails and Twitter if there isn't. I realise the correspondence will be archived when the process is finished, but if the emails were suggestions and comments rather than questions (which you answered), it's not clear how these were taken into consideration.

 

[Trish]

Hilda has partly answered this herself. I think a proportion of the workforce involved would have been non-negotiable in practice. The real task here is to confront Cochrane on its own 'cognitive dissonance'. I think the way to do that is to mix all the disparate viewpoints, including one or two people who really understand the problem, so that everyone at least knows where everyone else is coming from.

I am not saying that I think that a sensible conclusion will be reached but what actually gets written in this review is probably of less importance than trying to get into the public domain how much of a mess things are in, with the possibility of getting people to face up to how muddled their thinking is.

The objectivity thing is probably a red herring. Both NICE and Cochrane have scored the evidence low or very low I think. The problem is that some people feel unable to not give a recommendation for non-pharmacological treatments like this even if the basis is as poor as it gets. Glasziou has said words to the effect that 'oh, but we know it works from clinical experience'. Yet as I pointed out to NICE you can see from PACE that nobody could know that. And what is the point in a systematic review if trumped by anecdote? The cultural influence here is in not being able to resist recommending a treatment you have faith in, in the face of clear lack of usable evidence.

I find that post very depressing. Are you saying that all we can expect is people who believe more in their own therapy than in scientific oversight of whether that therapy has any positive effect should have a part in the review process? Why? It's not a debating society where all views, however self serving and unscientific need to be heard and included in the review? Or that we have to accept yet another flawed review because the problem is really with Cochrane's failed systems and that has to be tackled first?

This paragraph makes no sense to me as a patient affected by this:
I am not saying that I think that a sensible conclusion will be reached but what actually gets written in this review is probably of less importance than trying to get into the public domain how much of a mess things are in, with the possibility of getting people to face up to how muddled their thinking is.

When and how do you realistically expect such enlightenment to dawn? And if the process of turning the ship around isn't started by a good review team showing the way by doing a scientifically literate review, how else will it happen?

@Hilda Bastian

I'd be grateful if you can please confirm email addresses (also by direct message if need be) for

1) you/the independent group
2) the person who carries the responsibility within Cochrane for commissioning/monitoring the review
3) the person within Cochrane who is responsible for final sign off/chairing final sign off of the final review

There will be open correspondence regarding the pertinent legalities to Cochrane and those three parties will be copied. Your best intentions/approach are not in question and the letter is not regarding the independent group.

Many thanks.

Just to clarify, this is the first I have heard of this. Whatever Infinite Rubix is planning in the way of letters (or legal action?) is not a forum project, though of course any member is welcome to use the forum for ME advocacy communication. Perhaps, @InfiniteRubix, you could clarify what you are planning when you are ready. As I'm sure you know, open letters as part of ME advocacy can be posted here:
https://www.s4me.info/forums/open-letters-and-replies.100/

Under the circumstances, I don't think it would be wise for me to reply to further posts until I've gotten advice, and I don't know how long that will take. Sorry, everyone.

Thank you for reading our posts and doing your best to reply. I do understand that you may feel under constraints because you have been employed by Cochrane to carry out a specified role.

 

[InfiniteRubix]

@Hilda Bastian

I'd be grateful if you can please confirm email addresses (also by direct message if need be) for

1) you/the independent group
2) the person who carries the responsibility within Cochrane for commissioning/monitoring the review
3) the person within Cochrane who is responsible for final sign off/chairing final sign off of the final review

There will be open correspondence regarding the pertinent legalities to Cochrane and those three parties will be copied. Your best intentions/approach are not in question and the letter is not regarding the independent group.

Many thanks.

Many thanks indeed Hilda and for your role on the IAG, including outreach. It's valuable, which ensuring discipline of the entire process does not detract from.

As per @Trish's message, this is indeed not an S4ME or an advocacy communication that will be sent.

 

[rvallee]

The cultural influence here is in not being able to resist recommending a treatment you have faith in, in the face of clear lack of usable evidence.

Oh I would love to see this written in the sky, perhaps with fireworks and other glitzy things like giant confetti, just to be sure the message gets across, maybe with fighter jets doing the colored smoke thing and maybe a wild kangaroo, I don't know.

Without faith in this, the evidence amounts to the same as there is for astrology. Stars exist. The mind exists. They influence things, therefore they should do X, where X is the belief system. Nothing more to it. Without the belief, it amounts to nothing, not even worth looking at. This is a true socially spread belief in illness, because as with everything BPS ideologues, it's all projection.

 

[Lidia Thompson]

In reality, the crux of the issue is surely this:
https://www.mdpi.com/2227-9032/9/6/683

There is a group of patients who suffer from fatigue, who actually improve with exercise.

There is another group of patients (who can look similar) who have the cardinal symptom of PEM/PENE/PESE ... whatever you want to call it ... and will deteriorate if they push beyond the energy that their bodies are able to generate. This is objectively verifiable. This is the group for whom GET is contraindicated.

Anecdotally: Those (who do get PEM) who go on pushing over a protracted period of time, deteriorate very substantially and sometimes permanently.

Ipso facto, if one continually run trials of exercise that do not differentiate between the two groups, is it any wonder that we come to such a polarised view?

Why, oh why, is this so difficult to understand?

@Hilda Bastian
If NICE and Cochrane don't get this right (i.e. fail to distinguish between the two groups) and GET continues to be prescribed to the ME/CFS group (and to the Long Covid group that is similarly exhibiting PEM) then GRAVE HARMS will inevitably be perpetuated.

 

[Lidia Thompson]

Similar results were seen in female patients btw:
https://www.mdpi.com/2227-9032/9/6/682

 

[Jonathan Edwards]

Are you saying that all we can expect is people who believe more in their own therapy than in scientific oversight of whether that therapy has any positive effect should have a part in the review process? Or that we have to accept yet another flawed review because the problem is really with Cochrane's failed systems and that has to be tackled first?

Not quite either. I think we may have to accept under the circumstances that to have people who believe in therapies involved may get things further forward than not. A key point here is maybe that the right answer to the review - that the evidence is no good - has already been admitted by the advisers to Tovey who recommended it was withdrawn. And of course NICE has done its own review and come to the same conclusion. I don't actually think, now that NICE has met, that any new review will have any impact on anything. The purpose of the exercise lies elsewhere.

But I am not totally pessimistic that the re-review will be as flawed as the first. To repeat the claim that exercise is worth recommending even the believers are going to have to fly in the face of some pretty robust argument from Mary D and George F amongst others. And even if it is flawed I don't think anybody has to 'accept' it. It is no more than Cochrane trying to get into some sackcloth and ashes while the rest of the world knows what they did.

 

[cassava7]

And of course NICE has done its own review and come to the same conclusion. I don't actually think, now that NICE has met, that any new review will have any impact on anything. The purpose of the exercise lies elsewhere.

This may apply to the UK, however the Cochrane review has a significant impact abroad — especially in countries where no official clinical guidelines have been established for ME/CFS yet —. Clinicians, including physiotherapists, often bring up the current review to justify delivering (graded) exercise therapy to people with a diagnosis of CFS.

 

[Hutan]

In reality, the crux of the issue is surely this:

I don't think it's as clear cut as that. That study appears to have done CPET studies on a number of male patients over a period of time. The CPETs were part of a diagnostic process, which classed people as having ME/CFS or not. I think it's highly likely that anyone having a positive CPET test would be assessed as having PEM, and the fact that they are in the clinic means they have fatigue. So probably all of the people with a positive CPET got the label 'ME/CFS', and the people with a negative CPET got the label 'ICF'. The researchers did not actually do new CPETS, so we don't know how replicable CPETs are. (We need studies on that.)

It's very possible that some of the people showing a good response to exercise in the paired CPETs were just having a good period in their illness. Perhaps their trigger for PEM was higher than 8 minutes on a bicycle. Perhaps some of them were really well rested before the test and some of them weren't.

So, it's quite likely that people with a positive CPET in this clinic were classed as having ME/CFS. And so when charting the results of the people diagnosed with ME/CFS and people diagnosed as without ME/CFS, it is hardly surprising that the people with ME/CFS are found to have a positive CPET. It's circular.

edit: link to the discussion thread on this paper
Comparing Idiopathic Chronic Fatigue and ME/CFS in Males: Response to Two-Day CPET, 2021, van Campen and Visser


Trials with mixtures of people with fatigue meeting and not meeting criteria requiring PEM have not shown GET is helpful. Why could that be?

One possibility is that processes for assigning people to various criteria aren't very good. It might be hard to know if you have PEM, if no one describes it to you well, or if your life is not very regimented, so that it's hard to see patterns. The doctor asking the questions might just have been really useless. So some of the people who are recorded as not meeting more stringent criteria might actually get PEM, and might really have ME/CFS.

Another possibility is that graded exercise therapy isn't helpful in more fatiguing conditions than just ME/CFS.

Another possibility is related to the fact that the efficacy of GET relies on two assumptions - one is that exercise really does improve the health condition. The second is that telling someone that they need to exercise more actually makes them exercise more. So, even if, in theory, exercise could make some of the people with fatigue feel better, if telling them to exercise doesn't make them exercise more, then telling them to exercise is not a useful treatment.

Another possibility is that GET artificially reduces the amount of exercise a person is doing to start with - to establish the baseline before gradually increasing it. The end result might therefore be some people feeling a bit better initially because of less activity and then feeling as bad as ever as the level of activity goes back to what it was. But, overall, the amount of exercise the participants do doesn't change from trial start date, and so even the people who might benefit from exercise don't experience a benefit.

 

[Hutan]

This may apply to the UK, however the Cochrane review has a significant impact abroad — especially those where no official clinical guidelines have been established for ME/CFS yet —. Clinicians, including physiotherapists, often bring up the current review to justify delivering (graded) exercise therapy to people with a diagnosis of CFS.

This. The Cochrane reviews are a major obstacle to getting better guidelines in place around the world. A finding that GET is useful for ME/CFS would be a tremendous set-back. I feel uncomfortable gambling on that, for the sake of Cochrane sorting out its internal issues. Cochrane seems to have been fine with Paul Garner departing from evidence-based medicine by telling the world that his personal experience proves ME/CFS is a mind over matter issue. As far as I am aware, he is still the Cochrane Coordinating Editor of the Infectious Diseases Group - so I don't have much confidence in Cochrane management's ability to ensure good science is done.

We don't know how the IAG is going to work, but I've seen no evidence yet of processes that will allow members of that IAG to exert useful influence. Even if they can, you have the likes of Peter Gladwell in that group - so we are still relying on the review group to be sensible about how they use IAG input.

If you look at the latest CDC evidence review (which Mary Dimmock contributed to), you will see that she, on her own, can't ensure sensible outcomes (no one could). So we really need the systematic reviewers to be smart and unbiased. I don't know anything about them.

Review team

• Consumers: Mary Dimmock (USA, carer, person with ME/CFS advocacy background), Kay Hallsworth (UK, person with ME/CFS);
• Clinicians: Todd Davenport (USA, physiotherapy), Julia Newton (UK, medical);
• Systematic reviewers with relevant methodological expertise: David Nunan (UK, experience with systematic reviews of exercise as an intervention); Argie-Angeliki Veroniki (Greece, statistician);
• Systematic reviewers from Cochrane response: Hanna Bergman (Israel), Nicholas Henschke (Germany, lead author).

 

[mango]

This may apply to the UK, however the Cochrane review has a significant impact abroad — especially those where no official clinical guidelines have been established for ME/CFS yet —. Clinicians, including physiotherapists, often bring up the current review to justify delivering (graded) exercise therapy to people with a diagnosis of CFS.

Thank you, cassava7. I agree. This is still happening in Sweden. Not just clinicians but the Social Insurance Agency too. Cochrane's reviews have been cited over the years as evidence*, in some cases in the appeals court, for example. And by the people looking into developing clinical and insurance related guidelines, whether regional or national.

Edited to clarify:
* The Social Insurance Agency sometimes uses Cochrane's reviews in order to deny the ME patient prolonged sick pay, disability pension etc, by claiming that the patient hasn't tried all available effective treatments.

ETA: Swedish researchers are using Cochrane's reviews to strengthen their own hypotheses and models as well (for example ACT/graded activity studies by Olsson, Wicksell, Jonsjö et al).

 

[Lidia Thompson]

I don't think it's as clear cut as that.

Too clever by far for me Hutan!

However, what this study (and others like it) does help to illustrate, is that the reporting of harms from GET in ME/CFS has a clear physiological basis.
One should not continue to exert when there is an inadequate uptake of oxygen.
In asthma and heart disease this is a given. It really is time it was a given in ME/CFS.

My point: Cochrane should recognise that exercise should not continue to be endorsed before we understand all the parameters and limitations involved.

As for Trudie Chalder ... after decades ... for her to still to be saying that spending 2 weeks in bed completely wiped out after exertion is 'an extreme behavioural response' is beyond the pail.

Why does she (and others like her) still get a say? Really?

 

[Snowdrop]

This may apply to the UK, however the Cochrane review has a significant impact abroad — especially in countries where no official clinical guidelines have been established for ME/CFS yet —. Clinicians, including physiotherapists, often bring up the current review to justify delivering (graded) exercise therapy to people with a diagnosis of CFS.

I expect though that it is justification after the fact. They are physios after all and if someone with ME shows up a physio is going to do physio. So the problem comes before that in maybe referrals.

Even if cochrane doesn't keep up the pretense I expect there will be those in a position to do so that will continue to carry on as usual. It's not meant as an excuse for cochrane but there is so much institutional investment here. And yes, people may continue to be harmed but I don't think that's all down to cochrane.

 

[Hutan]

However, what this study (and others like it) does help to illustrate, is that the reporting of harms from GET in ME/CFS has a clear physiological basis.
...
My point: Cochrane should recognise that exercise should not continue to be endorsed before we understand all the parameters and limitations involved.

Yes, I agree.
I'm not completely sure that the CPET studies are rock-solid yet (and I've done a CPET and shown the performance drop, so I'm biased towards them being evidence) - we could use more replication from some independent exercise physiology researchers. But it is does look like evidence worth considering.

And it's not even that Cochrane needs to recognise that exercise should not be endorsed - I mean we all do activity to a greater or lesser degree. And I think we all agree that activity is good - it gets stuff done and it helps keep our bodies working. It's the 'exercise therapy' that's the problem - the assumption that we aren't doing enough activity to start with, even if that is as much as many healthy people our age; and the assumption that increasing activity levels has any positive impact on the illness.

 

[Hutan]

Even if cochrane doesn't keep up the pretense I expect there will be those in a position to do so that will continue to carry on as usual. It's not meant as an excuse for cochrane but there is so much institutional investment here. And yes, people may continue to be harmed but I don't think that's all down to cochrane.

Where I am, there isn't really anything in the way of services for people with ME/CFS at all. Hardly anyone is going to actually be prescribed GET; it's more likely the doctor will just say 'perhaps you could try getting out and about a bit more?' and not advocate for help with household tasks or mobility aids in case that would be facilitating your laziness. There's no institutional investment at all, and I'm sure that applies to large chunks of the world.

But what the Cochrane Exercise Therapy Review has allowed, is the unquestioning adherence to the idea that ME/CFS is a behavioural issue. And that means that when you try to argue with politicians for research funds, they just consult their doctor friends, and they come back looking pitying at you and say that ME/CFS research is not a priority. I've had 'Cochrane says..' quoted at me. Or when you are trying to get care for some other health problem, the doctors see 'ME/CFS' in your notes and don't believe anything you say.

Cochrane's reviews have been absolutely fundamental in propping up the 'ME/CFS is a behavioural issue' idea and preventing progress.

 

[Snowdrop]

Cochrane's reviews have been absolutely fundamental in propping up the 'ME/CFS is a behavioural issue' idea and preventing progress.

Yes.

I'm hesitant to post more, I need to think how to be more clear in what I'm meaning. It's past whatever amounts to prime thinking time for me today.