Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[Hutan]

I realize to analyse a trial in sufficient detail to really understand the issues is a big task but it feels necessary.

Maybe, when the list of trials to be looked at is decided (and we can probably identify them now) we can make sure we have threads on them, and forum members can analyse them. Probably not to feed into the process directly, but anyone involved could look at the forum comments, and it would help us be prepared to challenge anything that needs challenging. We could have a thread with a master list that links to each of the threads. I expect we have threads for most of the studies already.

 

[petrichor]

Why not? Research methodologists who deal mostly with drug interventions rather than behavioural interventions would probably understand the issues with PACE -- surely Cochrane have some. Expert patients who understand the methodological issues would understand the issues with PACE very well. I don't see any difficulty in having people who can deal with the sorts of issues in PACE apart from having a willingness to have those people in the team.

It does seem likely methodologists who deal with drug interventions would understand the issues with PACE better, but I just assume they may not have quite the right experience for the review or may not be regarded as having the right experience.

The Cochrane reviewers have an interest in not criticising methods that they've marked as acceptable on other reviews.

Julia Newton has an interest in not criticising methods she has used, or assumptions that she has made, in her research. Have you read much of Dr Newton's work? I actually argued before that most people understand that exercise doesn't fix everything, but Newton's paper is pretty strongly pro-exercise. I suggest there is a bit there to suggest that she is ideologically opposed to criticism of the universal utility of exercise.

I'm not that familiar with her work but based on what you've quoted it seems like she does have an almost ideological pro-exercise stance, which is concerning. And the cochrane reviews do have that inherent COI, which I wouldn't call an outright COI, but it's an unfortunate drawback. Like I sort of mentioned, to take account of the flaws in PACE, it seems like they would need to take account of things which they aren't used to looking at when doing reviews.

How can you tell that Kay is a 'fairly good pick'? What do you know about her skills in research methodology, or explaining to others the issues with GET trials, or standing up to other members of the panel about the problems with BPS methodology? How can you be sure that she does not have an association of some sort with BPS people? How can you be sure that Kay doesn't attribute her 'high-functioning status' to the fact that she does yoga and keeps active, and thinks that people with lower levels of function need some help in the form of GET to get motivated and active too? Of course I'm speculating and she probably doesn't, but I don't think we know.

My judgement on Kay is based mainly on looking at her twitter. She's talked about her negative experience of being told how ME is in her head and she just needs to exercise more, and has actually recently liked some tweets from people (Keith Geraghty and Sten Helmfrid) criticising the evidence base for CBT for ME. I agree there doesn't seem to be anything to indicate she has a good understanding of research methodology or issues with GET trials. But for playing the role to represent the experience of having ME, she seems good enough, and she doesn't seem to be pro-GET or anything like that. And I also just get the impression she seems smart and fairly competent, so maybe she could pick up on and understand the issues at stake.

 

[Trish]

There are many researchers, clinicians and scientists who have looked at the trials that will be the subject of this review and found exactly the same problems with the research as the expert patients found - the numbers of expert critics is ever expanding now with the advent of Long Covid.

This is not about consumers versus providers, or about anti-exercise opinions versus pro-exercise opinions, or amateur patients versus expert clinicians.

The idea that it's about opinions about exercise, so therefore you need people with different opinions on the value of exercise on the review panel is crazy. None of us is anti exercise. We are anti anything that makes us sicker and has no evidence of efficacy. We want effective treatment, whatever that is. I would be thrilled if I could get better by going for longer walks each week. It would be wonderful. None of us would be here on this forum if all we needed to do was be a bit more active each week. I dream of being able to do a simple thing like walking to the local shop without spending the next week in bed with crashing headaches and feeling too sick to eat and with my muscles in so much pain and so weak I can't stand up. I dream of seeing my daughter able to sit up long enough to do the handicrafts she used to enjoy before she got too sick, let alone being able to have a job, a family, a life...

Therapy professor, Trudie Chalder, one of the PACE researchers, after 30 years as a 'leader in the field' describes Post Exertional Malaise as "Health anxiety" and an "Extreme behavioural response" and states "Exercise is clearly not damaging" and recommends "Increasing exercise". (see this post). She has no idea.

And yet Hilda seems to think it's right to have a 'balance' and a 'dialogue' and that we mustn't have too many 'activists' up against those highly prejudiced and unscientific views.

I want scientifically competent reviewers, not a 'balance of opinions' on the panel.

 

[Sean]

I want scientifically competent reviewers, not a 'balance of opinions' on the panel.

This.

 

[Ariel]

I would be thrilled if I could get better by going for longer walks each week. It would be wonderful.

Same. It would change all of our lives. Has anyone here *not* tried this at some point in their illness? I doubt you will find many patients who have not tried gradually increasing activity.

Zooming out a bit, it is so bizarre that there is still discussion around whether or not doing the very thing that makes us ill - it's characteristic of the condition - might be good for us, actually.
There is nothing "balanced" or sensible about discussing whether exercise might treat ME/CFS. It's just a misunderstanding of the condition. I have to admit that the title of this review made me wince. It just sounds so absurd. Much of the literature uses criteria for inclusion/ outdated and incorrect case definitions that do not include PEM, and that fact cannot be emphasised enough. It's unclear how many participants even had ME/CFS. Speaking loosely of "ME/CFS" without clarity about the case definition is hopeless and harmful.

 

[Hutan]

But for playing the role to represent the experience of having ME, she seems good enough, and she doesn't seem to be pro-GET or anything like that.

The process should not be a negotiation between two sides. It's a review of the evidence against the standards of good science. If there are expert patients (and there are), at least one should be used, but all it really takes is people expert in research methodology who aren't prejudiced against people with ME/CFS. If you had decent people, you'd only need three - an expert patient, a good Cochrane person familiar with Cochrane protocols and an independent research methodologist. I think this has turned into something that is way more of a production that it needs to be.

There's no need to have someone there just to be 'the person with ME', especially one that seems to have uncritically accepted BPS ideas. What are they going to say that has any bearing on the analysis?

Zooming out a bit, it is so bizarre that there is still discussion around whether or not doing the very thing that makes us ill - it's characteristic of the condition - might be good for us, actually.

I know I keep going on about that Newton paper, but 10 years ago Julia Newton reported that the ME/CFS cohort was walking an average of 7000 steps per day (way over the UK average), and their pattern of lengths of sedentary time (as in getting up periodically) was no different to the healthy controls. And yet she still concluded that it was vital that the people with ME/CFS be encouraged to exercise. And here we are, still trying to get rid of people who think we would be well, if only we increased our physical activity by 10% per week....

 

[Sly Saint]

@Hilda Bastian just wondering if you considered asking Dr Nina Muirhead?

 

[Suffolkres]

Just as a point of clarity:

Obviously, I can't comment on discussions or outcomes, but I can reassure you that we see everything. There are no secrets.

The summaries are presented because we're often dealing with, as you can see from the evidence reviews, hundreds to thousands of pages of evidence and material at a time.

The summaries are provided in addition to giving us access to the raw info those summaries are based on, not instead of.

Many thanks Adam. I will let HJB know!

 

[Ariel]

I know I keep going on about that Newton paper, but 10 years ago Julia Newton reported that the ME/CFS cohort was walking an average of 7000 steps per day (way over the UK average), and their pattern of lengths of sedentary time (as in getting up periodically) was no different to the healthy controls. And yet she still concluded that it was vital that the people with ME/CFS be encouraged to exercise. And here we are, still trying to get rid of people who think we would be well, if only we increased our physical activity by 10% per week....

I have really appreciated your posts about this and it's really important to keep talking about it. It's something that should be easy to grasp.

 

[Invisible Woman]

Another point which I don't believe has been mentioned yet with regard to a patient representative who is not part of the patient community -

One of the issues I have observed in over 2 decades of having ME is that some ME patients who don't mix with others of different severity levels do not understand the different battles & difficulties they face.

For example - when I was merely moderately affected and mixed with mild to moderate patients I often heard the attitude expressed that the difference between the two groups was a lack of effort, will, gumption or the right advice.

The individuals expressing these views really struggled - some still struggling to work, albeit part time, still struggling to exercise etc. The more severe patients were sometimes viewed as people who wanted an easy answer or had simply given up. Like that campaign for natural childbirth the severe were dismissed as "too posh to push" through.

On the other hand severe patients who don't get much input from others can sometimes think of the more mildly affected who can still hold down jobs (albeit part time), exercise, carry out the day to day tasks of self care and so on as having a much easier time of it.

Neither viewpoint is true, but without interaction with the wider ME community - even if you don't take part in advocacy - then you are less likely to understand the difficulties faced by patients who are either more or less severely affected than you.

In my own case, for the first 10 years or so I was able to exercise, as long as I was very careful & sacrificed other things in order to be able to do so. While I hope I wasn't too judgemental of others, I certainly had no concept of what severe ME felt like.

As I've mentioned upthread one of the key messages impressed upon the newly ill is not to join patients groups. That the groups are militant, anti science, etc. However, not being part of the wider community means you don't have the opportunity to learn about others with different levels of function and less likely to hear the other side of the discussion on the BPS point of view.

So, in my opinion, a patient representative who is not a member of the wider ME community is less likely to question establishment points of view and less likely to understand the difficulties other patients might face.

 

[BurnA]

I would be thrilled if I could get better by going for longer walks each week.

Wouldn't we all.
This should be an obvious reason that exercise is futile for ME patients.

If exercise made us better we would all be doing it. Unless of course we didn't want to get better. In which case, proving that exercise can make us better is pointless, our problem is we don't want to get better.

So the question must be, will exercise make us want to get better? And if we want to get better, will we get better?

 

[JemPD]

Neither viewpoint is true, but without interaction with the wider ME community - even if you don't take part in advocacy - then you are less likely to understand the difficulties faced by patients who are either more or less severely affected than you

The individuals expressing these views really struggled - some still struggling to work, albeit part time, still struggling to exercise etc. The more severe patients were sometimes viewed as people who wanted an easy answer or had simply given up. Like that campaign for natural childbirth the severe were dismissed as "too posh to push" through.

These points are a big concern for me. The times i hear comments such as "i dont have the choice to stay in bed i have to get up to look after my children/pay the bills" etc on places such as ME Association's facebook page. They really have no concept that those who are bed bound have no choice but to stay there - that's what 'bound' means. I once gently pointed this out to someone, and she said "my nurse tells me i have a very severe form of the disease, i had to cut down my work hours", she had absolutely zero idea that there were people who hadn't been able to sit up or turn on the light for decades regardless of the consequences.
Thats what worries me about having patient reps who arent involved in th community. Even 'mild' ME/CFS is a complete nightmare, but it's a wild fantasy to those who are very severe. And if you dont have a patient rep who understands that, then we are in very big trouble

 

[MEMarge]

@Hilda Bastian just wondering if you considered asking Dr Nina Muirhead?

I think that sadly she would be too unwell to be part of the process. Alongside working part-time and a young family, she is already working hard on research, advocacy and medical education.

However listening to some of the videos she has produced, such as the Canadian one would be really useful to improve the understanding of this IA Group, as would the Invisible Illness videos and compulsory reading of @Jonathan Edwards expert testimony to NICE.

To be honest a fair amount of Cochrane's own videos/blogs etc on stats/bias etc isn't bad. It just doesn't seem to apply to any papers on ME.

 

[Snowdrop]

I think that sadly she would be too unwell to be part of the process. Alongside working part-time, a young family she is already working hard on research, advocacy and medical education.

However listening to some of the videos she has produced, such as the Canadian one would be really useful to improve the understanding of this IA Group, as would the Invisible Illness videos and compulsory reading of @Jonathan Edwards expert testimony to NICE.

To be honest a fair amount of Cochrane's own videos/blogs etc on stats/bias etc isn't bad. It just doesn't seem to apply to any papers on ME.

I would be interested to have links provided for the above mentioned videos. I did a quick google and nothing came up that was related.

 

[Adrian]

But for playing the role to represent the experience of having ME, she seems good enough, and she doesn't seem to be pro-GET or anything like that.

So what I don't understand is if there is such a role in a review process. They are not designing trials but reviewing what was done and aggregating results.

 

[Woolie]

So what do you do when people are deeply entrenched in opposed camps of people who have convinced each other without any doubt that they are right, and they each have an influence on the ground, where everyday decisions and actions are happening. Something has to shift, including in people's minds - and not just in one country. I don't believe you get there by people not talking to each other, not listening to each other, and not being seen to do so.

What's all this talk about entrenched points of view? As if we're talking about a political issues? Where talking about what constitutes reliable evidence for a treatment benefit.

A question for you, @Hilda Bastian: if you read a homeopathic study of ME which reported benefits on self-reported measures within a few months after the (12 week treatment), and the study was open-label, would you accept that as evidence that homeopathy works?

If your answer is no, would any time chatting with homeopaths be likely to change your mind? I'd say it wouldn't, because you're not against homeopathy per se, you just know too much about the biases that can be introduced in open label trials. You are unlikely to change you view on this at all during the discussion. Your view is based on your understanding of bias in science.

Now would your answer be any different if the design was the same but the treatment was exercise?
If your answer is still no, then that's again because you have a strong view that blinding is important in treatment trials, and that even if a treatment can't be blinded, we cannot just ignore the problem altogether. We need find some other way to minimise these sources of bias or alternatively, asses their impact on the outcomes. If that's your view, then I expect no amount of chat with exercise therapy providers is likely to dissuade you of that opinion, its based on your understanding of science.

 

[Adrian]

Now would your answer be any different if the design was the same but the treatment was exercise?

A blind review of trials could be interesting where the intervention is unknown to those looking at the results. But it doesn't work as the intervention sets expectations (like telling people if they ignore symptoms they will get better) and in turn this biases results

 

[Hutan]

if you read a homeopathic study of ME which reported benefits on self-reported measures within a few months after the (12 week treatment), and the study was open-label, would you accept that as evidence that homeopathy works?

That's a great example @Woolie.

And then, if you were conducting a review of whether homeopathy helped ME, you would not need to have a clinician who is a big fan of homeopathy on the review panel, nor would you need to have a person with ME who likes homeopathy on there either, to explain to the others what ME is like. What are they going to say, to add to the process? Probably, 'in my experience, homeopathy is great'. You'd just need a few people who are good at analysing intervention studies and writing up a review.

You wouldn't stress abut the fact that a negative result might upset the homeopaths. You wouldn't put someone on the review panel who has had a history of running unblinded homeopathy trials that involve telling the person that the treatment will almost certainly cure them.

 

[Hutan]

The more the process is politicised, the more it deviates from a straight-forward scientific assessment (what evidence is there? is the evidence any good?; what does the good evidence tell us about benefits and harms?), the more it includes people who don't know what they are doing in a review, the bigger the chance there is for the process to be manipulated by those who want a particular answer.

Sure, run a couple of online workshops with stakeholders ahead of time for the assessors to pick up anything unusual about the review; have a peer review team; have a public consultation period afterwards, to pick up any mistakes and publicise the findings. That's enough.

I know we have been going on about getting the right people into the process, but that's only necessary because there are people there who aren't right, and so we have to try to get balance.

I get annoyed with these suggestions that ME/CFS is special, that it has to be handled in some special way, to 'bring all sides together'. No. All we need is good science - competent work by qualified people with no bias. That is enough to get the result that is needed here.

 

[Hilda Bastian]

I’m a bit puzzled why none of this was included in monthly progress reports from March 2020, as decisions were being made. That is what I had assumed would happen. That way, we could have scrutinised these decisions and processes before people were appointed.

When we heard nothing for 14+ months I assumed that work had stalled. Instead, it appears that a great deal of work was going on but for some reason we weren’t being told about it. Please can you clarify why that was, Hilda?

Since I joined this Forum in May, I assured people that we were working hard and making progress: because reporting was delayed didn't mean that progress and involvement of people with ME/CFS wasn't happening. The March report stated explicitly that we were engaged in establishing an IAG and author team, an address for questions was provided, and questions were answered. And I was answering questions here on this Forum, too.

After the report of 20 March 2020, the pandemic hit its stride, slowing people down generally, and pretty much everything that wasn't pandemic response. For me, trying to ensure that all this didn't end up with the review taking months longer - and those pandemic months turned into a lot of months - meant prioritizing doing what was needed to try to avoid large time over-runs. Without the enormous upheaval of a pandemic, it would have gone differently: this was a very bad time to be trying to undertake something like this. Unfortunately, that's the short straw this process drew.

The time that goes into development and feedback rounds of a substantive report, and setting aside time to respond to questions and feedback about that report when released, was a priority. However, the work on the engagement with the existing critiques and feedback, was a higher one - for the actual review work, but also as it is critical to the consultation process we will be starting. It would have been better to at least formalize as brief updates what I was communicating here and on Twitter.

It was clear the process used to establish these teams, and their members, would be criticized. Issuing a substantive report on the teams was going to require time to respond after that, and I considered allowing that time part of the task, not just its development and discussions about drafts.

Issuing brief reports that we were progressing with what we reported on in March 2020 wouldn't have satisfied what I'm hearing here: if I'm understanding correctly, that seems to be wanting to ensure there was a process more similar, for example, to that undertaken for clinical guideline committees at some major public agencies. Various recruitment options will be discussed by the IAG and in Cochrane as part of considerations and recommendations at the end of the process.