Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[Hilda Bastian]

When you analyse the NICE review, will you critically appraise its scope and methods (such as use of GRADE) or check the data and reproduce the analyses - or both?

I will be looking at the methods and evidence related only to the subject Cochrane is reviewing. I'm not going to check the numbers in their GRADE tables, for example, and I'm not going to try to reproduce their analyses of qualitative or other research. My focus is on the quality of the Cochrane review and supporting stakeholder engagement in the process.

 

[Hilda Bastian]

Were researchers who work on the quality (methodology, reproducibility or replication) of trials of exercise and physical activity in chronic illnesses asked to participate, either as reviewers or to sit on the IAG?

Someone who is familiar with ME/CFS and the evidence base for exercise therapy in CFS is Rosie Twomey, and while others in the field may not (immediately) be, they have likely seen the methodological issues that have been raised with the review elsewhere and understand them.

Yes - and thanks for the suggestion.

 

[Ariel]

I agree it has ended up as a hot potato. But the blame for that lies squarely with one side - the people who did poor quality research, twisted the results to fit their political agenda and personal disdain for pwME, spent public money trying to withhold data from proper scrutiny, and demonised those who pointed out the faults. We need to be completely clear that we are the victims here. I think victim blaming is despicable. That Cochrane should join in that victim blaming is shameful.

I have actually been quite shocked at reading the victim-blaming here on this thread; non-sequiturs and comments regarding "actions" of a "minority" without an explanation of how this bears on the matter at hand. Just trying to shift the blame to pwME for their own mistreatment. Sad and disturbed to see this narrative repeated again, particularly from a group claiming scientific legitimacy.

I urge anyone who believes that ill patients are somehow "playing their part" in a dynamic that has resulted in abysmal care, harmful "exercise treatments", and further unnecessary suffering to rethink where they first heard this narrative about very ill patients, and about whose interests it has ultimately served. We are not "part of the problem". We have been harmed by a confluence of politics and bad science, among other factors.

 

[Hilda Bastian]

@Hilda Bastian, I sorry to have to keep going on about this, I'm really tired of this and I'm sure you are too.

But you still haven't explained how you came to select Kay Hallsworth. I think we've established that she expressed an interest in being on the writing team. And then, as there is absolutely nothing online that would indicate that she has a particular interest and expertise in ME/CFS research (other than she has ME/CFS) and you yourself are not an expert in ME/CFS research and politics, you must have asked around. I just can't see that any informed patient or patient advocacy organisation would have said 'ah, yes, Kay. She is exactly the person you need to have on the writing team to represent patients and inform the rest of the team about the problems with the ME/CFS exercise research, definitely the best person for the job'. I expect they would have said 'who?'. So what organisation was it that supported Kay for the job? She's a patient representative on a process that should be accountable to stakeholders, particularly the stakeholders with the least power and to who the outcome matters most. I think you must be able to say, even if only in general terms, who endorsed her. If I'm wrong, and a credible patient advocacy organisation endorsed her, please do say.

I think we have established that Kay has links with BPS proponents including Wessely and parts of the military interested in managing contested illnesses. If Kay has never met Wessely and White and Etherington, and does not support the statements they have made about ME/CFS and GET, then could she please make a clear statement about that? If she has met any of them and/or does support their views, I think she should also state that, so we all know where we are.

Meanwhile, it seems that you have chosen a BPS proponent to represent people with ME/CFS on a review that was prompted by people who view the BPS proponents as having caused people with ME/CFS great harm. I've appreciate you engaging here, but I'm not getting any clear sense from you that you understand how problematic that is. From here it looks as though you have done something that goes against all that you stand for.

If you hope to make this process credible, there needs to be an expert patient on the writing team. I'm sure that S4ME could help you locate a suitable person.

I've replied previously that I will expand on the process in a methods report.

Kay Hallsworth did not seek this role. I found her via internet searching. No person proposed for the author team was endorsed by any organization: it's a team of authors, not a committee, and no one is a representative, although I appreciate that perceptions about this differ.

Edit: PS - Just realized I didn't explain what it means when I say I'll expand on the process in a methods report: I'll be gathering questions and suggestions, to inform the development of the report (with further questions and suggestions welcomed on a draft).

 

[Hilda Bastian]

Sorry Hilda, could you be clearer please? Are you saying that you accept that among the ME/CFS patient experts there are some with useful technical knowledge and analytical and written skills, including those you specifically named upthread, but among all of those ME/CFS patient experts, there are none who are nice enough to work in an author team? And that is why you had to appoint a patient representative who does not appear to have technical knowledge but whom you judged to be nice?

[edit - crossposted with Woolie, we appear to both be interested in ideas about being nice. )

No, I'm not saying any of that. (Indeed I've said that I had invited one who declined on grounds of illness.) And as I've also said, I would not have proposed for authorship any person who I didn't think would do the job well. Thanks for the suggestions.

 

[Hilda Bastian]

Hilda, you have now been asked a version of this question at least 3 times and haven’t answered. Please can you tell us who these people are and what they have done? If you are not willing or are unable to tell us, please can you explain why?

I don’t speak for the forum but I’m fairly sure that if there are people claiming to speak on behalf of people with ME who are behaving inappropriately or unethically we would all like to know who they are and what they are doing so that we can all condemn their actions and dissociate ourselves from them. However, repeatedly referring to them without saying who they are or what they are doing denies us this opportunity and perpetuates prejudices. That in itself is an unethical behaviour in my view.

I am grateful to you taking the time to answer questions on here, and very much hope that you will not continue to dodge this one.

[Edit: crossed with @Woolie ’s post above]

Thanks. I am reading the thread. If I don't answer a question that's been asked several times, it's not because I've overlooked it: it's because I'm either still thinking about how to answer it or I'm not going to answer it. I did not describe behavior as inappropriate or unethical: I talked about behavior having consequences, and about being effective. I don't think my elaborating on this here is going to be helpful. I'm sure there are people genuinely interested in what behavior I think crosses the line into being counterproductive - everybody has opinions about that, so of course I do. My views are probably very different to those of many people responding in this thread, and I'll be leaving it at that. I'll keep reading though.

 

[Jonathan Edwards]

I am not sure if you prefer not to read my posts @Hilda Bastian, or whether you just find them too close to home to respond, but sadly this looks to me more and more like the blind leading the blind.

What is needed is clear thinking. Clear thinking is easily recognised, as is muddled thinking based on fixed assumptions and vested interest. Discussions here show that remarkably well.

Looking at the recent Newton paper on obesity it is clear that it is muddled thinking - the reasons are given in the thread on it. I had always thought Julia Newton's approach was muddled. She was involved in a trial of rituximab in fatigue in primary biliary cirrhosis. She implied to me that there was no great need for another trial for ME because all fatigue is the same and if the drug worked in PBC it would show it worked on fatigue.

But we know that the PBC, and its fatigue, are mediated by antibodies. Rituximab removes antibodies. We have little evidence for fatigue in ME being mediated by antibodies. So there is no way to extrapolate from any effect of rituximab in PBC to ME. That is a simple clear argument that everyone reading this can understand but it seems that Newton did not understand.

It is now clear that Newton has a presumption that somehow exercise is likely to help ME by reducing obesity when it makes much more sense to think the obesity is a result of the inactivity of ME rather than the other way around.

Pretty much all the Cochrane people I have interacted with seem to think in a similarly muddled way - about exercise and about trials. You might say surely all these experts are not going to be muddled. But these are self-appointed experts with an agenda that from my perspective is not mainstream clear-thinking science but something much more to do with medical politics.

What is the purpose of this re-review? It seems just for Cochrane to be able to feel that it has ticked a box at present left not quite ticked. Yet Cochrane could have ticked the box long ago by following the advice of those consulted by David Tovey - which was to withdraw, presumably because people could see that there was no useable information on which to base an analysis.

Inviting muddled thinkers to re-review seems to me just to perpetuate the problem.

There is nothing difficult about this. People here have no difficulty identifying muddled thinking. Sadly, muddled thinking ais all too common if there are competing interests like research funding.

 

[Wyva]

Well, in connection with the right and wrong patient "activist" behaviour: AIDS activists back then poured fake blood on people, they stormed schools with pamphlets that warned teenagers of the dangers, they put a giant condom on a senator's house, etc etc etc, the list is really long. When have we done anything like these seriously? I think we are actually very tame compared to them.

Now: they have a disease that is quite well manageable and they can live a close to normal life. To me that looks like a success.

Sorry but I don't see the kind of causality Hilda Bastian talks about. Unless it only applies to pwME. (I don't know what kind of wrong attitude a mysterious minorty of us has, but I doubt it is as out of line as the above examples.)

 

[Hoopoe]

We need to understand what is going on internally at Cochrane. If it's a simple misunderstanding of ME (erroneously viewing it as a state of deconditioning) then it should be easy to fix if people are willing to listen.

 

[Caroline Struthers]

I will be looking at the methods and evidence related only to the subject Cochrane is reviewing. I'm not going to check the numbers in their GRADE tables, for example, and I'm not going to try to reproduce their analyses of qualitative or other research. My focus is on the quality of the Cochrane review and supporting stakeholder engagement in the process.

Yes, I know you are only going to be looking at the exercise stuff. Are you going to check the GRADE ratings - ie. to check if you agree that the NICE reviewers' grading of low and very low quality is correct? Are the Cochrane author team going to be looking at the NICE review as well before they write the new protocol?

 

[Hilda Bastian]

I am not sure if you prefer not to read my posts @Hilda Bastian, or whether you just find them too close to home to respond, but sadly this looks to me more and more like the blind leading the blind.

What is needed is clear thinking. Clear thinking is easily recognised, as is muddled thinking based on fixed assumptions and vested interest. Discussions here show that remarkably well.

Looking at the recent Newton paper on obesity it is clear that it is muddled thinking - the reasons are given in the thread on it. I had always thought Julia Newton's approach was muddled. She was involved in a trial of rituximab in fatigue in primary biliary cirrhosis. She implied to me that there was no great need for another trial for ME because all fatigue is the same and if the drug worked in PBC it would show it worked on fatigue.

But we know that the PBC, and its fatigue, are mediated by antibodies. Rituximab removes antibodies. We have little evidence for fatigue in ME being mediated by antibodies. So there is no way to extrapolate from any effect of rituximab in PBC to ME. That is a simple clear argument that everyone reading this can understand but it seems that Newton did not understand.

It is now clear that Newton has a presumption that somehow exercise is likely to help ME by reducing obesity when it makes much more sense to think the obesity is a result of the inactivity of ME rather than the other way around.

Pretty much all the Cochrane people I have interacted with seem to think in a similarly muddled way - about exercise and about trials. You might say surely all these experts are not going to be muddled. But these are self-appointed experts with an agenda that from my perspective is not mainstream clear-thinking science but something much more to do with medical politics.

What is the purpose of this re-review? It seems just for Cochrane to be able to feel that it has ticked a box at present left not quite ticked. Yet Cochrane could have ticked the box long ago by following the advice of those consulted by David Tovey - which was to withdraw, presumably because people could see that there was no useable information on which to base an analysis.

Inviting muddled thinkers to re-review seems to me just to perpetuate the problem.

There is nothing difficult about this. People here have no difficulty identifying muddled thinking. Sadly, muddled thinking ais all too common if there are competing interests like research funding.

I read all the posts, but won't be replying to all of them.

If I was bothered by this, I wouldn't have accepted this role, or the invitation to be a guest here.

 

[Jonathan Edwards]

If I was bothered by this, I wouldn't have accepted this role, or the invitation to be a guest here.

Trouble is, I cannot really make anything of these generalised cryptic responses.
This is a real life situation and although professional colleagues may see me as breaking 'protocol' it seems to me that for the benefit of the patients we need to call a spade a spade.

I cannot understand why anyone could not be bothered by Cochrane being dominated by muddled thinking - it doesn't really make sense. Did you mean 'If I was not bothered by this'?

 

[Hilda Bastian]

Yes, I know you are only going to be looking at the exercise stuff. Are you going to check the GRADE ratings - ie. to check if you agree that the NICE reviewers' grading of low and very low quality is correct? Are the Cochrane author team going to be looking at the NICE review as well before they write the new protocol?

The inadequacy and cherry-picking in attending to other major systematic reviews is one of the criticisms I made of the amended review in October 2019, so, yes.

 

[Hilda Bastian]

Trouble is, I cannot really make anything of these generalised cryptic responses.
This is a real life situation and although professional colleagues may see me as breaking 'protocol' it seems to me that for the benefit of the patients we need to call a spade a spade.

I cannot understand why anyone could not be bothered by Cochrane being dominated by muddled thinking - it doesn't really make sense. Did you mean 'If I was not bothered by this'?

I was replying to this: "I am not sure if you prefer not to read my posts @Hilda Bastian, or whether you just find them too close to home to respond".

 

[Adrian]

I don't believe PWME and their advocates bear joint responsibility for the unpopularity of ME topics, or for the stigma that flows from all being judged by the behavior of a minority. I don't believe criticizing, or being persistent in criticizing, is a problem: not for me, and not for Cochrane either.

I think we need to be clear that there are those in the medical and scientific community who have purposely tires do stigmatize patients to avoid criticism. That is, to my mind, where a lot of the stigma within the academic community comes from. And its not just odd comments it is press campaigns against patients with journalists writing stories with unverified claims of threats. When someone dug into the stories of threats that had become urban myth within the CDC it was found that there was no evidence and just repeated rumour attributed to a comment from a journalist. Within the UK we have seen claims for example from Crawley that she received a letter (looking like the ransom notes you find on TV) which was on the front page of the sunday times magazine a few years earlier - but the journalist had admited to that being a made up image to illustrate the story.

We have people like Wessely saying he feels safer in afghanistan that with ME patients and using his platform in the press to permiate this story. Lets be clear he is not nieve this is a planned attempt and way to chalenge critics. Interestingly when academics talk about styles of criticism they don't cover the denegration of a group to dismiss their criicism.

I tend to think the talk of minority then comes in to say 'a few weirdos' so don't take them seriously as well as making people who carry out these campaigns feel better and 'ok' about their actions. But patients will tell you that when these stories happen it affects how people think about them.

So please stop perpetuating the myth by talking about minority of patients. If there are specific things in advocacy of ME patients that you think are wrong then be specific around what actions you think are bad and why (if it is not obvious).

 

[Sean]

Both the advisory and writing groups in this review need to have a solid understanding of methodology in general, and of the specific issues with its application in this area of medicine, including for safety.

The psychosocial school is demanding, and often simply implementing without any real debate, a gross lowering of methodological standards in both research and clinical practice.

Adequate control (blinding or objective measures), falsification, adequate reporting of harms, distinguishing correlation from causation, even randomisation, are all being argued as either irrelevant or too high a standard to meet for psychosocial studies, and even an inappropriate and counter-productive standard, often under the guise of 'personalised medicine'.

Yet those doing this also demand that the 'evidence' thus produced is given the same epistemological status, and hence practical authority over patients' lives and broader policy advice, as that which does meet the highest of methodological standards.

And they also simultaneously demand the highest possible methodological standard for biomedical studies and claims, and their critics' refutations of their own methodologically inferior psychosocial studies.

This is technically and ethically utterly unacceptable.

I certainly hope the dreadful abusive "few" does not turn out to be people who were angry on twitter or who asked angry questions at conferences or who wrote letters critiquing published articles. Or worse still, those who made FOI requests for data from various trials. Those are all to me, entirely understandable responses to the situation.

Understandable, and legitimate.

 

[Hutan]

@Hilda Bastian, have you had time to read the thread on the 2011 paper by Julia Newton that I tagged you into? The link is here; the thread is not terribly long.
Physical activity intensity but not sedentary activity is reduced in CFS and is associated with autonomic regulation, 2011, Newton et al
It is an old paper, but we see the problems in that paper in newer work by Julia and others. @cassava7 listed the threads on the newer work for you earlier in the thread here:Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Julia is by no means the worst researcher producing work supporting BPS ideas. But I can't see how a researcher who has worked on these issues for years and still produces highly flawed work can be thought competent enough to review the evidence for the exercise review. There is also the conflict of interest, as she would be considering the quality of her own work, and that of others who produce very similar output.

If there is anything in those threads that you don't follow, please ask us. I'm confident that any intelligent person with no BPS career to protect who spends some time reading the papers and the critiques will come away with the understanding that the BPS literature is tremendously problematic. If the exercise review is to have any credibility, you need to understand the issues with the BPS papers.

I think once you understand just how very poor the evidence base is, you may have a better understanding of how manipulative the BPS proponents have been to sell it as valid and suitable for building clinical guidance on. And perhaps you will be able to see the conversations in this thread, and the decisions made about the composition of the writing team in a different light.

 

[cassava7]

I think a range of abilities are needed for specific roles - and there are abilities in common across multiple roles, but some that are specific. For example, you don't need particular values and beliefs to provide a really valuable peer review. What you find out about people in interviews, or from people they've worked with, or asking what their track record is with community groups and so on is irrelevant to peer review if they have the technical knowledge and analytical and written skills you require: but it's highly relevant to people in an author team.

Other than sufficient knowledge of the topic and methodological rigor, no particular set of values and beliefs are necessary to provide valuable peer review, but those held are biases that influence peer review. Dr Newton seems to adhere to the biopsychosocial model of ME/CFS and this is highly likely to influence her peer review.

In your previous post, you said that your choice of Dr Newton was made based on the positive feedback you have received, but you now say that this is irrelevant to peer review. Thus the choice should have fallen back onto assessing her "technical knowledge and analytical and written skills" alone, which should have included taking into account her published work on physical activity/exercise in ME/CFS. Had it been, given its poor methodological quality, it seems unlikely that Dr Newton would have been chosen.

Did this body of work not come to your attention or was it overlooked otherwise?

I did not say that some Cochrane members are reluctant to scientific criticism being shared openly for every to review. I just pointed out that if I wanted to make everybody at Cochrane happy, I wouldn't start off with a long list of unvetted criticisms without responses to them, from the starting point of looking only for critiques not supporters' articles or favorable commentaries about how great it is. I have faced no headwind from Cochrane for doing this, much less opposition.

Edit: PS - Cochrane does promote post-publication peer review, and is better than most journals at publishing it. (Doesn't mean that every editor and author is good at responding to it, though. That will be something the IAG will make recommendations about down the line.)

Cochrane may well publish post-publication peer reviews, but the review not being withdrawn after patients highlighted serious flaws, which has instead led to this drawn-out update process, the dead-end responses from Cochrane to @Caroline Struthers' repeated queries, the change of the retraction policy are all examples that it fails and is unwilling to act upon it.

Thanks. I am reading the thread. If I don't answer a question that's been asked several times, it's not because I've overlooked it: it's because I'm either still thinking about how to answer it or I'm not going to answer it. I did not describe behavior as inappropriate or unethical: I talked about behavior having consequences, and about being effective. I don't think my elaborating on this here is going to be helpful. I'm sure there are people genuinely interested in what behavior I think crosses the line into being counterproductive - everybody has opinions about that, so of course I do. My views are probably very different to those of many people responding in this thread, and I'll be leaving it at that. I'll keep reading though.

To date, in the context of ME/CFS, comments on the "behavior of a minority" have come from proponents of exercise and CBT and their only purpose has been to stifle scientific criticism of their research -- and the harms of these treatments --. This has not been limited to the UK but has happened internationally (e.g. in Norway). While judicial scrutiny found these claims to be baseless, they have been amplified by public relations campaigns, and have been used to criticize the update of the Cochrane review.

Therefore, now that you have put it forward, clarification on your definition of this "behavior" and who is the "minority" is not just helpful, it is essential in keeping this update process fact-based. Unclear and unsubstantiated claims about patients have no place in a process that claims to be consumer-centric.

I believe that your involvement in the process is the closest commitment from Cochrane to openness/outreach that we will get. I am thankful for the work you have put in, but I am disappointed that once again, Cochrane, through your voice, chooses to ignore or dodge evasively specific (and reasonable) questions asked by patients whose lives have been, are and will continue to be impacted by the review. I hope you'll be able to answer these questions in time. The importance of transparency cannot be overstated, but so far, the process is opaquely being conducted behind-the-scenes.

 

[Sean]

Are the prominent medicos who have driven the whole CBT/GET/BPS agenda not also a small, vocal, and unduly influential minority?

 

[Hilda Bastian]

Other than sufficient knowledge of the topic and methodological rigor, no particular set of values and beliefs are necessary to provide valuable peer review, but those held are biases that influence peer review. Dr Newton seems to adhere to the biopsychosocial model of ME/CFS and this is highly likely to influence her peer review.

In your previous post, you said that your choice of Dr Newton was made based on the positive feedback you have received, but you now say that this is irrelevant to peer review. Thus the choice should have fallen back onto assessing her "technical knowledge and analytical and written skills" alone, which should have included taking into account her published work on physical activity/exercise in ME/CFS. Had it been, given its poor methodological quality, it seems unlikely that Dr Newton would have been chosen.

Did this body of work not come to your attention or was it overlooked otherwise?


Cochrane may well publish post-publication peer reviews, but the upholding of the review after patients highlighted serious flaws, which has instead led to this drawn-out update process, the dead-end responses from Cochrane to @Caroline Struthers' repeated queries, the change of the retraction policy are all examples that it fails and is unwilling to act upon it.


To date, in the context of ME/CFS, the only purpose of comments on the "behavior of a minority" has been to stifle scientific criticism of research on exercise therapy and CBT -- and their harms -- by their proponents (not just in the UK, but also notably in Norway). While judicial scrutiny found them to be baseless, they have been amplified by public relations campaigns, and have been used to criticize the update of the Cochrane review.

Therefore, now that you have put it forward, clarification on your definition of this "behavior" and who is the "minority" is not just helpful, it is essential in keeping this update process fact-based. Unclear and unsubstantiated claims about patients have no place in a process that claims to be consumer-centric.

I believe that your involvement in the process is the closest commitment from Cochrane to openness/outreach that we will get. I am thankful for the work you have put in, but I am disappointed that once again, Cochrane, through your voice, chooses to ignore or reply in a very generic manner to specific (and reasonable) questions asked by patients whose lives have been, are and will continue to be impacted by the review. The process remains opaque and is being conducted behind-the-scenes. I hope you'll be able to answer these questions in time -- the importance of transparency cannot be overstated --.

Just to clarify, Julia Newton is a member of the author team, not a peer reviewer. In addition to her clinical and research roles, she is a medical advisor to Action For ME and ME Research UK.

What the report says about this:

Since Cochrane posted in March 2020 that we were considering authors and establishing the IAG, expressions of interest and suggestions about potential contributors have been coming in. We are very grateful to everyone for their interest and willingness to contribute. Everyone has been considered for each of the roles, and will be considered for the range of other activities currently planned for the review – for example, peer review and a discussion group of ME/CFS groups.