Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[rvallee]

Attached is recent correspondence between me and Hilda, initiated by me after the post-COVID fatigue advice leaflets published and then withdrawn by NHS Trusts, notably Oxford Health.

Thank you for this. It's especially problematic that the reliability of the review was confirmed as inadequate with no relevant warnings while it is still being cited in clinical practice without those issues being acknowledged, and not in a small way but by NHS trusts themselves.

Frankly that leaflet makes the strongest point, among many others, for the failure of doing an update while not notifying that it's because the review is flawed beyond repair. Really trying to eat their cake and having it too. Which is systemic in BPS research. PACE is the definitive trial that shows GET is "effective" but no one does PACE-style GET anymore so don't worry about it and also other experiments have shown the same (null results but whatever) even though they have the same flaws or worse. There is no self-consistency within any single part of this, and even less so between the parts.

There should be prominent warnings over not only the review page on the Cochrane library but also for every copy used elsewhere. This is fundamental work to a publishing company and although I expect BMJ-type of nonsensical response ("our CMS does not allow us, a publishing company, to do basic publishing things") this is fundamental to their entire business model and frankly arguing otherwise is insulting.

I think Bastian is in a tough spot here where she has no influence over those things but can't admit to the privately-discussed justifications, since they are indefensible. But that makes the whole experiment especially difficult as it gives us every reason to doubt Cochrane's intentions here. Bastian has been open so far but Cochrane continues to be as opaque and unresponsive as ever.

Maybe it's because they still don't believe that our concerns are over substance and legitimate? Too much concern for their colleagues' ego?

 

[Caroline Struthers]

Thank you for this. It's especially problematic that the reliability of the review was confirmed as inadequate with no relevant warnings while it is still being cited in clinical practice without those issues being acknowledged, and not in a small way but by NHS trusts themselves.

Frankly that leaflet makes the strongest point, among many others, for the failure of doing an update while not notifying that it's because the review is flawed beyond repair. Really trying to eat their cake and having it too. Which is systemic in BPS research. PACE is the definitive trial that shows GET is "effective" but no one does PACE-style GET anymore so don't worry about it and also other experiments have shown the same (null results but whatever) even though they have the same flaws or worse. There is no self-consistency within any single part of this, and even less so between the parts.

There should be prominent warnings over not only the review page on the Cochrane library but also for every copy used elsewhere. This is fundamental work to a publishing company and although I expect BMJ-type of nonsensical response ("our CMS does not allow us, a publishing company, to do basic publishing things") this is fundamental to their entire business model and frankly arguing otherwise is insulting.

I think Bastian is in a tough spot here where she has no influence over those things but can't admit to the privately-discussed justifications, since they are indefensible. But that makes the whole experiment especially difficult as it gives us every reason to doubt Cochrane's intentions here. Bastian has been open so far but Cochrane continues to be as opaque and unresponsive as ever.

Maybe it's because they still don't believe that our concerns are over substance and legitimate? Too much concern for their colleagues' ego?

I know that Hilda attended at least one meeting with Cochrane about the review in the time after I met Karla in July 2019 and when the amended review was published in October. But of course that doesn't mean she was involved in the decision not to withdraw the review and not to publicize the changed conclusions of the amended review.

 

[Caroline Struthers]

Attached is recent correspondence between me and Hilda, initiated by me after the post-COVID fatigue advice leaflets published and then withdrawn by NHS Trusts, notably Oxford Health.

Hilda not hopeful Cochrane will spend time defending the indefensible - which admittedly would take an awful lot of time. It would be better not to have done the indefensible in the first place, or apologize for doing it.

17 May 2020

Dear Caroline

I wouldn't expect effort is going to be put into further statements about past decisions, given that a new process is underway and that process itself requires time - especially not considering how much people have on their plates with the pandemic and evidence responses to it on top of everything else. But of course I will pass it on in case I'm wrong.

Best wishes

Hilda

 

[rvallee]

Hilda not hopeful Cochrane will spend time defending the indefensible - which admittedly would take an awful lot of time. It would be better not to have done the indefensible in the first place, or apologize for doing it.

17 May 2020

Dear Caroline

I wouldn't expect effort is going to be put into further statements about past decisions, given that a new process is underway and that process itself requires time - especially not considering how much people have on their plates with the pandemic and evidence responses to it on top of everything else. But of course I will pass it on in case I'm wrong.

Best wishes

Hilda

Those past decisions are the only ones that affect the present and future, until other decisions supersede them anyway. They are the very reason this is happening in the first place.

I'm not surprised that Cochrane hopes to hinge everything on future decisions but this is not compatible with past decisions continuing to affect the present and future of millions of people subjected to medical malpractice. A choice that Cochrane and only Cochrane make as an institution, now and in the past. Decisions made strictly for the benefit of people with no stake or relevance in the matter.

Pretty much by the laws of the universe only past decisions are relevant for the present, there is no such thing as a future decision that affects the present so all the relevant decisions are always in the past. This is some Sharpe-level attempt at philosophising a practical problem with real-life consequences. If that's the hope then we have some very irreconcilable differences between saving the egos of incompetent charlatans vs. millions of lives.

The thing about being a professional is that you are accountable for decisions made, past present and future. That's the cost of directly influencing the lives of complete strangers. Trying to skirt around accountability while remaining in complete control of decision-making is exactly why past decisions were so thoroughly disastrous. Cochrane's decision-making appears to follow the same patterns that created the current disaster. It certainly does not help that the process is highly secretive, another feature that lead to poor decision-making in the past.

 

[Barry]

Hilda not hopeful Cochrane will spend time defending the indefensible - which admittedly would take an awful lot of time. It would be better not to have done the indefensible in the first place, or apologize for doing it.

17 May 2020

Dear Caroline

I wouldn't expect effort is going to be put into further statements about past decisions, given that a new process is underway and that process itself requires time - especially not considering how much people have on their plates with the pandemic and evidence responses to it on top of everything else. But of course I will pass it on in case I'm wrong.

Best wishes

Hilda

[my bold]

Trouble is of course those past decisions continue to have ongoing major adverse effects on health policy, with no end to that in sight.

 

[Andy]

[Asking generally] Given that the CFQ critique is now being included in this list, would it be useful to submit our PACE briefing document?

 

[MSEsperanza]

https://twitter.com/hildabast/status/1261207728929689600

 

[MSEsperanza]

There are too many good reasons to list them here why some of us have lost any hope that Cochrane could achieve anything good for people with ME.

So thank you all who, despite these reasons, and thank you all who, despite struggling with the additional burden that the pandemic means for people with a severe chronic illnees, keep their comments going.

Also many thanks to Hilda Bastian for answering Caroline's e-mails as well as getting back to forum members via Twitter, even to those who aren't on Twitter themselves.

I hope I will be able to check if I'd like to add anything to the 'commentaries and critiques' list , but so far haven't been able to read all the comments.

Also, couldn't find a link to the updated/ most recent version of Hilda Bastian's list.

[Asking generally] Given that the CFQ critique is now being included in this list, would it be useful to submit our PACE briefing document?

It's the biggest study that's part of the review and it displays some general issues with execise studies for ME that I think still haven't been akcnowledged by the reviewers or Cochrane yet.

Also I think most exercise studies done after the PACE trial reference to it mostly without taking the criticism into account.

So I think it's relevant and worth a try.

 

[MSEsperanza]

Attached is recent correspondence between me and Hilda, initiated by me after the post-COVID fatigue advice leaflets published and then withdrawn by NHS Trusts, notably Oxford Health.

Thank you for your persistence in pursuing this and all kinds of related issues, Caroline.

I would also like an explanation for the decision Karla took not to withdraw the review when the previous Editor-in-Chief said in the May 2019 correspondence with Fretheim “it is my judgement that my successors are unlikely to be willing to publish a future version of this review”.

I think it could be a bit misleading to quote that statement from the previous Editor-in-Chief. Apparently, he accepted some amendments after the review authors had repeatedly protested and eventually did his best to find a compromise with them.

So whereas I agree that it's sad that the new Editor-in-Chief decided against withdrawing and even against adding a clear and immediately visible notice about the extremely limited validity to the current version of the review, I don't think that this snippet of what her predecessor said in May 2019 is of relevance in this particular regard.

I also don't understand why Hilda Bastian, if I understood correctly, dismisses your criticism and suggestions about a clear and visible addition to the review.

The current addition is too easy to ignore both content-wise and with regard to its visibility. You need to click on "What's new" to see it.

However, the authors' conclusion is accessible without any notice of concern, and the discussion even states:

"This update provides valuable additional information when compared with the original review, and results reported in the original review are largely confirmed in this update. Moreover, the results reported here correspond well with those of other systematic reviews (Bagnall 2002; Larun 2011; Prins 2006) and with existing guidelines (NICE 2007)." [emphasis added]

https://www.cochranelibrary.com/cds...is|syndrome|fatigu|withdrawn|exercise|chronic

The thing about being a professional is that you are accountable for decisions made, past present and future. That's the cost of directly influencing the lives of complete strangers. Trying to skirt around accountability while remaining in complete control of decision-making is exactly why past decisions were so thoroughly disastrous.

Trouble is of course those past decisions continue to have ongoing major adverse effects on health policy, with no end to that in sight.

I find it hard to have confidence in the new update process if this isn't taken into account by the lead for the IAG, Hilda Bastian, even though she seems to take the criticism of the review seriously.

Did anyone make her aware of this thread?

https://www.s4me.info/threads/the-influence-of-the-cochrane-review-on-get.11768/

These are only the guidelines; there are many more references in papers and books.

We patients shouldn't be the ones who have to keep track.

In my view, there now is a chance for Cochrane to finally take people with ME, their and others' well-founded criticism seriously. Cochrane could:

1) Withdraw the review or at least put a clear and immediately visible addition on every version of the review that the current version both needs an update and doesn't justify any claims that exercise is safe or helpful for people with ME/CFS.

2) Track the references in papers, books or guidelines to all versions of the exercise review and inform the corresponding journals, editors and health authorities about 1).

Edited for clarity.

 

[Amw66]

Thank you for your persistence in pursuing this and all kinds of related issues, Caroline.



I think it could be a bit misleading to quote that statement from the previous Editor-in-Chief. Apparently, he accepted some amendments after the review authors had repeatedly protested and eventually did his best to find a compromise with them.

So whereas I agree that it's sad that the new Editor-in-Chief decided against withdrawing and even against adding a clear and immediately visible notice about the extremely limited validity to the current version of the review, I don't think that this snippet of what her predecessor said in May 2019 is of relevance in this particular regard.

I also don't understand why Hilda Bastian, if I understood correctly, dismisses your criticism and suggestions about a clear and visible addition to the review.

The current addition is too easy to ignore both content-wise and with regard to its visibility. You need to click on "What's new" to see it.

However, the author's conclusion is accessible without any notice of concern, and the discussion even states:

"This update provides valuable additional information when compared with the original review, and results reported in the original review are largely confirmed in this update. Moreover, the results reported here correspond well with those of other systematic reviews (Bagnall 2002; Larun 2011; Prins 2006) and with existing guidelines (NICE 2007)." [emphasis added]





I find it hard to have confidence in the new update process if this isn't taken into account by the lead for the IAG, Hilda Bastian, even though she seems to take the criticism of the review seriously.

Did anyone make her aware of this thread?

https://www.s4me.info/threads/the-influence-of-the-cochrane-review-on-get.11768/

These are only the guidelines; there are many more references in papers and books.

We patients shouldn't be the ones who have to keep track.

In my view, there now is a chance for Cochrane to finally take people with ME, their and others' well-founded criticism seriously if they:

1) Withdraw the review or at least put a clear and immediately visible addition on every version of the review that the current version both needs an update and doesn't justify any claims that exercise is safe or helpful for people with ME/CFS.

2) Track the references to all versions of the exercise review and inform journals, editors and health authorities that published papers, books or guidelines with references to any version of the Cochrane review about 1).

This, x10

 

[Caroline Struthers]

Thank you for your persistence in pursuing this and all kinds of related issues, Caroline.



I think it could be a bit misleading to quote that statement from the previous Editor-in-Chief. Apparently, he accepted some amendments after the review authors had repeatedly protested and eventually did his best to find a compromise with them.

So whereas I agree that it's sad that the new Editor-in-Chief decided against withdrawing and even against adding a clear and immediately visible notice about the extremely limited validity to the current version of the review, I don't think that this snippet of what her predecessor said in May 2019 is of relevance in this particular regard.

I also don't understand why Hilda Bastian, if I understood correctly, dismisses your criticism and suggestions about a clear and visible addition to the review.

The current addition is too easy to ignore both content-wise and with regard to its visibility. You need to click on "What's new" to see it.

However, the authors' conclusion is accessible without any notice of concern, and the discussion even states:

"This update provides valuable additional information when compared with the original review, and results reported in the original review are largely confirmed in this update. Moreover, the results reported here correspond well with those of other systematic reviews (Bagnall 2002; Larun 2011; Prins 2006) and with existing guidelines (NICE 2007)." [emphasis added]

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub8/full?highlightAbstract=syndrom|fatigue|exercis|syndrome|fatigu|withdrawn|exercise|chronic





I find it hard to have confidence in the new update process if this isn't taken into account by the lead for the IAG, Hilda Bastian, even though she seems to take the criticism of the review seriously.

Did anyone make her aware of this thread?

https://www.s4me.info/threads/the-influence-of-the-cochrane-review-on-get.11768/

These are only the guidelines; there are many more references in papers and books.

We patients shouldn't be the ones who have to keep track.

In my view, there now is a chance for Cochrane to finally take people with ME, their and others' well-founded criticism seriously. Cochrane could:

1) Withdraw the review or at least put a clear and immediately visible addition on every version of the review that the current version both needs an update and doesn't justify any claims that exercise is safe or helpful for people with ME/CFS.

2) Track the references in papers, books or guidelines to all versions of the exercise review and inform the corresponding journals, editors and health authorities about 1).

Edited for clarity.

Thanks so much for your feedback - I take the point about the relevance of quoting the previous Editor-in-Chief - absolutely. I am trying to get hold of the correspondence between Cochrane and The Institute of Public Health after 1 June, but I am not sure I will get anywhere via https://einnsyn.no. I have written to them anyway. I can't submit an Freedom of Information request to Cochrane for the correspondence because they are a charity, not a "public authority". There are moves to extend the FOI act to include charities, but it is not law yet, and I am not sure it ever will be. So Cochrane can continue to be as cloak and dagger as they like!

I have attached a very long and tortuous attempted correspondence with Karla, about this issue dating back to July 2019. I plan to post it on my blog. Unless you think it's just too long, boring and pointless to do so....

 

[Trish]

I have attached a very long and tortuous attempted correspondence with Karla, about this issue dating back to July 2019. I plan to post it on my blog. Unless you think it's just too long, boring and pointless to do so....

Thanks for your efforts. Good to have this on record.

 

[rvallee]

Thanks for your efforts. Good to have this on record.

Radical transparency is the way. Put it all out there, lift the veil of secrecy. That's what's so powerful about Tuller's work, it's like open source journalism, it's accountability by sunlight, the best and strongest disinfectant (not actually true but whatever bear with me here).

One day PhDs will be written on this, likely books and documentaries. Let's make it easy for people to find out what happened, archived for posterity. I'm already thinking ahead on an archiving effort once we pass the damn breakthrough and I can regain some brain function. A big well-categorized database of all the malfeasance, the evidence, the begging and pleading, the expert testimony, the thousands and thousands of complaints, the newspaper reports and just the overwhelming evidence of a human rights disaster in plain sight. And, of course, the discussions from people involved showing they are aware of all those problems yet choose to do nothing at all, in fact even enable the cover-up.

The facts speak for themselves, they just need a massive boom box.

 

[Caroline Struthers]

Radical transparency is the way. Put it all out there, lift the veil of secrecy. That's what's so powerful about Tuller's work, it's like open source journalism, it's accountability by sunlight, the best and strongest disinfectant (not actually true but whatever bear with me here).

One day PhDs will be written on this, likely books and documentaries. Let's make it easy for people to find out what happened, archived for posterity. I'm already thinking ahead on an archiving effort once we pass the damn breakthrough and I can regain some brain function. A big well-categorized database of all the malfeasance, the evidence, the begging and pleading, the expert testimony, the thousands and thousands of complaints, the newspaper reports and just the overwhelming evidence of a human rights disaster in plain sight. And, of course, the discussions from people involved showing they are aware of all those problems yet choose to do nothing at all, in fact even enable the cover-up.

The facts speak for themselves, they just need a massive boom box.

I think it's going to have to be Hollywood blockbuster, no? "Cochrane...we have a problem...."

 

[Caroline Struthers]

I have now posted the correspondence with Cochrane on my blog. Hilda is not going to list it because it's not what this particular list is about. . Her reply to me doesn't really make sense although maybe there's a word missing...anyway, do retweet if you wish https://healthycontrolblog.wordpres...rcise-for-cfs-review-july-2019-february-2020/

 

[MSEsperanza]

https://twitter.com/hildabast/status/1262140974995628032

To Hilda Bastian:

I'm not going to try to develop a list of the huge literature on aspects of Cochrane methodology generally, as I'm concentrating on the exercise/ME/CFS review.

Hm, isn't the exercise review so bad in part because the Cochrane methodology has its shortcomings that are particularly relevant for assessing the quality of evidence of therapist-delivered treatments, e.g. exercise ?

https://twitter.com/hildabast/status/1262152707093098496

After all this is analyzed, there will still of course be plenty of opportunities for people to raise any issues with the review. A reminder that people can send their thoughts to cochrane.IAG[at]gmail[dot]com

Those who now submit suggestions to your list might not be able to engage with the IAG later, so how about creating an additional google-doc now where you collect submissions relevant to the review process in general, e.g criticism of trial / review methodology, in particular regarding Cochrane tools or handbooks?

And perhaps even a third document that lists all other submissions you don't find relevant at the moment, but you or others in the IAG might find relevant later?

(I don't feel up to e-mailing to the IAG due to diverse reasons, one of them is that living with ME means having to pace all activity, also typing and cognitive activity, very carefully. In addition, I frequently have flare-ups that don't allow me to pursue any ME advocacy work for a couple of weeks or months, so I just leave my suggestions on the forum.)

Edited for clarity. Apologies for repeated editing.

 

[Caroline Struthers]

https://twitter.com/hildabast/status/1262140974995628032

To Hilda Bastian:



Hm, isn't the exercise review so bad in part because the Cochrane methodology has its shortcomings that are particularly relevant for assessing the quality of evidence of therapist-delivered treatments, e.g. exercise ?

https://twitter.com/hildabast/status/1262152707093098496

Those who now submit suggestions to your list might not be able to engage with the IAG later, so how about creating an additional google-doc now where you collect submissions relevant to the review process in general, e.g criticism of trial / review methodology, in particular regarding Cochrane tools or handbooks?

And perhaps even a third document that lists all other submissions you don't find relevant at the moment, but you or others in the IAG might find relevant later?

(I don't feel up to e-mailing to the IAG for diverse reasons, one of them is that living with ME means having to pace all activity, also typing and cognitive activity, very carefully. In addition, I frequently have flare-ups that don't allow me to pursue any ME advocacy work for a couple of weeks or months, so I just leave my suggestions on the forum.)

Edited for clarity. Apologies for repeated editing.

Hi there - did you send this to Hilda? I can forward it to her if you wish? Or will she see it on here...?

 

[MSEsperanza]

Hi there - did you send this to Hilda? I can forward it to her if you wish?

Thank you once again Caroline -- I didn't send it and am not sure Hilda will spot everything posted here, so yes, would be great if you could do this.

 

[Sly Saint]

this Cochrane review (2015) was in a tweet relating to 'rehabilitation of post-covid patients'; thought it appropriate here
Exercise rehabilitation following intensive care unit discharge for recovery from critical illness

All six studies assessed functional exercise capacity, although we noted wide variability in the nature of interventions, outcome measures and associated metrics and data reporting. Overall quality of the evidence was very low.

Similar variability was evident with regard to findings for the primary outcome of health‐related quality of life. Only two studies evaluated this outcome. Individually, neither study reported differences between intervention and control groups for health‐related quality of life due to the intervention. Overall quality of the evidence was very low.

Authors' conclusions
At this time, we are unable to determine an overall effect on functional exercise capacity, or on health‐related quality of life, of an exercise‐based intervention initiated after ICU discharge for survivors of critical illness. Meta‐analysis of findings was not appropriate because the number of studies and the quantity of data were insufficient. Individual study findings were inconsistent. Some studies reported a beneficial effect of the intervention on functional exercise capacity, and others did not. No effect on health‐related quality of life was reported. Methodological rigour was lacking across several domains, influencing the quality of the evidence. Wide variability was noted in the characteristics of interventions, outcome measures and associated metrics and data reporting.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD008632.pub2/full

 

[rvallee]

Hm, isn't the exercise review so bad in part because the Cochrane methodology has its shortcomings that are particularly relevant for assessing the quality of evidence of therapist-delivered treatments, e.g. exercise ?

100%

A broken process produces broken results. The process is broken and cannot be relied upon to produce better results until the process itself is corrected. The process allowed an irrelevant and misleading question to be given way more attention that is deserved.

There is simply no basis to ask whether CBT or GET or astrology or urine soaking therapy or any other random pseudoscience belief system out there to be evaluated when nothing has ever indicated that they are relevant. Charlatans choosing to waste their career on nonsense is not a reason to evaluate the product of their beliefs. If Cochrane can't ask the right questions then they can just go away and leave things to people who know what they're doing. This is not normal and has to be fixed moving forward.

There is no progress without acknowledging that this catastrophic failure was a product of a broken decision-making process. If Cochrane expect that to happen they will be very disappointed because it's simply not going to work. When you get caught robbing a bank you can't just return the money and call it even. Life doesn't work like that. Get you shit together Cochrane and start taking your job seriously.