Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Valerie Eliot Smith said:
@Caroline Struthers and @Robert 1973

A bit of background stuff: this is from a blog post I wrote two years ago:

"[M]ost public/private institutions are required to have a formal complaints and/or appeals procedure. Unfortunately, these are frequently (and, some might argue, deliberately) very difficult to access and inherently opaque, laborious and time/energy-consuming. However, unless the full internal procedure is followed closely – often involving many stages of escalation – it is inevitable that the complaint/appeal will be rejected."

So the first rule is to follow the internal process (which will always be cumbersome and frustrating by design) to its conclusion. The next stage after that would be the Charity Commission (and I see you've already been in touch with them, Caroline). The requirement there seems to be for the charity itself - not the original complainant - to report a "serious incident" and this is unlikely to meet the threshold for Cochrane to do that, from what I can see.


Once all the internal procedures have been exhausted, the position changes and it is appropriate to give consideration to litigation. Even then, I think that legal action targeted at Cochrane is a very long shot. Hypothetically, this is the next logical step but I have nothing left with which to write more now.
Click to expand...

I am happy to persist with Cochrane's internal procedures as long as it takes. And if eventually they have to report themselves to the charity commission (which they did over the dramatic Peter Gotzsche incident in 2018) that would be good. As you say, it is doubtful this will ever escalate to "serious incident", but you never know.
 
Valerie Eliot Smith said:
Thanks, Caroline. Yes, you're quite right; I transposed the word "serious" from preceding "error" to preceding "harm".

The same sticking-point remains, however, that Cochrane does not accept that the review could result in harm to patients - bizarrely - and therefore there is no serious error to report.

I suspect that the get-out clause they may rely on is this: "Following the conclusions of the review could result in harm to patients or populations of interest (other than known adverse effects)."

This is quite ridiculous, of course, but they're clearly not backing down.

Again, I think that the case for withdrawal will become stronger after the final publication of the NICE guideline in April (assuming there is nothing particularly unexpected in it).
Click to expand...
One of the only scenarios that make sense is that a lot is happening behind the scenes to undo the NICE guidance and Cochrane is taking a back seat to whether it succeeds. It's not as if the evidence grading will change, it is done. So Cochrane is not working from evidence, rather on what can be sold, regardless of evidence. Because it's irrelevant what is in the final guidance, it's the evidence they are supposed to be working from.

If the updated guidance is rejected back to its current version by political influence, Cochrane will have the cover to do nothing, regardless of what the evidence says or how poor it is. Which has been the formula so far, shown here again by simply refusing to grade the evidence for the CBT review and just slapping a weak label that won't appear in most published and re-hosted versions.

It frankly appears to be the formula that works for us: demand neutral evidence reviews. This is what happened with the IOM report. And again with NICE. And Cochrane is showing the way in how to deal with it: refuse to do an evaluation based on their own criteria. Because frankly they know it doesn't stand up to scrutiny or even their own criteria, which dismiss non-controlled open label trials with subjective outcomes as worthless, which make up all the evidence base here.

Neutral evidence reviews are the kryptonite here. We should push for more, to have institutions do real evaluations. It works. We can even see with recent updates in countries like Belgium and Finland. They are bad because they did not have to do a full evaluation, they simply cherry-picked what they wanted with a few concessions. With a full evaluation of the evidence, it would not have worked.
 
Last edited:
Here's the response from Cochrane. Unsurprising. I responded asking for a wee bit more detail on the criteria the editors used, which I don't suppose I will get. Next stop complaint I guess. And will probably contact the Charity Commission again. Does the fun ever stop?!

Dear Caroline,

The Editors have reviewed the information within the NICE guideline and the associated evidence review, and they have agreed that the criteria for a serious error has not been met. Therefore, the reviews on exercise for ME/CFS and cognitive behavioural therapy for ME/CFS will remain on the Cochrane Library, along with editorial notes explaining the status of each review. The investigation has now been concluded.

With best regards, Rachel

Rachel Marshall
Senior Editorial Officer
Editorial & Methods Department | Cochrane Central Executive
 
Caroline Struthers said:
Here's the response from Cochrane. Unsurprising. I responded asking for a wee bit more detail on the criteria the editors used, which I don't suppose I will get. Next stop complaint I guess. And will probably contact the Charity Commission again. Does the fun ever stop?!

Dear Caroline,

The Editors have reviewed the information within the NICE guideline and the associated evidence review, and they have agreed that the criteria for a serious error has not been met. Therefore, the reviews on exercise for ME/CFS and cognitive behavioural therapy for ME/CFS will remain on the Cochrane Library, along with editorial notes explaining the status of each review. The investigation has now been concluded.

With best regards, Rachel

Rachel Marshall
Senior Editorial Officer
Editorial & Methods Department | Cochrane Central Executive
Click to expand...
So... publishing a review with very low quality evidence is not a serious error. Interesting perspective. In the sense of "may you live in interesting times" but still.

I would be interested in the precise meaning of "editors have reviewed". How, exactly? Experience tells us that when such a thing happens in secrecy, it doesn't really amount to much.

This is the "gold standard" of medical evidence. Really says it all. Decades of stagnation in improving clinical outcomes and almost no one but the people who are forced to live with it is bothered by it. I don't get it. I guess that's what happens when a profession operates without accountability. Even in the best circumstances everything becomes self-serving.
 
Valerie Eliot Smith said:
As far as I can tell, @Caroline Struthers is doing everything possible so far within the limitations of the complaints procedure. Efforts to jump ahead will only be knocked back. When the Charity Commission suggested seeking legal advice, I'm not sure what they would have had in mind.
Click to expand...
Valerie Eliot Smith said:
The same sticking-point remains, however, that Cochrane does not accept that the review could result in harm to patients - bizarrely - and therefore there is no serious error to report.
Click to expand...

I think I may have raised the idea of legal action not because I think it is appropriate at this stage but because I think Cochrane don't take their responsibilities seriously and I suspect some of that is that they feel they will not be held accountable for the way they act or for putting their beliefs and prejudice before the evidence.

So I wonder if there is a question that if say someone was harmed by GET could they sue those advising them and those such as Cochrane who are providing bad information. Or would a doctor relying on cochrane sue them if they were the subject of legal action. Now NICE are changing their guidance and Cochrane are basically the only advice body supporting GET then I assume it makes their position weaker?
 
Cochrane UK is funded by the National Institute for Health Research (NIHR). The NIHR is funded by the Department of Health to improve the health and wealth of the nation through research.
Click to expand...

https://uk.cochrane.org/our-funders-and-partners

Funny how ordinary people are held accountable for their deeds all the time yet so called organisations tasked with something as important as this just "make stuff up and carry on".

And, it would seem, they get pretty snippy at being asked to account.
 
Cochrane UK is funded by the National Institute for Health Research (NIHR). The NIHR is funded by the Department of Health to improve the health and wealth of the nation through research.
Click to expand...

I think I recall seeing that there is a Cochrane Australia that receives funding from Australian taxpayers (although I could be wrong). Is it worth writing to MPs and Ministers and asking them why they are funding an organisation which has no regard for rigorous science and for the safety of marginalised patients? (and copying all correspondence to Cochrane) Other Cochrane funders could similarly be targeted.

Given that they don't respond to reasoned argument, perhaps it's now time to exert political pressure on Cochrane.
 
Last edited:
Caroline Struthers said:
Nope. Hilda said a while back she was behind due to COVID and other commitments but working on it and going to announce the members of the IAG soon. She was also working on the collection of critiques etc.
Click to expand...
Yes, I got horribly behind on this, and am working on ideas to try to make up time. Critique process nearly there, though, and progress report coming along too.
 
rvallee said:
Is it? Only person we ever heard is involved is Bastian. Who are these people and why was there never a public announcement or something to staff that group? Or is this false information?
Click to expand...
It's not their fault: it's on me - too ambitious about all the things I wanted done in the middle of a pandemic before making an announcement. High priority now.
 
Caroline Struthers said:
Here's the response from Cochrane. Unsurprising. I responded asking for a wee bit more detail on the criteria the editors used, which I don't suppose I will get. Next stop complaint I guess. And will probably contact the Charity Commission again. Does the fun ever stop?!

Dear Caroline,

The Editors have reviewed the information within the NICE guideline and the associated evidence review, and they have agreed that the criteria for a serious error has not been met. Therefore, the reviews on exercise for ME/CFS and cognitive behavioural therapy for ME/CFS will remain on the Cochrane Library, along with editorial notes explaining the status of each review. The investigation has now been concluded.

With best regards, Rachel

Rachel Marshall
Senior Editorial Officer
Editorial & Methods Department | Cochrane Central Executive
Click to expand...

Hello again
This is what I replied last night - also copied the whole lot to Hilda so she knows the latest

Dear Rachel

Please could you confirm the criteria the editors used to come to this conclusion?

I have been clear about why these reviews should be withdrawn, which is the continued risk of harm to patients. As a charity whose beneficiaries are patients and the public, I expect Cochrane to give me more detail on how the conclusion that keeping these reviews on the library could not cause harm to patients was reached.

With best wishes

Caroline
 
Hilda Bastian said:
It's not their fault: it's on me - too ambitious about all the things I wanted done in the middle of a pandemic before making an announcement. High priority now.
Click to expand...

Its easy for someone who is busy to take the blame but as an organization they should have an eye on progress and if things are delayed then they should be looking at what they can do and additional resourcing.
 
Adrian said:
I think I may have raised the idea of legal action not because I think it is appropriate at this stage but because I think Cochrane don't take their responsibilities seriously and I suspect some of that is that they feel they will not be held accountable for the way they act or for putting their beliefs and prejudice before the evidence.

So I wonder if there is a question that if say someone was harmed by GET could they sue those advising them and those such as Cochrane who are providing bad information. Or would a doctor relying on cochrane sue them if they were the subject of legal action. Now NICE are changing their guidance and Cochrane are basically the only advice body supporting GET then I assume it makes their position weaker?
Click to expand...

Thanks, Adrian.

A brief response: this is the process for considering the merits of litigation:
  1. Identify potential claimant(s)
  2. Identify appropriate defendant(s)
  3. Examine the cause(s) of action which might be available
  4. Assess the evidence
  5. Consider the likely defences
  6. Form an initial opinion on the merits of the case
None of this can happen without the benefit of a full written case history.

In this case, the first three stages of that process are not straightforward and present a wide range of challenges. It would take an enormous amount of time and energy to go through it in detail. I'm completely wiped out by the Paul Garner saga of last week so I'm not up to it at the moment.

(In any event, anyone who was seriously considering pursuing any form of litigation in relation to this matter would be well-advised to get an expert opinion from a currently practising specialist lawyer.)

In order to form a proper opinion, it is necessary to approach the matter objectively and not from any partisan viewpoint. A great deal of responsible legal practice consists of advising clients of things they don't want to hear. That can be very painful - but it's the only right thing to do. I'm not saying that this would go nowhere - I don't know yet - but there would be many obstacles to overcome on the way.

OK - I've now used the word "thing/s" twice in one sentence. I know that means it's time to stop talking......!

[ETA: I've adjusted a couple of words in this post. It doesn't change anything overall but, as so often happens, looking back at it afterwards, it felt as though the wording could have been slightly better - eg. "not straightforward" was previously "highly complex".]
 
Last edited:
Hilda Bastian said:
It's not their fault: it's on me - too ambitious about all the things I wanted done in the middle of a pandemic before making an announcement. High priority now.
Click to expand...
Good to hear and understandable, especially given how important clinical guidance is during a pandemic and how much work that must involve for you. But at least it ties in nicely given the massive overlap between ME and Long Covid.

Although Cochrane's behavior on the existing reviews remains baffling, especially with the NICE evidence review being completed.
 
Andy said:
Leading evidence-based group blames pandemic for 9-month delay pulling flawed cancer review
https://retractionwatch.com/2021/02...r-9-month-delay-pulling-flawed-cancer-review/
Click to expand...

Cochrane draws a distinction between “withdrawal” and “retraction,” as we’ve noted in a previous Retraction Watch post. Cochrane Library reviews are updated frequently to account for new evidence that weighs for, or against, a specific medical treatment, but can be withdrawn as they become outdated.
Click to expand...

Really? That's not what Cochrane keep saying with regard to the CBT and GET reviews.
 
I think it is important to point out comments like the one discussed here
https://www.s4me.info/threads/nice-...-10th-november-2020.17687/page-27#post-323599
on the NICE guidelines use a very flawed cochrane review to try to discredit the NICE review. I can't imagine anyone would take it seriously but if they did it could be very harmful to many patients for a very long time.

To take things from @Kalliope's post

https://oslo-universitetssykehus.no...te-for-cfsme/Documents/NICEcomments-MECFS.pdf

The committee has produced consultation documents consisting of 2634 pages. In our opinion, the committee has done a biased review of the evidence, and a biased assessment of patient. The committee has discarded Cochrane reviews on exercise and CBT, and has suggested radical changes in recommendations compared with previous NICE guidance, without a balanced review of why. Lillebeth Larun and other colleagues at NIPH revised their Cochrane review on exercise for patients with CFS/ME based on critical comments in 2019, in dialogue with Cochrane chief editors.

The primary studies on the different interventions are presented in tables over hundreds of pages, with standardized assessments regarding downgrading for indirectness, risk of bias etc.
The evidence is downgraded for indirectness when inappropriate diagnostic criteria not including PEM are used, without providing evidence to support the hypothesis that effect should be mediated by type of criteria (on the contrary there is evidence that the effect is similar across different criteria). It is strange to see how evidence from well conducted trials is reduced whereas undue weight is given to evidence from qualitative studies (even for questions on effectiveness/harm).
Click to expand...
 
You must log in or register to reply here.
Back
Top Bottom