Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I'm happy to look into it further, but as the review has not been retracted it seems that Cochrane is standing behind the conclusions.

He is not technically correct here. Cochrane have not retracted because their rules did not allow them to if the authors disagreed. What is happening now is not so much standing behind conclusions as trying to pretend the mess on the floor isn't there.
 
Jonathan Edwards said:
Cochrane have not retracted because their rules did not allow them to if the authors disagreed. What is happening now is not so much standing behind conclusions as trying to pretend the mess on the floor isn't there.
Click to expand...

I so wish I could remember the name of the German theatre company doing the very funny non-verbal show at the Edinburgh Children's Festival, where they made a mess on the floor with paper and then tried to conceal it with more and more paper. I'd like to ask them where they got the idea.

But anyway, who's going to do the review of Cochrane's rules to establish whether or not they ensure the worst science is firmly embedded as mainstream?
 
Kitty said:
I so wish I could remember the name of the German theatre company doing the very funny non-verbal show at the Edinburgh Children's Festival, where they made a mess on the floor with paper and then tried to conceal it with more and more paper. I'd like to ask them where they got the idea.

But anyway, who's going to do the review of Cochrane's rules to establish whether or not they ensure the worst science is firmly embedded as mainstream?
Click to expand...

Well @dave30th certainly enjoyed ripping up a copy or two of PACE...
 
Kitty said:
I so wish I could remember the name of the German theatre company doing the very funny non-verbal show at the Edinburgh Children's Festival, where they made a mess on the floor with paper and then tried to conceal it with more and more paper. I'd like to ask them where they got the idea.

But anyway, who's going to do the review of Cochrane's rules to establish whether or not they ensure the worst science is firmly embedded as mainstream?
Click to expand...

I am planning to write to COPE (of which Cochrane is a member) to complain (https://publicationethics.org/complaints-and-concerns-about-our-members) The new Cochrane withdrawal policy was changed suddenly in August 2019 with no transparent decision-making process behind the change (https://documentation.cochrane.org/display/EPPR/2019/10). I think it was decided in a hurry because of the pressure by me and others to withdraw the CBT and Exercise reviews. Of course I will never be able to prove that.

Now one of the very limited reasons given for withdrawing a review is a "serious error" which means following the conclusions might cause harm to patients. (https://documentation.cochrane.org/display/EPPR/Withdrawing+published+Cochrane+Reviews)

On the one hand this makes it look like Cochrane care about patients. On the other, they rejected my request to withdraw the reviews because they decided there was no specific (ie. statistical, or other technical) error. So the fact that harm may be caused to patients by their publications because they are out of date and/or poorly conducted, or contradict more recent thorough evidence reviews, as done by NICE in this case, is not their concern.

This doesn't seem to be ethical publication practice to me, especially for an charitable organisation whose motto is "Trusted evidence....informed decisions...better health". I am fairly sure COPE will disagree and think it's all absolutely fine, but you never know. Even if they do agree, they can only make recommendations to their members, which their members can ignore. Hey ho. Keeps me off the streets.
 
Caroline Struthers said:
I am planning to write to COPE (of which Cochrane is a member) to complain (https://publicationethics.org/complaints-and-concerns-about-our-members) The new Cochrane withdrawal policy was changed suddenly in August 2019 with no transparent decision-making process behind the change (https://documentation.cochrane.org/display/EPPR/2019/10). I think it was decided in a hurry because of the pressure by me and others to withdraw the CBT and Exercise reviews. Of course I will never be able to prove that.
Click to expand...

Perhaps it wasn't transparent to you because you weren't part of the organization? A Cochrane policy like this one takes a lot of consultation and debate among the 50+ editorial groups and others before a major editorial policy is changed, and they don't tend to be changed at short intervals as a result.

This one was part of the whole process of updating policy, not just the issue of withdrawals. And it took was under discussion for years, as I recall. (As the policy around withdrawals was directly relevant to my work when I was at the National Library of Medicine, and to my PhD studies, I used to check in every now and then to see if it was getting close to being finalized.)

I think, to be honest, the backstory of these reviews and Cochrane isn't that there's a history of them being so sensitive about them that they would do something drastic organization-wide because of them - but more that they weren't a priority, I'm afraid. Most healthcare issues suffer from this, everywhere, not just Cochrane. There are many healthcare issues that attract more attention, powerful stakeholders, and controversy than ME/CFS.
 
Hilda Bastian said:
Perhaps it wasn't transparent to you because you weren't part of the organization? A Cochrane policy like this one takes a lot of consultation and debate among the 50+ editorial groups and others before a major editorial policy is changed, and they don't tend to be changed at short intervals as a result.

This one was part of the whole process of updating policy, not just the issue of withdrawals. And it took was under discussion for years, as I recall. (As the policy around withdrawals was directly relevant to my work when I was at the National Library of Medicine, and to my PhD studies, I used to check in every now and then to see if it was getting close to being finalized.)

I think, to be honest, the backstory of these reviews and Cochrane isn't that there's a history of them being so sensitive about them that they would do something drastic organization-wide because of them - but more that they weren't a priority, I'm afraid. Most healthcare issues suffer from this, everywhere, not just Cochrane. There are many healthcare issues that attract more attention, powerful stakeholders, and controversy than ME/CFS.
Click to expand...
Yes. I understand what you're saying. I guess then I am suggesting the withdrawal policy was possibly finalised in a hurry. I don't think it was a good policy change as it now allows out of date reviews to remain in a trusted database of what is assumed by most to be the best available evidence. Cochrane authors used to make a commitment to update the review every two years, unless that Cochrane policy has also changed. Now if they decide they don't want to update the review, then that's also accepted as ok. There has been no effort from Cochrane, as far as I know, in the case of the CBT review to recruit a new author team. Or to approach Vink, who has already done a review of CBT, to Cochranize his.

There is no information about when a review is being updated because of its outdated methods and assumptions about the condition, or so out of date it shouldn't be used, displayed on the public-facing version of the reviews. This is the version that comes up first in a Google search.
 
Caroline Struthers said:
Yes. I understand what you're saying. I guess then I am suggesting the withdrawal policy was possibly finalised in a hurry.
Click to expand...

I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.
 
Hilda Bastian said:
I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.
Click to expand...

To be honest @Hilda Bastian, for those of us who have got fed up with Cochrane being disingenuous about these reviews there is every reason to think that this decision was disingenuous as well, even if it is a coincidence. If Cochrane wants to restore any credibility for itself it needs to start making valid decisions. One thing they could usefully do is get rid of GRADE and RoB2, which are clearly unfit for purpose. The more I look at this the more the whole system looks to have lost the plot.
 
Hilda Bastian said:
I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.
Click to expand...
David Tovey announced the withdrawal of the exercise review under the old policy and then changed his mind because of pressure from the authors. He said explicitly that the GRADE rating of "moderate" for the quality of the evidence for the fatigue outcome was not defensible.

A month after I and two patients met with Karla to beg her once again to withdraw the review rather than publish the amendment, the policy was changed. Of course it could be coincidental. It doesn't really matter. I still think it was a change in the wrong direction. And not just for people with ME but for everyone.
 
Hilda Bastian said:
I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.
Click to expand...

Cochrane withdrew the Chinese herbal medicine for CFS review in 2018 under the old policy because it was out of date. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006348.pub3/full

When I asked for the CBT for CFS review to be withdrawn for the same reason - it's even more out of date - they said the policy had changed so they wouldn't. https://www.cochranelibrary.com/cds....pub2/detailed-comment/en?messageId=261273171
 
Hilda Bastian said:
I don't see any indication that it was hurried, nor any link with these particular reviews. They didn't need to suddenly change the policy to not withdraw these reviews - they'd been not withdrawing them under the previous policy, too.
Click to expand...

Transparancy is about Cochrane publishing evidence that they went through a proper process (i.e. publishing all the meeting minutes). Otherwise we can have no idea of why they decided to change their policy or when they did it. Given it seems like such a bad policy as it means old stuff stands even if there is contradictory evidence then Cochrane should explain in detail - otherwise it just seems like another reason to not take Cochrane evidence seriously (in that they don't care if it is out of date!).

Cochrane should withdraw the review because it is simply a bad review and likely to harm patients. The methodology is poor and their results on fatigue rely on a questionnaire that isn't worth the paper it is printed on. Given this I don't see how anyone can take any review done by Cochrane seriously as the review and editorial processes are too weak to ensure adaquate quality.
 
Caroline Struthers said:
I am planning to write to COPE (of which Cochrane is a member) to complain (https://publicationethics.org/complaints-and-concerns-about-our-members) The new Cochrane withdrawal policy was changed suddenly in August 2019 with no transparent decision-making process behind the change (https://documentation.cochrane.org/display/EPPR/2019/10). I think it was decided in a hurry because of the pressure by me and others to withdraw the CBT and Exercise reviews. Of course I will never be able to prove that.

Now one of the very limited reasons given for withdrawing a review is a "serious error" which means following the conclusions might cause harm to patients. (https://documentation.cochrane.org/display/EPPR/Withdrawing+published+Cochrane+Reviews)

On the one hand this makes it look like Cochrane care about patients. On the other, they rejected my request to withdraw the reviews because they decided there was no specific (ie. statistical, or other technical) error. So the fact that harm may be caused to patients by their publications because they are out of date and/or poorly conducted, or contradict more recent thorough evidence reviews, as done by NICE in this case, is not their concern.

This doesn't seem to be ethical publication practice to me, especially for an charitable organisation whose motto is "Trusted evidence....informed decisions...better health". I am fairly sure COPE will disagree and think it's all absolutely fine, but you never know. Even if they do agree, they can only make recommendations to their members, which their members can ignore. Hey ho. Keeps me off the streets.
Click to expand...

I have now made a submission to COPE via https://publicationethics.org/facilitation-and-integrity-subcommittee. I have covered the specific issues around Cochrane's refusal to withdraw the two CFS reviews, or to independently review my appeal against that decision. And more generally about having editorial notes such as "do not use for decision-making" on reviews which are not visible from the plain language summary and author conclusion sections of the review which are made accessible on Cochrane's main website. This is the version which pops up first in a Google search. I view this as a clear breach of publication ethics. It will be interesting to see if they agree.
 
Caroline Struthers said:
I have now made a submission to COPE via https://publicationethics.org/facilitation-and-integrity-subcommittee. I have covered the specific issues around Cochrane's refusal to withdraw the two CFS reviews, or to independently review my appeal against that decision. And more generally about having editorial notes such as "do not use for decision-making" on reviews which are not visible from the plain language summary and author conclusion sections of the review which are made accessible on Cochrane's main website. This is the version which pops up first in a Google search. I view this as a clear breach of publication ethics. It will be interesting to see if they agree.
Click to expand...
PS - please don't Tweet about the fact I have made a submission to COPE as one of the condition is that they are left alone to get on with investigating...if they do investigate of course
 
Caroline Struthers said:
Cochrane withdrew the Chinese herbal medicine for CFS review in 2018 under the old policy because it was out of date. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006348.pub3/full
Click to expand...
And they begin in such a nice manner:

Supporting better health for those receiving care is central to Cochrane’s mission to promote evidence‐informed health decision‐making by producing high quality, relevant, accessible systematic reviews and other synthesized research evidence.
Click to expand...
"high quality" should well include also to go through open questions and even contradictions, especially when they have already been discussed.

If not, it must be considered to be of low quality -


Maybe I am naive, but the review is not much better done than by cribbing.
(@Hilda Bastian)
 
Caroline Struthers said:
And more generally about having editorial notes such as "do not use for decision-making" on reviews which are not visible from the plain language summary and author conclusion sections of the review which are made accessible on Cochrane's main website.
Click to expand...

This was the strategy BMJ used to hide for over a year the fact that the Lightning Process study was being examined for serious ethical and methodological violations. They put an editorial note on the paper, but it was on a completely different page that no one would have thought to look at. So no one reading the paper knew there were problems. I complained about this directly to Dr Godlee and she wrote back and said the BMJ's IT systems made it challenging to move the notice to the same page as the article itself. Yeah, right.
 
You must log in or register to reply here.
Back
Top Bottom