Cochrane ME/CFS GET review temporarily withdrawn

Thanks. Do you know if it is normal practice for Cochrane to choose someone who isn't a medical doctor and who has no or little previous experience in clinical trials to lead such a review?

Edit: And surely anyone leading such a review should have a considerable knowledge of statistics?

 

I'm shocked when I didn't think I could be shocked anymore. I'm now led to think that Cochrane hasn't exercise proper oversight with regard to ME/CFS.

edit - for tone. sorry.

 

My understanding is that Cochrane does not so much choose someone as someone offers and Cochrane approves. I wrote a review on treatment of heel pain. The review team was led by a podiatrist who had worked with me. There was only one reasonably well structured trial - which we had done. The podiatrist had not done other trails but was highly intelligent and critical.

So it is quite normal for people to do reviews of their own work and I do not think there is any requirement for leads to be doctors.

One of the things I think recent events of various sorts has brought to light is that although Cochrane by and large seems to be a well meaning project with some quality control there is absolutely no guarantee that any articular review is any good. Or that doing reviews like this is a particularly useful thing anyway. By and large you only need systematic reviews in areas where things are not very clear. And it might be best just to admit that things are not very clear and do some definitive studies.

 

Thanks for the explanation.

You said that "....Cochrane does not so much choose someone as someone offers and Cochrane approves " does that not leave things very open to manipulation especially in an area of contention?

I find that quite shocking actually, surely any supposed gold standard review, whatever the area of medicine, should be seen to be scrupulously beyond such possibilities of manipulation?

 

It's apparent that I've misunderstood how Cochrane works.

This.

 

Er, yes. Everyone should have remembered that all along. Maybe some people did remember it and made use of it.

Of course if Cochrane did the choosing things might be much worse - since Cochrane was probably set up by a group of GPs and other anti-Pharma enthusiasts who were probably as biased as sin.

It all boils down to peer review, which is a broken system too. Fortunately, there is some sign that it might be working a bit better recently.

 

This surprises me. Marking one's own homework would seem susceptible to honest but very human bias at best, and decidedly dishonest and inhuman bias at worst.

 

Yes, the process is explained here, it's the authors who choose an available topic:

https://community.cochrane.org/revi...roposing-and-registering-new-cochrane-reviews

Proposing and registering new reviews

What’s the process for writing a review?

All Cochrane systematic reviews are first registered as titles with a Cochrane Review Group (CRG). Registering your title is important to prevent duplication of effort with other authors, and to make sure your topic is appropriate for a Cochrane review. There are 52 CRGs that each focus on a particular area of health. CRGs are coordinated by an editorial team who edit and publish completed reviews in the Cochrane Library. Unlike other journals, your CRG will provide support and advice throughout the review process.

1. The author team chooses an available topic and registers a title with the appropriate CRG.
2. Authors prepare a protocol and submit it to the CRG for peer review and publication.
3. When the protocol has been approved, authors begin searching for studies and analysing their results.
4. Authors prepare the final review report and submit it for peer review and publication.
5. After the review is published, the authors continue to update it periodically.


So, did Larun et al decide for themselves, completely unsolicited, to do this review or was there any involvement from the the PACE authors/BPS brigade/MRC ?

 

I think that just about says it all. Why haven't we ever done the Cochrane review ourselves as a community?

Larun was obviously put up by the BPS crowd.

 

Looks like Larun's PhD thesis was on 'Chronic fatigue syndrome - Knowledge, activity and challenges'

https://translate.google.com/transl..._Lillebeth%20Larun.pdf?sequence=1&prev=search

From 2011.

 

Well - I’d pay not to read that.

 

It seems that by 2011 Larun had already done a 'systematic review' of exercise in CFS, but using a technique called 'meta-ethnography', which seems to have more to do with feeling good than science. At least part of the study includes 'qualitative studies' which are essentially of no value in terms of efficacy evidence. The abstract seems to start with the assumption that exercise is useful and to conclude the same without any critical analysis of evidence.

Of note in her abstract she notes that the quality of documentation in the studies was moderate to low on the GRADE system.

At some point others joined her for a Cochrane review, including a member of Michael Sharpe's department. That would make it pretty impossible not to produce a favourable conclusion.

 

Who funds Cochrane?

 

Grief I hadn't realised one the authors is from MS's department, well there's a surprise......

I'm assuming this is acceptable procedure for Cochrane?

It seems quite appalling to me.

 

Thanks Trish, I'd not spotted that.

It looks as if the potential for bias is build into the Cochrane model.

 

I think it's actually the opposite. Price did the 2004 version: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub2/full

Larun then joined, and used the 2004 version of the basis/protocol for her review (although she then deviated from that protocol in the manner noted by Courtney).

 

Not surprised!
So maybe Larun based her PhD on Price?

 

From what @Jonathan Edwards says, this is normal practice.

My response being ...

 

Which is a shame, because that's why patient surveys are dismissed so readily (I know, I know, there are other issues too).

But all of these studies are still based on qualitative data. They've simply taken qualitative data (the questions in the SF-36, CFQ etc) and produced a number out of them, that they can then claim is 'quantitative'. Bingo! That it actually strips out all the interesting information is disregarded.

The reason that 'qualitative studies' are considered to be of 'no value' is because they are incredibly hard to analyse properly.