Cochrane review and the PACE trial

Sly Saint

Sly Saint

Senior Member (Voting Rights)
In the debate on the PACE trial yesterday, the Cochrane review was again cited as validation of the PACE trial results
".........scientists as part of normal research collaboration, including the internationally respected research organisation Cochrane, which independently validated the findings."

I thought it would be useful to have a dedicated thread to discuss this/post relevant info.

Exercise as treatment for patients with chronic fatigue syndrome
"This review is an update of a previous Cochrane review from 2004, which showed that exercise therapy was a promising treatment for adults with CFS. Since the review, additional studies investigating the effectiveness and safety of exercise therapy for patients with CFS have been published."

http://www.cochrane.org/CD003200/DEPRESSN_exercise-treatment-patients-chronic-fatigue-syndrome

(I see that it comes under depression........hmm).

eta:
old article
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub4/full
most recent
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full



 
Last edited:
They refused data to Keith Geraghty and 3 of them sat on the cochrane review.



They also cited that the authors have published responses to criticism, but were not able/didn’t manage to publish anything meaningful in the PACE special edition JoHP. Can’t remember their reason though.
 
I think we need to stress over and over again that the main criticism of the PACE study concerns the basic design flaws. They used subjective outcomes in a study that was not blinded, and they conflated chronic fatigue with ME/CFS. Yes, they also manipulated the results to boost the subjective improvement and recovery rates, and they screwed up the sub-group analysis of different diagnostic criteria. However, even if we disregard the manipulations and “mistakes”, the conclusion still is that there is no improvement in objective secondary outcomes to back up the modest increase in subjective scores. It is a null result.

The Cochrane reviews on cognitive behavioral therapy and graded exercise therapy for ME/CFS are meta-analyses based on studies with the same design flaws. Pooling over multiple studies reduces statistical errors, but does not change the inclusion criteria in the studies and does not convert subjective outcomes to objective ones. We need to emphasize that there is no evidence of objective improvement in the Cochrane reports.
 
I strongly believe that the Cochrane review is a mega-important issue for us to focus on at the moment. Even more so than PACE.

The PACE authors will present the Cochrane review to NICE and claim that it upholds their own findings.They will have a strong case that will need careful, analytical and rigorous rebuttal.

We won't easily be able to challenge the use of CBT and GET as long as the Cochrane review stands.

Either we have to challenge the Cochrane review or we have to make sure that NICE understand that the Cochrane review relied on the Oxford criteria; meaning that it doesn't investigate ME/CFS. The Cochrane review shouldn't claim to examine chronic fatigue syndrome while relying on the Oxford criteria; it should claim only to investigate 'fatigue'.

It does also seem to be a conflict-of-interest if the pace authors sat on the Cochrane review panel. (Although I'm not quite sure what that is.)

Importantly, there are many technical flaws and factual errors in the Cochrane review which have been highlighted in the letters submitted and published in the latest version. These need to be followed up and pursued, vigorously.
 
Last edited:
Bob said:
Either we have to challenge the Cochrane review or we have to make sure that NICE understand that the Cochrane review relied on the Oxford criteria; meaning that it doesn't investigate ME/CFS. The Cochrane review shouldn't claim to examine chronic fatigue syndrome while relying on the Oxford criteria; it should claim only to investigate 'fatigue'.
Click to expand...
It might be worthy to highlight the extreme vetting the NIH is doing for the intra-mural study. Dr. Nath reports have to screen 200 suspected ME/CFS patients, just to find 20 study participants. Also, the purpose of the formation of Common Data Elements is to ensure that trial outcomes can be compared.

I used to work as a therapist using an evidence-based model. Because the model is evidence-based, agencies all over the county dish out huge grants to employ agencies that use the model. Every case had tremendous amounts of data collection and reporting. But, guess who generated the assessment forms, analyzed the data, and published? The same group of psychologists who created and patented the model. It was obvious working from the inside how skewed their interpretation of positive outcomes was. Evidence-based psychology is junk science without quality control, objective measures, and independent review.

I'm afraid we live in an era where facts are not reliably facts, since they are manufactured outcomes...PACE being the worst example.
 
Sly Saint said:
Exercise as treatment for patients with chronic fatigue syndrome
"This review is an update of a previous Cochrane review from 2004, which showed that exercise therapy was a promising treatment for adults with CFS. Since the review, additional studies investigating the effectiveness and safety of exercise therapy for patients with CFS have been published."
Click to expand...


Am I right in thinking that this is in effect the penultimate review and that another review is 'in the pipeline'. I was asked to review a new version with authors including I think all of White, Sharpe and Chalder some time last summer. I made it clear that I thought this was poor quality. What I do not know is what has happened since.
 
Bob said:
I strongly believe that the Cochrane review is a mega-important issue for us to focus on at the moment. Even more so than PACE.

The PACE authors will present the Cochrane review to NICE and claim that it upholds their own findings.They will have a strong case that will need careful, analytical and rigorous rebuttal.

We won't easily be able to challenge the use of CBT and GET as long as the Cochrane review stands.

Either we have to challenge the Cochrane review or we have to make sure that NICE understand that the Cochrane review relied on the Oxford criteria; meaning that it doesn't investigate ME/CFS. The Cochrane review shouldn't claim to examine chronic fatigue syndrome while relying on the Oxford criteria; it should claim only to investigate 'fatigue'.

It does also seem to be a conflict-of-interest if the pace authors sat on the Cochrane review panel. (Although I'm not quite sure what that is.)

Importantly, there are many technical flaws and factual errors in the Cochrane review which have been highlighted in the letters submitted and published in the latest version. These need to be followed up and pursued, vigorously.
Click to expand...

The PACE trial is now seen as a big piece of shit, null results on a subjective non blinded study with a made up criteria. If the Cochrane review supposedly backs up the findings of the BPS crew from the PACE trial doesn't that mean that the review itslef found all the other papers to be a big bag of shit too?

One cant use Cochrane review to advocate GET and CBT using PACE and visa versa, if PACE is now being seen as a null trial.

It seems to be that the BPS crowd are saying, "Ok well PACE the £5 million definitive trial may be crap but our pre PACE prejudices that we couldn't prove via PACE are backed up by other studies, after all we authored a lot of those other papers then sat on the Cochrane review panel".
 
Bob said:
Either we have to challenge the Cochrane review or we have to make sure that NICE understand that the Cochrane review relied on the Oxford criteria; meaning that it doesn't investigate ME/CFS. The Cochrane review shouldn't claim to examine chronic fatigue syndrome while relying on the Oxford criteria; it should claim only to investigate 'fatigue'.

It does also seem to be a conflict-of-interest if the pace authors sat on the Cochrane review panel. (Although I'm not quite sure what that is.)
Click to expand...
The other issue is that the Cochrane mental health group which lays claim to ME/CFS was co-founded by Simon Wessely. Some members of the group likely continue to have a vested interest in the outcomes of their own reviews (CBT/GET clinics and/or insurance ties), and I also suspect that Wessely and others have maintained an influence in it.
 
Jonathan Edwards said:
Am I right in thinking that this is in effect the penultimate review and that another review is 'in the pipeline'. I was asked to review a new version with authors including I think all of White, Sharpe and Chalder some time last summer. I made it clear that I thought this was poor quality. What I do not know is what has happened since.
Click to expand...
There is a new review in the pipeline which will analyse individual patient data for exercise therapy. [Exercise therapy for chronic fatigue syndrome (individual patient data)] At the moment only the protocol has been published, which was authored by the PACE authors. I suspect you were asked to review this review.

The existing review (exercise therapy for chronic fatigue syndrome) was first published years ago and was republished a couple of years ago with new data. It doesn't analyse individual patient data but only looks at effect sizes. The review includes substantial errors, untruths, and outcome switching etc. (Similar to PACE but maybe even worse?) The PACE authors aren't officially listed as reviewers for this review.
 
large donner said:
The PACE trial is now seen as a big piece of shit, null results on a subjective non blinded study with a made up criteria. If the Cochrane review supposedly backs up the findings of the BPS crew from the PACE trial doesn't that mean that the review itslef found all the other papers to be a big bag of shit too?

One cant use Cochrane review to advocate GET and CBT using PACE and visa versa, if PACE is now being seen as a null trial.

It seems to be that the BPS crowd are saying, "Ok well PACE the £5 million definitive trial may be crap but our pre PACE prejudices that we couldn't prove via PACE are backed up by other studies, after all we authored a lot of those other papers then sat on the Cochrane review panel".
Click to expand...
Unfortunately the PACE trial hasn't been retracted. This means that the study and its results are still a legitimate part of the scientific literature even though it is a contested paper.

The main complaints about PACE are that the data have been promoted and interpreted incorrectly; that it uses subjective outcomes; that the recruitment criteria aren't appropriate; and that the basis of the therapies aren't appropriate. On the other hand, no one has actually contested the data itself; Only the interpretations of the data have been challenged.

Cochrane can legitimately claim that it is appropriate for them to review the PACE data in an analysis of subjective outcomes from nonblinded trials that investigate 'fatigue' (using the Oxford criteria). I think it's legitimate as long as they clarify that bias is inheritant in all of the studies and that it doesn't investigate chronic fatigue syndrome. I don't think it is legitimate to claim that the review investigates chronic fatigue syndrome when the studies recruited patients based on a 'fatigue' criteria.

The Cochrane review is an analysis of subjective outcomes from non-blinded sstudies using Oxford criteria, or similar, for recruitment. In that respect it is no better than PACE. It is legitimate (as long as it is transparently stated in its aims, which it isn't) to carry out such an analysis of psychological therapies to treat 'fatigue', but it does seem like a rather pointless exercise, and it isn't relevant to ME/CFS.

Our task needs to be to challenge the Cochrane review directly, and/or to make sure that NICE understands the failings of the Cochrane review in relation to ME/CFS.

i.e. the failings are that it is completely irrelevant to ME/CFS patients, because the studies investigated used non-ME/CFS recruitment criteria; that the trial investigates subjective outcomes from non-blinded studies, so there is inherent bias; and that there are flaws and errors in the Cochran review, which have not been corrected.

Conflicts of interest could also be raised but this is a bit of a tricky area, and will probably be impossible to challenge unless we manage to bring down the entire scientific establishment.
 
Last edited:
Bob said:
I don't think it is legitimate to claim that the review investigates chronic fatigue syndrome when the studies recruited patients based on a 'fatigue' criteria.
Click to expand...

re Oxford Criteria I recall another discussion:

Valentijn said: ↑
Oxford requires fatigue to be the principal symptom, which is not the experience of many ME patients. Oxford also requires the presence of mental fatigue.
Click to expand...

Jonathan Edwards said:
I thought that fatigue had to be a feature with all these criteria. And most PWME on the forums seem to have mental fatigue so I would bet that nearly all the patients here fit Oxford if they fit the other sets.
Click to expand...

large donner said: ↑
Earlier you posted on the "phenomenon" as you descried it wherby people would be convinced they had RA when they simply didn't met the criteria, would you selectively include such people in an RA study then report on the findings as an RA study and would this be scientifically valid?
Click to expand...

Jonathan Edwards said:
No but that is not the point. They might well be included in a study of joint pain. And studies of joint pain can be applied to people with RA, because people with RA fall into that category. We are not talking about doing a study of Canadian criteria CFS and slipping in some Oxford patients. The study announces itself as studying Oxford patients so its results can be applied to anyone who fits Oxford. It might turn out that within that there is a Canadian subgroup that in fact responds differently, but until that was shown the general rule would be legitimate to apply.
Click to expand...

quoted from: https://www.s4me.info/threads/iime-...recommended-treatments.1949/page-3#post-34540 #54 & #56
 
This succinct article "Time to Reject the PACE Study" criticizes both PACE and the Cochrane review of PACE as well:
In a recent Cochrane review on exercise for ME/CFS based on PACE and seven other similar studies, all methodological issues were ignored [31]. The authors collaborated closely with the PACE team and have publicly been accused of conflicts of interest [32, 33]. A meta-analysis is only as reliable as the original studies that it is based on. The Cochrane review must therefore be questioned in the same way as PACE.
Click to expand...
The Cochrane review conflicts of interest in particular relate to the role of Dr Lillebeth Larun, which Prof James Coyne details in these blog articles.
 
Bob said:
and will probably be impossible to challenge unless we manage to bring down the entire scientific establishment.
Click to expand...

Lets do it!


Hip said:
The Cochrane review conflicts of interest in particular relate to the role of Dr Lillebeth Larun, which Prof James Coyne details in these blog articles.
Click to expand...

Thanks, I'd forgotten about some of those Coyne blogs. What was the COI stuff? Was it just her involvement with the IPD review? There's so much to read at the moment - my eyes have been complaining.
 
Bob said:
Conflicts of interest could also be raised but this is a bit of a tricky area, and will probably be impossible to challenge unless we manage to bring down the entire scientific establishment.
Click to expand...
It might be easier to just bring down the mental health Cochrane group, especially in specific regards to ME/CFS. The various Cochrane groups seem to act with fairly little oversight, so most of the blame can land on that one in particular without bringing the wider Cochrane brand into the discussion too much. Though Cochrane is failing to enforce quality controls, and their failure to deal with a rogue group would suggest that they have some serious faults in general.
 
Jonathan Edwards said:
Am I right in thinking that this is in effect the penultimate review and that another review is 'in the pipeline'. I was asked to review a new version with authors including I think all of White, Sharpe and Chalder some time last summer. I made it clear that I thought this was poor quality. What I do not know is what has happened since.
Click to expand...
This is the most recent:
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD003200.pub7/full

25 April 2017
(I added it to my first post shortly after I'd posted the excerpt)

Esther12 said:
Coyne blogs. What was the COI stuff?
Click to expand...
This was their reply to Coyne:
http://community.cochrane.org/news/cochrane-and-conflict-interest

eta: the PACE trial authors also responded but the full article is behind a paywall,
anyone have access to it?
"
Controversy over exercise therapy for chronic fatigue syndrome: continuing the debate
In a recent Round the Corner, Mitchell commented on a Cochrane Review of exercise therapy for chronic fatigue syndrome (CFS). One of the trials included in that review, and discussed by Mitchell, was the PACE trial. In this month’s Round the Corner we are publishing a response we received from authors of the PACE trial (Chalder, White & Sharpe), together with Mitchell’s reply. Ed."

Sept 2017

http://apt.rcpsych.org/content/23/5/288
 
Last edited:
Esther12 said:
Thanks, I'd forgotten about some of those Coyne blogs. What was the COI stuff? Was it just her involvement with the IPD review? There's so much to read at the moment - my eyes have been complaining.
Click to expand...

My brain is the organ that complains the most about the workload (isn't it ironic that when you are too ill to work in a salaried job, you actually find you work longer hours due to involvement with ME/CFS research and politics than you ever did in regular employment).

I only briefly glanced at Coyne's blog, but this is what he says about the COI:
The conflicts of interest of Lillebeth Larun

A co-author of Lillebeth Larun on the systematic review under discussion is a colleague in the department of the investigators whose trial is being evaluated. Larun is a co-author on another protocol with these investigators. Examination of the acknowledgments that protocol indicates that the investigators provided both data and funding for meetings:

The author team held three meetings in 2011, 2012 and 2013 which were funded as follows:

2011 via Paul Glasziou, NIHR senior research fellow fund, Oxford Department of primary care.
2012 via Hege R Eriksen, Uni Research Health, Bergen.
2013 via Peter D White’s academic fund (Professor of Psychological Medicine, Centre for Psychiatry, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London).
So, the both the systematic review under discussion and the other protocol were conducted among “families and friends”. In dismissing concerns about risk of bias for a particular trial, Lillebeth Larun is ignoring the obvious strong bias for her associates.

She has no business conducting this review nor dismissing the high risk of bias of inclusion of their study.
Click to expand...
 
You must log in or register to reply here.
Back
Top Bottom