A general thread on the PACE trial!

[Sid]

Nah. Some trial protocols are flawed and need to be changed during a trial. There is nothing unethical per se about a protocol amendment. The issue here is that they knew what treatment everyone was on, they saw that the treatments weren’t working and they made protocol changes to rescue their pet hypothesis.

 

[Sean]

One justification given by the PACE investigators, and also separately by Wessely, for the changes was that it made the results consistent with those from both the previous trials and their 'clinical experience', and that otherwise no PACE patient would be classed as improved, let alone recovered.

But the whole point of PACE, its justification, was to provide a "definitive" test of those previous underpowered non-definitive trials and clinical experience. So they can't use them as the benchmark for PACE results. It is the other way around. PACE was the benchmark, and it failed to validate those previous methodologically inferior results.

Furthermore, how did they know that the results from the original protocol didn't validate those previous results, and hence claim that as a justification for weakening the protocol, if they had not seen the results before they changed the protocol?

https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-018-0218-3
https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-019-0288-x
https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-019-0296-x

 

[Dolphin]

Extract from:
Carolyn Wilshire, Tom Kindlon & Simon McGrath (2017) PACE trial claims of recovery are not justified by the data: a rejoinder to Sharpe, Chalder, Johnson, Goldsmith and White (2017), Fatigue: Biomedicine, Health & Behavior, 5:1, 62-67, DOI: 10.1080/21641846.2017.1299358

Can previous data be used to support the changes to the recovery definition?

Sharpe et al. [3] refer to a number of studies performed by them and their collaborators which they believe provide independent support for their final recovery definition. They refer to several studies where the recovery rates following CBT, and/or following medical care alone, were numerically similar to the corresponding rates in their recovery paper (e.g. [8,9]). These observations are claimed as support for the use of their revised definition. Putting aside the issue of whether these studies are indeed representative of the literature, there are serious problems with this reasoning. First, most of the cited studies defined recovery differently to PACE, and if wemake even a reasonable attempt to equate the definitions, then the convergence amongst recovery rates does not look quite so good.2

For example, the definition of recovery used in the studies by Deale et al. [8] and Knoop et al. [9] were a lot stricter than the revised criteria used in PACE. If the Deale et al. recovery criteria are applied to the PACE data, for example (it is possible to use three of the four criteria), the PACE recovery rates fall to a maximum of 9% for CBT, which is very different from the 24% for CBT cited in Deale et al.

.

 

[Dolphin]

https://twitter.com/user/status/1748105901322477752

 

[Dolphin]

https://twitter.com/user/status/1759997145539805613

 

[poetinsf]

Posts moved from Opinion: ‘You don’t want to get better’: the outdated treatment of ME/CFS patients is a national scandal - George Monbiot
_______________

I'm sure this is old, but I couldn't help but stare at Sharpe's slide on PACE that showed a marginal improvement in the first 24 weeks and then nothing further. That is a proof that GET only reconditioned patients within the limit imposed by ME/CFS, and not a fundamental lifting of the ME/CFS condition or PEM threshold. And of course, APT lacks in the conditioning department since it aims at doing 50-70% to avoid PEM. It's in the same vein as Walitt's finding: they both are holding pacing responsible for dysfunction. They just don't understand that nobody wants to walk further and faster than ME/CFS patients.

 

[Hutan]

That is a proof that GET only reconditioned patients within the limit imposed by ME/CFS, and not a fundamental lifting of the ME/CFS condition or PEM threshold.

Not even proof of that.

The main PACE trial paper shows that 160 people were in the GET group at baseline, at 24 weeks there were only 150. I assume you are talking about differences in fatigue as measured by the Chalder Fatigue Questionnaire and physical function as measured by the SF-36. Both of those are subjective measures, so really biased by the nature of the interventions, especially when we are only talking about small differences over the SMC control, which we are. GET encouraged people to make light of their symptoms and suggested that getting better was the result of effort - so we would expect people to report that their capacity had improved after the intervention, somewhat regardless of how they were.

We have also heard that people cut out some of their pre-existing activities in order to be able to do the GET activities, so there was not necessarily any improvement in fitness. The six minute walking test is quite subject to bias too, with results depending on different levels of effort and familiarity. The improvement reported for that test at 52 weeks was modest and for only 110 people. I don't think we can say that the PACE trial is proof that GET made people with CFS fitter, up to some limit imposed by ME/CFS.

And of course, APT lacks in the conditioning department since it aims at doing 50-70% to avoid PEM.

The APT was not pacing. It was a complicated lot of joy-sapping scheduling. It didn't work either of course. It had the expectation that participants would increase their activity as they were able. So, actually it was quite similar to the symptom-contingent Graded Activity Therapy interventions that are currently being promoted at NIH clinics (and that have also been shown not to work e.g. recently in the Gaunt et al paediatric trial).

Therapeutic strategies consisted of identifying links between activity and fatigue by use of a daily diary, with corresponding encouragement to plan activity to avoid exacerbations, developing awareness of early warnings of exacerbation, limiting demands and stress, regularly planning rest and relaxation, and alternating different types of activities, with advice not to undertake activities that demanded more than 70% of participants' perceived energy envelopes. Increased activities were encouraged, if the participant felt able, and as long as they did not exacerbate symptoms.

I am by no means an expert in the PACE trial, others here might want to correct what I have written or add to it.

 

[Sean]

The Self-Paced Step Test in PACE also gave a null result, as did employment and welfare use.

Every objective measure in PACE was null, except the GET arm on the 6MWT, which did not reach clinical significance, barely increased the absolute distance patients could walk after a year of therapy, still leaving them scoring down among the worst of all medical conditions, and had a significant drop out rate. IOW, it is not even clear that single, extremely modest result was real, let alone of practical meaningful benefit to patients.

Total score on objective measures in PACE was effectively zero, and using the original protocol the results on the self-report measure were null as well.

Even if you accept the initial results on self-report measures according to the post-hoc modified protocol used in the published paper, the long-term follow-up (LTFU) results at 2.5 years were still null (no significant difference between any of the arms on the two primary measures).

Furthermore, an interesting feature of the response curves revealed at LTFU is that extrapolating the curve slopes suggests that the SMC & APT arms were continuing to improve, while the GET & CBT arms had tapered off in an asymptotic fashion, with no further gains, as you would expect from natural decay curves.

The PACE authors of course couldn't admit that, so they described the LTFU results not as null (no advantage for CBT or GET over APT or SMC) as they should have, but instead as CBT & GET 'maintaining their initial benefit'. Which is just not how comparative trials work at all.

In short, PACE showed no meaningful objective or sustained benefit from CBT or GET over APT or SMC. Even by PACE's own problematic standards the psycho-behavioural model it was testing failed completely.

And that is before we get into other serious issues with PACE, such as patient selection, harms reporting, dropping actimeters for outcome, and no doubt others I can't recall offhand.

 

[Sean]

Also, the PACE data that was released (and only when ordered to by a court) was only partial, and the authors have flatly refused to release the rest of it, despite legitimate requests for it from established researchers.

Which is about as unscientific, disgraceful, and cowardly as it gets in science.

 

[Peter T]

Also, the PACE data that was released (and only when ordered to by a court) was only partial, and the authors have flatly refused to release the rest of it, despite legitimate requests for it from established researchers.

Which is about as unscientific, disgraceful, and cowardly as it gets in science.

They had supposedly put the data on an ‘open access’ platform, but last I heard no one had managed to down load any of it. Does anyone have any up to date information?

 

[Midnattsol]

They had supposedly put the data on an ‘open access’ platform, but last I heard no one had managed to down load any of it. Does anyone have any up to date information?

I thought it was a platform where one could apply to be given the data, and the application needed you to write a valid reason or some such thing (+ I think it helped to have an affiliation with a university or other institution). There was the two data sleuths that didn't get access since reproducing analyses was not good enough?

 

[Sean]

If I understand it correctly, the mandatory retention period for the PACE data is 20 years, after which it can be destroyed. It is now 13 years since the publication of the main PACE paper (Feb 2011), so that leaves only 7 years for anybody to be able to access it before there can be no more re-assessment based on the unreleased data, nor hence on the full data set.

That is a situation that should never be allowed to occur. There needs to be serious pressure, particularly from within the medical scientific community, for that data to be secured and made available.

 

[MrMagoo]

If I understand it correctly, the mandatory retention period for the PACE data is 20 years, after which it can be destroyed. It is now 13 years since the publication of the main PACE paper (Feb 2011), so that leaves only 7 years for anybody to be able to access it before there can be no more re-assessment based on the unreleased data, nor hence on the full data set.

That is a situation that should never be allowed to occur. There needs to be serious pressure, particularly from within the medical scientific community, for that data to be secured and made available.

This would be something that would be great for George Monbiot to highlight, as he seems to have gone all in on this, could he be advised the clock is ticking?

 

[MBailey]

I thought it was a platform where one could apply to be given the data, and the application needed you to write a valid reason or some such thing (+ I think it helped to have an affiliation with a university or other institution). There was the two data sleuths that didn't get access since reproducing analyses was not good enough?

If anyone knows which platform this is on, that'd be really helpful...

 

[Robert 1973]

They had supposedly put the data on an ‘open access’ platform, but last I heard no one had managed to down load any of it. Does anyone have any up to date information?

I think @JohnTheJack was looking into this before Covid struck him down. Do you know the latest, John?

 

[Andy]

If anyone knows which platform this is on, that'd be really helpful...

I think these are the details that we have, PACE trial data

 

[JohnTheJack]

Yes, Vivli is the one they have been promoting.

It's important to note they have not released data as if it were in response to an FOI. They have released data for researchers only.

As @Midnattsol points out, some highly qualified researchers have tried and then gave up.

This is what one of them said to me after he gave up:

Getting the data from Vivli seems like a very convoluted process. I'm not surprised the PACE authors put it there... [his ellipsis]
Yeah, it's just gatekeeping. The opposite of OSF and proper open science
I have since found a bit more. It's like it was designed by a particularly sadistic bureaucrat. I'm trying to imagine who would put their data on there who didn't have something to hide.

 

[Jonathan Edwards]

Many of the following posts have been moved from
Paul Garner on Long Covid and ME/CFS - BMJ articles and other media. after Paul Garner tweeted in an attempt to suggest that AfME's later criticism of the PACE trial was not based on science and simply because people didn't like the findings of the trial.



I wonder if the campaign to stop PACE originally was based on the poor trial design?
My memory is that the failings trial design did not really surface until 2015. Prior to that most of the complaints were about the diagnostic criteria for entry, which may not have suited PWME/CFS but were scientifically legitimate. Even the US committee that advised against GET in ?2015 seemed to base its decision on that.

 

[dave30th]

My memory is that the failings trial design did not really surface until 2015.

It was clear from the beginning that the outcomes differed from the protocol ones. Patients criticized that right away. The overlap in outcome measures was noticed right away--no one knew how many would fall into that category, but later it turned out to be 13% for physical function. But the fact that it was possible was clear from The Lancet paper. The "normal range" analysis was shown to be flawed because they misrepresented the underlying population it was based on. etc etc. Patients were criticizing lots of things besides the case definitions.

ADD: When the recovery paper was published in 2013, then. it was even clearer what they had done--lowered all four of the "recovery" thresholds, and without any apparent committee approvals. It was only when the data were released in 2016 after Alem's suit that it was prove-able that everything patients were saying was true, so then the pressure really intensified.

 

[FMMM1]

Thanks for your reply.

But the published protocol paper, published in 2007, long after they started planning and indeed doing the trial, does not list actigraphy as an outcome measure, only as a predictor.

Interesting @Jonathan Edwards plausibly suggested that PACE may have had its origins in the preparation of the previous NICE Guideline [experts from a UK University - which uni?]. Basically all of the studies indicated that these interventions [exercise (& CBT?)] worked; however, one of those on the panel had published a research paper highlighting that they were all methodologically flawed - so potentially they highlighted that a better study was required!
What is the earliest business case/protocol in the public domain and what does it say?
What does "protocol paper---does not list actigraphy as an outcome measure, only as a predictor" mean in practice?

In the Trial Management Group minutes in 2004-5, you can see they they did consider whether or not they should make actigraphy not just a predictor, but also an outcome measure, and opted not to for well-dodgy reasons.

Wow - would you expect a contractor to decide what you should get as an outcome? As per my previous comment I'd like to see the original business case and see what the intention re outcome indicators was. It seems the contract was shambolic - you can trust an inhouse (civil servant in this case) expert - but relying on the contractor to set outcome indicators/purpose of the study --- beggars belief!

In the minutes the primary outcome measures were always questionnaires - Chalder Fatigue and SF36 physical function - right from 2003 (see the Trial Management Group minutes Trial Management Group minutes Meeting No. 2).

Begs the question what the propose of the study was!