PACE trial TSC and TMG minutes released

Sample size calcs should be based on what would be considered clinically important, not just statistically significant. They seem to have based it on the results that previous trials got, and haven't looked at whether any of that was clinically relevant or not.
No, I think what they did was correct for a power analysis. Which is to estimate likely effect sizes based on previous studies and ensure sufficient numbers of patients to detect effects of that magnitude. Power analyses are about preventing the possibility of a type II error (missing an effect that's really there). The issue of power is different from that around whether an effect is clinically important.
 
Last edited:
No, I think what they did was correct for a power analysis.

I am pretty much with @Lucibee n this point. What matters, if you are claiming equipoise, is that the cognoscenti have significant doubt about whether a treatment is clinically useful and that you have reasons, either purely theoretical or from inconclusive previous studies, to think it might be.

Power studies are certainly to avoid type II errors but you want to avoid a type II error of clinical significance, not just a type II error that matches a rather measly result someone got before.

I have always been surprised by the size of PACE. When I did a four arm trial we reckoned we needed about 160 patients (and so recruited 160). For treatments to be much use for chronic disease you want the sort of level of benefit that gives you that sort of number. For things like prevention of cardiac problems with aspirin things are different because only a small proportion of recruits will ever get ill. I may be wrong but it seems to me that a trial that has to recruit 640 patients is looking for something hardly worth bothering with in the first place.

On the hand I think you can claim equipoise and still state which treatments you expect to do best. In a sense it is just as unethical to do a trial if you have no reason to expect something to do best. All that equipoise requires is that there is genuine doubt. That would mean for instance, that you have problems leaving patients on a placebo if you knew there was an proven alternative effective treatment they could be having during the period of the trial.
 
If I were them, I would have kept the trial as simple as possible and done a straight comparison between what is offered in the clinic and what patients say works for them. They "controlled" it in completely the wrong way by expecting pts to attend clinic for their "pacing" intervention. But that just shows how poorly they understand the condition.

That pacing involved a therapist and visit to the clinic was an attempt to move it closer to CBT/GET in terms of placebo effect (not the best choice of words but you know what I mean). We have no idea how strong the placebo effect component was in each treatment arm. Ultimately the comparison of pacing and CBT/GET cannot be made with subjective measures because CBT/GET intentionally tries to introduce bias, while pacing does not. It's like comparing apples and oranges.
 
The adaptive pacing therapy manual for therapists in the PACE also made a complicated therapy out of the simple idea of pacing. It mentions "homework", "activity and fatigue diaries", "daily and weekly planning", "activity analysis sheets" and so on.

Pacing is in my view not clearly defined but I have never seen pacing described as such a complicated activity.

https://www.qmul.ac.uk/wolfson/media/wolfson/current-projects/1.apt-therapist-manual.pdf
 
I may be wrong but it seems to me that a trial that has to recruit 640 patients is looking for something hardly worth bothering with in the first place.

I was under the impression that larger sample sizes are not about establishing efficacy, they're about uncovering potential rare adverse events. You are correct that if it takes 160 patients per arm to achieve minimal statistical significance, the effect is unlikely to be clinically significant.
 
That pacing involved a therapist and visit to the clinic was an attempt to move it closer to CBT/GET in terms of placebo effect (not the best choice of words but you know what I mean). We have no idea how strong the placebo effect component was in each treatment arm. Ultimately the comparison of pacing and CBT/GET cannot be made with subjective measures because CBT/GET intentionally tries to introduce bias, while pacing does not. It's like comparing apples and oranges.

But if visiting the clinic causes exacerbation of symptoms, then you can't assess it properly. If they had been truly concerned about harms, they would have started from a completely different place. It was never a fair comparison, but I don't think they ever really cared about that.
 
Well done @Lucibee, I vaguely remember fiddling about with the formula they give for the step test some time ago and finding it more than a little confusing - I put lots of figures in and they didn't make much sense.

It sounds like the PACE researchers didn't have a clue what they were doing. My sympathy is with the patients who felt forced to do such a test for no good reason. Though the outcome, as shown on the graph, does say whatever they were hoping to prove didn't work.
 
For those who expressed an interest... (@Robert 1973 @Graham @Adrian @Trish @Esther12 @Luther Blissett @BruceInOz @Barry @Sasha @Jonathan Edwards )
I've now done a step test blog: https://lucibee.wordpress.com/2018/07/06/pace-trial-tiptoeing-around-the-step-test/

It's a bit rough and ready, so amendments and additions are likely. :whistle:

The step test was in the secondary outcomes for the protocol but was silently dropped from the stats analysis plan. I don't think they ever gave any reason for it. I see this as challenging their claims that they are compliant with the consort guidelines.

It may be hard to calculate the numbers but it wouldn't surprise me if they just got it wrong in Chadler's paper. I don't think they have any notion of quality control, or looking back in the protocol to find the right equation.
 
A bit OT: I've just seen a load of FOIs to the MRC I'd previously missed (or forgotten about) that seem a bit relevant to these minutes so I thought I'd post about them in here.

When were the MRC informed, in accordance with their guidelines, that the PACE Trial (PD White et al) Primary Outcome Measures of 'Positive Outcome' and 'Recovery' were to be eliminated from the Research Protocol?

Please provide any of the following which you have available:

1/ the date.
2/ a copy of the communication.
3/ the explanation/justification for the changes.
4/ the date and correspondence when the MRC approved the changes.

[along with more]
As a funder of the trial the MRC was a member of the Trial Steering
Committee (TSC) in a non-voting capacity. Any changes to the trial
protocol would have been discussed at the TSC and therefore the MRC would
have received notification through this route. The minutes of the TSC
meetings do refer to discussions around the detail of the protocol and
analysis plan, but again do not mention the terms “ recovery” or “positive
outcome”.

The Trial Steering Committee discussed the protocol in detail at a meeting held on 22 April 2004.

The Trial Steering Committee joined a meeting of the Analysis Strategy Group held on 04 September 2009. It was at this meeting that the change in the analysis of the Chalder Fatigue Questionnaire was discussed and agreed i.e. the planned secondary outcome measure of the scale using the Likert (0,1,2,3) scale would become part of the primary outcomes instead of the bimodal (0,0,1,1) scale.

The MRC considers that attendance at a meeting of the Trial Steering Committee and/or receipt of a copy of the minutes to be notification of any changes made.

https://www.whatdotheyknow.com/request/when_were_the_mrc_informed_that#incoming-881123)

Is the bottom line that the PACE trial principal investigators
and adviser, had no legal requirement to comply with their published trial
protocol, in regards to how they spent the 5 million pounds of tax payers
money, that was awarded to them, for the PACE trial?

In my emails of 5 ^ July and 18 July I have already outlined how changes
to an agreed protocol for an MRC funded study were managed. Approval from
the Trial Steering Committee and an NHS Research Ethics Committee would be
required before any changes could be implemented. You can find further
information about how amendments are currently managed from the Health
Research Authority:
[1]http://www.hra.nhs.uk/resources/after-yo....
Once approved any amendments would be rolled out, including new versions
of trial documentation.

https://www.whatdotheyknow.com/request/cfs_pace_permission_to_drop_more
 
But was this a change to the management of the trial, or simply to the ways they intended to analyse it?

The removal of those two measures and the change to CFQ didn't require any change to be made to any data that was collected from patients, so may not constitute a material change in the trial itself. They were still working out their analysis plan way after the trial and the roles of the TSG/TMG had ended.
 
This comment from the PACE team is pretty funny to look back upon:

Re: People want to learn as much as possible from the PACE trial for chronic fatigue syndrome

01 October 2013

Peter D White

Professor of Psychological Medicine

T Chalder, M Sharpe, T Johnson, K Goldsmith

Barts and the London Medical School

St Bartholomew's Hospital, London EC1A 7BE



Tom Kindlon states that access to the committee minutes of the PACE trial is needed to “..to find out why outcome measures were changed”.1 We disagree.

First, the primary outcome measures were the same ones as described in the protocol – fatigue and physical disability.2 Second, details and explanations of independently approved changes to the scoring system and analysis of the outcomes are already in the public domain; both in the published papers and on the PACE trial website (http://www.pacetrial.org/faq/).3,4

Third, his suggestion that there were problems in the reporting of harms is also incorrect.1 Unusually for a non-drug trial, we adopted the same stringent procedures as recommended by the European Union Clinical Trials Directive for Pharmacological Interventions. We measured safety by serious adverse reactions and events, non-serious adverse events, withdrawals from treatment because of worsening, self-rated global worsening, and a composite measure of serious deterioration.2,3 All adverse events were reviewed by an independent panel.

Finally, readers should know that the Information Tribunal’s unanimous judgement on the appeal was that: “The tribunal has no doubt that properly viewed in its context, this request should have been seen as vexatious - it was not a true request for information - rather its function was largely polemical.”5

PD White 1, T Chalder 2, M Sharpe 3, T Johnson 4, K Goldsmith 5

1 Wolfson Institute of Preventive Medicine, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London,
2 Academic Department of Psychological Medicine, King's College London,
3 Department of Psychiatry, University of Oxford, Oxford, UK
4 MRC Clinical Trials Unit at UCL, London
5 Biostatistics Department, Institute of Psychiatry, King’s College London,

1. Kindlon T. People want to learn as much as possible from the PACE trial for chronic fatigue syndrome. BMJ 2013;347:f5731.

2. White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn R; PACE trial group. Protocol for the PACE trial: a randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise, as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol 2007;7:6.

3. White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O’Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.

4.W hite PD, Johnson AL, Goldsmith K, Chalder T, Sharpe MC. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychol Med 2013;1-9, published online 31 Jan. doi:10.1017/S0033291713000020.

5. General Regulation Chamber (Information Rights) First Tier Tribunal. Mitchell versus Information Commissioner. EA 2013/0019. http://www.informationtribunal.gov.uk/DBFiles/Decision/i1069/20130822%20....

Competing interests: PDW has done voluntary and paid consultancy work for the United Kingdom government and a reinsurance company. TC has received royalties from Sheldon Press and Constable and Robinson. MS has done voluntary and paid consultancy work for the United Kingdom government, has done consultancy work for an insurance company, and has received royalties from Oxford University Press. KAG and ALJ declare that they have no conflicts of interests.

https://www.bmj.com/content/347/bmj.f5731/rr/664559

This being the BMJ, the PACE teams' misleading response to Kindlon was printed in their journal: https://www.bmj.com/content/347/bmj.f5963

Kindlon's response to them was not published in the journal: https://www.bmj.com/content/347/bmj.f5963/rr/667107

There were quite a lot of good responses to White's reply actually, and some funny stuff from Sean Lynch offering a semi defence of PACE, eg:

We may have to agree to differ about how the rationale for the changes in the trial protocol and analyses is perceived, as I feel I would have to trust the independence and integrity of the Steering Committee and statistical analysis, unless there was compelling evidence to the contrary. I would stand by my views that an independent secondary analysis is one possible option in interpreting trial data if there are serious concerns.

https://www.bmj.com/content/347/bmj.f5963/rr/671597
 
Just looking through these minutes. At meeting #8 of TSC, [redacted, who we know is Paul Dieppe] retires from the DMEC but is not replaced.

The next meeting is a joint one of the TSC and DMEC where the changes to the trial outcomes are approved. [Redacted, my guess Astrid Fletcher] from the DMEC sends apologies.

So when the changes were approved by 'two independent oversight committees', only one member of the Data Monitoring and Ethics Committee was present.
 
Just looking through these minutes. At meeting #8 of TSC, [redacted, who we know is Paul Dieppe] retires from the DMEC but is not replaced.

The next meeting is a joint one of the TSC and DMEC where the changes to the trial outcomes are approved. [Redacted, my guess Astrid Fletcher] from the DMEC sends apologies.

So when the changes were approved by 'two independent oversight committees', only one member of the Data Monitoring and Ethics Committee was present.


And the one left on this independent committee was psychiatrist Charlotte Feinmann who had co-authored with... Wessely.
 
Do we know why Dieppe retired? He should have known there were problems from the outset.

I haven't seen or heard anything.

Dieppe is on the dark side. He's into all sorts of quackery:

The Clinical Disability Paradox as a Window to Health and Wellbeing, Health & Wellbeing Network, University of Exeter, programme 2012.
https://cedar.exeter.ac.uk/media/un...work/H_and_W_invite_and_programme_v2.indd.pdf

Paul Dieppe on spirituality, ritual and the healing response, The History of Emotions Blog.
https://emotionsblog.history.qmul.a...spirituality-ritual-and-the-healing-response/
 
Back
Top Bottom