Trial By Error: A Letter to Archives of Disease in Childhood

[guest001]

I don’t want to over-egg the pudding, but it would be a shame if that prospective / retrospective registration tweet by Trish Groves, BMJ Open’s Editor in Chief and past COPE council member, about the ethical and scientific importance of prospective registration of clinical trials, suddenly disappeared...... wouldn’t it?

 

[Barry]

The BMJ group is a founder member of COPE, and COPE have the same requirement re prospective registration as the ICMJE (not a coincidence presumably, but the connection eludes me at the moment):

All of our journals have the same policy on trial registration—for studies started before July 2005, we permit retrospective registration (providing it was done before submission) but for trials started after July 2005, we require registration to have been done before enrolment of the first participant.

https://publicationethics.org/case/it-unethical-reject-unregistered-or-late-registered-trials

If a member persistently flouts this requirement, then surely their continued membership should be reconsidered, founder or otherwise?

Edit: By 'persistently' I mean either: new instances after an instance has been pointed out, or refusal to properly address a single instance after it has been pointed out.

 

[Awol]

I do wonder if a complaint to the ethics committees would be a good idea. I think the Exeter ethic committee acted inappropriately in converting a feasibility study to a full trial (and doing so via a subcommittee of 2 people). I think the ethics committee should have bounced the changes and said its a new trial reapply. But then its not a new trial but it is being presented as one but under the same ethics approval as the feasibility study. Its all very dodgy.

The ethics committee had also acted inappropriately by approving the feasibility study, as they knew the study involved participants undergoing the 'treatment' (as it was repeatedly referred to) so they should have rejected the application at the outset and advised Esther Crawley that she should apply using the (supposedly more stringent) process for clinical treatment trials. Processing SMILE as a feasibility study was evidently a way of doing a paediatric clinical trial through the back door, but with the compliance of the ethics committee.

The ethics committee undertook a review of their original decision to approve the feasibility study because of the "weight of correspondence" they received raising a number of concerns, but chose to uphold their original decision (bizarrely issuing a public apology to Phil Parker in the process).

Subsequent complaint to the National Research Ethics Service that Standard Operating Procedures had not been followed by NRES in their handling of the matter was unsurprisingly not upheld and, given the "potentially vexatious nature of correspondence and complaints", the panel agreed that ‘Understanding Animal Research’ should be contacted to explore areas of common ground in handling correspondence with regard to conducting research in contentious areas. This was March 2011, and the media frenzy over that summer, orchestrated by the UK Science Media Centre slamming ME patients and critics of research as dangerous activists, included Esther Crawley as well as the PACE trial team.

NRES seemed so intent that the SMILE study should go ahead that it was interpreted by the writer of this NZ article that:
"The process, developed in the late 1990s, is proving so successful that the UK's National Health Research Ethics Committee is doing a study on it." http://www.stuff.co.nz/auckland/local-news/east-bays-courier/5052840/Therapy-a-path-to-new-life

 

[Esther12]

The ethics committee had also acted inappropriately by approving the feasibility study, as they knew the study involved participants undergoing the 'treatment' (as it was repeatedly referred to) so they should have rejected the application at the outset and advised Esther Crawley that she should apply using the (supposedly more stringent) process for clinical treatment trials. Processing SMILE as a feasibility study was evidently a way of doing a paediatric clinical trial through the back door, but with the compliance of the ethics committee.

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

 

[Adrian]

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

I have issues with the feasibility study because of the trying it on children first.

But I think the ethics committee really go it wrong in terms of allowing the change fro a feasibility study to a full trial. They should have treated it as a new application - also checking things like power calculations were properly done and that the protocols made sense. Instead two people on a subcommittee basically approved it.

 

[Alvin]

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

I would guess that the logic behind this is selecting patients you already know will respond stacks the deck and science is supposed to be the hunt for the truth not cherry picking. And if a disease has more sufferers then the people your already looking at then building a random representative sample should be achievable. If there are only a few patients in the world with a disease and your already looking at them then you can't do a proper blinded objective trial, you can only report your subjective findings (and declare it as such)

 

[Jonathan Edwards]

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

I agree. Feasibility studies in the sense of preliminary studies involving treatment, or in other words pilot phase 1 studies, can be an essential component of getting to a position where a formal trial is ethical. Approving the feasibility study seemed to me entirely proper. But what you absolutely cannot do is then merge that into a formal study.

 

[dave30th]

I think the issue is mainly the combination of extending the feasibility trial PLUS swapping the outcomes. If everything had been kept exactly the same, then extending the feasibility trial would be much less problematic. My guess is the authors will argue that they didn't change the outcomes, they just swapped them, and they reported both outcomes, so they didn't do anything wrong--and on top of that they got ethical approval. One reason they might have done it this way is that substantial amendments to protocols only require a subcommittee of two, while an application for a new study would have required a quorum of the research ethics committee membership. The journal's editor has promised to review the matter and respond. It will be interesting to see whether Archives lets them get away with making bogus arguments, especially in light of the new BMJ Open study about how awful it is when studies aren't registered prospectively.

 

[Valentijn]

If a member persistently flouts this requirement, then surely their continued membership should be reconsidered, founder or otherwise?

COPE exists to protect its members from public claims of acting inappropriately. It's pretending to be something which it is not, and there's really no point in complaining about what its members are doing.

 

[Awol]

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

You're right of course, thanks. I do think it would have been reasonable for the REC to give an unfavourable opinion for this particular feasibility study particularly as it involved minors, and at the most basic level, EC failed to justify why the intervention should be tested on children before being tested on adults. I got muddled up by the answer to Q2 on the NHS REC form, which looks to me as though a better fit would have been "other clinical trial or clinical investigation" as the study involved the participants undergoing treatment. http://www.bristol.ac.uk/media-library/sites/ccah/documents/Ethics - NHS REC Form.pdf

Maybe what I should have said is that I think the original application should have been reviewed by an REC flagged for Research Into Children, but, on re-reading the NPSA/NRES SOPS for Research Ethics Committees, that's probably just my own opinion, rather than a breach of SOPs, and it wouldn't have necessarily altered the decision, as they may still have relied upon the information given by the PI. NRES said that the regional ethics committee had expertise to review paediatric research, and during the second review of their favourable opinion for the feasilibilty study, they were supported by a NRES Research Advisor also in post as an NHS Consultant Paediatrician.

I give the REC credit for making some important suggestions before giving their original decision, such as raising the lower age of entry criteria for partipants from 8 years of age to 12, but I do think they were remiss on other important points, such as disclosure of potential risks to undergoing the intervention, which was only included after the second review of their favourable opinion, and failing to grasp the problems in subjecting young participants to a treatment designed to train them to ignore their symptoms and to say that they feel well.

 

[Cheshire]

New letter from @dave30th to Archives of Disease in Childhood:

Trial By Error: My Exchange With Archives of Disease in Childhood

On January 30th, Professor Racaniello e-mailed a letter of concern to Archives of Disease in Childhood about a clinical trial of the Lightning Process in children with CFS/ME (as the study called the illness.) The letter, signed by 21 experts and academics, documented the trial’s questionable methodological choices and the investigators’ failure to disclose exactly what they had done.

In particular, the letter noted that the investigators extended a feasibility trial into a full trial while at the same time swapping primary and secondary outcome measures—a recipe for bias. As a result, they reported much more impressive-sounding results than would have been possible without the outcome measure swap.

http://www.virology.ws/2018/03/07/trial-by-error-my-exchange-with-archives-of-disease-in-childhood/

 

[Alvin]

New letter from

[SIZE=4][B]Trial By Error: My Exchange With Archives of Disease in Childhood[/B]
[URL]http://www.virology.ws/2018/03/07/trial-by-error-my-exchange-with-archives-of-disease-in-childhood/[/URL][/SIZE]

Given that, signatories to the Archives letter are somewhat perplexed that your review of the matter has already lasted more than a month. The facts are straightforward

Indeed, but alternative facts can be a minefield and need to be stress tested before being released for public consumption

 

[Esther12]

I just happened to get linked to Phil Parker's facebook page, and found this really sciencey explanation of CFS. I especially like how point 3 builds up to "basically everything." What I didn't like about this is that it got 67 shares!

 

[Allele]

And so it naturally follows that standing on a magic circle (instead of the vicious circle, of course) and shouting STOP is the answer to "basically everything."

 

[Alvin]

And so it naturally follows that standing on a magic circle (instead of the vicious circle, of course) and shouting STOP is the answer to "basically everything."

It doesn't beat the number 42

 

[Jonathan Edwards]

This is very similar to the molecular mimicry theory of autoimmunity and also to Mark Edwards's theory of functional disorders.

You have a negative feedback homeostatic system.
The explanation for disease X is that the negative feedback system acts as a positive feedback system. It is a bit like the heating engineer saying 'Oh yes, it's obvious - your house is boiling hot because the thermostat has gone into positive feedback' - without giving any explanation of how that might be.

An actual explanation of a disease in these terms has to explain why a system is homeostatic in 99.8% of the population and a vicious cycle for 0.2%. In autoimmunity precise molecular mechanisms for such a reversal are available. B cells use a chain reaction normally for rapid amplification of a response, just as white blood cells do in an abscess to localise events. If the local positive loops interfere with the general negative loops then problems arise. But without specific mechanisms you have a sort of anti-theory:

There is a negative control loop.
So the illness is explained by a positive uncontrol loop.
Duh!