Trial By Error: A Letter to Archives of Disease in Childhood

I don’t want to over-egg the pudding, but it would be a shame if that prospective / retrospective registration tweet by Trish Groves, BMJ Open’s Editor in Chief and past COPE council member, about the ethical and scientific importance of prospective registration of clinical trials, suddenly disappeared...... wouldn’t it?

 

The BMJ group is a founder member of COPE, and COPE have the same requirement re prospective registration as the ICMJE (not a coincidence presumably, but the connection eludes me at the moment):

https://publicationethics.org/case/it-unethical-reject-unregistered-or-late-registered-trials

If a member persistently flouts this requirement, then surely their continued membership should be reconsidered, founder or otherwise?

Edit: By 'persistently' I mean either: new instances after an instance has been pointed out, or refusal to properly address a single instance after it has been pointed out.

 

The ethics committee had also acted inappropriately by approving the feasibility study, as they knew the study involved participants undergoing the 'treatment' (as it was repeatedly referred to) so they should have rejected the application at the outset and advised Esther Crawley that she should apply using the (supposedly more stringent) process for clinical treatment trials. Processing SMILE as a feasibility study was evidently a way of doing a paediatric clinical trial through the back door, but with the compliance of the ethics committee.

The ethics committee undertook a review of their original decision to approve the feasibility study because of the "weight of correspondence" they received raising a number of concerns, but chose to uphold their original decision (bizarrely issuing a public apology to Phil Parker in the process).

Subsequent complaint to the National Research Ethics Service that Standard Operating Procedures had not been followed by NRES in their handling of the matter was unsurprisingly not upheld and, given the "potentially vexatious nature of correspondence and complaints", the panel agreed that ‘Understanding Animal Research’ should be contacted to explore areas of common ground in handling correspondence with regard to conducting research in contentious areas. This was March 2011, and the media frenzy over that summer, orchestrated by the UK Science Media Centre slamming ME patients and critics of research as dangerous activists, included Esther Crawley as well as the PACE trial team.

NRES seemed so intent that the SMILE study should go ahead that it was interpreted by the writer of this NZ article that:
"The process, developed in the late 1990s, is proving so successful that the UK's National Health Research Ethics Committee is doing a study on it." http://www.stuff.co.nz/auckland/local-news/east-bays-courier/5052840/Therapy-a-path-to-new-life

 

But shouldn't there be a way of doing feasibility studies before starting a treatment trial?

The way Crawley rolled her feasibility study into a full trial, after changing primary outcomes, and just the loopiness of LP, was all dodgy, but I don't think that it's necessarily wrong to approve of a feasibility study because participants undergo treatment.

 

I would guess that the logic behind this is selecting patients you already know will respond stacks the deck and science is supposed to be the hunt for the truth not cherry picking. And if a disease has more sufferers then the people your already looking at then building a random representative sample should be achievable. If there are only a few patients in the world with a disease and your already looking at them then you can't do a proper blinded objective trial, you can only report your subjective findings (and declare it as such)

 

I agree. Feasibility studies in the sense of preliminary studies involving treatment, or in other words pilot phase 1 studies, can be an essential component of getting to a position where a formal trial is ethical. Approving the feasibility study seemed to me entirely proper. But what you absolutely cannot do is then merge that into a formal study.

 

I think the issue is mainly the combination of extending the feasibility trial PLUS swapping the outcomes. If everything had been kept exactly the same, then extending the feasibility trial would be much less problematic. My guess is the authors will argue that they didn't change the outcomes, they just swapped them, and they reported both outcomes, so they didn't do anything wrong--and on top of that they got ethical approval. One reason they might have done it this way is that substantial amendments to protocols only require a subcommittee of two, while an application for a new study would have required a quorum of the research ethics committee membership. The journal's editor has promised to review the matter and respond. It will be interesting to see whether Archives lets them get away with making bogus arguments, especially in light of the new BMJ Open study about how awful it is when studies aren't registered prospectively.

 

COPE exists to protect its members from public claims of acting inappropriately. It's pretending to be something which it is not, and there's really no point in complaining about what its members are doing.

 

You're right of course, thanks. I do think it would have been reasonable for the REC to give an unfavourable opinion for this particular feasibility study particularly as it involved minors, and at the most basic level, EC failed to justify why the intervention should be tested on children before being tested on adults. I got muddled up by the answer to Q2 on the NHS REC form, which looks to me as though a better fit would have been "other clinical trial or clinical investigation" as the study involved the participants undergoing treatment. http://www.bristol.ac.uk/media-library/sites/ccah/documents/Ethics - NHS REC Form.pdf

Maybe what I should have said is that I think the original application should have been reviewed by an REC flagged for Research Into Children, but, on re-reading the NPSA/NRES SOPS for Research Ethics Committees, that's probably just my own opinion, rather than a breach of SOPs, and it wouldn't have necessarily altered the decision, as they may still have relied upon the information given by the PI. NRES said that the regional ethics committee had expertise to review paediatric research, and during the second review of their favourable opinion for the feasilibilty study, they were supported by a NRES Research Advisor also in post as an NHS Consultant Paediatrician.

I give the REC credit for making some important suggestions before giving their original decision, such as raising the lower age of entry criteria for partipants from 8 years of age to 12, but I do think they were remiss on other important points, such as disclosure of potential risks to undergoing the intervention, which was only included after the second review of their favourable opinion, and failing to grasp the problems in subjecting young participants to a treatment designed to train them to ignore their symptoms and to say that they feel well.

 

Indeed, but alternative facts can be a minefield and need to be stress tested before being released for public consumption

 

I just happened to get linked to Phil Parker's facebook page, and found this really sciencey explanation of CFS. I especially like how point 3 builds up to "basically everything." What I didn't like about this is that it got 67 shares!

 

And so it naturally follows that standing on a magic circle (instead of the vicious circle, of course) and shouting STOP is the answer to "basically everything."

 

It doesn't beat the number 42

 

This is very similar to the molecular mimicry theory of autoimmunity and also to Mark Edwards's theory of functional disorders.

You have a negative feedback homeostatic system.
The explanation for disease X is that the negative feedback system acts as a positive feedback system. It is a bit like the heating engineer saying 'Oh yes, it's obvious - your house is boiling hot because the thermostat has gone into positive feedback' - without giving any explanation of how that might be.

An actual explanation of a disease in these terms has to explain why a system is homeostatic in 99.8% of the population and a vicious cycle for 0.2%. In autoimmunity precise molecular mechanisms for such a reversal are available. B cells use a chain reaction normally for rapid amplification of a response, just as white blood cells do in an abscess to localise events. If the local positive loops interfere with the general negative loops then problems arise. But without specific mechanisms you have a sort of anti-theory:

There is a negative control loop.
So the illness is explained by a positive uncontrol loop.
Duh!