David Tuller - Trial By Error: My Exchange With Archives of Disease in Childhood

[Kalliope]

My exchange with archives of disease in childhood

On January 30th, Professor Racaniello e-mailed a letter of concern to Archives of Disease in Childhood about a clinical trial of the Lightning Process in children with CFS/ME (as the study called the illness.) The letter, signed by 21 experts and academics, documented the trial’s questionable methodological choices and the investigators’ failure to disclose exactly what they had done.
...

Archives of Disease in Childhood is part of the BMJ group. The journal’s editor, Dr. Nick Brown, assured Professor Racaniello that he took the matter seriously and that it would be reviewed. Since then, another BMJ journal, BMJ Open, has published a paper that touches directly on the issue. The new study examined whether studies submitted to The BMJ and not accepted because they were not properly prospectively registered were subsequently published elsewhere, among other things. The study concluded that “improperly registered trials are almost always published, suggesting that medical journal editors may not actively enforce registration requirements.”

Last week, I sent the following e-mail to Dr. Brown

 

[Daisymay]

Excellent, put the pressure on them, it's definitely necessary and shows them up.

If they try and wriggle out of this clear cut case they're going to look so bad.

But who notices?

That's one problem we have, who cares?

 

[Kalliope]

Excellent, put the pressure on them, it's definitely necessary and shows them up.

If they try and wriggle out of this clear cut case they're going to look so bad.

But who notices?

That's one problem we have, who cares?

We the patients care. This is about our health and our lives, for most of them it is just a job. I think this underlines how utmost important it is with proper patient representation and cooperation with patient organisations on all levels in medical research. Perhaps there should even be some patient representation in the journals?

 

[Awol]

I think this underlines how utmost important it is with proper patient representation and cooperation with patient organisations on all levels in medical research. Perhaps there should even be some patient representation in the journals?

The problem in UK is that patient organisations are used as a vehicle to enable and justify poor practice in medical research.

 

[Esther12]

I wonder if it could be better to be more politey-polite in correspondence, at least until they've has been really shit.

Has the Archives of Disease in Childhood editor being particularly shit yet, or just slow (after wrongly publishing a papers from a study with a number of clear flaws, and then praising it in the issue's introduction - which sadly, I don' think this counts as particularly shit for medical journal editors)?

eg: I doubt there's much advantage to bits like this, and they might put some people's back up:

As I mentioned in my last e-mail, I’m working on this project in journalism time, not academic-publishing time.

I was reading this half-interesting/half-annoying blog on debunking from Nick Brown:

There are things in there that can be rightly criticised, and seem to reflect a level of conservative pragmatism that I think is verging on immoral, but that it is coming from people presented as radical 'data thugs' is a reminder of what the culture is like within large parts of the medical research community. I get the impression that a lot of people in medicine are easily affronted, and happy to use that as a justification for ignoring problems within the research literature.

This maybe isn't the right thread for discussing this, as Tuller's letter was far from a radical assault, but I just happened to be reading both pieces at the same time. Also, as Tuller says, the facts are clear. The SMILE study violates the BMJ's policies and should not have been published. But then again, the BMJ's policies were only raised on 19th Feb (although the fact SMILE was not prospectively registered was raised earlier): http://www.virology.ws/2018/02/19/trial-by-error-a-letter-to-bmj-open/ If the paper were to be retracted, working out the details of that might be expected to take 'academic-publishing time'. I have to admit to a suspicion that the delay is a result of trying to find a way to help Crawley out, but who knows?

 

[dave30th]

As you say, I think they're probably stonewalling and trying to figure out a way around it. What is there to investigate? Even if I didn't know about the COPE and BMJ policies when the letter was first sent, they certainly were aware of those policies. So seeing the documentation of what had happened in this case--that the investigators had clearly violated those policies--should have set off immediate alarms, if they weren't already aware of the data manipulation. The reference to academic time comes across more snarky than intended. It was in response to the phrasing in his e-mail, which I didn't include, and a reference to an earlier e-mail I'd sent.

 

[Luther Blissett]

As you say, I think they're probably stonewalling and trying to figure out a way around it. What is there to investigate? Even if I didn't know about the COPE and BMJ policies when the letter was first sent, they certainly were aware of those policies. So seeing the documentation of what had happened in this case--that the investigators had clearly violated those policies--should have set off immediate alarms, if they weren't already aware of the data manipulation. The reference to academic time comes across more snarky than intended. It was in response to the phrasing in his e-mail, which I didn't include, and a reference to an earlier e-mail I'd sent.

We're talking about a group of people who are seriously pretending that a situation with the logical complexity of a light switch is somehow difficult. A light switch! I'd talk to them at the level of cognitive development that implies.

If they are unable to deal with such complexity, why should anyone take their journal seriously? Why would someone want them to publish anything? Not exactly a badge of honour to be in their journal if that is the standard of oversight.

I wouldn't worry about any snarkiness, intended or not. They deserve to be treated the same way as they've treated anyone who tried to question them.

 

[TiredSam]

I was reading this half-interesting/half-annoying blog on debunking from Nick Brown:

There are things in there that can be rightly criticised, and seem to reflect a level of conservative pragmatism that I think is verging on immoral, but that it is coming from people presented as radical 'data thugs' is a reminder of what the culture is like within large parts of the medical research community. I get the impression that a lot of people in medicine are easily affronted, and happy to use that as a justification for ignoring problems within the research literature.

That is a very interesting article, and probably deserves a thread of its own. I would love to read a review of it by someone like Christopher Hitchens, but unfortunately he has been sadly gathered. Plenty of arguments to be made on both sides I'd say.