https://www.cochranelibrary.com/cds....pub3/detailed-comment/en?messageId=421959275
1) The largest included study was published in the Christmas issue of the BMJ (1, which is characterized by articles written in jest. This was also the case for the study in question (2). It carries 75% of the weight in one of the main meta‐analyses of the review where the authors report a statistically significant effect on death, relative risk 0.88, 95% confidence interval 0.80 to 0.97. However, nowhere did the authors of the Cochrane review mention that this study evaluated the effect of prayer taking place 4‐10 years after the patients had either left the hospital alive or had died from their bloodstream infection. Thus, the study randomised dead patients and then studied whether they were dead or alive. The authors argued that we cannot assume "...that God is limited by a linear time".
5) When discussing the effect of prayer on what the authors call “clinical state”, they attempt to explain the lack of effect as being because the participants only received prayer for 14 days, which may not have been enough for prayer to be effective. The authors do not mention the far more plausible interpretation that the lack of effect of prayer is because prayer has no effect.
---
Author reply
1. Comments made about the Christmas issue of the BMJ and the
Leibovici 2001 study in particular are not fully accurate. Several articles in the late December issues of the BMJ are written with humour and some in pure spoof. Most are not. They may be written with humour and have an odd perspective, but are, nevertheless, interesting and well thought out research. The
Leibovici 2001 was not in jest. It is a rather serious paper, intended as a challenge.
We disagree that this review does not live up to the scientific standards one can reasonably expect of a Cochrane review.
It is a complex review, and like many others has been and remains imperfect. With successive revisions we endeavour to improve it.
---
In 2011:
https://www.cochranelibrary.com/cds....pub3/detailed-comment/en?messageId=421959262
we do think it is problematic to include a study written in jest if there is evidence in favour of this. That was the case for the
Leibovici 2001 study. We provided the evidence in our first response to the Cochrane review in a reference to a rapid response in the BMJ.
Leibovici clearly described his trial as a ‘non‐study’ and emphasised that his aim was not to test intercessory prayer, but to illustrate the limitations of which interventions can be sensibly tested in randomised trials:
‘() if the pre‐trial probability is infinitesimally low, the results of the trial will not really change it, and the trial should not be performed. This, to my mind, turns the article into a non‐study, although the details provided in the publication (randomisation done only once, statement of a wish, analysis, etc) are correct. The article has nothing to do with religion. I believe that prayer is a real comfort and help to a believer. I do not believe it should be tested in controlled trials.’ (4)
The review authors should therefore have had easy access to evidence that our assertion of the
Leibovici 2001 study is correct.
We recently wrote to authors of articles published in the BMJ Christmas issue to clarify the intentions behind them and investigate if they were given undue weight when they were subsequently quoted (5). The authors, including Leibovici, were given two options and could characterise their study as either:
A) a serious study (conducted to answer a scientific question, and could have been published in another scientific journal), or
B) a spoof study (written specifically for the BMJ Christmas issue with no attempt of addressing the proposed scientific question, but with an implicit ironic or tongue‐in‐cheek objective).”
The
answer from Leibovici was unmistakable: ‘Certainly B ‐ although the details of the 'study' that was done (basically dividing an old database into 2 random parts and comparing them) were described correctly in the article.’ Not all authors of papers in the BMJ Christmas issue chose option B, however.
Author reply —
We note the further comment from our colleagues in Denmark, who continue to argue against the inclusion of
Leibovici 2001.
The objectives of the original trialists when they complete a study must not influence its inclusion in a review for which the methods of the study make it eligible. To ensure that Cochrane reviews are rigorous they are preceded by a protocol – as was the case with this particular review. This protocol was open to comment and criticism. We received no comments on the inclusion criteria.
It is clearly not part of the reviewers' remit to judge the motivation of those conducting eligible trials. We are satisfied that we have applied our protocol as objectively and rigorously as possible. We are keen that all studies meeting the clearly stated inclusion criteria should be reported (even if later stated to have been "written in jest"), rather than being kept hidden and perpetuating publication bias.
