BMJ Peer review of Wilshire et al re-analysis of PACE paper

[Esther12]

Split from the twitter thread. Some discussion of the peer review of the PACE re-analysis article submitted to the BMJ starts from post #1745 of that thread.

edit: I also want to point out that this review misrepresents a number of aspects of the paper. The paper submitted to the BMJ was later released (after some modifications) here: https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-018-0218-3 The paper went on to get mainstream media coverage from the BBC, page 2 of the Times, etc: https://www.thetimes.co.uk/article/findings-of-5m-me-chronic-fatigue-study-worthless-89z8x0xzr

So that I have just two posts to bookmark, I'm going to quote the exchange with the BMJ about that ridiculous peer review report here. Possibly it deserves its own thread? It's easy for people to miss things in this huge rambling one.

Background:

We submitted an earlier version of this paper to BMJ. It did not include as much reanalysis detail, it mainly focused on the recovery data and the arguments concerning the interpretation of subjective measures in non-blinded trials. The authors were myself, Tom Kindlon, Robert Courtney, Keith Geraghty, and Alem Matthees.

We had two reviewers, one reviewer recommended publication, describing our piece as a "very well-written and incisive analysis" and made some useful minor suggestions. The other reviewer, who is a Scottish psychiatrist that has previously coauthored publications with the Sharpe/White/Wessely group, was (unsurprisingly) very critical, and pretended that he wanted to see a paper this bad being made available publicly so he and his cronies could fully criticise it! It is worth reading the full rant, to really appreciate what passes for peer review at this journal. I would add that bits and pieces from this diatribe were quoted by the editor as justifications for rejecting the paper. What a shabby excuse for a journal.

Here is the full diatribe:

One of the nice things about providing a referee's report (and it is even nicer when I am the recipient) is when a fresh look at a manuscript provides simple suggestions that lead to clear improvements in the eventual published paper. For the 1st time in the three decades that I have been refereeing scientific papers I am going to recommend publication of a manuscript but only on the condition that no changes are made based on any of my comments or questions or criticisms or praise. This is because the tone of this paper and certain of its content provide a fascinating illustration of some of the problems that surround the management of people with severe and prolonged fatigue states. As well as making some quite interesting points these authors provide examples of what people with severe and prolonged fatigue have to put up with - even from other people who themselves have severe and prolonged fatigue. If this manuscript is published in the BMJ this will give the chance for correspondents in the Rapid Responses section to point out some of its flaws and hopefully a valuable debate will follow that will contribute to a more thoughtful approach to this whole difficult field.

There has been and continues to be a great deal of scientific controversy surrounding the PACE trial of the treatment of chronic fatigue syndrome(s) using graded exercise therapy vs cognitive behavioural therapy vs adaptive pacing therapy all in addition to standardised care delivered by doctors with specialist experience of chronic fatigue syndrome(s). The back and forth scientific controversy makes fascinating reading as does this manuscript by Dr Wilshire, Mr Kindlon, Mr Courtney, Dr Geraghty and Mr Matthees.

In my opinion, the differing scientific interpretations of this trial have little or nothing to do with the participants' scientific training and expertise. Rather, scientific stances are dependent on people's personal background and/or their clinical training and/or their clinical experience of assessing and treating patients with severe fatigue states and/or their own personal experience of ill health and/or the illness experience of family members and/or their personal experience of clinical care especially the care they have received from doctors. I will try - briefly and I hope not too boringly - to weave into this referee's report some of my own background which will perhaps give some understanding of why, overall, I dislike this paper and yet still want it to be published in a widely-read journal.

Whether to publish this paper will not be a straightforward decision for the editors. It is certainly not an original contribution to the scientific literature. A paper with similar content by some of the same authors appeared only a couple of months ago in the journal "Fatigue, Biomedicine Health & Behaviour". However, if it is published in the BMJ then this would provide a readily accessible update on some of the continuing controversies surrounding the diagnosis, prognosis and treatment of people with severe and prolonged fatigue. Because of the Rapid Responses/Correspondence section of the BMJ it will give the investigators in the PACE trial the opportunity to put their different scientific interpretations to a wider audience. They have already published these different interpretations in “Fatigue, Biomedicine, Health & Behaviour". I am still waiting for that paper from my librarian but I am certain that these scientists are more than able to defend their trial. An up to date defence of the PACE trial with balanced consideration of its strengths and weaknesses will be very helpful for clinicians and policy makers if it is published in the readily accessible BMJ.com. This will, in turn, make it easier when clinicians and health service managers try to improve services for the wide range of patients with severe and prolonged fatigue.

The main reason I would like this paper to be accepted is because of some careless use of language by the authors. This can happen to anybody and I am sure these authors do not really mean what they say. They have however insulted and demeaned a subgroup of people with severe and prolonged fatigue (see below). This is in spite of the fact that Mr Kindlon, Mr Courtney and Mr Matthees are themselves in poor health due to severely fatiguing illnesses. I am sure this will be picked up by the PACE investigators and other readers of the BMJ and highlighted in the Rapid Responses. If not, then I will write in and tell the authors that they should know better and I think I would end with a "Shame on you".

If this manuscript is published in the BMJ and the authors receive the appropriate criticism (and credit) for some the things they have said I hope they will then get on with applying their skills and intelligence and insights to some proper work that will have a chance of genuinely benefiting patients with chronic fatigue syndromes and other overlapping disorders. It is about time that they moved on from their obsessive (in the non-psychiatric use of the term) poring over the results of a good (albeit imperfect) randomised controlled trial (…and name me one perfect trial that exists in any clinical field).

Here are some comments on the manuscript that may be helpful to the editors in deciding whether or not to publish this non-original paper in a major general journal. I am hoping that these comments will help draw out how interesting this paper will be to many readers of the BMJ.

1) These authors make no attempt whatsoever to acknowledge the heterogeneity of patients who are labelled with a diagnosis of chronic fatigue syndrome (and, in my experience, the even greater heterogeneity of the smaller number of patients who are labelled using the diagnostic concept of myalgic encephalomyelitis).

2) These authors make a big deal of the fact that their paper is the result of collaboration between patients and scientists. I am still unsure whether that should be clinical scientists. Dr Wilshire and Dr Geraghty – do you have any clinical training and experience?

3) From what I can gather from this paper and their other writings Dr Wilshire, Mr Kindlon, Mr Courtney and Mr Matthees and to a lesser extent Dr Geraghty seem to believe that the best definition of chronic fatigue syndrome is whatever condition it is that has led to ill health in Mr Kindlon and Mr Courtney and Mr Matthees.

4) I am sorry to hear of these authors’ ill health. I hope they will not be upset when I say that I do not accept their diagnosis of chronic fatigue syndrome. I do not accept at face value anybody’s declared diagnosis of chronic fatigue syndrome until I have done my own history and examination. The reasons for this are as follows. When I was a junior registrar in neurology in 1981/1982 our team investigated referrals – including self referrals – of patients with severe fatigue. They received a battery of investigations from brain scans through lumbar puncture through visual evoked responses to muscle biopsy. Every test would be normal. We never took a social history and never carried out a mental state examination. Come to think of it we never even took a proper past medical history. The patients would then be told that they had a condition called myalgic encephalomyelitis and would be sent home to rest with the prognosis that they would not improve but that there may be a cure in the future with advances in neurovirology. As I have written before (Pelosi, British Journal of General Practice, January 2000) and I have thought to myself on many occasions since - may God forgive me for the part I played in destroying the lives of some of these vulnerable patients.

I later did clinical work with referrals with severe fatigue in three different clinical and geographical settings between 1988 and 2011. I found then that a substantial minority (for a period it was the majority) of patients with a diagnosis of chronic fatigue syndrome and myalgic encephalomyelitis had readily diagnosable conditions using basic knowledge of general medicine and general psychiatry. Sometimes my rediagnosis/reformulation would lead on to effective treatment. However, some patients would reject these alternative explanations. Some, sadly, would be angry towards me and state that I was not taking their illness seriously. It was very disconcerting and it raised major worries about the inadequacies of my clinical communication skills when I would tell a patient with a diagnosis of chronic fatigue syndrome or, more usually, myalgic encephalomyelitis that their profound fatigue and other serious symptoms were better explained by, for example, their recurrent diabetic ketoacidosis; or the systemic effects of their known severe rheumatoid arthritis and its treatment; or their severe psychotic illness and its treatment; or profound depressive illness; or crippling panic disorder; or Reiter’s disease; or obsessive compulsive disorder with co-morbid depression; or malnutrition due to anorexia nervosa; or the temporary aftermath of newly diagnosed and treated severe thyroid disease…….I could go on and on - and then be told by the patient that I was not taking their ill health seriously.

I accept that my experience must have been unrepresentative since I was working from psychiatric outpatient clinics and for most of this time I was based in the West of Scotland that was the epicentre of myalgic encephalomyelitis movement. However, I cannot believe that I am the only doctor who encountered this phenomenon. How can I be sure that Mr Kindlon, Mr Courtney and Mr Matthees do not have ill health that could be much better categorised using any one of a range of much more straightforward diagnoses?

5) Who diagnosed Mr Kindlon’s, Mr Courtney’s and Mr Matthees’ condition? I do not expect an answer since their medical history is and should remain confidential. However I have to raise the possibility that they are self diagnosed. For a while at my clinic a clear majority of diagnoses of chronic fatigue syndrome or myalgic encephalomyelitis had been made by the patient themselves or by a relative or friend or neighbour and not by any doctor or other clinician. Many (but not all) of these patients had other obvious reasons for their fatigue after taking a full history and doing a physical examination and mental state examination. I accept that such high frequency of self diagnosis/lay diagnosis may not be found in other clinics but, once again, I cannot believe that I am the only clinician who has encountered this phenomenon.

6) If they have been diagnosed with chronic fatigue syndrome by a doctor was this by their own doctor? There certainly used to be a phenomenon whereby patients with a self diagnosis of chronic fatigue syndrome and (even more so) myalgic encephalomyelitis used to go doctor shopping until they found a doctor who agreed with their own diagnosis. In my experience many (but not all) of these patients had readily diagnosable and fairly straightforward alternative diagnoses.

7) I notice in the manuscript that Dr Wilshire says she has “recently co-authored a major critical review of the concept of psychological causation in medicine…”. Dr Wilshire, I mean no disrespect, but is it for you to say that your review is a major one? I hope you will not mind my saying that I found it to be very unbalanced and highly selective in its use of the medical literature. Once again, I want this manuscript as it stands to appear in a widely-read journal so that interested readers will be able to get a clearer view of Dr Wilshire’s thinking and I hope it will be criticised appropriately.

8) I think Wilshire and colleagues are right to question the number of subjects in whom there has been “recovery”. They may not be aware that the word “recovery” has been hijacked throughout mental health services by the very influential and international Recovery Movement. This is a positive example of a powerful and creative collaboration between service users and clinicians. Wilshire and her colleagues should look it up. The Recovery Movement has contributed to improved care for many patients especially those with chronic psychotic illnesses. I think I was the last person involved in mental health services in Scotland – patient or clinician – to argue against some uncritical approaches of the Recovery Movement but I ended up giving in. I find myself talking to certain patients about recovery - who have no chance of recovery in the dictionary sense of the word. The PACE investigators will easily be able to defend themselves in regard to recovery in their trial since they have operationally defined what they meant by their use of this word and they have been very clear about changes they made in their definitions. Nevertheless, Wilshire and colleagues do have a point about how reported “recovery rates” could be misleading.

9) Wilshire and colleagues are right that the PACE trial is not definitive. I have only seen the PACE investigators say this on one occasion and in one paper although I may have missed some other instances. I think this was just some careless use of language. Give them a break.

10) There is something unpleasant in the tone of this article (although I am also being influenced here by outpourings on the Internet). I cannot help but get the impression that Wilshire and her colleagues were punching the air when they thought they had come up with support for their views that certain treatment methods for certain people with serious ill health were not as helpful as others (both patients and clinicians) had hoped. Are Mr Kindlon, Mr Courtney and Mr Matthees absolutely sure that they are writing about syndromes of chronic fatigue? Are they sure they are not simply writing about themselves?

11) This brings me to the insulting language about certain patients with severe and prolonged fatigue states. Mr Kindlon, Mr Courtney and Mr Matthees are absolutely certain that their condition is not influenced by psychological troubles or social stressors. This does not give them and their co-authors any right to belittle other patients and understate the severity of illness in those with a diagnosis of chronic fatigue syndrome where such factors are playing a major part in their ill health. These authors write “…the CBT programme considers patients’ concerns about exercise to be merely ‘fearful cognitions’ that need addressing”. Earlier in the article they state “This model proposes that there is no major ongoing disease process in CFS – merely deconditioning due to recent inactivity, and its various consequences”. Merely? Have they never met anybody with near 100% disability due to mere fearful cognitions? I have. Have they never spoken with patients who are at risk of dying due to mere fearful cognitions and their consequences? I have. Wilshire and her colleagues should be ashamed of themselves and I think their shameful language should be published and then condemned. They should then be given the opportunity to write what they really mean and apologise to those patients about whom they have been so offensive. This is the main reason why I want this article to appear in a widely read general journal that has an active Correspondence/Rapid Responses section.

I hope these comments are helpful to the editors. There is such widespread and heated debate around this subject that I think it would be helpful for some of it to appear in a widely-read journal. I hope the editors will agree with me and publish this article – then stand back and see how the debate unfolds in the Rapid Responses/Correspondence. I think this will lead in the end to more people with severe and chronic fatigue getting better care packages - which will include for some cognitive behavioural therapy and for others graded exertion therapy and for yet others perhaps even adaptive pacing therapy and for others none of the above but with high quality informed medical care for everybody. Hopefully it will also contribute to efforts to raise funding for more research projects that will join the PACE trial in giving us guidance about how to treat and how not to treat individual patients with severe fatigue be it explained or unexplained or simple or complicated.

 

[Esther12]

Dear Navjoyt,

Thanks for your reply. I do understand you cannot accept every paper.

However, with this submission, something does seem to have gone very wrong in the review process. There is a very serious problem with one of the reviews. Reviewer 2 comes across strongly as having a personal agenda or viewpoint, and makes a number of statements that are highly emotive or otherwise inappropriate in the context of a journal review (e.g., ad hominem attacks on me or my co-authors). Many of these are also are ill-informed. The review also completely fails to directly address the core evidence and arguments presented in the paper. The reviewer is also incorrect that the work has already been published. Our previous publication considered only one aspect of the trial (claims of recovery). It was designed to pave the way for this broader article that evaluates the trial as a whole, and delves into considerably more detail – including presentation of the much-awaited novel findings from the reanalysis of the PACE trial.

It is the quality of the review, not its final outcome, that concerns us here. This review appears, at least on the surface, to recommend publication. But this makes it no less acceptable.

Reviewer 1’s review, although positive, was also rather scant. In this situation, it should have been clearly evident to the editors that an additional reviewer was needed.

In fact, I urge you to consider whether it was proper to forward Review 2 to us at all. Forwarding a review to the authors endorses its legitimacy as a source of valid information about the manuscript. By sending this review out, the BMJ is communicating the message that insults and emotive statements are acceptable substitutes for proper scholarly analysis of its manuscripts.

I have attached some brief notes that are intended to highlight the concerns about Review 2. I understand the need for confidentiality of reviewers. However, we will be making public some of these notes, including original quotes from Reviewer 2 and our accompanying responses. We feel this is important to maintaining an open process. No reviewers’ names will ever be revealed. The BMJ’s name won’t mentioned, at least not immediately. You need a chance to speak in your own defence first.

We gave BMJ the opportunity to consider this paper at their leisure, and had good faith that they would apply a fair and reasonable process. On submitting a paper to BMJ, we do not expect acceptance, but after nearly seven months’ wait, we do expect a proper scholarly evaluation of our manuscript. We do not feel that this has happened, and it is very disappointing.

Sincerely,

Carolyn Wilshire



Examples of Inappropriate Content in Review 2

“… these authors provide examples of what people with severe and prolonged fatigue have to put up with - even from other people who themselves have severe and prolonged fatigue.”

It is not for a journal reviewer to make wide claims about the views and potential reactions of patients, but rather to assess the content of the paper within its scope (which is patients diagnosed with CFS by Oxford criteria). Accusations of perceived wrong-doing, unsupported by evidence - are also clearly inappropriate.

This comment also displays an alarming lack of understanding of the patient perspective. Amongst CFS patients, there is widespread dissatisfaction with the treatments currently on offer (CBT and GET). The patient-organisation ME Action recently obtained more than 12,000 signatures on a petition calling for the misleading claims of the PACE trial to be withdrawn (http://www.meaction.net/pace-trial).

“I am sorry to hear of these authors’ ill health. I hope they will not be upset when I say that I do not accept their diagnosis of chronic fatigue syndrome. I do not accept at face value anybody’s declared diagnosis of chronic fatigue syndrome until I have done my own history and examination.”

Extraordinary comment for a scholarly review. Such ad hominem criticisms are not only irrelevant to the assessment of a scientific article; they are frankly insulting.

“These authors make a big deal of the fact that their paper is the result of collaboration between patients and scientists. I am still unsure whether that should be clinical scientists. Dr Wilshire and Dr Geraghty – do you have any clinical training and experience?”

Again, it was our hope that the BMJ would judge this paper on content rather than on ad hominem criticism of the authors. We do not make a “big deal” of the patient status of the article. It is mentioned in the author note. We do not use our status at any point in the paper to support our arguments; instead, we base our arguments on solid evidence and reasoning.

The reviewer goes on to express his wish that the “flaws” of our article, if published, will be revealed in the BMJ Rapid Responses, preferably by the authors of the original article. However, at no point does he attempt to identify these flaws himself, despite this being the primary task of a journal reviewer. Why is there no engagement in this review with the actual content of the manuscript? And how was it possible that the BMJ overlooked this key requirement for any review?

“Whether to publish this paper will not be a straightforward decision for the editors. It is certainly not an original contribution to the scientific literature. A paper with similar content by some of the same authors appeared only a couple of months ago in the journal "Fatigue, Biomedicine Health & Behaviour"….I am still waiting for that paper from my librarian…”

Before making a claim that two papers are heavily overlapping in content, a reviewer really does need to read both papers. Our papers are all available online without cost – no need to wait for the “librarian”. Just type in the names into Google scholar!

The paper in Fatigue paper does NOT cover the same material as the current ms, but deals in detail with the issue of recovery. The current paper has a much broader scope, delves into theory as well as evidence, and also presents the much-awaited results from the reanalysis of the PACE trial data that was recently made publicly available.

“The main reason I would like this paper to be accepted is because of some careless use of language by the authors. This can happen to anybody and I am sure these authors do not really mean what they say. They have however insulted and demeaned a subgroup of people with severe and prolonged fatigue (see below). This is in spite of the fact that Mr Kindlon, Mr Courtney and Mr Matthees are themselves in poor health due to severely fatiguing illnesses.”

Here are some further, highly emotive accusations of wrong –doing on our parts. Also, accusing us of “careless usage of language” is itself a careless accusation, if it is not supported by references to sentences in the document and an argument as to why they should be considered careless.

“If not, then I will write in and tell the authors that they should know better and I think I would end with a "Shame on you".”

Again, rather stunned to as the tone of language here that would normally be considered unacceptable in a journal review.

“It is about time that they moved on from their obsessive (in the non-psychiatric use of the term) poring over the results of a good (albeit imperfect) randomised controlled trial…”

This is a first-class example of the highly emotive language and style of this review, which seems to be based largely on strongly-held opinions and beliefs about the subject matter in general and/or its authors, and not about the material in the manuscript itself. Also, a professional who works with CFS patients (as he declares later) should not need to be told why such a critique is important. This trial continues to affect the treatment options of CFS patients both in the UK and overseas, and this is of enormous concern to patients. Therefore, attempting to negatively characterise myself and my co-authors as “obsessive” is not only unscholarly, but it betrays a profound lack of respect for the patient point of view.

“From what I can gather from this paper and their other writings Dr Wilshire, Mr Kindlon, Mr Courtney and Mr Matthees and to a lesser extent Dr Geraghty seem to believe that the best definition of chronic fatigue syndrome is whatever condition it is that has led to ill health in Mr Kindlon and Mr Courtney and Mr Matthees.”

Further examples of unseemly statements that should never have appeared in a journal review. No this is incorrect, we specifically define CFS in the article, using the primary definition from the trial. At no point do we discuss any of our own experiences.

“Wilshire and colleagues are right that the PACE trial is not definitive. I have only seen the PACE investigators say this on one occasion and in one paper although I may have missed some other instances. I think this was just some careless use of language. Give them a break.”

The exhortation to “give them a break”, referring to the authors of the original trial being critiqued is another type of polemic that has no place in a formal journal review. I understand these researchers are his friends and co-authors (that much is clear from his CV), but one has to put aside such things in a review and act in a professional manner. Or else, declare a conflict of interest.

“There is something unpleasant in the tone of this article (although I am also being influenced here by outpourings on the Internet). I cannot help but get the impression that Wilshire and her colleagues were punching the air when they thought they had come up with support for their views that certain treatment methods for certain people with serious ill health were not as helpful as others (both patients and clinicians) had hoped. Are Mr Kindlon, Mr Courtney and Mr Matthees absolutely sure that they are writing about syndromes of chronic fatigue? Are they sure they are not simply writing about themselves? “

Again, I was struck by the irony here, given the emotive and sometimes insulting and patronising tone of this review. Note here there is a further appeal to ad hominem arguments.

“This brings me to the insulting language about certain patients with severe and prolonged fatigue states. Mr Kindlon, Mr Courtney and Mr Matthees are absolutely certain that their condition is not influenced by psychological troubles or social stressors. This does not give them and their co-authors any right to belittle other patients and understate the severity of illness in those with a diagnosis of chronic fatigue syndrome where such factors are playing a major part in their ill health. These authors write “…the CBT programme considers patients’ concerns about exercise to be merely ‘fearful cognitions’ that need addressing”. Earlier in the article they state “This model proposes that there is no major ongoing disease process in CFS – merely deconditioning due to recent inactivity, and its various consequences”. Merely? Have they never met anybody with near 100% disability due to mere fearful cognitions? I have. Have they never spoken with patients who are at risk of dying due to mere fearful cognitions and their consequences? I have. Wilshire and her colleagues should be ashamed of themselves and I think their shameful language should be published and then condemned.”

In the context of a formal scholarly review that discussed only evidence and make not statements about individuals, it is quite astounding to be personally accused of “belittl(ing) other patients and understat(ing) the severity of illness in those with a diagnosis of chronic fatigue syndrome”. Two of our co-authors will not be able to read this review at all because they are currently entirely bedbound and cannot use a computer. In this context, the assumption that we lack understanding of severe illness is particularly offensive.

Dear Dr Wilshire,

Many thanks for getting in touch with us with your concerns. I am so sorry that the review process fell short of the standard you expected from The BMJ.

From your email to us on 8 April 2017, you explain that you do not feel that a proper scholarly evaluation of your manuscript took place, with specific concerns about the quality of referee reports and the appropriateness of forwarding reviewer 2's comments to the author team

The usual process for Analysis articles is that they are screened initially by an editor and then sent out for external peer review if deemed to be on a topic that is potentially relevant, interesting, and important to readers. Once we have the referee reports - ideally a minimum of two - the paper is listed for discussion at a fortnightly manuscript meeting. We select reviewers with different types of expertise and experience, and as you know we are committed to a peer review process that identifies reviewers to the authors (and, in the event of publication, to readers). At the meeting, a group of 3-5 editors discuss a batch of Analysis articles, in turn going through each paper's substance, strengths, and what it adds to previous work. Editors will also be mindful of a paper's suitability for the section and how it compares with other articles we are considering at the moment. We take the reviewers' comments into account, but the final decision rests with the editors and the degree to which reviewers' reports influence a decision will vary.

On this occasion, a thorough internal review of your paper by five editors gave us sufficient and convincing reasons to reject it. I’m afraid that all editors felt that the paper did not add enough to the debate for our audience of general clinical readers and raised more questions than it answered decisively. When this is the case, we tend to the view that the debate is better aired in the more specialist literature, where those most able to weigh the concerns will be sure to read the article. Our decision was informed by the reviewers but not driven by them, particularly because, as you note, on this occasion the two reports were each quite unusual: one was very short, and the other recommended publication so as to air in public many of the reviewer’s criticisms of the article. We did not feel that either reviewer persuaded us of the case for publication.

In terms of the time taken to review, and the possibility of seeking other reviewers’ reports,

there were, I am sorry to say, several delays that occurred during the editorial process for this paper, in part due to the length of time taken to send the article out for peer review, difficulty in obtaining reports from suitable reviewers, and finally a short wait in the queue of papers for the manuscript meeting.

While perhaps in another situation we may have sought additional reviews, when weighed against the unanimous editorial view for this paper and the time already taken to reach a decision, we did not think that in this instance we would be helping you by delaying the process further, given that it was very unlikely that any further reviewer would alter the editorial assessment of the article.

We do understand that you may find reviewer 2's comments upsetting and I’m very sorry for any offence they might have caused you. We passed these on to you in the spirit of our open review process but I regret that we did not do better to signpost them to you in our letter. Please do be assured that in evaluating the comments of both reviewers, the editors are used to considering the substance beyond the tone, and looking specifically at the issues that allow us to assess the article’s suitability for our particular audience.

I must also remind you that although we have an open peer review process where authors and reviewers are aware of one another’s identities, we do expect authors to keep reviews confidential, as we state in the decision letter. This blog explains our thinking: http://blogs.bmj.com/bmj/2016/05/16...-why-the-bmj-rejected-a-weekend-effect-paper/

In summary, while I do understand your comments about the quality of the referee reports, I am confident that a proper scholarly evaluation of the article took place. I unreservedly apologise once again for the time taken to reach a decision and for any distress the review process might have caused.

Yours sincerely,

Navjoyt

Thank you for your reply, Navjoyt.

It is most interesting that you chose to take the stance you did.

I probably don’t need to point out the problems with your explanation. The most obvious one is that if you were able to make up your mind about the paper without reviews, why did you send it out for review and makes us wait seven months? That is a very long time.

Also, you say “in evaluating the comments of both reviewers, the editors are used to considering the substance beyond the tone”. But the evidence from the decision letter contradicts this claim. Some of Reviewer’s 2 phrases appeared directly in your justification of rejection - you note “methodological issues, such as patient selection and heterogeneity” - but there’s no indication that you understand what these phrases actually mean. Think about it for a minute. This paper reanalyses data from a previous trial, and we do not have the privilege of fine-grained information about the clinical similarities and differences between the patients they selected. So it isn’t really possible for us to consider heterogeneity, is it? Yes, it is true to original authors could be critiqued for not themselves considering this issue in greater detail (I think Reviewer 1 was hoping we’d add that point in a revision). But we can hardly “fix” the trial for them! Statements like this do not place your editors in a good light.

Given your lack of personal expertise and the lack of any real substance in the reviews, your only option was to seek out further information from other experts. You failed to do that.

I also have no doubt that you were aware Reviewer 2 was a close friend and frequent collaborator with the researchers whose trial was being criticised in this article (I see he is well known to the BMJ). You perhaps should have asked yourself whether that person was a good choice in the first place. Sure, it is possible to put aside personal beliefs and alliances to write a review, but its abundantly clear this particular reviewer was unable to do so.

But in any case, I’m not writing now to continue the debate, and I’m not seeking a response. I just want to have these points on record.

On the confidentiality issue, as I said earlier, we will never be make the reviewers’ names public – they will remain confidential. But dated correspondence from BMJ and the reviews themselves will be made available to academics interested in the review process, to share as they see fit. I do hope something good can come out of this, in terms of improving BMJ editorial processes and/or drawing attention to the way reviewer beliefs and biases can operate to silence good quality critical analysis in this area.

CFS is not my usual area of research, but it has been fascinating to see the way in which professional connections and strongly held beliefs amongst a powerful group of UK psychiatrists and psychologists silence high quality debate in this area. We offered the BMJ the opportunity to publish a critical but carefully balanced analysis of the entire trial (it notes the many strengths of the trial, as well as its limitations). But instead of being part of the solution here, the BMJ instead chose to be part of the problem.

Sincerely,

Carolyn Wilshire

 

[Skycloud]

I've been thinking about this and I'm pretty disgusted with the BMJ to be honest, not because they didn't publish the paper, but because they accepted that mean and self indulgent nonsense as a review. I realise that reviewing for journals is done voluntarily and without remuneration, and someone can submit a pompous rant for their own pleasure if they choose instead of critiquing a paper. However the BMJ chose to accept it as one of just 2 reviews. Surely there were legitimate grounds for rejecting it outright because it says nothing relevant about the content, which is what is wanted in a review. A poor job on the part of the BMJ, and I wonder why. It would serve the 'scottish psychiatrist' and the BMJ right if this episode were to become more widely known. I really hope it does.



edit - tone

 

[Esther12]

A big part of the problem for me is that this is far from an isolated example from the BMJ. There's a long history of appalling behaviour and misrepresentations of the evidence relating to ME/CFS, and they seem to embrace it as 'standing up for science', while simultaneously avoiding debate with those patients (and academics) who know better than them.

PS: Hope it was okay to start a new thread on this, was unsure whether greater attention would be a good or bad thing. That review was appalling though.

 

[Dolphin]

I've been thinking about this and I'm pretty disgusted with the BMJ to be honest, not because they didn't publish the paper, but because they accepted that mean and self indulgent nonsense as a review. I realise that reviewing for journals is done voluntarily and without remuneration, and someone can submit pompous wankery if they choose instead of critiquing a paper. However the BMJ chose to accept it as one of just 2 reviews. Surely there were legitimate grounds for rejecting it outright because it says nothing relevant about the content, which is what is wanted in a review. A poor job on the part of the BMJ, and I wonder why. It would serve the 'scottish psychiatrist' and the BMJ right if this episode were to become more widely known. I really hope it does.

Patients writing about their health condition were abused by a peer reviewer and silenced by The BMJ
https://jcoynester.wordpress.com/20...bused-by-a-peer-reviewer-and-silenced-by-bmj/

 

[LightHurtsME]

pompous wankery

A useful phrase!

 

[large donner]

pompous wankery .

Isn't that unbecoming or is it the inclusion of the pompous part that's unbecoming. Or is wankery unbecoming on its own, pardon the pun.

Or is it the combination of the two that's unbecoming? Maybe just being pompous is unbecoming or maybe its a crucial part of becoming wankery.

Who knows, the whole last few days has been .

 

[Graham]

Following on from the debate at Westminster Hall, I wonder whether this should be passed on to the minister (Steve Brine?) as evidence of the sort of difficulty we face in getting the message out there, and of the entrenched opposition in many medical quarters.

 

[Sasha]

Following on from the debate at Westminster Hall, I wonder whether this should be passed on to the minister (Steve Brine?) as evidence of the sort of difficulty we face in getting the message out there, and of the entrenched opposition in many medical quarters.

I certainly think the BMJ deserves some public shaming - it was disgraceful to allow such a 'review' to go forward to the authors, considering the personal abuse it contained.

 

[chrisb]

A big part of the problem for me is that this is far from an isolated example from the BMJ. There's a long history of appalling behaviour and misrepresentations of the evidence relating to ME/CFS, and they seem to embrace it as 'standing up for science', while simultaneously evading debate with those patients (and academics) who know better than them.

PS: Hope it was okay to start a new thread on this, was unsure whether greater attention would be a good or bad thing. That review was utterly appalling though.

I recently came across complaints in the correspondence section of the BMJ in 1993 about their approach to ME or it might have been PVFS. If I rediscover it I will post it. However when following up on that I came across these criticisms by Ellen (EDIT for typo) Goudsmit.

http://www.axfordsabode.org.uk/me/bmj-bias.htm

Chronic fatigue syndrome: bias in the BMJ.
Ellen Goudsmit. PhD. C.Psychol.

Health psychologist and archivist.



Introduction
Chronic fatigue syndrome (CFS) is a common, potentially disabling illness, which carries a substantial socio-economic burden1. The true prevalence of this disorder is unknown but may have been underestimated in the past2. Without doubt, the condition deserves serious consideration both from the research scientists and physicians in clinical practice.

The British Medical Journal (BMJ) is the official organ of the British Medical Association, the largest professional body of physicians in the UK. Readers expect the journal to publish original research on CFS and to keep them up-to-date with developments documented elsewhere. The publication of a broad range of views allows practitioners to make informed decisions and is an essential part of the scientific process. For clinical and epidemiological purposes, patients with CFS are currently defined by the CDC criteria, which were developed by the International study group and modified in 19943.

In September 2000, I reviewed the content of the publications on CFS, which have appeared in the BMJ since 1995 to see if the nature of the papers reflected the global research, clinical opinion and changes in the diagnostic approach in relation to this condition


This is not a new problem.

 

[Daisymay]

Following on from the debate at Westminster Hall, I wonder whether this should be passed on to the minister (Steve Brine?) as evidence of the sort of difficulty we face in getting the message out there, and of the entrenched opposition in many medical quarters.

Well I sent it on to Lady Mar earlier and she may or may not send it on to Carol Monaghan and co, it's up to her, but maybe someone could send it on to Steve Brine?

 

[Daisymay]

This is not a new problem.

No and you can well imagine over the decades, literally decades, how researchers and activists will have been similarly attacked, smeared and invalidated in private to those in positions of power or influence.

 

[Starlight]

I find it hard to believe that the BMJ would publish such waffle and that they would expect people to wade through it is unreal. I find it hard to believe that one of their staff read this and decided to publish it. This deserves all the attention it can get and for all the wrong reasons.

 

[Adrian]

I find it hard to believe that the BMJ would publish such waffle and that they would expect people to wade through it is unreal. I find it hard to believe that one of their staff read this and decided to publish it. This deserves all the attention it can get and for all the wrong reasons.

They didn't publish it it was a peer review for the paper on PACE that was submitted.

 

[Barry]

I certainly think the BMJ deserves some public shaming - it was disgraceful to allow such a 'review' to go forward to the authors, considering the personal abuse it contained.

Valid, providing any lay audience clearly understand what a review of a scientific paper is supposed to be, and not be.

 

[Dolphin]

A possibly interesting angle to this is the BMJ now has open peer review so if the paper had been accepted, the peer review comments would have been made public.

 

[Barry]

A possibly interesting angle to this is the BMJ now has open peer review so if the paper had been accepted, the peer review comments would have been made public.

I appreciate it pulls in a whole raft of issues of its own, but its almost as if reviewer's comments need to be open to public scrutiny if a paper is not published, given that a very unjustified review could prevent publication of a good paper. But like I say, I appreciate the impracticality issues.

 

[Robert 1973]

@Carolyn Wilshire I know this forum is public but are you happy for the review and correspondence to be shared on social media and beyond?

 

[Carolyn Wilshire]

@Carolyn Wilshire I know this forum is public but are you happy for the review and correspondence to be shared on social media and beyond?

Yes. I'm happy to share this material publicly.

Its important to keep in mind that the manuscript being reviewed here was a little different from the final one that got published in BMC Psychology. It was written with a tight word limit in mind, and it included some data from our reanalyses but few details on how we performed the reanalyses.

But ALL the data and key arguments in that manuscript also appeared in the final BMC publication . So the reviewer's criticisms of material in the manuscript are also effectively criticisms of the final BMC publication.

The sentence that contained the word "merely", the one the reviewer claimed was "belittling patients" also appeared in the final version:

The behavioural-deconditioning model, on which the treatments were based, assumes that there is no underlying disease process in CFS, and that patients’ concerns about exercise are merely “fearful cognitions” that need addressing...

 

[Sean]

I recently came across complaints in the correspondence section of the BMJ in 1993 about their approach to ME or it might have been PVFS. If I rediscover it I will post it. However when following up on that I came across these criticisms by Ellen Gouldsmit.

http://www.axfordsabode.org.uk/me/bmj-bias.htm

I recommend reading EG's work from that period (mostly from 1997-2010), archived on Axford's Abode. It is good, and she was almost the only person doing anything at that level at the time.

Also, it is Goudsmit. No 'l'.