Caroline Struthers' letter to Cochrane Governing Board

Presumably the only potentially hopeful sign was that Larun felt there was a need to leak to the Reuter's journalist that Cochrane were considering withdrawing in some fashion this review, and to brief against them with the false 'bullying patients' narrative.

Presumably Larun and the BPS crew were feeling threatened in some way, given putting this all in the public domain was not without risk to them.

 

Seems this type of fiasco is right up her ally of work.

 

This could be based on Cochrane deciding to take at face value the authors' estimate of when the imminent resubmission would be made. If so this leaves plenty of scope for footdragging and Cochrane being surprised that the resubmission is taking longer than the authors' initially estimated.

 

Looks like the review was updated today, after 5:30 -

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub7/information#history

 

This should probably be copied to the main Cochrane thread too?

 

https://twitter.com/user/status/1082258944930865158

 

I'm sure this is the kind of reassuring talk people want to hear from a medical review organisation. That will go down well with their policy that it's perfectly fine for researchers to review their own work and grade it as flawless and perfect and there's nothing wrong with that.

Found a new slogan for Cochrane: objectivity is overrated. Alternatively: because it feels true.

Literally centuries of scientific lessons flushed down the toilet. Make phrenology great again?

 

I think we need to work on how to convince people of the how flawed it is to combine subjective outcomes with unblinded trials. There is clearly a good deal of convincing still to do. I seem to recall @Jonathan Edwards saying some time back on PR that quite a few people took a lot of convincing initially when he raised the point. Coming to all this from scratch, once pointed out it seemed terribly obvious to me and still does. But I think there is a lot of inertia in establishment mind set that really doesn't see otherwise. How to change that. For psych therapies aimed at genuine psych conditions, the notion may hold some water, but when targeted toward physical conditions with objective outcomes available, it's junk. Something for 2019.

 

https://healthycontrolblog.wordpres...iew-of-exercise-for-chronic-fatigue-syndrome/

 

My quick thoughts:

1) Tovey confirms that Cochrane planned to temporarily withdraw the Larun et al. GET review because the authors did not adequately respond to criticism in time. This was what the Reuters article was about. "A subsequent meeting with two of the authors” made the Cochrane editors change course.

Yet Toveys reponse suggests that personal reasons were involved in the authors failure to respond in time and he says that Cochrane only wanted to withdraw the review temporarily, "to give them time without pressure to address the issues raised fully." I suppose it's possible that one of the authors had personal reasons (sickness or deceased family member or something like that) that made it difficult to respond in time and that the Cochrane editors underestimated this.

2) Tovey says that currently the process of re-submitting, reassessment and peer review is pending, but it is not clear what this means as the authors have already offered a re-submission that was deemed unfit. To me it seems that the authors are allowed to make lots of mistakes and do-overs. Are they now doing a re-re-submission? Is there going to be a re-re-resubmission?

3) The most controversial point in Tovey’s response concerns the issue of “using only subjective outcomes in unblindable trials”. “To describe this as ‘terrible methodology’ is simply an opinion”, Tovey says, “and it is not shared by independent methodologists who we consulted". Weird.

Tovey also claims that “objective measures such as resource use were reported when they were identified.” That doesn’t seem right. As Vink showed objective outcomes mostly showed null results and were not considered in the Cochrane assessment of GET. https://journals.sagepub.com/doi/full/10.1177/2055102918805187

 

Surely the point about subjective trials is that there is HIGH BIAS in them, and the Cochrane review says it's low bias. Do the independent methodologists consider it low or high?
What is the current thinking on bias in subjective trials among psychologists @Brian Hughes ?

 

Haven't been able to read the blogpost yet - but this is more or less exactly what is the current mission statement (or something like that) for Cochrane now.

 

It seems to me that the response to point 3 regarding subjective measures in unblinded trials focussed only on the subjective measures and did not consider the consequences of combining that with lack of blinding. I think this is the strongest point with the weakest response. It should be made again but stronger with references to eliminate the "it's just an opinion" rebuttal.

 

Low reading ability-day today, so still haven't read the bofpost, just some tweets. But really surprised that Tovey would actually put that bit about - "saying that reporting only subjective outcomes in unblindable trials is "terrible methodology" is just an opinion" - in writing. And referencing to some un-named "methodology expert".

To Struthers?

I'm glad he did though, even if it makes me nervous about the fate of the ME exercise reviews. Because this will be so obvious wrong (or so I'm hoping....), that it's getting the attention of others.

https://twitter.com/user/status/1082383175853064192

 

Hilda Bastian is a well respected skeptic who has a blog on Plos One about science and research methodology. Would be great if she got involved as well.

 

Yeah, this response makes me think that Cochrane is even worse than I'd feared and leaves me dreading whatever they're going to do next. So many annoying and misguided things about it. I'm going to have to re-read it with lowered expectations so I can avoid being distracted by my eyes rolling back into my head.