Blog: Hilda Bastian, "A Classic Case of Science “He Said”, “She Said”: How Psychologists Trying to Prevent PTSD Got Controversial"

[Andy]

A familiar issue..

While several factors play a role in different review conclusions, the conclusion about harm in the first Cochrane review depended mostly on a single trial that had extensive weaknesses. That’s a risky thing to rely on.

That one trial got good quality ratings by the Cochrane reviewers. But that decision-making including one of the trial’s authors, who was an author of both the Cochrane reviews concluding psychological intervention can do harm. That’s a red flag, and I’ll write more about this issue in my next post.

It was far from being a reliable study, though. As I wrote here, it suffered from several serious biases. The group getting debriefed was much larger (64 versus 46 people) because of the way they randomized and because they stopped the trial early. (I’ve written more about the mess trials stopped early can leave behind here.)

The 2 comparison groups ended up different as a result – including being at higher risk of PTSD. And there was a high attrition rate (>22%): 7 people left the hospital before debriefing and 23 were lost to follow-up. That’s enough bias to materially tilt the results.

What’s more, the number of “events” – people who got PTSD – was low in both the trial’s groups. That’s what makes a study too small to reliably assess differences between groups.

What about the finding of harm in the second Cochrane review on multiple sessions for prevention? That was even weaker: there was no statistically significant evidence of an increase in PTSD. There was just something the author’s called a trend towards harm in one measure only – self-reported symptoms, not clinical assessment of PTSD. And the 2 studies carrying the weight of this “trend” were fraught with problems. For example, both trials also had issues in how people were recruited that were so severe, there was imbalance in the comparison groups – 82 versus 65 people in one, and 66 versus 83 in the other.

https://blogs.plos.org/absolutely-m...sts-trying-to-prevent-ptsd-got-controversial/

 

[Esther12]

A familiar Wessely...

So why did this research and the conclusion about possible harm become so controversial? Is it just because people shot the messenger when they didn’t like the message? That’s definitely what at least one of the first review’s authors want us to believe [PDF]. I don’t think so. I think the Cochrane reviews are seriously problematic. They were at high risk of causing what I once called evidence-based mistakes – where we are led astray by what claims to be strong evidence, but isn’t. In this case, it raises questions, I think, about author bias.

I have to admit to an instinctive bias against debriefing (which I can view as an interference in peoples' lives by professionals who often don't seem to really know what they're doing). I've always taken those Cochrane reviews at face value, probably partly because of that.

 

[Jonathan Edwards]

I have to admit to an instinctive bias against debriefing

I initially took that to be a comment on Bastian's 'debriefing' of us on the 'debriefing' provided by Cochrane on the studies of debriefing of trauma victims. When reading the piece I started to wonder what hidden agendas Bastian herself might have. But the message is clear enough - another instance of received wisdom based on pretty uninterpretable data.