Reanalysis of STAR*D trial (PACE-like outcome switching scandal)

The largest and most impactful trial of depression of all time has now been reanalysed using individual participant data obtained from the NIMH. The STAR*D trial was originally published in the American Journal of Psychiatry in 2006 and has amassed over 5,000 citations. It cost the US taxpayers 35 million USD.

The study purported to show that following four sequential (and increasingly aggressive) treatment steps, the cumulative remission rate is 67%. These findings have become accepted wisdom in psychiatry and pretty much every paper on depression cites this trial.

A new reanalysis of protocol-specified outcome found that the investigators switched the outcome from the blinded observer-rated Hamilton Depression Rating Scale to the unblinded self-reported QIDS, a novel (at the time) scale that the investigators made up. This resulted in near doubling of the published remission rate (protocol-stipulated rate 35% vs 67% in the outcome-switched published paper).

Another bit that made me chuckle and reminded me of PACE is that, as a result of outcome switching, they ended up including in the analysis patients who already met Hamilton remission criteria before the treatment.

Cherry on top:

At least these authors are keeping their mouths shut instead of making a spectacle of themselves like White, Chalder et al with their incessant wailing and flailing.

https://bmjopen.bmj.com/content/13/7/e063095

 

Well well well.
My oh my.
Goodness gracious me.

Depression already.
No more of this flimflamary!!!

 

Interesting, thanks for sharing.

The original trial had severe limitations though. This paper writes: "To mimic clinical practice, STAR*D used an open-label research design with no control group during any phase of the study."

 

At this point, it's probably guaranteed that this is accepted standard practice. And explains the mass indifference to doing the same thing with PACE. If they don't cheat, they have nothing. So they always do. The psychiatric discipline produces almost no effective outcomes outside of sedating people, or otherwise blunting their senses. Or simply lying about producing effective outcomes, and somehow getting away with it.

If the star trial of the profession features this much cheating, and we've seen it everywhere for years, it really is all they've got, and they simply can't accept this much failure. So they just keep covering up the same way, and think nothing of doing it again, it's how they always do it anyway.

I'd even suggest that the reanalyzed rates here are still highly inflated, simply because of all the bias inherent in how they rate outcomes, then how they evaluate them, then how they review and summarize them in guidelines. They're able to take a tiny 1% statistical significance of dubious origin and inflate it into a 100% surefire cure. It's a hype profession for the most part. Not entirely, but for the most part. They have an important mission to address, but have simply always been massively out of their depth.

I'm pretty sure that similar reanalysis of most drug trials, especially antidepressants, would yield similarly distorted outcomes. And it's not surprising, medicine works best when they understand the biology and target it directly. The further they are from understanding both the pathology and the drug to target it, the worse it is. And in depression, which has no test and no known pathology, just as with ME, they understand neither and mostly make stuff up instead.

It's really absurd how this stuff, especially psychosomatics, is invading all the nooks and crannies of medicine with their BS about MUS, or FND or BDD, or whatever. They literally can't tell when their expertise applies, and somehow even though they deliver almost nothing of value, they always want more. It's so damn weird, and clearly acts as a release valve for the massive inadequacies of medicine. Especially for liability and legal obligations. Just declare the patient insane, liability waived entirely. Problem buried alive.

More than anything, though, what they have done is block progress. And that is immoral. Fully and entirely immoral, and corrupt to a level that is simply impossible to believe for anyone who isn't familiar with it. It's not as if it's well-known that the brain is the most complicated organ in the body. And still they have the hubris, the utter shamelessness, to pretend that they can claim to whenever convenient. It's simply too big a scandal to blow up, it threatens the very credibility of all medicine, which is entirely complicit in allowing this.

Worth looking into, @dave30th.

 

What is RIAT?

https://restoringtrials.org/whatisriat/

eta:
now if only someone could get hold of the full data...
eta2: for the PACE trial.

 

Raters were blinded and were administering the Hamilton Depression Rating Scale. These blinded ratings were supposed to be used as the primary outcome. However, the investigators switched to an unblinded self-rated scale in their trial report.

There was no placebo control group and the patients were not blinded. In step 2 of the trial, CBT was one of the arms so blinding wouldn’t have been possible anyhow.

 

Yes, the 35% remission rate includes passage of time, regression toward the mean, “placebo effect” and various biases. As there was no placebo control group, we have no idea how the 35% remission rate compares to doing nothing or howling at the moon or taking a sugar pill.

 

Full data for what? The reanalysis is based on the original dataset which NIMH handed over.

 

sorry I was thinking of the PACE trial data and forgot to write it in my post.

 

Ah I see. To be honest, the limited dataset that’s out there already following FOIA victory is enough to establish that the treatments don’t work.

 

Wikipedia t:ldr for the original paper https://en.wikipedia.org/wiki/STAR*D

 

Oof. It looks exactly as bad as PACE. Seriously, this is standard operating practice and it's why we're getting nowhere denouncing it for us. They need to cheat to produce fake outcomes:

I have a feeling that anyone looking closely enough would find even more issues with it. And likely the response from institutions and MDs would be to band together in defense. There is clearly a huge bunker mentality applying here that views criticism of this kind of work as invalid, simply because it's indefensible and the only response is to attack critics and DARVO all the way.

It's going to be extremely revealing once AIs can automate this kind of work and reveal just how research in medical psychology is. We already know how excessively bad it is, and it's even worse. The whole discipline of psychiatry is in for a very rough few years, and they will deserve every bit of it.

 

yes, but the ongoing saga of the 'full data now being available' continues and yet (afaik) no one who has applied appears to have been deemed 'eligible' to retrieve it even
though the cat is out of the bag thanks to the FOI data.

(Who knows what else might be lurking in the data that we don't know about.)

 

I wonder if anyone in the depression community is going to campaign for retraction or at least a corrigendum. This is as shocking as PACE but it affects many more people because depression is vastly more common than ME/CFS.

 

Most of the data for SSRIs come from large drug trials. I doubt this trial actually has that much influence, frankly. Maybe in the psychiatric profession it's held up as some powerful evidence for what they do, but it's very small compared to the many large drug trials and systematic reviews. Hundreds of millions have taken SSRIs already. 4K people is nothing compared to this. And most MDs feel that they work, and that's even more influential.

I also don't think there is such a thing as the depression community. Lots of people who may or may not have something labeled as such, but little in the way of organized advocacy. This isn't something patients should be doing, however, and probably wouldn't succeed at. We debunked PACE and it made no real difference.

Depression in general terms (which includes the reckless application to us) is so much larger than us, it's hundreds of billions in drug revenues per year, lots and lots of MDs, way more than with us, who have been dispensing them for decades and genuinely think it would be disastrous to acknowledge basic facts like this. That it would scare people away from them. The strong belief is that it's worth giving them to 100x as many people as need it on the off chance that a few are helped. And it makes sense to them since they deny all the side effects, reattribute them to some psychological reaction, or whatever.

Really, I don't see how any of this is resolved until AIs automate the work of reviewing studies, and rate studies like this as worthless, or worse. It has to come from authority, and patients have basically negative authority here. Once AIs become a trusted authority, it will come quickly. It will be a sight to behold, and for sure it will happen much faster than any reaction to this reanalysis, because no one actually has to do anything here, there's no authority forcing anyone to.

 

The STAR*D Scandal: Scientific Misconduct on a Grand Scale
https://www.madinamerica.com/2023/09/the-stard-scandal-scientific-misconduct-on-a-grand-scale/

That happened to be the same year that psychologist Ed Pigott and colleagues published their deconstruction of the STAR*D trial. Pigott had filed a Freedom of Information Act request to obtain the STAR*D protocol and other key documents, and once he and his collaborators had the protocol, they were able to identify the various ways the NIMH investigators had deviated from the protocol to inflate the remission rate. They published patient data that showed if the protocol had been followed, the cumulative remission would have been 38%. The STAR*D investigators had also failed to report the stay-well rate at the end of one year, but Pigott and colleagues found that outcome hidden in a confusing graphic that the STAR*D investigators had published. Only 3% of the 4041 patients who entered the trial had remitted and then stayed well and in the trial to its end.
...
The protocol violations and publication of a fabricated “principal outcome”—the 67% cumulative remission rate—are evidence of scientific misconduct that rises to the level of fraud. Yet, as Pigott and colleagues have published their papers deconstructing the study, the NIMH investigators have never uttered a peep in protest. They have remained silent, and this was the case when Pigott and colleagues, in August of this year, published their latest paper in BMJ Open. In it, they analyzed patient-level data from the trial and detailed, once again, the protocol violations used to inflate the results. As BMJ Open wrote in the Rapid Responses section of the online article, “we invited the authors of the STAR*D study to provide a response to this article, but they declined.”
...
Thus, given the documented record of scientific misconduct, in the largest and most important trial of antidepressants ever conducted, there is only one conclusion to draw: In American psychiatry, scientific misconduct is an accepted practice.​

IMO none of this is especially egregious compared to what we've seen in the last few years in psychosomatic medicine. For sure PACE was far more fraudulent than this, but those are clearly standard practices and not just in psychiatry, but about anything that can be blamed on mental illness. I commonly see worse than this, the main difference is the scale. The cost and hype around STAR*D are similar to why PACE is so harshly defended.

Small trials used to justify decisions for years are less embarrassing when 97%+ of them simply get filtered out as being too awful to even consider. Even though by cost and numbers of patients they add up to far more, also true in years wasted. But they can be endlessly funded because the whole system is broken and can't stop doing harm, since admitting to doing harm is the worse thing that can happen. Harm is fine, admitting it is forbidden.

The fraud seen in STAR*D is terrible, of course, but this is indeed standard practice, including in anything that falls under clinical psychology, and even worse in non-pharmaceutical interventions. It's easy to see why pieces like PACE can't fall, the whole structure of mental health care is tied together with fraud and deceit on a mass scale.

Including the common framing of equating antidepressant prescriptions with cases of mental illness, even though they are commonly prescribed for any physical illness that can't be easily/cheaply figured out, and how more often than not, most "antidepressant" drugs are not ceased, even if by then natural remission could have occurred, since withdrawal symptoms are not only awful, but also denied on a mass scale.

Obviously the pharmaceutical industry is a major player in this, they made trillions out of this fraud, but the medical profession as a whole deserves very strong condemnation for doing the equivalent of parents getting their toddlers drunk just so they can have some peace and quiet. At its core, the very notion of "do no harm" is just as fraudulent, the premise is simply a giant lie.