The evidence for CBT in any condition, population or context... A meta-review... and panoramic meta-analysis, 2021, Fordham et al.

The evidence for cognitive behavioural therapy in any condition, population or context: a meta-review of systematic review and panoramic meta-analysis by Fordham et al.

Abstract
The majority of psychological treatment research is dedicated to investigating the effectiveness of cognitive behavioural therapy (CBT) across different conditions, population and contexts. We aimed to summarise the current systematic review evidence and evaluate the consistency of CBT's effect across different conditions.

We included reviews of CBT randomised controlled trials in any: population, condition, format, context, with any type of comparator and published in English. We searched DARE, Cochrane, MEDLINE, EMBASE, PsycINFO, CINAHL, CDAS, and OpenGrey between 1992 and January 2019. Reviews were quality assessed, their data extracted and summarised.

The effects upon health-related quality of life (HRQoL) were pooled, within-condition groups. If the across-condition heterogeneity was I2 < 75%, we pooled effects using a random-effect panoramic meta-analysis. We summarised 494 reviews (221 128 participants), representing 14/20 physical and 13/20 mental conditions (World Health Organisation's International Classification of Diseases). Most reviews were lower-quality (351/494), investigated face-to-face CBT (397/494), and in adults (378/494). Few reviews included trials conducted in Asia, South America or Africa (45/494).

CBT produced a modest benefit across-conditions on HRQoL (standardised mean difference 0.23; 95% confidence intervals 0.14–0.33, I2 = 32%). The effect's associated prediction interval −0.05 to 0.50 suggested CBT will remain effective in conditions for which we do not currently have available evidence. While there remain some gaps in the completeness of the evidence base, we need to recognise the consistent evidence for the general benefit which CBT offers.

 

A pity I had not been able to see this before answering questions at NICE committee - i.e. did I think CBT trials were just as rubbish in other conditions (shock horror).

The fact that this review is unable to identify good quality evidence in anything and pick it out and has to bundle everything into a 'works a bit although data are not that good' conclusion makes me pretty confident that CBT is a scam from start to finish. Yes, talking to helpful people may be helpful but CBT as a concept looks to be a complete scam.

 

But strangely they do not mention risk of bias when they discuss the limitation of assessing RCTs at the review level:

This is unfortunate. According to the article, the systematic reviews of RCTs for CBT that met the eligibility criteria were rated using AMSTAR 2 (tool, guidance). The published article on AMSTAR 2 [1] suggests classifying the quality of a review as follows:

With critical domains being those:

Hopefully all of the 143 reviews classified as "higher" quality (AMSTAR 2 moderate or high quality) did employ a tool for analyzing risk of bias from individual studies, and correctly so, as they should per the above. Some, but I suspect many, of the 351 "lower" quality reviews (AMSTAR 2 low and critically low quality) may be suffering from high risk of bias and more issues, at which point one can ask whether they can legitimately be included. As an example, a French meta-analysis of studies (not reviews of RCTs) on hydroxychloroquine for COVID-19 excluded those with a critical risk of bias according to the Cochrane RoB2 tool. [2] Here, unfortunately, the individual AMSTAR 2 ratings are not given, even in the supplementary material, so we can't know more about the critical flaws in the included reviews.

As @Hutan pointed out, biases around CBT are not discussed in the article and the non-superiority to other interventions (active comparators) is not addressed. There is no mention of spin, even though it has been shown to be a problem in psychotherapy trials whether they are registered prospectively or not [3]. Overall, the discussion doesn't show critical thinking with respect to CBT...

Taking these issues together, this meta-review arguably spins its findings:

There are more than just "some gaps" in the evidence base for CBT: the low quality evidence does not support the authors' conclusions. If we consider the AMSTAR 2 critical domains, this review scores low on "Consideration of risk of bias when interpreting the results of the review" and could be considered as low quality.

[1] Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. Published 2017 Sep 21. doi:10.1136/bmj.j4008

[2] Fiolet T, Guihur A, Rebeaud ME, Mulot M, Peiffer-Smadja N, Mahamat-Saleh Y. Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis. Clin Microbiol Infect. 2021;27(1):19-27. doi:10.1016/j.cmi.2020.08.022

[3] Stoll M, Mancini A, Hubenschmid L, et al. Discrepancies from registered protocols and spin occurred frequently in randomized psychotherapy trials-A meta-epidemiologic study. J Clin Epidemiol. 2020;128:49-56. doi:10.1016/j.jclinepi.2020.08.013

 

hmm what is happening in practice (IAPT) would suggest otherwise

"No trial of CBT has ever been conducted with the average dosage delivered in IAPT."

https://www.s4me.info/threads/uk-im...ies-iapt-articles-blogs-and-discussion.14318/

 

I might write to Edzard Ernst and ask him why he doesn't add it to his SCAM (so-called alternative medicines) list as the evidence for CBT is just as bad as for any of the treatments he bangs on about. He has ignored my hints to that effect so far...and I think claims lack of expertise in non-SCAM (ie. mainstream medicine) means he can't possibly comment. Not sure. It's annoying.

 

Yes, it's a good thing that there are all these things to get annoyed about. Otherwise lockdown would get pretty dull.

 

And another thing....what the hell is a "panoramic meta-analysis"?

 

It's what greater adjutant storks do on the rubbish tip at Guwahati.

 

Rather than providing evidence to support the effectiveness of CBT, I think the authors have accidentally done just the opposite.

 

For some patients, a treatment that does nothing might very well be far better than the other treatments.

Maybe even better would be to stop expecting patients who have health problems that nobody understands to do everything to recover.

 

Confirms what has been obvious since... forever: that CBT is an instrument that can be used to slightly change answers on questionnaires, with greater "efficacy" on changing answers by crafting deliberately vague, non-specific questions and the more rigorous methodology is, the smaller the effect, all the way down to indistinguishable from the natural noise one gets when using subjective ratings on questionnaires that are deliberately ambiguous. After all, how precise and accurate is a 7/10 rating on a tomato sauce? Very open to manipulation by underheating it, microwaving it at max temp for too long or adding too much salt or simply not adding any. There is no right answer, so no way to ever actually confirm anything, make-believe built on more make-believe.

In addition to doing obvious manipulation of results by arbitrarily using larger numbers to give the illusion of a great effect, an example of which is the touting of the 50-point increase on the IBS scale for the CBT for IBS study, never hinting at the fact that on a 500 point scale it means a less than 10% impact, something obviously not significant that was still touted as such.

It's basically institutionally-backed snake oil, built on quantity of evidence with no concern for quality, deliberate attempts to use bias in a way to amplify false effects and a general state of denial that this is all obviously an illusion caused by the imprecision that comes from not using valid measurements.

Fitting that while this review obviously destroys all pretense that any of this is legitimate, of course people like Howard actually pretend it says the opposite. Escalation of commitment is unforgiving, building up an obvious lie means the lying has to keep on lying. And of course those are the same people, like Greenhalgh and Sharpe, who routinely write and tweet about not falling for exactly this bias, which they selectively support or not depending entirely on their own perception.

This is anti-science, the textbook opposite of science. It will eventually be recognized as such, which makes it all that much more ridiculous.

 

In a way, these studies test certain assumptions common in CBT, for example that the problems of the patient are due to faulty beliefs. Maybe beliefs have simply little to do with whatever problems patients have.

 

That's a very good point. I can think of lots of people with faulty beliefs who seem to do pretty well.

Luciano Pavarotti had a faulty belief that he was not a very good singer, for instance.

 

Maybe he became a good singer because he often felt inadequate.

 

I have a close relative with anxiety who has attended CBT group therapies (which is the first line of treatment offered by a major US health plan) and she has completed workbooks in CBT and has also diligently attended groups in one of CBT's contemporary offshoots.

She has spent several years doing this, and her anxiety levels continue to be high (on onset of several years past now).

If CBT were effective on anxiety, I'm sure I would know. The phone calls would diminish and her decision-making would cease to be a world crisis.

 

It annoys me that skeptics like https://sciencebasedmedicine.org/make lists of fake illness / fake diagnosis yet go on to claim illnesses like ME are somatization or psychosomatic. It's as if as soon as psychogenic attribution rears it's ugly head, science and skepticism go out the window leaving different standards to be applied.

How is somatizing any less fake than Chronic Lyme or Electromagnetic Hypersensitivity? Where is the evidence for it? Why do they never weigh up the pros and cons of psychogenic attribution?