1. The latest crowdfunder to support David Tuller's work has opened. To donate click here.
    Dismiss Notice
  2. Guest, the two part 'News in Brief' for the week beginning 11th October 2021 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

The evidence for CBT in any condition, population or context... A meta-review... and panoramic meta-analysis, 2021, Fordham et al.

Discussion in 'Health News and Research unrelated to ME/CFS' started by Kalliope, Jan 18, 2021.

  1. Kalliope

    Kalliope Senior Member (Voting Rights)

    Messages:
    4,586
    Location:
    Norway
    The evidence for cognitive behavioural therapy in any condition, population or context: a meta-review of systematic review and panoramic meta-analysis by Fordham et al.

    Abstract
    The majority of psychological treatment research is dedicated to investigating the effectiveness of cognitive behavioural therapy (CBT) across different conditions, population and contexts. We aimed to summarise the current systematic review evidence and evaluate the consistency of CBT's effect across different conditions.

    We included reviews of CBT randomised controlled trials in any: population, condition, format, context, with any type of comparator and published in English. We searched DARE, Cochrane, MEDLINE, EMBASE, PsycINFO, CINAHL, CDAS, and OpenGrey between 1992 and January 2019. Reviews were quality assessed, their data extracted and summarised.

    The effects upon health-related quality of life (HRQoL) were pooled, within-condition groups. If the across-condition heterogeneity was I2 < 75%, we pooled effects using a random-effect panoramic meta-analysis. We summarised 494 reviews (221 128 participants), representing 14/20 physical and 13/20 mental conditions (World Health Organisation's International Classification of Diseases). Most reviews were lower-quality (351/494), investigated face-to-face CBT (397/494), and in adults (378/494). Few reviews included trials conducted in Asia, South America or Africa (45/494).

    CBT produced a modest benefit across-conditions on HRQoL (standardised mean difference 0.23; 95% confidence intervals 0.14–0.33, I2 = 32%). The effect's associated prediction interval −0.05 to 0.50 suggested CBT will remain effective in conditions for which we do not currently have available evidence. While there remain some gaps in the completeness of the evidence base, we need to recognise the consistent evidence for the general benefit which CBT offers.

     
    Last edited by a moderator: Jan 19, 2021
    sebaaa, rvallee, Barry and 6 others like this.
  2. Kalliope

    Kalliope Senior Member (Voting Rights)

    Messages:
    4,586
    Location:
    Norway
  3. Hutan

    Hutan Moderator Staff Member

    Messages:
    16,501
    Location:
    New Zealand
    You would think, given the extremely high rate of low quality reviews, that there might have been some comment about what was wrong with the reviews, or some comment about how future reviews might be done. But, no.

    There was nothing at all said about the reliance on subjective outcomes with interventions that are extremely hard to blind in the trials.
    But this is telling:
    When CBT is compared to something like a wait list, the benefit looks quite good - SMD 0.31
    When CBT is compared to anything else that might include a sympathetic therapist, an expectation of benefit, and/or group support, the benefit looks much smaller - SMD 0.09


    It doesn't seem to matter what sort of CBT is done - face to face sessions or online DIY courses. It all miraculously produces the same effect in the aggregate.


    The only mention of adherence and dropouts is when they are talking about doing more research to increase the benefit of CBT. There's no discussion about how high rates of dropouts might invalidate results.


    There was no comment about harms, no mention of the possibility that the very modest reported benefit for those participants who stuck with the treatment long enough to fill out the questionnaires might be more than outweighed by the harm done to those people who dropped out, scarred by the experience of being told that their way of looking at the world is flawed, or feeling like failures because they could not think their way out of their condition.
     
    Last edited: Jan 19, 2021
  4. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    10,043
    A pity I had not been able to see this before answering questions at NICE committee - i.e. did I think CBT trials were just as rubbish in other conditions (shock horror).

    The fact that this review is unable to identify good quality evidence in anything and pick it out and has to bundle everything into a 'works a bit although data are not that good' conclusion makes me pretty confident that CBT is a scam from start to finish. Yes, talking to helpful people may be helpful but CBT as a concept looks to be a complete scam.
     
    Chezboo, sous-bois, sebaaa and 30 others like this.
  5. cassava7

    cassava7 Senior Member (Voting Rights)

    Messages:
    424
    But strangely they do not mention risk of bias when they discuss the limitation of assessing RCTs at the review level:
    This is unfortunate. According to the article, the systematic reviews of RCTs for CBT that met the eligibility criteria were rated using AMSTAR 2 (tool, guidance). The published article on AMSTAR 2 [1] suggests classifying the quality of a review as follows:
    With critical domains being those:
    Hopefully all of the 143 reviews classified as "higher" quality (AMSTAR 2 moderate or high quality) did employ a tool for analyzing risk of bias from individual studies, and correctly so, as they should per the above. Some, but I suspect many, of the 351 "lower" quality reviews (AMSTAR 2 low and critically low quality) may be suffering from high risk of bias and more issues, at which point one can ask whether they can legitimately be included. As an example, a French meta-analysis of studies (not reviews of RCTs) on hydroxychloroquine for COVID-19 excluded those with a critical risk of bias according to the Cochrane RoB2 tool. [2] Here, unfortunately, the individual AMSTAR 2 ratings are not given, even in the supplementary material, so we can't know more about the critical flaws in the included reviews.

    As @Hutan pointed out, biases around CBT are not discussed in the article and the non-superiority to other interventions (active comparators) is not addressed. There is no mention of spin, even though it has been shown to be a problem in psychotherapy trials whether they are registered prospectively or not [3]. Overall, the discussion doesn't show critical thinking with respect to CBT...

    Taking these issues together, this meta-review arguably spins its findings:
    There are more than just "some gaps" in the evidence base for CBT: the low quality evidence does not support the authors' conclusions. If we consider the AMSTAR 2 critical domains, this review scores low on "Consideration of risk of bias when interpreting the results of the review" and could be considered as low quality.

    [1] Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. Published 2017 Sep 21. doi:10.1136/bmj.j4008

    [2] Fiolet T, Guihur A, Rebeaud ME, Mulot M, Peiffer-Smadja N, Mahamat-Saleh Y. Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis. Clin Microbiol Infect. 2021;27(1):19-27. doi:10.1016/j.cmi.2020.08.022

    [3] Stoll M, Mancini A, Hubenschmid L, et al. Discrepancies from registered protocols and spin occurred frequently in randomized psychotherapy trials-A meta-epidemiologic study. J Clin Epidemiol. 2020;128:49-56. doi:10.1016/j.jclinepi.2020.08.013
     
    Last edited: Jan 19, 2021
    Chezboo, sebaaa, TrixieStix and 11 others like this.
  6. Kalliope

    Kalliope Senior Member (Voting Rights)

    Messages:
    4,586
    Location:
    Norway
  7. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    6,852
    Location:
    UK
    sebaaa, alktipping, MEMarge and 4 others like this.
  8. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    441
    Location:
    Oxford UK
    I might write to Edzard Ernst and ask him why he doesn't add it to his SCAM (so-called alternative medicines) list as the evidence for CBT is just as bad as for any of the treatments he bangs on about. He has ignored my hints to that effect so far...and I think claims lack of expertise in non-SCAM (ie. mainstream medicine) means he can't possibly comment. Not sure. It's annoying.
     
    Chezboo, Sly Saint, sebaaa and 14 others like this.
  9. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    10,043
    Yes, it's a good thing that there are all these things to get annoyed about. Otherwise lockdown would get pretty dull.
     
  10. Caroline Struthers

    Caroline Struthers Senior Member (Voting Rights)

    Messages:
    441
    Location:
    Oxford UK
    And another thing....what the hell is a "panoramic meta-analysis"?
     
  11. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    10,043
    It's what greater adjutant storks do on the rubbish tip at Guwahati.
     
    Chezboo, sebaaa, TrixieStix and 11 others like this.
  12. Michiel Tack

    Michiel Tack Senior Member (Voting Rights)

    Messages:
    2,661
    Location:
    Belgium
    Rather than providing evidence to support the effectiveness of CBT, I think the authors have accidentally done just the opposite.
     
  13. strategist

    strategist Senior Member (Voting Rights)

    Messages:
    3,880
    For some patients, a treatment that does nothing might very well be far better than the other treatments.

    Maybe even better would be to stop expecting patients who have health problems that nobody understands to do everything to recover.
     
  14. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    7,489
    Location:
    Canada
    Confirms what has been obvious since... forever: that CBT is an instrument that can be used to slightly change answers on questionnaires, with greater "efficacy" on changing answers by crafting deliberately vague, non-specific questions and the more rigorous methodology is, the smaller the effect, all the way down to indistinguishable from the natural noise one gets when using subjective ratings on questionnaires that are deliberately ambiguous. After all, how precise and accurate is a 7/10 rating on a tomato sauce? Very open to manipulation by underheating it, microwaving it at max temp for too long or adding too much salt or simply not adding any. There is no right answer, so no way to ever actually confirm anything, make-believe built on more make-believe.

    In addition to doing obvious manipulation of results by arbitrarily using larger numbers to give the illusion of a great effect, an example of which is the touting of the 50-point increase on the IBS scale for the CBT for IBS study, never hinting at the fact that on a 500 point scale it means a less than 10% impact, something obviously not significant that was still touted as such.

    It's basically institutionally-backed snake oil, built on quantity of evidence with no concern for quality, deliberate attempts to use bias in a way to amplify false effects and a general state of denial that this is all obviously an illusion caused by the imprecision that comes from not using valid measurements.

    Fitting that while this review obviously destroys all pretense that any of this is legitimate, of course people like Howard actually pretend it says the opposite. Escalation of commitment is unforgiving, building up an obvious lie means the lying has to keep on lying. And of course those are the same people, like Greenhalgh and Sharpe, who routinely write and tweet about not falling for exactly this bias, which they selectively support or not depending entirely on their own perception.

    This is anti-science, the textbook opposite of science. It will eventually be recognized as such, which makes it all that much more ridiculous.
     
  15. Hutan

    Hutan Moderator Staff Member

    Messages:
    16,501
    Location:
    New Zealand
    Actually I could be quite easily persuaded to believe that CBT does have some uses - in conditions where the world might in fact be being viewed unhelpfully e.g. aggression, anxiety disorders and phobias. And, in Figure 2 of this meta-review, reviews of studies of anxiety disorders do appear to report slightly more positive outcomes than the rest. Phobias aren't mentioned and CBT was reported to have no impact on aggression.

    Yes, it's quite remarkable that, even with all of the biases and poor quality, the benefits for virtually all of the conditions appear so trivial.
     
    Last edited: Jan 19, 2021
  16. strategist

    strategist Senior Member (Voting Rights)

    Messages:
    3,880
    In a way, these studies test certain assumptions common in CBT, for example that the problems of the patient are due to faulty beliefs. Maybe beliefs have simply little to do with whatever problems patients have.
     
  17. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    10,043
    That's a very good point. I can think of lots of people with faulty beliefs who seem to do pretty well.

    Luciano Pavarotti had a faulty belief that he was not a very good singer, for instance.
     
  18. strategist

    strategist Senior Member (Voting Rights)

    Messages:
    3,880
    Maybe he became a good singer because he often felt inadequate.
     
    alktipping, Peter Trewhitt and FMMM1 like this.
  19. shak8

    shak8 Senior Member (Voting Rights)

    Messages:
    1,157
    Location:
    Northen California
    I have a close relative with anxiety who has attended CBT group therapies (which is the first line of treatment offered by a major US health plan) and she has completed workbooks in CBT and has also diligently attended groups in one of CBT's contemporary offshoots.

    She has spent several years doing this, and her anxiety levels continue to be high (on onset of several years past now).

    If CBT were effective on anxiety, I'm sure I would know. The phone calls would diminish and her decision-making would cease to be a world crisis.
     
  20. DigitalDrifter

    DigitalDrifter Senior Member (Voting Rights)

    Messages:
    512
    It annoys me that skeptics like https://sciencebasedmedicine.org/make lists of fake illness / fake diagnosis yet go on to claim illnesses like ME are somatization or psychosomatic. It's as if as soon as psychogenic attribution rears it's ugly head, science and skepticism go out the window leaving different standards to be applied.

    How is somatizing any less fake than Chronic Lyme or Electromagnetic Hypersensitivity? Where is the evidence for it? Why do they never weigh up the pros and cons of psychogenic attribution?
     

Share This Page