1. It is interesting that Paul Garner chose to praise this particular study, as it has many of the flaws that are commonly seen in psychosomatic and alternative therapies research. NICE recently rated the outcomes of research with these flaws as being of low or very low quality.
2. Paul Garner says that "changing beliefs cured chronic back pain". By the end of the intervention, use of alcohol and opioids had increased in the group with the 'changing beliefs' treatment. They were still using, on average, over 4 times the amount of cannabis that the people in the control treatment cohort were using.
3. The researchers fervently believed in the 'changing beliefs' treatment. There was no state of equipoise.
4. The research was funded by the 'Study of the Therapeutic Encounter Foundation', which has
a website that appears to have been hastily created and is not finished. The 'About Me' section says 'I am a paragraph. Click here to add your own text'. The research was also funded by the Psychophysiologic Disorders Society. Authors of the study went on to become consultants to United Health Group, a very large health care and health insurance company that has a substantial financial interest in reducing claims by people with chronic back pain.
5. Trial participants were recruited in a way that made it much more likely that they were favourably disposed to the idea that changing their beliefs would reduce their pain.
6. There were no reliable objective outcomes. It is a fundamental flaw in experimental design when unblinded treatments are combined with only subjective outcomes. There is a huge body of evidence showing that multiple biases combine to make the findings of such flawed studies worthless in assessing treatment effectiveness. This is particularly true when combined with participants who believe the treatment will work, and with researchers who are incentivised to find that the treatment works. It is even more true when the treatment involves learning to deny having pain and the outcome is the answer to the question 'so, how much pain do you have?'.
7. The people in the 'changing beliefs' group were more likely to be in full time employment prior to the treatment than the people in the other two treatment arms, so they had a better chance of reducing work hours in order to reduce pain. There is no report of hours worked by the end of the treatment, or at followup.
8. The study claims that gains were largely maintained at followup. However, the protocol suggested that participants in the control treatments would be given access to instruction in 'unlearning their pain'. Therefore, the followup results are actually largely uninterpretable.
8. The protocol suggested that participants would use an app to record opioid use for collection at follow-up timepoints because 'We are interested in medications most specifically because of the current opioid crisis, and the potential of the mind-body treatments under investigation here to reduce opioid use'. In an echo of the PACE researchers abandoning activity monitors supposedly because they were too burdensome for the trial participants, this study reported regarding measures of the use of alcohol, opioids and cannabis that 'these measures were not collected at follow-up timepoints to reduce the burden on patients'.
9. Multiple interpretations can be applied to fMRI data and, with a large number of variables, the analysis is particularly susceptible to cherry-picking and researcher bias.
10. Even if it really was true that chronic back pain can be cured by changing beliefs, that does not mean that other health conditions would respond in the same way.
11. It is particularly concerning that Paul Garner, who works in medical evidence analysis and has a role in Cochrane, seems to think that this study was useful. The problem created by the combination of unblinded treatments, a lack of objective outcomes and researcher bias is one that even students of experimental design should be able to identify.
