Effectiveness of distant healing for patients with chronic fatigue syndrome, 2008, Walach et al

This post has been copied from a petitions thread; five following posts have been moved from that thread.

off topic/ fun fact: this trial once got funding by the EU https://www.ncbi.nlm.nih.gov/pubmed/18277062 if I remember correctly 300 000 DM 150 000 € .

 

Thanks, I hadn't heard of this trial before! Reading this makes me desperate.

 

Hey, just like PACE and every other psychosocial trial! What a coincidence that a sham treatment has the same outcome as other sham treatments.

 

This is actually an interesting trial. It's one of the few that tested the effects of blinding and probably the only one that tested this in patients with CFS.

It's also a very large study. 409 CFS patients were randomized into four groups:

• Those who were blinded and received the intervention (distant healing)
• Those who weren't blinded and received the intervention (distant healing)
• Those who were blinded and were put on the waiting list control.
• Those who weren't blinded and were put on the waiting list control.

So there were approximately 100 patients in each group. The main outcome was the mental health component summary of the Sf-36 and that didn't show anything. But the secondary outcome the physical health component summary (PHCS), showed something interesting results.



The first three groups showed some small improvements, but the group that knew they were not being treated didn't improve. Although the difference was very small, the authors reported that it was significant (a result of their large sample size?).

So this seems to constitute evidence that the expectation of (possibly) receiving the intervention is sufficient to create a significant difference in reported physical health in a proper randomized trial, even if the intervention is something implausible as distant healing.

This was supported by a post-hoc analysis. The authors had asked the blinded patient groups whether they thought they were in the intervention or waiting-list control group. The mere belief of being in the intervention group caused a significant change in how patients reported their physical health:

The authors wrote that they "consider an effect size of d = 0.20 as clinically relevant."

So it seems that in an unblinded trial, expectations of possibly receiving the intervention suffice to create changes in reported physical health that look like clinically relevant effects, but disappear when patients are properly blinded.

Interested in hearing your thoughts on this trial @Jonathan Edwards @strategist @Snow Leopard @Esther12 and others

 

@Michiel Tack - As you say, the changes are pretty small and I'm not sure anyone would dispute that these sorts of changes could occur. That the 'treatment' is so ridiculous, doesn't involve real contact with the supposed 'healer'/therapist, doesn't involve promoting a model that encourages the view that an improvement in health reflects well on the participant, etc all mean that I'd expect less biasing of outcomes in a trial like this than in CBT/GET trials.

I was glad to see your notes on it but I'm not sure it's that valuable as something to refer to. Interested in what others think.

 

Nice find. My thoughts are that this might not convince proponents of something like CBT or LP. It might just reinforce their belief that modifying the psychological state of patients can treat CFS. These results aren't inconsistent with it.

 

To avoid misunderstanding , the results are also consistent with biased reporting and no actual improvement in health. A proponent of CBT and LP would choose the other interpretation however.

It's probably futile to debate with them as their theories cannot really be shown to be false. Even if the cause of ME/CFS was identified and turn out to be say aberrant immune cell activity, they would still claim that there are important psychological aspects that respond to CBT or LP and they would be able to point to the usual studies that fail to control for biased self-reporting of health. If they firmly committed to a testable claim this would be different but they keep their ideas vague and flexible to avoid exposing themselves to the risk of falsification.

The way we have to proceed in my opinion is to convince the rest of society that what the CBT/GET/LP people are doing is pseudoscience because their theories don't follow from the data and they test their treatments in a way that virtually guarantees positive results even if the treatment didn't actually work. It's important to attack the methodology because they can easily invent some new treatment to replace the old one in case it gets discredited.

CBT/GET/LP proponents will probably always believe in it, but if they are surrounded by an environment of people that are increasingly aware of the scientific problems, then these therapies will wither and eventually disappear.

 

The changes are smaller than in PACE but it is worth remembering that the changes in PACE were small (maybe three times the changes here but still trivial).

I think this could be quite a useful illustration of the problem with these trials. It goes along with the main case I made for NICE, that the fact that you can get the same result with anything presented the right way means that we have no reason to think any of these treatments has a specific effect of its own.

It is also another illustration, like SMILE, of the fact that you get a result however ridiculous the treatment, as long as you recruit people who are ready to believe it might work.

 

If they had not used blinding they would have got a statistically significant reduction in physical health in the distant healing group. What I think the trial shows is that when you don't use blinding, you create bias because of the expectation of getting the intervention or not. And when you use a large enough sample size you can inflate that small but consistent difference into something that gives you a p-value below 0.05 in a randomized trial. I think this is a study that can be cited to illustrate that.

I'm not sure if anyone would dispute that these sorts of changes can occur. Everyone seems to take small but statistically significant differences in unblinded randomized trials very seriously. In the new Rob2 tool, there's no section where response bias due to knowledge of (not) receiving the intervention. Suppose the authors would have done this trial without the blinded groups. As I understand it, because the lack of blinding didn't lead to changes or a failure to implement the assigned intervention that are inconsistent with the trial protocol or arose because of the trial context, this distant healing study could have been rated as low risk of bias using Rob2.

Exactly, all these factors would further increase bias and it isn't difficult to see that this could amount to the changes found in the PACE trial.

• If people have direct contact with trained health professionals in 12-15 sessions, then surely one could expect the response bias to be even higher than is the case with distant healing.
• And if people are getting instructions on how they should interpret their symptoms and how they should stop focusing on them, that would lead to a larger difference with the control group without reflecting a change in health.
• And if the treatment is something that is being offered to other health conditions and is prescribed by health authorities such as the CMO working group or later NICE, then that would lead to larger expectations than something implausible such as distant healing.

These effects aren't directly about blinding, more about not having a matched control group.

 

Sure, but I don't have much hope of convincing them given all that has happened. It might influence people who take CBT proponents seriously.

The fact that pseudoscience treatments such as distant healing or the lightning process are able to report statistically significant improvements in unblinded and uncontrolled randomized trials, gives a strong indication that this method is not reliable in measuring treatment effects.

I'm surprised by how many are willing to defend LP, given that it is pseudoscience, but I suspect that someone like Wessely, who likes to present himself as a sceptic, isn't too happy about it.

While he argued that patients need to be treated by a professional health psychologist with 12 sessions of CBT, it seems that someone trained in neurolinguistic programming can do the same in just three days. Fast as lightning! Wessely's carefully crafted a 'pragmatic' approach using the distinction between precipitating and perpetuating factors, doesn't seem to be any better than the pseudoscience Phil Parker came up with. That probably says it all.

 

It seems that the authors of the distant healing study aren't sceptics but really into alternative medicine.

There's some irony in this. Millions are spent on unblinded randomized trials using subjective outcomes without anyone studying the reliability of this approach (the team of Asbjørn Hróbjartsson seems to be the only exception).

So it's the people who take this approach too far by applying it to obvious pseudosciences such as LP and distant healing who unintentionally remove the veil and point out the flaws.

 

It seems to be worse according to their metrics .

Maybe Wessely isn't too happy about it but Sharpe and I believe someone else, maybe Moss Morris, tweeted about the lightning process. They seem to view it as further evidence supporting their position.

 

Did they tweet about LP or recovery Norway? In the latter, they might have underestimated how much the site promotes LP rather than ME/CFS patients who have recovered.

I would like to know if Sharpe is willing to defend LP as an effective treatment for ME/CFS. I suspect that for Peter White and Simon Wessely they answer would be no.

 

He might argue that he is not defending SMILE-just attacking anyone who attacks papers on non-drug treatments for CFS.

Regardless of their quality.

There may be depths to which he is unwilling to sink.

 

Judging by his various public pity parties in the media the last few months about horrible 'activists' being mean to him and his work, it looks like there may well not be any depth to which he is unwilling to sink!

 

This seems to suggest that the SF-36 is a lousy measure. Obviously, but I guess it had to be shown.

The very most generous interpretation suggests that bias for hope inflates the SF-36 score by about 3 points on average. And since it's not even clear that it is at all useful as a measurement, it's a very misleading tool at best. Obviously, but not surprising considering we are still firmly stuck at basic vocabulary problems and the meaning of common words.

 

Sure, but thus remains the problem that those subjective measures by questionnaire are never matched by objective measures, clearly showing subjective measures are misleading and should not be relied on.

It adds up precisely to a magical weight loss program that boasts of people feeling like they lost weight when actual measurements show an increase. This is clear evidence that vague subjective questionnaires are a misleading outcome measure.