Interventions that manipulate how patients report symptoms as a separate form of bias

[Sean]

I offer a free organic disposal service loving home for all unwanted black rice pudding.

 

[FMMM1]

I had the same impression. Haven't looked at it closely but I have a suspicion that the "hard science" Benedetti refers to (his own work presumably) is rather weak.

I haven't seen anything yet that shows the placebo effect isn't simply due to response bias.

I haven't read the study being discussed - here's my "usual" rant!

There's a thread on a study which compared actimetry with self reported questionnaires. Questionnaires overestimate activity/improvement if you like.
Real world scenario:
You work in policy and you are part of a team overseeing a study your Government is funding i.e. to see if intervention X works [think PACE]. Your job is to ensure that you have good quality evidence to present to the Minister so you'd (logically) go for actimetry and quote the relevant study(s) which set out how this method is objective and robust/defensible. Study ends, you write a summary which e.g. says intervention wasn't shown to be effective ---- Minster, on the face of it, has two options - highlight that the available evidence does not support the use of this treatment or say nothing.
Problem is our experience doesn't look anything like that scenario - poor quality studies just keep getting funded by the UK Government.

As for why questionnaires are crap, that seems to be irrelevant --- I think it could be the Hawthorne effect* and that in turn reflects inadequate blinding in studies [raised by Professor Hughes, Galway University, and I guess many others] - but I'm not sure you could ever blind a CBT/GET study effectively. @Simon had a really simple summary - unblinded then use objective outcome indicators, blinded then subjective indicators OK (you have an adequate control group).

Bottom line is if there's a method which works [actimetry] then why consider why the other method doesn't [questionnaires]?
*https://en.wikipedia.org/wiki/Hawthorne_effect


@Keela Too @Caroline Struthers

 

[Mich]

Hi. I've very much enjoyed reading through this forum so far. There are some extremely intelligent and knowledgeable contributors. I'm truly humbled to allowed to voice my meager opinions on the same platform.

Note: some of this is reflective of my spouse's opinions, as I read some of it aloud to them to get their opinion too.

Also: I'm aware of how old this post is, but I decided to start from "the beginning." I've also been reading new posts as they've come in since I joined, and from what I can tell so far, a lot of it's still relevant.

Caveat: I do have some college level education in psychology, and have found CBT and similar modalities to be helpful on occasion (especially REBT--but dicidely not anything bearing traces of Aaron Beck's CBT, with the exception of some of David Burns' stuff). However, for the most part, I would not call myself a proponent of CBT genrerally, despite my dangerous love affair with William James and incongruous fascination with the historical roots of "New Thought." But they way the coporate world glommed on to CBT (and it's cousin during the mindfulness hysteria of the Y210s), and the way Beck just adjusted his bow-tie and with a twinkle of they smiled away through it being co-opted by the business rats, and the ogseqious entrepreneurs and sycophant venture capitalist was such a vomit show.

My spouse, OTH, still denies it has ever done anything for their pain, PEM, or the deep grief and sorrow over mourning the loss of living much beyond the bedroom or our flat; yet, even then, once in a while, they'll begrudgingly admit that REBT has helped them now and then with their "stinkin' thinkin,'" For the most part, though, if it's brought up, they maintains that, "it's a bunch of fucking bullshit."

...."self-efficacy", which we know very well means not trying hard enough but they just pretend they don't mean that, even though we all know they do and so do they.

That's not how Bandura meant it. Do you mean like on self report questionnaires, and in interviews and such? This is an old thread, so maybe it's different now?

@Barry:

So to me the fundamental problem is that CBT and GET are formulated to be bias-inducing treatments, it's their very purpose. So it is hardly surprising they induce bias in the way people think, and to self report with significant bias on their physical illness.

Though no two CBTs were created equal, they're all against any type of bias. They aim to reduce or eliminate bias.

I like this. Or maybe "treatment-induced cognitive bias"

Then why not go for broke and call it iatrogenic bias or maybe even just (a sub-species of) iatrogenesis?

Or we could just call it what it is.....loading the dice. Loaded dice bias?

Except that we'd be committing a stereotyping or caricaturizing fallacy.

...So there are 3 stages:
1. Change your beliefs about your symptoms
2. Change your behaviour by increasing activity
3. Be able to sustain increased activity, and thereby recover back to normal fitness and employment levels.

For my partner, there was also a fourth stage, which involved returning my blank "fear-avoidance, "Questionnaire" to their physio's admin team, and if they asked why they didn't fill it out, being prepared to say, "Because I'm not going to let you use your misunderstanding of Bandura, social learning theory, CBT, etc... as a victim blaming, gaslighting instrument and down-the-road documentation that I'm non-compliant when your PT fails, which it' s about a coin flip that it will." Luckily it only came to getting the office manager to admit I didn't have to have to fill it out to receive treatment simply by asking a her a few times if I had to fill it out.

@Adrian
...it is important for any assessment/review to understand what is being measured...."

In terms of research method and design, I've always struggled with this. If we understand what we're measuring, do we really need to do a trial----especially over already overly replicated trials? When do we move the knowledge our research project has given us from the category of "verifying" to "verified" or "good enough"? IMHO, may of the problems of medical and social science research stem from its base in inductivist methods, especially the hypothetico-deductive form. Inductivsm, despite all the sweeping under the rug/look over there shenanigans it's inspired since your eminent Mr. Hume initially complained about, induction is still one hell of a problem. Someone early mentioned incestuous reasoning or something of that sort? That's not an artifact of inductivsm. It shows our lack of imagination as a cognitive species, and especially of scientist, whether of the citizen or career variety. As one at moments in both worlds, but mostly in latter (especially now), I for one have not spent enough time asking how can we do better when I was saying just look what these idiotic psychologists have done or projects.

@Adrian
....to distinguish between patients' interpretation/perception of their symptoms, and their reporting.....

True. But's let's not forget that this relates to other huge problems in science and epistemology (which involves the limits knowing), and it's why we still have other even bigger problems in these fields, like the pragmatist's rejection the object/subject, fact/value, rational/irrational dualisms Let's also not forget that this very problem brought us Skinner, and there would be no CBT without his contributions. In the case of science, we don't have a pendulum uni-directinally swayng to and fro at a stead rate, but several pendula at different speeds and sizes going all at once.

@Invisible Woman: So even before they participate in a study patients are primed to believe if only they do it right, have faith, recovery will happen.

GOOD CBT therapy ALSO essentially involves learning how to accept and cope with recovery when it does not happen without self blaming.

 

[Jonathan Edwards]

That's not how Bandura meant it.

But how Mr Bandura wanted to mean it is not the issue. It is how it is used.

Though no two CBTs were created equal, they're all against any type of bias. They aim to reduce or eliminate bias.

The creators of CBT may have this belief but the poster is right. CBT is designed to bias. Peter White of the PACE trial actually wrote this in a paper with Hans Knoop - that CBT works as a placebo. If the people who have promoted CBT for ME/CFS believe this is what they are doing then that is what they are doing.

It is not what is usually meant by iatrogenic because we are not talking about causing harm and why use jargon when you can use English?

If we understand what we're measuring, do we really need to do a trial-

Yes, because you need controlled observations even to apply inductive reasoning. Hume was worried that even if you repeatedly observe one thing under one set of conditions and another under another set you cannot deduce causation for sure. If you don't even have another set of conditions to check a negative finding on you have no reason to deduce anything at all.

But's let's not forget that this relates to other huge problems in science and epistemology

It might relate to the sort of armchair semantic debates you mention but in medical science the situation is uniquely bad for psychological interventions - for all the reasons the thread posters have given. I spent my life designing, executing, reviewing and making use of trials. Bisas problems crop up all the time but this is a unique situation where users admit that their treatment is designed to bias yet refuse to admit that there is any bias. The contradiction explicit in White and Knoop's paper shows just how little these people understand about what they are doing. And from what I have seen this is a completely general problem for psychological therapies.

 

[rvallee]

That's not how Bandura meant it. Do you mean like on self report questionnaires, and in interviews and such?

More of a general use of the term in how it's applied to chronic illness some perceive as psychosomatic. There is this widespread idea that, since there is "nothing actually wrong" with us, absence of evidence being used as affirmative evidence, then all we pretty much lack is something that can be lumped into the concept of self-efficacy. This is not about CBT but rather the generic biopsychosocial/psychosomatic models.

This idea that we're just people who "can't cope with modern society" is common in the literature, has been around since before electrification, no less, we should be be able to just do normal stuff, but we don't, because we lack this "self-efficacy". We should be able to "pick ourselves up", but we don't because *gesticulates widly* reasons.

Problem is that in the general area touching the problems medicine hasn't got a handle on, words are routinely twisted to mean something different. It's a big problem because it's used explicitly to hide the true meaning. It takes a long time to accept that this is really how it's done, and it's hard to point at specific examples precisely because deceit is the point.

This is an old thread, so maybe it's different now?

Sadly not. This thread contains a great example of how it plays out, here on the usage of the word 'functional', where it's explicitly admitted that it's a preferred term because more patients think it means the opposite: https://www.s4me.info/threads/diagnostic-labels-in-functional-disorders-2026-novak.48670/post-672279.

 

[Barry]

Though no two CBTs were created equal, they're all against any type of bias. They aim to reduce or eliminate bias.

The PACE trial pursued a "positive thinking" therapy regime on the assumption patients needed nothing more than to rethink their illness perceptions, and to thereby do more positive things such as exercising "sensibly" and not resting too much, etc. Essentially seeking to bias their thinking away from what they believed their illness to be. So even if they did not in truth improve their physical fitness, or maybe even deteriorated, they would very likely be feeling a bit more positive about themselves psychologically, and therefore very likely to give more positive ratings in any kind of subjective assessment. But the CBT did not fix their illness, at best only supportive. Properly formulated CBT has its rightful place in therapy, but not here, not in this form, not masquerading as a way to a cure.

This is from page 9 of the PCE trial's participant CBT manual, and is just one part of the manual which seeks to implant bias into participant's minds regarding their illness, prior to even starting the trial. The strong, unavoidable implication here in the manual, is that the reason the participant's illness has not been detected is because there isn't one to detect, and that giving in to that belief will just make the symptoms worse, unless physical rehabilitation is undertaken. And remember this is the CBT manual, not the GET one. [my bold]

PHYSIOLOGICAL ASPECTS OF CFS/ME
Many people with CFS/ME are concerned that their distressing symptoms may be related to a disease that hasn’t been detected. Others are concerned that a virus (if one occurred at onset) is still present or has caused physical damage to the body. Intensive research has tried to establish whether disease, deficiencies or any other abnormal changes in the body may explain the very distressing and debilitating symptoms experienced by people with CFS/ME. To date, it appears that there is no one cause of CFS/ME. A variety of different triggers are reported, e.g. different types of infection or stressful life events. Some people can pinpoint the exact date that it started. For others the onset is more gradual.

Difficulty maintaining previous activity levels is common to all sufferers. Some sufferers feel so ill that they rest for long periods and give up many of their previous activities including work, social activities and managing the home. Others may be able to function at a reasonable level, e.g. go to work or look after their family, but due to pushing themselves so hard in the day, may do very little in the evenings or at weekends. Others tend to do too much on “good days” and push themselves too hard for their level of fitness, they then rest for long periods on other days.

Over time, reduced or irregular activity and increased periods of rest causes physical changes in the body. These changes cause unpleasant sensations and symptoms that can be very distressing. It is important to point out that these changes are reversible with physical rehabilitation and/or exercise.

Research has looked at the effects of rest in healthy people when they reduce their activities and many similarities between people with CFS/ME and healthy inactive people have been noted.

 

[Utsikt]

So even if they did not in truth improve their physical fitness, or maybe even deteriorated, they would very likely be feeling a bit more positive about themselves psychologically, and therefore very likely to give more positive ratings in any kind of subjective assessment.

Why is that very likely? Changing their answers, sure. Changing how they actually feel, not so much.

But the CBT did not fix their illness, at best only supportive.

How would CBT be supportive if it tries to make you believe in a reality that doesn’t exist?

Properly formulated CBT has its rightful place in therapy,

Do you have any examples?

 

[Barry]

Why is that very likely? Changing their answers, sure. Changing how they actually feel, not so much.

How would CBT be supportive if it tries to make you believe in a reality that doesn’t exist?

Do you have any examples?

But that was the thing with PACE I believe, I don't think the questions were confined to how people felt, but required subjective answers to how much better they were. Given the flawed premise (bordering on presumption) was that the fundamental issue was a psychological one holding people back, once they answered subjectively they maybe believed to possibly be improved, it was taken as a done deal. And that was at trial end, not later.

Supportive CBT. There is a more technical term for it I've forgotten for the moment. If someone has a well understood but debilitating condition, cancer maybe, then I'm sure it must be mentally devastating. I'm not pretending to be an expert, but there will be plenty here who are, but how people cope with such an illness will to a fair degree depend on their mental state. My understanding is that in some cases supportive CBT can be beneficial as part of a coping strategy. No pretence, by therapist or to patient, that the CBT will cure anything, but that it can help a patient cope better.

But like I say, there others here who understand these things better.

 

[Jonathan Edwards]

My understanding is that in some cases supportive CBT can be beneficial as part of a coping strategy.

I think the point is that we have no reliable evidence for 'CBT' being any better than the man in the pub or someone who claims to have counselling skills but does not call them something fancy.

 

[bobbler]

When we discuss the problems with trials on cognitive behavioral therapy (CBT) and graded exercise therapy (GET) we usually mention things like lack of blinding + subjective outcomes or the lack of a credible control group etc. These are methodological weaknesses that are generally regarded as leading to high risk of bias.

But as most of you know, the problems with trials on CBT and GET go further because the intervention consists of influencing/manipulating how patients view and report their symptoms. Usually, code-words are used for this like self-efficacy, reduced symptom focussing and catastrophizing, tackling fear-avoidance etc.

Many have already highlighted this issue but I was wondering if we should argue for a separate risk of bias domain for interventions that influence how patients report their symptoms. I think that might make the problem clearer to those who haven't taken it seriously thus far.

Perhaps we should try to expand the scope to behavioral interventions outside ME/CFS. 'Pain education' is another example I can think of that seems to do the same thing. I suspect most forms of CBT suffer the same problem, although to a lesser extent than in ME/CFS.

It would be good if we could collect examples from other fields (including alternative medicine and quackery).

I agree. Good idea

 

[bobbler]

I agree that this is another level of bias that does not appear to be taken into account.
I am personally against any use of 'tools' though. It irrational and as long as it is considered OK the system will be manipulated.

Using a tool works like this:

1. Gather reliable evidence for factors affecting bias from all possible sources.
2. Derive a set of rules that seems to cover those factors and put it in a tool.
3. Use the tool.

A better approach is:

1. Gather reliable evidence for factors affecting bias from all possible sources.
2. Use it, case by case.

Since in the first method 2 can only be at best a rough approximation to 1 it has to be better to use 1.

The practical reason for having a tool is to make it possible to employ people with no real understanding of clinical trial psychology to apply some rules. I think that should not happen.

I get the ideal

But I think we’ve a chicken and egg issue

Even those who are good probably get forced by this culture to conform so good is seen as bad (not using the tools) and vice versa.

Up and comers get taught this lesson of what good looks like and so on

Maybe there is some hybrid that at least insists if said tools are used if that bias isn’t included in x way it’s useless - in order to make critique simpler and point of the patterns in a loggabld way but also lift the dregs a bit up from the bottom of the barrel

Whilst the discussion saying this isn’t the ideal etc

 

[bobbler]

I thought you were serious- such good idea, although I was a bit shocked you admitted to it.
But seriously, names for this bias: something like 'psychosocial bias'? or 'cognitive behavioural bias'? or even 'cognitive manipulation bias'? Or something more allegorical like 'Animal Farm bias'? (maybe not so seriously).

Spoonfeeding bias comes to mind as the issue seems to involve teaching to the test

But in a way that isn’t like teaching someone maths just telling them their pain of 10 isn’t socially acceptable ‘could the phrase it more positively’ then it’s brow-beating or social pressure but in a more direct and organised way that means someone can’t ’just Not care what people think’ as if each time you say 10 someone corrects you with 8 eventually it is a form of programming

So a lot of tricks lie under this bug perhaps with similar themes we can get a ‘does what it says in the tin ‘ nsne fit - which I think is needed for this to not get twisted eg in press etc (where bps just switch and being saying it’s terrible and the solution is doing more of what they’ve been doing because they get away with using words to run readers in circles)

 

[bobbler]

Treatment induced bias?

Persuasion bias?

Is it almost like ‘planting’ ? And not far off the old scandal of when people were going to therapists and being persuaded into false memories of abuse - which was fully acknowledged and the big issue in 2000s much as it is belittled now as if it was a blip and in doubt the debates were of the death of psych if it didn’t publicly clean up its act due to this


ironically just when bps was selling CBT into nhs for this type of thing claiming it was different but the type they were selling was actually turning out to not be cognitive therapy based but psychoanalytic and behavioural hiding under that CBT term - a switch and bait


Anyway I think it makes sense that if the themes are similar like priming and planting and controlling are to the above ‘last scandal’ that it is considered that the term reflects that - patterns matter

Oh and plus of course it brands the need for it to be included and guarded against as a bias as ‘savings psych/cleaning it up’ rather than then twisting it to be as if there is a theoretical issue with mental health in general. No just the harmful crap and anyone with respect for health or mental health shouldn’t be ok with harm or manipulation etc and it’s bs for them to justify it as it’s ok if they refuse to look so they can pretend it’s good intentions (when it’s callous indifference but really they know mostly…)

Something that flags its conning the system and the taxpayer money etc but also harmful and devious and not ‘therapeutic’ in the true health giving sense of the word but coercion and using them as tools to achieve ends (so something about lack of care as to how they really feel or are by putting words in their mouth they feel pressured to conform to. )

 

[bobbler]

Yes I was thinking yesterday along the lines of a bias modification bias, as at least certain aspects of CBT address cognitive distortions or processing biases which supposedly lead to biased or irrational thoughts and beliefs and resulting behaviours (although also goal setting, developing coping strategies) . GET likewise is contingent on the participant reconciling to the idea that increasing is possible without harm and resultant increases in symptoms are expected and not indicative of harm. Although 'Cognitive bias modification' is a psychological approach in its own right and distinguishable from CBT.

I think we start with things like pain and me/cfs

But stay right out of things where that argument about ‘but it could be good for x,y,z’ type conditions ends up getting thrown back

However it is useful to anticipate the arguments - even if I don’t agree with most and there’s some old-fashioned I believe lack of questioning in just how ‘caught in bias/unable to think/just needing coaching’ the cause of many if those things are. Or the best support would be. It seems to no longer be used gently as a sticking plaster whilst practical support and counselling is being waited for and thats taken it into the realm of nonsense non psychology by people not monitoring the content or that it’s now been slid in over time to be assumed it was ever ‘a treatment’ for those rather than only safe for temporary sticking plaster and plus the content changed so it’s just used as a vessel for stuff they think is common sense but isn’t the cure gif theses things (like eat better go out more behave better based on assumed norms not what cures x,y,z)

There’s no justification for this bias in some of these they are trying to push into and I think if the band chosen focuses on the fraudy coercive aspect that’s important - as these things and people are indeed going to use that distraction nonsense of ‘bias is good’ tosh.

No one should ever have given them permission that their treatment they enforce with powers that coerce it into people’s lives can be justified based on ‘evidence’ coerced out of people

And if it really was ‘any use’ what they do then why would they be engaging in such dodgy stuff and indeed campaigning for even less rigour?

 

[rvallee]

Why is that very likely? Changing their answers, sure. Changing how they actually feel, not so much.

I honestly have no idea what people mean by that when they talk about CBT, how it could even possibly make people feel any better or cope with illness better, as opposed to the obvious: they're just reporting differently from how they feel. What relation CBT has to coping with illness is honestly totally baffling.

CBT is basically given a sort of weird deference, like the old days when brainwashing and lie detectors were fashionable, even though neither actually exist. It's possible to find people who can be manipulated in such ways, but that's a very different thing from being able to apply such a process to anyone, or even most people. Somehow it's always asserted as a given, as a "well, this is what it says on the box".

Does Red Bull give people wings? Not at all. Does CBT help people cope with illness? Why would that be the case, when it doesn't even apply to most of the problems caused by illness, especially symptoms, which doesn't even make sense.

 

[Sean]

But the CBT did not fix their illness, at best only supportive. Properly formulated CBT has its rightful place in therapy, but not here, not in this form, not masquerading as a way to a cure.

I remain deeply and increasingly skeptical of the whole CBT show.

 

[Barry]

I remain deeply and increasingly skeptical of the whole CBT show.

And very rightly so. My wife has had ME now for 20 years, and I marvel at her determination to make the best of her situation. She is relatively lucky in that her ME is moderate, possibly toward the milder end of moderate, but she has still been hit hard regarding the things she can no longer do. We were lucky with the consultant we saw back then, who had a good in depth discussion with my wife - I was there - and he concluded that he felt CBT would be no use to her anyway; I think he maybe had an inkling even then about ME and CBT.

My wife and I first started going out together just over 49 years ago, so I have known her through many years of good health, plus all too many with ME, and whatever it is that is causing her ME, it most certainly is not induced by any psychological aversion to getting on with things, to exercise. As I've said many times here and elsewhere, my wife always does as much as she can, never as little as she can. The only difference now is that she is significantly more limited in what she can do, but the mental drive to survive has never left her. I admire her greatly.