RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) Sterne et al.

Both the PACE authors and Larun et al. have argued that bias due to lack of blinding is not really a big concern because patients don't like GET. In the PACE-trial expectations for GET were no higher than for APT. I suspect researchers might use such measurements of expectations before the trial to argue that bias due to lack of blinding is not really a big concern.
Click to expand...

Going from memory, in PACE the expectations for GET and APT were not different, but this was at baseline. Using this to claim that expectation bias was not a concern implies also claiming that the different treatments received later infused exactly the same expectations in participants, which seems rather dubious.

At the time, CBT/GET was already recommended by health authorities as most effective treatment and it promised a cure. The illness model of CBT/GET describes reducing activities in response to symptoms (an integral part of pacing) as maladaptive behaviour. Therapists would probably favor CBT/GET simply because it provided more job opportunities (pacing is usually not therapist delivered, and I think at the time CBT/GET was already available on the NHS).
 
Last edited:
Jonathan Edwards said:
It is actually wrong where the original tool assumed people new how bias works.
Click to expand...
Could you give an example or do you mean that the more a tool tries to specify with rules how bias should be assessed, the more it gives the impression that no judgement is needed?

Jonathan Edwards said:
The general risk of bias is always worse than the worst specific risk.
Click to expand...
I don't think anyone would disagree with that, it's just that they have chosen to use only 3 possible outcomes for the risk of bias assessment: low risk of bias, some concerns or high risk of bias.

In a sense, that means that a trial can be downgraded in only two steps to a high risk of bias. I think that it explains why it is now more difficult to rate a trial as high risk of bias for a particular domain. Do you think they should have used more than three outcomes?
 
Esther12 said:
So the good news is that both tools seem terrible, and therefore RoB2.0 may be less of a step back that we thought. Phew.
Click to expand...
This was only for blinding though.

For selective reporting, it seems that deviations from the protocol are taken less seriously in the new tool. In the old tool a trial had to follow the protocol in reporting the outcome. It reads:
The old Cochrane risk of bias tool said:
The study protocol is available and all of the study’s pre-specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre-specified way
Click to expand...

In the new tool, this is assessed at question 5.1. In the elaboration they write:
New risk of bias tool said:
Changes to analysis plans that were made before unblinded outcome data were available [...] do not raise concerns about bias in selection of the reported result.
Click to expand...
So that seems a problem, as others have already noted. They should have probably maintained the old description, as a failure to follow the protocol, would only result in some concerns, not a high risk of bias.
 
Barry said:
View attachment 8521

That is gobbledegook surely. Risk of bias due to being an open label trial is orthogonal to risk of bias due to deviations from intended intervention? Mashing the two into a single supposedly explanatory sentence seems illogical. It sounds as if they are trying to take two sources of bias they want to whitewash, and make it sound like they can make it OK. Or am I missing something?
Click to expand...

Precisely. It is non sequitur and deliberately so.
 
Jonathan Edwards said:
The general risk of bias is always worse than the worst specific risk.
Click to expand...
I think they may be confusing risk with consequence. There are many scenarios where the consequence of any one event occurring, out of a set of possible events, will be as serious as any of the other events. e.g. Any one of the following can kill you: being hit by lightning, falling off a cliff, being shot, being struck by flying debris, etc. The worst-case consequence of any one of them is effectively the same as all of them happening to you - you die. It's a logical OR.

But the risk of being killed by such events is clearly less for the person who is simply in a thunderstorm, compared to another person who is walking along a cliff top in a thunderstorm, with gales blowing stuff (including the person) all over the place, and being threatened by a gunman into the bargain.

Risk of an event occurring is distinct from the consequence of an event occurring. I'm not sure they appreciate that. Or maybe they do, but want to mislead people into thinking risk can be treated that way.

ETA: Corrected "It's a logical AND" to "It's a logical OR". Wrote it in too much of a hurry. Any one of the events means you are dead; more events does not make you more dead.
 
Last edited:
Barry said:
I think they may be confusing risk with consequence. There are many scenarios where the consequence of any one event occurring, out of a set of possible events, will be as serious as any of the other events. e.g. Any one of the following can kill you: being hit by lightning, falling off a cliff, being shot, being struck by flying debris, etc. The worst-case consequence of any one of them is effectively the same as all of them happening to you - you die. It's a logical OR.

But the risk of being killed by such events is clearly less for the person who is simply in a thunderstorm, compared to another person who is walking along a cliff top in a thunderstorm, with gales blowing stuff (including the person) all over the place, and being threatened by a gunman into the bargain.

Risk of an event occurring is distinct from the consequence of an event occurring. I'm not sure they appreciate that. Or maybe they do, but want to mislead people into thinking risk can be treated that way.

ETA: Corrected "It's a logical AND" to "It's a logical OR". Wrote it in too much of a hurry. Any one of the events means you are dead; more events does not make you more dead.
Click to expand...
Simple risk assessment - risk x likelihood of adverse effect It would no doubt be illuminating to see any of those for BPS input.
 
Michiel Tack said:
Both the PACE authors and Larun et al. have argued that bias due to lack of blinding is not really a big concern because patients don't like GET. In the PACE-trial expectations for GET were no higher than for APT. I suspect researchers might use such measurements of expectations before the trial to argue that bias due to lack of blinding is not really a big concern.
In my view question 4.5 in the new Rob tool is redundant and at risk of being misused, so it would better be deleted.
Click to expand...

I think Larun's argument is ridiculous because the patients who really don't like GET would never consent to participate in such a trial in the first place.

Participants who are willing to participate in such a trial likely have "an open mind" if you know what I mean and even if they have preconceived ideas, participants will still be subject to the usual set of biases as with any trial. But notably, these therapies were explicitly designed to challenge cognitions hence GET and CBT are inherently more subject to self reporting biases than APT that made patients more conscious of their limitations and lack of improvement.
 
They struggles to get patients to enter the trial which is why they changed the entry score from 60 to 65 and put out word to doctors that people with fibro could be referred.

So people who did not want to do GET never went near the trial in the first place.
 
Barry said:
Irrespective of whether these things get heeded or not, I think it is so very important they get into the written record, as your response has done.
Click to expand...

Yes. Very important for future historians of medicine to know what happened in the early 21st century. Unfortunately, journal correspondence (whether in print or online) does very little to change anything if the original authors don't want to make corrections. And it's very, very rare for the journal to insist that they do so.

Today, I've just been looking back at the huge amount of correspondence that a certain article in the Journal of the Royal College of GPs generated 30 years ago...
 
Tha t certai article shows the dangers inherent in the use an d misuse of language. It is strange how they "propose a model" but then use the indicative mood to express it, rather than offering some degree of conditionality. I am sure the perpetrators passed their "Use of English" exams and knew what they were doing.
 
You must log in or register to reply here.
Back
Top Bottom