BMJ: Rapid response to 'Updated NICE guidance on CFS', 2021, Jason Busse et al, Co-chair and members of the GRADE working group

Michiel Tack said:
Click to expand...


I have a suspicion that he is talking through his hat. I think I have seen it written that GRADE is not about recommendation for service policy but about quality of evidence. Anyway if he is right then GRADE is absolutely unfit for purpose.

I have a feeling that Dr Busse may be digging a hole for himself.
I like the way Michiel keeps a spade handy for these people.
 
Last edited:
Jonathan Edwards said:
I think I have seen it written that GRADE is not about recommendation for service policy but about quality of evidence.
Click to expand...

It's both, but assessing the quality of evidence and recommendation for service use are two separate things, even according to GRADE.

Nightsong said:
https://gdt.gradepro.org/app/handbook/handbook.html#h.qmsmbyw3yxsl
Click to expand...

From the GRADE Handbook:
A number of criteria should be used when moving from evidence to recommendations (see Chapter on Going from evidence to recommendations). During that process, separate judgements are required for each of these criteria.

In particular, separating judgements about the confidence in estimates or quality of evidence from judgements about the strength of recommendations is important as high confidence in effect estimates does not necessarily imply strong recommendations, and strong recommendations can result from low or even very low confidence in effect estimates (insert link to paradigmatic situations for when strong recommendations are justified in the context of low or very low confidence in effect estimates).
Click to expand...

Grading systems that fail to separate these judgements create confusion, while it is the defining feature of GRADE.
Click to expand...
 
Jonathan Edwards said:
I think I have seen it written that GRADE is not about recommendation for service policy but about quality of evidence.
Click to expand...

No, this was Cochrane. It seems that GRADE does deal with recommendations. But notably it regards low quality evidence as 'unconvincing' and also notes that since resources are always limited cost-effectiveness must be considered and recommendations should not be made where they might impact on more useful services.

I have not yet seen any suggestion that low quality evidence should lead to a weak recommendation to use the treatment.

Edit: it looks from the crossed posts above that this does not follow, although there may be more detail buried in the haystack of confusion.
 
@Jonathan Edwards : crossposted.

From the GRADE handbook:
(insert link to paradigmatic situations for when strong recommendations are justified in the context of low or very low confidence in effect estimates)
Click to expand...

That seems to be a missing link?

But there's an example:
Example 3: Strong recommendation based on low or very low quality evidence

The principle of administering appropriate antibiotics rapidly in the setting of severe infection or sepsis has not been tested against its alternative of no rush of delivering antibiotics in randomized controlled trials. Yet, guideline panels would be very likely to make a strong recommendation for the rapid use of antibiotics in this setting on the basis of available observational studies rated as low quality evidence because the benefits of antibiotic therapy clearly outweigh the downsides in most patients independent of the quality assessment (Schünemann et al. AJRCCM 2006)..
Click to expand...
 
Yes, it all comes back to me. I have read this in detail. I remember the example.
But I doubt there is anything that says that weak evidence should, as a rule, be followed by any recommendation. The only rational response to 'unconvincing' evidence is not to recommend.

All of this illustrates how despite the GRADE people thinking that they have invented some sort of robust objective system all they have really done is rather clunkily tried to explain how they think they make decisions themselves. The antibiotic example simply indicates that they realise that the system needs to be adjusted for common sense. So why not just leave people to use common sense?
 
It looks as if Dr Busse is clutching at straws for some reason. First he says NICE got it wrong. Then he says if they got it right they should still let him recommend GET (weakly). There is a strong impression that it matters to Dr Busse that GET is recommended.
 
Nightsong said:
he link is broken in the handbook, but it's section 6.3.2.
Click to expand...
I think it's interesting because if I understood properly this is mostly about weighing the evidence of an estimated desired effect against the evidence of an estimated undesired effect, i.e., mostly, harm.

Edit: On the other hand, it looks strange to me that very low quality evidence of benefit in the absence of harm generally should still lead to a recommendation in favor of the treatment, even if a weak one? That seems to be what GRADE advices though:

From section 6.1:

When a guideline panel is uncertain whether the balance is clear or when the relevant information about the various factors that influence the strength of a recommendation is not available, a guideline panel should be more cautious and in most instances it would opt to make a weak recommendation.
Click to expand...

(Edited for clarity.)
 
Last edited:
There is pretty much low quality evidence for anything you can imagine. It's literally what separates things that are real from those that aren't. By creating a standard that is below ground, everything goes: all the pseudosciences and alternative approaches, no matter what the evidence, they all have some low-quality evidence to "support" them. Cochrane has plenty of reviews for stuff like acupuncture and other woo. If low-quality evidence is sufficient, all that it takes for anyone to claim something is effective is to get it published and that's it. Doesn't even need to relate to reality at all.

This is what happens when exemptions are created in a system: it encourages to work only with those exemptions as they are far easier to meet than the normal standards. Why bother researching anything if you can simply try a bunch of stuff that doesn't work and promote it saying the evidence isn't good but it's there and that means it is 100% useful, somehow?

So there is a fork in the road ahead: whether medicine sticks to science or abandons it entirely. This process isn't even alternative medicine, it's alternative to medicine, a cheap knock-off. Like a smaller version of the post-truth era in politics, medicine is engaging in its own post-science era. And it's happening seemingly without notice. And the people who value truth are frankly not seeing that there is a split going on.

Safe to say that without Covid all of our efforts would have been cancelled entirely. Even with it's not even safe to say politics won't win the day. That's a huge lesson for Long Covid, they still don't pay attention to this stuff, can't imagine medicine is so utterly broken. Problem is how to make them aware of it without it being so demoralizing they brush it off, hoping it will be different for them.
 
MSEsperanza said:
That seems to be what GRADE advices though:
Click to expand...
No I think this is simply a precaution to not make strong recommendations when things are unclear.

The full section reads (my bolding)
For a guideline panel or others making recommendations to offer a strong recommendation they have to be certain about the various factors that influence the strength of a recommendation. The panel also should have the relevant information at hand that supports a clear balance towards either the desirable effects of an intervention (to recommend an action) or undesirable effects (to recommend against an action).

When a guideline panel is uncertain whether the balance is clear or when the relevant information about the various factors that influence the strength of a recommendation is not available, a guideline panel should be more cautious and in most instances it would opt to make a weak recommendation.
Click to expand...
so it seems that the 'more cautious' refers to more cautious than making a strong recommendation. If the evidence was unclear or unconvincing the cautious option would simply not to make a recommendation for or against.
 
Very low quality is the lowest the GRADE system goes. That's the level of case reports etc. See for example:
Case series and case reports are observational studies that investigate only patients exposed to the intervention. Source of control group results is implicit or unclear, thus, they will usually warrant downgrading from low to very low quality evidence.
Click to expand...
So if very low quality evidence should lead to a recommendation, then NICE would have to recommend everything (carnitine, LDN, Ampligen) that has scientific studies in support, no matter how bad those studies are.
 
You must log in or register to reply here.
Back
Top Bottom