Survey:Improving NICE guidance

just 'presenting' it at all would be a start.

I would suggest that, for any recommendations of treatment or advice to be given to patients, there should be a statement about what level of evidence there is.

My understanding is that there is some sort of code in the guideline language, with 'offer <a treatment>' meaning a higher level of evidence backs the recommendation than 'consider <a treatment>'. But most readers will miss that; and it's not at all clear.

I understand that sometimes recommendations that aren't evidence-based will be made, for reasons of political expediency or clinical necessity. But I'd like to see an unambiguous statement beside any recommendation or intervention. Something like, 'this recommendation is supported by substantial evidence of high quality', or 'this recommendation is currently not based on trial evidence', or 'this recommendation is supported by high quality evidence for adults, but there is no evidence in children and young people'.

If clinicians are expected to present benefits and risks of a treatment or behaviour change accurately to patients, they should be given some guidance on the strength of the evidence for that intervention. It would also make it abundantly clear where the gaps in research are.

 

I did this, although it took me a lot longer than 15 minutes. It's interesting; NICE seems to be testing out providing more information related to recommendations, and how the information is laid out.

The example recommendation they gave for comment included a short sentence saying it was based on low quality evidence, but did not say why the evidence was poor. And it might as well have been high quality evidence for all the difference it made to the recommendation. The example just carried right on with making the recommendation.

I was a bit like a broken record saying 'you can't base a recommendation on low quality evidence'. Because we know that 'low quality evidence' actually means that it probably isn't telling us anything reliable about the outcome. 'Low quality evidence' is only evidence that the researchers of that particular question need to do a lot better. The problem is, if you haven't had the experience that we have had with NICE guidelines, then I don't think you'd give much weight to the brief statement hidden in all the detail of the recommendation saying the evidence is low quality. You'd assume that the recommendation is right and that it is evidence-based.

On the outside chance that this has something to do with how NICE presents the ME/CFS recommendations, it would be great if some of our members could do the survey, and keep hammering home the point that low quality evidence is probably as useless as no evidence. And that we need to know what was wrong with the evidence. After all, several very small but good studies that all found the same thing might be useful. But one large study that is hopelessly biased, with no blinding and subjective outcomes probably isn't.

 

Oddly, I have been asked to spend an hour online with a NICE representative discussing the topic of the survey. The call will be later in my day; tomorrow morning UK time. If people have more views about this topic, please do post on this thread today.

 

I do find it encouraging that my idea of good answers to the question of how could NICE improve its guidance (answers which are aligned with the thoughts of many of us here) are regarded by NICE as worth talking about.

 

I've done the call, there were 3 women from NICE who were , well, very nice. They do genuinely want to make guidelines more useful. I guess that could mean that they are focused on the form rather than the substance, but getting form right is still an advance, and this team seemed to be trying hard.

There's a new format that is being trialled, first with some Covid-related guidelines. I think it's an improvement. Recommendations that are not based on evidence are clearly labelled as 'consensus recommendations'. There will be times when a consensus recommendation is a good thing, but it's good for it to be clear when a recommendation is just based on 'the committee thinks this is a good idea'. More information about why a recommendation was made is included along side the recommendation.

I did try, several times, to explain the problems with GRADE, like research with fatal flaws still being used to support recommendations. And the fact that it's all very well making much of the detail behind a recommendation easily available but, if the recommendation information says 'This recommendation is based on low quality evidence', most people won't realise that that could mean that actually the evidence is uninterpretable. I think the NICE women accepted that it could be good to explain actual problems with the evidence base (e.g. 'high risk of bias due to subjective outcomes in unblinded trials') beside the recommendation itself. But dealing with GRADE was outside their scope.

We talked about the guideline being a source document where the rationale for recommendations can be dug into. If doctors want a brief document with just the recommendations, there are online summaries they can access. Whereas I think the guideline should have a lot of the information relevant to a recommendation either in the recommendation or able to be clicked through from the recommendation - rather than having to go find sections at the end of the document or in another document with hundreds of pages.