Survey:Improving NICE guidance

NICE survey said:
how we ... present our guidance.
Click to expand...
:rofl: just 'presenting' it at all would be a start.

I would suggest that, for any recommendations of treatment or advice to be given to patients, there should be a statement about what level of evidence there is.

My understanding is that there is some sort of code in the guideline language, with 'offer <a treatment>' meaning a higher level of evidence backs the recommendation than 'consider <a treatment>'. But most readers will miss that; and it's not at all clear.

I understand that sometimes recommendations that aren't evidence-based will be made, for reasons of political expediency or clinical necessity. But I'd like to see an unambiguous statement beside any recommendation or intervention. Something like, 'this recommendation is supported by substantial evidence of high quality', or 'this recommendation is currently not based on trial evidence', or 'this recommendation is supported by high quality evidence for adults, but there is no evidence in children and young people'.

If clinicians are expected to present benefits and risks of a treatment or behaviour change accurately to patients, they should be given some guidance on the strength of the evidence for that intervention. It would also make it abundantly clear where the gaps in research are.
 
Last edited:
Hutan said:
:rofl: just 'presenting' it at all would be a start.

I would suggest that, for any recommendations of treatment or advice to be given to patients, there should be a statement about what level of evidence there is.

My understanding is that there is some sort of code in the guideline language, with 'offer <a treatment>' meaning a higher level of evidence backs the recommendation than 'consider <a treatment>'. But most readers will miss that; and it's not at all clear.

I understand that sometimes recommendations that aren't evidence-based will be made, for reasons of political expediency or clinical necessity. But I'd like to see an unambiguous statement beside any recommendation or intervention. Something like, 'this recommendation is supported by substantial evidence of high quality', or 'this recommendation is currently not based on trial evidence', or 'this recommendation is supported by high quality evidence for adults, but there is no evidence in children and young people'.

If clinicians are expected to present benefits and risks of a treatment or behaviour change accurately to patients, they should be given some guidance on the strength of the evidence for that intervention. It would also make it abundantly clear where the gaps in research are.
Click to expand...
I think this is a big problem with the NICE guidelines. I've never looked at most of them but, for example, with the autism ones this information is in a much larger guideline that is available on a search, but not the main guideline that clinicians and patients use (now also called a 'Quality Standard'). The more in depth guideline is about 400 pages long and it explains the quality of evidence and the limitations of the various diagnostic measures and tests, where research is limited etc. But that is not going to be read even by the Clinical Psychologists carrying out autism diagnosis (based on my experience).

I've just looked up the more recent children's version and the full guideline on that is nearly 800 pages in contrast to the 32 page main guideline:

https://www.nice.org.uk/guidance/cg170/evidence/full-guideline-pdf-248641453

https://www.nice.org.uk/guidance/cg...19s-support-and-management-pdf-35109745515205

I have no idea if there will be a similar 'full guideline' for the 2021 (22/23/24?) ME/CFS NICE guideline.
 
Last edited:
I did this, although it took me a lot longer than 15 minutes. It's interesting; NICE seems to be testing out providing more information related to recommendations, and how the information is laid out.

The example recommendation they gave for comment included a short sentence saying it was based on low quality evidence, but did not say why the evidence was poor. And it might as well have been high quality evidence for all the difference it made to the recommendation. The example just carried right on with making the recommendation.

I was a bit like a broken record saying 'you can't base a recommendation on low quality evidence'. Because we know that 'low quality evidence' actually means that it probably isn't telling us anything reliable about the outcome. 'Low quality evidence' is only evidence that the researchers of that particular question need to do a lot better. The problem is, if you haven't had the experience that we have had with NICE guidelines, then I don't think you'd give much weight to the brief statement hidden in all the detail of the recommendation saying the evidence is low quality. You'd assume that the recommendation is right and that it is evidence-based.

On the outside chance that this has something to do with how NICE presents the ME/CFS recommendations, it would be great if some of our members could do the survey, and keep hammering home the point that low quality evidence is probably as useless as no evidence. And that we need to know what was wrong with the evidence. After all, several very small but good studies that all found the same thing might be useful. But one large study that is hopelessly biased, with no blinding and subjective outcomes probably isn't.
 
Hutan said:
My understanding is that there is some sort of code in the guideline language, with 'offer <a treatment>' meaning a higher level of evidence backs the recommendation than 'consider <a treatment>'. But most readers will miss that; and it's not at all clear.
Click to expand...

These are excerpts from section 9 (Writing the guideline) of Developing NICE guidelines: the manual Process and methods [PMG20]
https://www.nice.org.uk/process/pmg20/chapter/writing-the-guideline#wording-the-recommendations

NICE reflects the strength of the recommendation in the wording (see the section on wording the recommendations). NICE uses 'offer' (or words such as 'measure', 'advise', or 'refer') to reflect a strong recommendation, usually where there is clear evidence of benefit. NICE uses 'consider' to reflect a recommendation for which the evidence of benefit is less certain.
Click to expand...

Reflect the strength of the recommendation

The 'strength' of a recommendation (see the section on interpreting the evidence to make recommendations) should be reflected in the consistent wording of recommendations within and across guidelines. In recommendations on activities or interventions that should (or should not) be offered, use directive language such as 'offer' (or 'do not offer'), 'advise', or 'ask about'.

In keeping with the principles of shared decision-making, people may choose whether or not to accept what they are offered or advised. If there is a closer balance between benefits and harms (activities or interventions that could be used), use 'consider'. If there is a legal duty to apply a recommendation, or the consequences of not following a recommendation are extremely serious, the recommendation should use 'must' or 'must not' and be worded in the passive voice.

Box 9.1 Examples of recommendations made with 3 different levels of certainty
Recommendations for activities or interventions that must or must not be used
- Ultra rapid detoxification under general anaesthesia or heavy sedation (where the airway needs to be supported) must not be used. This is because of the risk of serious adverse events, including death.
- Patient group directions must be authorised only by an appropriate authorising body in line with legislation.
​
Recommendations for activities or interventions that should or should not be offered
- Offer a trial of supervised pelvic floor muscle training of at least 3 months' duration as a first treatment to women with stress or mixed urinary incontinence.
- If a smoker's attempt to quit is unsuccessful using NRT, varenicline or bupropion, do not offer a repeat prescription within 6 months, unless special circumstances have hampered the person's initial attempt to stop smoking, when it may be reasonable to try again sooner.
- Record the person's blood pressure every 6 months.
- Do not offer lamotrigine to treat mania.
- Do not offer tigecycline to treat diabetic foot infections, unless other antibiotics are unsuitable.​

Recommendations for activities or interventions that could be offered
- Consider pelvic MRI to assess the extent of deep endometriosis involving the bowel, bladder or ureter.
- Consider referring people to other behavioural support services within the local health and care network (for example, to voluntary or community services) for interventions that are not available in the pharmacy.​
Click to expand...
​
Click to expand...
 
Is that related to this somehow?

Click to expand...


Code: 
https://twitter.com/NICEComms/status/1435634030196576259

https://indepth.nice.org.uk/methods-review/index.html#group-section-Share-your-views-67wiwXR1qw


Methods, processes and topic selection for health technology evaluation: proposals for change | NICE guidance | Our programmes |

https://www.nice.org.uk/about/what-...dance/chte-methods-and-processes-consultation

We are holding a series of virtual events to explain our proposals:
  • Methods focus (Tuesday 24 August, 4 to 5pm).
  • Highly specialised technologies criteria focus (Wednesday 1 September, 4 to 5pm).
  • Our process and topic selection proposals (Wednesday 8 September, 1 to 2pm).
  • Methods review: a focus on modifiers (Thursday 9 September, 2 to 3pm).
Find out more about the events and register to attend.
 
Last edited:
I'm curious to know if anyone has an opinion on who is likely to see this survey and who other than some people here is likely to fill it out. In other words, it would be good if some (by that I mean a lot) people with common sense and some basic integrity see it and give their thoughts.*

*I noticed that Jo Edwards has already done so.
 
Oddly, I have been asked to spend an hour online with a NICE representative discussing the topic of the survey. The call will be later in my day; tomorrow morning UK time. If people have more views about this topic, please do post on this thread today.
 
I do find it encouraging that my idea of good answers to the question of how could NICE improve its guidance (answers which are aligned with the thoughts of many of us here) are regarded by NICE as worth talking about.
 
Hutan said:
I do find it encouraging that my idea of good answers to the question of how could NICE improve its guidance (answers which are aligned with the thoughts of many of us here) are regarded by NICE as worth talking about.
Click to expand...

I posted here* re NICE's approach of using Cochrane review ---- if they're inviting comments then it might be an opportunity to highlight that Cochrane rated PACE etc. as "moderate" quality evidence and NICE (reasonably) found these studies to be "low and very low quality".

*
https://www.s4me.info/threads/uk-house-of-lords-house-of-commons-questions.707/page-24#post-376674
 
I've done the call, there were 3 women from NICE who were , well, very nice. They do genuinely want to make guidelines more useful. I guess that could mean that they are focused on the form rather than the substance, but getting form right is still an advance, and this team seemed to be trying hard.

There's a new format that is being trialled, first with some Covid-related guidelines. I think it's an improvement. Recommendations that are not based on evidence are clearly labelled as 'consensus recommendations'. There will be times when a consensus recommendation is a good thing, but it's good for it to be clear when a recommendation is just based on 'the committee thinks this is a good idea'. More information about why a recommendation was made is included along side the recommendation.

I did try, several times, to explain the problems with GRADE, like research with fatal flaws still being used to support recommendations. And the fact that it's all very well making much of the detail behind a recommendation easily available but, if the recommendation information says 'This recommendation is based on low quality evidence', most people won't realise that that could mean that actually the evidence is uninterpretable. I think the NICE women accepted that it could be good to explain actual problems with the evidence base (e.g. 'high risk of bias due to subjective outcomes in unblinded trials') beside the recommendation itself. But dealing with GRADE was outside their scope.

We talked about the guideline being a source document where the rationale for recommendations can be dug into. If doctors want a brief document with just the recommendations, there are online summaries they can access. Whereas I think the guideline should have a lot of the information relevant to a recommendation either in the recommendation or able to be clicked through from the recommendation - rather than having to go find sections at the end of the document or in another document with hundreds of pages.
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom