1. Guest, the 'News in Brief' for the week beginning 19th September 2022 is here.
    Dismiss Notice
  2. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Survey:Improving NICE guidance

Discussion in 'Other health news and research' started by Sly Saint, Aug 21, 2021.

  1. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    7,825
    Location:
    UK
    "Introduction

    Please complete this short survey to help us improve how we write and present our guidance. Your feedback is key to help us ensure our guidance is useful, useable and used.

    We think it will take you around 15 minutes to complete."

    https://nice.researchfeedback.net/s/improving_guidance
     
    MSEsperanza, Barry, Hutan and 6 others like this.
  2. Wonko

    Wonko Senior Member (Voting Rights)

    Messages:
    6,102
    Location:
    UK
    Are they havin' a larf?
     
    Yvonne, Jan, MSEsperanza and 15 others like this.
  3. Hutan

    Hutan Moderator Staff Member

    Messages:
    20,424
    Location:
    Aotearoa New Zealand
    :rofl: just 'presenting' it at all would be a start.

    I would suggest that, for any recommendations of treatment or advice to be given to patients, there should be a statement about what level of evidence there is.

    My understanding is that there is some sort of code in the guideline language, with 'offer <a treatment>' meaning a higher level of evidence backs the recommendation than 'consider <a treatment>'. But most readers will miss that; and it's not at all clear.

    I understand that sometimes recommendations that aren't evidence-based will be made, for reasons of political expediency or clinical necessity. But I'd like to see an unambiguous statement beside any recommendation or intervention. Something like, 'this recommendation is supported by substantial evidence of high quality', or 'this recommendation is currently not based on trial evidence', or 'this recommendation is supported by high quality evidence for adults, but there is no evidence in children and young people'.

    If clinicians are expected to present benefits and risks of a treatment or behaviour change accurately to patients, they should be given some guidance on the strength of the evidence for that intervention. It would also make it abundantly clear where the gaps in research are.
     
    Last edited: Aug 21, 2021
    Ash, Nellie, Jan and 23 others like this.
  4. Simbindi

    Simbindi Senior Member (Voting Rights)

    Messages:
    2,279
    Location:
    Somerset, England
    I think this is a big problem with the NICE guidelines. I've never looked at most of them but, for example, with the autism ones this information is in a much larger guideline that is available on a search, but not the main guideline that clinicians and patients use (now also called a 'Quality Standard'). The more in depth guideline is about 400 pages long and it explains the quality of evidence and the limitations of the various diagnostic measures and tests, where research is limited etc. But that is not going to be read even by the Clinical Psychologists carrying out autism diagnosis (based on my experience).

    I've just looked up the more recent children's version and the full guideline on that is nearly 800 pages in contrast to the 32 page main guideline:

    https://www.nice.org.uk/guidance/cg170/evidence/full-guideline-pdf-248641453

    https://www.nice.org.uk/guidance/cg...19s-support-and-management-pdf-35109745515205

    I have no idea if there will be a similar 'full guideline' for the 2021 (22/23/24?) ME/CFS NICE guideline.
     
    Last edited: Aug 22, 2021
    Ash, MSEsperanza, Wyva and 10 others like this.
  5. Hutan

    Hutan Moderator Staff Member

    Messages:
    20,424
    Location:
    Aotearoa New Zealand
    I did this, although it took me a lot longer than 15 minutes. It's interesting; NICE seems to be testing out providing more information related to recommendations, and how the information is laid out.

    The example recommendation they gave for comment included a short sentence saying it was based on low quality evidence, but did not say why the evidence was poor. And it might as well have been high quality evidence for all the difference it made to the recommendation. The example just carried right on with making the recommendation.

    I was a bit like a broken record saying 'you can't base a recommendation on low quality evidence'. Because we know that 'low quality evidence' actually means that it probably isn't telling us anything reliable about the outcome. 'Low quality evidence' is only evidence that the researchers of that particular question need to do a lot better. The problem is, if you haven't had the experience that we have had with NICE guidelines, then I don't think you'd give much weight to the brief statement hidden in all the detail of the recommendation saying the evidence is low quality. You'd assume that the recommendation is right and that it is evidence-based.

    On the outside chance that this has something to do with how NICE presents the ME/CFS recommendations, it would be great if some of our members could do the survey, and keep hammering home the point that low quality evidence is probably as useless as no evidence. And that we need to know what was wrong with the evidence. After all, several very small but good studies that all found the same thing might be useful. But one large study that is hopelessly biased, with no blinding and subjective outcomes probably isn't.
     
    ukxmrv, Louie41, MSEsperanza and 12 others like this.
  6. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    11,466
    Location:
    London, UK
    Yes, I did the survey. It is about something rather different but it gives one the opportunity to raise some questions about why NICE feel the need to give recommendations when there is no good evidence base.
     
    Ash, Joh, ukxmrv and 17 others like this.
  7. Sarah

    Sarah Senior Member (Voting Rights)

    Messages:
    1,478
    These are excerpts from section 9 (Writing the guideline) of Developing NICE guidelines: the manual Process and methods [PMG20]
    https://www.nice.org.uk/process/pmg20/chapter/writing-the-guideline#wording-the-recommendations

     
    Louie41, Simbindi, Michelle and 5 others like this.
  8. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    7,825
    Location:
    UK
  9. MSEsperanza

    MSEsperanza Senior Member (Voting Rights)

    Messages:
    2,394
    Location:
    betwixt and between
    Is that related to this somehow?

    Code:
    https://twitter.com/NICEComms/status/1435634030196576259
    https://indepth.nice.org.uk/methods-review/index.html#group-section-Share-your-views-67wiwXR1qw


    Methods, processes and topic selection for health technology evaluation: proposals for change | NICE guidance | Our programmes |

    https://www.nice.org.uk/about/what-...dance/chte-methods-and-processes-consultation

    We are holding a series of virtual events to explain our proposals:
    • Methods focus (Tuesday 24 August, 4 to 5pm).
    • Highly specialised technologies criteria focus (Wednesday 1 September, 4 to 5pm).
    • Our process and topic selection proposals (Wednesday 8 September, 1 to 2pm).
    • Methods review: a focus on modifiers (Thursday 9 September, 2 to 3pm).
    Find out more about the events and register to attend.
     
    Last edited: Sep 9, 2021
  10. Snowdrop

    Snowdrop Senior Member (Voting Rights)

    Messages:
    2,134
    Location:
    Canada
    I'm curious to know if anyone has an opinion on who is likely to see this survey and who other than some people here is likely to fill it out. In other words, it would be good if some (by that I mean a lot) people with common sense and some basic integrity see it and give their thoughts.*

    *I noticed that Jo Edwards has already done so.
     
  11. Hutan

    Hutan Moderator Staff Member

    Messages:
    20,424
    Location:
    Aotearoa New Zealand
    Oddly, I have been asked to spend an hour online with a NICE representative discussing the topic of the survey. The call will be later in my day; tomorrow morning UK time. If people have more views about this topic, please do post on this thread today.
     
    Ash, Sly Saint, FMMM1 and 13 others like this.
  12. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    11,466
    Location:
    London, UK
    You must have given good answers.
     
    Nellie, FMMM1, ahimsa and 6 others like this.
  13. Hutan

    Hutan Moderator Staff Member

    Messages:
    20,424
    Location:
    Aotearoa New Zealand
    I do find it encouraging that my idea of good answers to the question of how could NICE improve its guidance (answers which are aligned with the thoughts of many of us here) are regarded by NICE as worth talking about.
     
    Ash, Nellie, FMMM1 and 12 others like this.
  14. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    11,466
    Location:
    London, UK
    Indeed. I may have been a bit corrosive in my responses, which might have got me off the list as usual.
     
    Ash, FMMM1, shak8 and 5 others like this.
  15. FMMM1

    FMMM1 Senior Member (Voting Rights)

    Messages:
    1,898
    I posted here* re NICE's approach of using Cochrane review ---- if they're inviting comments then it might be an opportunity to highlight that Cochrane rated PACE etc. as "moderate" quality evidence and NICE (reasonably) found these studies to be "low and very low quality".

    *
    https://www.s4me.info/threads/uk-house-of-lords-house-of-commons-questions.707/page-24#post-376674
     
  16. FMMM1

    FMMM1 Senior Member (Voting Rights)

    Messages:
    1,898
    Entirely reasonably I'm sure ----. I wonder if they're looking for feedback on some superfluous crap rather than focusing on the quality of the guidelines?
     
    Hutan and Peter Trewhitt like this.
  17. Hutan

    Hutan Moderator Staff Member

    Messages:
    20,424
    Location:
    Aotearoa New Zealand
    I've done the call, there were 3 women from NICE who were , well, very nice. They do genuinely want to make guidelines more useful. I guess that could mean that they are focused on the form rather than the substance, but getting form right is still an advance, and this team seemed to be trying hard.

    There's a new format that is being trialled, first with some Covid-related guidelines. I think it's an improvement. Recommendations that are not based on evidence are clearly labelled as 'consensus recommendations'. There will be times when a consensus recommendation is a good thing, but it's good for it to be clear when a recommendation is just based on 'the committee thinks this is a good idea'. More information about why a recommendation was made is included along side the recommendation.

    I did try, several times, to explain the problems with GRADE, like research with fatal flaws still being used to support recommendations. And the fact that it's all very well making much of the detail behind a recommendation easily available but, if the recommendation information says 'This recommendation is based on low quality evidence', most people won't realise that that could mean that actually the evidence is uninterpretable. I think the NICE women accepted that it could be good to explain actual problems with the evidence base (e.g. 'high risk of bias due to subjective outcomes in unblinded trials') beside the recommendation itself. But dealing with GRADE was outside their scope.

    We talked about the guideline being a source document where the rationale for recommendations can be dug into. If doctors want a brief document with just the recommendations, there are online summaries they can access. Whereas I think the guideline should have a lot of the information relevant to a recommendation either in the recommendation or able to be clicked through from the recommendation - rather than having to go find sections at the end of the document or in another document with hundreds of pages.
     
    Last edited: Sep 22, 2021

Share This Page