Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Wessely always maintained that he was drwn to the subject because he had the patients' interests at heart. I see no reason to disbelieve him.

 

Really just trying to flag up the issues that the use of GRADE either directly by NICE or indirectly via Cochrane (Cochrane uses GRADE) is concerning - unblinded studies, which use subjective outcome criteria being labelled as "moderate" quality evidence that an intervention works (GET, CBT ----). There's an exchange here between @strategist and @Jonathan Edwards which sets out the problem*. OK NICE point out that they appraise the evidence independently - we want to ensure that happens (compare to the 2007 guideline).

I work at a very junior level in policy (not health policy), in a UK devolved administration. Some senior folks gave a policy workshop and one explained their experience of working with a Minister. Minister sent home with briefing papers --- key issues (according to senior officials). Monday morning, senior official meets Minister ---- turns out the Ministers questions relate to --- what can we do about X (and X is not an issue in the briefing papers). Turns out the Minister bumped into someone and they asked about X; so that is now an issue to be addressed. The lesson was the Ministers agenda isn't simply set via the formal route (briefing by officials) there are all sorts of other routes.

That's one of my motivations i.e. to try to influence. I copy my emails to NICE to the APPG etc.

Th other thing to bear in mind is that e.g. @Caroline Struthers excellent submission** might just be the opportunity NICE officials are looking for i.e. to highlight that linking up with Cochrane (and the use of GRADE) is causing problems re maintaining public confidence in NICE - production of evidenced based guidelines.

So really just having the conversations we have here, on this forum, with influencers (or those who are supposed to be)/decision makers.


*
https://www.s4me.info/threads/indep...ed-by-hilda-bastian.13645/page-80#post-390309

**https://healthycontrol.org/2021/11/26/letter-to-toby-lasserson-cochrane/

 

So only people who know nothing at all about an illness are qualified to understand it. Well, that explains an awful lot of the last 30 years.

 

Na, lived experience is good, as long as its "correct". e.g. Paul Garner, Recovery Norge.

All lived experiences are equal, but some are more equal than others.

 

Been avoiding sending this reply to (Katy) NICE* --- finally got around to it. Thank you all for your comments etc.

*"Katy,
thank you for your response.
Regarding your comment: "Each review question in the NICE guideline has its own review protocol agreed by the guideline committee. These protocols are different to those used by Cochrane reviews (for example, different inclusion and exclusion criteria), and the NICE guideline conducted its own independent systematic reviews based on Developing NICE guidelines: the manual<https://www.nice.org.uk/process/pmg20/chapter/introduction>"
I think these measures are essential i.e. so that NICE can produce evidence based reviews. I reiterate that the NICE guideline committee, the support technical staff and others in NICE, who participated in the development of the new ME/CFS guideline, have indeed reviewed the evidence and produced a sound guideline.

Leading figures in GRADE/Cochrane, such as Professor Paul Garner, have attacked the new ME/CFS guideline and praised others (involved in the discredited Lightening Process) who have also criticised the new guideline [Note*]. While such unwarranted, and unevidenced, criticisms are often considered the norm on social media, the fact that they are made by a leading figure in GRADE/Cochrane is evidence that NICE's collaborative agreement with Cochrane, and indeed NICE's use of GRADE, are not appropriate.

Understandably the ME/CFS community, and indeed those who view NICE as an example of how to produce clinical guidelines, are concerned by the unevidenced attacks on the new ME/CFS guideline.

To summarise, while I understand that NICE will attempt to manage the risks arising from the use of GRADE/Cochrane reviews, the evidence so far indicates that NICE's role is not compatible with the use of GRADE/Cochrane reviews. Indeed NICE's use of GRADE either directly, or indirectly (via Cochrane), is not compatible with the maintenance of public confidence in the system for producing guidelines.

Finally, NICE's review has identified that most of the (research) studies were unsuitable for evaluating interventions (exercise and CBT in this case). Typically the studies were unblinded and used subjective outcome criteria (questionnaires). The continued use of subjective outcome criteria is frustrating i.e. since actimetry (FitBit type devices) can be used to objectively measure the outcome of the intervention. Indeed, the PACE trial was supposed to use actimetry. The main funder of this research was the Department of Health and Social Care. It would be useful if NICE raised the issue of the poor quality of the research base with the Department of Health and Social Care - joined up Government.

Regards
Francis

*Note:
Paul Garner, a co-founder and influential member of Cochrane, said on Twitter “Nice (sic) lost the plot with GRADE with ME/CFS, according to the people that actually developed the methods: “a disastrous misapplication of GRADE methodology is the source of the problem” (https://twitter.com/user/status/1462734200797384709
).
Garner also describes Live Landmark, a Norwegian PhD student and Lighting Process practitioner as “Somebody being honest about the perversion of the GRADE process by Nice (sic) on ME/CFS” (https://twitter.com/user/status/1462095868832665601
)

 

https://twitter.com/user/status/1470380913431392260

 

Hi Folks

I have fiinally managed to write a follow-up to Gill Leng about the problems with the Cochrane/NICE collaboration idea after the response from Cochrane to my message to Toby Lasserson....will probably wait a bit before Tweeting it myself but you are welcome to...I can't tweak the blog as it is the email I sent. The Legal and General (old job) email addresses for Samantha Roberts bounced, so I may send it to her via Twitter...

https://healthycontrol.org/2021/12/31/letter-to-gill-leng-nice/

This letter is discussed in the thread "NICE and Cochrane sign collaborative agreement to deliver ‘living’ guideline recommendations" that can be found here.

 

posted on the PG thread
I found this podcast from 2014, Paul Garner :

https://cidg.cochrane.org/news/professor-paul-garner-discusses-his-work-cidg-coordinating-editor

 

So highly opinionated people, are complicated in systematic reviews because they tend to pull the interpretation of the results in a way that they think is right.

Says one of the most opinionated people I have ever met. I think "complicated" is rather understating the problem. Opinionated people also affect the methods of the review, such as choice of outcome measures and the application of GRADE, Risk of Bias etc. So they affect the results of the review, as well as the interpretation of the results.

 

Cochrane training manual:
Chapter 5: Collecting data

https://training.cochrane.org/handbook/current/chapter-05#_Ref515528070

(many issues raised do not appear to have been adhered to in previous reviews).

 

Don't want to take thread off topic but don't think starting a new thread is best either.

We haven't heard from Hilda recently but she has been posting on her blog still.

I thought some might be interested if they haven't seen them. Here's a link to some recent entries:

https://absolutelymaybe.plos.org/2021/10/31/the-metascience-movement-needs-to-be-more-self-critical/

https://absolutelymaybe.plos.org/2021/11/30/peeling-back-the-layers-of-medical-conservativism/

https://absolutelymaybe.plos.org/2021/12/31/5-things-we-learned-about-journal-peer-review-in-2021/

 

Interesting blogs.
There has been no "monthly" update about the review since August. I did chase Cochrane for an update once, but retracted that chase when i heard about hilda's bereavement. I have forwarded both the letter to Toby Lasserson and to Gill Leng to Hilda so she can add them to her repository of commentaries on the exercise review.

 

I just got an email alerting me to an event on tomorrow...for members of Oxford University only. It seems to be happening between 1pm and 1pm. I will email and ask for more details. Do let me know if you have any questions for Professor Montgomery ;-)

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams and is for members of the University only. Please email claire.murray@ethox.ox.ac.uk for further details.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics

 

They have now made it open to anyone on Teams @JohnTheJack !!!

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics

 

PS - It's from 1-2pm

 

Yes. It was not really relevant. I left before the end.

But I have just been to the Gill Leng thing, which was VERY relevant! https://zoom.us/webinar/register/WN_WxcfsQBkRGiC5rFegR_Xnw. I put a question (about Cochrane and its "black box" approach to reviews contrasting with NICE's transparency which she is rightfully proud of), but I was too late, and it didn't get put to her. I missed her answer to a question from Charles Shepherd about political influence because I was busy composing my own question. She talked a lot about her work with Cochrane, and she was forced to talk a bit about the ME guideline which I could tell she would rather not have done. Hopefully a recording will be made available. I didn't realise that her husband died in April last year, and she was very brave and dignified talking in some detail about that.


Mod note: There is further discussion about the Gill Leng interview here:
NICE and Cochrane sign collaborative agreement to deliver ‘living’ guideline recommendations