Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Esther12 said:
What are you hoping to achieve with this correspondence?
Click to expand...

Really just trying to flag up the issues that the use of GRADE either directly by NICE or indirectly via Cochrane (Cochrane uses GRADE) is concerning - unblinded studies, which use subjective outcome criteria being labelled as "moderate" quality evidence that an intervention works (GET, CBT ----). There's an exchange here between @strategist and @Jonathan Edwards which sets out the problem*. OK NICE point out that they appraise the evidence independently - we want to ensure that happens (compare to the 2007 guideline).

I work at a very junior level in policy (not health policy), in a UK devolved administration. Some senior folks gave a policy workshop and one explained their experience of working with a Minister. Minister sent home with briefing papers --- key issues (according to senior officials). Monday morning, senior official meets Minister ---- turns out the Ministers questions relate to --- what can we do about X (and X is not an issue in the briefing papers). Turns out the Minister bumped into someone and they asked about X; so that is now an issue to be addressed. The lesson was the Ministers agenda isn't simply set via the formal route (briefing by officials) there are all sorts of other routes.

That's one of my motivations i.e. to try to influence. I copy my emails to NICE to the APPG etc.

Th other thing to bear in mind is that e.g. @Caroline Struthers excellent submission** might just be the opportunity NICE officials are looking for i.e. to highlight that linking up with Cochrane (and the use of GRADE) is causing problems re maintaining public confidence in NICE - production of evidenced based guidelines.

So really just having the conversations we have here, on this forum, with influencers (or those who are supposed to be)/decision makers.


*
https://www.s4me.info/threads/indep...ed-by-hilda-bastian.13645/page-80#post-390309

**https://healthycontrol.org/2021/11/26/letter-to-toby-lasserson-cochrane/
 
adambeyoncelowe said:
It's also worth bearing in mind that the very idea that membership in a patient group is "activism", "militancy" and "bias" is inherently ableist. If advocating for ourselves is considered verboten by Cochrane and the IAG, that is institutionalising our disempowerment and a Victorian "doctor knows best" attitude.
Click to expand...
Wessely asserted early on that people who have the lived experience of a disease/condition are not best placed to understand it.

Apparently only people who have no lived experience of it can do that. :rolleyes:
 
Sean said:
Wessely asserted early on that people who have the lived experience of a disease/condition are not best placed to understand it.
Click to expand...

Midnattsol said:
Wyller said in an opinion piece that being close to someone with the illness is also disqualifying.
Click to expand...

So only people who know nothing at all about an illness are qualified to understand it. Well, that explains an awful lot of the last 30 years.
 
FMMM1 said:
Finally got around to drafting a reply* to NICE (Katy's) reply**. Grateful for your suggestions.

I've plagiarised some of your quotes @Caroline Struthers !

*Me to NICE (Katy):
"Regarding your comment: "Each review question in the NICE guideline has its own review protocol agreed by the guideline committee. These protocols are different to those used by Cochrane reviews (for example, different inclusion and exclusion criteria), and the NICE guideline conducted its own independent systematic reviews based on Developing NICE guidelines: the manual<https://www.nice.org.uk/process/pmg20/chapter/introduction>"

I think this is essential i.e. so that NICE can produce evidenced based reviews. I reiterate that the NICE guideline committee, the support technical staff and others in NICE, who participated in the development of the new ME/CFS guideline, have indeed reviewed the evidence and produced a sound guideline. However, leading figures in GRADE/Cochrane, such as Professor Paul Garner, have attacked the new guideline and praised others (involved in the discredited Lightening Process) who have also criticised the new guideline [Note 1]. While such unwarranted, and unevidenced, criticisms are often considered the norm on social media, the fact that they are made by a leading figure in GRADE/Cochrane is evidence that NICEs collaborative agreement is not appropriate.

Understandably the ME/CFS community, and indeed those who view NICE as an example of how to produce clinical guidelines, are concerned by the unevidenced attacks on the new ME/CFS guideline.

To summarise, while I understand that NICE will attempt to manage its use of GRADE/Cochrane reviews, the evidence so far indicates that NICE's role is incompatible with the use of GRADE/Cochrane reviews and indeed the maintenance of public confidence in the system for producing guidelines.

EDIT - I think I need to encourage NICE to highlight that much research funded by the Department of Health and Social Care (which also funds NICE) is unsuitable for use in guidelines - unblinded + subjective outcomes. Joined up Government and all that!

Regards

NOTE -
Paul Garner, a co-founder and influential member of Cochrane, said on Twitter “Nice (sic) lost the plot with GRADE with ME/CFS, according to the people that actually developed the methods: “a disastrous misapplication of GRADE methodology is the source of the problem” (16). Garner also describes Live Landmark, a Norwegian PhD student and Lighting Process practitioner as “Somebody being honest about the perversion of the GRADE process by Nice (sic) on ME/CFS” (17)


**NICE (Katy) to me:
Click to expand...

Been avoiding sending this reply to (Katy) NICE* --- finally got around to it. Thank you all for your comments etc.

*"Katy,
thank you for your response.
Regarding your comment: "Each review question in the NICE guideline has its own review protocol agreed by the guideline committee. These protocols are different to those used by Cochrane reviews (for example, different inclusion and exclusion criteria), and the NICE guideline conducted its own independent systematic reviews based on Developing NICE guidelines: the manual<https://www.nice.org.uk/process/pmg20/chapter/introduction>"
I think these measures are essential i.e. so that NICE can produce evidence based reviews. I reiterate that the NICE guideline committee, the support technical staff and others in NICE, who participated in the development of the new ME/CFS guideline, have indeed reviewed the evidence and produced a sound guideline.

Leading figures in GRADE/Cochrane, such as Professor Paul Garner, have attacked the new ME/CFS guideline and praised others (involved in the discredited Lightening Process) who have also criticised the new guideline [Note*]. While such unwarranted, and unevidenced, criticisms are often considered the norm on social media, the fact that they are made by a leading figure in GRADE/Cochrane is evidence that NICE's collaborative agreement with Cochrane, and indeed NICE's use of GRADE, are not appropriate.

Understandably the ME/CFS community, and indeed those who view NICE as an example of how to produce clinical guidelines, are concerned by the unevidenced attacks on the new ME/CFS guideline.

To summarise, while I understand that NICE will attempt to manage the risks arising from the use of GRADE/Cochrane reviews, the evidence so far indicates that NICE's role is not compatible with the use of GRADE/Cochrane reviews. Indeed NICE's use of GRADE either directly, or indirectly (via Cochrane), is not compatible with the maintenance of public confidence in the system for producing guidelines.

Finally, NICE's review has identified that most of the (research) studies were unsuitable for evaluating interventions (exercise and CBT in this case). Typically the studies were unblinded and used subjective outcome criteria (questionnaires). The continued use of subjective outcome criteria is frustrating i.e. since actimetry (FitBit type devices) can be used to objectively measure the outcome of the intervention. Indeed, the PACE trial was supposed to use actimetry. The main funder of this research was the Department of Health and Social Care. It would be useful if NICE raised the issue of the poor quality of the research base with the Department of Health and Social Care - joined up Government.

Regards
Francis

*Note:
Paul Garner, a co-founder and influential member of Cochrane, said on Twitter “Nice (sic) lost the plot with GRADE with ME/CFS, according to the people that actually developed the methods: “a disastrous misapplication of GRADE methodology is the source of the problem” ().
Garner also describes Live Landmark, a Norwegian PhD student and Lighting Process practitioner as “Somebody being honest about the perversion of the GRADE process by Nice (sic) on ME/CFS” ()
 
Hi Folks

I have fiinally managed to write a follow-up to Gill Leng about the problems with the Cochrane/NICE collaboration idea after the response from Cochrane to my message to Toby Lasserson....will probably wait a bit before Tweeting it myself but you are welcome to...I can't tweak the blog as it is the email I sent. The Legal and General (old job) email addresses for Samantha Roberts bounced, so I may send it to her via Twitter...

https://healthycontrol.org/2021/12/31/letter-to-gill-leng-nice/

This letter is discussed in the thread "NICE and Cochrane sign collaborative agreement to deliver ‘living’ guideline recommendations" that can be found here.
 
Last edited by a moderator:
posted on the PG thread
I found this podcast from 2014, Paul Garner :

...
People had spent an enormous amount of time doing primary research about the effects of healthcare but hadn’t invested the same scientific methods in producing the summaries of that.

So this group said ‘this is a scandal’ and that it needs fixing.
And so it was an organisation that came together around all aspects of healthcare. We’ve just got one little bit of the picture that we help co-ordinate from here. The systematic reviews are different from other publications because they are published entirely electronically and they use standard methods that the whole collaboration has produced, and they are updated over time. In other words as new trials appear, these are incorporated into new editions of the review that are then published.
The other critical point about Cochrane reviews is they are very clear about interpreting the evidence in the light of the quality of that evidence which not all reviews do.
The main group of people producing a review are the review team.
So the review team are critical to the process. The review team consists of several authors. They need people with topic experience, understanding of the topic. It needs people with statistical knowledge, and methological knowledge.. So this group of people need to get together.
The authors of these reviews need to be fastidious, careful, people, that are not highly opinionated. So highly opinionated people, are complicated in systematic reviews because they tend to pull the interpretation of the results in a way that they think is right.
Click to expand...
https://cidg.cochrane.org/news/professor-paul-garner-discusses-his-work-cidg-coordinating-editor
 
Sly Saint said:
posted on the PG thread
I found this podcast from 2014, Paul Garner :


https://cidg.cochrane.org/news/professor-paul-garner-discusses-his-work-cidg-coordinating-editor
Click to expand...
So highly opinionated people, are complicated in systematic reviews because they tend to pull the interpretation of the results in a way that they think is right.

Says one of the most opinionated people I have ever met. I think "complicated" is rather understating the problem. Opinionated people also affect the methods of the review, such as choice of outcome measures and the application of GRADE, Risk of Bias etc. So they affect the results of the review, as well as the interpretation of the results.
 
Cochrane training manual:
Chapter 5: Collecting data

5.1 Introduction#section-5-1
Systematic reviews aim to identify all studies that are relevant to their research questions and to synthesize data about the design, risk of bias, and results of those studies. Consequently, the findings of a systematic review depend critically on decisions relating to which data from these studies are presented and analysed. Data collected for systematic reviews should be accurate, complete, and accessible for future updates of the review and for data sharing. Methods used for these decisions must be transparent; they should be chosen to minimize biases and human error. Here we describe approaches that should be used in systematic reviews for collecting data, including extraction of data directly from journal articles and other reports of studies.
Click to expand...

https://training.cochrane.org/handbook/current/chapter-05#_Ref515528070

(many issues raised do not appear to have been adhered to in previous reviews).
 
Don't want to take thread off topic but don't think starting a new thread is best either.

We haven't heard from Hilda recently but she has been posting on her blog still.

I thought some might be interested if they haven't seen them. Here's a link to some recent entries:

https://absolutelymaybe.plos.org/2021/10/31/the-metascience-movement-needs-to-be-more-self-critical/

https://absolutelymaybe.plos.org/2021/11/30/peeling-back-the-layers-of-medical-conservativism/

https://absolutelymaybe.plos.org/2021/12/31/5-things-we-learned-about-journal-peer-review-in-2021/
 
Snowdrop said:
Don't want to take thread off topic but don't think starting a new thread is best either.

We haven't heard from Hilda recently but she has been posting on her blog still.

I thought some might be interested if they haven't seen them. Here's a link to some recent entries:

https://absolutelymaybe.plos.org/2021/10/31/the-metascience-movement-needs-to-be-more-self-critical/

https://absolutelymaybe.plos.org/2021/11/30/peeling-back-the-layers-of-medical-conservativism/

https://absolutelymaybe.plos.org/2021/12/31/5-things-we-learned-about-journal-peer-review-in-2021/
Click to expand...
Interesting blogs.
There has been no "monthly" update about the review since August. I did chase Cochrane for an update once, but retracted that chase when i heard about hilda's bereavement. I have forwarded both the letter to Toby Lasserson and to Gill Leng to Hilda so she can add them to her repository of commentaries on the exercise review.
 
JohnTheJack said:
After Montgomery's letter to the committee, I sent the attached response to him. He passed it on to the HRA who replied as follows:



I have copied the letter (attached) to a few people but nothing came of it. Misleading parliament is a very serious matter. I would note in particular in this regard points 9 and 13.
Click to expand...
I just got an email alerting me to an event on tomorrow...for members of Oxford University only. It seems to be happening between 1pm and 1pm. I will email and ask for more details. Do let me know if you have any questions for Professor Montgomery ;-)

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams and is for members of the University only. Please email claire.murray@ethox.ox.ac.uk for further details.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics
 
Caroline Struthers said:
I just got an email alerting me to an event on tomorrow...for members of Oxford University only. It seems to be happening between 1pm and 1pm. I will email and ask for more details. Do let me know if you have any questions for Professor Montgomery ;-)

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams and is for members of the University only. Please email claire.murray@ethox.ox.ac.uk for further details.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics
Click to expand...
They have now made it open to anyone on Teams @JohnTheJack !!!

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics
 
Caroline Struthers said:
They have now made it open to anyone on Teams @JohnTheJack !!!

Richard Doll Seminar, Tuesday 25 January 2022, 1:00 – 1:00pm

This seminar will take place on MS Teams.


Speaker: Professor Jonathan Montgomery, Professor of Healthcare Law, University College London and Visiting Professor at the Ethox Centre

Title: ‘Lay’ and expert evidence in public bioethics
Click to expand...
PS - It's from 1-2pm
 
Adrian said:
Did you go?
Click to expand...
Yes. It was not really relevant. I left before the end.

But I have just been to the Gill Leng thing, which was VERY relevant! https://zoom.us/webinar/register/WN_WxcfsQBkRGiC5rFegR_Xnw. I put a question (about Cochrane and its "black box" approach to reviews contrasting with NICE's transparency which she is rightfully proud of), but I was too late, and it didn't get put to her. I missed her answer to a question from Charles Shepherd about political influence because I was busy composing my own question. She talked a lot about her work with Cochrane, and she was forced to talk a bit about the ME guideline which I could tell she would rather not have done. Hopefully a recording will be made available. I didn't realise that her husband died in April last year, and she was very brave and dignified talking in some detail about that.


Mod note: There is further discussion about the Gill Leng interview here:
NICE and Cochrane sign collaborative agreement to deliver ‘living’ guideline recommendations
 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top Bottom