Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[FMMM1]

I think it would be interesting if there were legal liabilities associated knowingly with publishing poor medical advice. It would probably lead to significant changes in the way Cochrane do things and Journals would need to take corrections seriously

I think the key one is NICE's responsibility to produce evidence based guidelines. Also, the political accountability where NICE fails to produce evidence based guidelines --- I know, trying to make Teflon coated Departments/Ministers responsible for ensuring that guidelines are evidence based ----.

As for Cochrane, difficult to see how you could judicially review Cochrane ---- you'd have the great and the good coming out about "evidence". However, as others here have pointed out, those who claim to be "experts" re "evidence evaluation" inevitably seem to be entirely unreliable --- even about the basics.

 

[FMMM1]

The reference to "unpublished data" sounds a bit worrying - reminiscent of PACE (withholding data). I'm sure NICE would state that that the process of producing guidelines is transparent; however, it's one of those things --- you need to know the dodges/how to analyse the evidence before you can be confident that the outcome is evidence based.

 

[FMMM1]

Four YEARS of procrastination. How does Cochrane, even in their wildest fantasies, think ME sufferers believe there are honest, sincere or justified reasons for the inexcusable delay in removing the review.

Sounds reasonable maybe the challenge should be to NICE i.e. now it's signed up to a co-operative agreement!

Possibly a Parliamentary Question and/or an email to the APPGs (ME/CFS & Covid).

 

[Hutan]

A number of posts have been copied to The HANDI Working Group

 

[Caroline Struthers]

I only asked because of this mention in the linked article:

"In an exciting new development, the Royal College of General Practitioners (RCGP) is partnering with the RACGP to provide access to HANDI for UK GPs. This is a free resource so why not take a look?1 In addition, the RCGP will be contributing ideas for treatments to be included, helping with the review process, and linking RCGP resources to HANDI and creating UK-focused versions of HANDI advice."

Post copied to The HANDI Working Group

Good grief - this is awful

 

[Kalliope]

Apparently things have been moving forward according to Hilda Bastian on Twitter:

It has been moving forward: lack of progress reports does not necessarily mean lack of review progress - these involve different groups of people.

 

[Trish]

If I remember correctly, the first stage was intended to be the production by the review writing group of a protocol in consultation with the group Hilda appointed, then a public consultation on this protocol before going ahead with the review. I wonder where that process is up to. I guess we'll have to wait until some sort of update is provided.

 

[Kalliope]

Todd Davenport confirms to Bastian's tweet. But yes, would be nice with a proper update.

 

[Jonathan Edwards]

There is something very bizarre about the secrecy surrounding this process.
Normally confidentiality around a policy review would be to protect people who had taken on some sort of official duty to produce an independent report.

Cochrane might have seemed vaguely official in the past but it isn't. It is a group of self-appointed 'experts' who produce stuff with no official remit. I can see no reason why this process should be confidential, at any stage. The process is phoney. Recent outbursts from people like Flottorp and Garner, together with contributions by Glasziou make it clear that in the area of non-pharmacological treatments Cochrane hasn't a clue how to assess evidence quality. Its members are also indulging in misinformation and marketing of baseless treatments.

My only reason for thinking this exercise was worthwhile was that confronting some of the Cochrane people with the idiocy of the first review might lead to a slightly more enlightened approach. It is long past time that might have happened. It cannot take this long to not yet reach a decision that the evidence is worthless.

 

[Keela Too]

Another question is why Cochrane can’t accept the evidence reviews of the new guideline from NICE?

Why repeat the effort?

 

[Adrian]

If I remember correctly, the first stage was intended to be the production by the review writing group of a protocol in consultation with the group Hilda appointed, then a public consultation on this protocol before going ahead with the review. I wonder where that process is up to. I guess we'll have to wait until some sort of update is provided.

They have been talking about doing a new review for over 2 years (I heard them talk at the CMRC conference just over 2 years ago) and they still have not come up with a protocol let alone a review. It does lead to questions as to why things are so slow and what the delay is. If Cochrane were really serious about a new review they would have pushed things through much quicker.

 

[FMMM1]

public consultation

My initial reaction was that the responses to the consultation would be pretty damming (polite phrase) - even allowing for some delusion, and/or defences based on self interest, i.e. those responding on behalf of Cochrane, the whole idea of a consultation seems like a car crash with Cochrane the casualty.

I'm beginning to generate a bit of respect, for Cochrane, as I type --- if you know the responses to the consultation will be problematic, then don't ask the (consultation) questions/run the consultation.

Forget the idea of open minds and being prepared to make changes ----.

 

[CRG]

They have been talking about doing a new review for over 2 years (I heard them talk at the CMRC conference just over 2 years ago) and they still have not come up with a protocol let alone a review. It does lead to questions as to why things are so slow and what the delay is. If Cochrane were really serious about a new review they would have pushed things through much quicker.

I guess a lot of those who might be involved have had a major diversion of attention/resources* because of 30 months of pandemic. Certainly Hilda produced a lot of very valuable work on vaccine efficacy. Even people we are not well disposed towards may have experienced very profound impacts on both professional and personal lives, and frustrating as it is, it would be a bit ironic if as group of people whose lives have been shaped by illness, we didn't recognise the impacts of a pandemic on others.

*yes I know - a lot of that diversion of resources has been writing dodgy papers psychologising COVID

 

[Trish]

I understand all that, CRG. So why not a brief update saying that instead of silence.

I compare this to DecodeME which has had delays too, some due to the pandemic and some to the team including pwME. All this has been explained with regular updates and none of us are complaining that it's taking a bit longer than originally hoped.

Tweeted vague reassurance that something with the Cochrane review is happening is not adequate.

 

[Jonathan Edwards]

I guess a lot of those who might be involved have had a major diversion of attention/resources* because of 30 months of pandemic.

I see no need to make excuses for these people.
As eloquently explained by Peter Barry, the evidence now shows there is too little effect to be useful - we have a positive negative if you like, not just a don't know.
The Knoop paper showing that questionnaires do not reflect actual activity levels add to that a confirmation of the absurdity of the outcome measures used. But all the problems with these measures are problems of likely overoptimistic results. Since the results even so are not enough to bother with the whole thing is dead and buried.

It didn't take more than a couple of minutes to type that. A committee does not need more than half an hour to check the relevant data.

The whole thing is a Lilliputian Morris Dance to nowhere.

 

[CRG]

I see no need to make excuses for these people.
As eloquently explained by Peter Barry, the evidence now shows there is too little effect to be useful - we have a positive negative if you like, not just a don't know.
The Knoop paper showing that questionnaires do not reflect actual activity levels add to that a confirmation of the absurdity of the outcome measures used. But all the problems with these measures are problems of likely overoptimistic results. Since the results even so are not enough to bother with the whole thing is dead and buried.

It didn't take more than a couple of minutes to type that. A committee does not need more than half an hour to check the relevant data.

The whole thing is a Lilliputian Morris Dance to nowhere.

You are right of course. But - committees ! They have a life of their own, and are almost impossible to kill, they just go on and on unless someone decides to actively put them down. In the case of this advisory group, until its collective membership has decided it has done enough advising, or until someone gives it an absolute end date, it'll just continue to exist without acknowledging that it has no real purpose.

 

[Trish]

The group Hilda leads is an advisory group, but they are there to advise review team who will actually write the new Cochrane review. It would be possible for that team to give an update, and to publish the protocol as soon as it's ready as it surely should be by now.

 

[Adrian]

I guess a lot of those who might be involved have had a major diversion of attention/resources* because of 30 months of pandemic. Certainly Hilda produced a lot of very valuable work on vaccine efficacy. Even people we are not well disposed towards may have experienced very profound impacts on both professional and personal lives, and frustrating as it is, it would be a bit ironic if as group of people whose lives have been shaped by illness, we didn't recognise the impacts of a pandemic on others.

*yes I know - a lot of that diversion of resources has been writing dodgy papers psychologising COVID

Cochrane had choices if Hilda was not able to progress this as she was doing other things that she considered a higher priority then they could have looked for someone else to take on the task of somebody to support Hilda doing it.

If they wanted to delay they could have issued a schedule and withdrawn the previous review.

I would argue this review is important in the context of covid due to the numbers getting Long Covid and the likelihood that the Cochrane review is used to suggest ineffective and potentially harmful treatments to an increasing number of patients.

This is an issue with how the Cochrane editor chooses to prioritize things (leaving a harmful review up whilst allowing its replacement to drift). That may be a concious decision or it maybe that she simply does not consider it important enough to track. Either way it refects very badly on Cochrane as an organization.

 

[Adrian]

You are right of course. But - committees ! They have a life of their own, and are almost impossible to kill, they just go on and on unless someone decides to actively put them down. In the case of this advisory group, until its collective membership has decided it has done enough advising, or until someone gives it an absolute end date, it'll just continue to exist without acknowledging that it has no real purpose.

Its always good to give a date to a committee (with a goal). It may slip but that then becomes a concious decision and reasons and new proposed times need to be given.

I also wonder if they could have done a quick evidence review before trying to define a protocol. If they did they may simply have been able to say that they believe there does not exist a reasonable protocol to make sense of the evidence are provided by the current trials. (i.e. the evidence is too poor quality for inclusion into any compentant review).

 

[Caroline Struthers]

Another question is why Cochrane can’t accept the evidence reviews of the new guideline from NICE?

Why repeat the effort?

Exactly!! There is absolutely no reason for it at all. Cochrane have even signed a collaboration agreement with NICE to share data and analyses with each other to avoid the research waste this tortuous and weirdly secretive process of writing a new review with involvement of patients exemplifies. NICE involved patients in the process. Arguably more meaningfully than Cochrane is doing. And everything is publicly accessible, unlike this drip feed of reports which has stalled due to COVID and Hilda's understandable lack of availability. I have said that repeating the work NICE has done is pointless over and over to Cochrane directly and via Hilda. I have also said that it is unethical to keep the Exercise review on the Cochrane Library because the findings contradict NICE's evidence review. But it seems to make Cochrane dig its heels in even more. Trying not to take it personally ;-)

[edited at 22:39 to expand and hopefully clarify my points]