Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[InfiniteRubix]

My understanding is that Cochrane is morally and probably legally obliged

Precisely

There are zero contexts (outside of criminality) where legal obligations are not absolutely supreme. The remit of this review makes the requirements you outlined necessary for the creation of lawfully implementable conclusions.

The mirror image would imply that medics define law. They do not, however much we often insist on submitting to that falsity.

 

[InfiniteRubix]

Given the specifically habitual and normalised nature of unlawful clinical behaviour and administrative decision-making it is fundamental that this factor be made explicit

Malpractice is a norm in this field

 

[Caroline Struthers]

2 systematic reviewers from Cochrane Response were allocated to that work: I never considered that they weren't well-enough skilled and positioned to do it, or that it wasn't a good way to do this. I'm not going to discuss reasons for any individual not being part of the review team.

Do you know if the option was considered by the 2 systematic reviewers who did the work? I don't need to know the reasons why the option wasn't taken forward but I would like to know if it was considered or not.

 

[BurnA]

All I can say is the same thing again: the group includes people with different perspectives. I understand some are opposed to that concept completely

Can you explain this? It seems quite dismissive.
What do you mean by perspectives, we are all focused on science here, are you implying it's OK for someone to have a non scientific opinion because, hey that's just another perspective?
What concept are you referring to that you think you understand people are opposed to? Can you back that up please.? Who are you referring to specifically and what evidence supports your claim?
Thanks.

 

[Hilda Bastian]

This causes me serious concern @Hilda Bastian.

What we need from the exercise review is the best possible judgment on whether or not a set of trials reflect some real beneficial effect specific to certain treatments. Whether or not they do have an effect is a matter of fact, even if very difficult to ascertain. I do not like the word objective, which tends to get muddled in post-modernist type language but I think in this context we can see that there is an objective fact (hard to ascertain) as to whether exercise is beneficial in ME - and various subsidiary related facts of detail. There is therefore an objective fact of the matter as to whether the trials reflect a real effect.

My understanding is that Cochrane is morally and probably legally obliged to report on the evidence entirely on the a basis of the best possible rational case for whether the trials can be taken to reliably reflect a true effect. The reliability is also an objective fact just as the reliability of the weather forecast on wind speed and precipitation is an objective fact (but easier to ascertain). The lack of reliability of open trials with subjective endpoints in identifying a real effect as also based in good evidence.

To me it would be shocking and I think a breach of human rights to allow that points of view, or sociological factors other than simply the best possible attempt to ascertain objective truth here should have any influence at all. To do so would be to admit that it was legitimate for Cochrane to favour a treatment even if the was no good evidence that it worked or to discourage a treatment even if there was evidence that it did work. There is absolutely no case for having people with 'different points of view' authoring a review.

I hope I have misunderstood what you were suggesting but it does sound tantamount to an admission that Cochrane does not follow the policy it advertises and which it is very likely bound in law to stick to.

Would anyone with an official position in Cochrane endorse the view I think is being suggested?

People working together and talking to each other is a social process, and they're a little community when they work together for a long time - that's the "society" I was referring to.

 

[InfiniteRubix]

people with different perspectives

A perspective that is only relevant to unlawful clinical judgement or decision making is not a mere perspective.

It is contributory to that unlawful behaviour, which can also be unindemnified and/or criminal (context dependent).

The loose analogy, and the fact, in extremis, is that such a "perspective" would actually make someone an accessory.

 

[Jonathan Edwards]

People working together and talking to each other is a social process, and they're a little community when they work together for a long time - that's the "society" I was referring to.

So what on earth has that to do with remarks about objectivity and cultural differences and all that stuff?

I don't buy that as a response at all I am afraid. I would echo @InfiniteRubix 's echo of what I said. This is really serious legal stuff.

I actually think that in the context of the Cochrane background you have picked what might be close to an optimal mix of people and may well be an inspired choice. I don't want to derail that, but talk of nothing objective, sociological and cultural factors and that dreadful word 'embedded' (which as an editor for a multidisciplinary journal always gives me the jeeps) seems to me way outside what is legitimate.

Would anyone at Cochrane endorse these comments?
I am happy if you don't answer. I shall take this as an aberration.

 

[InfiniteRubix]

I will be rudely facetious to make a point. Organised crime is also a social process.

People merely holding a view does not mean there is any lawful merit, just because they have a view.

Merely having a perspective does not negate it's unlawfulness either and/or its contribution to law breaking.

Unlawful conduct stemming from a perspective is not santised legally by either normalised malpractice or habitual low standards.

Excess and unmerited respect contributes to the liability profile.

 

[Hoopoe]

My view is that subjective measures of health in an unblinded situation can easily be demonstrated to be misleading. In ME/CFS research we see over and over again how positive effects in such studies are treated as evidence that the treatment is effective. There may be an acknowledgement somewhere that the results aren't entirely reliable but the de facto they are still treated as if we could be sure that that the positive effect contains at least some treatment effects. The possibility that there is no treatment effect at all is virtually never considered and it is one of those facts that are too inconvenient and difficult to acknowledge for many.

One of the negative effects this has on ME/CFS research is that incompetent researchers continue to receive funding which slows down progress and has probably already delayed the day that diagnostic tests and genuinely effective treatments are available (maybe by decades). This would all stop if organizations like Cochrane stopped giving credibility to studies with such demonstratably misleading methods. Cochrane is very much part of the problem.

I'm sure there are people who have different perspectives on this but I don't believe that they could demonstrate that they can somehow estimate the true treatment effect from such trials (such claims to me look like complete pseudoscience to be honest).

PS: and the reason assumptions about treatment effects are a bigger problem in the context of ME/CFS is because the illness objectively doesn't respond as one would expect to exercise. In this context it's plausible that the true composition of an effect seen in subjective outcomes is not a positive treatment effect plus positive nonspecific effects but a negative treatment effect that is hidden by positive nonspecific effects.

 

[InfiniteRubix]

I will be rudely facetious to make a point. Organised crime is also a social process.

People merely holding a view does not mean there is any lawful merit, just because they have a view.

Merely having a perspective does not negate it's unlawfulness either and/or its contribution to law breaking.

Unlawful conduct stemming from a perspective is not santised legally by either normalised malpractice or habitual low standards.

Excess and unmerited respect contributes to the liability profile.

It is also worth adding that membership of an advisory group is not a place for people to acquire the minimal educational standards that may be lacking amongst some "perspectives."

Sufficient literacy should have been prerequisite. Merely holding a view despite scientific standards is not a sign of adequate literacy and implies prejudicial determination of group membership.

This further informs the legal context acutely when judging the process and outcomes.

 

[MEMarge]

I think that sadly she would be too unwell to be part of the process. Alongside working part-time and a young family, she is already working hard on research, advocacy and medical education.

However listening to some of the videos she has produced, such as the Canadian one would be really useful to improve the understanding of this IA Group, as would the Invisible Illness videos and compulsory reading of @Jonathan Edwards expert testimony to NICE.

To be honest a fair amount of Cochrane's own videos/blogs etc on stats/bias etc isn't bad. It just doesn't seem to apply to any papers on ME.

Sorry for slow reply. I was posting in haste and meant Neglected Illness as in Natalie Boulton's dialogues for a neglected illness: https://www.dialogues-mecfs.co.uk/videos/

The talk that Nina gave was a few months ago for a Canadian ME group/organisation, maybe BC. I haven't been able to find the link in a brief search today. Maybe @rvallee or someone else can remember more.

Thanks @Sly Saint for responding sooner, you are always great at locating articles etc.

 

[Hilda Bastian]

Do you know if the option was considered by the 2 systematic reviewers who did the work? I don't need to know the reasons why the option wasn't taken forward but I would like to know if it was considered or not.

Your question included considering whether particular named individuals would be "part of" the team.

None of the authors participated in any of the meetings to discuss who the authors would be. I have no idea what the 2 systematic reviewers who are doing the work have thought about. They work for Cochrane Response, and agreed to participate when asked.

Edit: missed a word.

 

[Hutan]

It is important to note that Dr Newton recommends activity-based approaches such as graded exercise and advice on deconditioning as a first-line intervention for ME/CFS.

She recommends the Lightning Process, CBT and mindfulness (27:46 timestamp in the video below).

Thanks so much @cassava7 for the 2012 video. I found a video of Professor Julia Newton speaking about GET this year - link here
Professor Julia Newton on GET. My concern about her presence on the writing team grows. There is no evidence that the interventions related to activity that she applies to people wth fatigue (the gradual 10% increase in activity) make any difference to their illness trajectories.

I actually think that in the context of the Cochrane background you have picked what might be close to an optimal mix of people and may well be an inspired choice.

I've seen you say this a couple of times, and I been puzzled both times. Given your attitudes to the idea that it is possible to discern something approximating the truth about the evidence for GET, why would you be in favour of including people in the team who have demonstrated that they cannot discern that truth even when they have spent a career investigating GET and applying it to patients? Do you think that politics makes it necessary, and the risks from having people without the skills to properly evaluate the GET trials will be sufficiently well managed?

 

[Adrian]

Thanks so much @cassava7 for the 2012 video. I found a video of Professor Julia Newton speaking about GET this year - link here
Professor Julia Newton on GET. My concern about her presence on the writing team grows. There is no evidence that the interventions related to activity that she applies to people wth fatigue (the gradual 10% increase in activity) make any difference to their illness trajectories.

I assume that it is a conflict of interest if her clinic is busy doing GET (by what ever name she chooses to call it) and a potential serious issue of NICE say its not effective and she plans to keeps billing the work to the NHS.

Of course perhaps since that video (which was a long time ago) she has wondered why patients seem to get worse or don't come back?

 

[Hutan]

Of course perhaps since that video (which was a long time ago) she has wondered why patients seem to get worse or don't come back?

The second video is from February of this year. So, if she wondered at any point, that did not stop her carrying on. I guess enough patients got better to make her think that she was helping. But, with the published results from trials, and her own studies, I don't know how she could not realise most people in the first couple of years with post-viral fatigue syndrome/ME/CFS actually do recover on their own.

 

[InfiniteRubix]

Thanks so much @cassava7 for the 2012 video. I found a video of Professor Julia Newton speaking about GET this year - link here
Professor Julia Newton on GET. My concern about her presence on the writing team grows. There is no evidence that the interventions related to activity that she applies to people wth fatigue (the gradual 10% increase in activity) make any difference to their illness trajectories.

I've seen you say this a couple of times, and I been puzzled both times. Given your attitudes to the idea that it is possible to discern something approximating the truth about the evidence for GET, why would you be in favour of including people in the team who have demonstrated that they cannot discern that truth even when they have spent a career investigating GET and applying it to patients? Do you think that politics makes it necessary, and the risks from having people without the skills to properly evaluate the GET trials will be sufficiently well managed?

Something may be politically necessary to accommodate others' unlawful behaviour or their willingness to maintain legally unsustainable standards. But avoiding the enforcement of reality does not remove implications of unlawfulness due to that mere political or practical expediency. Accomodation or facilitation of unlawful acts are exactly that - an ugly and common coping mechanism in our world.

Expediency and our "battered wife syndrome" are two sides of the same coin - keeping legal realities in box, for fear of hurting the sensitivities of the illiterate and/or the morally dubious.

 

[Sly Saint]

The talk that Nina gave was a few months ago for a Canadian ME group/organisation, maybe BC. I haven't been able to find the link in a brief search today. Maybe @rvallee or someone else can remember more.

Thanks @Sly Saint for responding sooner, you are always great at locating articles etc.

https://www.s4me.info/threads/news-from-canada.18994/#post-332638

 

[Caroline Struthers]

Your question included considering whether particular named individuals would be "part of" the team.

None of the authors participated in any of the meetings to discuss who the authors would be. I have no idea what the 2 systematic reviewers who are doing the work have thought about. They work for Cochrane Response, and agreed to participate when asked.

Edit: missed a word.

I revised the question to ask whether the suggestion of inviting Vink and Vink-niese as authors who had previously published a convertible review had been discussed. If I never made the suggestion before, there's no reason why it would have been. I can't remember whether I submitted it to the IAG email address or via Twitter or at all, and wondered if there was a log of de-duplicated suggestions anywhere. I can look back over my own emails and Twitter if it's easier. I didn't realise that an author team outside Cochrane staff was being assembled as I think the March 2020 report only said that there would be an IAG advising a Cochrane response author team (I haven't looked back at what was said in March 2020). It's great there is a more diverse author team but no one knew anything about it until recently.

The review which inspired the ACTIVE project which I was happy to be involved in (https://training.cochrane.org/resource/involving-consumers-cochrane-reviews-learning-active-project) and subsequently this pilot for high-profile reviews is this one https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001920.pub3/

There was no IAG, but a panel of stakeholders (not authors) who together made key decisions during the review process. It's an inspiring story and I think a model which should and could be adopted for all reviews, not only high profile ones. It is practical, sustainable and the review author team received funding for it specifically because of their commitment to the meaningful involvement of as many patients/carers and other stakeholders as possible.

This is a report about how they did things. https://training.cochrane.org/sites...volvement in a Cochrane systematic review.pdf

I think I suggested Alex Pollock (lead of the review and the ACTIVE project) as a good person to be involved in this pilot too. I will look back over the correspondence and Twitter.

 

[Jonathan Edwards]

Given your attitudes to the idea that it is possible to discern something approximating the truth about the evidence for GET, why would you be in favour of including people in the team who have demonstrated that they cannot discern that truth even when they have spent a career investigating GET and applying it to patients? Do you think that politics makes it necessary, and the risks from having people without the skills to properly evaluate the GET trials will be sufficiently well managed?

Hilda has partly answered this herself. I think a proportion of the workforce involved would have been non-negotiable in practice. The real task here is to confront Cochrane on its own 'cognitive dissonance'. I think the way to do that is to mix all the disparate viewpoints, including one or two people who really understand the problem, so that everyone at least knows where everyone else is coming from.

I am not saying that I think that a sensible conclusion will be reached but what actually gets written in this review is probably of less importance than trying to get into the public domain how much of a mess things are in, with the possibility of getting people to face up to how muddled their thinking is.

The objectivity thing is probably a red herring. Both NICE and Cochrane have scored the evidence low or very low I think. The problem is that some people feel unable to not give a recommendation for non-pharmacological treatments like this even if the basis is as poor as it gets. Glasziou has said words to the effect that 'oh, but we know it works from clinical experience'. Yet as I pointed out to NICE you can see from PACE that nobody could know that. And what is the point in a systematic review if trumped by anecdote? The cultural influence here is in not being able to resist recommending a treatment you have faith in, in the face of clear lack of usable evidence.

 

[Hilda Bastian]

I revised the question to ask whether the suggestion of inviting Vink and Vink-niese as authors who had previously published a convertible review had been discussed. If I never made the suggestion before, there's no reason why it would have been. I can't remember whether I submitted it to the IAG email address or via Twitter or at all, and wondered if there was a log of de-duplicated suggestions anywhere. I can look back over my own emails and Twitter if it's easier. I didn't realise that an author team outside Cochrane staff was being assembled as I think the March 2020 report only said that there would be an IAG advising a Cochrane response author team (I haven't looked back at what was said in March 2020). It's great there is a more diverse author team but no one knew anything about it until recently.

The review which inspired the ACTIVE project which I was happy to be involved in (https://training.cochrane.org/resource/involving-consumers-cochrane-reviews-learning-active-project) and subsequently this pilot for high-profile reviews is this one https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001920.pub3/

There was no IAG, but a panel of stakeholders (not authors) who together made key decisions during the review process. It's an inspiring story and I think a model which should and could be adopted for all reviews, not only high profile ones. It is practical, sustainable and the review author team received funding for it specifically because of their commitment to the meaningful involvement of as many patients/carers and other stakeholders as possible.

This is a report about how they did things. https://training.cochrane.org/sites/training.cochrane.org/files/public/uploads/resources/downloadable_resources/Pollock et al 2015 User involvement in a Cochrane systematic review.pdf

I think I suggested Alex Pollock (lead of the review and the ACTIVE project) as a good person to be involved in this pilot too. I will look back over the correspondence and Twitter.

Thanks for the links about the ACTIVE project.

Here's the text from March 2020 - the page is archived here:

The review’s revision and update will be conducted by systematic reviewers from Cochrane Response (Cochrane’s evidence consultancy unit), and additional new co-authors are being considered.

(Edit: removed italics)