Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

Apologies for just popping in.

Don't know where else to put this so leaving some notes here regarding caveats when critiquing most common features of some researchers' claims about trial methodology and alleged benefits from therapist-delivered treatments for ME/CFS.

- Objective outcomes:

It's true that objectivity with regard to outcome measures is always a 'spectrum'.

And even an indisputably objective outcome like death needs to be interpreted: Is it caused / delayed due to the investigated treatment or due to an other cause?

So, there is a hierarchy of robustness with regards to objective outcome measures and those outcomes need to be analyzed properly, but on the other hand, there are subjective outcomes that clearly are subjective, and only subjective -- both subjectively reported and only subjectively assessable.

The point that certain objective outcomes are not entirely objective is not valid with regard to differentiating between subjective and objective outcomes in the context of reliable trial methodology.

- Improvement versus recovery

This point is somewhat related to objective outcomes like going back to work or school.

It there is no cure, of course every relevant improvement should also be counted as benefit, even if it doesn't lead to restored working ability.

I think that needs to be taken into account when evaluating treatment trials that claim to have found evidence for improvement of symptoms but not for restoring working ability.

If there were any reliably measurable, relevant improvements, that actually would be better than nothing. But behavorial treatment and physiotherapy ME/CFS trials I'm aware of didn't show evidence of improvement beyond a short-term placebo effect in a minority of patients.

I think though that selling placebo as evidenced-based treatment could be counter-productive to coping with a life-altering illness and could cause long term mental suffering and other harm.

Perhaps it would be worthwhile to discuss what people with ME see as relevant improvements if they can't go back to work or school?

This post has been copied to start a new discussion thread:Clinical trial outcome measures of improvement and recovery in ME/CFS - which ones are useful? Discussion thread
 
Last edited by a moderator:
Robert 1973 said:
“At the appropriate juncture, in due course, in the fullness of time” as Sir Humphrey might say.

As far as I’m aware we’ve not had any news of progress since Hilda was appointed IAG lead over a year ago. Last April we were told there would be a progress report “soon”. In February we were told “A progress report is a high priority, and plan to have it out well before the end of this month.”

@Hilda Bastian Can you tell us whether any members of the IAG have been appointed yet?
Click to expand...
I think we’ve moved on from long grass to tumbleweed
 
One of my jobs was trying to get people in an organisation to do a better job on addressing complaints rather than ignoring and stonewalling. We used to get people to put together a timeline covering what had generated the complaint and also the steps involved in dealing with it. This would help identify any issues that could have prevented the complaint arising in the first place and also any issues that prevented the complaint being addressed in a reasonable timeframe. Tempting to have a go at a timeline for the complaint about the exercise in MECFS review.

As a public sector organisation we were subject to review by the ombudsman so bad examples of poor work would eventually be taken up by the ombudsman’s office for investigation and anonymised case studies would be included in their published reports to Parliament. So we were very aware of the accountability for our organisation’s actions.

What strikes me about these Science organisations is despite the influence their work has there appears to be at best minimal accountability, in practice no accountability.
 
NelliePledge said:
One of my jobs was trying to get people in an organisation to do a better job on addressing complaints rather than ignoring and stonewalling. We used to get people to put together a timeline covering what had generated the complaint and also the steps involved in dealing with it. This would help identify any issues that could have prevented the complaint arising in the first place and also any issues that prevented the complaint being addressed in a reasonable timeframe. Tempting to have a go at a timeline for the complaint about the exercise in MECFS review.

As a public sector organisation we were subject to review by the ombudsman so bad examples of poor work would eventually be taken up by the ombudsman’s office for investigation and anonymised case studies would be included in their published reports to Parliament. So we were very aware of the accountability for our organisation’s actions.

What strikes me about these Science organisations is despite the influence their work has there appears to be at best minimal accountability, in practice no accountability.
Click to expand...
That sounds a lot like quality assurance. I love QA, in software development it is a critical part of the process. One that few appreciate, because all they do is point out mistakes. Like it's their job. Which it is!

I see no evidence of quality assurance anywhere in the process of clinical evidence or medical practice. QA does not end on release, it is a continuous process that not only works with feedback but is proactive in gathering information, all information, so as not to miss anything. It is obsessed with not deleting information. The rare good kind of obsession.

This process does not exist in medicine, or I have seen no evidence of it anywhere. I personally think it's the systemic change that is needed, one that assumes that mistakes will be made, rather than categorically rejecting the premise of there being a mistake as vexatious and bound for the shredding cabinet.

Sure would be nice. Life-saving, even. In software we do this for junk like marketing campaigns and we do it proudly, because we know we make mistakes all the damn time. The reverse approach, I must say, is very, very bad. Boeing learned the same, but they have financial pressure. Medicine has no such pressure, doesn't even take the main stakeholders, the patients, into account at any point. What an odd system.
 
The announcement of this review was made early 2020 the timeframe given as 2 years to completion, so early 2022.

The first step after the independent advisory group lead appointment was to be setting up the group. work on this to start March 2020

we have no information whatsoever since March 2020 about the membership of the IAG, terms of reference, how it will conduct wider engagement, whether there is any possibility of meeting the announced timeframe.


Communication to, let alone engaging with, consumers is not occurring in this pilot of a new approach to engaging consumers.........
 
Last edited:
Robert 1973 said:
“At the appropriate juncture, in due course, in the fullness of time” as Sir Humphrey might say.

As far as I’m aware we’ve not had any news of progress since Hilda was appointed IAG lead over a year ago. Last April we were told there would be a progress report “soon”. In February we were told “A progress report is a high priority, and plan to have it out well before the end of this month.”

@Hilda Bastian Can you tell us whether any members of the IAG have been appointed yet?
Click to expand...
Hi Robert given it is now nearly 2 weeks since you asked this question maybe it would be worth asking on Twitter because other questions asked to @Hilda Bastian there are getting replies.
 
In the absence of communication people naturally start to draw their own conclusions about Cochrane’s position. Based on the little we know and the approach taken so far it appears to me that engagement with people with ME is not considered to be of any importance in this process.

i retain some hope that whatever the report consists of will turn out to be a piece of work that out of public view has had input from some knowledgeable ME advocates and is of some value in addressing the problems with the exercise review.

However I am not holding my breath
 
Robert Saunders: Thanks. Can you clarify, are you referring to a progress report or the new review of exercise therapy for ME/CFS? If it’s the former, what is holding it up? This was billed as a “pilot project for engaging stakeholders (such as consumers...)”. As a consumer I feel let down.

Hilda Bastian: Progress report. I'm sorry to have let you down.

Click to expand...
 
Did we misunderstand the part about this being part of some sort of pilot project for patient engagement? Because it looks identical to the old patient engagement of there being none whatsoever. Still in the wrong category (which of course is intentional and meaningful). Still the wrong definition and framing. Still harmful advice based on awful evidence, all blocked by people with no stake in the matter. Still astrology-level pseudoscience with conclusions detached from even the mediocre evidence.

However it's Cochrane who let us down. Or more accurately is failing in their mission, whatever it is because frankly I can't tell at this point. This is a strictly volunteer thing and a big ask for one person who is genuinely busy with important things, which I can't fault, Bastian's work on vaccine evidence really is important and influential in the right way. But it's true that very few people can actually take on this task and it must be hard to get people interested to contribute to it, tangible career risk and all. Makes for very broken incentives.

5 years and nothing to show for it, with an arbitrary veto from the only people who stand to lose from it, and not even any significant loss. With millions of lives hanging in the balance this is just terrible all around, a system broken beyond any hope of repair.
 
I'm going to post all the information about this that I can find, because the whole thing has been as clear as mud so far.

On October 2, 2019 - Cochrane announced publication of the amended version of the Exercise review, and also made this statement (https://www.cochrane.org/news/cfs):
“We have decided, therefore, that a new approach to the publication of evidence in this area is needed; and, today we are committing to the production of a full update of this Cochrane Review, beginning with a comprehensive review of the protocol, which will be developed in consultation with an independent advisory group that we intend to convene. This group will involve partners from patient-advocacy groups from different parts of the world who will help us to embed a patient-focused, contemporary perspective on the review question, methods and findings.”

She added, “By forging better relationships with patients, as well as the groups that represent them and the clinicians who seek to treat them, we can improve the way in which future Cochrane Reviews in this area address important questions and meet patients’ and clinicians’ needs. I can confirm that work will begin on this new review at the beginning of 2020 and that we will keep patient groups regularly informed of progress during the subsequent months.”
Click to expand...

On 13 February 2020, Hilda Bastian was announced as lead for the Independent Advisory Group (IAG): https://www.cochrane.org/news/appointment-lead-independent-advisory-group

We were promised further updates - but nothing so far has been forthcoming. Many of us emailed the group, but have not received any further information.

This is the document that Hilda Bastian posted on 9 April 2020 (which I've only just seen): https://community.cochrane.org/orga...eholder-engagement-high-profile-reviews-pilot

The relevant section on the IAG is here:
Why is there an Independent Advisory Group?
The goals of the Independent Advisory Group (IAG) are to:
  • ensure and facilitate responsiveness to stakeholders’ concerns; and
  • improve the relevance, accessibility, and credibility of the revised and updated review.
The involvement of representatives of advocacy groups for people living with ME/CFS from different parts of the world is intended to help ensure a patient-focused perspective on the review’s questions.

The lead for the IAG, Hilda Bastian, was chosen by the Cochrane Editor in Chief. Hilda has been a health consumer advocate and researcher, and has held many roles with healthcare bodies. She was chosen for the broad support seen for her writing on the specific concerns of people with ME/CFS related to this review, as well as her understanding of the methodology of systematic reviews.

The IAG will be supported by Cochrane staff, in particular:
  • Rachel Marshall, a Senior Cochrane Editorial Officer who is the update’s project manager; and
  • Richard Morley, Cochrane’s Consumer Engagement Officer.
More on the background: announcement of the appointment of the lead for the IAG
More on Cochrane’s position on consumer involvement: The statement of principles for consumer involvement in Cochrane
Click to expand...

Will there be ways for the views of people other than members of the IAG to be heard?
Yes. Cochrane has a commenting function that can be used to submit comments on the updated protocol and review when they are published. The protocol will set out, in detail, the proposed scope and methods for conducting the review. The drafts of both the new protocol and the updated review will be peer reviewed before publication.

People can communicate directly with the IAG via the email address, Cochrane.IAG@gmail.com. The email account is not monitored by members of Cochrane’s staff. When the IAG finishes its work, the contents of the account will be archived at Cochrane.

In addition to the IAG itself, the IAG lead will convene an email discussion group for groups representing people living with ME/CFS. The IAG lead, in consultation with other participants, will also develop other opportunities for stakeholder engagement.
Click to expand...

Who will be the members of the IAG and how will they be chosen?
Cochrane’s Editor in Chief appointed the IAG lead, Hilda Bastian. There will be a further nine members. Eight of those members will be chosen by the lead, in liaison with the update’s project manager, and the final member will be chosen after advice from the IAG at its first meeting.

In addition to the lead, the members of the IAG will have a range of expertise, including:
  • People living with ME/CFS;
  • Clinicians; and
  • Researchers, including a systematic review methodologist.
Individuals for these positions will include some representatives nominated by advocacy groups for people living with ME/CFS, some people who have been involved in discussion or review of the current Cochrane review, and clinicians and researchers active in the field. Unsolicited expressions of interest and suggestions are also being considered.

Suggestions have been received since the announcement of IAG lead on 13 February 2020. Members are being invited and appointed in several waves starting in March 2020, with a concern for gender balance and the range of geographical locations of members.

Members of the IAG who are living with ME/CFS may nominate a proxy, and can expect reasonable adjustments for their participation. There will be support to engage from the IAG lead and the Cochrane Consumer Engagement Officer, and if desired, technical and training material support provided by Cochrane staff.

Members of the IAG and nominated proxies will be required to complete a declaration of interests form, and declarations will be made publicly available. They will be expected to comply with Cochrane’s policy conflicts of interest and Cochrane Reviews applicable to authors.

Nominating groups will also be asked to declare interests.
Click to expand...

As far as I'm aware, the members of the IAG have yet to be appointed. So the process hasn't even started yet.

What will the IAG do?
The IAG will advise the authors of the review. It is anticipated that the IAG will be:
  • Advising on the language used to describe ME/CFS;
  • Advising on key issues for revising the review’s current protocol;
  • Reviewing and providing an IAG response to the drafts of the protocol and review;
  • Suggesting peer reviewers;
  • Reviewing the revisions made by the author team following peer review;
  • Considering issues raised by peer reviewers, stakeholder groups, and other readers about the protocol, the process for updating the review, and the review.
The IAG will shape its own role, consider gaps in its membership, and determine communication processes and confidentiality in the update of the review. Meetings will be conducted by teleconference and most communication will likely be over email.

The IAG lead will recruit members in liaison with the Cochrane project manager, and is the IAG’s spokesperson. She will facilitate its discussions and chair its meetings, and coordinate its responses to the authors. The IAG lead will also convene an advocacy group email discussion group and other means of stakeholder engagement. She has been offered an honorarium.
Click to expand...

Where can we get updates and answers to other questions?
Click to expand...

Has anyone actually signed up for updates and heard anything? I emailed the IAG directly in March last year, thinking that would trigger updates, but that was probably a mistake, since I have not heard anything since. From Hilda Bastian's recent tweets, it seems that nothing has actually happened yet.
 
Last edited:
Lucibee said:
Has anyone actually signed up for updates
Click to expand...
I have.
Lucibee said:
and heard anything?
Click to expand...
No.

@Hilda Bastian
I understand that there are situations where priorities shift.

Not even letting us know in a formal manner, inform us on the webpage etc is what really annoys me, though.

If you want to take people with ME seriously, then you have to do something that takes us seriously.

At the very least, why can't you just put a simple notice on the webpage and let subscribers know?

I don't understand why you need to finish a report to let people know that there will be a substantial delay.

Also, I don't understand why you need a report to let us know if people have been appointed to the IAG.[*]

Ideally, you could consult patients how to proceed if you won't have sufficient time for your job for a while.


Edit: [*] Or if invitations have been sent -- see this post
 
Last edited:
I signed up and have never heard anything at all. I have written several times to the IAG address with suggestions. I have received acknowledgement every time, and once, in the early days, a reply from Hilda to my questions - will dig that out.

Cochrane have rejected my request to withdraw the "current" reviews (Exercise and CBT) despite the fact they could cause harm to patients if the findings were followed. I sent details of that to Hilda.

I have submitted a complaint about Cochrane's refusal to withdraw the reviews, or to independently review this decision when I appealed against it, to the Committee for Publication Ethics (www.publicationethics.org). I did this a month ago, but have heard nothing back from them yet.
 
Who will be the members of the IAG and how will they be chosen?
Cochrane’s Editor in Chief appointed the IAG lead, Hilda Bastian. There will be a further nine members. Eight of those members will be chosen by the lead, in liaison with the update’s project manager, and the final member will be chosen after advice from the IAG at its first meeting.
[...]
Suggestions have been received since the announcement of IAG lead on 13 February 2020. Members are being invited and appointed in several waves starting in March 2020, with a concern for gender balance and the range of geographical locations of members.
Click to expand...

This is the part that confuses me most. It says advocacy groups will be involved. S4ME asked to be involved. Yet we have received no information on how and when we should make our suggestions for IAG members. So where have these 'suggestions' come from? Random individuals? Twitter? Private invitation to selected advocacy groups? Public invitation?
 
You must log in or register to reply here.
Back
Top Bottom