Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

I have just read through that correspondence. Thanks @Caroline Struthers.

It all seems so bizarre when what they are arguing about is the Chalder Fatigue Questionnaire which is a back of the envelope list of vague statements.

As we all know, the CFQ is designed to have a strong ceiling effect, and to be highly influencable by placebo and therapist effects, and has the bizarre option of 'better than normal' designed to bamboozle participants. It is not a linear scale measuring something objective, it's a tool for therapists to pretend to their employers that they are achieving something useful by persuading their clients to reinterpret their symptoms. Others have expanded on all this much better than I can.

I can't help wondering whether the editors are aware just what junk data they are discussing so seriously.

To suggest that a change of 1 response on the 11 descriptor list, or 3 points on the 33 point version has any clinical meaning at all is just plain daft.

And on all this nonsense hangs a review that has such strong influence over the way millions of people are treated. How dare Cochrane treat us so irresponsibly and all to keep happy a bunch of reviewers with an agenda.

It's insane.
 


Capable of withstanding rough treatment from readers is exactly what a protocol for a systematic review has to be. And glue is apt for what should follow if a systematic review is to be trustworthy: the reviewers should stick to the protocol, with transparency and very solid justification for any time they don’t. We should be far more serious about how to achieve all of this. It’s critical to confidence in a systematic review’s integrity.

https://absolutelymaybe.plos.org/20...bout-bias-in-protocols-of-systematic-reviews/
https://absolutelymaybe.plos.org/20...bout-bias-in-protocols-of-systematic-reviews/
ETA:
A fundamental problem with the original exercise therapy and ME/CFS protocol, it seems to me, was evident author bias in the text about both the intervention and the condition. Some of that bias in the text is actually encouraged by the current structure and guidance for Cochrane protocols, and it shouldn’t be.
 
Last edited:
Responses on Twitter from Hilda Bastian to some comments here.

@s4me_info Responses to issues directed to me & thanks for passing them on. MSEsperanza: I should spot everything - I read the thread carefully. Thx for suggestions about Google docs: acted on @ozfish I don't think I can post as a guest, hence this route; happy to if possible 1/x
Click to expand...

@s4me_info Jonathan Edwards: your peer review report was for a different review, but as you've given your consent for me to see it, I will be reading it. Thanks! 2/2
Click to expand...

 
Thanks to Hilda Bastian for taking so much interest.
My understanding was that the individualised patient data review I reported on was produced with the intention of addressing perceived weaknesses in the review still in place and would have superseded it. So it seemed to me to be an update of the same review - except that, presumably because the reports from a wider range of referees were so negative - the IPD review was pulled and never appeared.
 
Kalliope said:


Capable of withstanding rough treatment from readers is exactly what a protocol for a systematic review has to be. And glue is apt for what should follow if a systematic review is to be trustworthy: the reviewers should stick to the protocol, with transparency and very solid justification for any time they don’t. We should be far more serious about how to achieve all of this. It’s critical to confidence in a systematic review’s integrity.

https://absolutelymaybe.plos.org/20...bout-bias-in-protocols-of-systematic-reviews/
ETA:
A fundamental problem with the original exercise therapy and ME/CFS protocol, it seems to me, was evident author bias in the text about both the intervention and the condition. Some of that bias in the text is actually encouraged by the current structure and guidance for Cochrane protocols, and it shouldn’t be.
Click to expand...

Very insightful and necessary discussion. It only marginally concerns us but only because this is about the much broader issues in evidence-based medicine. That line about author bias, though, absolutely perfect. The underlying problems are as fundamental as it gets and are far from unique or even particular to the excessively poor evidence base for ME-BPS.

There really does seem to be a kind of suspension of disbelief that anything produced from EBM can only be good since they follow a protocol, never asking themselves whether the protocol is any good or if it was even followed at all, which we actually know is not even close to be the case. This is the kind of thinking that must happen if medicine is the least bit interested in solving the crisis of replicability.

Very little of the internal Cochrane discussion reflects an understanding of the magnitude of the problem, even when they involved the editor-in-chief. This thinking here is what's needed moving forward. I'm very much looking forward to what shape it takes as applied specifically on the issue of ME, especially the fundamental problem that even asking the question of exercise therapy in a disease defined by significant deterioration from even mild exertion is completely absurd and all problems derive from that original failure of asking the wrong questions for the wrong reasons and with the worst possible execution possible by the very worst possible people to do the job, followed by the very worst possible excuses not to fix that egregious mistake.
 
Jonathan Edwards said:
My understanding was that the individualised patient data review I reported on was produced with the intention of addressing perceived weaknesses in the review still in place and would have superseded it. So it seemed to me to be an update of the same review - except that, presumably because the reports from a wider range of referees were so negative - the IPD review was pulled and never appeared.
Click to expand...

Hi, @Hilda Bastian!

yes, Cochrane rejected the IPD review, based on scathing comments from Jonathan and others outside the usual suspects on whom Cochrane had previously relied to vet these seriously flawed CFS reviews. And then it also withdrew the published protocol for the IPD review.

I reported on that here: https://www.virology.ws/2018/12/10/trial-by-error-and-cochrane-makes-another-move/

Apart from Cochrane: I would love you to look into the Lightning Process study published in 2017 in a BMJ journal--Archives of Disease in Childhood--that now has a 3,000-word correction appended. More than 70 experts from Columbia, Berkeley, Stanford, UCL, Queens Mary University of London, etc signed a letter of protest to Fiona Godlee at BMJ.

The investigators from Bristol University recruited more than half of the participants before trial registration, swapped primary and secondary outcome measures after gathering data from these participants, and published the findings without disclosing these changes. BMJ did not retract the study but let the original findings stand while acknowledging all the errors I had documented on Virology Blog.

This flawed study is now being cited by Norwegian researchers to justify a study of the Lightning Process there.

Here's an article about the BMJ study that I wrote last fall for STAT:
https://www.statnews.com/2019/12/13...awed-chronic-fatigue-syndrome-research-paper/
 
dave30th said:
Hi, @Hilda Bastian!

yes, Cochrane rejected the IPD review, based on scathing comments from Jonathan and others outside the usual suspects on whom Cochrane had previously relied to vet these seriously flawed CFS reviews. And then it also withdrew the published protocol for the IPD review.

I reported on that here: https://www.virology.ws/2018/12/10/trial-by-error-and-cochrane-makes-another-move/

Apart from Cochrane: I would love you to look into the Lightning Process study published in 2017 in a BMJ journal--Archives of Disease in Childhood--that now has a 3,000-word correction appended. More than 70 experts from Columbia, Berkeley, Stanford, UCL, Queens Mary University of London, etc signed a letter of protest to Fiona Godlee at BMJ.

The investigators from Bristol University recruited more than half of the participants before trial registration, swapped primary and secondary outcome measures after gathering data from these participants, and published the findings without disclosing these changes. BMJ did not retract the study but let the original findings stand while acknowledging all the errors I had documented on Virology Blog.

This flawed study is now being cited by Norwegian researchers to justify a study of the Lightning Process there.

Here's an article about the BMJ study that I wrote last fall for STAT:
https://www.statnews.com/2019/12/13...awed-chronic-fatigue-syndrome-research-paper/
Click to expand...

Thanks, Dave - will read.
 
rvallee said:
.... all problems derive from that original failure of asking the wrong questions for the wrong reasons and with the worst possible execution possible by the very worst possible people to do the job, followed by the very worst possible excuses not to fix that egregious mistake.
Click to expand...

I agree and challenge the assumption that updating a review which asks the wrong question is appropriate. https://healthycontrolblog.wordpress.com/2020/05/22/alternative_proposal_for_overview/.

The CBT for CFS review - another wrong question for the wrong reasons by the wrong people - is still up saying CBT is effective. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD001027.pub2/full.

There has been no move by Cochrane either to update or amend the CBT review as they have done with the Exercise review. The Chinese medicine one has been withdrawn (https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006348.pub3/full), as part of a "strategic portfolio assessment of all existing and planned reviews and protocols, including those relating to Chronic Fatigue Syndrome (CFS)" But the Exercise and CBT reviews have not been withdrawn as part of this "strategic portfolio assessment". It is inconsistent.
 
strategist said:
If the wrong questions have been asked, what would be the right questions?

One good question is: How can doctors best support patients with ME/CFS even when there's no proven treatment yet?
Click to expand...

Indeed! This is why Cochrane reviews (other than overviews) are often not useful to doctors or patients as they tend to focus on individual interventions or intervention types. If it's not a drug, device or surgical procedure, there seems to be an assumption that it probably doesn't do any harm, even if any benefit is marginal or unproven.

Patients and doctors are interested in the range of options available to them, and the risks and benefits of each when compared with each other. Cochrane reviews force people to look at individual reviews in isolation, each with a different author team, different inclusion criteria in terms of diagnosis, outcomes etc. And sometimes biases in favour of the intervention under scrutiny which is near impossible to spot. It would be better to do evidence overviews focussed on patient (or public) populations, defined by diagnosis, location, health system, patient-important outcomes, resources, language etc. NICE are not perfect, but they may end up doing a better job with answering the question above. I hope so.
 
Something like a condensed, practice-oriented IOM report would be very useful.

PS: there is a primer for pediatric ME/CFS which I like

Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is a complex disease that affects children and adolescents as well as adults. The etiology has not been established. While many pediatricians and other health-care providers are aware of ME/CFS, they often lack essential knowledge that is necessary for diagnosis and treatment. Many young patients experience symptoms for years before receiving a diagnosis. This primer, written by the International Writing Group for Pediatric ME/CFS, provides information necessary to understand, diagnose, and manage the symptoms of ME/CFS in children and adolescents.
Click to expand...

Note the bolded parts.

https://www.frontiersin.org/articles/10.3389/fped.2017.00121/full
 
Thank you @Hilda Bastian for your article, you highlighted many relevant issues. Hopefully the Cochrane IAG will correct them so that the new review is not biased towards positive views on GET, which it currently is.

If possible at all, some input from @PhysiosforME and Workwell seems quite crucial as they are physiotherapists with an expertise on ME.

strategist said:
If the wrong questions have been asked, what would be the right questions?

One good question is: How can doctors best support patients with ME/CFS even when there's no proven treatment yet?
Click to expand...

In the absence of a treatment, the very least doctors could do would be to heavily insist on the importance of pacing. They could provide leaflets from national associations (or translated documents if no national association exists in a given country) and recommendations of books on pacing. This is simple, easy to implement, but it isn't done yet (especially not in France...).
 
cassava7 said:
In the absence of a treatment, the very least doctors could do would be to heavily insist on the importance of pacing. They could provide leaflets from national associations (or translated documents if no national association exists in a given country) and recommendations of books on pacing.
Click to expand...
Unfortunately, the PACE trial effectively prevented that from happening even tho it has never been acknowledged that the APT supposedly being 'tested' is not what is generally considered by patients as pacing.
Subsequently, 'pacing' has been incorrectly interpreted as a kind of 'gentle' GET in many quarters including AfME (a UK charity) and most, if not all, of the 'specialist' clinics.
 
Barry said:
It maybe warrants a thought experiment. Imagine the PACE authors really had proven to be good scientists, and reported honestly, openly and fully about their findings in 2011. How different would the ME/CFS landscape be today?
Click to expand...

Hard to predict but:

The forced separation of children with ME from their family, because they decline (further) GET treatment would most likely disappear.

The admission that CBT/GET as line of research has ended would mark the end of an era and probably stimulate and vitalize other research efforts in various ways.

Health insurance companies would try to find some new creative ways to delay or deny payment to patients with ME/CFS.

There would probably be some new quackery trying to establish itself.

PS: I forgot a really important effect. There would be a shift in how the illness is perceived by society.
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom