2025: The 2019/24 Cochrane Larun review Exercise Therapy for CFS - including IAG, campaign, petition, comments and articles

[Hutan]

Here's the asthma study @Medfeb
https://www.nejm.org/doi/full/10.1056/nejmoa1103319


Subjective improvement: from the left: inhaler with asthma drug, sham inhaler, sham acupuncture, no intervention
Only no intervention does not "work"


Objective improvement: same order of interventions
Only the inhaler with asthma drug improved breathing

Basically, if you have a poor trial design, with respect to outcomes and controls, the results are worthless and misleading. If you have a wide selection, you can still say something, possibly useful, about that wide group that you selected. And, if your trial is big enough, you can do some post hoc analysis to work out what trials would be useful to do next to deal with subgroups with different responses.

 

[Hutan]

@Medfeb, could you help us understand why Hilda and MEAction talk primarily about the diagnostic criteria as being the fault with BPS trials and ignore that trial design issue that so many of us see as the real problem? I find it such a puzzle that we aren't on the same page about this.

I get that it is face-saving approach to BPS people, as in 'we understand that you use the Fukuda criteria back then, but things have moved on and we now know about PEM, so those trials are too indirect, so please can we have a new review'. It's a lot more conciliatory than 'even back then, people knew how to design good trials, understood the importance of blinding and controls. I think Wessely even wrote a book about it. So, when you chose to make trials with only subjective outcomes for unblinded treatments and no controls, either you really did not understand how to do research or you were actively trying to mislead'.

I also get that there are an awful lot of trials that use this poor design, and so if that is the flaw that is focussed on, then getting agreement to move forward with a new review is a lot harder, because it may (and in my view hopefully will) shine a light on the rottenness of so many trials and the reviews that accept these flaws without much consequence.

I also get that indirectness is a fairly easy concept to explain to people, including the broader ME/CFS community. 'They didn't study our disease, they just studied people with chronic fatigue!'. Whereas I still haven't worked out a really snappy way to describe the problem that I think is the main one; I use the vague 'poor trial design' when I can't be bothered going through the list of subjective outcomes, blindedness, controls.

Do you think the people who talk primarily about the diagnostic criteria understand the danger in suggesting that that is the big problem that has to be fixed? For us and especially for the people yet to be diagnosed with Long Covid and FND? If so, why don't they talk about it?


I'm also interested to hear what you think the new review writing group will be doing with the protocol.

 

[Utsikt]

Here's the asthma study @Medfeb
https://www.nejm.org/doi/full/10.1056/nejmoa1103319

I would argue that the most plausible explanation of this study is that people with asthma are terrible at rating their asthma in a blinded study environment.

Asthma is an objectively measureable phenomenon, so any perceived benefit is irrelevant if they don’t improve the underlying condition.

Furthermore, the sham inhaler was not blinded to the patients. There’s absolutely no reason for not blinding it.

Using other diagnostic criteria is in itself entirely legitimate scientifically and to claim otherwise is to reveal a lack of understanding that threatens to be fatal to credibility.

I’m testing my own understanding:

The problem isn’t that they are using other criteria, the problem is that they are trying to generalize results from their wider criteria to the population in the narrower criteria.

Ideally, they should recognize that the results are worthless due to the methodological flaws. If they don’t do that, they should still recognize the very limitied generalizability of the findings.

I believe that both issues (methods and generalization) are highly relevant, and Cochrane needs to up their standards for both.

 

[Utsikt]

I also get that indirectness is a fairly easy concept to explain to people, including the broader ME/CFS community. 'They didn't study our disease, they just studied people with chronic fatigue!'. Whereas I still haven't worked out a really snappy way to describe the problem that I think is the main one; I use the vague 'poor trial design' when I can't be bothered going through the list of subjective outcomes, blindedness, controls.

A lot of people simply don’t understand the concept of scientific methodology. They believe that an opinion has the same value as evidence.

This is something I frequently argue with my family about. If I say that a study they read about in the news is nonsense (95 % are BPS), they almost get angry at me for dismissing the opinions of «experts».

This is not unique for ME or LC, it happens for every topic. «You should listen to everyone» is a common phrase.

This is a serious issue when it comes to public health, and it’s something the MEAs have to address in general. This misconception is prevalent in the patient community as well.

Which further highlights the need to get Cochrane to do a proper review, because we simply need some leverage to discredit the studies and interventions amongst the patients and their carers. IMO, just pulling the review is worse than pulling it and writing a proper one. Writing a new bad one is the worst option.

 

[Jonathan Edwards]

The problem isn’t that they are using other criteria, the problem is that they are trying to generalize results from their wider criteria to the population in the narrower criteria.

That argument is incorrect. You can legitimately apply results from wider criteria to a population with narrower criteria. It is the opposite, true generalisation that is not legitimate.

If bones are good for dogs you can assume they are good for whippets, unless you have evidence otherwise. You cannot assume they are good for all mammals.

If a narrower population has a feature that suggests it does not follow the general pattern then it is sensible to invoke caution but you still need evidence that it matters. As Hutan says, inxremental exercise might actually be good for exercise-induced asthma. After all, the whole idea of de-sensitisation comes from asthma and allergy.

The trials that are available tell us nothing about any difference in efficacy of GET and CBT for those with exertion intolerance and those without. But the whole thing is irrelevant because none of the trials show any reliable evidence of efficacy for anybody.

It is a bit like saying 'Oh I don't know if these eggs are any good because they have been delivered in the wrong boxes when there aren't even any eggs there.

 

[Trish]

Writing a new bad one is the worst option.

I agree, which is why I have come round to the idea that the only way forward now is to get Cochrane to see sense, withdraw the bogus 2024 version, preferably also the 2019 version too, but at least put an honest editors note on it saying it's outdated and shouldn't be used for medical care.

Since we haven't seen the draft protocol that Cochrane rejected last month, we can have no idea what it contained.

Ideally it would specify objective primary outcomes, given that all the trials were unblinded, and a wide search for harms data to include research based on large surveys of people who were prescribed exercise interventions. If they included subjective outcomes as secondary outcome measures, they should use the longest time since treatment, not the immediate effect that is likely to be too strongly influenced by the therapist and hope.

If that were the basis, I would be fairly confident that they would show that regardless of diagnostic criteria used, the trials were either useless or showed no benefit, and surveys showed evidence of harm.

What I fear is that Cochrane is so wedded to its psychological trials for lots of conditions using subjective outcomes in unblinded trials, that a protocol like I suggest above would be rejected, and they would insist on using fatigue scales as the primary outcome measure, knowing they would end up with the same outcome as Larun did.

A properly done review would show exercise therapy is ineffective and potentially harmful for anyone with a fatiguing condition, regardless which diagnostic criteria are used. This should therefore put a stop to any further trials, which would be a good thing.

My concern about withdrawing the Larun review from use, either on grounds of being out of date or of indirectness, gives the green light to the BPS people to keep doing more trials on the grounds that they need to test the intervention with better defined groups, or that they need to use more up to date outcome meaures, perhaps, horror of horrors, Sarah Tyson's efforts.

 

[Jonathan Edwards]

Would it be acceptable for a study of an asthma intervention to use any old criteria,

Absolutely it would. In clinical pharmacology we use different categorisations all the time. I have done studies on rheumatoid arthritis, inflammatory arthritis, seropositive rheumatoid arthritis, active rheumatoid arthritis, arthritis in general - you name it. Asthma is an outdated concept. These days you would be talking of reversible airflow obstruction, which, as Hutan says, you might want to qualify with allergen sensitivity, exercise induction or whatever. A bronchodilator like Ventolin is used for airflow obstruction even more widely on the basis that even partial reversibility might be helped.

Asthma and ME/CFS are not fixed things. They are useful categories amongst many other useful categories that can be wider or narrower.

If PACE had been properly designed and showed an objective benefit for GET it should be recommended unless there is evidence that a subgroup does not benefit. We cannot criticise the BPS people for muddled thinking if our own thinking is just as muddled.

This is not a valid methodological issue.

As indicated above, the whole issue is irrelevant because there are no eggs anyway.

 

[Jonathan Edwards]

I am doubtful that my Rapid Response will appear. There are none up and I suspect this indicates that none will be put up. The potato is too hot.

 

[Utsikt]

That argument is incorrect. You can legitimately apply results from wider criteria to a population with narrower criteria. It is the opposite, true generalisation that is not legitimate.

I don’t understand this.

If the narrow population (NP) is such that the entire population fits inside the wider population (WP), then it’s still possible that a study on the WP doesn’t include any patients from the NP.

If you study only men, you can’t say that the results also apply to women just because they are all human.

If you can’t prove that your selection also includes women, then you’ve only studied men. Claiming to have studied humans is incorrect and misleading.
(Edit: then you could potentially have only studied men)

By studying chronic fatigue patients and not properly testing for PEM (assuming it can even be done), you can’t claim that you’ve studied the entire CF group. You could have studied the non-PEM CF group only.

 

[Jonathan Edwards]

If the narrow population (NP) is such that the entire population fits inside the wider population (WP), then it’s still possible that a study on the WP doesn’t include any patients from the NP.

But that would mean that the 'wider' population was in fact the narrower population in that regard.

 

[Jonathan Edwards]

It may be worth reminding people that when I presented my thoughts about the PACE trial to the University College London Division of Medicine (cardiologists, endocrinologists, haematologists, clinical pharmacologists etc.) nearly ten years ago now the audience unanimously agreed that PACE was a non-starter methodologically because it was open label nd with subjective outcomes. This isn't a minor issue. It is the bedrock of trial design. In other branches of medicine we simply do not take any notice of trials like this.

 

[Sasha]

I agree, which is why I have come round to the idea that the only way forward now is to get Cochrane to see sense, withdraw the bogus 2024 version, preferably also the 2019 version too, but at least put an honest editors note on it saying it's outdated and shouldn't be used for medical care.

The problem is not that it's outdated, though - this is the obscurantist reason that the editor-in-chief gave to not pull the review in the first place. It allowed her to avoid saying it was rubbish. Something that is merely 'outdated' is fine. My sofa is outdated but I'm still sitting on it.

An honest editor's note would give the real reason why the review is so bad that it needed to be withdrawn (open label, subjective measures) but that would involve admitting that they were wrong to publish it in the first place and that the rest of the BPS literature is a house of cards.

 

[Sasha]

I am doubtful that my Rapid Response will appear. There are none up and I suspect this indicates that none will be put up. The potato is too hot.

Maybe they just need a bit longer to think about it, given the heat of the potato.

 

[Midnattsol]

It may be worth reminding people that when I presented my thoughts about the PACE trial to the University College London Division of Medicine (cardiologists, endocrinologists, haematologists, clinical pharmacologists etc.) nearly ten years ago now the audience unanimously agreed that PACE was a non-starter methodologically because it was open label nd with subjective outcomes. This isn't a minor issue. It is the bedrock of trial design. In other branches of medicine we simply do not take any notice of trials like this.

But when is this taught? I've taken methodology classes with medical students, and I've only ever really heard about this problem in the ME patient community. The go-to professor on stats at my medical faculty is a co-author on a study with just this problem (on ME patients to boot).

 

[Sasha]

It may be worth reminding people that when I presented my thoughts about the PACE trial to the University College London Division of Medicine (cardiologists, endocrinologists, haematologists, clinical pharmacologists etc.) nearly ten years ago now the audience unanimously agreed that PACE was a non-starter methodologically because it was open label nd with subjective outcomes. This isn't a minor issue. It is the bedrock of trial design. In other branches of medicine we simply do not take any notice of trials like this.

Can you offer a guest editorial? I honestly can't understand how this situation prevails.

It's frankly weird how doctors are expected to be 'collegiate' in this situation and not criticise other doctors. What is the point of doctors who are loyal to each other rather than to patients?

 

[Utsikt]

But that would mean that the 'wider' population was in fact the narrower population in that regard.

Which means that the choice of criteria is relevant because of the possibility of studying a non-generalizable selection of the CF population.

The other methdological flaws are more than enough to disquslify the studies, but the criteria are still relevant.

 

[Jonathan Edwards]

But when is this taught?

It is taught in clinical pharmacology courses, if I remember rightly at the preclinical stage of basic sciences. When I was taught it we used Desmond Laurence's textbook (Desmond was at UCL). It is part of the basic explanation of why we do double-blind trials. Every medical student has heard of double blind trials and ought to have an understanding of why they are done. The let out is that if you have truly objective endpoints you may not need to double blind, but that is the exception.

 

[Utsikt]

But when is this taught? I've taken methodology classes with medical students, and I've only ever really heard about this problem in the ME patient community. The go-to professor on stats at my medical faculty is a co-author on a study with just this problem (on ME patients to boot).

I learned about it at VGS (upper secondary/high school). Might have been my statistics teacher that was very into methodology.

 

[Peter T]

Still no comments showing on the MEAction blog

Still no comments appeared on the MEAction blog over twenty four hours later.

 

[Midnattsol]

I learned about it at VGS (upper secondary/high school). Might have been my statistics teacher that was very into methodology.

I can't remember if it was mentioned at VGS, but I've had teachers skipping whole chapters/topics based on "this class wouldn't understand/I think we need to spend more time on another topic so this has to go" so my education pre-university might be lacking From university I have 60+ ECTS in methodology, mostly from medical faculties (three different), and it hasn't been brought up.

Edit: It has been brought up that studies on things like physical activity/nutrition/sleep etc. can't be blinded, but not the problematic combination pf unblinded + subjective outcomes.