Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019 and new date December 2024

I was curious to see how "probably" is specifically defined, and what certainty it gives.

Really none.

The Compact Oxford English Dictionary defines probably as: " almost certainly".

The word "almost" is defined as: "very nearly".

Here is a WordReference.com discussion on what "probably" means to members of that forum; one of the suggestions is "anything greater than 50%.":

https://forum.wordreference.com/threads/probably-what-percent.2949969/


I would not feel confident in the certainty of a medical outcome if a practitioner told me the therapy, or procedure would "probably" work.

There are no real guarantees in medicine, but if something is maybe going to work only 50.1% of the time, the risks of the procedure may outweigh the benefits.

On the other hand, I don't have a broad enough knowledge in this; perhaps there are many medical procedures that will only probably work - and how certain that is, is up for interpretation.
 
Last edited:
"Selective reporting (reporting bias) Low risk
The protocol and the statistical analysis plan were not formally published prior to recruitment of participants, and some readers therefore claim the study should be viewed as being a post hoc study. The study authors oppose this, and have published a minute from a Trial Steering Committee (TSC) meeting stating that any changes made to the analysis since the original protocol was agreed by TSC and signed oN before the analysis commenced."


Unpublished protocol and stat analysis; I thought this was imperative to have up front - not done in a private meeting.

More exceptions allowed for these types of studies.

No one in medicine should be reassured about referring their patients for this "therapy".
 
Probably is certainly way less than would normally be considered adequate evidence for efficacy of a treatment. I think we would in medicine generally expect to be 90% sure, preferably 95%. But probably at least has a standard meaning to medics. If we say someone probably has measles it is understood that we think it is more likely than not.

But if we think on the basis of pilot studies that there is a 50+% chance that the treatment works that is quite different from saying it works in 50+% of people - these are unrelated. We may well think that there is a 50+% chance of it working in20% of people (which for some things might still be very useful) or in 85% of people.
 
I've only skimmed through it, but what a depressing read. They do have to note the lack of evidence for benefit at follow-up, but people Cochrane clearly haven't taken this review seriously, and chose to let the authors get away with some utter BS. The selective reporting section on PACE shows what these authors, and Cochrane, are. The general presumption that these questionnaires are valid for reliably measuring things like fatigue seems to go completely unexplored.

Some smaller things I thought I'd pull out that I haven't seen others mention but I might want to come back to:

For "Work and employment status" they say NS (not stated) for White 2011 - but doesn't their cost-effectiveness paper provide that info?

Looks like we've got some fun reading:

We have updated the methods according to recommendations provided in the 2011 version of the Cochrane Handbook for Systematic
Reviews of Interventions (Higgins 2011c). For the first version of this review (Edmonds 2004), the review authors conducted assessment
of methodological quality according to contemporary criteria of the handbook of The Cochrane Collaboration (Alderson 2004). Review
authors rated the adequacy of allocation concealment as adequate (A), unclear (B) or inadequate (C) or as not used (D), and applied the
Cochrane Collaboration Depression, Anxiety and Neurosis (CCDAN) Quality Rating System (Moncrieff 2001). For this update, we re-extracted
data on risk of bias to comply with current recommendations, and we used concealment of allocation as the main quality criterion for
included studies (Higgins 2011a).
Click to expand...

It looks like they used standard deviations from the Crawley paper on patients attending clinics. Someone mentioned that this is a populations with an artificially constrained SD as at the top end there's a cut off for eligibility, and at the bottom end there's a requirement to be able to attend the clinics. I wonder if this is an issue that's addressed in Schünemann 2011?

Re-expressing standardised mean differences and defining minimal important differences

If available studies measured the same outcome using different scales or different versions of the same scale, we presented the pooled
effect estimates in terms of standardised mean differences (SMD). SMD units may be hard to interpret, and based on feedback from the
Cochrane Editorial and Methods Department we have also calculated/re-expressed SMD using more familiar instruments. Chapter 12 in
the Cochrane Handbook for Systematic Reviews of Interventions (Schünemann 2011), recommends using standard deviation from a representative
observational study as a basis for the recalculation, and we therefore use standard deviations reported in Crawley 2013 for this
purpose.
Click to expand...

Their MID stuff seems to have come up with a way of classing small changes as an important difference. They've cited work that's unrelated to CFS to do this, and I'm not familiar with it. A lot of the research I've seen on MID is very poor anyway, but it seems particularly odd using research on a completely different patient group to define MID in CFS:

Post hoc, we have also been encouraged to define minimal important differences (MID) for commonly used outcome measures. MID thresholds
and relevant research literature are reported under Measures of treatment effect in the Methods section.
Click to expand...

Clinical studies and meta-analysis can detect small differences in
outcomes with little or no importance to individual participants.
Moreover, the interpretation of what is considered an important
difference may vary between patients, researchers and clinical experts
(Wyrwich 2007). We therefore identified research literature to
help quantify minimal important differences (MID) for important
outcome measures. For fatigue, one study among people with systemic
lupus erythematosus (Goligher 2008), reported a threshold
around 2.3 points for a minimally important change on the 33-point
Chalder Fatigue Scale, an effect size that corresponds to an SMD of
about 0.36 (Goligher 2008).

Studies in people with rheumatoid arthritis or chronic heart disease
suggest that the threshold for MID on the physical functioning
subscale of SF-36 can be set around 7 points (Ward 2014; Wyrwich
2007). Studies based on data from patients with chronic obstructive
pulmonary disease have also investigated MID for HADS and
suggest MIDs around 1.5 points for the HADS anxiety and the HADS
depression scale (Puhan 2008; Smid 2017). We did not detect studies
that established a common MID for the Jenkins Sleep Scale, but
decided to view a 20% change in sleep scores as a clinically important
difference.
Click to expand...
 
Last edited:
The review says that exercise therapy probably reduces fatigue at the end of treatment in comparison to a passive control.

A passive control is inadequate and it's impossible to say how much of the difference between the treament and control groups is due to specific treatment effect as opposed to nonspecific things.
 
:banghead::banghead::banghead:
Esther12 said:
There is moderate‐certainty evidence that exercise therapy probably increases the number of people who report at least some degree of improvement in self‐perception of overall health at end of treatment
Click to expand...
Impressive linguistic gymnastics if nothing else.
Hutan said:
So maybe the most effective thing to do to ensure that this review goes into the dustbin of history is to get a Cochrane review done on studies of exercise therapy that selected people with ME/CFS (ie with PEM)? Is anyone working on that? I think there was talk about it.
Click to expand...
today we are committing to the production of a full update of this Cochrane Review, beginning with a comprehensive review of the protocol, which will be developed in consultation with an independent advisory group that we intend to convene. This group will involve partners from patient-advocacy groups from different parts of the world who will help us to embed a patient-focused, contemporary perspective on the review question, methods and findings.
Click to expand...
So would our best bet now be to make sure the 'right' patient-advocacy groups get into the promised independent advisory group - we can probably (oops, did I say 'probably'?) think of some 'wrong' ones we don't want to be involved? They, the 'right' patient-advocacy groups, could then push for any new protocol to specify that studies included need to look at PwME with PEM, plus attempt to address all the other weakness in the current review - they'll have their work cut out...
 
so the house of cards continues until after the NICE review, how convenient.

It would be interesting to know how many other illnesses with the same type of evidence base are under/ due for review in the next couple of years

I generally find most medics have a poorer grasp of statistics and I have ( so very poor); they tend to just hoover things up that confirm their existing cognitive constructs.

Things seem destined to follow the British tradition for controversial change - death or retirement as as precursor.
 
Blog from Hilda Bastian
It’s tough, watching this play out slowly, step by painful step. As I’ve blogged before, ME/CFS consumer advocates have been hammering away to get this Cochrane systematic review on exercise and CFS fundamentally changed since 2015. I think they’re right. And now, they have achieved something pretty remarkable…

Kudos to Robert Courtney (now deceased), who got this ball rolling, and those who picked up the baton, especially Tom Kindlon and George Faulkner. But why do I think it’s only a start?

Firstly, I need to disclose that I’ve been directly involved in recent negotiations around the fate of this review, although I didn’t see this version till after it was published. The organization released a statement along with the amended review, making it clear that this is only a first step.
Click to expand...
https://blogs.plos.org/absolutely-m...n-of-the-cochrane-review-on-exercise-and-cfs/
 
And article from Emerge Aus on the news
Is this good news or bad news for people with ME/CFS?
It is good news that Cochrane recognised the seriousness of the complaint and the issues with GET research. Unfortunately, the updated review doesn’t really address them. The updated review concludes that there is low-moderate certainty about the evidence that GET may alleviate some symptoms of ME/CFS and recommends more research.

The updated review does acknowledge that the evidence only applies to patients diagnosed with the (very broad) Oxford or Fukuda diagnostic criteria and may not apply to people diagnosed with other criteria. Emerge Australia follows the recommendation of the NHMRC ME/CFS Advisory Committee, that ME/CFS research should be based on either the Canadian Consensus Criteria or International Consensus Criteria, which both require post-exertional malaise to be included as a key symptom for diagnosis. Oxford and Fukuda diagnostic criteria rely on fatigue as the primary symptom and therefore may capture people with a very wide range of fatiguing conditions, including depression and mental illness. It is good that Cochrane have acknowledged that the evidence is only applicable for people diagnosed with these broader criteria, but we feel that this distinction will be of little help to patients sitting with their doctor (or applying for DSP or NDIS) and trying to explain that exercise hasn’t helped or has made them feel worse.

The updated review also acknowledges that the “evidence regarding adverse effects is uncertain”. This is an improvement on the previous version of the review, which concluded that “no evidence suggests that exercise therapy may worsen outcomes”, although we believe that this still does not adequately reflect the experience of many patients.
Click to expand...
https://emerge.org.au/cochrane-has-...ew-of-graded-exercise-therapy-get-for-me-cfs/
 
Esther12 said:
They do have to note the lack of evidence for benefit at follow-up, but people Cochrane clearly haven't taken this seriously,
Click to expand...
This is particularly disappointing and disturbing.

The long-term outcome is the only serious metric for chronic conditions, as far as I am concerned. If they are not delivering on that, then they got nothing.
 
Thanks for posting Hilda’s post about the update, @Andy. I think her conclusion really sums the whole thing up really well, especially that Cochrane’s decision to not withdraw this review looks political, that Bob’s complaint has been absolutely crucial to getting any change and that Cochrane has been resistant to consumer involvement.

The previous editor-in-chief, David Tovey, had originally proposed that the review be withdrawn, but that wasn’t well received, and it didn’t happen. An editorial note on the review points out that it is “substantially out of date”. Once that determination has been made, Cochrane reviews are supposed to be either updated or withdrawn. It’s now nearly a year since then, and a replacement review isn’t going to be galloping over the horizon any time soon. Looking at it today, with all the statements in there that look current – like saying this version has had trials added – withdrawing it seems reasonable, and not withdrawing it seems political.

This review over-estimated the strength of the evidence and its applicability, and while better, it’s still doing that. That has been having a damaging effect on people to whom the results don’t apply, but on whom the consequences of this review are visited. Yet, until Courtney’s complaint got Tovey involved, consumers had been resolutely locked out of this Cochrane review, and indeed, this amendment process too. That was against Cochrane’s stated ethos. It’s great that Soares-Weiser is going to rectify that. Consumer participation isn’t only for when it’s smooth sailing. When it’s going to rock the boat may be the most important time of all to do it.
Click to expand...

https://blogs.plos.org/absolutely-m...n-of-the-cochrane-review-on-exercise-and-cfs/
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom