Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019 and new date December 2024

Hilda Bastian’s blog makes a lot of important points but, in her own words, it does not go nearly far enough. Presumably there is some value in being polite to the new editor but somewhere it needs to be stated that the flaws in the analysis are far deeper than she addresses.

When I refereed the individualised patient data version that got binned I simply said
A) none of these studies are controlled trials in a valid sense so should not even have been analysed according to the authors’ protocol.
B) the authors had no business to be writing the review since they have powerful competing interests in a controversial area
 
I think the main issue I have with this, particularly over the SMD debacle, is that although Tom and Bob were correct to pull them up on not using SMD for the combined fatigue scores, because that's what the authors said they would do (and then didn't do it because it made the results look bad), combining the results in this way was always flawed.

FSS and CFQ are completely different scales, and measure different things to produce a number (the scale goes in the same direction, but that's about it). But it's like combining distance and speed - FSS might be a proxy for absolute fatigue, but CFQ measures change in fatigue (how much worse have you got in the past 3 months or before you were ill or whenever you can last remember). They should NOT ever be combined. Even combining individual participant results within a study is dodgy, because it's individual mean difference that is important and not difference in group means (what they are referring to here as "mean difference" confusingly). Combining the 3 different ways of recording CFQ is not valid because the likert and binomial scores do not exactly correspond.

One would hope that by making them do it again, as they originally said they would, they would realise why it is a problem, but clearly they haven't.
 
How will this affect any publications/applications for research etc that cite the previous version of the Cochrane review?

eta: also can someone explain what's going on here
Ongoing studies

We are not aware of any relevant ongoing studies.

Studies awaiting classification
Two studies that were ongoing when we ran our search for literature in May 2014 (Marques 2012; White 2012), are now published. The publications based on these studies (Clarke 2017; Marques 2015), need to be assessed for eligibility next time this review is updated.

New studies found at this update
We have added three new studies in this updated review (Jason 2007; Wearden 2010; White 2011).
Click to expand...
so if this 'new' review is based on the search they did in 2014,
how come they added 3 studies?

and 'no ongoing studies'?
 
Last edited:
Jonathan Edwards said:
I do not see how an editor could have approved ‘moderate certainty evidence that probably...’ This is meaningless. Moderate certainty cannot then be qualified by probably.
Click to expand...
Precisely. An officer in the forces once responded to a concern of mine with "I think I can assure you ...". Probably a politician now.

Esther12 said:
From their correspondence, it looks like 'probably' is standard for moderate quality evidence in Cochrane reviews:

"Some practical issues will arise, e.g. which term shall we use to reflect the grading? Usually
“probably” is used for moderate level evidence, while “may” is used for low level. We are willing to
accept “may”."

https://www.s4me.info/threads/cochr...evelopments-2018-19.10030/page-16#post-205744
Click to expand...
Sort of Chinese whispers methodology.
 
Having only skimmed through noting the overwhelming 'uncertain', 'low-certainty evidence' along the way I was gobsmacked when I read this para in the final discussion section re Agreements and disagreements with previous studies or reviews
This update provides valuable additional information when compared with the original review, and results reported in the original review are largely confirmed in this update. Moreover, the results reported here correspond well with those of other systematic reviews (Bagnall 2002; Larun 2011; Prins 2006) and with existing guidelines (NICE 2007). One meta‐analysis of CBT and graded exercise therapy (Castell 2011), suggests that the two treatments are equally efficacious, especially for people with co‐morbid anxiety or depressive symptoms.
Click to expand...

how the hell did they come up with that conclusion?
 
Tom Kindlon said:
They continue to classify the PACE Trial as having a low risk of bias in terms of selective reporting which is hard to accept.
Click to expand...

It's downright incompetent of them (putting it charitably). Indeed their own risk of bias is massive I think. Perhaps one day in a court room they may begin to realise it.

Outcome Reporting Bias ...
Selective reporting can occur through different ways:
1. omitting outcomes which are deemed to be unfavourable or statistically insignificant
2. adding new outcomes based on collected data to favour statistical significance
3. including only a subset of the analysed data in the published study
4. failing to report data that was analysed in the trial (such as adverse effects)
5. changing outcomes of interest (from primary outcomes to secondary outcomes if they do not yield significant results)
Click to expand...

PACE trial ... most of the above. Hugely exacerbated by being fully unblinded.

There truly must be some Machiavellian moves in play behind the scenes here.
 
S-VV said:
Bump. I was under the impression it was only classified as mental health, so this may be a positive change
Click to expand...
Actually I think it is. One of the big beefs not so long back was that ME being classified only under mental health. Maybe this touches on what @Michiel Tack posted, about Cochrane saying another review being considered from the ground up. The first step for that would be to have a sensible classification for where it would be done.
 
How long before the BPS crew pump out gloating tweets claiming PACE has been proven a low risk of bias in terms of selective reporting?

We all know what happens next, the new review they are promising gets a "multi disciplinary panel of experts", to spend five years muddying the waters in an attempt to make everyone forget about the Larun saga.

Then they just pump out another Larun like crap paper in around 2024 "updated" using five more years of gravy train crap of new psuedoscience and data manipulation from the BPS library that is being funded as we speak.

They have already stated publicly that subjective endpoints unblinded outcome reporting is good practice, so why will the next review be any different?
 
https://cmd.cochrane.org/our-scope

What we cover
The Cochrane Common Mental Disorders Review Group assists in the production and maintenance of Cochrane systematic reviews that assess the effects (including harm) of treatments in the prevention and management of a range of common mental disorders. We also engage in knowledge translation activities to support the use of our evidence in decision making.

We mainly undertake intervention reviews, and we also produce reviews using multiple treatment meta-analysis. We look at interventions that prevent, manage and treat common mental disorders.

Our reviews cover a range of conditions including depression, anxiety, eating disorders and post-traumatic stress disorder (PTSD). See our glossary of conditions for more details.

etc.

----------------------------

https://cmd.cochrane.org/glossary

Glossary

(Please note we are updating this glossary between Oct 2018 - Feb 2019)

Adjustment Disorder
Maladaptive reactions to identifiable psychosocial stressors occurring within a short time after onset of the stressor. They are manifested by either impairment in social or occupational functioning or by symptoms (depression, anxiety, etc.) that are in excess of a normal and expected reaction to the stressor.

Affective Disorders
Long-standing illnesses characterized by either sustained or intermittent disturbance in mood with no psychotic features, such as delusions, hallucinations, etc.

Affective symptoms
Mood or emotional responses dissonant with or inappropriate to the behavior and/or stimulus.

Anorexia Nervosa
Syndrome in which the primary features include excessive fear of becoming overweight, body image disturbance, significant weight loss, refusal to maintain minimal normal weight, and amenorrhea. This disorder occurs most frequently in adolescent females. (APA Thesaurus of Psychological Index Terms, 7th ed)

Anxiety
Persistent feeling of dread, apprehension, and impending disaster.

Anxiety Disorders
Disorders in which anxiety (persistent feelings of apprehension, tension, or uneasiness) is the predominant disturbance.

Attempted Suicide
The unsuccessful attempt to kill oneself.

Bipolar Disorder
A major affective disorder marked by severe mood swings (manic or major depressive episodes) and a tendency to remission and recurrence.

Bulimia
Episodic binge eating associated with the fear of not being able to stop eating, depressed mood, and self-deprecating thoughts. Binges are frequently terminated by self-induced vomiting. The bulimic episodes are not due to ANOREXIA NERVOSA or any known physical disorder.

Chronic Fatigue Syndrome
Cochrane has recently created eight new Networks of Cochrane Review Groups (CRGs). The formation of these networks provides a timely opportunity to review the scope of all CRGs and to consider changes where appropriate. In response to concerns raised by members of the CFS/ME community, Cochrane has been considering repositioning the editorial oversight of CFS/ME reviews. The Cochrane Common Mental Disorders Review Group currently sits within the Brain, Nerves and Mind (BNM) Network. In the future, reviews on this topic might sit with another Cochrane Review Group within the BNM Network, or they might transfer to another Network altogether, such as the Long Term Conditions and Ageing 2 Network. This is currently under consideration and a decision is anticipated before the end of 2018.

Combat Disorder
Neurotic reactions to unusual, severe, or overwhelming military stress.

Conversion Disorder
A disorder whose predominant feature is a loss or alteration in physical functioning that suggests a physical disorder but that is actually a direct expression of a psychological conflict or need.

Depression
Depressive states usually of moderate intensity in contrast with major depression present in neurotic and psychotic disorders.

etc.

------------------

Review Group Networks

https://www.cochrane.org/about-us/our-global-community/review-group-networks
 
I know @Michiel Tack already quoted some of Cochrane’s Editor-in-Chief, Dr Karla Soares-Weiser's comment, but reproducing here as I think it is really important (as well as ensuring it does not get lost if Cochrane ever decide to pull it down, but hopefully not that).

“Cochrane recognizes the importance of providing the best available evidence on interventions for ME/CFS to enable patients and clinicians across the world to make well-informed decisions about treatment. This amended review is still based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS from the 1990s. Having heard different views expressed about the evidence base for this condition, we acknowledge that the publication of this amended review will not resolve all the ongoing questions about this globally important health topic.

“We have decided, therefore, that a new approach to the publication of evidence in this area is needed; and, today we are committing to the production of a full update of this Cochrane Review, beginning with a comprehensive review of the protocol, which will be developed in consultation with an independent advisory group that we intend to convene. This group will involve partners from patient-advocacy groups from different parts of the world who will help us to embed a patient-focused, contemporary perspective on the review question, methods and findings.”

She added, “By forging better relationships with patients, as well as the groups that represent them and the clinicians who seek to treat them, we can improve the way in which future Cochrane Reviews in this area address important questions and meet patients’ and clinicians’ needs. I can confirm that work will begin on this new review at the beginning of 2020 and that we will keep patient groups regularly informed of progress during the subsequent months.”
Click to expand...
To me this clearly signals the beginning of a changing of the guard ("today we are committing to"). The change will not, cannot, happen instantly, and the fact they appear to to be applying due process to it I find encouraging. But of course it's actions not words that count, and we shall see. The statement "today we are committing to" I would see as meaning they should be reaching out to advocacy organisations and patient groups very soon (within a month?), at least as a gesture of good faith, even if only for the purpose of establishing lines of contact. But I'm unfamiliar with how long the basic infrastructure would take to set up, so maybe 1 month is not enough. But it would be good to soon have some initial outreach timings from them.

I would also hope it will all be done within the neurological classification, but I cannot imagine that with the above statement they could be daft enough to still try and do it within mental health.

I would think, given that we need to foster positive outcomes wherever we can, we need to segregate out this new forthcoming review from the existing one in terms of our relations with Cochrane, and see if it really can be a new start for all concerned. By all means still lambaste the existing one for the tripe that it is. But track and positively engage with the new review and its new process starting with a clean slate, hopefully having much less need to lambaste it, though if that is what it needs then so be it.

Cochrane’s Editor-in-Chief does seem to want to reboot the whole thing, and I think we should positively support that if it truly is her intent.

Hopefully this might be the beginning of the end of the old BPS empire within Cochrane, even if only the first few µs.

@dave30th, @Jonathan Edwards
 
You must log in or register to reply here.
Back
Top Bottom