Cochrane Review: 'Exercise therapy for chronic fatigue syndrome', Larun et al. - New version October 2019 and new date December 2024

The two primary outcome measures 15,16 are valid and reliable and have been used in previous trials. 4–7

15 Chalder T, Berelowitz G, Hirsch S, Pawlikowska T, Wallace P, Wessely S. Development of a fatigue scale. J Psychosom Res 1993; 37: 147–53.
Click to expand...
Given the glaring flaws in this measuring instrument, and the fact out of range readings may be misconstrued as valid in-range readings, I would love to see this measuring instrument properly and rigorously reassessed. If it were something you bought in a high street shop, it would be deemed not fit for purpose, and I'm sure fall foul of consumer protection laws.

If the limitations of this measurement scale were properly and formally exposed, and its use where no careful account were taken of those limitations, then that alone would crucify PACE, as well as those who invented it ... oh yes, they were also PACE authors.

Is there a way to get an independent assessment of medical measuring instruments, by bodies with the clout to say yay or nay?
 
Barry said:
I would love to see this measuring instrument properly and rigorously reassessed.
Click to expand...

It is not a measuring instrument. It's a diagnostic instrument. And it doesn't even do that job properly, because Oxford requires 6 months of fatigue, not 3.

[edit: Sorry, I'm not remembering things correctly. No mention of 3 months. The instructions say this:

"We would like to know more about any problems you have had with feeling tired, weak or lacking in energy in the last month. Please answer ALL the questions by ticking the answer which applies to you most closely. If you have been feeling tired for a long while, then compare yourself to how you felt when you were last well."

Options are "Less than usual", "No more than usual", "More than usual", & "Much more than usual".

It does not define what "usual" is. It does not define what "a long while" is. "When you were last well" may not be the same thing as "before you were ill".]
 
Last edited:

Attachments

Lucibee said:
It is not a measuring instrument. It's a diagnostic instrument. And it doesn't even do that job properly, because Oxford requires 6 months of fatigue, not 3.
Click to expand...
OK, so my medical terminology is incorrect. But whatever it is correctly called, it is nonetheless being used to measure fatigue in scenarios it is not valid for. An instrument of any kind is only valid when used within its limitations; one of the first things an engineer learns.

What is the difference between the two please?

ETA: I can take a guess. Something that diagnoses you have or do not have ME, will not necessarily have the sensitivity nor the range to measure differences in your ME severity between to points in time. Is that close?
 
Last edited:
Lucibee said:
Diagnostic tools should not be used as endpoint measures in this way. It certainly shouldn't have been used as a repeated measures tool with such poorly defined baseline comparison point (as I keep saying - sorry for sounding like a stuck record). And to use it as a repeated measures tool in trials where the researchers are trying to capture *improvement*, it is pretty much useless. Why? Because most of the scoring emphasis is about "getting worse", not better. And yet they have ploughed on with it for 30+ years without realising that. It beggers belief really. :banghead:
Click to expand...
This is well-known problem in many areas, not just medicine. That it is so well-known and still allowed to be used is frankly malfeasance.

In the early 2000's, the US education system made a big shift towards standardized testing in order to determine allocation of funds for schools. School that repeatedly had low scores on those tests were defunded. The end result was predictably that schools started building their curriculum strictly around those tests. They did not teach a normal curriculum anymore, instead most class time was spent on material straight from the standardized testing pool.

Many schools resorted to systematically cheating, literally helping their students to cheat on the tests, as otherwise they would be defunded and so out of a job. The end result was also predictable: students didn't learn much since all they did was prepare for the tests, not by learning but by memorizing as many answers as possible.

Using the very thing that is ultimately tested as an ongoing target means everything is done to optimize that endpoint. It creates the environment for maximum bias. In this area of research, it additionally compounds to bias from the researchers themselves, who are frankly borderline cultists in their beliefs about the BPS ideology, bias from the questionnaires, which are mediocre tools at best, and maximally biasing participants by making the "treatments" all about influencing responses on the questionnaires and telling them explicitly that it is their improved responses on those questionnaires that will free them of this horrible illness.

As usual, the problem isn't even with the PACE gang, it is with people who know better and let them whatever they want. It's a failure of oversight and perverse incentives. None of this is real science and even less real medicine and I am pretty certain that everyone involved understand what they are doing, they just think it's all fine since we're not actually sick so it makes no difference either way because Simon said so.
 
Last edited:
Barry said:
OK, so my medical terminology is incorrect. But whatever it is correctly called, it is nonetheless being used to measure fatigue in scenarios it is not valid for. An instrument of any kind is only valid when used within its limitations; one of the first things an engineer learns.

What is the difference between the two please?

ETA: I can take a guess. Something that diagnoses you have or do not have ME, will not necessarily have the sensitivity nor the range to measure differences in your ME severity between to points in time. Is that close?
Click to expand...

It's not you, it's them!

The problem is that any time someone puts a number on something, folks assume that it must therefore "measure" something. In this case, the number is supposed to give an indication of fatigue severity (more tired than usual vs much more tired than usual), but what does that actually mean? And once it has a number, all semblance of meaning is transferred and potentially lost. The number becomes more meaningful than what it originally signified. Subjecting it to analysis gives it yet more meaning. (But without it actually meaning anything at all.)

At least with the PF (physical function) scale, the number has some sort of a definition behind it (being able to walk more than a block or a mile is quantifiable thing), but what on earth does "feeling a bit more tired than usual" mean in that context? And all the questions in the CFQ are incredibly vague (Do you feel weak? How's your memory?). Relating/comparing that to another person is impossible. And even within a person, your impression of fatigue will change over time, particularly if you are undergoing a programme of fatigue re-education to actively redefine what is "normal".
 
Lucibee said:
It's not you, it's them!

The problem is that any time someone puts a number on something, folks assume that it must therefore "measure" something. In this case, the number is supposed to give an indication of fatigue severity (more tired than usual vs much more tired than usual), but what does that actually mean? And once it has a number, all semblance of meaning is transferred and potentially lost. The number becomes more meaningful than what it originally signified. Subjecting it to analysis gives it yet more meaning. (But without it actually meaning anything at all.)

At least with the PF (physical function) scale, the number has some sort of a definition behind it (being able to walk more than a block or a mile is quantifiable thing), but what on earth does "feeling a bit more tired than usual" mean in that context? And all the questions in the CFQ are incredibly vague (Do you feel weak? How's your memory?). Relating/comparing that to another person is impossible. And even within a person, your impression of fatigue will change over time, particularly if you are undergoing a programme of fatigue re-education to actively redefine what is "normal".
Click to expand...
Ah yes, I see what you mean. Arbitrary in the extreme, yet treated as if accurate and precise ... when no one even knows what the reference is in the first place.

Seems to be the first thing this breed of psychiatrists seem to do. Ascribe a number to something totally woolly to give it some parasitic credibility.
 
Last edited:
Lucibee said:
At least with the PF (physical function) scale, the number has some sort of a definition behind it (being able to walk more than a block or a mile is quantifiable thing), but what on earth does "feeling a bit more tired than usual" mean in that context? And all the questions in the CFQ are incredibly vague (Do you feel weak? How's your memory?). Relating/comparing that to another person is impossible. And even within a person, your impression of fatigue will change over time, particularly if you are undergoing a programme of fatigue re-education to actively redefine what is "normal".
Click to expand...

In case of interest to you or anybody:
Stouten 2017. Figure 1.png

J Health Psychol. 2017 Aug;22(9):1192-1197. doi: 10.1177/1359105317707531. Epub 2017 May 12.
PACE-GATE: An alternative view on a study with a poor trial protocol.
Stouten B.

https://journals.sagepub.com/doi/full/10.1177/1359105317707531
 
Dolphin said:
In case of interest to you or anybody:
View attachment 8656

J Health Psychol. 2017 Aug;22(9):1192-1197. doi: 10.1177/1359105317707531. Epub 2017 May 12.
PACE-GATE: An alternative view on a study with a poor trial protocol.
Stouten B.

https://journals.sagepub.com/doi/full/10.1177/1359105317707531
Click to expand...
Wow. This picture speaks way, way more than 1,000 words and most of those words are profanity. As fraud goes, this makes the MMR paper and everything that followed it seem honest and good.
 
@Sly Saint What do you mean by 3 added studies?

As far as I know, no studies have been added to the data.

In their discussion section, they mention 2 relevant RCT's (GETSET and Marques et al. 2015) that have been published since their literature search but the data of these aren't included in the meta-analysis.
 
Last edited:
Barry said:
See quoted section in @Sly Saint's post ...
Click to expand...
Do you mean this: "We have added three new studies in this updated review (Jason 2007; Wearden 2010; White 2011)"?

This is a statement that was already in the previous version.

These studies (Wearden et al. 2010 is the FINE trial and White et al. 2011 the PACE trial) were added to the review done by Edmonds et al. back in 2004. So Larun et al. did an update of that version.

After problems were noted with the Larun et al. update, they have made an amendment to correct for some mistakes, but no new studies/data were added to the meta-analysis as far as I can see.
 
Thank you all who are working on the Cochrane case.
I'm not able to read much or word a substantial critique.

However, I think the easiest and quickest measures Cochrane now could undertake as a first setp to take evidence-based medicine and patients seriously were to add the editor-in-chief's comment on the review, so that it's immediately visible to anyone looking at both the review and its abstract.

I thought to write to the new editor in chief something around the following lines:

Dear ......
Thank you very much for your comment on the publication of the revised edition of the Cochrane review on exercise therapy for 'Chronic Fatigue Syndrome'.

Even though some amendments of the revised review have to be acknowledged, to us and other critics the review still appears strongly misleading, the evidence of the included studies is still overstated. Moreover, the authors' conclusions do not sufficiently reflect the restrictions worded in other parts of the review. For example, we consider it a grave neglect hat the review doesn't specify the inclusion criteria for patients more precisely than "able to participate in exercise therapy".

The plans announced for a full update sound promising. This process will take considerable time though, during which the still misleading parts of the review will continue to inform health care decisions with a high potential to harm patients. Given the high impact of Cochrane reviews for health care decisions around the globe, we think immediate action is needed.

We propose as a first step to add your announcement of the plans for a full update to the revised review, so that it's immediately visible to anyone looking at both the review and its abstract.

(add some formal e-mail language I'm currently not able to word.)

Edited to add: Maybe add some more criticsm about the "certainty of evidence" and announce that comments will be submitted as soon as possible, but a visible notice by the editor-in-chief would at least acknowledge that there are serious issues with the current review that have to be dealt with also in Cochrane's view.)

I'm aware this would still be far from ideal, but...

Is this something S4ME would like to consider setting up and sending?

@Andy
 
Last edited:
Michiel is now even more famous than he was before :)
Cochrane’s republication last week of its seriously problematic exercise-for-CFS systematic review has triggered an outpouring of comment about the organization’s flawed decision-making and low-quality scientific reasoning. One very smart member of the Science For ME forum, Michiel Tack, posted an excellent overview of the changes between the prior version and the one published last week.

I am reposting it below with his permission. A big disappointment in the review is the last point Michiel makes–that the new version maintains that the evidence for fatigue reduction through exercise is of “moderate” quality. Yet from e-mails exchanged in May between Cochrane and the Norwegian institute representing the review authors, it seemed likely that the quality or certainty of the evidence would be downgraded. David Tovey, Cochrane’s recently retired editor-in-chief, stated clearly in that e-mail exchange that the evidence for fatigue reduction could only be described as of “low” quality or certainty, as I recently reported. Perhaps Karla Soares-Weiser, the new editor-in-chief, disagreed with that position. If so, she is clearly wrong.
Click to expand...
http://www.virology.ws/2019/10/09/trial-by-error-more-on-the-revised-cochrane-exercise-review/
 
You must log in or register to reply here.
Back
Top Bottom