Sid
Senior Member (Voting Rights)
Trial investigators themselves.
-
New Forum Software Installed (Still a few issues but reopening the forum) More details.
Clinical effectiveness of an online group physical & mental health rehab programme for post-covid-19 condition REGAIN study, 2024, McGregor+
- Thread starter SNT Gatchaman
- Start date
Trial investigators themselves.
David Putrino goes on a Twitter rant. Excerpts —
(The paper said: "we prospectively monitored for post-exertional symptom exacerbation in the intervention arm during each contact of participants with the intervention team." and "We routinely monitored for signs and symptoms of post-exertional symptom exacerbation but did not observe any instances during the trial or follow-up period, indicating that the intervention was well tolerated."
That sounds more like their patients didn't suffer from PEM, or more likely the ones that did have PEM dropped out.)
(The paper said: "we prospectively monitored for post-exertional symptom exacerbation in the intervention arm during each contact of participants with the intervention team." and "We routinely monitored for signs and symptoms of post-exertional symptom exacerbation but did not observe any instances during the trial or follow-up period, indicating that the intervention was well tolerated."
That sounds more like their patients didn't suffer from PEM, or more likely the ones that did have PEM dropped out.)
Andy
Retired committee member
Trial By Error: New Long Covid Exercise-and-Therapy Study Claims Success Despite Clinically Insignificant Findings
A new study of an online group physical and psychological rehabilitation program for Long Covid confirms once again that people given an intervention purporting to help them are more likely to tell investigators that they feel better than those given nothing of the kind. People, this is not a surprising result! It certainly wasn’t necessary for the UK’s National Institute for Health Research to spend £1,200,000 to find out the answer. The real news here is that, even though the study was designed in a way guaranteed to generate an unknown amount of bias, the findings for the primary outcome were so miniscule that they apparently failed to reach the level believed to represent clinical significance.
In other words, don’t believe the abstract when it declares that the intervention was “clinically effective.” The data suggest otherwise–as the investigators themselves acknowledged in the bowels of the discussion section. No peer-reviewer should have allowed a claim of clinical effectiveness to pass unchallenged.
https://virology.ws/2024/02/20/tria...ss-despite-clinically-insignificant-findings/
A new study of an online group physical and psychological rehabilitation program for Long Covid confirms once again that people given an intervention purporting to help them are more likely to tell investigators that they feel better than those given nothing of the kind. People, this is not a surprising result! It certainly wasn’t necessary for the UK’s National Institute for Health Research to spend £1,200,000 to find out the answer. The real news here is that, even though the study was designed in a way guaranteed to generate an unknown amount of bias, the findings for the primary outcome were so miniscule that they apparently failed to reach the level believed to represent clinical significance.
In other words, don’t believe the abstract when it declares that the intervention was “clinically effective.” The data suggest otherwise–as the investigators themselves acknowledged in the bowels of the discussion section. No peer-reviewer should have allowed a claim of clinical effectiveness to pass unchallenged.
https://virology.ws/2024/02/20/tria...ss-despite-clinically-insignificant-findings/
even though the study was designed in a way guaranteed to generate an unknown amount of bias,
This is the core problem running through all these types of studies.
It is not that there is bias. While bias should ideally be removed in the design of the study, that is not always possible.
But what must happen for a study to be legitimate is for a trial design (including outcome measures) that reveal the type and amount of bias so that it can be accounted for in the analysis phase. They must be made overt.
That is, adequate control of relevant significant variables, including potential bias and confounders.
This is the core problem running through all these types of studies.
It is not that there is bias. While bias should ideally be removed in the design of the study, that is not always possible.
But what must happen for a study to be legitimate is for a trial design (including outcome measures) that reveal the type and amount of bias so that it can be accounted for in the analysis phase. They must be made overt.
That is, adequate control of relevant significant variables, including potential bias and confounders.
Last edited:
"No peer-reviewer should have allowed a claim of clinical effectiveness to pass unchallenged."
Peer-review is clearly broken in this area of medicine, and the profession itself is clearly incapable of reforming it.
Reform is going to have to be imposed upon them from outside.
Peer-review is clearly broken in this area of medicine, and the profession itself is clearly incapable of reforming it.
Reform is going to have to be imposed upon them from outside.
Dolphin
Senior Member (Voting Rights)
Apologies if this New Scientist article has been highlighted before. It's not open access.
https://www.sciencedirect.com/science/article/abs/pii/S0262407924002987
News & Technology
Analysis: Coronavirus recovery
Exercising can help people with long covid – at least a bit
An eight-week exercise programme modestly improved the quality of life of people with long covid, but it may not benefit everyone with the condition, finds Clare Wilson
Section snippets
Building up activity
Until recently, the main treatments in the UK for ME/CFS were rehabilitation programmes combined with talking therapy, but these are unpopular with some patient activists, who say they imply the condition is all in the mind.
Campaigners also say that encouraging people to raise their exercise levels can trigger post-exertional fatigue, something that can also be experienced by those with long covid.
In the new study, which involved 585 people with long covid, Gordon McGregor at University
https://www.sciencedirect.com/science/article/abs/pii/S0262407924002987
News & Technology
Analysis: Coronavirus recovery
Exercising can help people with long covid – at least a bit
An eight-week exercise programme modestly improved the quality of life of people with long covid, but it may not benefit everyone with the condition, finds Clare Wilson
Section snippets
Building up activity
Until recently, the main treatments in the UK for ME/CFS were rehabilitation programmes combined with talking therapy, but these are unpopular with some patient activists, who say they imply the condition is all in the mind.
Campaigners also say that encouraging people to raise their exercise levels can trigger post-exertional fatigue, something that can also be experienced by those with long covid.
In the new study, which involved 585 people with long covid, Gordon McGregor at University
Andy
Retired committee member
Dr Nicola Clague-Baker on Twitter
"A heavily edited letter that we produced in response to the New Scientist article - https://newscientist.com/letter/mg26134790-900-on-the-use-of-exercise-therapy-for-long-covid/"
On the use of exercise therapy for long covid
Published 21 February 2024
From Charles Shepherd, honorary medical adviser, ME Association; Sonya Chowdhury, CEO of Action for ME; and Nicola Baker at the University of Liverpool, UK
You report on the use of exercise for long covid. This condition and myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) have some important clinical and pathological overlaps and a significant proportion of people with long covid also meet diagnostic criteria for ME/CFS (17 February, p 14).
In new guidelines on ME/CFS in 2021, England’s National Institute for Health and Care Excellence decided to no longer recommend graded exercise therapy – fixed, incremental hikes in activity – but to recommend activity and energy management based on pacing, staying at levels that feel right for the individual. This followed a thorough review of all the clinical trials that had involved exercise therapy. People with ME/CFS have been harmed by exercise programmes. We hope those with long covid won’t experience the same fate.
https://www.newscientist.com/letter/mg26134790-900-on-the-use-of-exercise-therapy-for-long-covid/
"A heavily edited letter that we produced in response to the New Scientist article - https://newscientist.com/letter/mg26134790-900-on-the-use-of-exercise-therapy-for-long-covid/"
On the use of exercise therapy for long covid
Published 21 February 2024
From Charles Shepherd, honorary medical adviser, ME Association; Sonya Chowdhury, CEO of Action for ME; and Nicola Baker at the University of Liverpool, UK
You report on the use of exercise for long covid. This condition and myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) have some important clinical and pathological overlaps and a significant proportion of people with long covid also meet diagnostic criteria for ME/CFS (17 February, p 14).
In new guidelines on ME/CFS in 2021, England’s National Institute for Health and Care Excellence decided to no longer recommend graded exercise therapy – fixed, incremental hikes in activity – but to recommend activity and energy management based on pacing, staying at levels that feel right for the individual. This followed a thorough review of all the clinical trials that had involved exercise therapy. People with ME/CFS have been harmed by exercise programmes. We hope those with long covid won’t experience the same fate.
https://www.newscientist.com/letter/mg26134790-900-on-the-use-of-exercise-therapy-for-long-covid/
Andy
Retired committee member
Full letter can be seen here, https://meassociation.org.uk/2024/02/new-scientist-on-the-use-of-exercise-therapy-for-long-covid/
Andy
Retired committee member
Trial By Error: Letter to BMJ Seeking Correction in Study of Long Covid Physical-and-Mental Rehabilitation Program
Last month, The BMJ published a study of a rehab intervention for Long Covid in which the authors made claims that were not borne out by the data. The study was called “Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial” and was led was led by researchers from the University of Warwick in Coventry, England. (I critiqued it here.) In particular, the paper declared that the intervention was shown to be “clinically effective”—even though the results for the primary outcome did not meet the acknowledged threshold for a “minimally important” or “clinically important” difference.
This contradiction was noted in both a comment on the Science For ME forum and a rapid response appended to the article itself. Today, I sent the following letter to The BMJ’s editor-in-chief, Dr Kamran Abbasi. I cc’d Professor Gordon McGregor, the lead author, as well as my eleven co-signatories.
https://virology.ws/2024/03/11/tria...d-physical-and-mental-rehabilitation-program/
Last month, The BMJ published a study of a rehab intervention for Long Covid in which the authors made claims that were not borne out by the data. The study was called “Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial” and was led was led by researchers from the University of Warwick in Coventry, England. (I critiqued it here.) In particular, the paper declared that the intervention was shown to be “clinically effective”—even though the results for the primary outcome did not meet the acknowledged threshold for a “minimally important” or “clinically important” difference.
This contradiction was noted in both a comment on the Science For ME forum and a rapid response appended to the article itself. Today, I sent the following letter to The BMJ’s editor-in-chief, Dr Kamran Abbasi. I cc’d Professor Gordon McGregor, the lead author, as well as my eleven co-signatories.
https://virology.ws/2024/03/11/tria...d-physical-and-mental-rehabilitation-program/
Snow Leopard
Senior Member (Voting Rights)
God forbid they actually ask patients what threshold they think would be clinically significant.
Anyone who tries to define "minimally clinically important differences" based on some statistical definition, related to the standard deviation or other measure affected by signal-to-noise ratio of the scale is a charlatan.
Could very well be true!
But taking them at their word...if anyone does cite a minimally important difference and claims their findings meet that threshold, and it turns out that's not the case, they are charlatans as well, and need to correct their paper.
Andy
Retired committee member
Trial By Error: We Asked BMJ to Correct a Paper; BMJ Requested a Rapid Response; We Have Declined
Two weeks ago, I sent a letter to The BMJ on behalf of myself and 12 colleagues seeking a correction in a study published last month. The study, called Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” claimed that the intervention under investigation had been shown to be “clinically effective.” As our letter pointed out, this claim is untrue. In fact, the results for the primary outcome, health-related quality of life, fell below the threshold for what is considered to be a clinically important or minimally important difference on that measure. In other words, any measured benefits were so small as to be essentially meaningless even if they were statistically significant.
Last week, I received a response from Dr Nazrul Islam, an editor at The BMJ. He requested that we submit a rapid response to the article outlining our concerns. He indicated that The BMJ would ask the authors to respond, and that the journal would then act in accordance with its policies. After considering the invitation to submit a rapid response, our group has decided to decline it. I have sent Dr Islam a follow-up, explaining our position and again requesting that The BMJ take the necessary steps to correct the paper. Our response is below.
https://virology.ws/2024/03/24/tria...-requested-a-rapid-response-we-have-declined/
Two weeks ago, I sent a letter to The BMJ on behalf of myself and 12 colleagues seeking a correction in a study published last month. The study, called Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” claimed that the intervention under investigation had been shown to be “clinically effective.” As our letter pointed out, this claim is untrue. In fact, the results for the primary outcome, health-related quality of life, fell below the threshold for what is considered to be a clinically important or minimally important difference on that measure. In other words, any measured benefits were so small as to be essentially meaningless even if they were statistically significant.
Last week, I received a response from Dr Nazrul Islam, an editor at The BMJ. He requested that we submit a rapid response to the article outlining our concerns. He indicated that The BMJ would ask the authors to respond, and that the journal would then act in accordance with its policies. After considering the invitation to submit a rapid response, our group has decided to decline it. I have sent Dr Islam a follow-up, explaining our position and again requesting that The BMJ take the necessary steps to correct the paper. Our response is below.
https://virology.ws/2024/03/24/tria...-requested-a-rapid-response-we-have-declined/
rvallee
Senior Member (Voting Rights)
"If someone says it's raining and another person says it's dry, it's not your job to quote them both," goes a quote often attributed to journalism professor Jonathan Foster. "Your job is to look out of the f—-ing window and find out which is true."
Is 3>4, though? Who really knows? What kind of advanced math could really tell us for sure?
And 4 is just the barest minimum. Who gets by in their job always striving for the bare minimum, and more often than not not even meeting it? This is a race to the bottom, and no one's winning it.
Is 3>4, though? Who really knows? What kind of advanced math could really tell us for sure?
And 4 is just the barest minimum. Who gets by in their job always striving for the bare minimum, and more often than not not even meeting it? This is a race to the bottom, and no one's winning it.
Andy
Retired committee member
Trial By Error: My Letter to Lead REGAIN Trial Investigator Seeking Correction of Bogus Claims of Clinical Effectiveness
Two weeks ago, I sent a letter to The BMJ on behalf of myself and 12 colleagues seeking a correction in a study published last month. The study, called Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” claimed that the intervention under investigation had been shown to be “clinically effective.” As our letter pointed out, this claim is indefensible; the results for the primary outcome, health-related quality of life, fell below the threshold for what is considered a minimally important or clinically important difference.
Last week, The BMJ responded by asking us to submit a rapid response. We declined, as I reported earlier this week. Today I sent the following letter directly to Dr Gordon McGregor, a clinical exercise physiologist at the University of Warwick, who was the lead trial investigator as well as the lead author of the published report.
https://virology.ws/2024/03/28/tria...on-of-bogus-claims-of-clinical-effectiveness/
Two weeks ago, I sent a letter to The BMJ on behalf of myself and 12 colleagues seeking a correction in a study published last month. The study, called Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” claimed that the intervention under investigation had been shown to be “clinically effective.” As our letter pointed out, this claim is indefensible; the results for the primary outcome, health-related quality of life, fell below the threshold for what is considered a minimally important or clinically important difference.
Last week, The BMJ responded by asking us to submit a rapid response. We declined, as I reported earlier this week. Today I sent the following letter directly to Dr Gordon McGregor, a clinical exercise physiologist at the University of Warwick, who was the lead trial investigator as well as the lead author of the published report.
https://virology.ws/2024/03/28/tria...on-of-bogus-claims-of-clinical-effectiveness/
rvallee
Senior Member (Voting Rights)
There was a NHS-organized Long Covid conference (clearly to the surprise of everyone, it doesn't seem to have been announced beforehand) and McGregor presented the results of REGAIN as being positive, he even included the truncated axis that is a thing only done shadily, so I doubt he or BMJ will budge from pretending that 3 is bigger than 4, nor that always striving for the barest minimum just isn't good or useful science.
Sid
Senior Member (Voting Rights)
This should be banned by journals. It’s so misleading.
REGAIN Team has published a bullshit rapid response to the concerns raised. The handling editor wrote our group back with the link. I sent back this response:
Dear Dr Islam--
Thank you for the update. We will review the response in detail, but I will say from a first glance that it is too clever by half and clearly skates over key criticisms. Just one glaring example: In their effort to rebut the argument that they have extrapolated the findings way beyond the population studied, the authors' reference the conclusion of the full paper rather than the conclusion of the abstract.
In our letter, we referenced the abstract, which reads: "In adults with post-covid-19 condition, an online, home based, supervised, group physical and mental health rehabilitation programme was clinically effective at improving health related quality of life at three and 12 months compared with usual care." Furthermore, the box highlighting "What is already. known on this topic" and "What this study adds" includes zero mention of the highly salient detail that these patients had been hospitalized. As you undoubtedly know, readers are far more likely to review the abstract and highlights sections than they are to read the paper to the very end.
Had McGregor et al added "at least three months after hospital discharge for covid-19" to the sentence in the abstract and to the box highlighting the findings, we would not have raised this particular issue. Apparently, McGregor et al and The BMJ's editorial team believe it is entirely appropriate for investigators to offer more sweeping statements in the conclusions in a paper's abstract and in its highlights section as long as the very last paragraph of the paper's text includes more accurate information.
We strongly disagree. Such flagrant internal contradictions are not acceptable in reporting scientific findings. First-year epidemiology students at Berkeley would be reprimanded for this transparent effort to convey the impression in the paper's most high-profile sections that the findings are broadly applicable to the entire population of patients with post-covid-19 condition rather than limited to the much smaller cohort of those who were hospitalized.
The responses to our other concerns are equally disingenuous and inadequate--more on that to come. Readers of The BMJ deserve better.
Best--David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
Dear Dr Islam--
Thank you for the update. We will review the response in detail, but I will say from a first glance that it is too clever by half and clearly skates over key criticisms. Just one glaring example: In their effort to rebut the argument that they have extrapolated the findings way beyond the population studied, the authors' reference the conclusion of the full paper rather than the conclusion of the abstract.
In our letter, we referenced the abstract, which reads: "In adults with post-covid-19 condition, an online, home based, supervised, group physical and mental health rehabilitation programme was clinically effective at improving health related quality of life at three and 12 months compared with usual care." Furthermore, the box highlighting "What is already. known on this topic" and "What this study adds" includes zero mention of the highly salient detail that these patients had been hospitalized. As you undoubtedly know, readers are far more likely to review the abstract and highlights sections than they are to read the paper to the very end.
Had McGregor et al added "at least three months after hospital discharge for covid-19" to the sentence in the abstract and to the box highlighting the findings, we would not have raised this particular issue. Apparently, McGregor et al and The BMJ's editorial team believe it is entirely appropriate for investigators to offer more sweeping statements in the conclusions in a paper's abstract and in its highlights section as long as the very last paragraph of the paper's text includes more accurate information.
We strongly disagree. Such flagrant internal contradictions are not acceptable in reporting scientific findings. First-year epidemiology students at Berkeley would be reprimanded for this transparent effort to convey the impression in the paper's most high-profile sections that the findings are broadly applicable to the entire population of patients with post-covid-19 condition rather than limited to the much smaller cohort of those who were hospitalized.
The responses to our other concerns are equally disingenuous and inadequate--more on that to come. Readers of The BMJ deserve better.
Best--David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
Does anyone by any chance have a pdf of REGAIN as it was published? I'm preparing a talk and, from what I can tell, they have added a reference in the "conclusion" in the abstract and in the "What's New" box about the fact that these patients were hospitalized. One of the criticisms of the study was that these statements did not make that clear, so it seems, if I'm right, that they added that slyly and without saying anything after-the-fact.
Andy
Retired committee member
The Wayback Machine internet archive stored a version on the 17th April, https://web.archive.org/web/2024041....com/content/bmj/384/bmj-2023-076506.full.pdf - does that help?
A correction was published on 1st May 2024, https://www.bmj.com/content/385/bmj.q988
"Several sections of this paper by McGregor and colleagues (BMJ 2024;385:e076506, doi:10.1136/bmj-2023-076506, published 7 February 2024) have been updated for clarity of the study population. The conclusion in the abstract and the first and second points of the “What this study adds” subsection of the box should have made it clear that in adults with post-covid-19 condition “at least three months after hospital discharge for covid-19” the online, home based, supervised, group physical and mental health rehabilitation programme (REGAIN) showed clinical benefits and lack of harm."
A correction was published on 1st May 2024, https://www.bmj.com/content/385/bmj.q988
"Several sections of this paper by McGregor and colleagues (BMJ 2024;385:e076506, doi:10.1136/bmj-2023-076506, published 7 February 2024) have been updated for clarity of the study population. The conclusion in the abstract and the first and second points of the “What this study adds” subsection of the box should have made it clear that in adults with post-covid-19 condition “at least three months after hospital discharge for covid-19” the online, home based, supervised, group physical and mental health rehabilitation programme (REGAIN) showed clinical benefits and lack of harm."