Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian

[rvallee]

Trawling would be something like subgroup analyses that had never been part of the protocol in the first place

Then you're in luck!

This is literally what the PACE authors recently published in their own journal (the board of this journal is basically the BPS folks responsible for most of the ME-BPS literature): https://www.s4me.info/threads/patte...ndrome-2020-chalder-sharpe-white-et-al.15313/. And they plan on publishing more. We have no idea why. It doesn't appear to serve any purpose. But this is typical of BPS experiments.

In a recent CBT experiment by Wyller, one of the 2 peer reviewers plainly stated they only read the abstract and when pointed out BMJ finds no issue with that. PACE was sold on the promise that it would have objective outcomes and no one but us found issue with dropping that even though it was the whole point. After years of being pressed to justify this change (the real reason is that another trial had a null result) they simply stated that they prefer this outcome because it aligns with their expectations. I don't know what we're supposed to do with something like that but it's wrong.

In a recent trial of CBT for "dissociative seizures", which for years promised only the objective outcome of number of seizures would be significant, and yet lo and behold, they recently announced success in their endeavor even though the much-promised primary outcome, which they assured is the only outcome that counts, was null. There is simply no actual rigor in this field, only the appearance of. It's bona fide alternative medicine using the technical language of science but none of the substance.

 

[Hoopoe]

@rvallee the correct term is pseudoscience. It looks sort of like science but it's actually a belief system.

 

[dave30th]

And a newly published tool suggests a more lenient approach to bias!

I wrote about that here: https://www.virology.ws/2019/08/31/...nes-new-bias-guideline-is-lp-study-co-author/

One of the authors is a co-author of many studies with Professor Crawley. One was the Lightning Process study, which violated all sorts of rules designed to protect against bias. The other was the school absence study, in which the investigators interviewed more than 100 minors and their families and then published it without ANY ethical review by declaring it to be "service evaluation."

 

[Robert 1973]

I’ve only managed to a read a few posts in this thread, so apologies if I’m misunderstanding or repeating anything that has been said.

First, thanks to @Hilda Bastian for taking on this task and engaging in this forum.

Quite early on, I said I had trouble with the statement as it stood, and explained why people would disagree with it, but thanks for clarifying. I still don't agree, but yes, I disagree with that version a lot less!

I apologise if I have missed the explanation in this thread but please can you you clarify exactly what it is that you disagree about and why?

I truly understand the point with the particular types of trials you are speaking about, and I have criticized the PACE trial on the grounds of inadequate measurement tools. But you aren't going to convince people who have battled for exactly the opposite - for people's voices about their care to be taken seriously and given weight - to sign up to dismissing patients' reports so categorically.

I don’t see a conflict here. I want patients’ voices about our care to be heard. I want doctors and researchers to listen and take heed of my account of my symptoms and experience. And I want my accounts of my symptoms to be recorded in clinical trials. But, having listened to patients, I want scientists to test their hypothesis by designing clinical trials which produce useful, interpretable results. Where trials cannot be blinded that means using at least one objective outcome measure against which more subjective measures can be compared.

The Rituximab trials clearly demonstrated how difficult it can be to interpret the results from unblinded trials in ME/CFS. The results from the unblinded trials produced far better results than PACE, but the subsequent double-blind RCT showed the drug treatment to be no better than the placebo.

And subjective outcome measures have a higher risk of bias - that just doesn't necessarily make them useless, and it doesn't doom the trial they're in to uselessness either, especially if other sources of bias are low.

If we accept that subjectivity is a scale, can we agree there is a point on that scale where results in unblinded trials become uninterpretable (if they are not supported by at least one objective outcome measure)?

Fact is that actigraphy was initially supposed te an outcome measure of the 'definitive' PACE trial, being recommended by a patient group. Actigraphy was indeed measured at baseline, but was dismissed as a outcome measure during the trial. Those of us who have seen the PACE steering committee notes are rather suspicious of the rationale behind dropping it - after the results of Dutch colleagues were discussed and found that activity levels did not improve despite self-reports stating the contrary.

I’m guessing you know about this Hilda, but if not please read @Lucibee ’s blog: on “Whatever happened to Actigraphy?” Snow Leopard uses diplomatic language. To me the minutes provide convincing evidence that the investigators were not interested in genuinely testing the effectiveness of the therapies but rather seeking to design a trial that produced results which supported their preconceptions.

This view is supported by the correspondence in BMC Psychology in which Sharpe et al confessed: “We prefer the definitions of recovery we used to those used by Wilshire et al. [those specified in the protocol] as they give absolute rates more consistent both with the literature, and with our clinical experience.” (https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-019-0288-x)

I appreciate the genuine difficulties of trying to test the effectiveness of treatments which cannot be blinded for conditions which are difficult to assess objectively, but that does not mean that we should accept bad methodology. And PACE was worse than that because the investigators appear to have deliberately chosen to drop objective outcome measures and diverted from the protocol in order to produce results which are “more consistent with [their] clinical experience.”

[edit - missing word "tiral" and changed "interpretable" to "uninterpretable"]

 

[MSEsperanza]

I wrote about that here: https://www.virology.ws/2019/08/31/...nes-new-bias-guideline-is-lp-study-co-author/

One of the authors is a co-author of many studies with Professor Crawley. One was the Lightning Process study, which violated all sorts of rules designed to protect against bias. The other was the school absence study, in which the investigators interviewed more than 100 minors and their families and then published it without ANY ethical review by declaring it to be "service evaluation."

There is also the worrying role the Sciene Media Centre (UK) played in presenting the Lightning Process trial (SMILE):

https://www.virology.ws/2017/12/18/trial-by-error-my-questions-for-the-science-media-centre/

What I don't recall now is whether the trial investigators reported how school attendance was measuerd. On the SMC webiste it's said it was an objective measure:

Dr James Thompson, Honorary Senior Lecturer in Psychology, UCL, said:
“The treatment in this study looks like it had an effect, at least by the standard of most clinical trials. To be extra robust I would have liked to see more objective measures, [...]
“One limitation is that self-report scales can be subject to placebo effects, however if the patients feel better in the experimental condition in which they receive extra help, even if everyone knows it, then that is something and the pupils miss less school, which is an objective measure. In this case it may not have been the CBT element of the treatment, but it looks like it.” (My bolding.)

https://www.sciencemediacentre.org/expert-reaction-to-controversial-treatment-for-cfsme/

 

[dave30th]

What I don't recall now is whether the trial investigators reported how school attendance was measuerd. On the SMC webiste it's said it was an objective measure

This is slightly complicated. The protocol promised they would have self-reported school attendance and they would vet that against official records. In the trial, they used self-reported school attendance and forgot to mention anything about the promise to vet those findings against official data. That didn't matter for the former primary outcome--school attendance at six months. That outcome had null results as a self-reported outcome, but the news was buried. In contrast, school attendance at 12 months had a positive result, so they highlighted that in promoting the study-without ever mentioning that they did not check that against official school records. The whole thing was a methodological and ethical disaster that has been thoroughly whitewashed by BMJ.

 

[rvallee]

This view is supported by the correspondence in BMC Psychology in which Sharpe et al confessed: “We prefer the definitions of recovery we used to those used by Wilshire et al. [those specified in the protocol] as they give absolute rates more consistent both with the literature, and with our clinical experience.” (https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-019-0288-x)

The change in question was the lowering of the secondary-outcome-newly-promoted-to-primary-outcome SF-36 from 85 down to 60 as the threshold for recovery. The cutoff point to participate in PACE was SF-36 <65, so 13% of participants met the definition for recovery of SF-36 >=60 published in the paper on the day they entered the trial (although this is something that fluctuates a lot so single scores are unreliable). This major issue has been dismissed by the authors with sophistry and no one but us seems to find issue with overlapping subjective entry and recovery criteria changed mid-way in an open label trial of behavioral manipulation.

Peter White, the primary investigator, has previously defined his own threshold of a SF-36 below 75 qualifying as severe impairment. Obviously 60 is significantly below White's own threshold of 75 to qualify as severe impairment, yet he published a threshold of 60 in PACE as satisfying a notion of recovery. 60 is about the average score for an octogenarian population.

Despite this, people keep claiming that PACE "proved" ME is psychological. A pragmatic randomized trial, which cannot infer cause. Even though it had null results. Which is dismissed by saying many small trials show the same. Even though PACE was sold as the definitive trial of those smaller prior trials and its interpretation modified to align with prior expectations, which the trial was meant to test, as admitted by the authors themselves.

A later PACE paper acknowledged there was an actual increase in disability benefits and that it is not curative, only possibly beneficial to some. The authors and Richard Horton went on a PR tour to tout the CBT/GET treatments as a cure, for those who want it, based on the PACE results. There is a wide gap between the basic facts and how they are presented, published and discussed. And they all bias in the same direction, without fail.

 

[BurnA]

why did it take me so long to dig into and then engage with this issue?

there was no ethical obligation for me to spend a lot of time and stick my neck out for this particular one.

Hi Hilda, I hope you weren't pressurised and I would not like to think you are here or performing this task for any reason other than its your free choice.

Presumably one gets to be in a position like yours from a general desire and motivation to promote better science for the greater good.

I'd be interested to know at what point this becomes "sticking your neck out" , or why that would be a consideration.

 

[Hilda Bastian]

Take a look around at other social movements. How polite are they? It took BOTH Malcom X and MLK to make progress with civil rights in the USA. Then the progress stopped when many black people started to be nice again.

Why do you think we have to be extra polite not to offend idiots who don't understand the scientific method?

That's a very different discussion. I was talking about scientific weakness of an argument. It really doesn't bother me personally when you label me an idiot who doesn't understand the scientific method.

 

[Hilda Bastian]

Hi Hilda, I hope you weren't pressurised and I would not like to think you are here or performing this task for any reason other than its your free choice.

Presumably one gets to be in a position like yours from a general desire and motivation to promote better science for the greater good.

I'd be interested to know at what point this becomes "sticking your neck out" , or why that would be a consideration.

Oh yes, it was totally my choice to enter the discussion about ME/CFS and the Cochrane review, and there was no pressure or incentive to agree to lead the IAG.

It was sticking my neck out because it would make me a target for prolific aggressive behavior, both directed at me (for not agreeing with everything or acceding to demands for what I should do), and because I'd be tagged into a flood of conversation that included it. Longterm commitment to an issue means you can't manage that so easily (eg by muting/blocking/diverting to spam, particular words like the issue's name and people/email accounts). That's a drag on your communication media & psyche, and a disproportionate time cost, when there are so many other incredibly worthwhile and more gratifying things to do that don't carry those productivity costs. (And productivity loss when you're a freelancer is also economic loss.)

The reason I got involved, knowing my taking a strong public position was likely to result in longterm engagement over the Cochrane review, was the general desire you raise, plus thinking at the time that the others who could make a difference didn't look like they were going to do it.

 

[Snow Leopard]

That's a very different discussion. I was talking about scientific weakness of an argument. It really doesn't bother me personally when you label me an idiot who doesn't understand the scientific method.

If it wasn't clear, I'm not referring to you, but the hypothetical people who "clock out", as soon as we mention points of "everyone knowing" about the importance of participant blinding in controlling biases and other methodological flaws leading to high risk of bias.

 

[Sean]

It feels as if the researchers have been given all the benefit of the doubt at every step.

And then some. It is astounding just how protected a species these guys are. Clearly various powers that be regard this, er, project as too important to be allowed to fail.

I’m guessing you know about this Hilda, but if not please read @Lucibee ’s blog: on “Whatever happened to Actigraphy?” Snow Leopard uses diplomatic language. To me the minutes provide convincing evidence that the investigators were not interested in genuinely testing the effectiveness of the therapies but rather seeking to design a trial that produced results which supported their preconceptions.

It is anti-science.

 

[Hilda Bastian]

If it wasn't clear, I'm not referring to you, but the hypothetical people who "clock out", as soon as we mention points about "everyone knowing" about the importance of participant blinding in controlling biases and other methodological flaws leading to high risk of bias.

If the claim that started this discussion was always framed as you phrased it here, then it wouldn't have been a good example of what I mean, so the discussion wouldn't have taken place. But that's not the statement I've been taking issue with. Nor is the statement I mentioned an unusual aberration. The discourse around the Cochrane review often includes variants on the argument that all unblinded trials with subjective endpoints are known by everyone to be valueless - often citing/linking to the Students4BestEvidence blog post since that came out. (And it's just one example of a weak argument that doesn't do justice to a very strong case with plenty of very strong arguments that doesn't need to overplay anything.)

The thread about "clocking out" came from me saying I was one of the people who clocked out because of claims that I could see were problematic. So whether you intended to include me in the category or not, that was a logical conclusion to draw from the thread. (Thanks for saying you didn't mean that.)

It does no justice to serious complex topics, I know, but that's life in a time of information overload. When you're a casual bystander to debates about a gazillion topics, you inevitably form quick impressions about the strength of a case based on the strength of the arguments you see made for it, and that can drive whether or not it's something you want to learn more about or not. It's superficial, it's unfair, but no one can give every single subject the attention it deserves.

 

[Hilda Bastian]

I’ve only managed to a read a few posts in this thread, so apologies if I’m misunderstanding or repeating anything that has been said.

First, thanks to @Hilda Bastian for taking on this task and engaging in this forum.

I apologise if I have missed the explanation in this thread but please can you you clarify exactly what it is that you disagree about and why?

Thanks! I'm really sorry to everyone who picks this up at a later point, but while I'll continue to read this thread for the long haul, I genuinely don't think I can make my point of view on the very narrow issues I was discussing any clearer than I already have.

There's going to be plenty of opportunity in the coming months, though, to thoroughly discuss issues as they pertain specifically to evidence around ME/CFS and the Cochrane review. And I'm really grateful to all of you for contributing your knowledge and time to this.

 

[spinoza577]

It is astounding just how protected a species these guys are. Clearly various powers that be regard this, er, project as too important to be allowed to fail.

it should be a long tradition. Maybe anybody can tell me where psychiatry is located, as it doesn´t seem to be psychology nor neurology? Or is it a technical issue, like neurology for degenerative diseases, and psychiatry for functional diseases in the brain? At least this were understandable.

But as I see it, psychiatrists are sitting on their tradition with people or patients not taken seriously in the wider past and defend anything mysterious. But maybe it´s simply too difficult to investigate dysfunctions? And it´s simply much easier to work with any CBT from the psychology, claiming that deeds influence the function of the nerves than to investigate the machinery where all deeds and believes need to rely upon? An additional problem is that medicals are not well understood and are potentially harmful of course (which may be, of course, difficult to weight against the benefit).

The point would be that psychiatrists would not be driven by themselves to clarify their position, as it would diminish the freedom, the room of possibilities (including not justified ones).

 

[MSEsperanza]

Hi Hilda,

I don't know whether I will be able to reply to some of your and others valuable posts within reasonable time, but I'd like to let some non-substantial thoughts here.

I wrote this over the last days, so apologies if some points are now redundant.

To facilitate reading, I will post this in two parts.

First, I'd like to second others comments expressing their appreciation for your engagement with us on the forum.

Also, apologies for my mix-up of questions and addressees in my previous comments.

Open exchange and taken seriously as a patient and at the same time as an individual wanting and being capable to roughly understand the reasoning behind the research on the illness I suffer from and the healthcare I have been offered is something I very rarely encountered in the medical world since I was diagnosed with ME/CFS 22 years ago.

From a certain network of medical and other health care professionals that invented or adhere to the cognitive-behavioral approach to ME and all 'MUS', people with ME and their advocates are confronted with prejudices about what they collectively think, what their motivations are to engage in ME research, that they don't know the real business of psychiatry, psychology, trial design etc. and therefore their (our) criticism is of no value.

Cochrane acted as a protector of this network and still does. I see that you and maybe even Cochrane's editor-in-chief can't do much in this regard even if you wanted, because there is lack of consensus at Cochrane. We patients have to live or die with the consequences for at least two more years and at the same time the only chance to change things is to engage with the IAG even if there still is no clear signal we could trust Cochrane.

It's difficult to believe how much irrationality, factual errors, and blaming is applied by some people of the CBT-approach-to-ME-network to disdain criticism by patient organizations and by simply all people, be it civil scientists or professionals, challenging their work.

Many of the critics are S4ME forum members. I find it particularly unjustified if they are presented as opponents of psychology per se, given that on S4ME all ME research is being discussed and criticized, and also much if not most biomedical ME research is assessed as lacking robust evidence or over-interpreting their findings, whereas the little proportion assessed as good research, including psychological, is both highly appreciated and constructively criticized (often people find crucial things could be done in a better way even in 'good' research).

Mostly, there is no urgent need to publicly criticize bad biomedical research on ME by pwME on other platforms than here because, except by some fringe practitioners, no general treatment suggestions are derived from that research.

Also, many S4ME members are at the same time staff of other international, national or regional patient organizations, and these don't always have the same stance on many issues. I think "agree to disagree" is daily practice on S4ME and it's worthwhile to engage in a discussion to be clear about what exactly we disagree.

I'm aware that there is much irrationality and hate out there on social media in general, and patients and their advocates are no exception to this general phenomenon. I don't mean to defend those. It's a not-organized minority without any institutional affiliation though, and I find it irresponsible and much more harmful if some highly influential healthcare professionals and their allies in some medical journals and certain media outlets associate all well-founded criticism with the general phenomenon of internet trolls or adherents of diverse anti-science or pseudo-science stances. Healthcare professionals' or other academics' and civil scientists' professional expertise is being belittled or ignored if they challenge the cognitive-behavioral approach to ME.

I want to understand and even though my illness and related personal circumstances restrict the speed and scope of what I'm able to take in, I still want and am able to learn. S4ME is a place where this happens. We all have different professional backgrounds and expertise, including medical professors and other scientists, doctors, psychologists , and we learn from each other, also and often from controversies.

I appreciate the forum culture that encourages everyone to ask and point to potential misunderstandings before they accumulate to a communication barrier. I'm always happy when people point me to factual or logical errors in my comments.

Unfortunately, when I see such an accumulation of potential misunderstanding emerging but feel not up to commenting, I sometimes become impatient and comment nevertheless. Apologies if that results sometimes in muddled posts and multiple editing.

 

[MSEsperanza]

(Continued from my previous post.)

So what I basically wanted to say or ask is:

1. It's good that you're here. I see that the language we use regarding blinding and subjective and (potential) objective outcomes/ endpoints etc. could be more precise to avoid misunderstandings. From the forum discussion I also gained some new insights regarding these issues and that I used some terms in a misleading or maybe even erroneous way.

However, I don't know if I will be able to take part in the discussion in the future, and anyway, I'm not one of the forum's experts with regard to trial design and the methodology of systematic reviews and relevant related issued. But maybe I'm not alone in genuinely wanting to understand the point you made here in response to Jonathan Edwards:

That said, however, I see that sentiment, without any qualifications, repeatedly elsewhere: and I've seen people dissed for saying they disagree, and I've seen the impact of that.

I think you referred to Jo's sentence "The use of subjective outcomes (alone) from unblinded trials to judge the usefulness of a treatment is valueless".

In my opinion, it would be extremely helpful to work with this sentence and to know with which amendments of this sentence you could agree.(*)

2. Many forum members never got genuine responses to their criticism of the Cochrane reviews on exercise and Cochrane's review methodology in general. Most of this criticism is posted and being discussed on other threads.

In addition, many forum members, due to their illness and related personal circumstances, aren't on the forum regularly, a couple of members have to refrain completely from posting for an unforeseeable period of time. Also, I can't omit to once again mention the late Robert Courtney. Other knowledgeable forum members recovered or found out that they were misdiagnosed and understandably now are busy with other things. So, most people who contributed to the discussion before the IAG was constituted, won't be able to engage with the IAG.

So I'd like to ask you: Are you interested in reading what forum members think is the most relevant of the criticism posted and discussed on S4ME and would you like to comment here on some points you find crucial?

(I'm not a staff member of S4ME and only speak for myself.)

(*)Edited to temporarily remove my own suggestion of amendments.

 

[Hilda Bastian]

Hi Hilda,

I don't know whether I will be able to reply to some of your and others valuable posts within reasonable time, but I'd like to let some non-substantial thoughts here.

I wrote this over the last days, so apologies if some points are now redundant.

To facilitate reading, I will post this in two parts.

First, I'd like to second others comments expressing their appreciation for your engagement with us on the forum.

Also, apologies for my mix-up of questions and addressees in my previous comments.

Open exchange and taken seriously as a patient and at the same time as an individual wanting and being capable to roughly understand the reasoning behind the research on the illness I suffer from and the healthcare I have been offered is something I very rarely encountered in the medical world since I was diagnosed with ME/CFS 22 years ago.

From a certain network of medical and other health care professionals that invented or adhere to the cognitive-behavioral approach to ME and all 'MUS', people with ME and their advocates are confronted with prejudices about what they collectively think, what their motivations are to engage in ME research, that they don't know the real business of psychiatry, psychology, trial design etc. and therefore their (our) criticism is of no value.

Cochrane acted as a protector of this network and still does. I see that you and maybe even Cochrane's editor-in-chief can't do much in this regard even if you wanted, because there is lack of consensus at Cochrane. We patients have to live or die with the consequences for at least two more years and at the same time the only chance to change things is to engage with the IAG even if there still is no clear signal we could trust Cochrane.

It's difficult to believe how much irrationality, factual errors, and blaming is applied by some people of the CBT-approach-to-ME-network to disdain criticism by patient organizations and by simply all people, be it civil scientists or professionals, challenging their work.

Many of the critics are S4ME forum members. I find it particularly unjustified if they are presented as opponents of psychology per se, given that on S4ME all ME research is being discussed and criticized, and also much if not most biomedical ME research is assessed as lacking robust evidence or over-interpreting their findings, whereas the little proportion assessed as good research, including psychological, is both highly appreciated and constructively criticized (often people find crucial things could be done in a better way even in 'good' research).

Mostly, there is no urgent need to publicly criticize bad biomedical research on ME by pwME on other platforms than here because, except by some fringe practitioners, no general treatment suggestions are derived from that research.

Also, many S4ME members are at the same time staff of other international, national or regional patient organizations, and these don't always have the same stance on many issues. I think "agree to disagree" is daily practice on S4ME and it's worthwhile to engage in a discussion to be clear about what exactly we disagree.

I'm aware that there is much irrationality and hate out there on social media in general, and patients and their advocates are no exception to this general phenomenon. I don't mean to defend those. It's a not-organized minority without any institutional affiliation though, and I find it irresponsible and much more harmful if some highly influential healthcare professionals and their allies in some medical journals and certain media outlets associate all well-founded criticism with the general phenomenon of internet trolls or adherents of diverse anti-science or pseudo-science stances. Healthcare professionals' or other academics' and civil scientists' professional expertise is being belittled or ignored if they challenge the cognitive-behavioral approach to ME.

I want to understand and even though my illness and related personal circumstances restrict the speed and scope of what I'm able to take in, I still want and am able to learn. S4ME is a place where this happens. We all have different professional backgrounds and expertise, including medical professors and other scientists, doctors, psychologists , and we learn from each other, also and often from controversies.

I appreciate the forum culture that encourages everyone to ask and point to potential misunderstandings before they accumulate to a communication barrier. I'm always happy when people point me to factual or logical errors in my comments.

Unfortunately, when I see such an accumulation of potential misunderstanding emerging but feel not up to commenting, I sometimes become impatient and comment nevertheless. Apologies if that results sometimes in muddled posts and multiple editing.

Thanks - and no worries: the misunderstandings will work themselves out. It's an inevitable process when someone from a different perspective arrives. The misunderstandings should be sifted from the genuine disagreements before anything is finalized - but also trying to ensure that this doesn't stretch out to a very long time.

 

[Hilda Bastian]

(Continued from my previous post.)

So what I basically wanted to say or ask is:

1. It's good that you're here. I see that the language we use regarding blinding and subjective and (potential) objective outcomes/ endpoints etc. could be more precise to avoid misunderstandings. From the forum discussion I also gained some new insights regarding these issues and that I used some terms in a misleading or maybe even erroneous way.

However, I don't know if I will be able to take part in the discussion in the future, and anyway, I'm not one of the forum's experts with regard to trial design and the methodology of systematic reviews and relevant related issued. But maybe I'm not alone in genuinely wanting to understand the point you made here in response to Jonathan Edwards:



I think you referred to Jo's sentence "The use of subjective outcomes (alone) from unblinded trials to judge the usefulness of a treatment is valueless".

In my opinion, it would be extremely helpful to work with this sentence and to know with which amendments of this sentence you could agree.(*)

2. Many forum members never got genuine responses to their criticism of the Cochrane reviews on exercise and Cochrane's review methodology in general. Most of this criticism is posted and being discussed on other threads.

In addition, many forum members, due to their illness and related personal circumstances, aren't on the forum regularly, a couple of members have to refrain completely from posting for an unforeseeable period of time. Also, I can't omit to once again mention the late Robert Courtney. Other knowledgeable forum members recovered or found out that they were misdiagnosed and understandably now are busy with other things. So, most people who contributed to the discussion before the IAG was constituted, won't be able to engage with the IAG.

So I'd like to ask you: Are you interested in reading what forum members think is the most relevant of the criticism posted and discussed on S4ME and would you like to comment here on some points you find crucial?

(I'm not a staff member of S4ME and only speak for myself.)

(*) My suggestion would start with something like that -- but please ignore if it's not helpful:

An unblinded or single-blinded trial that uses only subjective outcomes to judge the usefulness of a treatment has a high risk of bias that cannot be sufficiently controlled.

Unblinded or single-blinded trials therefore should include adequate prespecified objective outcomes.

What constitutes an adequate objective outcome depends on diverse factors. To determine these factors, patients' perspectives should be taken into account.

Thanks, again! Point 1: no, it was a different statement by Jonathan that I discussed, and then a version of it from someone else who dropped the "primary" endpoint part. I think I've understood what the points were that people were making - although for some of them it took a while (like realizing that some thought systematic reviews could only ever be of primary endpoints of trials, as though systematic reviews were themselves hypothesis testing as trials are). I'll note this, and will come back at a later point and follow up on your suggestion of a list of points, carefully expressed - great idea!

Your point 2:

Many forum members never got genuine responses to their criticism of the Cochrane reviews on exercise and Cochrane's review methodology in general. Most of this criticism is posted and being discussed on other threads.

In addition, many forum members, due to their illness and related personal circumstances, aren't on the forum regularly, a couple of members have to refrain completely from posting for an unforeseeable period of time. Also, I can't omit to once again mention the late Robert Courtney. Other knowledgeable forum members recovered or found out that they were misdiagnosed and understandably now are busy with other things. So, most people who contributed to the discussion before the IAG was constituted, won't be able to engage with the IAG.

I agree there never was a proper response to Robert Courtney and others, and I find that egregious (and I've communicated my anger about that very clearly). The issues you raise here are why my starting point is gathering the published critiques, and beginning the IAG's work with a proper discussion of the issues raised in them. My plan is to have a public draft for discussion to see if we have missed or misinterpreted anything, and then we can build on that. It's one way to make sure people's contributions to date won't be lost, with proper follow-through and acknowledgment.

 

[Trish]

I took a day off from the forum and find this thread has raced ahead so fast I don't have the energy to read all the posts, so please forgive me if I repeat things already said.

It occured to me that the thread was diverted into discussions of epidurals and incontinence and whether outcome measures in them were objective or not, but these proved rather fruitless because of the specifics of those conditions, so I'd prefer we park those examples as not particularly helpful to this particular discussion.

In the asthma trial mentioned earlier, patients thought their dummy inhaler and sham acupuncture improved their breathing, but objective measurement showed they did not. This is a clear demonstration that a subjective measure gave a different outcome from an objective one, even when blinded, so subjective measures need to be treated with extra caution, and on their own cannot be relied on as measures of improvement in function, only as measures of improvement in perceived function - two very different things.

I hope we can all agree that perceived function and actual function are different enough to need to be distinguished - in the asthma case, leaving patients in danger of worsening to a dangerous level without realising in time if they take a medication that is ineffective but that they 'perceive' to be effective.

In the ME case, we can 'perceive' a treatment to be effective for a while, some people apparently getting significantly better following things like LP or CBT, but after a few days, weeks, or even months, crashing badly to a state as bad or worse than when they started.

If a trial is set up to find out whether telling people to increase their exercise in regular increments regardless of symptoms, which is what GET is set up to do, then we have several possible problems:

1. Inappropriate cohort selection: Use of old diagnostic criteria that don't include the requirement of PEM - and genuine PEM not post exertional fatigue that can happen in deconditioned individuals.

2. Non-compliance: Patients quickly realising that the increases in activity make them sicker, and either pretending to comply while not actually doing so, or cutting back other activities in order to manage the prescribed daily walk of increasing duration, or getting into a push-crash cycle trying to comply but being stopped by PEM and restarting.

3. Too short duration of trial: GET usually starts with cutting back activity to find a 'base level' less than the patient's energy ceiling. The following incremental increases may not reach that ceiling over the time span of the trial. Or the patient may be able to sustain pushing a bit beyond their ceiling for the duration, but crashes badly later, and this isn't recorded.

Cohort selection should be a key factor in whether trials are included in the review. I think only those trials specifically set up using PEM inclusive diagnosis should be included, or alternatively the review should be relabelled as a review of exercise therapy of idiopathic chronic fatigue, not applicable to ME/CFS.

Non compliance and too short duration mean GET hasn't actually been properly tested. Therapist effect, placebo effect, and persuasion to interpret symptoms as part of 'normal' reactions to exercise in deconditioned individuals are all likely to skew significantly perceived improvement in symptoms and function that is not confirmed by objective measures of fitness, stamina and ability to function.

I conclude from the above that subjective outcome measures in ME/CFS trials of exercise therapy are worse than useless. They give a false impression of compliance, and only measure perceived improvement, not actual improvement in function and symptoms.

Regardless of whether subjective outcomes are ever valid measures in any unblinded trials of other conditions, they are clearly not valid in ME/CFS exercise trials.

I would therefore hope that the IAG will make it absolutely clear that the primary outcome measures used in the Cochrane review of exercise therapy must be measures that come as near as possible to objective, and trials that have purely subjective primary outcome measures should not be included. Nor should trials using a patient cohort that is defined by the Oxford, Fukuda or any other diagnostic criteria where PEM is optional.

The question of whether secondary outcome measures can be cherry picked to show something useful is fraught with problems, not least the question of why the triallists defined them as secondary rather than primary outcomes. To me that shows they have an agenda and a belief about the nature of ME.

If this means there are no trials to include, then that could be stated as an empty review with full explanation given of the lack of validity of trials that don't meet these criteria.