COMET initiative: Core Outcome Set for Chronic Fatigue Syndrome/Myalgic Encephalomyelitis

[Andy]

Can someone explain what a "core outcome set" would look like? Are they trying to decide which metrics are most important to measure? Would it be like, thru this process we've found that the best measures of this illness are: the fatigue scale, and employment status? I mean, what is the end product supposed to be exactly?

From the homepage, https://www.comet-initiative.org/
"A core outcome set (COS) is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific areas of health or health care."
which obviously could be a sensible and useful list that helps to determine whether a treatment makes an actual difference to a patient, rather than just getting them to change their answer for something subjective. Or, and given the company he seems to keep this is the likely outcome in my opinion, it will be a confirmation of all the subjective measures that the BPSers have been using for years.

 

[Jonathan Edwards]

Can someone explain what a "core outcome set" would look like?

Yes, a core outcome set would be a set of measures that trial lists would be encouraged to collect in all trials. They might find they could not publish if they didn't. So it might be fatigue score, six minute walking distance, work status etc.

It is not quite the same as a standardised composite outcome scale - like the ACR criteria of improvement in rheumatoid or the EULAR grading for rheumatoid, which make use of 'core data' in a specific composite way.

It may be quite interesting that Busse is looking at this. If you set up a core outcome set then you have to have reasons for not including something obvious - like actimetry maybe.

 

[ME/CFS Skeptic]

Busse has a background in insurance medicine. He previously worked as a consultant to Prisma Health Canada, which is a private incorporated company funded by employers and insurers that consults on and manages long-term disability claims. He mentioned this as a COI in previous papers, for example here: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0049760&type=printable

 

[dave30th]

Yes, a core outcome set would be a set of measures that trial lists would be encouraged to collect in all trials. They might find they could not publish if they didn't. So it might be fatigue score, six minute walking distance, work status etc.

I get this of course. But I ask because I'm really perplexed--it just seems like such a stupid thing in this context for this illness by people who really seem to know nothing about it, or about core outcomes, or about anything, and have no record of publishing on it except to complain about NICE.

I guess my question is, has someone seen this as a big hole in the current literature? The CBT/GET ideologues staked out their core outcomes--fatigue and physical function--decades ago. Do these people think they're now going to reaffirm those as the "core outrcomes," or what?

 

[Hutan]

I think it is a hole in the current approach to ME/CFS research - done well, a set of agreed outcome measures would be great e.g. a good subjective fatigue measure, actimetry measures (e.g. time with feet on the ground, daily steps), some subjective global health measure, maybe a cognition measure... with requirements about how long the actimetry is done for, not just a week or even a month, requirements for followup to check that the answers on subjective surveys weren't just an expression of hope/gratitude/politeness immediately after an intervention. Currently, even good researchers use different surveys, making comparisons difficult, and can get actimetry outcomes wrong. Things like the 6 minute walk test are too prone to varying motivation.

But yes, it looks like some BPS/insurance people have latched onto that Metablind study and want to set in concrete their existing approach of just subjective surveys, probably including the CFQ and HADS, and sweep any inconvenient criticism away.

 

[Esther12]

we will use the Delphi process involving patient partners and clinicians in this area

https://www.comet-initiative.org/Studies/Details/2103

It would be nice to know more details about all this.

 

[Trish]

The home page of the Comet initiative says this:

The COMET Initiative brings together people interested in the development and application of agreed standardised sets of outcomes, known as ‘core outcome sets’ (COS). These sets represent the minimum that should be measured and reported in all clinical trials of a specific condition, but COS are also suitable for use in routine care, clinical audit and research other than randomised trials. You can read the core outcome set/COMET plain language summary here. The existence or use of a core outcome set does not imply that outcomes in a particular study should be restricted to those in the relevant core outcome set. Rather, there is an expectation that the core outcomes will be collected and reported, making it easier for the results of studies to be compared, contrasted and combined as appropriate; while researchers continue to explore other outcomes as well. COMET aims to collate and stimulate relevant resources, both applied and methodological, to facilitate exchange of ideas and information, and to foster methodological research in this area.

• 
  
  
• 
  
  
• 
  
  
• 

Reading more of their website:
It was set up from a meeting in Liverpool UK in 2010 and expanded to be more international.

It seems to me to be a bit like Cochrane in that people who write the Cochrane reviews, and people who do these core outcome sets for COMET, seem to be self selected and free to insert their own biases into the process.

 

[Snow Leopard]

This is exactly the wrong way to go about it.

I am hoping that we might be able to do something better but we will need some input from intelligent ME clinicians.

I suggest we start an alternative, else we get stuck with the above.

What will it take to get 'something better' off the ground?

I know many of us were interested in the idea of composite outcome measures that also explicitly consider baseline measures, rather than pretending that some numerical improvement on a scale has the same outcomes for different patients regardless of their underling level of heath.

 

[MSEsperanza]

I suggest we start an alternative, else we get stuck with the above.

I think that would be very worthwhile if we had some people left with the resources to be involved.

I imagine the Cochrane IAG discuss this topic as well and perhaps Busse et al want to be quicker than the IAG / influence the Cochrane people involved.

As shown in the other thread, the Busse / GRADE team has strong links to Cochrane and overlaps, respectively.

Another detail from Busse's CV:

[...]
He is a Director of the Cochrane Insurance Medicine Field Group (http://insuremed.cochrane.org/our-team).

And https://www.s4me.info/threads/bmj-r...-grade-working-group.19317/page-8#post-328665

No idea if there's any reason to still be hopeful that the IAG will open up the discussion in any meaningful / effective way so that people on S4ME, their critique and suggestions made here on the forum as well as in academic papers and other publications could participate.

There still seem to be people at Cochrane who have the courage to publicly criticize the endeavor to give up reasonable methodological standards -- see e.g. Andrew Moore (*) -- could there be some dissent and prudence at Cochrane that we just aren't aware of?


(*) Andrew Moore on Twitter @Healthy_Control @cochranecollab @susan_bewley .:

"Yes, they have been particularly resistant to experience and knowledge originating from outside the centre of Cochrane: "There's no knowledge but I know it". It was a built in fault as they had no effective quality control. @metanutter said it - adjust RoB etc depending......" / Twitter

Thank you to @Caroline Struthers for your perseverance in getting people involved to question bad science / research practices with regard to ME/CFS.

And Andrew Moore, if you happen to read this, thank you for speaking out.


(Edited to add links.)

 

[MSEsperanza]

The authors of the MetaBlind study had a more cautious interpretation of their findings than Busse. They wrote:

"We are unclear to what extent our results show that blinding is less important than previously believed, show the limitations of the meta-epidemiological approach (eg, residual confounding), or show a lack of precision in the comparisons made."​

Yes, Busse's tweet is an extreme misrepresentation of the findings of the Metablind study.

I wonder if the authors should be informed about that.

Haven't read Moustgaard's et al follow-up paper [*] but already in response to the reaction on their bmj paper they repeatedly warned against misinterpreting their findings.

E.g., they acknowledged Andrew Moore's critique:

"As discussed in our paper (1), and our follow-up paper (2), there are strong theoretical reasons to expect bias in non-blinded trials, backed by previous empirical studies (see our general comment ‘MetaBLIND in context’). These reasons may be particularly convincing in the case of pain trials, where lack of blinding of patients in particular would be expected to have a large impact.

"Andrew Moore cites empirical evidence to back this, in the form of comparisons of effect estimates from blinded and non-blinded trials of acupuncture for pain. We agree that there is especially strong empirical evidence to support blinding of patients in acupuncture pain trials. A systematic review of 12 trials that directly randomized patients to otherwise identical blind and nonblind sub-trials, found 10 acupuncture sub-trials, and reported substantially exaggerated effects sizes (3).

"In our study, we aimed to evaluate empirically to what extent lack of blinding impacts on effect estimates, for trials in general. We found no evidence for an impact of lack of blinding, on average. [...]

"What may explain the discrepancy in results between the MetaBLIND study and previous empirical research, for example (3) and (4), is not clear, though we note that we included no trials of acupuncture for pain in MetaBLIND."

https://www.bmj.com/content/368/bmj.l6802/rr-8

And another response by Moustgaard et al:

Moderation of risk of bias criteria for randomized trials is not warranted based on the MetaBLIND study

https://www.bmj.com/content/368/bmj.l6802/rr-7

"We interpret our results with caution, and find that moderation of the risk of bias criteria regarding blinding, based on our results, would be premature. We discuss this in our main paper (1) and in our follow-up publication (2). Replication of our study, as well as other further research is warranted to elucidate the role of blinding in clinical trials. [...]"



As others have pointed out, there's still more to criticize. It's a pity that the authors didn't reply to Michiel Tack's detailed critique, published in the Journal of Health Psychology:

Tack M. Problems with the MetaBLIND study: An examination of data on blinding patients in trials with patient-reported outcomes. Journal of Health Psychology. 2021;0(0). doi:10.1177/13591053211059391
Researchgate: https://www.researchgate.net/public...ents_in_trials_with_patient-reported_outcomes



[*] Moustgaard H, Jones HE, Savović J, Clayton GL, Sterne JA, Higgins JP, Hróbjartsson A. Ten questions to consider when interpreting results of a meta-epidemiological study-the MetaBLIND study as a case. Res Synth Methods. 2020 Mar;11(2):260-274. doi: 10.1002/jrsm.1392. Epub 2020 Jan 20. PMID: 31851427.
https://www.researchgate.net/public...logical_study_-_the_MetaBLIND_study_as_a_case


(Edited to add researchgate links)

 

[Jonathan Edwards]

We seem to have two threads on this - one from @Snow Leopard on a poll. I have posted there but this may be the right place for COMET specific comments.

The fact that COMET is UK derived - maybe Liverpool derived - is another telltale sign (@dave30th ) that this is a deliberate counter-attack by people who think they know a lot about ME and want to reinforce their ramparts against activity snipers. This looks like McMaster trying to bolster their reputation in the woo-woo field of therapist delivered treatments under guise of quality control. The Canadian equivalent of Glasziou I guess.

 

[Jonathan Edwards]

I suggest we start an alternative, else we get stuck with the above.

What will it take to get 'something better' off the ground?

I have mentioned this on the poll thread. It will take a grant application and a fair wind on the politics side. That might mean making sure people like Stephen Holgate are informed and in favour. Some of the potential funding bodies include people who might be resistant but if a good professional mathematician is included in a grant proposal it might get through even at MRC level. It need not be a huge grant in cost terms. What matters is getting a peer reviewed approved project and some good people prepared to devote time. That will have to include someone who has skills in grant writing, which is not me.

 

[Dolphin]

 

[MSEsperanza]

Thanks for posting -- both to Scott and Dolphin.

Transcript:

Heads up! A threat to our community: A known psychologizer of #MEcfs and #LongCovid with connections to the insurance industry is approaching patient organizations to support their 'research' project known as "Core Outcome Set for CFS/ME" or "Comet-initiative".

Have there been any replies?

(Can't see, don't have Twitter/X account)