Review Effectiveness of non-pharmacological interventions for fibromyalgia and quality of review methods: an overview of Cochrane Reviews, 2023 Bidonde et al

[Andy]

Abstract

Background
Fibromyalgia syndrome (FMS) is defined as chronic widespread pain associated with sleep disorders, cognitive dysfunction, and somatic symptoms present for at least three months and cannot be better explained by another diagnosis.

Objectives
To examine efficacy and safety of non-pharmacological interventions for FMS in adults reported in Cochrane Reviews, and reporting quality of reviews.

Methods
Systematic reviews of randomised controlled trials (RCTs) of non-pharmacological interventions for FMS were identified from the Cochrane Database of Systematic Reviews (CDSR 2022, Issue 3 and CDSR 2023 Issue 6). Methodological quality was assessed using the AMSTAR-2 tool and a set of methodological criteria critical for analgesic effects. The primary efficacy outcomes of interest were clinically relevant pain relief, improvement in health-related quality of life (HRQoL), acceptability, safety, and reduction of mobility difficulties as reported by study participants. No pooled analyses were planned. We assumed a clinically relevant improvement was a minimal clinically important difference (MCID) between interventions and controls of 15%, or a SMD of more than 0.2, or a MD of more than 0.5, on a 0 to 10 scale.

Results
Ten Cochrane reviews were eligible, reporting 181 randomized or quasi- randomized trials (11,917 participants, average trial size 66 participants). The reviews examined exercise training, acupuncture, transcutaneous electrical nerve stimulation, and psychological therapies.

One review was rated moderate according to AMSTAR 2, seven were rated low and two were rated critically low. All reviews met most of the additional methodological quality criteria. All reviews included studies with patient-reported outcomes for pain.

We found low certainty evidence of clinically relevant positive effects of aerobic and mixed exercise training and for cognitive behavioural therapies (CBTs) at reducing mobility difficulties and for mixed exercise training and CBTs for improving HRQoL at the end of the intervention. Number needed to treat for an additional beneficial outcome (NNTB) values for a MCID of 15% ranged between 4 and 9. We found low certainty evidence that was clinically relevant for mixed exercise and CBTs for reducing mobility difficulties at an average follow up of 24 weeks. We found low certainty evidence of clinically relevant positive effects of mixed exercise on HRQoL at an average follow up of 24 weeks. NNTB values for a MCID of 15% ranged from 5 to 11. The certainty of evidence of the acceptability (measured by dropouts) of the different non-pharmacological interventions ranged from very low to moderate and the dropout rate for any reason did not differ across the interventions or the controls, except for biofeedback and movement therapies. All the systematic reviews stated that the reporting of adverse events was inconsistent in the studies analysed (very low certainty evidence).

Authors' conclusions
There is low certainty evidence of clinically relevant reduction of mobility difficulties and of improvement of HRQoL among individuals with FMS by aerobic and mixed exercise training and by CBTs at the end of the intervention. There is low certainty evidence that CBTs and mixed exercise training reduces mobility difficulties post-treatment and that mixed exercise training improves HRQoL at follow-up by clinically meaningful scores.

Open access, https://www.sciencedirect.com/science/article/pii/S0049017223000902

 

[Adrian]

It would be interesting to compare the methodology and trial ratings with that of the Cochrane exercise review.

One review was rated moderate according to AMSTAR 2, seven were rated low and two were rated critically low. All reviews met most of the additional methodological quality criteria. All reviews included studies with patient-reported outcomes for pain.

It looks like they are rating Cochrane reviews themselves rather than trials and not rating them very well

@Caroline Struthers any opinions?

 

[Joan Crawford]

"The primary efficacy outcomes of interest were clinically relevant pain relief, improvement in health-related quality of life (HRQoL), acceptability, safety, and reduction of mobility difficulties as reported by study participants."

This is a bit of a guddle....

There is no evidence that pwFMS can experience pain relief. Meds at best take the edge off, that's about it. Pacing can help patients to manage their pain by pulling back from pushing through it to taking a more paced approach when they can. But there is no expectation in pain management programmes, for example, that pain relief is possible. The hint is in the title.

If the research reviewed (sorry not had time to look) is a guddle of trying to reduce pain and at the same time improving HRQoL, and increasing activity (reducing mobility issues), then this will fail. It is just not possible to achieve these aims with medical and non-pharma methods currently.

At best patient can pace (often by reviewing goals, delegating / eliminating tasks, doing things in different ways, e.g., using mobility scooters to get out and about and so on...). It is often about identifying that they are perhaps pressuring themselves to do what they did before (especially on good days), flaring up pain in an unhelpful cycle, to stop and think, and adapt and adjust. The goal is adapting and adjusting to living well despite chronic pain - which is what no one wants. Pts want to be pain free. That option does not exist currently.

Ideally, this is a discussion can occurs before referral into chronic pain services. Ideally, there is honesty and expectation management. By overselling the BPS approach as a possible cure - i.e. 'GET for FMS' / 'CBT to think your way to less pain' etc then this is wholly misleading as there is no objective evidence I'm aware of that doing more activity with FMS = less pain in pwFMS. Quite the opposite. Keeping gently active whilst being able to sustain some small scale strengthening exercises along with making choices and assertively pushing back on soaicalital nonsense, i.e., society doesn't like ambiguity and witnessing ongoing suffering, so it tends to pretend it ain't happening. And so on.

No surprise is there is a guddle in thinking there will be low quality evidence. If GET for FMS worked then for sure this would be straightforward to objectively measure. It's not been done I don't think. If anything, I think the objective measures would show reduction in activity - or at least intensity of activity.

Some brief measures like 6 min walk test, or sit to stands in 1 minute are often used - and these can be increased after pain management - but pwFMS's functioning is nowhere like age, sex matched healthy people without FMS.

 

[Caroline Struthers]

It would be interesting to compare the methodology and trial ratings with that of the Cochrane exercise review.

It looks like they are rating Cochrane reviews themselves rather than trials and not rating them very well

@Caroline Struthers any opinions?

I have only skimmed this, but only one of the 10 Cochrane reviews was rated moderate quality, the rest low, or critically low quality (go Cochrane!). I don't understand then how they can draw any conclusions at all from an overview of 10 mainly useless reviews. What do they mean when they say all reviews met most of the "additional methodological quality criteria". What are these additional criteria, and why does meeting "most" of them justify ignoring the AMSTAR 2 rating?

 

[Hutan]

BPS approaches are getting a bit of a blitzing lately, with that spinal pain paper finding many flawed studies, that affected the findings of reviews and guidelines that based recommendations on them.

Unfortunately, RCTs included in reviews are often (predominantly) of high risk of bias [3,4]. These factors, alone and especially together, suggest a high degree of caution is important when assessing the impact of interventions for a condition as complex and difficult as FMS.

Most primary studies of people with FMS have reported the outcomes (e.g., pain intensity, HRQoL) as mean values. However, the standards used to assess evidence in chronic pain trials have changed substantially in the last 10 years. The most important change is the move from using average pain scores, or average change in pain scores, to the number of people who have a substantial (by at least 50%) or a moderate (by at least 30%) decrease in pain and who continue in treatment, ideally in trials of 8 to 12 weeks or longer.

The protocol was developed in collaboration with Cochrane Neuromuscular Diseasesand Cochrane Musculoskeletal groups

Cochrane seems to be part of this improvement.

Systematic review quality assessment: AMSTAR-2
Shea BJ, Reeves BC, Wells GA, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017; 358: j4008.

Additional validity check elements

• Did the review use defined diagnostic criteria for fibromyalgia?

• Did the review include only studies in which patients made their own assessment of pain? Professionals' and patients' assessment of pain often differ, with professionals significantly underestimating patients’ pain[28].

• Did the review use studies with defined minimum pain intensity of moderate or severe pain as an inclusion criterion?

• Did the review examine study size as a confounding factor in any analysis of efficacy? Systematic reviews have been criticised for being overconfident about results with inadequate data[29]; there is increasing evidence of the importance of small trial size, both because of random chance and as an important source of bias[30]

• Did the review examine susceptibility to publication bias?

• Did the review examine or comment upon imputation methods for missing data as a potential source of bias?Did the review analyse the inclusion and exclusion criteria of the studies and discuss the utility of the study results to people with FMS in routine clinical care?

• Did the review perform subgroup analysis according to the type of control group?

All reviews met five or more of nine additional methodological criteria (see Appendix D). All reviews stated that they included studies with FMS established according to diagnostic criteria but six did not explicitly mention the diagnostic used [40], [41], [42], [43], [44], [45]. All reviews included studies with patient-reported outcomes of pain although this was not a predefined criterion for inclusion in the individual reviews, but no review required for inclusion that studies had established pain intensity at baseline should be at least moderate or reported participants' pain intensity at baseline. All reviews used predefined criteria of a minimal clinically important difference (MCID) between study groups (15%) for outcomes.

 

[Hutan]

Lots of interesting points in the discussion- here are some:

Cochrane reviews have a special place in the assessment of the clinical efficacy of interventions, because they concentrate on methodological issues designed to reduce the impact of bias. For pain-related topics, most forms of bias tend to increase, often significantly, estimates of treatment effects. There is good evidence for this for trials that are not randomised, not blind, that are small, and where imputation methods used may be inappropriate [51], [52], [53], [54], [55]. But significant treatment effects can also be hidden by inappropriate methods, for example, by trying to estimate efficacy where baseline pain is low [56] or different analytical strategies [57]. Inadequate methods used in many systematic reviews in pain and anaesthesia limit confidence in their results; Cochrane reviews typically provide moderate or high confidence [3].

This overview of Cochrane reviews of non-pharmacological interventions for FMS examined 10 reviews published between 2013 and 2019. One provided medium confidence, but nine provided low or critically low confidence. The common cause was failure to meet the generic 2017 AMSTAR-2 requirement to discuss the impact of possible publication bias and/or considering risk of bias when interpreting or discussing the results. Additional methodological issues more specific to pain were met commonly, though minimum pain intensity or reported pain intensity at baseline.

Deficiencies that limit confidence probably reflect the fact that these reviews were published before the publication of the AMSTAR-2 criteria in September 2017. The methodological requirements of a high quality and trustworthy review become increasingly more stringent, as our knowledge of factors detracting from confidence increase. For example, fabrication is now known to be an important issue that can have major effects on review results, some with relevance to FMS [58], and some Cochrane review groups, and journals have begun to introduce assessments of potential fabrication.

Beyond systematic review methods, attention is increasingly focussed on the RCTs included within reviews. Most RCTs included in Cochrane reviews have high or unknown risks of bias and are usually small in size, and questions arise about how much weight can be given to assessments of efficacy based on them [4,59]. When results from small studies with methodological issues are compared with those from single large and high quality RCTs in meta-analyses, they are often found to overestimate treatment effects [60]. For example, a very large randomised trial of paracetamol (acetaminophen) for low back pain (1650 patient trial) showed it to be no more effective than placebo [61]

Because FMS is a difficult clinical condition without any dominating effective therapy, it is difficult to comment on how results of clinical research can used in clinical practice. The results of this overview suggest that caution and wisdom are needed. Whether these findings support recommendations of evidence- and consensus based guidelines that non-pharmacological therapies should be preferred in the long-term management of FMS is an interesting point beyond the scope of this paper [11,12].

The implication for future systematic reviews, both within and outside Cochrane, is that attention to a host of details is required to produce evidence of which we can be confident. Beyond that, the fact that few available RCTs are without some high or unknown risk of bias might give us pause. Do we continue to produce research of low confidence and trust, or do we find ways to conduct large, high quality RCTs that would best support patients and their professional carers?

 

[Hutan]

The authors appear to have partially understood the enormous bias that comes from subjective outcomes in unblinded trials, especially where the treatment aims to reduce the importance participants attribute to the symptoms being measured as outcomes. But when it comes to recommendations for clinical practice, they still can't quite accept the results.

The GRADE rating of the certainty of evidence for pain was low to very low in nine reviews and moderate in one review. Some Cochrane authors have argued there should be a halt on trial registration, until the quality and focus of studies with psychological therapies in chronic pain radically improve, because of a significant threat of research waste [70].

Future randomized clinical trials of non-pharmacological interventions delivered by health care professionals (e.g. physiotherapy, psychological therapies) should improve the implementation of the intervention and provide clearer reporting of blinding status, particularly of trial personnel who can be easily blinded such data managers, the data safety and monitoring committee, statisticians and conclusion makers [71].

But still, this is the conclusion for clinical practice. It's as if they can't accept that nothing helps very much:

Non-pharmacological therapies are aimed at multidimensional targets of FMS. Some types of exercise (aerobic, aquatic and mixed exercise training) and CBTs, including ACT, can have positive effects on multiple domains of symptoms (pain, fatigue, psychological distress) of some people with FMS and can reduce mobility difficulties at the end of the intervention and at follow-up up to an average of 24 weeks after the end of the interventions. These findings support the recommendations of evidence- and consensus based guidelines that non-pharmacological therapies should be preferred in the long-term management of FMS [11,12]. FMS symptoms persist in nearly all patients during their lifetimes. Different types of exercise and types of psychological techniques, such as mindfulness, acceptance and commitment and stress reduction, can be used as self-management by people with FMS after instruction by health care professionals.

Studies should collect, analyse and report responder rates (e.g.30% or greater symptom relief or 20% or greater improvement in HRQoL) according to suggested responder definitions for FMS clinical trials [2].

I think this recommendation will help, as it's quite hard for an ineffective treatment to move an individual's survey results by 30%.

We thank the Norwegian Institute of Public Health for supporting this overview open access publication.

This is the organisation that has been supporting the Lightning Process?

 

[Midnattsol]

This is the organisation that has been supporting the Lightning Process?

At least the study at NTNU. And a number of individuals from NIPH have made general comments on it being "interesting", "provides hope" and similar.

 

[Joan Crawford]

"FMS symptoms persist in nearly all patients during their lifetimes. Different types of exercise and types of psychological techniques, such as mindfulness, acceptance and commitment and stress reduction, can be used as self-management by people with FMS after instruction by health care professionals."

While these types of techniques can be useful for pwFMS (some exercises - usually very minimal, gentle stretching and strengthening exercises - not aerobic exercises / mindfulness, acceptance and commitment and stress reduction), they most certainly don't need "instruction by health care professionals." People are not daft - many work out how to manage themselves just fine. The vast majority of pwFMS self manage themselves. It is a minority that are seen in pain clinics and who take part in pain management programmes. And this can be because they have not had a good medical process, picked up blaming/unhelpful attitudes form medics and others in their life and are needing more support to know more about their condition to manage it well. That can take a minority of patients some time to figure out and ask for support with this. Many get on with minimal input, beyond diagnosis, just fine.

 

[rvallee]

So the evidence for treatments in widespread usage is bad, at best, but the language used still encourages to just keep using them on the pretense that, whatever, there's evidence for some of it, even though it's bad? With some added recognition that seems to give points to trials simply for being trials that checked some bureaucratic boxes?

Especially since the pretense that those are effective is the universal excuse to do nothing more, there are zero efforts being channeled towards developing effective treatments, since the industry has been stuck in a loop of doing the exact same stuff over and over again. Which despite the attempt at improvement, is still the case with this study and AMSTAR-2, which is still spinning the same loop of perpetual failure.

And as much as they can talk about numbers-to-treat, they cannot pretend to be naïve enough and not be aware that when patients are stuck 1:1 in a consult room with an MD, they are told explicitly that those so-called treatments are fully effective for everyone, and that this means that their "illness" is psychological, even though this explicitly cannot be inferred from pragmatic trials. It's this detachment from reality more than anything that ruins everything, how a measly 1% of some heavily biased thing is hyped into a 100% safe and effective for everyone. Basically: it's the lies, the lies!

A bar set so low you can simply roll people over it, even when they're unconscious. No wonder decades go by with zero progress. You could easily arrive at the same conclusions for homeopathy, astrological projection and any ridiculous thing. And worst of all, this doesn't bother anyone. The complete lack of deliverables in healthcare outcomes, just as the complete lack of evidence used to support them, is no one's problem. Well, I guess it's the patients', who are blamed for being failed because no one can accept it and they can get away with passing it on.

Evidence-based medicine makes dysfunctional politics look sane and very high performance by comparison, since even though there is a similar level of waste and corruption, things do get delivered: housing and roads get built, electrical power gets delivered, sewerage gets treated, and so on. While here it's absolutely nothing, complete theater with nothing at all delivered other than jobs. It's so damn weird.

 

[Hutan]

I don't disagree @rvallee, but, from that very low bar, this is progress. It's not nothing.

The GRADE rating of the certainty of evidence for pain was low to very low in nine reviews and moderate in one review. Some Cochrane authors have argued there should be a halt on trial registration, until the quality and focus of studies with psychological therapies in chronic pain radically improve, because of a significant threat of research waste [70].

Future randomized clinical trials of non-pharmacological interventions delivered by health care professionals (e.g. physiotherapy, psychological therapies) should improve the implementation of the intervention and provide clearer reporting of blinding status, particularly of trial personnel who can be easily blinded such data managers, the data safety and monitoring committee, statisticians and conclusion makers [71].

 

[Hutan]

this is progress. It's not nothing.

Just to argue with myself, I note that the reference #70 above in the quote in my post above is from a 2017 paper, which itself is quoting a 2013 paper:

From 101 RCTs, the best conclusion we can draw is that there is low-quality evidence of small to moderate effects of CBT for chronic pain, meaning that the effect estimates could easily change with new evidence.

Perhaps the next 101 trials will help us. Without change, however, we believe not. In 2013, we argued that there should be a halt on trial registration, until the quality and focus radically improve, because of a significant threat of research waste9. There should be no new trials until three critical problems are addressed.

The 2017 paper, by Christopher Ecclestone and Geert Crombez, is a muddle of useful observations like the one quoted above and all of the usual self-delusions that non-pharmacological interventions could help, if only they were done better. It's interesting to see people acknowledging the poor evidence base but struggling so hard with the 'but, but, the idea that psychological and behavioural change will heal these people must be right'.

e.g.

(1) Like surgery, psychological intervention is dependent on the skill, training, and experience of the operator; is manufactured in the moment; and is tailored to the individual case. The overall small- to moderate-effect sizes of CBT hide a heterogeneity of content, operator characteristics, exposure time, and therapist allegiance, which go largely unreported. Individual candidate measures of process are often investigated, but common features of treatment go undiscussed. Burns has argued recently that attention to common mechanisms, in particular ‘behavioural activation’—actively engaging in practicing or experimenting with meaningful physical change—is a good candidate10,11. A further example is the need to account for parental distress, and parenting, in the treatment of adolescent chronic pain12. A novel target for therapy development is in making the non-specific specific.

(2) Despite a large evidence base, there are critical gaps. Individual differences and the importance of pain presentation are rarely investigated. There is some consideration of sex differences in therapy outcomes13, of chronic pain in later life14, and of delivery in particular settings such as the workplace15. But these are rare cases. Odder still is the absence of data for specific populations. For example, we found only three RCTs of psychological interventions eligible for a Cochrane Review in chronic neuropathic pain16. Exactly how the form and content of pain shape psychological experience is largely unexplored, hidden within large compound variables such as anxiety, depression, or disability. The extent to which, for example, worry about headache is critically different from worry about pelvic pain is important, as is the history of the meaning of that pain (for example, whether it was related to previous disease). A novel target for therapy development is establishing illness-specific psychological theory.

Edit - So, it's not nothing, but it was being said in 2013, and nothing much has happened yet.

 

[rvallee]

That is definitely what's needed. I still feel it's too slow for progress, but it is some suggestion of potential future progress. The halt in trial registration would be significant, and very much needed. It's one thing to speak of the future, but there is a huge mass of this in the past that has real life power over millions of people, including some happening right now. The exact same stuff. Done the same way. With the same biases. With the same intent and rationalization that "it must be true".

I really don't think there is much they can do in the future that will make it acceptable, because the only real conclusion here is that basically none of this works, and it's too much to accept. It has been sold as a done deal for decades, and argued that it means that MUS/FND/BDD/whatever is psychological, which is simply too embarrassing. It has been fought too violently, too disrespectfully.

I don't know how much it can counter the movement in the exact opposite direction, which is still very potent. The future of this discipline is being held back by those who made a career out of this easily-gamed process, and some of them can easily nip this in the bud. Cochrane has so far sided with them, as much of their non-pharmaceutical reviews suffer from all those flaws, and then some.

Could they bear retracting much of that catalogue? When a few powerful ideologues can stop the process for the exercise review? In part by being public about going to war with them? Even when Cochrane's own conclusions are that it should happen. We already have our answer to that. They're still allowing the charade of not retracting it while excusing it on a stalled "independent" process that is radio silent, and likely doing nothing at all.

It also only speaks of chronic pain. There is so much more to it, including but not exclusive to us. The whole non-pharmaceutical evidence base is invalid, as far as I'm concerned. I think that it's obvious enough to many, but so few dare call out the naked emperor. Especially since he's been obviously naked all along and people have been imprisoned and tortured over saying so.