Multidisciplinary rehabilitation treatment is not effective for ME/CFS: A review of the FatiGo trial, 2018, Vink & Vink-Niese

[Esther12]

Thanks for those notes. I've still not found time to read this (I'm falling ever further behind with Tuller's blogs too). Thanks to Vink & Vink-Niese too.

Patients were selected between December 2008 and January 2011. Therapy lasted 6 months. The protocol was submitted on 17 March 2011 (accepted on the 16 April 2012 and published on the 30 May 2012 (Vos-Vromans et al., 2012) even though ‘a fundamental principle in the design of randomized trials involves setting out in advance the endpoints that will be assessed in the trial, as failure to prespecify endpoints can introduce bias into a trial and creates opportunities for manipulation’ (Evans, 2007). Therefore a protocol should be published before the start of a trial and not when it (in FatiGo’s case) has (almost) finished.

That point reminded me that it was explicitly marked as a retrospectively registered trial on ISRCTN:

http://www.isrctn.com/ISRCTN77567702

 

[Dolphin]

Analysis of the activity monitor results shows that patients’ physical activity level had objectively improved by 5.8 (MRT) and 6.5 per cent (CBT) at 52 weeks respectively. The subjective fatigue scores had improved by 33.4 (MRT) and 21.5 per cent (CBT) (Vos-Vromans et al., 2016a). There is an inverse relation between fatigue and activity (Rongen-van Dartel et al., 2014). The more tired you are the less active you become and when your tiredness decreases your activity level will increase. Therefore the percentage of subjective decrease of fatigue should be the same or similar to the increase in activity. The activity monitor results however show that this wasn’t the case.

 

[Dolphin]

The aforementioned PACE trial, which used an adaptive pacing therapy (APT) and also a specialist medical care (SMC) control group, showed that after treatment there were no clinically significant differences according to the step test and the six minute walk test between the 4 groups in the study (CBT, GET, SMC and APT). According to the 6 minute walk test patients in all four groups would have still been ill enough to be on the waiting list for a lung transplant (Vink, 2016; White et al., 2011). The number of patients that were able to work had decreased and the number of patients receiving illness and disability benefits had increased. Also there was a 100 per cent increase in the proportion of participants in receipt of income protection or private pensions in the CBT and GET groups.

Just to point out in the PACE trial, there was a statistically significant improvement in the six minute walk test for the graded exercise therapy group although it could be argued that it was not clinically significant.

 

[Dolphin]

Dropout rate

Even though only 14.8 per cent (18/122) dropped out, at 52 weeks, activity monitor results were not available for 34.4 per cent (42/122) of participants. Patients who drop out of therapy are not a random sub-sample of all clients. Those who do not improve or suffer adverse reactions are the ones most likely to drop out of treatment. Yet many researchers and studies do not take this into account, and as a result ‘may conclude erroneously that their treatments are effective merely because their remaining clients are those that have improved’ (Lilienfeld et al., 2014).

So activity monitor results not available for a significant number.

 

[Dolphin]

The economic evaluation In its economic evaluation, FatiGo concluded that MRT is more cost-effective for fatigue and CBT for the quality of life, if the EQ-5D-3L quality of life scores of their secondary outcome are used (Vos-Vromans et al., 2017). Yet, as previously discussed, after MRT, patients were only minimally better than the severely fatigued. A study by Olesen et al. (2016), consisting of 20,220 adult patients, found a mean EQ-5D-3L quality of life score of 0.84 for the total population and 0.93 for people without a chronic condition. The mean EQ-5D-3L quality of life scores in FatiGo after CBT (0.61) and MRT (0.69) were still worse than in stroke (0.71), ischaemic heart disease (0.72) or colon cancer (0.74) (higher scores indicating a better quality of life) (Hvidberg et al., 2015). Moreover, a score of 0.69 (MRT) equals that of people with four chronic health conditions and a score of 0.61 (CBT) is almost the same as the score (0.60) for people with five or more chronic health conditions (Olesen et al., 2016). This confirms that neither MRT nor CBT are effective and ineffective treatments cannot be cost-effective.

This puts the results in some context. However I don't accept it proves that these treatments are not cost-effective. No cost effectiveness calculations were proffered in the paper. If a drug got these results, it could be useful and cost-effective.

 

[Dolphin]

Discussion

The FatiGo trial concluded that MRT is more effective for CFS/ME in the long term than CBT (Vos-Vromans et al., 2016a). It also concluded that MRT is more cost-effective for fatigue and CBT for the quality of life (Vos-Vromans et al., 2017). However, analysis of the study shows that it suffered from a number of serious methodological flaws. The unblinded trial used two subjective primary outcomes (fatigue and quality of life). This combination is known to lead to the erroneous inference of efficacy in its absence. The likelihood of this was made even bigger because of the large difference in contact hours between the two groups (44.5 vs 16) even though these should be the same. The only way to correct for these problems in unblinded trials is by using well-designed control groups and objective primary outcomes (Edwards, 2017; Lilienfeld et al., 2014).

Seems fair.

 

[Dolphin]

Another fundamental design flaw of the trial was that it compared CBT against MRT which was CBT plus a number of things. However these were not properly specified, as they were tailored to the individual needs. Yet, in a properly designed trial, patients in a treatment group should all receive the same treatment.

While it is not ideal, I'm not sure if it means the trial is not properly designed.

Furthermore, the trial did not have a ‘placebo’ control group (for example relaxation therapy, specialist medical care or pacing) to correct for the placebo effect and other confounding factors.

Another group could have been useful even a no-therapy group so one could see what natural improvements might have occurred over the time period.

 

[Dolphin]

Furthermore, the fatigue scores showed that neither MRT nor CBT were effective

Well, there were improvements. Just because neither group had normal levels doesn't mean they were necessarily not effective.

and the mean EQ-5D-3L quality of life scores after CBT and MRT were the same as for people with five or more (CBT) or four chronic health conditions (MRT) (Olesen et al., 2016). Moreover, quality of life was still worse than in stroke, ischaemic heart disease, colon cancer (Hvidberg et al., 2015), the total population or in people without a chronic condition (Olesen et al., 2016).

An interesting comparison

 

[Dolphin]

Also FatiGo ignored the results of the activity monitor, its only objective outcome measure. Analysis of these results showed that CBT and MRT at best only led to a minimal objective improvement of 5–6 per cent. The trial suffered from many methodological problems as discussed before. For example, post-exertional malaise, the cardinal feature of the disease was not compulsory for diagnosis.

Seems fair

 

[Dolphin]

Patients with co-morbid depression or anxiety were not excluded from the study even though that has been recommended by an international group of experts including the main proponents of the biopsychosocial model in 2003. It was recommended by consensus because ‘the presence of a medical or psychiatric condition that may explain the chronic fatigue state excludes the classification as CFS in research studies because overlapping pathophysiology may confound findings specific to CFS’ (Reeves et al., 2003).

Lots of CFS studies include some patients who also have depression or anxiety. I don't see it as a full exclusion.

 

[Dolphin]

Moreover, other trials that used a ‘placebo’ control group and objective outcomes did not show any objective or clinical significant improvement (Vink, 2016; White et al., 2011).

Just to point out again that there was a statistically significant improvement on the six minute walking test in the PACE trial (White et al 2011) though it could be argued that improvement wasn't clinically significant.

 

[Sly Saint]

 

[Trish]

I came across the trial this paper is critiquing recently and want to highlight the lack of clinically significant improvement in objectively measured physical activity which the authors show in a table but make no comment on.

This is the trial:
Multidisciplinary rehabilitation treatment versus cognitive behavioural therapy for patients with chronic fatigue syndrome: a randomized controlled trial
Vos-Vromans et al.

Vink and Vink Niese do a detailed analysis of the data from the electronic activity monitors, and this is the bit that stood out for me, not just because Vos-Vromans report the results but don't include them in their conclusions, but just how feeble the results were. I've added in the cost in Euros per person from their economic analysis paper:

Treatment ...................Multidisciplinary rehab ............... CBT

Treatment Cost/person 8,989.06..................... 3,308.43
Societal cost/person 14,307.95 ..................... 8,845.71

Activity meter readings:
Baseline 206233.65 (40264.16).......................... 202033.66 (43379.41)

26 weeks 227283.24 (45698.55).......................210019.75(48068.09)
change from baseline ... up 10.2% ......................... up 3.9%
52 weeks 218214.41 (48564.30)........... .............215262.14 (57074.22)
change from baseline... up 5.8% ........................... up 6.5%
change from 26 weeks .. down 4.0% ....................... up 2.5%
__________________

Given that most definitions of ME/CFS require a level of fatigue that restricts daily activity to less than half what they were doing before, and given the average steps/day for women in the UK is around 5000, that would mean the trial participants are likely to be on less than 2500 steps/day. Given that they are able to attend lots of treatment sessions, they are probably at the upper end of ME/CFS functioning, so say they average 2000 steps/day at the start.
If we assume the changes in activity levels measured are a reasonable proxy for steps/day, what would this mean?

For someone who did the rehab:
They start on 2000 steps/day.
After 6 months 2204 steps/day,
At 12 months 2116 steps/day

For someone who did CBT:
They start on 2000 steps/day.
After 6 months, 2078 steps/day
After 12 months, 2139 steps/day.

So the net result from either treatment is that someone walking 2000 steps per day before treatment is now walking about 100 steps a day more. I think we can all agree that's nowhere near clinically significant. No wonder they don't even mention, let alone discuss or draw conclusions from this objective data in their papers.

But it gets worse, as Vink and Vink-Niese point out. A significant number of participants didn't wear the actometers at 6 months, and even less at 12 months, some of them saying it was because they couldn't travel to the clinic to collect and return the actometer. That seems to me such a huge red flag it makes even this data likely to be an over inflated result.
They presumably used the data from the whole cohort to calculate the average activity level at the start, and then only those who wore the actometer at the later stages, so we have no idea what the starting points were for those giving data at the end. What if, as seems likely, those who wore the actometers at the end where from the upper end of the activity levels at the start. So their average at the start may even have been higher than at the end.

To quote Vink and Vink-Niese

Even though only 14.8 per cent (18/122) dropped out, at 52 weeks, activity monitor results were not available for 34.4 per cent (42/122) of participants. Patients who drop out of therapy are not a random sub-sample of all clients. Those who do not improve or suffer adverse reactions are the ones most likely to drop out of treatment. Yet many researchers and studies do not take this into account, and as a result ‘may conclude erroneously that their treatments are effective merely because their remaining clients are those that have improved’ (Lilienfeld et al., 2014).

Yes, I know Vink and Vink Niese have drawn attention to all this already, and far more eloquently and completely than I can, but I have seen recently the subjective data from this trial being used to justify these treatments, so it's not over yet.

That's got that off my chest!

 

[Mark Vink]

Thank you, that trial is a typical example of labelling ineffective treatment as effective and ignoring your own results. Or to put a differently, it is a typical example of opinion based medicine.

I came across the trial this paper is critiquing recently and want to highlight the lack of clinically significant improvement in objectively measured physical activity which the authors show in a table but make no comment on.

This is the trial and its abstract:
Multidisciplinary rehabilitation treatment versus cognitive behavioural therapy for patients with chronic fatigue syndrome: a randomized controlled trial
Vos-Vromans et al.

Vink and Vink Niese do a detailed analysis of the data from the electronic activity monitors, and this is the bit that stood out for me, not just because Vos-Vromans report the results but don't include them in their conclusions, but just how feeble the results were. I've added in the cost in Euros per person from their economic analysis paper:

Treatment ...................Multidisciplinary rehab ............... CBT

Treatment Cost/person 8,989.06..................... 3,308.43
Societal cost/person 14,307.95 ..................... 8,845.71

Activity meter readings:
Baseline 206233.65 (40264.16).......................... 202033.66 (43379.41)

26 weeks 227283.24 (45698.55).......................210019.75(48068.09)
change from baseline ... up 10.2% ......................... up 3.9%
52 weeks 218214.41 (48564.30)........... .............215262.14 (57074.22)
change from baseline... up 5.8% ........................... up 6.5%
change from 26 weeks .. down 4.0% ....................... up 2.5%
__________________

Given that most definitions of ME/CFS require a level of fatigue that restricts daily activity to less than half what they were doing before, and given the average steps/day for women in the UK is around 5000, that would mean the trial participants are likely to be on less than 2500 steps/day. Given that they are able to attend lots of treatment sessions, they are probably at the upper end of ME/CFS functioning, so say they average 2000 steps/day at the start.
If we assume the changes in activity levels measured are a reasonable proxy for steps/day, what would this mean?

For someone who did the rehab:
They start on 2000 steps/day.
After 6 months 2204 steps/day,
At 12 months 2116 steps/day

For someone who did CBT:
They start on 2000 steps/day.
After 6 months, 2078 steps/day
After 12 months, 2139 steps/day.

So the net result from either treatment is that someone walking 2000 steps per day before treatment is now walking about 100 steps a day more. I think we can all agree that's nowhere near clinically significant. No wonder they don't even mention, let alone discuss or draw conclusions from this objective data in their papers.

But it gets worse, as Vink and Vink-Niese point out. A significant number of participants didn't wear the actometers at 6 months, and even less at 12 months, some of them saying it was because they couldn't travel to the clinic to collect and return the actometer. That seems to me such a huge red flag it makes even this data likely to be an over inflated result.
They presumably used the data from the whole cohort to calculate the average activity level at the start, and then only those who wore the actometer at the later stages, so we have no idea what the starting points were for those giving data at the end. What if, as seems likely, those who wore the actometers at the end where from the upper end of the activity levels at the start. So their average at the start may even have been higher than at the end.

To quote Vink and Vink-Niese


Yes, I know Vink and Vink Niese have drawn attention to all this already, and far more eloquently and completely than I can, but I have seen recently the subjective data from this trial being used to justify these treatments, so it's not over yet.

That's got that off my chest!

 

[Sean]

But it gets worse, as Vink and Vink-Niese point out. A significant number of participants didn't wear the actometers at 6 months, and even less at 12 months, some of them saying it was because they couldn't travel to the clinic to collect and return the actometer. That seems to me such a huge red flag it makes even this data likely to be an over inflated result.

Yes, those very weak numbers are the best possible case scenario. Which is an appalling fact.