Since this thread has generated more interest than I expected for an only mildly promising retrospective clinic review, I decided to look more closely at the article, and have come up with some questions:
1. Is it a legitimate approach to clinical data to divide patients retrospectively into those who have 'improved' and those who haven't - and to base conclusions on this?
2. Is it legitimate in reporting on outcomes to include those who have only had a very short time on the treatment so far, so it's too early to tell whether the effects are lasting?
3. Is it legitimate to only include those who returned for a second visit. What about all those who only attended once, found the treatment ineffective or had too many side effects, and never went back. This possiblilty, not addressed, could significantly impact the numbers reporting lack of effect and/or side effects, and is simply unknown.
4. Is it legitimate to class as 'improvement' tiny increments on a ten point subjective scale? Surely they need to decide what is clinically significant improvement, and only include those as 'improved'.
5. Is it legitimate to include changes in symptoms that might have been affected by other changes made to treatment or management at the same time? For example, sleep - it seems likely in the clinic visit sleep meds will have been reviewed, and possibly new medications prescribed at the same time as the drug being assessed here. Similarly it seems likely that lifestyle management advice will be given, including pacing to reduce frequency of PEM. And both these factors would also have an impact on the other 2 symptoms being rated - fatigue and cognitive function, both of which are likely to improve with better sleep and pacing. There is no mention in the article of any assessment of such possible confounding factors.
6. Is it legitimate in a study of this kind to report numbers who suffered side effects or adverse events during, and possibly after, taking the drug, if no attempt is made to follow up patients who did not return to the clinic. There is no indication in the article that any attempt was made to address this, even with a simple phone call to ask the patient.
7. Is frequency of episodes of PEM a legitimate test of illness severity? My experience is that as I have got sicker I have been forced to change my life in a way that actually means I am more able to pace and avoid crashes. So for me, frequency of PEM is inversely related to severity of illness.
I ask all these question because it seems to me that even for a retrospective review of clinic data, the person doing the review makes choices about how to analyse the data that can significantly skew the outcomes reported.
What if they had analysed the data differently?
For example:
1. When assessing effectiveness, only assess as effective if the change cannot be explained by other changes in medication or daily symptom management. So omit patients whose improvements can be attributed to concurrent changes in sleep medication and/or pacing.
2. When asssessing effectiveness, only include as effective if the reported improvement is clinically significant (eg 2 or more of the listed symptoms improve by 2 or more points on the 10 point scale, and the patient reports that they have been able to significantly increase their cognitive and/or physical activity)
3. When assessing effectiveness, only include those whose improvement has been sustained for a given time, say 6 months.
4. When assessing worsening, include those have adverse events, whose symptoms get significantly worse, or who give up because of side effects, including tracking down those who dropped out.
Note: I'm as keen as anyone to find a treatment that works for a significant number of pwME. I just what honest reporting of reality, not sugar coating.