Harvard's "med" and "ed" is an actual disgrace

https://www.health.harvard.edu/diseases-and-conditions/understanding-chronic-fatigue-syndrome

no reviews yet

 

How educational would it be if people who were given or forced CBT/GET were to state their outcomes?

 

Yes, I appreciate that. I was more musing on the general phenomenon.

And I guess this is why they do not say evidence-based because they know it isn't yet suggest it all the same. This is what I think would be particularly useful for the NICE committee to see. It shows that what appears to be an evidence-based academic consensus is in fact just the line of least resistance.

Things like the IOM report are important but they can be painted as partisan. This looks like the reasonable neutral academic establishment view - but it is transparently fudged. It is 'Oh OK there is no evidence but please let's not rock the boat'. The fact that Harvard can do this is I think relevant to the debate to be had at NICE.

 

I have always assumed that Eisenberg and Kleinman must have been major influences on what was thought at Harvard. It is difficult to see how their influence would have been benign.

 

Given its replicability crisis, this is something of a self-own.

 

I think its partly the line of least resistance - what I might call their following the predominant narrative about ME/CFS being related to deconditioning, patient fears, and anxiety and depression and the general narrative that all exercise is good for all people. When in doubt, tell patients to eat better, get lots of exercise and sunshine, and reduce the stress.

But I also think the clinical guidance providers have either not understood the problems with the "CFS" evidence base where such diverse CFS, ME, and ME/CFS definitions are all lumped together as the same clinical entity or they have struggled with how to deal with it.

What has helped to tease that apart here are the conclusions that AHRQ reached in its reanalysis after excluding Oxford studies because they included patients with other diseases. Those conclusions were:
a) there was a lack of evidence of efficacy for GET and barely any for CBT once Oxford was excluded
b) in spite of a lack of harms reporting in these studies, there was evidence of harms
c) studies using case definitions that require hallmark features such as PEM were "blatantly missing" from the CBT and GET evidence base.

AHRQ never went back and reanalyzed the quality of these trials in light of the PACE criticisms, which is unfortunate.

But even without that, AHRQ's conclusions of the lack of evidence of efficacy specifically in ME definition studies combined with the evidence of harms (AHRQ, patient surveys, IOM findings on aerobic metabolism impairment) has played a role in getting some - not all - of the clinical education providers to reevaluate their statements about CBT and GET.

 

Mm. As you probably know I see that as a red herring. There is no evidence for efficacy of CBT or GET for any group as far as I can see. The Oxford studies excluded did not provide reliable evidence either. I was surprised that this was not obvious to AHRQ.

The lesson of the Harvard document for me is that these so-called university based opinion leaders do not really understand how to judge evidence. What I think may be useful for NICE discussions is pointing out just how prevalent this is on both sides of the Atlantic.

It ought to be so simple. Unblinded trials with subjective outcomes are never considered reliable evidence in any of the branches of medicine I have worked in - and for good reasons all academic medics should understand.

 

It seems to me this is a risky strategy that could easily backfire, unless you are absolutely sure that only evidence and not eminence holds any sway at NICE. Could it not be interpreted by some that if the eminent Harvard thinks this, there must be some truth to it?

 

I agree that there's no evidence of efficacy in light of the methodological issues, problems with switched outcomes, bias, use of subjective outcomes in unblinded trials, etc. Poorly run trials should not be accepted as the basis of recommendations for a given disease.

But I think its also significant that the population that they used to do these studies is not representative of patients with ME. IMO, its of questionable ethicality to base recommendations for treatment in ME on patients who do not have it.

I don't know why this would be considered a red herring. I also dont see that this in any way negates the impact of bad studies - it adds to the impact. The combination of bad studies and the wrong population is especially damning and such studies have no business informing treatment recommendations for ME

Regarding AHRQ not seeing the obvious problems - In 2014, they rated PACE a good study in spite of extensive input from the community on the issues with the trial. In 2016, post Tuller's first set of articles, we asked to have the quality of PACE reassessed and also to have the impact of Oxford studies on efficacy claims evaluated since AHRQ highlighted Oxford as a bad definition. But AHRQ only looked at the impact of Oxford and did not reevaluate the quality of all the studies.

 

The NICE committee consists of a good number of members who have a clear understanding of the evidence. There may be a similar number who have a muddled idea of the evidence but they will have to argue on the basis of evidence, because the NICE rules require that. My understanding s that last time nobody was in a position to challenge muddled accounts the evidence. This time they are.

What I think is useful about the Harvard document is that it indicates a complete muddle. It ends up suggesting chiropractic, where there is no evidence at all. It lays bare the fact that so many 'eminent' people are bullshitting.

 

There are more complex reasons why the population recruited in PACE were inappropriate but the argument about being unethical to apply results from a wider group to a narrower group does not work in general in logical or scientific terms. Nobody knows whether the term ME defines a group with particular treatment needs. It probably does but without evidence we do not know. So it is the category issue that is the red herring, not that the studies are bad, which is the evidence.

As a disinterested medical expert witness from another discipline I do not have any means to argue that the Oxford criteria are wrong, any more than 'osteoarthritis' is wrong. It is arbitrary. But what I can argue about is the reliability of the available data.

Indeed, so why was that? In 2014 I posted on PR that PACE looked to a newcomer like me like it was being hyped (not quite in those words but transparently). I only had to glance at the abstract to see that it was pretty much a non-starter. And of course plenty of others had seen defects before that. In the broader public debate the AHRQ decision has been helpful, there is no doubt, but if we want the NICE committee to see that there is no reliable evidence base for recommending CBT or GET for anyone with fatigue as things stand then it is important to get the arguments right.

NICE has to consider a wider range of fatiguing illness for reasons relating to the way guidelines operate in practice. The most likely reasoning for keeping CBT or GET will be that 'they may be helpful for some people'. The key argument is that we do not have evidence even for that. If AHRQ still maintain the trials are OK but just not for a narrower group that seems to me to muddy things badly. In contrast, Harvard, as I understand it, are saying the trials are not OK but why not do some jolly exercise or chiropractic anyway. I am not suggesting anyone follows their advice but I think it reveals how muddled 'authorities' are, without potentially defending the quality of weak trials.

 

AHRQ originally lumped together evidence from any CFS, ME, ME/CFS, etc definition and treated them all as representing the same clinical entity. But then both AHRQ and the NIH rejected Oxford as an acceptable definition because it includes patients with other fatiguing conditions. And AHRQ pointedly said that studies using definitions that required hallmarks like PEM were blatantly missing. Leaving aside the issue of study quality for the moment, this is not about a broader versus a narrower group of patients. This is about treatment recommendations being made based on studies in a group that could be missing the key hallmark of the disease under consideration. And if the study participants are missing key features such as PEM, we can reasonably expect they might react quite differently to the treatments being evaluated. IMO, this is a important question of whether this particular evidence is even medically relevant to people with ME/CFS, no matter how well it was executed.

I understood that NICE is developing guidelines for "ME/CFS" so I would expect they first will have to be clear on what that is and what evidence will be accepted. Will they be accepting evidence from studies that poorly defined cohorts? This isnt just an issue with PACE. There are other studies out there that use poor cohorts, including some that go beyond treatment trials to statements about prevalence, nature of risk factors, etc.

I'd hope that NICE would grapple with this issue and not just assume that all "CFS" studies using any definition are valid evidence for ME/CFS, that they are the same clinical entity, just differences in levels of severity.

 

I'm not defending them on this point. This is something that many of us called out when they first released the protocol and again during the comment period.

But I also see this is an example of the challenge that "evidence-based" medicine and practitioners of evidence based reviews have when the evidence base is as screwed up as this one is with its poorly executed studies and the conflation of overly broad and specific definitions as the same thing. AHRQ is not the only group to have accepted PACE as a good study.

 

Indeed and in addition my n=1 evidence says that it does not treat or cure ME.
But it does help my back pain

 

The nature of a guideline of this sort is that it has to be a guideline for anyone where diagnosis X is a major option or perhaps the closest available option to one of any number of clinical presentations. The guidelines has to cover everyone in a ball park. Otherwise there will be people for whom there is no guideline. Clinical care works on probabilities, not on pigeonholes.

Within that ball park, sorting groups into definite diagnosis, possible or doubtful or whatever is needed where there is evidence that indicates the difference leads to different management. As I understand it none of the trials available tell us anything about that. There are common sense arguments about who might suffer with exercise but they do not amount to reviewable evidence.

My understanding from the scoping meetings is that the above is the position of the chair of the guideline group.

They should be looking at anything possibly relevant but I don't think it matters because the evidence is not usable. Epidemiology and suchlike is not of great importance to the guidelines - they are essentially about best management. So far we do not know whether cohort definition affects that.

 

Just found this: https://www.health.harvard.edu/staying-healthy/yoga-for-chronic-fatigue-syndrome.

Harvard med-ed continuing its spiritual merger as a goop.com subsidiary.

There is a link at the bottom for the aforementioned guide on how to maximize your energy and vitality and eliminate all your spiritual toxins and impure thoughts causing your fatigue. Top. Medicine.

 

I liked hatha yoga because we lay a lot on the mat.