Estimating Prevalence, Demographics and Costs of ME/CFS Using Large Scale Medical Claims Data and Machine Learning, 2018, Valdez, Proskauer et al

[ME/CFS Skeptic]

So when G93.3 and R53.82 are used to select records for analyses, the resulting cohorts also include the poorly defined PVFS (in the "ME" cohort) and the symptom of chronic fatigue (in the "CFS" cohort). This could result in inflated prevalence estimates. And as we know, even aside from the ICD issue, a "CFS" diagnosis itself has sometimes been used as a wastebin by the medical community which could further compound the issue of overestimating prevalence

Yes. I basically ignored the results of ME + CFS diagnoses because of the problems with the ICD-CM codes for CFS and I focused on ME-results only, for the reasons you mention.

As you explained the ME in this paper is not ICC-defined ME but something else. Nonetheless I think it’s a relatively reliable selection, considering the enormous problems with diagnosis in the field of ME/CFS. There is a supplement to the paper that explains this a little further.

• In the ICD-CM 9 ME appeared in the index only of the code 323.9, Unspecified causes of encephalitis, myelitis, and encephalomyelitis.
• In the ICD-CM 10 ME falls under G93.3 which is labelled Postviral fatigue syndrome.

So there is a danger that other diagnoses such as unspecified causes of encephalitis or a passing PVFS have been given the same code as ME, confusing matters for the analysis. But the authors looked at continuous enrollment of 2 to 4 years, so the patients apparently had a chronic condition.

So I suppose it's reasonable to assume that most of the 14.000 patients who received this label, had what we call ME/CFS. Other prevalence estimates that we currently have, also had seriously flaws such as using the Fukuda criteria. So I think this study provides us useful information.

I see it as a different way of looking at ME/CFS and coming up with more or less the same estimated prevalence. The direct prevalence of the ME- ICD codes was 0,12%. I was quite surprised how high this number was because this is most certainly an underestimation as few physicians make the diagnosis of ME or use these ICD-codes. Because of the large dataset, the authors were able to estimate the number of ME patients that have not (yet) been diagnosed. Total prevalence then increased to around 0,85%.

So somewhere between 0,12 and 0,85% That is in agreement with community-based prevalence studies (using Fukuda-criteria) of 0,24-0,42% and with the data from the UK biobank (0,44%).

The percentage of patients being female (60%) is a bit lower than the community-based prevalence studies found higher rates (83% for Wichita and 71% for Chicago), the UK biobank study (70%), and a 2016 Australian prevalence study (around 75%), but other studies have found even lower rates such as Naculs prevalence study (51%).

 

[Webdog]

Just recently, Kaiser Permanente decided to start using ME G93.3 and slowly move away from CFS R53.82. Eventually CFS will should be abandoned by Kaiser. This paper supports that decision.

"The authors recommend use of the ME diagnosis code (G93.3) rather than CFS (R53.82), which defaults to 'Chronic fatigue, unspecified,' for better tracking where symptoms warrant."

 

[Webdog]

Some cost quotes from the study, suggesting that healthcare providers can actually reduce costs by training doctors and providing more effective care.

• Medical costs for ME/CFS in this study are double those for either Lupus or MS and four times higher than for the general insured (reference) population.
• Despite high direct medical costs, these [ME/CFS] patients often have significant unmet health care needs or forgo routine medical care.
• The data from this study illustrate the high costs of the illness [ME/CFS], and point to the potential for cost control if patients are diagnosed and provided with the most effective care.
• Many patients cite a long and costly journey to receiving an accurate diagnosis of ME/CFS. Further, once diagnosed, most patients struggle to find primary care providers who are knowledgeable about the condition and well versed in the best practices for managing the symptoms. These twin challenges in diagnosis and treatment are certainly contributors to added cost in the healthcare system.

 

[Medfeb]

Agree with both @Michiel Tack and @Webdog's posts. The ME cohort is certainly better than the CFS cohort. And Kaiser's decision is encouraging

But I'm not so sure that we can conclude that most in the "ME" cohort have what we call ME/CFS especially since the US medical community has not used the term ME. If I remember correctly, I've heard at least one of our disease experts use the term PVFS when the duration was not long enough or when some other criteria were not met. So I expect at least some of these cases could have resolved shortly and not been the kind of chronic illness seen in ME.

That said, its all we have at this point and we need to exploit whatever insights we can gain.

 

[Webdog]

Essentially, the approach that Kaiser Permanente is taking is to map IOM/SEID criteria to ME G93.3. Though nothing should prevent a doctor from using ICC if they prefer.

It seems a reasonable way to proceed at this time. Certainly better than the current practice of mapping everyone to CFS R53.82.

 

[ME/CFS Skeptic]

I've heard at least one of our disease experts use the term PVFS when the duration was not long enough or when some other criteria were not met. So I expect at least some of these cases could have resolved shortly and not been the kind of chronic illness seen in ME.

Didn't the fact that they looked at codes with continuous enrollment from 2 to 4 years partially solve that problem of PVFS cases resolving in a short timespan, or am I misinterpreting this?

You also have to compare it to what we currently had as estimates. You can have Fukuda CFS and recover after a few months. Data from the epidemiological study in Wichita, Kansas indicated that at three year follow-up almost 80% no longer fulfilled the diagnostic criteria. And if I remember correctly, in Dubbo-like studies 10% of infectious cases meet Fukuda-criteria at 6 months, but this decreased to 4% as time went on. So 60% of the original CFS sample no longer had CFS at long-term follow up.

So what we call ME/CFS (unfortunatedly) is highly variable, uncertain and ill defined.

 

[Medfeb]

@Michiel Tack - good point about continuous enrollment. That would certainly help that.

And you are right about the US epi studies - remarkably few kept a CFS diagnosis even two years in a row in that study.

 

[JaimeS]

Using the current US population, a prevalence of 857/100,000 calculates to roughly 2.8 million. That's 7x the prevalence of MS. I do find that hard to believe.

In the US, many providers won't allow an ME diagnosis, muddying the waters even further. For example, Kaiser Permanente currently only allows their doctors to make a "CFS" diagnosis (ICD-10 R53.82, Chronic fatigue unspecified), regardless of which criteria are used.

Whoa, seriously?

That's out-and-out weird. Any rationale?

 

[Snow Leopard]

Whoa, seriously?

That's out-and-out weird. Any rationale?

"roughly 2.8 million" is only a guess because it fails to account for demographics, but 0.8% was the pooled prevalence of "clinically assessed" cases worldwide in this systematic analysis:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3616604/

I'd expect the USA to have lower prevalence due to lower infectious burden than other countries, but still be in the same ballpark.

The key problem with all of these estimates is the same as the debate about the various case criteria - there is no strict demarcation, leading to slippage of estimates if researchers or clinicians are sloppy. If all you need is unexplained long term fatigue, then estimates blow out to over 3%.

 

[Forbin]

I don't think it accounts for everything, but... Leonard Jason's figure of 422/100,000 (.422%) is only intended as an estimate of the adult US population with CFS. If this current study is not confined to adults, you'd expect it to come up with a somewhat larger number - though a doubling of the previous estimate seems unlikely to be due to this factor alone.

 

[JaimeS]

"roughly 2.8 million" is only a guess because it fails to account for demographics, but 0.8% was the pooled prevalence of "clinically assessed" cases worldwide in this systematic analysis:

No, I'm wondering why Kaiser won't use the ME coding.

 

[JaimeS]

The key problem with all of these estimates is the same as the debate about the various case criteria - there is no strict demarcation, leading to slippage of estimates if researchers or clinicians are sloppy. If all you need is unexplained long term fatigue, then estimates blow out to over 3%.

But we've never used Oxford in the US. The only reason unexplained fatigue could be categorized as "CFS" is sheer laziness here. Our weakest criteria is Fukuda, which requires at least four of eight symptoms no matter how you slice it.

 

[Adrian]

Approximately 80% of individuals in this dataset were insured by commercial health insurance; close to 20% had coverage from Medicare, the U.S. government program for individuals age 65 and over and for certain individuals with disabilities. The dataset did not include any claims associated with Medicaid, the U.S. government program for low-income individuals. For some of the topics in this study, sub-sets of the dataset were used.

I don't know much about the US insurance system but could the 20% coverage of people with medicare bias the results they get in that it covers individuals with disabilities which could include people with ME. I think the question is are there more likely to be people with ME in the data set due to this?

 

[Medfeb]

But we've never used Oxford in the US. The only reason unexplained fatigue could be categorized as "CFS" is sheer laziness here. Our weakest criteria is Fukuda, which requires at least four of eight symptoms no matter how you slice it.

But most of those 8 symptoms are not unusual or distinctive and there are 163 different combinations of them when you specify any 4 of 8 symptoms. Nacul pointed out that only 20% (if I remember correctly) of those combinations require PEM. So in practice, lots of conditions can be stuffed into the Fukuda bucket as long as they have "medically unexplained chronic fatigue" - the only thing actually required.

 

[Adrian]

I'm still reading through the paper. But what I think they have done from the ML perspective is used the ME diagnostic codes from a subset as labels for a supervised training algorithm (a boosted tree). Then they use the model (after testing that it works on a test set (and selecting the model hyper parameters using a verification set) on features from the full data set which is based on other codes associated with patients along with the age, gender etc.

This gives them estimated for those who have similar patterns. But I'm not sure how good their model is they quote some figures but I would have liked an analysis of some of the errors around any primary diagnosis code on false positives as they could be picking up on other chronic conditions and this (as relatively rare things) being hidden in quite good overall results. This would result in over estimates. But I could be wrong as I've not read everything carefully yet.

The other thing that struck me was they will pick up those people who are seeing doctors but not those patients who have given up on doctors as they don't do anything. There is no temporal model in their data (I think) so they are unlikely to pickup on such trends where one year a patient sees the doctor then gives up.

The other thing that struck me was the struggled to model ME+CFS which could just be a feature of their ML modelling but it suggests there is a certain diversity or noise in the wider data set. It could be due to better diagnostics with ME but it could be due to some more knowledgeable doctors using the ME code and others with less experience using the CFS code.

 

[Adrian]

But most of those 8 symptoms are not unusual or distinctive and there are 163 different combinations of them when you specify any 4 of 8 symptoms. Nacul pointed out that only 20% (if I remember correctly) of those combinations require PEM. So in practice, lots of conditions can be stuffed into the Fukuda bucket as long as they have "medically unexplained chronic fatigue" - the only thing actually required.

From what I understand these aren't necessarily using any particular criteria but are really down to a doctors judgement and preference for particular codes. But the fact they can model the ME codes but struggled with ME+CFS does suggest something about the consistency and usage of the codes. What would be nice would be to see some sort of cluster analysis of their data to see if there are groups of common patterns of their selected features.

 

[Simon M]

I read this for another purpose and thought I'd post my comments here.


It makes sense to explore the value of the huge insurance claims database, with thousands of cases of "CFS" and "ME", But both the symptoms/factors selected by machine learning and the demographics look implausible for ME/CFS.

The code for CFS includes the dustbin diagnosis of “chronic fatigue, unspecified”, which means it isn't much use. And it’s unsurprising that they failed to construct a machine learning model for it (that was the basis for claiming it was “less homogeneous“ than ME).

ME/CFS diagnosis is not taught in medical schocyol. As the authors comment, those familiar with ME/CFS are probably the ones using the ME label to cover both ME and CFS. It’s likely that other physicians are putting all sorts of crap under the same label.

The prevalence for ME alone is 0.12%, which is a more credible rate than the headline 0.8% for CFS and ME combined. However, the details cast doubt on the accura of diagnosis:

The top symptoms/factors selected by machine learning include disorders of the urinary system, vitamin D deficiency, and diagnostic procedures for abdomen. Breathing difficulties are on a par with headaches. Neither post-exertional malaise nor disabling fatigue are collected symptoms. This does not look like ME/CFS. The authors acknowledge that machine learning did not validate the diagnosis.

It’s not surprising that the demographics look so weird:



Other data and clinical experience indicates that the gender difference emerges during adolescence and remains at around 3:1, and that peak incidence years are the teens and the 30s. ME/CFS is rarely diagnosed in people over the age of 65 because there are so many other potential causes of the symptoms.

By contrast, the graph implies* peak incidence from age 60, not in adolescent years and people‘s 30s. And only a modest female bias.
* The big jumps in prevalence indicate high rates of incidence

I think that, unfortunately, this is a case of garbage in, garbage out.

 

[ME/CFS Skeptic]

Neither post-exertional malaise nor disabling fatigue are collected symptoms.

Wasn't this a problem of the database and ICD-codes rather than of the diagnosis itself? Could be that doctors used these symptoms to make the diagnosis but that this was not recored in the data.

the graph implies* peak incidence from age 60

Good point. This is weird. Could doctors be using the PVFS code for old people not recovering from a virus?

I think that, unfortunately, this is a case of garbage in, garbage out.

Maybe it could be useful to take a sample of the 14.000 patients who got the ME/PVFS code and perform a diagnostic physical examination on them. It could tell us if this kind of approach is usefull or deceiving. This is probably not easy to do because of privacy reasons.

 

[Trish]

By contrast, the graph implies* peak incidence from age 60, not in adolescent years and people‘s 30s.

Is it new diagnoses, or all people who have an ME diagnosis? I am in my 60's and still have an ME/CFS diagnosis, even though that diagnosis was first made nearly 30 years ago.

 

[Medfeb]

By contrast, the graph implies* peak incidence from age 60, not in adolescent years and people‘s 30s. And only a modest female bias.
* The big jumps in prevalence indicate high rates of incidence

Interesting point. Thanks for posting

I'd think that up to certain age, the high rate in prevalence in older patients could also reflect the poor prognosis/low rates of recovery - that is, patients at age 60 include those who became ill at 20, 30, 40, etc as well as 60.

But if that were the explanation, I would have expected the rates to fall off in the 70s and 80s especially if the disease is associated with an earlier morbidity as a few studies suggested. The continued rise into the 70s and 80s is weird.