Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing, Morgan et al, 2021

Key Points

Question Do practitioners understand the probability of common clinical diagnoses?

Findings In this survey study of 553 practitioners performing primary care, respondents overestimated the probability of diagnosis before and after testing. This posttest overestimation was associated with consistent overestimates of pretest probability and overestimates of disease after specific diagnostic test results.

Meaning These findings suggest that many practitioners are unaccustomed to using probability in diagnosis and clinical practice. Widespread overestimates of the probability of disease likely contribute to overdiagnosis and overuse.

Abstract

Importance Accurate diagnosis is essential to proper patient care.

Objective To explore practitioner understanding of diagnostic reasoning.

Design, Setting, and Participants In this survey study, 723 practitioners at outpatient clinics in 8 US states were asked to estimate the probability of disease for 4 scenarios common in primary care (pneumonia, cardiac ischemia, breast cancer screening, and urinary tract infection) and the association of positive and negative test results with disease probability from June 1, 2018, to November 26, 2019. Of these practitioners, 585 responded to the survey, and 553 answered all of the questions. An expert panel developed the survey and determined correct responses based on literature review.

Results A total of 553 (290 resident physicians, 202 attending physicians, and 61 nurse practitioners and physician assistants) of 723 practitioners (76.5%) fully completed the survey (median age, 32 years; interquartile range, 29-44 years; 293 female [53.0%]; 296 [53.5%] White).

Pretest probability was overestimated in all scenarios.

Probabilities of disease after positive results were overestimated as follows:

pneumonia after positive radiology results, 95% (evidence range, 46%-65%; comparison P < .001);
breast cancer after positive mammography results, 50% (evidence range, 3%-9%; P < .001);
cardiac ischemia after positive stress test result, 70% (evidence range, 2%-11%; P < .001);
and urinary tract infection after positive urine culture result, 80% (evidence range, 0%-8.3%; P < .001).

Overestimates of probability of disease with negative results were also observed as follows:

pneumonia after negative radiography results, 50% (evidence range, 10%-19%; P < .001);
breast cancer after negative mammography results, 5% (evidence range, <0.05%; P < .001);
cardiac ischemia after negative stress test result, 5% (evidence range, 0.43%-2.5%; P < .001);
and urinary tract infection after negative urine culture result, 5% (evidence range, 0%-0.11%; P < .001).

Probability adjustments in response to test results varied from accurate to overestimates of risk by type of test (imputed median positive and negative likelihood ratios [LRs] for practitioners for chest radiography for pneumonia: positive LR, 4.8; evidence, 2.6; negative LR, 0.3; evidence, 0.3; mammography for breast cancer: positive LR, 44.3; evidence range, 13.0-33.0; negative LR, 1.0; evidence range, 0.05-0.24; exercise stress test for cardiac ischemia: positive LR, 21.0; evidence range, 2.0-2.7; negative LR, 0.6; evidence range, 0.5-0.6; urine culture for urinary tract infection: positive LR, 9.0; evidence, 9.0; negative LR, 0.1; evidence, 0.1).

Conclusions and Relevance This survey study suggests that for common diseases and tests, practitioners overestimate the probability of disease before and after testing. Pretest probability was overestimated in all scenarios, whereas adjustment in probability after a positive or negative result varied by test. Widespread overestimates of the probability of disease likely contribute to overdiagnosis and overuse.

https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2778364

To clarify: in the probabilities above, the "evidence ranges" denote the rates of diagnosis for each situation according to the medical literature.

 

Meaning These findings suggest that 'experts' are unaccustomed to using probability in diagnosis and clinical practice.