Does anybody know how reliable the current antibody tests are? Or does that depend on the manufacturer?
I actually just came across this article from Stat news about this very subject!
https://www.statnews.com/2020/04/24...to-be-released-heres-how-to-kick-their-tires/
If you scroll down to the subheading:
“So about those false positives and false negatives?
No test is perfect. And the sheer number of antibody tests — Dutch virologist Marion Koopmans recently saw nearly 275 on a list maintained by the WHO — makes it very tough at this stage to know how good any of them actually are. The WHO is working with a number of labs trying to validate tests, said Van Kerkhove, who added: “Unfortunately that takes a little bit of time.”
In particular, the rapid tests appear not to perform well at all. Koopmans, the head of virology at Erasmus Medical Center in Rotterdam, said the Dutch national serology task force has recommended that people not use the rapid tests, because of the risk that people will get a false result and assume — if it was a positive — that they have protection they do not in fact have.
Every serology test is going to produce some erroneous results. Some people who were truly sick will test negative — that’s a false negative. Some people who were not sick will test positive — that’s a false positive.
Each commercial test comes with guidance from the manufacturer about how “sensitive” it is — in other words, what percentage of true positive cases it will detect — as well as how “specific” it is, meaning how good it is at not generating false positive results.
Those estimates are especially important when the rate of infection in an area is likely low. Even a small over-estimate — say a 5% false positive rate — can vastly increase the final projection of how many people in a location had been infected.
Michael Osterholm, director of the Center for Infectious Diseases Research and Policy at the University of Minnesota, drew up a chart to explain how different rates of sensitivity and specificity will impact a serology study in an area with 1 million people, using a test that had 95% sensitivity (caught all but 5% of true positives) and 95% specificity (designated as positive only 5% of people who were actually negative).
If 5% of the population had been infected with SARS-CoV-2, there would have been 50,000 infected people. This test would find 47,500 (the true positives) but it would miss 2,500 (the false negatives). And it would detect 47,500 false positives — as many false positives as true positives. If the rate of infection in the community was smaller, the percentage of wrong results would rise.
If the rate of infection in the community increased, the errors become less substantial. If 15% of the community — 150,000 — had been infected, this test would find 142,500 true positives, 42,500 false positives, and would miss 7,500 cases — the false negatives.
Applying this knowledge to Thursday’s results from New York puts the picture in sharper focus. The release from the state doesn’t disclose the sensitivity of the test used, but it does note the specificity is between 93% and 100%, a “
huge range,” Ashish Jha, head of Harvard’s Global Health Institute, noted on Twitter. If the test performed at the low end of that range, New York’s infection rate would be
closer to 7%— half the figure Cuomo announced — and nearly one out of every two positives would have been a false positive, Jha said.
“These tests don’t perform like people think they do and so there are a lot of crazy results,” Osterholm said. “You can often find more than half of the positives you do document are actually false positives.”
There’s a lot more in the article after that.
