Sensitivity and Specificity

How the accuracy of a diagnostic or screening test is measured against a reference standard. The 2 x 2 table, sensitivity and specificity as conditional probabilities and why accuracy is a poor summary; what changes when the reference standard is itself a definition, shown by reading the same table with each test as the reference; Wilson intervals and the number of cases a study needs; how sensitivity depends on the severity and make-up of the cases, spectrum bias in studies of severe cases and healthy controls, and verification bias with the Begg-Greenes correction. Worked on HbA1c as a screen for undiagnosed diabetes, defined by fasting glucose, in NHANES adults.

A national health survey drew blood from 1,975 adults who had never been told they had diabetes. In 115 of them the fasting glucose was in the diabetic range, and in 79 the glycated hemoglobin was. Only 54 met both. Two accepted definitions of the same disease, applied to the same blood, disagree about 86 people. How well does one measurement detect a condition defined by another, and how should "how well" be measured?

The two measurements come from the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018. It drew blood from a subsample of adults who had fasted overnight.

  • Fasting plasma glucose (FPG) is the concentration of glucose in the blood after at least eight hours without food.
  • Glycated hemoglobin (HbA1c) is the percentage of hemoglobin that has glucose attached to it. A red blood cell lives about three months, so HbA1c reflects average glucose over that period.

The American Diabetes Association (ADA) accepts either as a criterion for diabetes: an FPG of 126 mg/dL or more, or an HbA1c of 6.5% or more (ADA, 2025, Table 2.1). Medicine answers the question above with two numbers. Sensitivity is the share of people with the condition whom the test flags. Specificity is the share of people without it whom the test clears. The later articles of this unit build on both. They turn them into the probability of disease after a given result, summarize a result by a likelihood ratio, and trace the trade-off between the two as the cut-off moves.

First we fix what is being tested and against what, and why the "truth" here is itself a definition. Then we define the two measures on the 2 by 2 table and see why the plain share of correct results misleads. Next we read the same table with the roles of the two tests reversed, and put confidence intervals on the measures. Finally we ask what sensitivity depends on, and how two common study designs distort it. The measures are conditional probabilities, as defined in Conditional Probability and the Law of Total Probability.

The test, the condition and the reference standard

Three things have to be fixed before any accuracy can be measured.

  • The index test is the procedure being evaluated, together with its rule for calling a result positive. Here it is HbA1c, positive at 6.5% or more.
  • The target condition is what the test is meant to detect: diabetes not yet diagnosed.
  • The reference standard is the best available way of deciding who has the target condition. Here it is a single FPG of 126 mg/dL or more.

The adults studied are those who would be screened in practice, adults without a diagnosis. NHANES has 2,366 nonpregnant adults aged 18 to 80 who met its fasting criterion of 8 to 24 hours. All had both blood measurements and an answer to the diabetes question. Of these, 391 already had a diagnosis and are set aside. Their glucose reflects treatment, and nobody screens a person already known to have the disease. The remaining 1,975 adults are treated as a random sample of adults without a diagnosis. NHANES is a complex survey, and its sampling weights would be needed for estimates about the U.S. population. As in the other units of this course, the weights are ignored, so the figures describe these adults (NCHS, 2020a, 2020b, 2020c).

The reference standard deserves suspicion before it is used. Diabetes is a disease of chronically raised blood glucose, and its diagnostic criteria are thresholds on glucose measurements, chosen by expert committees. "Diabetes" in this article therefore means "FPG of 126 mg/dL or more on the morning of the NHANES examination." That is a definition rather than an independent fact about the patient. Three consequences follow.

  • One value is not a diagnosis. Without unequivocal hyperglycemia, the ADA requires two abnormal results. They may be the same test on two occasions, or two different tests, possibly from the same sample (ADA, 2025, Table 2.1). A single NHANES glucose is not a clinical diagnosis.
  • The reference varies. FPG varies from day to day, so some of the 115 would not meet the threshold on another morning.
  • The two tests measure different things. HbA1c is a three-month average and FPG a morning snapshot. Their disagreement is partly a difference between two definitions, not only error in one of them. The ADA acknowledges that the cut-points of the different tests identify groups with incomplete concordance (ADA, 2025).

Choosing FPG as the reference is a modeling decision. It is made here because it lets the methods be shown on real data, and a later section reverses it.

The 2 by 2 table

Each adult falls into one of four cells, according to the test result and the reference standard. Following the layout of the course notes, the test result is in the rows and the reference in the columns:

HbA1c FPG $\ge$ 126 (diabetes) FPG $<$ 126 (no diabetes) Total
$\ge 6.5\%$ (positive) 54 (true positives, TP) 25 (false positives, FP) 79
$< 6.5\%$ (negative) 61 (false negatives, FN) 1,835 (true negatives, TN) 1,896
Total 115 1,860 1,975

The prevalence of the condition among the adults tested is $(\mathrm{TP} + \mathrm{FN})/N = 115/1{,}975 = 0.0582$.

Write $D$ for the event that a randomly chosen adult has the condition, $D^c$ for its complement, and $T^+$ and $T^-$ for a positive and a negative test. The two standard measures of accuracy condition on the truth.

Sensitivity is the probability that the test is positive in a person who has the condition:

$$\mathrm{Se} = P(T^+ \mid D) = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}} = \frac{54}{115} = 0.4696.$$

Specificity is the probability that the test is negative in a person who does not:

$$\mathrm{Sp} = P(T^- \mid D^c) = \frac{\mathrm{TN}}{\mathrm{TN} + \mathrm{FP}} = \frac{1{,}835}{1{,}860} = 0.9866.$$

Their complements are the two error rates. The false negative rate, $1 - \mathrm{Se} = 0.5304$, is the share of the condition the test misses. The false positive rate, $1 - \mathrm{Sp} = 0.0134$, is the share of people without the condition whom it flags.

Both denominators are column totals, and that is the point of the definitions. Sensitivity is computed entirely within the diseased column and specificity entirely within the other. So neither depends on how the population is divided between the two columns. Each is a property of how the test behaves in one kind of person. Yerushalmy (1947), writing on the assessment of diagnosis by X-ray, gave the two quantities the names in use today.

Check your understanding. Suppose the survey had examined twice as many adults with normal fasting glucose, and the test behaved the same way in them. Which of sensitivity, specificity and prevalence would change?

Only the prevalence. Doubling the second column doubles the 25 false positives and the 1,835 true negatives, so specificity stays at 0.9866. The diseased column is untouched, so sensitivity stays at 0.4696. The prevalence roughly halves. This is what "does not depend on prevalence" means.

HbA1c at 6.5% is a specific test with low sensitivity. It almost never flags an adult whose fasting glucose is normal. But it misses more than half of the adults whose fasting glucose is in the diabetic range. The figure shows why: the HbA1c values of the two groups overlap. The threshold sits in the upper tail of the larger group and near the middle of the smaller.

HbA1c among the 1,860 adults with FPG below 126 mg/dL (top) and the 115 with FPG of 126 or more (bottom), each as a share of its own group; the dashed line is the 6.5% cut-off, and bars to its right are test positives. Notice that almost all of the top group lies left of the line (specificity 0.987), but so does more than half of the bottom group (sensitivity 0.470): the median HbA1c of the cases, 6.4%, is below the cut-off.
HbA1c among the 1,860 adults with FPG below 126 mg/dL (top) and the 115 with FPG of 126 or more (bottom), each as a share of its own group; the dashed line is the 6.5% cut-off, and bars to its right are test positives. Notice that almost all of the top group lies left of the line (specificity 0.987), but so does more than half of the bottom group (sensitivity 0.470): the median HbA1c of the cases, 6.4%, is below the cut-off.

Why not accuracy?

A single number would be simpler. The obvious candidate is the proportion of correct results, $(\mathrm{TP} + \mathrm{TN})/N = 1{,}889/1{,}975 = 0.9565$, which looks impressive.

Check your understanding. A "test" that calls everybody negative finds no cases at all. What is its accuracy in these 1,975 adults?

It is $1{,}860/1{,}975 = 0.9418$, because it is correct for every adult without the condition. That is barely below HbA1c's 0.9565.

The reason is that accuracy is a weighted average of sensitivity and specificity, $\pi\,\mathrm{Se} + (1 - \pi)\,\mathrm{Sp}$, with $\pi$ the prevalence. When the condition is uncommon the weight falls almost entirely on specificity. Accuracy hides exactly the failure that matters in screening, the missed cases. It also changes with prevalence even when the test does not. The library's Binary Classification Metrics makes the same point for machine-learning classifiers. The course wiki's Evaluating Classifiers gives the information retrieval vocabulary, in which sensitivity is called recall.

The same table, read the other way

So far FPG has been the truth. Suppose instead that HbA1c of 6.5% or more is taken as the definition of diabetes, and FPG is the test under evaluation. Nothing in the data changes; the table is transposed.

Check your understanding. With HbA1c as the reference, will FPG's sensitivity be higher or lower than HbA1c's 0.470?

Higher. The 54 adults who meet both definitions are the true positives either way. As the reference, HbA1c has only 79 positives to divide by instead of 115. The sensitivity of FPG is $54/79 = 0.6835$, and its specificity is $1{,}835/1{,}896 = 0.9678$.

Index test Reference Sensitivity Specificity
HbA1c $\ge 6.5\%$ FPG $\ge 126$ 0.470 0.987
FPG $\ge 126$ HbA1c $\ge 6.5\%$ 0.684 0.968

Each test looks imperfect when the other is called the truth. HbA1c misses 53% of the adults whose FPG is diabetic, and FPG misses 32% of the adults whose HbA1c is. Whichever test is chosen as the reference is perfect by definition. Neither statement is wrong; each is a statement about agreement with a chosen reference.

When the reference is itself a measurement with error, every disagreement is booked as an error of the index test. So an imperfect reference makes a test look worse than it is. A test that shares the reference's errors can look better than it is; a second glucose measurement taken the same morning is an example. Reports of diagnostic accuracy should therefore always name the reference standard. A claim such as "HbA1c misses half of diabetes" should be read as "HbA1c at 6.5% misses about half of the adults whose single fasting glucose is 126 or more." Where no test is an acceptable reference, latent class models estimate the accuracy of several tests at once by treating the true state as unobserved. They are beyond this course.

Confidence intervals

The 115 cases are a sample, so 0.4696 is an estimate. How far might it be from the sensitivity in the population screened?

Sensitivity and specificity are proportions, each estimated from its own column. Sensitivity is $x = 54$ successes in $n = 115$ trials, where $n$ is the number of people with the condition, not the size of the study. All the methods of Confidence Intervals for a Proportion apply, and so do its warnings. The Wald interval $\hat{p} \pm 1.96\sqrt{\hat{p}(1-\hat{p})/n}$ covers less often than it claims when $n$ is small or $\hat{p}$ is near 0 or 1. Both are common for accuracy estimates. Wilson's score interval does not have that defect.

Measure Estimate Wald 95% Wilson 95%
Sensitivity, $54/115$ 0.4696 (0.3784, 0.5608) (0.3808, 0.5603)
Specificity, $1{,}835/1{,}860$ 0.9866 (0.9813, 0.9918) (0.9802, 0.9909)
Sensitivity, ages 18–39, $6/12$ 0.500 (0.217, 0.783) (0.254, 0.746)

With 115 cases the two methods nearly agree for sensitivity. For specificity, close to 1, the Wilson interval is shifted downward, because a proportion near 1 is more uncertain below than above. For the 12 cases aged 18 to 39, both intervals are centered on 0.5 and very wide, Wald's wider still. With so few cases, neither says more than that the sensitivity in this group lies somewhere between about a quarter and three quarters.

The width of the sensitivity interval depends on the number of cases, and in a screening population cases are scarce. Solving $1.96\sqrt{p(1-p)/n} = h$ for $n$ gives the number of cases needed for a margin of error $h$. With $p = 0.470$ it is 383 cases for $h = 0.05$ and 96 for $h = 0.10$. At a prevalence of 5.8%, collecting 383 cases means screening about 6,578 adults. A study that wants a precise sensitivity in a low-prevalence setting must be large, or it must recruit cases separately. Recruiting them separately brings the problem of the next section.

What sensitivity depends on

Sensitivity and specificity do not depend on the prevalence. Does that make them fixed properties of the test? It does not. Sensitivity is an average over the people who have the condition. If the test detects some of them more easily than others, the average depends on who they are.

Splitting the 115 cases by their fasting glucose shows how strongly the case mix matters:

FPG of the case (mg/dL) Cases HbA1c $\ge 6.5\%$ Sensitivity (Wilson 95%) Median HbA1c
126–139 59 15 0.254 (0.161, 0.378) 6.1%
140–179 39 24 0.615 (0.459, 0.751) 6.8%
180 and over 17 15 0.882 (0.657, 0.967) 8.7%

HbA1c detects severe hyperglycemia reliably and mild hyperglycemia rarely. Half of the screen-detected cases are mild, so the overall sensitivity is low. In a population where undiagnosed diabetes is typically found late and severe, the same test would be far more sensitive.

By age the differences are smaller. Sensitivity is 0.500 (6 of 12) at ages 18 to 39, 0.439 (18 of 41) at 40 to 59 and 0.484 (30 of 62) at 60 and over. The intervals overlap heavily. Specificity is higher in the youngest group:

  • 0.9945 (727 of 731) at ages 18 to 39;
  • 0.9813 (576 of 587) at 40 to 59;
  • 0.9815 (532 of 542) at 60 and over.

Adults over 40 whose fasting glucose is below the threshold cross 6.5% more often than younger ones. The course notes write this dependence as $\mathrm{Sensitivity}(x) = P(T^+ \mid D, X = x)$ for a patient characteristic $x$, which is the right way to think of it. A reported sensitivity is an average over the case mix of the study that produced it.

Sensitivity of HbA1c at 6.5% with 95% Wilson intervals; the count of positives over cases is at the right. Read down the panels: by age group the sensitivity barely changes, by the fasting glucose of the case it rises from 0.25 to 0.88, and among adults already diagnosed whose FPG is 126 or more, a group of severe cases, it is 0.84.
Sensitivity of HbA1c at 6.5% with 95% Wilson intervals; the count of positives over cases is at the right. Read down the panels: by age group the sensitivity barely changes, by the fasting glucose of the case it rises from 0.25 to 0.88, and among adults already diagnosed whose FPG is 126 or more, a group of severe cases, it is 0.84.

Spectrum bias

The dependence becomes a bias when a study of a test recruits a different spectrum of patients from the one in which the test will be used. The cheapest design for a new test collects clear cases, often from a specialty clinic, and clear non-cases, often healthy volunteers, and tests both. This is a diagnostic case-control study. Ransohoff and Feinstein (1978) drew attention to the problem under the name of the "spectrum" of patients studied. It is now usually called spectrum bias.

The NHANES data can imitate the design.

  • Cases: the 264 adults who had already been diagnosed with diabetes and whose fasting glucose was still 126 or more (median FPG 165 mg/dL, median HbA1c 7.55%).
  • Controls: the 444 adults aged 18 to 39 with no diagnosis and an FPG below 100, the normal range.

Check your understanding. Compared with the screening population, will this design make HbA1c look more sensitive, less sensitive, or about the same? And its specificity?

Both look better. HbA1c at 6.5% is positive in 223 of the cases and negative in 443 of the controls:

Design Sensitivity Specificity
Screening population (1,975 adults) 0.470 (0.381, 0.560) 0.9866 (0.9802, 0.9909)
Severe cases and young healthy controls 0.845 (0.796, 0.883) 0.9977 (0.9874, 0.9996)

These cases imitate the design only roughly. Most adults with a diagnosis are being treated, and treatment lowers both glucose and HbA1c. So the cases are the diagnosed adults whose glucose stayed high, treated or not, and their HbA1c reflects months of treatment as well as the disease. The comparison shows the direction and rough size of the bias, not the accuracy a particular case-control study would report.

The test has not changed. The cases are easier to detect because they are further from the threshold. The controls are easier to clear because the hard non-cases, older adults with glucose just below the diabetic range, were left out. A reader who took the second row to a screening program would expect HbA1c to catch 84% of undiagnosed diabetes, and would find it catches under half. The defense is in the design. Evaluate a test in a consecutive or random series of the patients in whom it will be used, as NHANES does. Then report the spectrum of the patients studied.

Verification bias

A second bias arises when the reference standard is not applied to everyone. In clinical studies the reference is often costly or invasive (a biopsy, an angiogram), and clinicians tend to order it for patients whose index test was positive. If only the verified patients are analyzed, the cases that the test missed are mostly never found, and the test looks more sensitive than it is. This is verification bias, also called work-up bias.

NHANES measured FPG on everyone, so its data have no verification bias. The script imposes an invented verification rule on them to show the effect. Every adult with a positive HbA1c was "verified" by having the FPG reference recorded. Each adult with a negative HbA1c was verified with probability 0.10.

Check your understanding. Which way will the sensitivity computed from the verified adults move, and which way the specificity?

Sensitivity rises and specificity falls. Over 30 seeds, the sensitivity from the verified adults alone had a median of 0.900 (interquartile range 0.885 to 0.915), nearly double the true 0.470. Specificity fell to a median of 0.880 (0.875 to 0.885). All 25 false positives were kept, but only about a tenth of the 1,835 true negatives and of the 61 false negatives.

Can the bias be undone? Here is the idea. Suppose the decision to verify depends only on the test result. Then within each result the verified patients are a fair sample, and the whole population can be rebuilt from everyone's test results.

The formal version. Begg and Greenes (1983) gave the correction for this case. Within each test result, the verified patients are a random sample of all patients with that result. So $P(D \mid T^+)$ and $P(D \mid T^-)$ can be estimated from the verified patients, while $P(T^+)$ and $P(T^-)$ can be estimated from everyone tested. Bayes' theorem, from Conditional Probability and the Law of Total Probability, recombines them:

$$\mathrm{Se} = P(T^+ \mid D) = \frac{P(T^+)\,P(D \mid T^+)}{P(T^+)\,P(D \mid T^+) + P(T^-)\,P(D \mid T^-)},$$

and similarly for specificity with $D^c$ in place of $D$. In the simulation the corrected sensitivity had a median of 0.481 (0.438 to 0.518) and the corrected specificity 0.9866, centered on the true values. The spread in sensitivity comes from estimating $P(D \mid T^-)$ from about 190 verified negatives. The correction fails if verification also depends on something related to disease that the analyst does not see, such as how ill the patient looked.

Screening and confirmation

Back to the question of whether HbA1c is a good test. Which property matters more depends on what the test is for.

  • A screening test is applied to people without symptoms, to find those who should be examined further. A missed case may not be found again for years, while a false positive costs a confirmatory test. So screening favors sensitivity.
  • A confirmatory test is applied to people who screened positive, to decide on a diagnosis and treatment. A false positive now means a wrong diagnosis, so confirmation favors specificity. The ADA's rule that a diagnosis needs two abnormal results is a confirmation step of this kind (ADA, 2025).

On that view HbA1c at 6.5% is a poor screen and a reasonable confirmer. Its positive results are usually confirmed by fasting glucose, but it misses most cases. A lower HbA1c cut-off, such as the prediabetes boundary of 5.7%, trades specificity for sensitivity. ROC Curves for Diagnostic Tests traces that trade-off over every cut-off.

Corrections to the course notes

The unit's notes define sensitivity and specificity, the 2 by 2 table and the false positive rate as here. Their list of the factors that move sensitivity (disease stage, patient characteristics, laboratory conditions) agrees with the sections above, as does their call to account for spectrum bias. Their confidence interval for sensitivity is the Wald interval $p \pm 1.96\sqrt{p(1-p)/n}$, with $n$ the number of diseased people. That is correct as an approximation. Stated more precisely, it is unreliable for the small numbers of cases and the proportions near 1 typical of accuracy studies, where the Wilson interval is the better choice. The notes' formula for the probability that at least one of several tests detects a case, $1 - \prod(1 - \mathrm{Se}_i)$, holds when the tests' results are independent within the diseased. Likelihood Ratios for Diagnostic Tests shows how far that can fail for two real measurements.

Summary

Quantity Definition Here
Prevalence $(\mathrm{TP} + \mathrm{FN})/N$ 0.0582
Sensitivity $P(T^+ \mid D) = \mathrm{TP}/(\mathrm{TP} + \mathrm{FN})$ 0.470
Specificity $P(T^- \mid D^c) = \mathrm{TN}/(\mathrm{TN} + \mathrm{FP})$ 0.987
False negative rate $1 - \mathrm{Se}$ 0.530
False positive rate $1 - \mathrm{Sp}$ 0.013
Accuracy $\pi\,\mathrm{Se} + (1 - \pi)\,\mathrm{Sp}$ 0.956

What we have learned

Sensitivity and specificity describe a test by conditioning on the truth: HbA1c at 6.5% flags 47.0% of the adults whose fasting glucose is diabetic and clears 98.7% of the others. Because each is computed within one column, neither depends on prevalence, which is why accuracy, a prevalence-weighted average of the two, misleads. They are still not constants. They depend on the reference standard chosen and on the severity of the cases: sensitivity runs from 0.25 to 0.88 across glucose levels. Study designs that pick easy cases and controls, or verify only the positives, distort them further.

Neither number answers the patient's question. The adult with a positive HbA1c knows the result, not the disease status, and needs the probability conditioned the other way. Predictive Values and Bayes' Theorem reverses the conditioning, and shows how strongly the answer depends on how common the condition is.

In Python

from scipy.stats import binomtest

tp, fp, fn, tn = 54, 25, 61, 1835            # HbA1c >= 6.5% against FPG >= 126 mg/dL
for name, x, n in [('sensitivity', tp, tp + fn), ('specificity', tn, tn + fp)]:
    ci = binomtest(x, n).proportion_ci(confidence_level=0.95, method='wilson')
    print(name, round(x / n, 4), round(ci.low, 4), round(ci.high, 4))
# sensitivity 0.4696 0.3808 0.5603
# specificity 0.9866 0.9802 0.9909

Every number in this article is computed by tools/experiments/applied-statistics/sensitivity-and-specificity.py in the site's repository, from the NHANES files loaded by screendata.py; the intervals were checked against scipy 1.15.2.

References

  • American Diabetes Association Professional Practice Committee (2025). 2. Diagnosis and classification of diabetes: Standards of Care in Diabetes–2025. Diabetes Care, 48(Suppl. 1), S27–S49. https://doi.org/10.2337/dc25-S002 record · 3 pages
  • Begg, C. B., and Greenes, R. A. (1983). Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics, 39(1), 207–215. record
  • National Center for Health Statistics (2020a). NHANES 2017–2018 Laboratory Data: Plasma Fasting Glucose (GLU_J). Data documentation and codebook. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/Nchs/Data/Nhanes/Public/2017/DataFiles/GLU_J.htm record
  • National Center for Health Statistics (2020b). NHANES 2017–2018 Laboratory Data: Glycohemoglobin (GHB_J). Data documentation and codebook. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/Nchs/Data/Nhanes/Public/2017/DataFiles/GHB_J.htm record
  • National Center for Health Statistics (2020c). National Health and Nutrition Examination Survey 2017–2018: DEMO_J and DIQ_J data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 45 pages
  • Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages
  • Ransohoff, D. F., and Feinstein, A. R. (1978). Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England Journal of Medicine, 299(17), 926–930. record · 2 pages
  • Yerushalmy, J. (1947). Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Reports, 62(40), 1432–1449. record