Give the same positive HbA1c to an adult aged 18 to 39 and to an adult over 60. With the test's overall accuracy, Bayes' theorem puts the probability of a diabetic fasting glucose at 0.36 for the first and 0.80 for the second. The test is the same; only the patient's starting point differs. How much of the final probability is the test's doing, and can that contribution be stated once, for any patient?
In Predictive Values and Bayes' Theorem, that probability came from a formula that mixes the prevalence, which describes the patient's population, with the sensitivity and specificity, which describe the test. A likelihood ratio separates the two. It summarizes the evidence of a test result in a single number. Bayes' theorem in odds form then applies it to any patient by one multiplication. The odds of disease after the test are the odds before the test times the likelihood ratio.
First we define the ratios for a positive and a negative result. Then we prove the odds form of Bayes' theorem, check it by age group and draw it as Fagan's nomogram. Next come confidence intervals, and a ratio for each band of HbA1c values. Finally we ask when the ratios of two tests can be multiplied. The article assumes the odds and the log-scale intervals of Relative Risk and the Odds Ratio.
The examples continue with the unit's 1,975 adults from the National Health and Nutrition Examination Survey (NHANES) 2017 to 2018, of whom 115 have a fasting plasma glucose (FPG) of 126 mg/dL or more. Sensitivity and Specificity discusses the limits of that reference definition. The test is HbA1c, either positive at 6.5% or more or as its measured value.
Likelihood ratios of a positive and a negative result
The likelihood ratio of a test result $r$ is the probability of that result in people with the condition, divided by its probability in people without it:
$$\mathrm{LR}(r) = \frac{P(r \mid D)}{P(r \mid D^c)}.$$
A result with a likelihood ratio above 1 is more common among people with the condition, and so is evidence for it. Below 1, it is evidence against; exactly 1, no evidence. For a test with two outcomes the two ratios have names. The positive likelihood ratio is
$$\mathrm{LR}^+ = \frac{P(T^+ \mid D)}{P(T^+ \mid D^c)} = \frac{\mathrm{Se}}{1 - \mathrm{Sp}} = \frac{54/115}{25/1{,}860} = \frac{0.4696}{0.0134} = 34.94,$$
and the negative likelihood ratio is
$$\mathrm{LR}^- = \frac{P(T^- \mid D)}{P(T^- \mid D^c)} = \frac{1 - \mathrm{Se}}{\mathrm{Sp}} = \frac{61/115}{1{,}835/1{,}860} = 0.5377.$$
A positive HbA1c is 35 times as common among adults with a diabetic fasting glucose as among the others, which is strong evidence. A negative HbA1c is 0.54 times as common among them. That is weak evidence against, because more than half of them have a negative HbA1c.
Like sensitivity and specificity, likelihood ratios condition on the truth, and so they do not depend on the prevalence. They inherit all the dependence on the case mix that Sensitivity and Specificity described. Deeks and Altman (2004) describe likelihood ratios above 10 and below 0.1 as providing strong evidence to rule a diagnosis in or out in most circumstances. By that rule a positive HbA1c rules in, and a negative one does not rule out.
Bayes' theorem in odds form
Turning one number per result into a probability for a particular patient needs odds.
The odds of an event with probability $p$ are $p/(1 - p)$, and a probability is recovered from odds $o$ as $o/(1 + o)$, as in Relative Risk and the Odds Ratio. Before the test, the patient's probability of the condition is the pre-test probability $P(D)$, often the prevalence in a group the patient belongs to. After a result $r$ it is the post-test probability $P(D \mid r)$.
Here is the plan: write $P(D \mid r)$ and $P(D^c \mid r)$ by Bayes' theorem and divide, so that their common denominator cancels.
Proposition. For any test result $r$ with $P(r \mid D^c) > 0$,
$$\underbrace{\frac{P(D \mid r)}{P(D^c \mid r)}}_{\text{post-test odds}} = \underbrace{\frac{P(r \mid D)}{P(r \mid D^c)}}_{\mathrm{LR}(r)} \times \underbrace{\frac{P(D)}{P(D^c)}}_{\text{pre-test odds}}.$$
Proof. By Bayes' theorem, $P(D \mid r) = P(D)\,P(r \mid D)/P(r)$ and $P(D^c \mid r) = P(D^c)\,P(r \mid D^c)/P(r)$. Dividing the first by the second cancels $P(r)$, which is the awkward term of the probability form, and leaves the stated product. $\square$
The proposition separates the two sources of information exactly. Everything about the patient before testing is in the pre-test odds, and everything the test result adds is in the likelihood ratio.
Take an adult aged 60 or over. Among the NHANES adults of that age without a diagnosis, 62 of 604 meet the reference definition. That is a pre-test probability of 0.1026, and pre-test odds of $0.1026/0.8974 = 0.1144$.
Check your understanding. A positive HbA1c multiplies the odds by 34.94. Does it also multiply this adult's probability of the condition by about 35?
It cannot, since 35 times 0.1026 is more than 1. The odds become $0.1144 \times 34.94 = 3.996$, a post-test probability of $3.996/4.996 = 0.800$. The probability rises by a factor of 7.79, not 35. Odds and probability are close only when both are small. A negative result multiplies the odds by 0.5377, to 0.0615, a probability of 0.058.
The same calculation for each age group, against what was actually observed in it:
| Age | Pre-test probability | After positive: predicted | Observed | After negative: predicted | Observed |
|---|---|---|---|---|---|
| 18–39 | 0.0162 | 0.364 | 6/10 = 0.600 | 0.0087 | 6/733 = 0.0082 |
| 40–59 | 0.0653 | 0.709 | 18/29 = 0.621 | 0.0362 | 23/599 = 0.0384 |
| 60 and over | 0.1026 | 0.800 | 30/40 = 0.750 | 0.0579 | 32/564 = 0.0567 |
These are the predictions of the previous article reached by a different route, and they share its limitation. The likelihood ratios were computed over all ages, and the positive likelihood ratio is not constant across them. Specificity is higher among the young, so their own $\mathrm{LR}^+$ is $0.500/0.0055 = 91.4$, and the oldest group's is $0.484/0.0185 = 26.2$. The negative predictions are close in every group, because $\mathrm{LR}^-$ varies much less, from 0.50 to 0.57.
The log scale and Fagan's nomogram
Taking logarithms turns the multiplication into an addition:
$$\ln(\text{post-test odds}) = \ln(\text{pre-test odds}) + \ln \mathrm{LR}.$$
On the log-odds scale, each test result moves the patient by a fixed distance, $\ln \mathrm{LR}$, whatever the starting point. A positive HbA1c moves the log-odds up by $\ln 34.94 = 3.55$, and a negative one down by $0.62$.
Check your understanding. The positive result moves the young adult (pre-test 0.0162) and the adult over 60 (pre-test 0.1026) by the same distance on the log-odds scale. Which one gains more in probability?
The older adult: from 0.103 to 0.800, a gain of about 0.70, against 0.016 to 0.364 for the young adult, about 0.35. The same distance is a large change in probability near 0.5 and a small one near 0 or 1. That is why a strong test changes the management of a patient with an uncertain diagnosis, and barely affects one who is almost certainly well or almost certainly ill.
Fagan (1975) turned the addition into a nomogram. Here is the idea: a straight line across three equally spaced vertical scales passes through the middle one at the average of its heights on the outer two. So if the middle scale carries half the log of the likelihood ratio, the line adds.
The construction. Draw three parallel vertical scales at equal spacing.
- On the left scale, mark each pre-test probability $p$ at the height $-\ln[p/(1-p)]$.
- On the right, mark each post-test probability $q$ at the height $+\ln[q/(1-q)]$.
- On the middle scale, mark each likelihood ratio at the height $\tfrac12 \ln \mathrm{LR}$.
A line from the pre-test probability on the left, through the likelihood ratio in the middle, meets the right scale at height $2 \cdot \tfrac12 \ln \mathrm{LR} - (-\ln \text{pre-test odds})$. That is the log of the post-test odds. Deeks and Altman (2004) reproduce Fagan's nomogram. They use it to update a pre-test probability of 0.1 to a post-test probability of 0.7 for a result with a likelihood ratio of about 20.
Confidence intervals
How precise are the estimates 34.94 and 0.5377?
The positive likelihood ratio is the ratio of two independent proportions: the proportion positive among the 115 cases, and the proportion positive among the 1,860 non-cases. In the language of Relative Risk and the Odds Ratio, it is the relative risk of a positive result in the diseased group against the non-diseased group. Deeks and Altman (2004) note that likelihood ratios can be given confidence intervals in the same way as risk ratios. The log-scale interval of that article applies, with $x_1$ positives of $n_1$ cases and $x_0$ positives of $n_0$ non-cases:
$$\mathrm{SE}(\ln \widehat{\mathrm{LR}}) = \sqrt{\frac{1}{x_1} - \frac{1}{n_1} + \frac{1}{x_0} - \frac{1}{n_0}}, \qquad \widehat{\mathrm{LR}}\, e^{\pm 1.96\,\mathrm{SE}}.$$
For the positive likelihood ratio, $x_1 = 54$, $n_1 = 115$, $x_0 = 25$ and $n_0 = 1{,}860$. The standard error is 0.2220 and the interval is $(22.6, 54.0)$. For the negative likelihood ratio the negatives are counted instead ($x_1 = 61$, $x_0 = 1{,}835$); the standard error is 0.0878 and the interval $(0.453, 0.639)$. The interval for $\mathrm{LR}^+$ is wide in relative terms, because it rests on only 25 false positives. Even its lower end is above 10.
The course notes write the same variance as $(1 - \mathrm{Se})/(\mathrm{Se}\, n_1) + \mathrm{Sp}/[(1 - \mathrm{Sp})\, n_2]$. The two forms are equal, since $(1 - \mathrm{Se})/(\mathrm{Se}\,n_1) = 1/x_1 - 1/n_1$ when $\mathrm{Se} = x_1/n_1$, and similarly for the second term. The script evaluates both and gets 0.2220.
The diagnostic odds ratio
Dividing the two likelihood ratios gives the diagnostic odds ratio,
$$\mathrm{DOR} = \frac{\mathrm{LR}^+}{\mathrm{LR}^-} = \frac{\mathrm{Se}/(1 - \mathrm{Se})}{(1 - \mathrm{Sp})/\mathrm{Sp}} = \frac{\mathrm{TP} \cdot \mathrm{TN}}{\mathrm{FP} \cdot \mathrm{FN}} = \frac{54 \times 1{,}835}{25 \times 61} = 65.0.$$
It is the odds ratio of the 2 by 2 table, with Woolf's interval from the same article. It is a single summary of a test's discrimination, convenient for combining studies in a meta-analysis. But like any single number it hides the split between the two kinds of error. A test with sensitivity 0.9 and specificity 0.6, and one with sensitivity 0.6 and specificity 0.9, have the same DOR, 13.5, and very different uses.
Likelihood ratios for bands of a continuous result
Dichotomizing at 6.5% treats 6.5% and 9% alike, and 6.4% and 4.8% alike. Deeks and Altman (2004) point out that computing likelihood ratios does not require dichotomizing the result at all.
The table splits HbA1c into the bands the ADA uses: below 5.7%, the prediabetes range of 5.7% to 6.4%, and 6.5% or more. The two outer bands are divided further.
Check your understanding. An HbA1c from 6.0% to 6.4% is "negative" at the 6.5% cut-off. Is it evidence for the condition or against it?
For it. The band's likelihood ratio is 2.70:
| HbA1c band | Cases (share of 115) | Non-cases (share of 1,860) | LR (95% CI) | Post-test probability at prevalence 5.8% |
|---|---|---|---|---|
| below 5.7% | 17 (0.148) | 1,265 (0.680) | 0.217 (0.140, 0.337) | 0.013 |
| 5.7–5.9% | 12 (0.104) | 378 (0.203) | 0.513 (0.298, 0.884) | 0.031 |
| 6.0–6.4% | 32 (0.278) | 192 (0.103) | 2.70 (1.95, 3.73) | 0.143 |
| 6.5–6.9% | 26 (0.226) | 21 (0.011) | 20.0 (11.6, 34.5) | 0.553 |
| 7.0% and over | 28 (0.243) | 4 (0.002) | 113 (40, 317) | 0.875 |
The single positive likelihood ratio, 34.9, is an average of the two upper bands' ratios, 20.0 and 113. The weights are the shares of the false positives each band contains ($21/25$ and $4/25$), because both its numerator and its denominator are sums over the bands. Read band by band, an HbA1c of 6.6% carries about half the evidence that "positive" suggests, and one of 7.5% more than three times as much. The band from 6.0% to 6.4% raises the probability from 0.058 to 0.143.
Interval likelihood ratios keep the information that a cut-off throws away. Their cost is wider intervals, since each band has fewer people. ROC Curves for Diagnostic Tests shows that the likelihood ratio of a band is the slope of the ROC curve over that band.
Predictive values from likelihood ratios
Writing prevalence as odds turns the predictive values of the previous article into short formulas. With prevalence $\pi$, the post-test odds after a positive result are $\mathrm{LR}^+ \pi/(1 - \pi)$, so
$$\mathrm{PPV} = \frac{\pi\,\mathrm{LR}^+}{\pi\,\mathrm{LR}^+ + (1 - \pi)}, \qquad 1 - \mathrm{NPV} = \frac{\pi\,\mathrm{LR}^-}{\pi\,\mathrm{LR}^- + (1 - \pi)},$$
and so $\mathrm{NPV} = (1 - \pi)/[(1 - \pi) + \pi\,\mathrm{LR}^-]$. At the NHANES prevalence, these give 0.6835 and 0.9678, the values read from the table.
Combining two tests
If one test result multiplies the odds by $\mathrm{LR}_1$, a second result applied afterwards multiplies the new odds by its own likelihood ratio. It is tempting to write
$$\text{post-test odds} = \mathrm{LR}_2 \times \mathrm{LR}_1 \times \text{pre-test odds}.$$
The odds form of Bayes' theorem holds for any result, including the pair of results $(r_1, r_2)$. The pair's likelihood ratio is $P(r_1, r_2 \mid D)/P(r_1, r_2 \mid D^c)$. That ratio factors as $\mathrm{LR}_1 \times \mathrm{LR}_2$ exactly when
$$P(r_1, r_2 \mid D) = P(r_1 \mid D)\,P(r_2 \mid D) \quad\text{and}\quad P(r_1, r_2 \mid D^c) = P(r_1 \mid D^c)\,P(r_2 \mid D^c),$$
that is, when the two results are conditionally independent given the disease status.
This is a different and much stronger condition than independence of the two tests in the population. Two informative tests for the same disease are generally not independent in the population, since both are more often positive in the diseased. What the multiplication needs is that, among the diseased and separately among the non-diseased, one result carries no information about the other. If both tests respond to severity, or to the same nuisance factor, the condition fails, and multiplying counts the shared evidence twice. The same assumption underlies naive Bayes classification, which multiplies the likelihood ratios of many words.
The NHANES data allow a check with two inexpensive screening measurements: HbA1c of 5.7% or more (the prediabetes threshold), and a body mass index (BMI) of 30 or more (obesity). On the 1,942 adults with a BMI (113 cases):
- HbA1c at 5.7% has $\mathrm{LR}^+ = 2.652$ and $\mathrm{LR}^- = 0.221$;
- BMI at 30 has $\mathrm{LR}^+ = 1.741$ and $\mathrm{LR}^- = 0.582$.
Check your understanding. For an adult positive on both, will the product of the two likelihood ratios overstate or understate the evidence?
Overstate. The four combinations:
| HbA1c $\ge 5.7\%$ | BMI $\ge 30$ | Cases | Non-cases | Joint LR | Product of LRs |
|---|---|---|---|---|---|
| yes | yes | 61 | 260 | 3.797 | 4.617 |
| yes | no | 35 | 326 | 1.738 | 1.542 |
| no | yes | 10 | 400 | 0.405 | 0.385 |
| no | no | 7 | 843 | 0.134 | 0.129 |
The product overstates the evidence of two positive results by 22% (4.617 against 3.797). It also makes two negative results look slightly stronger evidence against than they are (0.129 against 0.134).
The reason is visible within each group. Among the cases, the odds ratio between the two tests is 1.22, and among the non-cases 1.68. The first rests on only 7 and 10 cases in the two cells where HbA1c is below 5.7%, so it is imprecise. Even among adults whose fasting glucose is on the same side of 126, those with a BMI of 30 or more are more likely to have an HbA1c of 5.7% or more. So the second result partly repeats the first. The overstatement is modest here because the association is moderate. For two tests of the same kind, such as two glucose measurements, it would be much larger.
Serial and parallel testing
The same failure affects the two standard rules for combining tests. Under serial testing, a person is positive only if both tests are positive. Under parallel testing, a person is positive if either is. With conditional independence:
- serial testing has sensitivity $\mathrm{Se}_1\mathrm{Se}_2$ and specificity $1 - (1 - \mathrm{Sp}_1)(1 - \mathrm{Sp}_2)$;
- parallel testing has sensitivity $1 - (1 - \mathrm{Se}_1)(1 - \mathrm{Se}_2)$, the course notes' formula for the probability of detection, and specificity $\mathrm{Sp}_1\mathrm{Sp}_2$.
For the two tests here, the sensitivities are 0.850 and 0.628 and the specificities 0.680 and 0.639:
| Rule | Sensitivity: observed | if independent | Specificity: observed | if independent |
|---|---|---|---|---|
| Serial (both positive) | 0.540 | 0.534 | 0.858 | 0.884 |
| Parallel (either positive) | 0.938 | 0.944 | 0.461 | 0.434 |
Serial testing gains less specificity, and parallel testing less sensitivity, than the independence formulas promise. Positively associated tests tend to make their errors together. The right way to combine two tests is to estimate the joint likelihood ratios of their combinations from data, as in the table above. Another is to fit a model such as logistic regression that allows for the association.
Corrections to the course notes
The unit's notes define the two likelihood ratios, the odds form of Bayes' theorem and interval likelihood ratios as here. Their scale for interpreting magnitudes (above 10 large, 5 to 10 moderate, 2 to 5 small, and the mirror images below 1) is consistent with the rule of thumb in Deeks and Altman (2004). Their variance for $\ln \mathrm{LR}^+$ is correct and equals the risk-ratio form. Their formula for the positive predictive value in terms of $\mathrm{LR}^+$ is correct.
Their formula for the negative predictive value, $\mathrm{Sp}/(\mathrm{LR}^- \pi + \mathrm{Sp})$, has the specificity where the probability of no disease, $1 - \pi$, belongs. Stated more precisely, it is $(1 - \pi)/[(1 - \pi) + \pi\,\mathrm{LR}^-]$, which follows from the PPV formula with $\mathrm{LR}^-$ in place of $\mathrm{LR}^+$. The two differ little when the condition is rare (0.9692 against 0.9678 here), but substantially when it is common. At a prevalence of 0.5 the notes' form gives 0.7859 against the correct 0.6503. The notes state that likelihood ratios multiply "if the tests are independent." The condition is conditional independence given disease status, and the HbA1c and BMI example shows the size of the error when it fails.
Summary
| Quantity | Formula | HbA1c $\ge 6.5\%$ |
|---|---|---|
| Positive likelihood ratio | $\mathrm{Se}/(1 - \mathrm{Sp})$ | 34.9 (22.6, 54.0) |
| Negative likelihood ratio | $(1 - \mathrm{Se})/\mathrm{Sp}$ | 0.538 (0.453, 0.639) |
| Odds form of Bayes' theorem | post-test odds $= \mathrm{LR} \times$ pre-test odds | |
| Diagnostic odds ratio | $\mathrm{LR}^+/\mathrm{LR}^- = \mathrm{TP}\cdot\mathrm{TN}/(\mathrm{FP}\cdot\mathrm{FN})$ | 65.0 |
| $\mathrm{SE}(\ln \mathrm{LR})$ | $\sqrt{1/x_1 - 1/n_1 + 1/x_0 - 1/n_0}$ | 0.2220, 0.0878 |
What we have learned
A likelihood ratio states once what a test result adds, separately from the patient. A positive HbA1c multiplies the odds of a diabetic fasting glucose by 34.9, and a negative one by 0.54. On the log-odds scale the update is an addition, which Fagan's nomogram draws with a ruler. The ratios are relative risks of a result, so they take the log-scale intervals of the previous unit. Giving each band of HbA1c its own ratio recovers evidence that a single cut-off discards. The ratios of two tests multiply only when the tests are conditionally independent, which HbA1c and BMI are not.
The bands raise a question of their own. If every HbA1c value carries evidence, where should the line between positive and negative go, and how well does the measurement separate the two groups over all possible lines? ROC Curves for Diagnostic Tests answers both.
In Python
import numpy as np
def lr_ci(x1, n1, x0, n0, z=1.96):
"""Likelihood ratio (x1/n1)/(x0/n0) with its log-scale 95% interval."""
lr = (x1 / n1) / (x0 / n0)
se = np.sqrt(1 / x1 - 1 / n1 + 1 / x0 - 1 / n0)
return lr, lr * np.exp(-z * se), lr * np.exp(z * se)
print(lr_ci(54, 115, 25, 1860)) # LR+ 34.94 (22.6, 54.0)
print(lr_ci(61, 115, 1835, 1860)) # LR- 0.538 (0.453, 0.639)
pre = 62 / 604 # adults aged 60 and over
odds = pre / (1 - pre) * 34.94
print(odds / (1 + odds)) # post-test probability 0.800
Every number in this article is computed by tools/experiments/applied-statistics/likelihood-ratios-for-diagnostic-tests.py in the site's repository, from the NHANES files loaded by screendata.py.
References
- Deeks, J. J., and Altman, D. G. (2004). Diagnostic tests 4: Likelihood ratios. BMJ, 329(7458), 168–169. record · 2 pages
- Fagan, T. J. (1975). Nomogram for Bayes's theorem (letter). New England Journal of Medicine, 293, 257. record
- National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: DEMO_J, DIQ_J, BMX_J, GLU_J and GHB_J data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 45 pages
- Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages