A study samples 200 obese and 200 non-obese adults from this unit's NHANES population and finds diabetes 2.188 times as common among the obese. In the whole population the relative risk is 1.794. How far from the population's value can such an estimate fall, and how is a comparison of two groups adjusted for a third variable such as age? The first question is answered by a confidence interval built on the logarithmic scale, which for this sample runs from 1.252 to 3.822. The second is answered by the Mantel–Haenszel estimators, which average the ratios found within the age groups.
This article continues Relative Risk and the Odds Ratio, which defines the two ratios and shows which study designs can estimate them, and it uses that article's notation throughout. A $2 \times 2$ table has the cells $a$ and $b$, the exposed with and without the outcome, and $c$ and $d$, the unexposed with and without it. The risks are $p_1 = a/(a + b)$ and $p_2 = c/(c + d)$. The relative risk is $\text{RR} = p_1/p_2$, and the odds ratio is $\text{OR} = ad/bc$. For obesity and diagnosed diabetes among the 5,107 adults the four cells are 432, 1,667, 345 and 2,663, so that $p_1 = 0.2058$, $p_2 = 0.1147$, $\text{RR} = 1.794$ and $\text{OR} = 2.000$.
First we find the approximate variance of the logarithm of a proportion and of an odds, from a limit theorem. Then we build the Katz interval for the relative risk and Woolf's interval for the odds ratio, work an example and compute how often the two intervals cover the true value. Finally we adjust a comparison for a confounder with the Mantel–Haenszel estimators, and give their standard errors and a test.
Confidence intervals on the log scale
A study reports an estimated ratio, and like any estimate it needs a confidence interval. For a ratio the interval is built on the logarithmic scale, for the following reason.
A ratio estimated from a sample is a ratio of random quantities, and its sampling distribution is skewed: it cannot fall below 0, but it can be large. The logarithm removes the skew. On the log scale, a ratio of 2 and a ratio of 1/2 are equally far from 1, at $\pm\ln 2$. And the log of a ratio is a difference, $\ln \widehat{\text{RR}} = \ln \hat{p}_1 - \ln \hat{p}_2$, to which the reasoning for differences applies. The standard intervals are therefore built for the log of the ratio and transformed back.
Here is the plan. First we find the variance of the log of a proportion and of the log of an odds. Then we add the two groups' variances, build a normal interval for the log ratio, and exponentiate its ends.
The delta method
Here is the idea in one sentence: near $\theta$ a smooth function is almost a straight line of slope $g'(\theta)$, and a straight line multiplies deviations by its slope, so it multiplies variances by the slope squared. If $\hat\theta$ is close to $\theta$, a first-order Taylor expansion gives $g(\hat\theta) \approx g(\theta) + g'(\theta)(\hat\theta - \theta)$, which suggests
$$\operatorname{Var}\big(g(\hat\theta)\big) \approx g'(\theta)^2 \operatorname{Var}(\hat\theta).$$
This is the delta method, already used for the log of a rate ratio in Crude Rates and Their Precision. The expansion is a heuristic. The precise result is a statement about limiting distributions.
Theorem. Let $Y_1, Y_2, \ldots$ be random variables and $\theta$ and $\sigma > 0$ numbers such that $\sqrt{n}\,(Y_n - \theta)$ converges in distribution to the normal distribution with mean 0 and variance $\sigma^2$. Let $g$ be a function that is defined at every value the $Y_n$ can take and differentiable at $\theta$, with $g'(\theta) \ne 0$. Then $\sqrt{n}\,\big(g(Y_n) - g(\theta)\big)$ converges in distribution to the normal distribution with mean 0 and variance $g'(\theta)^2\sigma^2$.
Convergence in distribution means that the probability of every interval approaches the normal probability of that interval as $n$ grows. The theorem is cited here, not proved: Casella and Berger (2002) state and prove it, and van der Vaart (1998) gives a version for functions of several variables. Its proof makes the Taylor step exact with a remainder term and shows, with Slutsky's theorem (also in Casella and Berger, 2002), that the remainder does not affect the limit. Both tools rest on the theory of convergence in distribution, which lies beyond this unit, whose rules concern events. The hypothesis is supplied by the central limit theorem: for a sample proportion $\hat{p} = x/n$ from $n$ independent trials with $0 < p < 1$, $\sqrt{n}\,(\hat{p} - p)$ converges in distribution to the normal distribution with variance $\sigma^2 = p(1 - p)$.
Four points of the statement matter in practice.
- The function must be defined at every value of the estimate, and the logarithm is not. A sample can have $x = 0$, where $\ln \hat{p}$ is not defined, so the theorem does not apply to $g = \ln$ as it stands. The repair is to give $g$ a value at 0: define $g(0)$ as any fixed number, for example 0, and $g(y) = \ln y$ for $y > 0$. This $g$ is defined everywhere and is still differentiable at $p > 0$, so the theorem applies to it. The choice of $g(0)$ does not matter. Two choices give statistics that differ only when $x = 0$, an event of probability $(1 - p)^n$, so the probabilities of any interval under the two choices differ by at most $(1 - p)^n$, which tends to 0. The same repair, at $x = 0$ and at $x = n$, serves for the log odds.
- It is a statement about the limiting distribution, not about the variance itself. Because of the samples with $x = 0$, $\ln \hat{p}$ has no variance in the strict sense, and with a value assigned at 0 its variance would depend on that value. The "variance" below is the variance of the normal distribution that approximates $\ln \hat{p}$ in large samples. The remedy for zero cells in practice is described under exact coverage below.
- The function must be differentiable at the true value, with a nonzero derivative. The logarithm is, at every $p > 0$, and the log odds at every $p$ strictly between 0 and 1.
- The approximation is good when the sample is large enough for $\hat{p}$ to stay where $g$ is nearly linear. How large is checked below by exact calculation.
The log of a proportion. With $\hat{p} = x/n$, the limiting variance is $\sigma^2 = p(1 - p)$, so the approximate variance of $\hat{p}$ is $p(1 - p)/n$. The function $g(p) = \ln p$ has $g'(p) = 1/p$. By the theorem, with $g$ given a value at 0 as above, $\ln \hat{p}$ is approximately normal with mean $\ln p$ and variance
$$\operatorname{Var}(\ln \hat{p}) \approx \frac{1}{p^2} \cdot \frac{p(1-p)}{n} = \frac{1 - p}{np}.$$
Estimating $p$ by $x/n$ gives $(1 - x/n)/x = 1/x - 1/n$. Replacing $p$ by its estimate inside the variance does not change the limiting normal distribution of the standardized statistic. By the law of large numbers (Grinstead and Snell, 1997, Theorem 8.2), $\hat{p}$ is within any fixed margin of $p$ with probability tending to 1, so the estimated variance divided by the true one tends to 1 in the same sense, and Slutsky's theorem then applies (Moment Generating Functions and Slutsky's Theorem proves this case of it, a factor that tends to 1).
Back to the running example. For a sample of $n = 200$ obese adults, with $p = 0.2058$, the formula gives $(1 - p)/(np) = 0.0193$. The exact variance of $\ln \hat{p}$, computed from the binomial distribution over the samples with at least one case, is 0.0200, about 4% larger. (A sample of 200 with no case has probability below $10^{-19}$.) With $n = 1{,}000$ the two agree to within 1%.
The log of the odds. For $g(p) = \ln[p/(1-p)] = \ln p - \ln(1 - p)$, $g'(p) = 1/p + 1/(1 - p) = 1/[p(1 - p)]$, so
$$\operatorname{Var}\big(\ln \tfrac{\hat{p}}{1 - \hat{p}}\big) \approx \frac{1}{p^2(1-p)^2} \cdot \frac{p(1-p)}{n} = \frac{1}{np(1-p)}.$$
Estimating $p$ by $x/n$ gives $n/[x(n - x)]$, and since $1/x + 1/(n - x) = (n - x + x)/[x(n - x)]$, this equals $1/x + 1/(n - x)$.
The Katz interval for the relative risk
The two groups are independent samples, so $\ln \hat{p}_1$ and $\ln \hat{p}_2$ are independent, and each is approximately normal by the theorem above. A difference of independent normal variables is normal, with variance the sum of the two variances: The Sampling Distribution of the Mean cites the proof for sums, and Random Variables, Expectation and Variance proves that the variances add. So $\ln \widehat{\text{RR}} = \ln \hat{p}_1 - \ln \hat{p}_2$ is approximately normal with mean $\ln \text{RR}$ and variance the sum of the two variances. Take $x = a$, $n = a + b$ in the exposed group and $x = c$, $n = c + d$ in the unexposed:
$$\operatorname{SE}(\ln \widehat{\text{RR}}) = \sqrt{\frac{1}{a} - \frac{1}{a + b} + \frac{1}{c} - \frac{1}{c + d}}.$$
The $100(1 - \alpha)\%$ interval for the relative risk is then
$$\exp\!\big(\ln \widehat{\text{RR}} \pm z_{1-\alpha/2}\operatorname{SE}\big) = \widehat{\text{RR}} \times e^{\pm z_{1-\alpha/2}\operatorname{SE}}.$$
This is the interval of Katz, Baptista, Azen and Pike (1978).
Woolf's interval for the odds ratio
The log odds ratio is the difference of the two groups' log odds, $\ln \widehat{\text{OR}} = \ln(a/b) - \ln(c/d)$. The two log odds are independent and approximately normal, so their variances add. By the formula for the log of the odds, the exposed group ($x = a$, $n - x = b$) contributes $1/a + 1/b$ and the unexposed group ($x = c$, $n - x = d$) contributes $1/c + 1/d$:
$$\operatorname{SE}(\ln \widehat{\text{OR}}) = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d}},$$
with the interval $\widehat{\text{OR}} \times e^{\pm z_{1-\alpha/2}\operatorname{SE}}$. Breslow and Day (1980) give it as the logit limits, from the normal approximation of Woolf (1955). The formula is symmetric in the four cells. So it is the same whether the table was sampled by exposure (rows) or by outcome (columns), which makes it the interval for case-control studies as well.
A worked example
Suppose a study samples 200 obese and 200 non-obese adults from the NHANES population and records diabetes. One such sample (seed 7) gave $a = 35$ diabetic among the obese and $c = 16$ among the others, so $b = 165$ and $d = 184$. The risks are $\hat{p}_1 = 0.175$ and $\hat{p}_2 = 0.080$.
Relative risk. $\widehat{\text{RR}} = 0.175/0.080 = 2.188$, and $\ln 2.188 = 0.7828$.
$$\operatorname{SE} = \sqrt{\frac{1}{35} - \frac{1}{200} + \frac{1}{16} - \frac{1}{200}} = 0.2847,$$
so the interval for $\ln \text{RR}$ is $0.7828 \pm 1.96 \times 0.2847 = (0.2247,\ 1.3408)$. Exponentiating, the 95% interval for the relative risk is $(1.252,\ 3.822)$.
Odds ratio. $\widehat{\text{OR}} = (35 \times 184)/(165 \times 16) = 2.439$, and $\ln 2.439 = 0.8917$.
$$\operatorname{SE} = \sqrt{\frac{1}{35} + \frac{1}{165} + \frac{1}{16} + \frac{1}{184}} = 0.3203,$$
so the interval for $\ln \text{OR}$ is $(0.2641,\ 1.5194)$, and for the odds ratio $(1.302,\ 4.570)$.
Both intervals contain the population values, 1.794 and 2.000, and both exclude 1.
Example (an interval that is not symmetric). The interval for the log relative risk is symmetric about 0.7828, but the interval for the relative risk, $(1.252,\ 3.822)$, is not symmetric about 2.188. It extends 0.936 below 2.188 and 1.635 above. Exponentiating stretches the upper end and compresses the lower one, so the asymmetry is inherited from the log scale. It matches the skew of the ratio's sampling distribution, which was the reason for working on the log scale in the first place.
scipy's relative_risk and odds_ratio (with kind='sample') give the same intervals to the digits shown. Each interval also gives a test. The statistics $\ln \widehat{\text{RR}}/\operatorname{SE} = 2.749$ and $\ln \widehat{\text{OR}}/\operatorname{SE} = 2.784$ are referred to the standard normal. Both are close to the pooled two-proportion statistic $z = 2.848$ of Comparing Two Proportions, and all give $p$-values below 0.01.
Woolf's interval also serves the simulated case-control sample of Relative Risk and the Odds Ratio, which had 86 obese and 64 non-obese cases, 56 obese and 94 non-obese controls, and an odds ratio of 2.256. Its Woolf interval is $2.256 \times e^{\pm 1.96 \times 0.2361} = (1.420,\ 3.583)$, with $\operatorname{SE} = \sqrt{1/86 + 1/56 + 1/64 + 1/94} = 0.2361$. No interval for the relative risk can be computed from it.
Exact coverage
Both intervals rest on large-sample approximations. An exact calculation shows how well they keep their 95% in samples of realistic size: at the two pairs of risks taken from the NHANES adults they cover at least 95% of the time, and at one other pair both fall slightly short. Because the counts in each group are binomial, the actual coverage can be computed exactly, as for the intervals of Confidence Intervals for a Proportion. Sum, over every pair of counts $(a, c)$, the probability of that pair whenever its interval contains the true ratio.
A table with a zero cell has no finite log ratio. A common remedy, used here, adds 0.5 to every cell of such a table. The table gives the coverage for $n$ adults per group, at the population risks of obesity and diabetes and at the much smaller risks of adults aged 18 to 39:
| $n$ per group | Katz, all ages | Woolf, all ages | Katz, ages 18–39 | Woolf, ages 18–39 | P(a zero cell), 18–39 |
|---|---|---|---|---|---|
| 25 | 0.980 | 0.972 | 0.999 | 0.999 | 0.853 |
| 50 | 0.969 | 0.964 | 0.995 | 0.995 | 0.636 |
| 100 | 0.957 | 0.953 | 0.985 | 0.985 | 0.334 |
| 200 | 0.954 | 0.954 | 0.974 | 0.974 | 0.099 |
| 400 | 0.951 | 0.951 | 0.970 | 0.970 | 0.010 |
| 1,000 | 0.951 | 0.950 | 0.956 | 0.956 | 0.000 |
At these risks the log intervals err on the side of caution. Their coverage is at or above 95%, approaching it as the samples grow. With rare outcomes they are very conservative until the groups are large. Most small samples then contain few or no cases, and the intervals are very wide.
These results are for two particular pairs of risks and for samples of fixed size per exposure group. They should not be read as a general guarantee.
- With a common outcome and no association, both risks equal to one half, Woolf's interval covers the true odds ratio with probability 0.943 at 50 adults per group, 0.944 at 100 and 0.949 at 200. Katz's interval covers the true relative risk with probability 0.960, 0.956 and 0.949 at the same three sizes. So Woolf's interval is slightly below 95% at all three, and Katz's at 200.
- Breslow and Day (1980) report Gart and Thomas's finding that the logit limits for the odds ratio are generally too narrow when calculated from small samples. They describe exact methods for small tables, based on the conditional distribution of the table given its margins.
Adjusting for a confounder
Back to smoking. Conditional Probability and the Law of Total Probability found that former smokers have more diabetes than adults who never smoked, and that part of the difference is due to age. In ratio terms:
| Age group | Former: diabetic / total | Never: diabetic / total | RR | OR |
|---|---|---|---|---|
| 18–39 | 6 / 205 | 21 / 1,145 | 1.596 | 1.614 |
| 40–59 | 49 / 300 | 115 / 929 | 1.319 | 1.382 |
| 60 and over | 216 / 677 | 258 / 944 | 1.167 | 1.246 |
| All ages (crude) | 271 / 1,182 | 394 / 3,018 | 1.756 | 1.981 |
The crude relative risk is 1.756, with 95% interval $(1.528,\ 2.019)$. It exceeds the ratio in every age group, because former smokers are concentrated at older ages. A single summary that adjusts for age combines the age-specific tables instead.
The idea is to average the age-specific ratios, with weights that favor the tables carrying the most information. The Mantel–Haenszel estimator of a common odds ratio across $K$ tables does this. With $a_k, b_k, c_k, d_k$ the cells and $n_k$ the total of table $k$, it is
$$\widehat{\text{OR}}_{\text{MH}} = \frac{\sum_k a_k d_k/n_k}{\sum_k b_k c_k/n_k}.$$
It is a weighted average of the table odds ratios $a_k d_k/(b_k c_k)$, with weights $b_k c_k/n_k$, when no table has $b_k c_k = 0$. Each numerator term is the table's odds ratio times its weight,
$$\frac{a_k d_k}{n_k} = \frac{a_k d_k}{b_k c_k} \cdot \frac{b_k c_k}{n_k},$$
and the denominator is the sum of the weights. Mantel and Haenszel (1959) proposed it. Breslow and Day (1980) recommend it for its simplicity, its consistency even with many small tables, and its immunity to zero cells.
The estimator of the same form for the relative risk is
$$\widehat{\text{RR}}_{\text{MH}} = \frac{\sum_k a_k (c_k + d_k)/n_k}{\sum_k c_k (a_k + b_k)/n_k}.$$
It is a weighted average of the table relative risks with weights $c_k(a_k + b_k)/n_k$, when no table has $c_k = 0$, by the same step: each numerator term is the table's relative risk times its weight,
$$\frac{a_k (c_k + d_k)}{n_k} = \frac{a_k/(a_k + b_k)}{c_k/(c_k + d_k)} \cdot \frac{c_k (a_k + b_k)}{n_k},$$
and the denominator is the sum of the weights. The same estimator is computed by riskratio_pooled in the StratifiedTable class of statsmodels, beside the Mantel–Haenszel odds ratio, oddsratio_pooled.
Example (where the summary can fall). The three age-specific relative risks are 1.596, 1.319 and 1.167. The Mantel–Haenszel relative risk can fall only between 1.167 and 1.596, and so cannot be near the crude 1.756, since a weighted average with positive weights cannot leave the range of what it averages. For the smoking data,
$$\widehat{\text{OR}}_{\text{MH}} = \frac{128.860}{99.955} = 1.289, \qquad \widehat{\text{RR}}_{\text{MH}} = \frac{167.917}{139.013} = 1.208.$$
Adjusting for age, in three broad groups, removes most of the crude association. The relative risk falls from 1.756 to 1.208, close to the 1.23 found by direct standardization in that article. The weights favor the oldest group, which has the most diabetic adults and so the most information: its weights are 107.752 of the 139.013 for the relative risk and 73.373 of the 99.955 for the odds ratio.
Intervals and a test for the summary
A summary estimate needs a standard error as much as a single table's does. The formulas below are cited, not derived: their derivations rest on the large-sample theory of sums over many tables. Each is checked here in three ways. The script confirms that each reduces to the single-table formula of this article when there is one table. It confirms that resampling the smoking tables 20,000 times reproduces both standard errors to within 2%. And the intervals are computed on the smoking data.
The odds ratio. Robins, Breslow and Greenland (1986) gave a variance estimator for $\ln \widehat{\text{OR}}_{\text{MH}}$ that is consistent both for a few large tables and for many small ones. With
$$P_k = \frac{a_k + d_k}{n_k}, \quad Q_k = \frac{b_k + c_k}{n_k}, \quad R_k = \frac{a_k d_k}{n_k}, \quad S_k = \frac{b_k c_k}{n_k}, \quad R = \sum_k R_k, \quad S = \sum_k S_k,$$
so that $\widehat{\text{OR}}_{\text{MH}} = R/S$, it is
$$\widehat{\operatorname{Var}}\big(\ln \widehat{\text{OR}}_{\text{MH}}\big) = \frac{\sum_k P_k R_k}{2R^2} + \frac{\sum_k (P_k S_k + Q_k R_k)}{2RS} + \frac{\sum_k Q_k S_k}{2S^2}.$$
With a single table the three terms are $P/(2R) + [P/(2R) + Q/(2S)] + Q/(2S) = P/R + Q/S$, which is
$$\frac{a + d}{ad} + \frac{b + c}{bc} = \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d},$$
Woolf's variance. For the smoking data the standard error is 0.0930, and the 95% interval is $1.289 \times e^{\pm 1.96 \times 0.0930} = (1.074,\ 1.547)$.
The relative risk. Greenland and Robins (1985) gave the corresponding estimator for $\ln \widehat{\text{RR}}_{\text{MH}}$:
$$\widehat{\operatorname{Var}}\big(\ln \widehat{\text{RR}}_{\text{MH}}\big) = \frac{\sum_k \big[(a_k + b_k)(c_k + d_k)(a_k + c_k) - a_k c_k n_k\big]/n_k^2}{\Big(\sum_k a_k (c_k + d_k)/n_k\Big)\Big(\sum_k c_k (a_k + b_k)/n_k\Big)}.$$
With a single table the numerator is $[ad(a + b) + bc(c + d)]/n^2$, since $(a + b)(c + d)(a + c) - ac(a + b + c + d) = ad(a + b) + bc(c + d)$ on expanding. The denominator is $ac(a + b)(c + d)/n^2$. The quotient is
$$\frac{d}{c(c + d)} + \frac{b}{a(a + b)} = \frac{1}{c} - \frac{1}{c + d} + \frac{1}{a} - \frac{1}{a + b},$$
the variance of the Katz interval. For the smoking data the standard error is 0.0686, and the 95% interval is $1.208 \times e^{\pm 1.96 \times 0.0686} = (1.056,\ 1.382)$. Both adjusted intervals exclude 1. The adjusted interval for the relative risk lies entirely below the crude one, $(1.528,\ 2.019)$.
The Mantel–Haenszel test. Mantel and Haenszel (1959) also gave a test of the hypothesis that exposure and outcome are unrelated in every table (a common odds ratio of 1). Given the four margins of table $k$, that hypothesis makes $a_k$ a count with mean and variance
$$E_k = \frac{(a_k + b_k)(a_k + c_k)}{n_k}, \qquad V_k = \frac{(a_k + b_k)(c_k + d_k)(a_k + c_k)(b_k + d_k)}{n_k^2\,(n_k - 1)},$$
those of the hypergeometric distribution, the distribution behind Fisher's exact test in Comparing Two Proportions. Breslow and Day (1980) present the test for case-control data, and the script confirms the two formulas against the exact distribution for the youngest age group. The statistic compares the total of the $a_k$ with the total expected:
$$z_{\text{MH}} = \frac{\sum_k a_k - \sum_k E_k}{\sqrt{\sum_k V_k}},$$
referred to the standard normal distribution, or its square to the chi-square distribution with one degree of freedom. Mantel and Haenszel subtracted $\tfrac12$ from the absolute value of the numerator as a continuity correction. For the smoking data, 271 former smokers have diabetes, against $4.10 + 40.03 + 197.96 = 242.10$ expected, with $\sum_k V_k = 3.41 + 26.24 + 81.62 = 111.28$:
$$z_{\text{MH}} = \frac{271 - 242.10}{\sqrt{111.28}} = 2.74,$$
and 2.69 with the correction. The two-sided $p$-values are 0.006 and 0.007. After adjustment for age the association is much weaker than the crude one but still distinguishable from none. With one table, the uncorrected statistic is the pooled two-proportion $z$ of Comparing Two Proportions multiplied by $\sqrt{(n - 1)/n}$, which the script confirms on three tables of this unit.
Three cautions apply. The first two carry over from the article on conditional probability.
- The data are cross-sectional, so they cannot show whether smoking preceded diabetes.
- Age is not the only difference between former and never smokers.
- A summary is meaningful only if the ratio is roughly constant across the groups. Here it falls with age, from about 1.6 to 1.2, and reporting the age-specific ratios is more informative than any single number.
Adjusting for several variables at once, continuous ones included, is the work of Logistic Regression. Its coefficients are log odds ratios, the measure's most important use today.
Summary
| Quantity | Formula |
|---|---|
| Katz interval | $\widehat{\text{RR}}\, e^{\pm z\sqrt{1/a - 1/(a+b) + 1/c - 1/(c+d)}}$ |
| Woolf interval | $\widehat{\text{OR}}\, e^{\pm z\sqrt{1/a + 1/b + 1/c + 1/d}}$ |
| Mantel–Haenszel | $\widehat{\text{OR}}_{\text{MH}} = \sum a_k d_k/n_k \big/ \sum b_k c_k/n_k$; $\widehat{\text{RR}}_{\text{MH}} = \sum a_k (c_k + d_k)/n_k \big/ \sum c_k (a_k + b_k)/n_k$ |
| Mantel–Haenszel test | $z_{\text{MH}} = \big(\sum a_k - \sum E_k\big)\big/\sqrt{\sum V_k}$ |
Worked problems
These problems are posed on the unit's own data: tables of the 5,107 NHANES adults and a simulated sample that the script draws from them. Each is solved from the results above; none is taken from the course text. The script recomputes every number.
Problem 1. The case-control study of Relative Risk and the Odds Ratio with three controls per case (the same seed 7) drew 150 diabetic adults, of whom 86 were obese, and 450 non-diabetic adults, of whom 181 were obese. Estimate the odds ratio with a 95% interval. Then compute the naive "relative risk" from the same table, and say what it estimates.
The odds ratio. The odds of obesity are $86/64 = 1.3438$ among the cases and $181/269 = 0.6729$ among the controls, so
$$\widehat{\text{OR}} = \frac{86 \times 269}{64 \times 181} = \frac{23{,}134}{11{,}584} = 1.997, \qquad \ln 1.997 = 0.6917.$$
Woolf's standard error uses the four cells:
$$\operatorname{SE} = \sqrt{\frac{1}{86} + \frac{1}{64} + \frac{1}{181} + \frac{1}{269}} = \sqrt{0.01163 + 0.01563 + 0.00552 + 0.00372} = \sqrt{0.03650} = 0.1910.$$
The interval for $\ln \text{OR}$ is $0.6917 \pm 1.96 \times 0.1910 = (0.3173,\ 1.0661)$, and exponentiating gives $(1.373,\ 2.904)$ for the odds ratio. It contains the population value, 2.000, and it is narrower than the interval from 150 controls, $(1.420,\ 3.583)$, because the two cells of controls are larger and their reciprocals smaller.
The naive ratio. Reading the table as if it were a cohort, the share of cases among the obese is $86/(86 + 181) = 0.3221$ and among the others $64/(64 + 269) = 0.1922$, a ratio of 1.676. It estimates no population quantity. With one control per case the same sample of cases gave 1.495, and the expected values under the two designs are 1.409 and 1.675, as that article reports. The population relative risk is 1.794.
Problem 2. The crude table for smoking and diabetes has 271 diabetic adults among 1,182 former smokers and 394 among 3,018 never smokers. Treating the two groups as independent samples, derive the 95% interval for the relative risk quoted in the article, and find the odds ratio with its interval.
The table is $a = 271$, $b = 911$, $c = 394$, $d = 2{,}624$, with $\hat{p}_1 = 0.2293$ and $\hat{p}_2 = 0.1306$.
Relative risk. $\widehat{\text{RR}} = 0.2293/0.1306 = 1.756$ and $\ln \widehat{\text{RR}} = 0.5632$ (from the unrounded ratio 1.7562). The Katz standard error is
$$\operatorname{SE} = \sqrt{\frac{1}{271} - \frac{1}{1{,}182} + \frac{1}{394} - \frac{1}{3{,}018}} = \sqrt{0.003690 - 0.000846 + 0.002538 - 0.000331} = \sqrt{0.005051} = 0.0711.$$
The interval for $\ln \text{RR}$ is $0.5632 \pm 1.96 \times 0.0711 = (0.4239,\ 0.7024)$, and exponentiating gives $(1.528,\ 2.019)$, the interval quoted above.
Odds ratio. $\widehat{\text{OR}} = (271 \times 2{,}624)/(911 \times 394) = 711{,}104/358{,}934 = 1.981$ and $\ln \widehat{\text{OR}} = 0.6837$. Woolf's standard error is
$$\operatorname{SE} = \sqrt{\frac{1}{271} + \frac{1}{911} + \frac{1}{394} + \frac{1}{2{,}624}} = \sqrt{0.003690 + 0.001098 + 0.002538 + 0.000381} = \sqrt{0.007707} = 0.0878.$$
The interval for $\ln \text{OR}$ is $(0.5116,\ 0.8557)$, and for the odds ratio $(1.668,\ 2.353)$. The odds ratio is farther from 1 than the relative risk, as the identity $\text{OR} = \text{RR}\,(1 - p_2)/(1 - p_1)$ of Relative Risk and the Odds Ratio requires, and its standard error is larger. On the log scale the two estimates are about equally many standard errors from zero: $0.5632/0.0711 = 7.92$ and $0.6837/0.0878 = 7.79$.
Problem 3. Obesity and diabetes, within the three age groups:
| Age group | Obese: diabetic / total | Not obese: diabetic / total | RR | OR |
|---|---|---|---|---|
| 18–39 | 22 / 640 | 12 / 1,040 | 2.979 | 3.050 |
| 40–59 | 140 / 697 | 69 / 860 | 2.503 | 2.881 |
| 60 and over | 270 / 762 | 264 / 1,108 | 1.487 | 1.754 |
| All ages (crude) | 432 / 2,099 | 345 / 3,008 | 1.794 | 2.000 |
Find the Mantel–Haenszel odds ratio and relative risk with their 95% intervals, carry out the Mantel–Haenszel test, and compare the adjusted ratios with the crude ones.
The terms. For each table, with $n_k$ its total:
| Age group | $n_k$ | $a_k d_k/n_k$ | $b_k c_k/n_k$ | $a_k(c_k + d_k)/n_k$ | $c_k(a_k + b_k)/n_k$ | $E_k$ | $V_k$ |
|---|---|---|---|---|---|---|---|
| 18–39 | 1,680 | 13.462 | 4.414 | 13.619 | 4.571 | 12.95 | 7.86 |
| 40–59 | 1,557 | 71.124 | 24.684 | 77.328 | 30.888 | 93.56 | 44.77 |
| 60 and over | 1,870 | 121.861 | 69.459 | 159.979 | 107.576 | 217.60 | 92.16 |
| Sum | 206.447 | 98.557 | 250.926 | 143.036 | 324.11 | 144.79 |
For the first row, for example, $a_1 d_1/n_1 = 22 \times 1{,}028/1{,}680 = 13.462$ and $E_1 = 640 \times 34/1{,}680 = 12.95$, where 34 is the number of diabetic adults in the age group.
The estimates.
$$\widehat{\text{OR}}_{\text{MH}} = \frac{206.447}{98.557} = 2.095, \qquad \widehat{\text{RR}}_{\text{MH}} = \frac{250.926}{143.036} = 1.754.$$
The intervals. The Robins–Breslow–Greenland formula gives a standard error of 0.0835 for $\ln \widehat{\text{OR}}_{\text{MH}}$, so the interval is $2.095 \times e^{\pm 1.96 \times 0.0835} = (1.778,\ 2.467)$. The Greenland–Robins formula gives 0.0634 for $\ln \widehat{\text{RR}}_{\text{MH}}$, and the interval $1.754 \times e^{\pm 1.96 \times 0.0634} = (1.549,\ 1.986)$.
The test. In all, 432 obese adults have diabetes, against 324.11 expected if obesity and diabetes were unrelated within every age group:
$$z_{\text{MH}} = \frac{432 - 324.11}{\sqrt{144.79}} = \frac{107.89}{12.03} = 8.97.$$
A standard normal variable exceeds 8.97 in absolute value with a probability far below any conventional level, so the hypothesis of no association within age groups is rejected.
The comparison. Adjustment for age changes little here: the odds ratio moves from 2.000 to 2.095 and the relative risk from 1.794 to 1.754. This is unlike smoking, where the relative risk fell from 1.756 to 1.208. Problem 4 of Conditional Probability and the Law of Total Probability explains the difference. When the two exposure groups have the same age distribution, the crude relative risk is itself a weighted average of the age-specific ones, and age cannot distort it. The two groups have the same age distribution exactly when the exposed share is the same at every age. Former smokers were much older than never smokers, but the obese share is similar in the three age groups: $640/1{,}680 = 38.1\%$, $697/1{,}557 = 44.8\%$ and $762/1{,}870 = 40.7\%$. The summary still hides something. The age-specific odds ratios fall from 3.05 to 1.75, so the association is not the same at every age, and the three ratios say more than either summary.
What we have learned
An estimated ratio gets its confidence interval on the log scale, where a ratio becomes a difference and the skew of its sampling distribution is removed. The approximate variances $1/x - 1/n$ for the log of a proportion and $1/x + 1/(n - x)$ for the log of an odds give the Katz interval for the relative risk and Woolf's interval for the odds ratio. A sample of 200 adults per group gave a relative risk of 2.188 with interval $(1.252,\ 3.822)$ and an odds ratio of 2.439 with interval $(1.302,\ 4.570)$. Exact calculation puts the coverage of both intervals at or above 95% at the NHANES risks, and slightly below it at one other pair of risks. Mantel–Haenszel weights adjust either ratio for a confounder: for former against never smokers the relative risk falls from 1.756 to 1.208 after adjustment for age, with interval $(1.056,\ 1.382)$.
The next unit, Screening and Diagnostic Tests, asks the same questions of a medical test. A test's accuracy is described by conditional probabilities given the true state, and the patient needs the reverse, which Bayes' theorem supplies. Its likelihood ratios turn out to be relative risks of a positive result, with the same log-scale intervals. The unit begins with Sensitivity and Specificity.
In Python
from scipy.stats.contingency import relative_risk, odds_ratio
a, b, c, d = 35, 165, 16, 184 # 200 obese, 200 not obese
rr = relative_risk(a, a + b, c, c + d)
print(rr.relative_risk, rr.confidence_interval()) # 2.188, (1.252, 3.822)
orr = odds_ratio([ [a, b], [c, d] ], kind='sample')
print(orr.statistic, orr.confidence_interval()) # 2.439, (1.302, 4.570)
scipy's default kind='conditional' returns the conditional maximum likelihood estimate of the odds ratio, which differs slightly from $ad/bc$. Every number in this article is computed by tools/experiments/applied-statistics/relative-risk-and-odds-ratio.py in the site's repository, the script it shares with Relative Risk and the Odds Ratio, from the NHANES files loaded by probdata.py; the worked intervals were checked against scipy 1.17.1.
References
- Breslow, N. E., and Day, N. E. (1980). Statistical Methods in Cancer Research, Volume I: The Analysis of Case-Control Studies. IARC Scientific Publications No. 32. International Agency for Research on Cancer. record · 4 pages
- Casella, G., and Berger, R. L. (2002). Statistical Inference (2nd ed.). Duxbury. Slutsky's theorem and the limiting distribution of a differentiable function of an estimator. record · 3 pages
- Greenland, S., and Robins, J. M. (1985). Estimation of a common effect parameter from sparse follow-up data. Biometrics, 41(1), 55–68. record
- Grinstead, C. M., and Snell, J. L. (1997). Introduction to Probability (2nd rev. ed.), §8.1 (Theorem 8.2, the law of large numbers). American Mathematical Society. record · 13 pages
- Katz, D., Baptista, J., Azen, S. P., and Pike, M. C. (1978). Obtaining confidence intervals for the risk ratio in cohort studies. Biometrics, 34(3), 469–474. record
- Mantel, N., and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719–748. record
- National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: DEMO_J, BPX_J, BPQ_J, BMX_J, DIQ_J and SMQ_J data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 52 pages
- Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 48 pages
- Robins, J., Breslow, N., and Greenland, S. (1986). Estimators of the Mantel–Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42(2), 311–323. record
- SciPy documentation.
scipy.stats.contingency.relative_riskandscipy.stats.contingency.odds_ratio. https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.contingency.odds_ratio.html record - statsmodels documentation.
statsmodels.stats.contingency_tables.StratifiedTable. https://www.statsmodels.org/stable/generated/statsmodels.stats.contingency_tables.StratifiedTable.html record - van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press. The same result for functions of several variables. record · 4 pages
- Woolf, B. (1955). On estimating the relation between blood group and disease. Annals of Human Genetics, 19(4), 251–253. record