The Mantel–Haenszel Test

The theory under the Mantel–Haenszel test and estimator for a series of 2 x 2 tables. The noncentral hypergeometric distribution of a table given its margins, derived; the test as the conditional score test, with the exact variance; its normal limit for many small strata, proved by Lyapunov's condition; matched pairs, where the test is McNemar's, the estimator is the conditional maximum likelihood estimate and the unconditional one converges to its square; the estimator's unbiased estimating equation and its consistency for few large strata and for many small ones; its variance; and tests of homogeneity and trend in the odds ratio. Worked on smoking, obesity and diabetes among NHANES adults in strata from three age groups to single matched pairs, and checked against Breslow and Day's published analysis, with worked problems on an exact conditional estimate, Cochran's variance, test-based limits and the one-table case.

Among the NHANES adults of 2017 to 2018, former smokers have diagnosed diabetes more often than adults who never smoked, and they are also older. Intervals and Adjustment for Relative Risks and Odds Ratios adjusts the comparison for age in three groups and finds a Mantel–Haenszel odds ratio of 1.289 with a test statistic of 2.74. That page states the test's mean and variance and confirms them by computation, and it cites the variance of the estimator. Why is the variance of each table the hypergeometric one, why may the test be trusted when the strata are many and small, and what happens to the estimate when the three age groups are refined to single years of age and sex, or to matched pairs?

The Mantel–Haenszel test compares the total count of exposed people with the outcome, over a series of $2 \times 2$ tables, with the total expected if exposure and outcome were unrelated within every table. Mantel and Haenszel (1959) proposed it, with a summary relative risk, for retrospective studies "subclassified by factors controlled in the analysis" (their abstract).

First we derive the distribution of a table given its margins, and show that the test is the score test of the likelihood that this distribution defines. Then we prove that the statistic is approximately normal when the strata are many and small, and that with one matched pair per stratum it is McNemar's test. Next we turn to the estimator: why it is consistent in both kinds of limit, why the ordinary maximum likelihood estimate is not, and how its variance is estimated. Finally we test whether a common odds ratio exists at all. The article assumes the notation and results of the course page above, and the score test of Likelihood Ratio, Wald and Score Tests.

The setting

The data are $K$ tables, one per stratum. In table $k$, $a_k$ and $b_k$ are the exposed with and without the outcome, $c_k$ and $d_k$ the unexposed with and without it, and $n_k = a_k + b_k + c_k + d_k$. Write the margins as

$$n_{1k} = a_k + b_k, \quad n_{0k} = c_k + d_k, \quad m_{1k} = a_k + c_k, \quad m_{0k} = b_k + d_k,$$

the exposed and unexposed totals and the totals with and without the outcome. The odds ratio of table $k$ is $\psi_k$, and the hypothesis of a common odds ratio is $\psi_k = \psi$ for every $k$.

The course page gives three things, which this article takes as known: the estimator

$$\hat\psi_{\text{MH}} = \frac{R}{S}, \qquad R = \sum_k \frac{a_k d_k}{n_k}, \quad S = \sum_k \frac{b_k c_k}{n_k};$$

the test statistic

$$z_{\text{MH}} = \frac{\sum_k a_k - \sum_k E_k}{\sqrt{\sum_k V_k}}, \qquad E_k = \frac{n_{1k}m_{1k}}{n_k}, \quad V_k = \frac{n_{1k}n_{0k}m_{1k}m_{0k}}{n_k^2(n_k - 1)};$$

and the variance estimator of Robins, Breslow and Greenland (1986) for $\ln\hat\psi_{\text{MH}}$. For former against never smokers in three age groups, $\hat\psi_{\text{MH}} = 1.289$ and $z_{\text{MH}} = 2.74$. For obese against non-obese adults, $\hat\psi_{\text{MH}} = 2.095$ and $z_{\text{MH}} = 8.97$.

The distribution given the margins

The test conditions on all four margins of every table. Here is why that is useful, in one sentence: given the margins, the distribution of a table depends on the odds ratio and on nothing else, so the many nuisance parameters, one baseline risk per stratum, drop out.

Theorem 1 (the noncentral hypergeometric distribution). Let $a \sim \text{Binomial}(n_1, p_1)$ and $c \sim \text{Binomial}(n_0, p_0)$ be independent, with $0 < p_0, p_1 < 1$, and let $\psi = p_1(1 - p_0)/[p_0(1 - p_1)]$ be the odds ratio. Given $a + c = m_1$,

$$P(a = u \mid a + c = m_1) = \frac{\binom{n_1}{u}\binom{n_0}{m_1 - u}\psi^u}{\sum_v \binom{n_1}{v}\binom{n_0}{m_1 - v}\psi^v},$$

for $\max(0, m_1 - n_0) \le u \le \min(n_1, m_1)$, the sum running over the same range.

Proof. By independence,

$$P(a = u,\ c = m_1 - u) = \binom{n_1}{u}p_1^u(1 - p_1)^{n_1 - u}\binom{n_0}{m_1 - u}p_0^{m_1 - u}(1 - p_0)^{n_0 - m_1 + u}.$$

Collect the factors that depend on $u$: $p_1^u(1 - p_1)^{-u}p_0^{-u}(1 - p_0)^{u} = \psi^u$. What remains, $(1 - p_1)^{n_1}p_0^{m_1}(1 - p_0)^{n_0 - m_1}$, does not depend on $u$. So $P(a = u, c = m_1 - u) = C\binom{n_1}{u}\binom{n_0}{m_1 - u}\psi^u$ for a constant $C$. The conditional probability is this divided by its sum over the possible $u$, and $C$ cancels. $\blacksquare$

Three remarks complete the picture.

  • Case-control sampling gives the same distribution. In a case-control study the outcome totals $m_1, m_0$ are fixed by design and the exposure counts are binomial within them. Then the same argument conditions on the exposure totals. The two weights agree up to a constant: $\binom{n_1}{u}\binom{n_0}{m_1 - u} = n_1!\,n_0!/(a!\,b!\,c!\,d!)$ and the case-control weight is $m_1!\,m_0!/(a!\,b!\,c!\,d!)$, where $a = u$ and the other cells follow from the margins. Their ratio $n_1!\,n_0!/(m_1!\,m_0!)$ is fixed once the margins are, so it cancels from the conditional probability. Breslow and Day (1980, §4.2, equation 4.2) make the same point: the formula is unchanged on interchanging the two pairs of margins.
  • At $\psi = 1$ it is the hypergeometric distribution of Fisher's exact test, whose mean $E_k$ and variance $V_k$ The Chi-Square Test of Independence derives. The variance has $n_k - 1$ in its denominator because the count is drawn without replacement.
  • The odds ratio is the only parameter left. The baseline risk $p_0$ of the stratum has disappeared. That is what makes the conditional distribution suitable for many strata, each with its own baseline.

For the youngest age group of the smoking data, 6 of 205 former smokers and 21 of 1,145 never smokers have diabetes, so $m_1 = 27$ and $a$ can range from 0 to 27. At $\psi = 1$ the observed $a = 6$ has probability 0.1146, and the mean and variance are $E_1 = 4.10$ and $V_1 = 3.41$, the values on the course page.

Example (the conditional estimate of one table). The value of $\psi$ that maximizes the conditional probability of the observed $a$ is the conditional maximum likelihood estimate. For this table it is 1.613, against the table's odds ratio $ad/bc = 1.614$. The two differ because the conditional likelihood is not the binomial one. In large tables the difference is negligible; Problem 1 works a small table in which it is not.

The test as a conditional score test

Here is the plan. The product over strata of the conditional probabilities is a likelihood for $\beta = \ln\psi$ alone. Its derivative at $\beta = 0$ is the numerator of $z_{\text{MH}}$, and its information there is the denominator. So the Mantel–Haenszel test is the score test of this likelihood.

The formal version. With a common $\psi = e^\beta$, the conditional log-likelihood of the $K$ independent tables is

$$\ell(\beta) = \sum_k \Big[a_k\beta + \ln\tbinom{n_{1k}}{a_k}\tbinom{n_{0k}}{m_{1k} - a_k} - \ln Z_k(\beta)\Big], \qquad Z_k(\beta) = \sum_u \tbinom{n_{1k}}{u}\tbinom{n_{0k}}{m_{1k} - u}e^{u\beta}.$$

Differentiate $\ln Z_k$. Since $Z_k'(\beta) = \sum_u u\,w_u e^{u\beta}$, with $w_u$ the binomial weights, $Z_k'/Z_k$ is the mean of $u$ under the conditional distribution: $E_\beta(a_k)$. Differentiating again, $(\ln Z_k)'' = Z_k''/Z_k - (Z_k'/Z_k)^2 = E_\beta(a_k^2) - E_\beta(a_k)^2 = \operatorname{Var}_\beta(a_k)$. Therefore

$$\ell'(\beta) = \sum_k \big[a_k - E_\beta(a_k)\big], \qquad -\ell''(\beta) = \sum_k \operatorname{Var}_\beta(a_k).$$

At $\beta = 0$ these are $\sum_k(a_k - E_k)$ and $\sum_k V_k$, and the score statistic $\ell'(0)/\sqrt{-\ell''(0)}$ is $z_{\text{MH}}$. For the smoking data the script confirms by numerical differentiation of $\ell$ that the score at 0 is $271 - 242.10 = 28.90$ and the information 111.28.

Three things follow.

  • The variance is the exact conditional one. Cochran (1954) proposed the same comparison of observed and expected totals, with the variance $n_{1k}n_{0k}m_{1k}m_{0k}/n_k^3$ in each table. Breslow and Day (1980, §4.4, p. 139) write that the summary test "was developed by Cochran (1954) and by Mantel and Haenszel (1959), with the latter suggesting use of the exact variances". The two differ by the factor $(n_k - 1)/n_k$ in each table, which matters only when tables are small. Problem 2 compares them.
  • Setting the score to zero gives the conditional maximum likelihood estimate of a common odds ratio: the $\psi$ at which the total observed equals the total expected, $\sum_k a_k = \sum_k E_\psi(a_k)$ (Breslow and Day, 1980, equation 4.25). In general it has no closed form and is found numerically; matched pairs (Theorem 3) and very small tables (Problem 1) are exceptions.
  • The test is one-directional. A score test of $\beta = 0$ is most sensitive to a common shift of every table in the same direction. Tables with odds ratios on both sides of 1 can cancel, and the test can miss an association that reverses between strata. It tests "no association in any table" against "a common association".

Why the statistic is approximately normal

Under the hypothesis, and given the margins, the counts $a_1, \ldots, a_K$ are independent, with means $E_k$ and variances $V_k$. The numerator of $z_{\text{MH}}$ is a sum of independent centered terms that are not identically distributed. Its normal limit can arrive in two ways.

  • A few large strata. When every cell of every table is large, each $a_k$ is itself approximately normal (Breslow and Day, 1980, §4.3), and a sum of independent approximately normal terms is approximately normal.
  • Many small strata. When the tables stay small but their number grows, no single $a_k$ is near normal, and the limit has to come from the number of terms.

The second case is the one that matters for fine stratification and matching, and it can be proved. The Sign Test and the Wilcoxon Signed-Rank Test states Lyapunov's theorem (Billingsley, 1995): for independent centered terms $X_k$ with variances summing to $s^2$, the standardized sum converges in distribution to the standard normal if $\sum_k E|X_k|^3/s^3 \to 0$.

Theorem 2 (many small strata). Suppose the table sizes are bounded, $n_k \le M$ for all $k$, and that $\sum_k V_k \to \infty$ as $K \to \infty$. Then, under the hypothesis $\psi = 1$ and given the margins, $z_{\text{MH}}$ converges in distribution to the standard normal.

Proof. Let $X_k = a_k - E_k$. Since $a_k$ and $E_k$ both lie between 0 and $n_k$, $|X_k| \le n_k \le M$. So $E|X_k|^3 \le M\,E(X_k^2) = MV_k$, and

$$\frac{\sum_k E|X_k|^3}{\big(\sum_k V_k\big)^{3/2}} \le \frac{M\sum_k V_k}{\big(\sum_k V_k\big)^{3/2}} = \frac{M}{\sqrt{\sum_k V_k}} \to 0.$$

Lyapunov's condition holds, and the standardized sum, which is $z_{\text{MH}}$, has a standard normal limit. $\blacksquare$

The condition $\sum_k V_k \to \infty$ holds as soon as infinitely many tables have both an exposed and an unexposed member and both outcomes. With bounded sizes there are finitely many possible sets of margins, so each such table has $V_k$ at least the smallest positive value among them. A table with a zero margin has $V_k = 0$ and $a_k = E_k$: it contributes nothing to either the numerator or the denominator, and it can be dropped. The theorem is a statement given the margins, which is how the test is used: its null distribution is the conditional one. An unconditional version would need more: that the random margins make $\sum_k V_k$ grow without bound with probability tending to 1, and an argument that the conditional convergence is uniform enough to survive averaging over the margins. That version is not proved here.

A practical check comes from Mantel and Fleiss (1980), as reported by Breslow and Day (1980, §4.4, p. 139): the approximation should be adequate for p-values of about 0.05 and above if the null mean $\sum_k E_k$ lies at least five units from both the smallest and the largest value that $\sum_k a_k$ can take given the margins. For the smoking data those extremes are 0 and 665, and the null mean is 242.10, so the rule is met with a wide margin.

Matched pairs: the smallest strata

A matched pair is a stratum of two people. Here the general theory meets McNemar's Test for Paired Proportions, and the comparison of estimators becomes exact.

Consider pairs of one exposed and one unexposed person. Each pair is a table with $n_{1k} = n_{0k} = 1$. Write $n_{11}$ for the pairs in which both have the outcome, $n_{10}$ for those in which only the exposed person has it, $n_{01}$ for those in which only the unexposed person has it, and $n_{00}$ for those in which neither does.

Theorem 3 (pairs). For one-to-one matched pairs:

  1. $z_{\text{MH}} = (n_{10} - n_{01})/\sqrt{n_{10} + n_{01}}$, McNemar's statistic;
  2. $\hat\psi_{\text{MH}} = n_{10}/n_{01}$, and this is also the conditional maximum likelihood estimate;
  3. the unconditional maximum likelihood estimate, from a logistic model with one intercept per pair and a common log odds ratio, is $(n_{10}/n_{01})^2$.

Proof. (1) A pair with both or neither having the outcome has $m_{1k} = 2$ or $0$, so $a_k$ is fixed by the margins: $V_k = 0$ and $a_k = E_k$. A discordant pair has $m_{1k} = 1$ and $n_k = 2$, so $E_k = \tfrac12$ and $V_k = 1 \cdot 1 \cdot 1 \cdot 1/(4 \cdot 1) = \tfrac14$. Summing over the $n_{10} + n_{01}$ discordant pairs, $\sum(a_k - E_k) = n_{10} - \tfrac12(n_{10} + n_{01}) = \tfrac12(n_{10} - n_{01})$ and $\sum V_k = \tfrac14(n_{10} + n_{01})$. The ratio is McNemar's $z$.

(2) In a pair, $a_kd_k/n_k$ is $\tfrac12$ when only the exposed person has the outcome ($a_k = d_k = 1$) and 0 otherwise; $b_kc_k/n_k$ is $\tfrac12$ when only the unexposed person has it. So $R = n_{10}/2$, $S = n_{01}/2$ and $R/S = n_{10}/n_{01}$. For the conditional estimate, by Theorem 1 a discordant pair has $a_k = 1$ with probability $\psi/(1 + \psi)$ (the support is $\{0, 1\}$ with weights 1 and $\psi$). So $n_{10}$ given the number of discordant pairs is binomial with probability $\psi/(1 + \psi)$, maximized at $\psi/(1 + \psi) = n_{10}/(n_{10} + n_{01})$, that is, $\psi = n_{10}/n_{01}$. Concordant pairs have conditional probability 1 whatever $\psi$ is.

(3) Model pair $k$ by $\operatorname{logit} P(\text{outcome}) = \alpha_k + \beta x$, with $x = 1$ for the exposed member. A concordant pair can be fitted exactly by letting $\alpha_k \to \pm\infty$, and contributes nothing to the estimate of $\beta$. For a discordant pair in which only the exposed member has the outcome, the log-likelihood is $f = (\alpha + \beta) - \ln(1 + e^{\alpha + \beta}) - \ln(1 + e^{\alpha})$. Setting $\partial f/\partial\alpha = 0$ gives $\sigma(\alpha + \beta) + \sigma(\alpha) = 1$, with $\sigma(x) = 1/(1 + e^{-x})$. Since $\sigma(-x) = 1 - \sigma(x)$ and $\sigma$ is increasing, the solution is $\alpha + \beta = -\alpha$, so $\alpha = -\beta/2$, and the profile log-likelihood is $\beta/2 - g(\beta)$ with $g(\beta) = \ln(1 + e^{\beta/2}) + \ln(1 + e^{-\beta/2})$. A pair in which only the unexposed member has the outcome gives, by the same steps, $-\beta/2 - g(\beta)$. The total profile log-likelihood is

$$\tfrac12(n_{10} - n_{01})\beta - (n_{10} + n_{01})\,g(\beta).$$

Since $g'(\beta) = \tfrac12[\sigma(\beta/2) - \sigma(-\beta/2)] = \sigma(\beta/2) - \tfrac12$, setting the derivative to zero gives $\sigma(\beta/2) = n_{10}/(n_{10} + n_{01})$, so $e^{\beta/2} = n_{10}/n_{01}$ and $e^{\beta} = (n_{10}/n_{01})^2$. $\blacksquare$

Breslow and Day (1980, §7.1, p. 250) derive the same squaring, with a reference to Andersen, and conclude that "a relative risk of 2 will tend to be estimated as 4 by this approach". The reason is that the unconditional estimate fits one intercept per pair, and the number of parameters grows with the number of pairs. Andersen (1970) proved that conditional maximum likelihood estimates are consistent and asymptotically normal in this situation of infinitely many incidental parameters.

An experiment with matched pairs. The script matches each former smoker at random to a never smoker of the same sex and the same age in years, without replacement. Of the 1,182 former smokers, 1,049 find a partner and 133 do not, in every seed. For seed 19,000,000 the pairs give $n_{11} = 56$, $n_{10} = 168$, $n_{01} = 157$ and $n_{00} = 668$. Then $\hat\psi_{\text{MH}} = 168/157 = 1.070$, the unconditional estimate is $1.070^2 = 1.145$, McNemar's $z = 0.61$, and the standard error of $\ln\hat\psi_{\text{MH}}$ is $\sqrt{1/168 + 1/157} = 0.1110$. Over 30 random matchings (seeds 19,000,000 to 19,000,029) the Mantel–Haenszel estimate had median 1.081 (interquartile range 1.057 to 1.124), the unconditional estimate 1.168 (1.117 to 1.263), and McNemar's $z$ 0.68 (0.48 to 1.04). Matching on sex and exact age leaves little association between former smoking and diabetes, and the next section finds the same by stratifying.

The Mantel–Haenszel estimator

Why should $R/S$ estimate a common odds ratio when each table is too small to estimate its own? Here is the idea. Each table contributes $a_kd_k - \psi b_kc_k$, a quantity whose expectation is exactly zero at the true $\psi$, and the estimator is the solution of "sum of these quantities equals zero".

Theorem 4 (an unbiased estimating function). Let $a_k \sim \text{Binomial}(n_{1k}, p_{1k})$ and $c_k \sim \text{Binomial}(n_{0k}, p_{0k})$ be independent, $b_k = n_{1k} - a_k$, $d_k = n_{0k} - c_k$, and let $\psi$ be the table's odds ratio. Then $E[a_kd_k - \psi b_kc_k] = 0$. The same holds when the outcome totals are fixed and the exposure counts are binomial within them (case-control sampling).

Proof. By independence, $E[a_kd_k] = E[a_k]E[d_k] = n_{1k}p_{1k} \cdot n_{0k}(1 - p_{0k})$ and $E[b_kc_k] = n_{1k}(1 - p_{1k}) \cdot n_{0k}p_{0k}$. Their ratio is $p_{1k}(1 - p_{0k})/[(1 - p_{1k})p_{0k}] = \psi$, so $E[a_kd_k] = \psi E[b_kc_k]$. Under case-control sampling, with $q_1$ and $q_0$ the exposure probabilities among those with and without the outcome, $a_k \sim \text{Binomial}(m_{1k}, q_1)$ and $b_k \sim \text{Binomial}(m_{0k}, q_0)$ independently, $c_k = m_{1k} - a_k$ and $d_k = m_{0k} - b_k$. Then $E[a_kd_k] = m_{1k}q_1 \cdot m_{0k}(1 - q_0)$ and $E[b_kc_k] = m_{0k}q_0 \cdot m_{1k}(1 - q_1)$, with ratio $q_1(1 - q_0)/[q_0(1 - q_1)]$, the exposure odds ratio. Relative Risk and the Odds Ratio shows that it equals the outcome odds ratio $\psi$. $\blacksquare$

The script confirms the identity exactly, with rational arithmetic, for two small tables. Dividing by $n_k$ does not disturb it, so $R - \psi S$ has expectation zero, and $\hat\psi_{\text{MH}} = R/S$ is the root of the estimating equation $R - \psi S = 0$.

Theorem 5 (consistency). Suppose every table has odds ratio $\psi$.

  1. Many small strata. If $n_k \le M$ for all $k$, the tables are independent, and $E[S]/K \ge s_0 > 0$ for all $K$, then $\hat\psi_{\text{MH}} \to \psi$ in probability as $K \to \infty$.
  2. A few large strata. If $K$ is fixed and in each table $n_k \to \infty$ with the shares $n_{1k}/n_k$ and $n_k/\sum_j n_j$ fixed and positive, then $\hat\psi_{\text{MH}} \to \psi$ in probability.

Proof. (1) Each term $T_k = (a_kd_k - \psi b_kc_k)/n_k$ has mean zero by Theorem 4 and satisfies $|T_k| \le (1 + \psi)n_k \le (1 + \psi)M$, since $a_kd_k \le n_k^2$ and $b_kc_k \le n_k^2$. The terms are independent, so $\operatorname{Var}\big(\tfrac1K\sum_k T_k\big) \le (1 + \psi)^2M^2/K \to 0$, and by Chebyshev's inequality $(R - \psi S)/K \to 0$ in probability. The same argument applied to $S$, whose terms are bounded by $M$, gives $(S - E[S])/K \to 0$, so $S/K$ is at least $s_0/2$ with probability tending to 1. On that event $|\hat\psi_{\text{MH}} - \psi| = |R - \psi S|/S \le 2|R - \psi S|/(Ks_0)$, which tends to 0 in probability.

(2) By the law of large numbers, $a_k/n_{1k} \to p_{1k}$ and $c_k/n_{0k} \to p_{0k}$ in probability. Write $r_k = n_{1k}/n_k$ and $w_k = n_k/\sum_j n_j$. Then

$$\frac{R}{\sum_j n_j} = \sum_k w_k\,\frac{a_k}{n_k}\,\frac{d_k}{n_k} \to \sum_k w_k\,r_kp_{1k}(1 - r_k)(1 - p_{0k}),$$

and likewise $S/\sum_j n_j \to \sum_k w_k\,r_k(1 - p_{1k})(1 - r_k)p_{0k}$, which is positive. Each term of the first limit is $\psi$ times the corresponding term of the second, by the definition of $\psi$. The ratio of the two limits is $\psi$, and by the continuous mapping theorem (Convergence in Distribution and the Proof of the Central Limit Theorem) $R/S \to \psi$. $\blacksquare$

The unconditional maximum likelihood estimate fails the first property: Theorem 3 gave a case with many small strata in which it converges to $\psi^2$. With a few large strata the number of its parameters stays fixed, and its behavior is a question for the ordinary large-sample theory of maximum likelihood (Asymptotics of the Maximum Likelihood Estimator), which is not applied to it here. Breslow (1981) studied four estimators in the limit where the number of tables grows with the possible margins fixed. He found that "neither the unconditional maximum likelihood nor the empirical logit estimators converge to the true odds ratio". The conditional estimator and the Mantel–Haenszel estimator are consistent, and their asymptotic variances have a ratio that "tends to one under the null hypothesis" (his abstract).

Checking the formulas on a published analysis

Breslow and Day (1980, §4.4 and Table 4.3) analyze the association of daily alcohol consumption (80 g or more) with oesophageal cancer in six age strata of a case-control study. The script reproduces their published figures with the functions it uses for the NHANES data: the corrected summary chi-square 83.22, the Mantel–Haenszel estimate 5.158, the conditional maximum likelihood estimate 5.251, the unconditional estimate 5.312, the logit estimate 5.022 (with $\tfrac12$ added to every cell), the homogeneity statistic 9.32 at the unconditional estimate, the logit homogeneity statistic 6.93 and the trend statistic 0.32. The script also confirms by a direct numerical fit of the logistic model with six intercepts that the solution of Breslow and Day's equation 4.25 with asymptotic means is the unconditional maximum likelihood estimate. With the Mantel–Haenszel estimate in place of the unconditional one, Breslow and Day report fitted values totaling 95.16 and a homogeneity statistic of 9.28. The script obtains 95.18 and 9.32. The difference is small, and its source has not been found.

The three age groups

Back to the NHANES tables. For the two exposures, in the three age groups:

Former vs never smokers Obese vs not obese
$z_{\text{MH}}$ 2.74 8.97
Mantel–Haenszel estimate 1.289 2.095
Conditional ML estimate 1.291 2.092
Unconditional ML estimate 1.291 2.093
SE of $\ln\hat\psi_{\text{MH}}$, Robins–Breslow–Greenland 0.0930 0.0835
SE of $\ln\hat\psi_{\text{MH}}$, estimating function 0.0309 0.1496

With three large strata the three estimators agree to the second decimal. Breslow and Day (1980, §4.4, p. 141) report the same kind of agreement: "When the data in each stratum are more extensive it yields results which are in good agreement with the MLEs", citing Gart (1971) and McKinlay (1978). Theorem 5 proves only that the Mantel–Haenszel estimator is consistent here. The two standard errors do not agree, and the next section explains why.

The variance from the estimating function. The estimating equation suggests a variance estimator directly. Since $\hat\psi_{\text{MH}} - \psi = (R - \psi S)/S$, and $\ln\hat\psi - \ln\psi \approx (\hat\psi - \psi)/\psi$ to first order,

$$\ln\hat\psi_{\text{MH}} - \ln\psi \approx \frac{R - \psi S}{\psi S} = \frac{\sum_k T_k}{\psi S}.$$

The $T_k$ are independent with mean zero, so $\operatorname{Var}(\sum_k T_k) = \sum_k E[T_k^2]$. Estimating each $E[T_k^2]$ by the observed $T_k^2$ at $\hat\psi$ and $\psi S$ by $R$ gives

$$\widehat{\operatorname{Var}}_{\text{EF}}(\ln\hat\psi_{\text{MH}}) = \frac{\sum_k \big(a_kd_k - \hat\psi b_kc_k\big)^2/n_k^2}{R^2}.$$

This is a "sandwich" estimator. With many strata it averages many squared terms, each an unbiased estimate of its own mean at the true $\psi$, and it is consistent by the same argument as Theorem 5(1). With three strata it estimates a variance from three squared numbers, and one root of the estimating equation has already been spent making their weighted sum zero. This is the situation in which it gives 0.0309 for smoking and 0.1496 for obesity, against 0.0930 and 0.0835 from the estimator of Robins, Breslow and Greenland (1986). Their estimator was built to be "consistent in both sparse data and large-strata limiting models" (their abstract); the course page checks its value on the smoking tables by resampling, to within 2%. With many strata (the sex-by-age strata below) the two estimators agree closely, and for matched pairs they agree exactly, both reducing to $1/n_{10} + 1/n_{01}$ (the script checks this).

Many small strata

Refine the strata. Splitting the former and never smokers by sex and single year of age gives 124 strata that contain both groups, with a median of 31 adults and 4,109 adults in all. The estimates are:

Sex by single year of age
$z_{\text{MH}}$ 1.09
Mantel–Haenszel estimate 1.113
Conditional ML estimate 1.115
Unconditional ML estimate 1.118
Logit estimate, $\tfrac12$ added to each cell 1.350
SE of $\ln\hat\psi_{\text{MH}}$, Robins–Breslow–Greenland 0.0989
SE of $\ln\hat\psi_{\text{MH}}$, estimating function 0.0906

The strata are small but not tiny, and the Mantel–Haenszel, conditional and unconditional estimates still agree closely. The logit estimate does not: adding $\tfrac12$ to each of four cells in 124 small tables moves it to 1.350. Breslow and Day (1980, §4.4, p. 139) warn that the logit estimate with this correction "is subject to unacceptable bias when combining information from large numbers of strata, each containing only a few cases or controls". With the sandwich and Robins–Breslow–Greenland standard errors now close, the estimating-function variance is serviceable, as the argument above predicts for many strata.

Example (where did the association go?). With three age groups the summary odds ratio was 1.289 and $z = 2.74$; with sex and single years it is 1.113 and $z = 1.09$. Intermediate stratifications locate the change. Single years of age alone (63 strata) give 1.259 with $z = 2.45$, and sex by the three age groups gives 1.157 with $z = 1.51$. So most of the change comes from sex. Among the former smokers 64.1% are men, against 40.2% of the never smokers, and among all 5,107 adults diagnosed diabetes is more common in men (17.3%) than in women (13.2%). Within the three age groups the ages also differ somewhat: in the youngest, former smokers average 30.1 years and never smokers 27.1, while in the other two groups the means differ by less than a year (50.1 against 49.9, and 70.4 against 69.6). The matched pairs of the previous section, matched on both sex and age, gave a similar answer, 1.081 at the median. These are cross-sectional data, and as the course page cautions, sex and age are not the only differences between former and never smokers.

Is the odds ratio the same in every stratum?

A summary odds ratio describes the data only if the strata share one. Breslow and Day (1980, §4.4) give the test that bears their names. Fit the common odds ratio, compute for each table the fitted count $A_k(\hat\psi)$, the value of $a_k$ that, with the margins held fixed, gives a table whose odds ratio is exactly $\hat\psi$, and compare:

$$X^2_{\text{BD}} = \sum_k \frac{\big(a_k - A_k(\hat\psi)\big)^2}{\operatorname{Var}(a_k; \hat\psi)}, \qquad \operatorname{Var}(a_k; \hat\psi) = \Big(\frac1{A_k} + \frac1{B_k} + \frac1{C_k} + \frac1{D_k}\Big)^{-1},$$

where $A_k$ is the root of $A(n_k - n_{1k} - m_{1k} + A) = \hat\psi(n_{1k} - A)(m_{1k} - A)$ between the bounds of the table, and $B_k, C_k, D_k$ fill in the table from its margins (their equations 4.10, 4.11, 4.13 and 4.30). Under homogeneity it is approximately chi-squared with $K - 1$ degrees of freedom when the sample is large relative to the number of strata. Breslow and Day note two limits: with many thin strata the chi-squared approximation may fail, and the test ignores the order of the strata, so it lacks power against a trend. For a trend in the odds ratio with a stratum score $x_k$ they give a one-degree-of-freedom statistic (their 4.31),

$$X^2_{\text{trend}} = \frac{\big[\sum_k x_k\big(a_k - A_k(\hat\psi)\big)\big]^2}{\sum_k x_k^2 V_k(\hat\psi) - \big[\sum_k x_kV_k(\hat\psi)\big]^2\big/\sum_k V_k(\hat\psi)},$$

with a continuity correction of $\tfrac12$ in the numerator when the scores are equally spaced. A third statistic uses Woolf's logit approach: the weighted sum of squared deviations of the tables' log odds ratios from their weighted mean, with $\tfrac12$ added to each cell (their 4.33). Tarone (1985) derived a modification of the score-based heterogeneity test for use with the Mantel–Haenszel estimate; it is cited and not computed here.

The obesity tables. At $\hat\psi_{\text{MH}} = 2.095$ the fitted counts of obese adults with diabetes are 19.0, 126.6 and 286.5 in the three age groups, against 22, 140 and 270 observed, with variances 8.2, 43.3 and 92.6. The young and middle groups have more diabetes among the obese than a common odds ratio predicts, and the oldest group less. Then:

Obesity Smoking
Breslow–Day $X^2$ (2 df) 8.16, $p = 0.017$ 0.47, $p = 0.792$
Logit $X^2$ (2 df) 7.99, $p = 0.018$ 0.63, $p = 0.730$
Trend, scores 0, 1, 2 7.47, $p = 0.006$ 0.40, $p = 0.526$
Trend, corrected 7.09, $p = 0.008$ 0.30, $p = 0.585$

For obesity the hypothesis of a common odds ratio is rejected, and the trend statistic locates the departure: the odds ratio falls with age. The course page saw the decline, 3.05, 2.88 and 1.75, and concluded that "the three ratios say more than either summary"; the tests confirm that the decline is larger than chance would produce. For smoking there is no evidence against a common odds ratio across the three age groups. The course page saw the smoking ratios fall from about 1.6 to 1.2; with these numbers of cases, the tests find a decline of that size well within what chance produces.

Beyond a series of 2 × 2 tables

Mantel (1963) extended the procedure to tables with more than two rows or columns, where rows and columns are ordered and given scores: in each table a "deviation of the sum of cross products from expectation, and its variance conditioned on all marginal totals" are computed and summed (his abstract). Among the applications he lists are comparisons of age-adjusted death rates and life-table analyses. The single-table case with ordered exposure is the trend test of Trends in Proportions: the Cochran–Armitage Test, and the life-table case is the log-rank test of the Survival Analysis unit, The Log-Rank Test, which forms one $2 \times 2$ table at each event time.

What is proved, what is cited, what the experiment shows

Proved here. The noncentral hypergeometric distribution of a table given its margins, and its invariance between cohort and case-control sampling (Theorem 1); the test as the conditional score test, with its exact variance; the normal limit for many bounded strata given the margins, from Lyapunov's theorem (Theorem 2); McNemar's test, the conditional estimate and the squared unconditional estimate for matched pairs (Theorem 3); the unbiased estimating function (Theorem 4); consistency in both limits (Theorem 5); the estimating-function variance and its consistency with many strata; the one-table identity of Problem 4.

Cited. Lyapunov's theorem (Billingsley, 1995, through the linked page); the hypergeometric mean and variance, through The Chi-Square Test of Independence; the normal approximation for large tables, the homogeneity, trend and logit statistics and the rule of five (Breslow and Day, 1980); the variance estimator of Robins, Breslow and Greenland (1986); the consistency of conditional estimators in general (Andersen, 1970); Breslow's (1981) results on the four estimators; Tarone's (1985) modification.

Computed. All test statistics, estimates and fitted values are exact computations on the NHANES tables. The matched pairs are 30 seeded random matchings, and their medians describe those matchings. The comparison of estimators in sex-by-age strata is one data set, not a study of the estimators.

Worked problems

These problems were written for this article. Problem 1 uses the small table of Breslow and Day (1980, §4.2), the others the NHANES tables. The script tools/experiments/statistics/mantel-haenszel-test.py recomputes every number.

Problem 1 (an exact conditional estimate). Breslow and Day (1980, equation 4.3) use the table with $a = 2$, $b = 1$, $c = 3$, $d = 1$. Find the conditional distribution of $a$ given the margins, its mean and variance at $\psi = 1$, and the conditional maximum likelihood estimate of $\psi$.

Solution. At $\psi = 1$ the probabilities of $a = 1, 2, 3$ are $1/7$, $4/7$ and $2/7$ (0.143, 0.571, 0.286), with mean 2.143 and variance 0.408; the conditional estimate is $1/\sqrt2 = 0.707$, against the table's odds ratio $2/3 = 0.667$.

The margins are $n_1 = 3$, $n_0 = 4$, $m_1 = 5$, $m_0 = 2$. So $a \ge m_1 - n_0 = 1$ and $a \le \min(n_1, m_1) = 3$. By Theorem 1 the weights are $\binom{3}{u}\binom{4}{5 - u}\psi^u$: $3 \cdot 1 \cdot \psi = 3\psi$ for $u = 1$, $3 \cdot 4\psi^2 = 12\psi^2$ for $u = 2$ and $1 \cdot 6\psi^3 = 6\psi^3$ for $u = 3$. Dividing by $3\psi$, the probabilities are proportional to $1$, $4\psi$ and $2\psi^2$, so

$$P(a = 1) = \frac{1}{1 + 4\psi + 2\psi^2}, \quad P(a = 2) = \frac{4\psi}{1 + 4\psi + 2\psi^2}, \quad P(a = 3) = \frac{2\psi^2}{1 + 4\psi + 2\psi^2},$$

the expressions Breslow and Day give. At $\psi = 1$ the denominator is 7. The mean is $(1 + 8 + 6)/7 = 15/7 = 2.143$. The second moment is $(1 + 16 + 18)/7 = 5$, so the variance is $5 - (15/7)^2 = 20/49 = 0.408$. The hypergeometric formula gives the same: $3 \cdot 4 \cdot 5 \cdot 2/(7^2 \cdot 6) = 120/294 = 20/49$.

The conditional estimate solves $E_\psi(a) = 2$, by the score equation:

$$\frac{1 + 8\psi + 6\psi^2}{1 + 4\psi + 2\psi^2} = 2 \iff 1 + 8\psi + 6\psi^2 = 2 + 8\psi + 4\psi^2 \iff \psi^2 = \tfrac12,$$

and the positive root is $\psi = 1/\sqrt2 = 0.707$. The empirical odds ratio is $(2 \times 1)/(1 \times 3) = 0.667$. The two are different estimators, and in a table this small the difference shows.

Problem 2 (Mantel–Haenszel against Cochran). For the obesity tables, compute the test with the exact conditional variance and with Cochran's variance $n_{1k}n_{0k}m_{1k}m_{0k}/n_k^3$, and explain the size of the difference.

Solution. The total observed is 432 and the total expected 324.11. The exact variances sum to 144.79 and Cochran's to 144.71. The statistics are 8.966 and 8.969.

In each table the two variances differ by the factor $n_k/(n_k - 1)$. The tables have 1,680, 1,557 and 1,870 adults, so the factors are within 0.07% of 1, and the ratio of the two standard deviations is 1.00029. The distinction matters for small tables: for a matched pair, $n_k = 2$, Cochran's variance is half the exact one, and the statistic would be $\sqrt2$ times McNemar's.

Problem 3 (test-based limits). Breslow and Day (1980, equation 4.29) give the "test-based" interval $\hat\psi^{\,1 \pm z_{0.975}/z}$, with $z$ the uncorrected statistic. Compute it for the smoking data and compare with the Robins–Breslow–Greenland interval.

Solution. The test-based interval is 1.075 to 1.546, the other 1.074 to 1.547.

With $\hat\psi = 1.289$, $\ln\hat\psi = 0.2540$ and $z = 2.74$, the limits are $\exp\{0.2540(1 \pm 1.96/2.74)\}$, which gives 1.075 and 1.546. Writing them as $\hat\psi\,e^{\pm 1.96\,\ln\hat\psi/z}$ shows the standard error they imply: $\ln\hat\psi/z = 0.0927$. That is the standard error of the log odds ratio if the test statistic and the estimate measure the same distance from 1 in the same units, which is close to true when $\psi$ is near 1 and fails as it moves away (Breslow and Day, 1980, §4.4). Here the Robins–Breslow–Greenland standard error is 0.0930 and the interval $1.289e^{\pm 1.96 \times 0.0930}$ runs from 1.074 to 1.547.

Problem 4 (one table). Prove that for a single table the uncorrected $z_{\text{MH}}$ satisfies $z_{\text{MH}}^2 = X^2(N - 1)/N$, where $X^2 = N(ad - bc)^2/(n_1n_0m_1m_0)$ is Pearson's statistic for the table.

Solution. First, $aN - n_1m_1 = a(a + b + c + d) - (a + b)(a + c) = ad - bc$. So $a - E = (ad - bc)/N$, and

$$z_{\text{MH}}^2 = \frac{(a - E)^2}{V} = \frac{(ad - bc)^2}{N^2} \cdot \frac{N^2(N - 1)}{n_1n_0m_1m_0} = \frac{(N - 1)(ad - bc)^2}{n_1n_0m_1m_0} = \frac{N - 1}{N}X^2.$$

Comparing Two Proportions proves that $X^2$ is the square of the pooled two-proportion $z$, so $z_{\text{MH}} = z\sqrt{(N - 1)/N}$, the relation the course page confirmed numerically. The factor is the ratio of Cochran's variance to the exact one, as in Problem 2.

Summary

Result Statement
Distribution given margins $P(a = u) \propto \binom{n_1}{u}\binom{n_0}{m_1 - u}\psi^u$
The test score test of the conditional likelihood at $\psi = 1$: $\sum(a_k - E_k)/\sqrt{\sum V_k}$
Normal limit holds for many bounded strata when $\sum V_k \to \infty$ (Lyapunov)
Matched pairs $z_{\text{MH}}$ = McNemar's $z$; $\hat\psi_{\text{MH}} = n_{10}/n_{01}$; unconditional MLE $= (n_{10}/n_{01})^2$
Estimating function $E[a_kd_k - \psi b_kc_k] = 0$; $\hat\psi_{\text{MH}}$ solves $R - \psi S = 0$
Consistency for many small strata and for a few large ones
Homogeneity $\sum_k (a_k - A_k(\hat\psi))^2/\operatorname{Var}(a_k; \hat\psi)$, $K - 1$ df

What we have learned

Given its margins, a $2 \times 2$ table depends only on its odds ratio, and the product of these conditional distributions over strata is a likelihood free of the strata's baseline risks. The Mantel–Haenszel test is its score test at an odds ratio of 1, which explains the exact hypergeometric variance, and Lyapunov's condition makes it valid for many small strata as well as for a few large ones. The estimator solves an equation whose terms have expectation exactly zero, which makes it consistent in both limits, while the unconditional maximum likelihood estimate converges to the square of the odds ratio in matched pairs. On the NHANES data, refining the adjustment from three age groups to sex and single years of age moved the odds ratio for former smoking from 1.289 to 1.113 and the test from 2.74 to 1.09. For obesity, the odds ratios differ between age groups by more than chance would produce, and they fall with age.

Every test so far asked whether two variables are associated. The last article of the unit, Measures of Agreement: Cohen's Kappa, asks a different question of a paired table: not whether two classifications are related, but how often they agree, beyond what chance would give.

References