Trends in Proportions: the Cochran–Armitage Test

A one-degree-of-freedom test for a proportion that rises or falls across ordered groups. The statistic derived three ways, as a standardized weighted regression slope, as the score statistic of a binary regression model with any link, and as n times a squared correlation; its normal limit proved; the exact conditional variance that gives Mantel's (n - 1) r^2 form and an exact test; the r x 2 chi-square split exactly into a trend part and a departure from linearity; the choice of scores; and the power gained over the chi-square test. Worked on diagnosed diabetes across body mass index categories in NHANES. Worked problems on diabetes by age group, a U-shaped table the test cannot see, and the effect of the scores.

A sample of 150 adults drawn at random from the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018 was divided into five groups by body mass index (BMI): under 25, 25 to 30, 30 to 35, 35 to 40, and 40 and over (kg/m²). The share with diagnosed diabetes was 1 of 42, 6 of 50, 4 of 22, 5 of 21 and 3 of 15. The chi-square test of the $5 \times 2$ table, from The Chi-Square Test of Independence, finds no significant difference among the five proportions ($p = 0.102$). Yet the proportions mostly rise with BMI. Does the sample show that the proportion with diabetes increases across the ordered groups? It does: the test of this article gives $p = 0.0106$.

A test for trend in proportions asks whether a proportion changes steadily across groups that have a natural order, by measuring the slope of the proportions against scores assigned to the groups. The version derived here is the Cochran–Armitage test. Cochran (1954) argued that the usual chi-square tests "are not directed against any specific alternative" and proposed supplementary tests aimed at particular patterns, and Armitage (1955) devoted a paper to tests for linear trends in proportions (both from their abstracts). Armitage's opening example is of the same kind as ours: a carrier rate that increases with tonsil size, for which the usual test of the $2 \times 3$ table gives $\chi^2 = 7.89$ on 2 degrees of freedom, a probability of about 0.02, and which he expected a test designed for a trend to show as much more significant. The test spends one degree of freedom on the trend, where the chi-square test spends $I - 1$ on every pattern.

First we set up the groups and derive the statistic as a weighted regression slope, then prove its normal limit. Then we show that the same statistic is the score test of every binary regression model in the score, and $n$ times a squared correlation. Next come the exact conditional version, the exact split of the chi-square statistic into trend and departure, the choice of scores, and the power gained. The article builds on The Chi-Square Test of Independence and uses the score statistic of Likelihood Ratio, Wald and Score Tests.

The setting

There are $I$ groups in a fixed order. Group $i$ has $n_i$ individuals, of whom $y_i$ have the outcome, with $n = \sum_i n_i$. The groups are independent binomial samples, $y_i \sim \text{Bin}(n_i, p_i)$, as in the independent-rows design of the independence article; with multinomial sampling, Theorem 1 of that article turns the design into this one by conditioning on the group sizes. Each group carries a score $x_i$, a number that expresses its position, such as $1, 2, \ldots, I$ or a typical value of the variable that defines the groups. The null hypothesis is that the proportions are equal, $H_0\colon p_1 = \cdots = p_I$.

Write $\hat p_i = y_i/n_i$ for the group proportions, $\hat p = \sum_i y_i/n$ for the pooled proportion, $\bar x = \sum_i n_ix_i/n$ for the mean score per individual, and

$$S_{xx} = \sum_{i=1}^{I} n_i(x_i - \bar x)^2.$$

The running example has $I = 5$, scores $x_i = 1, \ldots, 5$, and

BMI under 25 25–30 30–35 35–40 40 and over
$n_i$ 42 50 22 21 15
$y_i$ 1 6 4 5 3
$\hat p_i$ 0.024 0.120 0.182 0.238 0.200
Population share 0.072 0.152 0.183 0.228 0.233

The last row is the share with diagnosed diabetes among all 5,107 adults of the unit's risk-factor population (described on the topic page), which rises across the first four groups and levels off. The pooled proportion is $\hat p = 19/150 = 0.1267$, the mean score $\bar x = 2.4467$ and $S_{xx} = 253.073$.

The statistic as a regression slope

Here is the idea. Fit a straight line to the five proportions against the scores, weighting each proportion by its group size, and ask whether its slope is further from zero than chance allows. The slope's numerator is a linear combination of the counts, and under $H_0$ its variance is known exactly up to the common proportion.

The weighted least squares slope of $\hat p_i$ on $x_i$, with weights $n_i$, is

$$b = \frac{\sum_i n_i(x_i - \bar x)(\hat p_i - \hat p)}{S_{xx}} = \frac{T}{S_{xx}}, \qquad T = \sum_{i=1}^{I}(x_i - \bar x)\,y_i,$$

because $n_i\hat p_i = y_i$ and $\sum_i n_i(x_i - \bar x)\hat p = 0$. Under $H_0$ with common proportion $p$, $E[T] = p\sum_i n_i(x_i - \bar x) = 0$, and, since the $y_i$ are independent with variances $n_ip(1 - p)$,

$$\operatorname{Var}(T) = p(1 - p)\sum_i (x_i - \bar x)^2 n_i = p(1 - p)\,S_{xx}.$$

The Cochran–Armitage statistic standardizes $T$ with the pooled proportion in place of $p$:

$$z = \frac{T}{\sqrt{\hat p(1 - \hat p)\,S_{xx}}} = b\,\sqrt{\frac{S_{xx}}{\hat p(1 - \hat p)}}.$$

For the running sample, $T = (1 - 2.4467)(1) + (2 - 2.4467)(6) + \cdots + (5 - 2.4467)(3) = 13.513$, its estimated variance is $0.1267 \times 0.8733 \times 253.073 = 27.996$, and

$$z = \frac{13.513}{\sqrt{27.996}} = 2.554.$$

The two-sided $p$-value from the standard normal is $2P(Z \ge 2.554) = 0.0106$, and the one-sided value for an increasing trend is 0.0053. The fitted slope is $b = 0.0534$, about five percentage points per BMI group.

The null distribution

Theorem 1 (normal limit). Let the scores not all be equal, and let $n \to \infty$ with $n_i/n \to \omega_i > 0$ for every group. Under $H_0$ with common proportion $0 < p < 1$, $z \xrightarrow{d}$ standard normal, and so $z^2 \xrightarrow{d} \chi^2_1$. ($z$ is defined when $0 < \sum_i y_i < n$, an event whose probability tends to 1; its value off that event does not affect the limit.)

The proof uses the Lindeberg–Feller theorem for sums of independent variables that are not identically distributed (Durrett, 2019, Theorem 3.4.10), stated in The Chi-Square Goodness-of-Fit Test, section "Power and the noncentral chi-squared distribution".

Proof. Number the individuals $l = 1, \ldots, n$, let $Y_l$ be 1 if individual $l$ has the outcome, and let $x_{(l)}$ be the score of its group. Since $\sum_i n_i(x_i - \bar x) = 0$,

$$\frac{T}{\sqrt{n}} = \sum_{l=1}^{n}\frac{(x_{(l)} - \bar x)(Y_l - p)}{\sqrt{n}} = \sum_{l=1}^{n}X_{n,l}.$$

The $X_{n,l}$ are independent with mean 0. Each is at most $M/\sqrt{n}$ in absolute value, with $M = 2\max_i\lvert x_i\rvert$, because $\lvert Y_l - p\rvert \le 1$ and $\lvert x_{(l)} - \bar x\rvert \le 2\max_i\lvert x_i\rvert$; so for every $\eta > 0$ the Lindeberg sum is 0 once $M/\sqrt{n} < \eta$. Their variances add to $p(1 - p)S_{xx}/n$, which tends to $v = p(1 - p)\sum_i\omega_i(x_i - \bar x_\omega)^2$, where $\bar x_\omega = \sum_i\omega_ix_i$; $v > 0$ because the scores are not all equal and every $\omega_i > 0$. So $T/\sqrt{n}$ converges in distribution to the normal with mean 0 and variance $v$. Next, $\hat p \xrightarrow{p} p$ by the weak law of large numbers and $S_{xx}/n \to \sum_i\omega_i(x_i - \bar x_\omega)^2$, so $\sqrt{\hat p(1 - \hat p)S_{xx}/n} \xrightarrow{p} \sqrt{v}$. By Slutsky's theorem (Convergence in Distribution and the Proof of the Central Limit Theorem), $z = (T/\sqrt{n})/\sqrt{\hat p(1 - \hat p)S_{xx}/n}$ converges to the standard normal. $\square$

The script checked the level by drawing samples from the population with the diabetes labels shuffled, once for each of 30 seeds, so that BMI group and diabetes were independent with the population's margins. With 2,000 samples of 150 for each seed (seeds 5,800,000 + $s$), the test at the 5% level rejected in a share with median 0.0495 (interquartile range 0.0460 to 0.0564), and with samples of 300 (seeds 5,850,000 + $s$), 0.0505 (0.0445 to 0.0574). The chi-square test of the $5 \times 2$ table, applied to the same samples, rejected in shares with medians 0.0473 (0.0394 to 0.0519) and 0.0463 (0.0446 to 0.0554). The spread across seeds is wide because each seed shuffles its own null population and draws only 2,000 samples.

The same statistic, two more ways

The score test of a binary regression model

Let the proportions follow a model $p_i = h(\alpha + \beta x_i)$, where $h$ is a differentiable function with $h' > 0$ that takes every value in $(0, 1)$: the logistic function $h(\eta) = 1/(1 + e^{-\eta})$ of Logistic Regression, the standard normal distribution function (the probit model), or any other. The null hypothesis $\beta = 0$ is the hypothesis of equal proportions. Here is the idea of the next theorem: at $\beta = 0$ every group has the same linear predictor, so every derivative of $h$ appears as the same constant, and the constants cancel from the score statistic.

Theorem 2 (the score statistic for any link). Let $0 < \sum_i y_i < n$. For every such $h$, the score statistic for $\beta = 0$, with $\alpha$ estimated under the null hypothesis, is $z^2$.

Proof. The log-likelihood is $\ell(\alpha, \beta) = \sum_i[y_i\ln h(\eta_i) + (n_i - y_i)\ln(1 - h(\eta_i))]$ with $\eta_i = \alpha + \beta x_i$. Differentiating,

$$\frac{\partial\ell}{\partial\alpha} = \sum_i\frac{(y_i - n_ih(\eta_i))\,h'(\eta_i)}{h(\eta_i)(1 - h(\eta_i))}, \qquad \frac{\partial\ell}{\partial\beta} = \sum_i x_i\,\frac{(y_i - n_ih(\eta_i))\,h'(\eta_i)}{h(\eta_i)(1 - h(\eta_i))}.$$

Under $\beta = 0$ all $\eta_i$ equal $\alpha$, and the first equation set to zero gives $h(\hat\alpha_0) = \hat p$, which has a solution because $h$ takes every value in $(0, 1)$, as the logistic, probit and complementary log-log functions do. Write $\mu = \hat p$ and $\gamma_1 = h'(\hat\alpha_0)/(\mu(1 - \mu))$. At $(\hat\alpha_0, 0)$ the score is $U_\alpha = 0$ and

$$U_\beta = \gamma_1\sum_i x_i(y_i - n_i\mu) = \gamma_1\sum_i(x_i - \bar x)(y_i - n_i\mu) = \gamma_1T,$$

using $\sum_i(y_i - n_i\mu) = 0$. The Fisher information is the covariance matrix of the score (Fisher Information and the Cramér–Rao Bound). At $\beta = 0$ each $y_i$ has variance $n_i\mu(1 - \mu)$, so with $\gamma_2 = h'(\hat\alpha_0)^2/(\mu(1 - \mu))$ the entries of the information matrix $\mathcal{I}$ are

$$\mathcal{I}_{\alpha\alpha} = \gamma_2n, \qquad \mathcal{I}_{\alpha\beta} = \gamma_2\sum_i n_ix_i, \qquad \mathcal{I}_{\beta\beta} = \gamma_2\sum_i n_ix_i^2.$$

The score statistic with a nuisance parameter is $\mathbf{U}^{\mathsf T}\mathcal{I}^{-1}\mathbf{U}$. With $U_\alpha = 0$ only the $(\beta, \beta)$ entry of the inverse matters, which for a $2 \times 2$ matrix is $1/(\mathcal{I}_{\beta\beta} - \mathcal{I}_{\alpha\beta}^2/\mathcal{I}_{\alpha\alpha})$. Here $\mathcal{I}_{\beta\beta} - \mathcal{I}_{\alpha\beta}^2/\mathcal{I}_{\alpha\alpha} = \gamma_2[\sum_i n_ix_i^2 - (\sum_i n_ix_i)^2/n] = \gamma_2S_{xx}$. So the statistic is

$$\frac{\gamma_1^2T^2}{\gamma_2S_{xx}} = \frac{h'^2}{\mu^2(1 - \mu)^2}\cdot\frac{\mu(1 - \mu)}{h'^2}\cdot\frac{T^2}{S_{xx}} = \frac{T^2}{\hat p(1 - \hat p)S_{xx}} = z^2. \qquad \square$$

The theorem says nothing about the distribution of the score statistic in a two-parameter model, which the site does not prove in general; Theorem 1 supplies it directly. The script fitted the logistic model to the running sample by Newton's method and computed the three statistics of Likelihood Ratio, Wald and Score Tests for the slope. The score statistic is $z^2 = 6.523$; the Wald statistic, with the fitted slope 0.452 and standard error 0.183, is 6.080 ($p = 0.0137$); and the likelihood ratio statistic is 6.171 ($p = 0.0130$). The fitted odds of diabetes are multiplied by $e^{0.452} = 1.57$ per BMI group. The script also checked Theorem 2 numerically: the score statistics of the logistic, probit and complementary log-log models all equal $z^2$ to eight decimal places.

n times a squared correlation

List the 150 individuals with two numbers each: the score $x_{(l)}$ of the group and the outcome $Y_l$, 0 or 1. Their Pearson correlation is $r$.

Theorem 3 (the correlation form). $z^2 = n\,r^2$.

Proof. The mean of the scores over individuals is $\bar x$ and the mean of the outcomes is $\hat p$. The cross-product sum is $\sum_l (x_{(l)} - \bar x)(Y_l - \hat p) = \sum_i(x_i - \bar x)y_i - \hat p\sum_i n_i(x_i - \bar x) = T$. The sum of squares of the scores is $S_{xx}$, and that of the outcomes is $\sum_l(Y_l - \hat p)^2 = n\hat p(1 - \hat p)$, because $Y_l^2 = Y_l$. So $r^2 = T^2/(S_{xx}\,n\hat p(1 - \hat p)) = z^2/n$. $\square$

For the running sample $r = 0.2085$ and $150 \times 0.2085^2 = 6.523 = 2.554^2$. The trend test is a test that a correlation is zero, with one variable binary and the other a set of group scores. Correlation: Pearson, Spearman and Kendall treats the correlation coefficient itself.

The exact conditional test

The variance of $T$ above was computed for independent binomial groups. Conditioning on the total number with the outcome, as Fisher's exact test does, changes it slightly and gives an exact test.

Theorem 4 (the conditional moments of T). Under $H_0$, given the group sizes and the total $y = \sum_i y_i$,

$$E[T] = 0, \qquad \operatorname{Var}(T) = \hat p(1 - \hat p)\,S_{xx}\,\frac{n}{n - 1}.$$

Proof. By Theorem 2 of the independence article with two columns, given both margins the $y$ outcomes are assigned at random to the $n$ individuals, every set of $y$ individuals being equally likely. Let $I_l$ indicate that individual $l$ is one of them, and $a_l = x_{(l)} - \bar x$, so that $T = \sum_l a_lI_l$ and $\sum_l a_l = 0$. Each $I_l$ has mean $q = y/n = \hat p$, and for $l \ne l'$, $\operatorname{Cov}(I_l, I_{l'}) = -q(1 - q)/(n - 1)$, as in the proof of Theorem 5 of that article. So $E[T] = q\sum_l a_l = 0$ and

$$\operatorname{Var}(T) = q(1 - q)\sum_l a_l^2 - \frac{q(1 - q)}{n - 1}\sum_{l \ne l'}a_la_{l'} = q(1 - q)\Big[\sum_l a_l^2 + \frac{1}{n - 1}\sum_l a_l^2\Big] = q(1 - q)S_{xx}\frac{n}{n - 1},$$

using $\sum_{l \ne l'}a_la_{l'} = (\sum_l a_l)^2 - \sum_l a_l^2 = -\sum_l a_l^2$. $\square$

With this variance the squared statistic becomes $z^2(n - 1)/n = (n - 1)r^2$, by Theorem 3. That is the form of Mantel (1963), who, by his abstract, computed for tables with scored rows and columns "a deviation of the sum of cross products from expectation, and its variance conditioned on all marginal totals". For the running sample $(n - 1)r^2 = 6.479$, $p = 0.0109$, against $z^2 = 6.523$; the factor $(n - 1)/n$ matters only in small samples.

The conditional distribution itself is computable. Given the group sizes and the total, the counts $(y_1, \ldots, y_I)$ have probabilities proportional to $\prod_i\binom{n_i}{y_i}$, by the formula of Theorem 2 of the independence article with two columns. The script summed these weights group by group to obtain the exact distribution of $\sum_i x_iy_i$, which differs from $T$ by the constant $\bar x y$. For the running sample the observed value is 60. Its exact conditional upper-tail probability is 0.0085, and the two-sided probability of a value at least as far from the mean as 60 is 0.0109, close to the normal approximation's 0.0106. As a check, the weights summed to $\binom{150}{19}$ (Vandermonde's identity), and the exact mean and variance matched Theorem 4.

Example (a small sample). In a sample of 40 adults (seed 53101) the five BMI groups held 11, 11, 10, 5 and 3 adults, with diabetes in 0, 2, 1, 1 and 0. The observed $\sum_i x_iy_i = 11$, against a conditional mean of 9.80. The normal approximation gives $z = 0.517$ and two-sided $p = 0.6053$, or, with the conditional variance of Theorem 4, $z = 0.510$ and $p = 0.6098$. The exact conditional two-sided probability is 0.6786 (one-sided 0.3735). With four outcomes among 40 adults the statistic takes few values, the exact and approximate answers differ in the first decimal, and both say the sample shows nothing.

Splitting the chi-square statistic

The trend test and the chi-square test of the $I \times 2$ table are related by an exact identity. Problem 1 of the independence article shows that the chi-square statistic of an $I \times 2$ table is

$$X^2 = \frac{\sum_i n_i(\hat p_i - \hat p)^2}{\hat p(1 - \hat p)}.$$

The numerator is the weighted sum of squares of the proportions about their pooled value, and least squares splits it into the part explained by the fitted line and the residual part. Let $\tilde p_i = \hat p + b(x_i - \bar x)$ be the fitted line.

Theorem 5 (the partition). $X^2 = z^2 + X^2_{\text{dep}}$, where

$$X^2_{\text{dep}} = \frac{\sum_i n_i(\hat p_i - \tilde p_i)^2}{\hat p(1 - \hat p)}$$

measures the departure of the proportions from a straight line. Under $H_0$, with $I \ge 3$ and the conditions of Theorem 1, $z^2$ and $X^2_{\text{dep}}$ converge jointly in distribution to independent $\chi^2_1$ and $\chi^2_{I-2}$ variables.

Proof. Write $\hat p_i - \hat p = (\tilde p_i - \hat p) + (\hat p_i - \tilde p_i)$ and expand the weighted sum of squares. The cross term is $\sum_i n_i(\tilde p_i - \hat p)(\hat p_i - \tilde p_i) = b\sum_i n_i(x_i - \bar x)(\hat p_i - \tilde p_i)$, and

$$\sum_i n_i(x_i - \bar x)(\hat p_i - \tilde p_i) = \sum_i n_i(x_i - \bar x)(\hat p_i - \hat p) - b\sum_i n_i(x_i - \bar x)^2 = T - bS_{xx} = 0.$$

So $\sum_i n_i(\hat p_i - \hat p)^2 = \sum_i n_i(\tilde p_i - \hat p)^2 + \sum_i n_i(\hat p_i - \tilde p_i)^2$. The first term is $b^2S_{xx} = T^2/S_{xx}$, which divided by $\hat p(1 - \hat p)$ is $z^2$. This proves the identity.

For the limit, let $W_i = \sqrt{n_i}(\hat p_i - p)/\sqrt{p(1 - p)}$. The $W_i$ are independent and each converges in distribution to a standard normal variable, so by Lemma 2 of the independence article and the Cramér–Wold device the vector $\mathbf{W}$ converges in distribution to a vector $\mathbf{Y}$ of independent standard normal variables. Let $\mathbf{w}$ have entries $\sqrt{n_i/n}$ and $\mathbf{v}$ entries $\sqrt{n_i}(x_i - \bar x)/\sqrt{S_{xx}}$. Both are unit vectors, and they are perpendicular, since $\mathbf{w}\cdot\mathbf{v} = \sum_i n_i(x_i - \bar x)/\sqrt{nS_{xx}} = 0$. A short calculation gives, exactly,

$$\frac{T^2}{p(1 - p)\,S_{xx}} = (\mathbf{v}\cdot\mathbf{W})^2, \qquad \frac{\sum_i n_i(\hat p_i - \hat p)^2}{p(1 - p)} = \lVert(\mathrm{Id} - \mathbf{w}\mathbf{w}^{\mathsf T})\mathbf{W}\rVert^2,$$

the first because $T = \sum_i n_i(x_i - \bar x)(\hat p_i - p) = \sqrt{p(1 - p)S_{xx}}\;\mathbf{v}\cdot\mathbf{W}$, and the second because $\sqrt{n_i}(\hat p_i - \hat p)/\sqrt{p(1 - p)} = W_i - \sqrt{n_i/n}\,(\mathbf{w}\cdot\mathbf{W})$. So, with $p(1 - p)$ in place of $\hat p(1 - \hat p)$, the trend part is $\lVert\mathbf{v}\mathbf{v}^{\mathsf T}\mathbf{W}\rVert^2$ and the departure part is $\lVert(\mathrm{Id} - \mathbf{w}\mathbf{w}^{\mathsf T} - \mathbf{v}\mathbf{v}^{\mathsf T})\mathbf{W}\rVert^2$, where $\mathrm{Id}$ is the identity matrix of size $I$. The matrices $\mathbf{v}\mathbf{v}^{\mathsf T}$ and $\mathrm{Id} - \mathbf{w}\mathbf{w}^{\mathsf T} - \mathbf{v}\mathbf{v}^{\mathsf T}$ are projections of ranks 1 and $I - 2$ whose product is 0, and they converge to the corresponding matrices for the limiting weights $\omega_i$. By the continuous mapping argument of Theorem 3 of the independence article, the pair converges in distribution to the pair of squared lengths of the two projections of $\mathbf{Y}$, which by Theorem 2 of Sampling Distributions from the Normal are independent $\chi^2_1$ and $\chi^2_{I-2}$ variables. Replacing $p(1 - p)$ by $\hat p(1 - \hat p) \xrightarrow{p} p(1 - p)$ does not change the limit (Slutsky's theorem). $\square$

For the running sample,

$$X^2 = 7.728 = z^2 + X^2_{\text{dep}} = 6.523 + 1.205.$$

The departure from linearity, on 3 degrees of freedom, has $p = 0.752$: the line fits the five proportions well, and nearly all of the chi-square statistic is trend. The chi-square test spreads that trend over 4 degrees of freedom and does not reach significance, while the trend test concentrates it in one. In the whole population the same split gives $X^2 = 125.32$ and $z^2 = 113.98$, so the trend carries a share 0.910 of the population's departure from equal proportions. Cochran's (1954) abstract describes this use of supplementary tests: the remedy for a test "not directed against any specific alternative" is to supplement it "by additional tests that help to reveal the significant type of deviation".

Example (a significant trend is not a linear relationship). The test asks whether the slope of the fitted line is zero. A rejection says the proportions tend to rise (or fall) with the scores; it does not say they lie on a line, and the population's proportions, which level off after the fourth group, do not. Whether a line describes them is the separate question that $X^2_{\text{dep}}$ answers.

The choice of scores

The test depends on the scores, but not on their location or unit. If every $x_i$ is replaced by $c + dx_i$ with $d > 0$, then $x_i - \bar x$ becomes $d(x_i - \bar x)$, $T$ becomes $dT$ and $S_{xx}$ becomes $d^2S_{xx}$, so $z$ is unchanged; with $d < 0$ it changes sign. Scores $1, 2, 3, 4, 5$ and $0, 10, 20, 30, 40$ give the same test.

The spacing matters. For the BMI groups, natural scores are the median BMI of each group in the population: 22.4, 27.4, 32.1, 36.8 and 44.0. They are nearly equally spaced except for the open-ended last group, and they give $z = 2.471$, $p = 0.0135$, against $z = 2.554$ with equally spaced scores. Problem 3 shows a coarser choice. Scores should be chosen before the data are seen, from the meaning of the groups. Choosing them afterwards to make $z$ largest turns a one-degree-of-freedom test into a search, and its level is no longer $\alpha$.

How much power the trend test gains

How much power the trend test gains depends on the alternative, and here it is measured, not derived. (On another example, the section "Two categories" of The Chi-Square Goodness-of-Fit Test shows a one-degree-of-freedom test and a four-degree test reaching different conclusions on one sample, and its Problem 4 compares their powers.) For the BMI groups the script drew samples from the population itself, in which the population's own proportions are the truth. With 2,000 samples for each of 30 seeds:

Sample size Trend test, power Chi-square test, power
150 0.4465 (0.4416 to 0.4524) 0.3145 (0.3101 to 0.3203)
300 0.7398 (0.7330 to 0.7444) 0.5880 (0.5841 to 0.5966)

Each entry is the median rejection rate at the 5% level over the seeds, with the interquartile range. As an observation for this population, the trend test had the higher power at both sizes. For proportions with no linear component in the scores ($\sum_i\omega_i(x_i - \bar x_\omega)p_i = 0$), the population value of the trend part is zero: the trend test has nothing to detect, while the departure part, and so the chi-square test, can respond; Problem 2 shows a table of that kind, where $z = 0$ and $X^2 = 6.000$. No power was computed for such an alternative.

What is proved here, what is cited, and what the experiments show

Proved here: the mean and variance of $T$ under $H_0$; the normal limit of $z$ (Theorem 1); that $z^2$ is the score statistic of every binary regression model in the scores (Theorem 2) and $n$ times a squared correlation (Theorem 3); the exact conditional variance and Mantel's $(n - 1)r^2$ form (Theorem 4); the exact partition of the $I \times 2$ chi-square statistic and the joint limit of its parts (Theorem 5); the invariance of the test under changes of location and unit of the scores.

Cited: the Lindeberg–Feller theorem and the Cramér–Wold device (Durrett, 2019), through the linked articles; the continuity theorem for characteristic functions, through Lemma 2 of the independence article; the history from the abstracts of Cochran (1954), Armitage (1955) and Mantel (1963).

What the experiments show: the level of the test, and its power and that of the chi-square test, in samples of 150 and 300 from this population; the exact conditional distribution on two samples; the agreement of the score statistics of three link functions. Each is an observation about these data.

Worked problems

The problems are posed on samples from the unit's NHANES population and on a constructed table, to exercise the computation, the limits of a trend test and the choice of scores. Every number is computed by the script named in the last section.

Problem 1 (diabetes by age group). The script drew 120 adults (seed 53201). With diabetes in 1 of 37 aged 18 to 39, 6 of 37 aged 40 to 59 and 11 of 46 aged 60 and over, test for a trend with scores 1, 2, 3, and split the chi-square statistic.

Solution. The proportions are 0.027, 0.162 and 0.239. The pooled proportion is $\hat p = 18/120 = 0.1500$. The mean score is $\bar x = (37 + 74 + 138)/120 = 2.0750$, and

$$T = (1 - 2.075)(1) + (2 - 2.075)(6) + (3 - 2.075)(11) = -1.075 - 0.450 + 10.175 = 8.650,$$

$$S_{xx} = 37(1.075)^2 + 37(0.075)^2 + 46(0.925)^2 = 42.758 + 0.208 + 39.359 = 82.325.$$

So $z = 8.650/\sqrt{0.15 \times 0.85 \times 82.325} = 2.670$, with two-sided $p = 0.00759$. The $3 \times 2$ chi-square statistic is $X^2 = 7.298$ on 2 degrees of freedom, which splits as $z^2 = 7.128$ for the trend and $0.169$ for the departure from linearity on 1 degree of freedom. The rise with age is almost entirely linear in the scores, as far as 120 adults can tell.

Problem 2 (a pattern the trend test cannot see). Three ordered groups of 50 have 20, 10 and 20 with the outcome. Compute the trend statistic and the chi-square statistic.

Solution. The pooled proportion is $50/150 = 0.3333$, and with scores 1, 2, 3 the mean score is 2, so $T = (1 - 2)(20) + (2 - 2)(10) + (3 - 2)(20) = 0$ and $z = 0$. The trend test sees nothing. The chi-square statistic is

$$X^2 = \frac{50(0.4 - 0.3333)^2 + 50(0.2 - 0.3333)^2 + 50(0.4 - 0.3333)^2}{0.3333 \times 0.6667} = 6.000,$$

on 2 degrees of freedom, $p = 0.0498$, all of it departure from linearity. A test aimed at one direction has no power in the directions perpendicular to it. When a U-shaped or other non-monotone pattern is possible, the chi-square test, or the departure term of Theorem 5, is the one that can detect it.

Problem 3 (the effect of the scores). For the running sample, compare the trend test with scores 1 to 5, with the population medians of BMI, and with scores 0, 0, 1, 1, 1 (BMI under 30 against 30 and over).

Solution. The three give $z = 2.554$ ($p = 0.0106$), $z = 2.471$ ($p = 0.0135$) and $z = 2.346$ ($p = 0.0190$). The last choice turns the test into the comparison of two proportions, diabetes among those under 30 against those at 30 or over. With only two distinct scores, $T$ and $S_{xx}$ depend on the data only through the two merged groups, so $z$ is the trend statistic of the collapsed table of two groups. For two groups the fitted line passes through both proportions, the departure term of Theorem 5 is 0, and $z^2$ equals the $X^2$ of the $2 \times 2$ table, which Comparing Two Proportions proves equal to the squared two-proportion $z$ statistic. All three reject at the 5% level here. In the population, the trend part carries a share 0.910 of $X^2$ with equal spacing, 0.877 with the medians and 0.635 with the split at 30: of the three, equal spacing follows these population proportions most closely. That is an observation about these scores and this population; no power was computed for the three choices, and none should be chosen by looking at the sample.

Summary

Quantity Formula or result
Statistic $z = T/\sqrt{\hat p(1 - \hat p)S_{xx}}$, $T = \sum_i(x_i - \bar x)y_i$, $S_{xx} = \sum_i n_i(x_i - \bar x)^2$
Null limit standard normal (Theorem 1)
Regression reading $z = b\sqrt{S_{xx}/(\hat p(1 - \hat p))}$, $b$ the weighted slope
Score test $z^2$ for every link $h$ (Theorem 2)
Correlation reading $z^2 = nr^2$; conditional form $(n - 1)r^2$ (Mantel)
Partition $X^2_{I \times 2} = z^2 + X^2_{\text{dep}}$, limits $\chi^2_1$ and $\chi^2_{I-2}$, independent
Scores invariant to location and unit; spacing matters; choose in advance

What we have learned

When groups are ordered, a test of equal proportions can aim at the alternative that matters, a steady rise or fall. The Cochran–Armitage statistic is the standardized slope of the group proportions on the scores, the score statistic of every binary regression model in those scores, and $n$ times the squared correlation of score and outcome. It is one component of the chi-square statistic of the table, and the rest measures departure from a line. For 150 NHANES adults it found the rise of diabetes with BMI that the chi-square test, spending four degrees of freedom, missed.

The trend test compares independent groups. When the two classifications are made on the same individuals, as with a blood pressure measured twice, the observations are paired and the comparison changes; McNemar's Test for Paired Proportions treats that case.

In Python

import numpy as np
from scipy import stats

n_i = np.array([42, 50, 22, 21, 15]); y_i = np.array([1, 6, 4, 5, 3]); x = np.arange(1, 6)
p = y_i.sum() / n_i.sum(); xbar = (n_i * x).sum() / n_i.sum()
T = ((x - xbar) * y_i).sum(); Sxx = (n_i * (x - xbar) ** 2).sum()
z = T / np.sqrt(p * (1 - p) * Sxx)
print(z, 2 * stats.norm.sf(abs(z)))          # 2.554, 0.0106

Every number in this article is computed and checked against the page by tools/experiments/statistics/cochran-armitage-trend-test.py in the site's repository.

References