The Log-Rank Test

How to test whether two survival curves differ when follow-up is censored. Builds the log-rank test as a Mantel–Haenszel test over the 2 x 2 tables formed at each event time, derives the hypergeometric mean and variance of each table, and proves in discrete time that the tables' deviations are uncorrelated martingale increments, so that their variances add although the tables are not independent (normal limit cited); links the proof that it is the Cox model's score test; proves that the observed-over-expected chi-square never exceeds it, that Gehan's generalized Wilcoxon test is the log-rank test weighted by the number at risk, and that the most powerful weights are proportional to the log hazard ratio (drift heuristic); covers stratified tests and the number of events a trial needs, by Schoenfeld's and Freedman's formulas. Worked on the 6-MP leukemia trial and on NHANES mortality by diagnosed diabetes and smoking, where stratifying by age cuts one comparison by more than half and strengthens the other, with simulations (30 seeds) of level and power under proportional, early, late and crossing hazards and of trials sized by the two formulas. Worked problems on one event time by hand, Gehan's identity on four patients, and an unequal allocation.

Among the 4,114 NHANES adults of 1999 to 2000 that this unit follows, 273 of the 416 with diagnosed diabetes died by the end of 2019. Suppose diabetes made no difference to the risk of death at any moment. Then, given who was still alive at each death, the people with diabetes would have been expected to account for 92.1 of the deaths. The Kaplan–Meier Estimator drew the two survival curves; they are far apart. But people with diabetes were also about 16 years older on average. How do we test whether two survival curves differ, when follow-up is censored, and how do we keep age out of the comparison?

The log-rank test answers both. At every time at which someone dies, it compares the number of deaths in one group with the number expected if the two groups shared the same hazard. Then it adds the differences over time. Its null hypothesis is that the two groups have the same survival function, and it can be computed within strata, such as age groups, and summed.

First we set up the $2 \times 2$ table at each event time and derive its distribution under the null hypothesis, which makes the test a Mantel–Haenszel test. Then we prove the fact that makes the sum valid although the tables are not independent. We link the proof that the test is the score test of the Cox model. Next come the weighted versions, which alternatives each detects best, stratification, and the number of events a trial needs. Finally we measure level and power by simulation, including hazards that cross. The article assumes The Kaplan–Meier Estimator and The Mantel–Haenszel Test, and the notation of Censoring, Survival Functions and Hazards.

The table at each event time

Two groups are followed: group 1 (for example, people with diabetes) and group 0. As in the rest of the unit, each person contributes an observed time $Y_i$ and an event indicator $\delta_i$. The distinct event times of the two groups together are $t_{(1)} < \cdots < t_{(k)}$. At $t_{(j)}$ write

  • $n_{1j}$ and $n_{0j}$ for the numbers at risk in the two groups, $n_j = n_{1j} + n_{0j}$;
  • $d_{1j}$ and $d_{0j}$ for the numbers of deaths, $d_j = d_{1j} + d_{0j}$.

These four numbers form a $2 \times 2$ table, group by died-or-not, among those at risk:

died at $t_{(j)}$ survived $t_{(j)}$ at risk
group 1 $d_{1j}$ $n_{1j} - d_{1j}$ $n_{1j}$
group 0 $d_{0j}$ $n_{0j} - d_{0j}$ $n_{0j}$
total $d_j$ $n_j - d_j$ $n_j$

The null hypothesis is $H_0\!: S_1(t) = S_0(t)$ for all $t$, or equivalently that the two groups have the same hazard at every time. People censored in either group leave the tables after their censoring time. That is why censoring is no obstacle, as long as it is independent of the lifetime within each group.

Why the count at each time is hypergeometric

Here is the idea. If the two groups share a hazard, then at each event time the deaths fall at random on the people at risk, without regard to group. So given how many die, which ones die is like drawing that many names from a hat, and the number drawn from group 1 is hypergeometric.

To make this exact, take the discrete-time model of The Kaplan–Meier Estimator (section "The Nelson–Aalen estimator"). Times lie on a grid, the hazards are $h_j$, and censoring is independent and happens after the events of its time. Under $H_0$ both groups have hazard $h_j$ at $t_{(j)}$. Given everything observed before $t_{(j)}$, each of the $n_j$ people at risk dies there with probability $h_j$, independently. Any particular set of $d$ people dies, and the rest survive, with probability $h_j^d(1 - h_j)^{n_j - d}$, which depends only on $d$. So given $d_j$, every set of $d_j$ people from the risk set is equally likely to be the set that died. The count $d_{1j}$ of group 1 members among them is then hypergeometric:

$$P(d_{1j} = u) = \frac{\binom{n_{1j}}{u}\binom{n_{0j}}{d_j - u}}{\binom{n_j}{d_j}}.$$

The same distribution appeared in The Mantel–Haenszel Test (Theorem 1, at an odds ratio of 1), and The Chi-Square Test of Independence derives its moments, as does the proof of Theorem 3 in The Cox Proportional Hazards Model. The derivation is short, and the variance is the one the test uses, so here it is in this setting.

Lemma 1 (hypergeometric moments). If $d$ of $n$ people are chosen at random without replacement and $n_1$ of the $n$ are in group 1, the number $D_1$ chosen from group 1 has

$$E[D_1] = \frac{n_1 d}{n}, \qquad \operatorname{Var}(D_1) = \frac{n_1 n_0\, d\,(n - d)}{n^2(n - 1)}, \qquad n_0 = n - n_1.$$

Proof. Write $D_1 = \sum_{i \in \text{group } 1} I_i$, where $I_i = 1$ if person $i$ is chosen. Each person is chosen with probability $d/n$, so $E[D_1] = n_1d/n$. Two given people are both chosen with probability $\frac{d}{n}\cdot\frac{d - 1}{n - 1}$, so for $i \ne i'$

$$\operatorname{Cov}(I_i, I_{i'}) = \frac{d(d - 1)}{n(n - 1)} - \frac{d^2}{n^2} = \frac{d\,[n(d - 1) - d(n - 1)]}{n^2(n - 1)} = -\frac{d(n - d)}{n^2(n - 1)}.$$

Adding $n_1$ variances $\frac{d}{n}(1 - \frac{d}{n}) = \frac{d(n - d)}{n^2}$ and $n_1(n_1 - 1)$ covariances,

$$\operatorname{Var}(D_1) = \frac{n_1 d(n - d)}{n^2}\Big[1 - \frac{n_1 - 1}{n - 1}\Big] = \frac{n_1 d(n - d)}{n^2}\cdot\frac{n - n_1}{n - 1}. \qquad \square$$

Write $E_j = n_{1j}d_j/n_j$ and $V_j = n_{1j}n_{0j}d_j(n_j - d_j)/[n_j^2(n_j - 1)]$ for the conditional mean and variance at $t_{(j)}$. When only one person dies, $d_j = 1$, the variance is $V_j = n_{1j}n_{0j}/n_j^2$: a single Bernoulli trial with probability $n_{1j}/n_j$ that the death falls in group 1. When everyone at risk dies, $V_j = 0$: the table is fixed by its margins. When only one person is at risk, $n_j = 1$, the formula has a zero denominator, and $V_j = 0$ by definition, since again the table is fixed.

For example, at 8 weeks in the leukemia trial below, 16 children on 6-MP and 12 on placebo are at risk and 4 relapse, all on placebo. If the treatment made no difference, the chance that all 4 relapses fall in the placebo group would be $\binom{12}{4}/\binom{28}{4} = 0.024$.

The statistic

Sum the deviations and the variances over the event times:

$$U = \sum_{j=1}^{k}(d_{1j} - E_j) = O_1 - E_1, \qquad V = \sum_{j=1}^{k} V_j, \qquad Z = \frac{U}{\sqrt V}, \qquad X^2 = \frac{U^2}{V}.$$

Here $O_1 = \sum_j d_{1j}$ is the observed number of deaths in group 1 and $E_1 = \sum_j E_j$ the number expected under $H_0$. The test rejects at the two-sided level $\alpha$ when $|Z| > z_{1-\alpha/2}$, or when $X^2$ exceeds the $1 - \alpha$ quantile of the chi-squared distribution with one degree of freedom. The sign of $U$ says which group fares worse. In form this is exactly the Mantel–Haenszel statistic of The Mantel–Haenszel Test, with one stratum per event time. Mantel (1963) listed "life-table analyses" among the applications of his extension of the Mantel–Haenszel procedure (his abstract). Harrington and Fleming's (1982) abstract credits the log-rank test to Mantel (1966) and Cox (1972).

Why the variance is still a sum

There is a difference from the Mantel–Haenszel setting, and it is the heart of the matter. There, the strata were different people and the tables were independent, so the variances added. Here the tables are formed from the same people at successive times. Who is at risk at $t_{(j)}$ depends on who died earlier, so the tables are dependent. Why may their variances still be added?

Theorem 1 (the log-rank statistic is a sum of conditional Mantel–Haenszel terms). In the discrete-time model under $H_0$, index by the grid times $\tau_1 < \cdots < \tau_m$ rather than by the event times (a grid time with no death contributes $X = 0$ and $V = 0$, so the sums are unchanged). For each grid time, which we still write $t_{(j)}$, let $\mathcal{P}_j$ denote everything observed before it, and let $X_j = d_{1j} - E_j$. Then $E[X_j \mid \mathcal{P}_j, d_j] = 0$ and $E[X_j^2 \mid \mathcal{P}_j, d_j] = V_j$; the $X_j$ are uncorrelated; and

$$E[U] = 0, \qquad \operatorname{Var}(U) = E[V].$$

Proof. Given $\mathcal{P}_j$ and $d_j$, $d_{1j}$ is hypergeometric (previous section), so by Lemma 1 its conditional mean is $E_j$ and its conditional variance $V_j$, both determined by $\mathcal{P}_j$ and $d_j$. This gives the first two claims. For $j < j'$, $X_j$ is determined by what is observed before $t_{(j')}$, so by the averaging property of conditional expectation (Conditional Expectation), $E[X_jX_{j'}] = E\big[X_j\,E[X_{j'} \mid \mathcal{P}_{j'}, d_{j'}]\big] = 0$. Hence $E[U] = \sum_j E[X_j] = 0$ and $\operatorname{Var}(U) = \sum_j E[X_j^2] = \sum_j E\big[E[X_j^2 \mid \mathcal{P}_j, d_j]\big] = E[V]$. $\square$

The partial sums of the $X_j$ form a martingale (Random Walks and Martingales): each increment has mean zero given the past. That is enough for the variance; for the normal limit it is not, and the theorem needed is a central limit theorem for martingales. Harrington and Fleming (1982) used "martingale theory ... to establish asymptotic normality" of a class of statistics containing this one (their abstract). Aalen (1978) defined and studied tests of this kind for counting processes, and Andersen, Borgan, Gill and Keiding (1993) give the theory at book length. In continuous time with no ties the same structure holds, with each death a Bernoulli trial among those at risk.

The script checks Theorem 1 by enumerating every possible history of a small cohort. It has three people in each group and hazard 1/3 at each of three times, and each survivor is censored after each time with probability 1/4. All 7,056 possible sequences of death and censoring counts in the two groups, weighted by their exact probabilities, give $E[U] = 0$ and $E[U^2] = E[V] = 31513/61440 = 0.5129$, in exact fractions.

Two cautions follow from the theorem. First, the test needs independent censoring. If leaving the study depended on frailty differently in the two groups, the risk sets would not reflect the groups' hazards (The Kaplan–Meier Estimator, section "When censoring is informative"). Second, the null hypothesis is equality of the whole curves. A rejection says that the hazards differ somewhere; it does not say that one group fares worse at every time.

The 6-MP trial, table by table

Freireich et al. (1963) randomized children with acute leukemia in remission to 6-mercaptopurine or placebo. The remission times are listed in The Kaplan–Meier Estimator; all 21 placebo patients relapsed, and 9 of the 21 on 6-MP. With group 1 = 6-MP, there are 17 distinct event times:

$t$ (weeks) $n_{1j}$ $n_{0j}$ $d_{1j}$ $d_{0j}$ $E_j$ $V_j$
1 21 21 0 2 1.000 0.488
2 21 19 0 2 1.050 0.486
3 21 17 0 1 0.553 0.247
4 21 16 0 2 1.135 0.477
5 21 14 0 2 1.200 0.466
6 21 12 3 0 1.909 0.651
7 17 12 1 0 0.586 0.243
8 16 12 0 4 2.286 0.871
10 15 8 1 0 0.652 0.227
11 13 8 0 2 1.238 0.448
12 12 6 0 2 1.333 0.418
13 12 4 1 0 0.750 0.188
15 11 4 0 1 0.733 0.196
16 11 3 1 0 0.786 0.168
17 10 3 0 1 0.769 0.178
22 7 2 1 1 1.556 0.302
23 6 1 1 1 1.714 0.204

At the first event time all 42 children are at risk, half in each group, and 2 relapse, so $E_1 = 21 \times 2/42 = 1$ and $V_1 = 21 \cdot 21 \cdot 2 \cdot 40/(42^2 \cdot 41) = 0.4878$. The 6-MP group had $O_1 = 9$ relapses against $E_1 = 19.25$ expected; the placebo group 21 against 10.75. So $U = -10.25$, $V = 6.257$, $Z = -4.10$ and $X^2 = 16.79$, with a two-sided p-value of $4.2 \times 10^{-5}$. Remissions lasted longer on 6-MP.

The dataset records the children in pairs, with the remission status at randomization (partial or complete). The analysis here ignores the pairing and treats the two groups as independent samples. A test stratified by remission status, as in the section "Stratified tests", would use part of that information.

Example (the observed-over-expected chi-square). A shortcut sometimes used is the Pearson-like sum $(O_1 - E_1)^2/E_1 + (O_0 - E_0)^2/E_0$. For the trial it gives 15.23, smaller than 16.79. That is no accident.

Proposition 1 (the shortcut never exceeds the log-rank statistic). $(O_1 - E_1)^2/E_1 + (O_0 - E_0)^2/E_0 \le U^2/V$.

Proof. Since $O_1 + O_0 = E_1 + E_0 = D$, the total number of deaths, $O_0 - E_0 = -U$, and the left side is $U^2(1/E_1 + 1/E_0) = U^2D/(E_1E_0)$. So it suffices that $V \le E_1E_0/D$. Write $\pi_j = n_{1j}/n_j$, so that $E_j = \pi_jd_j$ and $d_j - E_j = (1 - \pi_j)d_j$. Since $(n_j - d_j)/(n_j - 1) \le 1$, $V_j \le \pi_j(1 - \pi_j)d_j$. With weights $w_j = d_j/D$, which sum to 1, $\sum_j \pi_j(1 - \pi_j)d_j = D(\bar\pi - \overline{\pi^2})$, where the bars are $w$-weighted means, while $E_1E_0/D = D\,\bar\pi(1 - \bar\pi)$. The difference $D[\bar\pi(1 - \bar\pi) - \bar\pi + \overline{\pi^2}] = D[\overline{\pi^2} - \bar\pi^2]$ is $D$ times a weighted variance, which is nonnegative. $\square$

So the shortcut is conservative. It loses most when the shares $\pi_j$ change a lot over time, as they do when one group dies faster.

The test as a score test

Is the log-rank test merely a sensible construction, or is it the answer to a likelihood question? The Cox Proportional Hazards Model answers this (Theorem 3, section "The score test at zero is the log-rank test"). Put the group indicator $x_i \in \{0, 1\}$ into the model $h(t \mid x) = h_0(t)e^{\beta x}$. Under the exact discrete likelihood for tied times (the discrete logistic model) and under Breslow's approximation, the score of the partial likelihood at $\beta = 0$ is $O_1 - E_1$; under Efron's approximation, in general it is not, when there are ties. With the exact discrete likelihood the observed information at 0 is the hypergeometric $V$, so the score test of $\beta = 0$ is the log-rank test. With Breslow's approximation for ties the information at 0 is $\sum_j d_jn_{1j}n_{0j}/n_j^2$, which lacks the factor $(n_j - d_j)/(n_j - 1)$ and is larger when there are ties.

For the 6-MP trial, with its ties, the Breslow information is 6.596 against $V = 6.257$, and the score statistic is $-3.99$ against $-4.10$. With no ties the two coincide. The same structure appeared in The Mantel–Haenszel Test, where the Mantel–Haenszel test is the conditional score test for a common odds ratio. Here the common parameter is a hazard ratio (under the discrete logistic model, an odds ratio of the discrete hazards, which is close to the hazard ratio when the hazards are small), and the score test is local to $\beta = 0$ (Likelihood Ratio, Wald and Score Tests). The score-test reading predicts which alternatives the test detects best: those in which the hazard ratio is the same at every time, proportional hazards. The next section makes that precise.

Weighted log-rank tests

The log-rank test gives every event time the same weight. A weighted log-rank test chooses weights $w_j$, computed from the data before $t_{(j)}$, and uses

$$U_w = \sum_j w_j(d_{1j} - E_j), \qquad V_w = \sum_j w_j^2V_j, \qquad Z_w = U_w/\sqrt{V_w}.$$

Theorem 1 carries over word for word, since $w_j$ is fixed given $\mathcal{P}_j$: $E[U_w] = 0$ and $\operatorname{Var}(U_w) = E[V_w]$. Three weights are common.

  • Gehan's test, $w_j = n_j$: early event times, when many are at risk, count most.
  • The Peto–Peto test, $w_j = \hat S(t_{(j)}^-)$, the pooled Kaplan–Meier estimate just before $t_{(j)}$. Harrington and Fleming's abstract says that their class contains a test essentially equivalent to Peto and Peto's (1972) generalization of the Wilcoxon test. The member usually identified with it, and used here under that name, has this weight; we did not check the form in either paper. Again early times count most, but the weight falls with survival rather than with the number at risk, so censoring does not lower it.
  • A late-weighted test, $w_j = 1 - \hat S(t_{(j)}^-)$: zero at the start, growing as deaths accumulate.

Harrington and Fleming (1982) proposed a class of rank statistics that contains the log-rank test and "a test essentially equivalent to Peto & Peto's (1972) generalization of the Wilcoxon test", as their abstract says; the abstract does not give the class's weights, and we did not check them in the paper. Weights $\hat S(t^-)^\rho$ run between the two: $\rho = 0$ gives the log-rank test and $\rho = 1$ the Peto–Peto weight. Weights $\hat S(t^-)^\rho[1 - \hat S(t^-)]^\gamma$ add the late-weighted test. We write these as FH($\rho$, $\gamma$), after Harrington and Fleming, so that the late-weighted test is FH(0, 1) and the Peto–Peto test FH(1, 0). The script computes the weights from the pooled Kaplan–Meier estimate itself; versions that modify it slightly differ in small samples.

Proposition 2 (Gehan's test is the log-rank test weighted by the number at risk). Gehan (1965) proposed "an extension of the Wilcoxon test to samples with arbitrary censoring on the right" (his abstract). In its pairwise form, every pair of one person from each group is scored. The score is $+1$ if the group 1 member is known to outlast the group 0 member, $-1$ if known to die first, and 0 if the order cannot be determined. A pair's order is known when the earlier of the two times is a death and is strictly earlier than the other time. It is also known when a death and a censoring fall at the same time, since by the unit's convention the death comes first. The sum of the scores, $W$, equals $-\sum_j n_j(d_{1j} - E_j)$.

Proof. Count the pairs whose order is decided at $t_{(j)}$. A pair is scored $+1$ at $t_{(j)}$ when the group 0 member dies at $t_{(j)}$ while the group 1 member is at risk there and does not die. There are $d_{0j}(n_{1j} - d_{1j})$ such pairs. A pair is scored $-1$ at $t_{(j)}$ in $d_{1j}(n_{0j} - d_{0j})$ pairs. Pairs dying together at $t_{(j)}$ score 0, as do pairs in which the earlier time is a censoring. Every scored pair is counted exactly once, at the earlier death. So

$$W = \sum_j\big[d_{0j}(n_{1j} - d_{1j}) - d_{1j}(n_{0j} - d_{0j})\big] = \sum_j\big[d_{0j}n_{1j} - d_{1j}n_{0j}\big] = \sum_j\big[(d_j - d_{1j})n_{1j} - d_{1j}(n_j - n_{1j})\big],$$

which is $\sum_j (d_jn_{1j} - d_{1j}n_j) = -\sum_j n_j(d_{1j} - n_{1j}d_j/n_j)$. $\square$

For the 6-MP trial the pairwise scores sum to 271, and $\sum_j n_j(d_{1j} - E_j) = -271$, as the proposition requires. The weighted log-rank form standardizes by $V_w$. With these weights the trial gives $Z = -3.67$ (Gehan), $-3.80$ (Peto–Peto) and $-3.61$ (FH(0, 1)), against $-4.10$ for the log-rank test.

Which weight detects which difference? Here is the idea in one sentence: the weighted test is most powerful when its weights follow the log hazard ratio over time. Suppose that at $t_{(j)}$ the log hazard ratio of group 1 to group 0 is a small number $\theta_j$. To keep the algebra short, let one person die at each event time. Given the past, the death falls in group 1 with probability $n_{1j}e^{\theta_j}/(n_{1j}e^{\theta_j} + n_{0j})$, which to first order in $\theta_j$ is $\pi_j + \pi_j(1 - \pi_j)\theta_j$, with $\pi_j = n_{1j}/n_j$ (the derivative of $\pi e^\theta/(\pi e^\theta + 1 - \pi)$ at $\theta = 0$ is $\pi(1 - \pi)$). So $E[d_{1j} - E_j \mid \mathcal{P}_j] \approx \theta_jv_j$ with $v_j = \pi_j(1 - \pi_j) = V_j$, and the standardized statistic has mean about

$$\mu_w = \frac{\sum_j w_j\theta_jv_j}{\sqrt{\sum_j w_j^2v_j}}.$$

Proposition 3 (the best weights). For fixed positive $v_j$, real $\theta_j$ not all zero and real weights $w_j$ not all zero, $\mu_w \le \sqrt{\sum_j\theta_j^2v_j}$, with equality exactly when the weights are a positive multiple of $\theta_j$. For a two-sided test, whose power depends on $|\mu_w|$, any nonzero multiple of $\theta_j$ is optimal.

Proof. By the Cauchy–Schwarz inequality applied to the vectors $(w_j\sqrt{v_j})$ and $(\theta_j\sqrt{v_j})$, $\sum_j w_j\theta_jv_j \le \sqrt{\sum_j w_j^2v_j}\sqrt{\sum_j\theta_j^2v_j}$, with equality exactly when one vector is a nonnegative multiple of the other. Since both vectors are nonzero and $v_j > 0$, that means $w = c\theta$ with $c > 0$. Applying the same inequality to $-w$ gives $|\mu_w| \le \sqrt{\sum_j\theta_j^2v_j}$, with equality for $w = c\theta$, $c \ne 0$. $\square$

The first-order drift is a heuristic; the optimization is exact. Schoenfeld (1981) derived the asymptotic distribution of this class of statistics under alternatives (his abstract). Harrington and Fleming (1982) present a class of distributions matched to their class of statistics: for each distribution there is a statistic in the class with "some optimal properties" for detecting location alternatives from it (their abstract). The consequences:

  • Proportional hazards, $\theta_j$ constant: equal weights are best, and the log-rank test is the most powerful member of the class.
  • A difference early in follow-up: Gehan and Peto–Peto weights, large early, do better.
  • A difference late in follow-up: the late-weighted FH(0, 1) does better.
  • Crossing hazards, $\theta_j$ changing sign: the best weights change sign too. Every test with positive weights adds an early excess and a late deficit with the same sign convention, and the two can cancel, wholly or in part, depending on how the weights treat early and late times. With equal weights they nearly cancel in the simulation below.

Level and power, simulated

The script compares the four tests at the two-sided 5% level, with 100 people per group. Group 0 has hazard 1, and censoring is uniform on $(0, 3)$ and independent of everything. There are 30 runs (seeds 83,000,000 to 83,000,029) of 1,000 data sets. Group 1 has:

  • no difference: hazard 1;
  • proportional: hazard 0.6 throughout;
  • early: hazard 0.4 before $t = 0.5$, then 1;
  • late: hazard 1 before $t = 0.75$, then 0.4;
  • crossing: hazard 2 before $t = 0.3$, then 0.5. Group 1 then does worse at first, and its survival curve crosses group 0's at $t = 0.9$, where both are 0.407.

Rejection rates, medians over the 30 runs with interquartile ranges:

group 1 log-rank Gehan Peto–Peto FH(0, 1)
no difference 0.052 (0.048 to 0.057) 0.050 (0.046 to 0.053) 0.051 (0.045 to 0.056) 0.054 (0.050 to 0.060)
proportional 0.805 (0.797 to 0.811) 0.709 (0.704 to 0.722) 0.755 (0.749 to 0.763) 0.667 (0.659 to 0.675)
early 0.550 (0.542 to 0.558) 0.805 (0.799 to 0.812) 0.758 (0.751 to 0.765) 0.114 (0.108 to 0.121)
late 0.251 (0.244 to 0.262) 0.075 (0.069 to 0.078) 0.103 (0.096 to 0.109) 0.527 (0.521 to 0.538)
crossing 0.056 (0.051 to 0.063) 0.353 (0.342 to 0.359) 0.254 (0.238 to 0.259) 0.377 (0.368 to 0.386)
The five simulated comparisons: survival of group 0 (hazard 1) and group 1, with each test's rejection rate. Read across the panels: each weight wins where its emphasis matches the difference, and with crossing curves the log-rank test rejects about as often as under no difference at all.
The five simulated comparisons: survival of group 0 (hazard 1) and group 1, with each test's rejection rate. Read across the panels: each weight wins where its emphasis matches the difference, and with crossing curves the log-rank test rejects about as often as under no difference at all.

All four tests hold their level, which Theorem 1 and its weighted version predict. Under proportional hazards the log-rank test is the most powerful, as Proposition 3 predicts. Gehan's test wins when the difference is early and the late-weighted test when it is late; each loses heavily in the other's scenario. When the hazards cross, the log-rank test rejects in 0.056 of the data sets, essentially its level. The early excess of deaths in group 1 and its later deficit cancel in $U$. The weighted tests each see one half of the crossing and reject 25% to 38% of the time, in opposite directions. Gehan's and the Peto–Peto test reject with group 1 worse in 0.353 and 0.254 of the data sets, and in the other direction in a median of 0.000; the late-weighted test rejects with group 1 better in 0.376, and with it worse in a median of 0.000. The log-rank test's rejections split 0.037 worse and 0.018 better. These are observations of this experiment. Schoenfeld's (1981) abstract adds a less obvious one from theory, that "in certain situations the power of the log rank test falls as the amount of censoring decreases".

Example (choosing the weight after seeing the data). Running all four tests and reporting the smallest p-value does not keep the level. Under no difference the four statistics are correlated but not identical, and the minimum of their p-values falls below 0.05 more often than any one of them. In the simulation above, with no difference between the groups, at least one of the four tests rejected in 0.101 (0.096 to 0.106) of the data sets. That is twice the nominal level. The weight belongs in the analysis plan, chosen from what kind of difference is expected. When the shape of the difference is unknown, there are two alternatives. One is a test that combines several weights with a correction for the maximum. The other is a summary that does not need proportional hazards, such as the difference in restricted mean survival (The Kaplan–Meier Estimator, section "The tail of the curve").

Stratified tests

When a third variable, such as age, affects survival and differs between the groups, the comparison should be made within its levels. The stratified log-rank test computes $U_s$ and $V_s$ within each stratum $s$, with risk sets formed only from that stratum, and uses

$$Z = \frac{\sum_s U_s}{\sqrt{\sum_s V_s}}.$$

Its null hypothesis is that within each stratum the two groups share a hazard; the hazards may differ freely between strata. Theorem 1 applies within each stratum, and the strata are independent groups of people, so $E[\sum_s U_s] = 0$ and $\operatorname{Var}(\sum_s U_s) = \sum_s E[V_s]$. This is the Mantel–Haenszel idea applied twice: over event times within a stratum, and over strata.

Diabetes, smoking and death in NHANES

The unit's NHANES cohort is the 4,114 adults aged 20 to 84 examined in 1999 and 2000 who answered the diabetes and smoking questions. Deaths to the end of 2019 come from the National Death Index linkage (NCHS, 2022), and time is counted in months. The Kaplan–Meier Estimator describes the cohort. Monthly times give many ties, which the hypergeometric variance handles exactly. Age is grouped as 20 to 39, 40 to 59, 60 to 74 and 75 to 84.

Diabetes. The 416 adults with diagnosed diabetes (mean age 62.8, against 47.2 for the others) had 273 deaths.

expected deaths $O_1/E_1$ $Z$
crude 92.1 2.97 19.6
stratified by four age groups 170.8 1.60 8.54
stratified by ten-year age and sex 169.6 1.61 8.68

Within age groups, people with diabetes are expected to account for 170.8 deaths rather than 92.1, because they are older. The ratio of observed to expected falls from 2.97 to 1.60. The difference that remains is still far beyond chance. Weighted versions within the four age groups give $Z = 9.03$ (Gehan), 8.98 (Peto–Peto) and 5.29 (FH(0, 1)).

Smoking. Comparing the 865 current smokers with the 2,168 who never smoked, the current smokers had 271 deaths.

expected deaths $O_1/E_1$ $Z$
crude 217.1 1.25 4.29
stratified by four age groups 159.3 1.70 10.11
stratified by ten-year age and sex 161.5 1.68 10.13

Here stratification works the other way. Current smokers were younger: mean age 44.3, against 46.8 for never smokers. Only 3.6% of them were 75 or older, against 9.5% of never smokers, and the 392 adults aged 75 to 84 account for 28.6% of all deaths in the cohort. In the pooled risk sets the current smokers are compared with an older group, so their expected deaths, 217.1, are too high. Within age groups the expected count falls to 159.3, the excess becomes clear, and the statistic more than doubles. Within the four age groups separately, the statistics are 3.76, 6.85, 7.55 and 0.59, the last for ages 75 to 84. Weighted within age groups: 10.69 (Gehan), 11.22 (Peto–Peto) and 3.53 (FH(0, 1)).

In both comparisons the late-weighted test gives much the smallest statistic. By Proposition 3 that is what happens when the hazard ratio is larger early in follow-up than late. It is also what happens when late event times carry little of the comparison's information, and the tests alone do not separate the two. Whether the hazard ratio changes over time is a question for The Cox Proportional Hazards Model, which tests proportional hazards directly. As on the course pages, sampling weights are not used, so these are comparisons among these adults. Age groups this coarse leave some confounding within them, but the ten-year strata change little.

How many events a trial needs

A trial to be analyzed by the log-rank test is sized in events, not people, because the information in $V$ comes only from deaths. Each death contributes at most $1/4$ (when the groups are equal in size at that time). Here is the derivation, in the heuristic spirit of Proposition 3.

Allocate a share $p$ of patients to group 1, assume proportional hazards with ratio $\psi = e^\theta$, and suppose $D$ deaths are observed. Suppose the share of group 1 in the risk set stays near $p$, which is approximately true when $\theta$ is near 0 and censoring is the same in both groups. Then each death contributes about $\theta p(1 - p)$ to the mean of $U$ and $p(1 - p)$ to its variance. So $Z$ is approximately normal with mean $\theta\sqrt{Dp(1 - p)}$ and variance 1. Power $1 - \beta$ at two-sided level $\alpha$ requires $|\theta|\sqrt{Dp(1 - p)} = z_{1-\alpha/2} + z_{1-\beta}$, which gives Schoenfeld's formula

$$D = \frac{(z_{1-\alpha/2} + z_{1-\beta})^2}{p(1 - p)(\ln\psi)^2},$$

which is $4(z_{1-\alpha/2} + z_{1-\beta})^2/(\ln\psi)^2$ for equal allocation. This is the event count usually attributed to Schoenfeld. His 1981 abstract states that under proportional hazards and equal censoring in the two groups "the sample size formula for the F test used to compare exponential samples is shown to be valid for the log rank test", and his 1983 abstract that a formula was derived when covariates are incorporated. We did not check the printed form in either paper.

A second version, labeled Freedman's below, keeps the probability exact instead of linearizing it. With equal shares at risk, a death falls in group 1 with probability $\psi/(1 + \psi)$, so its contribution to the mean of $U$ is $\psi/(1 + \psi) - 1/2 = (\psi - 1)/[2(1 + \psi)]$, with variance $1/4$. Then $Z$ has mean $\sqrt D\,(\psi - 1)/(1 + \psi)$, and

$$D = (z_{1-\alpha/2} + z_{1-\beta})^2\Big(\frac{1 + \psi}{1 - \psi}\Big)^2.$$

Freedman (1982) published tables of the numbers of patients required for trials analyzed by the log-rank test (his abstract). The formula above is the event count that this derivation gives. It carries his name here for that connection only; his paper was not available to check whether this is the formula behind his tables. The number of patients is $D$ divided by the probability that a patient's death is observed during the trial.

For $\psi = 0.6$, 5% level and 80% power, Schoenfeld's formula gives 120.3 events and Freedman's 125.6; for $\psi = 0.75$, 379.4 and 384.6. Both are approximations, and simulation shows how good. The script runs trials with equal allocation, $2D$ patients ($D$ per arm, all entering at time 0), exponential lifetimes and no other censoring, stopped at the planned number of events $D$ (rounded up), so half the patients have died when a trial stops. There are 30 runs (seeds from 84,000,000) of 1,000 trials each:

hazard ratio formula events power
0.6 Schoenfeld 121 0.778 (0.768 to 0.787)
0.6 Freedman 126 0.797 (0.790 to 0.803)
0.75 Schoenfeld 380 0.801 (0.786 to 0.807)
0.75 Freedman 385 0.806 (0.796 to 0.814)

At a hazard ratio of 0.6 Schoenfeld's count falls a little short of 80% power in this experiment. Unbalanced risk sets are not the reason. When the trials stop, group 1 makes up 0.59 of those still at risk, but with five times as many patients per arm, where it makes up 0.51, the same 121 events give power 0.779 (0.772 to 0.789), no better. Most of the shortfall comes from the linearization. With balanced risk sets a death contributes $(\psi - 1)/[2(1 + \psi)] = -0.1250$ to the mean of $U$, not $\theta/4 = -0.1277$, and with that drift 121 events predict power 0.785. Using the variance per death under the alternative, $\psi/(1 + \psi)^2$, instead of $1/4$ raises the prediction to 0.793, and the exact binomial calculation for 121 deaths with balanced risk sets gives 0.782; the rest of the gap is within the spread of the simulation. Freedman's formula uses the exact drift. Theory agrees that the drift of the risk sets is not the cause: with one death at a time the drift per unit of variance is $(\psi - 1)/(\pi\psi + 1 - \pi)$, which for $\psi < 1$ grows in size as group 1's share $\pi$ rises, so the imbalance in these trials costs no information per death. At 0.75 the two drifts are closer and both counts are close to 80%. Power Analysis for Evaluations treats the general logic of choosing a sample size from an effect, a level and a power.

What is proved, what is cited, what the experiment shows

Proved here. The hypergeometric distribution of each table under the null hypothesis in discrete time, with its mean and variance (Lemma 1); that the deviations are uncorrelated martingale increments, so $E[U] = 0$ and $\operatorname{Var}(U) = E[V]$, and the same for weighted and stratified versions (Theorem 1); that the observed-over-expected chi-square never exceeds the log-rank statistic (Proposition 1); Gehan's pairwise statistic as the log-rank test weighted by the number at risk (Proposition 2); the optimal weights given the drift (Proposition 3, exact once the drift is granted). Schoenfeld's and Freedman's event counts are derived heuristically.

Cited. The log-rank test as the score test of the Cox model, with the exact and Breslow informations (The Cox Proportional Hazards Model, Theorem 3); the normal limit by martingale theory and the class of weighted statistics (Harrington and Fleming, 1982; Aalen, 1978; Andersen et al., 1993); the asymptotic distribution under alternatives and the validity of the exponential sample-size formula for the log-rank test (Schoenfeld, 1981) and its extension with covariates (Schoenfeld, 1983); Freedman's (1982) tables; the origins of the tests (Mantel, 1963; Peto and Peto, 1972; Gehan, 1965).

Computed. The 6-MP and NHANES statistics are exact computations, checked against the Cox partial likelihood of the unit's fitting code at $\beta = 0$. Theorem 1 is checked exactly by enumeration on a small cohort. The level, power and sample-size results are seeded simulations reported as medians and interquartile ranges over 30 runs; they describe the scenarios simulated.

Worked problems

These problems were written for this article; they use the 6-MP trial and small invented samples. The script tools/experiments/statistics/log-rank-test.py recomputes every number.

Problem 1 (one event time by hand). At 6 weeks in the 6-MP trial, 33 children are at risk, 21 on 6-MP, and 3 relapse, all on 6-MP. Compute $E_j$, $V_j$ and the contribution to $U$, and say what the time contributes to the evidence.

Solution. $E_j = 21/11 = 1.9091$, $V_j = 0.6508$, and the contribution to $U$ is $3 - 1.9091 = 1.0909$.

By Lemma 1 with $n = 33$, $n_1 = 21$, $n_0 = 12$ and $d = 3$: $E_j = 21 \times 3/33 = 21/11$, and $V_j = 21 \times 12 \times 3 \times 30/(33^2 \times 32) = 22680/34848 = 0.6508$. All three relapses at this time are on 6-MP, more than the 1.91 expected, so this time counts against the treatment. It is outweighed by the other times: in total the 6-MP group had 9 relapses against 19.25 expected.

Problem 2 (Gehan's identity on four patients). Group 1 has a death at 3 and a censoring at 5; group 0 has deaths at 2 and 4. Compute Gehan's pairwise score sum and the log-rank and Gehan-weighted $U$.

Solution. The pairwise sum is $W = 2$, the Gehan-weighted $U_w = -2$, and the log-rank $U = -2/3$.

Pairs (group 1, group 0): (3, 2): the group 0 member dies first, $+1$. (3, 4): the group 1 member dies first, $-1$. (5+, 2): $+1$. (5+, 4): the group 0 member dies at 4 while the group 1 member is still under observation, $+1$. So $W = 2$. The event times are 2, 3 and 4, with $(n_j, n_{1j}, d_{1j})$ equal to $(4, 2, 0)$, $(3, 2, 1)$ and $(2, 1, 0)$, one death each. So $E_j$ is $1/2$, $2/3$ and $1/2$, and the deviations are $-1/2$, $1/3$ and $-1/2$, which sum to $U = -2/3$. Weighting by $n_j$: $4(-1/2) + 3(1/3) + 2(-1/2) = -2 = -W$, as Proposition 2 says.

Problem 3 (unequal allocation). Two patients are assigned to the new treatment for each one assigned to control. How many deaths does the trial need to detect a hazard ratio of 0.5 with 90% power at the two-sided 5% level, and how does that compare with equal allocation?

Solution. 98.4, so 99 deaths, against 87.5 (88 deaths) with equal allocation.

Schoenfeld's formula with $p = 2/3$, $p(1 - p) = 2/9$, $z_{0.975} = 1.960$, $z_{0.9} = 1.2816$ and $(\ln 0.5)^2 = 0.4805$ gives $D = (3.2415)^2/(0.2222 \times 0.4805) = 98.4$. With $p = 1/2$, $p(1 - p) = 1/4$ and $D = 87.5$. The ratio is $(1/4)/(2/9) = 9/8$: unequal allocation costs 12.5% more events, the price of the smaller control arm. The formula linearizes the drift and assumes the risk set keeps the allocation ratio. With a hazard ratio as far from 1 as 0.5, the simulation in the section "How many events a trial needs" suggests rounding up generously.

Summary

Result Statement
Table at each event time under $H_0$, $d_{1j}$ given the past and $d_j$ is hypergeometric: mean $n_{1j}d_j/n_j$, variance $n_{1j}n_{0j}d_j(n_j - d_j)/[n_j^2(n_j - 1)]$
Statistic $Z = (O_1 - E_1)/\sqrt{\sum_j V_j}$; $X^2 = Z^2$ on 1 degree of freedom
Why the sum is valid deviations are uncorrelated martingale increments: $\operatorname{Var}(U) = E[V]$
Score test the Cox model's score test at $\beta = 0$ (exact ties likelihood)
Shortcut $\sum(O - E)^2/E \le X^2$
Weights Gehan $n_j$ (= generalized Wilcoxon), Peto–Peto $\hat S(t^-)$, late $1 - \hat S(t^-)$; best weight $\propto$ log hazard ratio
Stratified sum $U_s$ and $V_s$ over strata
Events needed $D = (z_{1-\alpha/2} + z_{1-\beta})^2/[p(1 - p)(\ln\psi)^2]$ (Schoenfeld)

What we have learned

The log-rank test forms a $2 \times 2$ table at every death and adds the deviations of the observed deaths in one group from their hypergeometric expectations. The tables are dependent, yet their variances add, because each deviation has mean zero given the past: a martingale increment. The test is the Cox model's score test, and, to first order (Proposition 3), it is most powerful when the hazard ratio is constant. Weighted versions trade that power for sensitivity to early or late differences, and when hazards cross, the log-rank test can miss a difference entirely. On the 6-MP trial it gives $X^2 = 16.79$. On the NHANES adults, stratifying by age cut the diabetes comparison from $Z = 19.6$ to 8.54 and raised the smoking comparison from 4.29 to 10.11. Those are the two directions in which confounding by age can act.

The test says whether two curves differ, not by how much. It adjusts for one grouped variable at a time. The Cox Proportional Hazards Model estimates the hazard ratio itself, adjusts for several variables at once, and tests whether the ratio is constant.

References