Ten patients each took two sleeping drugs, and the paired t-test found that the second drug added sleep, with $p = 0.0028$. That $p$-value rests on a model: the differences are a sample from a normal population. The $t$-tests and the analysis of variance of the previous units all rest on such a model, with normal populations that share a variance. The tests hold up well when the model is roughly right. Small skewed samples, wild values and measurements that are only ordinal (a pain score, a stage of disease) strain them. Can we test the same questions without assuming any shape for the population at all?
Nonparametric methods do it by replacing the measurements with their signs or their ranks. If the groups are the same, every assignment of the signs or ranks is equally likely. The null distribution of the test statistic can then be found by counting, and it does not depend on the shape of the population. The tests are exact for any sample size and any continuous population.
They are not free. A test that uses only ranks gives up some information. When the data really are normal it is slightly less powerful, and the unit measures how much: about 5% more observations for the rank tests, and over half again as many for the sign test. When the data have heavy tails or skewness the comparison reverses, often by a wide margin.
A rank test also answers a question of its own:
- the sign test asks whether positive and negative differences are equally common;
- the signed-rank test asks whether the differences are symmetric about zero;
- the rank-sum and Kruskal–Wallis tests ask whether values in one group tend to exceed those in another.
Each of these coincides with "are the centers equal?" only under further assumptions. The sign test needs continuous data, the signed-rank test symmetric differences, and the others populations of the same shape. The unit shows examples in which a rank test and a test of means reach different conclusions, and both are correct.
The unit follows the earlier designs in turn:
- Paired data: the sign and signed-rank tests, and with them the unit's tools of exact counting, tie corrections and efficiency.
- Two independent groups: the rank-sum test, which is Mann and Whitney's $U$.
- Several groups: the Kruskal–Wallis test, the analysis of variance of the ranks.
The examples continue those of the earlier units. Student's (1908) sleep data serve the paired tests. Blood pressures from the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018 serve the comparisons of independent groups, including the four age groups of the analysis of variance. The course's formula sheets end with the analysis of variance, so the notation is the standard one of the original papers:
- $W^+$ and $W^-$ for the signed-rank sums;
- $W$ and $U$ for the rank-sum statistics;
- $H$ for Kruskal and Wallis's statistic;
- $n$ (or $n_1$, $n_2$, $N$) for sample sizes.
Every number comes from a script in the site's repository.
Articles
- The Sign Test and the Wilcoxon Signed-Rank Test: how can paired differences be tested by their signs alone, or by their signs and ranks? The sign test as an exact binomial test. Wilcoxon's signed-rank statistic and its exact null distribution from the $2^n$ equally likely sign patterns, enumerated for drug 1 of the sleep data ($p = 0.359$). Its mean $n(n+1)/4$ and variance $n(n+1)(2n+1)/24$ derived, with the corrections for ties and zero differences, worked on repeated NHANES readings. Wilcoxon's (1945) maize example recomputed. What each test tests, shown on a population whose median difference is zero but whose positive differences outnumber the negative ones. Efficiencies $3/\pi$ and $2/\pi$ under normality, and power for five shapes by simulation.
- The Wilcoxon Rank-Sum Test: how do we compare two independent groups by ranks, and what does a rejection mean? The rank sum $W$ and Mann and Whitney's (1947) $U$ shown to be the same test, with $U/(n_1 n_2)$ as an estimate of $P(X > Y)$. The exact distribution from Mann and Whitney's recurrence and the normal approximation with ties, worked on 8 men and 8 women from NHANES. Why the test compares medians only under a shift model, with simulations of equal medians and of unequal spreads. Asymptotic relative efficiency $3/\pi = 0.955$ for normal data and up to 3 for skewed data, and the effect of one gross error.
- The Kruskal-Wallis Test: how do we compare several groups by ranks? $H$ derived as the analysis of variance of the ranks, with its computing formula and tie correction. Its chi-squared approximation, checked by exact enumeration of Kruskal and Wallis's (1952) example and by simulation, and its reduction to the rank-sum test for two groups. The four NHANES age groups worked by hand ($H = 9.141$, $p = 0.027$). Its behavior when spreads differ, its power against the classical and Welch $F$ tests, and Dunn's (1964) comparisons of mean ranks.
Related topics
Each test has a counterpart in an earlier unit:
- the signed-rank test in the paired t-test;
- the rank-sum test in the two-sample t-test;
- the Kruskal–Wallis test in one-way analysis of variance, whose follow-up comparisons use the corrections of Multiple Comparisons and the Bonferroni Correction.
The binomial distribution behind the sign test returns in the next unit, Inference on Proportions, which begins with the normal approximation to the binomial. In information retrieval, Significance Testing applies the sign and signed-rank tests to comparisons of search engines over the same queries. It sets them beside the randomization test, which computes a null distribution by the same reasoning from the original values instead of their ranks.
References
- Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3), 241–252. record · 2 pages
- Fay, M. P., and Proschan, M. A. (2010). Wilcoxon–Mann–Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics Surveys, 4, 1–39. record · 3 pages
- Kruskal, W. H., and Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621. record · 4 pages
- Mann, H. B., and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50–60. record · 2 pages
- NIST/SEMATECH. e-Handbook of Statistical Methods, §7.4.1 (the Kruskal-Wallis test). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/ record · 30 pages
- Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages
- Student (1908). The probable error of a mean. Biometrika, 6(1), 1–25. record · 11 pages
- Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83. record · 3 pages