A study measures the systolic blood pressure of 8 men and 8 women and asks whether one group tends to run higher than the other. The two-sample t-test compares the two sample means, and for small samples it needs populations that are close to normal. Blood pressure is right-skewed, and a single unusually high reading moves a mean of eight values a long way. An alternative is to put the sixteen values in order. Do the men's values sit higher in that ordering than chance would put them?
That is the question of the Wilcoxon rank-sum test. Frank Wilcoxon (1945) proposed it for "unpaired experiments" in the same short paper that introduced the signed-rank test: rank all the observations together and add up the ranks of one group. Two years later H. B. Mann and D. R. Whitney (1947) proposed a statistic $U$ counting how often an observation of one group exceeds one of the other. They worked out its exact distribution by a recurrence, proved that it is asymptotically normal, and noted that "Wilcoxon proposed an equivalent test". The two statistics differ by a constant. The test goes by both names: the rank-sum test, the Mann–Whitney $U$ test, or the Wilcoxon–Mann–Whitney test.
First we show that $W$ and $U$ are the same test, and what $U$ estimates. Then we work the 8 men and 8 women by hand, find the exact null distribution by counting arrangements, and derive the normal approximation with its correction for ties. Then come two points that the shorthand "nonparametric test of medians" obscures: what hypothesis the test actually examines, and how it compares in power with the $t$-test, under normality and otherwise. The article assumes The Sign Test and the Wilcoxon Signed-Rank Test, where exact counting, the tie correction and asymptotic relative efficiency are introduced.
The setting and the hypotheses
Two independent random samples are drawn: $x_1, \dots, x_{n_1}$ from a population with distribution function $F$, and $y_1, \dots, y_{n_2}$ from one with distribution function $G$. There are $N = n_1 + n_2$ observations in all. Independence is the same requirement as for the two-sample t-test: no observation in one group is matched to one in the other. For paired data, the signed-rank test is the counterpart.
The null hypothesis is that the two populations are the same,
$$H_0: F = G.$$
Under $H_0$ the $N$ observations are $N$ independent draws from one distribution, which is all the exact distribution of the test needs.
The alternative that Mann and Whitney (1947) had in mind was that one variable is stochastically larger than the other. $X$ is stochastically larger than $Y$ if $P(X > a) \ge P(Y > a)$ for every value $a$, with strict inequality somewhere. Values of $X$ then tend to be higher across the whole range. Their motivating study measured how long infected rats survived with and without a protective drug. There, "the only relevant alternative" to no effect was that survival times with the drug were stochastically larger. A shift in location, $G(a) = F(a + \Delta)$, is the simplest case. What the test detects when the alternative is neither is the subject of a later section.
Ranks, W and U
Rank all $N$ observations together from 1 (smallest) to $N$ (largest), tied values receiving the mean of the ranks they span. The rank-sum statistic is
$$W = \sum_{i=1}^{n_1} \operatorname{rank}(x_i),$$
the sum of the ranks of the first sample. The Mann–Whitney statistic counts pairs:
$$U = \#\{(i, j): x_i > y_j\} + \tfrac12\,\#\{(i, j): x_i = y_j\},$$
the number of the $n_1 n_2$ pairs, one observation from each group, in which the $x$ is the larger, with ties counted as one half.
The two are the same test. Here is the idea in one sentence: an $x$'s rank counts the observations at or below it, and the $y$'s among them are exactly the pairs that $U$ counts.
Order the first sample, $x_{(1)} < x_{(2)} < \cdots < x_{(n_1)}$, and suppose for the moment that there are no ties. The rank of $x_{(k)}$ in the combined sample is the number of observations at or below it. That is $k$ from its own sample (itself and the $k - 1$ smaller $x$'s), plus the number of $y$'s below it, $c_k$. So
$$W = \sum_{k=1}^{n_1}(k + c_k) = \frac{n_1(n_1+1)}{2} + \sum_{k=1}^{n_1} c_k = \frac{n_1(n_1+1)}{2} + U,$$
because $\sum c_k$ counts every pair $(x, y)$ with $x > y$ exactly once. With ties, a value tied with $y$'s gets the average of the ranks it shares with them. That adds one half for each tied $y$, matching the half counts in $U$. Ties within the first sample shift ranks among themselves without changing their sum. Hence
$$U = W - \frac{n_1(n_1+1)}{2}$$
always. Since $n_1$ is fixed, any test based on one is a test based on the other. The smallest possible $W$, when the $x$'s occupy the lowest ranks, is $n_1(n_1+1)/2$, which is where $U = 0$. Counting pairs the other way gives $U' = n_1 n_2 - U$, the number of pairs in which the $y$ is larger.
Mann and Whitney (1947, §3) defined $U$ as the number of times a $y$ precedes an $x$ in the ordered sequence, which is this $U$. They gave its relation to Wilcoxon's rank sum $T$ of the $y$'s as $U = mn + m(m+1)/2 - T$, with $n$ $x$'s and $m$ $y$'s. It is the same identity, written for the other sample.
The second form explains what the statistic estimates. Each of the $n_1 n_2$ pairs contributes 1, $\tfrac12$ or 0 to $U$, and each pair's expected contribution is
$$\theta = P(X > Y) + \tfrac12 P(X = Y),$$
so $U/(n_1 n_2)$ is an unbiased estimate of $\theta$. Under $H_0$, $\theta = \tfrac12$ by symmetry. Fay and Proschan (2010, §5.1.1) call $\theta$ the Mann–Whitney functional; it is also called the probability of superiority. Two values from the NHANES population of 2017 to 2018 set the scale:
- a randomly chosen man has a higher systolic blood pressure than a randomly chosen woman with probability $\theta = 0.552$ (ties, at 1.0% of pairs, split);
- an adult aged 60 or over is higher than one aged 18 to 39 with $\theta = 0.825$.
Worked example: 8 men and 8 women
Now the 8 men and 8 women. The script drew them at random from the NHANES adults (seed 1). Their systolic pressures, in mm Hg, are
- men: 140.7, 109.3, 124.7, 109.3, 143.3, 170.7, 124.0, 124.7;
- women: 115.3, 137.3, 140.0, 107.3, 119.3, 121.3, 112.7, 108.0.
Ranked together (M for a man, W for a woman):
| Value | 107.3 | 108.0 | 109.3 | 109.3 | 112.7 | 115.3 | 119.3 | 121.3 |
|---|---|---|---|---|---|---|---|---|
| Group | W | W | M | M | W | W | W | W |
| Rank | 1 | 2 | 3.5 | 3.5 | 5 | 6 | 7 | 8 |
| Value | 124.0 | 124.7 | 124.7 | 137.3 | 140.0 | 140.7 | 143.3 | 170.7 |
|---|---|---|---|---|---|---|---|---|
| Group | M | M | M | W | W | M | M | M |
| Rank | 9 | 10.5 | 10.5 | 12 | 13 | 14 | 15 | 16 |
Two pairs of men share values and receive midranks 3.5 and 10.5. The men's ranks sum to $W = 82$ and the women's to 54; together they make $16 \cdot 17/2 = 136$. Then
$$U = 82 - \frac{8 \cdot 9}{2} = 46,$$
and counting directly, the men are higher in 46 of the 64 man–woman pairs. Each of the three highest men is above all 8 women, the man at 124.0 and the two at 124.7 are above 6 women each, and the two at 109.3 above 2 each: $3 \times 8 + 3 \times 6 + 2 \times 2 = 46$. The women are higher in $U' = 64 - 46 = 18$ pairs. The estimate of $\theta$ is $46/64 = 0.72$.
Check your understanding. The men are higher in 72% of the pairs, where chance alone would give about 50%. With 8 people per group, is that strong evidence that men run higher?
It is not, as the exact distribution now shows.
The exact null distribution
Under $H_0$ every way of choosing which 8 of the 16 positions in the ordering belong to men is equally likely. There are $\binom{16}{8} = 12{,}870$ of them, and the null distribution of $U$ is the distribution of the pair count over all of them. This is the counting argument of the signed-rank test, with arrangements of group labels in place of sign patterns.
Mann and Whitney (1947, §4) found the distribution with a recurrence. The idea is to look at the largest observation: it belongs to one group or the other, and either way what remains is a smaller problem of the same kind. Code each $x$ as 0 and each $y$ as 1, and let $p_{n_1, n_2}(u)$ be the probability that a random sequence of $n_1$ zeros and $n_2$ ones has $U = u$. Look at the last element of the sequence.
- It is an $x$ with probability $n_1/(n_1 + n_2)$. Then that $x$ is above all $n_2$ $y$'s, contributing $n_2$ to $U$, and the remaining sequence is a random arrangement of $n_1 - 1$ zeros and $n_2$ ones.
- It is a $y$ with probability $n_2/(n_1+n_2)$, contributing nothing.
So
$$p_{n_1, n_2}(u) = \frac{n_1}{n_1 + n_2}\,p_{n_1 - 1,\, n_2}(u - n_2) + \frac{n_2}{n_1 + n_2}\,p_{n_1,\, n_2 - 1}(u),$$
starting from $p_{n_1, 0}$ and $p_{0, n_2}$, which put all their probability on $u = 0$. Mann and Whitney used it to tabulate $P(U \le u)$ for all sample sizes up to 8. The script's implementation reproduces their Table I: for example, $P(U \le 13) = 0.025$ and $P(U \le 12) = 0.019$ for 8 and 8, and $P(U \le 4) = 0.048$ for 5 and 5.
The distribution is symmetric about $n_1 n_2/2 = 32$. For the observed $U = 46$, or equivalently $U' = 18$, the two-sided $p$-value is
$$p = 2\,P(U \le 18) = 2 \times 0.0803 = 0.161.$$
This sample has ties, and the recurrence assumes none. The exact distribution conditional on the observed midranks treats every one of the 12,870 assignments of these sixteen ranks to the two groups as equally likely. It can be computed by a similar recursion over the ranks, adding them one at a time as for the signed-rank test, and it gives $p = 0.152$. Either way, 8 men and 8 women give no evidence that the populations differ, and the pooled $t$-test agrees: $t = 1.267$, $p = 0.226$. The population difference is real, $\theta = 0.552$, but small. As for the $t$-test, 16 people are far too few to see it.
The recurrence also gives critical values. For two samples of 8, a two-sided test at $\alpha = 0.05$ rejects when $\min(U, U') \le 13$, with exact level 0.0499. Very small samples cannot reject at all. With 3 in each group the most extreme arrangement has two-sided probability $2/\binom{6}{3} = 0.1$. Fay and Proschan (2010, §5.3.3) note that at $\alpha = 0.05$ the exact test needs at least 4 per group to have any possibility of rejecting, where the $t$-test needs only 2.
The normal approximation
Counting 12,870 arrangements is easy for a computer, but the count grows quickly with the sample sizes. For larger samples the distribution of $W$ is approximated by a normal distribution with its exact mean and variance. Here is the idea: under $H_0$ the first group's ranks are just a random draw from the numbers 1 to $N$, so $W$ behaves like the total of a sample drawn without replacement.
The formal version. Kruskal and Wallis (1952, §3.1) used this argument for their test. Under $H_0$ the ranks of the first sample are a random sample of $n_1$ drawn without replacement from the integers $1, \dots, N$. That population has mean $(N+1)/2$ and variance $(N^2 - 1)/12$. The sum of $n_1$ values drawn without replacement has mean $n_1$ times the population mean. Its variance is $n_1$ times the population variance times the finite population correction $(N - n_1)/(N - 1)$:
$$E(W) = \frac{n_1(N+1)}{2}, \qquad \operatorname{Var}(W) = n_1 \cdot \frac{N^2 - 1}{12} \cdot \frac{N - n_1}{N - 1} = \frac{n_1 n_2 (N+1)}{12}.$$
Since $U$ differs from $W$ by a constant, $E(U) = n_1 n_2/2$ and $\operatorname{Var}(U) = n_1 n_2(N+1)/12$. These are the values Mann and Whitney derived from their recurrence (their equations 2 and 5). They proved that the limiting distribution is normal however $n_1$ and $n_2$ grow, by showing that every moment of the standardized $U$ approaches the corresponding moment of the standard normal. They noted that for 8 and 8 it already "differs only a negligible amount from the normal distribution". The standardized statistic, with a continuity correction of one half, is
$$z = \frac{\left|W - \dfrac{n_1(N+1)}{2}\right| - \dfrac12}{\sqrt{\dfrac{n_1 n_2 (N+1)}{12}}}.$$
Ties lower the variance of the ranks, and so of $W$. As the signed-rank article derives, each group of $t$ tied values reduces the total sum of squares of the ranks by $(t^3 - t)/12$. The population variance of the ranks therefore falls by $\sum(t^3 - t)/(12N)$, and
$$\operatorname{Var}(W) = \frac{n_1 n_2}{12}\left[(N + 1) - \frac{\sum (t^3 - t)}{N(N - 1)}\right],$$
the sum running over the groups of tied values in the combined sample.
Check your understanding. The 8 men and 8 women include two tied pairs. Will the tie correction make $z$ larger or smaller?
Slightly larger, because the variance in the denominator shrinks. Here $E(W) = 8 \cdot 17/2 = 68$ and $\operatorname{Var}(W) = 64 \cdot 17/12 = 90.67$ without the correction. The two tied pairs reduce it to 90.40, and
$$z = \frac{|82 - 68| - 0.5}{\sqrt{90.40}} = \frac{13.5}{9.508} = 1.420, \qquad p = 0.156,$$
close to the exact values 0.161 and 0.152. At the exact critical value for 8 and 8, $U = 13$, the normal approximation with continuity correction gives a two-sided probability of 0.052 against the exact 0.0499.
Two larger samples
The approximation is meant for larger samples, and the earlier units supply two. The two-sample t-test article drew 20 men and 20 women (seed 1) and found $t = 0.674$, $p = 0.504$. The rank-sum test on the same 40 values gives $W = 432.5$ for the men, so $U = 432.5 - 210 = 222.5$ of 400 pairs. That is $U/400 = 0.556$, near the population's 0.552. The other ingredients are:
- $E(W) = 410$;
- six groups of ties (five pairs and a triple), reducing the variance from 1366.67 to 1365.51;
- the continuity correction.
Then
$$z = \frac{|432.5 - 410| - 0.5}{\sqrt{1365.51}} = 0.595, \qquad p = 0.552.$$
The exact conditional $p$-value is 0.551. Without the continuity correction $z = 0.609$ and $p = 0.543$.
The same article's second sample, 10 adults aged 60 and over and 30 aged 18 to 39, gave Welch's $t$-test $p = 0.0068$. The older adults' ranks sum to $W = 301$, so $U = 301 - 55 = 246$ of 300 pairs. In 82% of the pairs the older adult has the higher pressure, close to the population's 82.5%. Then $z = 2.984$, $p = 0.0028$, and the exact conditional $p$-value is 0.0019. Here the rank-sum test gives the stronger evidence of the two, for a reason taken up under robustness below.
When the test rejects, as it does here, what exactly has it shown?
What the test tests
The rank-sum test is often described as a test of whether two medians are equal. That is correct under an assumption that is easy to forget, and wrong without it.
Under a shift model the second population is the first moved by a constant $\Delta$: $G(a) = F(a - \Delta)$. Then the two medians differ by $\Delta$, and so do the two means. $\theta$ is above, at or below one half exactly as $\Delta$ is above, at or below zero. The test of $F = G$ is then a test of $\Delta = 0$, which is at once a test of equal medians and of equal means. This is the setting in which the test was derived, and in which its efficiency is computed below.
Without a shift model the test responds to $\theta$. Mann and Whitney (1947, §5) proved that the test is consistent against every alternative in which one variable is stochastically larger than the other: its power tends to 1 as both samples grow. The underlying reason is that $U/(n_1 n_2)$ estimates $\theta$. Whenever $\theta \ne \tfrac12$, the statistic drifts away from its null mean faster than its standard deviation grows. Fay and Proschan (2010) list this among several interpretations of the same decision rule: under the assumption that $F = G$ when the null holds, rejecting means $\theta \ne \tfrac12$. Medians do not enter. Two consequences follow, and the script checked both by simulation (10,000 pairs of samples for each of 30 seeds; medians of the rejection rates).
Equal medians, different shapes. Take an exponential distribution shifted to have median 0 ($X = E - \ln 2$, with $E$ exponential with mean 1) and a standard normal $Y$. Both have median 0.
Check your understanding. As the samples grow, will the rank-sum test reject the equality of these two populations about 5% of the time, or more and more often?
More and more often. $P(X > Y) = 0.557$, because the exponential's long right tail puts much of its probability above the normal's typical values, while its left side is cut off at $-0.69$. The rank-sum test detects that difference. It rejected in 9.4% of samples with 20 per group, 16.6% with 50 and 50.5% with 200. Each rejection is a correct statement that $\theta \ne \tfrac12$, and an incorrect one if reported as "the medians differ".
The NHANES populations give the converse. Shifting every woman's value up by 3.33 mm Hg makes the women's median equal the men's, 124.67 mm Hg. For these two populations $\theta = 0.496$, very close to one half. The rank-sum test rejected in 5.0%, 5.1% and 5.2% of samples of 50, 200 and 800 per group: it sees no difference, correctly. Welch's $t$-test compares means, 127.41 against 128.70 after the shift. It rejected in 6.0%, 9.5% and 25.1%, also correctly, because the means differ. The two tests are both valid; they answer different questions and so give different answers.
Equal centers, different spreads. If the two populations are normal with the same mean but standard deviations 1 and 3, then by symmetry $\theta = \tfrac12$ exactly, yet $F \ne G$. Is the rank-sum test a valid test of $\theta = \tfrac12$ here?
Check your understanding. The analysis of variance rejected too rarely when the largest groups had the largest spreads. With 10 observations at SD 1 and 40 at SD 3, will the rank-sum test reject too often or too rarely?
Too rarely, like the pooled $t$-test. The simulation says the rank-sum test is not valid here:
| Sizes (SD 1, SD 3) | Rank-sum | Welch $t$ | Pooled $t$ |
|---|---|---|---|
| 25 and 25 | 0.065 | 0.050 | 0.053 |
| 10 and 40 | 0.009 | 0.050 | 0.001 |
| 40 and 10 | 0.135 | 0.051 | 0.257 |
The variance formula for $U$ assumed $F = G$. When the spreads differ, the true variance of $U$ is different: larger when the more variable population has the smaller sample. The test's level then moves away from $\alpha$ in the same direction as the pooled $t$-test's, if less extremely. This is the Behrens–Fisher problem of the two-sample t-test article in rank form. Fay and Proschan (2010, §5.1.2) call it the nonparametric Behrens–Fisher problem. They note that the rank-sum test is not valid for it, and describe a test with a variance estimate that does not assume $F = G$ (Brunner and Munzel's).
The practical point is that switching from the $t$-test to the rank-sum test does not protect against unequal variances. With equal group sizes the distortion is modest (0.065 here); with unequal sizes it can be serious.
The summary is a rule for reporting:
- Under a shift model, or when the populations are thought to have the same shape, the test compares locations and may be reported as comparing medians (or means).
- Otherwise it should be reported as showing, or failing to show, that values in one group tend to be larger than in the other, with $U/(n_1 n_2)$ as the estimate of how often.
In the NHANES example, "older adults had higher blood pressure than younger adults in an estimated 82% of pairs" is both accurate and informative. Under a shift model, the median of the $n_1 n_2$ differences $x_i - y_j$ estimates the shift. Inverting the test gives a confidence interval for it, a construction due to Hodges and Lehmann (Fay and Proschan, 2010, §5.1.1).
When the shift model does hold, the remaining question is the one the signed-rank test raised: how much power does ranking cost?
Efficiency and robustness
Asymptotic relative efficiency
The measure is the asymptotic relative efficiency defined in the signed-rank article: the limiting ratio of the sample sizes two tests need for the same power against small shifts. Under a shift model with a continuous density $f$ of variance $\sigma^2$, the efficiency of the rank-sum test to the two-sample $t$-test is
$$\text{ARE} = 12\,\sigma^2\left(\int f(x)^2\,dx\right)^2$$
(Fay and Proschan, 2010, §5.3.4). It is the same expression as for the signed-rank test against the paired $t$-test, and the slope argument there carries over, with $U/(n_1 n_2)$ estimating $\theta$ in place of the Walsh averages. For the normal distribution, $\int f^2 = 1/(2\sigma\sqrt\pi)$, so
$$\text{ARE} = 12\sigma^2 \cdot \frac{1}{4\pi\sigma^2} = \frac{3}{\pi} = 0.955.$$
When the data are normal, where the $t$-test is the best possible test, the rank-sum test needs only about 5% more observations to match it. For other shapes the script evaluated the integral numerically:
| Shape | ARE, rank-sum vs $t$ |
|---|---|
| Normal | 0.955 |
| Uniform | 1.000 |
| Logistic | 1.097 |
| Contaminated normal (10% with three times the SD) | 1.373 |
| Laplace (double exponential) | 1.500 |
| $t$ with 3 degrees of freedom | 1.900 |
| Exponential | 3.000 |
The efficiency can be far above 1 and never falls far below it. Fay and Proschan (2010, §5.3.4) give the smallest possible value over all shapes, $108/125 = 0.864$, attained by a parabolic density. They also show that for $t$ distributions the rank-sum test is the more efficient for 18 or fewer degrees of freedom. In their recommendations (§5.3.6) they cite a review of high-quality real measurements, which found that $t$ distributions with 3 to 10 degrees of freedom often fit better than the normal. They conclude that the rank-sum test "will often be asymptotically more powerful than the t-test for real high quality data".
Power at 20 per group
Do the asymptotic numbers hold at a realistic size? The script checked the efficiencies with 20 observations per group, with the same design as for the signed-rank test:
- the shift was $0.909\,\sigma$, which gives the pooled $t$-test power 0.80 for normal data at $\alpha = 0.05$;
- every shape was scaled to the same standard deviation;
- the rank-sum test used its exact critical value, $\min(U, U') \le 127$, with exact level 0.0491.
| Shape | Power, pooled $t$ | Power, rank-sum | Level of $t$ / rank-sum when $\Delta = 0$ |
|---|---|---|---|
| Normal | 0.799 | 0.778 | 0.050 / 0.050 |
| Uniform | 0.800 | 0.736 | 0.051 / 0.050 |
| Logistic | 0.800 | 0.820 | 0.050 / 0.049 |
| Laplace | 0.801 | 0.880 | 0.049 / 0.048 |
| Contaminated normal | 0.805 | 0.891 | 0.047 / 0.048 |
| $t$, 3 df | 0.843 | 0.950 | 0.046 / 0.050 |
| Exponential | 0.804 | 0.934 | 0.047 / 0.050 |
Under a shift model both tests hold their level for every shape. The $t$-test does so by the central limit theorem, slightly conservatively for heavy tails; the rank-sum test does so exactly. Their powers differ in the way the efficiencies predict. The $t$-test's power is nearly the same for every shape, because it depends on the data mainly through the means and the standard deviation, which have the same values in every scenario. The rank-sum test's power depends on how concentrated the distribution is, which is what $\int f^2$ measures. It is high for peaked and skewed shapes and lowest for the flat uniform.
One gross error
Robustness in a narrower sense is the effect of a few bad values. In the sample of 10 older and 30 younger adults, suppose the highest older value, 169.3 mm Hg, had been entered as 1693.3, a slipped decimal point.
Check your understanding. The error is in the direction of the true difference: it makes the older group look higher still. What happens to each test's $p$-value?
Welch's $t$-test gave $p = 0.0068$ on the correct data. It now gives $t = 1.152$ and $p = 0.279$. The error raises the older group's mean from 142.3 to 294.7 mm Hg, which makes the difference larger. But it raises the group's standard deviation from 24.3 to 491.9, which swamps the difference. The rank-sum test gives exactly the same result as before, $U = 246$ and $p = 0.0028$, because 1693.3 has the same rank as 169.3: it is the largest value either way.
Fay and Proschan (2010, §5.3.5) show the same effect in a simulation with a single outlier of 1000 in samples of 100. The $t$-test's $p$-values rose from below $5 \times 10^{-6}$ to between 0.26 and 0.29, while the rank-sum test still rejected. Even an error in the direction of the true difference can destroy the $t$-test's power. An error cannot move a rank by more than the number of observations it passes.
Choosing between the tests
- When the question is about means (total cost, mean blood pressure reduction), use a test of means: Welch's $t$-test in general. The rank-sum test does not test means unless the shapes are the same.
- When the populations plausibly differ by a shift and may have heavy tails, outliers or skewness, the rank-sum test is the more powerful choice and loses little if the data turn out normal.
- When the data are ordinal (pain scores, stages, ratings), means may not be meaningful at all, and the rank-sum test's $\theta$ is a natural summary.
- When the spreads differ and the group sizes are unequal, neither the pooled $t$-test nor the rank-sum test holds its level. Welch's $t$-test does for means; for $\theta$, a test with a variance estimate that does not assume equal distributions is needed.
- Choosing the test after a preliminary test of normality is discouraged, like choosing between the pooled and Welch tests after a preliminary test of variances. Fay and Proschan (2010, §5.3.6) advise that the choice "should not be based on a test of normality", since such tests have little power in small samples and too much in large ones.
What we have learned
Ranking two samples together and adding one group's ranks gives Wilcoxon's $W$. Subtracting its minimum gives Mann and Whitney's $U$, the number of pairs in which the first group's value is the larger. Under the null hypothesis every arrangement of the group labels is equally likely, so the exact distribution comes from counting, and the normal approximation with its tie correction is close by 8 per group. On 8 men and 8 women the men were higher in 72% of pairs, but $p = 0.161$.
A rejection means that $\theta = P(X > Y) + \tfrac12 P(X = Y)$ differs from one half. It means the medians differ only under a shift model. The test does not repair unequal spreads, and with unequal group sizes it fails in the same direction as the pooled $t$-test. In return it loses about 5% of the sample size under normality, gains a great deal under heavy tails and skewness, and is unmoved by a single gross error.
What about more than two groups? The Kruskal–Wallis test ranks all of them together and runs an analysis of variance on the ranks, and for two groups it reduces to this test.
In Python
from scipy import stats
men = [140.7, 109.3, 124.7, 109.3, 143.3, 170.7, 124.0, 124.7] # 8 men (mm Hg)
women = [115.3, 137.3, 140.0, 107.3, 119.3, 121.3, 112.7, 108.0] # 8 women
res = stats.mannwhitneyu(men, women) # ties present: normal approximation
print(res.statistic, res.pvalue) # U = 46, p = 0.156
perm = stats.mannwhitneyu(men, women, method=stats.PermutationMethod(n_resamples=20000, random_state=1))
print(perm.pvalue) # 0.1515, all 12,870 splits enumerated
print(res.statistic / (8 * 8)) # estimate of theta: 0.719
mannwhitneyu reports $U$ for the first sample; its documentation notes that the $p$-value is identical to the Wilcoxon rank-sum test's and that $W = U + n_1(n_1+1)/2$. Its exact method assumes no ties, so with tied data it uses the normal approximation with the tie correction, and a permutation method gives the exact conditional $p$-value. The outputs shown were checked with scipy 1.17.1, and every number in this article is computed by tools/experiments/applied-statistics/wilcoxon-rank-sum-test.py in the site's repository, which implements Mann and Whitney's recurrence and the exact conditional distribution directly.
References
- Fay, M. P., and Proschan, M. A. (2010). Wilcoxon–Mann–Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics Surveys, 4, 1–39. record · 3 pages
- Kruskal, W. H., and Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621. record · 4 pages
- Mann, H. B., and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50–60. record · 2 pages
- National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: Demographic Variables and Sample Weights (DEMO_J) and Blood Pressure (BPX_J) data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 45 pages
- SciPy.
scipy.stats.mannwhitneyu. SciPy reference documentation. https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.mannwhitneyu.html record - Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83. record · 3 pages