One-Sided Confidence Bounds and Sample Size

Confidence bounds that limit a mean from one side only, derived from the same pivot as two-sided intervals and worked on Student's sleep data; their link to one-sided tests, why the side must be chosen before the data are seen (choosing afterwards doubles the miss rate), and how to choose a sample size for a target margin of error, with the t iteration and the chance that the finished interval is as narrow as planned.

Some questions about a mean have only one interesting direction. A trial of a sleeping drug wants to know how much sleep the drug adds at the least; that it might add a great deal is welcome, but not the point. A safety study of a contaminant wants to know how high the average exposure could be; a low value needs no warning.

A two-sided interval answers both directions at once, and so it spends half its error allowance on the side nobody asked about. Can we spend all of it on the side that matters? A one-sided confidence bound does exactly that.

This article answers two practical questions that come up once intervals are in use. The first is how to state a one-sided result. We derive one-sided bounds from the same pivots as the two-sided intervals of Confidence Intervals and Coverage and Confidence Intervals for a Mean, work them on Student's sleep data, and show by simulation why the side must be chosen in advance. The second question turns the interval around. Instead of asking how wide an interval a given sample produces, we ask how large a sample a study needs for an interval of a given width.

Deriving a one-sided bound

The idea fits in a sentence. A two-sided 95% interval leaves 2.5% in each tail; a one-sided 95% bound leaves all 5% in one tail, so its critical value is smaller and its single endpoint sits closer to the sample mean.

To make it exact, start from the pivot for a mean when the standard deviation $\sigma$ is known,

$$Z = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}},$$

which has a standard normal distribution (exactly for a normal population, approximately for large samples from others). Here $\bar{X}$ is the mean of a sample of $n$ observations and $\mu$ is the population mean. A two-sided interval uses the event $-z_{1-\alpha/2} \le Z \le z_{1-\alpha/2}$, which leaves $\alpha/2$ in each tail. A one-sided bound uses an event that leaves all of $\alpha$ in one tail:

$$P\left(\frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \le z_{1-\alpha}\right) = 1 - \alpha,$$

where $z_{1-\alpha}$ is the point with probability $1 - \alpha$ below it under the standard normal curve. For 95%, $z_{0.95} = 1.645$. Multiplying by $\sigma/\sqrt{n}$ and rearranging,

$$P\left(\mu \ge \bar{X} - z_{1-\alpha}\frac{\sigma}{\sqrt{n}}\right) = 1 - \alpha.$$

So the random quantity $\bar{X} - z_{1-\alpha}\,\sigma/\sqrt{n}$ falls below $\mu$ with probability $1 - \alpha$. Once the sample is in, it gives the lower confidence bound. The event in the other tail gives the upper confidence bound:

$$\mu \ge \bar{x} - z_{1-\alpha}\frac{\sigma}{\sqrt{n}}, \qquad \mu \le \bar{x} + z_{1-\alpha}\frac{\sigma}{\sqrt{n}}.$$

When $\sigma$ is estimated by the sample standard deviation $s$, the pivot $(\bar{X} - \mu)/(S/\sqrt{n})$ has a $t$ distribution with $n - 1$ degrees of freedom for a normal population. The same steps give

$$\mu \ge \bar{x} - t_{n-1,\,1-\alpha}\frac{s}{\sqrt{n}}, \qquad \mu \le \bar{x} + t_{n-1,\,1-\alpha}\frac{s}{\sqrt{n}}.$$

These are the one-sided intervals of the NIST/SEMATECH e-Handbook of Statistical Methods (§7.2.2.1). The handbook writes the lower bound as $\bar{Y} + t_{\alpha,\,N-1}\,s/\sqrt{N}$, with the negative quantile $t_{\alpha,\,N-1} = -t_{1-\alpha,\,N-1}$. Written as intervals they are $[\bar{x} - t_{n-1,\,1-\alpha}\,s/\sqrt{n},\ \infty)$ and $(-\infty,\ \bar{x} + t_{n-1,\,1-\alpha}\,s/\sqrt{n}]$: one end is a number, the other is unbounded.

The only change from the two-sided case is the critical value: $z_{1-\alpha}$ instead of $z_{1-\alpha/2}$, that is 1.645 instead of 1.960 at 95%. Two consequences are worth remembering:

  • a 95% one-sided bound is the same number as the corresponding end of a 90% two-sided interval, since both use $z_{0.95}$ or $t_{n-1,\,0.95}$;
  • conversely, a 95% two-sided interval is the pair of 97.5% one-sided bounds.

Student's sleep data, one side at a time

Back to the ten patients of Confidence Intervals for a Mean. Each took two sleeping drugs (Student, 1908). The differences in hours of sleep gained, drug 2 minus drug 1, have mean $\bar{d} = 1.580$, standard deviation $s_d = 1.2300$ and standard error $s_d/\sqrt{10} = 0.3890$ hours. The natural one-sided question is: how much more sleep does drug 2 give, at the least?

With 9 degrees of freedom, $t_{9,\,0.95} = 1.8331$, and the 95% lower bound is

$$\mu \ge 1.580 - 1.8331 \times 0.3890 = 1.580 - 0.713 = 0.867 \text{ hours}.$$

In words: the data are consistent with drug 2 adding at least about 0.87 hours over drug 1, with 95% confidence in the procedure. Here is how it compares with the other statements we could make:

Statement Critical value Result (hours)
95% two-sided interval $t_{9,\,0.975} = 2.2622$ $[0.700,\ 2.460]$
95% lower bound $t_{9,\,0.95} = 1.8331$ $\mu \ge 0.867$
95% upper bound $t_{9,\,0.95} = 1.8331$ $\mu \le 2.293$
99% lower bound $t_{9,\,0.99} = 2.8214$ $\mu \ge 0.483$
Three 95% statements about the mean extra sleep from drug 2 over drug 1, from the same ten patients. The two-sided interval runs from 0.70 to 2.46 hours. Notice that the lower bound, 0.87, sits higher than the interval's lower end: it spends all of its 5% error allowance on one side, and in exchange says nothing about how large the effect might be. The upper bound mirrors it. The dotted line is the sample mean, 1.58.
Three 95% statements about the mean extra sleep from drug 2 over drug 1, from the same ten patients. The two-sided interval runs from 0.70 to 2.46 hours. Notice that the lower bound, 0.87, sits higher than the interval's lower end: it spends all of its 5% error allowance on one side, and in exchange says nothing about how large the effect might be. The upper bound mirrors it. The dotted line is the sample mean, 1.58.

The lower bound is tighter on its side than the two-sided interval, 0.867 against 0.700. The price is silence about the other side. That trade is worth making only when the other side genuinely does not matter to the decision.

The bound and the one-sided test

A lower bound answers the same question as a one-sided test, and the two always agree. Suppose the question is whether $\mu$ exceeds some value $\mu_0$, with null hypothesis $H_0: \mu \le \mu_0$ and alternative $H_A: \mu > \mu_0$. The test of Hypothesis Tests for a Mean rejects $H_0$ at level $\alpha$ when $t = (\bar{x} - \mu_0)/(s/\sqrt{n}) > t_{n-1,\,1-\alpha}$. Multiplying through by $s/\sqrt{n}$ and rearranging gives the same inequality as

$$\mu_0 < \bar{x} - t_{n-1,\,1-\alpha}\frac{s}{\sqrt{n}}.$$

So the test rejects exactly when $\mu_0$ lies below the $1 - \alpha$ lower bound. That article shows the same agreement for two-sided tests and intervals.

For the sleep data with $\mu_0 = 0$, the lower bound stays above zero up to a confidence level of 0.99858. That makes the one-sided $p$-value $1 - 0.99858 = 0.00142$. The confidence level 0.99858 agrees with the 0.9985 that Student reported in 1908 as the probability that drug 2 is the better soporific.

Coverage, and a hazard of one-sided bounds

Does a one-sided bound keep its promise? The script one-sided-confidence-bounds-and-sample-size.py checks by simulation, drawing samples of 10 (10,000 samples for each of 30 seeds; medians reported). For a normal population, the 95% lower bound was below the true mean in 0.9496 of samples, and the upper bound above it in 0.9498. That is what the derivation promises.

The NHANES blood pressure population of Confidence Intervals and Coverage tells a different story. It is skewed to the right, and its two sides behave differently: the lower bound covered in 0.9748 of samples, the upper bound in only 0.9114.

A two-sided interval partly hides this, because its misses on the two sides add up. A one-sided bound has only one side and inherits that side's error in full. Confidence Intervals for a Mean explains the cause: for a right-skewed population, samples that miss the long tail have both a low mean and a small standard deviation, so intervals err downward much more often than upward. For skewed data, an upper bound computed from a small sample deserves extra caution.

The side must be chosen in advance

The guarantee of a one-sided bound depends on the side being fixed before the data are seen. It is tempting to look first. If $\bar{x}$ came out above some reference value $\mu_0$, report a lower bound to show how far above; if below, report an upper bound.

Check your understanding. An analyst follows that rule and always reports a "95%" bound. Suppose the true mean equals the reference value $\mu_0$. How often does the reported bound miss $\mu$?

The answer is about 10%, twice the advertised rate. A lower bound misses when $\bar{x}$ lands far above $\mu$; an upper bound misses when it lands far below. Choosing after looking means using the lower bound exactly in the samples where $\bar{x}$ is high, which are the samples in which a lower bound is liable to miss. The same holds for the upper bound, so both 5% tails are exposed.

The simulation confirms it. For a normal population with samples of 10, the script chose the side according to whether $\bar{x}$ exceeded $\mu_0$, and recorded how often the reported 95% bound missed the true mean $\mu$.

True mean minus $\mu_0$, in standard errors $\sigma/\sqrt{n}$ 0 0.5 1 2 3
Miss rate of the "95%" bound 0.1006 0.1006 0.0996 0.0692 0.0518

When the true mean is at or near the reference value, the miss rate doubles from 5% to 10%. Only when the true mean is far from $\mu_0$, so that the side is almost always the same, does the rate fall back toward 5%. A "95%" bound chosen this way is really a 90% procedure. That is why the direction of a one-sided analysis belongs in the study protocol, written before the data arrive.

How large a sample?

So far the sample was given and we asked how wide an interval it produces. Planning a study runs the other way. It usually starts from the precision wanted: a margin of error $E$, the half-width of the interval. How many observations does that take?

The answer comes from solving the margin-of-error formula for $n$. With $\sigma$ known, the two-sided interval's half-width is $z_{1-\alpha/2}\,\sigma/\sqrt{n}$. Setting it equal to $E$,

$$z_{1-\alpha/2}\frac{\sigma}{\sqrt{n}} = E \quad\Longrightarrow\quad n = \left(\frac{z_{1-\alpha/2}\,\sigma}{E}\right)^2,$$

rounded up to the next whole number, since rounding down would give a margin slightly larger than $E$. The NIST handbook (§7.2.2.2) gives the 95% case, $N \ge (1.96/\delta)^2\sigma^2$, in its own symbols. For a one-sided bound no further than $E$ from $\bar{x}$, the critical value is $z_{1-\alpha}$:

$$n = \left(\frac{z_{1-\alpha}\,\sigma}{E}\right)^2.$$

Two features of the formula govern every study budget.

  • The sample size grows with the square of $\sigma/E$. Halving the margin of error quadruples the sample.
  • It does not depend on the size of the population. A survey of a city and a survey of a country need the same $n$ for the same precision, provided the sample is a small fraction of the population.

The second can seem surprising. It follows from the sampling distribution of the mean: the standard error $\sigma/\sqrt{n}$ involves the sample size, not the population size, once the sample is a small share of the whole.

Planning a blood pressure survey

Suppose a health survey wants to estimate mean adult systolic blood pressure to within $E = 2$ mm Hg with 95% confidence. It takes $\sigma = 20.12$ mm Hg from an earlier survey (the NHANES value). Then

$$n = \left(\frac{1.960 \times 20.12}{2}\right)^2 = 388.77, \quad\text{so } n = 389.$$

A one-sided 95% bound within 2 mm Hg of the estimate needs $(1.645 \times 20.12/2)^2 = 273.8$, so 274.

Sample size needed to estimate mean systolic blood pressure (sigma 20.12 mm Hg) to within E mm Hg, for a two-sided 95% interval (solid) and a one-sided 95% bound (dashed), on a logarithmic axis. Follow the solid line leftward: each halving of E multiplies n by four, from 98 at E = 4 to 389 at E = 2 and 1,556 at E = 1.
Sample size needed to estimate mean systolic blood pressure (sigma 20.12 mm Hg) to within E mm Hg, for a two-sided 95% interval (solid) and a one-sided 95% bound (dashed), on a logarithmic axis. Follow the solid line leftward: each halving of E multiplies n by four, from 98 at E = 4 to 389 at E = 2 and 1,556 at E = 1.
Half-width $E$ (mm Hg) 1 2 3 4 5 8 10
$n$, two-sided 95% 1,556 389 173 98 63 25 16
$n$, one-sided 95% 1,096 274 122 69 44 18 11

When sigma will be estimated: the t iteration

The formula used $z$, but the finished study will estimate $\sigma$ by $s$ and use the $t$ interval. Its critical value $t_{n-1,\,1-\alpha/2}$ is larger than $z_{1-\alpha/2}$, and it depends on the very $n$ being chosen. The usual remedy is to iterate:

  1. compute $n$ with $z$;
  2. look up $t$ for that $n$ and recompute;
  3. repeat until $n$ stops changing.

The NIST handbook (§7.2.2.2) uses the same iteration for the sample size of a test.

As an example, suppose a new trial of drug 2 against drug 1 wants a 95% interval for the mean extra sleep with half-width $E = 0.5$ hours. It plans with $\sigma = 1.23$ hours, the standard deviation of Student's ten differences.

Step $n$ used for $t$ $t_{n-1,\,0.975}$ $(t\,\sigma/E)^2$ Rounded up
start ($z = 1.960$) 23.25 24
1 24 2.0687 25.90 26
2 26 2.0595 25.67 26

The iteration settles at 26 patients, two more than the $z$ formula gave. For the blood pressure survey, the same iteration moves from 389 to 392; with hundreds of observations, $t$ and $z$ barely differ.

Will the interval actually be that narrow?

Suppose the sleep trial enrolls its 26 patients. Will its interval come out at $\pm 0.5$ hours?

A little over half the time. The formula makes the half-width equal to $E$ when the sample standard deviation $s$ happens to equal the planning value $\sigma$. But $s$ is random. It comes out above $\sigma$ in roughly half of all samples, and then the finished interval is wider than planned.

The probability that the realized half-width is at most $E$ is sometimes called the assurance of the design. For a normal population it can be computed. The half-width $t_{n-1,\,1-\alpha/2}\,s/\sqrt{n}$ is at most $E$ exactly when

$$\frac{(n-1)s^2}{\sigma^2} \le \frac{(n-1)\,n\,E^2}{t_{n-1,\,1-\alpha/2}^2\,\sigma^2}.$$

The left side has a chi-squared distribution with $n - 1$ degrees of freedom (fact 2 of Confidence Intervals for a Mean), so the probability is a chi-squared tail area.

For the sleep trial with $n = 26$, the assurance is 0.556, and the script's simulation of normal samples agrees at 0.556. Nearly half of such trials would end with an interval wider than the planned $\pm 0.5$ hours. Raising $n$ until the assurance reaches 90% requires 34 patients, 31% more. For the blood pressure survey, $n = 392$ gives an assurance of 0.521, and 90% assurance needs 426 adults.

Real populations add a further effect. The chi-squared calculation assumes normality, and the sample standard deviation varies more in populations with heavy tails. The NHANES blood pressure population has a kurtosis of 4.68, against 3 for a normal distribution. When the script drew samples of 426 from it, only 0.834 of the intervals came out within $\pm 2$ mm Hg, not 0.90. A planning value of $\sigma$ that is itself an estimate from a small pilot study adds yet more uncertainty. The sample size formula is the right starting point, and a careful plan adds a margin on top of it.

Summary

Goal Formula
Two-sided interval, $\sigma$ known $\bar{x} \pm z_{1-\alpha/2}\,\sigma/\sqrt{n}$
Two-sided interval, $\sigma$ estimated $\bar{x} \pm t_{n-1,\,1-\alpha/2}\,s/\sqrt{n}$
Lower bound $\mu \ge \bar{x} - z_{1-\alpha}\,\sigma/\sqrt{n}$ or $\bar{x} - t_{n-1,\,1-\alpha}\,s/\sqrt{n}$
Upper bound $\mu \le \bar{x} + z_{1-\alpha}\,\sigma/\sqrt{n}$ or $\bar{x} + t_{n-1,\,1-\alpha}\,s/\sqrt{n}$
Sample size, two-sided, half-width $E$ $n = (z_{1-\alpha/2}\,\sigma/E)^2$, rounded up; iterate with $t$
Sample size, one-sided, distance $E$ $n = (z_{1-\alpha}\,\sigma/E)^2$, rounded up

A one-sided bound moves all of the error allowance to the side that matters, which makes it tighter there and silent on the other side. It keeps its promise only if the side is fixed in advance, and for skewed data its one side can be noticeably less reliable than a two-sided interval suggests. Choosing a sample size is the margin-of-error formula solved for $n$. The answer is a starting point that the variability of $s$ and the shape of the population will push upward.

This closes the topic on confidence intervals. The next topic, hypothesis testing, uses the same pivots to ask a different question: are the data consistent with a stated value of the mean? It starts with Hypothesis Tests for a Mean. Sample sizes chosen to give a test enough power to detect a given difference, rather than an interval a given width, follow the same pattern with an extra term and are in Power and Sample Size for a Test of a Mean.

In Python

import math
from statistics import NormalDist
from scipy import stats

dbar, sd, n = 1.580, 1.2300, 10
print(dbar - stats.t.ppf(0.95, n - 1) * sd / math.sqrt(n))   # lower bound, about 0.867

sigma, E = 1.23, 0.5
m = math.ceil((NormalDist().inv_cdf(0.975) * sigma / E) ** 2)   # 24
while True:
    nxt = math.ceil((stats.t.ppf(0.975, m - 1) * sigma / E) ** 2)
    if nxt == m:
        break
    m = nxt
print(m)                                                        # 26
t = stats.t.ppf(0.975, m - 1)
print(stats.chi2.cdf((m - 1) * m * E**2 / (t * sigma) ** 2, m - 1))  # assurance, about 0.556

Every number in this article is computed by tools/experiments/applied-statistics/one-sided-confidence-bounds-and-sample-size.py in the site's repository.

References