Sampling Distributions

The bridge from probability to inference, the course unit on the sampling distribution of the mean. A sample mean is a random quantity with a distribution of its own, centered on the population mean, narrower than the population by a factor of root n (with a finite population correction when sampling without replacement), and close to normal for large samples whatever the shape of the population, by the central limit theorem; how large is large enough, the Berry–Esseen bound, and where the theorem fails. Worked on NHANES blood pressure and C-reactive protein.

A health department measures the blood pressure of 25 adults drawn at random and reports their mean. A different 25 would have given a different mean. So how much does a sample mean vary from sample to sample, and what shape does that variation take?

The answer needs a change of viewpoint. A statistic computed from a random sample, such as the sample mean, is itself a random quantity. Its distribution over all possible samples is its sampling distribution. Almost every method in the rest of the course is a statement about one.

The units before this one describe data and the distribution of a single observation. The units after it estimate a population mean with a stated margin of error and test claims about it. This unit connects them, with three facts about the sample mean:

  1. Its sampling distribution is centered on the population mean, so the sample mean is unbiased.
  2. Its standard deviation, the standard error, is the population standard deviation divided by $\sqrt{n}$. Precision improves only with the square root of the sample size, and a correction applies when a large share of a finite population is sampled without replacement.
  3. Its shape approaches the normal curve as $n$ grows, whatever the shape of the population. This is the central limit theorem.

The first two facts follow from the rules of expectation and variance. The third is a limit theorem, so the practical questions are how fast the limit is reached and when it is not reached at all.

Two examples run through the unit. The 5,247 adults with blood pressure measurements in the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018 serve as a population whose parameters are known, so the theory can be checked by drawing samples from it. The same survey's measurements of C-reactive protein, a marker of inflammation with a skewness of 7.9, show how slowly the central limit theorem can work in practice. Every number comes from a script in the site's repository.

Articles

  • The Sampling Distribution of the Mean: where are sample means centered, and how widely do they spread? The population, the sample and the sampling distribution told apart. Every sample of a five-person population listed. Proofs that $E[\bar{X}] = \mu$ and that the standard error is $\sigma/\sqrt{n}$ for independent draws. The finite population correction $\sqrt{(N - n)/(N - 1)}$, derived from the covariance of draws without replacement and checked by simulation up to a sample of the whole population. The standard error versus the standard deviation, and the estimated standard error as a statistic with its own spread. How the skewness of the mean falls with $n$, and the sampling distribution of the median for comparison. Later units build on this article for the sampling distribution and the standard error.
  • The Central Limit Theorem: why are sample means nearly normal, and how large is large enough? The Lindeberg–Lévy theorem and what it does and does not claim. Exact distributions of exponential means. A proof that the skewness of a mean is $\gamma/\sqrt{n}$, and blood pressure against C-reactive protein, for which means of 30 are still strongly skewed. The Berry–Esseen bound. Failures under infinite variance (Cauchy and Pareto) and dependence. A proof sketch by moment generating functions, and the theorem's history.

This unit builds on earlier ones and leads into the rest of the course.

References