Moment Generating Functions and Slutsky's Theorem

The proofs behind the normal limit of a sample mean. The weak law of large numbers, proved from Chebyshev's inequality; the limit theorem for the standardized mean, proved with moment generating functions for every population that has one, with an exact list of what is assumed, proved and cited; and Slutsky's theorem, with the case proved that lets the sample standard deviation s replace sigma. Checked on the exponential distribution and on NHANES blood pressures.

Draw 400 adults at random, with replacement, from the 5,247 adults with a blood pressure reading in the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018, and standardize their mean with the sample standard deviation $s$. The result falls outside $\pm 1.96$ in 5.07% of samples, close to the 5% of the normal curve, although the blood pressures are skewed and $s$ is only an estimate. Why does the normal curve appear, and why does it survive the replacement of $\sigma$ by $s$? Because of two theorems, both proved here: the limit theorem for a standardized mean, proved for every population that has a moment generating function, and the case of Slutsky's theorem that covers a factor settling at 1.

The Central Limit Theorem states the first of them, measures how fast it works and shows where it fails. For independent observations $X_1, X_2, \ldots$ with a common distribution of mean $\mu$ and finite variance $\sigma^2 > 0$, and $\bar{X}_n$ their mean, the standardized mean

$$Z_n = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$$

satisfies $P(Z_n \le z) \to \Phi(z)$ for every $z$, where $\Phi$ is the standard normal distribution function. That is the Lindeberg–Lévy theorem, and its kind of convergence is called convergence in distribution. This article holds the proofs that article refers to.

First we prove the weak law of large numbers: the sample mean settles at $\mu$. Then we prove the limit theorem under one extra assumption, with moment generating functions. Finally we state Slutsky's theorem and prove the case that replaces $\sigma$ by $s$. The article assumes The Sampling Distribution of the Mean, The Normal Distribution and the expectation rules of Random Variables, Expectation and Variance.

The mean settles at mu

The law of large numbers

The standard deviation of a sample mean, $\sigma/\sqrt{n}$, goes to zero as the sample grows, so the mean settles at $\mu$. Here is the precise statement. A sequence of random variables $W_n$ converges in probability to a constant $c$ if, for every $\varepsilon > 0$, $P(\lvert W_n - c\rvert \ge \varepsilon) \to 0$ as $n \to \infty$.

Theorem (weak law of large numbers). Let $X_1, X_2, \ldots$ be independent random variables with a common mean $\mu$ and a common finite variance $\sigma^2$. Then $\bar{X}_n$ converges in probability to $\mu$: for every $\varepsilon > 0$,

$$P\big(\lvert \bar{X}_n - \mu\rvert \ge \varepsilon\big) \le \frac{\sigma^2}{n\varepsilon^2} \to 0.$$

The idea: the variance of the mean, $\sigma^2/n$, goes to zero, and a variable with a small variance is rarely far from its mean.

Proof. The second half of the idea is Chebyshev's inequality: a random variable $W$ with mean $m$ and variance $v$ satisfies $P(\lvert W - m\rvert \ge \varepsilon) \le v/\varepsilon^2$. (Standard Scores, Chebyshev's Inequality and Grouped Data proves it for a list of values; the proof for a random variable is the same idea.) Let $I$ be the indicator of the event $\lvert W - m\rvert \ge \varepsilon$, equal to 1 when the event occurs and 0 otherwise. Then $(W - m)^2 \ge \varepsilon^2 I$ always: when the event occurs the left side is at least $\varepsilon^2$, and when it does not the right side is 0. The difference $(W - m)^2 - \varepsilon^2 I$ is a variable that is never negative, so its expected value is not negative, and by linearity

$$v = E\big[(W - m)^2\big] \ \ge\ \varepsilon^2 E[I] = \varepsilon^2\,P(\lvert W - m\rvert \ge \varepsilon).$$

Now take $W = \bar{X}_n$, which has mean $\mu$ and variance $\sigma^2/n$ by The Sampling Distribution of the Mean. $\square$

For the blood pressures, with $\sigma^2 = 405.01$, the bound says that a mean of 400 adults misses $\mu$ by 2 mm Hg or more with probability at most $405.01/(400 \times 4) = 0.253$. The script's samples did so 4.6% of the time (median of 30 seeds, 10,000 samples each), and the normal approximation gives 4.7%. The inequality is crude, since it must hold for every distribution, but it is enough to prove the law.

The law holds under weaker conditions than the proof uses. The mean of independent observations with a common distribution converges to $\mu$ whenever the mean $\mu$ exists, with no assumption on the variance, and it does so in a stronger sense, the strong law of large numbers. Those results are cited, not proved here (Billingsley, 1995, in its treatment of the laws of large numbers).

Why it is true: a proof for populations with a moment generating function

The proof of the Lindeberg–Lévy theorem in full generality uses characteristic functions and a continuity theorem from analysis (Billingsley, 1995, in its section on the central limit theorem). The argument below follows the approach that Grinstead and Snell (1997, §10.3) sketch for bounded variables, using moment generating functions instead; the remainder estimate in Step 3 is given here in full, for any population whose moment generating function is finite near 0. It is a complete proof under one extra assumption, with one theorem cited. To be exact about what is done:

  • Assumed, beyond the hypotheses of the theorem: the population has a moment generating function that is finite near 0, as defined in the next paragraph.
  • Proved: the full conclusion of the theorem, $P(Z_n \le z) \to \Phi(z)$ for every $z$, for every such population.
  • Cited, not proved: the continuity theorem for moment generating functions (Curtiss, 1942), stated below. Every other step is derived here.

Here is the plan in one sentence. The moment generating function of the standardized mean is the $n$th power of one observation's, evaluated at $t/\sqrt{n}$, and expanding it near zero shows that the power tends to $e^{t^2/2}$, the normal's.

The extra assumption. The moment generating function of a random variable $X$ is $M_X(t) = E[e^{tX}]$. Assume it is finite for all $t$ in some interval $(-h, h)$ around 0. This is stronger than a finite variance. It implies that all moments of $X$ are finite, and it fails for some distributions whose variance is finite, such as the lognormal. It holds for the exponential (for $t < 1$), the normal, and every bounded variable, including a draw from any finite population. (The theorem's hypothesis of independent draws then means sampling with replacement.)

The cited theorem. The continuity theorem (Curtiss, 1942): if $M_{Z_n}(t) \to M_Z(t)$ for every $t$ in an interval around 0, and $M_Z$ is finite there, then $P(Z_n \le z) \to P(Z \le z)$ at every $z$ where the distribution function of $Z$ is continuous. For a normal limit that is every $z$. The theorem contains the fact that a moment generating function that is finite near 0 determines the distribution, since otherwise the limit could not be identified from $M_Z$. The rest is elementary.

Step 1: the target. For a standard normal $Z$, completing the square in the exponent,

$$M_Z(t) = \int_{-\infty}^{\infty} e^{tz}\,\frac{e^{-z^2/2}}{\sqrt{2\pi}}\,dz = e^{t^2/2}\int_{-\infty}^{\infty} \frac{e^{-(z - t)^2/2}}{\sqrt{2\pi}}\,dz = e^{t^2/2},$$

because the last integrand is the normal density with mean $t$, which integrates to 1 (The Normal Distribution proves it).

Step 2: independence turns a sum into a power. Standardize each observation, $Y_i = (X_i - \mu)/\sigma$, so that $E[Y_i] = 0$ and $E[Y_i^2] = 1$, and write the standardized mean as

$$Z_n = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} = \frac{1}{\sqrt{n}}\sum_{i=1}^n Y_i.$$

The standardized observation has a moment generating function too: $M_Y(t) = E[e^{t(X - \mu)/\sigma}] = e^{-\mu t/\sigma}M_X(t/\sigma)$, which is finite for $\lvert t\rvert < h\sigma$. For independent variables the expected value of a product of functions of them factors, so

$$M_{Z_n}(t) = E\Big[\prod_{i=1}^n e^{tY_i/\sqrt{n}}\Big] = \prod_{i=1}^n E\big[e^{tY_i/\sqrt{n}}\big] = \Big[M_Y\Big(\frac{t}{\sqrt{n}}\Big)\Big]^n.$$

(For discrete variables the product rule is the computation of $E[XY]$ in Random Variables, Expectation and Variance with $n$ factors in place of two: the joint probabilities of $n$ independent variables are products, so the $n$-fold sum of $\prod g_i(x_i)\prod P(X_i = x_i)$ splits into the product of $n$ single sums. For continuous variables it is the rule cited in The Normal Distribution.) The right side is finite whenever $\lvert t\rvert/\sqrt{n} < h\sigma$, so for every $n$ on the interval $\lvert t\rvert < h\sigma$.

Step 3: expand near zero. The claim is that

$$M_Y(s) = 1 + \frac{s^2}{2} + r(s), \qquad \frac{r(s)}{s^2} \to 0 \text{ as } s \to 0.$$

Taylor's theorem for the exponential function, with the remainder in Lagrange's form, gives for every real $x$

$$e^x = 1 + x + \frac{x^2}{2} + R(x), \qquad \lvert R(x)\rvert \le \frac{\lvert x\rvert^3 e^{\lvert x\rvert}}{6}.$$

Put $x = sY$ and take expected values. Since $E[Y] = 0$ and $E[Y^2] = 1$, linearity gives $M_Y(s) = 1 + s^2/2 + r(s)$ with $r(s) = E[R(sY)]$, and

$$\lvert r(s)\rvert \le \frac{\lvert s\rvert^3}{6}\,E\big[\lvert Y\rvert^3 e^{\lvert sY\rvert}\big].$$

It remains to bound the last expected value by a constant. Fix $c$ with $0 < c < h\sigma$. The function $y^3e^{-cy/2}$ is bounded on $y \ge 0$, say by $K$, so $\lvert Y\rvert^3 \le Ke^{c\lvert Y\rvert/2}$. For $\lvert s\rvert \le c/2$,

$$\lvert Y\rvert^3 e^{\lvert sY\rvert} \le Ke^{c\lvert Y\rvert/2}\,e^{c\lvert Y\rvert/2} = Ke^{c\lvert Y\rvert} \le K\big(e^{cY} + e^{-cY}\big),$$

and the expected value of the right side is $K\big(M_Y(c) + M_Y(-c)\big)$, finite by the assumption. Call it $B$. Then $\lvert r(s)\rvert \le B\lvert s\rvert^3/6$ for $\lvert s\rvert \le c/2$, so $\lvert r(s)\rvert/s^2 \le B\lvert s\rvert/6 \to 0$. This bound is where the extra assumption does its work.

Step 4: take the limit. Fix $t$ and let $a_n = t^2/(2n) + r(t/\sqrt{n})$, so that $M_{Z_n}(t) = (1 + a_n)^n$. Then $a_n \to 0$, and

$$n\,a_n = \frac{t^2}{2} + t^2\,\frac{r(t/\sqrt{n})}{(t/\sqrt{n})^2} \to \frac{t^2}{2}$$

(for $t = 0$ everything equals 1 trivially). For $t \ne 0$ the limit $t^2/2$ is positive, so $a_n$ is positive, and in particular not zero, for all large $n$. Since $\ln(1 + a)/a \to 1$ as $a \to 0$,

$$\ln M_{Z_n}(t) = n\ln(1 + a_n) = n a_n \cdot \frac{\ln(1 + a_n)}{a_n} \to \frac{t^2}{2},$$

so $M_{Z_n}(t) \to e^{t^2/2} = M_Z(t)$. By the continuity theorem, $P(Z_n \le z) \to \Phi(z)$ for every $z$. $\square$

Where each hypothesis enters. Independence is used once, in Step 2. Without it the generating function of the sum does not factor, and the shared-component example of The Central Limit Theorem shows that the conclusion can fail. The finite variance fixes the $s^2/2$ term in Step 3, the only term that survives the limit. Every feature of the population beyond its mean and variance is in the remainder $r$, which vanishes. That is the precise sense in which the normal limit forgets the shape of the population.

The exponential as a test case. The exponential with mean 1 has $M_X(t) = \int_0^\infty e^{tx}e^{-x}\,dx = 1/(1 - t)$ for $t < 1$, so $Y = X - 1$ has $M_Y(t) = e^{-t}/(1 - t)$. At $t = 0.5$, the values of $[M_Y(0.5/\sqrt{n})]^n$ for $n$ = 1, 10, 100, 1,000 and 10,000 are 1.2131, 1.1502, 1.1381, 1.1347 and 1.1336, approaching $e^{0.125} = 1.1331$.

Dropping the extra assumption. This paragraph describes the general proof and does not give it; the proof is in Billingsley (1995, in its section on the central limit theorem, as the Lindeberg–Lévy theorem). Replace $e^{tY}$ by $e^{itY}$, with $i = \sqrt{-1}$. The resulting characteristic function exists for every distribution, because $\lvert e^{itY}\rvert = 1$. The general proof follows the same four steps, with Lévy's continuity theorem in place of Curtiss's and a Taylor expansion in Step 3 that needs only a finite second moment. The Cauchy distribution shows what happens without one. Its characteristic function is $e^{-\lvert t\rvert}$, which has no quadratic term at 0, and the characteristic function of the mean of $n$ Cauchy variables is $(e^{-\lvert t\rvert/n})^n = e^{-\lvert t\rvert}$ again (Grinstead and Snell, 1997, §10.3).

Replacing sigma by s: Slutsky's theorem

The theorem standardizes the mean with the true $\sigma$. In practice $\sigma$ is unknown, and the statistic actually computed is

$$T_n = \frac{\bar{X}_n - \mu}{s_n/\sqrt{n}} = Z_n \cdot \frac{\sigma}{s_n},$$

where $s_n$ is the sample standard deviation and $Z_n$ the standardized mean of the theorem. For small samples its distribution is not normal: for a normal population it is Student's $t$ (The Chi-Squared and t Distributions). For large samples it has the same normal limit as $Z_n$. The reason is that the factor $\sigma/s_n$ settles at 1, and a factor that settles at 1 does not change a limit in distribution. The second half of that sentence is a theorem.

Theorem (Slutsky). Let $Z_n$ be random variables whose distribution functions converge to that of $Z$ at every point where the latter is continuous, and let $A_n$ and $B_n$ be random variables that converge in probability to constants $a$ and $b$. Then the distribution function of $A_nZ_n + B_n$ converges to that of $aZ + b$ at every point where the latter is continuous.

The general theorem is cited, not proved here: van der Vaart (1998, in its chapter on stochastic convergence, as Slutsky's lemma) proves the statements for a product and for a sum that combine to give it. What this article needs is the case $a = 1$, $b = 0$ with a normal limit, and that case is proved here in full, in three parts.

Proposition 1 (a factor that settles at 1). Let $Z_n$ be random variables with $P(Z_n \le z) \to \Phi(z)$ for every $z$, and let $R_n$ be positive random variables that converge in probability to 1. Then $P(R_nZ_n \le z) \to \Phi(z)$ for every $z$.

The idea: when $R_n$ is within $\varepsilon$ of 1, the event $R_nZ_n \le z$ is caught between two events about $Z_n$ alone, with cut-offs close to $z$, and $\Phi$ is continuous.

Proof. Fix $z$ and $\varepsilon$ with $0 < \varepsilon < 1$. Let $a$ be the smaller and $b$ the larger of the two numbers $z/(1 + \varepsilon)$ and $z/(1 - \varepsilon)$. Let $G_n$ be the event $\lvert R_n - 1\rvert \le \varepsilon$. On $G_n$, $R_n$ lies between $1 - \varepsilon$ and $1 + \varepsilon$, so $z/R_n$ lies between $a$ and $b$. Since $R_n > 0$, the event $R_nZ_n \le z$ is the event $Z_n \le z/R_n$. So on $G_n$,

$$Z_n \le a \quad\Longrightarrow\quad R_nZ_n \le z \quad\Longrightarrow\quad Z_n \le b.$$

The first implication says that the event $\{Z_n \le a\}$, apart from outcomes outside $G_n$, is contained in $\{R_nZ_n \le z\}$. The second says that $\{R_nZ_n \le z\}$, apart from outcomes outside $G_n$, is contained in $\{Z_n \le b\}$. Writing $q_n = P(\lvert R_n - 1\rvert > \varepsilon)$ for the probability of the outcomes outside $G_n$,

$$P(Z_n \le a) - q_n \ \le\ P(R_nZ_n \le z) \ \le\ P(Z_n \le b) + q_n.$$

As $n \to \infty$, $q_n \to 0$ because $R_n$ converges in probability to 1, and $P(Z_n \le a) \to \Phi(a)$ and $P(Z_n \le b) \to \Phi(b)$ by hypothesis. Now let $\eta > 0$. Because $\Phi$ is continuous and $a$ and $b$ both tend to $z$ as $\varepsilon \to 0$, we can choose $\varepsilon$ so small that $\Phi(a) > \Phi(z) - \eta$ and $\Phi(b) < \Phi(z) + \eta$. With that $\varepsilon$, the displayed inequality shows that for all large $n$

$$\Phi(z) - 2\eta \ <\ P(R_nZ_n \le z) \ <\ \Phi(z) + 2\eta.$$

Since $\eta$ was arbitrary, $P(R_nZ_n \le z) \to \Phi(z)$. $\square$

Proposition 2 (the sample standard deviation settles at sigma). Let $X_1, X_2, \ldots$ be independent with a common distribution that has mean $\mu$, variance $\sigma^2 > 0$ and a finite fourth moment $E[(X - \mu)^4]$. Then $s_n^2/\sigma^2$ converges in probability to 1, and so does $\sigma/s_n$.

The idea: $s_n^2$ is, up to a factor $n/(n - 1)$ and a small correction, the mean of the independent variables $(X_i - \mu)^2$, whose mean is $\sigma^2$, and the law of large numbers applies to that mean.

Proof. Write $V_n = \frac{1}{n}\sum_{i=1}^n (X_i - \mu)^2$. Expanding $(X_i - \bar{X}_n)^2 = \big((X_i - \mu) - (\bar{X}_n - \mu)\big)^2$ and summing over $i$, the cross term is $-2(\bar{X}_n - \mu)\sum(X_i - \mu) = -2n(\bar{X}_n - \mu)^2$, so

$$(n - 1)s_n^2 = \sum_{i=1}^n (X_i - \bar{X}_n)^2 = nV_n - n(\bar{X}_n - \mu)^2, \qquad s_n^2 = \frac{n}{n - 1}\Big(V_n - (\bar{X}_n - \mu)^2\Big).$$

The variables $(X_i - \mu)^2$ are independent, with mean $\sigma^2$ and variance $E[(X - \mu)^4] - \sigma^4$, finite by assumption. So the weak law of large numbers proved above applies to their mean: $V_n$ converges in probability to $\sigma^2$. It also applies to $\bar{X}_n$ itself. Now fix $\delta$ with $0 < \delta < 1$, and suppose that three things hold:

$$\lvert V_n - \sigma^2\rvert \le \frac{\delta\sigma^2}{3}, \qquad (\bar{X}_n - \mu)^2 \le \frac{\delta\sigma^2}{3}, \qquad \frac{n}{n - 1} \le 1 + \frac{\delta}{3}.$$

Then $V_n - (\bar{X}_n - \mu)^2$ lies between $\sigma^2(1 - 2\delta/3)$ and $\sigma^2(1 + \delta/3)$, and multiplying by $n/(n-1)$, which is between 1 and $1 + \delta/3$,

$$1 - \delta \ \le\ 1 - \frac{2\delta}{3} \ \le\ \frac{s_n^2}{\sigma^2} \ \le\ \Big(1 + \frac{\delta}{3}\Big)^2 = 1 + \frac{2\delta}{3} + \frac{\delta^2}{9} \ \le\ 1 + \delta.$$

The third condition holds for all large $n$. The first fails with a probability that tends to 0, by the law for $V_n$. The second fails exactly when $\lvert\bar{X}_n - \mu\rvert > \sigma\sqrt{\delta/3}$, also with a probability that tends to 0. So $P(\lvert s_n^2/\sigma^2 - 1\rvert > \delta) \to 0$, which is the first claim.

For the second, $\sigma/s_n = (s_n^2/\sigma^2)^{-1/2}$, and the function $u \mapsto u^{-1/2}$ is continuous at $u = 1$. So for every $\varepsilon > 0$ there is a $\delta$ between 0 and 1 such that $\lvert u - 1\rvert \le \delta$ implies $\lvert u^{-1/2} - 1\rvert \le \varepsilon$. Then $P(\lvert \sigma/s_n - 1\rvert > \varepsilon) \le P(\lvert s_n^2/\sigma^2 - 1\rvert > \delta) \to 0$. Here the ratio needs a definition when $s_n = 0$, which has positive probability when the draws come, with replacement, from a finite population: all $n$ draws can be the same person. Define $R_n = \sigma/s_n$ when $s_n > 0$ and $R_n = 1$ when $s_n = 0$. The event $s_n = 0$ is contained in the event $\lvert s_n^2/\sigma^2 - 1\rvert > \delta$ for every $\delta < 1$, so its probability tends to 0, and the inequality above holds for this $R_n$: outside the event $\lvert s_n^2/\sigma^2 - 1\rvert > \delta$, $s_n$ is positive and $\lvert R_n - 1\rvert \le \varepsilon$. $\square$

Corollary (the central limit theorem with s). Under the hypotheses of Proposition 2, and whenever the central limit theorem holds for $Z_n$,

$$\lim_{n \to \infty} P\left(\frac{\bar{X}_n - \mu}{s_n/\sqrt{n}} \le z\right) = \Phi(z) \quad \text{for every } z.$$

Proof. When $s_n > 0$, $T_n = R_nZ_n$ with the $R_n$ of Proposition 2. On the event $s_n = 0$, whose probability tends to 0, $T_n$ is undefined; give it the value $Z_n$ there, as $R_n = 1$ does, so that $T_n = R_nZ_n$ always. The choice does not matter: two events $A$ and $B$ that differ only on outcomes with $s_n = 0$ satisfy $\lvert P(A) - P(B)\rvert \le P(s_n = 0)$, so a change on events of vanishing probability does not alter a limit of probabilities. The central limit theorem gives $P(Z_n \le z) \to \Phi(z)$, Proposition 2 gives $R_n \to 1$ in probability, and Proposition 1 gives the conclusion. $\square$

What is proved and what is cited. The corollary is proved here for every population with a moment generating function finite near 0, since such a population has moments of every order (each power $\lvert y\rvert^k$ is bounded by a multiple of $e^{c\lvert y\rvert}$, as $\lvert y\rvert^3$ was in Step 3), so both the proof of the central limit theorem above and Proposition 2 apply. That covers sampling with replacement from every finite population, the two NHANES populations among them (the script's samples are drawn with replacement). For a population with only a finite variance the corollary is still true, by citation at two points: the central limit theorem itself, and the law of large numbers for $V_n$ without a finite fourth moment (both in Billingsley, 1995, in its treatments of the central limit theorem and of the laws of large numbers). Proposition 1 needs no change.

The blood pressures as a test case. The limit is the same with $s$ as with $\sigma$, but the approach to it differs. The script drew 10,000 samples from the blood pressures for each of 30 seeds and standardized each mean both ways (medians over the seeds):

$n$ With $\sigma$: above $+1.96$ below $-1.96$ With $s$: above $+1.96$ below $-1.96$ Both tails with $s$
30 0.0299 0.0195 0.0161 0.0496 0.0657
400 0.0263 0.0232 0.0211 0.0296 0.0507

At $n = 400$ the two-sided probability with $s$ is 5.07%, close to the limit of 5%. At $n = 30$ it is 6.57%, and the heavier tail has changed sides: with $\sigma$ the upper tail is too heavy, with $s$ the lower. That reversal is an observation from this experiment, not a result proved here. A reason consistent with it is that in a right-skewed population a sample with a low mean tends to have missed the large values, so its $s$ is small as well, which pushes $T_n$ further below zero. The tails with $\sigma$, by contrast, are close to what the Edgeworth term quoted in The Central Limit Theorem predicts, although that expansion was stated for continuous populations and the blood pressures are not one: with $\gamma = 1.038$ and $n = 30$ the term $\gamma(1 - z^2)\varphi(z)/(6\sqrt{n})$ at $z = \pm 1.96$ gives tails of 0.0302 above and 0.0198 below, against 0.0299 and 0.0195 simulated. In samples of 30, $s$ was more than 10% away from $\sigma$ in 57% of samples; in samples of 400, in 3.6%.

Summary

Result Statement
Normal MGF $E[e^{tZ}] = e^{t^2/2}$
Weak law of large numbers $P(\lvert\bar{X}_n - \mu\rvert \ge \varepsilon) \le \sigma^2/(n\varepsilon^2) \to 0$
With $s$ for $\sigma$ (Slutsky) $P\big(\frac{\bar{X} - \mu}{s/\sqrt{n}} \le z\big) \to \Phi(z)$

What we have learned

The sample mean converges in probability to $\mu$, by Chebyshev's inequality. Its standardized form has the normal limit: the proof here is complete for every population with a moment generating function finite near 0, with one cited theorem, and the general case is cited. A factor that converges in probability to 1 does not change a normal limit, so the limit is unchanged when $\sigma$ is replaced by $s$. For means of 400 blood pressures the two-sided tail probability with $s$ is 5.07%; for means of 30 it is 6.57%, and the approach to the limit differs from the one with $\sigma$.

With the sampling distribution of the mean established, the next unit turns it into an interval estimate, beginning with Confidence Intervals and Coverage.

Every number in this article is computed by tools/experiments/applied-statistics/central-limit-theorem.py in the site's repository (its section on the moment generating function and the checks under "Reference pass"), which downloads the NHANES files from the CDC and checks their checksums.

References

  • Billingsley, P. (1995). Probability and Measure (3rd ed.). Wiley. The law of large numbers for independent identically distributed variables with a finite mean; the section on the central limit theorem (the Lindeberg–Lévy theorem, and a central limit theorem for dependent variables). record · 3 pages
  • Curtiss, J. H. (1942). A note on the theory of moment generating functions. Annals of Mathematical Statistics, 13(4), 430–433. record
  • Grinstead, C. M., and Snell, J. L. (1997). Introduction to Probability (2nd rev. ed.). American Mathematical Society. §10.3 (the moment generating function argument and the Cauchy average). record · 13 pages
  • National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: Demographic Variables and Sample Weights (DEMO_J) and Blood Pressure (BPX_J) data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 52 pages
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press. The chapter on stochastic convergence (Slutsky's lemma). record · 4 pages