Ten patients in Student's 1908 sleep trial slept on average 1.58 hours longer on one drug than on another, and the sample standard deviation of their ten differences was 1.23 hours. An interval for the population mean $\mu$ built from those two numbers rests on the ratio
$$T = \frac{\bar{X} - \mu}{S/\sqrt{n}},$$
the distance between the sample mean $\bar{X}$ and $\mu$ in units of the estimated standard error, where $S$ is the sample standard deviation and $n$ the sample size. What is the distribution of that ratio? For a sample from a normal population it is Student's $t$ distribution with $n - 1$ degrees of freedom, whatever the mean and standard deviation of the population are.
This article proves that statement and defines the two distributions it needs. The chi-squared distribution is the distribution of a sum of squares of independent standard normal variables. The $t$ distribution is the distribution of a standard normal variable divided by the square root of an independent chi-squared variable over its degrees of freedom.
First we state three facts about normal samples. Then we prove all three with a single rotation of coordinates. Next we derive the chi-squared density. Then we divide out $\sigma$ to reach the theorem on Student's ratio. Finally we derive the density of $t$ and its variance. The article assumes the normal distribution and the sampling distribution of the mean.
This article is where the course defines the chi-squared and $t$ distributions. Later articles that use them, such as The Two-Sample t-Test and One-Way Analysis of Variance, build on the facts proved here. Confidence Intervals for a Mean shows why the ratio is needed, and uses the theorem to build the $t$ interval and work it on the sleep data.
Three facts about normal samples
Suppose $X_1, \dots, X_n$ are independent draws from a normal population with mean $\mu$ and standard deviation $\sigma$, with $n \ge 2$. Let $\bar{X}$ be their mean and $S^2$ their sample variance (with divisor $n - 1$).
1. The mean is normal. $\bar{X}$ has a normal distribution with mean $\mu$ and standard deviation $\sigma/\sqrt{n}$, so
$$Z = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}}$$
is standard normal. This is the pivot of the known-$\sigma$ interval in Confidence Intervals and Coverage.
2. The scaled variance is chi-squared. We need a name for the distribution of a sum of squared normals. A chi-squared distribution with $\nu$ degrees of freedom, written $\chi^2_\nu$, is the distribution of
$$Z_1^2 + Z_2^2 + \cdots + Z_\nu^2,$$
where $Z_1, \dots, Z_\nu$ are independent standard normal variables. Three properties follow at once from this definition, and later articles rely on them:
- Its mean is $\nu$. Each $Z_j^2$ has expectation $\operatorname{Var}(Z_j) = 1$, and there are $\nu$ of them.
- Its variance is $2\nu$. For a standard normal $Z$ with density $\phi$, integrate by parts with $u = z^3$ and $dv = z\phi(z)\,dz$, for which $v = -\phi(z)$ because $\phi'(z) = -z\phi(z)$. The boundary term $-z^3\phi(z)$ vanishes at $\pm\infty$, so $E(Z^4) = \int z^4\phi(z)\,dz = 3\int z^2\phi(z)\,dz = 3$. Hence $\operatorname{Var}(Z^2) = E(Z^4) - [E(Z^2)]^2 = 3 - 1 = 2$, and the variances of the $\nu$ independent terms add.
- Independent chi-squared variables add. If $U$ is $\chi^2_a$ and $V$ is an independent $\chi^2_b$, then $U + V$ is a sum of $a + b$ independent squared standard normals, so it is $\chi^2_{a+b}$. The degrees of freedom add.
For a normal sample, the scaled sample variance
$$V = \frac{(n-1)S^2}{\sigma^2} = \sum_{i=1}^{n}\left(\frac{X_i - \bar{X}}{\sigma}\right)^2$$
has a $\chi^2_{n-1}$ distribution. It looks like a sum of $n$ squared standardized deviations, but the deviations are not independent. Deviations from the sample mean always sum to zero, so once $n - 1$ of them are known, the last is determined. One degree of freedom is used up by estimating $\mu$ with $\bar{X}$, and $n - 1$ remain. As a check, the mean of $V$ is $n - 1$, which says $E(S^2) = \sigma^2$; the section on the divisor in Confidence Intervals for a Mean proves the same thing another way.
3. The mean and the variance are independent. For a normal population, and only for a normal population, $\bar{X}$ and $S^2$ are statistically independent. Knowing how spread out a sample is tells nothing about where its mean fell. Student derived the distribution of the sample standard deviation of a normal sample in Section I of his paper, and showed in Section II that it is uncorrelated with the sample mean. The independence is proved below. The "only" is cited, not proved here. Geary (1936) and Lukacs (1942) showed that if $\bar{X}$ and $S^2$ of an independent sample of size $n \ge 2$ are independent, the population must be normal, under conditions on the population's moments that the two papers state. (A population concentrated at one value is the trivial exception, or the normal with $\sigma = 0$: its $S^2$ is always 0.)
Proof of the three facts: a rotation
Here is the idea. Standardize the observations, so that the sample becomes a point $Z = (Z_1, \dots, Z_n)$ in $n$-dimensional space whose coordinates are independent standard normal variables. The density of such a point depends only on its distance from the origin, so it is the same after any rotation of the axes. Choose a rotation whose first axis points along $(1, 1, \dots, 1)$. The first new coordinate then carries the sample mean, the other $n - 1$ carry the deviations from it, and the new coordinates are again independent standard normal variables. The three facts can be read off.
Three properties of joint densities and independence are used and cited, not proved here (Casella and Berger, 2002, the chapter "Multiple Random Variables"):
- The product rule. Random variables $Z_1, \dots, Z_n$ with densities are independent exactly when their joint density is the product of their individual densities.
- The linear change of variables. If the random vector $Z$ has joint density $f$ and $Q$ is an invertible $n \times n$ matrix, then $W = QZ$ has joint density $f(Q^{-1}w)/\lvert\det Q\rvert$.
- Functions of independent variables. If $(Y_1, \dots, Y_k)$ and $(Y_{k+1}, \dots, Y_n)$ are independent random vectors, then any function of the first is independent of any function of the second. In particular, functions $g_i(Y_i)$ of mutually independent variables $Y_i$ are mutually independent.
Everything else in the proof is written out.
Step 1: rotating independent standard normal variables gives independent standard normal variables. Let $Z_1, \dots, Z_n$ be independent standard normal variables and let $Q$ be an orthogonal matrix, one with $QQ^{\mathsf{T}} = I$. Then the components of $W = QZ$ are independent standard normal variables.
Proof. By the product rule, the joint density of $Z$ is
$$f(z) = \prod_{i=1}^{n}\frac{1}{\sqrt{2\pi}}\,e^{-z_i^2/2} = (2\pi)^{-n/2}\,e^{-\lVert z\rVert^2/2}, \qquad \lVert z\rVert^2 = z_1^2 + \cdots + z_n^2.$$
It depends on $z$ only through the squared length $\lVert z\rVert^2$. For an orthogonal matrix, $Q^{-1} = Q^{\mathsf{T}}$, so $Q^{\mathsf{T}}Q = I$ as well. Its determinant is $\pm 1$, because $(\det Q)^2 = \det(QQ^{\mathsf{T}}) = 1$. And it preserves length: $\lVert Q^{\mathsf{T}}w\rVert^2 = w^{\mathsf{T}}QQ^{\mathsf{T}}w = \lVert w\rVert^2$. By the linear change of variables, the joint density of $W$ is
$$f(Q^{\mathsf{T}}w) = (2\pi)^{-n/2}\,e^{-\lVert w\rVert^2/2} = \prod_{i=1}^{n}\frac{1}{\sqrt{2\pi}}\,e^{-w_i^2/2}.$$
This is a product of $n$ standard normal densities. Integrating out all coordinates but one shows that each $W_i$ has the standard normal density, since each factor integrates to 1. The joint density is therefore the product of the individual densities, and by the product rule the $W_i$ are independent. $\blacksquare$
This step is where normality is used: the joint density of independent standard normal coordinates depends on the point only through its distance from the origin.
Step 2: an orthogonal matrix whose first row carries the mean. Let $Q$ be the $n \times n$ matrix with rows
$$q_1 = \frac{1}{\sqrt{n}}\,(1, 1, \dots, 1), \qquad q_k = \frac{1}{\sqrt{k(k-1)}}\,(\underbrace{1, \dots, 1}_{k-1},\ -(k-1),\ 0, \dots, 0), \quad k = 2, \dots, n.$$
Row $k$ has $k - 1$ ones, then the entry $-(k-1)$ in position $k$, then zeros. For $n = 3$ the rows are $(1, 1, 1)/\sqrt{3}$, $(1, -1, 0)/\sqrt{2}$ and $(1, 1, -2)/\sqrt{6}$. This is Helmert's matrix. Its rows are orthonormal, by three checks:
- Each row has length 1. For $q_1$, $n \cdot \tfrac{1}{n} = 1$. For $q_k$, the squares sum to $\dfrac{(k-1) + (k-1)^2}{k(k-1)} = \dfrac{(k-1)\,k}{k(k-1)} = 1$.
- Each $q_k$ with $k \ge 2$ is perpendicular to $q_1$. The entries of $q_k$ sum to $\dfrac{(k-1) - (k-1)}{\sqrt{k(k-1)}} = 0$.
- Two rows $q_j$ and $q_k$ with $2 \le j < k$ are perpendicular. The nonzero entries of $q_j$ lie in positions 1 to $j$, where $q_k$ is constant, equal to $1/\sqrt{k(k-1)}$. So $q_j \cdot q_k$ is that constant times the sum of the entries of $q_j$, which is 0.
Orthonormal rows mean $QQ^{\mathsf{T}} = I$, so $Q$ is orthogonal.
Step 3: reading off the three facts. Let $Z_i = (X_i - \mu)/\sigma$. The $Z_i$ are independent, because the $X_i$ are and each $Z_i$ is a function of its own $X_i$ (the third cited property), and each is standard normal (standardization). Set $W = QZ$ with the matrix of step 2. By step 1, $W_1, \dots, W_n$ are independent standard normal variables. Write $\bar{Z}$ for the mean of the $Z_i$, so that $\bar{Z} = (\bar{X} - \mu)/\sigma$ and $Z_i - \bar{Z} = (X_i - \bar{X})/\sigma$.
The first coordinate is
$$W_1 = q_1 \cdot Z = \frac{1}{\sqrt{n}}\sum_{i=1}^{n} Z_i = \sqrt{n}\,\bar{Z} = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}}.$$
The rotation preserves length, $\lVert W\rVert^2 = Z^{\mathsf{T}}Q^{\mathsf{T}}QZ = \lVert Z\rVert^2$. Expanding the square, $\sum_i (Z_i - \bar{Z})^2 = \sum_i Z_i^2 - 2\bar{Z}\sum_i Z_i + n\bar{Z}^2 = \sum_i Z_i^2 - n\bar{Z}^2$. So
$$\frac{(n-1)S^2}{\sigma^2} = \sum_{i=1}^{n}(Z_i - \bar{Z})^2 = \lVert Z\rVert^2 - n\bar{Z}^2 = \lVert W\rVert^2 - W_1^2 = W_2^2 + W_3^2 + \cdots + W_n^2.$$
Now the three facts follow.
- Fact 1. $W_1$ is standard normal, and $W_1$ is exactly the $Z$ of fact 1. So $\bar{X} = \mu + (\sigma/\sqrt{n})\,W_1$ is normal with mean $\mu$ and standard deviation $\sigma/\sqrt{n}$.
- Fact 2. $(n-1)S^2/\sigma^2$ is a sum of the squares of $n - 1$ independent standard normal variables, $W_2, \dots, W_n$. By definition that is $\chi^2_{n-1}$.
- Fact 3. $\bar{X}$ is a function of $W_1$ alone, and $S^2 = \sigma^2(W_2^2 + \cdots + W_n^2)/(n-1)$ is a function of $W_2, \dots, W_n$ alone. The variable $W_1$ is independent of the vector $(W_2, \dots, W_n)$: their joint density $\phi(w_1)\prod_{i \ge 2}\phi(w_i)$ factors into a function of $w_1$ times a function of the rest. By the third cited property, $\bar{X}$ and $S^2$ are therefore independent. $\blacksquare$
The count of degrees of freedom is now literal. The sample has $n$ independent coordinates. After the rotation one of them, $W_1$, holds the mean, and the remaining $n - 1$ hold the variance.
The identity behind fact 2 holds for any numbers, not only for random ones: if $w = Qx$, then $w_1 = \sqrt{n}\,\bar{x}$ and $w_2^2 + \cdots + w_n^2 = \sum_i (x_i - \bar{x})^2$. The script checks it on the ten sleep differences worked in Confidence Intervals for a Mean. Their first rotated coordinate is $\sqrt{10} \times 1.58 = 4.996$, and the squares of the other nine sum to 13.616, the sum of squared deviations found there. One-Way Analysis of Variance uses the same argument, with a rotation whose first axis is weighted by the group sizes, for its between-groups sum of squares.
The chi-squared density
The definition of $\chi^2_\nu$ as a sum of squares is enough for the three facts. Computing probabilities, and deriving the density of $t$ below, needs its density as well. A chi-squared variable with $\nu$ degrees of freedom has density
$$f_\nu(x) = \frac{x^{\nu/2 - 1}\,e^{-x/2}}{2^{\nu/2}\,\Gamma(\nu/2)}, \qquad x > 0,$$
and 0 for $x \le 0$. Here $\Gamma$ is the gamma function, $\Gamma(a) = \int_0^\infty u^{a-1}e^{-u}\,du$ for $a > 0$. Integration by parts gives $\Gamma(a + 1) = a\,\Gamma(a)$, and $\Gamma(1) = 1$, so $\Gamma(k) = (k-1)!$ for whole numbers $k$: the gamma function extends the factorial. The substitution $u = x/2$ turns its definition into the form used below,
$$\int_0^\infty x^{a-1}e^{-x/2}\,dx = 2^a\,\Gamma(a). \qquad (1)$$
The idea of the proof is induction on $\nu$: find the density of one squared normal directly, then add one squared normal at a time. One more property of densities is cited, not proved here, the convolution formula (Casella and Berger, 2002, the chapter "Properties of a Random Sample"): if $U$ and $Y$ are independent and positive with densities $f_U$ and $f_Y$, then $U + Y$ has density $\int_0^x f_U(u)\,f_Y(x - u)\,du$ at $x > 0$.
Proof. One degree of freedom. For $x > 0$, $P(Z^2 \le x) = P(-\sqrt{x} \le Z \le \sqrt{x}) = 2\Phi(\sqrt{x}) - 1$. Differentiating, the density of $Z^2$ is
$$2\phi(\sqrt{x}) \cdot \frac{1}{2\sqrt{x}} = \frac{x^{-1/2}\,e^{-x/2}}{\sqrt{2\pi}}.$$
A density integrates to 1, so $\int_0^\infty x^{-1/2}e^{-x/2}\,dx = \sqrt{2\pi}$. By (1) with $a = \tfrac12$ the same integral is $2^{1/2}\,\Gamma(\tfrac12)$. Hence $\Gamma(\tfrac12) = \sqrt{\pi}$, and the density of $Z^2$ is $f_1$.
From $\nu$ to $\nu + 1$. Suppose the formula holds for $\nu$. A $\chi^2_{\nu+1}$ variable is $U + Y$ with $U$ distributed as $\chi^2_\nu$ and $Y = Z_{\nu+1}^2$ an independent $\chi^2_1$. By the convolution formula its density at $x > 0$ is
$$\int_0^x \frac{u^{\nu/2-1}e^{-u/2}}{2^{\nu/2}\Gamma(\nu/2)} \cdot \frac{(x-u)^{-1/2}e^{-(x-u)/2}}{\sqrt{2\pi}}\,du = \frac{e^{-x/2}}{2^{\nu/2}\Gamma(\nu/2)\sqrt{2\pi}}\int_0^x u^{\nu/2-1}(x-u)^{-1/2}\,du.$$
Substituting $u = xs$ in the last integral gives $x^{\nu/2 - 1} \cdot x^{-1/2} \cdot x\int_0^1 s^{\nu/2-1}(1-s)^{-1/2}\,ds = B\,x^{(\nu+1)/2 - 1}$, where $B$ is a positive constant that does not depend on $x$. It is finite because both exponents exceed $-1$. So the density of $U + Y$ is $C\,x^{(\nu+1)/2-1}e^{-x/2}$ for some constant $C$. The constant is fixed by the requirement that a density integrate to 1: by (1) with $a = (\nu+1)/2$, $C = 1/\left[2^{(\nu+1)/2}\,\Gamma\!\left(\tfrac{\nu+1}{2}\right)\right]$. That is $f_{\nu+1}$. $\blacksquare$
Two consequences are used later. Let $V$ be $\chi^2_\nu$. By (1) and $\Gamma(a + 1) = a\,\Gamma(a)$,
$$E\!\left(\frac{1}{V}\right) = \frac{\int_0^\infty x^{\nu/2-2}e^{-x/2}\,dx}{2^{\nu/2}\Gamma(\nu/2)} = \frac{2^{\nu/2-1}\,\Gamma(\nu/2 - 1)}{2^{\nu/2}\,\Gamma(\nu/2)} = \frac{1}{2(\nu/2 - 1)} = \frac{1}{\nu - 2} \quad (\nu > 2),$$
$$E\!\left(\sqrt{V}\right) = \frac{\int_0^\infty x^{(\nu+1)/2-1}e^{-x/2}\,dx}{2^{\nu/2}\Gamma(\nu/2)} = \frac{\sqrt{2}\;\Gamma\!\left(\frac{\nu+1}{2}\right)}{\Gamma\!\left(\frac{\nu}{2}\right)}.$$
For $\nu \le 2$ the first integral diverges at 0. The second gives the average of the sample standard deviation of a normal sample. Since $S = \sigma\sqrt{V/(n-1)}$ with $V$ distributed as $\chi^2_{n-1}$, $E(S) = \sigma\sqrt{2/(n-1)}\;\Gamma(n/2)/\Gamma((n-1)/2)$. That is $0.886\,\sigma$ for $n = 3$ and $0.973\,\sigma$ for $n = 10$: slightly below $\sigma$, as the example in Confidence Intervals for a Mean on why the errors of a too-small and a too-large $s$ do not cancel requires.
Dividing out sigma
The $t$ distribution with $\nu$ degrees of freedom is defined as the distribution of
$$T = \frac{Z}{\sqrt{V/\nu}},$$
where $Z$ is standard normal, $V$ is $\chi^2_\nu$, and the two are independent. In words: a standard normal divided by the square root of an independent chi-squared over its degrees of freedom.
Facts 1 to 3 supply exactly these ingredients, with $\nu = n - 1$: the $Z$ of fact 1 is standard normal, the $V$ of fact 2 is $\chi^2_{n-1}$, and by fact 3 they are independent, since $Z$ is a function of $\bar{X}$ and $V$ of $S^2$. Substituting them,
$$\frac{Z}{\sqrt{V/(n-1)}} = \frac{(\bar{X} - \mu)\big/(\sigma/\sqrt{n})}{\sqrt{S^2/\sigma^2}} = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \cdot \frac{\sigma}{S} = \frac{\bar{X} - \mu}{S/\sqrt{n}}.$$
The unknown $\sigma$ cancels. That is the whole point, and it gives the result this article set out to reach.
Theorem (Student's ratio). If $X_1, \dots, X_n$ are independent draws from a normal population with mean $\mu$ and standard deviation $\sigma$, and $n \ge 2$, then
$$T = \frac{\bar{X} - \mu}{S/\sqrt{n}}$$
has the $t$ distribution with $n - 1$ degrees of freedom, whatever the values of $\mu$ and $\sigma$.
It is therefore a pivot: a quantity involving the unknown $\mu$ whose distribution is completely known. What has been proved and what has been cited can now be stated exactly. The three facts and the theorem are proved in full above, given the three cited properties of joint densities and independence. Casella and Berger (2002), in the chapter "Properties of a Random Sample", prove the three facts by a different route.
The density of t
The $t$ distribution with $\nu$ degrees of freedom has density
$$f(t) = \frac{\Gamma\!\left(\frac{\nu+1}{2}\right)}{\sqrt{\nu\pi}\;\Gamma\!\left(\frac{\nu}{2}\right)}\left(1 + \frac{t^2}{\nu}\right)^{-(\nu+1)/2}.$$
This is the density given in the NIST handbook (§1.3.6.6.4), written there with the beta function, which is equivalent. It follows from the chi-squared density derived above, and so it rests on the cited convolution formula as well as the product rule. The idea: fix the value of $V$, where $T \le t$ is an event about $Z$ alone, then average over $V$.
Derivation. Let $T = Z/\sqrt{V/\nu}$ with $Z$ standard normal and $V$ an independent $\chi^2_\nu$. By the product rule the joint density of $(Z, V)$ is $\phi(z)\,f_\nu(v)$. The event $T \le t$ is the event $Z \le t\sqrt{V/\nu}$, so integrating the joint density over it, first in $z$,
$$P(T \le t) = \int_0^\infty \Phi\!\left(t\sqrt{v/\nu}\right) f_\nu(v)\,dv.$$
Differentiate with respect to $t$ under the integral sign. (That is allowed here because the derivative of the integrand, $\phi(t\sqrt{v/\nu})\sqrt{v/\nu}\,f_\nu(v)$, is at most $\sqrt{v/\nu}\,f_\nu(v)/\sqrt{2\pi}$ for every $t$, and that bound has a finite integral.) With $\phi(z) = e^{-z^2/2}/\sqrt{2\pi}$ and the formula for $f_\nu$,
$$f(t) = \int_0^\infty \frac{e^{-t^2 v/(2\nu)}}{\sqrt{2\pi}}\sqrt{\frac{v}{\nu}}\cdot\frac{v^{\nu/2-1}e^{-v/2}}{2^{\nu/2}\Gamma(\nu/2)}\,dv = \frac{1}{\sqrt{2\pi\nu}\;2^{\nu/2}\,\Gamma(\nu/2)}\int_0^\infty v^{(\nu+1)/2 - 1}\,e^{-\frac{v}{2}\left(1 + t^2/\nu\right)}\,dv.$$
Substitute $x = v\,(1 + t^2/\nu)$. The integral becomes $(1 + t^2/\nu)^{-(\nu+1)/2}\int_0^\infty x^{(\nu+1)/2-1}e^{-x/2}\,dx$, which by (1) equals $(1 + t^2/\nu)^{-(\nu+1)/2}\,2^{(\nu+1)/2}\,\Gamma\!\left(\tfrac{\nu+1}{2}\right)$. The powers of 2 combine to $2^{(\nu+1)/2}/(\sqrt{2}\cdot 2^{\nu/2}) = 1$, which leaves the stated density. $\blacksquare$
Two special cases can be read from the formula. With $\nu = 1$, $\Gamma(1) = 1$ and $\Gamma(\tfrac12) = \sqrt{\pi}$ give $f(t) = 1/[\pi(1 + t^2)]$, the Cauchy density, whose tails are so heavy that it has no mean: $\int_0^\infty t/(1 + t^2)\,dt$ diverges. As $\nu$ grows, the factor $(1 + t^2/\nu)^{-(\nu+1)/2}$ tends to $e^{-t^2/2}$, the shape of the normal density, by the limit $(1 + a/m)^m \to e^a$. The table of critical values in Confidence Intervals for a Mean shows the approach to the normal numerically, and the figure there draws the densities with 1, 4 and 9 degrees of freedom against the normal.
For $\nu > 2$ the variance of $T$ is $\nu/(\nu - 2)$, for example $9/7 = 1.29$ with 9 degrees of freedom, against 1 for the standard normal. The proof is short. $T$ has mean 0 by symmetry, so its variance is $E(T^2) = E(Z^2 \cdot \nu/V)$. Since $Z$ and $V$ are independent, the expectation of the product is the product of the expectations, $E(Z^2) \cdot \nu \cdot E(1/V) = 1 \cdot \nu \cdot \tfrac{1}{\nu - 2}$, by the value of $E(1/V)$ found with the chi-squared density. For $\nu \le 2$ that expectation is infinite, and so is the variance of $T$.
Worked problems
Both problems are posed on Student's sleep data, to exercise the rotation and the chi-squared distribution once each. Every number is recomputed by the article's script, tools/experiments/applied-statistics/chi-squared-and-t-distributions.py.
Problem 1 (the rotation by hand). For the first three sleep differences, 1.2, 2.4 and 1.3 hours, compute the rotated coordinates $w = Qx$ with Helmert's matrix for $n = 3$. Verify that $w_1 = \sqrt{3}\,\bar{x}$, that $w_2^2 + w_3^2$ is the sum of squared deviations, and that the rotation preserves the squared length.
Solution. All three hold: $w = (2.829, -0.849, 0.408)$, and $w_2^2 + w_3^2 = 0.8867 = \sum_i (x_i - \bar{x})^2$. The rows of $Q$ are $(1, 1, 1)/\sqrt{3}$, $(1, -1, 0)/\sqrt{2}$ and $(1, 1, -2)/\sqrt{6}$, so
$$w_1 = \frac{1.2 + 2.4 + 1.3}{\sqrt{3}} = \frac{4.9}{\sqrt{3}} = 2.829, \qquad w_2 = \frac{1.2 - 2.4}{\sqrt{2}} = -0.849, \qquad w_3 = \frac{1.2 + 2.4 - 2.6}{\sqrt{6}} = 0.408.$$
The mean is $\bar{x} = 4.9/3 = 1.6333$, and $\sqrt{3} \times 1.6333 = 2.829 = w_1$. The deviations from the mean are $-0.4333$, $0.7667$ and $-0.3333$, with squares summing to $0.1878 + 0.5878 + 0.1111 = 0.8867$. On the other side, $w_2^2 + w_3^2 = 0.7200 + 0.1667 = 0.8867$. For the length, $w_1^2 + w_2^2 + w_3^2 = 8.0033 + 0.8867 = 8.89$ and $1.2^2 + 2.4^2 + 1.3^2 = 1.44 + 5.76 + 1.69 = 8.89$. The sample variance of the three differences is therefore $0.8867/2 = 0.4433$, computed from the two coordinates that do not carry the mean.
Problem 2 (an interval for the standard deviation). Assuming the ten sleep differences are a normal sample, use fact 2 to find a 95% confidence interval for the population standard deviation $\sigma$ of the differences.
Solution. The interval is $[0.846, 2.245]$ hours. By fact 2, $V = (n-1)S^2/\sigma^2$ is $\chi^2_9$, whatever $\sigma$ is, so $V$ is a pivot for $\sigma$. Let $a$ and $b$ be the points with probability 0.025 below and 0.975 below under the $\chi^2_9$ distribution. The script finds $a = 2.700$ and $b = 19.023$ from the distribution function. Then
$$P\left(a \le \frac{(n-1)S^2}{\sigma^2} \le b\right) = 0.95.$$
Taking reciprocals reverses the inequalities, and multiplying by $(n-1)S^2 > 0$ gives the same event as
$$\frac{(n-1)S^2}{b} \le \sigma^2 \le \frac{(n-1)S^2}{a}.$$
For the sleep data $(n-1)s^2 = 13.616$, the sum of squared deviations. So the interval for $\sigma^2$ is $[13.616/19.023,\ 13.616/2.700] = [0.716, 5.042]$, and taking square roots, $[0.846, 2.245]$ for $\sigma$. The interval is not symmetric about $s = 1.230$, because the chi-squared distribution is skewed to the right. It is also wide: ten differences determine the standard deviation only to within a factor of about 2.7 from one end to the other. That is the uncertainty in $s$ that the $t$ critical value pays for, and the reason One-Sided Confidence Bounds and Sample Size treats a planning value of $\sigma$ from a small study with caution. Its exactness depends on the normality of the differences, more heavily than the $t$ interval's does. The variance of $S^2$ depends on the population's kurtosis (One-Sided Confidence Bounds and Sample Size gives the formula), and the chi-squared interval allows only for the variance that a normal population gives. For a population with heavier tails than the normal, $S^2$ varies more than that at every sample size, so a larger sample does not repair the interval as it repairs the $t$ interval. The simulations of Confidence Intervals for a Mean checked the $t$ interval against non-normal populations, not this one.
What we have learned
A sample from a normal population, standardized and rotated so that its first axis points along $(1, \dots, 1)$, is again a set of independent standard normal coordinates. The first carries the sample mean and the other $n - 1$ carry the sample variance. So the mean is normal, the scaled variance $(n-1)S^2/\sigma^2$ is chi-squared with $n - 1$ degrees of freedom, and the two are independent. Dividing the standardized mean by the square root of the scaled variance over its degrees of freedom cancels $\sigma$ and leaves Student's ratio, which therefore has the $t$ distribution with $n - 1$ degrees of freedom. The densities of both distributions follow, the chi-squared by induction and the $t$ from it. Three properties of joint densities and independence, and the convolution formula, are cited; everything else is proved here.
Confidence Intervals for a Mean turns the theorem into the $t$ interval, tabulates its critical values and tests it on populations that are not normal. Later units use the same facts: The Two-Sample t-Test adds two chi-squared variables, and One-Way Analysis of Variance takes the ratio of two to define the $F$ distribution.
References
- Casella, G., and Berger, R. L. (2002). Statistical Inference (2nd ed.). Duxbury. The chapters "Multiple Random Variables" (independence, joint densities and transformations of random vectors) and "Properties of a Random Sample" (the convolution formula; the mean and variance of a normal sample). record · 3 pages
- Geary, R. C. (1936). The distribution of "Student's" ratio for non-normal samples. Supplement to the Journal of the Royal Statistical Society, 3(2), 178–184. record
- Lukacs, E. (1942). A characterization of the normal distribution. The Annals of Mathematical Statistics, 13(1), 91–93. record
- NIST/SEMATECH. e-Handbook of Statistical Methods, §1.3.6.6.4 (t distribution). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/ record · 32 pages
- Student (1908). The probable error of a mean. Biometrika, 6(1), 1–25. record · 12 pages