Draw 10 adults at random, with replacement, from the 5,107 adults of the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018 used in the Probability unit, and count how many are obese. Of those adults, 2,099 have a body mass index of 30 or more, a share of $p = 0.4110$. The count could be anything from 0 to 10. A count of 4 is the most likely single value, with probability 0.2502. A count of 7 or more has probability 0.0634, and none at all, 0.0050. Where do such numbers come from, and do they still hold when the people counted are not independent of each other?
The count is a random variable, a number determined by the outcome of a random experiment. The list of its possible values with their probabilities is its distribution. The binomial distribution is the distribution of the number of successes in a fixed number of independent trials with the same probability of success. It answers the first question exactly.
First we set out the general ideas: random variables, their distributions, and the two numbers that summarize a distribution, its expected value and its variance, with the rules for computing them. Then we derive the binomial distribution by counting, prove its mean and variance two ways, and describe its shape. Finally we ask what happens when the conditions behind it fail, as they often do in medical data, and measure the consequences on real data.
This article is where the course defines random variables, expected values and variances. Later articles, from The Poisson Distribution to The Sampling Distribution of the Mean, use the rules proved here.
Random variables
A random variable is a function that assigns a number to each outcome of a random experiment. Drawing one adult from NHANES is the experiment. The adult's body mass index, the number of household members, and an indicator that is 1 for obesity and 0 otherwise are all random variables defined on it. Random variables are written with capital letters, $X$, $Y$, $K$, and the values they take with lower-case letters.
A random variable is discrete when its possible values can be listed, usually as whole numbers: counts of events, of people, of positive tests. It is continuous when it can take any value in an interval, as a measurement such as blood pressure or height can, at least in principle. The two kinds need different mathematical tools: sums for discrete variables and integrals for continuous ones. This article treats discrete variables, and The Normal Distribution treats continuous ones.
The probability mass function
A first example comes from the way NHANES measures blood pressure. After the participant has rested quietly for five minutes, the examiner takes three consecutive readings (National Center for Health Statistics, 2020, BPX_J documentation). Of the 5,247 examined adults with at least one systolic reading, 4,730 have all three. Pick one of them at random and let $K$ be the number of the three readings that are 130 mm Hg or higher. $K$ takes the values 0, 1, 2 and 3. Its probabilities are the shares of adults with each count:
| $k$ | Adults | $P(K = k)$ | $F(k) = P(K \le k)$ |
|---|---|---|---|
| 0 | 2,778 | 0.5873 | 0.5873 |
| 1 | 269 | 0.0569 | 0.6442 |
| 2 | 268 | 0.0567 | 0.7008 |
| 3 | 1,415 | 0.2992 | 1.0000 |
The function $p(k) = P(K = k)$ is the probability mass function (pmf) of $K$. Every pmf has two properties, which follow directly from the axioms of The Rules of Probability. Each value is between 0 and 1. And the values add to 1, because the events $\{K = 0\}, \{K = 1\}, \ldots$ are mutually exclusive and together make up the whole sample space.
The distribution function
The cumulative distribution function (CDF) of a random variable $X$ is
$$F(x) = P(X \le x),$$
defined for every real number $x$, not only the values $X$ takes. For a discrete variable it is the running total of the pmf, as in the last column above. Its graph is a staircase that jumps by $p(k)$ at each value $k$ and is flat between them.
Every CDF rises from 0 at the far left to 1 at the far right and never decreases. The reason is the monotonicity rule: the event $\{X \le a\}$ is contained in $\{X \le b\}$ when $a < b$. Probabilities of ranges come from differences: $P(a < X \le b) = F(b) - F(a)$. For $K$, $P(1 \le K \le 2) = F(2) - F(0) = 0.7008 - 0.5873 = 0.1135$, which agrees with $0.0569 + 0.0567$ up to rounding.
The same table could have come from a model rather than from counting. That is the point of a theoretical distribution: a formula for $p(k)$, with one or two parameters, that describes many different situations at once. Before building one, we need the two numbers that summarize any distribution.
Expectation and variance
The expected value
The expected value, or mean, of a discrete random variable is the average of its values weighted by their probabilities:
$$E[X] = \mu = \sum_x x\, p(x).$$
For $K$,
$$E[K] = 0(0.5873) + 1(0.0569) + 2(0.0567) + 3(0.2992) = 1.0677,$$
computed with the unrounded shares (the rounded ones give 1.0679).
When the probabilities are shares of a finite population, as here, $E[K]$ is simply the population mean of $K$: the average adult has 1.0677 of three readings at or above 130. In the relative-frequency view of The Rules of Probability, it is the value the average of many independent draws of $K$ settles toward.
Check your understanding. No adult can have 1.0677 high readings. In what sense, then, is 1.0677 the "expected" value of $K$?
Not as a typical value: the most common value of $K$ is 0, and 1 is one of the rarest. It is a long-run average, the balance point of the distribution, and it need not be a value the variable can take.
Three rules for expected values are used constantly. For constants $a$ and $b$ and random variables $X$ and $Y$ on the same experiment,
$$E[aX + b] = aE[X] + b, \qquad E[X + Y] = E[X] + E[Y], \qquad E[g(X)] = \sum_x g(x)\,p(x).$$
The idea of the proof is that an expected value can be computed outcome by outcome, and sums can then be split term by term.
Proof. Take a finite sample space with outcomes $\omega$ of probability $P(\omega)$. The expected value of any random variable $Z$ can be computed outcome by outcome, $E[Z] = \sum_\omega Z(\omega) P(\omega)$, because grouping the outcomes by the value $Z$ gives them recovers $\sum_z z\,p(z)$. Then
$$E[X + Y] = \sum_\omega \big(X(\omega) + Y(\omega)\big) P(\omega) = \sum_\omega X(\omega) P(\omega) + \sum_\omega Y(\omega) P(\omega) = E[X] + E[Y].$$
In the same way $E[aX + b] = a\sum_\omega X(\omega)P(\omega) + b\sum_\omega P(\omega) = aE[X] + b$, since the probabilities add to 1. The third rule is the outcome-by-outcome formula applied to $Z = g(X)$ and grouped by the value of $X$. $\square$
The second rule, linearity, needs no independence. The expected value of a sum is the sum of the expected values, whatever the relation between the terms. The same results hold for infinite sample spaces whenever the sums converge (Grinstead and Snell, 1997, §6.1).
The variance
The mean says where a distribution is centered. How spread out is it? The variance is the expected squared distance from the mean,
$$\operatorname{Var}(X) = \sigma^2 = E\big[(X - \mu)^2\big] = \sum_x (x - \mu)^2 p(x),$$
and the standard deviation $\sigma$ is its square root, in the units of $X$. Expanding the square and using linearity gives the computing formula
$$\operatorname{Var}(X) = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - \mu^2.$$
For $K$, $E[K^2] = 1(0.0569) + 4(0.0567) + 9(0.2992) = 2.9759$ from the unrounded shares. So $\operatorname{Var}(K) = 2.9759 - 1.0677^2 = 1.8360$, and the standard deviation is 1.3550. These are the population variance and standard deviation of Measures of Center and Spread, with divisor $N$, written for a distribution rather than a list of values.
Adding a constant shifts a distribution without changing its spread. Multiplying by $a$ multiplies distances from the mean by $|a|$:
$$\operatorname{Var}(aX + b) = E\big[(aX + b - a\mu - b)^2\big] = a^2 \operatorname{Var}(X).$$
For sums, independence matters. Two discrete random variables are independent when $P(X = x, Y = y) = P(X = x)\,P(Y = y)$ for all values $x$ and $y$. In words, every event about $X$ is independent, in the sense of The Rules of Probability, of every event about $Y$. For independent variables the expected value of a product factors:
$$E[XY] = \sum_{x, y} xy\, P(X = x)P(Y = y) = \Big(\sum_x x\, P(X = x)\Big)\Big(\sum_y y\, P(Y = y)\Big) = E[X]\,E[Y].$$
Now expand the variance of a sum, writing $\mu_X$ and $\mu_Y$ for the means:
$$\operatorname{Var}(X + Y) = E\big[\big((X - \mu_X) + (Y - \mu_Y)\big)^2\big] = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2E\big[(X - \mu_X)(Y - \mu_Y)\big].$$
The last expectation is the covariance of $X$ and $Y$. By linearity it equals $E[XY] - \mu_X \mu_Y$, which is zero for independent variables. So
$$\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) \quad \text{for independent } X, Y.$$
By induction, the variance of a sum of any number of independent variables is the sum of their variances (Grinstead and Snell, 1997, §6.2). Unlike linearity of expectation, this rule fails for dependent variables. That failure is the subject of the last section of the article.
Check your understanding. If $X$ and $Y$ are independent, what is the variance of the difference $X - Y$?
It is $\operatorname{Var}(X) + \operatorname{Var}(Y)$, not their difference. Write $X - Y = X + (-1)Y$, a sum of independent variables, and note that $\operatorname{Var}(-Y) = (-1)^2\operatorname{Var}(Y)$. Subtracting a variable adds its uncertainty, just as adding one does.
Bernoulli trials
The simplest random variable takes only the values 0 and 1. Drawing one adult and recording $Y = 1$ for obesity and $Y = 0$ otherwise gives a Bernoulli variable with success probability $p = P(Y = 1)$. Its mean and variance are
$$E[Y] = 0(1 - p) + 1 \cdot p = p, \qquad \operatorname{Var}(Y) = E[Y^2] - p^2 = p - p^2 = p(1 - p),$$
using $Y^2 = Y$ for a variable that is 0 or 1. The words "success" and "failure" are labels for the two outcomes and carry no judgment. In medicine the "success" being counted is often a disease or a death.
A sequence of Bernoulli trials is a sequence of such experiments with four properties (Grinstead and Snell, 1997, §3.2):
- The number of trials, $n$, is fixed in advance.
- Each trial has two possible outcomes, success and failure.
- The probability of success, $p$, is the same on every trial.
- The trials are independent: the outcome of one trial does not change the probabilities for any other.
Drawing 10 adults with replacement and recording obesity meets all four. The number of trials is 10. Each draw is a fresh, equally likely choice among the same 5,107 adults. So each has the same probability $p = 0.4110$ of producing an obese adult, whatever the earlier draws produced.
Check your understanding. Treat an adult's three blood pressure readings as three trials, with "at or above 130 mm Hg" as a success. Is $K$, the number of successes, a sequence of Bernoulli trials? Which condition is in doubt?
The doubtful condition is the fourth, independence. A person whose first reading is high probably has high blood pressure, so the next reading is likely to be high too. The last section measures the consequences.
The binomial probability
Let $X$ be the number of successes in $n$ Bernoulli trials. What is $P(X = x)$? Here is the idea in one sentence. Every particular order of $x$ successes and $n - x$ failures has the same probability, so $P(X = x)$ is the number of such orders times the probability of one of them.
First take one particular sequence with $x$ successes, say the one in which the first $x$ trials are successes and the remaining $n - x$ are failures. By independence, its probability is the product of the probabilities of the individual outcomes:
$$\underbrace{p \cdot p \cdots p}_{x} \cdot \underbrace{(1 - p)\cdots(1 - p)}_{n - x} = p^x (1 - p)^{n - x}.$$
Every other sequence with exactly $x$ successes has the same probability, since it is the same product taken in a different order. The sequences differ only in which $x$ of the $n$ positions hold the successes. The number of ways to choose $x$ positions from $n$ is the binomial coefficient
$$\binom{n}{x} = \frac{n!}{x!\,(n - x)!}.$$
The sequences are mutually exclusive, so by the addition axiom their probabilities add:
$$P(X = x) = \binom{n}{x} p^x (1 - p)^{n - x}, \qquad x = 0, 1, \ldots, n.$$
This is the binomial distribution with parameters $n$ and $p$, written $X \sim \text{Bin}(n, p)$. The two parameters determine the whole distribution. The probabilities add to 1, as a pmf's must, by the binomial theorem:
$$\sum_{x=0}^{n} \binom{n}{x} p^x (1 - p)^{n - x} = \big(p + (1 - p)\big)^n = 1^n = 1.$$
That theorem is where the distribution's name comes from.
A worked case
For the 10 adults, $n = 10$ and $p = 0.4110$. The probability of exactly 4 obese adults is
$$P(X = 4) = \binom{10}{4}(0.4110)^4(0.5890)^6 = 210 \times 0.001191 = 0.2502.$$
The factor 0.001191 is the probability of one particular order, such as four obese adults followed by six who are not. The 210 orders together make 0.2502. The full distribution:
| $x$ | $\binom{10}{x}$ | $P(X = x)$ | $P(X \le x)$ |
|---|---|---|---|
| 0 | 1 | 0.0050 | 0.0050 |
| 1 | 10 | 0.0351 | 0.0401 |
| 2 | 45 | 0.1101 | 0.1502 |
| 3 | 120 | 0.2049 | 0.3551 |
| 4 | 210 | 0.2502 | 0.6053 |
| 5 | 252 | 0.2095 | 0.8148 |
| 6 | 210 | 0.1218 | 0.9366 |
| 7 | 120 | 0.0486 | 0.9852 |
| 8 | 45 | 0.0127 | 0.9979 |
| 9 | 10 | 0.0020 | 0.9999 |
| 10 | 1 | 0.0001 | 1.0000 |
Questions about ranges are answered by adding rows or by subtracting values of the CDF:
- at most 2 obese adults: $F(2) = 0.1502$;
- 7 or more: $1 - F(6) = 1 - 0.9366 = 0.0634$;
- 3 to 5: $F(5) - F(2) = 0.6646$.
The complement rule makes "at least one" easy: $P(X \ge 1) = 1 - P(X = 0) = 1 - 0.5890^{10} = 0.9950$.
Does the formula match the population itself? The script drew 100,000 samples of 10 adults with replacement for each of 30 random seeds. The share of samples with exactly 4 obese adults had median 0.2502 across the seeds (interquartile range 0.2493 to 0.2508), matching the formula.
Check your understanding. Count the adults among the 10 who are not obese. What is the distribution of that count, and what is the probability that exactly 6 of the 10 are not obese?
The count is $10 - X$, which is $\text{Bin}(10, 0.5890)$, since each draw is a Bernoulli trial for "not obese" too. Exactly 6 non-obese adults is the same event as exactly 4 obese ones, so its probability is again 0.2502.
Mean and variance
Ten draws, each obese with probability 0.411, should produce about 4.11 obese adults on average. In general, the mean of a binomial count is $np$ and its variance $np(1 - p)$. There are two proofs, and both are worth knowing.
Through indicators. Write $X = Y_1 + Y_2 + \cdots + Y_n$, where $Y_i$ is 1 if trial $i$ is a success and 0 if not. Each $Y_i$ is a Bernoulli variable with mean $p$ and variance $p(1 - p)$. By linearity,
$$E[X] = E[Y_1] + \cdots + E[Y_n] = np,$$
and since the trials are independent, the variances add:
$$\operatorname{Var}(X) = \operatorname{Var}(Y_1) + \cdots + \operatorname{Var}(Y_n) = np(1 - p). \qquad \square$$
The first statement uses only linearity, so it holds even when the trials are dependent. The second uses independence, and the last section shows what happens without it.
From the formula. The identity $x\binom{n}{x} = n\binom{n-1}{x-1}$ holds because both sides count the ways to choose a committee of $x$ from $n$ people and then a chair from the committee. It gives
$$E[X] = \sum_{x=1}^{n} x \binom{n}{x} p^x (1-p)^{n-x} = np \sum_{x=1}^{n} \binom{n-1}{x-1} p^{x-1} (1-p)^{n-x} = np,$$
because the last sum, with $j = x - 1$, is the total of the $\text{Bin}(n - 1, p)$ probabilities. Applying the identity twice gives $E[X(X-1)] = n(n-1)p^2$ in the same way, and then
$$\operatorname{Var}(X) = E[X(X - 1)] + E[X] - (E[X])^2 = n(n-1)p^2 + np - n^2p^2 = np(1 - p). \qquad \square$$
Back to the 10 adults. $E[X] = 10 \times 0.4110 = 4.110$ and $\operatorname{Var}(X) = 10 \times 0.4110 \times 0.5890 = 2.4208$, a standard deviation of 1.5559. Summing $x\,p(x)$ and $(x - 4.110)^2 p(x)$ over the table gives the same values. In the simulation, the mean of the 100,000 counts had median 4.1099 across the 30 seeds, and their variance 2.4228.
Dividing the count by $n$ gives the sample proportion, with mean $p$ and standard error $\sqrt{p(1-p)/n}$. That consequence is developed in The Normal Approximation to the Binomial.
Shape
The mean and variance locate and scale the distribution. What does it look like?
Three facts describe the shape.
The mode. The ratio of successive probabilities is
$$\frac{P(X = x + 1)}{P(X = x)} = \frac{n - x}{x + 1} \cdot \frac{p}{1 - p}.$$
It exceeds 1 exactly when $(n - x)p > (x + 1)(1 - p)$, that is, when $x < (n + 1)p - 1$. So the probabilities rise while $x$ is below $(n + 1)p - 1$ and fall after it. The most likely value is the whole number $x$ with $(n+1)p - 1 \le x \le (n + 1)p$. If $(n + 1)p$ is itself a whole number, it and the value one below tie (NIST/SEMATECH, §1.3.6.6.18). For the 10 adults, $(n + 1)p = 4.521$, and the mode is 4.
Symmetry and reflection. Counting failures instead of successes turns $X$ into $n - X$, which is $\text{Bin}(n, 1 - p)$, as the non-obese count showed. So the distribution for $p = 0.9$ is the mirror image of the one for $p = 0.1$. For $p = 0.5$ the distribution is its own mirror image: $P(X = x) = P(X = n - x)$, for example $P(X = 3) = P(X = 7) = 0.1172$ when $n = 10$.
Skewness. For $p < 0.5$ the distribution has a longer right tail, because the count cannot fall below zero but can rise to $n$. The skewness, derived in The Normal Approximation to the Binomial, is $(1 - 2p)/\sqrt{np(1 - p)}$. It is 0.843 for $n = 10$ and $p = 0.1$, 0.114 for $p = 0.411$, and 0.057 for $n = 40$ and $p = 0.411$. It shrinks like $1/\sqrt{n}$, which is why the binomial approaches the normal curve as $n$ grows. When $n$ is large and $p$ small, so that the mean $np$ stays moderate, the binomial approaches a different limit, the Poisson distribution.
Computing binomial probabilities
The formula can be evaluated directly for small $n$, and printed tables of binomial probabilities served when that was laborious. Software now computes exact probabilities for any $n$. It takes care with the very large and very small numbers in $\binom{n}{x}p^x(1-p)^{n-x}$, so there is no longer a reason to approximate a binomial probability in a calculation. The normal approximation keeps its importance for a different reason. It describes how the count behaves, and it yields the closed-form intervals and tests of Inference on Proportions.
When the conditions fail
So far every count has come from draws with replacement, which meet all four conditions by construction. The binomial distribution is exact only under its four conditions. In medical data the first two rarely cause trouble: the number of patients or subjects is usually fixed by design, and an outcome can be made binary by definition. The other two often fail, and in different directions.
Sampling without replacement
A real study samples people without replacement. Each person drawn slightly changes the composition of those who remain, so the trials are not quite independent. The exact distribution is then the hypergeometric, with the same mean and a variance smaller by the factor $(N - n)/(N - 1)$. For a sample that is a small fraction of the population the difference is negligible. The Normal Approximation to the Binomial measures it for samples of 50 from 1,696 young adults.
Unequal probabilities
Suppose a study recruits 4 adults aged 18 to 39, 3 aged 40 to 59 and 3 aged 60 or over, each drawn at random within their age group, and counts how many have hypertension. In the NHANES population of this article, the probabilities of hypertension in the three groups are 0.2280, 0.5825 and 0.7984. The trials are independent, but $p$ is not constant, so the count is not binomial.
Its exact distribution comes from combining the trials one at a time (a convolution). Its mean is still $\sum p_i = 5.055$, by linearity. Since the trials are still independent, its variance is
$$\operatorname{Var}(X) = \sum_{i=1}^{n} p_i(1 - p_i).$$
How does that compare with a binomial? Write $\bar{p}$ for the average of the $p_i$ and $V = \frac{1}{n}\sum (p_i - \bar{p})^2$ for their spread. Then $\sum p_i^2 = n(\bar{p}^2 + V)$, and so
$$\sum_{i=1}^{n} p_i(1 - p_i) = n\bar{p} - n(\bar{p}^2 + V) = n\bar{p}(1 - \bar{p}) - nV.$$
Unequal probabilities among independent trials make the count less variable than a binomial with the same average probability. Here $\bar{p} = 0.5055$, and the binomial variance would be 2.4997. The actual variance is 1.9165, smaller by $nV = 0.5832$. Extreme counts become rarer. No hypertensive adults has probability 0.0002, against 0.0009 under the binomial; all ten, 0.0003 against 0.0011. A design that fixes the number from each group removes the variation between groups from the count. That is the idea behind stratified sampling.
Dependence within a person
Return to the three readings of each adult. Suppose the readings of one person were independent trials with a common probability. Then $K$ would be $\text{Bin}(3, p)$, where $p$ is the share of all readings at or above 130: of the 14,190 readings, that share is 0.3559. The mean of $K$ would be $3p = 1.0677$. That matches the data exactly, as linearity guarantees with or without independence. Everything else fails.
| $k$ | Observed | $\text{Bin}(3, 0.3559)$ |
|---|---|---|
| 0 | 2,778 | 1,264.0 |
| 1 | 269 | 2,095.2 |
| 2 | 268 | 1,157.6 |
| 3 | 1,415 | 213.2 |
The binomial predicts that most adults have one or two high readings out of three. In fact most have none or all three. The readings of one person are not independent trials. A person whose blood pressure is high tends to have it high on all three readings, and the probability of a high reading differs greatly from person to person. The variance of $K$ is 1.8360, against $3p(1 - p) = 0.6877$ for the binomial, a ratio of 2.670.
Clustering
The same mechanism operates whenever trials come in clusters: patients within the same hospital, children within the same school, people within the same household or neighborhood. NHANES itself samples people in geographic clusters. For variance estimation NCHS releases 30 masked variance units, 15 pseudo-strata of 2 pseudo-PSUs each (variables SDMVSTRA and SDMVPSU of the DEMO_J file). Each is an aggregate of the survey's lower-level sampling units, grouped and masked to protect participants' confidentiality, and the article treats each as one location. Among the adults of this article, the share who are obese ranges from 0.2659 in one group to 0.5479 in another.
Suppose a study chooses one of the 30 groups at random and then draws 10 adults from it. How much more variable is the count? Here is the idea before the algebra. Given the group, the count is binomial with that group's probability, but the probability itself is now random, and its variation adds to the binomial variance.
The formal version. Write $P$ for the group's probability, with mean $\bar{p}$ and variance $\operatorname{Var}(P)$ over the groups. Then $E[X \mid P] = nP$ and $\operatorname{Var}(X \mid P) = nP(1 - P)$. The law of total variance, $\operatorname{Var}(X) = E[\operatorname{Var}(X \mid P)] + \operatorname{Var}(E[X \mid P])$, gives
$$\operatorname{Var}(X) = n\bar{p} - nE[P^2] + n^2\operatorname{Var}(P) = n\bar{p}(1 - \bar{p}) + n(n - 1)\operatorname{Var}(P),$$
using $E[P^2] = \operatorname{Var}(P) + \bar{p}^2$. (The law of total variance follows from $\operatorname{Var}(X) = E[X^2] - (E[X])^2$ by writing $E[X^2] = E\big[E[X^2 \mid P]\big]$ and $E[X^2 \mid P] = \operatorname{Var}(X \mid P) + (E[X \mid P])^2$.) The extra term is positive whenever the clusters differ. Define $\rho = \operatorname{Var}(P)/[\bar{p}(1 - \bar{p})]$, the correlation between the outcomes of two people in the same cluster. The variance becomes
$$\operatorname{Var}(X) = n\bar{p}(1 - \bar{p})\,\big[1 + (n - 1)\rho\big].$$
The factor $1 + (n - 1)\rho$ is the design effect of sampling in clusters of size $n$.
The check on NHANES. For obesity across the 30 groups, $\bar{p} = 0.4073$, $\operatorname{Var}(P) = 0.006413$ (a standard deviation of 0.0801 between groups) and $\rho = 0.0266$. A correlation that small looks harmless, but it is multiplied by $n - 1$:
- For 10 adults from one location, the variance is $2.4140 + 90 \times 0.006413 = 2.9912$ instead of 2.4140, a design effect of 1.239.
- For 50 adults from one location, the design effect would be 2.302. The clustered sample would carry about as much information about the population as 22 adults drawn from it at random.
In the tails, no obese adults among the 10 has probability 0.0110, against 0.0054 for the binomial. Seven or more has probability 0.0838, against 0.0604. A simulation of 20,000 such samples for each of 30 seeds gave a variance with median 2.9927 (interquartile range 2.9769 to 3.0090), in agreement with the formula.
Check your understanding. Does sampling from one location change the mean number of obese adults among the 10?
No. By linearity, $E[X] = E\big[E[X \mid P]\big] = n\bar{p} = 4.073$. (It differs from 4.110 only because the 30 locations get equal weight.) Clustering leaves the center alone and widens the spread, which is what makes it easy to overlook.
For the three readings, the same formula with $n = 3$ and a ratio of 2.670 gives $\rho = (2.670 - 1)/2 = 0.835$. Readings of the same person are very strongly correlated, as they should be if they measure the same blood pressure.
The two departures act in opposite directions:
- Unequal probabilities among independent trials, fixed in advance, make counts less variable than the binomial.
- Probabilities that vary at random between clusters, and are shared within them, make counts more variable.
The second is far more common in practice and far more dangerous. A method that assumes binomial variance then understates its standard errors. Survey analyses such as those of NHANES use the design variables for this reason. Studies that randomize clinics or communities rather than people plan their sample size with the design effect. The Poisson distribution has an exact counterpart, overdispersion, treated in The Poisson Distribution.
Corrections to the course notes
The unit's notes give the binomial pmf, mean and variance, and the four conditions, as this article does. They state the condition for the normal approximation as $np > 10$ and $n(1-p) > 10$, while the course's formula sheet uses 5. Both thresholds are in use, and The Normal Approximation to the Binomial measures what each guarantees. The notes' caution about "misinterpreting independent trials" is made concrete in the last section: dependence leaves the mean $np$ unchanged and changes everything else about the distribution.
Summary
| Quantity | Formula |
|---|---|
| pmf and CDF | $p(x) = P(X = x)$; $F(x) = P(X \le x) = \sum_{k \le x} p(k)$ |
| Expected value | $E[X] = \sum_x x\,p(x)$; $E[aX + b] = aE[X] + b$; $E[X + Y] = E[X] + E[Y]$ |
| Variance | $\operatorname{Var}(X) = E[(X - \mu)^2] = E[X^2] - \mu^2$; $\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$ |
| Independent sums | $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$ |
| Bernoulli | mean $p$, variance $p(1 - p)$ |
| Binomial pmf | $P(X = x) = \binom{n}{x}p^x(1 - p)^{n - x}$, $x = 0, \ldots, n$ |
| Binomial mean, variance | $np$, $np(1 - p)$ |
| Mode | the whole number(s) $x$ with $(n + 1)p - 1 \le x \le (n + 1)p$ |
| Unequal $p_i$, independent | variance $n\bar{p}(1 - \bar{p}) - nV$ |
| Clusters of size $n$ | variance $n\bar{p}(1 - \bar{p})[1 + (n - 1)\rho]$ |
What we have learned
Expectations add whatever the dependence; variances add only for independent variables. Counting sequences of independent trials gives the binomial distribution, with mean $np$ and variance $np(1 - p)$: for 10 adults from NHANES, 4.110 obese adults on average, with a standard deviation of 1.5559. The mean survives any failure of the conditions; the variance does not. Unequal probabilities fixed in advance shrink it. Dependence inflates it: an adult's three readings vary 2.670 times as much as the binomial allows, and drawing from one survey location multiplies the variance by 1.239.
When $n$ is large and $p$ small, as for deaths in a large population, the binomial needs $n$ and $p$ separately, and often neither is known. What distribution do such counts follow? That is The Poisson Distribution.
In Python
from scipy import stats
X = stats.binom(10, 0.4110) # obese adults among 10
print(X.pmf(4)) # P(X = 4): about 0.2502
print(X.cdf(2), X.sf(6)) # P(X <= 2) about 0.1502; P(X >= 7) about 0.0634
print(X.mean(), X.var()) # 4.110, about 2.4208
sf(x) is the survival function $P(X > x)$, so $P(X \ge 7)$ is sf(6). Every number in this article is computed by tools/experiments/applied-statistics/binomial-distribution.py in the site's repository, from the NHANES files loaded by descdata.py and probdata.py.
References
- Grinstead, C. M., and Snell, J. L. (1997). Introduction to Probability (2nd rev. ed.), §3.2 (Bernoulli trials and binomial probabilities), §6.1 (expected value) and §6.2 (variance). American Mathematical Society. record · 10 pages
- National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: DEMO_J (including the masked variance units SDMVPSU and SDMVSTRA), BPX_J (three consecutive readings after five minutes' rest), BPQ_J, BMX_J, DIQ_J and SMQ_J data files and documentation. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 45 pages
- NIST/SEMATECH. e-Handbook of Statistical Methods, §1.3.6.6.18 (Binomial distribution). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/ record · 30 pages
- Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages