Of 1,696 young adults examined in a national health survey, 389 had hypertension. A random sample of 40 of them contained 10. How far from the truth could a share of 10 in 40 be? How would we test a claim about the true share, or compare it with another group's? This unit answers those questions for outcomes that are simply yes or no.
Many outcomes in medicine are of that kind: the patient has the condition or does not, recovered or did not, tested positive or negative. The quantity of interest is then a proportion, the share of a population with the outcome, and the data are counts. The earlier units built confidence intervals and tests for means of measurements. A proportion is also a mean, the mean of zeros and ones, so much of that machinery carries over. Two features of binary data change the details:
- The variance of a yes-or-no observation is fixed by its mean, $p(1-p)$. There is no separate spread to estimate, and a test of a stated proportion knows its standard error exactly.
- The count is a whole number. Its distribution is a lattice of separate probabilities, which a normal curve can only approximate.
The second feature has a consequence that surprises most people who meet it. The interval for a proportion found in nearly every textbook, $\hat{p} \pm z\sqrt{\hat{p}(1-\hat{p})/n}$, often covers the true proportion less often than its stated confidence level. Sometimes it covers much less, and not only in small samples. Because the count takes only $n + 1$ values, the coverage of any interval can be computed exactly rather than estimated by simulation. The unit uses that to compare the textbook interval with three alternatives:
- Wilson's score interval of 1927;
- the Agresti–Coull interval, built on Wilson's center;
- the Clopper–Pearson "exact" interval of 1934.
The same calculation measures the actual Type I error rates of the tests.
The unit takes four steps:
- how a sample proportion varies from sample to sample, and when a normal curve describes it well;
- intervals for one proportion, and why the textbook interval falls short;
- tests for one proportion, their power, and the sample size a study needs;
- the comparison of two proportions, including the chi-square test of a $2 \times 2$ table.
The examples use hypertension among adults examined in the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018. Hypertension is defined as the National Center for Health Statistics defined it for those years: a mean systolic pressure of at least 130 mm Hg, a mean diastolic pressure of at least 80 mm Hg, or current use of blood pressure medication. The 5,182 nonpregnant adults with blood pressure readings serve as a population with known proportions. These run from 14.6% among women aged 18 to 39 to 80.0% among adults aged 60 and over. The notation follows the course's formula sheets:
- $x$ events in a sample of $n$, and $\hat{p} = x/n$;
- a null value $p_0$ and an alternative $p_1$;
- for two groups, $\hat{p}_1$ and $\hat{p}_2$, and the pooled proportion $\hat{p}$.
Every number comes from a script in the site's repository.
Articles
- The Normal Approximation to the Binomial: how far can a sample proportion stray from the truth? The NHANES hypertension data and their variables. The sample proportion as a mean of Bernoulli variables, with its mean and standard error derived. Sampling without replacement and the finite population correction, checked by simulation. The normal approximation and the continuity correction, against exact binomial and hypergeometric probabilities. The skewness of the binomial derived, and what the rule $np \ge 5$ does and does not guarantee.
- Confidence Intervals for a Proportion: why does the textbook interval fall short, and what should replace it? The Wald interval and its coverage computed exactly, reproducing the published values of Brown, Cai and DasGupta (2001), and the two reasons it fails. Wilson's interval derived by inverting the test, and the Agresti–Coull interval. The Clopper–Pearson interval from the binomial tails, with the proof that it never undercovers. Coverage and length of all four across sample sizes and proportions, and the rule of three for samples with no events.
- Tests and Sample Size for a Proportion: does a proportion differ from a stated value, and how many observations does it take to tell? The one-sample $z$ test and why its standard error uses $p_0$. Its exact correspondence with the Wilson interval. The exact binomial test, and the actual Type I error rates of both, computed exactly. Power and the sample size formula derived, with the sawtooth that discreteness puts into exact power, and the sample size for estimating a proportion to a given margin of error.
- Comparing Two Proportions: do two groups differ, and by how much? The pooled two-proportion $z$ test, and the interval for the difference with the unpooled standard error. The chi-square test of a $2 \times 2$ table, with the proof that it is the square of the $z$ test. Yates' correction and Fisher's exact test. Risk difference, relative risk and odds ratio, and the sample size per group, checked by exact power and simulation.
Related topics
This is the course's last unit, and it draws on most of the earlier ones:
- it applies the reasoning of Confidence Intervals and Hypothesis Testing to counts;
- its sample size formulas parallel those for means in Power and Sample Size for a Test of a Mean and Sample Size for Comparing Two Means;
- it rests on the binomial distribution, the normal distribution and the central limit theorem;
- the sign test in Nonparametric Methods is its exact binomial test with $p_0 = 1/2$.
Relative risks and odds ratios, introduced in Relative Risk and the Odds Ratio, are estimated with adjustment for other variables by the library's Logistic Regression. Sensitivity, specificity and predictive values are proportions too; they are in Screening and Diagnostic Tests. The library's notes on error bars for evals apply the standard error of a proportion to language model benchmarks.
References
- Brown, L. D., Cai, T. T., and DasGupta, A. (2001). Interval estimation for a binomial proportion. Statistical Science, 16(2), 101–133. record · 4 pages
- Clopper, C. J., and Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4), 404–413. record · 2 pages
- NIST/SEMATECH. e-Handbook of Statistical Methods, §7.2.4 (Does the proportion of defectives meet requirements?) and §7.3.3 (How can we determine whether two processes produce the same proportion of defectives?). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/ record · 30 pages
- Ostchega, Y., Fryar, C. D., Nwankwo, T., and Nguyen, D. T. (2020). Hypertension Prevalence Among Adults Aged 18 and Over: United States, 2017–2018. NCHS Data Brief No. 364. National Center for Health Statistics. record · 6 pages
- Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages
- Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158), 209–212. record · 3 pages