Theoretical Probability Distributions

Random variables and the three distributions that describe much of the data in medicine. Probability mass and density functions, distribution functions, expectation and variance; the binomial for successes in a fixed number of independent trials, the Poisson for counts of rare events and events in time, and the normal for measurements; each derived and proved, and each tested on real data, including what happens when trials are clustered, when a rate varies, and when measurements are skewed. Worked on NHANES adults, Bortkiewicz's horse-kick deaths and NCHS weekly death counts.

Draw 10 adults at random from a survey population in which 41% are obese, and count the obese among them. Count the deaths in a state in one week. Measure one man's height. Each result is a number that depends on chance. How likely is each possible value? The rules of probability alone cannot say.

A random variable is a number whose value depends on the outcome of a random experiment. Its distribution gives the probability of each value, or of each range of values. This unit builds on Probability and adds three theoretical distributions. Each is derived from a short list of assumptions, and together they describe a large share of the data in medicine and public health.

  • The binomial distribution counts successes in a fixed number of independent trials that share one probability of success: patients who respond to a treatment, sampled adults who are obese.
  • The Poisson distribution counts events that are individually rare in a large population, or that occur at random in time at a constant rate: deaths from a cause in a year, cases of a disease in a week.
  • The normal distribution describes measurements that vary symmetrically around a mean. More important for the rest of the course, it describes the means of samples.

The three are linked. The Poisson is the limit of the binomial when trials are many and successes rare, and both approach the normal as their counts grow.

The unit runs in three steps:

  1. First, the language of random variables, expected values and variances, and the binomial distribution derived by counting sequences of trials.
  2. Then, the Poisson distribution as a limit of the binomial and as a model of events in time, and how to tell when real counts vary more than it allows.
  3. Finally, continuous random variables and the normal curve, and how to tell whether a set of measurements is close enough to normal.

Each article proves its distribution's mean and variance and tests it on real data, because the assumptions are where models fail in practice. The running examples are:

  • adults of the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018, with their obesity, their three blood pressure readings and their heights;
  • Bortkiewicz's counts of Prussian soldiers killed by horse kicks;
  • the weekly death counts that the National Center for Health Statistics (NCHS) publishes for every state.

Every number comes from a script in the repository.

Articles

  1. The Binomial Distribution: how likely is each count of successes among a fixed number of trials, and what happens when the trials are not independent? Random variables, probability mass functions and distribution functions. Expected value and variance, with linearity of expectation and the additivity of variances for independent variables proved. Bernoulli trials, and the binomial probability derived by counting sequences. The mean $np$ and variance $np(1-p)$ proved two ways, and the mode, symmetry and skewness. Then the conditions failing: sampling without replacement; unequal probabilities, which shrink the variance; and dependence and clustering, which inflate it. The design effect $1 + (n-1)\rho$ is derived from the law of total variance and measured on repeated blood pressure readings and survey locations in NHANES.
  2. The Poisson Distribution: what distribution do counts of rare events follow, and how can we tell when real counts vary more than it allows? The Poisson probability as the limit of the binomial, proved, with the speed of convergence measured. Mean and variance both equal to $\lambda$, and sums of independent Poisson counts. The Poisson process, time scaling and exponential waiting times. Bortkiewicz's horse-kick deaths and Fisher's index of dispersion. Overdispersion in NCHS weekly death counts for 51 states and other areas: cancer deaths behave as Poisson counts, while deaths from all causes, with their winter peaks, vary nearly four times as much as the model allows in Iowa. A varying rate explains the excess; the negative binomial models it. Last, the normal approximation and its unequal tails.
  3. The Normal Distribution: when do two numbers, a mean and a standard deviation, describe a whole set of measurements, and how can we tell? Continuous random variables, densities and distribution functions. The normal density, with proofs that it integrates to 1 and has mean $\mu$ and variance $\sigma^2$. Standardization, tables and software, and the 68–95–99.7 rule. Checking normality with histograms, normal probability plots and the Shapiro–Wilk test, on men's heights, which follow the curve closely, and systolic blood pressure, which does not. Log, reciprocal and Box–Cox transformations of blood pressure. Why the normal curve arises, with a forward link to the central limit theorem. Later units build on this article whenever they assume normal data or a normal sampling distribution.

This unit builds on earlier ones and leads into the rest of the course.

References

  • Bortkiewicz, L. von (1898). Das Gesetz der kleinen Zahlen. B. G. Teubner, Leipzig. record · 2 pages
  • Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh (2nd ed., 1928). record · 6 pages
  • Grinstead, C. M., and Snell, J. L. (1997). Introduction to Probability (2nd rev. ed.). American Mathematical Society. record · 10 pages
  • National Center for Health Statistics (2021). Weekly Counts of Deaths by State and Select Causes, 2014–2019. data.cdc.gov dataset 3yf8-kanr. record · 5 pages
  • NIST/SEMATECH. e-Handbook of Statistical Methods, §1.3.6.6 (Gallery of distributions). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/ record · 30 pages
  • Pagano, M., Gauvreau, K., and Mattie, H. (2022). Principles of Biostatistics (3rd ed.). CRC Press. The course text. record · 35 pages