Categorical Data Analysis

Tests and measures for counts in categories, beyond the single 2 x 2 table of the course. Six articles, on goodness of fit, independence in tables of any size, trends across ordered groups, paired proportions, stratified tables and agreement between raters, with the chi-squared limits proved by projection, the exact conditional tests derived, and the running examples computed on NHANES 2017-2018 adults, together with Bortkiewicz's horse-kick deaths, Breslow and Day's published strata, a published small study of paired proportions and this site's relevance grades.

The applied statistics course on this site ends with counts of a yes-or-no outcome in two groups: Comparing Two Proportions tests whether two proportions differ, shows that the chi-square test of a $2 \times 2$ table is the same test, and gives Fisher's exact test for small samples. Much categorical data does not fit that frame. An answer may fall in five categories rather than two, a table may have several rows and columns, the groups may have a natural order, the two classifications may be made on the same person, a comparison may need to be made within strata of a third variable, or the question may be how well two raters agree rather than whether two groups differ. This unit treats those cases.

Three ideas run through it. The first is Pearson's statistic, the sum over cells of squared differences between observed and expected counts divided by the expected counts, together with its companion the likelihood ratio statistic. The second is the counting of directions: the deviations of the counts from their expected values are confined by every total that is fixed or fitted, and the degrees of freedom of the chi-squared limit are the dimension of the space that remains. The third is conditioning on margins, which removes unknown parameters from the null distribution, makes exact tests possible and connects the different ways a table can be sampled.

What the unit assumes

The chi-squared distribution and its properties from The Chi-Squared and t Distributions; the projection theorem and the noncentral chi-squared distribution from Sampling Distributions from the Normal; convergence in distribution, Slutsky's theorem and the central limit theorem from Convergence in Distribution and the Proof of the Central Limit Theorem; the score and likelihood ratio statistics from Likelihood Ratio, Wald and Score Tests; and the Kullback–Leibler divergence and mutual information from the Information Theory unit, which supply the likelihood ratio statistic's other reading.

What is proved and what is cited

Proved in the unit: the moments of multinomial counts; Pearson's statistic as a Mahalanobis distance and as a score statistic; its chi-squared limit with $k - 1$ degrees of freedom for any number of categories, and $(r - 1)(c - 1)$ for an $r \times c$ table, by projections; the vanishing difference between Pearson's and the likelihood ratio statistic; the noncentral limit that gives the power; the loss of a degree of freedom for one parameter estimated from the counts; the conditional distributions of a table given its margins and the exact tests built on them; the exact conditional variance of a cell; the Cochran–Armitage statistic as a regression slope, a score statistic and a correlation, and the exact partition of the chi-square statistic into trend and departure; and the results on paired, stratified and agreement data listed with their articles below.

Cited: the Cramér–Wold device, the Lindeberg–Feller theorem and the continuity theorem for characteristic functions (Durrett, 2019), through the articles that use them; the several-parameter form of Wilks's theorem; the behavior of the goodness-of-fit test when parameters are estimated from ungrouped data (Chernoff and Lehmann, 1954). Assumed, neither proved nor cited: the several-parameter linear representation of the maximum likelihood estimate, on which the degrees-of-freedom count for $s \ge 2$ estimated parameters rests (the site proves it for one parameter).

Most running examples use the NHANES 2017–2018 adults of the course, treated as a finite population from which samples are drawn with stated seeds. Self-rated health by age group comes from the 5,242 adults with a blood pressure reading and an answer on general health. Diabetes, smoking, age group and body mass index come from the course's risk-factor population: the 5,107 nonpregnant adults with blood pressure readings, a body mass index, an answer on diagnosed diabetes (borderline counted as no) and a known smoking status. The paired articles use the repeated blood pressure readings. Other examples are Bortkiewicz's horse-kick deaths (goodness of fit), Breslow and Day's published strata (Mantel–Haenszel), a published small study (McNemar's test) and this site's relevance grades (agreement). Every number is computed and checked against its page by a script in tools/experiments/statistics/.

Notation

Pearson's statistic is $X^2$ and the likelihood ratio statistic $G^2 = 2\sum O\ln(O/E)$, with observed counts $O$ and expected counts $E$. "Chi-square test" names the test and "chi-squared distribution" the distribution, written $\chi^2_\nu$, with upper point $\chi^2_{\nu, 1-\alpha}$. A sample of $n$ observations in $k$ categories has counts $O_j$ and category probabilities $\pi_j$, with null values $\pi_{0j}$. An $r \times c$ table has counts $n_{ij}$, row and column totals $n_{i+}$ and $n_{+j}$, and estimated expected counts $E_{ij} = n_{i+}n_{+j}/n$. A $2 \times 2$ table written in the course's form keeps its cells $a, b, c, d$, and strata are indexed by $k$. Group proportions compared across groups keep the course's $p_i$, $\hat p_i$ and pooled $\hat p$. Cohen's coefficient of agreement is written $\kappa$ in its own article only; elsewhere in the collection $\kappa$ is the kurtosis.

Articles

  1. The Chi-Square Goodness-of-Fit Test: whether counts in $k$ categories agree with proposed probabilities. Proves the covariance of multinomial counts, that Pearson's statistic is a Mahalanobis distance and the score statistic, its $\chi^2_{k-1}$ limit by a projection, the agreement of the likelihood ratio statistic, the noncentral limit for power, and the loss of one degree of freedom for a parameter estimated from the counts.
  2. The Chi-Square Test of Independence: tables of any size. Connects the three sampling designs by conditioning, derives the expected counts as maximum likelihood estimates, proves Fisher's $(r - 1)(c - 1)$ degrees of freedom as the rank of a product of two projections, and derives the exact conditional variance of a cell behind the adjusted residuals, the permutation test for small counts and the bound behind Cramér's $V$.
  3. Trends in Proportions: the Cochran–Armitage Test: a proportion that rises or falls across ordered groups. Derives the one-degree-of-freedom statistic as a weighted regression slope, as the score statistic of every binary regression model in the scores, and as $n$ times a squared correlation; proves its normal limit, its exact conditional form and the exact partition of the chi-square statistic into trend and departure from linearity.
  4. McNemar's Test for Paired Proportions: two classifications of the same individuals. Why only the discordant pairs carry information about the difference of the two proportions, the exact binomial form of the test, and the variance of a difference of dependent proportions.
  5. The Mantel–Haenszel Test: an association within strata of a third variable. The theory beneath the test and the estimator that the course page uses: the test as a conditional score test, its limit with many small strata, and why the Mantel–Haenszel estimator is consistent.
  6. Measures of Agreement: Cohen's Kappa: agreement between two raters beyond what chance produces, its standard error, weighted versions and its paradoxes.

Where the unit leads

The likelihood ratio statistic for a table is a difference of deviances, and models for counts with several explanatory variables, logistic and log-linear regression, are the subject of the planned Generalized Linear Models unit. The information retrieval course already uses the chi-square statistic of a $2 \times 2$ table to select features (Feature Selection), and agreement between relevance assessors is the subject of Relevance Judgments.

References