Matching and Propensity Scores

How one number, the probability of treatment given the covariates, can stand in for all of them when estimating a causal effect from observational data. The balancing property of the propensity score and the inheritance of unconfoundedness are proved (Rosenbaum and Rubin, 1983), with the characterization of every balancing score; inverse-probability weighting is proved unbiased with the known score and connected to the Horvitz–Thompson estimator, its variance is derived, and an example shows it can be infinite under positivity; with a discrete covariate the estimated score turns weighting into subclassification exactly, and the estimated score is shown to beat the true one in a single cell; weighting for the effect on the treated, exact matching on the score and the double robustness of the augmented estimator are proved. Former and never smokers among NHANES adults are compared with eighteen-term score and outcome models, with balance checks, bootstrap intervals and an E-value; simulations with a known answer compare eight estimators when each working model is right or wrong, and under poor overlap with trimming. Worked problems reweight controls to look like the treated, price the true score in one cell, take a doubly robust estimator apart, and construct an unbiased estimator with infinite variance.

Former smokers among 4,200 adults in the National Health and Nutrition Examination Survey (NHANES) of 2017 to 2018 report diagnosed diabetes more often than adults who never smoked, 0.229 against 0.131. Confounding and Causal Diagrams adjusted for age and sex by standardization, using six strata, and the risk ratio fell from 1.76 to 1.13. Race and Hispanic origin, education and income also differ between the groups and may affect diabetes. Crossing all five covariates in broad categories gives 900 strata, and the adults fill them so thinly that 136 strata hold a single adult. How can the adjustment be made when the strata outnumber the data?

The propensity score is the probability of treatment given the covariates, $e(x) = P(W = 1 \mid X = x)$. Rosenbaum and Rubin (1983) proved that this single number does the work of all the covariates: units with the same score have the same distribution of covariates whether treated or not, and if comparing like with like on the covariates removes confounding, comparing like with like on the score does too. The score can then be used to weight, to stratify or to match. This article proves those facts, derives what each method estimates and how its precision depends on the score, and applies them to the smoking data.

First the setting and its two assumptions. Then the balancing property and its consequences (Theorems 1 to 3). Then weighting by the inverse of the score, its link to survey sampling, its variance, and why the estimated score is better than the true one. Then subclassification and matching. Then the doubly robust estimator, which combines a model for the score with a model for the outcome. Then overlap, the NHANES analysis, and simulations in which the true effect is known. The article assumes Potential Outcomes and the Average Treatment Effect and the adjustment formula of Confounding and Causal Diagrams (its Theorem 5), and uses logistic regression to estimate scores.

Too many strata

The covariates are age, sex, race and Hispanic origin (six groups), education (five levels) and the ratio of family income to the poverty line (five levels: under 1, 1 to 2, 2 to 4, 4 and over, and not reported, which applies to 527 adults). NHANES records adults' education (variable DMDEDUC2) from age 20; for the 231 adults aged 18 or 19 it is in the youth variable DMDEDUC3, whose codes are mapped to the same five levels (grades up to 8 and the codes for less than 5th and less than 9th grade to less than 9th grade, grades 9 to 12 without a diploma to 9th to 11th grade, a diploma or GED to high school, more than high school to some college). The 5 adults aged 20 or over who refused or did not know are counted with the high-school level. In three age groups the full cross-classification has $3 \times 2 \times 6 \times 5 \times 5 = 900$ cells. The 4,200 adults occupy 704 of them, 136 cells hold a single adult, and only 353 cells contain both a former and a never smoker. Those 353 cells hold a share 0.766 of the adults, so standardization over all five covariates would discard almost a quarter of the data, and finer categories would discard more.

The difficulty is the curse of dimensionality of exact adjustment: the number of cells grows as the product of the numbers of levels. A model for the outcome in terms of all covariates is one way through it. The propensity score is another, which models the treatment instead.

The setting

Units are independent copies of $(X, Y(1), Y(0), W)$, with treatment $W \in \{0, 1\}$, potential outcomes $Y(1), Y(0)$, observed outcome $Y = W\,Y(1) + (1 - W)\,Y(0)$ and pre-treatment covariates $X$. The targets are the average treatment effect $\tau = E[Y(1) - Y(0)]$ and the average effect on the treated $\tau_{\mathrm t} = E[Y(1) - Y(0) \mid W = 1]$. Two assumptions are made throughout.

  1. Unconfoundedness: $\{Y(1), Y(0)\} \perp\!\!\!\perp W \mid X$.
  2. Positivity (or overlap): $0 < e(X) < 1$ with probability 1.

Together they are what Rosenbaum and Rubin (1983) call strongly ignorable treatment assignment (in modern usage the two are often stated separately; Ding, 2024, §11.2.1). Neither depends on the outcome model, and unconfoundedness is not testable from the data; Confounding and Causal Diagrams shows how a causal diagram argues for it through the backdoor criterion. Under the two assumptions its Theorem 5 identifies $E[Y(w)] = E[\mu_w(X)]$, with $\mu_w(x) = E[Y \mid W = w, X = x]$. Everything below is a different way to compute or estimate that quantity.

The propensity score balances the covariates

Here is the idea in one sentence: once the probability of treatment is fixed, knowing the covariates tells nothing more about who was treated, so the treated and untreated with that probability are alike in their covariates.

Theorem 1 (balancing; Rosenbaum and Rubin, 1983, Theorem 1). $W \perp\!\!\!\perp X \mid e(X)$.

Proof. $W$ is binary, so it suffices to show $P(W = 1 \mid X, e(X)) = P(W = 1 \mid e(X))$. The left side is $P(W = 1 \mid X) = e(X)$, since $e(X)$ is a function of $X$. For the right side use the tower property and then the definition of $e$:

$$P(W = 1 \mid e(X)) = E\big[P(W = 1 \mid X) \mid e(X)\big] = E\big[e(X) \mid e(X)\big] = e(X).$$

The two sides agree. $\square$

The theorem uses no assumption about outcomes: it is a property of the score and the covariates alone (Ding, 2024, §11.3, makes the same point). It is also checkable, since both $W$ and $X$ are observed, and that is the basis of the balance checks in the NHANES section.

A balancing score is any function $b(X)$ with $W \perp\!\!\!\perp X \mid b(X)$. The covariates themselves are one, the score is another, and the next theorem says exactly which functions are.

Theorem 2 (every balancing score; Rosenbaum and Rubin, 1983, Theorem 2). A function $b(X)$ is a balancing score if and only if $e(X) = f(b(X))$ for some function $f$; that is, $b$ is at least as fine as $e$.

Proof. If $e(X) = f(b(X))$, then by the tower property

$$P(W = 1 \mid b(X)) = E[e(X) \mid b(X)] = E[f(b(X)) \mid b(X)] = f(b(X)) = e(X) = P(W = 1 \mid X, b(X)),$$

so $W$ is independent of $X$ given $b(X)$. Conversely, if $b$ is a balancing score, then $P(W = 1 \mid X, b(X)) = P(W = 1 \mid b(X))$. The left side is $e(X)$. The right side is a function of $b(X)$; call it $f(b(X))$. Then $e(X) = f(b(X))$. $\square$

So the score is the coarsest balancing score: any summary of the covariates that balances them carries at least the information in $e(X)$. Rosenbaum and Rubin (1983, Theorem 2) prove both directions; so does Ding (2024, Theorem 11.5).

Theorem 3 (unconfoundedness passes to the score; Rosenbaum and Rubin, 1983, Theorem 3). If $\{Y(1), Y(0)\} \perp\!\!\!\perp W \mid X$, then $\{Y(1), Y(0)\} \perp\!\!\!\perp W \mid e(X)$. Positivity given $X$ holds if and only if $0 < e(X) < 1$, which is positivity given $e(X)$.

Proof. Write $\mathbf Y = \{Y(1), Y(0)\}$. It suffices to show $P(W = 1 \mid \mathbf Y, e(X)) = P(W = 1 \mid e(X))$. By the tower property, conditioning first on the finer information $(\mathbf Y, X)$,

$$P(W = 1 \mid \mathbf Y, e(X)) = E\big[P(W = 1 \mid \mathbf Y, X) \mid \mathbf Y, e(X)\big] = E\big[e(X) \mid \mathbf Y, e(X)\big] = e(X),$$

where the middle step uses unconfoundedness, $P(W = 1 \mid \mathbf Y, X) = P(W = 1 \mid X) = e(X)$. By Theorem 1's computation $P(W = 1 \mid e(X)) = e(X)$ as well. For positivity, $P(W = 1 \mid e(X)) = e(X)$, so the treatment probability given the score lies strictly between 0 and 1 exactly when $e(X)$ does. $\square$

By Theorem 2 the same argument works for any balancing score: the proof used only $P(W = 1 \mid b(X)) = e(X)$. Hernán and Robins (2020, Technical Point 15.1) add a graphical proof: the score is a node between $X$ and $W$ that intercepts every backdoor path.

Corollary 1 (comparisons at a fixed score; Rosenbaum and Rubin, 1983, Theorem 4). Under strong ignorability, for every value $s$ taken by $e(X)$,

$$E[Y \mid W = 1, e(X) = s] - E[Y \mid W = 0, e(X) = s] = E[Y(1) - Y(0) \mid e(X) = s].$$

Proof. By consistency $E[Y \mid W = 1, e(X) = s] = E[Y(1) \mid W = 1, e(X) = s]$, and by Theorem 3 this equals $E[Y(1) \mid e(X) = s]$. The same steps apply to the untreated. Positivity makes both conditioning events possible. $\square$

Averaging the corollary over the distribution of $e(X)$ gives $\tau$; averaging it over the distribution of $e(X)$ among the treated gives $\tau_{\mathrm t}$. Subclassification and matching estimate these averages; weighting reaches the same targets without forming groups at all.

Weighting by the inverse of the score

Here is the idea in one sentence: a treated unit with score 0.25 stands for itself and three untreated units like it, so weighting it by $1/0.25 = 4$ rebuilds what the whole population would have looked like had everyone been treated.

Theorem 4 (inverse-probability weighting; Ding, 2024, Theorem 11.2). Under strong ignorability,

$$E\!\left[\frac{W Y}{e(X)}\right] = E[Y(1)], \qquad E\!\left[\frac{(1 - W) Y}{1 - e(X)}\right] = E[Y(0)].$$

Consequently, with the true score known, the estimator

$$\hat\tau_{\mathrm{HT}} = \frac1n\sum_{i=1}^n \frac{W_iY_i}{e(X_i)} - \frac1n\sum_{i=1}^n \frac{(1 - W_i)Y_i}{1 - e(X_i)}$$

is unbiased for $\tau$.

Proof. On $W = 1$, consistency gives $WY = WY(1)$. Condition on $X$ and use unconfoundedness to split the conditional expectation of the product:

$$E\!\left[\frac{W Y(1)}{e(X)}\right] = E\!\left[\frac{E[W \mid X]\,E[Y(1) \mid X]}{e(X)}\right] = E\big[E[Y(1) \mid X]\big] = E[Y(1)],$$

using $E[W \mid X] = e(X) > 0$. The second identity follows in the same way with $1 - W$ and $1 - e(X)$. Each term of $\hat\tau_{\mathrm{HT}}$ is an average of $n$ identically distributed terms, so its expectation is the expectation of one of them. $\square$

The same identity as the Horvitz–Thompson estimator. Condition on the $n$ units' covariates and potential outcomes. By unconfoundedness and independence across units, the treatments are then independent with $P(W_i = 1) = e(X_i)$: the treated form a Poisson sample from the $n$ units with inclusion probabilities $\pi_i = e(X_i)$. The first sum of $\hat\tau_{\mathrm{HT}}$, times $n$, is $\sum_{i:\,W_i = 1} Y_i(1)/\pi_i$, the Horvitz–Thompson estimator of the total $\sum_i Y_i(1)$. Survey Weights and the Horvitz–Thompson Estimator proves (its Theorem 1) that this estimator is unbiased for the total under any design with every $\pi_i > 0$. So $\hat\tau_{\mathrm{HT}}$ is unbiased even conditionally on the units, for the average effect among them, $\frac1n\sum_i[Y_i(1) - Y_i(0)]$. Horvitz and Thompson (1952) introduced the estimator for sampling; Ding (2024, §11.2.2) notes that it is the same estimator.

The estimator has a defect that survey statisticians also met. Adding a constant $c$ to every outcome changes it by $c$ times $\frac1n\sum_i W_i/e(X_i) - \frac1n\sum_i (1 - W_i)/(1 - e(X_i))$, a difference of two estimates of 1 that is not 0 in any particular sample (Ding, 2024, Proposition 11.1). Dividing each sum by its own sum of weights removes the defect:

$$\hat\tau_{\mathrm{H\acute ajek}} = \frac{\sum_i W_iY_i/e(X_i)}{\sum_i W_i/e(X_i)} - \frac{\sum_i (1 - W_i)Y_i/(1 - e(X_i))}{\sum_i (1 - W_i)/(1 - e(X_i))}.$$

This is the Hájek estimator, the ratio form that Survey Weights and the Horvitz–Thompson Estimator treats as its Theorem 4. It is consistent but not exactly unbiased, and Ding (2024, §11.2.2) reports that numerical studies find it more stable. It is the weighting estimator used below unless the Horvitz–Thompson form is named.

Proposition 1 (weights for the effect on the treated). Under $Y(0) \perp\!\!\!\perp W \mid X$ and $e(X) < 1$,

$$E[Y(0) \mid W = 1] = \frac{E\big[(1 - W)\,Y\,e(X)/(1 - e(X))\big]}{E\big[(1 - W)\,e(X)/(1 - e(X))\big]}.$$

Proof. By consistency and then unconfoundedness, $E[(1 - W)Y\,e(X)/(1 - e(X)) \mid X] = e(X)\,E[Y(0) \mid X]$, since $E[1 - W \mid X] = 1 - e(X)$. Unconfoundedness also gives $e(X)E[Y(0) \mid X] = E[W \mid X]\,E[Y(0) \mid X] = E[W\,Y(0) \mid X]$, so the numerator is $E[W\,Y(0)] = P(W = 1)\,E[Y(0) \mid W = 1]$. The same computation with $Y$ replaced by 1 gives the denominator $E[W] = P(W = 1)$. Divide. $\square$

So the treated keep weight 1 and each untreated unit gets weight $e(X)/(1 - e(X))$, the odds of treatment: untreated units that look like the treated count most. Problem 1 carries this out on a three-point covariate.

How variable is weighting? The terms of $\hat\tau_{\mathrm{HT}}$ have a simple second moment.

Proposition 2 (variance with the known score). Under strong ignorability, with $E[Y(1)^2] < \infty$,

$$\operatorname{Var}\!\left(\frac{W Y}{e(X)}\right) = E\!\left[\frac{E[Y(1)^2 \mid X]}{e(X)}\right] - \big(E[Y(1)]\big)^2,$$

and the first sum of $\hat\tau_{\mathrm{HT}}$ has $1/n$ times this variance. Both sides may be $+\infty$: the second moment is the expectation of a nonnegative variable, and the mean is finite by Theorem 4.

Proof. $W^2 = W$, so $(WY/e(X))^2 = W\,Y(1)^2/e(X)^2$. Conditioning on $X$ and using unconfoundedness, $E[W\,Y(1)^2 \mid X] = e(X)\,E[Y(1)^2 \mid X]$, which divided by $e(X)^2$ gives the first term. Subtract the squared mean from Theorem 4. The terms are independent across units. $\square$

Two consequences follow. First, units with small $e(X)$ enter with weight $1/e(X)$, so the variance is governed by how close the score comes to 0 (and, for the second sum, to 1); the section on overlap returns to this. Second, the variance contains $E[Y(1)^2 \mid X]$, not the conditional variance of $Y(1)$, so the level of the outcome inflates it; this is the location defect again, which the Hájek form removes.

Estimated versus true scores

In an experiment the score is known; in an observational study it is estimated, usually by logistic regression of $W$ on $X$. It would seem that using the estimate costs precision. It does the opposite. Here is the idea in one sentence: the estimated score corrects the chance imbalance between treated and untreated that a particular sample happens to have, while the true score corrects only the imbalance expected on average.

Proposition 3 (a discrete covariate). Let $X$ take finitely many values, and estimate the score in each cell by the share treated, $\hat e(x) = n_{1x}/n_x$, where $n_x$ units have $X = x$ and $n_{1x}$ of them are treated, with $0 < n_{1x} < n_x$ in every cell. Then the Horvitz–Thompson and Hájek forms coincide and equal the subclassification estimator:

$$\frac1n\sum_i \frac{W_iY_i}{\hat e(X_i)} = \sum_x \frac{n_x}{n}\,\bar Y_{1x},$$

where $\bar Y_{1x}$ is the mean outcome of the treated in cell $x$, and the same holds for the untreated.

Proof. Group the sum by cell. In cell $x$ every treated unit has weight $1/\hat e(x) = n_x/n_{1x}$, so the cell contributes $\frac1n \cdot \frac{n_x}{n_{1x}}\sum_{i:\,X_i = x,\,W_i = 1} Y_i = \frac{n_x}{n}\bar Y_{1x}$. The sum of weights $\frac1n\sum_i W_i/\hat e(X_i)$ is $\sum_x (n_x/n) = 1$, so the Hájek form is the same. $\square$

With the estimated score, weighting is exactly the g-formula with the sample's own covariate distribution as the standard. With the true score it is not: the cell contributes $\frac{1}{n\,e(x)}\sum Y_i$ over the treated, whose expectation is right but whose value moves with the random number treated.

Proposition 4 (one cell). Let $X$ be constant, so that $e(X) = e$, and let $Y(1)$ have mean $\mu$ and variance $\sigma^2$. With the true score, $n\operatorname{Var}\big(\frac1n\sum_i W_iY_i/e\big) = (\sigma^2 + \mu^2)/e - \mu^2$. With the estimated score the estimator is the mean of the treated, whose variance given $n_1 \ge 1$ treated is $\sigma^2/n_1$; $n$ times this conditional variance tends to $\sigma^2/e$ with probability 1, since $n_1/n \to e$. The true score costs

$$\frac{\sigma^2 + \mu^2}{e} - \mu^2 - \frac{\sigma^2}{e} = \mu^2\,\frac{1 - e}{e} \ \ge\ 0.$$

Proof. The first variance is Proposition 2 with $E[Y(1)^2] = \sigma^2 + \mu^2$. Given $n_1$, the treated outcomes are $n_1$ independent draws of $Y(1)$, by unconfoundedness, so their mean has variance $\sigma^2/n_1$; and $n_1/n \to e$ with probability 1 by the strong law of large numbers, so $n\sigma^2/n_1 \to \sigma^2/e$. The difference is algebra. $\square$

The gap vanishes only when $\mu = 0$ or $e = 1$. Rosenbaum (1987) explains that estimated scores beat true ones for almost the same reason that direct adjustment can outperform the sample mean in a simple random sample, and Hirano, Imbens and Ridder (2003) prove that weighting by a nonparametric estimate of the score, rather than the true score, yields an efficient estimator of the average treatment effect (both from their abstracts). Hernán and Robins (2020, §15.2) put it briefly: in a randomized experiment the estimated score adjusts for random imbalances in the covariates that the true score ignores. The simulations below measure the gain: with a correctly specified score model the standard deviation over 30 data sets was 0.154 with the estimated score and 0.225 with the true one.

Subclassification and matching on the score

Subclassification. Corollary 1 suggests grouping units by their score and comparing treated and untreated within groups. Rosenbaum and Rubin (1984) did this with five groups of the estimated score, on observational data on treatments for coronary artery disease, and found that the five groups balanced 74 covariates (their abstract); Rosenbaum and Rubin (1983, p. 52) describe five subclasses of equal size at the quintiles of the estimated score for the same data. They cite Cochran's (1968) observation that five subclasses suffice "to remove at least 90% of the bias for many continuous distributions" as the reason for using five. The estimate is

$$\hat\tau_{\mathrm{sub}} = \sum_{j=1}^{5} \frac{n_j}{n}\big(\bar Y_{1j} - \bar Y_{0j}\big),$$

with $n_j$ units in group $j$. It is Proposition 3 with the groups as cells, and it leaves the bias that comes from treating units with different scores as equal. In the simulations below, with a correct score model, quintile subclassification left a bias of about 0.24 on an effect of 1, the largest of any method with a correct model. More groups reduce that bias at a cost in variance; Lunceford and Davidian (2004) compare subclassification with weighting in detail.

Matching. For the effect on the treated, pair each treated unit with the untreated unit whose score is closest, and compare their outcomes. Exact matching on the score is conditionally unbiased, by the next result.

Proposition 5 (exact matching). Assume $Y(0) \perp\!\!\!\perp W \mid X$ and positivity, and write $\tau_{\mathrm t}(s) = E[Y(1) - Y(0) \mid W = 1, e(X) = s]$. Suppose each treated unit $i$ is matched to an untreated unit $j(i)$ with $e(X_{j(i)}) = e(X_i)$, chosen using only the units' scores and treatments, with ties broken by an independent randomization. Then, given the scores and treatments of all units and that every treated unit has a match, the mean of $Y_i - Y_{j(i)}$ over the $n_1 \ge 1$ treated units has expectation $\frac{1}{n_1}\sum_{i:\,W_i = 1}\tau_{\mathrm t}(e(X_i))$. Given only the treatments, with $n_1 \ge 1$, the treated units' scores are independent draws from the distribution of $e(X)$ among the treated, so this sample quantity has expectation $\tau_{\mathrm t}$. The estimator is defined only when every treated unit has a match, an event that depends on the scores; conditioning on it can shift the treated units' scores, so the estimator is conditionally unbiased for the sample quantity rather than exactly unbiased for $\tau_{\mathrm t}$. When the score takes finitely many values, the probability of the event tends to 1 as $n$ grows.

Proof. Condition on the scores and treatments of all units and on the tie-breaking randomization; the matching $j(\cdot)$ is then fixed. Units are independent, so given these, unit $j$'s outcome depends only on its own score and treatment: its conditional mean is $E[Y(0) \mid W = 0, e(X) = s]$, where $s$ is its score, and it does not depend on which unit chose it. The step needs the selection to use no covariate beyond the score: Theorem 3 gives $Y(0) \perp\!\!\!\perp W \mid e(X)$, not independence of $Y(0)$ and $X$ given $e(X)$, so a rule that picked, among tied untreated units, the one with the largest value of some covariate could select on $Y(0)$. By Theorem 3 (applied to $Y(0)$ alone, which the same proof covers), $E[Y(0) \mid W = 0, e(X) = s] = E[Y(0) \mid e(X) = s] = E[Y(0) \mid W = 1, e(X) = s]$. The treated unit's outcome has conditional mean $E[Y(1) \mid W = 1, e(X) = s]$. Subtracting gives $\tau_{\mathrm t}(e(X_i))$ for each treated unit; averaging gives the first claim. Given the treatment vector, the units are independent and each treated unit's covariates have the distribution of $X$ given $W = 1$, so each $\tau_{\mathrm t}(e(X_i))$ has expectation $E[\tau_{\mathrm t}(e(X)) \mid W = 1] = \tau_{\mathrm t}$ by the tower property, and so does their average. Finally, if the score takes finitely many values $s$, positivity gives $q_s = P(W = 0, e(X) = s) > 0$ for each, and the probability that no untreated unit has score $s$ is $(1 - q_s)^n$; by the union bound the probability that some value lacks an untreated unit is at most $\sum_s (1 - q_s)^n \to 0$, and when every value has an untreated unit every treated unit has a match. $\square$

Exact matches on a continuous score do not exist, and the matched unit's score differs a little. Abadie and Imbens (2006) show that matching estimators on the covariates with a fixed number of matches are in general not $\sqrt n$-consistent, and do not attain the efficiency bound even when they are, and they give a consistent variance estimator (their abstract). Abadie and Imbens (2016) prove that estimating the score first changes the large-sample distribution of propensity score matching estimators and derive the correction to the variance. Ding (2024, §15.3) presents their bias-corrected matching estimator, which adds a regression adjustment for the remaining differences in covariates. Matching with replacement uses some untreated units many times, which costs precision; the NHANES section shows how much.

Doubly robust estimation

Weighting needs a correct model for the score and nothing about the outcome. Regression (the g-formula with a fitted $\hat\mu_w$) needs a correct outcome model and nothing about the score. The augmented inverse-probability-weighted (AIPW) estimator combines them so that either suffices. Let $\pi(x)$ be a working model for the score and $m_1(x)$, $m_0(x)$ working models for the outcome means; either may be wrong. The estimator is

$$\hat\tau_{\mathrm{DR}} = \frac1n\sum_i \left[m_1(X_i) + \frac{W_i\{Y_i - m_1(X_i)\}}{\pi(X_i)}\right] - \frac1n\sum_i \left[m_0(X_i) + \frac{(1 - W_i)\{Y_i - m_0(X_i)\}}{1 - \pi(X_i)}\right].$$

Read the first bracket as the regression prediction corrected by a weighted average of its errors on the treated, or as the weighting estimator corrected by the predictions. The estimator goes back to Robins, Rotnitzky and Zhao (1994), as Hernán and Robins (2020, Technical Point 13.3) attribute it; Bang and Robins (2005) define double robustness as remaining consistent when either model is correct.

Theorem 5 (double robustness; Ding, 2024, Theorem 12.1). Under strong ignorability, for fixed functions $\pi$ (with $0 < \pi < 1$) and $m_1$,

$$E\!\left[m_1(X) + \frac{W\{Y - m_1(X)\}}{\pi(X)}\right] - E[Y(1)] = E\!\left[\frac{\{e(X) - \pi(X)\}\{\mu_1(X) - m_1(X)\}}{\pi(X)}\right],$$

where $\mu_1(x) = E[Y(1) \mid X = x]$. The bias is zero if $\pi = e$ or if $m_1 = \mu_1$, and the same holds for the second bracket with $1 - e$, $1 - \pi$ and $\mu_0$.

Proof. By consistency $W\{Y - m_1(X)\} = W\{Y(1) - m_1(X)\}$. Subtract $Y(1) = m_1(X) + \{Y(1) - m_1(X)\}$ inside the expectation:

$$E\!\left[m_1(X) + \frac{W\{Y(1) - m_1(X)\}}{\pi(X)} - Y(1)\right] = E\!\left[\frac{W - \pi(X)}{\pi(X)}\,\{Y(1) - m_1(X)\}\right].$$

Condition on $X$. By unconfoundedness $W$ and $Y(1)$ are independent given $X$, so the conditional expectation of the product is the product of $E[W - \pi(X) \mid X] = e(X) - \pi(X)$ and $E[Y(1) - m_1(X) \mid X] = \mu_1(X) - m_1(X)$, divided by $\pi(X)$. $\square$

The bias is a product of two errors. Hernán and Robins (2020, Technical Point 13.2) call this second-order bias and note that it allows flexible estimates of both functions to be combined with small bias. When fitted models replace the fixed functions, the theorem applies to their probability limits, which gives consistency if either model is correctly specified. When both are wrong the product need not be small, and nothing says it is smaller than either error alone. Kang and Schafer (2007), estimating a population mean from incomplete data in a simulation where both models were incorrect but neither grossly misspecified, found that many doubly robust methods performed better than simple inverse-probability weighting but that none improved on simple regression-based prediction; in their words, "two wrong models are not better than one" (their abstract). The simulations below find the same pattern in a sharper form.

Overlap and extreme weights

Positivity says the score is never exactly 0 or 1. It does not say how close it comes, and Proposition 2 shows that closeness is what decides the variance. Problem 4 constructs an estimator that is unbiased and has infinite variance although positivity holds everywhere. Ding (2024, §11.2.3) notes that many asymptotic results assume strong overlap, the score bounded away from 0 and 1, and that even then the estimated score can come close to them.

Three remedies are common, and each changes the question.

  • Trimming drops the units whose estimated score lies outside an interval. Crump, Hotz, Imbens and Mitnik (2009) derive the subpopulation in which the effect can be estimated most precisely and report that the rule of thumb of discarding units with estimated scores outside $[0.1, 0.9]$ approximates their optimal rule across a wide range of distributions. The estimand becomes the effect in the retained subpopulation.
  • Truncation replaces scores below $L$ by $L$ and above $U$ by $U$, which keeps all units but biases the weights (Ding, 2024, §11.2.3).
  • Changing the estimand to the effect on the treated: by Proposition 1 only untreated units get weights, and only through the odds $e/(1 - e)$, so scores near 0 do no harm; scores near 1 still do.

A useful summary of the weights is the effective sample size $n_{\mathrm{eff}} = (\sum_i w_i)^2/\sum_i w_i^2$. For independent values with common variance $\sigma^2$, the weighted mean $\sum_i w_iy_i/\sum_i w_i$ has variance $\sigma^2\sum_i w_i^2/(\sum_i w_i)^2 = \sigma^2/n_{\mathrm{eff}}$, the variance of an unweighted mean of $n_{\mathrm{eff}}$ values. Survey Weights and the Horvitz–Thompson Estimator discusses the same quantity for survey weights as Kish's effective sample size, and what it does not measure.

Hernán and Robins (2020, §15.4) warn that restricting to a range of the estimated score defines a population that is hard to describe ("individuals do not come with a propensity score tattooed on their forehead") and recommend restricting on real covariates where possible.

What the simulations show. In the design described in the section on simulations below, with the linear predictor of the true score multiplied by 3, the score treats a share 0.314 of the population, and a median of 628 of 2,000 units were treated. The largest weight in a data set had median 158, and the effective sample size of the treated had median 81. The Horvitz–Thompson estimator with the true score gave a median of 1.176 (0.970 to 1.346; here and below, the interquartile range over the 30 data sets, seeds 32,000,000 to 32,000,029) for an effect of 1, and the Hájek estimator with a correctly specified estimated score 1.207 (1.043 to 1.316). The doubly robust estimator gave 0.980 (0.943 to 1.037). Trimming at $[0.1, 0.9]$ kept a median share 0.454 of the units (the true score lies in the interval for a share 0.447 of the population) and gave 1.029 (0.974 to 1.100) for the Hájek estimator and 1.010 (0.959 to 1.077) for the doubly robust one. The trimmed population's own average effect, computed by numerical integration, is 1.005, close to 1 here because the effect varies little with the covariates; in general it can be far from the effect in the whole population.

The NHANES example

Back to the 4,200 adults. The treatment is having smoked at least 100 cigarettes and having quit ($W = 1$, 1,182 adults) against never having smoked ($W = 0$, 3,018); the outcome is diagnosed diabetes. The covariates are those of the first section, coded for the models as a cubic in $(\text{age} - 50)/10$, an indicator for men, five indicators for race and Hispanic origin (non-Hispanic White as reference), four for education (high school as reference) and four for income (four or more times the poverty line as reference). The score model is a logistic regression with 18 coefficients. The outcome models for regression and the doubly robust estimator are two logistic regressions with the same terms, one fitted to each group. Body mass index is left out: Confounding and Causal Diagrams explains why it may be a mediator. The survey's sampling weights are not used: the adults are treated as a finite population, as in the course pages, so the estimates describe these examined adults rather than the U.S. population. Survey Weights and the Horvitz–Thompson Estimator computes the weighted version of the crude comparison.

Overlap. The estimated scores run from 0.031 to 0.730 among former smokers and from 0.015 to 0.730 among never smokers, with means 0.389 and 0.239. No score is near 1, and the smallest are not extreme: the largest inverse-probability weight is 32.1. The effective sample sizes are 709 of 1,182 former smokers and 2,791 of 3,018 never smokers, so weighting costs the former smokers about two fifths of their information.

Left: the estimated propensity scores of former and never smokers. Notice that the two distributions cover nearly the same range, with former smokers shifted toward higher scores; apart from a few never smokers with scores below 0.03, there are no regions where only one group appears. Right: the standardized difference of each covariate column between former and never smokers, before and after weighting by the inverse of the estimated score. Notice that age and sex differ by about half a standard deviation before weighting and that every column is within 0.05 after.
Left: the estimated propensity scores of former and never smokers. Notice that the two distributions cover nearly the same range, with former smokers shifted toward higher scores; apart from a few never smokers with scores below 0.03, there are no regions where only one group appears. Right: the standardized difference of each covariate column between former and never smokers, before and after weighting by the inverse of the estimated score. Notice that age and sex differ by about half a standard deviation before weighting and that every column is within 0.05 after.

Balance. The standardized difference of a covariate is the difference of its group means divided by the square root of the average of the two group variances. Before weighting, 7 of the 17 covariate columns differ by more than 0.1; the largest differences are age (0.61) and sex (0.49). After weighting by the inverse of the estimated score the largest absolute difference is 0.047, and age differs by $-0.003$. Theorem 1 predicts balance with the true score; balance with the estimated score is evidence that the score model is adequate for the measured covariates, and says nothing about unmeasured ones.

Estimates. The risk difference is in percentage points.

Method Target Risk, former Risk, never Difference Ratio
Crude none 0.229 0.131 9.9 1.76
Outcome regression $\tau$ 0.173 0.154 1.9 1.12
Weighting (Hájek) $\tau$ 0.170 0.153 1.8 1.12
Doubly robust $\tau$ 0.174 0.155 1.9 1.12
Quintile subclasses $\tau$ 0.175 0.150 2.5 1.17
Matching, 1 neighbor $\tau_{\mathrm t}$ 0.229 0.188 4.1 1.22
Weighting by the odds $\tau_{\mathrm t}$ 0.229 0.209 2.0 1.10
Outcome regression $\tau_{\mathrm t}$ 0.229 0.214 1.5 1.07

For the effect on the treated the risk of former smokers is their observed risk, and the "never" column is the estimate of $E[Y(0) \mid W = 1]$: the risk former smokers would have had if they had never smoked. Matching paired the 1,182 former smokers with 726 distinct never smokers, many used more than once.

Uncertainty was measured by a nonparametric bootstrap: 500 resamples of the 4,200 adults (seed 31,000,000), refitting every model in each, with 2.5% and 97.5% percentiles as the interval.

Method Difference, 95% interval Ratio, 95% interval
Crude 7.1 to 12.7 1.49 to 2.05
Outcome regression ($\tau$) −0.5 to 4.4 0.97 to 1.31
Weighting ($\tau$) −0.5 to 4.4 0.97 to 1.30
Doubly robust ($\tau$) −0.5 to 4.5 0.97 to 1.32
Matching ($\tau_{\mathrm t}$) −4.0 to 6.0 0.84 to 1.34

Four observations. The three estimators of $\tau$ that use a fitted model agree closely, at a ratio of 1.12 and a difference of about 1.8 to 1.9 points, far below the crude 1.76 and 9.9. Quintile subclassification is higher, as residual imbalance within subclasses would predict. Matching on one neighbor is the least precise: its interval for the difference is about twice as wide as those of the other adjusted estimators (it also targets $\tau_{\mathrm t}$, not $\tau$). Its point estimate is also fragile. Coding the 231 adults aged 18 and 19 at the high-school level instead of using their recorded education, a change to 5.5% of the adults, moves the matching estimate from 4.1 to −0.5 points (777 of the 1,182 former smokers get a different nearest never smoker), while the doubly robust estimate stays at 1.9. One nearest neighbor per former smoker makes the estimate depend on which never smoker happens to be closest. And the bootstrap treats the adults as independent draws, while NHANES sampled them in clusters, so the intervals are probably too narrow; Survey Sampling Designs explains why.

How strong would an unmeasured confounder have to be? VanderWeele and Ding (2017) define the E-value of an observed risk ratio $\mathrm{RR} > 1$ as $\mathrm{RR} + \sqrt{\mathrm{RR}(\mathrm{RR} - 1)}$, the minimum strength of association, on the risk-ratio scale, that an unmeasured confounder would need with both treatment and outcome, beyond the measured covariates, to explain the observed ratio away (Ding, 2024, Definition 17.1). For the doubly robust ratio 1.12 the E-value is 1.49; for the crude ratio 1.76 it is 2.91. VanderWeele and Ding also recommend computing it for the confidence limit closest to the null; here that limit is already below 1, so no unmeasured confounding at all is needed for the interval to include no effect.

What this analysis cannot say. Physical activity, diet, alcohol and family history of diabetes are not in the model, and each plausibly affects both quitting and diabetes; an association of 1.49 with each would be enough to remove the estimated excess. The data are cross-sectional, and some adults may have quit because they were diagnosed, an arrow from outcome to treatment that no adjustment set repairs. "Former smoker" lumps together people who quit decades ago and last month, while Yeh and colleagues (2010) found the risk highest in the first years after quitting. The analysis estimates an adjusted association under stated assumptions; calling it the effect of having quit requires believing those assumptions.

Simulations with a known answer

The NHANES analysis cannot be checked against the truth. A simulation can. In each of 30 data sets (seeds 30,000,000 to 30,000,029) 2,000 units have two independent standard normal covariates $X_1, X_2$; treatment follows the score

$$\operatorname{logit} e(x) = -0.4 + 0.6\,x_1 - 0.6\,x_2 + 0.5\,(x_1^2 - 1),$$

which treats a share 0.404 of the population; and the outcomes are $Y(0) = x_1 + x_2 + (x_1^2 - 1) + \varepsilon$ and $Y(1) = Y(0) + 1 + 0.5\,x_1$ with $\varepsilon$ standard normal. The average effect is $\tau = 1$ exactly, since $E[X_1] = 0$; the effect on the treated is 1.137 (by numerical integration), larger because the treated have larger $X_1$ on average. The quadratic term enters both the score and the outcome, so a model that leaves it out is wrong for both.

Each working model is fitted either correctly (with $x_1^2$) or incorrectly (linear in $x_1$ and $x_2$ only): logistic regression for the score, least squares in each group for the outcome. Matching pairs each unit with the nearest unit of the other group on the logit of the estimated score, with replacement, and averages the imputed differences, so that it estimates $\tau$. The table gives the median over the 30 data sets and the interquartile range.

Estimator Both models right Outcome model wrong
Crude 1.805 (1.734 to 1.903) 1.805 (1.734 to 1.903)
Regression 1.001 (0.975 to 1.028) 1.815 (1.733 to 1.857)
Weighting, true score (HT) 1.056 (0.886 to 1.125) 1.056 (0.886 to 1.125)
Weighting, estimated (HT) 1.051 (0.971 to 1.105) 1.051 (0.971 to 1.105)
Weighting, estimated (Hájek) 1.057 (0.970 to 1.106) 1.057 (0.970 to 1.106)
Doubly robust 0.999 (0.980 to 1.037) 1.045 (0.983 to 1.092)
Quintile subclasses 1.238 (1.206 to 1.268) 1.238 (1.206 to 1.268)
Matching 1.028 (0.930 to 1.104) 1.028 (0.930 to 1.104)
Estimator Score model wrong Both models wrong
Crude 1.805 (1.734 to 1.903) 1.805 (1.734 to 1.903)
Regression 1.001 (0.975 to 1.028) 1.815 (1.733 to 1.857)
Weighting, true score (HT) 1.056 (0.886 to 1.125) 1.056 (0.886 to 1.125)
Weighting, estimated (HT) 1.836 (1.749 to 1.953) 1.836 (1.749 to 1.953)
Weighting, estimated (Hájek) 1.788 (1.719 to 1.885) 1.788 (1.719 to 1.885)
Doubly robust 1.000 (0.973 to 1.040) 2.061 (1.964 to 2.178)
Quintile subclasses 1.651 (1.603 to 1.705) 1.651 (1.603 to 1.705)
Matching 1.621 (1.548 to 1.695) 1.621 (1.548 to 1.695)
The eight estimators in the four scenarios: median over 30 simulated data sets with the interquartile range, and the true effect of 1 as a vertical line. Notice that the doubly robust estimator is on the line whenever one of its two models is right and is the worst of all when both are wrong; and that every estimator relying only on the estimated score (weighting, subclasses, matching) is pulled toward the crude 1.805 when the score model omits the quadratic term, while weighting with the true score is unaffected but has the widest spread.
The eight estimators in the four scenarios: median over 30 simulated data sets with the interquartile range, and the true effect of 1 as a vertical line. Notice that the doubly robust estimator is on the line whenever one of its two models is right and is the worst of all when both are wrong; and that every estimator relying only on the estimated score (weighting, subclasses, matching) is pulled toward the crude 1.805 when the score model omits the quadratic term, while weighting with the true score is unaffected but has the widest spread.

The table bears out the theory and shows its limits.

  • Each method is near 1 when its own model is right and far from 1 when it is wrong; the crude difference of 1.805 shows the size of the confounding.
  • The doubly robust estimator was near 1 in three scenarios, as Theorem 5 says. With both models wrong it gave 2.061, further from the truth than any other estimator, the sharper form of Kang and Schafer's finding. Theorem 5 explains why. Evaluating the bias formula at the probability limits of the two wrong working models (by numerical integration), the treated bracket overstates $E[Y(1)]$ by 0.72 and the control bracket understates $E[Y(0)]$ by 0.35, a total of 1.07, against limiting biases of 0.80 for regression alone and 0.84 for Horvitz–Thompson weighting with the wrong score alone. The score error $e - \pi$ and the outcome error $\mu_1 - m_1$ have the same sign for 90% of the population. Units whose working score is below 0.2 are 10.5% of the population but carry 58% of the treated bracket's bias: the factor $1/\pi$ magnifies the product of two same-signed errors where $\pi$ is small.
  • Weighting with the true score had the widest interquartile range of the estimators near 1, and its standard deviation over the 30 data sets was 0.225 against 0.154 with the estimated score (interquartile ranges 0.238 and 0.134), as Proposition 4 suggests. Its mean over the data sets was 1.005, with a standard error of 0.041, consistent with Theorem 4's unbiasedness; the median of 1.056 reflects skewness from large weights.
  • Quintile subclassification left a bias of about 0.24 even with the right score, and matching was the least precise of the estimators near 1.

These are observations of this design at this sample size, not general rankings.

What is proved, what is cited, what the experiment shows

Proved on this page: the balancing property (Theorem 1); the characterization of balancing scores (Theorem 2); the inheritance of unconfoundedness and positivity by the score (Theorem 3) and the unbiasedness of comparisons at a fixed score (Corollary 1); the inverse-probability-weighting identities and the unbiasedness of the known-score estimator (Theorem 4); the weights for the effect on the treated (Proposition 1); the variance of the known-score estimator (Proposition 2); the identity of weighting with an estimated cell score and subclassification (Proposition 3); the cost of the true score in one cell (Proposition 4); exact matching (Proposition 5); the bias formula of the doubly robust estimator and its double robustness (Theorem 5).

Cited: the results' original sources (Rosenbaum and Rubin, 1983; Robins, Rotnitzky and Zhao, 1994; Bang and Robins, 2005), the identification of the score-weighted estimator with the Horvitz–Thompson estimator, whose unbiasedness Survey Weights and the Horvitz–Thompson Estimator proves; the efficiency of the estimated score (Rosenbaum, 1987; Hirano, Imbens and Ridder, 2003), the large-sample theory of matching (Abadie and Imbens, 2006, 2016), the trimming rule (Crump et al., 2009), the double-robustness simulation of Kang and Schafer (2007), Cochran's 90% (through Rosenbaum and Rubin, 1983, p. 52), and the E-value (VanderWeele and Ding, 2017, through Ding, 2024).

What the experiments show: in the simulation each estimator was near the truth when its model was right; the doubly robust estimator survived either model being wrong and failed worst when both were; the estimated score gave a smaller spread than the true score; subclassification on quintiles kept a bias of about a quarter of the effect. Under poor overlap the true-score weighting estimator was unstable and trimming at $[0.1, 0.9]$ stabilized it. In NHANES, adjustment for age, sex, race and Hispanic origin, education and income reduced the risk ratio of former against never smokers from 1.76 to 1.12, with a bootstrap interval that includes no difference.

Worked problems

The problems were written for this article; every number is computed exactly and checked by the article's script.

Problem 1 (making controls look like the treated). A covariate takes three values with probabilities $1/2$, $3/10$ and $1/5$, and the score is $1/5$, $1/2$ and $4/5$ at the three values. Find the distribution of the covariate among the treated and the untreated, and show that weighting the untreated by the odds $e/(1 - e)$ turns their distribution into that of the treated.

Solution. $P(W = 1) = \frac12 \cdot \frac15 + \frac{3}{10} \cdot \frac12 + \frac15 \cdot \frac45 = 0.41$. By Bayes' theorem $P(X = x \mid W = 1) = P(X = x)e(x)/0.41$, which gives $10/41 = 0.2439$, $15/41 = 0.3659$ and $16/41 = 0.3902$; and $P(X = x \mid W = 0) = P(X = x)(1 - e(x))/0.59$, which gives $40/59 = 0.6780$, $15/59 = 0.2542$ and $4/59 = 0.0678$. The odds are $1/4$, $1$ and $4$. Multiplying the untreated distribution by them gives $10/59$, $15/59$ and $16/59$, whose total is $41/59 = 0.6949$, the odds of treatment $P(W = 1)/P(W = 0)$. Dividing by the total gives $10/41$, $15/41$ and $16/41$, the distribution among the treated. This is Proposition 1 at work: the weighted untreated mean of any function of the covariates, or of $Y(0)$ under unconfoundedness, is its mean among the treated.

Problem 2 (the price of the true score). In a single cell with treatment probability $e = 1/4$, the treated outcome has mean $\mu = 10$ and standard deviation $\sigma = 2$. Compare $n$ times the variance of the true-score weighting estimator of $E[Y(1)]$ with that of the mean of the treated.

Solution. By Proposition 4 the first is $(\sigma^2 + \mu^2)/e - \mu^2 = (4 + 100) \times 4 - 100 = 316$, and the second tends to $\sigma^2/e = 16$. The true-score estimator needs about 20 times as many units for the same precision. The excess $\mu^2(1 - e)/e = 300$ comes entirely from the level of the outcome: the true-score estimator multiplies each treated outcome by 4 and so inherits the random count of treated units, while the mean of the treated divides by that count. Shifting the outcome to mean 0 would remove the excess, which is the location defect that the Hájek form repairs.

Problem 3 (taking a doubly robust estimator apart). Use the covariate and score of Problem 1, with $E[Y(1) \mid X = x]$ equal to 2, 5 and 9 at the three values. Compute the limit of the first bracket of the doubly robust estimator when the score model is $\pi \equiv 1/2$ and the outcome model is $m_1 \equiv 0$, and check it against Theorem 5.

Solution. $E[Y(1)] = \frac12 \cdot 2 + \frac{3}{10} \cdot 5 + \frac15 \cdot 9 = 4.30$. With $m_1 \equiv 0$ the bracket is $WY/\pi$, whose mean is $\sum_x P(x)\,e(x)\,\mu_1(x)/\pi(x) = 2\big(\frac12 \cdot \frac15 \cdot 2 + \frac{3}{10} \cdot \frac12 \cdot 5 + \frac15 \cdot \frac45 \cdot 9\big) = 2 \times 2.39 = 4.78$. Theorem 5 gives the bias as $\sum_x P(x)\{e(x) - \frac12\}\mu_1(x)/\frac12 = 2\big(\frac12 \cdot (-\frac{3}{10}) \cdot 2 + 0 + \frac15 \cdot \frac{3}{10} \cdot 9\big) = 2(-0.3 + 0.54) = 0.48$, and $4.30 + 0.48 = 4.78$. With the correct score $\pi = e$ the bracket's mean is 4.30 whatever $m_1$ is; with the correct outcome model $m_1 = \mu_1$ it is 4.30 whatever $\pi$ is; and the outcome model $m_1 \equiv 0$ alone, used as a regression estimator, gives 0.

Problem 4 (unbiased with infinite variance). Let $X$ be uniform on $(0, 1)$, $e(x) = x$ and $Y(1) \equiv 1$. Show that $WY/e(X)$ has mean 1 and infinite variance, although positivity holds.

Solution. Positivity holds, since $0 < x < 1$ for every $x$ in the support. By Theorem 4 the mean is $E[Y(1)] = 1$. By Proposition 2 the variance is $E[1/e(X)] - 1 = \int_0^1 \frac{dx}{x} - 1$, and the integral diverges. The average of $n$ such terms is unbiased, but it has infinite variance for every $n$, so the usual standard error, which estimates that variance, describes nothing. Strong overlap ($e$ bounded away from 0) would rule this out; positivity alone does not.

Summary

Result Statement
Balancing $W \perp\!\!\!\perp X \mid e(X)$
Balancing scores $b(X)$ balances iff $e(X) = f(b(X))$
Unconfoundedness given $X$ implies given $e(X)$
Weighting $E[WY/e(X)] = E[Y(1)]$; Horvitz–Thompson in the units
Effect on the treated untreated weighted by $e/(1 - e)$
Variance, known score $E[E(Y(1)^2 \mid X)/e(X)] - (E\,Y(1))^2$
Estimated score cell frequencies give subclassification; cost of true score $\mu^2(1 - e)/e$
Double robustness bias $E[(e - \pi)(\mu_1 - m_1)/\pi]$
NHANES risk ratio 1.76 crude, 1.12 adjusted; E-value 1.49

What we have learned

The propensity score compresses the covariates into the one number that matters for comparability: given it, treatment is independent of the covariates, and unconfoundedness given the covariates becomes unconfoundedness given the score. Weighting by its inverse is the Horvitz–Thompson estimator of survey sampling applied to the treated as a sample of the population; subclassification and matching compare like with like on it; and the doubly robust estimator combines it with an outcome model so that either can be wrong. Every one of these methods needs both a correct model and enough overlap, and none can say anything about confounders that were not measured. In the smoking data adjustment removes most of the crude association, and what remains could be explained by an unmeasured confounder of moderate strength.

Weighting by inverse probabilities came from survey sampling, where the probabilities are known because the statistician chose them. Survey Sampling Designs takes up those designs, and Survey Weights and the Horvitz–Thompson Estimator proves the theorem that weighting rests on.

References

  • Abadie, A., and Imbens, G. W. (2006). Large sample properties of matching estimators for average treatment effects. Econometrica, 74(1), 235–267. https://doi.org/10.1111/j.1468-0262.2006.00655.x record
  • Abadie, A., and Imbens, G. W. (2016). Matching on the estimated propensity score. Econometrica, 84(2), 781–807. https://doi.org/10.3982/ECTA11293 record
  • Bang, H., and Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4), 962–973. https://doi.org/10.1111/j.1541-0420.2005.00377.x record
  • Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24(2), 295–313. https://doi.org/10.2307/2528036. Not seen; cited as Rosenbaum and Rubin (1983, p. 52) state its finding. record
  • Crump, R. K., Hotz, V. J., Imbens, G. W., and Mitnik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1), 187–199. https://doi.org/10.1093/biomet/asn055 record
  • Ding, P. (2024). A First Course in Causal Inference. CRC Press. arXiv:2305.18793 (v2, 2023). Chapters 11, 12, 15 and 17. record · 6 pages
  • Hernán, M. A., and Robins, J. M. (2020). Causal Inference: What If. Chapman and Hall/CRC. Version of 19 August 2026 from the authors' site, https://miguelhernan.org/whatifbook. Sections 15.2 to 15.4, Technical Points 13.2, 13.3 and 15.1. record · 3 pages
  • Hirano, K., Imbens, G. W., and Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4), 1161–1189. https://doi.org/10.1111/1468-0262.00442 record
  • Horvitz, D. G., and Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260), 663–685. https://doi.org/10.1080/01621459.1952.10483446 record · 3 pages
  • Kang, J. D. Y., and Schafer, J. L. (2007). Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22(4), 523–539. https://doi.org/10.1214/07-STS227 (arXiv:0804.2958) record
  • Lunceford, J. K., and Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Statistics in Medicine, 23(19), 2937–2960. https://doi.org/10.1002/sim.1903 record
  • National Center for Health Statistics (2020). National Health and Nutrition Examination Survey 2017–2018: Demographic Variables and Sample Weights (DEMO_J), Blood Pressure (BPX_J), Blood Pressure and Cholesterol questionnaire (BPQ_J), Body Measures (BMX_J), Diabetes (DIQ_J) and Smoking (SMQ_J) data files. Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2017 record · 106 pages
  • Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427), 846–866. https://doi.org/10.1080/01621459.1994.10476818. Not seen; cited as Hernán and Robins (2020, Technical Point 13.3) attribute the estimator to it. record
  • Rosenbaum, P. R. (1987). Model-based direct adjustment. Journal of the American Statistical Association, 82(398), 387–394. https://doi.org/10.1080/01621459.1987.10478441 record
  • Rosenbaum, P. R., and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41. Read in the technical report version (MRC Technical Summary Report 2305, Mathematics Research Center, University of Wisconsin–Madison, December 1981, DTIC ADA114514), which has the same theorem numbering; the published version (pp. 44–46 for Theorems 1 to 4, p. 52 for the subclasses) was checked by the site's citation audit. record · 2 pages
  • Rosenbaum, P. R., and Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79(387), 516–524. https://doi.org/10.1080/01621459.1984.10478078 record
  • VanderWeele, T. J., and Ding, P. (2017). Sensitivity analysis in observational research: introducing the E-value. Annals of Internal Medicine, 167(4), 268–274. https://doi.org/10.7326/M16-2607. Abstract read; the formula as Ding (2024, Definition 17.1) gives it. record
  • Yeh, H.-C., Duncan, B. B., Schmidt, M. I., Wang, N.-Y., and Brancati, F. L. (2010). Smoking, smoking cessation, and risk for type 2 diabetes mellitus: a cohort study. Annals of Internal Medicine, 152(1), 10–17. https://doi.org/10.7326/0003-4819-152-1-201001050-00005 record · 2 pages