Iowa's weekly deaths from all causes, 313 weeks from 2014 to 2019, have a correlation of 0.66 between each week and the next. Iowa's weekly deaths from cancer, over the same weeks, have a correlation of 0.01. A correlation between a series and a shifted copy of itself is an autocorrelation, and it raises two questions at once. When does such a number estimate a property of the process that made the data, and what does a correlation of 0.66 do to the statistics that assume independent observations: the standard error of a mean, a t-test, a test of whether the series is noise?
The first answer is stationarity: the statistical behavior of the series does not change when the time origin moves, so that the correlation between weeks $h$ apart is one number $\rho(h)$, the same in every part of the series, and the data from all pairs $h$ apart estimate it together. The second answer is a set of exact formulas and limit theorems. A positive autocorrelation makes the mean more variable than independence predicts, and a nominal 5% t-test then rejects far more often than 5% of the time. The sample autocorrelations of noise scatter around zero with standard deviation about $1/\sqrt{n}$, which gives a test of whether a series is noise.
Here is the route. First we define strict and weak stationarity and the autocovariance function, and prove which functions can be autocovariances. Then we derive the variance of a mean for any covariances, and the exact large-sample level of a t-test on an autoregressive series. Next we study the sample autocovariance: its bias, its nonnegative definiteness, and the distribution of the sample autocorrelations of independent noise, through a central limit theorem for dependent sequences that we prove. That distribution gives the portmanteau tests. Finally we treat trend, seasonality and differencing, and measure all of it on Iowa's deaths. The article assumes the variance of the mean of a stationary series, Theorem 1 of Resampling Dependent and Clustered Data, and the convergence results of Convergence in Distribution and the Proof of the Central Limit Theorem. Autoregressive and Moving Average Models builds models on the autocovariances defined here.
The data and the notation
The weekly deaths. The National Center for Health Statistics publishes weekly counts of deaths by state of occurrence and by the week in which they occurred (National Center for Health Statistics, 2021). Iowa has 313 weeks from 2014 to 2019: the year 2014 has 53 weeks in the MMWR calendar the counts follow, and the other five years 52. All-cause deaths average 570.5 a week with standard deviation 46.8 (divisor $n$); cancer deaths average 122.7. The same series appear in The Poisson Distribution, Overdispersion in Count Data, Overdispersion and Quasi-Likelihood and Resampling Dependent and Clustered Data. The last uses the first 312 weeks; this article uses all 313, so its sample autocorrelations can differ from that page's in the second decimal. Every number below comes from tools/experiments/statistics/autocorrelation-and-stationarity.py.
Notation. A time series is a sequence of random variables $X_t$ indexed by integer times $t$; the data are one stretch $x_1, \dots, x_n$ of one realization, with mean $\bar x$. The mean function is $\mu_t = E(X_t)$, and the covariance of two times is $\operatorname{Cov}(X_s, X_t)$. White noise $Z_t \sim \mathrm{WN}(0, \sigma^2)$ means $E(Z_t) = 0$, $\operatorname{Var}(Z_t) = \sigma^2$ and $\operatorname{Cov}(Z_s, Z_t) = 0$ for $s \ne t$. Independent noise $Z_t \sim \mathrm{IID}(0, \sigma^2)$ adds that the $Z_t$ are independent and identically distributed. The backshift operator is $BX_t = X_{t-1}$, the difference operator $\nabla = 1 - B$, so $\nabla X_t = X_t - X_{t-1}$, and the lag-$d$ difference $\nabla_d X_t = X_t - X_{t-d}$. The notation is shared with the rest of the Stochastic Processes and Time Series unit, whose main text is Brockwell and Davis (2016), with chapters on stationary processes (chapter 2) and on ARMA models (chapter 3).
Stationarity
Here is the idea. A process is stationary when shifting it in time changes nothing about its probabilities. The strong form asks this of the whole joint distribution; the weak form asks it only of means, variances and covariances, which is all that the second-moment results of this article use.
Definition (strict stationarity). $\{X_t\}$ is strictly stationary if for every $k \ge 1$, every times $t_1, \dots, t_k$ and every shift $h$, the vectors $(X_{t_1}, \dots, X_{t_k})$ and $(X_{t_1+h}, \dots, X_{t_k+h})$ have the same joint distribution.
Definition (weak stationarity). $\{X_t\}$ with $E(X_t^2) < \infty$ for all $t$ is weakly stationary if $\mu_t = \mu$ does not depend on $t$ and $\operatorname{Cov}(X_t, X_{t+h})$ depends only on $h$. Its autocovariance function is then $$\gamma(h) = \operatorname{Cov}(X_t, X_{t+h}),$$ and its autocorrelation function (ACF) is $\rho(h) = \gamma(h)/\gamma(0)$, when $\gamma(0) > 0$. Taking $t = s - h$ shows $\gamma(-h) = \operatorname{Cov}(X_{s-h}, X_s) = \gamma(h)$, so both functions are even. In this unit "stationary" alone means weakly stationary.
The Markov chains of Markov Chains and Stationary Distributions use the word for a distribution: a stationary distribution $\pi$ is one that the chain's transitions leave unchanged. The two uses meet. A chain started in its stationary distribution has the same distribution at every time, and its joint distributions over any $k$ times are then unchanged by a shift; the chain is a strictly stationary process (Durrett, 2019, Example 6.1.2).
How the two definitions relate.
- Strict implies weak, given finite variances. If $E(X_t^2) < \infty$, strict stationarity makes $X_t$ and $X_{t+h}$ share a distribution, so the mean is constant, and makes $(X_t, X_{t+h})$ and $(X_0, X_h)$ share a joint distribution, so the covariance depends only on $h$.
- Weak does not imply strict. Let the $X_t$ be independent, standard normal for odd $t$ and equal to $+1$ or $-1$ with probability one half each for even $t$. Every $X_t$ has mean 0 and variance 1 and different times are uncorrelated, so the process is weakly stationary (it is white noise), but $X_1$ and $X_2$ have different distributions.
- For Gaussian processes they agree. A process is Gaussian if every vector $(X_{t_1}, \dots, X_{t_k})$ is multivariate normal.
Proposition 1 (Gaussian processes). A weakly stationary Gaussian process is strictly stationary.
Proof. The vectors $(X_{t_1}, \dots, X_{t_k})$ and $(X_{t_1+h}, \dots, X_{t_k+h})$ are multivariate normal with the same mean vector $(\mu, \dots, \mu)$ and the same covariance matrix, whose $(i, j)$ entry is $\gamma(t_j - t_i)$ for both. A multivariate normal distribution is determined by its mean vector and covariance matrix (Theorem 1 of The Multivariate Normal Distribution). $\square$
Examples.
- Noise. $\mathrm{IID}(0, \sigma^2)$ is strictly stationary; $\mathrm{WN}(0, \sigma^2)$ is weakly stationary, with $\gamma(0) = \sigma^2$ and $\gamma(h) = 0$ for $h \ne 0$.
- A random walk is not stationary. $S_t = Z_1 + \dots + Z_t$ with $Z_t \sim \mathrm{IID}(0, \sigma^2)$ has $\operatorname{Var}(S_t) = t\sigma^2$, which grows with $t$. Random Walks and Martingales studies it.
- A moving average. $X_t = Z_t + \theta Z_{t-1}$ with $Z_t \sim \mathrm{WN}(0, \sigma^2)$ has $\gamma(0) = \sigma^2(1 + \theta^2)$, $\gamma(\pm1) = \operatorname{Cov}(Z_t + \theta Z_{t-1}, Z_{t+1} + \theta Z_t) = \theta\sigma^2$, and $\gamma(h) = 0$ for $\lvert h\rvert \ge 2$, because then the two sums share no $Z$. So $\rho(1) = \theta/(1 + \theta^2)$.
- An autoregression. $X_t = \phi X_{t-1} + Z_t$ with $\lvert\phi\rvert < 1$ has a stationary solution with $\gamma(h) = \sigma^2\phi^{\lvert h\rvert}/(1 - \phi^2)$, as Resampling Dependent and Clustered Data derives and Autoregressive and Moving Average Models proves in full.
- A cycle with a random phase. $X_t = A\cos(\omega t + U)$, with $A$ and $\omega$ fixed and $U$ uniform on $[0, 2\pi)$, has $E(X_t) = 0$ and, by $\cos a\cos b = \frac12[\cos(a - b) + \cos(a + b)]$, $$E(X_tX_{t+h}) = \frac{A^2}{2}E\big[\cos(\omega h) + \cos(2\omega t + \omega h + 2U)\big] = \frac{A^2}{2}\cos(\omega h),$$ because $\cos(c + 2U)$ averages to zero over a uniform $U$ for every constant $c$. So the process is stationary although each realization is a deterministic wave. Its autocorrelation $\cos(\omega h)$ never dies out. With a period of $2\pi/\omega = 52.18$ weeks it is $-1$ at a lag of about 26 weeks and $+1$ at 52.
The last example is a warning that will matter for Iowa. A seasonal cycle can be modeled as a fixed function of time, which makes the series nonstationary in its mean, or as a cycle with a random phase, which keeps it stationary. The data cannot tell the two apart from one stretch of six years, because the phase is drawn once. What changes is the question the analysis answers, as the section on Iowa shows.
Which functions are autocovariances
Here is the idea. A variance cannot be negative, and the variance of any weighted sum of the $X_t$ is a quadratic form in the autocovariances. So an autocovariance function must make every such quadratic form nonnegative. The converse also holds: every function with that property is the autocovariance of some process, and a Gaussian one can be built.
A real function $\kappa$ on the integers is nonnegative definite if $$\sum_{i=1}^k\sum_{j=1}^k a_i\,\kappa(t_i - t_j)\,a_j \ge 0$$ for every $k$, all integers $t_1, \dots, t_k$ and all real $a_1, \dots, a_k$. Equivalently, every matrix $[\kappa(t_i - t_j)]_{i,j}$ is positive semidefinite.
Theorem 1 (autocovariances are even and nonnegative definite). The autocovariance function $\gamma$ of a stationary process satisfies $\gamma(0) \ge 0$, $\gamma(-h) = \gamma(h)$, $\lvert\gamma(h)\rvert \le \gamma(0)$, and is nonnegative definite.
Proof. $\gamma(0) = \operatorname{Var}(X_t) \ge 0$, and evenness was shown above. By the Cauchy–Schwarz inequality for covariances, $\lvert\operatorname{Cov}(X_t, X_{t+h})\rvert \le \sqrt{\operatorname{Var}(X_t)\operatorname{Var}(X_{t+h})} = \gamma(0)$. For the last property, let $Y = \sum_i a_iX_{t_i}$. By the bilinearity of covariance, $$0 \le \operatorname{Var}(Y) = \sum_i\sum_j a_ia_j\operatorname{Cov}(X_{t_i}, X_{t_j}) = \sum_i\sum_j a_i\,\gamma(t_j - t_i)\,a_j,$$ and $\gamma(t_j - t_i) = \gamma(t_i - t_j)$. $\square$
Theorem 2 (every even nonnegative definite function is an autocovariance). If $\kappa$ is even and nonnegative definite, there is a strictly stationary Gaussian process with mean 0 and autocovariance function $\kappa$.
Proof. For each finite set of times $t_1 < \dots < t_k$, the matrix $K = [\kappa(t_j - t_i)]$ is symmetric (by evenness) and positive semidefinite (by assumption). So it has a factorization $K = AA^{\mathsf T}$, for instance from its spectral decomposition, and $AW$ with $W$ a vector of $k$ independent standard normal variables has the distribution $N_k(\mathbf{0}, K)$ (The Multivariate Normal Distribution). These finite-dimensional distributions are consistent: dropping a time from the list gives the normal distribution with that row and column deleted, which is the marginal of the larger one, by Theorem 3 of the same article (affine maps). Kolmogorov's extension theorem (Durrett, 2019, Theorem 2.1.21) then gives a single process $\{X_t\}$ on one probability space whose finite-dimensional distributions are these. It is Gaussian with mean 0 and $\operatorname{Cov}(X_s, X_t) = \kappa(t - s)$, so it is weakly stationary, and by Proposition 1 strictly stationary. $\square$
Durrett states the extension theorem for a sequence indexed by $1, 2, \dots$; a process indexed by all integers is the same thing after listing the integers as $0, 1, -1, 2, -2, \dots$, which is what we use. The theorem settles a practical question: to check that a proposed autocorrelation function is legitimate, check that it is nonnegative definite, nothing more.
Example (how large can a lag-1 correlation be, alone?). Suppose $\kappa(0) = 1$, $\kappa(\pm1) = r$ and $\kappa(h) = 0$ for $\lvert h\rvert \ge 2$. Is this an autocorrelation function for every $r$ in $[-1, 1]$? No: only for $\lvert r\rvert \le \frac12$. Take $a_t = (-1)^t$ for $t = 1, \dots, k$. The quadratic form is $k - 2(k - 1)r$, since each of the $k - 1$ neighboring pairs contributes $2a_ta_{t+1}r = -2r$. Nonnegativity for every $k$ requires $r \le k/(2(k - 1))$, and letting $k \to \infty$ gives $r \le \frac12$. With $a_t = 1$ the form is $k + 2(k - 1)r$, which requires $r \ge -\frac12$. Conversely, every $r$ with $\lvert r\rvert \le \frac12$ is attained by the moving average above: $\theta/(1 + \theta^2) = r$ has the real solution $\theta = (1 - \sqrt{1 - 4r^2})/(2r)$ for $0 < \lvert r\rvert \le \frac12$, and $\theta = 0$ gives $r = 0$. Worked problem 2 finds the same bound from the moving average's side.
The variance of a mean under dependence
Here is the idea. The variance of a sum is the sum of all the covariances among its terms. Independence keeps only the $n$ variances; dependence adds $n(n - 1)$ covariances, which can dominate. And the usual sample variance, which assumes a constant mean and independent terms, estimates neither the variance of a single observation nor that of the mean correctly when either assumption fails.
Theorem 3 (the mean and the sample variance, for any covariances). Let $X_1, \dots, X_n$ have means $\mu_t$, finite variances and covariances $c(s, t) = \operatorname{Cov}(X_s, X_t)$. Write $\bar\mu = \frac1n\sum_t\mu_t$. Then
- $\displaystyle\operatorname{Var}(\bar X) = \frac{1}{n^2}\sum_{s=1}^n\sum_{t=1}^n c(s, t)$;
- $\displaystyle E\sum_{t=1}^n(X_t - \bar X)^2 = \sum_{t=1}^n c(t, t) - n\operatorname{Var}(\bar X) + \sum_{t=1}^n(\mu_t - \bar\mu)^2$.
Proof. (1) is the bilinearity of covariance applied to $\bar X = \frac1n\sum_tX_t$. For (2), write $Y_t = X_t - \mu_t$, which has mean 0, so that $X_t - \bar X = (Y_t - \bar Y) + (\mu_t - \bar\mu)$. Squaring and summing, the cross term $2\sum_t(Y_t - \bar Y)(\mu_t - \bar\mu)$ has expectation 0. The identity $\sum_t(Y_t - \bar Y)^2 = \sum_tY_t^2 - n\bar Y^2$ gives $E\sum_t(Y_t - \bar Y)^2 = \sum_tc(t, t) - n\operatorname{Var}(\bar X)$, since $E(Y_t^2) = c(t, t)$ and $E(\bar Y^2) = \operatorname{Var}(\bar Y) = \operatorname{Var}(\bar X)$. $\square$
Three consequences follow, one per term.
- Stationary series. With $c(s, t) = \gamma(t - s)$, part 1 becomes $n\operatorname{Var}(\bar X) = \sum_{\lvert h\rvert < n}(1 - \lvert h\rvert/n)\gamma(h)$, Theorem 1 of Resampling Dependent and Clustered Data, which also proves the limit $n\operatorname{Var}(\bar X) \to \sigma_\infty^2 = \sum_h\gamma(h)$ when $\sum_h\lvert\gamma(h)\rvert < \infty$. The long-run variance $\sigma_\infty^2$ governs the error of an average along a Markov chain too, where Markov Chain Monte Carlo turns it into an effective sample size.
- The sample variance under dependence. For a stationary series, part 2 gives $$E(s^2) = \frac{n}{n - 1}\big(\gamma(0) - \operatorname{Var}(\bar X)\big), \qquad s^2 = \frac{1}{n - 1}\sum_t(X_t - \bar X)^2.$$ Only for uncorrelated observations, where $\operatorname{Var}(\bar X) = \gamma(0)/n$, is this $\gamma(0)$. Positive autocorrelation makes $\operatorname{Var}(\bar X)$ larger and so $s^2$ smaller on average. The usual standard error $s/\sqrt n$ is then too small twice over: it estimates $\gamma(0)/n$ instead of $\sigma_\infty^2/n$, and it underestimates even $\gamma(0)$.
- A changing mean. The last term of part 2 adds the spread of the means to the sample variance. A seasonal cycle or a trend therefore inflates $s^2$ as an estimate of the variance around the mean. For Iowa's all-cause deaths, a least-squares fit of a linear trend and an annual cycle (defined in the section on Iowa below) takes 0.60 of the sum of squares about the mean, the cycle alone 0.54, and leaves a remainder with standard deviation 29.6 against the raw 46.8. For cancer deaths the same fit takes 0.04.
Example (an autoregression with 52 observations). For the autoregression $X_t = 0.6X_{t-1} + Z_t$ scaled so that $\gamma(0) = 1$, with $n = 52$, part 1 gives $\operatorname{Var}(\bar X) = 0.0741$, nearly four times the $1/52 = 0.0192$ of independent observations. The expected sample variance with divisor $n$, the value at lag 0 in the table of the section on estimating the autocovariance function, is $1 - 0.0741 = 0.926$, not 1.
What dependence does to a t-test
The one-sample t-test of $H_0: \mu = \mu_0$ rejects at nominal level 5% when $\lvert T\rvert > t_{n-1,\,0.975}$, where $T = \sqrt n(\bar X - \mu_0)/s$. The Chi-Squared and t Distributions proves that $T$ has the $t$ distribution with $n - 1$ degrees of freedom for independent normal observations. A common reading is that serial correlation is a minor matter for a t-test, since the mean is still unbiased. The mean is unbiased, but the test's level is not 5%, and the next theorem says exactly what it is in large samples.
Here is the idea. In large samples $\sqrt n(\bar X - \mu)$ has variance $\sigma_\infty^2$, while $s^2$ estimates $\gamma(0)$. So $T$ is approximately normal with variance $\sigma_\infty^2/\gamma(0)$ instead of 1, and for the autoregression that ratio is $(1 + \phi)/(1 - \phi)$.
Theorem 4 (the level of the t-test for an autoregressive series). Let $\{X_t\}$ be a stationary Gaussian process with mean $\mu_0$ and $\gamma(h) = \gamma(0)\phi^{\lvert h\rvert}$, $\lvert\phi\rvert < 1$, the autocovariances of the autoregression. Then the probability that the nominal 5% two-sided t-test rejects $H_0: \mu = \mu_0$ tends, as $n \to \infty$, to $$\alpha(\phi) = 2\Big[1 - \Phi\Big(z_{0.975}\sqrt{\tfrac{1 - \phi}{1 + \phi}}\Big)\Big],$$ where $\Phi$ is the standard normal distribution function and $z_{0.975} = \Phi^{-1}(0.975) = 1.95996\ldots$, written 1.96 below.
Proof. Write $\kappa^2 = \sigma_\infty^2/\gamma(0) = (1 + \phi)/(1 - \phi)$; the sum $\sigma_\infty^2 = \gamma(0)(1 + \phi)/(1 - \phi)$ is the geometric series computed in Resampling Dependent and Clustered Data.
Step 1: the mean. $\bar X$ is a linear combination of jointly normal variables, so $\sqrt n(\bar X - \mu_0)$ is exactly normal with mean 0 and variance $v_n = n\operatorname{Var}(\bar X)$, and $v_n \to \sigma_\infty^2$ by the stationary case of Theorem 3. Hence $P(\sqrt n(\bar X - \mu_0) \le x) = \Phi(x/\sqrt{v_n}) \to \Phi(x/\sigma_\infty)$ for every $x$: $\sqrt n(\bar X - \mu_0) \xrightarrow{d} N(0, \sigma_\infty^2)$.
Step 2: the variance. Write $Y_t = X_t - \mu_0$. Then $s^2 = \frac{n}{n - 1}\big[\frac1n\sum_tY_t^2 - \bar Y^2\big]$. First, $E(\bar Y^2) = \operatorname{Var}(\bar X) \to 0$, so $\bar Y^2 \to 0$ in probability by Markov's inequality. Second, $\frac1n\sum_tY_t^2$ has mean $\gamma(0)$, and for jointly normal $Y_s, Y_t$ with mean 0, $$\operatorname{Cov}(Y_s^2, Y_t^2) = 2\gamma(t - s)^2.$$ To see this, write $Y_t = bY_s + W$ with $b = \gamma(t - s)/\gamma(0)$ and $W$ independent of $Y_s$ (the residual of a normal regression is uncorrelated with, hence independent of, the regressor). Then $E(Y_s^2Y_t^2) = b^2E(Y_s^4) + E(Y_s^2)E(W^2) = 3b^2\gamma(0)^2 + \gamma(0)E(W^2)$, using $E(Y_s^4) = 3\gamma(0)^2$ and the vanishing of the odd term $2bE(Y_s^3)E(W)$. Since $E(Y_t^2) = b^2\gamma(0) + E(W^2) = \gamma(0)$, the covariance is $3b^2\gamma(0)^2 + \gamma(0)E(W^2) - \gamma(0)^2 = 2b^2\gamma(0)^2 = 2\gamma(t - s)^2$. Therefore $$\operatorname{Var}\Big(\frac1n\sum_tY_t^2\Big) = \frac{2}{n^2}\sum_{s,t}\gamma(t - s)^2 \le \frac2n\sum_{h=-\infty}^\infty\gamma(h)^2 = \frac{2\gamma(0)^2}{n}\cdot\frac{1 + \phi^2}{1 - \phi^2} \to 0,$$ and Chebyshev's inequality gives $\frac1n\sum_tY_t^2 \to \gamma(0)$ in probability. So $s^2 \to \gamma(0)$ in probability.
Step 3: the statistic. By Slutsky's theorem (Convergence in Distribution and the Proof of the Central Limit Theorem), $T = \sqrt n(\bar X - \mu_0)/s \xrightarrow{d} N(0, \kappa^2)$. The critical value $c_n = t_{n-1,\,0.975}$ tends to $z_{0.975}$, so $z_{0.975}T/c_n$ has the same limit, again by Slutsky's theorem. The limit distribution is continuous, so $P(\lvert T\rvert > c_n) = P(\lvert z_{0.975}T/c_n\rvert > z_{0.975}) \to P(\lvert\kappa W\rvert > z_{0.975})$ with $W$ standard normal, which is $\alpha(\phi)$. $\square$
At $\phi = 0$ the formula gives exactly 0.05. For $\phi > 0$ the level exceeds 5%, and for $\phi < 0$ it falls below: negatively correlated observations make the mean less variable, and the test becomes conservative.
How good is the limit for 52 or 312 observations? The script simulated the test on stationary Gaussian autoregressions with $\gamma(0) = 1$, 2,000 series for each of 30 seeds, and recorded the share of series in which it rejected. It also computed a finite-sample approximation that keeps the exact $n\operatorname{Var}(\bar X)$ of Theorem 3 and the exact $E(s^2)$ above in place of their limits: $2[1 - \Phi(t_{n-1,\,0.975}\sqrt{E(s^2)/(n\operatorname{Var}(\bar X))})]$. This approximation is a heuristic, not a theorem. The table gives medians over the 30 seeds.
| $\phi$ | Limit $\alpha(\phi)$ | Approximation, $n = 52$ | Simulated, $n = 52$ | Approximation, $n = 312$ | Simulated, $n = 312$ |
|---|---|---|---|---|---|
| −0.50 | 0.001 | 0.001 | 0.002 | 0.001 | 0.001 |
| 0.0 | 0.050 | 0.045 | 0.050 | 0.049 | 0.051 |
| 0.2 | 0.110 | 0.101 | 0.108 | 0.108 | 0.109 |
| 0.5 | 0.258 | 0.249 | 0.255 | 0.256 | 0.259 |
| 0.66 | 0.375 | 0.370 | 0.379 | 0.374 | 0.379 |
| 0.9 | 0.653 | 0.667 | 0.681 | 0.656 | 0.659 |
The limit is already close at 312 observations. At $\phi = 0.66$, the lag-1 autocorrelation of Iowa's raw all-cause series, the nominal 5% test rejected a true null hypothesis in 37.9% of the simulated series of length 312 (interquartile range over the seeds 0.370 to 0.386). At $\phi = 0.9$ the standard error $s/\sqrt n$ is too small by the factor $\sqrt{1.9/0.1} = 4.36$ in large samples. Iowa's series is not an autoregression, and Worked problem 3 says what the theorem does and does not say about it. The remedy is a standard error built from the long-run variance, by a model (Autoregressive and Moving Average Models) or by resampling blocks (Resampling Dependent and Clustered Data).
Estimating the autocovariance function
From one stretch of data the autocovariances are estimated by averaging products of deviations $h$ apart: $$\hat\gamma(h) = \frac1n\sum_{t=1}^{n-h}(x_t - \bar x)(x_{t+h} - \bar x), \quad 0 \le h < n, \qquad \hat\gamma(-h) = \hat\gamma(h), \qquad \hat\rho(h) = \frac{\hat\gamma(h)}{\hat\gamma(0)}.$$ The plot of $\hat\rho(h)$ against $h$ is the correlogram. Two choices in this definition need defending: the divisor $n$, although only $n - h$ products are summed, and the use of $\bar x$ in place of the unknown mean.
Why divide by $n$. Dividing by $n - h$ looks more natural, and the next theorem shows what it would cost.
Theorem 5 (the sample autocovariance is nonnegative definite). For any data $x_1, \dots, x_n$, set $\hat\gamma(h) = 0$ for $\lvert h\rvert \ge n$. Then $\hat\gamma$ is nonnegative definite. If the $x_t$ are not all equal, every matrix $\hat\Gamma_m = [\hat\gamma(i - j)]_{i,j=1}^m$ is positive definite.
Proof. Let $d_t = x_t - \bar x$ and fix $m$. Let $D$ be the $m \times (n + m - 1)$ matrix whose $i$-th row is $i - 1$ zeros, then $d_1, \dots, d_n$, then $m - i$ zeros. Rows $i$ and $j$ overlap in the products $d_td_{t+\lvert i-j\rvert}$, so $(DD^{\mathsf T})_{ij} = \sum_{t=1}^{n-\lvert i-j\rvert}d_td_{t+\lvert i-j\rvert} = n\hat\gamma(i - j)$, which is 0 when $\lvert i - j\rvert \ge n$. Hence $\hat\Gamma_m = \frac1nDD^{\mathsf T}$ and $a^{\mathsf T}\hat\Gamma_ma = \frac1n\lVert D^{\mathsf T}a\rVert^2 \ge 0$. If the $x_t$ are not all equal, some $d_k \ne 0$; take the first such $k$. Row $i$ of $D$ has its first nonzero entry in column $k + i - 1$, a different column for each row, so the rows are linearly independent, $D^{\mathsf T}a \ne 0$ for $a \ne 0$, and the form is positive. $\square$
With the divisor $n - h$ the theorem fails. Worked problem 1 shows it for three observations. On Iowa's all-cause series, the $313 \times 313$ matrix built from the divisor $n - h$ has smallest eigenvalue −8,362, while the one built from the divisor $n$ has all eigenvalues positive. The culprit is the longest lags, where a few products are divided by a small number: at lag 312 the single product $(x_1 - \bar x)(x_{313} - \bar x)$ gives 3,960 with divisor 1 and 12.7 with divisor 313. A negative variance for some weighted sum of the series would follow from the larger matrix, which is why the divisor $n$ is the standard choice. The prediction equations of Autoregressive and Moving Average Models also need $\hat\Gamma_m$ to be invertible, which the theorem guarantees.
The bias. Neither divisor gives an unbiased estimate, because of $\bar x$.
Proposition 2 (the expectation of the sample autocovariance). Let $X_1, \dots, X_n$ have a common mean and covariances $c(s, t)$, and write $\bar c_t = \operatorname{Cov}(X_t, \bar X) = \frac1n\sum_sc(s, t)$. For $0 \le h < n$, $$E\hat\gamma(h) = \frac1n\sum_{t=1}^{n-h}\big[c(t, t + h) - \bar c_t - \bar c_{t+h} + \operatorname{Var}(\bar X)\big].$$ For white noise with variance $\sigma^2$ this is $\sigma^2(n - 1)/n$ at $h = 0$ and $-\sigma^2(n - h)/n^2$ for $h \ge 1$.
Proof. With a common mean $\mu$, $X_t - \bar X = (X_t - \mu) - (\bar X - \mu)$, so $E[(X_t - \bar X)(X_{t+h} - \bar X)] = c(t, t + h) - \operatorname{Cov}(X_{t+h}, \bar X) - \operatorname{Cov}(X_t, \bar X) + \operatorname{Var}(\bar X)$. Sum over $t$ and divide by $n$. For white noise, $c(t, t + h) = 0$ for $h \ge 1$, $\bar c_t = \sigma^2/n$ and $\operatorname{Var}(\bar X) = \sigma^2/n$, so each bracket is $-\sigma^2/n$ and there are $n - h$ of them; at $h = 0$ each bracket is $\sigma^2 - \sigma^2/n$ and there are $n$. $\square$
So the sample autocorrelations of white noise are centered slightly below zero, near $-1/n$, and those of a positively correlated series are pulled down further. For the autoregression with $\phi = 0.6$, $\gamma(0) = 1$ and $n = 52$, the script computed the exact expectation (the formula in matrix form, $\frac1n\operatorname{tr}$ of the lag-$h$ band of $C\Gamma C$, with $C$ the centering matrix and $\Gamma$ the covariance matrix) and checked it on 2,000 series for each of 30 seeds (median of the per-seed averages):
| Lag $h$ | $\gamma(h)$ | $\frac{n-h}{n}\gamma(h)$ | Exact $E\hat\gamma(h)$ | Simulated |
|---|---|---|---|---|
| 1 | 0.600 | 0.588 | 0.515 | 0.514 |
| 2 | 0.360 | 0.346 | 0.273 | 0.273 |
| 5 | 0.078 | 0.070 | 0.001 | 0.000 |
| 10 | 0.006 | 0.005 | −0.057 | −0.056 |
At lag 0 the exact expectation is 0.926, as Theorem 3 predicted. The factor $(n - h)/n$ accounts for little of the bias; centering at $\bar x$ accounts for most of it, and it is of order $\operatorname{Var}(\bar X)$, about $\sigma_\infty^2/n$, which is large when the dependence is strong. At lag 10 the true autocovariance is 0.006 and the estimate is centered at −0.057: in a short, strongly correlated series, small negative sample autocorrelations at moderate lags are expected even when the true ones are positive.
The sample autocorrelations of independent noise
The first use of the correlogram is to ask whether a series, or the residuals of a model, could be noise. That needs the distribution of $\hat\rho(h)$ when the series is noise. The sample autocorrelation is built from products $Z_tZ_{t+h}$, which are not independent of one another: $Z_tZ_{t+1}$ and $Z_{t+1}Z_{t+2}$ share $Z_{t+1}$. They are, however, dependent only over a short range, and that is enough for a central limit theorem.
A sequence $\{Y_t\}$ is $m$-dependent if for every $t$ the variables $\{Y_s : s \le t\}$ are independent of $\{Y_s : s \ge t + m + 1\}$. Independent sequences are 0-dependent; a moving average $Z_t + \theta Z_{t-1}$ of independent noise is 1-dependent; the products $Z_tZ_{t+h}$ are $h$-dependent.
Lemma 1 (approximation). Let $W_n$ and $Y_{n,k}$ be random variables. Suppose that for each $k$, $Y_{n,k} \xrightarrow{d} N(0, \tau_k^2)$ as $n \to \infty$; that $\tau_k^2 \to \tau^2 > 0$ as $k \to \infty$; and that for every $\varepsilon > 0$, $\lim_{k\to\infty}\limsup_{n\to\infty}P(\lvert W_n - Y_{n,k}\rvert > \varepsilon) = 0$. Then $W_n \xrightarrow{d} N(0, \tau^2)$.
Proof. Fix $x$ and $\varepsilon > 0$, and take $k$ large enough that $\tau_k > 0$. The event $W_n \le x$ is contained in the union of $Y_{n,k} \le x + \varepsilon$ and $\lvert W_n - Y_{n,k}\rvert > \varepsilon$, so $$P(W_n \le x) \le P(Y_{n,k} \le x + \varepsilon) + P(\lvert W_n - Y_{n,k}\rvert > \varepsilon).$$ Let $n \to \infty$: $\limsup_nP(W_n \le x) \le \Phi((x + \varepsilon)/\tau_k) + \limsup_nP(\lvert W_n - Y_{n,k}\rvert > \varepsilon)$. Let $k \to \infty$: the bound becomes $\Phi((x + \varepsilon)/\tau)$. Let $\varepsilon \to 0$: $\limsup_nP(W_n \le x) \le \Phi(x/\tau)$. In the same way, $Y_{n,k} \le x - \varepsilon$ implies that $W_n \le x$ or $\lvert W_n - Y_{n,k}\rvert > \varepsilon$, so $P(W_n \le x) \ge P(Y_{n,k} \le x - \varepsilon) - P(\lvert W_n - Y_{n,k}\rvert > \varepsilon)$, and the same three limits give $\liminf_nP(W_n \le x) \ge \Phi(x/\tau)$. $\square$
Theorem 6 (central limit theorem for $m$-dependent sequences). Let $\{Y_t\}$ be strictly stationary and $m$-dependent with $E(Y_t) = 0$, $E(Y_t^2) < \infty$ and autocovariances $\gamma_Y$, and suppose $\sigma_\infty^2 = \sum_{\lvert h\rvert \le m}\gamma_Y(h) > 0$. Then $$\frac{1}{\sqrt n}\sum_{t=1}^nY_t \xrightarrow{d} N(0, \sigma_\infty^2).$$
Here is the idea. Cut the time axis into long blocks of length $k$ separated by short gaps of length $m$. Blocks separated by gaps are independent and identically distributed, so the ordinary central limit theorem applies to their sums. The gaps carry a fraction of about $m/(k + m)$ of the terms, which becomes negligible as $k$ grows, and Lemma 1 lets $k$ grow after $n$.
Proof. Fix $k > m$, let $r = \lfloor n/(k + m)\rfloor$, and divide $1, \dots, r(k + m)$ into $r$ consecutive segments of length $k + m$. In segment $j$ let $U_j$ be the sum of the first $k$ terms (the long block) and $V_j$ the sum of the last $m$ (the gap); let $R_n$ be the sum of the fewer than $k + m$ terms after $r(k + m)$. Then $\sum_{t=1}^nY_t = \sum_jU_j + \sum_jV_j + R_n$.
The long blocks are i.i.d. The last index of block $j$ is $(j - 1)(k + m) + k$ and the first of block $j + 1$ is $j(k + m) + 1$, a difference of $m + 1$. So $U_r$ depends only on variables at least $m + 1$ steps after every variable in $U_1, \dots, U_{r-1}$, and $m$-dependence makes $U_r$ independent of $(U_1, \dots, U_{r-1})$. Repeating the argument for $U_{r-1}$ and so on shows that $U_1, \dots, U_r$ are mutually independent. By strict stationarity they have a common distribution, with mean 0 and variance $v_k = \operatorname{Var}(Y_1 + \dots + Y_k) = \sum_{\lvert h\rvert < k}(k - \lvert h\rvert)\gamma_Y(h)$, by part 1 of Theorem 3. Since $\gamma_Y(h) = 0$ for $\lvert h\rvert > m$ (independence), $v_k/k = \sum_{\lvert h\rvert \le m}(1 - \lvert h\rvert/k)\gamma_Y(h) \to \sigma_\infty^2$ as $k \to \infty$, so $v_k > 0$ for large $k$.
The long blocks satisfy a central limit theorem. Let $Y_{n,k} = n^{-1/2}\sum_{j=1}^rU_j = \sqrt{r/n}\cdot r^{-1/2}\sum_{j=1}^rU_j$. As $n \to \infty$, $r \to \infty$, the Lindeberg–Lévy theorem gives $r^{-1/2}\sum_jU_j \xrightarrow{d} N(0, v_k)$, and $r/n \to 1/(k + m)$, so by Slutsky's theorem $Y_{n,k} \xrightarrow{d} N(0, \tau_k^2)$ with $\tau_k^2 = v_k/(k + m)$. As $k \to \infty$, $\tau_k^2 = \frac{k}{k + m}\cdot\frac{v_k}{k} \to \sigma_\infty^2$.
The gaps are negligible. The gap sums $V_1, \dots, V_r$ are mutually independent by the same argument, since consecutive gaps are $k + 1 > m + 1$ indices apart, each with mean 0 and variance $v_m$. So $\operatorname{Var}(\sum_jV_j) = rv_m$. Every covariance is at most $\gamma_Y(0)$ in absolute value (Theorem 1), so $\operatorname{Var}(R_n) \le (k + m)^2\gamma_Y(0)$. Using $\operatorname{Var}(A + B) \le 2\operatorname{Var}(A) + 2\operatorname{Var}(B)$, $$E\Big[\Big(\frac{1}{\sqrt n}\sum_{t=1}^nY_t - Y_{n,k}\Big)^2\Big] \le \frac{2rv_m}{n} + \frac{2(k + m)^2\gamma_Y(0)}{n} \le \frac{2v_m}{k + m} + \frac{2(k + m)^2\gamma_Y(0)}{n}.$$ By Chebyshev's inequality, $\limsup_nP(\lvert n^{-1/2}\sum_tY_t - Y_{n,k}\rvert > \varepsilon) \le 2v_m/((k + m)\varepsilon^2)$, which tends to 0 as $k \to \infty$. Lemma 1 finishes the proof. $\square$
The theorem has a long history. Hoeffding and Robbins (1948) proved central limit theorems "for sequences of dependent random variables of a certain special type which occurs frequently in mathematical statistics", in the words of their abstract. Durrett (2019, Example 8.3.3) obtains the $m$-dependent case as an example of a central limit theorem for stationary sequences proved through martingales (his Theorem 8.3.2). The variance $\sigma_\infty^2$ is the long-run variance of Theorem 1 of Resampling Dependent and Clustered Data, which for an $m$-dependent sequence is a finite sum. Theorem 6 is one of the central limit theorems for sequences whose dependence dies away with distance that the section on dependence of The Central Limit Theorem mentions.
Theorem 7 (sample autocorrelations of independent noise). Let $Z_t \sim \mathrm{IID}(\mu, \sigma^2)$ with $0 < \sigma^2 < \infty$, and fix $H \ge 1$. Then $$\sqrt n\,\big(\hat\rho(1), \dots, \hat\rho(H)\big) \xrightarrow{d} N_H(\mathbf 0, I_H).$$ In words: for large $n$, the sample autocorrelations of independent noise at a fixed set of lags are approximately independent normal variables with mean 0 and variance $1/n$.
Here is the idea. Up to negligible terms, $\sqrt n\,\hat\rho(h)$ is $\frac{1}{\sigma^2\sqrt n}\sum_tZ_tZ_{t+h}$ for noise with mean 0. Any linear combination of these products over $h = 1, \dots, H$ is an $H$-dependent stationary sequence with no correlation between different times, so Theorem 6 applies, and the Cramér–Wold device turns the linear combinations into the joint statement.
Proof. Every $\hat\rho(h)$ is unchanged when a constant is added to all the $Z_t$, so take $\mu = 0$.
Step 1: products with the true mean. Let $\tilde\gamma(h) = \frac1n\sum_{t=1}^{n-h}Z_tZ_{t+h}$. Fix real $c_1, \dots, c_H$ and set $Y_t = \sum_{h=1}^Hc_hZ_tZ_{t+h}$. The sequence $\{Y_t\}$ is strictly stationary, because it is the same function of the shifted i.i.d. sequence at every $t$, and it is $H$-dependent, because $Y_t$ depends on $Z_t, \dots, Z_{t+H}$ only. It has mean 0. In $E(Y_tY_{t+j})$ for $j \ge 1$, every term is $E(Z_tZ_{t+h}Z_{t+j}Z_{t+j+h'})$ with $h, h' \ge 1$; the index $t$ is smaller than the other three, so $Z_t$ is independent of the other factors and the term is $E(Z_t)E(\cdots) = 0$. So $\gamma_Y(j) = 0$ for $j \ne 0$. For the variance, $E(Y_t^2) = \sum_{h,h'}c_hc_{h'}E(Z_t^2Z_{t+h}Z_{t+h'})$; when $h \ne h'$ the larger of $t + h$ and $t + h'$ appears once and the term vanishes, so $E(Y_t^2) = \sigma^4\sum_hc_h^2$. Theorem 6 gives $n^{-1/2}\sum_{t=1}^nY_t \xrightarrow{d} N(0, \sigma^4\sum_hc_h^2)$. This differs from $\sqrt n\sum_hc_h\tilde\gamma(h) = n^{-1/2}\sum_hc_h\sum_{t=1}^{n-h}Z_tZ_{t+h}$ by $n^{-1/2}\sum_hc_h\sum_{t=n-h+1}^nZ_tZ_{t+h}$, a sum of at most $H^2$ products divided by $\sqrt n$, whose second moment is at most $H^2\sigma^4\big(\sum_h\lvert c_h\rvert\big)^2/n \to 0$. By the converging-together lemma (Lemma 3 of Convergence in Distribution and the Proof of the Central Limit Theorem), $\sqrt n\sum_hc_h\tilde\gamma(h) \xrightarrow{d} N(0, \sigma^4\sum_hc_h^2)$.
Step 2: centering at $\bar Z$. Expanding the products, $$\hat\gamma(h) - \tilde\gamma(h) = -\bar Z\cdot\frac1n\sum_{t=1}^{n-h}(Z_t + Z_{t+h}) + \frac{n - h}{n}\bar Z^2.$$ Here $\sqrt n\bar Z$ has variance $\sigma^2$, so it is bounded in probability, and $\frac1n\sum_{t=1}^{n-h}(Z_t + Z_{t+h})$ and $\bar Z$ have variances at most $4\sigma^2/n$ and tend to 0 in probability. By Lemma 4 of the same article (bounded times vanishing), $\sqrt n(\hat\gamma(h) - \tilde\gamma(h)) \to 0$ in probability, and so $\sqrt n\sum_hc_h\hat\gamma(h)$ has the limit of Step 1.
Step 3: the denominator. $\hat\gamma(0) = \frac1n\sum_tZ_t^2 - \bar Z^2 \to \sigma^2$ in probability, by the weak law of large numbers for i.i.d. variables with a finite mean (The Laws of Large Numbers) applied to $Z_t^2$, and $\bar Z^2 \to 0$. Slutsky's theorem gives $$\sqrt n\sum_hc_h\hat\rho(h) = \frac{\sqrt n\sum_hc_h\hat\gamma(h)}{\hat\gamma(0)} \xrightarrow{d} N\Big(0, \sum_hc_h^2\Big).$$
Step 4: the joint limit. The right-hand side is the distribution of $\sum_hc_hW_h$ with $W \sim N_H(\mathbf 0, I_H)$. Since this holds for every $c$, the Cramér–Wold device (Durrett, 2019, Theorem 3.10.6) gives the joint limit. $\square$
The proof needs only a finite variance, not a finite fourth moment, because the products $Z_tZ_{t+h}$ of independent factors have variance $\sigma^4$ whatever the tails of $Z_t$. It does need independence, and white noise is not enough. Take $Z_t = W_tW_{t-1}$ with $W_t$ independent standard normal. This is white noise with variance 1: $E(Z_t) = 0$, and $E(Z_tZ_{t+j}) = 0$ for $j \ge 1$ because the earliest $W$ in the product appears once. But $Z_tZ_{t+1} = W_{t-1}W_t^2W_{t+1}$ has variance $E(W_{t-1}^2)E(W_t^4)E(W_{t+1}^2) = 3$, not $\sigma^4 = 1$. These products form a stationary 2-dependent sequence with no correlation between different times (again the earliest $W$ appears once), so Theorem 6 applies to them. Steps 2 and 3 carry over: $\operatorname{Var}(\sqrt n\bar Z) = 1$ because the $Z_t$ are uncorrelated, and the $Z_t^2 = W_t^2W_{t-1}^2$ are 1-dependent with finite variance, so their average tends to 1 by Chebyshev's inequality. Hence $\sqrt n\,\hat\rho(1) \xrightarrow{d} N(0, 3)$. The noise band is then too narrow by the factor $\sqrt3$ at lag 1. In the script, series of this kind of length 313 (2,000 for each of 30 seeds) gave a median $n\operatorname{Var}\hat\rho(1)$ of 2.79, and 0.242 of their lag-1 autocorrelations fell outside the band.
What this gives in practice. If a series is independent noise, about 95% of its sample autocorrelations should lie within $\pm1.96/\sqrt n$, the band drawn on most correlograms. The script checked this on independent standard normal noise and on centered exponential noise (skewness 2), 2,000 series of each length for each of 30 seeds; the table in the next section reports the median of the per-seed results. At $n = 313$ the share of $\hat\rho(h)$ outside the band, over lags 1 to 20, was 0.046 for normal noise and 0.043 for exponential noise. At $n = 52$ it was 0.026 and 0.021: the band is conservative for short series. The reason is the spread of $\hat\rho(h)$, which is below $1/\sqrt n$ in short series and falls further as the lag grows. For normal noise of length 52, $n\operatorname{Var}\hat\rho(h)$ had median 0.95 at lag 1, 0.78 at lag 10 and 0.60 at lag 20; at length 313 the values were 1.00, 0.97 and 0.93. (The centering of $\hat\rho(h)$ near $-1/n$, Proposition 2, works the other way: shifting the center away from 0 makes values outside the band more likely, not less.)
Bartlett's formula. For a stationary series that is not noise, the sample autocorrelations are still approximately normal in large samples, with a covariance matrix given by a formula named after Bartlett (1946), whose paper we did not read. Boshnakov and Ravagli (2021) state it as follows, citing Francq and Zakoïan (2010, Theorem 1.1; Boshnakov and Ravagli give the year as 2011): if $X_t = \sum_{l=-\infty}^\infty\psi_l\varepsilon_{t-l}$ with $\varepsilon_t$ i.i.d. with a finite fourth moment and $\sum_l\lvert\psi_l\rvert < \infty$, then $\sqrt n(\hat\rho(1) - \rho(1), \dots, \hat\rho(m) - \rho(m))$ tends to $N_m(\mathbf 0, W)$ with $$w_{ij} = \sum_{k=-\infty}^\infty\big[\rho(k + i)\rho(k + j) + \rho(k - i)\rho(k + j) + 2\rho(i)\rho(j)\rho(k)^2 - 2\rho(i)\rho(k)\rho(k + j) - 2\rho(j)\rho(k)\rho(k + i)\big].$$ They add that for such linear processes the fourth moment can be relaxed to a finite variance when $\sum_l\lvert l\rvert\psi_l^2 < \infty$. The script confirms numerically that this sum equals the product form $$w_{ij} = \sum_{k=1}^\infty\big[\rho(k + i) + \rho(k - i) - 2\rho(i)\rho(k)\big]\big[\rho(k + j) + \rho(k - j) - 2\rho(j)\rho(k)\big]$$ for the two processes below (to within $10^{-12}$). For noise, $\rho(k) = 0$ for $k \ne 0$, so $w_{ij} = 1$ when $i = j$ and 0 otherwise, which is Theorem 7. For the autoregression with $\phi = 0.6$ the product form gives $w_{11} = 1 - \phi^2 = 0.64$. For a moving average of order $q$ and a lag $i > q$, $\rho(i) = 0$ and $\rho(k + i) = 0$ for every $k \ge 1$, so the product form reduces to $w_{ii} = \sum_{k\ge1}\rho(k - i)^2 = 1 + 2\sum_{k=1}^q\rho(k)^2$. Beyond the order of a moving average, the sample autocorrelations are more variable than those of noise. We checked the formula, not the conditions, by simulation at $n = 1{,}000$, 2,000 series for each of 30 seeds:
| Process | $w_{11}$ | $n\operatorname{Var}\hat\rho(1)$, simulated | $w_{22}$ | $n\operatorname{Var}\hat\rho(2)$, simulated |
|---|---|---|---|---|
| AR(1), $\phi = 0.6$ | 0.640 | 0.645 | 1.331 | 1.347 |
| MA(1), $\theta = 0.8$ | 0.513 | 0.510 | 1.476 | 1.466 |
The moving average $X_t = Z_t + 0.8Z_{t-1}$ has $\rho(1) = 0.8/1.64 = 0.4878$. A positive autocorrelation at lag 1 makes the sample autocorrelation at lag 2 more variable than $1/n$, by the factor 1.476 here. Reading a correlogram of a correlated series against the noise band $\pm1.96/\sqrt n$ therefore overstates the evidence at the lags beyond the true correlations.
Portmanteau tests
Looking at 20 sample autocorrelations one at a time invites a false alarm: even for noise, about one in twenty falls outside the band. A portmanteau test combines them into one statistic. Box and Pierce (1970) used $$Q_{\mathrm{BP}} = n\sum_{h=1}^H\hat\rho(h)^2,$$ and Ljung and Box (1978) showed in their paper that "a substantially improved approximation results from a simple modification of this test" (their abstract), the statistic $$Q_{\mathrm{LB}} = n(n + 2)\sum_{h=1}^H\frac{\hat\rho(h)^2}{n - h}.$$ Hyndman and Athanasopoulos (2021, §5.4) give both in this form, call the Ljung–Box test "more accurate", and suggest $H = 10$ for non-seasonal data and $H = 2m$ for seasonal data with period $m$, capped at $n/5$.
Corollary (the reference distribution for noise). For $Z_t \sim \mathrm{IID}(\mu, \sigma^2)$ with finite $\sigma^2 > 0$ and fixed $H$, both $Q_{\mathrm{BP}}$ and $Q_{\mathrm{LB}}$ converge in distribution to the chi-squared distribution with $H$ degrees of freedom.
Proof. $Q_{\mathrm{BP}} = \lVert V_n\rVert^2$ with $V_n = \sqrt n(\hat\rho(1), \dots, \hat\rho(H))$. By Theorem 7, $V_n \xrightarrow{d} W \sim N_H(\mathbf 0, I_H)$. Durrett (2019, §3.10) defines convergence in distribution of random vectors by $E f(V_n) \to E f(W)$ for every bounded continuous $f$; for such $f$, $f(\lVert\cdot\rVert^2)$ is again bounded and continuous, so $\lVert V_n\rVert^2 \xrightarrow{d} \lVert W\rVert^2$, a sum of $H$ squared independent standard normal variables, which is chi-squared with $H$ degrees of freedom (The Chi-Squared and t Distributions). For the other statistic, $Q_{\mathrm{LB}} - Q_{\mathrm{BP}} = \sum_{h=1}^H n\hat\rho(h)^2\cdot\frac{h + 2}{n - h}$; each $n\hat\rho(h)^2$ converges in distribution and so is bounded in probability, and each factor $(h + 2)/(n - h) \to 0$, so the difference tends to 0 in probability and the converging-together lemma applies. $\square$
When the series is the residuals of a fitted model. Box and Pierce (1970) showed that the residual autocorrelations of a fitted autoregressive moving average model are approximately "a singular linear transformation of the autocorrelations of the errors" (their abstract), so that they are less variable than the noise band suggests. Their abstract continues: "Failing to allow for this results in a tendency to overlook evidence of lack of fit." The standard correction subtracts the number of fitted coefficients from the degrees of freedom: Hyndman and Athanasopoulos (2021, §9.7) use $H - K$ degrees of freedom with $K = p + q$ for a model with $p$ autoregressive and $q$ moving average coefficients. That correction is cited, not proved here; Autoregressive and Moving Average Models applies it.
How close is the chi-squared reference at 52 and 313 observations? The script applied both tests at $H = 10$ and nominal level 5% to the same simulated noise series as above (medians of the per-seed results over 30 seeds):
| Noise | $n$ | Mean $\hat\rho(1)$ | $-1/n$ | $n\operatorname{Var}\hat\rho(1)$ | Share outside the band, lags 1–20 | Box–Pierce rejects | Ljung–Box rejects |
|---|---|---|---|---|---|---|---|
| normal | 52 | −0.019 | −0.019 | 0.95 | 0.026 | 0.033 | 0.066 |
| normal | 313 | −0.002 | −0.003 | 1.00 | 0.046 | 0.047 | 0.054 |
| exponential | 52 | −0.018 | −0.019 | 0.88 | 0.021 | 0.023 | 0.050 |
| exponential | 313 | −0.003 | −0.003 | 0.97 | 0.043 | 0.044 | 0.051 |
For white noise, Proposition 2 gives $E\hat\gamma(1)/E\hat\gamma(0) = [-\sigma^2(n - 1)/n^2]/[\sigma^2(n - 1)/n] = -1/n$ exactly, which is $-1/52 = -0.0192$ at the shorter length. That is a ratio of expectations, not the expectation of the ratio $\hat\rho(1)$, but the simulated mean of $\hat\rho(1)$ matches it. In these simulations the Box–Pierce test rejected too rarely at $n = 52$ (0.033 and 0.023) and the Ljung–Box test was closer to 5% for exponential noise and too liberal for normal noise (0.066). At $n = 313$ both were within a point of 5%. The factor $(n + 2)/(n - h)$ in $Q_{\mathrm{LB}}$ inflates each term, and more at longer lags. That compensates for the spread of $\hat\rho(h)$, which in short series is below $1/\sqrt n$ and falls with the lag, as the values of $n\operatorname{Var}\hat\rho(h)$ at lags 1, 10 and 20 in the previous section show. The nonzero mean near $-1/n$ is not the reason: it adds to $E\hat\rho(h)^2$ and so works against the need for inflation. These are observations for these two noise distributions, these lengths and $H = 10$.
Trend, seasonality and differencing
Most series are not stationary as they stand. The classical decomposition writes $$X_t = m_t + s_t + Y_t,$$ with $m_t$ a slowly changing trend, $s_t$ a seasonal component with period $d$ ($s_{t+d} = s_t$, and $\sum_{j=1}^ds_j = 0$ so that the season does not shift the level) and $Y_t$ a stationary remainder. There are two ways to deal with $m_t$ and $s_t$: to estimate the components and subtract them, or to remove them by differencing.
Estimating them. A trend can be fitted as a polynomial in $t$ and a season as a sum of sines and cosines with the seasonal period, by least squares. This is a linear model, fitted as in The Geometry of Least Squares. For Iowa we use the model of Overdispersion and Quasi-Likelihood on the linear scale: with weeks numbered $t = 0, \dots, 312$ and $P = 365.25/7 \approx 52.18$ weeks, $$x_t = \beta_0 + \beta_1\frac{t - 156}{P} + \sum_{j=1}^3\Big[\gamma_j\cos\frac{2\pi jt}{P} + \delta_j\sin\frac{2\pi jt}{P}\Big] + y_t.$$ The period is a year, not 52 weeks, because MMWR years have 52 or 53 weeks; the harmonics follow the calendar, where a fixed lag of 52 weeks drifts by a week over a 53-week year.
Removing them by differencing. Here is the idea: differencing a polynomial lowers its degree, and differencing at the seasonal lag subtracts a periodic function from itself.
Proposition 3 (what differencing does).
- If $m_t$ is a polynomial in $t$ of degree $k \ge 1$, then $\nabla m_t$ is a polynomial of degree $k - 1$; so $\nabla^{k+1}m_t = 0$.
- If $s_t$ has period $d$, then $\nabla_ds_t = 0$; and $\nabla_d(a + bt) = bd$, a constant.
- If $Z_t \sim \mathrm{WN}(0, \sigma^2)$, then $\nabla Z_t$ is stationary with $\rho(1) = -\frac12$ and $\rho(h) = 0$ for $\lvert h\rvert \ge 2$; and $\nabla_dZ_t$ has $\rho(d) = -\frac12$ and $\rho(h) = 0$ for the other $h \ne 0$.
Proof. (1) By the binomial theorem, $t^k - (t - 1)^k = kt^{k-1} + (\text{terms of lower degree})$, and lower powers drop at least one degree, so the leading term of $\nabla m_t$ is $kb_kt^{k-1}$ with $b_k \ne 0$ the leading coefficient of $m_t$. Applying $\nabla$ $k$ times leaves the constant $k!\,b_k$, and once more gives 0. (2) $\nabla_ds_t = s_t - s_{t-d} = 0$ by periodicity, and $(a + bt) - (a + b(t - d)) = bd$. (3) $\operatorname{Var}(Z_t - Z_{t-1}) = 2\sigma^2$, $\operatorname{Cov}(Z_t - Z_{t-1}, Z_{t+1} - Z_t) = -\operatorname{Var}(Z_t) = -\sigma^2$, and differences two or more steps apart share no $Z$; so $\rho(1) = -\sigma^2/(2\sigma^2) = -\frac12$. The lag-$d$ case is the same with $d$ in place of 1. $\square$
Part 3 is the signature of over-differencing: differencing a series that was already stationary noise creates a lag-1 autocorrelation of $-\frac12$, and an autocorrelation near $-\frac12$ after differencing suggests the difference was not needed. Differencing a random walk $S_t$ (the example above) gives back its noise $Z_t$, which is why differencing is the natural remedy for a stochastic trend. Autoregressive and Moving Average Models builds the models (called integrated) that difference first and then model the result.
Iowa's weekly deaths, measured
The script computed the sample autocorrelations of five versions of each series: the raw counts; the residuals after fitting the annual cycle alone (the six harmonic terms and an intercept); the residuals after fitting the trend and the cycle; the first difference; and the lag-52 difference. The first difference has 312 values and the lag-52 difference 261. Lags are in weeks.
All causes:
| Series | 1 | 2 | 4 | 13 | 26 | 52 |
|---|---|---|---|---|---|---|
| raw | 0.66 | 0.64 | 0.52 | 0.06 | −0.52 | 0.50 |
| cycle removed | 0.28 | 0.28 | 0.13 | 0.18 | 0.00 | 0.04 |
| trend and cycle removed | 0.18 | 0.18 | 0.01 | 0.10 | −0.08 | −0.02 |
| first difference | −0.48 | 0.02 | −0.14 | 0.08 | −0.08 | 0.03 |
| lag-52 difference | 0.24 | 0.23 | 0.08 | 0.09 | −0.05 | −0.48 |
Cancer:
| Series | 1 | 2 | 4 | 13 | 26 | 52 |
|---|---|---|---|---|---|---|
| raw | 0.01 | 0.10 | −0.07 | 0.01 | 0.01 | 0.08 |
| cycle removed | −0.03 | 0.07 | −0.10 | 0.02 | 0.01 | 0.05 |
| trend and cycle removed | −0.03 | 0.07 | −0.11 | 0.02 | 0.01 | 0.05 |
| first difference | −0.55 | 0.13 | −0.04 | 0.09 | 0.00 | 0.06 |
| lag-52 difference | −0.01 | 0.02 | −0.15 | 0.01 | 0.04 | −0.34 |
Over the 312 weeks used in Resampling Dependent and Clustered Data the raw lag-52 value is 0.51; over all 313 it is 0.504.
All causes, raw. The correlogram is a slow wave with a period of a year, the autocorrelation of the random-phase cycle above blended with a short-range part. If a cycle independent of the rest of the series carried a share $c$ of the variance, its contribution at lags 26 and 52 would be about $-c$ and $+c$; the fitted cycle takes 0.54 of the sum of squares, and the raw values at those lags are −0.52 and 0.50. Read as a stationary process, the series has autocorrelations that never decay, so the summability assumed by Theorem 1 of Resampling Dependent and Clustered Data fails. That does not make the mean more variable. For the random-phase cycle, $\bar X = \frac An\operatorname{Re}\big(e^{iU}S\big)$ with $S = \sum_{t=1}^ne^{i\omega t}$, and since $E\cos^2(U + c) = \frac12$ for every constant $c$, $$\operatorname{Var}(\bar X) = \frac{A^2\lvert S\rvert^2}{2n^2} \le \frac{A^2}{2n^2\sin^2(\omega/2)},$$ using $\lvert S\rvert = \lvert\sin(n\omega/2)/\sin(\omega/2)\rvert$ (the sum of a geometric series), for $\omega$ not a multiple of $2\pi$. So the cycle contributes only of order $1/n^2$ to $\operatorname{Var}(\bar X)$, and nothing to the limit of $n\operatorname{Var}(\bar X)$: the sums $\sum_{\lvert h\rvert<n}(1 - \lvert h\rvert/n)\gamma(h)$ of the cycle's autocovariances tend to 0 although $\sum_h\gamma(h)$ itself does not converge. What the cycle does disturb is everything built from the autocovariances over short lags, such as the block bootstrap with blocks shorter than a year, which sees a large, slowly oscillating dependence. Read as a fixed cycle plus a remainder, the series is not stationary in its mean, and the cycle should be modeled.
The fitted cycle and trend. Removing the cycle alone leaves a lag-1 autocorrelation of 0.28, and removing the trend as well leaves 0.18, with standard deviation 29.6 for the remainder; Resampling Dependent and Clustered Data found the same values on 312 weeks. The trend and the cycle together take 0.60 of the sum of squares. The remainder is not noise. Its first three autocorrelations are 0.18, 0.18 and 0.17, the Ljung–Box statistic at 10 lags is 38.6, far above the 5% point 18.31 of the chi-squared distribution with 10 degrees of freedom, and 8 of the 52 lags lie outside the band, against about 2.6 expected for noise. The degrees of freedom are not corrected for the eight fitted regression coefficients, so the test is approximate; at 38.6 the conclusion does not depend on that. Epidemics that raise deaths for a few weeks at a time produce this short-range dependence, which Overdispersion and Quasi-Likelihood also found in its Pearson residuals (0.16 at lag 1, for a model of the log mean). Modeling the remainder is the job of Autoregressive and Moving Average Models.
Differencing. The first difference of the all-cause series has lag-1 autocorrelation −0.48, close to the $-\frac12$ of Proposition 3 for differenced noise, because the remainder is only weakly correlated and the first difference removes little of the cycle in one week. The lag-52 difference removes most of the cycle but leaves lag-1 autocorrelation 0.24 and creates −0.48 at lag 52, again close to $-\frac12$. The lag-52 difference also inherits the calendar problem: after the 53-week year 2014 it compares weeks that are a year and a week apart.
Cancer. The raw cancer series looks like noise: the Ljung–Box statistic is 12.1 at 10 lags ($p = 0.28$) and 72.0 at 62 lags ($p = 0.18$), and 3 of the 52 lags lie outside the band. Removing the trend and cycle changes little (Ljung–Box 15.1 at 10 lags, $p = 0.13$), since they take only 0.04 of the sum of squares. Differencing it is a mistake that Proposition 3 predicts exactly: the first difference has lag-1 autocorrelation −0.55, and the lag-52 difference has −0.34 at lag 52, both created by the differencing.
The Ljung–Box statistics, at 10 lags and at 62 lags (the cap $n/5$ of Hyndman and Athanasopoulos's rule, since $2m = 104$ exceeds it), and the number of the 52 lags outside the band:
| Series | Ljung–Box, 10 lags | Ljung–Box, 62 lags | Lags outside the band, of the 52 |
|---|---|---|---|
| all causes, raw | 771.3 | 2,922.6 | 43 |
| all causes, trend and cycle removed | 38.6 | 138.7 | 8 |
| cancer, raw | 12.1 | 72.0 | 3 |
| cancer, trend and cycle removed | 15.1 | 68.4 | 2 |
These are measurements of two series over six years. They say that all-cause deaths in Iowa have a strong annual cycle and weak short-range dependence around it, and that cancer deaths, a hundred-odd a week, behave like independent counts around a nearly constant mean, as Overdispersion and Quasi-Likelihood found for their variance.
What is proved here, what is cited, what the experiment shows
Proved here. That strict stationarity with finite variances implies weak stationarity, and the converse for Gaussian processes (Proposition 1); that autocovariance functions are even, bounded by $\gamma(0)$ and nonnegative definite (Theorem 1), and that every even nonnegative definite function is the autocovariance of a stationary Gaussian process (Theorem 2, given Kolmogorov's extension theorem); the bound $\lvert\rho(1)\rvert \le \frac12$ for a process correlated at lag 1 only; the variance of a mean and the expected sample variance for any means and covariances (Theorem 3); the large-sample level of the t-test for a Gaussian autoregression (Theorem 4); the nonnegative and positive definiteness of the sample autocovariance with divisor $n$ (Theorem 5) and its exact expectation (Proposition 2); the approximation lemma (Lemma 1) and the central limit theorem for $m$-dependent stationary sequences (Theorem 6), using the Lindeberg–Lévy theorem; the joint normal limit of the sample autocorrelations of independent noise with finite variance (Theorem 7) and the chi-squared limit of the Box–Pierce and Ljung–Box statistics for noise (Corollary); the effect of differencing on polynomials, periodic components and noise (Proposition 3).
Cited. Kolmogorov's extension theorem, the Cramér–Wold device, the definition of convergence in distribution for vectors and the martingale route to the $m$-dependent theorem (Durrett, 2019); the dependent central limit theorems of Hoeffding and Robbins (1948), at the level of their abstract; Bartlett's formula, in the statement of Boshnakov and Ravagli (2021) with its conditions as they give them (Francq and Zakoïan, 2010); the Box–Pierce and Ljung–Box statistics and the residual correction (Box and Pierce, 1970; Ljung and Box, 1978, at the level of their abstracts; Hyndman and Athanasopoulos, 2021, §5.4 and §9.7); the unit's main text (Brockwell and Davis, 2016), cited for its chapter topics only.
What the experiments show. On simulated Gaussian autoregressions, the nominal 5% t-test rejected in 0.2% to 68% of series as $\phi$ ran from −0.5 to 0.9, close to the limit of Theorem 4 already at $n = 312$. The exact bias of the sample autocovariance of an autoregression with $n = 52$ was confirmed by simulation. For normal and exponential noise of lengths 52 and 313, the share of sample autocorrelations outside the band and the rejection rates of the portmanteau tests were near 5% at 313 and off by up to two points at 52. Bartlett's formula matched simulated variances for one autoregression and one moving average. On Iowa's weekly deaths, the all-cause series is dominated by an annual cycle and keeps a lag-1 autocorrelation of 0.18 around a fitted cycle and trend, while the cancer series is indistinguishable from noise by these tests. These are observations of these processes, sizes and data.
Worked problems
The four problems are posed here, on small exact cases and on the article's data and processes; the article's script checks each number.
Problem 1 (a sample autocovariance that is not nonnegative definite). For the series 1, 2, 3, compute the sample autocovariances with divisor $n - h$ and with divisor $n$, form the $3 \times 3$ matrices $[\hat\gamma(i - j)]$, and decide whether each is positive semidefinite.
Solution. With divisor $n - h$ the matrix has a negative eigenvalue, $-\frac13$; with divisor $n$ its eigenvalues are $\frac13$, $\frac23$ and 1. The mean is 2 and the deviations are $-1, 0, 1$. The sums of products at lags 0, 1 and 2 are $1 + 0 + 1 = 2$, $(-1)(0) + (0)(1) = 0$ and $(-1)(1) = -1$. Divisor $n - h$: $\frac23$, 0 and $\frac{-1}{1} = -1$. Divisor $n$: $\frac23$, 0 and $-\frac13$. Both matrices have the form $\begin{pmatrix}a & 0 & c\\ 0 & a & 0\\ c & 0 & a\end{pmatrix}$, whose eigenvalues are $a$ (eigenvector $(0, 1, 0)$) and $a \pm c$ (eigenvectors $(1, 0, \pm1)$). With $a = \frac23$, $c = -1$ they are $\frac23$, $-\frac13$ and $\frac53$; with $c = -\frac13$ they are $\frac23$, $\frac13$ and 1. The negative eigenvalue means that the weighted sum $X_1 + X_3$ would have "variance" $2a + 2c = -\frac23$ under the divisor $n - h$, which no process allows (Theorem 1). Theorem 5 guarantees the divisor-$n$ matrix is positive definite, since the data are not constant.
Problem 2 (the largest lag-1 correlation of a moving average). For $X_t = Z_t + \theta Z_{t-1}$, find the range of $\rho(1)$ as $\theta$ runs over the real line, and the value of $\theta$ that attains each end.
Solution. The range is $[-\frac12, \frac12]$, with $\frac12$ at $\theta = 1$ and $-\frac12$ at $\theta = -1$. From the example in the section on stationarity, $\rho(1) = \theta/(1 + \theta^2)$. Since $(1 - \lvert\theta\rvert)^2 \ge 0$, $1 + \theta^2 \ge 2\lvert\theta\rvert$, so $\lvert\rho(1)\rvert \le \frac12$, with equality exactly when $\lvert\theta\rvert = 1$. Every value in between is attained, because $\theta/(1 + \theta^2)$ is continuous on $[-1, 1]$. This agrees with the bound found from nonnegative definiteness in the section on autocovariances: a process correlated at lag 1 only cannot have $\lvert\rho(1)\rvert > \frac12$, whatever its construction. The value 0.4878 for $\theta = 0.8$ in the Bartlett table lies just inside it.
Problem 3 (a t-test at Iowa's autocorrelation). Iowa's raw all-cause series has lag-1 autocorrelation 0.66. If a series of that length were a Gaussian autoregression with $\phi = 0.66$, what would be the large-sample level of the nominal 5% t-test for its mean, and by what factor would $s/\sqrt n$ understate the standard error? Does the answer apply to Iowa?
Solution. The level is 0.375, and the factor is about 2.21. By Theorem 4, the level is $2[1 - \Phi(z_{0.975}\sqrt{0.34/1.66})]$. Here $\sqrt{0.34/1.66} = 0.4526$, so the argument is $z_{0.975} \times 0.4526 = 0.887$, and $2[1 - \Phi(0.887)] = 0.375$. The variance ratio $\kappa^2 = 1.66/0.34 = 4.882$, so the true large-sample standard error is $\sqrt{4.882} = 2.21$ times $\sqrt{\gamma(0)/n}$, the quantity $s/\sqrt n$ estimates. The simulation at $n = 312$ found 0.379. The answer does not apply to Iowa as it stands. Iowa's series is not an autoregression: its correlogram is a cycle, and Theorem 4's hypothesis that $\gamma(h) = \gamma(0)\phi^{\lvert h\rvert}$ fails. Read as a stationary process with a random-phase cycle, its autocovariances never decay and are not summable, so the hypothesis of Theorem 4 fails and the theorem has nothing to say. (That does not make the mean's variance large: the section on Iowa shows that a random-phase cycle contributes only of order $1/n^2$ to $\operatorname{Var}(\bar X)$.) Read as a fixed cycle plus a remainder, the relevant series is the remainder, whose lag-1 autocorrelation is 0.18. The problem shows how large the distortion can be, not what it is for Iowa.
Problem 4 (a Ljung–Box test by hand). Iowa's raw cancer series ($n = 313$) has sample autocorrelations 0.0132, 0.1034 and −0.0585 at lags 1 to 3. Compute the Box–Pierce and Ljung–Box statistics with $H = 3$ and test at the 5% level whether the series is independent noise.
Solution. $Q_{\mathrm{BP}} = 4.48$ and $Q_{\mathrm{LB}} = 4.54$, both below the 5% point 7.81 of the chi-squared distribution with 3 degrees of freedom, so the test does not reject; the Ljung–Box p-value is 0.209. The squares are 0.000176, 0.010701 and 0.003427 (from the unrounded autocorrelations), with sum 0.014304, so $Q_{\mathrm{BP}} = 313 \times 0.014304 = 4.48$. For the Ljung–Box statistic each square is divided by $313 - h$: $0.000176/312 + 0.010701/311 + 0.003427/310 = 0.0000460$, and $313 \times 315 = 98{,}595$, so $Q_{\mathrm{LB}} = 98{,}595 \times 0.0000460 = 4.54$. It is a little larger than $Q_{\mathrm{BP}}$, since each term is inflated by $(n + 2)/(n - h) \approx 1.01$. The Corollary justifies the chi-squared reference for independent noise. Here the question is exactly whether the series is such noise, so no degrees of freedom are subtracted. Over 10 lags the conclusion is the same ($p = 0.28$, in the section on Iowa).
Summary
- A process is weakly stationary when its mean is constant and its covariances depend only on the lag; for Gaussian processes this is the same as strict stationarity. Autocovariance functions are exactly the even nonnegative definite functions.
- The variance of a mean is the average of all the covariances, and the usual sample variance is biased downward by positive correlation and upward by a changing mean.
- For a Gaussian autoregression the nominal 5% t-test has large-sample level $2[1 - \Phi(z_{0.975}\sqrt{(1 - \phi)/(1 + \phi)})]$, with $z_{0.975} \approx 1.96$: 0.375 at $\phi = 0.66$.
- The sample autocovariance with divisor $n$ is nonnegative definite; with divisor $n - h$ it need not be. Both are biased by centering at $\bar x$, by about $-\operatorname{Var}(\bar X)$.
- A central limit theorem for $m$-dependent sequences, proved from the independent one, gives the approximate $N(0, 1/n)$ law of the sample autocorrelations of independent noise and the chi-squared limit of the Box–Pierce and Ljung–Box statistics. Bartlett's formula gives the variances for correlated linear processes.
- Differencing removes polynomial trends and periodic components, and turns noise into a series with lag-1 autocorrelation $-\frac12$.
- Iowa's all-cause deaths are an annual cycle plus a remainder with lag-1 autocorrelation 0.18; cancer deaths pass the tests for noise.
What we have learned
Stationarity is what makes a single realization informative about its process: every pair of times $h$ apart estimates the same $\gamma(h)$. Dependence changes the arithmetic of the statistics built for independent data, and the changes are exact and computable: the variance of a mean, the level of a test, the scatter of the sample autocorrelations. For noise the scatter has a simple law, which turns the correlogram into a test. For Iowa's all-cause deaths, a fitted cycle and trend leave a remainder that fails that test. The remainder needs a model of its dependence, and the simplest such models, and the forecasts they give, are the subject of Autoregressive and Moving Average Models.
References
- Bartlett, M. S. (1946). On the theoretical specification and sampling properties of autocorrelated time-series. Journal of the Royal Statistical Society, Series B, 8(1), 27–41 (volume and title as Crossref lists them). https://doi.org/10.2307/2983611 record
- Boshnakov, G. N., and Ravagli, D. (2021). A signed power transformation with application to white noise testing. arXiv:2109.00280. https://doi.org/10.48550/arXiv.2109.00280 record
- Box, G. E. P., and Pierce, D. A. (1970). Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. Journal of the American Statistical Association, 65(332), 1509–1526. https://doi.org/10.1080/01621459.1970.10481180 record · 2 pages
- Brockwell, P. J., and Davis, R. A. (2016). Introduction to Time Series and Forecasting (3rd ed.). Springer Texts in Statistics. Springer. https://doi.org/10.1007/978-3-319-29854-2 record · 3 pages
- Durrett, R. (2019). Probability: Theory and Examples (5th ed.). Cambridge University Press. Theorem 2.1.21 (Kolmogorov's extension theorem), §3.10 and Theorem 3.10.6 (Cramér–Wold device), Example 6.1.2, Theorem 8.3.2 and Example 8.3.3 (M-dependent sequences); numbers of the author's version of January 11, 2019. record · 15 pages
- Francq, C., and Zakoïan, J.-M. (2010). GARCH Models: Structure, Statistical Inference and Financial Applications. John Wiley & Sons. https://doi.org/10.1002/9780470670057 (Cited through Boshnakov and Ravagli, 2021, who give the year as 2011; not read.) record
- Hoeffding, W., and Robbins, H. (1948). The central limit theorem for dependent random variables. Duke Mathematical Journal, 15(3), 773–780. https://doi.org/10.1215/S0012-7094-48-01568-3 record
- Hyndman, R. J., and Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.). OTexts, Melbourne. https://otexts.com/fpp3/ (accessed November 2026) record · 2 pages
- Ljung, G. M., and Box, G. E. P. (1978). On a measure of lack of fit in time series models. Biometrika, 65(2), 297–303. https://doi.org/10.1093/biomet/65.2.297 record
- National Center for Health Statistics (2021). Weekly Counts of Deaths by State and Select Causes, 2014–2019. Centers for Disease Control and Prevention, data.cdc.gov dataset 3yf8-kanr. https://data.cdc.gov/d/3yf8-kanr (accessed November 2026) record · 16 pages