Simple Linear Regression fitted a line through five patients from a diabetes study, predicting disease progression after one year from body mass index. The line missed each patient by some amount, its residuals: $-11.83$, $-48.03$, $-15.76$, $+68.95$ and $+6.67$ progression units. To compare this line with another, or to say how well it predicts, those five numbers must be reduced to one. Squaring them and averaging gives one answer; taking their absolute values and averaging gives another; and the two answers rank models differently when a few errors are large.
A loss function assigns a cost to each error, and the average cost over a set of cases is a regression metric. This article covers the standard metrics, mean squared error, its square root and mean absolute error, and what each one measures. It derives which prediction each loss favors, introduces losses that resist outliers, and compares several models on patients they were not fitted to. The running example is the diabetes study of Efron, Hastie, Johnstone and Tibshirani (2004): 442 patients, ten baseline measurements and a measure of disease progression one year later.
Three metrics by hand
Write $e_i = y_i - \hat{y}_i$ for the residual of case $i$, the observed value minus the prediction, over $n$ cases. The three standard metrics are
$$\mathrm{MSE} = \frac{1}{n}\sum_{i=1}^{n} e_i^2, \qquad \mathrm{RMSE} = \sqrt{\mathrm{MSE}}, \qquad \mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} |e_i|.$$
For the five patients:
| Patient | Residual $e$ | $e^2$ | $\lvert e \rvert$ |
|---|---|---|---|
| 1 | −11.83 | 139.8 | 11.83 |
| 2 | −48.03 | 2,306.7 | 48.03 |
| 3 | −15.76 | 248.4 | 15.76 |
| 4 | 68.95 | 4,753.9 | 68.95 |
| 5 | 6.67 | 44.4 | 6.67 |
| Sum | 0 | 7,493.22 | 151.23 |
So $\mathrm{MSE} = 7{,}493.22/5 = 1{,}498.64$, $\mathrm{RMSE} = \sqrt{1{,}498.64} = 38.71$ and $\mathrm{MAE} = 151.23/5 = 30.25$. (The squares are computed from unrounded residuals.) The sum of squared residuals is the RSS that least squares minimized, so the MSE of a least-squares fit on its own data is RSS$/n$.
The units differ. MSE is in squared progression units, which have no direct meaning. RMSE and MAE are in progression units, the scale of the outcome: the line misses these patients by about 30 to 39 units. MAE is literally the average size of a miss. RMSE is a root-mean-square size, which weights large misses more heavily.
The weighting differs. Patient 4's residual of 68.95 is 45.6% of the sum of absolute residuals but 63.4% of the sum of squared residuals. Squaring makes a residual twice as large count four times as much, so MSE and RMSE are dominated by the largest errors, and MAE much less so.
RMSE is never below MAE
For any set of residuals, $\mathrm{RMSE} \ge \mathrm{MAE}$. The difference of their squares is
$$\mathrm{RMSE}^2 - \mathrm{MAE}^2 = \frac{1}{n}\sum_i |e_i|^2 - \Bigl(\frac{1}{n}\sum_i |e_i|\Bigr)^2,$$
which is the variance of the absolute residuals, and a variance is never negative. The two are equal only when every residual has the same size. The more the sizes of the errors vary, the more RMSE exceeds MAE, so the ratio RMSE/MAE is a rough indicator of how heavy the tail of large errors is. For normally distributed errors with standard deviation $\sigma$, the mean absolute error is $\sigma\sqrt{2/\pi}$, so the ratio is $\sqrt{\pi/2} = 1.2533$. For the five patients it is $38.71/30.25 = 1.280$. In the other direction, $\mathrm{RMSE} \le \sqrt{n}\,\mathrm{MAE}$, with equality when one case carries all of the error.
Since RMSE is an increasing function of MSE, the two always rank models in the same order. They carry the same information, and RMSE is usually reported because its units are those of the outcome.
What each loss is estimating
A loss function does more than score a model. It determines what a model fitted by minimizing it will estimate. The simplest case shows this exactly: predict every patient with one constant $c$, and ask which $c$ minimizes each loss.
Squared loss. The average squared error, $\frac{1}{n}\sum (y_i - c)^2$, has derivative $-\frac{2}{n}\sum (y_i - c)$, which is zero when $c = \frac{1}{n}\sum y_i = \bar{y}$. The best constant under squared loss is the mean.
Absolute loss. The average absolute error, $\frac{1}{n}\sum |y_i - c|$, is not differentiable where $c$ equals some $y_i$, but away from those points its slope is $\frac{1}{n}$ times (the number of $y_i$ below $c$ minus the number above). The slope is negative while fewer than half the values lie below $c$ and positive once more than half do, so the minimum is where the counts balance: the median.
For the 442 patients, the mean progression is 152.13 and the median 140.5. The distribution has a longer right tail (skewness 0.44), which pulls the mean above the median.
| Constant prediction | MSE | RMSE | MAE |
|---|---|---|---|
| mean, 152.13 | 5,929.9 | 77.01 | 65.76 |
| median, 140.5 | 6,065.2 | 77.88 | 65.04 |
Each constant wins on its own loss. The same holds for models with inputs. A model fitted by minimizing squared error estimates the conditional mean of the outcome given the inputs, and one fitted by minimizing absolute error estimates the conditional median (Hastie, Tibshirani and Friedman, 2009, §2.4). The two coincide when the errors are symmetric about zero and differ when they are skewed. This is a reason to choose the loss deliberately: a hospital planning resources for an average patient wants the mean; a prediction meant to be "as likely too high as too low" for an individual is a median.
Each loss also corresponds to a model of the errors. Least squares is maximum likelihood when the errors are normal, the model behind the intervals of Linear Regression Assumptions. Least absolute deviations is maximum likelihood when the errors follow a Laplace (double exponential) distribution, whose heavier tails make large errors less surprising.
Quantile loss
Absolute loss is the special case $\tau = 1/2$ of the quantile or pinball loss. For a level $\tau$ between 0 and 1,
$$\rho_\tau(e) = \begin{cases} \tau\, e & e \ge 0, \\ (\tau - 1)\, e & e < 0, \end{cases}$$
which charges $\tau$ per unit for under-prediction and $1 - \tau$ per unit for over-prediction. The same balancing argument as for the median shows that the best constant is the $\tau$-quantile of the data: at $\tau = 0.9$, the value with 90% of observations below it. Koenker and Bassett (1978) introduced regression quantiles, linear models fitted by minimizing this loss; such a model describes how a chosen quantile of the outcome, rather than its mean, depends on the inputs. Quantile loss suits problems where the two directions of error have different costs, such as stocking enough supplies to meet demand nine days in ten.
Robustness to outliers
The weighting that makes squared error emphasize large residuals also makes it fragile. What a single value can do depends on how much each residual pulls on the fit, which is the derivative of the loss. For squared loss $e^2/2$, the pull is $e$ itself: a residual ten times larger pulls ten times harder, without limit. For absolute loss the pull is $\pm 1$ whatever the size of the residual. The Huber loss (Huber, 1964) is a compromise, quadratic for small residuals and linear for large ones:
$$\rho_\delta(e) = \begin{cases} \tfrac{1}{2} e^2 & |e| \le \delta, \\ \delta\,|e| - \tfrac{1}{2}\delta^2 & |e| > \delta. \end{cases}$$
Within $\delta$ of zero it behaves like least squares, which is efficient when errors are normal; beyond $\delta$ its pull is capped at $\delta$, as with absolute loss (Hastie et al., 2009, §10.6, who write the same loss scaled by 2). The threshold is stated in units of the residuals. scikit-learn's HuberRegressor estimates a scale for the residuals along with the coefficients and applies the threshold $\epsilon = 1.35$ (its default) to residuals divided by that scale (scikit-learn developers, 2024).
One bad value in the training data
To see the difference, the 442 patients were split at random into 294 for training and 148 for testing (scikit-learn's train_test_split, random_state=742), and progression was regressed on BMI three ways: by least squares, by least absolute deviations (median regression) and by Huber regression. Then one training value was corrupted. The training patient with the highest BMI, 42.2, had a progression of 242; it was changed to 2,420, the kind of error a misplaced decimal point produces. Each method was refitted, and every fit was scored on the 148 untouched test patients:
| Method | Training data | Intercept | BMI slope | Test RMSE | Test MAE |
|---|---|---|---|---|---|
| Least squares | clean | −105.97 | 9.974 | 59.39 | 49.99 |
| Least squares | one bad value | −245.37 | 15.528 | 64.16 | 52.02 |
| Least absolute deviations | clean | −153.98 | 11.531 | 58.00 | 47.70 |
| Least absolute deviations | one bad value | −164.40 | 12.000 | 58.41 | 47.87 |
| Huber | clean | −129.58 | 10.845 | 59.11 | 49.32 |
| Huber | one bad value | −141.19 | 11.315 | 59.33 | 49.27 |
One value in 294 raised the least-squares slope by more than half, from 9.97 to 15.53, and its test RMSE from 59.4 to 64.2. The corrupted patient has the highest BMI in the training data, which gives it high leverage: a point at the edge of the input range can tilt the line most, as Linear Regression Assumptions explains. The two robust fits moved by less than half a unit of slope and their test errors barely changed. On the clean data the three methods differ too. Least absolute deviations fits the conditional median, and its line lies below the least-squares line at low BMI (76.6 against 93.5 at a BMI of 20), as expected for an outcome whose distribution is skewed to the right.
Robust losses are not a substitute for looking at the data. A value of 2,420 where every other value is below 350 should be found and corrected before any model is fitted. Robust fitting matters where bad values cannot all be found, as in large automated pipelines, or where the error distribution really does have heavy tails.
One bad value in the test data
The same fragility affects evaluation. The model with three inputs (BMI, blood pressure and the serum measurement S5), fitted to the clean training data, has test MSE 3,085.0, RMSE 55.54, MAE 45.48 and median absolute error 38.51. Changing one test patient's progression from 163 to 1,630, with the model's prediction for that patient 235.3, changes them as follows:
| Metric | Clean test data | One bad value | Factor |
|---|---|---|---|
| MSE | 3,085.0 | 16,191.9 | 5.25 |
| RMSE | 55.54 | 127.25 | 2.29 |
| MAE | 45.48 | 54.41 | 1.20 |
| Median absolute error | 38.51 | 38.51 | 1.00 |
One value in 148 multiplied the MSE by five and more than doubled the RMSE. The MAE rose by a fifth, and the median absolute error did not move. A large gap between RMSE and MAE, or a sudden change in RMSE, is a reason to look at the largest residuals before drawing conclusions. The choice among these metrics should still follow from what the errors cost. If large errors are disproportionately harmful, RMSE measures what matters, and its sensitivity is the point. If every unit of error costs the same, MAE does.
Comparing models on held-out data
A metric computed on the data a model was fitted to measures fit, not prediction. Five least-squares models were fitted to the 294 training patients and evaluated on the 148 test patients. The models range from the training mean alone to all ten baseline variables with every square and pairwise product (degree 2, 65 columns), as in Multiple Linear Regression.
| Model | Columns | Training RMSE | Training MAE | Test MSE | Test RMSE | Test MAE | Test $R^2$ |
|---|---|---|---|---|---|---|---|
| training mean only | 0 | 78.77 | 67.69 | 5,476.2 | 74.00 | 64.10 | −0.053 |
| BMI | 1 | 64.13 | 53.44 | 3,526.9 | 59.39 | 49.99 | 0.322 |
| BMI, BP, S5 | 3 | 56.05 | 46.51 | 3,085.0 | 55.54 | 45.48 | 0.407 |
| all ten | 10 | 53.27 | 43.18 | 3,096.9 | 55.65 | 44.26 | 0.404 |
| all ten, degree 2 | 65 | 47.32 | 37.04 | 3,754.3 | 61.27 | 48.14 | 0.278 |
Training error falls with every added column, from an RMSE of 78.8 to 47.3. Test error does not. The three-input and ten-input models are nearly tied on the test patients, and the degree-2 model, which fits its training data best by a wide margin, predicts new patients worse than either: its test RMSE of 61.3 is close to that of BMI alone. The gap between its training RMSE (47.3) and test RMSE (61.3) is the signature of overfitting described in The Bias-Variance Tradeoff.
The three metrics disagree in one place. By test RMSE the three-input model is slightly better than the ten-input model (55.54 against 55.65); by test MAE the ten-input model is better (44.26 against 45.48). A disagreement like this means the models differ in the shape of their errors rather than their size: the ten-input model has smaller typical errors and a larger worst error (163.2 against 153.2). With 148 test patients, though, differences this small are well within what a different random split could reverse. Cross-Validation averages over many splits, and significance tests ask whether a difference is larger than chance.
$R^2$ on test data is $1 - \mathrm{MSE}/\mathrm{MSE}_{\text{baseline}}$, the share of the baseline's squared error that the model removes. It has two common versions, which differ in the baseline. scikit-learn's r2_score, used in the table, compares the model with the mean of the test outcomes, a prediction that could only be made after seeing them. Compared instead with the training mean, an honest baseline, the values are higher here: 0.356 for BMI, 0.437 for the three-input model, and 0.000 for the training mean itself. The test patients in this split happen to average 141.1 against 157.7 in training, which is why the training mean scores $-0.053$ by scikit-learn's version. Either way, $R^2$ on new data can be negative, as Simple Linear Regression noted, and the version should be stated.
Percentage errors
Errors relative to the size of the outcome are popular in forecasting. The mean absolute percentage error is
$$\mathrm{MAPE} = \frac{100\%}{n}\sum_{i=1}^{n} \left|\frac{y_i - \hat{y}_i}{y_i}\right|.$$
For the three-input model on the test patients it is 49.2%, while the median absolute percentage error is 29.2%. The mean is pulled up by patients with small progression values: one patient with progression 25 and a prediction of 140 contributes an error of 460%. The 18 test patients with progression below 60 account for 37.5% of the MAPE. Hyndman and Koehler (2006) describe these problems in forecasting: percentage errors are undefined when an actual value is zero, extremely large when it is near zero, and meaningful only for outcomes on a scale with a true zero. Dividing the MAE by the mean outcome avoids the first two problems; here it is $45.48/141.1 = 32.2\%$. Progression scores are an index rather than a quantity with a natural zero, so percentage errors are a poor fit for this example.
Training loss and evaluation loss
The loss used to fit a model and the metric used to judge it need not be the same. Least squares is fitted by minimizing squared error, and it is reasonable to report its MAE as well, because MAE describes the typical size of an error in a way readers grasp. What matters is that the evaluation metric reflects what errors cost in the application, and that it is computed on data the model has not seen. When the two losses differ, the model is not optimizing what is being measured. If the measure is MAE and the errors are skewed, a model fitted by least absolute deviations will usually do better on it, as the robust-fit table showed on the clean data.
In Python
import numpy as np
from sklearn.datasets import load_diabetes
from sklearn.linear_model import HuberRegressor, LinearRegression, QuantileRegressor
from sklearn.metrics import (mean_absolute_error, mean_squared_error,
median_absolute_error, r2_score, root_mean_squared_error)
from sklearn.model_selection import train_test_split
data = load_diabetes(scaled=False)
X, y = data.data[:, [2, 3, 8]], data.target # BMI, BP, S5
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=1/3, random_state=742)
pred = LinearRegression().fit(X_tr, y_tr).predict(X_te)
print(mean_squared_error(y_te, pred)) # 3085.0
print(root_mean_squared_error(y_te, pred)) # 55.54
print(mean_absolute_error(y_te, pred)) # 45.48
print(median_absolute_error(y_te, pred)) # 38.51
print(r2_score(y_te, pred)) # 0.407 (relative to the test-set mean)
# robust fits of progression on BMI alone
bmi_tr = X_tr[:, :1]
lad = QuantileRegressor(quantile=0.5, alpha=0, solver="highs").fit(bmi_tr, y_tr)
hub = HuberRegressor(epsilon=1.35, alpha=0).fit(bmi_tr, y_tr)
print(lad.intercept_, lad.coef_, hub.intercept_, hub.coef_) # -153.98 [11.53] -129.58 [10.85]
root_mean_squared_error was added in scikit-learn 1.4; in earlier versions, take the square root of mean_squared_error. QuantileRegressor with quantile=0.5 fits least absolute deviations, and alpha=0 turns off its default L1 penalty. HuberRegressor also has a small default L2 penalty, alpha=0.0001, set to zero here. The numbers in this article are computed by tools/experiments/data-science/regression-loss-functions.py in the site's repository.
References
- Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. The Annals of Statistics, 32(2), 407–499. record · 12 pages
- Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.). Springer. record · 23 pages
- Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101. record
- Hyndman, R. J., and Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679–688. record
- Koenker, R., and Bassett, G. (1978). Regression quantiles. Econometrica, 46(1), 33–50. record
- scikit-learn developers (2024). HuberRegressor. scikit-learn 1.6 documentation. https://scikit-learn.org/1.6/modules/generated/sklearn.linear_model.HuberRegressor.html record