enetLTS fits robust and sparse linear,
logistic and multinomial regression models. It combines the trimming
idea of the Least Trimmed Squares (LTS) estimator with the elastic net
penalty, so that a few outlying observations cannot spoil the fit, while
the penalty selects variables and handles correlated and
high-dimensional predictors (\(p >
n\)).
For \(h \le n\) observations, the estimator minimizes the penalized loss over the best subset \(H\) of size \(h\),
\[ Q(H,\beta) = \sum_{i \in H} \ell_i(\beta) + h\lambda P_\alpha(\beta), \qquad P_\alpha(\beta) = \frac{1-\alpha}{2}\|\beta\|_2^2 + \alpha\|\beta\|_1 , \]
where \(\ell_i\) is half the squared
residual (linear) or the negative log-likelihood contribution (logistic,
multinomial), so that \(\lambda\) has
the same scale as in glmnet (Kurnaz, Hoffmann and Filzmoser
(2018) write the linear loss without the factor \(1/2\), which doubles the scale of \(\lambda\)).
The subset \(H\) is searched with
concentration steps (C-steps) from many small starting subsets: each
C-step fits the elastic net on the current subset and keeps the \(h\) observations that fit it best. These
are the observations with the smallest loss (linear regression; logistic
regression within each class, so that the class proportions are kept) or
with the smallest robust distance within their class (multinomial
regression, see the section on the multinomial model). For logistic
regression, the best starting subsets are chosen with the robust
Bianco-Yohai criterion of Croux and Haesbroeck (2003), as in Kurnaz,
Hoffmann and Filzmoser (2018). A final reweighting step
raises the efficiency. The tuning parameters \(\alpha\) and \(\lambda\) are chosen by cross-validation,
and the elastic net fits themselves are computed with the
glmnet package (Friedman, Hastie and Tibshirani, 2010).
The package at a glance:
| Task | Function |
|---|---|
| Fit the model | enetLTS(x, y, family = ...) |
| Summary, coefficients | print(), coef(),
nonzeroCoef.enetLTS() |
| Flagged observations | outliers.enetLTS(), weights() |
| Predictions | predict(), fitted(),
residuals() |
| Plots | plotCoef.enetLTS(), plotResid.enetLTS(),
plotDiagnostic.enetLTS() |
| Performance on test data | assess.enetLTS(), confusion.enetLTS(),
roc.enetLTS() |
Three simulated data sets with known outliers,
GaussianExample, BinomialExample and
MultinomialExample, are used below; all of them are
high-dimensional (\(p > n\)).
We load a data set with \(n = 100\)
observations and \(p = 200\)
predictors, of which the first 10 observations are outliers, and fit the
model with the default settings. seed makes the random
starting subsets, and therefore the result, reproducible.
Printing the fit shows the nonzero coefficients (at most
maxprint of them), the selected tuning parameters and the
number of observations flagged as outliers:
fit
#> enetLTS estimator (family = "gaussian", reweighted fit)
#>
#> Call: enetLTS(xx = x, yy = y, seed = 1)
#>
#> Nonzero coefficients (35 of 200 predictors):
#> (Intercept) x1 x2 x3 x4 x5
#> 1.1590 0.7691 0.7048 0.6734 1.4873 1.0187
#> x6 x7 x8 x9 x10 x11
#> 0.6019 0.8707 1.8722 0.3109 1.2145 0.5719
#> x12 x13 x14 x15 x16 x17
#> 1.3227 0.6547 0.6478 1.6695 0.3159 1.5570
#> x18 x19
#> 0.9101 0.9740
#> ... 16 more; see coef(x)
#>
#> alpha = 0.65, lambda = 0.2463, lambdaw = 0.1594
#> Observations flagged as outliers: 15 of 100 (see outliers.enetLTS())The flagged observations, with their absolute residuals
(loss) and fitted values:
outliers.enetLTS(fit)
#> enetLTS outliers (reweighted fit, gaussian): 15 of 100 observations flagged
#> index loss observed predicted type
#> 1 256.226 635.593 379.367 outlier
#> 2 255.365 636.414 381.049 outlier
#> 3 255.126 624.066 368.940 outlier
#> 4 255.921 640.200 384.279 outlier
#> 5 256.717 635.644 378.927 outlier
#> 6 254.164 634.995 380.831 outlier
#> 7 255.416 628.734 373.318 outlier
#> 8 255.072 630.754 375.682 outlier
#> 9 256.706 631.616 374.910 outlier
#> 10 257.744 638.528 380.784 outlier
#> 14 2.461 -34.244 -31.783 outlier
#> 22 1.910 24.300 22.390 outlier
#> 41 2.931 0.986 -1.945 outlier
#> 58 3.286 -14.045 -10.759 outlier
#> 84 3.314 -11.911 -8.597 outlierPredictions for new observations:
predict(fit, newX = GaussianExample$xtest[1:5, ], type = "response")
#> $response
#> [1] 4.572018 10.274609 -5.736091 20.673360 -7.849757The rest of this vignette explains these steps for every family.
family = "gaussian" (default)The linear model is \(y_i = \beta_0 + x_i^\top\beta + \varepsilon_i\). enetLTS keeps the \(h\) observations with the smallest squared residuals and minimizes
\[\min_{|H|=h}\ \Big\{\frac12\sum_{i\in H}(y_i-\beta_0-x_i^\top\beta)^2 + h\lambda P_\alpha(\beta)\Big\}.\]
The intercept is not penalized. For \(\alpha=1\) this is the sparse LTS estimator of Alfons, Croux and Gelper (2013).
GaussianExample follows the simulation design of Kurnaz,
Hoffmann and Filzmoser (2018). The first 20 predictors are informative
(\(\beta_j = 1\)) and highly correlated
within two blocks; the other 180 are noise. For the first 10
observations the informative predictors are drawn from \(N(20, 1)\) and the response is shifted far
up, so they are leverage points and vertical outliers at the same time
(see ?GaussianExample). The data also contain a clean test
set.
Each estimate comes in two versions. The raw
estimator minimizes the trimmed penalized criterion and is the initial
robust fit. The reweighted estimator (the default) aims
at a higher statistical efficiency: it refits the model on the
observations that are consistent with the raw fit and flags only those
that are clearly inconsistent with the refitted model. The two versions
can differ in the coefficients, the selected variables and the flagged
observations; an observation trimmed by the raw fit is not guaranteed to
be flagged by the reweighted fit. Use vers = "raw" in
coef(), predict(),
outliers.enetLTS(), the plots and the assessment functions
to obtain the raw version.
b <- coef(fit) # intercept first
br <- coef(fit, vers = "raw")
c(reweighted = sum(b[-1] != 0), raw = sum(br[-1] != 0))
#> reweighted raw
#> 35 33The true model has 20 nonzero slopes. The reweighted fit selects 20 of them and 15 noise variables.
outliers.enetLTS() returns one row per observation; its
print method shows only the flagged ones, and the indices are the
index entries with flagged == TRUE. Since we
know the contaminated observations, we can check the result:
o <- outliers.enetLTS(fit)
fl <- o$index[o$flagged]
out <- GaussianExample$outliers
list(flagged = fl,
detected = intersect(fl, out),
missed = setdiff(out, fl),
clean_but_flagged = setdiff(fl, out))
#> $flagged
#> [1] 1 2 3 4 5 6 7 8 9 10 14 22 41 58 84
#>
#> $detected
#> [1] 1 2 3 4 5 6 7 8 9 10
#>
#> $missed
#> integer(0)
#>
#> $clean_but_flagged
#> [1] 14 22 41 58 8410 of the 10 contaminated observations are flagged. The clean
observations in the last line are flagged because their standardized
residual exceeds the cut-off qnorm(1 - del) = 2.24 of the
reweighting step. With normal errors this happens by chance for about
2*del = 2.5% of the clean observations, here about 2.2 of
90; 5 were flagged. Their absolute residuals (the loss
column above) are small compared with those of the outliers. The next
section shows how crossfit = TRUE reduces such false
alarms.
The argument del controls this trade-off: a smaller
value flags fewer clean observations, but also makes it easier for real
outliers to stay unflagged.
enetLTS() is the fitting function for all families. The
arguments below are used by every family; the family-specific ones are
listed in the sections on logistic and multinomial regression.
| Argument | Default | Meaning |
|---|---|---|
family |
"gaussian" |
"gaussian", "binomial" or
"multinomial". |
alphas |
seq(0, 1, length = 41) |
Grid of the elastic net mixing parameter \(\alpha\) (\(\alpha = 1\) lasso, \(\alpha = 0\) ridge). |
lambdas |
family-specific grid | Grid of the penalty parameter \(\lambda\) for the trimmed (raw) fit. |
lambdaw |
chosen by cross-validation | Penalty parameter of the reweighted fit. |
hsize |
0.75 |
Fraction of observations kept; up to about \(1-\)hsize of the data may be
contaminated. |
nsamp |
c(500, 10) |
Number of elemental starting subsets, and how many of the best are iterated to convergence. |
nfold, repl |
5, 1 |
Folds and repetitions of the cross-validation for \(\alpha\) and \(\lambda\). |
del |
0.0125 |
Cut-off of the reweighting step. For linear regression with normal
errors about a share 2*del (2.5%) of clean observations
would be flagged; in practice, and for the other families, the share can
be higher. |
intercept, scal |
TRUE, TRUE |
Include an intercept; standardize the predictors (mean and standard deviation) on each subset. |
ncores |
1 |
Number of cores for parallel computation. The random starts are drawn in the main process, so the results should not depend on it. |
seed |
NULL |
Makes the random starts, and therefore the results, reproducible. |
For the gaussian family the default lambdas grid runs
from lambda0, a value derived from robust (winsorized)
correlations between the response and each predictor, down to zero.
Example: crossfit = TRUE. With
cross-fitting, the observations kept by the trimmed fit are chosen by
cross-fitted residuals, so that an observation is never judged by a
model that has seen it. For the gaussian family it is off by default. On
our data:
fit.cf <- enetLTS(x, y, crossfit = TRUE, seed = 1)
ocf <- outliers.enetLTS(fit.cf)
flcf <- ocf$index[ocf$flagged]
list(detected = intersect(flcf, out), clean_but_flagged = setdiff(flcf, out))
#> $detected
#> [1] 1 2 3 4 5 6 7 8 9 10
#>
#> $clean_but_flagged
#> [1] 14 84
sapply(list(default = fit, crossfit = fit.cf), function(f)
assess.enetLTS(f, GaussianExample$xtest, GaussianExample$ytest, vers = "reweighted")$rmse)
#> default.reweighted crossfit.reweighted
#> 1.511869 1.370049All outliers are still flagged, fewer clean observations are flagged
(2 instead of 5), and the test error is smaller. In simulations with
this design (6 data sets) both settings flagged all contaminated
observations, and crossfit = TRUE flagged fewer clean
observations on average (1.2 against 2.7) with a slightly smaller test
error. For high-dimensional data, crossfit = TRUE is
therefore worth trying.
A smaller hsize gives more protection and less
efficiency; for example, if up to 40% of the data could be outlying:
Parallel computation:
predict() returns a list. type = "response"
gives the predicted values; type = "coefficients" and
type = "nonzero" return the coefficients and the indices of
the nonzero ones. fitted() and residuals()
work on the training data.
plotCoef.enetLTS() shows every slope as a needle from
zero, with a point for the nonzero ones; the intercept and the number of
nonzero slopes are given in the subtitle. The 20 informative predictors
are the first 20; the needles further right are noise variables with
small coefficients.
plot of chunk gaus-coef
plotResid.enetLTS() draws two plots: the standardized
residuals against the observation index and against the fitted values
\(x_i^\top\hat\beta\). Flagged
observations are red triangles; the gray lines at \(\pm\)qnorm(1 - del) are the
cut-offs of the reweighting step, so the points outside them are exactly
the flagged observations. The ten outliers have standardized residuals
far beyond the cut-offs, so the plot gets a second panel that zooms in
on the regular observations and on the flagged observations close to
them; there, the cut-off lines and the few clean observations just
outside them are visible.
Both plotResid.enetLTS() and
plotDiagnostic.enetLTS() add such a zoom panel
automatically when the flagged observations are far away from the
regular ones (zoom = TRUE, the default). To save space,
this vignette shows it here and for the logistic diagnostic plot below,
and draws the other plots with zoom = FALSE.
plot of chunk gaus-resid
plot of chunk gaus-resid
plotDiagnostic.enetLTS() plots the response against the
fitted values, with the identity line. The regular observations lie
close to the line; the outliers lie far above it.
plot of chunk gaus-diag
assess.enetLTS() computes performance measures on new
data, for the reweighted and the raw fit side by side, in the layout of
glmnet::assess.glmnet (Hastie, Qian and Tay, 2026):
assess.enetLTS(fit, GaussianExample$xtest, GaussianExample$ytest)
#> $mse
#> reweighted raw
#> 2.285748 2.995265
#> attr(,"measure")
#> [1] "Mean-Squared Error"
#>
#> $rmse
#> reweighted raw
#> 1.511869 1.730683
#> attr(,"measure")
#> [1] "Root Mean-Squared Error"
#>
#> $mae
#> reweighted raw
#> 1.211453 1.336816
#> attr(,"measure")
#> [1] "Mean Absolute Error"The errors of the test data have standard deviation 1, so a root mean
squared error close to 1 means that the model predicts about as well as
possible. In simulations with this design (6 data sets), all
contaminated observations were flagged in every data set, and the test
root mean squared error was about 1.4 for enetLTS and about 7 for the
classical elastic net (fitted with glmnet), which the
outliers pull away from the clean data.
family = "binomial"With \(y_i \in \{0, 1\}\), \(\eta_i = \beta_0 + x_i^\top\beta\) and \(\pi_i = \Pr(y_i = 1\mid x_i) = e^{\eta_i}/(1 + e^{\eta_i})\), the loss of observation \(i\) is \(d_i = d(\eta_i, y_i) = -y_i\log\pi_i-(1-y_i)\log(1-\pi_i) = -y_i\,\eta_i+\log(1+e^{\eta_i})\), which Kurnaz, Hoffmann and Filzmoser (2018) call the deviance of the observation (it is the negative log-likelihood contribution, without a factor 2). enetLTS minimizes the trimmed, penalized negative log-likelihood
\[\min_{|H|=h}\ \Big\{\sum_{i\in H} d_i + h\lambda P_\alpha(\beta)\Big\},\]
that is, the observations with the smallest negative log-likelihood contribution (equivalently, the smallest deviance) are kept, within each class. The penalty acts on the slopes of the standardized predictors.
BinomialExample also follows Kurnaz, Hoffmann and
Filzmoser (2018): \(n = 100\), \(p = 200\), 20 correlated informative
predictors and a latent linear model. The first 10% of the observations
of class 0 get informative predictors from \(N(20, 1)\) while their label stays 0. The
model would predict class 1 for them with near certainty, so they are
bad leverage points.
data(BinomialExample)
xb <- BinomialExample$x
yb <- BinomialExample$y
table(yb)
#> yb
#> 0 1
#> 49 51
fitb <- enetLTS(xb, yb, family = "binomial", seed = 1)
fitb
#> enetLTS estimator (family = "binomial", reweighted fit)
#>
#> Call: enetLTS(xx = xb, yy = yb, family = "binomial", seed = 1)
#>
#> Nonzero coefficients (37 of 200 predictors):
#> (Intercept) x2 x3 x4 x7 x8
#> 0.1861 0.1954 0.6743 0.6147 0.2327 0.7524
#> x9 x10 x11 x12 x14 x17
#> 0.5693 0.7304 0.2370 0.7247 0.7909 0.3695
#> x18 x19 x20 x25 x46 x47
#> 0.2304 0.0204 0.2005 0.0751 0.3272 0.2327
#> x48 x50
#> 0.4021 0.1099
#> ... 18 more; see coef(x)
#>
#> alpha = 0.75, lambda = 3e-04, lambdaw = 0.007862
#> Observations flagged as outliers: 6 of 100 (see outliers.enetLTS())For the classification families the type column
separates flagged observations that are classified as observed
(leverage point) from those that are classified differently
(possible mislabel / outlier). The latter are the
candidates for wrong labels and for bad leverage points.
ob <- outliers.enetLTS(fitb)
ob
#> enetLTS outliers (reweighted fit, binomial): 6 of 100 observations flagged
#> index loss observed predicted type
#> 1 123.080 0 1 possible mislabel / outlier
#> 3 129.998 0 1 possible mislabel / outlier
#> 5 2.222 1 0 possible mislabel / outlier
#> 7 125.044 0 1 possible mislabel / outlier
#> 13 125.421 0 1 possible mislabel / outlier
#> 53 2.624 1 0 possible mislabel / outlier
outb <- BinomialExample$outliers
list(contaminated = outb,
detected = intersect(ob$index[ob$flagged], outb),
clean_but_flagged = setdiff(ob$index[ob$flagged], outb))
#> $contaminated
#> [1] 1 3 7 13
#>
#> $detected
#> [1] 1 3 7 13
#>
#> $clean_but_flagged
#> [1] 5 53The loss column is the negative log-probability of the
observed class. The contaminated observations have losses of at least
123: the model is practically certain that they belong to class 1. The
flagged clean observations have a label that is unlikely, but not
impossible, under the model: it gives their observed class the
probabilities 0.11 and 0.07. They are candidates for wrong labels, not
verdicts.
| Argument | Default | Meaning |
|---|---|---|
yy |
– | A 0/1 vector (numeric, or a factor with levels 0 and 1). |
lambdas |
robust grid | Runs from lambda00, based on a robustified
point-biserial correlation between the response and each predictor, down
to zero (see lambda00()). |
del |
0.0125 |
An observation is flagged when its Pearson residual exceeds
qnorm(1 - del) in absolute value. |
crossfit |
ignored | Not available for the binomial family (a warning is given). |
type.response |
"link" |
Type of fitted.values: "link",
"response" (probabilities) or "class". |
In the plots of the logistic and multinomial models, the class is
shown by color and shape, and flagged observations are
drawn as filled symbols, so the plots can also be read without color.
The residual plot shows the negative log-likelihood of every observation
(the loss of outliers.enetLTS()), against the
index and against the estimated linear predictor \(\hat\eta_i\) (drawn here without the zoom
panel). The bad leverage points have a very large linear predictor (the
model predicts class 1) and a very large negative log-likelihood (they
are labeled 0).
plot of chunk bin-resid
plot of chunk bin-resid
The diagnostic plot shows the observed response against \(\hat\eta_i\), together with the fitted logistic curve. The zoom panel shows the range of the regular observations, where the S-shape of the logistic curve is visible.
plot of chunk bin-diag
assess.enetLTS(fitb, BinomialExample$xtest, BinomialExample$ytest)
#> $deviance
#> reweighted raw
#> 0.4493500 0.4667908
#> attr(,"measure")
#> [1] "Binomial Deviance"
#>
#> $class
#> reweighted raw
#> 0.105 0.095
#> attr(,"measure")
#> [1] "Misclassification Error"
#>
#> $auc
#> reweighted raw
#> 0.9695860 0.9699929
#> attr(,"measure")
#> [1] "AUC"
#>
#> $mse
#> reweighted raw
#> 0.07017962 0.07044182
#> attr(,"measure")
#> [1] "Mean-Squared Error"
#>
#> $mae
#> reweighted raw
#> 0.1453057 0.1194645
#> attr(,"measure")
#> [1] "Mean Absolute Error"The deviance is twice the mean negative log-likelihood,
as in glmnet. The confusion matrix and the ROC curve of the
reweighted fit:
confusion.enetLTS(fitb, BinomialExample$xtest, BinomialExample$ytest)
#> True
#> Predicted 0 1 Sum
#> 0 79 13 92
#> 1 8 100 108
#> Sum 87 113 200
#>
#> Percent correct: 0.895plot of chunk bin-roc
In simulations with this design (6 data sets) all contaminated observations were flagged in every data set, and the test misclassification rate was about 0.10 for enetLTS and about 0.49 for the classical elastic net.
family = "multinomial"Let \(g_i \in \{1, \dots, K\}\) be the class label of observation \(i\) and \(y_{il} = I(g_i = l)\) its indicator coding. Every class has an intercept and a slope vector, and the scores (values of the linear link function) are \(z_{il} = \beta_{l0} + x_i^\top\beta_l\) for \(l = 1, \dots, K\) (the linear predictors \(\eta_i\) of the logistic section, one per class). The class probabilities are \(p_l(x_i) = e^{z_{il}} / \sum_{j=1}^{K} e^{z_{ij}}\). The parameters are not identifiable without a penalty (adding the same vector to all \(\beta_l\) leaves the probabilities unchanged); with a reference class \(K\), the coefficients \(\beta_l - \beta_K\) are identified. The loss of observation \(i\) is the negative log-likelihood contribution
\[d_i = -\sum_{l=1}^{K} y_{il}\log p_l(x_i) = -\sum_{l=1}^{K} y_{il}\, z_{il} + \log\sum_{j=1}^{K} e^{z_{ij}},\]
where the second form follows from \(\log p_l(x_i) = z_{il} - \log\sum_j e^{z_{ij}}\) and \(\sum_l y_{il} = 1\); it is the form used in the computations. enetLTS minimizes the trimmed, penalized negative log-likelihood
\[\min_{|H|=h}\ \Big\{\sum_{i\in H} d_i + h\lambda P_\alpha(B)\Big\}, \qquad P_\alpha(B) = \frac{1-\alpha}{2}\sum_{j=1}^{p}\sum_{l=1}^{K}\beta_{jl}^2 + \alpha\sum_{j=1}^{p}\sum_{l=1}^{K}|\beta_{jl}|,\]
where \(B\) is the \((p+1)\times K\) coefficient matrix (Kurnaz
and Filzmoser, 2023); the penalty acts on the slopes of all \(K\) classes and not on the intercepts. This
is the ungrouped penalty of glmnet; the grouped one is used
with typegrouped = TRUE. Kurnaz and Filzmoser (2023) write
the loss as an average over \(H\)
(factor \(1/h\)), as in the
glmnet objective, with penalty \(h\lambda P_\alpha\) as printed in their
Equation (5); the form above has the scale of \(\lambda\) of the glmnet
objective, as in the code.
The subset is not chosen by this loss. In every C-step the centered scores are decomposed by a singular value decomposition (of rank \(r \le K-1\), because the probabilities sum to one), and the observations of each class are ranked by the robust Mahalanobis distance \(RD(u_i)\) of their singular vectors \(u_i\) (MCD estimates of location and scatter within the class). The distances are scaled within the class so that their median equals the \(0.5\)-quantile of the \(\chi^2_r\) distribution, \(RD_s(u_i) = RD(u_i)\sqrt{\chi^2_{r,0.5}/\mathrm{median}\,RD(u_i)}\). The new subset contains the \(h_l\) observations of class \(l\) with the smallest scaled distances, so that the class proportions are kept. Kurnaz and Filzmoser (2023) apply the decomposition to the scores of each class separately; the code applies it once to the scores of all observations. The code uses \(h_l = \lfloor n_l \cdot \texttt{hsize} + 1\rfloor\) and \(h = \sum_l h_l\); the paper defines \(h_l = (n_l + 1)h/n\), adjusted so that \(\sum_l h_l = h\). The formula above is the elastic net fit on the retained subset.
MultinomialExample has \(n =
150\) observations, \(p = 200\)
predictors and \(K = 3\) classes; each
class is driven by its own two predictors. The first 15 observations are
bad leverage points: predictors 1 and 2, which point to class 1, are
shifted by 5, and the label is set to class 3.
data(MultinomialExample)
xm <- MultinomialExample$x
ym <- MultinomialExample$y
table(ym)
#> ym
#> 1 2 3
#> 53 43 54
fitm <- enetLTS(xm, ym, family = "multinomial", seed = 1)
fitm
#> enetLTS estimator (family = "multinomial", reweighted fit)
#>
#> Call: enetLTS(xx = xm, yy = ym, family = "multinomial", seed = 1)
#>
#> Nonzero coefficients (39 of 200 predictors in at least one class):
#> 1 2 3
#> (Intercept) 0.1928 -0.0075 -0.0047
#> x1 0.8128 0.0000 0.0000
#> x2 0.8239 0.0000 0.0000
#> x3 -0.1525 0.9210 0.0000
#> x4 0.0000 0.9482 0.0000
#> x5 0.0000 0.0000 1.1420
#> x6 0.0000 0.0000 0.7731
#> x11 0.0000 0.2065 0.0000
#> x24 0.0000 0.1286 0.0000
#> x34 0.0000 0.0000 0.1890
#> x36 0.0255 0.0000 0.0000
#> x44 0.0000 0.0000 -0.0328
#> x47 0.1306 0.0000 0.0000
#> x49 -0.0876 0.0000 0.0000
#> x67 0.0000 0.0000 0.0217
#> x72 -0.2807 0.0000 0.0000
#> x77 -0.1855 0.0000 0.2084
#> x86 0.0000 0.0000 -0.0086
#> x87 0.0143 0.0000 0.0000
#> x88 0.0000 0.1805 0.0000
#> ... 20 more rows; see coef(x)
#>
#> alpha = 1, lambda = 0.009536, lambdaw = 0.03737
#> Observations flagged as outliers: 16 of 150 (see outliers.enetLTS())The printed matrix has one column per class; a row is shown when the predictor has a nonzero coefficient in at least one class.
om <- outliers.enetLTS(fitm)
om
#> enetLTS outliers (reweighted fit, multinomial): 16 of 150 observations flagged
#> index loss observed predicted type
#> 1 10.465 3 1 bad leverage point (screened)
#> 2 7.341 3 1 bad leverage point (screened)
#> 3 8.254 3 1 bad leverage point (screened)
#> 4 8.049 3 1 bad leverage point (screened)
#> 5 7.953 3 1 bad leverage point (screened)
#> 6 9.453 3 1 bad leverage point (screened)
#> 7 10.172 3 1 bad leverage point (screened)
#> 8 7.719 3 1 bad leverage point (screened)
#> 9 5.161 3 1 bad leverage point (screened)
#> 10 9.532 3 1 bad leverage point (screened)
#> 11 10.070 3 1 bad leverage point (screened)
#> 12 7.746 3 1 bad leverage point (screened)
#> 13 11.214 3 1 bad leverage point (screened)
#> 14 9.305 3 1 bad leverage point (screened)
#> 15 5.251 3 1 bad leverage point (screened)
#> 56 0.115 1 1 leverage point (screened)
outm <- MultinomialExample$outliers
list(detected = intersect(om$index[om$flagged], outm),
missed = setdiff(outm, om$index[om$flagged]),
clean_but_flagged = setdiff(om$index[om$flagged], outm))
#> $detected
#> [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
#>
#> $missed
#> integer(0)
#>
#> $clean_but_flagged
#> [1] 56All 15 contaminated observations are flagged. Observations that are
outlying in the predictors are screened before the fit, without using
the labels (screen = TRUE); they are listed as
bad leverage point (screened) (classified differently from
the observed class) or leverage point (screened), and
stored in fitm$screened. The screening uses a 97.5% cut-off
of a robust distance, so a few clean observations may be screened as
well. Here this happened to observation 56, which is classified
correctly and therefore listed as
leverage point (screened).
| Argument | Default | Meaning |
|---|---|---|
yy |
– | A factor (or a vector of class labels) with \(K \ge 3\) classes, or an \(n \times K\) indicator matrix. |
lambdas |
20 values | Log-spaced from the largest useful \(\lambda\) (lasso, robustly standardized data) down to 1% of it. |
screen |
TRUE |
Leaves out observations that are outlying in the predictors before
the fit, without using the labels; they are listed in
$screened. |
crossfit |
TRUE |
Chooses the retained observations by cross-fitted losses, so an observation is never judged by a model that has seen it. |
del |
0.0125 |
The reweighting flags an observation when the cross-fitted
probability of its observed class falls below a cut-off calibrated to
flag about 2*del (2.5%) of the observations if the model is
correct. |
typegrouped |
FALSE |
Use a grouped lasso penalty (all \(K\) coefficients of a predictor enter or leave together) in the fits after the subset search. |
An alternative reweighting rule (a robust median/MAD bound on the
cross-fitted loss within each class) can be used with
options(enetLTS.wrule = "mad").
The coefficient plot has one panel per class. The largest coefficients are those of the true predictors: 1 and 2 for class 1, 3 and 4 for class 2, and 5 and 6 for class 3.
plot of chunk mult-coef
The residual plot shows the negative log-likelihood of every observation against its index.
plot of chunk mult-resid
The diagnostic plot shows the first two components of the estimated scores.
plot of chunk mult-diag
assess.enetLTS(fitm, MultinomialExample$xtest, MultinomialExample$ytest)
#> $deviance
#> reweighted raw
#> 1.136182 1.351575
#> attr(,"measure")
#> [1] "Multinomial Deviance"
#>
#> $class
#> reweighted raw
#> 0.2066667 0.2966667
#> attr(,"measure")
#> [1] "Misclassification Error"
#>
#> $mse
#> reweighted raw
#> 0.3266918 0.4005329
#> attr(,"measure")
#> [1] "Mean-Squared Error"
#>
#> $mae
#> reweighted raw
#> 0.7724546 0.7083629
#> attr(,"measure")
#> [1] "Mean Absolute Error"
confusion.enetLTS(fitm, MultinomialExample$xtest, MultinomialExample$ytest)
#> True
#> Predicted 1 2 3 Sum
#> 1 76 19 17 112
#> 2 8 77 7 92
#> 3 6 5 85 96
#> Sum 90 101 109 300
#>
#> Percent correct: 0.7933The overall share of correct classifications hides differences between the classes. The share of each true class that is recognized:
hsizehsize is the fraction of observations used by the
trimmed fit; the estimator can cope with up to about \(1-\)hsize contamination. The
default 0.75 is a common compromise between robustness and
efficiency. When the amount of contamination is unknown,
hsize = "optimal" (experimental) fits all values in
hgrid and selects one of them as follows. For each value
the model is fitted; then, in a repeated 5-fold cross-validation (2
repetitions), the model is refitted on the retained observations of that
fit outside the test fold, and the loss (squared error for
"gaussian", negative log-likelihood otherwise) is computed
for all observations of the test fold. The criterion is
the mean of the smallest 75% of these cross-validated losses, so the
outliers, which have large losses, do not decide. The selected value is
the largest hsize whose criterion is
within 10% of the smallest one, which favors efficiency when several
values are about equally good. A warning is issued when the selected
value is below 0.7. Since the worst 25% of the losses are ignored, the
criterion cannot be relied on when more than about a quarter of the data
are contaminated. This option is experimental: use it as a guide and
compare the result with a fixed hsize.
type only marks
candidates.outliers.enetLTS() is a diagnostic, not a
universal outlier detector. What it reports depends on the
version ("raw" or "reweighted") and on the
fitted model. Remote observations that the model fits well (good
leverage points) are not flagged by the reweighted fit; compare with
vers = "raw".hsize, and look at the plots before interpreting the
coefficients.enetLTS() returns a list of class
"enetLTS". The most important components:
| Component | Meaning |
|---|---|
coefficients, raw.coefficients |
Reweighted and raw coefficients (intercept first). |
alpha, lambda, lambdaw |
Selected tuning parameters (raw fit, reweighted fit). |
wt, raw.wt |
0/1 weights; 0 marks a flagged observation. |
best |
Indices of the best subset of the raw fit. |
fitted.values, residuals |
On the training data (reweighted fit). |
screened |
Multinomial family: observations removed by the screening. |
inputs |
The data and the arguments of the call. |
Hastie, T., Qian, J. and Tay, K. (2026). An Introduction to
glmnet. Vignette of the R package glmnet. https://glmnet.stanford.edu/articles/glmnet.html
(accessed 16 November 2026).
Kurnaz, F. S. and Filzmoser, P. (2023). enetLTS: Robust and sparse methods for high dimensional linear, binary, and multinomial regression. Journal of Open Source Software, 8(82), 4773. doi:10.21105/joss.04773
Kurnaz, F. S. and Filzmoser, P. (2023). Robust and sparse multinomial regression in high dimensions. Data Mining and Knowledge Discovery, 37(4), 1609–1629. doi:10.1007/s10618-023-00936-6
Kurnaz, F. S., Hoffmann, I. and Filzmoser, P. (2018). Robust and sparse estimation methods for high-dimensional linear and logistic regression. Chemometrics and Intelligent Laboratory Systems, 172, 211–222. doi:10.1016/j.chemolab.2017.11.017
Alfons, A., Croux, C. and Gelper, S. (2013). Sparse least trimmed squares regression for analyzing high-dimensional large data sets. The Annals of Applied Statistics, 7, 226–248.
Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1–22.
Croux, C. and Haesbroeck, G. (2003). Implementing the Bianco and Yohai estimator for logistic regression. Computational Statistics & Data Analysis, 44, 273–295.