LiblineaR 2.50-1: New Features, Correctness, and Performance

One-class SVM: outlier and novelty detection

type=21 fits a one-class SVM. Unlike every other type, it is unsupervised – it learns what “normal” data looks like from data alone. target is still required as an argument (its length must match the number of rows of data) but its values are ignored. bias must be <= 0 (this solver has no bias-term concept), and nu (a number in (0, 1), default 0.5) is an upper bound on the fraction of training points that may be treated as outliers.

predict() returns 1 for inliers and -1 for outliers; decisionValues=TRUE additionally returns the raw signed distance to the decision boundary (positive means inlier – useful if you want to rank points by how anomalous they are, rather than just a yes/no call).

x <- iris[, 1:4]
y <- iris[, 5]

m_oc <- LiblineaR(x, y, type = 21, bias = -1, nu = 0.1)
p_oc <- predict(m_oc, newx = x, decisionValues = TRUE)
table(p_oc$predictions)
#> 
#>   1 
#> 150
head(p_oc$decisionValues)
#>        [,1]
#> [1,] 543.29
#> [2,] 506.37
#> [3,] 499.75
#> [4,] 492.30
#> [5,] 541.00
#> [6,] 589.24

regularizeBias: excluding the intercept from regularization

By default, LiblineaR’s bias term (the intercept) is penalized by the L1/L2 regularization exactly like every other coefficient. Setting regularizeBias=FALSE excludes it, so it is fit freely. This is only valid combined with bias=1 exactly, and only for type 0, 2, 5, 6 or 11; other combinations raise a clear error. Most users should leave this at its default (TRUE) – an unregularized intercept is a specialized choice, useful mainly when you have strong reasons to believe the regularization penalty should apply only to the feature weights, not to the overall offset.

m_default <- LiblineaR(x, y, type = 0, bias = 1)
m_unreg   <- LiblineaR(x, y, type = 0, bias = 1, regularizeBias = FALSE)

findC now supports SVR (type=11)

For perspective: heuristicC(data) and findC=TRUE both help pick a good cost, but they work very differently. heuristicC() is a fast, closed-form formula – it inspects data once and returns a single suggested cost, with no training involved at all. findC=TRUE is an actual empirical search – it repeatedly trains and cross-validates across a doubling sequence of cost values and returns whichever one empirically scored best. heuristicC() is a cheap starting guess; findC is expensive but data-driven tuning, and the two compose naturally: useInitC=TRUE (the default) uses the cost you pass in – often heuristicC()’s own suggestion – as findC’s starting point for the search.

findC=TRUE previously only searched for a good cost with type=0 and type=2. It now also supports type=11 (L2-regularized L2-loss SVR), additionally searching svr_eps at the same time. For this type only, findC returns a named list list(C=..., svr_eps=...) instead of a plain number, since there are now two values to report; type 0 and 2 are unaffected and still return a plain number.

xr <- as.matrix(iris[, 1:3])
yr <- iris[, 4]

best <- LiblineaR(xr, yr, type = 11, findC = TRUE, cross = 5)
#> Warning in LiblineaR(xr, yr, type = 11, findC = TRUE, cross = 5): No value
#> provided for svr_eps. Using default of 0.1
best
#> $C
#> [1] 1
#> 
#> $svr_eps
#> [1] 0

m_svr <- LiblineaR(xr, yr, type = 11, cost = best$C, svr_eps = best$svr_eps)

Method

Four engines were compared: the LIBLINEAR command-line tools at v2.10 and v2.50, and the LiblineaR R package before and after this upgrade. All four were trained on byte-identical data (the same train/test split of the same matrix) with matching hyperparameters (cost, bias, epsilon, svr_eps). Four datasets were used: two synthetic classification problems (2-class and 3-class, 600 rows each), one regression problem (600 rows), and one larger problem (8000 rows, 300 columns) reserved for timing.

A methodological caveat, confirmed empirically before trusting any of the numbers below: LIBLINEAR’s dual coordinate-descent solvers (and the L1-regularized solvers) shuffle their working set using a random number generator. LiblineaR routes this through R’s own RNG (set.seed()-controlled), while the plain command-line tools use C’s own unseeded rand(). For these solver types, coefficients are not expected to match between LiblineaR and the command-line tools, even at the identical LIBLINEAR version – they are two different, independent, equally valid solutions to the same problem. What is expected, and checked below, is that predictions agree almost exactly. Primal, non-randomized solvers (logistic regression and L2-loss SVM/SVR, type 0, 2, 11) use no randomness at all, so their coefficients are expected to match the command-line tools almost to floating-point precision – and are the real fidelity check.

Result 1: does LiblineaR still correctly wrap LIBLINEAR?

Maximum absolute coefficient difference between LiblineaR and the pristine command-line tools, same LIBLINEAR version, across the 3 correctness datasets (28 train/predict comparisons per version):

fidelity <- data.frame(
  type = c(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 11L, 12L, 13L),
  solver = c("L2R_LR (primal)", "L2R_L2LOSS_SVC_DUAL", "L2R_L2LOSS_SVC (primal)",
             "L2R_L1LOSS_SVC_DUAL", "MCSVM_CS", "L1R_L2LOSS_SVC", "L1R_LR",
             "L2R_LR_DUAL", "L2R_L2LOSS_SVR (primal)", "L2R_L2LOSS_SVR_DUAL",
             "L2R_L1LOSS_SVR_DUAL"),
  max_abs_diff_v210 = c(1.72e-10, 9.65e-03, 1.98e-10, 3.60e-02, 1.38e-02,
                         3.93e-01, 6.82e-01, 7.32e-03, 1.84e-10, 7.93e-02, 1.71e-01),
  max_abs_diff_v250 = c(1.75e-10, 9.65e-03, 1.93e-10, 3.60e-02, 1.38e-02,
                         3.93e-01, 6.82e-01, 7.32e-03, 1.59e-10, 7.93e-02, 1.71e-01)
)
knitr::kable(fidelity, digits = 4,
  col.names = c("type", "solver", "max|diff| vs CLI (2.10)", "max|diff| vs CLI (2.50)"))

Primal types (0, 2, 11) match the pristine command-line tools to 1e-10, at both versions – direct evidence that LiblineaR correctly passes data through to the same underlying solver, unaffected by the upgrade. Every dual/L1 type shows a larger difference, exactly as the caveat above predicts – and across all 54 such comparisons (both versions, all 3 datasets), minimum prediction agreement was 98.75%, with most exactly 100%, confirming these are two valid solutions to the same problem, not a correctness gap.

Result 2: what changed for an ordinary user

Maximum absolute coefficient difference between LiblineaR before and after the upgrade, same 3 datasets:

upgrade <- data.frame(
  type = c(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 11L, 12L, 13L),
  solver = fidelity$solver,
  max_abs_diff = c(0.0789, 0, 0.2281, 0, 0, 0, 0, 0, 0.0021, 0, 0)
)
knitr::kable(upgrade, digits = 4, col.names = c("type", "solver", "max|diff| (2.10-26 vs. 2.50-1)"))

Only the 3 primal Newton-solved types (0, 2, 11) show any drift at all: LIBLINEAR replaced its trust-region Newton solver with a faster-converging line-search variant partway through this version range, refined further by later releases. Every dual/L1 type was bit-for-bit identical on these ordinary, well-conditioned datasets – LIBLINEAR 2.50’s lower iteration cap on dual coordinate descent (see Result 3) was never actually reached on data this well-behaved. Predictive performance (accuracy for classification, mean squared error for regression) matched to 4 decimal places for every type on every dataset.

Result 3: what actually changed under harder conditions

LIBLINEAR 2.50 caps its dual coordinate-descent solvers at 300 iterations (down from 1000 in earlier versions) and, for three solver types, automatically switches to the primal solver instead of running out the clock if that cap is reached. Result 2 shows this is invisible on easy-to-fit data. It stops being invisible on a harder problem. On a larger, more weakly-separated dataset (8000 rows, 300 columns):

large <- data.frame(
  type = c(0L, 1L, 7L, 11L, 12L),
  solver = c("L2R_LR (primal)", "L2R_L2LOSS_SVC_DUAL", "L2R_LR_DUAL",
             "L2R_L2LOSS_SVR (primal)", "L2R_L2LOSS_SVR_DUAL"),
  metric_old = c(0.9188, 0.9219, 0.9200, 1.0234, 1.0502),
  metric_new = c(0.9181, 0.9188, 0.9200, 1.0235, 1.0234),
  time_old_s = c(0.133, 0.632, 1.035, 0.075, 3.400),
  time_new_s = c(0.147, 0.288, 0.998, 0.086, 1.201),
  speedup = c(0.90, 2.19, 1.04, 0.87, 2.83)
)
knitr::kable(large, digits = 4,
  col.names = c("type", "solver", "accuracy/MSE (2.10-26)", "accuracy/MSE (2.50-1)",
                "time 2.10-26 (s)", "time 2.50-1 (s)", "speedup"))

Two solver types where the iteration cap is actually reached on this dataset (type 1 and type 12, both dual) show a 2.2x and 2.8x speedup respectively – and type 12’s test-set mean squared error also improved (1.050 → 1.023), because the previous solver spent its full 1000-iteration budget grinding toward a barely-better optimum on a genuinely slow-converging problem, while the current one recognizes the cap, switches solver, and reaches a comparably good (here, slightly better) answer in a fraction of the time. The primal Newton types (0, 11) are essentially a wash in either direction (0.87-0.90x) – expected, since that solver’s algorithm changed but not its overall cost.