The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

TRexSelector R package guide

T-Rex+Tikhonov: a general penalty matrix

From the elastic net to a penalty matrix

The elastic net adds the ridge penalty \(\frac{\lambda_2}{2} \|\boldsymbol\beta\|_2^2\) to the lasso. Tikhonov regularization in general form [1], [2] replaces it by \(\frac{\lambda_2}{2} \|\boldsymbol\Gamma \boldsymbol\beta\|_2^2 = \frac{\lambda_2}{2} \boldsymbol\beta^\top \boldsymbol K \boldsymbol\beta\) with \(\boldsymbol K = \boldsymbol\Gamma^\top \boldsymbol\Gamma\). The matrix encodes what the analyst knows about the coefficients. \(\boldsymbol K = \boldsymbol I\) gives the ridge penalty of the elastic net, a first-difference operator \(\boldsymbol\Gamma\) gives the graph Laplacian of a chain and asks neighbouring coefficients to be equal (smoothness), a graph Laplacian in general [3], [4] couples the variables along the edges of a graph, a group-mean matrix reproduces the penalty of the informed elastic net, and a block-diagonal centering matrix asks the members of a group to share a coefficient. The Tikhonov T-Rex selector runs the terminating Tikhonov-net solver TIKNET, a pathwise homotopy algorithm, with such a matrix inside the random experiments.

A smooth coefficient profile

The active variables are a contiguous block with equal coefficients, the case a chain Laplacian is made for.

set.seed(2718)
n <- 120
p <- 60
true_support <- 21:30
beta <- numeric(p)
beta[true_support] <- 1
X <- matrix(rnorm(n * p), n, p)
y <- as.numeric(X %*% beta + rnorm(n))

trex_gamma_to_k() builds \(\boldsymbol K\) from an operator \(\boldsymbol\Gamma\) and returns a sparse matrix of the Matrix package. A dense base matrix is accepted everywhere a penalty matrix is.

gamma <- diff(diag(p))                # first differences, (p - 1) x p
if (requireNamespace("Matrix", quietly = TRUE)) {
  K <- trex_gamma_to_k(gamma)
  class(K)
} else {
  K <- crossprod(gamma)
}
#> [1] "dgCMatrix"
#> attr(,"package")
#> [1] "Matrix"
K[1:4, 1:4]
#> 4 x 4 sparse Matrix of class "dgCMatrix"
#>                 
#> [1,]  1 -1  .  .
#> [2,] -1  2 -1  .
#> [3,]  . -1  2 -1
#> [4,]  .  . -1  2

Running the selector

The Tikhonov control is required, because it carries \(\boldsymbol K\). The penalty weight is fixed here. A base control that is passed has to name the solver of the selector, "TIKNET". Without a base control the selector runs the defaults of trex_control() with that solver.

ctrl <- trex_control(solver = "TIKNET", K = 10)
tik <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
                                tik_control = trex_tikhonov_control(tikhonov_K = K, lambda_2 = 1),
                                control = ctrl)
tik$select()
tik$selected_indices
#>  [1] 21 22 23 24 25 26 27 28 29 30
trex_fdp(tik$selected_indices, true_support)
#> [1] 0
trex_tpp(tik$selected_indices, true_support)
#> [1] 1
tik$lambda2_used
#> [1] 1

TRexTikhonovSelector inherits the fields of TRexSelector and adds lambda2_used, lambda2_cv_diagnostics, lambda2_pilot and dummy_clusters. A selector runs once, as every selector does.

Choosing lambda_2

lambda_2 < 0 (the default) hands the choice to lambda2_method. Three rules exist.

Without a grid the rules run on trex_default_lambda2_grid(), the grid the C++ core holds for them.

A supplied lambda_2 is used as it is, on the scale of the standardized columns. A grid, pilot_grid or the grid of a power rule, is stated for columns of unit norm. Under scaling_mode = "ZSCORE" the columns have the squared norm \(n - 1\), so the rules run at the grid times \(n - 1\), and one grid means the same shrinkage under every scaling mode. Every rule reports its pick on the scale of a supplied value, so lambda2_used can be passed as lambda_2.

pilot <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
                                  tik_control = trex_tikhonov_control(
                                    tikhonov_K = K,
                                    pilot_grid = c(0, 1, 10), pilot_K = 4, pilot_T = 2),
                                  control = ctrl)
pilot$select()
pilot$lambda2_used
#> [1] 0
pilot$lambda2_pilot$grid
#> [1]  0  1 10
pilot$lambda2_pilot$counts
#> [1] 13 11 10
profile <- trex_tikhonov_lambda2_power_rule(X, y, tFDR = 0.1, seed = 4,
                                            tik_control = trex_tikhonov_control(tikhonov_K = K),
                                            control = ctrl, grid = c(0, 1, 10))
profile$lambda2
#> [1] 0
profile$counts
#> [1] 10 10 10

Dummies for a structured penalty

The false discovery rate (FDR) guarantee needs dummies that are exchangeable with the null variables under the penalty. Two mechanisms take care of that. cluster_dummies = TRUE (default) draws the dummies as per-cluster coloured layers on a correlation clustering of the variables (prior_groups, or hierarchical clustering with corr_max and hc_linkage), and every dummy joins the \(\boldsymbol K\)-coupling of the variable it mirrors (the folded coupling). cluster_dummies = FALSE exists for validation runs. The field dummy_clusters reports the clusters of a run.

Group centering

penalty_structure = "GROUP_CENTERING" lets the selector build \(\boldsymbol K\) itself as the block-diagonal centering penalty \(\boldsymbol I - \boldsymbol J / p_m\) over the groups of prior_groups, so the members of a group share a coefficient. The structure needs prior_groups (group numbers from 1, one per variable), and the selector refuses it without them. An all-singleton partition is legitimate, the penalty vanishes and the path is the lasso on per-variable cluster dummies. trex_build_group_centering_k() returns this matrix for a group assignment.

groups <- rep(1:6, each = 10)
gc <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
                               tik_control = trex_tikhonov_control(
                                 penalty_structure = "GROUP_CENTERING", prior_groups = groups,
                                 lambda_2 = 1),
                               control = ctrl)
gc$select()
gc$selected_indices
#>  [1] 21 22 23 24 25 26 27 28 29 30

The solver

The selector runs "TIKNET", the pathwise homotopy algorithm of the Tikhonov net. It is the solver of the base control that applies when control is left out. trex_control() names "TLARS" by default, which the selector refuses as every other solver, so a base control that is passed sets solver = "TIKNET".

Bibliography

[1]
A. N. Tikhonov, “Solution of incorrectly formulated problems and the regularization method,” Soviet Math. Dokl., vol. 4, pp. 1035–1038, 1963.
[2]
Y. Park, L. Reichel, G. Rodriguez, and X. Yu, “Parameter determination for Tikhonov regularization problems in general form,” J. Comput. Appl. Math., vol. 343, pp. 12–25, 2018, doi: 10.1016/j.cam.2018.04.049.
[3]
A. J. Smola and R. Kondor, “Kernels and regularization on graphs,” in Learning theory and kernel machines (COLT/kernel 2003), 2003, vol. 2777, pp. 144–158. doi: 10.1007/978-3-540-45167-9_12.
[4]
F. R. K. Chung, Spectral graph theory. American Mathematical Society, 1997.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.