The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
any_near0()
.NA_real
to FBM type integer on new
Macs.big_randomSVD()
and big_crossprodSelf()
(#52).backingfile
to
big_crossprodSelf()
and big_cor()
(#170).big_univLogReg()
(#137).ind.col
in
big_prodMat()
(#154).FBM.dir
(that defaults to
tempdir()
as before). This can be used to change the
default directory used to create FBMs when calling either
FBM()
, FBM.code256()
, as_FBM()
,
big_copy()
, or big_transpose()
. Note that, if
not using the temporary directory anymore, you must clean up the files
you do not want to keep.ARMA_64BIT_WORD
.$add_columns()
.as_scaling_fun()
to create
your own fun.scaling
parameters.pcor()
(with a warning).pcor()
now returns NAs (instead of 0s) for singular
systems.big_prodVec()
, big_cprodVec()
,
big_colstats()
and big_univLinReg()
have been
recoded.pcor()
for singular systems, e.g. when
x
has all the same values.summary()
and plot()
for old (<
v1.3) big_sp_list
models.pcor()
to compute partial
correlations.Add two options in big_spLinReg()
and
big_spLogReg()
; power_scale
for using a
different scaling for LASSO and power_adaptive
for using
adaptive LASSO (where larger marginal effects are penalized less). See
documentation for details.
big_(c)prodVec()
and big_(c)prodMat()
(re)gain a ncores
parameter. Note that for
big_(c)prodMat()
, it might be beneficial to use the BLAS
parallelism (with bigparallelr::set_blas_ncores()
) instead
of this parameter, especially when the matrix A
is
large-ish.
big_colstats()
can now be run in parallel
(added parameter ncores
).Functions big_(c)prodMat()
and
big_(t)crossprodSelf()
now use much less memory, and may be
faster.
Add covar_from_df()
to convert a data frame with
factors/characters to a numeric matrix using one-hot encoding.
Add a new column $all_conv
to output of
summary()
for big_spLinReg()
and
big_spLogReg()
to check whether all models have stopped
because of “no more improvement”. Also add a new parameter
sort
to summary()
.
Now warn
(enabled by default) if some models may not
have reached a minimum when using big_spLinReg()
and
big_spLogReg()
.
In .self$nrow * .self$ncol : NAs produced by integer overflow
.Make two different memory-mappings: one that is read-only (using
$address
) and one where it is possible to write (using
$address_rw
). This enables to use file permissions to
prevent modifying data.
Also add a new field $is_read_only
to be used to
prevent modifying data (at least with <-
) even when you
have write permissions to it. Functions creating an FBM now gain a
parameter is_read_only
.
Make vector accessors (e.g. X[1:10]
)
faster.
Move some code to new packages {bigassertr} and {bigparallelr}.
big_randomSVD()
gains arguments related to
matrix-vector multiplication.
assert_noNA()
is faster.
big_increment()
.In plot.big_SVD()
,
Can now plot many PCA scores (more than two) at once.
Use coord_fixed()
when plotting PCA scores because
it is good practice.
Use log-scale in scree plot to better see small differences in singular values.
Reexport cowplot::plot_grid()
to merge multiple
ggplots.
AUCBoot()
is now 6-7 times faster.center
and scale
to
products.big_univLogReg()
for variables with no
variation. IRLS was not converging, so glm()
was used
instead. The problem is that glm()
drops dimensions causing
singularities so that Z-score of the first covariate (or intercept) was
used instead of a missing value.Use mio instead of boost for memory-mapping.
Add a parameter base.row
to
predict.big_sp_list()
and automatically detect if needed
(as well as for covar.row
).
Possibility to subset a big_sp_list
without losing
attributes, so that one can access one model (corresponding to one
alpha) even if it is not the ‘best’.
Add parameters pf.X
and pf.covar
in
big_sp***Reg()
to provide different penalization for each
variable (possibly no penalization at all).
Add %*%
, crossprod
and
tcrossprod
operations for ‘double’ FBMs.
Now also returns the number of non-zero variables
($nb_active
) and the number of candidate variables
($nb_candidate
) for each step of the regularization paths
of big_spLinReg()
and big_spLogReg()
.
warn
and return.all
of
big_spLinReg()
and big_spLogReg()
are
deprecated; now always return the maximum information. Now provide two
methods (summary
and plot
) to get a quick
assessment of the fitted models.Check of missing values for input vectors (indices and targets) and matrices (covariables).
AUC()
is now stricter: it accepts only 0s and 1s for
target
.
$bm()
and $bm.desc()
have been added in
order to get an FBM
as a
filebacked.big.matrix
. This enables using {bigmemory}
functions.float
added.big_write
added.big_read
now has a filter
argument to
filter rows, and argument nrow
has been removed because it
is now determined when reading the first block of data.
Removed the save
argument from FBM
(and
others); now, you must use FBM(...)$save()
instead of
FBM(..., save = TRUE)
.
You can now fill an FBM using a data frame. Note that factors will be used as integers.
Package
{bigreadr} has been developed and is now used by
big_read
.
options(bigstatsr.downcast.warning = FALSE)
, or you can use
without_downcast_warning()
to disable this warning for one
call.big_read
so that it is faster (corresponding
vignette updated).possibility to add a “base predictor” for
big_spLinReg
and big_spLogReg
.
don’t store the whole regularization path (as a sparse
matrix) in big_spLinReg
and big_spLogReg
anymore because it caused major slowdowns.
directly average the K predictions in
predict.big_sp_best_list
.
only use the “PSOCK” type of cluster because “FORK” can leave
zombies behind. You can change this with
options(bigstatsr.cluster.type = "PSOCK")
.
Fix a bug in big_spLinReg
related to the computation
of summaries.
Now provides function plus
to be used as the
combine
argument in big_apply
and
big_parallelize
instead of '+'
.
options(bigstatsr.cluster.type = "PSOCK")
. Uses “PSOCK” in
0.4.0.big_spLinReg
and big_spLogReg
. One will be
chosen by grid-search.big_prodMat
when using a dimension of 1
or 0.big_crossprod
,
big_tcrossprod
, big_SVD
and
big_randomSVD
(before, there was no default at all)Integrate Cross-Model Selection and Averaging (CMSA)
directly in big_spLinReg
and big_spLogReg
, a
procedure that automatically chooses the value of the \(\lambda\)
hyper-parameter.
Speed up big_spLinReg
and
big_spLogReg
(issue
#12)
big.matrix
format of package
bigmemoryThese binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.