The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
bdCreate_hdf5_matrix() error paths.dim() on a closed HDF5Matrix returns
NULL and length() returns 0 instead of
signalling an error.hdf5_close_all() is silent when there is nothing to
close.impute_snps() could crash, deadlock or silently corrupt its
output on matrices with over 4000 columns.hdf5_apply() with "CrossProd" /
"tCrossProd" returned an all-zero result for inputs of
1,048,576 elements or more.svd() / prcomp() with
scale = TRUE now stop on a constant column instead of
silently returning all-zero singular values. scale() is
unchanged (NaN, as in base R).threads argument of svd() /
prcomp() is now honoured; it had no effect on the internal
matrix products.impute_snps() and the omics filters now propagate
dimnames, and the filters report a clear error when they remove every
row or column.crossprod(), %*% and
tcrossprod() between HDF5Matrix objects stored
in different files no longer fail.memory.size() (defunct since R 4.2.0), which inflated the
block-size and preload budget.checkHDF5File() uses the portable HDF5 C API instead of
the deprecated H5::H5File::isHdf5(), fixing a downstream
link failure with Rhdf5lib >= 1.32.filter_low_coverage(): pcent counts
entries equal to 3, not NA.filter_maf(): the documented threshold direction was
inverted (variants with MAF at or below the threshold are removed).svd() / prcomp() now record whether the
decomposition was exact or approximate (new attributes plus a
message()), and the new svd_auto_threshold()
returns the size boundary. See ?svd.HDF5Matrix.svd() / prcomp() warn when the requested
threads cannot be honoured.hdf5_remove() and HDF5Matrix$remove()
delete a dataset from a file.bdCorr_matrix(): fixed a C++ exception
(Expecting a single value: [extent=0]) that could abort the
R session under recent Rcpp versions. The returned list evaluated an
optional Nullable<bool> argument in a boolean
context, which now resolves to as<bool>(NULL); the
unwrapped logical values are used instead. This resolves the CRAN
r-devel example check failure."c++ exception" prefix from all
error messages so failures surface as ordinary R errors rather than text
that reads like a leaked C++ exception (per CRAN request).
Error-handling control flow is unchanged.cbind() / rbind(): documentation no longer
states that plain R matrices are accepted and auto-converted (they are
not). Convert inputs first with hdf5_matrix() /
hdf5_create_matrix().RSpectra from the README system-requirements
list (it is not a package dependency; the bundled C++ Spectra library is
used internally).Ops.HDF5Matrix definition
into a single documented method and documented the 1xn / nx1 broadcast
behavior.qr(..., method = "tsqr"): fixed incorrect results
(loss of orthogonality in Q) when the last row-block had fewer rows than
columns (m mod block_size < n). The last block is now
merged with the previous one, guaranteeing all blocks satisfy the rank
condition required by the local Householder QR step. Affected
configurations: any (m, n, threads) where
m mod max(2n, ceil(m / max(4, threads * 4))) < n.
Discovered via a systematic regression test covering 202 (m, n, threads,
block_size) combinations.
pseudoinverse() now correctly zeroes singular values
at or below the tolerance (1e-9), fixing incorrect results for
rank-deficient or near-singular matrices.
Fixed silent failure on matrix conversion errors in
bdCrossprod(), bdtCrossprod(), and
bd_wproduct(); these now raise informative errors.
Removed unreachable code in the correlation HDF5 backend (no functional effect).
pseudoinverse.matrix() for consistent in-memory
support alongside the existing HDF5Matrix method.pseudoinverse() and
prcomp(rank. =) documentation to match actual
behavior.crossprod(),
tcrossprod(), aggregation functions, scale(),
and solve(): when input fits within 20% of available RAM, a
single HDF5 read + BLAS call + single write replaces the block
loop.scale() PATH 2 (block-wise streaming) is now
parallelized with OpenMP.hdf5matrix_options(paral = TRUE) now forces PATH 2 (OMP
streaming) in %*%, crossprod(),
tcrossprod(), aggregations, and scale(),
giving explicit thread control regardless of matrix size.getAvailableMemoryMB() now uses platform-native
detection (macOS: host_statistics64; Linux:
/proc/meminfo), replacing the hardcoded 4 GB fallback.target_blocks = max(4, nthreads * 4)).rbind() and cbind() with 3 or more
HDF5Matrix arguments no longer produce dataset names
exceeding 1400 characters. Auto-generated names are now
rbind_N_<uid> with a random 8-character identifier,
also preventing “dataset already exists” collisions. Two-argument calls
are unchanged (A_rbind_B).paral = FALSE being silently ignored in
multiplication() PATH 2/3.#pragma omp critical(accessFile) in
crossprod(), tcrossprod(), and aggregation
PATH 2 (race condition fix).Cholesky_decomposition_intermediate_hdf5 on macOS ARM64
(removed throw from OpenMP parallel region).scale(): added paral, threads
parameters; PATH 1/PATH 2 @details;
@seealso hdf5matrix_options().split(), hdf5_apply(),
hdf5_reduce(): clarified calling conventions and two-level
access pattern. Fixed legacy example.hdf5_import
example.This is a major release of BigDataStatMeth. It introduces a new
HDF5Matrix user-facing interface, standard S3 methods for
HDF5-backed matrices, and an extended C++ infrastructure for block-wise
statistical computing with HDF5 files.
HDF5Matrix interface for working with
matrices stored in HDF5 files.dim(), [,
[<-, %*%, crossprod(),
tcrossprod(), scale(), cor(),
svd(), prcomp(), qr(),
chol(), and solve().hdf5matrix_options()
for common settings such as parallel execution, number of threads, block
size, and HDF5 compression.HDF5Matrix objects.%*%, crossprod(), tcrossprod(),
cbind(), and rbind().colSums(), rowSums(), colMeans(),
rowMeans(), colVars(), rowVars(),
colSds(), rowSds(), colMins(),
rowMins(), colMaxs(), rowMaxs(),
mean(), var(), and sd().scale(), sweep(), and cor().svd(), prcomp(),
qr(), chol(), solve(),
eigen(), and pseudoinverse().diag(), diag<-(), diag_op(),
and diag_scale().bd* utilities for
specialized workflows that do not map directly to standard R
generics.list_datasets() for inspecting datasets stored in
HDF5 files.hdf5_close_all() for closing open HDF5 handles
managed by the package.hdf5matrix_options().HDF5Matrix interface and standard R methods.help("BigDataStatMeth").extdata
directory.minimumBlockSize promoted from int to
long in
Cholesky_decomposition_intermediate_hdf5 and
Inverse_of_Cholesky_decomposition_intermediate_hdf5 to
prevent overflow before promotion to double for
sqrt(). Detected by gcc-ASAN on R-hub.matrixSvdBlock.hpp): wrong singular value was evaluated in
the rank-truncation check, causing incorrect nzeros counts.H5SL_insert from H5I_register) when the
number of hierarchical levels q >= 2:
Next_level_SvdBlock_decomposition_hdf5 is now sequential;
the outer block loop already provides sufficient parallelism.nev parameter not being applied to per-block
truncation in First_level_SvdBlock_decomposition_hdf5: the
parameter was declared in the function signature but never used, causing
unnecessarily large intermediate matrices and a final SVD over more
components than requested.bdCCA_hdf5_rcpp().These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.