The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
with() on a vimmi object returns a
mice-compatible mira (elements call,
call1, nmis, analyses) instead of
an anonymous list, so mice::pool(),
summary(pool(fits)) and mice::getfit() run
unchanged. Code that indexed the old list directly
(fits[[i]], lapply(fits, ...)) should use
mice::getfit(fits) or fits$analyses.vimpute() returns are type-stable: the
result is always the imputed data, classed like the input (data.frame
in, data.frame out; data.table in, data.table out). With
tune = TRUE or pred_history = TRUE the
diagnostics are attached as attr(result, "tuning_log") /
attr(result, "pred_history") instead of switching the
return to a bare list.vimpute()’s default uncert is now
"pmm" (random draw among the 5 nearest donors):
default numeric imputations are observed donor values with an honest
distribution, instead of deterministic conditional means. Set
uncert = "none" for the previous behaviour. Default
m > 1 runs are now stochastic between imputations.
rangerImpute(), xgboostImpute() and
regressionImp() keep their deterministic behaviour
(uncert = "none" pinned internally).vimpute(m > 1) now bootstraps by
default: boot defaults to TRUE for
multiple imputation – each of the m imputations refits its
models on a bootstrap sample, so parameter uncertainty is propagated
into the draws (approximately proper multiple imputation in combination
with the default PMM) – and to FALSE for single imputation.
Set boot = FALSE explicitly for the previous single-fit
behaviour.uncert = "pmm" and "midastouch"
now perform true predictive mean matching (Little 1988): donors
are matched on their predicted values – scored by the trained
model on the observed rows, out-of-bag for a no-bootstrap ranger fit –
instead of on their observed values. Matching on observed values
selected donors whose values happened to lie near the prediction
(including gross outliers from other covariate regions) and its donor
spread shrank with n; predicted-value matching carries residual-scale
variability, as in mice and Hmisc::aregImpute().uncert != "none": the imputed category is sampled
from the predicted class probabilities on every sweep (the convergence
criterion tracks the noise-free most-probable class), so early stopping
no longer strips factor variables of between-imputation variability;
under uncert = "none" factor imputation is now fully
deterministic (previously sequential = FALSE drew
stochastically regardless of uncert).diabetes is now a synthetic data set,
generated by Matthias Templ from
mlbench::PimaIndiansDiabetes2 with the synvey package: the
team that collected the original Pima Indians Diabetes data has asked
for redistribution to stop. Column names, types, dimensions and the
Outcome levels are unchanged, so code runs as before, but
the values – and any results computed from them – differ from earlier
VIM versions; 0 pregnancies is now a valid value rather than
NA.vimpute()’s
imputation methods are resolved through a package-level registry, and
register_vimpute_method() adds a user-defined method backed
by any mlr3 learner pair in one call –
e.g. register_vimpute_method("cart", learner = list(regr = "regr.rpart", classif = "classif.rpart"), packages = "rpart"),
or lightgbm via mlr3extralearners – usable everywhere the
built-in names work (global method, per-variable lists,
method-keyed learner_params, tune = TRUE via
an optional search_space hook).
vimpute_methods() lists the registered methods,
unregister_vimpute_method() removes user-added ones. The
six built-ins are seeded through the same contract; unsupported-method
errors now name the offending method and list what is registered.vimpute(data, spec = list(Sleep = vs_ranger(num.trees = 300, tune = TRUE), NonD = vs_robust(donorcond = ">= 0"), .default = vs_ranger()))
bundles a variable’s method, learner parameters,
formula/predictors, tune, PMM
settings, makeNA and donorcond into one object
instead of nine parallel arguments. The vs_*() constructors
(and vimpute_spec() for registered methods) validate
learner parameters eagerly against the method’s
parameter set, so a typo fails at the constructor call, not
mid-imputation. The flat per-variable arguments keep working unchanged;
specs compile to exactly them.vimpute(data, Sleep ~ Dream + Span | ranger(tune = TRUE), NonD ~ . | robust(donorcond = ">= 0"), .default = vs_ranger(), m = 20, seed = 1).
A plain-column right-hand side restricts the predictors
(works for every method, including ranger/xgboost); a right-hand side
with transformations (s(x), log(x),
I(x^2)) becomes a model formula
(formula-capable methods only). Grammar formulas compile to specs. Note:
vimpute()’s signature now has ... in second
position for the grammar – arguments after data must be
passed by name (they always were in all documented usage).vimpute() gains seed for whole-run
reproducibility (applied once at entry, as in mice; the m
imputations still differ from each other).vimpute() gains predictors: per-variable
predictor control – the equivalent of mice’s
predictorMatrix (named list or 0/1 matrix), working for
every method including ranger and xgboost
(where formula is rejected). A variable’s
formula takes precedence over its predictors
entry.vimpute() gains visit_sequence:
"asis", "increasing.na",
"decreasing.na", or an explicit permutation of the
NA-variables.vimpute() gains tuned_params to apply (and
reuse) tuned hyperparameters without running the tuner; each
tuning_log entry now carries its chosen parameters in
$params.vimpute() gains
tune_control = vimpute_tune_control(budget = , folds = , tuner = , batch_size = ):
the tuning evaluation budget, resampling folds, tuner and batch size are
user-controllable (defaults keep the built-in data-size heuristics);
tuning_log entries record the budget and folds used.
vimpute_search_space() exports the built-in per-learner
search spaces (the starting point for
register_vimpute_method() search_space hooks).
Nested resampling is intentionally not offered – the goal is good
imputations, not unbiased generalisation estimates.m > 1 and tune = TRUE, tuning now
runs once (in the first imputation) and the chosen
parameters are shared by all m imputations – previously
every imputation re-tuned independently, conflating tuner noise with
missing-data uncertainty. The vimmi object carries the
tuning report in $tuning_log.vimpute() warns (naming the variable) and falls
back to a featureless learner for that variable.vimmi objects store per-iteration chain statistics
(mean/variance of the imputed values per variable, iteration and
imputation) plus the seed, and plot(vimmi)
draws mice-style convergence trace plots. Single runs expose the same
trace data as attr(result, "chain").vim_as_mids(): the documented name for the
vimmi-to-mids conversion (as.mids.vimmi() is kept as the
historical alias; despite the dotted name it was never an S3 method –
mice::as.mids() is not a generic).makeMissing(): amputation generator
for simulation studies (the variable-wise mice::ampute()
counterpart) – MCAR/MAR/MNAR mechanisms with exact per-variable
proportions, driver weights, and an
attr(., "where") indicator that plugs directly into
evaluation()/nrmse()/pfc().overimpute(): model-agnostic
calibration diagnostic (the Amelia::overimpute() analogue)
– the observed cells of a variable are overimputed fold by fold with
multiple draws; print() reports the empirical interval
coverage and plot() draws the observed-vs-imputed
calibration plot. Works with any method, spec, or grammar
configuration.OOBerror analogue): every vimpute() run
reports NRMSE (numeric) / PFC (factor) per variable as
attr(result, "model_error") – out-of-bag for ranger,
honestly labelled in-sample for the other learners – and
print(vimmi) shows it.plot(vimmi, "density") and
plot(vimmi, "strip"): observed-vs-imputed
distribution diagnostics (the
mice::densityplot()/stripplot() analogues),
complementing the "chains" convergence traces.makeMissing() -> vimpute(m = 5) ->
chain/density diagnostics -> Rubin pooling via
with()/mice::pool() and
vim_as_mids() -> tuning with tune_control
-> overimpute() calibration ->
evaluation() against the simulated truth.makeMissing() scenarios, NRMSE on the amputed cells,
runtime) comparing vimpute ranger/robust and kNN() with
mice and missRanger (both guarded Suggests); precomputed at demo scale
(vignettes/precompute.R), paper-scale via one
constant.boot +
uncert = "pmm"), the boot-free PMM variant, the
textbook-proper (boot + normalerror), and a deliberately
improper (bootstrap without residual noise) configuration –
demonstrating the anti-conservative pooled SEs the improper setting
produces and the warning that guards against it.?vimpute states the missingness assumptions (MAR
incl. MCAR; MNAR caveat with pointers to makeMissing()
sensitivity simulation).imputeCellM() gains a chained-equations
data.frame interface: imputeCellM(data) now
imputes every variable with missing values – each is regressed on all
remaining variables, sweeps run with deterministic predictions until the
imputed values stabilise (maxit, eps), and the
requested uncert step is applied once after convergence –
returning the same
data_imputed/cellweights/converged/iterations
list as imputeCellIRMI().
imputeCellwise(data, method = "cellM") dispatches to it
instead of erroring (“cellM requires a formula”), and the formula
interface gains uncert = "none" (deterministic predictions;
most-probable category for factors).method = "restricted" no longer has uncertainty
draws layered over its constrained solutions: the 7.3.0 default
uncert = "pmm" replaced the solver’s rule-satisfying value
with a donor draw that knows nothing about the rules – under a rule
y >= 4 with observed donors 1 and 2, the “restricted”
imputation was 2 (CI-red on all platforms since the
restrictionRegression merge, which predated the new default). Registry
entries can now pin their variables’ uncertainty mechanism
(uncert_override): restricted pins "none",
silently for the default and with a warning when a draw mechanism was
requested explicitly. The per-variable model-quality report is also
skipped for restricted (model_error = FALSE in the registry
entry) – its in-sample predict re-ran the conic solver once more per
variable for a metric of marginal value, so
save_optimization_problem = TRUE again records exactly one
problem per imputed variable.uncert = "normalerror" / "resid"
no longer draw from an in-sample residual scale: for learners
that expose no scale of their own (ranger, xgboost),
sigma_hat and the residual pool were derived from the
model’s predictions on its own training rows. A forest’s
in-sample predictions are near-interpolating, so the estimated scale
came out at roughly half the true predictive spread (0.57 against a
held-out 1.16 on a linear DGP with residual sigma 1):
"normalerror" injected half the noise it should,
"resid" sampled from a residual pool that was far too
tight, and the pooled intervals of the textbook-proper
boot + normalerror configuration under-covered badly (0.83
/ 0.73 against a nominal 0.95 at 30% / 50% missingness). Where the
fitted model exposes out-of-bag predictions – ranger stores them for
free, in training-row order – both are now derived from those, which
track the predictive spread to ~1%; every other learner keeps the
in-sample fallback. Learners reporting a proper scale of their own
(lm, lmrob, glmrob,
gam, robgam) never took this path and are
unchanged, as is the default uncert = "pmm", which matches
donors on predicted values and needs no scale estimate at all. The same
residual pool feeds robustboot = "stratified" (the default)
and "residual", whose good/bad split was previously
computed on the same over-tight residuals. A regression test pins both
the scale and the residual pool against a held-out predictive SD.imputeCellIRMI() no longer errors with
“subscript out of bounds” when DDC refuses columns:
cellWise::DDC()’s checkDataSet() drops
constant, too-discrete and NA-heavy columns (on the colic
data 11 of 19 continuous columns), so stdResid covers only
the retained submatrix while the weight-initialisation loop indexed all
continuous columns. The DDC weights are now mapped back through
colInAnalysis/rowInAnalysis; refused rows and
columns keep univariate weights, and DDC’s console chatter (“The final
data set we will analyze has …”) is captured instead of leaking to the
console.imputeCellEM(), imputeCellIRMI(),
imputeCellMCD(), imputeCellMM(),
imputeCellReg()) built per-variable model formulas by
pasting column names, so with duplicated names – e.g. after
colnames(x) <- substr(colnames(x), 1, 15), which maps
mucous_membranes_col and
mucous_membranes_group to the same name – the response
resolved to the first column of that name: the model was fitted
on the wrong variable, factor imputations drew from the wrong level set
and were silently turned into NA by the levels-mismatch,
with repeated “response appeared on the RHS” multinom warnings. The
functions now fit under internal positional names and restore the user’s
names on exit.imputeCellM()’s
formula interface never received its weights:
model.frame() evaluates the weights argument
in data and then in environment(formula) – the
caller’s environment, where the internal row-weight vector does not
exist – so the cell-weighted fit always errored and silently
fell back to the unweighted model (with a “Multinomial model failed”
warning). The row weights are now passed as a column of the fitting
data.classif.glm_rob no longer inverts binary
classifications: the robust logistic learner behind
vimpute(method = "robust") attached the fitted binomial
probability – P of the second factor level, the
glm/glmrob convention – to the first
level, so binary factor imputations drew from inverted class
probabilities and the deterministic argmax picked the less
likely class: worse-than-chance imputation exactly where the signal is
strong (PFC 0.70 instead of 0.13 on the diabetes data). The multiclass
one-vs-rest path was oriented correctly and is unchanged; a direction
regression test now pins both paths."robust"/"robgam" provide).evaluation() no longer returns NaN when
one variable type has no missing cells (0/0 guarded).evaluation() gains where, the documented
name for the amputed-cell mask (matching makeMissing()’s
"where" attribute); the historical m keeps
working, supplying both errors.eps now bounds the per-variable
relative change (numeric: mean squared change of the imputed
values divided by the variance of the observed values; factors: share of
changed categories), and the run stops when the largest
per-variable change stays below eps – previously the raw
changes were summed, so eps was meaningless across data
scales (a variable measured in thousands could block convergence
forever, and one variable could mask another). The full
iterations-by-variables change matrix is returned as
attr(result, "convergence").batch_size = 1 for the random search).cellWeightsMCD() keeps exact-constant columns at weight 1
instead of feeding them to covMcd() (every subsample is
singular there, which produced repeated warnings on each outer
iteration), and cellIRWLS() skips the lmrob.S
initialisation for rank-deficient designs (e.g. a constant column
aliased with the intercept), using the ridge-regularised cell-weighted
OLS init instead.future plan active at entry is
restored instead of being forced to "sequential".kNN()/gowerD() used every core, which
CRAN’s incoming checks flag (“examples with CPU time > 2.5 times
elapsed time”). options(VIM.ncores = ) now sets the number
of threads; without the option VIM uses at most 2 threads under
R CMD check (CRAN’s two-core policy) and OpenMP’s default
otherwise. Results do not depend on the thread count. The
xgboostImpute() example was reduced to a single two-target
call (CRAN’s 5-second example limit).R CMD check within CRAN’s
10-minute budget (the incoming pre-test on Windows took 14–17 minutes,
330 s of it tests and 280 s vignette rebuilds). The two simulation
vignettes (Benchmarking imputation methods, Validating
multiple-imputation properness) are now precomputed – their code is
executed by vignettes/precompute.R in the source repository
and the output is embedded, so CRAN only renders text; the comparison
chunks of the vimpute vignette use single-sweep imputations
(sequential = FALSE) and the overimpute() demo
three folds. Long-running tests – hyperparameter tuning, the
multiple-imputation and vimpute-default regression tests, and the two
vimpute integration scripts under tests/ – now run only
when NOT_CRAN=true (set by
devtools::check()/test() and by the GitHub
Actions workflow); CRAN runs the fast core of the suite.vimpute() gains keep_all_columns (default
TRUE): the full dataset is returned, with columns excluded
via considered_variables passed through unchanged (matching
kNN()/hotdeck()/irmi()); set
FALSE for the previous considered-only shape.vimpute() warns when m > 1 cannot
produce between-imputation variability (no boot,
uncert, or stochastic pmm), so improper
multiple imputation is no longer silent.vimpute() returns ordered-factor columns as ordered
factors (previously flattened to plain factors; the
m > 1 path also lost the level order).vimpute(tune = TRUE) now runs with
sequential = FALSE (was a silent no-op).vimpute() handles per-variable method lists correctly:
a named length-1 list validates the variable name, and an unnamed
per-column list maps by column position.rangerImpute() and xgboostImpute() forward
their hyperparameters to the backend learner.regressionImp() uses lm/glm
as documented, falling back to regularized regression only when
needed.vim_complete().
VIM exports no complete() generic of its own:
mice and tidyr both export one, so an exported
VIM generic would mask theirs (and be masked by them) and would make
packages importing VIM and mice/tidyr
wholesale emit “replacing previous import” at load time. The same
function is registered as a method on mice::complete() and
tidyr::complete(), so complete(obj, 1) keeps
working whenever either package is attached.evaluation() supports
vartypes = "guess".irmi(mi > 1) returns a list of imputations again
(was a single mangled data.frame under the default
imp_var = TRUE).imputeRobust(): method = "gamRob" and
uncert = "wresid" now work (previously crashed);
uncert is validated with a clear error; the PMM donor pool
no longer includes initialised values.imputeRobustChain() repaired (previously imputed zeros
or crashed on most paths).imputeCellIRMI(),
imputeCellM(), imputeCellMCD()) no longer
scale the design matrix by cell weights, which had made them impute
worse than the median on clean data; the default
init_weights is now "ddc".imputeCellReg() cell-weight computation fixed (could
produce negative weights).imputeCellMCD(): documented that
boot = TRUE does not yet propagate parameter
uncertainty.kNN()/gowerD(): semi-continuous
(mixed) distance variables are now range-scaled like
numeric ones, so a large-scale mixed variable no longer dominates the
neighbour search.kNN(weightDist = TRUE) no longer produces
NaN imputations when distances exceed 1 (e.g. with
methodStand = "iqr").?kNN and ?gowerD document the distance
standardisation and the NA-sentinel convention.car moved from Imports to Suggests (Box-Cox implemented
natively).vimpute() compatibility and validationrangerImpute(), xgboostImpute(), and
regressionImp() delegate to vimpute()vimpute()mlr3 backend for a
flexible imputation method.hotdeck() when ord_var and variable
overlapirmi(), kNN(), hotdeck() and
regressionImp()). Thanks @wolfgangrannetbauer (#44,
#45)tableMiss(): A table that highlights missing and
imputed values via colors (#47).ranger::ranger() (#35)irmi() with logical and integer columns
(#42)gowerD()maxCat() and
sampleCat()collision addedimp_var and imp_suffix to
irmi(), so it is more consistent with the other functions
(#27)kNN(), to use random
forest in combination with kNN()matchImpute() for imputing
randomly within groupsimpNA in hotdeck()irmi() with factors (#13). Thanks @Deleetdkirmi() (#23)kNN() and data.tablehotdeck()data.frame is passed to
irmi() (#6)kNN(): weightDist to use
the distances for the k nearest neighbours as weightswhich.minN() is not used anymore,
instead there is a C++ function, kNN() is now about 1.6
times faster on a replication (100x) of the sleep datasethotdeck()hotdeck() if no
ord_var is definedhotdeck() with makeNAkNN() useImputedDist if the
imputed values of a variable should be used in subsequent imputation of
another variable.irmi() with newer version of nnet
(multinom) and if residual scale can not be computed (noise)modelFormula in irmi()irmi()hotdeck() based on data.table -> faster and
quite stablekNN()makeNA in
kNN()imp_var now updates existing TF imp_vars
(with warning)regressionImp()These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.