Data
To demonstrate the function, the sleep dataset from the
VIM package is used.
data <- as.data.table(VIM::sleep)
a <- aggr(sleep, plot = FALSE)
plot(a, numbers = TRUE, prop = FALSE)
The left plot shows the amount of missings for each column in the
dataset sleep and the right plot shows how often each combination of
missings occur. For example, there are 9 rows wich contain a missing in
both NonD and Dream.
dataDS <- sleep[, c("Dream", "Sleep")]
marginplot(dataDS, main = "Missing Values")
The red boxplot on the left
shows the distrubution of all values of Sleep where Dream contains a
missing value. The blue
boxplot on the left shows the distribution of the values of Sleep where
Dream is observed.
Advanced Options
Parameter method
(default: “ranger” for all variables)
Specifies the method used for imputation of each variable. If
method is not provided, vimpute() uses
"ranger" for all variables. In this example, different
imputation methods are specified for each variable. The
NonD variable uses a robust method, Dream and
Span are using ranger, Sleep uses xgboost,
Gest uses a regularized method and class uses
a robust method.
"robust": Robust Regression Models
lmrob for numeric variables: Implements MM-estimation
for resistance to outliersglmrob for factors: Robust GLM; binary via robust
logit, multiclass via one-vs-rest
"regularized": Regularized Regression
(glmnet)
- Uses elastic net regularization
- Automatically handles multicollinearity
"ranger": Random Forest
- Fast implementation of random forests
- Handles non-linear relationships well
"xgboost": Gradient Boosted Trees
- State-of-the-art tree boosting
- Handles mixed data types well
"gam": Generalized Additive Models
(mgcv)
- Supports smooth terms for numeric predictors
- Can be used for numeric and factor targets
"robgam": Robust GAM
- Adds robust weighting / refitting to GAM-style models
- Designed for settings with potential outliers
You can provide method globally as a single method name
(applies to all variables), or as a named list per variable.
Per-variable settings can also be bundled into spec
objects — one object per variable instead of coordinating the
parallel
method/learner_params/formula/tune/…
arguments — or written as a compact formula grammar.
Both compile to the classic arguments, so results are identical:
# spec objects: everything about a variable in one place,
# learner parameters validated at the constructor call
res <- vimpute(sleep_data,
spec = list(
Sleep = vs_ranger(num.trees = 300, tune = TRUE),
NonD = vs_robust(donorcond = ">= 0"),
.default = vs_ranger()
),
seed = 1)
# the same as formula grammar: target ~ predictors | method(...)
res <- vimpute(sleep_data,
Sleep ~ . | ranger(num.trees = 300, tune = TRUE),
NonD ~ . | robust(donorcond = ">= 0"),
.default = vs_ranger(),
seed = 1)
A plain-column right-hand side restricts the predictors (works for
every method, including ranger and xgboost); a right-hand side with
transformations such as s(x) or log(x) becomes
a model formula for the formula-capable methods.
Beyond the built-ins, any pair of mlr3 learners can be registered as
an additional method with a single call — no changes to VIM required.
The registered name then works everywhere the built-in names do (global
method, per-variable lists, method-keyed
learner_params, specs and grammar):
# CART trees via mlr3's rpart learners
register_vimpute_method("cart",
learner = list(regr = "regr.rpart", classif = "classif.rpart"),
packages = "rpart")
imp_cart <- vimpute(sleep_data, method = "cart")
vimpute_methods() # lists built-ins plus registered methods
result_mixed <- vimpute(
data = data,
method = list(NonD = "robust",
Dream = "ranger",
Sleep = "xgboost",
Span = "ranger",
Gest = "regularized"),
pred_history = TRUE,
sequential = FALSE
)
dataDS <- as.data.frame(result_mixed[, c("Dream", "Sleep", "Dream_imp", "Sleep_imp")])
marginplot(dataDS, delimiter = "_imp", main = "Imputation with different Models for each Variable")
The side-by-side margin plots compare the performance of two
imputation methods: xgboost (left) and regularized (right):
xgboost handles missing values with data-driven, uneven imputations
that capture complex patterns but may be less stable, while regularized
methods produce smoother, more conservative estimates that are less
prone to overfitting. The key difference lies in flexibility (xgboost)
versus shrinkage-based stability (regularization). Note that
regularization is not robustness in the statistical sense: it stabilises
coefficients against multicollinearity and overfitting, but does not
protect against outliers – for outlier-resistant imputation use the
"robust" or "robgam" methods.
Parameter pmm
(default: FALSE)
result <- vimpute(
data = data,
method = list(NonD = "robust",
Dream = "ranger",
Sleep = "xgboost",
Span = "ranger",
Gest = "regularized"),
pmm = list(NonD = FALSE, Dream = TRUE, Sleep = FALSE, Span = FALSE , Gest = TRUE)
)
If pmm = TRUE, this is applied only to numeric target
variables.
vimpute() first computes the model prediction for a missing
entry, then compares it to observed values of that target and selects
donor candidates by smallest absolute distance to the prediction.
If pmm_k = 1 (or NULL, which defaults to 1
when PMM is active), the closest observed value is used directly.
If pmm_k > 1, the final imputed value is derived from
the k nearest donors using pmm_k_method
(e.g. "mean", "median", "random"
or a custom function).
If pmm = FALSE, raw model predictions are used.
In sequential imputation, the convergence criterion is computed from
the raw model prediction when PMM is active. This avoids unstable
stopping behavior caused by stochastic or donor-based PMM values while
still returning the PMM-imputed values in the final data.
You can provide pmm globally as a single logical value
(applies to all numeric variables), or as a named list per variable.
Parameter pmm_k
(default: NULL)
pmm_k defines how many nearest donor candidates are
considered when PMM is enabled.
You can provide pmm_k globally as a single integer value
(applies to all variables where PMM is active), or as a named list per
variable.
Parameter pmm_k_method
(default: “mean”)
pmm_k_method controls how the final donor value is
derived when pmm_k > 1. Possible values are:
"mean" (default): uses the mean of the k
nearest observed donor candidates."median": uses the median of the k nearest
donor candidates (more robust to outliers)."random": randomly draws one value from the
k nearest donor candidates."custom function": a function that gets the
k nearest donor values and returns one numeric value.
If a variable-specific list contains NULL,
vimpute() falls back to "mean" for that
variable.
You can provide pmm_k_method globally as a single
value/function (applies to all numeric variables where
pmm = TRUE and pmm_k > 1), or as a named
list per variable.
result <- vimpute(
data = data,
pmm = list(Dream = TRUE, Sleep = TRUE, NonD = FALSE),
pmm_k = list(Dream = 5, Sleep = 3),
pmm_k_method = list(
Dream = "median",
Sleep = "random"
)
)
Custom aggregation functions are also possible. The function receives
the nearest donor values and must return exactly one non-missing numeric
value.
result <- vimpute(
data = data,
pmm = list(Dream = TRUE),
pmm_k = list(Dream = 5),
pmm_k_method = list(Dream = function(x) mean(x, trim = 0.2))
)
Parameter learner_params
(default: NULL)
Use learner_params to pass method-specific settings to
the underlying learners.
This is useful if different variables are imputed with different methods
and each method should receive its own parameter configuration.
You can provide learner_params globally (when one method
is used for all variables), as a method-level list, or as a named list
per variable.
result <- vimpute(
data = data,
method = list(Dream = "ranger", Sleep = "xgboost"),
learner_params = list(
Dream = list(num.trees = 700, min.node.size = 4),
Sleep = list(nrounds = 250, max_depth = 5, eta = 0.05)
)
)
Parameter makeNA
(default: NULL)
makeNA defines values that should be treated as missing
for selected variables. This is useful when special codes such as
-999, "unknown" or "not measured"
should be imputed, while regular NA values in the same
variable should remain untouched.
result <- vimpute(
data = data,
method = list(Dream = "ranger"),
makeNA = list(Dream = -999)
)
If a variable is listed in makeNA, only the matching
values are imputed for that variable. Variables not listed in
makeNA continue to use regular NA values as
the imputation target.
Parameter donorcond
(default: NULL)
donorcond restricts which observed values are allowed to
act as donors / training observations for a target variable. Conditions
are supplied as character strings and evaluated on the target values via
the temporary variable x.
result <- vimpute(
data = data,
method = list(Dream = "ranger"),
donorcond = list(Dream = "> quantile(x, 0.1, na.rm = TRUE)")
)
This can be useful when implausible observed values should not be
used for model fitting, while the target variable itself remains part of
the imputation workflow.
Parameters boot, robustboot and
uncert
(defaults: boot = FALSE,
robustboot = "stratified",
uncert = "none")
These arguments control additional imputation uncertainty.
boot = TRUE refits the imputation model on a bootstrap
sample.robustboot controls how bootstrap rows are sampled
("standard", "stratified" or
"residual").uncert adds uncertainty to numeric predictions:
"normalerror" adds normal noise using the model scale
estimate."resid" adds sampled residuals."pmm" uses score-based predictive mean matching."midastouch" uses PMM with covariate-distance
weighting.
result <- vimpute(
data = data,
method = list(Dream = "ranger"),
boot = TRUE,
robustboot = "standard",
uncert = "normalerror"
)
If explicit pmm = TRUE is used for a variable, it takes
precedence over uncert.
Parameter m
(default: 1)
m controls multiple imputation. If
m > 1, vimpute() returns a
vimmi object instead of a single completed dataset.
mi <- vimpute(
data = data,
method = list(Dream = "ranger"),
m = 5,
boot = TRUE,
uncert = "resid",
imp_var = FALSE
)
completed_1 <- vim_complete(mi, 1)
completed_all <- vim_complete(mi, "all")
completed_long <- vim_complete(mi, "long")
The vimmi object stores the original data and the
imputed values efficiently. It can be inspected with
print() and summary(), completed datasets can
be extracted with vim_complete() – or with the familiar
complete() when mice or tidyr is
attached, since VIM registers its method on their generic – and
plot() draws mice-style convergence trace plots of the
imputation chains. with() fits a model on each completed
dataset and returns a mice-compatible mira object, ready
for mice::pool(). If the mice package is
installed, vim_as_mids() (alias
as.mids.vimmi()) converts a vimmi object to a
mice::mids object for downstream pooling workflows.
Parameter tune
(default: FALSE)
result <- vimpute(
data = data,
tune = TRUE
)
Whether to perform hyperparameter tuning (only possible if seq =
TRUE):
- When TRUE:
- Conducts randomized parameter search (after half of the
iterations)
- Uses best performing configuration
- When FALSE:
- Uses default model parameters
- Recommended: TRUE for optimal performance
You can provide tune either as a single global
TRUE/FALSE value (applies to all variables), or as a named list per
variable.
When tuning is enabled, the tuning report is attached to the imputed
dataset as an attribute: attr(result, "tuning_log"). If
pred_history = TRUE is also enabled, both attributes
("pred_history" and "tuning_log") are present
– the return value is always the imputed data itself.
Parameters nseq and eps
(default: 10 and default: 0.005)
result <- vimpute(
data = data,
nseq = 20,
eps = 0.01
)
nseq describes the number of sequential imputation
iterations. Higher values:
- Allow more refinement
- Increase computation time
eps describes the convergence threshold for sequential
imputation:
- Stops early if changes between iterations < eps
- Smaller values: Require more precise convergence but may need more
iterations
Parameter imp_var
(default: TRUE)
result <- vimpute(
data = data,
imp_var = TRUE
)
Creating indicator variables for imputed values adds “_imp” columns
(TRUE/FALSE) to mark which data points were imputed. This is
particularly useful for tracking imputation effects and conducting
diagnostic analyses.
Parameter pred_history
(default: FALSE)
print(tail(attr(result, "pred_history"), 9))
#> iteration variable index predicted_values
#> <int> <char> <int> <num>
#> 1: 10 Sleep 62 10.6
#> 2: 10 Span 4 4.5
#> 3: 10 Span 13 4.7
#> 4: 10 Span 35 3.9
#> 5: 10 Span 36 6.0
#> 6: 10 Gest 13 12.0
#> 7: 10 Gest 19 252.0
#> 8: 10 Gest 20 63.0
#> 9: 10 Gest 56 30.0
When enabled (TRUE), this option saves prediction trajectories,
allowing users to track how imputed values evolve across iterations.
This feature is particularly useful for diagnosing convergence
issues.
Since VIM 7.3.0, vimpute() always returns the imputed
dataset itself (classed like the input); the prediction history rides
along as an attribute. Access it via
attr(result, "pred_history") — the result is the imputed
data directly.
result <- vimpute(data = data, pred_history = TRUE)
# Access imputed data
head(result)
# Access prediction history
tail(attr(result, "pred_history"), 9)
