The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Diagnostics, classification and uncertainty

Regression diagnostics

residual_diagnostics_plot() assembles the classic panel. influence_plot() and qq_plot() each show one of its checks in more detail. We fit a linear model to the crop-yield trial.

fit <- lm(yield ~ rainfall + fertiliser + soil_ph, data = crop_yield)
residual_diagnostics_plot(fit, title = "Crop-yield model")

influence_plot(fit)

The bubble area is Cook’s distance, the standard measure of an observation’s influence on the fitted coefficients (Cook, 1977).

vif_plot() checks for multicollinearity among the predictors. Because the crop-yield predictors are close to orthogonal, we create collinearity by adding a soil-moisture measure that is largely driven by rainfall.

set.seed(1)
collinear <- crop_yield
collinear$soil_moisture <- 0.05 * collinear$rainfall +
  rnorm(nrow(collinear), sd = 2)
vif_plot(lm(yield ~ rainfall + soil_moisture + fertiliser, data = collinear))

Rainfall and soil moisture each carry a variance inflation factor near 6, above the rule-of-thumb line at 5, while fertiliser stays at 1.

Diagnostics for a generalised linear model

For a binary glm, the raw residuals take only a few values, so a plain residual-versus-fitted plot is hard to read. binned_residual_plot() instead splits the data into equal-count bins of fitted values and plots the mean residual per bin (Gelman & Hill, 2007) against a band of +/- 2 standard errors. These are the standard errors that the fitted model implies for each bin’s mean. Under a correct model, about 95% of the points sit inside the band, scattered around zero.

We model the adverse-event outcome from the clinical trial on the baseline biomarker, age and treatment arm. The outcome is rare, with a base rate of about 10%.

gfit <- glm(adverse_event ~ biomarker + age + arm,
            data = clinical_trial, family = binomial)
binned_residual_plot(gfit, title = "Binned residuals: adverse-event model")

For a glm, residual_diagnostics_plot() replaces two of the classic panels with GLM-appropriate ones: binned residuals and a Q-Q plot of randomised quantile residuals, which are standard-normal under a correct model regardless of the response distribution.

residual_diagnostics_plot(gfit, title = "Adverse-event model")

Classification on an imbalanced outcome

The precision-recall, gains and lift charts are built for an imbalanced outcome such as the adverse event. depictr’s classification plots each read a binomial glm directly, a pair of vectors or, to compare several models, a named list of models.

The ROC curve reports the AUC. Passing youden = TRUE marks the Youden’s J operating point (the threshold maximising sensitivity + specificity - 1):

roc_curve_plot(gfit, youden = TRUE)

When the positive class is rare, the precision-recall curve is more informative than the ROC curve, because it ignores the many true negatives. The baseline is the positive prevalence, and f1 = TRUE marks the maximum-F1 operating point:

pr_curve_plot(gfit, f1 = TRUE)

Comparing two models in one figure

Passing a named list overlays one colour-coded curve per model, with a per-curve AUC (or average precision) in the legend. Here, the full model is compared with a biomarker-only model.

reduced <- glm(adverse_event ~ biomarker, data = clinical_trial,
               family = binomial)
models <- list(Full = gfit, `Biomarker only` = reduced)

An ROC curve rises towards the top-left corner, so its bottom-right corner is always empty. legend_inside = TRUE puts the per-model legend in that corner, and the figure then needs no right-hand margin.

roc_curve_plot(models, youden = TRUE, legend_inside = TRUE)

pr_curve_plot(models, f1 = TRUE)

For ranking and targeting tasks, the cumulative gains and lift charts show how many positive cases are captured as more of the score-ordered population is targeted. Both charts again overlay the two models.

gain_plot(models, legend_inside = TRUE)

lift_plot(models, legend_inside = TRUE)

Choosing an operating point

For choosing a threshold, threshold_plot() complements the ROC and PR curves. It sweeps the decision threshold across the range of scores and plots how each metric trades off against the others. A dashed line marks the Youden threshold and a dotted line the maximum-F1 threshold.

tp <- threshold_plot(gfit, title = "Metrics across the decision threshold")
tp

attr(tp, "thresholds")   # the Youden and max-F1 thresholds
   youden        f1 
0.1315540 0.2128417 

Calibration

calibration_plot() checks whether predicted probabilities match observed frequencies. Each bin’s observed rate carries a Wilson binomial confidence interval, so bins backed by few observations are not over-interpreted. Such bins are common in the sparse upper tail when the outcome is rare.

calibration_plot(gfit, bins = 6)

For the confusion-matrix heatmap, threshold = "youden" reuses the operating point that the ROC curve marks.

confusion_matrix_plot(gfit, threshold = "youden", normalise = "row")

Uncertainty

posterior_plot() summarises posterior, bootstrap or simulation draws as a distribution per parameter. The example shows the posterior draws from a Bayesian fit of the lexical-decision model as point-and-interval forests.

draws <- readRDS(
  system.file("extdata", "lexdec_draws.rds", package = "depictr")
)
posterior_plot(draws[c("conditionunrelated", "modalityauditory",
                       "word_frequency")],
               labels = c(conditionunrelated = "condition",
                          modalityauditory = "modality",
                          word_frequency = "word frequency"),
               style = "interval", title = "Posterior estimates (ms)")

Power curves

power_curve_plot() reads a simr::powerCurve() object or a tidy data frame, so a slow power simulation does not have to be re-run to redraw it. The package includes a powerCurve object produced by a simr analysis of the lexical-decision design, which simulated power for the word-frequency slope as the number of participants grows. We read it from disk.

pc <- readRDS(
  system.file("extdata", "powercurve_lexdec.rds", package = "depictr")
)
power_curve_plot(pc, x_lab = "Number of participants",
                 title = "Power for the word-frequency effect")

The analysis ran 100 simulations at each of 12, 24, 36, 48 and 60 participants. Power for the word-frequency slope is 88% with 12 participants and 99% with 24, and it stays at 98% or above for the larger samples. The confidence band is widest at the smallest sample size, where its lower limit falls on the 80% target line, so 24 participants is the first design the simulation clears comfortably.

Composing and saving

Combine any of these with arrange_plots() and save with save_plot():

panel <- arrange_plots(
  qq_plot(fit), influence_plot(fit),
  ncol = 2, title = "Diagnostics", tag_levels = "A"
)
panel

save_plot() writes it at a print-ready 300 dpi by default and creates any missing directories. A PDF or EPS embeds its fonts and a TIFF is LZW-compressed, as journals ask of submitted figures. The help page, ?save_plot, lists the formats that PLOS, Nature and Elsevier accept.

save_plot("figures/diagnostics.png", panel, width = 7, height = 3.2, dpi = 300)

References

Cook, R. D. (1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. https://doi.org/10.1080/00401706.1977.10489493
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press. https://doi.org/10.1017/CBO9780511790942

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.