The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Multivariate analysis and survival

Beyond regression, depictr covers principal component analysis, clustering (with quality diagnostics) and survival curves.

Principal component analysis

pca_plot() runs a PCA on the numeric columns of a data frame and draws a biplot: the observations projected onto two components, with the variable loadings as arrows. scree_plot() shows how much variance each component explains.

num <- c("rainfall", "fertiliser", "soil_ph", "yield")
pca_plot(crop_yield, cols = num, group = "treatment",
         title = "Crop-yield PCA")

scree_plot(crop_yield, cols = num)

Both functions also accept a ready-made prcomp() object, so you can analyse once and plot several views:

pc <- prcomp(crop_yield[num], scale. = TRUE)
pca_plot(pc, components = c(1, 3))

Clustering

cluster_plot() runs k-means and shows the clusters, with convex hulls and labelled centroids, on the first two principal components, so it works for any number of variables. dendrogram_plot() draws a hierarchical-clustering tree and can cut it into k groups.

cluster_plot(crop_yield, cols = num, k = 3, seed = 1,
             title = "Crop-yield clusters")

region_means <- aggregate(
  cbind(stress, sleep_hours, life_satisfaction, age, income) ~ region,
  data = wellbeing_survey, FUN = mean
)
rownames(region_means) <- region_means$region
dendrogram_plot(region_means[-1], k = 2, title = "Regions clustered")

Choosing the number of clusters

k_diagnostic() evaluates a cluster-quality criterion across a range of k and suggests a value. The criterion is the average silhouette width (the default, after Rousseeuw, 1987), the within-sum-of-squares elbow or the gap statistic (Tibshirani et al., 2001).

kd <- k_diagnostic(crop_yield, k_range = 2:6, cols = num,
                   method = "silhouette")
kd  # the diagnostic curve, with the suggested k marked

The suggested k and the underlying table are attached as attributes:

attr(kd, "suggested")
[1] 2
knitr::kable(attr(kd, "k_table"), digits = 3)
k avg_silhouette
2 0.242
3 0.237
4 0.220
5 0.227
6 0.224

The silhouette width is undefined for a single cluster, and its search starts at 2. The elbow’s default search starts at 1. Because the elbow is the candidate farthest from the line joining the first and last points of the curve, it never picks either end of the range. The gap statistic compares the within-cluster dispersion against a null reference, so, unlike the other two criteria, it can also support k = 1, and its default search starts there. It draws its reference samples at random, so the chunk is seeded for a reproducible build.

set.seed(1)
k_diagnostic(crop_yield, cols = num, method = "gap")

The gap statistic suggests k = 1, so the three-cluster partition drawn above is a description imposed on the data rather than evidence of groups within it.

silhouette_plot() shows the quality of a given clustering, with one bar per observation grouped by cluster. It labels the average silhouette width of each cluster and draws the overall average as a dashed line. Wide positive bars are well-placed observations, and a bar below zero marks an observation that may belong to a neighbouring cluster, because it sits closer to that cluster than to its own.

set.seed(1)
cl <- kmeans(scale(crop_yield[num]), centers = attr(kd, "suggested"),
             nstart = 10)$cluster
silhouette_plot(crop_yield, cl, cols = num,
                title = "Silhouette widths by cluster")

The overall average here is 0.24, and only six of the two hundred bars fall below zero, none of them by much. Kaufman and Rousseeuw’s (1990) rough guide treats an average at or below 0.25 as no substantial structure, so this figure agrees with the gap statistic. Since k-means returns clusters whether or not the data contains any, the silhouette is one way of asking which case you are in.

The same diagnostics apply to the numeric profile of the wellbeing survey:

wb_num <- c("age", "income", "stress", "sleep_hours",
            "exercise_days", "life_satisfaction")
wkd <- k_diagnostic(wellbeing_survey, k_range = 2:6, cols = wb_num,
                    method = "wss")
wcl <- kmeans(scale(na.omit(wellbeing_survey[wb_num])),
              centers = attr(wkd, "suggested"), nstart = 10)$cluster
silhouette_plot(na.omit(wellbeing_survey[wb_num]), wcl,
                title = sprintf("Wellbeing survey, k = %d",
                                attr(wkd, "suggested")))

Survival curves

survival_plot() draws Kaplan-Meier curves (Kaplan & Meier, 1958) with stepwise confidence limits and censoring marks. The limits come from Greenwood’s formula (Greenwood, 1926) on the log-log scale, which keeps them between 0 and 1, and they cover better in small samples than untransformed limits (Borgan & Liestøl, 1990). conf_type = "plain" draws the untransformed limits and conf_type = "log" the default of survival::survfit(). The estimate is computed in base R, so no modelling package is needed. You can pass follow-up times and an event indicator directly, a data frame or a survival::survfit object, whose limits are recomputed at the requested conf_level.

In the clinical_trial dataset, the treatment arm has a lower hazard than the placebo arm, so the curves separate. Turning on the three publication annotations gives a survminer-style figure. The annotations are a number-at-risk table beneath the curves, the log-rank test of the difference as a subtitle and dashed guides to the median survival of each arm whose curve reaches 0.5. A survival curve is monotone-decreasing, so its bottom-left corner is always empty, and legend_inside = TRUE puts the group legend there.

survival_plot(
  clinical_trial$time, clinical_trial$event, group = clinical_trial$arm,
  risk_table = TRUE, median_line = TRUE, logrank = TRUE, legend_inside = TRUE,
  x_lab = "Months", title = "Overall survival by arm"
)

The log-rank p-value is tiny. Median survival on placebo is 18.2 months, while the treatment arm’s curve stays above 0.5 to the 36-month study end, so its median is not reached and it gets no guide. Because its event times are longer, that arm is also more often right-censored at the study end, and its curve shows a denser run of censoring marks.

References

Borgan, Ø., & Liestøl, K. (1990). A note on confidence intervals and bands for the survival function based on transformations. Scandinavian Journal of Statistics, 17(1), 35–41.
Greenwood, M. (1926). A report on the natural duration of cancer (Vol. 33, pp. 1–26). His Majesty’s Stationery Office.
Kaplan, E. L., & Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282), 457–481. https://doi.org/10.1080/01621459.1958.10501452
Kaufman, L., & Rousseeuw, P. J. (1990). Finding groups in data: An introduction to cluster analysis. Wiley. https://doi.org/10.1002/9780470316801
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53–65. https://doi.org/10.1016/0377-0427(87)90125-7
Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(2), 411–423. https://doi.org/10.1111/1467-9868.00293

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.