The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
Quickstart: stress-testing a pre-specified
MMRM
A confirmatory trial has one pre-specified mixed model for repeated
measures (MMRM). ballast refits it under families of perturbations of
the modelling decisions and the data, judges every refit against the
pre-specified estimate, confidence interval and significance level, and
classifies the result as robust, sensitive or fragile.
The pre-specified analysis
Describe the analysis once. The example uses the
fev_data set that ships with mmrm, restricted to 100
subjects so the vignette builds quickly.
library(ballast)
dat <- mmrm::fev_data[mmrm::fev_data$USUBJID %in% paste0("PT", 1:100), ]
spec <- stress_spec(dat, response = "FEV1", subject = "USUBJID", visit = "AVISIT",
treatment = "ARMCD", reference = "PBO", covariates = ~ RACE + SEX + FEV1_BL,
contrast_visit = "VIS4", covariance = "us",
method = "Satterthwaite", vcov = "Asymptotic")
spec
#> <stress_spec> pre-specified
#> data: 400 rows, 100 subjects, visits VIS1 VIS2 VIS3 VIS4
#> model: FEV1 ~ RACE + SEX + FEV1_BL + ARMCD + AVISIT + ARMCD:AVISIT + us(AVISIT | USUBJID)
#> contrast: TRT - PBO at VIS4
#> inference: Satterthwaite df, Asymptotic vcov, alpha 0.05
stress_fit(spec)
#> family label kind role is_base converged note estimate
#> 1 base pre-specified alternative sensitivity TRUE TRUE 3.872365
#> stderr df lcl ucl p n_subj n_obs aicc param
#> 1 2.069905 70.88075 -0.2550291 7.999759 0.06550197 99 279 1754.542 NA
#> covariance method vcov covariates alpha
#> 1 us Satterthwaite Asymptotic RACE + SEX + FEV1_BL 0.05
If you already have an mmrm fit,
stress_spec_mmrm(fit, reference = "PBO") builds the same
object; from an ADaM BDS data set, stress_spec_adam()
does.
Families
Each family function returns rows in one schema. A few of them:
stress_covariance(spec, c("us", "cs", "ar1h"))[, c("label", "estimate", "stderr", "p", "is_base")]
#> label estimate stderr p is_base
#> 1 us 3.872365 2.069905 0.06550197 TRUE
#> 2 cs 3.532005 1.492399 0.01867436 FALSE
#> 3 ar1h 3.398204 2.088478 0.10824537 FALSE
stress_inference(spec)[, c("label", "estimate", "stderr", "df", "p")]
#> label estimate stderr df p
#> 1 Satterthwaite 3.872365 2.069905 70.88075 0.06550197
#> 2 Kenward-Roger 3.872365 2.018843 70.88075 0.05912569
#> 3 Kenward-Roger (linear) 3.872365 2.093147 70.88075 0.06847896
#> 4 Residual 3.872365 2.069905 267.00000 0.06246752
#> 5 Sandwich, between-within 3.872365 2.000699 174.00000 0.05454936
#> 6 Jackknife sandwich, between-within 3.872365 2.070826 174.00000 0.06316891
stress_covariates(spec)[, c("label", "estimate", "p")]
#> label estimate p
#> 1 without RACE 4.707980 0.02963354
#> 2 without SEX 3.832708 0.07144548
#> 3 without FEV1_BL 4.046893 0.05880214
#> 4 no covariates 4.829839 0.03027465
Deletions, re-randomisation, reference-based and delta-adjusted
multiple imputation, other estimators, a greedy search for the smallest
set of subjects that changes the conclusion, intercurrent-event
strategies and continuous perturbation curves are the other families;
see the reference index.
Pre-specify and run in one call
A plan fixes the families and their settings before unblinding;
printing it gives the text a statistical analysis plan can carry.
standard_plan() is the recommended set.
plan <- stress_plan(spec, covariance = list(structures = c("us", "cs", "ar1h")), inference = list(),
covariates = list(), weights = TRUE, loo = list(by = "RACE"), permute = list(n = 99),
reference = NULL, estimator = list(estimators = c("complete_cases", "ancova")),
fragility = list(max_k = 2))
plan
#> Pre-specified stress test
#>
#> Pre-specified analysis: FEV1 ~ RACE + SEX + FEV1_BL + ARMCD + AVISIT + ARMCD:AVISIT + us(AVISIT | USUBJID); contrast TRT minus PBO at VIS4; Satterthwaite df, Asymptotic variance; alpha 0.05; equal LS-mean weights.
#>
#> Perturbation families, run after unblinding exactly as listed; every refit is judged against the pre-specified estimate,
#> its 95% confidence interval and alpha. Classification: fragile if any counted alternative reverses direction or leaves the
#> pre-specified confidence interval; otherwise sensitive if any crosses alpha; otherwise robust. Specifications reached by two
#> families are counted once; tipping-point grids are searches and are not counted; deletions are counted separately.
#>
#> 1. Within-subject covariance structure: structures = us, cs, ar1h
#> 2. Degrees-of-freedom method and variance estimator: defaults
#> 3. Covariate set: defaults
#> 4. Least-squares-mean weighting (equal versus observed margins): defaults
#> 5. Leave-one-out deletions: by = RACE
#> 6. Re-randomisation test: n = 99
#> 7. Estimator class: estimators = complete_cases, ancova
#> 8. Smallest set of subjects that changes the conclusion: max_k = 2
res <- run_plan(plan, seed = 1)
res
#> Pre-specified stress test
#> 8 families, 22 rows, 17 counted; 0.1 min
#> Base: estimate 3.87, 95% CI -0.255 to 8, p = 0.0655 (alpha 0.05)
#> Counted: 14 alternative analyses, 3 deletions; 4 significant, 4 crossed alpha
#> Outside base CI: 0; direction reversed: 0; largest shift: 1.74 base SE
#> Verdict: SENSITIVE over all counted rows; SENSITIVE over the 17 sensitivity analyses (same estimand; 0 supplementary rows); marginal: base p near alpha
#> Rows that crossed alpha:
#> family label estimate p
#> covariance structure cs 3.53 0.0187
#> covariate set without RACE 4.71 0.0296
#> covariate set no covariates 4.83 0.0303
#> re-randomisation test 99 permutations 3.87 0.0300
stress_report() writes a Markdown report with the
figures, stress_tables() the Word or RTF tables and the
census as CSV, stress_html() an interactive page, and
stress_log() the audit trail with the plan’s hash.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.
