The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

IRT recovery, SBC, and misspecification evidence

Simulation-based calibration (SBC) and parameter recovery answer different software-validation questions. Recovery checks whether fitted estimates reproduce known generating quantities under declared scenarios; SBC checks calibration of posterior computation under the declared generative model.

design <- eyeprocess_irt_recovery_design(sample_size = 250L, n_items = 12L,
                                          missing_rate = c(0,.15), testlet_sd = c(0,.35),
                                          replications = 5L, seed = 20260811L)
head(design)
#>   scenario_id sample_size n_items missing_rate testlet_sd replications     seed
#> 1   IRTREC001         250      12         0.00       0.00            5 20260811
#> 2   IRTREC002         250      12         0.15       0.00            5 20260811
#> 3   IRTREC003         250      12         0.00       0.35            5 20260811
#> 4   IRTREC004         250      12         0.15       0.35            5 20260811
eyeprocess_irt_misspecification_suite()
#>                       scenario                perturbation
#> 1                    reference                        none
#> 2             local_dependence       testlet random effect
#> 3                  missingness               MCAR omission
#> 4 discrimination_heterogeneity    wider log-discrimination
#> 5              lower_asymptote    non-zero lower asymptote
#> 6               latent_mixture two-component theta mixture
#>                            target
#> 1            calibration baseline
#> 2              local independence
#> 3         missing-data robustness
#> 4              item heterogeneity
#> 5            guessing sensitivity
#> 6 latent distribution sensitivity

Exact recovery fitting currently requires mirt; absence produces a gated result. SBC rank summaries reuse eyeprocess’s rank-diagnostic infrastructure.

The SBC workflow follows the logic of Talts et al., Validating Bayesian Inference Algorithms with Simulation-Based Calibration: https://arxiv.org/abs/1804.06788.

Simulation-based calibration view

The figure below is generated from a small deterministic known-item simulation. It is a computational calibration diagnostic under the declared generative model, not evidence of empirical model adequacy.

viz_items <- data.frame(
  item_id = paste0('I', 1:8),
  a = seq(0.8, 1.5, length.out = 8),
  b = seq(-1.5, 1.5, length.out = 8),
  c = 0,
  d = 1
)

viz_sbc <- eyeprocess::run_eyeprocess_irt_ability_sbc(
  items = viz_items,
  replications = 20L,
  posterior_draws = 19L,
  theta_grid = seq(-5, 5, length.out = 201L),
  seed = 902L
)

stopifnot(
  inherits(viz_sbc, 'eye_irt_sbc_evidence')
)

plot(viz_sbc)
Simulation-based calibration diagnostic for the deterministic known-item example.

Simulation-based calibration diagnostic for the deterministic known-item example.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.