The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Validation for temporally dependent pupil data

Start with the prediction target

Temporally dependent samples should not automatically be treated as exchangeable observation-level units. create_pupil_validation_plan() makes the intended target explicit before computing validation.

sim <- simulate_pupil_timecourse(
  n_participants = 6,
  trials_per_participant = 4,
  sampling_frequency = 10,
  time_window = c(-0.3, 1),
  baseline_window = c(-0.3, 0),
  seed = 21
)
contract <- create_pupil_contract(
  outcome_col = "pupil_mm",
  participant_col = "participant_id",
  trial_col = "trial_id",
  condition_col = "condition",
  time_col = "event_time",
  pupil_unit = "millimetres",
  sampling_frequency = 10
)
prepared <- prepare_pupil_timecourse(sim$data, contract)

trial_plan <- create_pupil_validation_plan(
  prepared,
  target = "new_trial_known_participant",
  K = 4,
  seed = 21
)
trial_plan
#> <gp3bayes_pupil_validation_plan>
#>   Target: new_trial_known_participant
#>   Strategy: grouped_trial_kfold_within_participant
#>   Model-support rows: 330
#>   Leakage detected: FALSE
#>   Automatic strategy selection: FALSE

Other supported targets distinguish a new participant and a future time segment.

participant_plan <- create_pupil_validation_plan(
  prepared,
  target = "new_participant",
  K = 3,
  seed = 21
)
future_plan <- create_pupil_validation_plan(
  prepared,
  target = "future_segment",
  future_fraction = 0.2
)
participant_plan
#> <gp3bayes_pupil_validation_plan>
#>   Target: new_participant
#>   Strategy: grouped_participant_kfold
#>   Model-support rows: 330
#>   Leakage detected: FALSE
#>   Automatic strategy selection: FALSE
future_plan
#> <gp3bayes_pupil_validation_plan>
#>   Target: future_segment
#>   Strategy: leave_future_segment_out
#>   Model-support rows: 330
#>   Leakage detected: FALSE
#>   Automatic strategy selection: FALSE

Execution

trial_cv <- validate_pupil_model(
  pupil_fit,
  trial_plan,
  execute = TRUE
)
pupil_validation_table(trial_cv)
plot_pupil_validation(trial_cv)

Grouped K-fold uses complete validation groups. The future-segment target uses a chronological holdout and explicit refitting rather than presenting ordinary observation-wise PSIS-LOO as a universal time-series solution. Validation answers only the declared predictive question.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.