Fitting Wixted Lineup-Memory Models

What these models are

A witness viewing a simultaneous lineup sees several faces at once and must somehow reduce several impressions to one decision. Signal detection theory says each lineup member generates a memory strength — culprits tend to generate more than innocent members, but the distributions overlap. What it does not say is how the several strengths are combined into a single choice. That is an open theoretical question, and the models fitted here are three answers to it (Wixted, Vul, Mickes & Wilson, 2018).

Independent Observations. The witness attends to the strongest-feeling face and identifies it if that feeling passes a criterion. Each member is evaluated on its own, so the decision depends on the maximum strength and nothing else. This is the model most people have in mind when they picture a witness scanning a lineup.

Ensemble. The witness judges the best face relative to the lineup as a whole — a face must stand out from its neighbours, not merely feel familiar. Formally the decision variable is the maximum strength minus the lineup mean. This predicts something the first model does not: putting a suspect among high-quality foils should make identification harder, because the comparison standard rises.

Integration. The witness pools evidence across the whole lineup, summing strengths rather than picking out a maximum. It is included as a historically important comparator rather than a recommended default; Wixted et al. found it generally inferior.

BEST-Rest is available for exact replication work, but it is a linear rescaling of Ensemble rather than a distinct account. For a lineup of size \(k\),

\[ DV_{\text{Ensemble}} = \frac{k-1}{k}\, DV_{\text{BEST-Rest}} \]

and the criteria rescale by the same factor, so the two produce identical fits. Reporting both as though they were separate model-selection evidence would be double counting.

What this interface does and does not cover

fit_lineup_models() fits the fair simultaneous-lineup case: one procedure or condition per call, with trial-level target_present, identification and ordered numeric confidence.

Not covered, and what each would require:

The first two are the most common in practice, since real lineups are rarely perfectly fair. If your lineup has null foils — members nobody ever chooses — see lineup_table() and the effective-size measures in the main vignette before fitting anything here.

Installing the engine

pyWitness is optional and lives in an isolated virtual environment. Neither loading the package nor running its checks installs any Python software. Run this once, interactively — it is slow, and it is deliberately not executed when this vignette is built:

library(r4lineups)
install_pywitness()
check_pywitness_deps(initialize = TRUE)

The installer pins an audited upstream commit, and every fitted result records that revision along with the imported engine version. The bridge refuses to run against an environment whose version does not match.

check_pywitness_deps(initialize = FALSE)
#> pyWitness environment: r4lineups-pywitness 
#> Environment exists: TRUE 
#> Python initialized: TRUE 
#> pyWitness available: TRUE 
#> Audited revision: e726dcfc09423d0e0ff7f46c8e3a711040293eba

Preparing data

Data are simulated here at the level of individual foils. foil_bias gives each foil its own mean memory strength, so the foils are not interchangeable — some are plausible alternatives to the suspect and one is chosen only rarely.

condition_a <- as.data.frame(simulate_lineup_data(
  n_tp        = 500,
  n_ta        = 500,
  d_prime     = 1.6,
  c_criterion = 0.9,
  lineup_size = 6,
  conf_levels = 9,
  foil_bias   = c(0.4, 0.2, 0.0, -0.4, -1.0),
  seed        = 404
))

# Put confidence on the 0-100 scale used elsewhere in the package
condition_a$confidence <- round(seq(20, 100, length.out = 9))[condition_a$confidence]

str(condition_a[c("target_present", "identification", "confidence")])
#> 'data.frame':    1000 obs. of  3 variables:
#>  $ target_present: logi  FALSE FALSE TRUE FALSE FALSE FALSE ...
#>  $ identification: chr  "suspect" "filler" "suspect" "suspect" ...
#>  $ confidence    : num  50 30 50 20 20 40 20 20 40 20 ...

knitr::kable(
  as.data.frame.matrix(table(condition_a$target_present,
                             condition_a$identification)),
  caption = "Response counts by lineup type, simulated data (500 target-present and 500 target-absent lineups of size 6). Rows: FALSE = target-absent, TRUE = target-present."
)

Requirements: target_present logical or 0/1; identification one of "suspect", "filler", "reject"; confidence finite numeric with larger values meaning greater confidence. Under the supported fair-lineup design, target-absent suspect choices are pooled with target-absent filler choices. The upstream processor needs at least one observation in every aggregate TP/TA response category, though cells within individual confidence bins may be zero.

Fitting

fits <- fit_lineup_models(
  condition_a,
  models          = c("independent", "ensemble", "integration"),
  lineup_size     = 6,
  confidence_bins = c(0, 60, 80, 100),
  variance        = "equal",
  shared_variance = "estimated",
  control         = list(maxiter = 5000L, method = "Nelder-Mead")
)

knitr::kable(
  as.data.frame(fits$comparison)[, c("model", "converged", "n_parameters",
                                     "pearson_chisq", "df", "p_value")],
  digits = 4, row.names = FALSE,
  caption = "Model fits, simulated data generated by the MAX (Independent Observations) rule."
)

For an Independent-Observations model with zero shared variance use shared_variance = "zero", which selects pyWitness’s dedicated class for that exact limit. Shared variance cancels from Ensemble and BEST-Rest and is fixed at zero for them. Unequal signal variance is requested with variance = "unequal".

Starting values can be supplied per model:

fit_started <- fit_lineup_models(
  condition_a,
  models = c("independent", "ensemble"),
  confidence_bins = c(0, 60, 80, 100),
  starts = list(
    independent = c(targetMean = 1.5, c1 = 0.8, c2 = 1.4, c3 = 2.0),
    ensemble    = c(targetMean = 1.5, c1 = 0.7, c2 = 1.2, c3 = 1.7)
  )
)

fit_started$comparison$converged
#> [1] TRUE TRUE

Can the fit tell the generating rules apart?

An informative check is whether fitting picks out the rule that generated the data. simulate_lineup_data() implements the same three decision rules, so this can be explored directly. One mismatch should be kept in mind: the data below are simulated with unequal foils (foil_bias), while fit_lineup_models() assumes a fair lineup with exchangeable fillers. Every fitted model is therefore somewhat misspecified, even the one matching the generating rule.

simulate_under <- function(rule, seed = 404) {
  d <- as.data.frame(simulate_lineup_data(
    n_tp = 500, n_ta = 500, d_prime = 1.6, c_criterion = 0.9,
    lineup_size = 6, conf_levels = 9, decision_rule = rule,
    foil_bias = c(0.4, 0.2, 0.0, -0.4, -1.0), seed = seed
  ))
  d$confidence <- round(seq(20, 100, length.out = 9))[d$confidence]
  d
}

recovery <- do.call(rbind, lapply(c("max", "ensemble", "integration"), function(rule) {
  f <- fit_lineup_models(simulate_under(rule),
                         models = c("independent", "ensemble", "integration"),
                         lineup_size = 6, confidence_bins = c(0, 60, 80, 100))
  cc <- as.data.frame(f$comparison)
  get <- function(m, col) cc[[col]][cc$model == m]
  data.frame(
    `Generated by`     = rule,
    `Independent chi2` = round(get("independent", "pearson_chisq"), 1),
    `Independent p`    = signif(get("independent", "p_value"), 3),
    `Ensemble chi2`    = round(get("ensemble", "pearson_chisq"), 1),
    `Ensemble p`       = signif(get("ensemble", "p_value"), 3),
    `Integration chi2` = round(get("integration", "pearson_chisq"), 1),
    `Integration p`    = signif(get("integration", "p_value"), 3),
    check.names = FALSE
  )
}))

knitr::kable(
  recovery, row.names = FALSE,
  caption = "Pearson chi-square and its p-value by fitted model, for data generated under each decision rule. Simulated, 500 target-present and 500 target-absent lineups per row."
)

Read the p-values, not the chi-squares. The models do not have the same degrees of freedom — Independent Observations estimates five parameters against Ensemble’s four, leaving it one fewer residual degree of freedom — so the model with more parameters is favoured by a raw chi-square comparison purely by being more flexible. Ranking the models by chi-square alone is not a comparison the statistic licenses.

A p-value here is the probability of a discrepancy this large if that model were true; it is not the probability that the model is the correct one, and p-values from non-nested models do not rank them on a common scale. With that in mind, two things emerge.

In the MAX and Integration rows the model matching the generating rule fits acceptably and the others are rejected outright.

The Ensemble row does not. Independent Observations and Ensemble return essentially identical p-values, and neither can be rejected. This is not a case of the wrong model winning; it is a case of the two models being indistinguishable on these data.

Independent Observations and Ensemble make very similar predictions for a fair lineup. Separating them requires either much larger samples or a design built to separate them — most obviously one that manipulates foil quality, since it is precisely there that the two accounts disagree: Ensemble makes the comparison standard depend on the other lineup members and Independent Observations does not. simulate_lineup_data() can vary foil quality through foil_bias, so the question is directly explorable.

Wixted et al. reached a similar conclusion, finding both models plausible with a modest overall advantage for Ensemble. Treat a small chi-square difference between these two as uninformative, and note that a single dataset per rule — as here — is an illustration rather than a recovery study. A proper one would simulate many datasets per generating model and report how often each is selected.

Interpreting the output

knitr::kable(
  as.data.frame(fits$parameters), digits = 4, row.names = FALSE,
  caption = "Parameter estimates, simulated data."
)

fits$diagnostics
#>         model converged                                status iterations error
#> 1 independent      TRUE Optimization terminated successfully.        407  <NA>
#> 2    ensemble      TRUE Optimization terminated successfully.        167  <NA>
#> 3 integration      TRUE Optimization terminated successfully.        330  <NA>
fits$engine
#> $name
#> [1] "pyWitness"
#> 
#> $version
#> [1] "0.1.dev0+ge726dcfc0"
#> 
#> $repository
#> [1] "https://github.com/lmickes/pyWitness.git"
#> 
#> $audited_revision
#> [1] "e726dcfc09423d0e0ff7f46c8e3a711040293eba"
#> 
#> $installation_source
#> [1] "https://github.com/lmickes/pyWitness/archive/e726dcfc09423d0e0ff7f46c8e3a711040293eba.zip"
#> 
#> $environment
#> [1] "r4lineups-pywitness"
#> 
#> $objective_correction
#> [1] "Rejection-cell variance denominators corrected per fit; installed engine unchanged"
#> 
#> $durable_python_object
#> [1] FALSE

plot(fits, type = "comparison")

plot(fits, type = "fit")

plot(fits, type = "roc")

Technical Details

References

Further Reading

• ?fit_lineup_models and ?install_pywitness
• vignette("model_comparison") for the 2-HT, EIG and full-ROC models, which run without any Python engine
• vignette("simulation_power_analysis") for the simulator and its decision rules