Case study: industrial image inspection

Overview

Can a decision model find defects in images of manufactured parts when it sees only a few reference images? And are its probabilities calibrated, so that a probability of 0.8 means that about 8 of 10 such parts are defective?

This vignette is a complete case study. It goes from the wording of the descriptions to a one-time test, on five datasets of industrial images. If you only want to learn the image arguments on one small dataset, read vignette("images") first.

The case study repeats the steps of a study of decision models on these images (the original study; its record is public at https://fmegahed.github.io/decision-models-for-industrial-image-qc/). It tunes the descriptions of the answers, chooses the number of reference images and the image size, fits a calibrator, and saves each setup for one final test. It uses three models:

• Jev, from TypeSafe, with decider_typesafe(). The version is pinned to jev-1.14.0, so a later version of Jev does not change the results.
• gpt-6-luna, on the Decisions API of OpenAI, with decider_openai().
• Clef-Flash, from Cloudflare, on a local SGLang server, with decider_systemone().

The results below come from live calls on 2026-10-23 and 2026-10-24. The package saved them in extdata/study_again.rds of the installed package, so this vignette builds without a key, a network or the images. The script study_again.R in the GitHub repository (folder data-raw) made them. Its code is the code of this vignette.

Data

The five datasets come from a published study of image inspection with CLIP (Megahed et al., 2026). Each image is nominal (a good part) or defective.

The authors of Megahed et al. (2026) recorded the extrusion images and simulated the microstructure images. The pan images come from Megahed and Camelio (2012). The authors simulated the textile images with the R package spc4sts (Bui & Apley, 2021). The pipe staple images come from the Automotive Visual Inspection Dataset (AutoVI) of Renault (Carvalho et al., 2024).

git clone https://github.com/fmegahed/qe_genai.git

image_dir <- "qe_genai/data"
state_dir <- "state"
setup_dir <- "setups"
test_dir <- "test"

dir.create(state_dir)
dir.create(setup_dir)
dir.create(test_dir)

Rows

The table study_sets.csv gives each image one role. The image paths are relative to the folder data/ of fmegahed/qe_genai.

sets <- system.file("extdata", "study_sets.csv", package = "decisions") |>
  utils::read.csv() |>
  tibble::as_tibble() |>
  dplyr::mutate(
    truth = factor(
      dplyr::if_else(class == "nominal", "normal", "defective"),
      levels = c("normal", "defective")
    ),
    path = file.path(image_dir, image_path)
  )

by_dataset <- split(sets, sets$dataset)

table(sets$dataset, sets$role)
#>                 
#>                  calibration learning reference test
#>   extrusion               50       92         8  200
#>   microstructure          50       92         8  450
#>   pan_images               0       12         8   78
#>   pipe_staple             34       92         8  100
#>   textiles                50       92         8  200

The roles:

• learning: labeled images to tune on. The tuning steps choose the descriptions and the image settings on these images.
• reference: four images per class, with a rank from 1 to 4. A setup with k references per class uses the ranks 1 to k. The references come from the learning images. No step scores the references that its setup uses.
• calibration: labeled images to fit the calibrator on. The pans have too few images for this role. For the pans, the calibrator uses the learning images and the references that the chosen setup does not use.
• test: the images for the final estimate. They are scored one time only.

Each step of the tuning looks at the scores and keeps the better choice. A choice that looks better on the images that it was chosen on is, on average, less good on new images. So the estimate of a tuned setup on its own learning images is too high. The test images give an honest estimate only if no choice used them. So the test images are scored one time, at the end, after all choices are fixed. score_setup() refuses rows with the role test, and evaluate_once() scores them one time (see “Test”).

The original study drew the rows once, with fixed seeds.

Question

Each dataset gets the same choice question, with two answers. The instructions name inspection_image: that is the name of the input image in each request. The descriptions of the answers change during the tuning. The first candidate of each dataset is a generic description.

quality_question <- function(descriptions) {
  list(quality = question_choice(
    "Inspect `inspection_image` for manufacturing defects. Select exactly one category.",
    descriptions
  ))
}

generic <- c(
  normal = "No visible manufacturing defects; the part appears conforming.",
  defective = "At least one visible manufacturing defect or anomaly."
)

Models

deciders <- list(
  jev = decider_typesafe("jev-1.14.0"),
  openai = decider_openai("gpt-6-luna"),
  clef = decider_systemone(
    "http://localhost:30000", "Cloudflare/clef-flash", image_format = "images"
  )
)

decider_typesafe() reads the key from the environment variable TYPESAFE_API_KEY, and decider_openai() from OPENAI_API_KEY (for example, in ~/.Renviron). Clef-Flash runs on your computer, on a GPU, and needs no key. The shell command below starts its server with Docker (vignette("images") has the details). It is not run here.

docker run --gpus all --shm-size 8g --ipc=host -p 30000:30000 \
  lmsysorg/sglang:dev-clef \
  sglang serve --model-path Cloudflare/clef-flash --host 0.0.0.0 --port 30000

The code below runs one model at a time. model_name is "jev", "openai" or "clef".

model_name <- "jev"

Setups

A setup holds one complete decision: the decider, the question with its descriptions, the reference images and the image settings. make_setup() makes the setup of one model and dataset with k references per class. upscale = TRUE enlarges each image to max_size pixels on its longest side: TypeSafe refuses images under 280 pixels, and some images of these datasets are smaller.

score_rows() scores a setup on labeled rows with score_setup(). It gives score_setup() the role of each row, so a test row stops the call before any request. A reference row counts as a learning row. Each run keeps its answers in a state file (path): a run that stops continues and does not pay twice. score_rows() also keeps the image paths with the result, so a later step can check that its rows are in the same order as the saved answers.

make_setup <- function(model, dataset, descriptions, k, max_size) {
  references <- by_dataset[[dataset]] |>
    dplyr::filter(role == "reference", rank <= k) |>
    dplyr::arrange(rank)
  decision_setup(
    deciders[[model]], quality_question(descriptions),
    references = list(
      normal = references$path[references$truth == "normal"],
      defective = references$path[references$truth == "defective"]
    ),
    max_size = max_size, upscale = TRUE
  )
}

score_rows <- function(setup, rows, path) {
  score_setup(
    setup, image = rows$path, truth = rows$truth,
    role = dplyr::if_else(rows$role == "reference", "learning", rows$role),
    path = path
  ) |>
    dplyr::mutate(image_path = list(rows$image_path))
}

state_file <- function(...) {
  file.path(state_dir, paste0(paste(..., sep = "_"), ".json"))
}

Wording

saved <- readRDS(system.file("extdata", "study_again.rds", package = "decisions"))
candidates <- dplyr::select(
  saved$wording, model, dataset, candidate, normal, defective, reason
)

keep_rule <- function(auc, gain = 0.02) {
  best <- purrr::accumulate(auc, \(best, x) {
    if (round(x - best, 10) >= gain) x else best
  })
  c(TRUE, diff(best) > 0)
}

score_candidate <- function(model, dataset, candidate, normal, defective) {
  setup <- make_setup(
    model, dataset, c(normal = normal, defective = defective),
    k = 2, max_size = 768
  )
  design <- dplyr::filter(
    by_dataset[[dataset]],
    role == "learning" | (role == "reference" & rank > 2)
  )
  score_rows(setup, design, state_file("wording", model, dataset, candidate))
}

wording <- dplyr::filter(candidates, model == model_name)

wording <- wording |>
  dplyr::mutate(
    result = purrr::pmap(
      list(model, dataset, candidate, normal, defective), score_candidate
    ),
    auc = purrr::map_dbl(result, "estimate")
  )

wording <- dplyr::mutate(wording, kept = keep_rule(auc), .by = dataset)

final_wording <- wording |>
  dplyr::filter(kept) |>
  dplyr::slice_tail(n = 1, by = dataset) |>
  dplyr::select(model, dataset, normal, defective) |>
  dplyr::bind_rows(tibble::tibble(
    model = model_name, dataset = "pan_images",
    normal = generic[["normal"]], defective = generic[["defective"]]
  ))

Grid

The grid tries 6 setups for each model and dataset: k = 1, 2 or 4 references per class, and max_size = 512 or 768 pixels. Each setup uses the final wording. It scores the learning images that are not references: 92 per dataset, and 12 for the pans. The setup with the highest AUC is the chosen setup. A tie goes to the smaller k, then to the smaller size.

score_grid <- function(model, dataset, normal, defective, k, max_size) {
  setup <- make_setup(
    model, dataset, c(normal = normal, defective = defective), k, max_size
  )
  learning <- dplyr::filter(by_dataset[[dataset]], role == "learning")
  score_rows(setup, learning, state_file("grid", model, dataset, k, max_size))
}

grid <- final_wording |>
  dplyr::cross_join(tibble::tibble(
    k = rep(c(1, 2, 4), each = 2), max_size = rep(c(512, 768), times = 3)
  ))

grid <- grid |>
  dplyr::mutate(result = purrr::pmap(
    list(model, dataset, normal, defective, k, max_size), score_grid
  ))

grid <- dplyr::bind_cols(
  dplyr::select(grid, -result),
  purrr::list_rbind(grid$result) |>
    dplyr::select(-metric) |>
    dplyr::rename(auc = estimate)
)

chosen <- grid |>
  dplyr::arrange(dataset, dplyr::desc(round(auc, 10)), k, max_size) |>
  dplyr::slice_head(n = 1, by = dataset)

The AUC of each setup:

The chosen setups:

No setting is the best for all models and datasets. On the extrusion images, the chosen setups use k = 1 (Jev), 4 (gpt-6-luna) and 4 (Clef-Flash). On the microstructures, they use k = 4, 1 and 1. Images of 512 pixels need about 49 % of the input tokens of images of 768 pixels, and they were often as good. 22 of the 90 setups reach an AUC of 1, all on the pans and pipe staples. The tie rule then picks the setup with the fewest references. On the 12 learning images of the pans, an AUC of 1 says little.

These AUCs are on the images that the grid chose on, so they are too high (see “Rows”). The calibration and test images give fresher estimates.

Calibration

A calibrator corrects the probabilities of a model, so that they agree with the rates of defects. This vignette uses Platt scaling: a logistic regression of the truth on the logit of the probability of defective. It is fitted on the calibration images, with the chosen setup.

calibration_rows <- function(dataset, k) {
  rows <- by_dataset[[dataset]]
  if (dataset == "pan_images") {
    dplyr::filter(rows, role == "learning" | (role == "reference" & rank > k))
  } else {
    dplyr::filter(rows, role == "calibration")
  }
}

score_calibration <- function(model, dataset, normal, defective, k, max_size) {
  setup <- make_setup(
    model, dataset, c(normal = normal, defective = defective), k, max_size
  )
  rows <- calibration_rows(dataset, k)
  score_rows(setup, rows, state_file("calibration", model, dataset, k, max_size))
}

calibration <- dplyr::select(chosen, model, dataset, normal, defective, k, max_size)

calibration <- calibration |>
  dplyr::mutate(result = purrr::pmap(
    list(model, dataset, normal, defective, k, max_size), score_calibration
  ))

Platt scaling does not fit the labels 0 and 1. With N+ defective and N- normal calibration images, its target is (N+ + 1) / (N+ + 2) for a defective image and 1 / (N- + 2) for a normal image. These targets keep the fit finite when a model ranks the few calibration images almost perfectly. A fit on the labels 0 and 1 then has no finite solution: its slope grows without limit, and the calibrated probabilities become 0 or 1. The logistic calibrator of probably (probably::cal_estimate_logistic() with smooth = FALSE) fits the labels 0 and 1, so it saturates on so few, almost perfectly ranked calibration images. On the calibration images of this vignette, its glm() warns for 8 of the 15 setups, and its calibrated probabilities come within 1e-12 of 0 or 1 for 9 setups. The calibrators of probably fit well with more calibration images. The targets are not 0 or 1, so the fit uses family = quasibinomial. clip() keeps each probability between 0.005 and 0.995, the clamp of the original study, so that its logit stays between about -5.3 and 5.3.

clip <- function(prob) pmin(pmax(prob, 0.005), 0.995)

platt_fit <- function(prob, truth) {
  n_defective <- sum(truth == "defective")
  n_normal <- sum(truth == "normal")
  target <- dplyr::if_else(
    truth == "defective", (n_defective + 1) / (n_defective + 2), 1 / (n_normal + 2)
  )
  stats::glm(target ~ stats::qlogis(clip(prob)), family = stats::quasibinomial())
}

platt_apply <- function(prob, intercept, slope) {
  stats::plogis(intercept + slope * stats::qlogis(clip(prob)))
}

platt_row() fits the Platt scaling of one setup and keeps its two coefficients and the class counts.

platt_row <- function(model, dataset, k, result, ...) {
  truth <- calibration_rows(dataset, k)$truth
  fit <- platt_fit(result$answers[[1]]$.pred_defective, truth)
  tibble::tibble(
    model = model, dataset = dataset,
    n_normal = sum(truth == "normal"), n_defective = sum(truth == "defective"),
    intercept = stats::coef(fit)[[1]], slope = stats::coef(fit)[[2]]
  )
}

platt <- calibration |>
  purrr::pmap(platt_row) |>
  purrr::list_rbind()

The setups keep no calibrator. write_setup() saves each setup as one JSON file, and the Platt fits stay in the saved results. So evaluate_once() scores the raw probabilities, and platt_apply() maps them to calibrated ones. (A setup can hold a probably calibrator when there are enough calibration images for it. Set it with setup@calibrator <- calibrator; this works also for a setup from read_setup().)

calibration |>
  dplyr::select(model, dataset, normal, defective, k, max_size) |>
  purrr::pwalk(\(model, dataset, normal, defective, k, max_size) {
    setup <- make_setup(
      model, dataset, c(normal = normal, defective = defective), k, max_size
    )
    write_setup(setup, file.path(setup_dir, paste0(model, "_", dataset, ".json")))
  })

The Platt fits, with the smallest fitted value and the distance of the largest fitted value from 1, on the calibration images:

All 15 fits converged, and glm() gave no warning. No fitted value reaches 0 or 1: the fitted values lie between 3.1e-04 and 1 - 2.3e-06. No fitted value lies within 1e-6 of 0 or 1. The clamp limits the logit, so it also limits the calibrated probability of any raw probability. But a steep fit multiplies the logit by its slope (up to 6.9, for Clef-Flash on the textiles). So at the clamp limits, 2 fits (Clef-Flash on the pipe staples and Clef-Flash on the textiles) give a calibrated probability within 1e-12 of 0 or 1.

The calibrated metrics:

calibration_metrics <- function(model, dataset, k, result, intercept, slope,
                                ...) {
  answers <- dplyr::mutate(
    result$answers[[1]], truth = calibration_rows(dataset, k)$truth
  )
  calibrated <- dplyr::mutate(
    answers, .pred_defective = platt_apply(.pred_defective, intercept, slope)
  )
  metrics <- yardstick::metric_set(
    yardstick::roc_auc, yardstick::brier_class, yardstick::mn_log_loss
  )
  dplyr::bind_rows(
    before = metrics(answers, truth, .pred_defective, event_level = "second"),
    after = metrics(calibrated, truth, .pred_defective, event_level = "second"),
    .id = "probabilities"
  ) |>
    dplyr::mutate(model = model, dataset = dataset, n = nrow(answers), .before = 1)
}

calibration_summary <- calibration |>
  dplyr::left_join(platt, by = c("model", "dataset")) |>
  purrr::pmap(calibration_metrics) |>
  purrr::list_rbind()

The table gives the AUC of the chosen setup on the calibration images, and the Brier score and the log loss before and after Platt scaling. A lower Brier score and a lower log loss are better. Platt scaling is strictly increasing between the clamp limits (its slope is positive in all 15 fits), so it keeps the order of the images there. Raw probabilities below 0.005, or above 0.995, become equal. On the calibration images, the AUC after Platt scaling is the same as before for all 15 setups. The AUC of the test comes from the raw probabilities.

Platt scaling is fitted and measured on the same images, so the numbers “after” are too good. The test images give the honest numbers.

The calibration images are new to the chosen setup (not for the pans: their calibration rows include the grid images), so their AUC is a first check of the choices. On most datasets, the AUC stays close to the AUC of the grid. On the extrusion images, the AUC of Jev falls from 0.77 (grid) to 0.61. This agrees with the effect that “Rows” describes: the choices fit the learning images better than new images. Chance plays a part too, because 50 images give a rough AUC.

Before calibration, the probabilities of the models are often too extreme. For gpt-6-luna on the extrusion images, the log loss is 3.84: the model gave probabilities near 0 or 1 to images of the other class. Platt scaling moves such probabilities toward the rates of defects, and the log loss falls to 0.57.

For Jev on the pans, Jev on the pipe staples, gpt-6-luna on the pans, Clef-Flash on the pans, and Clef-Flash on the pipe staples, the chosen setup separates the two classes of the calibration images fully (AUC 1). Their Platt fits stay finite, so their Brier score and log loss “after” stay above 0. The targets of Platt pull the probabilities a little away from 0 and 1. Where the raw probabilities are already good, this can make the numbers “after” worse: the Brier score or the log loss goes up for Jev on the pans and gpt-6-luna on the pans.

Folds

How much does an AUC move from one sample of images to another? Cross-validation splits the learning images of the chosen setup into 5 folds, 10 times (rsample::vfold_cv(), with the classes in the same ratio in each fold). Each fold gives the AUC of its fifth of the images. No model is fitted, and no call is sent: the folds use the saved probabilities of the grid.

fold_auc <- function(model, dataset, answers) {
  learning <- dplyr::filter(by_dataset[[dataset]], role == "learning")
  scored <- dplyr::mutate(answers, truth = learning$truth)
  set.seed(2026)
  rsample::vfold_cv(scored, v = 5, repeats = 10, strata = truth) |>
    dplyr::mutate(
      model = model, dataset = dataset,
      auc = purrr::map_dbl(splits, \(split) {
        held_out <- rsample::assessment(split)
        yardstick::roc_auc_vec(
          held_out$truth, held_out$.pred_defective, event_level = "second"
        )
      })
    ) |>
    dplyr::select(model, dataset, id, id2, auc)
}

folds <- chosen |>
  dplyr::select(model, dataset, answers) |>
  purrr::pmap(fold_auc) |>
  purrr::list_rbind()

The mean of the fold AUCs is close to the AUC of the grid, as it should be: the folds split the same images. The spread is the useful part. On the extrusion images, the AUC of about 18 images has a standard deviation of about 0.09, and it goes from 0.54 to 1.00 over the three models. On the microstructures and the textiles, the standard deviation is 0.05 or less. The standard deviation of an AUC falls about as 1 / sqrt(n). So for the 92 learning images of the extrusion, it is about 0.09 / sqrt(5), that is about 0.04: a difference of a few hundredths between two setups can come from chance. The pans give an AUC of 1 in 100 % of the folds, and the pipe staples in 97 %, so their folds show almost no spread.

Test

The test images give the final estimate of each chosen setup. No step above used them: the descriptions, the image settings and the Platt fits were fixed before the test. Before the run, the script study_again.R checks that no learning, reference or calibration image has the path or the bytes (md5) of a test image. This check also shows that two extrusion test images are the same file, byte for byte; both stay in the test set. The numbers below come from one run on the test images, on 2026-10-24.

evaluate_once() reads the saved setup, which holds no calibrator, and scores the test images one time. So its AUC is the AUC of the raw probabilities. It writes the result to path and refuses to run again while that file exists. score_test() reads the result file when it exists, so a second run of this code sends no request and gives the same numbers.

test_file <- function(model, dataset) {
  file.path(test_dir, paste0(model, "_", dataset, ".rds"))
}

score_test <- function(model, dataset) {
  path <- test_file(model, dataset)
  if (file.exists(path)) {
    # The test rows were scored: read the result. No request goes out.
    return(dplyr::mutate(readRDS(path), calls = 0))
  }
  setup <- read_setup(file.path(setup_dir, paste0(model, "_", dataset, ".json")))
  test <- dplyr::filter(by_dataset[[dataset]], role == "test")
  evaluate_once(
    setup, image = test$path, truth = test$truth, event = "defective",
    path = path
  )
}

test <- chosen |>
  dplyr::select(model, dataset) |>
  dplyr::mutate(result = purrr::map2(model, dataset, score_test))

The Platt fit of each setup then gives the calibrated probabilities of the test images. test_metrics() computes with yardstick the AUC, the accuracy, the sensitivity (the share of defective images that the setup finds) and the specificity (the share of normal images that it passes) at a probability of 0.5, the Brier score and the log loss. It computes them before Platt scaling (raw) and after it (calibrated).

at_half <- function(prob) {
  factor(
    dplyr::if_else(prob >= 0.5, "defective", "normal"),
    levels = c("normal", "defective")
  )
}

test_metrics <- function(model, dataset, result, intercept, slope, ...) {
  truth <- dplyr::filter(by_dataset[[dataset]], role == "test")$truth
  metrics <- yardstick::metric_set(
    yardstick::roc_auc, yardstick::accuracy, yardstick::sensitivity,
    yardstick::specificity, yardstick::brier_class, yardstick::mn_log_loss
  )
  score <- function(prob) {
    scored <- tibble::tibble(truth = truth, prob = prob, predicted = at_half(prob))
    metrics(scored, truth, prob, estimate = predicted, event_level = "second")
  }
  raw <- result$answers[[1]]$.pred_defective
  dplyr::bind_rows(
    raw = score(raw),
    calibrated = score(platt_apply(raw, intercept, slope)),
    .id = "probabilities"
  ) |>
    dplyr::mutate(model = model, dataset = dataset, n = length(truth), .before = 1)
}

test_summary <- test |>
  dplyr::left_join(platt, by = c("model", "dataset")) |>
  purrr::pmap(test_metrics) |>
  purrr::list_rbind()

The table gives, for each model and dataset, the number of test images, the AUC and the accuracy (raw_accuracy) of the raw probabilities, and the accuracy, sensitivity, specificity, Brier score and log loss after Platt scaling.

The same threshold metrics, the Brier score and the log loss of the raw probabilities, before Platt scaling:

The AUC measures the ranking: it is the chance that a defective image gets a higher probability than a normal image. On the pans and the pipe staples, the AUC of each model is 0.995 or more. On the microstructures and the textiles, it lies between 0.934 and 1.000. The extrusion images are the hardest for all three models: their AUC lies between 0.733 and 0.842.

A good ranking does not give good decisions at a probability of 0.5. The raw probabilities of a model can fall mostly on one side of 0.5. For example, on the extrusion images, Clef-Flash gives no test image a raw probability of 0.5 or more (the highest is 0.495), although its AUC there is 0.842. So with the raw probabilities it calls every image normal, and its accuracy is 0.500. After Platt scaling, its accuracy is 0.745.

Platt scaling moves the probabilities toward the rates of defects of the calibration images. On most setups, this helps. The accuracy at 0.5 is higher after Platt scaling for 12 of the 15 setups, and the same for gpt-6-luna on the pipe staples. The Brier score and the log loss are both lower for 13 setups.

Platt scaling hurts Jev on the pans and gpt-6-luna on the pans. Their accuracy falls (Jev 0.987 to 0.885 and gpt-6-luna 0.808 to 0.654), their Brier score rises (Jev 0.053 to 0.075 and gpt-6-luna 0.118 to 0.163), and their log loss rises (Jev 0.236 to 0.287 and gpt-6-luna 0.356 to 0.465). The Platt fits of the pans come from only 18 images, and the grid chose the setup on most of these images (see “Calibration”), so these fits are rough.

The test sets are small. For the 78 pan images, one image on the wrong side changes the accuracy by 1.3 percentage points, and for the 100 pipe staple images by 1.0 points.

Cost and time

The tables below give the tuning steps (wording, grid and calibration); the test has its own table in “Test”. TypeSafe, OpenAI and the Clef-Flash server report no cost, so .cost is NA, and the reported cost below is NA too. All three report the input tokens. images is the number of scored images: one call each. seconds is the sum of the response times. The requests run in parallel (10 at a time), so a run takes less time than this sum.

The images by step:

The wording images of gpt-6-luna are the calls of its loops. No tuning step scored a test image.

Jev used about 2.6 times the input tokens of the other two models for the same images. The two hosted models answered in about 0.9 (Jev) and 1.1 (gpt-6-luna) seconds per image. Clef-Flash ran on one GPU (NVIDIA RTX 5000 Ada, 32 GB), and it took about 3.3 seconds per image, but it costs nothing per call. Once, during the wording step, the Clef-Flash server stopped answering. A restart of the server and a new run with the same state files continued the step: the finished answers came from the state files, and only the open images were sent again.

Image size

The setup here differs slightly from the paper of the original study. This vignette enlarges each image to 512 or 768 pixels on its longest side (upscale = TRUE; the large pan images are made smaller). The paper used 1,568 pixels, with padding.

Reruns

The answers of a model can change a little from one run to the next. A second run of the code of this vignette, on 2026-10-24, gave the same answers for gpt-6-luna at every step. For Jev (jev-1.14.0), the AUCs of the wording and grid steps differed by at most 0.03, and two close choices changed: the final wording of microstructure (candidate 3 in place of 2) and the number of references for extrusion (4 in place of 1). The test AUC of Jev on extrusion was 0.739 in place of 0.755. Clef-Flash, on a local GPU, changed less: its wording AUCs differed by at most 0.005, one close choice changed (the final wording of textiles, candidate 4 in place of 2), it chose the same k and image size for all five datasets, and its test AUCs differed by at most 0.007. So expect slightly different numbers for Jev and Clef-Flash when you run this code.

References

Bui, A. T., & Apley, D. W. (2021). spc4sts: Statistical process control for stochastic textured surfaces in R. Journal of Quality Technology, 53(3), 219-242. https://doi.org/10.1080/00224065.2019.1707730

Carvalho, P., Lafou, M., Durupt, A., Leblanc, A., & Grandvalet, Y. (2024). The Automotive Visual Inspection Dataset (AutoVI): A genuine industrial production dataset for unsupervised anomaly detection \[Data set\]. Zenodo. https://doi.org/10.5281/zenodo.10459003

Megahed, F. M., & Camelio, J. A. (2012). Real-time fault detection in manufacturing environments using face recognition techniques. Journal of Intelligent Manufacturing, 23(3), 393-408. https://doi.org/10.1007/s10845-010-0378-3

Megahed, F. M., Chen, Y.-J., Colosimo, B. M., Grasso, M., Jones-Farmer, L. A., Knoth, S., Sun, H., & Zwetsloot, I. M. (2026). Adapting OpenAI’s CLIP model for few-shot image inspection in manufacturing quality control: An expository case study with multiple application examples. Journal of Quality Technology, 58(4), 415-441. https://doi.org/10.1080/00224065.2026.2640171