screenllm quickstart

The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

This vignette walks through the six-call workflow screenllm implements. Everything below runs on a laptop against a locally-served Ollama backend. If you don’t yet have Ollama installed and the four default models pulled, check_setup() will tell you what’s missing.

1. Verify setup

library(screenllm)
check_setup()

2. Load a corpus

Any tibble with title and abstract columns works; read_records() also accepts CSV, XLSX, or RIS paths and normalises common column-name variations (Scopus, Web of Science, EndNote). The package ships with a 40-record toy dataset drawn from the Community-Based Fisheries Management (CBFM) review used in the manuscript, which we use here for demonstration.

library(screenllm)
#> screenllm loaded.
#>   * First time?          install_prereqs(preset = "light")   # installs Ollama + 4 small models
#>   * Ready to go:         launch_app()                         # open the browser workflow
#>   * Diagnose Ollama:     check_setup()
toy_path <- system.file("extdata", "toy_cbfm.csv", package = "screenllm")
records <- read_records(toy_path)
head(records[, c("id", "title")])
#> # A tibble: 6 × 2
#>   id         title                                                              
#>   <chr>      <chr>                                                              
#> 1 record_001 Evaluation of managed access approach in Anambas Islands Marine Re…
#> 2 record_002 Livelihoods and fisheries governance in a contemporary Pacific Isl…
#> 3 record_003 Guidelines for coastal and marine site-planning and examples of pl…
#> 4 record_004 Antimicrobial stewardship program (ASP): An effective implementing…
#> 5 record_005 Ecosystem services approach for community-based ecotourism: toward…
#> 6 record_006 Effects of community-based management on amazon river turtles: A c…

3. Define the inclusion criteria

criteria <- define_criteria(
  scope = "Articles potentially relevant to community-based fisheries management (CBFM) in Pacific Island contexts.",
  inclusions = c(
    "It is possible that the study includes a case study from a Pacific Island country (e.g. Fiji, Solomon Islands, Vanuatu, Papua New Guinea, Samoa, Tonga, or similar).",
    "It is possible that the study discusses fisheries and/or marine resource management.",
    "It is possible that the study discusses a community-based approach."
  )
)
print(criteria)
#> <screenllm_criteria 3 inclusions>
#> Scope: 
#>   Articles potentially relevant to community-based fisheries management
#>   (CBFM) in Pacific Island contexts.
#> 1. It is possible that the study includes a case study from a Pacific Island country (e.g. Fiji, Solomon Islands, Vanuatu, Papua New Guinea, Samoa, Tonga, or similar).
#> 2. It is possible that the study discusses fisheries and/or marine resource management.
#> 3. It is possible that the study discusses a community-based approach.

4. Rank the corpus

Real screening uses default_ensemble(), which talks to Ollama. For this vignette we use backend_mock() so the code runs without Ollama.

mock_ensemble <- custom_ensemble(
  models = c("gemma3:27b", "gpt-oss:20b"),
  replicates = 2,
  backend = backend_mock()
)
ranked <- rank_records(records, criteria, ensemble = mock_ensemble, verbose = FALSE)
head(ranked[, c("id", "title", "universal_best_score", "rank")])
#> # A tibble: 6 × 4
#>   id         title                                    universal_best_score  rank
#>   <chr>      <chr>                                                   <dbl> <int>
#> 1 record_023 The reimplementation of the Ra'ui: Cora…                 73       1
#> 2 record_010 Effectiveness of village-based marine r…                 72       2
#> 3 record_012 Marine Protected Areas in Fiji: A criti…                 71.5     3
#> 4 record_013 Enhancing community empowerment through…                 71.5     4
#> 5 record_004 Antimicrobial stewardship program (ASP)…                 71       5
#> 6 record_028 Fishers' perceptions, facilitating fact…                 71       6

For a real run, swap the mock for the default:

ranked <- rank_records(records, criteria, ensemble = default_ensemble())

5. Plan the human screening set

plan <- plan_screening(ranked)
plan
#> 
#> ── <screenllm_plan> ──
#> 
#> ℹ Stop at record 40 of 40 (expected workload 100.0%).
#> ℹ Settings: min coverage 50%, run length 50, spot check n = 200
#> ! Spot-check gate is placeholder (no labelled spot-check supplied). Once the reviewer labels the first ~200 records, re-run `plan_screening()` with `spot_check_labels` set to sharpen the stop point.

The to_screen element is the tibble of records the reviewer should inspect. Everything below the stopping point is treated as excluded.

6. Screen and report

Interactive screening via the Shiny app:

launch_screening_app(plan, ranked, out_file = "screening_decisions.csv")

Offline screening (spreadsheet round-trip):

export_worksheet(plan, path = "to_screen.xlsx")
# Reviewer fills in the human_decision column and saves as
# 'to_screen_completed.xlsx'.
decisions <- read_decisions("to_screen_completed.xlsx")

Summarise the run and surface any strong LLM-human disagreements as a manual audit queue:

report <- summarise_screening(ranked, decisions, plan = plan)
print(report)

disagreements <- audit_disagreements(ranked, decisions)
disagreements

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.