Package {AI4OfficialStats}

Audit Statistical Fidelity of AI-Mediated Official Statistics

Description

Tools for representing official-statistics references and AI-generated claims, auditing multiple dimensions of statistical fidelity, applying critical-error rules, creating proof bundles, tracing provenance, importing ordinary official statistics files with automatic English/Persian schema detection, retrieving bounded official data from supported providers including WHO, and generating robustness and red-team tests.

Details

The package is model-provider neutral and performs no mandatory network calls. Live provider access occurs only through explicit fetch functions. It can be used with outputs from generative AI, retrieval-augmented generation, or agentic systems after those outputs are represented as stat_claim objects. It complements process-level official-statistics quality frameworks such as GSBPM 5.2 by providing an executable claim-level fidelity-control layer rather than a replacement statistical production model. The package also provides offline GSBPM mapping and quality-report helpers via gsbpm52_map(), gsbpm_align(), and gsbpm_quality_report().

Author(s)

Hossein Hassani, Steve MacFeely, and Leila Marvian Mashhad

Convert Provider or Ordinary Data to an Official Statistical Reference

Description

Converts one provider or ordinary data row into a structured official-statistics reference while preserving the source row and retrieval provenance.

Usage

as_stat_reference(x, row = 1L, indicator = NULL, geo = NULL, time = NULL,
  unit = NULL, source = NULL, dataset = NULL, population = NULL,
  adjustment = NULL, frequency = NULL, methodology = NULL, revision = NULL,
  quality_flag = NULL, uncertainty = NULL, transformation = NULL,
  mapping = "auto")

Arguments

Value

A stat_reference object.

Component-Level Statistical Fidelity Audits

Description

Compares individual semantic, numerical, temporal, geographic, unit, provenance, revision, and uncertainty dimensions of an AI-mediated claim against an official-statistics reference.

Usage

semantic_audit(reference, claim, aliases = NULL, minimum_similarity = 0.65)
numeric_audit(reference, claim, abs_tol = 1e-08, rel_tol = 1e-06)
temporal_audit(reference, claim)
geo_audit(reference, claim, aliases = NULL)
unit_audit(reference, claim)
source_audit(reference, claim, missing_score = 0.4)
revision_audit(reference, claim)
uncertainty_audit(reference, claim)

Arguments

Value

A stat_audit_component object containing score, status, critical flag, error code, expected value, observed value, and details.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
numeric_audit(ref, clm)
unit_audit(ref, clm)

Audit and Summarize Statistical Fidelity

Description

Runs the full ten-component audit and applies geometric composite scoring plus non-compensatory critical-error rules. The package also records the audit's intended alignment with relevant GSBPM 5.2 Analyse, Quality Management, and Metadata Management activities.

Usage

audit_stat_ai(reference, claim, indicator_aliases = NULL,
  geo_aliases = NULL, numeric_abs_tol = 1e-08,
  numeric_rel_tol = 1e-06, weights = NULL,
  pass_threshold = 0.90, weak_threshold = 0.75)

fidelity(components, weights = NULL, pass_threshold = 0.90,
  weak_threshold = 0.75)

audit_batch(references, claims, ids = NULL, ...)

weakest_link(x)

Arguments

Value

audit_stat_ai() returns a stat_fidelity_audit with ten component audits, the certification decision, and an embedded GSBPM 5.2 alignment record; audit_batch() returns a batch summary data frame; fidelity() returns summary scores; weakest_link() returns the lowest component score.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
aud <- audit_stat_ai(ref, clm)
aud
weakest_link(aud)

Official Statistics AI Benchmark Tools

Description

Creates adversarial statistical minimal pairs and loads the offline OfficialStatAI-Bench starter cases.

Usage

statistical_minimal_pairs(reference)
benchmark_from_reference(reference)
officialstat_benchmark()

Arguments

Value

A data frame for minimal pairs or the packaged starter benchmark; benchmark_from_reference() returns a list.

Examples

ref <- stat_reference(5.1, "unemployment rate", "Exampleland", "2025", "percent", "NSO")
statistical_minimal_pairs(ref)
head(officialstat_benchmark())

Classify, Search, and Verify Natural-Language Statistical Claims

Description

Search-first tools for turning natural-language statistical statements into structured claims, selecting appropriate official-statistics providers, retrieving candidate evidence, matching statistical dimensions, and comparing claimed values.

Usage

classify_stat_claim(text, claim_index = 1L, parser = NULL)

suggest_stat_providers(x)

search_claim_evidence(x, providers = "auto", limit = 8L, strict = FALSE,
  cache = TRUE, cache_dir = NULL, max_age = 3600, timeout = 30)

verify_claim(text, providers = "auto", evidence = NULL, provider = NULL,
  mapping = "auto", claim_index = 1L, parser = NULL,
  max_candidates = 5L, min_evidence_score = 0.55, require_named_source = TRUE,
  abs_tol = NULL, rel_tol = 1e-6, cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30, strict = FALSE)

verify_text(text, evidence = NULL, max_claims = 20L, ...)

Arguments

Details

Automatic verification does not search for a matching number alone. Candidate observations are scored using the indicator, geography, requested time (or latest-time intent), unit, and aggregate/disaggregation context. The selected official observation and source URL are retained in the result for provenance.

Provider attribution is optional. When a claim explicitly names a supported source and require_named_source = TRUE, automatic routing respects that source. When no source is named, providers = "auto" chooses suitable providers from the statistical domain. providers = "all" runs separate checks against every provider with full automatic verification support. The printed verification always reports the source-selection mode and the official provider used when evidence is retrieved.

Reference time is also optional. If a claim contains neither a year nor an explicit latest-time phrase, verification defaults to the latest available matching official observation. The claim profile records this as an inferred time choice, and printed verification output reports the automatic latest-time policy together with the official reference year actually used.

The built-in natural-language parser is deliberately deterministic and provider-agnostic. A custom parser may be supplied when richer domain NLP is needed. Automatic live retrieval is limited to providers for which the package has both searchable metadata and a bounded observation connector; other organisations remain usable through supplied evidence and the generic provider registration/import layer. In version 0.2.0, full automatic catalogue-to-observation verification is enabled for World Bank, WHO, UNSD SDG, and Eurostat.

Value

classify_stat_claim() returns a stat_claim_profile. suggest_stat_providers() returns a provider plan. search_claim_evidence() returns stat_search_results. verify_claim() returns a stat_claim_verification with status SUPPORTED, MISMATCH, NOT_FOUND, AMBIGUOUS, or INCOMPLETE. With providers = "all", verify_claim() returns a stat_multi_source_verification. verify_text() returns a stat_verification_batch.

Examples

p <- classify_stat_claim("Iran's unemployment rate was 8.1 percent in 2025.")
p
suggest_stat_providers(p)

official <- data.frame(
  value = 7.8,
  indicator = "unemployment rate",
  country = "Iran",
  year = 2025,
  unit = "percent",
  source = "Example official source"
)
verify_claim("Iran's unemployment rate was 8.1 percent in 2025.",
             evidence = official)

# Source and year are optional; the latest matching official observation is used
# when no year is stated. Requires internet access:
# verify_claim("Iran's population was 91 million.", providers = "auto")

# Provider attribution is optional; automatic routing is the default.
# Requires internet access:
# verify_claim("Life expectancy in Iran was 77 years in 2023.", providers = "auto")
# verify_claim("Life expectancy in Iran was 77 years in 2023.", providers = "all")

Audit Statistical Context and Revision Propagation

Description

Audits dataset and methodological context and measures revision propagation lag, including right-censored lag when an AI system has not yet reflected a revision.

Usage

context_audit(reference, claim, missing_score = 0.5,
  methodology_min_similarity = 0.35)
revision_propagation_lag(official_revised_at, ai_reflected_at = NULL,
  observed_at = Sys.time())

Arguments

Value

context_audit() returns a component audit. revision_propagation_lag() returns a one-row data frame.

Easy import and audit of arbitrary official-statistics data

Description

Provides a beginner-facing layer that accepts ordinary data frames, files, URLs, Persian/English schemas, and plain-text claims without requiring users to construct the package's canonical objects manually.

Usage

detect_stat_schema(x, mapping = "auto")

detect_stat_provider(x, label = FALSE)

read_official_stats(x, provider = NULL, mapping = "auto", dataset = NULL,
  indicator = NULL, geo = NULL, time = NULL, unit = NULL, source = NULL,
  sheet = NULL, encoding = "UTF-8", cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30, quiet = FALSE)

as_stat_claim(x, row = 1L, mapping = "auto", hints = list(),
  claim_index = 1L)

audit_stats(reference, claim, row = 1L, provider = NULL,
  mapping = "auto", claim_mapping = "auto", claim_index = 1L,
  hints = list(), assume_context = TRUE, ...)

Arguments

Details

Automatic schema recognition includes common English and Persian field names, Persian/Arabic digits, and common Persian unit labels. CSV, TSV, TXT, JSON, RDS, and suitable ZIP files are supported with base package dependencies. Excel input is available when the suggested readxl package is installed.

Value

detect_stat_schema() returns a stat_schema; detect_stat_provider() returns a provider identifier/label or NA; read_official_stats() returns stat_provider_data; as_stat_claim() returns stat_claim; and audit_stats() returns stat_fidelity_audit.

Examples

d <- data.frame(
  Value = 10.4,
  Indicator = "unemployment rate",
  Country = "Spain",
  Year = 2025,
  Unit = "percent",
  Source = "Example NSO"
)

detect_stat_schema(d)
x <- read_official_stats(d)
audit_stats(d, "The reported value is 10.4%.")

Search the Eurostat Catalogue

Description

Discover Eurostat datasets and tables through the official Catalogue API table-of-contents feed.

Usage

eurostat_catalogue_url(lang = "en")

search_eurostat(query, limit = 20L, lang = "en",
  types = c("dataset", "table"), cache = TRUE,
  cache_dir = NULL, max_age = 86400, timeout = 30)

Arguments

Details

The search function uses Eurostat's Catalogue API for discovery. Observation retrieval remains bounded and is performed by fetch_eurostat() using Eurostat's Statistics API.

Value

eurostat_catalogue_url() returns a character URL. search_eurostat() returns stat_search_results.

Examples

eurostat_catalogue_url()
# Live catalogue search requires internet:
# search_eurostat("unemployment rate", limit = 5)

Extract Candidate Statistical Claims from Text

Description

Provides a conservative built-in numerical extractor and a parser interface for advanced NLP or LLM-based claim extraction without tying the package to any model vendor.

Usage

extract_stat_claims(text, hints = list(), parser = NULL)

Arguments

Value

A list of stat_claim objects.

Examples

extract_stat_claims("The rate was 10.4% in 2025.",
  hints = list(indicator = "rate", geo = "Example", time = "2025"))

Align AI4OfficialStats with GSBPM 5.2

Description

Offline helpers that map AI4OfficialStats's AI-mediated claim-level quality controls to relevant phases, sub-processes and overarching activities in the Generic Statistical Business Process Model (GSBPM) version 5.2.

Usage

gsbpm52_map(relationship = c("all", "direct", "supporting"))

gsbpm_align(x)

gsbpm_quality_report(audit)

Arguments

Details

The mapping is intended to make the relationship to GSBPM 5.2 explicit and auditable. It does not claim that AI4OfficialStats replaces the GSBPM or that using the package constitutes official UNECE certification of GSBPM conformance. The principal direct relationships are with Analyse 6.2 (Validate outputs), 6.5 (Finalise outputs), Disseminate 7.2/7.5, Evaluate 8.1/8.2, and the overarching Quality and Metadata Management activities.

Value

gsbpm52_map() and gsbpm_align() return data frames. gsbpm_quality_report() returns a stat_gsbpm_report object containing component-level evidence, the audit decision, and mapped GSBPM activities.

Examples

gsbpm52_map("direct")
gsbpm_align("semantic")
gsbpm_align("audit_stat_ai")

Official-Statistics API Connectors

Description

Build URLs and retrieve bounded data from Eurostat, the World Bank Indicators API, and the OECD Data Explorer SDMX API.

Usage

eurostat_url(dataset, filters = list(), lang = "EN")
fetch_eurostat(dataset, filters = list(), lang = "EN", cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30, max_cells = 1e6)
worldbank_url(indicator, country = "all", start = NULL, end = NULL,
  source = NULL, page = 1L, per_page = 1000L, metadata = FALSE)
fetch_worldbank(indicator, country = "all", start = NULL, end = NULL,
  source = NULL, per_page = 1000L, include_metadata = TRUE, cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30)
oecd_url(agency, dataset, selection = "all", version = NULL,
  start_period = NULL, end_period = NULL, labels = TRUE)
fetch_oecd(agency, dataset, selection = "all", version = NULL,
  start_period = NULL, end_period = NULL, labels = TRUE, cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30)

Arguments

Details

Network access occurs only after an explicit call to a fetch function. Tests use local fixtures rather than provider endpoints.

Value

URL builders return a character string. Fetchers return a stat_provider_data data frame.

Examples

eurostat_url("demo_pjan", list(geo = "ES", time = "2025"))
worldbank_url("SP.POP.TOTL", country = "FRA", start = 2020, end = 2025)
oecd_url("OECD.SDD.STES", "DSD_STES@DF_CLI", start_period = "2025")
# Live retrieval requires internet:
# wb <- fetch_worldbank("SP.POP.TOTL", country = "FRA", start = 2025, end = 2025)

Write, Read, and Verify Statistical Proof Bundles

Description

Serializes Proof-Carrying Statistical Answers as canonical JSON and verifies their SHA-256 payload fingerprint.

Usage

write_proof(proof, path, pretty = TRUE)
read_proof(path)
verify_proof(proof)

Arguments

Value

Writing returns the path invisibly; reading returns a proof list; verification returns hash validity diagnostics.

Provenance Graphs and Proof-Carrying Statistical Answers

Description

Creates traceable relationships between source statistics, transformations, and AI claims, and packages audit evidence into a reproducible proof object.

Usage

claim_graph(reference, claim)
trace_claim(graph)
proof_bundle(audit)
certify_claim(audit, require_pass = FALSE)

Arguments

Value

A lightweight graph list, or a stat_proof carrying a SHA-256 fingerprint and audit status.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
aud <- audit_stat_ai(ref, clm)
g <- claim_graph(ref, clm)
trace_claim(g)
certify_claim(aud)

Register a generic official-statistics provider profile

Description

Registers session-local metadata for an official organisation that does not need a dedicated package connector. Profiles may provide reusable column mappings and host/name patterns for provider detection.

Usage

register_official_provider(name, provider_label = name, mapping = list(),
  host_patterns = character(), base_url = NA_character_, overwrite = FALSE)

Arguments

Value

The registered profile, invisibly.

Examples

register_official_provider(
  "example_nso",
  "Example National Statistical Office",
  mapping = list(value = "Value", geo = "Country", time = "Year"),
  host_patterns = "data.example.gov",
  overwrite = TRUE
)

Search multiple official-statistics catalogues

Description

Runs explicit API/catalogue searches across supported official providers and combines ranked metadata results without treating unavailable providers as ground-truth failures.

Usage

search_official_stats(query,
  providers = c("worldbank", "ons", "unsd_sdg", "who", "eurostat"), limit = 10L,
  strict = FALSE, cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

Arguments

Value

A stat_search_results data frame. When strict = FALSE, provider errors are preserved in the provider_errors attribute.

Examples

# Live catalogue search requires internet:
# search_official_stats("unemployment", limit = 5)

Additional official-statistics API connectors and search tools

Description

Adds metadata discovery and bounded retrieval for official World Bank, UNSD SDG, UK ONS, and U.S. BLS services. The IMF helper builds URLs for the current IMF SDMX 2.1 and 3.0 services while leaving dataset-specific resource paths to the caller.

Usage

search_worldbank(query, source = NULL, limit = 20L, per_page = 1000L,
  max_pages = 20L, cache = TRUE, cache_dir = NULL, max_age = 86400,
  timeout = 30)

ons_search_url(query, content_type = "dataset", limit = 20L, offset = 0L,
  filters = list())

search_ons(query, content_type = "dataset", limit = 20L, offset = 0L,
  filters = list(), cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

unsd_sdg_series_url(all_releases = FALSE)

unsd_sdg_data_url(series_code, page = 1L, page_size = 1000L,
  release_code = NULL)

search_unsd_sdg(query, limit = 20L, all_releases = FALSE, cache = TRUE,
  cache_dir = NULL, max_age = 86400, timeout = 30)

fetch_unsd_sdg(series_code, page = 1L, page_size = 1000L, max_pages = 1L,
  release_code = NULL, cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

bls_url(series_id, latest = FALSE)

fetch_bls(series_id, latest = FALSE, cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30)

imf_api_url(path, version = c("3.0", "2.1"), query = list())

Arguments

Value

URL builders return character URLs. Search functions return stat_search_results. Data retrieval functions return stat_provider_data.

Examples

ons_search_url("unemployment", content_type = "dataset")
unsd_sdg_series_url()
unsd_sdg_data_url("SI_COV_SOCINS", page_size = 100)
bls_url("LNS14000000", latest = TRUE)
imf_api_url("dataflow/IMF/all/latest", version = "3.0")

# Live calls require internet:
# search_worldbank("unemployment rate")
# search_ons("unemployment")
# search_unsd_sdg("social protection")
# fetch_bls("LNS14000000", latest = TRUE)

Extensible REST API support and session-only API keys

Description

Provides explicit GET-only access to additional official JSON APIs without scraping HTML. Custom registrations are session-local and never store secrets. API keys can be supplied directly or through environment variables named STATFIDELITY_<PROVIDER>_API_KEY.

Usage

stat_api_key(provider, key = NULL, required = FALSE)

set_stat_api_key(provider, key)

clear_stat_api_key(provider)

api_get_json(base_url, path = NULL, query = list(), provider = "custom",
  api_key = NULL, api_key_name = NULL, cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30)

register_stat_api(name, base_url, search_path = NULL, query_param = "q",
  api_key_name = NULL, provider_label = name, overwrite = FALSE)

official_stat_providers(include_custom = TRUE)

fetch_stat_api(name, path, params = list(), parser = NULL, api_key = NULL,
  cache = TRUE, cache_dir = NULL, max_age = 3600, timeout = 30)

search_stat_api(name, query, params = list(), parser = NULL, api_key = NULL,
  cache = TRUE, cache_dir = NULL, max_age = 3600, timeout = 30)

Arguments

Value

API-key helpers return a key or invisibly return the environment-variable name. api_get_json() returns a stat_api_response. Registered fetch and search functions return parser output or a best-effort data frame. 'official_stat_providers()' reports separate search, fetch, automatic-verification, and generic-import capabilities.

Examples

official_stat_providers()

register_stat_api(
  "example_nso",
  base_url = "https://api.example.gov/v1",
  search_path = "search",
  query_param = "q"
)

Construct an AI-Mediated Statistical Claim

Description

Creates a structured representation of a statistical claim produced or mediated by an AI system.

Usage

stat_claim(value = NA_real_, indicator = NA_character_, geo = NA_character_,
  time = NA_character_, unit = NA_character_, source = NA_character_,
  dataset = NULL, population = NULL, adjustment = NULL, frequency = NULL,
  methodology = NULL, revision = NULL, quality_flag = NULL,
  uncertainty = NULL, transformation = NULL, answer = NULL,
  metadata = list())

Arguments

Value

An object of class stat_claim.

Examples

clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
clm

Construct an Official-Statistics Reference Object

Description

Creates a structured canonical representation of one official statistic used as ground truth for fidelity auditing.

Usage

stat_reference(value, indicator, geo, time, unit, source,
  dataset = NULL, population = NULL, adjustment = NULL,
  frequency = NULL, methodology = NULL, revision = NULL,
  quality_flag = NULL, uncertainty = NULL, transformation = NULL,
  retrieved_at = Sys.time(), source_url = NULL, version = NULL,
  metadata = list())

Arguments

Value

An object of class stat_reference.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
ref

Prompt Robustness, Statistical Red-Teaming, and Model Stability

Description

Utilities for testing prompt sensitivity, generating official-statistics failure traps, and measuring cross-model answer stability.

Usage

perturb_prompt(prompt, n = 8L, include_original = TRUE)
redteam_stats(reference)
model_stability(results, item_col = "item", model_col = "model",
  answer_col = "answer")

Arguments

Value

Prompt variants, a red-team data frame, or a stability summary list.

Examples

perturb_prompt("What is the unemployment rate in Spain in 2025?", n = 3)
ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
head(redteam_stats(ref))

Audit a Derived Statistical Transformation

Description

Checks whether the AI used the required statistical operation and, when possible, whether the derived result is numerically correct. Supported built-in operations include difference, percentage-point change, percent/percentage change, ratio, share, sum, and mean.

Usage

transformation_audit(reference, claim, abs_tol = 1e-08, rel_tol = 1e-06)

Arguments

Value

A stat_audit_component.

Examples

ref <- stat_reference(-0.7, "rate change", "A", "2024-2025",
  "percentage points", "Provider",
  transformation = list(operation = "percentage point change",
                        inputs = c(11.1, 10.4), result = -0.7))
clm <- stat_claim(-6.31, "rate change", "A", "2024-2025",
  "percent", "Provider",
  transformation = list(operation = "percent change",
                        inputs = c(11.1, 10.4), result = -6.31))
transformation_audit(ref, clm)

WHO OData connector

Description

Builds WHO OData requests, searches WHO indicator metadata, and retrieves bounded indicator observations in the common stat_provider_data format. The base URL is configurable because WHO is transitioning its OData infrastructure from the legacy GHO implementation to the World Health Data Hub.

Usage

who_odata_url(resource = "Indicator", filter = NULL, select = NULL,
  top = NULL, skip = NULL, base_url = NULL)

search_who(query, limit = 20L, base_url = NULL, cache = TRUE,
  cache_dir = NULL, max_age = 86400, timeout = 30)

fetch_who(indicator, country = NULL, year = NULL, unit = NULL,
  indicator_name = NULL, filters = list(), page_size = 1000L,
  max_pages = 20L, base_url = NULL, cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30)

Arguments

Value

who_odata_url() returns a URL, search_who() returns stat_search_results, and fetch_who() returns stat_provider_data.

Examples

who_odata_url(
  "WHOSIS_000001",
  filter = "SpatialDim eq 'IRN'",
  top = 100
)