Getting started with rmoriedata

The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

rmoriedata bundles the open-data fixtures used across the rmorie ecosystem and ships a small set of analyst-facing helpers for releasing aggregate statistics without re-identification risk. Everything shown here runs offline against data that installs with the package – no network required.

1. Browsing the bundled store

Every bundled table lives in a Parquet store. morie_data_catalog() lists what’s there, with row/column counts and the original source path.

cat <- morie_data_catalog()
table(cat$kind)
#> 
#> dictionary      table 
#>          8         99
head(cat[cat$kind == "table", c("slug", "n_rows", "n_cols")])
#>                                                          slug n_rows n_cols
#> 2 arsau_2020_2022_useofforce_agrregatesummarybyyear_2020_2022      5      6
#> 3        arsau_2020_2022_useofforce_detaileddataset_2020_2022      5    167
#> 5                           arsau_2023_uof_individual_records      5    112
#> 6                                 arsau_2023_uof_main_records      5     23
#> 7                          arsau_2023_uof_probe_cycle_records      5      3
#> 8                   arsau_2023_uof_weapon_records_invaliddata      5      5

Load any table by its slug:

iucr <- morie_data_load("chicago_iucr_codes")
head(iucr)
#>   iucr primary_description         secondary_description index_code active
#> 1 031A             ROBBERY               ARMED - HANDGUN          I   True
#> 2 031B             ROBBERY         ARMED - OTHER FIREARM          I   True
#> 3 033A             ROBBERY       ATTEMPT ARMED - HANDGUN          I   True
#> 4 033B             ROBBERY ATTEMPT ARMED - OTHER FIREARM          I   True
#> 5 041A             BATTERY          AGGRAVATED - HANDGUN          I   True
#> 6 041B             BATTERY    AGGRAVATED - OTHER FIREARM          I   True

Unknown slugs error with guidance rather than failing silently:

morie_data_load("no_such_dataset")
#> Error:
#> ! No dataset 'no_such_dataset'. See morie_data_catalog() for valid slugs.

Some tables ship a data dictionary (JSON). Find them via the catalogue’s kind column:

dict_slugs <- cat$slug[cat$kind == "dictionary"]
head(dict_slugs)
#> [1] "arsau_2020_2022_dictionary" "arsau_2023_dictionary"     
#> [3] "arsau_2024_dictionary"      "corrections_uof_dictionary"
#> [5] "cpads_data_provenance"      "otis_dictionary"

2. The named loaders

Chicago crime samples

Two CRAN-safe slices of the City of Chicago open data ship as lazy-loaded data objects, and load_chicago_data() wraps them (with an optional full = TRUE network fetch of the complete dataset).

comp <- load_chicago_data("complaints")
dim(comp)
#> [1] 25000    16
head(sort(table(comp$primary_type), decreasing = TRUE), 5)
#> 
#>           THEFT         BATTERY CRIMINAL DAMAGE         ASSAULT   OTHER OFFENSE 
#>            5825            4800            2362            1975            1760

arr <- load_chicago_data("arrests")
sort(table(arr$charge_type), decreasing = TRUE)
#> 
#>     M           F 
#> 12342  6464  6194

Ontario SIU director’s reports

The first open, machine-readable parse of the Ontario Special Investigations Unit director’s-report corpus – one row per report, 64 structured columns.

en <- load_siu_reports(lang = "en")
nrow(en)
#> [1] 2182
head(sort(table(en$police_service), decreasing = TRUE), 5)
#> 
#>          Toronto Police Service       Ontario Provincial Police 
#>                             467                             404 
#>            Peel Regional Police Niagara Regional Police Service 
#>                             188                              95 
#>         Hamilton Police Service 
#>                              91

CIHI data-table catalogue

A catalogue of the public CIHI data-table workbooks, each with a live URL and an Internet Archive fallback so the table stays retrievable if CIHI rotates the file. (fetch_cihi_table() does the actual download and is a network call, so it isn’t run here.)

tables <- load_cihi_data_tables()
nrow(tables)
#> [1] 232
head(tables$title, 3)
#> [1] "Injury and Trauma Emergency Department and Hospitalization Statistics, 2024–2025"
#> [2] "Wait Times for Priority Procedures in Canada, 2008 to 2025 — Data Tables"        
#> [3] "Health Workforce in Canada, 2024 — Quick Stats"

3. Data integrity

Verify you’re using the exact data slice the package shipped:

ck <- morie_data_checksums()
head(ck[order(-ck$bytes), c("file", "bytes")], 3)
#>                              file    bytes
#> 177             rmoriedata.sqlite 14835712
#> 29            describe_corpus.Rds  1712072
#> 183 siu_directors_reports.parquet  1035827

# The same compiled SHA256 kernel the whole ecosystem uses:
morie_core_sha256("abc")
#> [1] "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"

4. Differential privacy

Release counts, means, and histograms with calibrated noise. Smaller epsilon means stronger privacy and more noise.

set.seed(1)

# A private count of records matching a predicate.
morie_dp_laplace_count(true_count = 42, epsilon = 1.0)
#> [1] 41.36704

# The mechanism is unbiased -- averaging many releases recovers the truth.
mean(replicate(2000, morie_dp_laplace_count(42, epsilon = 1.0)))
#> [1] 41.98688

# A private mean of bounded data.
x <- runif(1000, 0, 1)
morie_dp_gaussian_mean(x, lower = 0, upper = 1, epsilon = 1.0)
#> [1] 0.4893552

# A private histogram straight from tabulated data.
counts <- as.integer(table(comp$year))
round(pmax(0, morie_dp_laplace_histogram(counts, epsilon = 1.0)))
#> [1] 24999     2

5. Re-identification risk

Check a release against the standard disclosure-control thresholds before publishing it.

df <- data.frame(
  age = c(25, 25, 25, 32, 32, 40),
  sex = c("F", "F", "F", "M", "M", "M")
)

# k-anonymity: every quasi-identifier combo must appear >= k times.
morie_k_anonymity_verify(df, c("age", "sex"), k = 2)$summary
#> [1] "k=2: VIOLATED (min class size=1; 1/3 classes below threshold)"

# l-diversity: each class must hold >= l distinct sensitive values.
df2 <- data.frame(
  age = c(25, 25, 25, 25, 32, 32, 32),
  sex = c("F", "F", "F", "F", "M", "M", "M"),
  dx  = c("A", "B", "C", "A", "X", "Y", "Z")
)
morie_l_diversity_verify(df2, c("age", "sex"), "dx", l = 3)$summary
#> [1] "l=3: SATISFIED (min diversity=3; 0/2 classes below threshold)"

Cell suppression hides small counts and (by default) applies complementary suppression so the hidden values can’t be recovered from the marginals:

tbl <- matrix(c(120, 3, 47, 88, 2, 99, 14, 51, 60), nrow = 3,
              dimnames = list(c("A", "B", "C"), c("X", "Y", "Z")))
res <- morie_cell_suppress(tbl, threshold = 5)
res$suppressed
#>     X  Y  Z
#> A 120 NA 14
#> B  NA NA NA
#> C  NA 99 60

Bridging to Python

load_chicago_data(..., as = "parquet_path") writes a Parquet file and returns its path – the recommended hand-off to pandas.read_parquet(). Because the store is Parquet throughout, the same tables load natively in R, Python, DuckDB, and Arrow.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.