The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

autocodebook

Automatic codebook and eligibility tracking for data preprocessing pipelines in R.

Write the mutate() — the codebook writes itself.

Built for large-scale epidemiological and social data pipelines using sparklyr, but works equally well with local data frames.

Installation

# From CRAN (after release)
install.packages("autocodebook")

# Development version
# install.packages("devtools")
devtools::install_github("patriciafortesm/autocodebook")

Why autocodebook?

In data preprocessing pipelines, documenting variables is duplicated work. You already wrote the case_when() with all the logic — but then you have to manually write the type, the source columns, the category labels, and the code again in a separate codebook table.

Before (manual codebook — you write everything twice):

# Step 1: Create the variable
df <- df %>%
  mutate(
    sex = case_when(
      cod_sex %in% c(0L, 99L) ~ NA_character_,
      cod_sex == 1L            ~ "Male",
      cod_sex == 2L            ~ "Female",
      TRUE                     ~ NA_character_
    )
  )

# Step 2: Manually document it (duplicated effort!)
register_var("sex",
  type       = "character",
  source     = "cod_sex",
  label      = "Sex",
  categories = "Male; Female; NA (codes 0 and 99)",
  code       = "case_when(cod_sex %in% c(0L, 99L) ~ NA_character_, ...)"
)

After (with autocodebook — you only write the label):

df <- auto_mutate(df,
  labels = list(sex = "Sex"),
  sex = case_when(
    cod_sex %in% c(0L, 99L) ~ NA_character_,
    cod_sex == 1L            ~ "Male",
    cod_sex == 2L            ~ "Female",
    TRUE                     ~ NA_character_
  )
)
# Done. Type, source, categories, and code are captured automatically.

The package uses introspection (rlang) to capture the source code of each expression and infer:

What you write vs. what is automatic

Quick example

library(dplyr)
library(autocodebook)

cb_init(id_col = "person_id")

df <- df %>%
  # Track raw data
  auto_filter(step = "1. Raw data", description = "All records", TRUE) %>%
  # Eligibility
  auto_filter(step = "2. Valid sex",
              description = "Exclude records with missing sex",
              !is.na(cod_sex)) %>%
  auto_filter(step = "3. Adults",
              description = "Restrict to age >= 18",
              age >= 18) %>%
  # Create derived variables (auto-documented)
  auto_mutate(
    labels = list(
      sex      = "Sex",
      race     = "Self-declared race / ethnicity",
      crowding = "Household crowding (people per room)"
    ),
    block = "Demographics",
    sex = case_when(
      cod_sex == 1L ~ "Male",
      cod_sex == 2L ~ "Female",
      TRUE          ~ NA_character_
    ),
    race = case_when(
      cod_race == 1L ~ "White",
      cod_race == 2L ~ "Black",
      cod_race == 3L ~ "Brown",
      cod_race == 5L ~ "Indigenous",
      TRUE           ~ NA_character_
    ),
    crowding = n_people / n_rooms
  )

# View and export
cb_render()                                              # Codebook as gt table
cb_export(file.path(tempdir(), "codebook.html"))         # Export to HTML
cb_export(file.path(tempdir(), "codebook.docx"))         # Editable Word table
cb_export(file.path(tempdir(), "codebook.xlsx"))         # Editable Excel spreadsheet
track_render()                                           # Eligibility flow as gt table

# Programmatic access
cb_get()      # Codebook as a tibble
track_get()   # Tracking log as a tibble

Standardized HTML report

A single call to generate_report() produces a complete dashboard with eligibility flowchart, codebook, and per-variable inspection — ready to share with collaborators or attach as a supplement.

generate_report(
  data        = df,
  type        = "longitudinal",         # or "cross_sectional"
  id_var      = "person_id",
  time_var    = "year",
  output_html = file.path(tempdir(), "report.html")
)

Eligibility section — automatic flowchart with N per step and number of records removed:

Codebook section — all derived variables with type, source, categories, and the exact code that produced them:

Variable inspection — distribution by period, missingness pattern, and within-subject variation (Fixed vs. Varies), per variable:

Editable exports for papers and supplements

The codebook can be exported as a fully editable Word table (for paper supplements) or Excel spreadsheet (with filters, for review before publication):

Word (.docx) — paste straight into supplementary material:

Excel (.xlsx) — filter, sort, edit, then re-import if needed:

CONSORT-style eligibility flowchart

For studies that split the cohort by exposure (and optionally by mediator), track_split() + track_outcomes() capture N and outcome counts at every subgroup combination. flow_diagram() then renders a publication-ready CONSORT-style flowchart directly from the eligibility steps (recorded by auto_filter()) and the flow tree — no manual positioning needed:

df %>%
  auto_filter(step = "age",   description = "Younger than 10 years", age >= 10) %>%
  auto_filter(step = "sinan", description = "No record of violence", has_violence) %>%
  track_split(by = "sgm", label = "SGM status",
              value_labels = c("0" = "Non-SGM", "1" = "SGM")) %>%
  track_outcomes(c("self_harm", "psych"),
                 labels = list(self_harm = "Self-harm",
                               psych     = "Psychiatric hospitalization"))

flow_diagram()       # publication-ready ggplot
flow_table()         # the same data as a tidy tibble (one row per leaf × outcome)

flow_diagram() — vertical trunk (baseline → aggregated exclusions → eligible cohort), one column per subgroup, and outcome boxes stacked beneath each subgroup:

flow_table() — the same information as a tidy tibble, ready for analysis or editable export (CSV, XLSX):

flow_diagram() returns a ggplot object, so it can be themed, embedded in the standardized report (which does so automatically), or saved with flow_diagram_export(). The export format follows the file extension:

flow_diagram_export("flow.png")    # raster image
flow_diagram_export("flow.pdf")    # vector (also .svg, .eps)
flow_diagram_export("flow.emf")    # editable vector for Word (needs 'devEMF')
flow_diagram_export("flow.docx")   # Word document with the flowchart embedded (needs 'officer')
flow_diagram_export("flow.pptx")   # PowerPoint, fully editable shapes (needs 'rvg' + 'officer')

For the .pptx output, right-click the figure in PowerPoint and choose Ungroup to edit each box and label as a native shape. The tidy table can still be piped into a dedicated diagramming package such as consort or DiagrammeR if you prefer.

Spark example

Works the same way with sparklyr — no API changes:

library(sparklyr)
library(dplyr)
library(autocodebook)

sc <- spark_connect(master = "local")
df <- copy_to(sc, my_data, "my_table")

cb_init(id_col = "person_id")
track_step(df, "1. Raw data")

df <- auto_mutate(df,
  labels = list(
    region_code = "Municipality code (7 digits)",
    state_code  = "State code (first 2 digits)"
  ),
  block = "Geographic variables",
  region_code = lpad(as.character(cod_munic), 7L, "0"),
  state_code  = substring(region_code, 1L, 2L)
)

cb_render()
spark_disconnect(sc)

Big-data optimizations

For large Spark pipelines, several helpers reduce wasted recomputation:

• cb_set_default_cache(TRUE) — caches intermediate results across the whole session.
• auto_filter(..., assume_unique = TRUE) — skips the n_distinct(id) call in tracking when the dataset is already unique by ID (orders of magnitude faster on multi-million-row data).
• cb_checkpoint(sdf, mode = "memory") — materializes a lazy tbl_spark to break long chains of transformations.
• generate_report(..., cache_data = TRUE) — persists the dataset once before computing all report aggregations.

API reference

Verb wrappers

Codebook

Eligibility tracking

Flow tree (CONSORT-style)

Reports and session options

Parameters for auto_mutate / auto_summarise

auto_mutate(.data,
  labels = list(var1 = "Label for variable 1"),  # only required field
  block  = "Section name",                        # optional: groups in codebook
  var1   = case_when(...)                          # your normal dplyr expressions
)

• labels: Named list mapping variable names to descriptions. If omitted, the variable name itself is used.
• block: Optional string. Groups variables into sections in the rendered codebook (e.g., "Demographics", "Migration flags").

Compatibility

• R >= 4.1
• Works with both sparklyr (tbl_spark) and local data frames
• Compatible with Spark SQL functions (lpad, substring, lag with window_order, etc.)
• No stringr dependency — uses only base R internally
• Report exports require rmarkdown, ggplot2, patchwork, scales (Suggests)
• Editable exports to .docx / .xlsx require officer, flextable, openxlsx (Suggests)

License

MIT

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.