Purpose

Larger occurrence workflows are useful only when their outputs can be checked, understood and reused. A completed run should show which rows were attempted, which rows completed, how many records were retained after each processing step, where exported files were written, and which settings were used.

This vignette focuses on audit and scaling steps after a biofetchR workflow has run. The workflow-specific vignettes cover terrestrial, freshwater, marine, spatial-overlay and origin-evidence workflows. Here, the emphasis is on reading the outputs, identifying rows that need review, rerunning selected rows, and keeping enough information to reproduce the run later.

The main principle is to inspect before analysis. Rows with clear summaries, sensible retention, expected output files and interpretable evidence fields can be carried forward. Rows with missing files, unexpected record loss, unresolved evidence or incomplete spatial assignment should be reviewed before they are used.

Recommended output structure

For small tests, outputs can be written to a temporary directory. For project work, use a stable folder structure that separates inputs, exported outputs, provider caches, scripts and reports.

A clear folder structure might look like this:

project/
  data_raw/
    species_country_input.csv
    marine_species_input.csv

  outputs/
    terrestrial_gadm_batch/
      gbif_summary.csv
      occurrence_files/
      origin_audit/
      taxonomy/

    marine_eez_batch/
      gbif_summary.csv
      occurrence_files/
      origin_audit/
      taxonomy/

    spatial_overlay_checks/
      gbif_summary.csv
      occurrence_files/

  cache/
    gadm/
    marine_overlays/
    griis/
    native_web/
    raster/
    wdpa/
    ramsar/

  scripts/
    01_run_smoke_test.R
    02_run_batch.R
    03_check_outputs.R
    04_rerun_failed_rows.R

  reports/
    sessionInfo.txt
    provider_citations.txt

The exact folder names can be changed. The important point is to keep final occurrence outputs separate from reusable provider caches, while keeping summary and audit files close to the processed records they describe.

The main summary file

The main file to inspect after most high-level workflows is gbif_summary.csv. This table usually contains one row per attempted species-region, species-country or species-overlay combination.

The exact columns depend on the pipeline and settings used, but common fields include:

#>          column_group
#> 1   Input identifiers
#> 2 Spatial identifiers
#> 3       Record counts
#> 4       Status fields
#> 5       Output fields
#> 6     Evidence fields
#>                                                    example_columns
#> 1                               species, iso2c, region_id, overlay
#> 2            region_source, gadm_unit, overlay, marine_region_name
#> 3 n_total, n_cleaned, n_thinned, n_with_overlay, n_without_overlay
#> 4                                  status, fail_stage, fail_reason
#> 5                                          output_file, output_dir
#> 6           native_filter_mode, griis_filter_mode, evidence_action
#>                                                              why_check
#> 1                                     Confirm which row was attempted.
#> 2                            Confirm which spatial framework was used.
#> 3                Check how many records survived each processing step.
#> 4 Identify rows that succeeded, failed, or returned no usable records.
#> 5                Confirm where exported occurrence files were written.
#> 6    Confirm how native-range or invasive-status evidence was handled.

Use this file as the first checkpoint. It gives a compact view of row status, record counts, output paths and any evidence or spatial settings recorded by the workflow.

Example summary table

The example below is offline and does not use live GBIF data. It mimics the kind of summary table produced by a mixed terrestrial and marine batch.

summary_example <- data.frame(
  species = c(
    "Xenopus laevis",
    "Trachemys scripta",
    "Rattus rattus",
    "Carcinus maenas",
    "Mnemiopsis leidyi",
    "Undaria pinnatifida"
  ),
  workflow = c(
    "terrestrial_gadm",
    "terrestrial_gadm",
    "terrestrial_gadm",
    "marine_eez",
    "marine_eez",
    "marine_lme"
  ),
  region_id = c(
    "FR",
    "GB",
    "GB",
    NA,
    NA,
    NA
  ),
  overlay = c(
    NA,
    NA,
    NA,
    "eez",
    "eez",
    "lme"
  ),
  n_total = c(
    1340,
    220,
    0,
    980,
    640,
    410
  ),
  n_cleaned = c(
    1309,
    198,
    0,
    944,
    612,
    389
  ),
  n_thinned = c(
    861,
    174,
    0,
    702,
    455,
    301
  ),
  n_with_overlay = c(
    861,
    174,
    0,
    690,
    430,
    292
  ),
  n_without_overlay = c(
    0,
    0,
    0,
    12,
    25,
    9
  ),
  status = c(
    "success",
    "success",
    "no_records",
    "success",
    "success",
    "success"
  ),
  fail_stage = c(
    NA,
    NA,
    "import",
    NA,
    NA,
    NA
  ),
  fail_reason = c(
    NA,
    NA,
    "No records returned",
    NA,
    NA,
    NA
  ),
  output_file = c(
    file.path(
      example_output_dir,
      "terrestrial_gadm_batch",
      "occurrence_files",
      "Xenopus_laevis_FR.csv"
    ),
    file.path(
      example_output_dir,
      "terrestrial_gadm_batch",
      "occurrence_files",
      "Trachemys_scripta_GB.csv"
    ),
    NA,
    file.path(
      example_output_dir,
      "marine_eez_batch",
      "occurrence_files",
      "Carcinus_maenas_eez.csv"
    ),
    file.path(
      example_output_dir,
      "marine_eez_batch",
      "occurrence_files",
      "Mnemiopsis_leidyi_eez.csv"
    ),
    file.path(
      example_output_dir,
      "marine_lme_batch",
      "occurrence_files",
      "Undaria_pinnatifida_lme.csv"
    )
  ),
  stringsAsFactors = FALSE
)

summary_example
#>               species         workflow region_id overlay n_total n_cleaned
#> 1      Xenopus laevis terrestrial_gadm        FR    <NA>    1340      1309
#> 2   Trachemys scripta terrestrial_gadm        GB    <NA>     220       198
#> 3       Rattus rattus terrestrial_gadm        GB    <NA>       0         0
#> 4     Carcinus maenas       marine_eez      <NA>     eez     980       944
#> 5   Mnemiopsis leidyi       marine_eez      <NA>     eez     640       612
#> 6 Undaria pinnatifida       marine_lme      <NA>     lme     410       389
#>   n_thinned n_with_overlay n_without_overlay     status fail_stage
#> 1       861            861                 0    success       <NA>
#> 2       174            174                 0    success       <NA>
#> 3         0              0                 0 no_records     import
#> 4       702            690                12    success       <NA>
#> 5       455            430                25    success       <NA>
#> 6       301            292                 9    success       <NA>
#>           fail_reason
#> 1                <NA>
#> 2                <NA>
#> 3 No records returned
#> 4                <NA>
#> 5                <NA>
#> 6                <NA>
#>                                                                                                         output_file
#> 1    D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/terrestrial_gadm_batch/occurrence_files/Xenopus_laevis_FR.csv
#> 2 D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/terrestrial_gadm_batch/occurrence_files/Trachemys_scripta_GB.csv
#> 3                                                                                                              <NA>
#> 4        D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_eez_batch/occurrence_files/Carcinus_maenas_eez.csv
#> 5      D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_eez_batch/occurrence_files/Mnemiopsis_leidyi_eez.csv
#> 6    D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_lme_batch/occurrence_files/Undaria_pinnatifida_lme.csv

This table is illustrative. In a real workflow, read gbif_summary.csv from the pipeline output directory.

summary_tbl <- read.csv(
  file.path(example_output_dir, "terrestrial_gadm_batch", "gbif_summary.csv"),
  stringsAsFactors = FALSE
)

Status checks

Start by checking the status distribution. A batch can complete overall while still containing rows that returned no records, stopped during import, stopped during spatial attribution, or need manual review.

table(summary_example$status, useNA = "ifany")
#> 
#> no_records    success 
#>          1          5

Rows that did not complete successfully should be inspected before analysis.

failed_rows <- summary_example[
  !summary_example$status %in% "success",
  ,
  drop = FALSE
]

failed_rows
#>         species         workflow region_id overlay n_total n_cleaned n_thinned
#> 3 Rattus rattus terrestrial_gadm        GB    <NA>       0         0         0
#>   n_with_overlay n_without_overlay     status fail_stage         fail_reason
#> 3              0                 0 no_records     import No records returned
#>   output_file
#> 3        <NA>

When a row is skipped, stops early or returns no records, use the summary fields to identify the likely cause. The issue may be a missing GBIF result, a taxon-name problem, coordinate filtering, provider access, output writing, or a spatial framework that does not match the retained records.

The most useful diagnostic fields are usually status, fail_stage and fail_reason.

select_existing(
  failed_rows,
  c(
    "species",
    "workflow",
    "region_id",
    "overlay",
    "status",
    "fail_stage",
    "fail_reason"
  )
)
#>         species         workflow region_id overlay     status fail_stage
#> 3 Rattus rattus terrestrial_gadm        GB    <NA> no_records     import
#>           fail_reason
#> 3 No records returned

Record-retention checks

Record counts show how much data remains after each processing step. Comparing n_total, n_cleaned, n_thinned and overlay-assignment counts helps identify rows that need closer inspection.

summary_example$retained_after_cleaning <- safe_rate(
  summary_example$n_cleaned,
  summary_example$n_total
)

summary_example$retained_after_thinning <- safe_rate(
  summary_example$n_thinned,
  summary_example$n_cleaned
)

summary_example$overlay_assignment_rate <- safe_rate(
  summary_example$n_with_overlay,
  summary_example$n_thinned
)

select_existing(
  summary_example,
  c(
    "species",
    "workflow",
    "n_total",
    "n_cleaned",
    "n_thinned",
    "n_with_overlay",
    "n_without_overlay",
    "retained_after_cleaning",
    "retained_after_thinning",
    "overlay_assignment_rate"
  )
)
#>               species         workflow n_total n_cleaned n_thinned
#> 1      Xenopus laevis terrestrial_gadm    1340      1309       861
#> 2   Trachemys scripta terrestrial_gadm     220       198       174
#> 3       Rattus rattus terrestrial_gadm       0         0         0
#> 4     Carcinus maenas       marine_eez     980       944       702
#> 5   Mnemiopsis leidyi       marine_eez     640       612       455
#> 6 Undaria pinnatifida       marine_lme     410       389       301
#>   n_with_overlay n_without_overlay retained_after_cleaning
#> 1            861                 0               0.9768657
#> 2            174                 0               0.9000000
#> 3              0                 0                      NA
#> 4            690                12               0.9632653
#> 5            430                25               0.9562500
#> 6            292                 9               0.9487805
#>   retained_after_thinning overlay_assignment_rate
#> 1               0.6577540               1.0000000
#> 2               0.8787879               1.0000000
#> 3                      NA                      NA
#> 4               0.7436441               0.9829060
#> 5               0.7434641               0.9450549
#> 6               0.7737789               0.9700997

Low retention after cleaning can indicate many invalid coordinates, missing coordinates, exact zero-zero records, institution coordinates, centroid records or records removed by coordinate-uncertainty filters.

Low retention after thinning usually indicates strong spatial clustering. This may be appropriate for the analysis, but it should be recorded when thinning is used in the final workflow.

Low overlay assignment rates should also be reviewed. For example, unmatched marine records may fall outside the selected overlay, sit near boundaries, occur in provider coverage gaps, or retain coordinate problems that were not removed during cleaning.

Rows needing review

Not every row needing attention has a non-success status. Some rows complete but still deserve review because they have low retention, many unmatched records, or evidence fields that affect interpretation.

summary_example$needs_review <- with(
  summary_example,
  status != "success" |
    retained_after_cleaning < 0.5 |
    retained_after_thinning < 0.5 |
    overlay_assignment_rate < 0.9
)

select_existing(
  summary_example[summary_example$needs_review, , drop = FALSE],
  c(
    "species",
    "workflow",
    "status",
    "n_total",
    "n_cleaned",
    "n_thinned",
    "n_with_overlay",
    "n_without_overlay",
    "retained_after_cleaning",
    "retained_after_thinning",
    "overlay_assignment_rate",
    "fail_stage",
    "fail_reason"
  )
)
#>         species         workflow     status n_total n_cleaned n_thinned
#> 3 Rattus rattus terrestrial_gadm no_records       0         0         0
#>   n_with_overlay n_without_overlay retained_after_cleaning
#> 3              0                 0                      NA
#>   retained_after_thinning overlay_assignment_rate fail_stage
#> 3                      NA                      NA     import
#>           fail_reason
#> 3 No records returned

The thresholds above are examples only. Choose review thresholds that match the taxa, spatial scale and purpose of the analysis.

Output-file checks

Successful rows should usually point to exported occurrence files. Before combining outputs, check that listed files exist and can be opened.

The example paths below are illustrative, so they are expected not to exist when this vignette is rendered.

summary_example$output_file_listed <- !is.na(summary_example$output_file) &
  nzchar(summary_example$output_file)

summary_example$output_file_exists <- ifelse(
  summary_example$output_file_listed,
  file.exists(summary_example$output_file),
  FALSE
)

select_existing(
  summary_example,
  c(
    "species",
    "status",
    "output_file_listed",
    "output_file_exists",
    "output_file"
  )
)
#>               species     status output_file_listed output_file_exists
#> 1      Xenopus laevis    success               TRUE              FALSE
#> 2   Trachemys scripta    success               TRUE              FALSE
#> 3       Rattus rattus no_records              FALSE              FALSE
#> 4     Carcinus maenas    success               TRUE              FALSE
#> 5   Mnemiopsis leidyi    success               TRUE              FALSE
#> 6 Undaria pinnatifida    success               TRUE              FALSE
#>                                                                                                         output_file
#> 1    D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/terrestrial_gadm_batch/occurrence_files/Xenopus_laevis_FR.csv
#> 2 D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/terrestrial_gadm_batch/occurrence_files/Trachemys_scripta_GB.csv
#> 3                                                                                                              <NA>
#> 4        D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_eez_batch/occurrence_files/Carcinus_maenas_eez.csv
#> 5      D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_eez_batch/occurrence_files/Mnemiopsis_leidyi_eez.csv
#> 6    D:/R_temp\\Rtmpo7rcUO/biofetchR-vignette-outputs/marine_lme_batch/occurrence_files/Undaria_pinnatifida_lme.csv

In a real run, a successful row with a missing output file should be checked before the exported dataset is used. The file may have been moved, the path may need to be resolved relative to the output directory, or the export may not have completed as expected.

Rerunning problem rows

In larger batches, rerun only the rows that need another pass. This avoids repeating completed rows and makes the new output easier to compare with the original run.

rows_to_rerun <- summary_example[
  !summary_example$status %in% "success",
  ,
  drop = FALSE
]

terrestrial_rows_to_rerun <- rows_to_rerun[
  grepl("^terrestrial|^freshwater", rows_to_rerun$workflow),
  ,
  drop = FALSE
]

marine_rows_to_rerun <- rows_to_rerun[
  grepl("^marine", rows_to_rerun$workflow),
  ,
  drop = FALSE
]

select_existing(
  rows_to_rerun,
  c(
    "species",
    "workflow",
    "region_id",
    "overlay",
    "fail_stage",
    "fail_reason"
  )
)
#>         species         workflow region_id overlay fail_stage
#> 3 Rattus rattus terrestrial_gadm        GB    <NA>     import
#>           fail_reason
#> 3 No records returned

For terrestrial and freshwater workflows, rerun the selected rows with the same spatial settings as the original run.

failed_terrestrial_input <- data.frame(
  species = terrestrial_rows_to_rerun$species,
  iso2c = terrestrial_rows_to_rerun$region_id,
  stringsAsFactors = FALSE
)

process_gbif_terrestrial_freshwater_pipeline(
  df = failed_terrestrial_input,

  output_dir = file.path(example_output_dir, "terrestrial_gadm_rerun"),

  user = Sys.getenv("GBIF_USER"),
  pwd = Sys.getenv("GBIF_PWD"),
  email = Sys.getenv("GBIF_EMAIL"),

  region_source = "gadm",
  gadm_unit = 1,

  batch_size = 1,
  apply_cleaning = TRUE,
  apply_thinning = TRUE,
  dist_km = 5,

  export_summary = TRUE,
  store_in_memory = FALSE,
  return_all_results = FALSE
)

For marine workflows, rerun the selected species with the same overlay and evidence settings as the original run.

failed_marine_input <- data.frame(
  species = marine_rows_to_rerun$species,
  stringsAsFactors = FALSE
)

process_gbif_marine_pipeline(
  df = failed_marine_input,

  output_dir = file.path(example_output_dir, "marine_eez_rerun"),

  user = Sys.getenv("GBIF_USER"),
  pwd = Sys.getenv("GBIF_PWD"),
  email = Sys.getenv("GBIF_EMAIL"),

  overlay = "eez",
  overlay_cache_dir = file.path(example_cache_dir, "marine_overlays"),

  batch_size = 1,
  apply_cleaning = TRUE,
  apply_thinning = TRUE,
  dist_km = 5,

  export_summary = TRUE,
  store_in_memory = FALSE,
  return_all_results = FALSE
)

Keep rerun outputs separate from the original batch. This makes it clear which rows were rerun and which settings were used.

Preserving evidence and spatial audit outputs

When taxonomy preparation, origin evidence, invasive-status evidence or spatial attribution are enabled, keep their supporting outputs with the processed occurrence files. These tables explain how rows were interpreted before they were filtered, exported or carried into later analyses.

A useful origin-evidence audit might look like this:

origin_audit_example <- data.frame(
  species = c(
    "Xenopus laevis",
    "Trachemys scripta",
    "Carcinus maenas",
    "Undaria pinnatifida"
  ),
  recipient_region = c(
    "FR",
    "GB",
    "GB",
    "NZ"
  ),
  evidence_source = c(
    "native_web_and_griis",
    "native_web_and_griis",
    "griis_country_and_native_web",
    "griis_country_and_native_web"
  ),
  native_range_decision = c(
    "outside_native_range",
    "outside_native_range",
    "outside_native_range",
    "outside_native_range"
  ),
  griis_decision = c(
    "listed_invasive",
    "listed_invasive",
    "listed_alien_or_invasive",
    "listed_alien_or_invasive"
  ),
  evidence_action = c(
    "retain",
    "retain",
    "retain",
    "review"
  ),
  stringsAsFactors = FALSE
)

origin_audit_example
#>               species recipient_region              evidence_source
#> 1      Xenopus laevis               FR         native_web_and_griis
#> 2   Trachemys scripta               GB         native_web_and_griis
#> 3     Carcinus maenas               GB griis_country_and_native_web
#> 4 Undaria pinnatifida               NZ griis_country_and_native_web
#>   native_range_decision           griis_decision evidence_action
#> 1  outside_native_range          listed_invasive          retain
#> 2  outside_native_range          listed_invasive          retain
#> 3  outside_native_range listed_alien_or_invasive          retain
#> 4  outside_native_range listed_alien_or_invasive          review

Rows marked for review should remain visible in an audit table, even when they are excluded from a final analysis dataset.

Spatial attribution can be checked in the same way. The example below records the spatial framework used and the number of records with and without an assignment.

spatial_audit_example <- data.frame(
  species = c(
    "Xenopus laevis",
    "Carcinus maenas",
    "Carcinus maenas",
    "Undaria pinnatifida"
  ),
  workflow = c(
    "terrestrial_gadm",
    "marine_eez",
    "marine_lme",
    "marine_ecs"
  ),
  spatial_framework = c(
    "GADM level 1",
    "Exclusive Economic Zones",
    "Large Marine Ecosystems",
    "Extended continental shelves"
  ),
  records_with_assignment = c(
    861,
    690,
    675,
    288
  ),
  records_without_assignment = c(
    0,
    12,
    27,
    13
  ),
  stringsAsFactors = FALSE
)

spatial_audit_example
#>               species         workflow            spatial_framework
#> 1      Xenopus laevis terrestrial_gadm                 GADM level 1
#> 2     Carcinus maenas       marine_eez     Exclusive Economic Zones
#> 3     Carcinus maenas       marine_lme      Large Marine Ecosystems
#> 4 Undaria pinnatifida       marine_ecs Extended continental shelves
#>   records_with_assignment records_without_assignment
#> 1                     861                          0
#> 2                     690                         12
#> 3                     675                         27
#> 4                     288                         13

The aim is not to duplicate every occurrence record in an audit table. It is to keep enough information to explain which evidence and spatial settings shaped the exported outputs.

Scaling, memory and runtime

Scale gradually. Start with a small run, inspect the summary, open the exported file, then increase the number of rows only after the settings behave as expected.

A practical sequence is:

run one species-region or one species-overlay row;
inspect gbif_summary.csv;
open the exported occurrence file;
run a short batch with the same settings;
check retention rates and rows needing review;
add optional overlays or evidence sources only when needed;
split very large projects by taxon group, region, realm or overlay;
keep all summary and audit outputs.

The example below shows a modest second-stage terrestrial input table.

terrestrial_test_batch <- data.frame(
  species = c(
    "Xenopus laevis",
    "Trachemys scripta",
    "Sturnus vulgaris",
    "Rattus rattus",
    "Myocastor coypus"
  ),
  iso2c = c(
    "FR",
    "GB",
    "DE",
    "GB",
    "IT"
  ),
  stringsAsFactors = FALSE
)

terrestrial_test_batch
#>             species iso2c
#> 1    Xenopus laevis    FR
#> 2 Trachemys scripta    GB
#> 3  Sturnus vulgaris    DE
#> 4     Rattus rattus    GB
#> 5  Myocastor coypus    IT

For marine workflows, a second-stage test might use a short species list and one overlay.

marine_test_batch <- data.frame(
  species = c(
    "Carcinus maenas",
    "Mnemiopsis leidyi",
    "Undaria pinnatifida",
    "Didemnum vexillum"
  ),
  stringsAsFactors = FALSE
)

marine_test_batch
#>               species
#> 1     Carcinus maenas
#> 2   Mnemiopsis leidyi
#> 3 Undaria pinnatifida
#> 4   Didemnum vexillum

For larger workflows, store_in_memory = FALSE is usually the safest setting. This writes processed occurrence files to disk rather than retaining all records in the R session. Keep batch_size conservative while testing, then adjust it only after the workflow is stable.

The example below shows a project-style batch call. It is not evaluated because paths and input tables are project-specific.

project_output <- file.path(example_output_dir, "terrestrial_gadm_batch")
project_cache <- example_cache_dir

batch_result <- process_gbif_terrestrial_freshwater_pipeline(
  df = terrestrial_test_batch,

  output_dir = project_output,

  user = Sys.getenv("GBIF_USER"),
  pwd = Sys.getenv("GBIF_PWD"),
  email = Sys.getenv("GBIF_EMAIL"),

  region_source = "gadm",
  gadm_unit = 1,
  cache_dir_gadm = file.path(project_cache, "gadm"),

  batch_size = 1,

  apply_cleaning = TRUE,
  apply_thinning = TRUE,
  dist_km = 5,

  export_summary = TRUE,
  store_in_memory = FALSE,
  return_all_results = FALSE,
  quiet = FALSE
)

Provider metadata and session information

biofetchR workflows may use GBIF occurrence data, administrative boundaries, marine overlays, ecoregions, protected-area layers, raster layers, native-range evidence, invasive-status evidence, WoRMS evidence, SInAS evidence and other provider resources.

For project outputs, keep a simple metadata table recording which sources were used and what should be saved with the results.

provider_metadata_template <- data.frame(
  provider = c(
    "GBIF",
    "GADM",
    "Marine Regions",
    "GRIIS",
    "SInAS",
    "WoRMS"
  ),
  used_for = c(
    "Occurrence records",
    "Administrative attribution",
    "Marine overlay attribution",
    "Species-country alien or invasive status evidence",
    "Native and alien distribution evidence",
    "Marine taxonomy or distribution evidence"
  ),
  record_with_outputs = c(
    "Download DOI or download key",
    "Version or access date",
    "Product name, version or access date, and overlay name",
    "Version or access date",
    "Dataset version and access date",
    "Dataset DOI, version, or access date"
  ),
  stringsAsFactors = FALSE
)

provider_metadata_template
#>         provider                                          used_for
#> 1           GBIF                                Occurrence records
#> 2           GADM                        Administrative attribution
#> 3 Marine Regions                        Marine overlay attribution
#> 4          GRIIS Species-country alien or invasive status evidence
#> 5          SInAS            Native and alien distribution evidence
#> 6          WoRMS          Marine taxonomy or distribution evidence
#>                                      record_with_outputs
#> 1                           Download DOI or download key
#> 2                                 Version or access date
#> 3 Product name, version or access date, and overlay name
#> 4                                 Version or access date
#> 5                        Dataset version and access date
#> 6                   Dataset DOI, version, or access date

Provider metadata should be stored alongside gbif_summary.csv, exported occurrence files, and any origin or spatial audit tables.

For reproducibility, record the R session information alongside final outputs. This helps identify the R version, package versions and optional dependencies used for the run.

sessionInfo()
#> R version 4.4.3 (2025-02-28 ucrt)
#> Platform: x86_64-w64-mingw32/x64
#> Running under: Windows 11 x64 (build 26200)
#> 
#> Matrix products: default
#> 
#> 
#> locale:
#> [1] LC_COLLATE=C                           
#> [2] LC_CTYPE=English_United Kingdom.utf8   
#> [3] LC_MONETARY=English_United Kingdom.utf8
#> [4] LC_NUMERIC=C                           
#> [5] LC_TIME=English_United Kingdom.utf8    
#> 
#> time zone: Europe/London
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] biofetchR_0.1.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] sass_0.4.10             generics_0.1.4          class_7.3-23           
#>  [4] xml2_1.5.2              KernSmooth_2.23-26      stringi_1.8.7          
#>  [7] lattice_0.22-6          rgbif_3.8.2             digest_0.6.37          
#> [10] magrittr_2.0.3          evaluate_1.0.4          grid_4.4.3             
#> [13] RColorBrewer_1.1-3      rnaturalearth_1.0.1     fastmap_1.2.0          
#> [16] plyr_1.8.9              jsonlite_2.0.0          whisker_0.4.1          
#> [19] e1071_1.7-16            CoordinateCleaner_3.0.1 DBI_1.2.3              
#> [22] httr_1.4.7              scales_1.4.0            oai_0.4.0              
#> [25] codetools_0.2-20        lazyeval_0.2.2          jquerylib_0.1.4        
#> [28] cli_3.6.4               rlang_1.2.0             units_0.8-7            
#> [31] cachem_1.1.0            yaml_2.3.10             tools_4.4.3            
#> [34] geosphere_1.5-20        dplyr_1.1.4             ggplot2_4.0.2          
#> [37] vctrs_0.6.5             R6_2.6.1                proxy_0.4-27           
#> [40] classInt_0.4-11         lifecycle_1.0.5         stringr_1.6.0          
#> [43] pkgconfig_2.0.3         terra_1.8-42            pillar_1.11.0          
#> [46] bslib_0.9.0             gtable_0.3.6            glue_1.8.0             
#> [49] data.table_1.17.0       Rcpp_1.0.14             sf_1.0-20              
#> [52] xfun_0.52               tibble_3.2.1            tidyselect_1.2.1       
#> [55] rstudioapi_0.17.1       knitr_1.50              dichromat_2.0-0.1      
#> [58] farver_2.1.2            htmltools_0.5.8.1       rmarkdown_2.29         
#> [61] compiler_4.4.3          S7_0.2.1                sp_2.2-0

In a project workflow, save this to a text file at the end of the run.

dir.create(example_output_dir, recursive = TRUE, showWarnings = FALSE)

writeLines(
  capture.output(sessionInfo()),
  file.path(example_output_dir, "sessionInfo.txt")
)

Final checklist

Before carrying outputs forward, confirm that the run can be traced from input to exported records.

A complete run folder should include:

the input table used for the run;
gbif_summary.csv;
exported occurrence files for successful rows;
any taxonomy, origin-evidence or spatial-attribution outputs produced by the selected settings;
provider-cache notes or paths;
provider citations or access details where needed;
saved R session information.

Rows should be held back for review if they stopped early, returned no records, lack an expected output file, have unexpectedly low retention, contain many unmatched spatial assignments, or include unresolved evidence that affects the analysis.

Once these checks are complete, the retained occurrence files can be combined, summarised or passed into the next stage of the project with a clear record of how they were produced.

Auditing and scaling biofetchR workflows