The data4health package streamlines the typical epidemiological data workflow: loading raw surveillance or clinical files, cleaning and standardising them, filtering the cases of interest, aggregating case counts over space and time, and finally saving the results for downstream analysis or reporting. It is designed to help users with the most common challenges encountered during health data wrangling, such as inconsistent column names, missing values, fragmented reporting periods, and the need for spatiotemporal aggregation.
The data4health package is designed to work in tandem with other packages of the 4health suite:
Together, these packages support the integration of environmental, social, and health data into a unified analytical workflow. More information about the toolkit can be found at the HARMONIZE website.
This vignette walks you through a complete worked example using synthetic dengue surveillance data to provide an overview of the data4health package workflow and its core functions.
This vignette is written for:
No prior knowledge of epidemiology is required, but basic familiarity with R and data frames is assumed.
For the workflows shown in this package, the input data must be organized as a line list dataset. A “line list” is a standard format in public health surveillance. It is simply a spreadsheet where each row represents one individual case (for example, one patient with dengue), and each column contains information about that individual (such as age, sex, location, and date of symptom onset).
In this vignette we work with a synthetic dataset that was simulated to mirror health data that public health authorities collect when monitoring infectious diseases:
head(example_dataset)
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO DENGUE_TYPE
#> 1 Agrabah 100089 LOC-3 1959-01-17 2024-07-14 F 1
#> 2 Agrabah 100002 LOC-5 2022-01-01 2023-01-24 F 1
#> 3 Motunui 100038 LOC-7 1974-01-13 2023-12-20 F 1
#> 4 Monstropolis 100009 LOC-6 1956-01-18 2024-09-02 M 1
#> 5 Zootopia 100056 LOC-16 1962-01-16 2023-08-01 M 1
#> 6 Atlantica 100075 LOC-13 2022-01-01 2024-10-28 M 1
#> HOSPITALISED RESULT_LABORATORY NOTES SEROTYPE
#> 1 N Positive <NA> <NA>
#> 2 N Positive <NA> <NA>
#> 3 N Positive <NA> <NA>
#> 4 N Negative <NA> 3
#> 5 N Positive Contacto domiciliario 3
#> 6 S Pending <NA> <NA>The dataset includes the following variables:
NAME_GEO: Name of the geographic area (e.g., city or district) where the case was reported
CODE_GEO: Numeric geographic code corresponding to the area
LOCALCOD: Local administrative code
DATE_BRTHD: Date of birth of the individual
DATE_INI: Date of symptom onset (when the illness began)
SEXO: Sex of the individual (M = Male, F = Female)
DENGUE_TYPE: Clinical classification of dengue severity (coded numerically)
HOSPITALISED: Whether the patient was hospitalised (S = Yes, N = No)
RESULT_LABORATORY: Laboratory test result (Positive, Negative, or Pending)
NOTES: Free-text notes (may contain additional comments)
SEROTYPE: Dengue virus serotype (when available)
d4h_load() → d4h_clean() → d4h_filter() → d4h_aggregate() → d4h_save()Each step produces a plain data.frame that you can inspect, subset, or pass directly into the next function. You can choose to use the full pipeline or just choose the functions that fit your needs.
The full workflow can be executed in one go, or you can choose to execute each step separately and inspect the intermediate results before moving on to the next step. This flexibility allows you to tailor the workflow to your specific needs and data.
d4h_load()d4h_load() accepts a path to either a single file or a character vector of paths and binds the results into one dataframe.
Supported formats include .csv, .rds, Excel (both .xls / .xlsx), .json, .dbf and .dbc. This package contains a few example datasets in different formats for you to load and test the function with. You can see all examples by passing the d4h_example function with no arguments:
d4h_example()
#> [1] "example_dataset.csv" "example_dataset.dbc"
#> [3] "example_dataset.json" "example_dataset.xlsx"
#> [5] "example_dataset_2023.rds" "example_dataset_2024.rds"The example below shows how to load any file with d4h_load(). The function will automatically detect the file format and load it accordingly.
csv_path <- d4h_example("example_dataset.csv")
head(d4h_load(csv_path))
#> ✅ Successfully loaded 1 file(s).
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO DENGUE_TYPE
#> 1 Arendelle 100097 LOC-12 1994-01-08 2024-05-06 F 1
#> 2 Arendelle 100008 LOC-15 1955-01-18 2023-07-05 M 3
#> 3 Agrabah 100072 LOC-16 2021-01-01 2023-06-05 F 1
#> 4 Agrabah 100003 LOC-7 1975-01-13 2023-05-14 F 1
#> 5 Radiator Springs 100081 LOC-13 1945-01-20 2024-06-05 M 1
#> 6 Arendelle 100010 LOC-9 1970-01-14 2024-04-19 F 1
#> HOSPITALISED RESULT_LABORATORY NOTES SEROTYPE
#> 1 N Negative <NA> 1
#> 2 N Positive Contacto domiciliario NA
#> 3 N Positive Imported case 1
#> 4 S Positive <NA> NA
#> 5 N Negative <NA> NA
#> 6 N Positive <NA> NAIt is also possible to load multiple files at once, as long as the column names are consistent across files. The function will automatically bind the rows together into one data frame.
rds2023_path <- d4h_example("example_dataset_2023.rds")
rds2024_path <- d4h_example("example_dataset_2024.rds")
# to check that both files were loaded correctly,
# we will check the number of rows of each file
nrow(d4h_load(rds2023_path))
#> ✅ Successfully loaded 1 file(s).
#> [1] 259
nrow(d4h_load(rds2024_path))
#> ✅ Successfully loaded 1 file(s).
#> [1] 241
# the number of rows corresponds to the sum of the rows in both files:
nrow(d4h_load(c(rds2023_path, rds2024_path)))
#> ✅ Successfully loaded 2 file(s).
#> [1] 500In the case of loading Excel files with multiple sheets, you can specify the sheet name to load. If not specified, the function will load the first sheet by default.
xlsx_path <- d4h_example("example_dataset.xlsx")
d4h_load(xlsx_path)
#> ⚠ The Excel file contains the following sheets: year2023, year2024. Since no sheet was specified, the first one was loaded.
#> ✅ Successfully loaded 1 file(s).
#> # A tibble: 259 × 11
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO
#> <chr> <dbl> <chr> <dttm> <dttm> <chr>
#> 1 Arendelle 100008 LOC-15 1955-01-18 00:00:00 2023-07-05 00:00:00 M
#> 2 Agrabah 100072 LOC-16 2021-01-01 00:00:00 2023-06-05 00:00:00 F
#> 3 Agrabah 100003 LOC-7 1975-01-13 00:00:00 2023-05-14 00:00:00 F
#> 4 Zootopia 100017 LOC-7 1956-01-18 00:00:00 2023-04-03 00:00:00 F
#> 5 <NA> 100081 LOC-16 1997-01-07 00:00:00 2023-05-19 00:00:00 M
#> 6 Monstropolis 100068 LOC-13 2006-01-05 00:00:00 2023-07-11 00:00:00 M
#> 7 Atlantica 100085 LOC-1 1962-01-16 00:00:00 2023-07-12 00:00:00 M
#> 8 Arendelle 100037 LOC-10 1955-01-18 00:00:00 2023-02-17 00:00:00 <NA>
#> 9 Atlantica 100097 LOC-2 1967-01-15 00:00:00 2023-04-13 00:00:00 F
#> 10 Motunui 100005 LOC-11 2001-01-06 00:00:00 2023-09-21 00:00:00 M
#> # ℹ 249 more rows
#> # ℹ 5 more variables: DENGUE_TYPE <chr>, HOSPITALISED <chr>,
#> # RESULT_LABORATORY <chr>, NOTES <chr>, SEROTYPE <chr>
# as you can see all dates in DATE_INI, are from 2023You will see that the function automatically alerts you that there are multiple sheets in the Excel file and that it is loading the first sheet by default. If you want to load a specific sheet, you can specify the sheet name as follows:
d4h_load(xlsx_path, sheet = "year2024")
#> ✅ Successfully loaded 1 file(s).
#> # A tibble: 241 × 11
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO
#> <chr> <dbl> <chr> <dttm> <dttm> <chr>
#> 1 Arendelle 100097 LOC-12 1994-01-08 00:00:00 2024-05-06 00:00:00 F
#> 2 Radiator Spr… 100081 LOC-13 1945-01-20 00:00:00 2024-06-05 00:00:00 M
#> 3 Arendelle 100010 LOC-9 1970-01-14 00:00:00 2024-04-19 00:00:00 F
#> 4 Zootopia 100040 LOC-3 1973-01-13 00:00:00 2024-10-18 00:00:00 F
#> 5 Motunui 100027 LOC-1 1958-01-17 00:00:00 2024-08-20 00:00:00 F
#> 6 Motunui 100062 LOC-6 1980-01-12 00:00:00 2024-01-27 00:00:00 F
#> 7 Arendelle 100088 LOC-20 1964-01-16 00:00:00 2024-08-22 00:00:00 M
#> 8 Motunui 100039 LOC-13 1971-01-14 00:00:00 2024-04-03 00:00:00 M
#> 9 Monstropolis 100019 LOC-15 1982-01-11 00:00:00 2024-02-29 00:00:00 M
#> 10 Monstropolis 100084 LOC-11 1985-01-10 00:00:00 2024-05-18 00:00:00 M
#> # ℹ 231 more rows
#> # ℹ 5 more variables: DENGUE_TYPE <chr>, HOSPITALISED <chr>,
#> # RESULT_LABORATORY <chr>, NOTES <chr>, SEROTYPE <chr>
# now, all dates in DATE_INI are from 2024d4h_load() checks whether the file contains a header or not. However you can force the function to treat the first row as a header or as data by using the header parameter.
head(d4h_load(rds2023_path)) # the function automatically detects the header
#> ✅ Successfully loaded 1 file(s).
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO DENGUE_TYPE
#> 2 Arendelle 100008 LOC-15 1955-01-18 2023-07-05 M 3
#> 3 Agrabah 100072 LOC-16 2021-01-01 2023-06-05 F 1
#> 4 Agrabah 100003 LOC-7 1975-01-13 2023-05-14 F 1
#> 8 Zootopia 100017 LOC-7 1956-01-18 2023-04-03 F 1
#> 10 <NA> 100081 LOC-16 1997-01-07 2023-05-19 M 1
#> 11 Monstropolis 100068 LOC-13 2006-01-05 2023-07-11 M 2
#> HOSPITALISED RESULT_LABORATORY NOTES SEROTYPE
#> 2 N Positive Contacto domiciliario <NA>
#> 3 N Positive Imported case 1
#> 4 S Positive <NA> <NA>
#> 8 N Positive <NA> <NA>
#> 10 S Positive Contacto domiciliario 1
#> 11 N Positive <NA> <NA>
# forcing the first row as header
head(d4h_load(rds2023_path, header = TRUE))
#> ✅ Successfully loaded 1 file(s).
#> NAME_GEO CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO DENGUE_TYPE
#> 2 Arendelle 100008 LOC-15 1955-01-18 2023-07-05 M 3
#> 3 Agrabah 100072 LOC-16 2021-01-01 2023-06-05 F 1
#> 4 Agrabah 100003 LOC-7 1975-01-13 2023-05-14 F 1
#> 8 Zootopia 100017 LOC-7 1956-01-18 2023-04-03 F 1
#> 10 <NA> 100081 LOC-16 1997-01-07 2023-05-19 M 1
#> 11 Monstropolis 100068 LOC-13 2006-01-05 2023-07-11 M 2
#> HOSPITALISED RESULT_LABORATORY NOTES SEROTYPE
#> 2 N Positive Contacto domiciliario <NA>
#> 3 N Positive Imported case 1
#> 4 S Positive <NA> <NA>
#> 8 N Positive <NA> <NA>
#> 10 S Positive Contacto domiciliario 1
#> 11 N Positive <NA> <NA>
# forcing the first row to be included in the data
head(d4h_load(rds2023_path, header = FALSE))
#> ⚠ A header will be introduced. Make sure to give it meaningful columnnames using the rename_columns parameters of d4h_clean().
#> ✅ Successfully loaded 1 file(s).
#> Column_1 Column_2 Column_3 Column_4 Column_5 Column_6 Column_7
#> 1 <NA> CODE_GEO LOCALCOD DATE_BRTHD DATE_INI SEXO DENGUE_TYPE
#> 2 Arendelle 100008 LOC-15 -5462 19543 M 3
#> 3 Agrabah 100072 LOC-16 18628 19513 F 1
#> 4 Agrabah 100003 LOC-7 1838 19491 F 1
#> 8 Zootopia 100017 LOC-7 -5097 19450 F 1
#> 10 <NA> 100081 LOC-16 9868 19496 M 1
#> Column_8 Column_9 Column_10 Column_11
#> 1 HOSPITALISED RESULT_LABORATORY NOTES SEROTYPE
#> 2 N Positive Contacto domiciliario <NA>
#> 3 N Positive Imported case 1
#> 4 S Positive <NA> <NA>
#> 8 N Positive <NA> <NA>
#> 10 S Positive Contacto domiciliario 1For more information about d4h_load() and its parameters, see ?d4h_load.
d4h_clean()Here we show a selection of use cases. d4h_clean() handles the most common pre-processing tasks in one call, applying each step in the order listed below. This means later steps can reference outputs from earlier ones — for example, a renamed column can already be referred to by its new name within the same call.
Column selection
Depending on the number of columns in your data and the analysis you plan to do, you can use one of two parameters:
cols_to_remove: only specifying columns which you want to eliminate from your data. This is useful when you want to keep most of your columns and only want to remove a few of them.cols_to_include: only specifying columns which you want to keep in your data. This is useful when you have many columns and you only want to keep a few of them.Currently, the dataset contains the following columns:
colnames(example_dataset)
#> [1] "NAME_GEO" "CODE_GEO" "LOCALCOD"
#> [4] "DATE_BRTHD" "DATE_INI" "SEXO"
#> [7] "DENGUE_TYPE" "HOSPITALISED" "RESULT_LABORATORY"
#> [10] "NOTES" "SEROTYPE"In our example, we want to remove the CODE_GEO, LOCALCOD, and NOTES columns, which are not relevant for our analysis.
dengue_clean <- d4h_clean(data = example_dataset,
cols_to_remove = c("CODE_GEO", "LOCALCOD", "NOTES"))
#> ✅ Removing columns completed successfully: The following columns were removed from your dataset: CODE_GEO, LOCALCOD, NOTES.If you check the column names of dengue_clean, you will see that the columns have been removed:
colnames(dengue_clean)
#> [1] "NAME_GEO" "DATE_BRTHD" "DATE_INI"
#> [4] "SEXO" "DENGUE_TYPE" "HOSPITALISED"
#> [7] "RESULT_LABORATORY" "SEROTYPE"Column renaming
You can either apply a rule to all column names, the options are:
If you want to change all column names to lowercase, you can use the following code:
dengue_clean <- d4h_clean(data = dengue_clean,
rename_columns = "lower")
#> ✅ Renaming columns completed successfully.Now, the column names of dengue_clean are:
colnames(dengue_clean)
#> [1] "name_geo" "date_brthd" "date_ini"
#> [4] "sexo" "dengue_type" "hospitalised"
#> [7] "result_laboratory" "serotype"If you want to rename column names to new names, you can use the following code:
dengue_clean <- d4h_clean(data = dengue_clean,
rename_columns = c(name_geo = "region",
date_brthd = "birthday_date",
date_ini = "onset_date"))
#> ✅ Renaming columns completed successfully.Now the column names are:
colnames(dengue_clean)
#> [1] "region" "birthday_date" "onset_date"
#> [4] "sexo" "dengue_type" "hospitalised"
#> [7] "result_laboratory" "serotype"Missing value filtering
Now we have to start to think about what variables are essential for our analysis. For example, if we want to do a time series analysis of dengue cases by region, we need to have a known onset date and a known region for each case.
You can check for missing values in each column using:
colSums(is.na(dengue_clean))
#> region birthday_date onset_date sexo
#> 20 0 0 26
#> dengue_type hospitalised result_laboratory serotype
#> 22 26 31 376As you can see onset_date is complete but region has 20 missing values. If we want to drop these rows with missing values in these key variables, we can use the remove_rows_missing parameter in d4h_clean(). The following will drop any row that has a missing value in either onset_date or region.
dengue_clean <- d4h_clean(data = dengue_clean,
remove_rows_missing = c("onset_date", "region"))
#> ✅ Removing rows with missing values completed successfully.
#> Removed 20 rows due to missing entries.The function always gives an indication of how many rows were removed. We can double-check this by running:
Sparse-column removal
Some columns may have a high proportion of missing values, which may reduce their usefulness for analysis. You can find the percentage of missing values with the following code:
colMeans(is.na(dengue_clean)) * 100
#> region birthday_date onset_date sexo
#> 0.000000 0.000000 0.000000 5.208333
#> dengue_type hospitalised result_laboratory serotype
#> 4.166667 5.208333 5.833333 74.375000In our example, the serotype column has 74.375% missing values.
You can choose to drop any column that has more than a certain percentage of missing values using the threshold_remove parameter. For example, the following will drop any column that has more than 50% missing values.
dengue_clean <- d4h_clean(data = dengue_clean, threshold_remove = 50)
#> ✅ The following columns were removed from your dataset because they contain more missing data than the established threshold of 50 : serotypeAs usual, the message printed in the console directly lets you know which columns were removed.
Category-recoding
Sometimes categorical variables are coded using abbreviations or numeric codes that are not human-readable (for example, ‘M’ instead of ‘Male’). You can use the rename_categories parameter to recode these values to more meaningful labels.
For example, the following will recode the sex variable from “M” and “F” to “Male” and “Female”, the dengue_type variable from “1”, “2”, “3” to “Dengue without warning signs”, “Dengue with warning signs”, and “Severe dengue”, and the hospitalised variable from “S” and “N” to “Yes” and “No”.
dengue_clean <- d4h_clean(data = dengue_clean,
rename_categories = list(
sexo = c("M" = "Male", "F" = "Female"),
dengue_type = c("1" = "Dengue without warning signs",
"2" = "Dengue with warning signs",
"3" = "Severe dengue"),
hospitalised = c("S" = "Yes", "N" = "No")
))
#> ✅ Renaming categories completed successfully.Date/time derivation
In epidemiology, time aggregation is essential because disease trends are typically analysed by epidemiological week, month, or year rather than by individual dates. In this package, we currently consider three timescales (epiweek, month, year) and two formats to represent time: date format (e.g., 2023-01-01) and number format (e.g., 2023-01).
The same date can be represented in different formats. In the table below you can see the different representations of the same date (2026-02-19) in different formats and timescales:
| date | number | |
|---|---|---|
| epiweek | 15.02.2026 | 2026-07 |
| month | 01.02.2026 | 2 |
| year | 01.01.2026 | 2026 |
Note: The epiweek number is always given in combination with a year since the epidemiological weeks sometimes can have overlap, e.g. the first of January of 2026, corresponds to the epiweek 53 of the year 2025. If you need more information on epiweeks please check ISO 8601.
Please note that the start date of the week is by default set to Sunday as a European standard. You can change the start date by adding the parameter week_start.
There are several functions to transform date columns from one to another:
date_to_weekdate: converts a date column to the date of the first day of the epiweekdate_to_weeknumber: converts a date column to the epiweek number (e.g., 2026-07)date_to_monthdate: converts a date column to date of the first day of the monthdate_to_monthnumber: converts a date column to the month number (e.g., 2 for February)date_to_yearnumber: converts a date column to the year number (e.g., 2026)dengue_clean <- d4h_clean(data = dengue_clean,
date_to_weekdate = "onset_date",
date_to_weeknumber = "onset_date",
date_to_monthdate = "onset_date",
date_to_monthnumber = "onset_date",
date_to_yearnumber = "onset_date"
)
#> ✅ Transforming date to weekdate completed successfully: New column 'onset_date_weekdate' was created.
#> ✅ Transforming date to weeknumber completed successfully: New column 'onset_date_epiyw' was created.
#> ✅ Transforming date to monthdate completed successfully: New column 'onset_date_monthdate' was created.
#> ✅ Transforming date to monthnumber completed successfully: New column 'onset_date_monthnumber' was created.
#> ✅ Transforming date to yearnumber completed successfully: New column 'onset_date_year' was created.You can see the results for one case here:
dengue_clean[1, ]
#> region birthday_date onset_date sexo dengue_type
#> 1 Agrabah 1959-01-17 2024-07-14 Female Dengue without warning signs
#> hospitalised result_laboratory onset_date_weekdate onset_date_epiyw
#> 1 No Positive 2024-07-14 2024-29
#> onset_date_monthdate onset_date_monthnumber onset_date_year
#> 1 2024-07-01 7 2024Clean in one step
It is also possible to execute d4h_clean() only once with all the parameters at once, as in the example below:
dengue_clean <- d4h_clean(
data = example_dataset,
cols_to_remove = c("CODE_GEO", "LOCALCOD", "NOTES"),
rename_columns = c(
DATE_BRTHD = "birthdate",
DATE_INI = "onset_date",
NAME_GEO = "region",
SEXO = "sex",
DENGUE_TYPE = "dengue_type",
HOSPITALISED = "hospitalised",
RESULT_LABORATORY = "lab_result",
SEROTYPE = "serotype"
),
remove_rows_missing = c("onset_date", "region"),
threshold_remove = 50,
rename_categories = list(
sex = c(M = "Male", F = "Female"),
dengue_type = c("1" = "Dengue without warning signs",
"2" = "Dengue with warning signs",
"3" = "Severe dengue"),
hospitalised = c(S = "Yes", N = "No")),
date_to_weekdate = "onset_date",
date_to_weeknumber = "onset_date",
date_to_monthdate = "onset_date",
date_to_monthnumber = "onset_date",
date_to_yearnumber = "onset_date"
)
#> ✅ Renaming columns completed successfully.
#> ✅ Removing columns completed successfully: The following columns were removed from your dataset: CODE_GEO, LOCALCOD, NOTES.
#> ✅ Removing rows with missing values completed successfully.
#> Removed 20 rows due to missing entries.
#> ✅ Renaming categories completed successfully.
#> ✅ Transforming date to weekdate completed successfully: New column 'onset_date_weekdate' was created.
#> ✅ Transforming date to weeknumber completed successfully: New column 'onset_date_epiyw' was created.
#> ✅ Transforming date to monthdate completed successfully: New column 'onset_date_monthdate' was created.
#> ✅ Transforming date to monthnumber completed successfully: New column 'onset_date_monthnumber' was created.
#> ✅ Transforming date to yearnumber completed successfully: New column 'onset_date_year' was created.
#> ✅ The following columns were removed from your dataset because they contain more missing data than the established threshold of 50 : serotype
# the resulting dataframe
head(dengue_clean)
#> region birthdate onset_date sex dengue_type
#> 1 Agrabah 1959-01-17 2024-07-14 Female Dengue without warning signs
#> 2 Agrabah 2022-01-01 2023-01-24 Female Dengue without warning signs
#> 3 Motunui 1974-01-13 2023-12-20 Female Dengue without warning signs
#> 4 Monstropolis 1956-01-18 2024-09-02 Male Dengue without warning signs
#> 5 Zootopia 1962-01-16 2023-08-01 Male Dengue without warning signs
#> 6 Atlantica 2022-01-01 2024-10-28 Male Dengue without warning signs
#> hospitalised lab_result onset_date_weekdate onset_date_epiyw
#> 1 No Positive 2024-07-14 2024-29
#> 2 No Positive 2023-01-22 2023-04
#> 3 No Positive 2023-12-17 2023-51
#> 4 No Negative 2024-09-01 2024-36
#> 5 No Positive 2023-07-30 2023-31
#> 6 Yes Pending 2024-10-27 2024-44
#> onset_date_monthdate onset_date_monthnumber onset_date_year
#> 1 2024-07-01 7 2024
#> 2 2023-01-01 1 2023
#> 3 2023-12-01 12 2023
#> 4 2024-09-01 9 2024
#> 5 2023-08-01 8 2023
#> 6 2024-10-01 10 2024After cleaning, every row has a known onset date and sex, column names follow a consistent lower-case convention, and several derived time columns are ready for aggregation.
For more information about d4h_clean() and its parameters, see ?d4h_clean.
d4h_filter()d4h_filter() can be used to filter the dataset by any column you choose. Depending on the column type, different filter conditions are available:
| Column type | Conditions |
|---|---|
| numeric | over, under, between, equal |
| Date | after, before, during |
| character | include, exclude, match, starts, ends |
To use d4h_filter, pass the column name you want to filter by as an argument, and the value of each argument is a list which contains the filter criteria.
dengue_filtered <- d4h_filter(
data = dengue_clean,
lab_result = list(include = "Positive"),
onset_date = list(during = as.Date(c("2023-01-01", "2023-12-31"))),
region = list(exclude = "Arendelle")
)
#> ✅ Successfully filtered the dataset.
nrow(dengue_filtered)
#> [1] 84For more information about d4h_filter() and its parameters, see ?d4h_filter.
d4h_aggregate()d4h_aggregate() aggregates by counting the number of cases within defined groups (for example, the number of dengue cases per week and per region). This step transforms individual-level data into summary statistics suitable for reporting, visualisation, or modelling. For example, instead of analysing 500 individual patient records, you might analyse 52 weekly case counts. It also supports additional stratification variables via add_col, and can fill in zero-count cells for missing time-space combinations with all_times and all_spaces.
Simple spatio-temporal aggregation
weekly_by_region <- d4h_aggregate(
data = dengue_filtered,
time_col = "onset_date_weekdate",
space_col = "region"
)
#> ✅ Aggregation completed successfully.
head(weekly_by_region)
#> onset_date_weekdate region cases
#> 1 2023-01-08 Agrabah 1
#> 2 2023-01-22 Agrabah 1
#> 3 2023-03-05 Agrabah 1
#> 4 2023-03-19 Agrabah 1
#> 5 2023-03-26 Agrabah 1
#> 6 2023-04-02 Agrabah 1Monthly counts stratified by dengue severity
This is the typical input format for an epidemic curve or a time-series model.
monthly_by_type <- d4h_aggregate(
data = dengue_filtered,
time_col = "onset_date_monthdate",
space_col = "region",
add_col = "dengue_type"
)
#> ✅ Aggregation completed successfully.
head(monthly_by_type, 12)
#> onset_date_monthdate region dengue_type cases
#> 1 2023-01-01 Agrabah Dengue with warning signs 0
#> 2 2023-01-01 Agrabah Dengue without warning signs 1
#> 3 2023-01-01 Agrabah Severe dengue 1
#> 4 2023-02-01 Agrabah Dengue with warning signs 0
#> 5 2023-02-01 Agrabah Dengue without warning signs 0
#> 6 2023-02-01 Agrabah Severe dengue 0
#> 7 2023-03-01 Agrabah Dengue with warning signs 1
#> 8 2023-03-01 Agrabah Dengue without warning signs 1
#> 9 2023-03-01 Agrabah Severe dengue 1
#> 10 2023-04-01 Agrabah Dengue with warning signs 1
#> 11 2023-04-01 Agrabah Dengue without warning signs 1
#> 12 2023-04-01 Agrabah Severe dengue 0Ensuring a complete time series
In surveillance data, weeks with zero reported cases are often not explicitly recorded. This can lead to misleading analyses if missing weeks are interpreted as missing data rather than true zero counts. Pass a full date sequence via all_times so that the aggregated output contains explicit zero rows rather than silently missing weeks.
all_weeks <- seq(
from = as.Date("2023-01-01"),
to = as.Date("2023-12-31"),
by = "week"
)
weekly_complete <- d4h_aggregate(
data = dengue_filtered,
time_col = "onset_date_weekdate",
space_col = "region",
all_times = all_weeks,
all_spaces = c("Lima", "Cusco", "Loreto")
)
#> ✅ Aggregation completed successfully.
# confirm zeros are present
sum(weekly_complete$cases == 0)
#> [1] 159For more information about d4h_aggregate() and its parameters, see ?d4h_aggregate.
d4h_save()d4h_save() writes a data.frame to disk. The parameters are:
data.frame to savename, the file name will automatically include a timestamp (current date and time)."csv", "csv2", "rds", "xls", "dbf", and "json".# Save the weekly aggregation as a CSV for sharing
d4h_save(
data = weekly_complete,
name = "dengue_weekly_2023"
)
# Save the cleaned individual-level data as RDS for fast reloading in R
d4h_save(
data = dengue_filtered,
name = "dengue_analysis_clean",
format = "rds"
)
# Save for teams that use Excel
d4h_save(
data = monthly_by_type,
name = "dengue_monthly_by_type",
format = "xls"
)For more information about d4h_save() and its parameters, see ?d4h_save.
Together, these five functions form a concise and readable data-processing pipeline. Using the base R pipe (|>, available since R 4.1) the full workflow above can be written as:
d4h_load(csv_path) |>
d4h_clean(
cols_to_remove = c("CODE_GEO", "LOCALCOD", "NOTES"),
rename_columns = c(
DATE_BRTHD = "birthdate",
DATE_INI = "onset_date",
NAME_GEO = "region",
SEXO = "sex",
DENGUE_TYPE = "dengue_type",
HOSPITALISED = "hospitalised",
RESULT_LABORATORY = "lab_result",
SEROTYPE = "serotype"
),
remove_rows_missing = c("onset_date", "region"),
threshold_remove = 50,
rename_categories = list(
sex = c(M = "Male", F = "Female"),
dengue_type = c("1" = "Dengue without warning signs",
"2" = "Dengue with warning signs",
"3" = "Severe dengue"),
hospitalised = c(S = "Yes", N = "No")),
date_to_weekdate = "onset_date",
date_to_weeknumber = "onset_date",
date_to_monthdate = "onset_date",
date_to_monthnumber = "onset_date",
date_to_yearnumber = "onset_date"
) |>
d4h_filter(
lab_result = list(include = "Positive"),
onset_date = list(during = as.Date(c("2023-01-01", "2023-12-31"))),
region = list(exclude = "Arendelle")
) |>
d4h_aggregate(
time_col = "onset_date_weekdate",
space_col = "region",
all_times = all_weeks,
all_spaces = c("Lima", "Cusco", "Loreto")
) |>
d4h_save(name = "dengue_weekly_confirmed", format = "csv")The workflow is concise and readable, making it easy to follow the process from data loading to final output. Each step is clearly defined and can be easily modified or extended as needed.
Once you have executed the full workflow, you will have a cleaned, filtered, and aggregated dataset that is ready for analysis or reporting. We recommend the following next steps:
The data4health package is designed to be flexible and adaptable to different types of health data and analysis needs. We encourage you to explore the documentation and experiment with the functions to see how they can best serve your specific use case. If you have any needs that are not currently met by the package, please feel free to reach out to us with suggestions for new features or improvements. We are committed to making this toolkit as useful and user-friendly as possible for the health data community.