Package {exeval}

Type:	Package
Language:	en-US
Title:	External Evaluation of Population Pharmacokinetic-Pharmacodynamic (popPKPD) Models
Version:	0.0.1
Description:	Provides tools to automate external pharmacokinetic model evaluation workflows, including Bayesian forecasting, predictive performance metrics, diagnostic plotting, and automated reporting.
License:	MIT + file LICENSE
Encoding:	UTF-8
LazyData:	true
Depends:	R (≥ 4.1)
Imports:	mapbayr, dplyr, ggplot2, ggpubr, mrgsolve (≥ 1.0.8), scales, stats, methods, rlang
Suggests:	knitr, rmarkdown,
URL:	https://github.com/Martin-Umpierrez/exeval
BugReports:	https://github.com/Martin-Umpierrez/exeval/issues
RoxygenNote:	7.3.3
NeedsCompilation:	no
Packaged:	2026-06-05 15:03:33 UTC; ialvarez
Author:	Manuel Ibarra [aut], Martin Umpierrez [aut, cre], Ignacio Alvarez Castro [aut], Nicolas Schmidt [aut]
Maintainer:	Martin Umpierrez <mumpierrez@fq.edu.uy>
Repository:	CRAN
Date/Publication:	2026-06-10 08:10:14 UTC

Plot combined model performance metrics

Description

Generates comparative visualizations of evaluation metrics across multiple externally evaluated models combined with combine_metrics().

Usage

combine_metric_plot(
  cmetrics,
  type = c("bias_barplot", "MAIPE_barplot", "IF20_plot", "IF30_plot")
)

Arguments

Details

This function is intended for model comparison workflows, allowing visual inspection of predictive performance across evaluation occasions..

Metrics are displayed separately for each evaluation occasion (OCC) using faceted plots, enabling direct visual comparison between candidate models.

Value

A ggplot2 object.

See Also

combine_metrics(), plot.EvalPPK()

Examples

# Fake  metrics
generate_fake_metrics <- function(n_occasions = 3) {
data.frame(
 OCC = rep(1:n_occasions),  # Simula varias ocasiones
 rBIAS = stats::rnorm(n_occasions, mean = 0, sd = 10),
 rBIAS_lower = stats::rnorm(n_occasions, mean = -5, sd = 5),
 rBIAS_upper = stats::rnorm(n_occasions, mean = 5, sd = 5),
 MAIPE = stats::runif(n_occasions, min = 10, max = 50),
 IF20 = stats::runif(n_occasions, min = 20, max = 80),
 IF30 = stats::runif(n_occasions, min = 30, max = 90)
)
}

simulation1 <- list(metrics_means = generate_fake_metrics())
simulation2 <- list(metrics_means = generate_fake_metrics())
# List of models
models_list <- list(
list(model_name = "Test_Model1", metrics_list = simulation1),
list(model_name = "Test_Model2", metrics_list = simulation2)
)
combined_results <- combine_metrics(models_list)

# plot results
combine_metric_plot(combined_results, type = 'bias_barplot')

Combine evaluation metrics across multiple models

Description

Combines model performance metrics from multiple externally evaluated models into a single data frame and identifies the top-performing models according to a selected metric and ranking criterion.

Usage

combine_metrics(
  models,
  metric = "MAIPE",
  top_n = 2,
  occ_eval = 2,
  rank_criteria = "min"
)

Arguments

Details

All models must have been evaluated using the same external evaluation strategy (eval_type), otherwise the function stops with an error.

Top-performing models are identified internally according to the selected metric and ranking criterion.

Value

A named list containing:

cmetrics

Combined data frame containing metrics for all input models, with an additional Model column.

topmodelspd

Data frame containing the top-performing models according to the selected ranking criteria.

See Also

metrics_occ()

Examples

#' # Fake  metrics
generate_fake_metrics <- function(n_occasions = 3) {
data.frame(
 OCC = rep(1:n_occasions),  # Simula varias ocasiones
 rBIAS = stats::rnorm(n_occasions, mean = 0, sd = 10),
 rBIAS_lower = stats::rnorm(n_occasions, mean = -5, sd = 5),
 rBIAS_upper = stats::rnorm(n_occasions, mean = 5, sd = 5),
 MAIPE = stats::runif(n_occasions, min = 10, max = 50),
 IF20 = stats::runif(n_occasions, min = 20, max = 80),
 IF30 = stats::runif(n_occasions, min = 30, max = 90)
)
}
simulation1 <- list(metrics_means = generate_fake_metrics())
simulation2 <- list(metrics_means = generate_fake_metrics())
# List of models
models_list <- list(
list(model_name = "Test_Model1", metrics_list = simulation1),
list(model_name = "Test_Model2", metrics_list = simulation2)
)

combined_results <- combine_metrics(models_list)
print(combined_results)

Built-in population PK/PKPD models

Description

Curated population pharmacokinetic (PK) and pharmacokinetic-pharmacodynamic (PKPD) models included in the package for external evaluation workflows.

Usage

exeval_models

Format

A data frame with 6 variables:

Label

Unique model identifier used to reference the model within the package.

Drug

Drug associated with the model.

Author

First author of the original publication.

Year

Publication year.

Ref

Reference title or citation for the original model publication.

Model_code

Model code stored as a character string in mrgsolve format.

Details

These models can be used directly in exeval_ppk() by supplying the corresponding Label as the model argument.

External evaluation workflow for population PK, PKPD models

Description

Runs the complete external evaluation workflow for population pharmacokinetic (popPK) or pharmacokinetic-pharmacodynamic (PKPD) models, including MAP estimation, posterior model updating, simulation, and prediction error metric calculation.

Prints a formatted representation of an EvalPPK object, including dataset characteristics, evaluation settings, and performance metrics.

Prints a formatted representation of a summary generated from an EvalPPK object, including metadata, applied summary settings, global performance metrics, fit distribution, and poorly fitted individuals.

Generates a structured summary of an EvalPPK object, including global performance metrics, fit quality classification, and identification of poorly fitted individuals.

Generates visualization plots for external evaluation results stored in an EvalPPK object, including prediction error metrics, fit quality distributions, and forecasting performance summaries.

Usage

exeval_ppk(
  model,
  data,
  model_name = NULL,
  drug_name = NULL,
  tool = "mapbayr",
  check_compile = TRUE,
  num_occ = NULL,
  num_ids = NULL,
  sampling = TRUE,
  occ_ref = NULL,
  evaluation_type = c("sequential_updating", "stepwise_updating",
    "sequential_reference_updating", "backward_reference_updating"),
  method = c("L-BFGS-B", "newuoa"),
  assessment = c("a_priori", "Bayesian_forecasting", "Complete"),
  verbose = FALSE
)

## S3 method for class 'EvalPPK'
print(x, ...)

## S3 method for class 'summary.EvalPPK'
print(x, ...)

## S3 method for class 'EvalPPK'
summary(
  object,
  occ = NULL,
  by_occ = TRUE,
  poor_threshold = 50,
  top_n = 10,
  ...
)

## S3 method for class 'EvalPPK'
plot(
  x,
  type = c("bias_barplot", "bias_pointrange", "MAIPE_barplot", "bias_boxplot",
    "bias_violin", "bias_dotplot", "bias_density", "IF20_plot", "IF30_plot", "IF_plot",
    "error_plot", "fit_class", "fit_histogram"),
  occ = NULL,
  signed = FALSE,
  ...
)

Arguments

Details

This function serves as the main high-level interface for the exeval workflow.

This function executes the complete external evaluation workflow:

1. MAP estimation via run_MAP_estimations()

2. posterior model updating via update_map_models()

3. PK simulations via run_pk_simulations()

4. prediction error metric calculation via metrics_occ()

The returned object is an EvalPPK object containing all intermediate results and summary metadata.

Summary outputs can be generated across all occasions, for a specific occasion, or stratified by occasion.

This method provides visualization tools for assessing predictive performance of external model evaluations, including bias, precision, forecasting success, and fit quality classification.

Value

An object of class EvalPPK containing:

metrics

Prediction error metrics returned by metrics_occ().

estimates

MAP estimation results returned by run_MAP_estimations().

updates

Posterior individualized models returned by update_map_models().

simulations

Simulation outputs returned by run_pk_simulations().

Additional workflow metadata are stored as object attributes.

An object of class summary.EvalPPK containing:

metadata

Evaluation metadata inherited from the original EvalPPK object.

global_metrics

Summary performance metrics across all observations or stratified by occasion.

fit_distribution

Distribution of fit quality categories based on absolute prediction error.

poor_fit_ids

Table of individuals exceeding the selected poor-fit threshold.

A ggplot2 object, except for "IF_plot", which returns a combined plot object generated with ggpubr.

See Also

run_MAP_estimations(), update_map_models(), run_pk_simulations(), metrics_occ()

Examples

data("tacrolimus_pk1_kidney", package = "exeval")
data("model_tacHAN2011", package = "exeval")

dd <- tacrolimus_pk1_kidney |> subset(ID < 6)

res <- exeval_ppk(model="TAC_Han2011",
                 data = dd,
                 evaluation_type= "sequential_updating",
                 assessment='Bayesian_forecasting' )

print(res) # Print the results

Compute external evaluation performance metrics

Description

Computes predictive performance metrics from simulation outputs generated during external model evaluation.

Usage

metrics_occ(
  simulations,
  assessment = c("a_priori", "Bayesian_forecasting", "Complete"),
  tool = "mapbayr",
  ...
)

Arguments

Details

This function compares simulated predictions with observed concentrations and calculates individual- and occasion-level prediction error metrics.

Individual predictions are matched with observed concentrations using subject identifier (ID), occasion (OCC), and observation time (TIME).

The following metrics are calculated:

• IPE: individual prediction error (%)

• APE: absolute prediction error (%)

• rRMSE: relative root mean squared error (%)

• rBIAS: relative bias (%)

• MAIPE: mean absolute individual prediction error (%)

• IF20: percentage of predictions within 20\

• IF30: percentage of predictions within 30\

Individual observations are additionally classified into fit quality categories (Excellent, Acceptable, Poor, Very Poor) based on absolute prediction error.

Value

A named list containing:

metrics

Data frame containing individual prediction errors and fit classifications for each subject, occasion, and observation time.

metrics_means

Data frame containing summary performance metrics aggregated by occasion.

See Also

run_pk_simulations(), plot.EvalPPK(), summary.EvalPPK()

Examples

data("exeval_models", package = "exeval")
data("tacrolimus_pk1_kidney", package = "exeval")

dd <- tacrolimus_pk1_kidney |> subset(ID < 6)

fit <- run_MAP_estimations(
  model = exeval_models$Model_code[[2]],
  model_name = "TAC_Zuo2013",
  data = dd,
  evaluation_type = "sequential_updating"
)

post <- update_map_models(
  map_results = fit,
  evaluation_type = "sequential_updating"
)

sim <- run_pk_simulations(
  individual_model = post,
  map_results = fit,
  assessment = "Complete"
)

mm <- metrics_occ(
  simulations = sim,
  assessment = "Complete"
)

Prepare input data for exeval

Description

Renames user-defined dataset columns to the standardized naming convention used internally by exeval.

Usage

prepare_data(
  data,
  name_id = NULL,
  name_time = NULL,
  name_occ = NULL,
  name_date = NULL,
  name_cmt = NULL,
  name_dv = NULL,
  name_mdv = NULL,
  name_amt = NULL,
  name_evid = NULL,
  name_ss = NULL,
  name_dur = NULL,
  name_lag = NULL,
  name_rate = NULL,
  name_ii = NULL,
  name_addl = NULL
)

Arguments

Details

This helper function allows external datasets with arbitrary column names to be reformatted for compatibility with the external evaluation workflow.

At minimum, ID and TIME mappings must be provided.

Additional columns can be optionally mapped depending on the analysis workflow and model requirements.

Value

A data frame with standardized column names compatible with exeval.

See Also

exeval_ppk()

Examples

df <- data.frame(
  patient = c(1, 1, 2),
  time = c(0, 12, 24),
  conc = c(NA, 8.4, 6.1),
  visit = c(1, 1, 2)
)

df_std <- prepare_data(
  data = df,
  name_id = "patient",
  name_time = "time",
  name_dv = "conc",
  name_occ = "visit"
)

head(df_std)

Run MAP Bayesian Estimation for External Model Evaluation

Description

Performs Maximum A Posteriori (MAP) Bayesian estimation using mapbayr for external evaluation of pharmacokinetic models across multiple dosing occasions.

Usage

run_MAP_estimations(
  model,
  model_name = NULL,
  tool = "mapbayr",
  check_compile = TRUE,
  data,
  num_occ = NULL,
  num_ids = NULL,
  sampling = TRUE,
  occ_ref = NULL,
  evaluation_type = c("sequential_updating", "stepwise_updating",
    "sequential_reference_updating", "backward_reference_updating"),
  method = c("L-BFGS-B", "newuoa")
)

Arguments

Details

The population model can be provided either as:

• a compiled mrgsolve::mrgmod object, or

• a character string containing mrgsolve model code.

When model code is supplied as a character string, the model is compiled internally using mrgsolve::mcode(). In this case, a model name must be provided via model_name.

The evaluation strategy defines which observations are used to inform each MAP estimation:

• "sequential_updating": cumulative observations up to each occasion (e.g., OCC1, OCC1+2, OCC1+2+3).

• "stepwise_updating": observations from each occasion treated independently.

• "sequential_reference_updating": cumulative observations up to the reference occasion occ_ref.

• "backward_reference_updating": sequential backward updating from occ_ref.

Value

A named list containing:

data_by_occ

List of input datasets partitioned according to the selected evaluation strategy, where each element contains the observations used for a specific MAP estimation.

treatments_by_occ

List of treatment/event datasets grouped by occasion and subject, used for posterior predictive simulations.

apriori_treatments

List of treatment/event datasets used for a priori predictive simulations.

map_estimations

List of MAP estimation objects returned by mapbayr::mapbayest() for each evaluation subset.

eval_type

Character string indicating the selected evaluation strategy.

pop_model

Compiled population model (mrgmod) used for estimation.

See Also

mapbayr::mapbayest(), mrgsolve::mcode()

Examples

data("exeval_models", package = "exeval")
data("tacrolimus_pk1_kidney", package = "exeval")

dd <- tacrolimus_pk1_kidney |> subset(ID < 6)

fit <- run_MAP_estimations(
  model = exeval_models$Model_code[[2]],
  model_name = "TAC_Zuo2013",
  data = dd,
  evaluation_type = "sequential_updating"
)

Run PK simulations for external model evaluation

Description

Simulates concentration-time profiles for external model evaluation using population predictions (a priori), individualized posterior predictions (Bayesian forecasting), or both, depending on the selected simulation strategy.

Usage

run_pk_simulations(
  individual_model,
  map_results,
  assessment = c("a_priori", "Bayesian_forecasting", "Complete"),
  seed = NULL,
  verbose = FALSE
)

Arguments

Details

This function performs pharmacokinetic simulations at the observation times available in the external evaluation dataset.

Depending on the selected assessment, simulations are performed using:

• the population model for a priori predictions,

• individualized posterior models for Bayesian forecasting,

• or both approaches for complete external evaluation.

This function represents the final simulation step in the external evaluation workflow following run_MAP_estimations() and, when posterior predictions are required, update_map_models().

Reproducibility of stochastic a priori simulations can be controlled using the seed argument.

Value

A named list containing:

simulation_results

List of simulated concentration-time profiles for each individual and evaluation occasion.

ttoocc

Treatment/event datasets grouped by occasion and used as simulation inputs.

eval_type

Character string indicating the evaluation strategy inherited from run_MAP_estimations().

events_tto

Event datasets used for each simulation.

assessment

Character string indicating the selected simulation strategy.

See Also

run_MAP_estimations(), update_map_models()

Examples

data("exeval_models", package = "exeval")
data("tacrolimus_pk1_kidney", package = "exeval")

dd <- tacrolimus_pk1_kidney |> subset(ID < 6)

fit <- run_MAP_estimations(
  model = exeval_models$Model_code[[2]],
  model_name = "TAC_Zuo2013",
  data = dd,
  evaluation_type = "sequential_updating"
)

post <- update_map_models(
  map_results = fit,
  evaluation_type = "sequential_updating"
)

sim <- run_pk_simulations(
  individual_model = post,
  map_results = fit,
  assessment = "Complete",
  seed = 123
)

Tacrolimus pharmacokinetic data in kidney transplant patients

Description

Pharmacokinetic and clinical data from adult kidney transplant recipients treated with tacrolimus, used for population pharmacokinetic model development, external evaluation, and methodological package examples.

Usage

tacrolimus_pk1_kidney

Format

A data.frame with 739 rows and 30 variables:

ID

Patient identifier

OCC

Number of the occasion

DD

Tacrolimus daily dose (mg)

AMT

Dose amount (mg)

TIME

Sequential time (hours)

POD

Post-operative days

DV

Observed tacrolimus concentration (ng/mL)

EVID

Event identifier

CMT

Compartment identifier

MDV

Missing dependent variable flag

II

Dosing interval (hours)

SS

Steady-state indicator

Creatine

Creatinine (mg/dL)

SCR

Serum creatinine (\mu mol/L)

eGFR

Estimated glomerular filtration rate (mL/min/1.73 m^2)

ClCrea

Creatinine clearance (Cockcroft-Gault, mL/min)

AGE

Age (years)

SEX

Sex

WT

Body weight (kg)

HCT

Hematocrit

CYP3A5

CYP3A5 polymorphism

EXPRESSION

CYP3A5 expresser status

PDN_DOSE

Prednisone dose (mg)

PDNXWT

Prednisone dose normalized by body weight (mg/kg)

Height

Height (cm)

Height..m.

Height (m)

BSA

Body surface area (m^2)

BMIcalc

Body mass index (kg/m^2)

LBW

Lean body weight (kg)

DMELITU

Diabetes mellitus indicator

Details

This dataset corresponds to a Uruguayan kidney transplant cohort.

Source

De-identified clinical dataset adapted for methodological research and package examples.

References

Umpierrez M, et al. (2025). Accelerating Tacrolimus Model-Informed Precision Dosing in Kidney Transplant Recipients: Model Evaluation and Refinement Strategies.

Update MAP estimation objects with posterior individual parameters

Description

Converts MAP estimation results obtained with run_MAP_estimations() into individualized posterior models using mapbayr. Depending on the selected evaluation strategy, posterior information is propagated across occasions to support posterior predictive simulations.

Usage

update_map_models(
  map_results,
  evaluation_type = c("sequential_updating", "stepwise_updating",
    "sequential_reference_updating", "backward_reference_updating")
)

Arguments

Details

This function applies mapbayr::use_posterior() to each MAP estimation object and returns a list of updated individual models that can be used for posterior predictive simulations.

The evaluation strategy must match the one originally used in run_MAP_estimations().

This function applies mapbayr::use_posterior() to each MAP estimation object contained in map_results, generating individualized posterior models for subsequent simulation.

Posterior information is propagated across occasions according to the selected evaluation_type, which must match the strategy originally used in run_MAP_estimations().

Posterior model objects are dynamically named following the pattern a.posteriori_occX_Y, where X and Y indicate the occasions linked by the posterior update.

The resulting objects are intended for use with run_pk_simulations().

Value

A named list containing:

ind_model

List of posterior individualized model objects created using mapbayr::use_posterior().

eval_type

Character string indicating the evaluation strategy used.

Examples

data("exeval_models", package = "exeval")
data("tacrolimus_pk1_kidney", package = "exeval")

dd <- tacrolimus_pk1_kidney |> subset(ID < 6)

fit <- run_MAP_estimations(
  model = exeval_models$Model_code[[2]],
  model_name = "TAC_Zuo2013",
  data = dd,
  evaluation_type = "sequential_updating"
)

post <- update_map_models(
  map_results = fit,
  evaluation_type = "sequential_updating"
)