4 Performance benchmarking of HaDeX and HaDeX2
For each pair of functions in the previous section, we can assess relative execution speed using the exemplary dataset as a controlled reference for comparison. We concentrate on six major tasks: reading data file, plotting (and preparing data) uptake curve for a single peptide, comparison plot of two biological states, differential Woods plot with difference between two states, reconstruction of the protein sequence and computation of confidence limits.
We performed the benchmark utilizing the code shown below.
library(HaDeX)
dat_HaDeX <- HaDeX::read_hdx(system.file(package = "HaDeX2", "HaDeX/data/alpha.csv"))
dat_HaDeX2 <- HaDeX2::read_hdx(system.file(package = "HaDeX2", "HaDeX/data/alpha.csv"))
version_benchmark <- microbenchmark(
list = alist(`HaDeX_1. Read input` = HaDeX::read_hdx(system.file(package = "HaDeX2",
"HaDeX/data/alpha.csv")),
`HaDeX2_1. Read input` = HaDeX2::read_hdx(system.file(package = "HaDeX2",
"HaDeX/data/alpha.csv")),
`HaDeX_2. Plot uptake curve` = {
HaDeX::calculate_kinetics(dat = dat_HaDeX,
sequence = "GFGDLKSPAGL",
state = "Alpha_KSCN",
start = 1, end = 11,
time_in = 0, time_out = 1440) %>%
HaDeX::plot_kinetics(kin_dat = .)},
`HaDeX2_2. Plot uptake curve` = {
HaDeX2::calculate_peptide_kinetics(dat = dat_HaDeX2,
sequence = "GFGDLKSPAGL",
state = "Alpha_KSCN",
start = 1, end = 11,
time_0 = 0, time_100 = 1440) %>%
HaDeX2::plot_uptake_curve(uc_dat = .)},
`HaDeX_3. Plot comparison` = {
HaDeX::prepare_dataset(dat = dat_HaDeX,
in_state_first = "Alpha_KSCN_0",
chosen_state_first = "Alpha_KSCN_1",
out_state_first = "Alpha_KSCN_1440",
in_state_second = "ALPHA_Gamma_0",
chosen_state_second = "ALPHA_Gamma_1",
out_state_second = "ALPHA_Gamma_1440") %>%
HaDeX::comparison_plot(calc_dat = .,
theoretical = FALSE,
relative = TRUE,
state_first = "Alpha_KSCN",
state_second = "ALPHA_Gamma")},
`HaDeX2_3. Plot comparison` = {
HaDeX2::create_state_comparison_dataset(dat = dat_HaDeX2,
states = c("Alpha_KSCN",
"ALPHA_Gamma"),
time_0 = 0, time_100 = 1440) %>%
HaDeX2::plot_state_comparison(uptake_dat = .,
theoretical = FALSE,
fractional = TRUE,
time_t = 1)},
`HaDeX_4. Plot Woods` = {
HaDeX::prepare_dataset(dat = dat_HaDeX,
in_state_first = "Alpha_KSCN_0",
chosen_state_first = "Alpha_KSCN_1",
out_state_first = "Alpha_KSCN_1440",
in_state_second = "ALPHA_Gamma_0",
chosen_state_second = "ALPHA_Gamma_1",
out_state_second = "ALPHA_Gamma_1440") %>%
HaDeX::woods_plot(calc_dat = .,
theoretical = FALSE,
relative = TRUE,
confidence_limit = 0.98,
confidence_limit_2 = 0.98)},
`HaDeX2_4. Plot Woods` = {
HaDeX2::calculate_diff_uptake(dat = dat_HaDeX2,
states = c("Alpha_KSCN", "ALPHA_Gamma"),
time_t = 1, time_0 = 0, time_100 = 1440) %>%
HaDeX2::plot_differential(diff_uptake_dat = .,
time_t = 1,
theoretical = FALSE,
fractional = TRUE,
show_houde_interval = TRUE,
confidence_level = 0.98)},
`HaDeX_5. Calculate confidence limit` = {
HaDeX::prepare_dataset(dat = dat_HaDeX,
in_state_first = "Alpha_KSCN_0",
chosen_state_first = "Alpha_KSCN_1",
out_state_first = "Alpha_KSCN_1440",
in_state_second = "ALPHA_Gamma_0",
chosen_state_second = "ALPHA_Gamma_1",
out_state_second = "ALPHA_Gamma_1440") %>%
HaDeX::calculate_confidence_limit_values(calc_dat = .,
confidence_limit = 0.98,
theoretical = FALSE,
relative = TRUE)},
`HaDeX2_5. Calculate confidence limit` = {
HaDeX2::calculate_diff_uptake(dat = dat_HaDeX2,
states = c("Alpha_KSCN", "ALPHA_Gamma"),
time_0 = 0, time_100 = 1440, time_t = 1) %>%
HaDeX2::calculate_confidence_limit_values(diff_uptake_dat = .,
confidence_level = 0.98,
theoretical = FALSE,
fractional = TRUE)},
`HaDeX_6. Reconstruct sequence` = HaDeX::reconstruct_sequence(dat = dat_HaDeX),
`HaDeX2_6. Reconstruct sequence` = HaDeX2::reconstruct_sequence(dat = dat_HaDeX2)
)
)The microbenchmark package operates by repeatedly executing each command 100 times o obtain a stable and representative estimate of execution time.
Figure 4.1: Benchmark results.
| task | HaDeX | HaDeX2 | Runtime ratio |
|---|---|---|---|
| 1. Read input | 34.83070 | 28.86460 | 0.8287115 |
| 2. Plot uptake curve | 171.86675 | 65.38155 | 0.3804200 |
| 3. Plot comparison | 186.86305 | 59.34140 | 0.3175663 |
| 4. Plot Woods | 201.53030 | 77.75905 | 0.3858430 |
| 5. Calculate confidence limit | 172.91570 | 53.16090 | 0.3074383 |
| 6. Reconstruct sequence | 24.18105 | 15.78765 | 0.6528935 |
Across all tasks, the reported values represent a runtime ratio (HaDeX2/HaDeX) consistently below one, indicating that HaDeX2 is faster than HaDeX for every measured operation. The strongest speedups, corresponding to the lowest ratios, are observed for plotting functions, calculating confidence limits, and plotting uptake curves, whereas input reading and sequence reconstruction show comparatively smaller, though still meaningful, reductions in execution time. In the case of input reading, the modest speed-up results from the fact that this functionality has substantially expanded in-built quality control in HaDeX2, where additional validation steps intentionally constrain maximal speed in favor of improved data integrity.