The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Validation and comparison

This vignette provides a lightweight, fully reproducible benchmark. It is designed for package documentation, not as a final simulation study for a journal article.

The benchmark includes:

Compared methods:

Metrics:

For paper-grade evidence, increase the number of repetitions, document the simulation design, and report uncertainty around the summarized metrics.

Run benchmark suite

result <- run_validation_suite(reps = 2, grid_size_1d = 60, grid_size_2d = 16)
summary_tbl <- summarise_validation(result)
summary_tbl
#>    scenario dimension method    ise_mean elapsed_mean  ise_median
#> 1        S1         1    ASH 0.032815853       0.0010 0.032815853
#> 2        S1         1  GLBFP 0.021953737       0.0015 0.021953737
#> 3        S1         1    KDE 0.002658040       0.0000 0.002658040
#> 4        S1         1   LBFP 0.021953737       0.0015 0.021953737
#> 5        S2         1    ASH 0.020182782       0.0010 0.020182782
#> 6        S2         1  GLBFP 0.013839910       0.0020 0.013839910
#> 7        S2         1    KDE 0.005633544       0.0000 0.005633544
#> 8        S2         1   LBFP 0.013839910       0.0020 0.013839910
#> 9        S3         1    ASH 0.029677125       0.0005 0.029677125
#> 10       S3         1  GLBFP 0.021946772       0.0015 0.021946772
#> 11       S3         1    KDE 0.006869287       0.0000 0.006869287
#> 12       S3         1   LBFP 0.021946772       0.0015 0.021946772
#> 13       S4         2    ASH 0.038106086       0.0035 0.038106086
#> 14       S4         2  GLBFP 0.018294805       0.0100 0.018294805
#> 15       S4         2    KDE 0.003425851       0.0000 0.003425851
#> 16       S4         2   LBFP 0.018294805       0.0095 0.018294805
#> 17       S5         2    ASH 0.047296014       0.0035 0.047296014
#> 18       S5         2  GLBFP 0.023920100       0.0095 0.023920100
#> 19       S5         2    KDE 0.004657292       0.0005 0.004657292
#> 20       S5         2   LBFP 0.023920100       0.0090 0.023920100
#>    elapsed_median
#> 1          0.0010
#> 2          0.0015
#> 3          0.0000
#> 4          0.0015
#> 5          0.0010
#> 6          0.0020
#> 7          0.0000
#> 8          0.0020
#> 9          0.0005
#> 10         0.0015
#> 11         0.0000
#> 12         0.0015
#> 13         0.0035
#> 14         0.0100
#> 15         0.0000
#> 16         0.0095
#> 17         0.0035
#> 18         0.0095
#> 19         0.0005
#> 20         0.0090

MISE ranking by scenario

mi <- summary_tbl[order(summary_tbl$scenario, summary_tbl$ise_mean), ]
mi[, c("scenario", "dimension", "method", "ise_mean", "elapsed_mean")]
#>    scenario dimension method    ise_mean elapsed_mean
#> 3        S1         1    KDE 0.002658040       0.0000
#> 2        S1         1  GLBFP 0.021953737       0.0015
#> 4        S1         1   LBFP 0.021953737       0.0015
#> 1        S1         1    ASH 0.032815853       0.0010
#> 7        S2         1    KDE 0.005633544       0.0000
#> 6        S2         1  GLBFP 0.013839910       0.0020
#> 8        S2         1   LBFP 0.013839910       0.0020
#> 5        S2         1    ASH 0.020182782       0.0010
#> 11       S3         1    KDE 0.006869287       0.0000
#> 10       S3         1  GLBFP 0.021946772       0.0015
#> 12       S3         1   LBFP 0.021946772       0.0015
#> 9        S3         1    ASH 0.029677125       0.0005
#> 15       S4         2    KDE 0.003425851       0.0000
#> 14       S4         2  GLBFP 0.018294805       0.0100
#> 16       S4         2   LBFP 0.018294805       0.0095
#> 13       S4         2    ASH 0.038106086       0.0035
#> 19       S5         2    KDE 0.004657292       0.0005
#> 18       S5         2  GLBFP 0.023920100       0.0095
#> 20       S5         2   LBFP 0.023920100       0.0090
#> 17       S5         2    ASH 0.047296014       0.0035

Sensitivity analysis (b, m)

result$sensitivity
#>   b_scale m mean_density max_density
#> 1     0.8 1   0.01875666   0.1965201
#> 2     1.0 1   0.01847158   0.1951697
#> 3     1.2 1   0.01785758   0.1301115
#> 4     1.0 2   0.01783902   0.1514042

Notes

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.