The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
PublicationBiasBenchmark is an R package for benchmarking publication bias correction methods through simulation studies. It provides:
All datasets, results, and measures are hosted on Zenodo: https://zenodo.org/communities/publicationbiasbenchmark/ (originally on OSF: https://doi.org/10.17605/OSF.IO/EXF3M).
For the methodology of living synthetic benchmarks please cite:
Bartoš, F., Pawel, S., & Siepe, B. S. (2025). Living synthetic benchmarks: A neutral and cumulative framework for simulation studies. arXiv Preprint. https://doi.org/10.48550/arXiv.2510.19489
For the publication bias benchmark R package please cite:
Bartoš F, Pawel S, Siepe BS, Čala P (2026). PublicationBiasBenchmark: Benchmark for publication bias correction methods. version 0.4.0, https://cran.r-project.org/package=PublicationBiasBenchmark.
Overviews of the benchmark results are available as articles on the package website:
Contributor guidelines for extending the package with data-generating mechanisms, methods, and results are available at:
Illustrations of how to use the precomputed datasets, results, and measures are available at:
The rest of this file overviews the main features of the package.
Install the released CRAN version
install.packages("PublicationBiasBenchmark")Install latest development version from GitHub
remotes::install_github("FBartos/PublicationBiasBenchmark")Addition or modification of a method or data-generating mechanisms is always reflected in a minor version update. Minor changes to infrastructure etc are reflected in patch updates.
library(PublicationBiasBenchmark)# Obtain a data.frame with pre-defined conditions
dgm_conditions("Stanley2017")
# simulate the data from the second condition
df <- simulate_dgm("Stanley2017", 2)
# fit a method
run_method("RMA", df)# download the pre-simulated datasets
# (the intended location for storing the package resources needs to be specified)
PublicationBiasBenchmark.options(resources_directory = "/path/to/files")
download_dgm_datasets("no_bias")
# retrieve first repetition of first condition from the downloaded datasets
retrieve_dgm_dataset("no_bias", condition_id = 1, repetition_id = 1)# download the pre-computed results
download_dgm_results("no_bias", method = "RMA")
# retrieve results the first repetition of first condition of RMA from the downloaded results
retrieve_dgm_results("no_bias", method = "RMA", condition_id = 1, repetition_id = 1)
# retrieve all results across all conditions and repetitions
download_dgm_results("no_bias")
retrieve_dgm_results("no_bias")# download the pre-computed measures
download_dgm_measures("no_bias", method = "RMA")
# retrieve measures of bias the first condition of RMA from the downloaded results
retrieve_dgm_measures("no_bias", measure = "bias", method = "RMA", condition_id = 1)
# retrieve all measures across all conditions and measures
download_dgm_measures("no_bias")
retrieve_dgm_measures("no_bias")# define sim setting
sim_settings <- list(
n_studies = 100,
mean_effect = 0.3,
heterogeneity = 0.1
)
# check whether it is feasible
# (defined outside of the function - not to decrease performance during simulation)
validate_dgm_setting("no_bias", sim_settings)
# simulate the data
df <- simulate_dgm("no_bias", sim_settings)
# fit a method
run_method("RMA", df)simulate_dgm(): Generates simulated data according to
specified data-generating mechanism and settings.dgm_conditions(): Lists prespecified conditions of the
data-generating mechanism.validate_dgm_setting(): Validates (custom) setting of
the data-generating mechanism.download_dgm_datasets(): Downloads pre-simulated
datasets from the Zenodo release.retrieve_dgm_dataset(): Retrieves the pre-simulated
dataset of a given condition and repetition from the downloaded
benchmark data.run_method(): Estimates method on a supplied data
according to the specified settings.method_settings(): Lists prespecified settings of the
method.download_dgm_results(): Downloads pre-computed results
from the Zenodo release.retrieve_dgm_results(): Retrieves the pre-computed
results of a given method, condition, and repetition from the downloaded
benchmark data.bias(), bias_mcse(), etc.: Functions to
compute performance measures and their Monte Carlo standard errors.download_dgm_measures(): Downloads pre-computed
performance measures from the Zenodo release.retrieve_dgm_measures(): Retrieves the pre-computed
performance measures of a given method, condition, and repetition from
the downloaded benchmark data.See methods("dgm") for the full list:
"no_bias": Generates data without publication bias (a
test simulation)"Stanley2017": Stanley et al. (2017)"Alinaghi2018": Alinaghi & Reed (2018)"Bom2019": Bom & Rachinger (2019)"Carter2019": Carter et al. (2019)See methods("method") for the full list:
"mean": Mean effects size"FMA": Fixed effects meta-analysis"RMA": Random effects meta-analysis"WLS": Weighted Least Squares"trimfill": Trim-and-Fill (Duval & Tweedie,
2000)"WAAPWLS": Weighted Least Squares - Weighted Average of
Adequately Power Studies (Stanley et al., 2017)"WILS": Weighted and Iterated Least Squares (Stanley
& Doucouliagos,
2024)"PET": Precision-Effect Test (PET) publication bias
adjustment (Stanley & Doucouliagos, 2014)"PEESE": Precision-Effect Estimate with Standard Errors
(PEESE) publication bias adjustment (Stanley & Doucouliagos,
2014)"PETPEESE": Precision-Effect Test and Precision-Effect
Estimate with Standard Errors (PET-PEESE) publication bias adjustment
(Stanley & Doucouliagos, 2014)"EK": Endogenous Kink (Bom & Rachinger, 2019)"SM": Selection Models (3PSM, 4PSM) (Vevea &
Hedges, 1995)"pcurve": P-curve (Simonsohn et al., 2014)"puniform": P-uniform (van Assen
et al., 2015) and P-uniform* (van Aert &
van Assen, 2025)"AK": Andrews & Kasy selection models (AK1, AK2)
(Andrews & Kasy,
2019)"RoBMA": Robust Bayesian Meta-Analysis (Bartoš et al.,
2023)"MAIVE": Meta Analysis Instrumental Variable Estimator
(Irsova et al., 2025)"MAN": Meta-Analysis of Nonaffirmative Studies (Mathur
& VanderWeele, 2020)"RTMA": Right-Truncated Meta-Analysis (Mathur,
2024)"MMPH": Mixture Model of P-Hacking (Moss & De Bin,
2023)See ?measures for the full list of performance measures
and their Monte Carlo standard errors/
The benchmark data are published in versioned releases, and each
package version uses a default release. Use
list_benchmark_releases() to see the available releases and
the release argument (e.g.,
release = "2026.2") to use a specific one. See Benchmark
Releases for details and for how to cite a release.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.