Bake Parallel Jobs in Background with bakerrr

The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Overview

The {bakerrr} S7 class enables efficient background and parallel job orchestration in R, making it easy to apply a function to multiple sets of arguments using configurable daemons. This vignette demonstrates usage with sample inputs and inspects job status and results.

Job Architecture

Daemons: Background R processes that execute jobs in parallel
Arguments List: Each element contains named arguments for the target function
Results Collection: Automatic aggregation with error capture and reporting

Setup

Define a simple function and generate a list of argument sets, introducing a mix of numeric and non-numeric (error-producing) cases.

fun <- function(x, y) {
  Sys.sleep(2)
  x + y
}

# Note how each list item is a set of arguments for the function above.
args_list <- list(
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = "p", y = ceiling(rnorm(1) * 10)),  # Intentional type error
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  # Add more sets as needed
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10)),
  list(x = ceiling(rnorm(1) * 10), y = ceiling(rnorm(1) * 10))
)

Creating and Running Jobs

Instantiate a bakerrr object and process jobs in parallel with configurable daemon count.

new_stirr <- bakerrr::bakerrr(
  fun,
  args_list,
  n_daemons = 4
) |>
  bakerrr::run_jobs(wait_for_results = TRUE)

Inspecting Status and Results

Print job summary and view results. Note that jobs triggering errors report the error messages as designed.

print(new_stirr)
new_stirr@results
#> [[1]]
#> [1] -2
#> 
#> [[2]]
#> Error in purrr::in_parallel: non-numeric argument to binary operator
#> 
#> [[3]]
#> [1] 16
#> 
#> [[4]]
#> [1] 30
#> 
#> [[5]]
#> [1] 28
#> 
#> [[6]]
#> [1] 3
#> 
#> [[7]]
#> [1] -12
#> 
#> [[8]]
#> [1] 4
#> 
#> [[9]]
#> [1] 17
#> 
#> [[10]]
#> [1] 8

Notes

Use n_daemons to control parallel worker count based on available cores.
Error handling is built in; results will include error messages for failed jobs.
You can pass additional background job options via bg_args.
For more details, see function-level documentation or explore extending with additional job handlers and reporting tools.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.