The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Parallel Workers Running in Linux Containers

Introduction

This vignette shows how to set up parallel workers running in Linux containers, e.g. Docker (https://www.docker.com/), Apptainer (https://apptainer.org/), and udocker (https://indigo-dc.github.io/udocker/).

Examples

Example: Two parallel workers running in Docker

This example sets up two parallel workers running Docker image ‘rocker/r-parallel’ (https://hub.docker.com/r/rocker/r-parallel).

library(parallelly)
cl <- makeClusterPSOCK(
  rep("localhost", times = 2L),
  ## Launch Rscript inside Linux container via Docker
  rscript = c(
    "docker", "run", "--net=host", "rocker/r-parallel",
    "Rscript"
  ),
  ## IMPORTANT: Because Docker runs inside a virtual machine (VM) on macOS
  ## and MS Windows (not Linux), when the R worker tries to connect back to
  ## the default 'localhost' it will fail, because the main R session is
  ## not running in the VM, but outside on the host.  To reach the host on
  ## macOS and MS Windows, make sure to use master = "host.docker.internal"
  master = if (.Platform$OS.type == "unix") NULL else "host.docker.internal",
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 4.3.3 (2024-02-29), platform x86_64-pc-linux-gnu)

Example: Two parallel workers running in Apptainer

This example shows how to set up two parallel workers running Docker image ‘rocker/r-parallel’ (https://hub.docker.com/r/rocker/r-parallel) via Apptainer (<https://apptainer.org/).

library(parallelly)
cl <- makeClusterPSOCK(
  rep("localhost", times = 2L),
  ## Launch Rscript inside Linux container via Apptainer
  rscript = c(
    "apptainer", "exec", "docker://rocker/r-parallel",
    "Rscript"
  )
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 3.6.1 (2019-07-05), platform x86_64-pc-linux-gnu)

Example: Two parallel workers running in udocker

This example shows how to set up two parallel workers running Docker image ‘rocker/r-parallel’ (https://hub.docker.com/r/rocker/r-parallel) via udocker (https://indigo-dc.github.io/udocker/).

library(parallelly)
cl <- makeClusterPSOCK(
  rep("localhost", times = 2L),
  ## Launch Rscript inside Linux container via Docker
  rscript = c(
    "udocker", "--quiet", "run", "rocker/r-parallel",
    "Rscript"
  )
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 3.6.1 (2019-07-05), platform x86_64-pc-linux-gnu)

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.