The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Introduction to mizu

mizu provides parallel computation and data exchange between R processes on the same machine: lock-free channels and work-stealing task pools over POSIX shared memory (Linux, macOS) or Win32 file mappings (Windows).

A channel is a two-way message link between an R session and a helper process that it spawns. A pool is a set of worker processes that divide submitted tasks among themselves.

In both, one process writes data into shared memory and the other reads it in place, with no copy through a socket, pipe, or file.

library(mizu)

p <- mizu_pool(n_workers = 4L)
t <- mizu_submit(p, sum(x) + y, x = 1:10, y = 100)
mizu_collect(t)
#> [1] 155
m <- mizu_map(p, 1:5, \(i) i * 2L, .template = integer(1))
mizu_collect(m)
#> [1]  2  4  6  8 10
mizu_pool_stop(p)

The articles

Benchmarks — measurements against mirai.
Channels — two-way message links between R processes: send and receive, batched forms, sentinel values, remote errors.
Task pools — work-stealing worker pools: submit and collect, nested tasks, waiting on several tasks, sizing and observing a pool.
Parallel map — mizu_map(): mapping over a pool with the function and data staged once, templates, reproducible random numbers, prepared maps.
Python interop — channels and pools shared with Python processes through pymizu, including the cross-language map.
Operations — sizing /dev/shm on Linux, the Linux memory allocator, and what happens when a process crashes.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.