The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

rush

Package website: release | dev

rush is a package designed to solve large-scale problems asynchronously across a distributed network. Employing a database centric model, rush enables workers to communicate tasks and their results over a shared Redis database. Key features include low task overhead, efficient caching, and robust error handling. The package powers asynchronous optimization algorithms in the bbotk and mlr3tuning packages.

Features

• Database centric model for robust scalability.
• Efficient communication between workers usingRedis.
• Maintains low overhead, limiting delays to just a millisecond per task.
• Reduces read/write operations with a lightweight and efficient caching system.
• Offers centralized system features, such as task queues.
• Provides fast data transformation from Redis to data.table.
• Simplifies local worker setup with processx.
• Enables scaling to large remote worker networks via the mirai package.
• Automatically detects and recovers from worker failures for high reliability.
• Logs worker messages directly into the Redis database using lgr.
• Designed with minimal dependencies for lightweight integration.

Install

Install the latest release from CRAN.

install.packages("rush")

Install the development version from GitHub.

pak::pak("mlr-org/rush")

And install Redis.

Related Work

• The rrq package is a task queue system for R using Redis.
• The future package provides a simple and uniform way of evaluating R expressions asynchronously across a range of backends.
• batchtools is a package for the execution of long-running tasks on high-performance computing clusters.
• The mirai package evaluates an R expression asynchronously in a parallel process, locally or distributed over the network.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.