The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Forecasting pools

Rolling-origin pools

MFF aggregates point forecasts. For forecasting, the validation forecasts must be produced the way the test forecasts will be: from origins that move through time, using only the observations available at each origin. rolling_train() builds such a pool for a univariate series. For every target month \(t\) of the validation and test blocks, each model is fitted to the observations up to \(t - h\) and its \(h\)-step-ahead forecast becomes the candidate prediction for \(t\).

The models below are seasonal specifications for the log of the monthly AirPassengers series, written with the forecast package. Each model is a function of the training window y and the horizon h; it may return a numeric vector or a forecast object.

y <- log(datasets::AirPassengers)
models <- list(
  ets_aaa = function(y, h) forecast::forecast(forecast::ets(y, model = "AAA"), h = h),
  airline = function(y, h) {
    forecast::forecast(forecast::Arima(y, order = c(0, 1, 1),
                                       seasonal = c(0, 1, 1)), h = h)
  },
  stl_ets = function(y, h) forecast::stlf(y, h = h),
  seasonal_naive = function(y, h) forecast::snaive(y, h = h)
)
pool <- rolling_train(y, models, horizon = 1, n_validation = 24, n_test = 24)
pool
#> MFF prediction pool
#>   Task: forecasting | Pool: heterogeneous 
#>   Candidates: 4 | Validation: 24 | Test: 24

The last 24 months are test targets and the 24 months before them are validation targets. Origins and settings are recorded with the pool.

info <- pool_info(pool)
range(info$metadata$validation_origins)
#> [1] 1956.917 1958.833
range(info$metadata$test_origins)
#> [1] 1958.917 1960.833

Weights are learned on the validation forecasts only.

fit <- tune_mff(pool, clusters = 2:3, method = "fcm", m = c(1.5, 2),
                control = mff_control(seed = 21))
selected_configuration(fit)[c("clusters", "m")]
#> $clusters
#> [1] 3
#> 
#> $m
#> [1] 2
weights(fit)
#>        ets_aaa        airline        stl_ets seasonal_naive 
#>   3.374312e-04   5.191592e-01   4.805033e-01   1.253406e-10
evaluate(predict(fit, pool), test_observed(pool))
#>             MAE       RMSE      MAPE    SMAPE         MSE      MedAE
#> [1,] 0.02510835 0.03316024 0.4109221 0.411017 0.001099601 0.02169879
plot(fit, type = "series", newdata = pool)

Different horizons are different forecasting tasks, so each horizon has its own pool, for example rolling_train(y, models, horizon = 3, ...). For \(h > 1\) the validation block ends at the origin of the first test target, \(h\) months before it, and the \(h - 1\) months in between are left out: otherwise validation outcomes observed after a test origin would influence the selected aggregation. window = "sliding" with window_size fits every model to a fixed number of recent observations instead of an expanding window. With mff_control(seed = ) each origin receives its own seed, and parallel = TRUE distributes origins over socket workers.

Forecasts from fable

Rolling-origin forecasts produced with fable on a stretched tsibble can be converted directly. horizon selects the forecast step taken from every origin and the last n_test target times form the test block; validation targets end at the first test origin, as above. The observed tsibble must contain only the forecast series.

ap <- tsibble::as_tsibble(datasets::AirPassengers)
stretched <- tsibble::stretch_tsibble(ap, .init = 108, .step = 1)
fable_fc <- fabletools::forecast(
  fabletools::model(stretched,
                    snaive = fable::SNAIVE(log(value)),
                    ets = fable::ETS(log(value) ~ error("A") + trend("A") +
                                       season("A")),
                    drift = fable::RW(log(value) ~ drift())),
  h = 1)
fable_pool <- as_mff_pool(fable_fc, data = ap, horizon = 1, n_test = 12)
fable_pool
#> MFF prediction pool
#>   Task: forecasting | Pool: heterogeneous 
#>   Candidates: 3 | Validation: 24 | Test: 12

Externally produced forecasts

Forecasts from other software enter through mff_pool() with task = "forecasting" and time indexes. The constructor checks that the indexes are strictly increasing and that validation precedes test; it cannot verify how the external forecasts were produced, so they should come from a rolling-origin scheme like the one above.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.