PredictTestbench: Testbench for Comparison of Data Prediction Models

Neeraj Bokde neerajdhanraj@gmail.com

2016-08-22

This Document is to introduce the R package ‘PredictTestbench’. It is an testing workbench for comparison of time series data prediction models/methods. It compares prediction methods with reference to RMSE, MAE or MAPE parameters. It allows to add new proposed methods to test bench and to compare with other methods. The function prediction_append() allows to add multiple numbers of methods to the existing methods available in test bench.

Following example describs the working of this package:

Consider a sample data ‘sunspots’ as follows:

datax <- sunspots

Import library for Package PredictTestbench as follows:

library(PredictTestbench)

The function prediction_errors() is used to compare imputing methods with reference to RMSE, MAE or MAPE parameters. Syntax of prediction_errors() as shown below:

prediction_errors(dataIn, nextVal, errorParameter, MethodPath, MethodName)

where,

At simplest form, function prediction_errors() can we used as:

q <- prediction_errors(datax, nextVal = 10)
q
## $Parameter
## [1] "RMSE Plot"
## 
## $Desired_Prediction
##  [1] 66.5 80.7 99.2 91.1 82.2 71.8 50.3 55.8 33.3 33.4
## 
## $ARIMA_Method_Prediction
##  [1] 70.02228 75.70179 75.86081 73.84612 71.08016 68.15879 65.32361
##  [8] 62.66504 60.20967 57.95770
## 
## $ARIMA_Method_Error
## [1] 16.1764
## 
## $ARIMA_Execution_Time_in_Seconds
## [1] 3.513408
## 
## $PSF_Method_Prediction
##  [1] 73.45000 73.71667 68.11667 67.91667 65.21667 58.50000 57.36667
##  [8] 50.61667 57.41667 50.20000
## 
## $PSF_Method_Error
## [1] 17.33982
## 
## $PSF_Execution_Time_in_Seconds
## [1] 2.650613
plot_predictions(q)

By default, this function compares two prediction methods, i.e. ARIMA and PSF https://cran.r-project.org/package=PSF. The plot_predictions() function is used to plot the comparison plots between different methods. This testbench allows to add one more data prediction method to compare with already existing methods. The only care is to be takes as, the new imputing method is to be designed in function format such that it could return predicted data as output. Suppose, following function is the desired method to add in testbench.

===============================

inter <- function(dataIn)

{

library(PSF)

d <- pred_for_w(dataIn, 4, 3, 10)

return(d)

}

===============================

Save this function in new R script file and save it and note its Source location similar to "source('~/PredictTestbench/R/inter.R')" and use prediction_errors() function as:

#aa <- prediction_errors(dataIn= datax, nextVal = 10, MethodPath = "source('~/PredictTestbench/R/inter.R')", MethodName = "Proposed_Method")

#aa
#plot_predictions(aa)

This above code is written in commented format, since this function is dependent on other function and its location, which is not included in this package.

If user wishes to add more than one imputation methods to test bench, the function prediction_append() is used as:

#bb <- prediction_append(existing_method = aa, dataIn= datax, nextVal = 10, MethodPath = "source('~/imputeTestbench/R/PSFimpute.R')", MethodName = "PSFimpute")

#bb
#plot_predictions(bb)

where

In similar way, user is allowed to remove unwanted prediction method in testbench, if any. This is done with prediction_remove() function.

# cc <- prediction_remove (existing_method = bb, index_number = 1)
# cc
# plot_predictions(cc)

where

One/two Step ahead forecasting is done with function

#a1 <- step_ahead_forecast(dataIn = datax, trainedData = 80, MethodPath = "source('~/imputeTestbench/R/PSFimpute.R')", errorParameter = 3, stepSize = 1)
#a1