---
title: "Introduction to 'ForecastTB' package"
author: "Neeraj Dhanraj Bokde (neerajdhanraj@eng.au.dk) and Gorm Bruun Andresen (gba@eng.au.dk)"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Introduction_to_ForecastTB}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
editor_options: 
  chunk_output_type: inline
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

## Demonstration of 'ForecastTB' package:

This document demonstates the R package 'ForecastTB'. It is intended for comparing the performance of forecasting methods. The package assists in developing background, strategies, policies and environment needed for comparison of forecasting methods. A comparison report for the defined framework is produced as an output. Load the package as following:

```{r setup}
library(ForecastTB)
```

The basic function of the package is `prediction_errors()`. Following are the parameters considered by this function:

* `data`: input time series for testing

* `nval`: an integer to decide number of values to predict (default:`12`)

* `ePara`: type of error calculation (RMSE and MAE are default), add an error parameter of your choice in the following manner: `ePara = c("errorparametername")`, where errorparametername is should be a source/function which returns desired error set. (default:`RMSE` and `MAE`)

* `ePara_name`: list of  names of error parameters passed in order (default:`RMSE` and `MAE`)

* `Method`: list of locations of function for the proposed prediction method (should be recursive) (default:`ARIMA`)

* `MethodName`: list of names for function for the proposed prediction method in order (default:`ARIMA`)

* `strats`: list of forecasting strategies. Available : `recursive` and `dirRec`. (default:`recursive`)

* `append_`: suggests if the function is used to append to another instance. (default:`1`)

* `dval`: last d values of the data to be used for forecasting (default: length of the `data`)


The `prediction_errors()` function returns, two slots as output. First slot is `output`, which provides `Error_Parameters`, indicating error values for the forecasting methods and error parameters defined in the framework, and `Predicted_Values` as values forecasted with the same foreasting methods. Further, the second slot is `parameters`, which returns the parameters used or provided to `prediction_errors()` function.


```{r}
a <- prediction_errors(data = nottem)  #`nottem` is a sample dataset in CRAN

a
```


The quick visualization of the object retuned with `prediction_errors()` function can be done with `plot()` function as below:

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
b <- plot(a)
```
--------
********

## Comparison of multiple methods:

As discussed above, `prediction_errors()` function evaluates the performance of `ARIMA` method. In addition, it allows to compare performance of distinct methods along with `ARIMA`. In following example, two methods (`LPSF` and `PSF`) are compared along with the `ARIMA`. These methods are formatted in the form of a function, which requires `data` and `nval` as input parameters and must return the `nval` number of frecasted values as a vector. In following code, `test1()` and `test2()` functions are used for `LPSF` and `PSF` methods, respectively.


```{r fig.height = 7, fig.width = 7, fig.align = "center"}
library(decomposedPSF)
test1 <- function(data, nval){
   return(lpsf(data = data, n.ahead = nval))
}

library(PSF)
test2 <- function(data, nval){
  a <- psf(data = data, cycle = 12)
  b <- predict(object = a, n.ahead = nval)
  return(b)
}
```

Following code chunk show how user can attach various methods in the `prediction_errors()` function. In this chunk, the `append_` parameter is assigned `1`, to appned the new methods (`LPSF` and `PSF`) in addition to the default `ARIMA` method. On contrary, if the `append_`parameter is assigned `0`, only newly added `LPSF` and `PSF` nethods would be compared.

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
a1 <- prediction_errors(data = nottem, nval = 48, 
                        Method = c("test1(data, nval)", "test2(data, nval)"), 
                        MethodName = c("LPSF","PSF"), append_ = 1)
a1@output$Error_Parameters
b1 <- plot(a1)
```
--------
********

### Appending new methods:

Consider, another function `test3()`, which is to be added to an already existing object `prediction_errors`, eg. `a1`.

```{r fig.height = 8, fig.width = 8, fig.align = "center"}
library(forecast)
test3 <- function(data, nval){
  b <- as.numeric(forecast(ets(data), h = nval)$mean)
  return(b)
}
```


For this purpose, the `append_()` function can be used as follows:

The `append_()` function have `object`, `Method`, `MethodName`, `ePara` and `ePara_name` parameters, with similar meaning as that of used in `prediction_errors()` function. Other hidden parameters of the `append_()` function automatically get synced with the `prediction_errors()` function. 

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
c1 <- append_(object = a1, Method = c("test3(data,nval)"), MethodName = c('ETS'))
c1@output$Error_Parameters
d1 <- plot(c1)
```
--------
********

### Removing methods:

When more than one methods are established in the environment and the user wish to remove one or more of these methods from it, the `choose_()` function can be used. This function takes a `prediction_errors` object as input shows all methods established in the environment, and asks the number of methods which the user wants to remove from it.

In the following example, the user supplied `4` as input, which reflects `Method 4: ETS`, and in response to this, the `choose_()` function provides a new object with updated method lists.

```{}
# > e1 <- choose_(object = c1)
# Following are the methods attached with the object:
#         [,1]    [,2]   [,3]  [,4] 
# Indices "1"     "2"    "3"   "4"  
# Methods "ARIMA" "LPSF" "PSF" "ETS"
#
# Enter the indices of methods to remove:4
#
# > e1@output$Error_Parameters
#            RMSE       MAE exec_time
# ARIMA 2.5233156 2.1280641 0.1963789
# LPSF  2.3915796 1.9361111 0.2990961
# PSF   2.2748736 1.8301389 0.1226711
```
--------

### Adding new Error metrics:

In default scenario, the `prediction_errors()` function compares forecasting methods in terms of `RMSE`, `MAE` and `MAPE`. In addition, it allows to append multiple new error metrics. The Percent change in variance (PCV) is an another error metric with following definition:

$PCV = \frac{\mid var(Predicted) - var(Observed) \mid}{var(Observed)}$

where $var(Predicted)$ and $var(Observed)$ are variance of predicted and obvserved values. Following chunk code is the function for PCV error metric:

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
pcv <- function(obs, pred){
  d <- (var(obs) - var(pred)) * 100/ var(obs)
  d <- abs(as.numeric(d))
  return(d)
}
```

Following chunk code is used to append PCV as a new error metric in existing `prediction_errors` object.

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
a1 <- prediction_errors(data = nottem, nval = 48, 
                        Method = c("test1(data, nval)", "test2(data, nval)"), 
                        MethodName = c("LPSF","PSF"), 
                        ePara = "pcv(obs, pred)", ePara_name = 'PCV',
                        append_ = 1)
a1@output$Error_Parameters
b1 <- plot(a1)
```

## A unique plot:

A unique way of showing forecasted values, especially if these are seasonal values, the following function can be used. This plot shows how forecatsed observations are behaving on an increasing number of seasonal time horizons.

```{r fig.height = 6, fig.width = 8, fig.align = "left"}
plot_circle(a1)
```
--------
********

## Monte-Carlo strategy:

Monte-Carlo is a popular strategy to compare the performance of forecasting methods, which selects multiple patches of dataset randomly and test performance of forecasting methods and returns the average error values.

The Monte-Carlo strategy ensures an accurate comparison of forecasting methods and avoids the baised results obtained by chance.

This package provides the `monte_carlo()` function as follows:

The parameters used in this function are:

  * `object`: output of 'prediction_errors()' function
  
  * `size`: volume of time series used in Monte Carlo strategy
  
  * `iteration`: number of iterations models to be applied
  
  * `fval`: a flag to view forecasted values in each iteration (default: 0, don't view values)
  
  * `figs`: a flag to view plots for each iteration (default: 0, don't view plots)
  
This function returns:

  * Error values with provided models in each iteration along with the mean values

```{r fig.height = 7, fig.width = 7, fig.align = "center"}
a1 <- prediction_errors(data = nottem, nval = 48, 
                        Method = c("test1(data, nval)"), 
                        MethodName = c("LPSF"), append_ = 1)
monte_carlo(object = a1, size = 180, iteration = 10)
```
--------
********

When `monte_carlo()` function with `fval` and `figs` ON flags:
```{r fig.height = 7, fig.width = 7, fig.align = "center"}
monte_carlo(object = a1, size = 144, iteration = 2, fval = 1, figs = 1)
```

## Functions in Future Versions:

* `plot.MC()`

* `bollinger_plot()`

* New simulation strategies in `prediction_errors()`