The package gmvarkit contains tools to estimate and work with Gaussian Mixture Vector Autoregressive (GMVAR) model (see Kalliovirta et. al. 2016). This vignette does not explain details about the model and it’s assumed that the reader is familiar with the cited article(s).
The GMVAR models in gmvarkit are defined as class gmvar objects, whose can be created with the estimation function fitGMVAR() or the constructor function GMVAR(). The created gmvar objects are then conveniently used as main arguments in many other functions and methods, that allow for example model diagnostics, simulations and forecasting. Therefore after estimating a GMVAR model it’s straight forward and easy to use the other functions for further analyses. However some tasks, such as applying general linear constraints or setting up initial population for the genetic algorithm employed by the estimation function, require accurate understanding on how the parameter vectors are constructed in this package.
The notations for (non-constrained) parameter vector are in line with the cited article by Kalliovirta et. al. (2016), but for clarity we will repeat these notations in the first section. In the second section we show how to apply general linear constraints to the autoregressive parameters of GMVAR model, and we illustrate the general formula with two examples. In the third section we have listed some useful functions and methods found in gmvarkit and briefly explain how they work and how to use them.
Defining a GMVAR model requires specifying the autoregressive degree denoted by symbol p and the number of mixture components M. The number of time series in the system is denoted by d, and it’s assumed d is larger than one.1 The form of the parameter vector depends on whether one is using regular or constrained model.
The paramater vector for regular model is size ((M(pd^2+d(d+1)/2+1)-1)x1) and has form \[\boldsymbol{\theta} = (\boldsymbol{\upsilon_1},...,\boldsymbol{\upsilon_M},\alpha_1,...,\alpha_{M-1}),\enspace \text{where}\quad\quad\enspace\quad\] \[\boldsymbol{\upsilon_m}=(\phi_{m,0},\boldsymbol{\phi_m},\sigma_m)\enspace ((pd^2+d(d+1)/2)x1),\quad\quad\enspace\] \[\boldsymbol{\phi_m}=(vec(A_{m,1}),...,vec(A_{m,p}))\enspace (pd^2x1), \enspace \text{and}\quad\enspace\;\] \[\sigma_m=vech(\Omega_m)\enspace ((d(d+1)/2)x1),\enspace m=1,...,M.\]
Above \(\phi_{m,0}\) denotes the intercept parameter of m:th mixture component, \(A_{m,i}\) denotes the coefficient matrix of m:th mixture component and i:th lag, \(\Omega_m\) denotes the (positive definite) error term covariance matrix and \(\alpha_m\) denotes the mixing weight parameter of m:th mixture component. \(vec()\) is a vectorization operator that stacks columns of a matrix into a vector and \(vech()\) stacks columns of a matrix from the main diagonal downwards (including the main diagonal) into a vector.
The parameter vector above has “intercept” parametrization, referring to the intercept terms \(\phi_{m,0}\). However it’s also possible to use “mean” parametrization, where the intercept terms are simply replaced by the regimewise means \(\mu_m=(I_d-\sum_{i=1}^pA_{m,i})^{-1}\phi_{m,0}\).
Employing linear constraints on the autoregressive parameters of GMVAR model is straigth forward. The constraints are expressed in a very general form allowing any arbitrary linear constraints, but one needs to take the time to construct the contraint matrix carefully for each particular case.
We consider constraints of form \[(\boldsymbol{\phi_1},...,\boldsymbol{\phi_M}) = \boldsymbol{C}\boldsymbol{\psi},\enspace \text{where}\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\enspace\] \[\boldsymbol{\phi_m}=(vec(A_{m,1}),...,vec(A_{m,p}))\enspace (pd^2x1), \enspace m=1,...,M,\] \(\boldsymbol{C}\) is known \((Mpd^2xq)\) constraint matrix (of full column rank) and \(\boldsymbol{\psi}\) is unkown \((qx1)\) parameter vector.
The paramater vector for constrained model is size ((M(d+d(d+1)/2+1)+q-1)x1) and has form \[\boldsymbol{\theta} = (\phi_{1,0},...,\phi_{M,0},\boldsymbol{\psi},\alpha_1,...,\alpha_{M-1}),\] where \(\boldsymbol{\psi}\) is the \((qx1)\) parameter vector containing contrained autoregressive parameters. As in the case of regular models, instead of the intercept parametrization that takes use of intercept terms \(\phi_{m,0}\), one may use the mean parametrization with regimewise means \(\mu_m\) instead (m=1,…,M).
There are two main uses for linear constraints: restricting the autoregressive parameters to be the same for all regimes and constraining some parameters to zero. Of course many other arbitrary constraints may be useful, but we will show illustrative examples of these two.
To restrict the AR-parameters to be the same for all regimes, we want \(\boldsymbol{\phi_m}\) to be the same for all m=1,…,M. The parameter vector \(\boldsymbol{\psi}\) \((qx1)\) then corresponds to any \(\boldsymbol{\phi_m}=\boldsymbol{\phi}\), and therefore \(q=pd^2\). For the constraint matrix we choose \[\boldsymbol{C} = [I_{pd^2}:\cdots:I_{pd^2}]' \enspace (Mpd^2xpd^2),\] that is, M pieces of \((pd^2xpd^2)\)-diagonal matrices stacked on top of each other, because then \[\boldsymbol{C}\boldsymbol{\psi}=(\boldsymbol{\psi},...,\boldsymbol{\psi})=(\boldsymbol{\phi},...,\boldsymbol{\phi}).\]
The previous example shows how to restrict the AR-parameters to be the same for all regimes, but let’s say we also want to constrain the non-diagonal elements of coefficient matrices \(A_{m,i}\) (m=1,…,M, i=1,…,p) to be zero. We have the constrained parameter \(\boldsymbol{\psi}\) \((qx1)\) representing the unconstrained parameters \((\boldsymbol{\phi_1},...,\boldsymbol{\phi_M})\), where by assumption \(\boldsymbol{\phi_m}=\boldsymbol{\phi}=(vec(A_1),...,vec(A_p))\) \((pd^2x1)\) and the elements of \(vec(A_i)\) (i=1,…,p) corresponding to the diagonal are zero.
For illustrative purposes, let’s consider a GMVAR model with autoregressive degree p=2, number of mixture components M=2 and number of time series in the system d=2. Then we have \[\boldsymbol{\phi}=(A_{1,(1,1)},0,0,A_{1,(2,2)},A_{2,(1,1)},0,0,A_{2,(2,2)}) \enspace (8x1) \enspace \text{and} \] \[\boldsymbol{\psi}=(A_{1,(1,1)},A_{1,(2,2)},A_{2,(1,1)},A_{2,(2,2)}) \enspace (4x1).\quad\quad\quad\quad\quad\enspace\] By a direct calculation, we can see that choosing the constraint matrix \[\boldsymbol{C}=\left[{\begin{array}{c} \boldsymbol{\tilde{c}} \\ \boldsymbol{\tilde{c}} \\ \end{array}}\right] \enspace (Mpd^2x4), \enspace \text{where}\]
\[\boldsymbol{\tilde{c}}=\left[{\begin{array}{cccc} 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array}}\right] \enspace (pd^2x4)\] satisfies \(\boldsymbol{C}\boldsymbol{\psi}=(\boldsymbol{\phi},...,\boldsymbol{\phi}).\)
In the package gmvarkit one may use the function fitGMVAR() to estimate a GMVAR model. The maximum likelihood estimation is performed in two phases: in the first phase fitGMVAR() uses a genetic algorithm to find starting values for gradient based variable metric algorithm, which it then uses to finalize the estimation in the second phase. It’s important to keep in mind that it’s not guaranteed, that the numerical estimation algorithms will end up in the global maximum point rather than a local one. Because of multimodality and challenging surface of the log-likelihood function, it’s actually expected that most of the estimation rounds won’t find the global maximum point. For this reason one should always perform multiple estimation rounds. The number of estimation rounds can be controlled with the argument ncalls, but this is also done by default and it takes use of parallel computing (number of cores used can be set with the argument ncores).
If the model estimates poorly, it is often because the number of mixture components M is chosen too large. One may also adjust the settings of the genetic algorithm employed, or set up an initial population with guesses for the estimates. This can by done by passing arguments in fitGMVAR() to the (non-exported) function GAfit() employing the genetic algorithm. To check the available settings, read the documentation ?GAfit. If the iteration limit is reached when estimating the model, the function iterate_more() can be used to finish the estimation.
The parameters of the estimated model are printed in an illustrative and easy to read form. In order to easily compare approximate standard errors to certain estimates, one can print the approximate standard errors of the estimes in the same form with the function print_std_errors(). Numerical approximation of the gradient and Hessian matrix of the log-likelihood at the estimates can be obtained conveniently with the functions get_gradient() and get_hessian().
The estimated objects have their own print, plot and summary methods to name a few.
The package gmvarkit considers model diagnostics based on multivariate extension of quantile residuals (see Kalliovirta and Saikkonen 2010), whose are under the correct model specification asymptotically multivariate standard normal distruted. The quantile residual tests introduced by Kalliovirta and Saikkonen (2010) can be performed with the function quantile_residuals_tests() by providing the estimated model (that is class gmvar object) as an argument.
For graphical diagnostics one may use the function diagostic_plot(), which enables one to plot the quantile residual time series, auto- and cross-correlation functions of quantile residuals or their squares, or quantile residual kernel destity estimates and normal QQ-plots.
One may wish to construct an arbitrary GMVAR model without any estimation process, for example in order to simulate from the particular process of interest. An arbitrary model can be created with the function GMVAR(). If one wants to add data to the model afterwards, it’s advisable to use the function add_data().
The function simulateGMVAR() is the one for the job. As the main argument it uses a gmvar-object generated by fitGMVAR() or GMVAR().
The package gmvarkit contains predict method predict.gmvar() for forecasting GMVAR processes. For one step predictions using the exact formula for conditional mean is supported, but the forecasts further than that are based on independent simulations. The predictions are either sample means or medians and the confidence intervals are based on sample quantiles. The objects generated by predict.gmvar() have their own plot method.
For univariate analysis, one may use the package uGMAR.↩