The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

1. Getting started with STEM models

library(Stem)

The model

Stem estimates hierarchical spatio-temporal models of the form

\[ z_t = X_t \beta + A y_t + e_t, \qquad e_t \sim N(0, \Sigma_e), \] \[ y_t = G y_{t-1} + \eta_t, \qquad \eta_t \sim N(0, \Sigma_\eta), \qquad y_0 \sim N(m_0, C_0), \]

for \(t = 1, \dots, T\). The observation vector \(z_t\) collects the measurements at \(d\) spatial locations; \(y_t\) is a \(p\)-dimensional latent temporal process shared by the network and loaded on the locations through the \(d \times p\) matrix \(A\); \(X_t\) holds \(r\) known covariates. The spatial covariance is

\[ \Sigma_e = \sigma^2_\varepsilon I + \sigma^2_\omega C(h; \theta), \qquad C(h; \theta) = \exp(-\theta h), \]

so that \(\sigma^2_\varepsilon\) is a nugget and \(\sigma^2_\omega\) the variance of the spatially correlated component. The parameter vector is \(\phi = (\beta, \sigma^2_\varepsilon, \sigma^2_\omega, \theta, G, \Sigma_\eta, m_0)\), estimated by maximum likelihood through Kalman filtering and smoothing inside an EM algorithm, with \(C_0\) held fixed.

Data

The package ships two datasets. pm10 is the small example of Fasso and Cameletti: 22 stations and 366 days of log PM10, with emissions and altitude as covariates. povalley is a larger, real network of 36 background stations in the Po Valley observed daily over 2019–2023.

data(pm10)

str(pm10, max.level = 1)
#> List of 3
#>  $ coords    : num [1:22, 1:2] 423 470 437 395 444 ...
#>   ..- attr(*, "dimnames")=List of 2
#>  $ covariates: num [1:8052, 1:3] 1 1 1 1 1 1 1 1 1 1 ...
#>   ..- attr(*, "dimnames")=List of 2
#>  $ z         : num [1:366, 1:22] 3.33 2.94 3.4 3.58 3.74 ...
#>   ..- attr(*, "dimnames")=List of 2

coordinates <- pm10$coords * 1000   # from km to meters
covariates <- pm10$covariates
z <- pm10$z

dim(z)            # T x d
#> [1] 366  22
dim(covariates)   # (T * d) x r
#> [1] 8052    3

The covariate matrix stacks the \(T \times r\) block of each station on top of the next: rows \(1, \dots, T\) belong to the first station, rows \(T+1, \dots, 2T\) to the second, and so on. Keeping this convention is essential, because every routine of the package relies on it to slice the data by location.

Building a model object

STEM_Model() assembles the data and the starting values into an object of class STEM_Model. The starting values are supplied through the list phi, which must have exactly the components of the parameter vector.

phi <- list(
  beta        = matrix(c(3.65, 0.046, -0.904), 3, 1),
  sigma2eps   = 0.1,
  sigma2omega = 0.2,
  theta       = 0.01,
  G           = matrix(0.77, 1, 1),
  Sigmaeta    = matrix(0.3, 1, 1),
  m0          = as.matrix(0),
  C0          = as.matrix(1)
)

A <- matrix(1, ncol(z), 1)   # a single latent process, loaded on every station

mod <- STEM_Model(
  z = z,
  covariates = covariates,
  coordinates = coordinates,
  phi = phi,
  A = A
)

class(mod)
#> [1] "STEM_Model"

Estimation

STEM_Estimation() runs the EM algorithm on the state-space form of the model (the Kalman filter and smoother make the E-step), accelerated by default by SQUAREM; STEM_control(algorithm = ) chooses among EM, ECME, SQUAREM and SQUAREM on ECME, which reach the same maximum. The iterations stop when the largest relative change of a parameter or the absolute change of the log-likelihood falls below its tolerance, 1e-3 by default; precision, used below to keep the example short, replaces the first. Progress messages are off by default and can be turned on with verbose = TRUE.

fit <- STEM_Estimation(mod, precision = 0.01, distance = "euclidean")

round(unlist(fit$estimates$phi.hat), 5)
#> sigma2omega       beta1       beta2       beta3           G    Sigmaeta 
#>     0.08451     5.22138     0.00273    -0.92158     0.98609     0.06749 
#>          m0          C0       theta   sigma2eps 
#>    -1.59211     1.00000     0.00002     0.08090

fit$estimates$loglik
#> [1] -3829.113
fit$estimates$convergence.par[c("conv.log", "conv.par", "iterEM")]
#> $conv.log
#> [1] FALSE
#> 
#> $conv.par
#> [1] TRUE
#> 
#> $iterEM
#> [1] 21

Use distance = "geo" when the coordinates are longitude and latitude, as in povalley; the geodesic distance is then computed with geodist.

The smoothed latent process is returned as a ts object:

plot(fit$estimates$y.smoothed, ylab = "smoothed latent state", xlab = "day")

Smoothed latent temporal process

Uncertainty: the parametric bootstrap

The EM algorithm returns point estimates but no standard errors. STEM_Bootstrap() simulates B datasets from the fitted model and re-estimates the parameters on each of them.

boot <- STEM_Bootstrap(fit, B = 100, distance = "euclidean")

npar <- 9
boot.estimates <- t(vapply(boot$boot.output, function(b) {
  p <- b$estimates$phi.hat
  c(p$beta, p$sigma2eps, p$sigma2omega, p$theta, p$G, p$Sigmaeta, p$m0)
}, numeric(npar)))

se <- sqrt(diag(stats::var(stats::na.omit(boot.estimates))))
round(se, 5)

Spatial prediction

STEM_Kriging() predicts the process at unobserved locations, given their coordinates, covariates and loading matrix.

Where to go next

The second vignette, Spatially-clustered STEM models, shows what to do when a single global regression is not plausible over the whole network, and the coefficients themselves should be allowed to vary across spatial regimes:

vignette("SCSTEM", package = "Stem")

A conceptual map of how the functions call one another is the third vignette:

vignette("function-map", package = "Stem")

References

Fasso, A., Cameletti, M. and Nicolis, O. (2007). Air quality monitoring using heterogeneous networks. Environmetrics, 18, 245–264.

Fasso, A. and Cameletti, M. (2010). A unified statistical approach for simulation, modeling, analysis and mapping of environmental data. Simulation, 86, 139–153.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.