The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

A-quick-tour-of-StMoE

Introduction

StMoE (Skew-t Mixture-of-Experts) provides a flexible and robust modelling framework for heterogenous data with possibly skewed, heavy-tailed distributions and corrupted by atypical observations. StMoE consists of a mixture of K skew-t expert regressors network (of degree p) gated by a softmax gating network (of degree q) and is represented by:

Model estimation/learning is performed by a dedicated expectation conditional maximization (ECM) algorithm by maximizing the observed data log-likelihood. We provide simulated examples to illustrate the use of the model in model-based clustering of heterogeneous regression data and in fitting non-linear regression functions.

It was written in R Markdown, using the knitr package for production.

See help(package="meteorits") for further details and references provided by citation("meteorits").

Application to a simulated dataset

Generate sample

n <- 500 # Size of the sample
alphak <- matrix(c(0, 8), ncol = 1) # Parameters of the gating network
betak <- matrix(c(0, -2.5, 0, 2.5), ncol = 2) # Regression coefficients of the experts
sigmak <- c(0.5, 0.5) # Standard deviations of the experts
lambdak <- c(3, 5) # Skewness parameters of the experts
nuk <- c(5, 7) # Degrees of freedom of the experts network t densities
x <- seq.int(from = -1, to = 1, length.out = n) # Inputs (predictors)

# Generate sample of size n
sample <- sampleUnivStMoE(alphak = alphak, betak = betak, sigmak = sigmak, 
                          lambdak = lambdak, nuk = nuk, x = x)
y <- sample$y

Set up StMoE model parameters

K <- 2 # Number of regressors/experts
p <- 1 # Order of the polynomial regression (regressors/experts)
q <- 1 # Order of the logistic regression (gating network)

Set up EM parameters

n_tries <- 1
max_iter <- 1500
threshold <- 1e-5
verbose <- TRUE
verbose_IRLS <- FALSE

Estimation

stmoe <- emStMoE(X = x, Y = y, K, p, q, n_tries, max_iter, 
                 threshold, verbose, verbose_IRLS)
## EM - StMoE: Iteration: 1 | log-likelihood: -358.341886247016
## EM - StMoE: Iteration: 2 | log-likelihood: -322.659951557355
## EM - StMoE: Iteration: 3 | log-likelihood: -320.693470431902
## EM - StMoE: Iteration: 4 | log-likelihood: -318.61107077321
## EM - StMoE: Iteration: 5 | log-likelihood: -316.212273528493
## EM - StMoE: Iteration: 6 | log-likelihood: -313.383764444746
## EM - StMoE: Iteration: 7 | log-likelihood: -310.065053962983
## EM - StMoE: Iteration: 8 | log-likelihood: -306.224283800106
## EM - StMoE: Iteration: 9 | log-likelihood: -301.880368911868
## EM - StMoE: Iteration: 10 | log-likelihood: -297.136415870862
## EM - StMoE: Iteration: 11 | log-likelihood: -292.198038917438
## EM - StMoE: Iteration: 12 | log-likelihood: -287.324695300884
## EM - StMoE: Iteration: 13 | log-likelihood: -282.746661707539
## EM - StMoE: Iteration: 14 | log-likelihood: -278.614147365268
## EM - StMoE: Iteration: 15 | log-likelihood: -274.996730286112
## EM - StMoE: Iteration: 16 | log-likelihood: -271.895041279085
## EM - StMoE: Iteration: 17 | log-likelihood: -269.280635577131
## EM - StMoE: Iteration: 18 | log-likelihood: -267.09149388565
## EM - StMoE: Iteration: 19 | log-likelihood: -265.271026320903
## EM - StMoE: Iteration: 20 | log-likelihood: -263.759092494974
## EM - StMoE: Iteration: 21 | log-likelihood: -262.502946815773
## EM - StMoE: Iteration: 22 | log-likelihood: -261.454038406018
## EM - StMoE: Iteration: 23 | log-likelihood: -260.574638252184
## EM - StMoE: Iteration: 24 | log-likelihood: -259.832146800618
## EM - StMoE: Iteration: 25 | log-likelihood: -259.201264232419
## EM - StMoE: Iteration: 26 | log-likelihood: -258.66209266237
## EM - StMoE: Iteration: 27 | log-likelihood: -258.200304380158
## EM - StMoE: Iteration: 28 | log-likelihood: -257.803020700055
## EM - StMoE: Iteration: 29 | log-likelihood: -257.459647457927
## EM - StMoE: Iteration: 30 | log-likelihood: -257.161109980419
## EM - StMoE: Iteration: 31 | log-likelihood: -256.902374475734
## EM - StMoE: Iteration: 32 | log-likelihood: -256.677764013955
## EM - StMoE: Iteration: 33 | log-likelihood: -256.481647583747
## EM - StMoE: Iteration: 34 | log-likelihood: -256.309532304488
## EM - StMoE: Iteration: 35 | log-likelihood: -256.157703550254
## EM - StMoE: Iteration: 36 | log-likelihood: -256.023316001022
## EM - StMoE: Iteration: 37 | log-likelihood: -255.903871921378
## EM - StMoE: Iteration: 38 | log-likelihood: -255.797646853338
## EM - StMoE: Iteration: 39 | log-likelihood: -255.702858065201
## EM - StMoE: Iteration: 40 | log-likelihood: -255.618465633147
## EM - StMoE: Iteration: 41 | log-likelihood: -255.543420022665
## EM - StMoE: Iteration: 42 | log-likelihood: -255.476395670569
## EM - StMoE: Iteration: 43 | log-likelihood: -255.416291553121
## EM - StMoE: Iteration: 44 | log-likelihood: -255.362320737453
## EM - StMoE: Iteration: 45 | log-likelihood: -255.313803352974
## EM - StMoE: Iteration: 46 | log-likelihood: -255.270074418416
## EM - StMoE: Iteration: 47 | log-likelihood: -255.230463560675
## EM - StMoE: Iteration: 48 | log-likelihood: -255.194449430696
## EM - StMoE: Iteration: 49 | log-likelihood: -255.161613595902
## EM - StMoE: Iteration: 50 | log-likelihood: -255.131581608645
## EM - StMoE: Iteration: 51 | log-likelihood: -255.104031659776
## EM - StMoE: Iteration: 52 | log-likelihood: -255.078686030772
## EM - StMoE: Iteration: 53 | log-likelihood: -255.055302250778
## EM - StMoE: Iteration: 54 | log-likelihood: -255.033928970051
## EM - StMoE: Iteration: 55 | log-likelihood: -255.01434724664
## EM - StMoE: Iteration: 56 | log-likelihood: -254.996347083498
## EM - StMoE: Iteration: 57 | log-likelihood: -254.979838144307
## EM - StMoE: Iteration: 58 | log-likelihood: -254.96469757084
## EM - StMoE: Iteration: 59 | log-likelihood: -254.950764911511
## EM - StMoE: Iteration: 60 | log-likelihood: -254.937902022156
## EM - StMoE: Iteration: 61 | log-likelihood: -254.925987986507
## EM - StMoE: Iteration: 62 | log-likelihood: -254.914882798003
## EM - StMoE: Iteration: 63 | log-likelihood: -254.904535254373
## EM - StMoE: Iteration: 64 | log-likelihood: -254.894864826624
## EM - StMoE: Iteration: 65 | log-likelihood: -254.885800788666
## EM - StMoE: Iteration: 66 | log-likelihood: -254.877280881681
## EM - StMoE: Iteration: 67 | log-likelihood: -254.869245760273
## EM - StMoE: Iteration: 68 | log-likelihood: -254.861646904019
## EM - StMoE: Iteration: 69 | log-likelihood: -254.854441631407
## EM - StMoE: Iteration: 70 | log-likelihood: -254.847592025442
## EM - StMoE: Iteration: 71 | log-likelihood: -254.841064308203
## EM - StMoE: Iteration: 72 | log-likelihood: -254.834828294085
## EM - StMoE: Iteration: 73 | log-likelihood: -254.828856906542
## EM - StMoE: Iteration: 74 | log-likelihood: -254.823125745702
## EM - StMoE: Iteration: 75 | log-likelihood: -254.81761269621
## EM - StMoE: Iteration: 76 | log-likelihood: -254.812297566296
## EM - StMoE: Iteration: 77 | log-likelihood: -254.807161750446
## EM - StMoE: Iteration: 78 | log-likelihood: -254.80218785685
## EM - StMoE: Iteration: 79 | log-likelihood: -254.797403497463
## EM - StMoE: Iteration: 80 | log-likelihood: -254.792855394281
## EM - StMoE: Iteration: 81 | log-likelihood: -254.788511319515
## EM - StMoE: Iteration: 82 | log-likelihood: -254.784341459075
## EM - StMoE: Iteration: 83 | log-likelihood: -254.780317790363
## EM - StMoE: Iteration: 84 | log-likelihood: -254.77641350682
## EM - StMoE: Iteration: 85 | log-likelihood: -254.772602954156
## EM - StMoE: Iteration: 86 | log-likelihood: -254.768863405824
## EM - StMoE: Iteration: 87 | log-likelihood: -254.76517125295
## EM - StMoE: Iteration: 88 | log-likelihood: -254.761506012367
## EM - StMoE: Iteration: 89 | log-likelihood: -254.757837544911
## EM - StMoE: Iteration: 90 | log-likelihood: -254.754161748511
## EM - StMoE: Iteration: 91 | log-likelihood: -254.750435218567
## EM - StMoE: Iteration: 92 | log-likelihood: -254.746629465947
## EM - StMoE: Iteration: 93 | log-likelihood: -254.742714669617
## EM - StMoE: Iteration: 94 | log-likelihood: -254.738659578476
## EM - StMoE: Iteration: 95 | log-likelihood: -254.734490817481
## EM - StMoE: Iteration: 96 | log-likelihood: -254.730192157155
## EM - StMoE: Iteration: 97 | log-likelihood: -254.725719853109
## EM - StMoE: Iteration: 98 | log-likelihood: -254.721031436972
## EM - StMoE: Iteration: 99 | log-likelihood: -254.716086365243
## EM - StMoE: Iteration: 100 | log-likelihood: -254.710846824594
## EM - StMoE: Iteration: 101 | log-likelihood: -254.705278656842
## EM - StMoE: Iteration: 102 | log-likelihood: -254.699352342126
## EM - StMoE: Iteration: 103 | log-likelihood: -254.693043955812
## EM - StMoE: Iteration: 104 | log-likelihood: -254.686335997371
## EM - StMoE: Iteration: 105 | log-likelihood: -254.679217983225
## EM - StMoE: Iteration: 106 | log-likelihood: -254.671686704349
## EM - StMoE: Iteration: 107 | log-likelihood: -254.663746075242
## EM - StMoE: Iteration: 108 | log-likelihood: -254.655406541352
## EM - StMoE: Iteration: 109 | log-likelihood: -254.646684061509
## EM - StMoE: Iteration: 110 | log-likelihood: -254.637598732378
## EM - StMoE: Iteration: 111 | log-likelihood: -254.628173166331
## EM - StMoE: Iteration: 112 | log-likelihood: -254.618430731211
## EM - StMoE: Iteration: 113 | log-likelihood: -254.608428384836
## EM - StMoE: Iteration: 114 | log-likelihood: -254.598186505404
## EM - StMoE: Iteration: 115 | log-likelihood: -254.587718089114
## EM - StMoE: Iteration: 116 | log-likelihood: -254.577032799573
## EM - StMoE: Iteration: 117 | log-likelihood: -254.566137142869
## EM - StMoE: Iteration: 118 | log-likelihood: -254.555035337377
## EM - StMoE: Iteration: 119 | log-likelihood: -254.543730791611
## EM - StMoE: Iteration: 120 | log-likelihood: -254.532228075951
## EM - StMoE: Iteration: 121 | log-likelihood: -254.520535176582
## EM - StMoE: Iteration: 122 | log-likelihood: -254.508665705371
## EM - StMoE: Iteration: 123 | log-likelihood: -254.496640769654
## EM - StMoE: Iteration: 124 | log-likelihood: -254.484490215664
## EM - StMoE: Iteration: 125 | log-likelihood: -254.472253026858
## EM - StMoE: Iteration: 126 | log-likelihood: -254.45997675758
## EM - StMoE: Iteration: 127 | log-likelihood: -254.447715961625
## EM - StMoE: Iteration: 128 | log-likelihood: -254.435565924281
## EM - StMoE: Iteration: 129 | log-likelihood: -254.423607788127
## EM - StMoE: Iteration: 130 | log-likelihood: -254.411895190203
## EM - StMoE: Iteration: 131 | log-likelihood: -254.400476823631
## EM - StMoE: Iteration: 132 | log-likelihood: -254.389393202635
## EM - StMoE: Iteration: 133 | log-likelihood: -254.37867404923
## EM - StMoE: Iteration: 134 | log-likelihood: -254.368336569733
## EM - StMoE: Iteration: 135 | log-likelihood: -254.358385555638
## EM - StMoE: Iteration: 136 | log-likelihood: -254.348812910957
## EM - StMoE: Iteration: 137 | log-likelihood: -254.339597972166
## EM - StMoE: Iteration: 138 | log-likelihood: -254.330710646008
## EM - StMoE: Iteration: 139 | log-likelihood: -254.322113858071
## EM - StMoE: Iteration: 140 | log-likelihood: -254.313766153391
## EM - StMoE: Iteration: 141 | log-likelihood: -254.305624190977
## EM - StMoE: Iteration: 142 | log-likelihood: -254.297627316357
## EM - StMoE: Iteration: 143 | log-likelihood: -254.289754304995
## EM - StMoE: Iteration: 144 | log-likelihood: -254.281967130667
## EM - StMoE: Iteration: 145 | log-likelihood: -254.274231823909
## EM - StMoE: Iteration: 146 | log-likelihood: -254.266518680729
## EM - StMoE: Iteration: 147 | log-likelihood: -254.258802087261
## EM - StMoE: Iteration: 148 | log-likelihood: -254.251060056927
## EM - StMoE: Iteration: 149 | log-likelihood: -254.243273588205
## EM - StMoE: Iteration: 150 | log-likelihood: -254.235425944923
## EM - StMoE: Iteration: 151 | log-likelihood: -254.227501935458
## EM - StMoE: Iteration: 152 | log-likelihood: -254.219487223213
## EM - StMoE: Iteration: 153 | log-likelihood: -254.211367644405
## EM - StMoE: Iteration: 154 | log-likelihood: -254.203128451916
## EM - StMoE: Iteration: 155 | log-likelihood: -254.1947533597
## EM - StMoE: Iteration: 156 | log-likelihood: -254.18622324363
## EM - StMoE: Iteration: 157 | log-likelihood: -254.177514369866
## EM - StMoE: Iteration: 158 | log-likelihood: -254.168596074928
## EM - StMoE: Iteration: 159 | log-likelihood: -254.159427917496
## EM - StMoE: Iteration: 160 | log-likelihood: -254.149956481982
## EM - StMoE: Iteration: 161 | log-likelihood: -254.140112305156
## EM - StMoE: Iteration: 162 | log-likelihood: -254.129807983281
## EM - StMoE: Iteration: 163 | log-likelihood: -254.118939720844
## EM - StMoE: Iteration: 164 | log-likelihood: -254.107396862317
## EM - StMoE: Iteration: 165 | log-likelihood: -254.095087362962
## EM - StMoE: Iteration: 166 | log-likelihood: -254.08198912314
## EM - StMoE: Iteration: 167 | log-likelihood: -254.068227669384
## EM - StMoE: Iteration: 168 | log-likelihood: -254.054144889453
## EM - StMoE: Iteration: 169 | log-likelihood: -254.040276614933
## EM - StMoE: Iteration: 170 | log-likelihood: -254.027150867197
## EM - StMoE: Iteration: 171 | log-likelihood: -254.015516270311
## EM - StMoE: Iteration: 172 | log-likelihood: -254.005428715919
## EM - StMoE: Iteration: 173 | log-likelihood: -253.996597298353
## EM - StMoE: Iteration: 174 | log-likelihood: -253.988680128159
## EM - StMoE: Iteration: 175 | log-likelihood: -253.982000739415
## EM - StMoE: Iteration: 176 | log-likelihood: -253.976458951876
## EM - StMoE: Iteration: 177 | log-likelihood: -253.971941901404
## EM - StMoE: Iteration: 178 | log-likelihood: -253.968330643806
## EM - StMoE: Iteration: 179 | log-likelihood: -253.965506908767
## EM - StMoE: Iteration: 180 | log-likelihood: -253.963358402817

Summary

stmoe$summary()
## ------------------------------------------
## Fitted Skew t Mixture-of-Experts model
## ------------------------------------------
## 
## StMoE model with K = 2 experts:
## 
##  log-likelihood df       AIC      BIC      ICL
##       -253.9634 12 -265.9634 -291.251 -291.283
## 
## Clustering table (Number of observations in each expert):
## 
##   1   2 
## 254 246 
## 
## Regression coefficients:
## 
##     Beta(k = 1) Beta(k = 2)
## 1   -0.00186094 -0.08398201
## X^1  2.57239459 -2.55715595
## 
## Variances:
## 
##  Sigma2(k = 1) Sigma2(k = 2)
##      0.4328539     0.6125303

Plots

Mean curve

stmoe$plot(what = "meancurve")

Confidence regions

stmoe$plot(what = "confregions")

Clusters

stmoe$plot(what = "clusters")

Log-likelihood

stmoe$plot(what = "loglikelihood")

Application to real dataset

Load data

library(MASS)
data("mcycle")
x <- mcycle$times
y <- mcycle$accel

Set up StMoE model parameters

K <- 4 # Number of regressors/experts
p <- 2 # Order of the polynomial regression (regressors/experts)
q <- 1 # Order of the logistic regression (gating network)

Set up EM parameters

n_tries <- 1
max_iter <- 1500
threshold <- 1e-5
verbose <- TRUE
verbose_IRLS <- FALSE

Estimation

stmoe <- emStMoE(X = x, Y = y, K, p, q, n_tries, max_iter, 
                 threshold, verbose, verbose_IRLS)
## EM - StMoE: Iteration: 1 | log-likelihood: -598.565702033975
## EM - StMoE: Iteration: 2 | log-likelihood: -586.594322769658
## EM - StMoE: Iteration: 3 | log-likelihood: -585.173591847768
## EM - StMoE: Iteration: 4 | log-likelihood: -582.369299935532
## EM - StMoE: Iteration: 5 | log-likelihood: -575.243758066658
## EM - StMoE: Iteration: 6 | log-likelihood: -569.121821835036
## EM - StMoE: Iteration: 7 | log-likelihood: -565.085413323596
## EM - StMoE: Iteration: 8 | log-likelihood: -563.483599829053
## EM - StMoE: Iteration: 9 | log-likelihood: -563.129391506102
## EM - StMoE: Iteration: 10 | log-likelihood: -563.019040796945
## EM - StMoE: Iteration: 11 | log-likelihood: -562.912050661351
## EM - StMoE: Iteration: 12 | log-likelihood: -562.762681987713
## EM - StMoE: Iteration: 13 | log-likelihood: -562.580039526022
## EM - StMoE: Iteration: 14 | log-likelihood: -562.411088197371
## EM - StMoE: Iteration: 15 | log-likelihood: -562.302582513181
## EM - StMoE: Iteration: 16 | log-likelihood: -562.247754786568
## EM - StMoE: Iteration: 17 | log-likelihood: -562.219882633222
## EM - StMoE: Iteration: 18 | log-likelihood: -562.201761352874
## EM - StMoE: Iteration: 19 | log-likelihood: -562.186310255718
## EM - StMoE: Iteration: 20 | log-likelihood: -562.17132753963
## EM - StMoE: Iteration: 21 | log-likelihood: -562.156107460835
## EM - StMoE: Iteration: 22 | log-likelihood: -562.140269866389
## EM - StMoE: Iteration: 23 | log-likelihood: -562.123772098625
## EM - StMoE: Iteration: 24 | log-likelihood: -562.106387456376
## EM - StMoE: Iteration: 25 | log-likelihood: -562.088214763658
## EM - StMoE: Iteration: 26 | log-likelihood: -562.069235287434
## EM - StMoE: Iteration: 27 | log-likelihood: -562.049326951292
## EM - StMoE: Iteration: 28 | log-likelihood: -562.028463524128
## EM - StMoE: Iteration: 29 | log-likelihood: -562.006369869854
## EM - StMoE: Iteration: 30 | log-likelihood: -561.983242167128
## EM - StMoE: Iteration: 31 | log-likelihood: -561.958818064529
## EM - StMoE: Iteration: 32 | log-likelihood: -561.932819250104
## EM - StMoE: Iteration: 33 | log-likelihood: -561.905285699747
## EM - StMoE: Iteration: 34 | log-likelihood: -561.876076243754
## EM - StMoE: Iteration: 35 | log-likelihood: -561.844868244808
## EM - StMoE: Iteration: 36 | log-likelihood: -561.811518798558
## EM - StMoE: Iteration: 37 | log-likelihood: -561.775763365654
## EM - StMoE: Iteration: 38 | log-likelihood: -561.737219240371
## EM - StMoE: Iteration: 39 | log-likelihood: -561.69570438925
## EM - StMoE: Iteration: 40 | log-likelihood: -561.650767022315
## EM - StMoE: Iteration: 41 | log-likelihood: -561.601875555183
## EM - StMoE: Iteration: 42 | log-likelihood: -561.548648250004
## EM - StMoE: Iteration: 43 | log-likelihood: -561.490390688554
## EM - StMoE: Iteration: 44 | log-likelihood: -561.426202767578
## EM - StMoE: Iteration: 45 | log-likelihood: -561.355051717415
## EM - StMoE: Iteration: 46 | log-likelihood: -561.275558122617
## EM - StMoE: Iteration: 47 | log-likelihood: -561.186007492833
## EM - StMoE: Iteration: 48 | log-likelihood: -561.084125214445
## EM - StMoE: Iteration: 49 | log-likelihood: -560.967134271983
## EM - StMoE: Iteration: 50 | log-likelihood: -560.83187601741
## EM - StMoE: Iteration: 51 | log-likelihood: -560.673877228403
## EM - StMoE: Iteration: 52 | log-likelihood: -560.489466300177
## EM - StMoE: Iteration: 53 | log-likelihood: -560.281949348375
## EM - StMoE: Iteration: 54 | log-likelihood: -560.073323321802
## EM - StMoE: Iteration: 55 | log-likelihood: -559.899889267887
## EM - StMoE: Iteration: 56 | log-likelihood: -559.773679074186
## EM - StMoE: Iteration: 57 | log-likelihood: -559.67349172942
## EM - StMoE: Iteration: 58 | log-likelihood: -559.584658109678
## EM - StMoE: Iteration: 59 | log-likelihood: -559.510562092578
## EM - StMoE: Iteration: 60 | log-likelihood: -559.462898651175
## EM - StMoE: Iteration: 61 | log-likelihood: -559.442679098028
## EM - StMoE: Iteration: 62 | log-likelihood: -559.436665069673
## EM - StMoE: Iteration: 63 | log-likelihood: -559.435030878558

Summary

stmoe$summary()
## ------------------------------------------
## Fitted Skew t Mixture-of-Experts model
## ------------------------------------------
## 
## StMoE model with K = 4 experts:
## 
##  log-likelihood df      AIC       BIC       ICL
##        -559.435 30 -589.435 -632.7903 -632.7874
## 
## Clustering table (Number of observations in each expert):
## 
##  1  2  3  4 
## 28 37 31 37 
## 
## Regression coefficients:
## 
##     Beta(k = 1) Beta(k = 2)  Beta(k = 3) Beta(k = 4)
## 1    -3.6985620   992.62382 -1889.618688 292.2075328
## X^1   0.9395230  -103.99905   117.331380 -12.1454859
## X^2  -0.0859998     2.43552    -1.775234   0.1246457
## 
## Variances:
## 
##  Sigma2(k = 1) Sigma2(k = 2) Sigma2(k = 3) Sigma2(k = 4)
##       15.06473      453.8693      491.3541      575.0638

Plots

Mean curve

stmoe$plot(what = "meancurve")

Confidence regions

stmoe$plot(what = "confregions")

Clusters

stmoe$plot(what = "clusters")

Log-likelihood

stmoe$plot(what = "loglikelihood")

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.