Semi-Confirmatory Structural Equation Modeling

Po-Hsien Huang

In this example, we will show how to use lslx to conduct semi-confirmatory structural equation modeling. The example uses data PoliticalDemocracy in the package lavaan. Hence, lavaan must be installed.

Model Sepcification

In the following specification, x1 - x3 and y1 - y8 is assumed to be measurements of 3 latent factors: ind60, dem60, and dem65.

model <-
'
fix(1) * x1 + x2 + x3      <=: ind60
fix(1) * y1 + y2 + y3 + y4 <=: dem60
fix(1) * y5 + y6 + y7 + y8 <=: dem65
dem60 <= ind60
dem65 <= ind60 + dem60
'

The operator <=: means that the RHS latent factors is defined by the LHS observed variables. In particular, the loadings are freely estimated. In this model, ind60 is measured by x1 - x3, dem60 is mainly measured by y1 - y4, and dem65 is mainly measured by y5 - y8. The operator <= means that the regression coefficients from the RHS variables to the LHS variables are freely estimated. In this model, dem60 is influenced by ind60 and dem65 is influenced by dem60 and ind60. Details of model syntax can be found in the section of Model Syntax via ?lslx.

Object Initialization

lslx is written as an R6 class. Everytime we conduct analysis with lslx, an lslx object must be initialized. The following code initializes an lslx object named r6_lslx.

library(lslx)
r6_lslx <- lslx$new(model = model,
                    sample_cov = cov(lavaan::PoliticalDemocracy),
                    sample_size = nrow(lavaan::PoliticalDemocracy))
NOTE: Because argument 'sample_cov' doesn't contain group name(s), default group name(s) is created.
NOTE: Because argument 'sample_mean' is missing, default 'sample_mean' is created.
An 'lslx' R6 class is initialized via 'sample_cov'. 
  Response Variable(s): x1 x2 x3 y1 y2 y3 y4 y5 y6 y7 y8 
  Latent Factor(s): ind60 dem60 dem65 

Here, lslx is the object generator for lslx object and new is the build-in method of lslx to generate a new lslx object. The initialization of lslx requires users to specify a model for model specification (argument model) and a sample moments to be fitted (argument sample_cov and sample_size). The sample moment must contains all the observed variables specified in the given model.

Model Respecification

After an lslx object is initialized, model can be respecified by free_coefficent, fix_coefficent, and penalize_coefficent methods. The following code sets y1<->y5, y2<->y4, y2<->y6, y3<->y7, y4<->y8, and y6<->y8 as penalized parameters.

r6_lslx$penalize_coefficient(name = c("y1<->y5",
                                      "y2<->y4",
                                      "y2<->y6",
                                      "y3<->y7",
                                      "y4<->y8",
                                      "y6<->y8"))
The relation y5<->y1 under G is set as PENALIZED with starting value = 0. 
The relation y4<->y2 under G is set as PENALIZED with starting value = 0. 
The relation y6<->y2 under G is set as PENALIZED with starting value = 0. 
The relation y7<->y3 under G is set as PENALIZED with starting value = 0. 
The relation y8<->y4 under G is set as PENALIZED with starting value = 0. 
The relation y8<->y6 under G is set as PENALIZED with starting value = 0. 

To see more methods for respecifying model, please check the section of Set-Related Method via ?lslx.

Model Fitting

After an lslx object is initialized, method fit_mcp can be used to fit the specified model into the given data with mcp penalty funtion.

r6_lslx$fit_mcp(lambda_grid = seq(.02, .30, .02),
                 delta_grid = Inf)
CONGRATS: The optimization algorithm converged under all specified penalty levels. 
  Specified Tolerance for Convergence: 0.001 
  Specified Maximal Number of Iterations: 100 

The fit_mcp requires users to specify the considerd penalty levels (argument lambda_grid and delta_grid). In this example, the lambda grid is seq(.02, .30, .02) and the delta grid is Inf. Note that in this example delta = Inf makes mcp to be equivalent to the lasso panalty. All the fitting result will be stored in the fitting field of r6_lslx.

Model Summarizing

Unlike traditional SEM analysis, lslx fit the model into data under all the penalty levels considered. To summarize the fitting result, a selector to determine an optimal penalty level must be specified. Availble selectors can be found in the section of Penalty Level Selection via ?lslx. The following code summarize the fitting result under the penalty level selected by Akaike information criterion (AIC).

r6_lslx$summarize(selector = "aic")
General Information                                                                                   
   number of observation                                                         75
   number of complete observation                                                75
   number of missing pattern                                                   none
   number of group                                                                1
   number of response                                                            11
   number of factor                                                               3
   number of free coefficient                                                    36
   number of penalized coefficient                                                6

Fitting Information                                                                                   
   penalty method                                                               mcp
   lambda grid                                                           0.02 - 0.3
   delta grid                                                                   Inf
   algorithm                                                                 fisher
   missing method                                                              none
   tolerance for convergence                                                  0.001

Saturated Model Information                                                                                   
   loss value                                                                 0.000
   number of non-zero coefficient                                            77.000
   degree of freedom                                                          0.000

Baseline Model Information                                                                                   
   loss value                                                                 9.739
   number of non-zero coefficient                                            22.000
   degree of freedom                                                         55.000

Numerical Condition                                                                                   
   lambda                                                                     0.020
   delta                                                                        Inf
   objective value                                                            0.630
   objective gradient absolute maximum                                        0.001
   objective hessian convexity                                                0.029
   number of iteration                                                        3.000
   loss value                                                                 0.519
   number of non-zero coefficient                                            42.000
   degree of freedom                                                         35.000
   robust degree of freedom                                                     NaN
   scaling factor                                                               NaN

Information Criteria                                                                                   
   Akaike information criterion (aic)                                        -0.415
   Akaike information criterion with penalty being 3 (aic3)                  -0.881
   consistent Akaike information criterion (caic)                            -1.963
   Bayesian information criterion (bic)                                      -1.496
   adjusted Bayesian information criterion (abic)                            -0.026
   Haughton Bayesian information criterion (hbic)                            -0.639
   robust Akaike information criterion (raic)                                   NaN
   robust Akaike information criterion with penalty being 3 (raic3)             NaN
   robust consistent Akaike information criterion (rcaic)                       NaN
   robust Bayesian information criterion (rbic)                                 NaN
   robust adjusted Bayesian information criterion (rabic)                       NaN
   robust Haughton Bayesian information criterion (rhbic)                       NaN

Fit Indices                                                                                   
   root mean square error of approximation (rmsea)                            0.038
   comparative fit indice (cfi)                                               0.994
   non-normed fit indice (nnfi)                                               0.991
   standardized root mean of residual (srmr)                                  0.046

Likelihood Ratio Test
                    statistic         df    p-value
   unadjusted          38.888     35.000      0.299
   mean-adjusted          -          -          -  

Root Mean Square Error of Approximation Test
                     estimate      lower      upper
   unadjusted           0.038      0.000      0.103
   mean-adjusted          -          -          -  

Coefficient Test (Standard Error = "observed_fisher", Alpha Level = 0.05)
  Factor Loading
                  type  estimate  std.error  z-value  p-value  lower  upper
    x1<-ind60    fixed     1.000        -        -        -      -      -  
    x2<-ind60     free     2.181      0.139   15.664    0.000  1.908  2.453
    x3<-ind60     free     1.819      0.152   11.940    0.000  1.520  2.117
    y1<-dem60    fixed     1.000        -        -        -      -      -  
    y2<-dem60     free     1.279      0.182    7.010    0.000  0.921  1.636
    y3<-dem60     free     1.061      0.148    7.172    0.000  0.771  1.350
    y4<-dem60     free     1.278      0.149    8.575    0.000  0.986  1.570
    y5<-dem65    fixed     1.000        -        -        -      -      -  
    y6<-dem65     free     1.198      0.169    7.074    0.000  0.866  1.530
    y7<-dem65     free     1.285      0.159    8.061    0.000  0.973  1.597
    y8<-dem65     free     1.277      0.162    7.884    0.000  0.960  1.595

  Regression
                  type  estimate  std.error  z-value  p-value  lower  upper
 dem60<-ind60     free     1.477      0.395    3.737    0.000  0.702  2.251
 dem65<-ind60     free     0.558      0.230    2.426    0.008  0.107  1.009
 dem65<-dem60     free     0.841      0.099    8.515    0.000  0.647  1.034

  Covariance
                  type  estimate  std.error  z-value  p-value  lower  upper
      y5<->y1      pen     0.593      0.354    1.675    0.047 -0.101  1.286
      y4<->y2      pen     1.054      0.644    1.637    0.051 -0.208  2.317
      y6<->y2      pen     1.855      0.660    2.809    0.002  0.561  3.149
      y7<->y3      pen     0.613      0.599    1.024    0.153 -0.560  1.786
      y8<->y4      pen     0.274      0.437    0.626    0.266 -0.583  1.130
      y8<->y6      pen     1.205      0.532    2.265    0.012  0.162  2.248

  Variance
                  type  estimate  std.error  z-value  p-value  lower  upper
ind60<->ind60     free     0.454      0.088    5.167    0.000  0.282  0.627
dem60<->dem60     free     3.983      0.946    4.212    0.000  2.129  5.836
dem65<->dem65     free     0.168      0.218    0.772    0.220 -0.259  0.594
      x1<->x1     free     0.083      0.020    4.139    0.000  0.044  0.122
      x2<->x2     free     0.121      0.071    1.709    0.044 -0.018  0.261
      x3<->x3     free     0.473      0.090    5.232    0.000  0.296  0.651
      y1<->y1     free     1.922      0.452    4.256    0.000  1.037  2.808
      y2<->y2     free     6.944      1.220    5.692    0.000  4.553  9.336
      y3<->y3     free     5.091      0.946    5.381    0.000  3.237  6.946
      y4<->y4     free     3.020      0.707    4.273    0.000  1.635  4.406
      y5<->y5     free     2.372      0.476    4.983    0.000  1.439  3.306
      y6<->y6     free     4.714      0.821    5.745    0.000  3.106  6.322
      y7<->y7     free     3.412      0.704    4.849    0.000  2.033  4.791
      y8<->y8     free     3.143      0.664    4.736    0.000  1.842  4.444

  Intercept
                  type  estimate  std.error  z-value  p-value  lower  upper
        x1<-1     free     0.000      0.085    0.000    0.500 -0.166  0.166
        x2<-1     free     0.000      0.174    0.000    0.500 -0.342  0.342
        x3<-1     free     0.000      0.162    0.000    0.500 -0.318  0.318
        y1<-1     free     0.000      0.303    0.000    0.500 -0.594  0.594
        y2<-1     free     0.000      0.448    0.000    0.500 -0.879  0.879
        y3<-1     free     0.000      0.377    0.000    0.500 -0.740  0.740
        y4<-1     free     0.000      0.385    0.000    0.500 -0.755  0.755
        y5<-1     free     0.000      0.302    0.000    0.500 -0.591  0.591
        y6<-1     free     0.000      0.385    0.000    0.500 -0.754  0.754
        y7<-1     free     0.000      0.379    0.000    0.500 -0.743  0.743
        y8<-1     free     0.000      0.373    0.000    0.500 -0.730  0.730

In this example, we can see that the PL estimate under the selected penalty level doesn’t contain any zero value, which indicates that all of the covariance of measurements are relevant. The summarize method also shows the result of significance tests for the coefficients. In lslx, the default standard errors are calculated based on sandwich formula whenever raw data is available. In this example, because raw data is not used for lslx object initialization, standarrd error is calculated by using observed Fisher information matrix. It may not be valid when the model is misspecified and the data are not normal. Also, it is generally invalid after choosing a penalty level.

Objects Extraction

In lslx, many quantities related to SEM can be extracted by extract-related method. For example, the coefficient estimate and its asymptotic variance can be obtained by

r6_lslx$extract_coefficient(selector = "bic")
        x1<-1|G         x2<-1|G         x3<-1|G         y1<-1|G         y2<-1|G         y3<-1|G 
         0.0000          0.0000          0.0000          0.0000          0.0000          0.0000 
        y4<-1|G         y5<-1|G         y6<-1|G         y7<-1|G         y8<-1|G  dem60<-ind60|G 
         0.0000          0.0000          0.0000          0.0000          0.0000          1.4766 
 dem65<-ind60|G  dem65<-dem60|G     x1<-ind60|G     x2<-ind60|G     x3<-ind60|G     y1<-dem60|G 
         0.5579          0.8408          1.0000          2.1806          1.8185          1.0000 
    y2<-dem60|G     y3<-dem60|G     y4<-dem60|G     y5<-dem65|G     y6<-dem65|G     y7<-dem65|G 
         1.2787          1.0606          1.2777          1.0000          1.1979          1.2851 
    y8<-dem65|G ind60<->ind60|G dem60<->dem60|G dem65<->dem65|G       x1<->x1|G       x2<->x2|G 
         1.2773          0.4545          3.9825          0.1679          0.0829          0.1214 
      x3<->x3|G       y1<->y1|G       y5<->y1|G       y2<->y2|G       y4<->y2|G       y6<->y2|G 
         0.4733          1.9225          0.5926          6.9445          1.0545          1.8546 
      y3<->y3|G       y7<->y3|G       y4<->y4|G       y8<->y4|G       y5<->y5|G       y6<->y6|G 
         5.0914          0.6127          3.0204          0.2737          2.3723          4.7141 
      y8<->y6|G       y7<->y7|G       y8<->y8|G 
         1.2050          3.4118          3.1431 
diag(r6_lslx$extract_coefficient_acov(selector = "bic"))
        x1<-1|G         x2<-1|G         x3<-1|G         y1<-1|G         y2<-1|G         y3<-1|G 
       0.007165        0.030431        0.026350        0.091945        0.201019        0.142481 
        y4<-1|G         y5<-1|G         y6<-1|G         y7<-1|G         y8<-1|G  dem60<-ind60|G 
       0.148528        0.091033        0.148099        0.143586        0.138817        0.156133 
 dem65<-ind60|G  dem65<-dem60|G     x1<-ind60|G     x2<-ind60|G     x3<-ind60|G     y1<-dem60|G 
       0.052874        0.009752              NA        0.019379        0.023198              NA 
    y2<-dem60|G     y3<-dem60|G     y4<-dem60|G     y5<-dem65|G     y6<-dem65|G     y7<-dem65|G 
       0.033274        0.021869        0.022200              NA        0.028675        0.025411 
    y8<-dem65|G ind60<->ind60|G dem60<->dem60|G dem65<->dem65|G       x1<->x1|G       x2<->x2|G 
       0.026244        0.007734        0.894013        0.047326        0.000401        0.005047 
      x3<->x3|G       y1<->y1|G       y5<->y1|G       y2<->y2|G       y4<->y2|G       y6<->y2|G 
       0.008185        0.204032        0.125237        1.488554        0.414961        0.435944 
      y3<->y3|G       y7<->y3|G       y4<->y4|G       y8<->y4|G       y5<->y5|G       y6<->y6|G 
       0.895235        0.358282        0.499646        0.190983        0.226696        0.673361 
      y8<->y6|G       y7<->y7|G       y8<->y8|G 
       0.282949        0.495052        0.440486 

The NA standard errors mean that the corresponding coefficients are fixed or identified as zero by penalty.