Introduction
Univariate meta-analysis
Multivariate meta-analysis
Three-level meta-analysis
Two-stage structural equation modeling (TSSEM)
One-stage meta-analytic structural equation modeling (OSMASEM)
Installation and help
- Potential issues with the OpenMx available at CRAN
- Help

Introduction

The metaSEM package provides functions for conducting univariate and multivariate meta-analysis using a structural equation modeling approach via the OpenMx package. It also implemented the two-stage structural equation modeling (TSSEM) approach to conducting a fixed- and random-effects meta-analytic structural equation modeling (MASEM) on correlation/covariance matrices.
The metaSEM package is based on the following papers:
- Cheung, M. W.-L. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13(3), 182–202. https://doi.org/10.1037/a0013163
- Cheung, M. W.-L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 16(2), 267–294. https://doi.org/10.1080/10705510902751291
- Cheung, M. W.-L. (2010). Fixed-effects meta-analyses as multiple-group structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 17(3), 481–509. https://doi.org/10.1080/10705511.2010.489367
- Cheung, M. W.-L. (2013). Implementing restricted maximum likelihood estimation in structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 20(1), 157–167. https://doi.org/10.1080/10705511.2013.742404
- Cheung, M. W.-L. (2013). Multivariate meta-analysis as structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 429–454. https://doi.org/10.1080/10705511.2013.797827
- Cheung, M. W.-L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19(2), 211–229. https://doi.org/10.1037/a0032968
- Cheung, M. W.-L. (2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), 29–40. https://doi.org/10.3758/s13428-013-0361-y
- Cheung, M. W.-L. (2015). metaSEM: an R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5(1521). https://doi.org/10.3389/fpsyg.2014.01521
- Cheung, M. W.-L. (2015). Meta-Analysis: A Structural Equation Modeling Approach. Wiley.
- Cheung, M.W.-L. (2018). Computing multivariate effect sizes and their sampling covariance matrices with structural equation modeling: Theory, examples, and computer simulations. Frontiers in Psychology, 9(1387). https://doi.org/10.3389/fpsyg.2018.01387
- Cheung, M. W.-L. (2018). Issues in solving the problem of effect size heterogeneity in meta-analytic structural equation modeling: A commentary and simulation study on Yu, Downes, Carter, and O’Boyle (2016). Journal of Applied Psychology, 103(7), 787–803. https://doi.org/10.1037/apl0000284
- Cheung, M. W.-L. (2018). Some reflections on combining meta-analysis and structural equation modeling. Research Synthesis Methods, 0(0). https://doi.org/10.1002/jrsm.1321
- Cheung, M. W.-L., & Chan, W. (2004). Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods, 7(2), 206–223. https://doi.org/10.1177/1094428104264024
- Cheung, M. W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10(1), 40–64. https://doi.org/10.1037/1082-989X.10.1.40
- Cheung, M. W.-L., & Chan, W. (2005). Classifying correlation matrices into relatively homogeneous subgroups: A cluster analytic approach. Educational and Psychological Measurement, 65(6), 954–979. https://doi.org/10.1177/0013164404273946
- Cheung, M. W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 28–53. https://doi.org/10.1080/10705510802561295
- Cheung, M. W.-L., & Cheung, S. F. (2016). Random-effects models for meta-analytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods, 7(2), 140–155. https://doi.org/10.1002/jrsm.1166
- Cheung, M. W.-L., & Hafdahl, A. R. (2016). Special issue on meta-analytic structural equation modeling: Introduction from the guest editors. Research Synthesis Methods, 7(2), 112–120. https://doi.org/10.1002/jrsm.1212
- Cheung, M. W.-L., & Hong, R. Y. (2017). Applications of meta-analytic structural equation modelling in health psychology: examples, issues, and recommendations. Health Psychology Review, 11(3), 265–279. https://doi.org/10.1080/17437199.2017.1343678
- Jak, S., & Cheung, M. W.-L. (2018). Testing moderator hypotheses in meta-analytic structural equation modeling using subgroup analysis. Behavior Research Methods, 50(4), 1359–1373. https://doi.org/10.3758/s13428-018-1046-3

Univariate meta-analysis

Random-effects model

## Load the library
library(metaSEM)

## Try to use more than one cores
mxOption(NULL, 'Number of Threads', parallel::detectCores()-2)

## Show the first few studies of the data set
head(Becker83)

  study    di   vi percentage items
1     1 -0.33 0.03         25     2
2     2  0.07 0.03         25     2
3     3 -0.30 0.02         50     2
4     4  0.35 0.02        100    38
5     5  0.69 0.07        100    30
6     6  0.81 0.22        100    45

## Random-effects meta-analysis with ML
summary( meta(y=di, v=vi, data=Becker83) )


Call:
meta(y = di, v = vi, data = Becker83)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate Std.Error    lbound    ubound z value Pr(>|z|)
Intercept1  0.174734  0.113378 -0.047482  0.396950  1.5412   0.1233
Tau2_1_1    0.077376  0.054108 -0.028674  0.183426  1.4300   0.1527

Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)   0.6718

Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 2
Degrees of freedom: 8
-2 log likelihood: 7.928307 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Fixed-effects model

## Fixed-effects meta-analysis by fixing the heterogeneity variance at 0
summary( meta(y=di, v=vi, data=Becker83, RE.constraints=0) )


Call:
meta(y = di, v = vi, data = Becker83, RE.constraints = 0)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate Std.Error    lbound    ubound z value Pr(>|z|)  
Intercept1  0.100640  0.060510 -0.017957  0.219237  1.6632  0.09627 .
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)        0

Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 1
Degrees of freedom: 9
-2 log likelihood: 17.86043 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Mixed-effects model

## Mixed-effects meta-analysis with "log(items)" as the predictor
summary( meta(y=di, v=vi, x=log(items), data=Becker83) )


Call:
meta(y = di, v = vi, x = log(items), data = Becker83)

95% confidence intervals: z statistic approximation
Coefficients:
              Estimate   Std.Error      lbound      ubound z value
Intercept1 -3.2015e-01  1.0981e-01 -5.3539e-01 -1.0492e-01 -2.9154
Slope1_1    2.1088e-01  4.5084e-02  1.2251e-01  2.9924e-01  4.6774
Tau2_1_1    1.0000e-10  2.0095e-02 -3.9386e-02  3.9386e-02  0.0000
            Pr(>|z|)    
Intercept1  0.003552 ** 
Slope1_1   2.905e-06 ***
Tau2_1_1    1.000000    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239

Explained variances (R2):
                           y1
Tau2 (no predictor)    0.0774
Tau2 (with predictors) 0.0000
R2                     1.0000

Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 3
Degrees of freedom: 7
-2 log likelihood: -4.208024 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Multivariate meta-analysis

Random-effects model

## Show the data set
Berkey98

  trial pub_year no_of_patients   PD    AL var_PD cov_PD_AL var_AL
1     1     1983             14 0.47 -0.32 0.0075    0.0030 0.0077
2     2     1982             15 0.20 -0.60 0.0057    0.0009 0.0008
3     3     1979             78 0.40 -0.12 0.0021    0.0007 0.0014
4     4     1987             89 0.26 -0.31 0.0029    0.0009 0.0015
5     5     1988             16 0.56 -0.39 0.0148    0.0072 0.0304

## Multivariate meta-analysis with a random-effects model
mult1 <- meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98) 

summary(mult1)


Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL), 
    data = Berkey98)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate  Std.Error     lbound     ubound z value  Pr(>|z|)
Intercept1  0.3448392  0.0536312  0.2397239  0.4499544  6.4298 1.278e-10
Intercept2 -0.3379381  0.0812480 -0.4971812 -0.1786951 -4.1593 3.192e-05
Tau2_1_1    0.0070020  0.0090497 -0.0107351  0.0247391  0.7737    0.4391
Tau2_2_1    0.0094607  0.0099698 -0.0100797  0.0290010  0.9489    0.3427
Tau2_2_2    0.0261445  0.0177409 -0.0086270  0.0609161  1.4737    0.1406
              
Intercept1 ***
Intercept2 ***
Tau2_1_1      
Tau2_2_1      
Tau2_2_2      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)   0.6021
Intercept2: I2 (Q statistic)   0.9250

Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 5
Degrees of freedom: 5
-2 log likelihood: -11.68131 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

## Plot the effect sizes
plot(mult1)

## Plot the effect sizes with the forest plots
## Load the library for forest plots 
library("metafor")

## Create extra panels for the forest plots
plot(mult1, diag.panel=TRUE, main="Multivariate meta-analysis",
axis.label=c("PD", "AL"))

## Forest plot for PD
forest( rma(yi=PD, vi=var_PD, data=Berkey98) )
title("Forest plot of PD")

## Forest plot for AL
forest( rma(yi=AL, vi=var_AL, data=Berkey98) )
title("Forest plot of AL")

Fixed-effects model

## Fixed-effects meta-analysis by fixiing the heterogeneity variance component at 
## a 2x2 matrix of 0.
summary( meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98,
              RE.constraints=matrix(0, nrow=2, ncol=2)) )


Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL), 
    data = Berkey98, RE.constraints = matrix(0, nrow = 2, ncol = 2))

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate Std.Error    lbound    ubound z value  Pr(>|z|)    
Intercept1  0.307219  0.028575  0.251212  0.363225  10.751 < 2.2e-16 ***
Intercept2 -0.394377  0.018649 -0.430929 -0.357825 -21.147 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)        0
Intercept2: I2 (Q statistic)        0

Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 2
Degrees of freedom: 8
-2 log likelihood: 90.88326 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Mixed-effects model

## Multivariate meta-analysis with "publication year-1979" as a predictor 
summary( meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98,
              x=scale(pub_year, center=1979)) )


Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL), 
    x = scale(pub_year, center = 1979), data = Berkey98)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate  Std.Error     lbound     ubound z value  Pr(>|z|)
Intercept1  0.3440001  0.0857659  0.1759020  0.5120982  4.0109 6.048e-05
Intercept2 -0.2918175  0.1312796 -0.5491208 -0.0345141 -2.2229   0.02622
Slope1_1    0.0063540  0.1078235 -0.2049762  0.2176842  0.0589   0.95301
Slope2_1   -0.0705888  0.1620966 -0.3882922  0.2471146 -0.4355   0.66322
Tau2_1_1    0.0080405  0.0101206 -0.0117955  0.0278766  0.7945   0.42692
Tau2_2_1    0.0093413  0.0105515 -0.0113392  0.0300218  0.8853   0.37599
Tau2_2_2    0.0250135  0.0170788 -0.0084603  0.0584873  1.4646   0.14303
              
Intercept1 ***
Intercept2 *  
Slope1_1      
Slope2_1      
Tau2_1_1      
Tau2_2_1      
Tau2_2_2      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0

Explained variances (R2):
                              y1     y2
Tau2 (no predictor)    0.0070020 0.0261
Tau2 (with predictors) 0.0080405 0.0250
R2                     0.0000000 0.0433

Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 7
Degrees of freedom: 3
-2 log likelihood: -12.00859 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Three-level meta-analysis

This section illustrates how to conduct three-level meta-analyses using the metaSEM package implemented in the R environment. The metaSEM package was written to simplify the procedures to perform a meta-analysis. Most readers may only need to use the metaSEM package to conduct the analysis. The next section shows how to conduct two- and three-level meta-analyses with the meta() and meta3() functions. The third section demonstrates more complicated three-level meta-analyses using a dataset with more predictors. This section shows how to implement three-level meta-analyses as structural equation models using the OpenMx package. It provides detailed steps on how three-level meta-analyses can be formulated as structural equation models.
This section also demonstrates the advantages of using the SEM approach to conduct three-level meta-analyses. These include flexibility on imposing constraints for model comparisons and construction of likelihood-based confidence interval (LBCI). I also demonstrate how to conduct three-level meta-analysis with restricted (or residual) maximum likelihood (REML) using the reml3() function and handling missing covariates with the full information maximum likelihood (FIML) using the meta3X() function. Readers may refer to Cheung (2015) for the design and implementation of the metaSEM package and Cheung (2014) for the theory and issues on how to formulate three-level meta-analyses as structural equation models.
Two datasets from published meta-analyses were used in the illustrations. The first dataset was based on Cooper et al. (2003) and Konstantopoulos (2011). Konstantopoulos (2011) selected part of the dataset to illustrate how to conduct a three-level meta-analysis. The second dataset was reported by Bornmann et al. (2007) and Marsh et al. (2009). They conducted a three-level meta-analysis on gender effects in peer reviews of grant proposals.

Comparisons between two- and three-level models with Cooper et al.’s (2003) dataset

As an illustration, I first conduct the tradition (two-level) meta-analysis using the meta() function. Then I conduct a three-level meta-analysis using the meta3() function. We may compare the similarities and differences between these two sets of results.

Inspecting the data

Before running the analyses, we need to load the metaSEM library. The datasets are stored in the library. It is always a good idea to inspect the data before the analyses. We may display the first few cases of the dataset by using the head() command.

#### Cooper et al. (2003)
library("metaSEM")

head(Cooper03)

  District Study     y     v Year
1       11     1 -0.18 0.118 1976
2       11     2 -0.22 0.118 1976
3       11     3  0.23 0.144 1976
4       11     4 -0.30 0.144 1976
5       12     5  0.13 0.014 1989
6       12     6 -0.26 0.014 1989

Two-level meta-analysis

Similar to other R packages, we may use summary() to extract the results after running the analyses. I first conduct a random-effects meta-analysis and then a fixed- and mixed-effects meta-analyses.

Random-effects model The Q statistic on testing the homogeneity of effect sizes was $578.86, df = 55, p < .001$. The estimated heterogeneity $\tau^2$ (labeled Tau2_1_1 in the output) and $I^2$ were 0.0866 and 0.9459, respectively. This indicates that the between-study effect explains about 95% of the total variation. The average population effect (labeled Intercept1 in the output; and its 95% Wald CI) was 0.1280 (0.0428, 0.2132).

#### Two-level meta-analysis

## Random-effects model  
summary( meta(y=y, v=v, data=Cooper03) )


Call:
meta(y = y, v = v, data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
           Estimate Std.Error   lbound   ubound z value  Pr(>|z|)    
Intercept1 0.128003  0.043472 0.042799 0.213207  2.9445  0.003235 ** 
Tau2_1_1   0.086537  0.019485 0.048346 0.124728  4.4411 8.949e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)   0.9459

Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 2
Degrees of freedom: 54
-2 log likelihood: 33.2919 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Fixed-effects model A fixed-effects meta-analysis can be conducted by fixing the heterogeneity of the random effects at 0 with the RE.constraints argument (random-effects constraints). The estimated common effect (and its 95% Wald CI) was 0.0464 (0.0284, 0.0644).

## Fixed-effects model
summary( meta(y=y, v=v, data=Cooper03, RE.constraints=0) )


Call:
meta(y = y, v = v, data = Cooper03, RE.constraints = 0)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate Std.Error    lbound    ubound z value Pr(>|z|)    
Intercept1 0.0464072 0.0091897 0.0283957 0.0644186  5.0499 4.42e-07 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)        0

Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 1
Degrees of freedom: 55
-2 log likelihood: 434.2075 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Mixed-effects model Year was used as a covariate. It is easier to interpret the intercept by centering the Year with scale(Year, scale=FALSE). The scale=FALSE argument means that it is centered, but not standardized. The estimated regression coefficient (labeled Slope1_1 in the output; and its 95% Wald CI) was 0.0051 (-0.0033, 0.0136) which is not significant at $\alpha=.05$. The $R^2$ is 0.0164.

## Mixed-effects model
summary( meta(y=y, v=v, x=scale(Year, scale=FALSE), data=Cooper03) )


Call:
meta(y = y, v = v, x = scale(Year, scale = FALSE), data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate  Std.Error     lbound     ubound z value  Pr(>|z|)
Intercept1  0.1259126  0.0432028  0.0412367  0.2105884  2.9145  0.003563
Slope1_1    0.0051307  0.0043248 -0.0033457  0.0136071  1.1864  0.235483
Tau2_1_1    0.0851153  0.0190462  0.0477856  0.1224451  4.4689 7.862e-06
              
Intercept1 ** 
Slope1_1      
Tau2_1_1   ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Explained variances (R2):
                           y1
Tau2 (no predictor)    0.0865
Tau2 (with predictors) 0.0851
R2                     0.0164

Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 31.88635 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Three-level meta-analysis

Random-effects model The Q statistic on testing the homogeneity of effect sizes was the same as that under the two-level meta-analysis. The estimated heterogeneity at level 2 $\tau^2_{(2)}$ (labeled Tau2_2 in the output) and at level 3 $\tau^2_{(3)}$ (labeled Tau2_3 in the output) were 0.0329 and 0.0577, respectively. The level 2 $I^2_{(2)}$ (labeled I2_2 in the output) and the level 3 $I^2_{(3)}$ (labeled I2_3 in the output) were 0.3440 and 0.6043, respectively. Schools (level 2) and districts (level 3) explain about 34% and 60% of the total variation, respectively. The average population effect (and its 95% Wald CI) was 0.1845 (0.0266, 0.3423).

#### Three-level meta-analysis
## Random-effects model
summary( meta3(y=y, v=v, cluster=District, data=Cooper03) )


Call:
meta3(y = y, v = v, cluster = District, data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Intercept  0.1844554  0.0805411  0.0265977  0.3423131  2.2902 0.022010 * 
Tau2_2     0.0328648  0.0111397  0.0110314  0.0546982  2.9502 0.003175 **
Tau2_3     0.0577384  0.0307423 -0.0025154  0.1179921  1.8781 0.060362 . 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                              Estimate
I2_2 (Typical v: Q statistic)   0.3440
I2_3 (Typical v: Q statistic)   0.6043

Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 16.78987 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Mixed-effects model Year was used as a covariate. The estimated regression coefficient (labeled Slope_1 in the output; and its 95% Wald CI) was 0.0051 (-0.0116, 0.0218) which is not significant at $\alpha=.05$. The estimated level 2 $R^2_{(2)}$ and level 3 $R^2_{(3)}$ were 0.0000 and 0.0221, respectively.

## Mixed-effects model
summary( meta3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE), 
               data=Cooper03) )


Call:
meta3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE), 
    data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Intercept  0.1780268  0.0805219  0.0202067  0.3358469  2.2109 0.027042 * 
Slope_1    0.0050737  0.0085266 -0.0116382  0.0217856  0.5950 0.551814   
Tau2_2     0.0329390  0.0111620  0.0110618  0.0548162  2.9510 0.003168 **
Tau2_3     0.0564628  0.0300330 -0.0024007  0.1153264  1.8800 0.060104 . 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Explained variances (R2):
                        Level 2 Level 3
Tau2 (no predictor)    0.032865  0.0577
Tau2 (with predictors) 0.032939  0.0565
R2                     0.000000  0.0221

Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 4
Degrees of freedom: 52
-2 log likelihood: 16.43629 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Model comparisons Many research hypotheses involve model comparisons among nested models. anova(), a generic function for comparing nested models, may be used to conduct a likelihood ratio test which is also known as a chi-square difference test.
Testing $H_0: \tau^2_{(3)} = 0$
- Based on the data structure, it is clear that a 3-level meta-analysis is preferred to a traditional 2-level meta-analysis. It is still of interest to test whether the 3-level model is statistically better than the 2-level model by testing $H_0: \tau^2_{(3)}=0$. Since the models with $\tau^2_{(3)}$ being freely estimated and with $\tau^2_{(3)}=0$ are nested, we may compare them by the use of a likelihood ratio test.
- It should be noted, however, that $H_0: \tau^2_{(3)}=0$ is tested on the boundary. The likelihood ratio test is not distributed as a chi-square variate with 1 df. A simple strategy to correct this bias is to reject the null hypothesis when the observed p value is larger than .10 for $\alpha=.05$.
- The likelihood-ratio test was 16.5020 (df =1), p < .001. This demonstrates that the three-level model is statistically better than the two-level model.

## Model comparisons
  
model2 <- meta(y=y, v=v, data=Cooper03, model.name="2 level model", silent=TRUE) 
#### An equivalent model by fixing tau2 at level 3=0 in meta3()
## model2 <- meta3(y=y, v=v, cluster=District, data=Cooper03, 
##                 model.name="2 level model", RE3.constraints=0) 
model3 <- meta3(y=y, v=v, cluster=District, data=Cooper03, 
                model.name="3 level model", silent=TRUE) 
anova(model3, model2)

           base    comparison ep minus2LL df       AIC   diffLL diffdf
1 3 level model          <NA>  3 16.78987 53 -89.21013       NA     NA
2 3 level model 2 level model  2 33.29190 54 -74.70810 16.50203      1
             p
1           NA
2 4.859793e-05

Testing $H_0: \tau^2_{(2)} = \tau^2_{(3)}$
- From the results of the 3-level random-effects meta-analysis, it appears the level 3 heterogeneity is much larger than that at level 2.
- We may test the null hypothesis $H_0: \tau^2_{(2)} = \tau^2_{(3)}$ by the use of a likelihood-ratio test.
- We may impose an equality constraint on $\tau^2_{(2)} = \tau^2_{(3)}$ by using the same label in meta3(). For example, Eq_tau2 is used as the label in RE2.constraints and RE3.constraints meaning that both the level 2 and level 3 random effects heterogeneity variances are constrained equally. The value of 0.1 was used as the starting value in the constraints.
- The likelihood-ratio test was 0.6871 (df = 1), p = 0.4072. This indicates that there is not enough evidence to reject $H_0: \tau^2_2=\tau^2_3$.

## Testing \tau^2_2 = \tau^2_3
modelEqTau2 <- meta3(y=y, v=v, cluster=District, data=Cooper03, 
                     model.name="Equal tau2 at both levels",
                     RE2.constraints="0.1*Eq_tau2", RE3.constraints="0.1*Eq_tau2") 
anova(model3, modelEqTau2)

           base                comparison ep minus2LL df       AIC
1 3 level model                      <NA>  3 16.78987 53 -89.21013
2 3 level model Equal tau2 at both levels  2 17.47697 54 -90.52303
     diffLL diffdf         p
1        NA     NA        NA
2 0.6870959      1 0.4071539

Likelihood-based confidence interval
- A Wald CI is constructed by $\hat{\theta} \pm 1.96 SE$ where $\hat{\theta}$ and $SE$ are the parameter estimate and its estimated standard error.
- An LBCI can be constructed by the use of the likelihood ratio statistic (e.g., Cheung, 2009; Neal & Miller, 1997).
- It is well known that the performance of Wald CI on variance components is very poor. For example, the 95% Wald CI on $\hat{\tau}^2_{(3)}$ in the three-level random-effects meta-analysis was (-0.0025, 0.1180). The lower bound falls outside 0.
- An LBCI on the heterogeneity variance is preferred. Since $I^2_{(2)}$ and $I^2_{(3)}$ are functions of $\tau^2_{(2)}$ and $\tau^2_{(3)}$, LBCI on these indices may also be requested and used to indicate the precision of these indices.
- An LBCI may be requested by specifying LB in the intervals.type argument.
- The 95% LBCI on $\hat{\tau}^2_{(3)}$ is (0.0198, 0.1763) that stay inside the meaningful boundaries. Regarding the $I^2$, the 95% LBCIs on $I^2_{(2)}$ and $I^2_{(3)}$ were (0.1274, 0.6573) and (0.2794, 0.8454), respectively.

## LBCI for random-effects model
summary( meta3(y=y, v=v, cluster=District, data=Cooper03, 
               I2=c("I2q", "ICC"), intervals.type="LB") )


Call:
meta3(y = y, v = v, cluster = District, data = Cooper03, intervals.type = "LB", 
    I2 = c("I2q", "ICC"))

95% confidence intervals: Likelihood-based statistic
Coefficients:
          Estimate Std.Error   lbound   ubound z value Pr(>|z|)
Intercept 0.184455        NA 0.012025 0.358179      NA       NA
Tau2_2    0.032865        NA 0.016331 0.060489      NA       NA
Tau2_3    0.057738        NA 0.019810 0.172714      NA       NA

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Heterogeneity indices (I2) and their 95% likelihood-based CIs:
                               lbound Estimate ubound
I2_2 (Typical v: Q statistic) 0.12754  0.34396 0.6565
ICC_2 (tau^2/(tau^2+tau^3))   0.13123  0.36273 0.7005
I2_3 (Typical v: Q statistic) 0.28173  0.60429 0.8451
ICC_3 (tau^3/(tau^2+tau^3))   0.29948  0.63727 0.8688

Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 16.78987 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

## LBCI for mixed-effects model
summary( meta3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE), 
               data=Cooper03, intervals.type="LB") )


Call:
meta3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE), 
    data = Cooper03, intervals.type = "LB")

95% confidence intervals: Likelihood-based statistic
Coefficients:
            Estimate Std.Error     lbound     ubound z value Pr(>|z|)
Intercept  0.1780268        NA  0.0056477  0.3512066      NA       NA
Slope_1    0.0050737        NA -0.0128322  0.0237932      NA       NA
Tau2_2     0.0329390        NA  0.0163748  0.0591198      NA       NA
Tau2_3     0.0564628        NA  0.0192365  0.1429149      NA       NA

Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0

Explained variances (R2):
                        Level 2 Level 3
Tau2 (no predictor)    0.032865  0.0577
Tau2 (with predictors) 0.032939  0.0565
R2                     0.000000  0.0221

Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 4
Degrees of freedom: 52
-2 log likelihood: 16.43629 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Restricted maximum likelihood estimation
- REML may also be used in the three-level meta-analysis. The parameter estimates for $\tau^2_{(2)}$ and $\tau^2_{(3)}$ were 0.0327 and 0.0651, respectively.

## REML
summary( reml1 <- reml3(y=y, v=v, cluster=District, data=Cooper03) )


Call:
reml3(y = y, v = v, cluster = District, data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
         Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Tau2_2  0.0327365  0.0110922  0.0109963  0.0544768  2.9513 0.003164 **
Tau2_3  0.0650619  0.0355102 -0.0045368  0.1346607  1.8322 0.066921 . 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Number of studies (or clusters): 56
Number of observed statistics: 55
Number of estimated parameters: 2
Degrees of freedom: 53
-2 log likelihood: -81.14044 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

We may impose an equality constraint on $\tau^2_{(2)}$ and $\tau^2_{(3)}$ and test whether this constraint is statistically significant. The estimated value for $\tau^2_{(2)}=\tau^2_{(3)}$ was 0.0404. When this model is compared against the unconstrained model, the test statistic was 1.0033 (df = 1), p = .3165, which is not significant.

summary( reml0 <- reml3(y=y, v=v, cluster=District, data=Cooper03,
                        RE.equal=TRUE, model.name="Equal Tau2") )


Call:
reml3(y = y, v = v, cluster = District, data = Cooper03, RE.equal = TRUE, 
    model.name = "Equal Tau2")

95% confidence intervals: z statistic approximation
Coefficients:
     Estimate Std.Error   lbound   ubound z value  Pr(>|z|)    
Tau2 0.040418  0.010290 0.020249 0.060587  3.9277 8.576e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Number of studies (or clusters): 56
Number of observed statistics: 55
Number of estimated parameters: 1
Degrees of freedom: 54
-2 log likelihood: -80.1371 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

anova(reml1, reml0)

                          base comparison ep  minus2LL df AIC   diffLL
1 Variance component with REML       <NA>  2 -81.14044 -2  NA       NA
2 Variance component with REML Equal Tau2  1 -80.13710 -1  NA 1.003336
  diffdf         p
1     NA        NA
2      1 0.3165046

We may also estimate the residual heterogeneity after controlling for the covariate. The estimated residual heterogeneity for $\tau^2_{(2)}$ and $\tau^2_{(3)}$ were 0.0327 and 0.0723, respectively.

summary( reml3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE),
               data=Cooper03) )


Call:
reml3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE), 
    data = Cooper03)

95% confidence intervals: z statistic approximation
Coefficients:
         Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Tau2_2  0.0326502  0.0110529  0.0109870  0.0543134  2.9540 0.003137 **
Tau2_3  0.0722656  0.0405349 -0.0071813  0.1517124  1.7828 0.074619 . 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Number of studies (or clusters): 56
Number of observed statistics: 54
Number of estimated parameters: 2
Degrees of freedom: 52
-2 log likelihood: -72.09405 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

More complex 3-level meta-analyses with Bornmann et al.’s (2007) dataset

This section replicates the findings in Table 3 of Marsh et al. (2009). Several additional analyses on model comparisons were conducted. Missing data were artificially introduced to illustrate how missing data might be handled with FIML.

Inspecting the data

The effect size and its sampling variance are logOR (log of the odds ratio) and v, respectively. Cluster is the variable representing the cluster effect, whereas the potential covariates are Year (year of publication), Type (Grants vs. Fellowship), Discipline (Physical sciences, Life sciences/biology, Social sciences/humanities and Multidisciplinary) and Country (United States, Canada, Australia, United Kingdom and Europe).

#### Bornmann et al. (2007)
head(Bornmann07)

  Id                       Study Cluster    logOR          v Year
1  1 Ackers (2000a; Marie Curie)       1 -0.40108 0.01391692 1996
2  2 Ackers (2000b; Marie Curie)       1 -0.05727 0.03428793 1996
3  3 Ackers (2000c; Marie Curie)       1 -0.29852 0.03391122 1996
4  4 Ackers (2000d; Marie Curie)       1  0.36094 0.03404025 1996
5  5 Ackers (2000e; Marie Curie)       1 -0.33336 0.01282103 1996
6  6 Ackers (2000f; Marie Curie)       1 -0.07173 0.01361189 1996
        Type                 Discipline Country
1 Fellowship          Physical sciences  Europe
2 Fellowship          Physical sciences  Europe
3 Fellowship          Physical sciences  Europe
4 Fellowship          Physical sciences  Europe
5 Fellowship Social sciences/humanities  Europe
6 Fellowship          Physical sciences  Europe

Model 0: Intercept

The Q statistic was 221.2809 (df = 65), p < .001. The estimated average effect (and its 95% Wald CI) was -0.1008 (-0.1794, -0.0221). The $\hat{\tau}^2_{(2)}$ and $\hat{\tau}^3_{(3)}$ were 0.0038 and 0.0141, respectively. The $I^2_{(2)}$ and $I^2_{(3)}$ were 0.1568 and 0.5839, respectively.

## Model 0: Intercept  
summary( Model0 <- meta3(y=logOR, v=v, cluster=Cluster, data=Bornmann07, 
                         model.name="3 level model") )


Call:
meta3(y = logOR, v = v, cluster = Cluster, data = Bornmann07, 
    model.name = "3 level model")

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)  
Intercept -0.1007784  0.0401327 -0.1794371 -0.0221198 -2.5111  0.01203 *
Tau2_2     0.0037965  0.0027210 -0.0015367  0.0091297  1.3952  0.16295  
Tau2_3     0.0141352  0.0091445 -0.0037877  0.0320580  1.5458  0.12216  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                              Estimate
I2_2 (Typical v: Q statistic)   0.1568
I2_3 (Typical v: Q statistic)   0.5839

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 3
Degrees of freedom: 63
-2 log likelihood: 25.80256 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing $H_0: \tau^2_3 = 0$ We may test whether the three-level model is necessary by testing $H_0: \tau^2_{(3)} = 0$. The likelihood ratio statistic was 10.2202 (df = 1), p < .01. Thus, the three-level model is statistically better than the two-level model.

## Testing tau^2_3 = 0
Model0a <- meta3(logOR, v, cluster=Cluster, data=Bornmann07, 
                 RE3.constraints=0, model.name="2 level model")
anova(Model0, Model0a)

           base    comparison ep minus2LL df        AIC   diffLL diffdf
1 3 level model          <NA>  3 25.80256 63 -100.19744       NA     NA
2 3 level model 2 level model  2 36.02279 64  -91.97721 10.22024      1
            p
1          NA
2 0.001389081

Testing $H_0: \tau^2_2 = \tau^2_3$ The likelihood-ratio statistic in testing $H_0: \tau^2_{(2)} = \tau^2_{(3)}$ was 1.3591 (df = 1), p = 0.2437. Thus, there is no evidence to reject the null hypothesis.

## Testing tau^2_2 = tau^2_3
Model0b <- meta3(logOR, v, cluster=Cluster, data=Bornmann07, 
                 RE2.constraints="0.1*Eq_tau2", RE3.constraints="0.1*Eq_tau2", 
                 model.name="tau2_2 equals tau2_3")
anova(Model0, Model0b)

           base           comparison ep minus2LL df       AIC   diffLL
1 3 level model                 <NA>  3 25.80256 63 -100.1974       NA
2 3 level model tau2_2 equals tau2_3  2 27.16166 64 -100.8383 1.359103
  diffdf        p
1     NA       NA
2      1 0.243693

Model 1: `Type` as a covariate

Conventionally, one level (e.g., Grants) is used as the reference group. The estimated intercept (labeled Intercept in the output) represents the estimated effect size for Grants and the regression coefficient (labeled Slope_1 in the output) is the difference between Fellowship and Grants.
- The estimated slope of Type (and its 95% Wald CI) was -0.1956 (-0.3018, -0.0894) which is statistically significant at $\alpha=.05$. This is the difference between Fellowship and Grants. The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.0693 and 0.7943, respectively.

## Model 1: Type as a covariate  
## Convert characters into a dummy variable
## Type2=0 (Grants); Type2=1 (Fellowship)    
Type2 <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
summary( Model1 <- meta3(logOR, v, x=Type2, cluster=Cluster, data=Bornmann07))


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type2, data = Bornmann07)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value  Pr(>|z|)
Intercept -0.0066071  0.0371125 -0.0793462  0.0661320 -0.1780 0.8587001
Slope_1   -0.1955869  0.0541649 -0.3017483 -0.0894256 -3.6110 0.0003051
Tau2_2     0.0035335  0.0024306 -0.0012303  0.0082974  1.4538 0.1460058
Tau2_3     0.0029079  0.0031183 -0.0032039  0.0090197  0.9325 0.3510704
             
Intercept    
Slope_1   ***
Tau2_2       
Tau2_3       
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0035335  0.0029
R2                     0.0692595  0.7943

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 4
Degrees of freedom: 62
-2 log likelihood: 17.62569 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Alternative model: Grants and Fellowship as indicator variables
- If we want to estimate the effects for both Grants and Fellowship, we may create two indicator variables to represent them. Since we cannot estimate the intercept and these coefficients at the same time, we need to fix the intercept at 0 by specifying the intercept.constraints=0 argument in meta3(). We are now able to include both Grants and Fellowship in the analysis. The estimated effects (and their 95% CIs) for Grants and Fellowship were -0.0066 (-0.0793, 0.0661) and -0.2022 (-0.2805, -0.1239), respectively.

## Alternative model: Grants and Fellowship as indicators  
## Indicator variables
Grants <- ifelse(Bornmann07$Type=="Grants", yes=1, no=0)
Fellowship <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
  
Model1b <- meta3(logOR, v, x=cbind(Grants, Fellowship), 
                 cluster=Cluster, data=Bornmann07,
                 intercept.constraints=0, model.name="Model 1")

## Rerun to remove error code
Model1b <- rerun(Model1b)

summary(Model1b)


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Grants, 
    Fellowship), data = Bornmann07, intercept.constraints = 0, 
    model.name = "Model 1")

95% confidence intervals: z statistic approximation
Coefficients:
           Estimate   Std.Error      lbound      ubound z value  Pr(>|z|)
Slope_1 -1.0441e-01  1.7816e+02 -3.4930e+02  3.4909e+02 -0.0006    0.9995
Slope_2 -2.0209e-01  3.9553e-02 -2.7962e-01 -1.2457e-01 -5.1094 3.232e-07
Tau2_2   3.5752e-03  2.4549e-03 -1.2363e-03  8.3867e-03  1.4563    0.1453
Tau2_3   2.7139e-03  2.8448e-03 -2.8618e-03  8.2896e-03  0.9540    0.3401
           
Slope_1    
Slope_2 ***
Tau2_2     
Tau2_3     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0035752  0.0027
R2                     0.0582930  0.8080

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 4
Degrees of freedom: 62
-2 log likelihood: 17.65814 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Model 2: `Year` and `Year^2` as covariates

When there are several covariates, we may combine them with the cbind() command. For example, cbind(Year, Year^2) includes both Year and its squared as covariates. In the output, Slope_1 and Slope_2 refer to the regression coefficients for Year and Year^2, respectively. To increase the numerical stability, the covariates are usually centered before creating the quadratic terms. Since Marsh et al. (2009) standardized Year in their analysis, I follow this practice here.
The estimated regression coefficients (and their 95% CIs) for Year and Year^2 were -0.0010 (-0.0473, 0.0454) and -0.0118 (-0.0247, 0.0012), respectively. The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.2430 and 0.0000, respectively.

## Model 2: Year and Year^2 as covariates
summary( Model2 <- meta3(logOR, v, x=cbind(scale(Year), scale(Year)^2), 
                         cluster=Cluster, data=Bornmann07,
                         model.name="Model 2") )


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(scale(Year), 
    scale(Year)^2), data = Bornmann07, model.name = "Model 2")

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value Pr(>|z|)
Intercept -0.08627312  0.04125581 -0.16713302 -0.00541322 -2.0912  0.03651
Slope_1   -0.00095287  0.02365224 -0.04731040  0.04540466 -0.0403  0.96786
Slope_2   -0.01176840  0.00659995 -0.02470407  0.00116727 -1.7831  0.07457
Tau2_2     0.00287389  0.00206817 -0.00117965  0.00692744  1.3896  0.16466
Tau2_3     0.01479446  0.00926095 -0.00335666  0.03294558  1.5975  0.11015
           
Intercept *
Slope_1    
Slope_2   .
Tau2_2     
Tau2_3     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0028739  0.0148
R2                     0.2430134  0.0000

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 5
Degrees of freedom: 61
-2 log likelihood: 22.3836 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing $H_0: \beta_{Year} = \beta_{Year^2}=0$ The test statistic was 3.4190 (df = 2), p = 0.1810. Thus, there is no evidence supporting that =Year= has a quadratic effect on the effect size.

## Testing beta_{Year} = beta_{Year^2}=0
anova(Model2, Model0)

     base    comparison ep minus2LL df       AIC   diffLL diffdf         p
1 Model 2          <NA>  5 22.38360 61  -99.6164       NA     NA        NA
2 Model 2 3 level model  3 25.80256 63 -100.1974 3.418955      2 0.1809603

Model 3: `Discipline` as a covariate

There are four categories of Discipline. multidisciplinary is used as the reference group in the analysis.
The estimated regression coefficients (and their 95% Wald CIs) for DisciplinePhy, DisciplineLife and DisciplineSoc were -0.0091 (-0.2041, 0.1859), -0.1262 (-0.2804, 0.0280) and -0.2370 (-0.4746, 0.0007), respectively. The $R^2_2$ and $R^2_3$ were 0.0000 and 0.4975, respectively.

## Model 3: Discipline as a covariate
## Create dummy variables using multidisciplinary as the reference group
DisciplinePhy <- ifelse(Bornmann07$Discipline=="Physical sciences", yes=1, no=0)
DisciplineLife <- ifelse(Bornmann07$Discipline=="Life sciences/biology", yes=1, no=0)
DisciplineSoc <- ifelse(Bornmann07$Discipline=="Social sciences/humanities", yes=1, no=0)
summary( Model3 <- meta3(logOR, v, x=cbind(DisciplinePhy, DisciplineLife, DisciplineSoc), 
                         cluster=Cluster, data=Bornmann07,
                         model.name="Model 3") )


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(DisciplinePhy, 
    DisciplineLife, DisciplineSoc), data = Bornmann07, model.name = "Model 3")

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value Pr(>|z|)
Intercept -0.01474783  0.06389945 -0.13998845  0.11049278 -0.2308  0.81747
Slope_1   -0.00913064  0.09949198 -0.20413134  0.18587005 -0.0918  0.92688
Slope_2   -0.12617957  0.07866274 -0.28035570  0.02799656 -1.6041  0.10870
Slope_3   -0.23695698  0.12123091 -0.47456520  0.00065125 -1.9546  0.05063
Tau2_2     0.00390942  0.00283949 -0.00165587  0.00947471  1.3768  0.16857
Tau2_3     0.00710338  0.00643210 -0.00550331  0.01971006  1.1044  0.26944
           
Intercept  
Slope_1    
Slope_2    
Slope_3   .
Tau2_2     
Tau2_3     
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0039094  0.0071
R2                     0.0000000  0.4975

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 6
Degrees of freedom: 60
-2 log likelihood: 20.07571 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing whether Discipline is significant
- The test statistic was 5.7268 (df = 3), p = 0.1257 which is not significant. Therefore, there is no evidence supporting that =Discipline= explains the variation of the effect sizes.

## Testing whether Discipline is significant
anova(Model3, Model0)

     base    comparison ep minus2LL df        AIC   diffLL diffdf
1 Model 3          <NA>  6 20.07571 60  -99.92429       NA     NA
2 Model 3 3 level model  3 25.80256 63 -100.19744 5.726842      3
          p
1        NA
2 0.1256832

Model 4: `Country` as a covariate

There are five categories in Country. The United States is used as the reference group in the analysis.
The estimated regression coefficients (and their 95% Wald CIs) for CountryAus, CountryCan, CountryEur, and CountryUK are -0.0240 (-0.2405, 0.1924), -0.1341 (-0.3674, 0.0993), -0.2211 (-0.3660, -0.0762) and 0.0537 (-0.1413, 0.2487), respectively. The $R^2_2$ and $R^2_3$ were 0.1209 and 0.6606, respectively.

## Model 4: Country as a covariate
## Create dummy variables using the United States as the reference group
CountryAus <- ifelse(Bornmann07$Country=="Australia", yes=1, no=0)
CountryCan <- ifelse(Bornmann07$Country=="Canada", yes=1, no=0)
CountryEur <- ifelse(Bornmann07$Country=="Europe", yes=1, no=0)
CountryUK <- ifelse(Bornmann07$Country=="United Kingdom", yes=1, no=0)
  
summary( Model4 <- meta3(logOR, v, x=cbind(CountryAus, CountryCan, CountryEur, 
                         CountryUK), cluster=Cluster, data=Bornmann07,
                         model.name="Model 4") )


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(CountryAus, 
    CountryCan, CountryEur, CountryUK), data = Bornmann07, model.name = "Model 4")

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Intercept  0.0025681  0.0597768 -0.1145923  0.1197285  0.0430 0.965732   
Slope_1   -0.0240109  0.1104328 -0.2404552  0.1924333 -0.2174 0.827876   
Slope_2   -0.1340800  0.1190667 -0.3674465  0.0992865 -1.1261 0.260127   
Slope_3   -0.2210801  0.0739174 -0.3659556 -0.0762046 -2.9909 0.002782 **
Slope_4    0.0537251  0.0994803 -0.1412527  0.2487030  0.5401 0.589157   
Tau2_2     0.0033376  0.0023492 -0.0012667  0.0079420  1.4208 0.155383   
Tau2_3     0.0047979  0.0044818 -0.0039862  0.0135820  1.0705 0.284379   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0033376  0.0048
R2                     0.1208598  0.6606

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 7
Degrees of freedom: 59
-2 log likelihood: 14.18259 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing whether Discipline is significant
- The test statistic was 11.6200 (df = 4), p = 0.0204 which is statistically significant.

  ## Testing whether Discipline is significant
  anova(Model4, Model0)

     base    comparison ep minus2LL df       AIC   diffLL diffdf
1 Model 4          <NA>  7 14.18259 59 -103.8174       NA     NA
2 Model 4 3 level model  3 25.80256 63 -100.1974 11.61996      4
           p
1         NA
2 0.02041284

Model 5: `Type` and `Discipline` as covariates

The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.3925 and 1.0000, respectively. The $\hat{\tau}^2_{(3)}$ was near 0 after controlling for the covariates.

## Model 5: Type and Discipline as covariates
Model5 <- meta3(logOR, v, x=cbind(Type2, DisciplinePhy, DisciplineLife, 
                DisciplineSoc), cluster=Cluster, data=Bornmann07,
                model.name="Model 5")

## Rerun to remove error code
Model5 <- rerun(Model5)

summary(Model5)


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, DisciplinePhy, 
    DisciplineLife, DisciplineSoc), data = Bornmann07, model.name = "Model 5")

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value
Intercept  6.7036e-02  1.8553e-02  3.0672e-02  1.0340e-01  3.6132
Slope_1   -1.9004e-01  4.0234e-02 -2.6890e-01 -1.1118e-01 -4.7233
Slope_2    1.9511e-02  6.5942e-02 -1.0973e-01  1.4875e-01  0.2959
Slope_3   -1.2779e-01  3.5914e-02 -1.9818e-01 -5.7400e-02 -3.5582
Slope_4   -2.3950e-01  9.4054e-02 -4.2384e-01 -5.5154e-02 -2.5464
Tau2_2     2.3062e-03  1.4270e-03 -4.9059e-04  5.1030e-03  1.6162
Tau2_3     1.0000e-10          NA          NA          NA      NA
           Pr(>|z|)    
Intercept 0.0003025 ***
Slope_1    2.32e-06 ***
Slope_2   0.7673210    
Slope_3   0.0003734 ***
Slope_4   0.0108849 *  
Tau2_2    0.1060586    
Tau2_3           NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0023062  0.0000
R2                     0.3925434  1.0000

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 7
Degrees of freedom: 59
-2 log likelihood: 4.66727 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing whether Discipline is significant after controlling for Type
- The test statistic was 12.9584 (df = 3), p = 0.0047 which is significant. Therefore, Discipline is still significant after controlling for Type.

## Testing whether Discipline is significant after controlling for Type
anova(Model5, Model1)

     base            comparison ep minus2LL df       AIC   diffLL diffdf
1 Model 5                  <NA>  7  4.66727 59 -113.3327       NA     NA
2 Model 5 Meta analysis with ML  4 17.62569 62 -106.3743 12.95842      3
            p
1          NA
2 0.004727388

Model 6: `Type` and `Country` as covariates

The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.3948 and 1.0000, respectively. The $\hat{\tau}^2_{(3)}$ was near 0 after controlling for the covariates.

## Model 6: Type and Country as covariates
Model6 <- meta3(logOR, v, x=cbind(Type2, CountryAus, CountryCan, 
                CountryEur, CountryUK), cluster=Cluster, data=Bornmann07,
                model.name="Model 6") 

## Rerun to remove error code
Model6 <- rerun(Model6)

summary(Model6)


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, CountryAus, 
    CountryCan, CountryEur, CountryUK), data = Bornmann07, model.name = "Model 6")

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value
Intercept  6.7507e-02  1.8933e-02  3.0399e-02  1.0461e-01  3.5656
Slope_1   -1.5167e-01  4.1113e-02 -2.3225e-01 -7.1092e-02 -3.6892
Slope_2   -6.9580e-02  8.5164e-02 -2.3650e-01  9.7339e-02 -0.8170
Slope_3   -1.4231e-01  7.5204e-02 -2.8970e-01  5.0879e-03 -1.8923
Slope_4   -1.6116e-01  4.0203e-02 -2.3995e-01 -8.2361e-02 -4.0086
Slope_5    9.0419e-03  7.0074e-02 -1.2830e-01  1.4639e-01  0.1290
Tau2_2     2.2976e-03  1.4407e-03 -5.2618e-04  5.1213e-03  1.5947
Tau2_3     1.0000e-10          NA          NA          NA      NA
           Pr(>|z|)    
Intercept 0.0003631 ***
Slope_1   0.0002250 ***
Slope_2   0.4139266    
Slope_3   0.0584498 .  
Slope_4   6.108e-05 ***
Slope_5   0.8973315    
Tau2_2    0.1107693    
Tau2_3           NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0022976  0.0000
R2                     0.3948192  1.0000

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 8
Degrees of freedom: 58
-2 log likelihood: 5.076592 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Testing whether Country is significant after controlling for Type
- The test statistic was 12.5491 (df = 4), p = 0.0137. Thus, Country is significant after controlling for Type.

## Testing whether Country is significant after controlling for Type
anova(Model6, Model1)

     base            comparison ep  minus2LL df       AIC  diffLL diffdf
1 Model 6                  <NA>  8  5.076592 58 -110.9234      NA     NA
2 Model 6 Meta analysis with ML  4 17.625692 62 -106.3743 12.5491      4
           p
1         NA
2 0.01370262

Model 7: `Discipline` and `Country` as covariates

The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.1397 and 0.7126, respectively.

## Model 7: Discipline and Country as covariates
summary( meta3(logOR, v, x=cbind(DisciplinePhy, DisciplineLife, DisciplineSoc,
                         CountryAus, CountryCan, CountryEur, CountryUK), 
                         cluster=Cluster, data=Bornmann07,
                         model.name="Model 7") )


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(DisciplinePhy, 
    DisciplineLife, DisciplineSoc, CountryAus, CountryCan, CountryEur, 
    CountryUK), data = Bornmann07, model.name = "Model 7")

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)  
Intercept  0.0029305  0.0576743 -0.1101090  0.1159700  0.0508  0.95948  
Slope_1    0.1742169  0.1702554 -0.1594775  0.5079114  1.0233  0.30618  
Slope_2    0.0826806  0.1599802 -0.2308749  0.3962360  0.5168  0.60528  
Slope_3   -0.0462265  0.1715774 -0.3825119  0.2900590 -0.2694  0.78761  
Slope_4   -0.0486321  0.1306918 -0.3047834  0.2075192 -0.3721  0.70981  
Slope_5   -0.2169132  0.1915703 -0.5923841  0.1585577 -1.1323  0.25751  
Slope_6   -0.3036578  0.1670721 -0.6311130  0.0237975 -1.8175  0.06914 .
Slope_7   -0.0605272  0.1809419 -0.4151668  0.2941124 -0.3345  0.73799  
Tau2_2     0.0032661  0.0022784 -0.0011994  0.0077317  1.4335  0.15171  
Tau2_3     0.0040618  0.0038459 -0.0034759  0.0115996  1.0562  0.29090  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0032661  0.0041
R2                     0.1396974  0.7126

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 10
Degrees of freedom: 56
-2 log likelihood: 10.31105 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Model 8: `Type`, `Discipline`, and `Country` as covariates

The $R^2_{(2)}$ and $R^2_{(3)}$ were 0.4466 and 1.0000, respectively. The $\hat{\tau}^2_{(3)}$ was near 0 after controlling for the covariates.

## Model 8: Type, Discipline and Country as covariates
Model8 <- meta3(logOR, v, x=cbind(Type2, DisciplinePhy, DisciplineLife, DisciplineSoc,
                           CountryAus, CountryCan, CountryEur, CountryUK), 
                           cluster=Cluster, data=Bornmann07,
                           model.name="Model 8") 
## There was an estimation error. The model was rerun again.
summary(rerun(Model8))


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, DisciplinePhy, 
    DisciplineLife, DisciplineSoc, CountryAus, CountryCan, CountryEur, 
    CountryUK), data = Bornmann07, model.name = "Model 8")

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value
Intercept  6.8563e-02  1.8630e-02  3.2049e-02  1.0508e-01  3.6802
Slope_1   -1.6885e-01  4.1545e-02 -2.5028e-01 -8.7425e-02 -4.0643
Slope_2    2.5329e-01  1.5814e-01 -5.6671e-02  5.6325e-01  1.6016
Slope_3    1.2689e-01  1.4774e-01 -1.6268e-01  4.1646e-01  0.8589
Slope_4   -8.3549e-03  1.5796e-01 -3.1795e-01  3.0124e-01 -0.0529
Slope_5   -1.1530e-01  1.1147e-01 -3.3377e-01  1.0317e-01 -1.0344
Slope_6   -2.6412e-01  1.6402e-01 -5.8559e-01  5.7344e-02 -1.6103
Slope_7   -2.9029e-01  1.4859e-01 -5.8152e-01  9.5296e-04 -1.9536
Slope_8   -1.5975e-01  1.6285e-01 -4.7893e-01  1.5943e-01 -0.9810
Tau2_2     2.1010e-03  1.2925e-03 -4.3226e-04  4.6342e-03  1.6255
Tau2_3     1.0000e-10          NA          NA          NA      NA
           Pr(>|z|)    
Intercept  0.000233 ***
Slope_1   4.818e-05 ***
Slope_2    0.109241    
Slope_3    0.390412    
Slope_4    0.957818    
Slope_5    0.300949    
Slope_6    0.107324    
Slope_7    0.050754 .  
Slope_8    0.326610    
Tau2_2     0.104051    
Tau2_3           NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0021010  0.0000
R2                     0.4466073  1.0000

Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 11
Degrees of freedom: 55
-2 log likelihood: -1.645211 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Handling missing covariates with FIML

When there are missing data in the covariates, data with missing values are excluded from the analysis in meta3(). The missing covariates can be handled by the use of FIML in meta3X(). We illustrate two examples on how to analyze data with missing covariates with missing completely at random (MCAR) and missing at random (MAR) data.

MCAR

About 25% of the level-2 covariate Type was introduced by the MCAR mechanism.

#### Handling missing covariates with FIML
  
## MCAR
## Set seed for replication
set.seed(1000000)
  
## Copy Bornmann07 to my.df
my.df <- Bornmann07
## "Fellowship": 1; "Grant": 0
my.df$Type_MCAR <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)

## Create 17 out of 66 missingness with MCAR
my.df$Type_MCAR[sample(1:66, 17)] <- NA
  
summary(meta3(y=logOR, v=v, cluster=Cluster, x=Type_MCAR, data=my.df))


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type_MCAR, data = my.df)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value
Intercept -0.00484542  0.03934421 -0.08195865  0.07226781 -0.1232
Slope_1   -0.21090081  0.05346218 -0.31568477 -0.10611686 -3.9449
Tau2_2     0.00446788  0.00549280 -0.00629780  0.01523356  0.8134
Tau2_3     0.00092884  0.00336489 -0.00566623  0.00752391  0.2760
           Pr(>|z|)    
Intercept    0.9020    
Slope_1   7.985e-05 ***
Tau2_2       0.4160    
Tau2_3       0.7825    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 151.643
Degrees of freedom of the Q statistic: 48
P value of the Q statistic: 1.115552e-12

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0042664  0.0145
Tau2 (with predictors) 0.0044679  0.0009
R2                     0.0000000  0.9361

Number of studies (or clusters): 20
Number of observed statistics: 49
Number of estimated parameters: 4
Degrees of freedom: 45
-2 log likelihood: 13.13954 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

There is no need to specify whether the covariates are level 2 or level 3 in meta3() because the covariates are treated as a design matrix. When meta3X() is used, users need to specify whether the covariates are at level 2 (x2) or level 3 (x3).

summary(meta3X(y=logOR, v=v, cluster=Cluster, x2=Type_MCAR, data=my.df))


Call:
meta3X(y = logOR, v = v, cluster = Cluster, x2 = Type_MCAR, data = my.df)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Intercept -0.0106318  0.0397685 -0.0885766  0.0673131 -0.2673 0.789207   
SlopeX2_1 -0.1753249  0.0582619 -0.2895162 -0.0611336 -3.0093 0.002619 **
Tau2_2     0.0030338  0.0026839 -0.0022266  0.0082941  1.1304 0.258324   
Tau2_3     0.0036839  0.0042817 -0.0047082  0.0120759  0.8604 0.389586   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0037965  0.0141
Tau2 (with predictors) 0.0030338  0.0037
R2                     0.2009069  0.7394

Number of studies (or clusters): 21
Number of observed statistics: 115
Number of estimated parameters: 7
Degrees of freedom: 108
-2 log likelihood: 49.76372 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

MAR

For the case of missing covariates with MAR, the missingness in Type depends on the values of Year. Type is missing when Year is smaller than 1996.

## MAR
Type_MAR <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
  
## Create 27 out of 66 missingness with MAR for cases Year<1996
index_MAR <- ifelse(Bornmann07$Year<1996, yes=TRUE, no=FALSE)
Type_MAR[index_MAR] <- NA
  
summary(meta3(logOR, v, x=Type_MAR, cluster=Cluster, data=Bornmann07))


Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type_MAR, data = Bornmann07)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate   Std.Error      lbound      ubound z value Pr(>|z|)
Intercept -0.01587052  0.03952546 -0.09333901  0.06159796 -0.4015 0.688032
Slope_1   -0.17573648  0.06328327 -0.29976940 -0.05170355 -2.7770 0.005487
Tau2_2     0.00259266  0.00171596 -0.00077056  0.00595588  1.5109 0.130811
Tau2_3     0.00278384  0.00267150 -0.00245221  0.00801989  1.0421 0.297388
            
Intercept   
Slope_1   **
Tau2_2      
Tau2_3      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 151.11
Degrees of freedom of the Q statistic: 38
P value of the Q statistic: 1.998401e-15

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0029593  0.0097
Tau2 (with predictors) 0.0025927  0.0028
R2                     0.1238926  0.7121

Number of studies (or clusters): 12
Number of observed statistics: 39
Number of estimated parameters: 4
Degrees of freedom: 35
-2 log likelihood: -24.19956 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

It is possible to include level 2 (av2) and level 3 (av3) auxiliary variables. Auxiliary variables are those that predict the missing values or are correlated with the variables that contain missing values. The inclusion of auxiliary variables can improve the efficiency of the estimation and the parameter estimates.

## Include auxiliary variable
summary(meta3X(y=logOR, v=v, cluster=Cluster, x2=Type_MAR, av2=Year, data=my.df))


Call:
meta3X(y = logOR, v = v, cluster = Cluster, x2 = Type_MAR, av2 = Year, 
    data = my.df)

95% confidence intervals: z statistic approximation
Coefficients:
            Estimate  Std.Error     lbound     ubound z value Pr(>|z|)   
Intercept -0.0264058  0.0572056 -0.1385266  0.0857151 -0.4616 0.644372   
SlopeX2_1 -0.2003999  0.0691093 -0.3358517 -0.0649481 -2.8998 0.003735 **
Tau2_2     0.0029970  0.0022371 -0.0013877  0.0073817  1.3396 0.180359   
Tau2_3     0.0030212  0.0032463 -0.0033415  0.0093839  0.9307 0.352033   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Explained variances (R2):
                         Level 2 Level 3
Tau2 (no predictor)    0.0049237  0.0088
Tau2 (with predictors) 0.0029970  0.0030
R2                     0.3913243  0.6571

Number of studies (or clusters): 21
Number of observed statistics: 171
Number of estimated parameters: 14
Degrees of freedom: 157
-2 log likelihood: 377.3479 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Two-stage structural equation modeling (TSSEM)

Inspect the data: Digman (1997)

The correlation matrices and the sample sizes were stored in Digman97$data and Digman97$n, respectively. We may list the first few cases of the data by using the head() command.

#### Load the metaSEM library for TSSEM
library(metaSEM)
  
#### Inspect the data for inspection
head(Digman97$data)

$`Digman 1 (1994)`
       A     C   ES     E    I
A   1.00  0.62 0.41 -0.48 0.00
C   0.62  1.00 0.59 -0.10 0.35
ES  0.41  0.59 1.00  0.27 0.41
E  -0.48 -0.10 0.27  1.00 0.37
I   0.00  0.35 0.41  0.37 1.00

$`Digman 2 (1994)`
       A    C   ES     E     I
A   1.00 0.39 0.53 -0.30 -0.05
C   0.39 1.00 0.59  0.07  0.44
ES  0.53 0.59 1.00  0.09  0.22
E  -0.30 0.07 0.09  1.00  0.45
I  -0.05 0.44 0.22  0.45  1.00

$`Digman 3 (1963c)`
      A     C   ES     E    I
A  1.00  0.65 0.35  0.25 0.14
C  0.65  1.00 0.37 -0.10 0.33
ES 0.35  0.37 1.00  0.24 0.41
E  0.25 -0.10 0.24  1.00 0.41
I  0.14  0.33 0.41  0.41 1.00

$`Digman & Takemoto-Chock (1981b)`
       A     C   ES     E     I
A   1.00  0.65 0.70 -0.26 -0.03
C   0.65  1.00 0.71 -0.16  0.24
ES  0.70  0.71 1.00  0.01  0.11
E  -0.26 -0.16 0.01  1.00  0.66
I  -0.03  0.24 0.11  0.66  1.00

$`Graziano & Ward (1992)`
      A    C   ES    E    I
A  1.00 0.64 0.35 0.29 0.22
C  0.64 1.00 0.27 0.16 0.22
ES 0.35 0.27 1.00 0.32 0.36
E  0.29 0.16 0.32 1.00 0.53
I  0.22 0.22 0.36 0.53 1.00

$`Yik & Bond (1993)`
      A    C   ES    E    I
A  1.00 0.66 0.57 0.35 0.38
C  0.66 1.00 0.45 0.20 0.31
ES 0.57 0.45 1.00 0.49 0.31
E  0.35 0.20 0.49 1.00 0.59
I  0.38 0.31 0.31 0.59 1.00

head(Digman97$n)

[1] 102 149 334 162  91 656

Fixed-effects TSSEM

Stage 1 analysis

To conduct a fixed-effects TSSEM, we may specify method=FEM argument (the default method) in calling the tssem1() function. The results are stored in an object named fixed1. It can be displayed by the summary() command. The $\chi^2(130, N=4,496) = 1,499.73, p < .001$, CFI=0.6825, RMSEA=0.1812 and SRMR=0.1750. Based on the test statistic and the goodness-of-fit indices, the assumption of homogeneity of correlation matrices was rejected.

## Fixed-effects model: Stage 1 analysis
fixed1 <- tssem1(Cov=Digman97$data, n=Digman97$n, method="FEM")
summary(fixed1)


Call:
tssem1FEM(Cov = Cov, n = n, cor.analysis = cor.analysis, model.name = model.name, 
    cluster = cluster, suppressWarnings = suppressWarnings, silent = silent, 
    run = run)

Coefficients:
       Estimate Std.Error z value  Pr(>|z|)    
S[1,2] 0.363278  0.013368 27.1760 < 2.2e-16 ***
S[1,3] 0.390375  0.012880 30.3076 < 2.2e-16 ***
S[1,4] 0.103572  0.015048  6.8830 5.862e-12 ***
S[1,5] 0.092286  0.015047  6.1330 8.621e-10 ***
S[2,3] 0.416070  0.012519 33.2344 < 2.2e-16 ***
S[2,4] 0.135148  0.014776  9.1464 < 2.2e-16 ***
S[2,5] 0.141431  0.014866  9.5135 < 2.2e-16 ***
S[3,4] 0.244459  0.014153 17.2723 < 2.2e-16 ***
S[3,5] 0.138339  0.014834  9.3259 < 2.2e-16 ***
S[4,5] 0.424566  0.012376 34.3070 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                     Value
Sample size                      4496.0000
Chi-square of target model       1505.4443
DF of target model                130.0000
p value of target model             0.0000
Chi-square of independence model 4471.4242
DF of independence model          140.0000
RMSEA                               0.1815
RMSEA lower 95% CI                  0.1736
RMSEA upper 95% CI                  0.1901
SRMR                                0.1621
TLI                                 0.6580
CFI                                 0.6824
AIC                              1245.4443
BIC                               412.0217
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

The pooled correlation matrix (the parameter estimates) can be extracted by the use of the coef() command.

coef(fixed1)

            A         C        ES         E          I
A  1.00000000 0.3632782 0.3903748 0.1035716 0.09228557
C  0.36327824 1.0000000 0.4160695 0.1351477 0.14143058
ES 0.39037483 0.4160695 1.0000000 0.2444593 0.13833895
E  0.10357155 0.1351477 0.2444593 1.0000000 0.42456626
I  0.09228557 0.1414306 0.1383390 0.4245663 1.00000000

Stage 2 analysis

As an illustration, I continued to fit the structural model based on the pooled correlation matrix. We may specify the two-factor model with the RAM formulation

## Factor covariance among latent factors
Phi <- matrix(c(1,"0.3*cor","0.3*cor",1), ncol=2, nrow=2)

## Error covariance matrix
Psi <- Diag(c("0.2*e1","0.2*e2","0.2*e3","0.2*e4","0.2*e5"))

## S matrix
S1 <- bdiagMat(list(Psi, Phi))

## This step is not necessary, but it is useful for inspecting the model.
dimnames(S1)[[1]] <- dimnames(S1)[[2]] <- c("A","C","ES","E","I","Alpha","Beta") 

## Display S1
S1

      A        C        ES       E        I        Alpha     Beta     
A     "0.2*e1" "0"      "0"      "0"      "0"      "0"       "0"      
C     "0"      "0.2*e2" "0"      "0"      "0"      "0"       "0"      
ES    "0"      "0"      "0.2*e3" "0"      "0"      "0"       "0"      
E     "0"      "0"      "0"      "0.2*e4" "0"      "0"       "0"      
I     "0"      "0"      "0"      "0"      "0.2*e5" "0"       "0"      
Alpha "0"      "0"      "0"      "0"      "0"      "1"       "0.3*cor"
Beta  "0"      "0"      "0"      "0"      "0"      "0.3*cor" "1"

## A matrix
Lambda <-
matrix(c(".3*Alpha_A",".3*Alpha_C",".3*Alpha_ES",rep(0,5),".3*Beta_E",".3*Beta_I"),
       ncol=2, nrow=5)
A1 <- rbind( cbind(matrix(0,ncol=5,nrow=5), Lambda),
             matrix(0, ncol=7, nrow=2) )

## This step is not necessary, but it is useful for inspecting the model.
dimnames(A1)[[1]] <- dimnames(A1)[[2]] <- c("A","C","ES","E","I","Alpha","Beta") 

## Display A1
A1

      A   C   ES  E   I   Alpha         Beta       
A     "0" "0" "0" "0" "0" ".3*Alpha_A"  "0"        
C     "0" "0" "0" "0" "0" ".3*Alpha_C"  "0"        
ES    "0" "0" "0" "0" "0" ".3*Alpha_ES" "0"        
E     "0" "0" "0" "0" "0" "0"           ".3*Beta_E"
I     "0" "0" "0" "0" "0" "0"           ".3*Beta_I"
Alpha "0" "0" "0" "0" "0" "0"           "0"        
Beta  "0" "0" "0" "0" "0" "0"           "0"

## F matrix to select the observed variables
F1 <- create.Fmatrix(c(1,1,1,1,1,0,0), as.mxMatrix=FALSE)

## Display F1
F1

     [,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    1    0    0    0    0    0    0
[2,]    0    1    0    0    0    0    0
[3,]    0    0    1    0    0    0    0
[4,]    0    0    0    1    0    0    0
[5,]    0    0    0    0    1    0    0

We may also specify the model using lavann’s syntax.

model1 <- "## Factor loadings
           Alpha=~A+C+ES
           Beta=~E+I
           ## Factor correlation
           Alpha~~Beta"

RAM1 <- lavaan2RAM(model1, obs.variables=c("A","C","ES","E","I"),
                   A.notation="on", S.notation="with")
RAM1

$A
      A   C   ES  E   I   Alpha         Beta       
A     "0" "0" "0" "0" "0" "0*AonAlpha"  "0"        
C     "0" "0" "0" "0" "0" "0*ConAlpha"  "0"        
ES    "0" "0" "0" "0" "0" "0*ESonAlpha" "0"        
E     "0" "0" "0" "0" "0" "0"           "0*EonBeta"
I     "0" "0" "0" "0" "0" "0"           "0*IonBeta"
Alpha "0" "0" "0" "0" "0" "0"           "0"        
Beta  "0" "0" "0" "0" "0" "0"           "0"        

$S
      A          C          ES           E          I         
A     "0*AwithA" "0"        "0"          "0"        "0"       
C     "0"        "0*CwithC" "0"          "0"        "0"       
ES    "0"        "0"        "0*ESwithES" "0"        "0"       
E     "0"        "0"        "0"          "0*EwithE" "0"       
I     "0"        "0"        "0"          "0"        "0*IwithI"
Alpha "0"        "0"        "0"          "0"        "0"       
Beta  "0"        "0"        "0"          "0"        "0"       
      Alpha             Beta             
A     "0"               "0"              
C     "0"               "0"              
ES    "0"               "0"              
E     "0"               "0"              
I     "0"               "0"              
Alpha "1"               "0*AlphawithBeta"
Beta  "0*AlphawithBeta" "1"              

$F
   A C ES E I Alpha Beta
A  1 0  0 0 0     0    0
C  0 1  0 0 0     0    0
ES 0 0  1 0 0     0    0
E  0 0  0 1 0     0    0
I  0 0  0 0 1     0    0

$M
  A C ES E I Alpha Beta
1 0 0  0 0 0     0    0

A1 <- RAM1$A
S1 <- RAM1$S
F1 <- RAM1$F

Since we are conducting a correlation structure analysis, the error variances are not free parameters. The chi-square test statistic of the Stage 2 analysis was $\chi^2(4, N=4,496) = 65.45, p < .001$, CFI=0.9802, RMSEA=0.0585 and SRMR=0.0284.

fixed2 <- tssem2(fixed1, Amatrix=A1, Smatrix=S1, Fmatrix=F1, 
                 diag.constraints=FALSE, intervals="LB",
                 model.name="TSSEM2 Digman97")
summary(fixed2)


Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
              Estimate Std.Error  lbound  ubound z value Pr(>|z|)
AonAlpha       0.56280        NA 0.53270 0.59302      NA       NA
ConAlpha       0.60522        NA 0.57525 0.63535      NA       NA
EonBeta        0.78141        NA 0.71872 0.85498      NA       NA
ESonAlpha      0.71920        NA 0.68876 0.75032      NA       NA
IonBeta        0.55137        NA 0.50000 0.60270      NA       NA
AlphawithBeta  0.36268        NA 0.31859 0.40646      NA       NA

Goodness-of-fit indices:
                                               Value
Sample size                                4496.0000
Chi-square of target model                   65.4526
DF of target model                            4.0000
p value of target model                       0.0000
Number of constraints imposed on "Smatrix"    0.0000
DF manually adjusted                          0.0000
Chi-square of independence model           3112.7228
DF of independence model                     10.0000
RMSEA                                         0.0585
RMSEA lower 95% CI                            0.0465
RMSEA upper 95% CI                            0.0713
SRMR                                          0.0284
TLI                                           0.9505
CFI                                           0.9802
AIC                                          57.4526
BIC                                          31.8088
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

plot(fixed2)

Fixed-effects TSSEM with cluster

Stage 1 analysis

There are 4 types of sample characteristics in the original cluster. We may group them into either younger or older sample.

#### Display the frequencies of "cluster"
table(Digman97$cluster)


  Adolescents      Children Mature adults  Young adults 
            1             4             6             3

#### Fixed-effects TSSEM with several clusters
#### Create a variable for different cluster
#### Younger participants: Children and Adolescents
#### Older participants: others
clusters <- ifelse(Digman97$cluster %in% c("Children","Adolescents"),
                   yes="Younger participants", no="Older participants")
  
#### Show the clusters
clusters

 [1] "Younger participants" "Younger participants" "Younger participants"
 [4] "Younger participants" "Younger participants" "Older participants"  
 [7] "Older participants"   "Older participants"   "Older participants"  
[10] "Older participants"   "Older participants"   "Older participants"  
[13] "Older participants"   "Older participants"

We may conduct a fixed-effects model by specifying the cluster=clusters argument. Fixed-effects TSSEM will be conducted according to the labels in the clusters. The goodness-of-fit indices of the Stage 1 analysis for the older and younger participants were $\chi^2(80, N=3,658) = 823.88, p < .001$, CFI=0.7437, RMSEA=0.1513 and SRMR=0.1528, and $\chi^2(40, N=838) = 344.18, p < .001$, CFI=0.7845, RMSEA=0.2131 and SRMR=0.1508, respectively.

## Example of Fixed-effects TSSEM with several clusters
cluster1 <- tssem1(Digman97$data, Digman97$n, method="FEM", 
                   cluster=clusters)
summary(cluster1)

$`Older participants`

Call:
tssem1FEM(Cov = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis, 
    model.name = model.name, suppressWarnings = suppressWarnings)

Coefficients:
       Estimate Std.Error z value  Pr(>|z|)    
S[1,2] 0.297471  0.015436 19.2716 < 2.2e-16 ***
S[1,3] 0.370248  0.014532 25.4774 < 2.2e-16 ***
S[1,4] 0.137694  0.016403  8.3944 < 2.2e-16 ***
S[1,5] 0.098061  0.016724  5.8637 4.528e-09 ***
S[2,3] 0.393692  0.014146 27.8307 < 2.2e-16 ***
S[2,4] 0.183045  0.016055 11.4009 < 2.2e-16 ***
S[2,5] 0.092774  0.016643  5.5743 2.485e-08 ***
S[3,4] 0.260753  0.015554 16.7645 < 2.2e-16 ***
S[3,5] 0.096141  0.016573  5.8009 6.596e-09 ***
S[4,5] 0.411756  0.013900 29.6225 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                     Value
Sample size                      3658.0000
Chi-square of target model        825.9826
DF of target model                 80.0000
p value of target model             0.0000
Chi-square of independence model 3000.9731
DF of independence model           90.0000
RMSEA                               0.1515
RMSEA lower 95% CI                  0.1424
RMSEA upper 95% CI                  0.1611
SRMR                                0.1459
TLI                                 0.7117
CFI                                 0.7437
AIC                               665.9826
BIC                               169.6088
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

$`Younger participants`

Call:
tssem1FEM(Cov = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis, 
    model.name = model.name, suppressWarnings = suppressWarnings)

Coefficients:
        Estimate Std.Error z value  Pr(>|z|)    
S[1,2]  0.604330  0.022125 27.3141 < 2.2e-16 ***
S[1,3]  0.465536  0.027493 16.9327 < 2.2e-16 ***
S[1,4] -0.031381  0.035940 -0.8732   0.38258    
S[1,5]  0.061508  0.034547  1.7804   0.07500 .  
S[2,3]  0.501432  0.026348 19.0311 < 2.2e-16 ***
S[2,4] -0.060678  0.034557 -1.7559   0.07911 .  
S[2,5]  0.320987  0.031064 10.3330 < 2.2e-16 ***
S[3,4]  0.175437  0.033675  5.2097 1.891e-07 ***
S[3,5]  0.305149  0.031586  9.6609 < 2.2e-16 ***
S[4,5]  0.478640  0.026883 17.8045 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                     Value
Sample size                       838.0000
Chi-square of target model        346.2810
DF of target model                 40.0000
p value of target model             0.0000
Chi-square of independence model 1470.4511
DF of independence model           50.0000
RMSEA                               0.2139
RMSEA lower 95% CI                  0.1939
RMSEA upper 95% CI                  0.2355
SRMR                                0.1411
TLI                                 0.7305
CFI                                 0.7844
AIC                               266.2810
BIC                                77.0402
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

The pooled correlation matrix for each cluster can be extracted by the use of the coef() command.

coef(cluster1)

$`Older participants`
            A          C         ES         E          I
A  1.00000000 0.29747123 0.37024803 0.1376942 0.09806125
C  0.29747123 1.00000000 0.39369157 0.1830450 0.09277411
ES 0.37024803 0.39369157 1.00000000 0.2607530 0.09614072
E  0.13769424 0.18304500 0.26075304 1.0000000 0.41175622
I  0.09806125 0.09277411 0.09614072 0.4117562 1.00000000

$`Younger participants`
             A          C        ES          E          I
A   1.00000000  0.6043300 0.4655359 -0.0313810 0.06150839
C   0.60433002  1.0000000 0.5014319 -0.0606784 0.32098713
ES  0.46553592  0.5014319 1.0000000  0.1754367 0.30514853
E  -0.03138100 -0.0606784 0.1754367  1.0000000 0.47864004
I   0.06150839  0.3209871 0.3051485  0.4786400 1.00000000

Stage 2 analysis

The goodness-of-fit indices of the Stage 2 analysis for the older and younger participants were $\chi^2(4, N=3,658) = 21.92, p < .001$, CFI=0.9921, RMSEA=0.0350 and SRMR=0.0160, and $\chi^2(4, N=838) = 144.87, p < .001$, CFI=0.9427, RMSEA=0.2051 and SRMR=0.1051, respectively.

cluster2 <- tssem2(cluster1, Amatrix=A1, Smatrix=S1, Fmatrix=F1, 
                   diag.constraints=FALSE, intervals.type="z")
#### Please note that the estimates for the younger participants are problematic.
summary(cluster2)

$`Older participants`

Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: z statistic approximation
Coefficients:
              Estimate Std.Error   lbound   ubound z value  Pr(>|z|)    
AonAlpha      0.512656  0.018206 0.476973 0.548340  28.158 < 2.2e-16 ***
ConAlpha      0.549967  0.017945 0.514795 0.585140  30.647 < 2.2e-16 ***
EonBeta       0.967136  0.058751 0.851986 1.082287  16.462 < 2.2e-16 ***
ESonAlpha     0.732173  0.018929 0.695074 0.769273  38.680 < 2.2e-16 ***
IonBeta       0.430649  0.029634 0.372568 0.488730  14.532 < 2.2e-16 ***
AlphawithBeta 0.349236  0.028118 0.294125 0.404346  12.420 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                               Value
Sample size                                3658.0000
Chi-square of target model                   21.9954
DF of target model                            4.0000
p value of target model                       0.0002
Number of constraints imposed on "Smatrix"    0.0000
DF manually adjusted                          0.0000
Chi-square of independence model           2273.3313
DF of independence model                     10.0000
RMSEA                                         0.0351
RMSEA lower 95% CI                            0.0217
RMSEA upper 95% CI                            0.0500
SRMR                                          0.0160
TLI                                           0.9801
CFI                                           0.9920
AIC                                          13.9954
BIC                                         -10.8233
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

$`Younger participants`

Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: z statistic approximation
Coefficients:
               Estimate Std.Error    lbound    ubound z value Pr(>|z|)    
AonAlpha       0.747647  0.023880  0.700842  0.794451 31.3081   <2e-16 ***
ConAlpha       0.911705  0.019864  0.872772  0.950638 45.8967   <2e-16 ***
EonBeta        0.152564  0.159168 -0.159400  0.464528  0.9585   0.3378    
ESonAlpha      0.677435  0.025864  0.626742  0.728128 26.1919   <2e-16 ***
IonBeta        3.283814  3.364059 -3.309620  9.877248  0.9761   0.3290    
AlphawithBeta  0.117258  0.125453 -0.128626  0.363141  0.9347   0.3500    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                               Value
Sample size                                 838.0000
Chi-square of target model                  145.6167
DF of target model                            4.0000
p value of target model                       0.0000
Number of constraints imposed on "Smatrix"    0.0000
DF manually adjusted                          0.0000
Chi-square of independence model           2480.2324
DF of independence model                     10.0000
RMSEA                                         0.2057
RMSEA lower 95% CI                            0.1778
RMSEA upper 95% CI                            0.2350
SRMR                                          0.1051
TLI                                           0.8567
CFI                                           0.9427
AIC                                         137.6167
BIC                                         118.6926
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

We may also plot the parameter estimates of these two groups.

library(semPlot)

## Convert the model to semPlotModel object with 2 plots
my.plots <- lapply(X=cluster2, FUN=meta2semPlot, latNames=c("Alpha","Beta"))

## Setup two plots
layout(t(1:2))
semPaths(my.plots[[1]], whatLabels="est", nCharNodes=10, color="green")
title("Younger") 
semPaths(my.plots[[2]], whatLabels="est", nCharNodes=10, color="yellow")
title("Older")

Random-effects TSSEM

Stage 1 analysis

Since there is not enough data to estimate the full variance component of the random effects, I estimate the variance component with a diagonal matrix in the RE.type="Diag" argument. The range of $I^2$ indices, the percentage of total variance that can be explained by the between study effect, are from .84 to .95.

#### Random-effects TSSEM with random effects on the diagonals
random1 <- tssem1(Digman97$data, Digman97$n, method="REM", RE.type="Diag")
summary(random1)


Call:
meta(y = ES, v = acovR, RE.constraints = Diag(paste0(RE.startvalues, 
    "*Tau2_", 1:no.es, "_", 1:no.es)), RE.lbound = RE.lbound, 
    I2 = I2, model.name = model.name, suppressWarnings = TRUE, 
    silent = silent, run = run)

95% confidence intervals: z statistic approximation
Coefficients:
               Estimate   Std.Error      lbound      ubound z value
Intercept1   0.38971908  0.05429256  0.28330762  0.49613054  7.1781
Intercept2   0.43265881  0.04145135  0.35141566  0.51390196 10.4377
Intercept3   0.04945628  0.06071078 -0.06953466  0.16844722  0.8146
Intercept4   0.09603708  0.04478710  0.00825598  0.18381818  2.1443
Intercept5   0.42724246  0.03911620  0.35057612  0.50390880 10.9224
Intercept6   0.11929317  0.04106204  0.03881306  0.19977328  2.9052
Intercept7   0.19292425  0.04757961  0.09966993  0.28617856  4.0548
Intercept8   0.22690163  0.03240893  0.16338130  0.29042196  7.0012
Intercept9   0.18105567  0.04258854  0.09758366  0.26452768  4.2513
Intercept10  0.43614968  0.03205960  0.37331403  0.49898533 13.6043
Tau2_1_1     0.03648372  0.01513055  0.00682838  0.06613905  2.4113
Tau2_2_2     0.01963096  0.00859789  0.00277942  0.03648251  2.2832
Tau2_3_3     0.04571435  0.01952284  0.00745029  0.08397841  2.3416
Tau2_4_4     0.02236119  0.00995083  0.00285793  0.04186445  2.2472
Tau2_5_5     0.01729071  0.00796404  0.00168149  0.03289994  2.1711
Tau2_6_6     0.01815481  0.00895896  0.00059557  0.03571406  2.0264
Tau2_7_7     0.02604880  0.01130265  0.00389600  0.04820159  2.3047
Tau2_8_8     0.00988761  0.00513713 -0.00018098  0.01995619  1.9247
Tau2_9_9     0.01988242  0.00895052  0.00233972  0.03742512  2.2214
Tau2_10_10   0.01049221  0.00501578  0.00066147  0.02032295  2.0918
             Pr(>|z|)    
Intercept1  7.068e-13 ***
Intercept2  < 2.2e-16 ***
Intercept3    0.41529    
Intercept4    0.03201 *  
Intercept5  < 2.2e-16 ***
Intercept6    0.00367 ** 
Intercept7  5.018e-05 ***
Intercept8  2.538e-12 ***
Intercept9  2.126e-05 ***
Intercept10 < 2.2e-16 ***
Tau2_1_1      0.01590 *  
Tau2_2_2      0.02242 *  
Tau2_3_3      0.01920 *  
Tau2_4_4      0.02463 *  
Tau2_5_5      0.02992 *  
Tau2_6_6      0.04272 *  
Tau2_7_7      0.02119 *  
Tau2_8_8      0.05426 .  
Tau2_9_9      0.02633 *  
Tau2_10_10    0.03645 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 1220.33
Degrees of freedom of the Q statistic: 130
P value of the Q statistic: 0

Heterogeneity indices (based on the estimated Tau2):
                              Estimate
Intercept1: I2 (Q statistic)    0.9326
Intercept2: I2 (Q statistic)    0.8872
Intercept3: I2 (Q statistic)    0.9325
Intercept4: I2 (Q statistic)    0.8703
Intercept5: I2 (Q statistic)    0.8797
Intercept6: I2 (Q statistic)    0.8480
Intercept7: I2 (Q statistic)    0.8887
Intercept8: I2 (Q statistic)    0.7669
Intercept9: I2 (Q statistic)    0.8590
Intercept10: I2 (Q statistic)   0.8193

Number of studies (or clusters): 14
Number of observed statistics: 140
Number of estimated parameters: 20
Degrees of freedom: 120
-2 log likelihood: -112.4196 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

The pooled correlation coefficients (fixed effects) and the variance components (the random effects) can be extracted by the use of the coef() command via the select argument.

## Fixed effects
coef(random1, select="fixed")

 Intercept1  Intercept2  Intercept3  Intercept4  Intercept5  Intercept6 
 0.38971908  0.43265881  0.04945628  0.09603708  0.42724246  0.11929317 
 Intercept7  Intercept8  Intercept9 Intercept10 
 0.19292425  0.22690163  0.18105567  0.43614968

## Random effects
coef(random1, select="random")

   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.036483716 0.019630961 0.045714353 0.022361193 0.017290714 0.018154815 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.026048796 0.009887608 0.019882422 0.010492210

Stage 2 analysis

The steps are exactly the same as those in the fixed-effects model. The chi-square test statistic of the Stage 2 analysis was $\chi^2(4, N=4,496) = 8.51, p < .001$, CFI=0.9911, RMSEA=0.0158 and SRMR=0.0463.

random2 <- tssem2(random1, Amatrix=A1, Smatrix=S1, Fmatrix=F1, 
                  diag.constraints=FALSE, intervals="LB")
summary(random2)


Call:
wls(Cov = pooledS, aCov = aCov, n = tssem1.obj$total.n, Amatrix = Amatrix, 
    Smatrix = Smatrix, Fmatrix = Fmatrix, diag.constraints = diag.constraints, 
    cor.analysis = cor.analysis, intervals.type = intervals.type, 
    mx.algebras = mx.algebras, model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
              Estimate Std.Error  lbound  ubound z value Pr(>|z|)
AonAlpha       0.56944        NA 0.46903 0.67529      NA       NA
ConAlpha       0.59063        NA 0.48968 0.69723      NA       NA
EonBeta        0.67996        NA 0.54690 0.72608      NA       NA
ESonAlpha      0.76045        NA 0.64855 0.89661      NA       NA
IonBeta        0.64184        NA 0.50459 0.70619      NA       NA
AlphawithBeta  0.37769        NA 0.28683 0.47386      NA       NA

Goodness-of-fit indices:
                                               Value
Sample size                                4496.0000
Chi-square of target model                    7.8204
DF of target model                            4.0000
p value of target model                       0.0984
Number of constraints imposed on "Smatrix"    0.0000
DF manually adjusted                          0.0000
Chi-square of independence model            501.6767
DF of independence model                     10.0000
RMSEA                                         0.0146
RMSEA lower 95% CI                            0.0000
RMSEA upper 95% CI                            0.0297
SRMR                                          0.0436
TLI                                           0.9806
CFI                                           0.9922
AIC                                          -0.1796
BIC                                         -25.8234
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

## Plot the parameter estimates
plot(random2, color="green")

Becker and Schram (1994)

Inspect the data

We may list the first few cases of the data by using the head() command.

#### Load the metaSEM library for TSSEM
library(metaSEM)
  
#### Inspect the data for inspection (not required for the analysis)
head(Becker94$data)

$`Becker (1978) Females`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.47      -0.21
Spatial        0.47    1.00      -0.15
SAT_Verbal    -0.21   -0.15       1.00

$`Becker (1978) Males`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.28       0.19
Spatial        0.28    1.00       0.18
SAT_Verbal     0.19    0.18       1.00

$`Berry (1957) Females`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.48       0.41
Spatial        0.48    1.00       0.26
SAT_Verbal     0.41    0.26       1.00

$`Berry (1957) Males`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.37       0.40
Spatial        0.37    1.00       0.27
SAT_Verbal     0.40    0.27       1.00

$`Rosenberg (1981) Females`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.42       0.48
Spatial        0.42    1.00       0.23
SAT_Verbal     0.48    0.23       1.00

$`Rosenberg (1981) Males`
           SAT_Math Spatial SAT_Verbal
SAT_Math       1.00    0.41       0.74
Spatial        0.41    1.00       0.44
SAT_Verbal     0.74    0.44       1.00

head(Becker94$n)

[1]  74 153  48  55  51  18

Fixed-effects TSSEM

Stage 1 analysis

The test statistic of the Stage 1 analysis based on the TSSEM approach was $\chi^2(27, N=538) = 62.50, p < .001$, CFI=0.7943, RMSEA=0.1565 and SRMR=0.2011. Based on the test statistic and the goodness-of-fit indices, the hypothesis of homogeneity of correlation matrices was rejected.

#### Fixed-effects model
## Stage 1 analysis
fixed1 <- tssem1(Becker94$data, Becker94$n, method="FEM")
summary(fixed1)


Call:
tssem1FEM(Cov = Cov, n = n, cor.analysis = cor.analysis, model.name = model.name, 
    cluster = cluster, suppressWarnings = suppressWarnings, silent = silent, 
    run = run)

Coefficients:
       Estimate Std.Error z value  Pr(>|z|)    
S[1,2] 0.379961  0.037123 10.2351 < 2.2e-16 ***
S[1,3] 0.334562  0.039947  8.3751 < 2.2e-16 ***
S[2,3] 0.176461  0.042334  4.1683 3.069e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                     Value
Sample size                       538.0000
Chi-square of target model         63.6553
DF of target model                 27.0000
p value of target model             0.0001
Chi-square of independence model  207.7894
DF of independence model           30.0000
RMSEA                               0.1590
RMSEA lower 95% CI                  0.1096
RMSEA upper 95% CI                  0.2117
SRMR                                0.1586
TLI                                 0.7709
CFI                                 0.7938
AIC                                 9.6553
BIC                              -106.1169
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

coef(fixed1)

            SAT_Math   Spatial SAT_Verbal
SAT_Math   1.0000000 0.3799605   0.334562
Spatial    0.3799605 1.0000000   0.176461
SAT_Verbal 0.3345620 0.1764610   1.000000

Stage 2 analysis

We may specify the model via the RAM formulation. If all variables are observed, there is no need to specify the F matrix. Since the df of the regression model is 0, the proposed model always fits the data perfectly.

#### Prepare models for stage 2 analysis
## A2: asymmetric matrix (regression coefficients)
A2 <- create.mxMatrix(c(0,"0.2*Spatial2Math","0.2*Verbal2Math",
                        0,0,0,
                        0,0,0), 
                        type="Full", ncol=3, nrow=3, as.mxMatrix=FALSE, byrow=TRUE)
dimnames(A2) <- list( c("Math", "Spatial", "Verbal"), c("Math", "Spatial", "Verbal") )
A2

        Math Spatial            Verbal           
Math    "0"  "0.2*Spatial2Math" "0.2*Verbal2Math"
Spatial "0"  "0"                "0"              
Verbal  "0"  "0"                "0"

## S2: symmetric matrix (variance covariance matrix among variables)
S2 <- create.mxMatrix(c("0.2*ErrorVarMath",0,0,1,"0.2*SpatialCorVerbal",1), 
                        type="Symm", as.mxMatrix=FALSE)
dimnames(S2) <- list( c("Math", "Spatial", "Verbal"), c("Math", "Spatial", "Verbal") )
S2

        Math               Spatial                Verbal                
Math    "0.2*ErrorVarMath" "0"                    "0"                   
Spatial "0"                "1"                    "0.2*SpatialCorVerbal"
Verbal  "0"                "0.2*SpatialCorVerbal" "1"

## Using lavaan's syntax
model2 <- "## Math is modeled by Spatial and Verbal
           Math ~ Spatial2Math*Spatial + Verbal2Math*Verbal
           ## Variances of predictors are fixed at 1
           Spatial ~~ 1*Spatial
           Verbal ~~ 1*Verbal
           ## Correlation between the predictors
           Spatial ~~ SpatialCorVerbal*Verbal
           ## Error variance
           Math ~~ ErrorVarMath*Math"

RAM2 <- lavaan2RAM(model2)
RAM2

$A
        Math Spatial          Verbal         
Math    "0"  "0*Spatial2Math" "0*Verbal2Math"
Spatial "0"  "0"              "0"            
Verbal  "0"  "0"              "0"            

$S
        Math             Spatial              Verbal              
Math    "0*ErrorVarMath" "0"                  "0"                 
Spatial "0"              "1"                  "0*SpatialCorVerbal"
Verbal  "0"              "0*SpatialCorVerbal" "1"                 

$F
        Math Spatial Verbal
Math       1       0      0
Spatial    0       1      0
Verbal     0       0      1

$M
  Math Spatial Verbal
1    0       0      0

## Stage 2 analysis
fixed2 <- tssem2(fixed1, Amatrix=RAM2$A, Smatrix=RAM2$S, diag.constraints=TRUE, 
                 intervals="LB")
summary(fixed2)


Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
                 Estimate Std.Error   lbound   ubound z value Pr(>|z|)
Spatial2Math     0.331238        NA 0.258216 0.404145      NA       NA
Verbal2Math      0.276111        NA 0.199501 0.352776      NA       NA
ErrorVarMath     0.781766        NA 0.714803 0.840626      NA       NA
SpatialCorVerbal 0.176461        NA 0.093482 0.259434      NA       NA

Goodness-of-fit indices:
                                            Value
Sample size                                538.00
Chi-square of target model                   0.00
DF of target model                           0.00
p value of target model                      0.00
Number of constraints imposed on "Smatrix"   1.00
DF manually adjusted                         0.00
Chi-square of independence model           160.47
DF of independence model                     3.00
RMSEA                                        0.00
RMSEA lower 95% CI                           0.00
RMSEA upper 95% CI                           0.00
SRMR                                         0.00
TLI                                          -Inf
CFI                                          1.00
AIC                                          0.00
BIC                                          0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

plot(fixed2)

Fixed-effects TSSEM with cluster

Stage 1 analysis

The goodness-of-fit indices of the Stage 1 analysis for the female and male participants were $\chi^2(12, N=235) = 42.41, p < .001$, CFI=0.7116, RMSEA=0.2327 and SRMR=0.2339, and $\chi^2(12, N=303) = 16.13, p = .1852$, CFI=0.9385, RMSEA=0.0755 and SRMR=0.1508, respectively.

#### Fixed-effects model with cluster
## Stage 1 analysis
cluster1 <- tssem1(Becker94$data, Becker94$n, method="FEM", cluster=Becker94$gender)
summary(cluster1)

$Females

Call:
tssem1FEM(Cov = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis, 
    model.name = model.name, suppressWarnings = suppressWarnings)

Coefficients:
       Estimate Std.Error z value  Pr(>|z|)    
S[1,2] 0.455896  0.051993  8.7685 < 2.2e-16 ***
S[1,3] 0.341583  0.061943  5.5144 3.499e-08 ***
S[2,3] 0.171931  0.064731  2.6561  0.007905 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                    Value
Sample size                      235.0000
Chi-square of target model        43.1898
DF of target model                12.0000
p value of target model            0.0000
Chi-square of independence model 123.4399
DF of independence model          15.0000
RMSEA                              0.2357
RMSEA lower 95% CI                 0.1637
RMSEA upper 95% CI                 0.3161
SRMR                               0.2141
TLI                                0.6405
CFI                                0.7124
AIC                               19.1898
BIC                              -22.3252
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

$Males

Call:
tssem1FEM(Cov = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis, 
    model.name = model.name, suppressWarnings = suppressWarnings)

Coefficients:
       Estimate Std.Error z value  Pr(>|z|)    
S[1,2] 0.318051  0.051698  6.1521 7.646e-10 ***
S[1,3] 0.328286  0.052226  6.2858 3.261e-10 ***
S[2,3] 0.179549  0.055944  3.2094   0.00133 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Goodness-of-fit indices:
                                    Value
Sample size                      303.0000
Chi-square of target model        16.4819
DF of target model                12.0000
p value of target model            0.1701
Chi-square of independence model  84.3496
DF of independence model          15.0000
RMSEA                              0.0786
RMSEA lower 95% CI                 0.0000
RMSEA upper 95% CI                 0.1643
SRMR                               0.1025
TLI                                0.9192
CFI                                0.9354
AIC                               -7.5181
BIC                              -52.0829
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

coef(cluster1)

$Females
            SAT_Math   Spatial SAT_Verbal
SAT_Math   1.0000000 0.4558958  0.3415826
Spatial    0.4558958 1.0000000  0.1719309
SAT_Verbal 0.3415826 0.1719309  1.0000000

$Males
            SAT_Math   Spatial SAT_Verbal
SAT_Math   1.0000000 0.3180507  0.3282856
Spatial    0.3180507 1.0000000  0.1795489
SAT_Verbal 0.3282856 0.1795489  1.0000000

Stage 2 analysis

## Stage 2 analysis
cluster2 <- tssem2(cluster1, Amatrix=A2, Smatrix=S2, diag.constraints=FALSE, intervals="LB")
summary(cluster2)

$Females

Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
                 Estimate Std.Error   lbound   ubound z value Pr(>|z|)
Spatial2Math     0.409265        NA 0.305292 0.512536      NA       NA
Verbal2Math      0.271217        NA 0.156125 0.386561      NA       NA
SpatialCorVerbal 0.171931        NA 0.045061 0.298800      NA       NA

Goodness-of-fit indices:
                                            Value
Sample size                                235.00
Chi-square of target model                   0.00
DF of target model                           0.00
p value of target model                      0.00
Number of constraints imposed on "Smatrix"   0.00
DF manually adjusted                         0.00
Chi-square of independence model           105.57
DF of independence model                     3.00
RMSEA                                        0.00
RMSEA lower 95% CI                           0.00
RMSEA upper 95% CI                           0.00
SRMR                                         0.00
TLI                                          -Inf
CFI                                          1.00
AIC                                          0.00
BIC                                          0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

$Males

Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), aCov = vcov.tssem1FEM(tssem1.obj), 
    n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix, 
    Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis, 
    intervals.type = intervals.type, mx.algebras = mx.algebras, 
    model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
                 Estimate Std.Error   lbound   ubound z value Pr(>|z|)
Spatial2Math     0.267739        NA 0.166906 0.368546      NA       NA
Verbal2Math      0.280213        NA 0.178023 0.382428      NA       NA
SpatialCorVerbal 0.179549        NA 0.069892 0.289197      NA       NA

Goodness-of-fit indices:
                                             Value
Sample size                                303.000
Chi-square of target model                   0.000
DF of target model                           0.000
p value of target model                      0.000
Number of constraints imposed on "Smatrix"   0.000
DF manually adjusted                         0.000
Chi-square of independence model            68.564
DF of independence model                     3.000
RMSEA                                        0.000
RMSEA lower 95% CI                           0.000
RMSEA upper 95% CI                           0.000
SRMR                                         0.000
TLI                                           -Inf
CFI                                          1.000
AIC                                          0.000
BIC                                          0.000
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

We may also plot the parameter estimates of these two groups.

## Convert the model to semPlotModel object with 2 plots
my.plots <- lapply(X=cluster2, FUN=meta2semPlot)

## Setup two plots
layout(t(1:2))
semPaths(my.plots[[1]], whatLabels="est", nCharNodes=10, color="green")
title("Females") 
semPaths(my.plots[[2]], whatLabels="est", nCharNodes=10, color="yellow")
title("Males")

Random-effects TSSEM

Stage 1 analysis

The $I^2$ indices for the correlations between spatial and math, verbal and math, and spatial and verbal are .00, .81 and .23, respectively.

#### Random-effects model
## Stage 1 analysis: A diagonal matrix for random effects 
random1 <- tssem1(Becker94$data, Becker94$n, method="REM", RE.type="Diag")
summary(random1)


Call:
meta(y = ES, v = acovR, RE.constraints = Diag(paste0(RE.startvalues, 
    "*Tau2_", 1:no.es, "_", 1:no.es)), RE.lbound = RE.lbound, 
    I2 = I2, model.name = model.name, suppressWarnings = TRUE, 
    silent = silent, run = run)

95% confidence intervals: z statistic approximation
Coefficients:
             Estimate  Std.Error     lbound     ubound z value  Pr(>|z|)
Intercept1  0.3777491  0.0395030  0.3003246  0.4551735  9.5625 < 2.2e-16
Intercept2  0.3807843  0.0784956  0.2269358  0.5346329  4.8510 1.228e-06
Intercept3  0.1704927  0.0513545  0.0698398  0.2711457  3.3199 0.0009004
Tau2_1_1    0.0005038  0.0042009 -0.0077298  0.0087374  0.1199 0.9045416
Tau2_2_2    0.0416264  0.0257388 -0.0088206  0.0920734  1.6173 0.1058209
Tau2_3_3    0.0067540  0.0102792 -0.0133928  0.0269008  0.6571 0.5111469
              
Intercept1 ***
Intercept2 ***
Intercept3 ***
Tau2_1_1      
Tau2_2_2      
Tau2_3_3      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Q statistic on the homogeneity of effect sizes: 61.02636
Degrees of freedom of the Q statistic: 27
P value of the Q statistic: 0.0001932117

Heterogeneity indices (based on the estimated Tau2):
                             Estimate
Intercept1: I2 (Q statistic)   0.0337
Intercept2: I2 (Q statistic)   0.7224
Intercept3: I2 (Q statistic)   0.2676

Number of studies (or clusters): 10
Number of observed statistics: 30
Number of estimated parameters: 6
Degrees of freedom: 24
-2 log likelihood: -22.61046 
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

coef(random1, select="fixed")

Intercept1 Intercept2 Intercept3 
 0.3777491  0.3807843  0.1704928

coef(random1, select="random")

    Tau2_1_1     Tau2_2_2     Tau2_3_3 
0.0005037989 0.0416264033 0.0067539585

Stage 2 analysis

## Stage 2 analysis
random2 <- tssem2(random1, Amatrix=A2, Smatrix=S2, diag.constraints=FALSE, intervals="LB")
summary(random2)


Call:
wls(Cov = pooledS, aCov = aCov, n = tssem1.obj$total.n, Amatrix = Amatrix, 
    Smatrix = Smatrix, Fmatrix = Fmatrix, diag.constraints = diag.constraints, 
    cor.analysis = cor.analysis, intervals.type = intervals.type, 
    mx.algebras = mx.algebras, model.name = model.name, suppressWarnings = suppressWarnings, 
    silent = silent, run = run)

95% confidence intervals: Likelihood-based statistic
Coefficients:
                 Estimate Std.Error  lbound  ubound z value Pr(>|z|)
Spatial2Math      0.32219        NA 0.23725 0.40432      NA       NA
Verbal2Math       0.32585        NA 0.16874 0.48257      NA       NA
SpatialCorVerbal  0.17049        NA 0.06984 0.27115      NA       NA

Goodness-of-fit indices:
                                            Value
Sample size                                538.00
Chi-square of target model                   0.00
DF of target model                           0.00
p value of target model                      0.00
Number of constraints imposed on "Smatrix"   0.00
DF manually adjusted                         0.00
Chi-square of independence model           110.81
DF of independence model                     3.00
RMSEA                                        0.00
RMSEA lower 95% CI                           0.00
RMSEA upper 95% CI                           0.00
SRMR                                         0.00
TLI                                          -Inf
CFI                                          1.00
AIC                                          0.00
BIC                                          0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)

We may also plot the models with the labels or the parameter estimates.

## Plot the model with labels
plot(random2, whatLabels="path", color="red")

## Plot the parameter estimates
plot(random2, color="green")

One-stage meta-analytic structural equation modeling (OSMASEM)

Nohe15

Data: Nohe, C., Meier, L. L., Sonntag, K., & Michel, A. (2015). The chicken or the egg? A meta-analysis of panel studies of the relationship between work–family conflict and strain. Journal of Applied Psychology, 100(2), 522–536. https://doi.org/10.1037/a0038012

Data preparation

## Convert correlation matrices into a dataframe
my.df <- Cor2DataFrame(Nohe15A1$data, Nohe15A1$n)

## Add the standardized Lag (moderator) to the data
my.df$data <- data.frame(my.df$data, Lag=scale(Nohe15A1$Lag), check.names=FALSE)
head(my.df$data)

                                       S1_W1 W2_W1 S2_W1 W2_S1 S2_S1 S2_W2
Britt...Dawson..2005.                   0.29  0.58  0.22  0.24  0.57  0.27
Demerouti.et.al...2004.                 0.53  0.57  0.41  0.41  0.68  0.54
Ford..2010.                             0.35  0.75  0.32  0.26  0.74  0.30
Hammer.et.al...2005...female.subsample  0.32  0.57  0.22  0.30  0.43  0.30
Hammer.et.al...2005...male.subsample    0.19  0.54  0.17  0.21  0.60  0.30
Innstrand.et.al...2008.                 0.42  0.63  0.31  0.30  0.62  0.44
                                       C(S1_W1 S1_W1) C(W2_W1 S1_W1)
Britt...Dawson..2005.                    0.0014224809   2.033649e-04
Demerouti.et.al...2004.                  0.0020763956   2.968506e-04
Ford..2010.                              0.0021207103   3.031879e-04
Hammer.et.al...2005...female.subsample   0.0029726171   4.249774e-04
Hammer.et.al...2005...male.subsample     0.0029726189   4.249797e-04
Innstrand.et.al...2008.                  0.0003112269   4.449432e-05
                                       C(S2_W1 S1_W1) C(W2_S1 S1_W1)
Britt...Dawson..2005.                    0.0008791263   0.0008736637
Demerouti.et.al...2004.                  0.0012832600   0.0012752855
Ford..2010.                              0.0013106490   0.0013025040
Hammer.et.al...2005...female.subsample   0.0018371447   0.0018257287
Hammer.et.al...2005...male.subsample     0.0018371479   0.0018257311
Innstrand.et.al...2008.                  0.0001923452   0.0001911499
                                       C(S2_S1 S1_W1) C(S2_W2 S1_W1)
Britt...Dawson..2005.                    2.047359e-04   0.0004890914
Demerouti.et.al...2004.                  2.988519e-04   0.0007139253
Ford..2010.                              3.052317e-04   0.0007291634
Hammer.et.al...2005...female.subsample   4.278435e-04   0.0010220716
Hammer.et.al...2005...male.subsample     4.278446e-04   0.0010220745
Innstrand.et.al...2008.                  4.479419e-05   0.0001070087
                                       C(W2_W1 W2_W1) C(S2_W1 W2_W1)
Britt...Dawson..2005.                    0.0007981114   3.750405e-04
Demerouti.et.al...2004.                  0.0011650035   5.474456e-04
Ford..2010.                              0.0011898672   5.591312e-04
Hammer.et.al...2005...female.subsample   0.0016678465   7.837350e-04
Hammer.et.al...2005...male.subsample     0.0016678480   7.837381e-04
Innstrand.et.al...2008.                  0.0001746202   8.205552e-05
                                       C(W2_S1 W2_W1) C(S2_S1 W2_W1)
Britt...Dawson..2005.                    3.671037e-04   1.042820e-04
Demerouti.et.al...2004.                  5.358598e-04   1.522195e-04
Ford..2010.                              5.472977e-04   1.554694e-04
Hammer.et.al...2005...female.subsample   7.671489e-04   2.179206e-04
Hammer.et.al...2005...male.subsample     7.671515e-04   2.179221e-04
Innstrand.et.al...2008.                  8.031894e-05   2.281582e-05
                                       C(S2_W2 W2_W1) C(S2_W1 S2_W1)
Britt...Dawson..2005.                    2.035394e-04   0.0016496834
Demerouti.et.al...2004.                  2.971050e-04   0.0024080433
Ford..2010.                              3.034474e-04   0.0024594371
Hammer.et.al...2005...female.subsample   4.253418e-04   0.0034474109
Hammer.et.al...2005...male.subsample     4.253444e-04   0.0034474151
Innstrand.et.al...2008.                  4.453244e-05   0.0003609368
                                       C(W2_S1 S2_W1) C(S2_S1 S2_W1)
Britt...Dawson..2005.                    0.0005769127   3.592787e-04
Demerouti.et.al...2004.                  0.0008421174   5.244382e-04
Ford..2010.                              0.0008600918   5.356322e-04
Hammer.et.al...2005...female.subsample   0.0012055941   7.507977e-04
Hammer.et.al...2005...male.subsample     0.0012055983   7.507998e-04
Innstrand.et.al...2008.                  0.0001262230   7.860683e-05
                                       C(S2_W2 S2_W1) C(W2_S1 W2_S1)
Britt...Dawson..2005.                    0.0008607808   0.0016601436
Demerouti.et.al...2004.                  0.0012564808   0.0024233109
Ford..2010.                              0.0012832977   0.0024750297
Hammer.et.al...2005...female.subsample   0.0017988066   0.0034692693
Hammer.et.al...2005...male.subsample     0.0017988103   0.0034692723
Innstrand.et.al...2008.                  0.0001883312   0.0003632253
                                       C(S2_S1 W2_S1) C(S2_W2 W2_S1)
Britt...Dawson..2005.                    3.727431e-04   0.0008739048
Demerouti.et.al...2004.                  5.440916e-04   0.0012756376
Ford..2010.                              5.557051e-04   0.0013028629
Hammer.et.al...2005...female.subsample   7.789345e-04   0.0018262322
Hammer.et.al...2005...male.subsample     7.789360e-04   0.0018262355
Innstrand.et.al...2008.                  8.155267e-05   0.0001912026
                                       C(S2_S1 S2_S1) C(S2_W2 S2_S1)
Britt...Dawson..2005.                    0.0007805646   1.951879e-04
Demerouti.et.al...2004.                  0.0011393904   2.849144e-04
Ford..2010.                              0.0011637073   2.909962e-04
Hammer.et.al...2005...female.subsample   0.0016311787   4.078899e-04
Hammer.et.al...2005...male.subsample     0.0016311792   4.078916e-04
Innstrand.et.al...2008.                  0.0001707810   4.270506e-05
                                       C(S2_W2 S2_W2)        Lag
Britt...Dawson..2005.                    0.0013981167 -0.6794521
Demerouti.et.al...2004.                  0.0020408312 -0.7711151
Ford..2010.                              0.0020843862 -0.8016694
Hammer.et.al...2005...female.subsample   0.0029217019 -0.1294740
Hammer.et.al...2005...male.subsample     0.0029217044 -0.1294740
Innstrand.et.al...2008.                  0.0003058961  0.6038301

## Display the pairwise no. of studies
pattern.na(my.df, show.na=FALSE, type="osmasem")

      S1_W1 W2_W1 S2_W1 W2_S1 S2_S1 S2_W2
S1_W1    32    32    32    32    32    32
W2_W1    32    32    32    32    32    32
S2_W1    32    32    32    32    32    32
W2_S1    32    32    32    32    32    32
S2_S1    32    32    32    32    32    32
S2_W2    32    32    32    32    32    32

## Proposed model
model1 <- 'W2 ~ w2w*W1 + s2w*S1
           S2 ~ w2s*W1 + s2s*S1
           W1 ~~ w1WITHs1*S1
           W2 ~~ w2WITHs2*S2
           W1 ~~ 1*W1
           S1 ~~ 1*S1
           W2 ~~ Errw2*W2
           S2 ~~ Errs2*S2'

## Plot the model
plot(model1, col="yellow")

## Convert the lavaan syntax into the RAM specification
RAM1 <- lavaan2RAM(model1, obs.variables=c("W1", "S1", "W2", "S2"))
RAM1

$A
   W1      S1      W2  S2 
W1 "0"     "0"     "0" "0"
S1 "0"     "0"     "0" "0"
W2 "0*w2w" "0*s2w" "0" "0"
S2 "0*w2s" "0*s2s" "0" "0"

$S
   W1           S1           W2           S2          
W1 "1"          "0*w1WITHs1" "0"          "0"         
S1 "0*w1WITHs1" "1"          "0"          "0"         
W2 "0"          "0"          "0*Errw2"    "0*w2WITHs2"
S2 "0"          "0"          "0*w2WITHs2" "0*Errs2"   

$F
   W1 S1 W2 S2
W1  1  0  0  0
S1  0  1  0  0
W2  0  0  1  0
S2  0  0  0  1

$M
  W1 S1 W2 S2
1  0  0  0  0

Fitting a just-identified model (df=0)

## Create the model implied correlation structure with implicit diagonal constraints
M0 <- create.vechsR(A0=RAM1$A, S0=RAM1$S)

## Create the heterogeneity variance-covariance matrix
T0 <- create.Tau2(RAM=RAM1, RE.type="Diag")

## Fit the OSMASEM
fit0 <- osmasem(model.name="No moderator", Mmatrix=M0, Tmatrix=T0, data=my.df)
summary(fit0)

Summary of No moderator 
 
free parameters:
       name  matrix row col    Estimate  Std.Error A    z value
1       w2w      A0  W2  W1  0.57247128 0.02226456    25.712220
2       w2s      A0  S2  W1  0.08023683 0.02484214     3.229868
3       s2w      A0  W2  S1  0.08584127 0.02479589     3.461915
4       s2s      A0  S2  S1  0.58612400 0.02079000    28.192594
5  w1WITHs1      S0  S1  W1  0.38045217 0.02256156    16.862850
6  w2WITHs2      S0  S2  W2  0.16888458 0.02523191     6.693293
7    Tau1_1 vecTau1   1   1 -2.15335880 0.14358419   -14.997186
8    Tau1_2 vecTau1   2   1 -2.36882273 0.14419315   -16.428123
9    Tau1_3 vecTau1   3   1 -2.47296546 0.15796631   -15.655018
10   Tau1_4 vecTau1   4   1 -2.47676198 0.15669549   -15.806211
11   Tau1_5 vecTau1   5   1 -2.46245713 0.14519714   -16.959405
12   Tau1_6 vecTau1   6   1 -2.19983813 0.14187315   -15.505669
       Pr(>|z|)
1  0.000000e+00
2  1.238472e-03
3  5.363474e-04
4  0.000000e+00
5  0.000000e+00
6  2.182032e-11
7  0.000000e+00
8  0.000000e+00
9  0.000000e+00
10 0.000000e+00
11 0.000000e+00
12 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             12                    180             -300.1701
   Saturated:             27                    165                    NA
Independence:             12                    180                    NA
Number of observations/statistics: 12906/192

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -660.1701              -276.1701                -276.1459
BIC:     -2003.9507              -186.5847                -224.7195
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:44 
Wall clock time: 0.4849949 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the estimated coefficients
coef(fit0)

       w2w        w2s        s2w        s2s   w1WITHs1   w2WITHs2 
0.57247128 0.08023683 0.08584127 0.58612400 0.38045217 0.16888458

## A matrix
mxEval(Amatrix, fit0$mx.fit)

           [,1]       [,2] [,3] [,4]
[1,] 0.00000000 0.00000000    0    0
[2,] 0.00000000 0.00000000    0    0
[3,] 0.57247128 0.08584127    0    0
[4,] 0.08023683 0.58612400    0    0

## S matrix
mxEval(Smatrix, fit0$mx.fit)

          [,1]      [,2]      [,3]      [,4]
[1,] 1.0000000 0.3804522 0.0000000 0.0000000
[2,] 0.3804522 1.0000000 0.0000000 0.0000000
[3,] 0.0000000 0.0000000 0.6275158 0.1688846
[4,] 0.0000000 0.0000000 0.1688846 0.6142363

## Extract the heterogeneity variance-covariance matrix
VarCorr(fit0)

           Tau2_1      Tau2_2      Tau2_3      Tau2_4      Tau2_5
Tau2_1 0.01347772 0.000000000 0.000000000 0.000000000 0.000000000
Tau2_2 0.00000000 0.008759246 0.000000000 0.000000000 0.000000000
Tau2_3 0.00000000 0.000000000 0.007112291 0.000000000 0.000000000
Tau2_4 0.00000000 0.000000000 0.000000000 0.007058491 0.000000000
Tau2_5 0.00000000 0.000000000 0.000000000 0.000000000 0.007263349
Tau2_6 0.00000000 0.000000000 0.000000000 0.000000000 0.000000000
           Tau2_6
Tau2_1 0.00000000
Tau2_2 0.00000000
Tau2_3 0.00000000
Tau2_4 0.00000000
Tau2_5 0.00000000
Tau2_6 0.01228132

Using `Lag` as a moderator on the A matrix

Ax <- matrix(c(0,0,0,0,
               0,0,0,0,
               "0*data.Lag","0*data.Lag",0,0,
               "0*data.Lag","0*data.Lag",0,0),
             nrow=4, ncol=4, byrow=TRUE)
Ax

     [,1]         [,2]         [,3] [,4]
[1,] "0"          "0"          "0"  "0" 
[2,] "0"          "0"          "0"  "0" 
[3,] "0*data.Lag" "0*data.Lag" "0"  "0" 
[4,] "0*data.Lag" "0*data.Lag" "0"  "0"

## Create the model implied correlation structure with the centered Lag as the moderator
M1 <- create.vechsR(A0=RAM1$A, S0=RAM1$S, Ax=Ax)

fit1 <- osmasem(model.name="Lag as moderator on Ax", Mmatrix=M1, Tmatrix=T0, data=my.df)
summary(fit1)

Summary of Lag as moderator on Ax 
 
free parameters:
       name  matrix row col    Estimate  Std.Error A     z value
1       w2w      A0  W2  W1  0.57303976 0.01839765    31.1474384
2       w2s      A0  S2  W1  0.07984495 0.02419489     3.3000754
3       s2w      A0  W2  S1  0.08539104 0.02393612     3.5674551
4       s2s      A0  S2  S1  0.58623426 0.01962560    29.8708888
5  w1WITHs1      S0  S1  W1  0.38118372 0.02282277    16.7019033
6  w2WITHs2      S0  S2  W2  0.16697481 0.02500439     6.6778203
7     w2w_1      A1  W2  W1 -0.06201558 0.01850573    -3.3511551
8     w2s_1      A1  S2  W1 -0.02593370 0.02096724    -1.2368677
9     s2w_1      A1  W2  S1 -0.00238280 0.02055897    -0.1159007
10    s2s_1      A1  S2  S1 -0.02780976 0.01974172    -1.4086798
11   Tau1_1 vecTau1   1   1 -2.13819052 0.14360102   -14.8898005
12   Tau1_2 vecTau1   2   1 -2.63051822 0.16155259   -16.2827361
13   Tau1_3 vecTau1   3   1 -2.52419385 0.16007540   -15.7687801
14   Tau1_4 vecTau1   4   1 -2.51990859 0.15983963   -15.7652300
15   Tau1_5 vecTau1   5   1 -2.53747582 0.14712278   -17.2473349
16   Tau1_6 vecTau1   6   1 -2.19886329 0.14144098   -15.5461544
       Pr(>|z|)
1  0.000000e+00
2  9.665887e-04
3  3.604651e-04
4  0.000000e+00
5  0.000000e+00
6  2.425216e-11
7  8.047522e-04
8  2.161362e-01
9  9.077312e-01
10 1.589299e-01
11 0.000000e+00
12 0.000000e+00
13 0.000000e+00
14 0.000000e+00
15 0.000000e+00
16 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             16                    176             -323.6921
   Saturated:             27                    165                    NA
Independence:             12                    180                    NA
Number of observations/statistics: 12906/192

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -675.6921              -291.6921                -291.6499
BIC:     -1989.6109              -172.2449                -223.0913
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:45 
Wall clock time: 0.8082128 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the estimated coefficients
coef(fit1)

        w2w         w2s         s2w         s2s    w1WITHs1    w2WITHs2 
 0.57303976  0.07984495  0.08539104  0.58623426  0.38118372  0.16697481 
      w2w_1       w2s_1       s2w_1       s2s_1 
-0.06201558 -0.02593370 -0.00238280 -0.02780976

## Extract the residual heterogeneity variance-covariance matrix
VarCorr(fit1)

           Tau2_1      Tau2_2      Tau2_3      Tau2_4      Tau2_5
Tau2_1 0.01389285 0.000000000 0.000000000 0.000000000 0.000000000
Tau2_2 0.00000000 0.005189923 0.000000000 0.000000000 0.000000000
Tau2_3 0.00000000 0.000000000 0.006419676 0.000000000 0.000000000
Tau2_4 0.00000000 0.000000000 0.000000000 0.006474932 0.000000000
Tau2_5 0.00000000 0.000000000 0.000000000 0.000000000 0.006251389
Tau2_6 0.00000000 0.000000000 0.000000000 0.000000000 0.000000000
           Tau2_6
Tau2_1 0.00000000
Tau2_2 0.00000000
Tau2_3 0.00000000
Tau2_4 0.00000000
Tau2_5 0.00000000
Tau2_6 0.01230528

## Calculate the R2
osmasemR2(fit1, fit0)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.013477716 0.008759246 0.007112291 0.007058491 0.007263349 0.012281315 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.013892849 0.005189923 0.006419676 0.006474932 0.006251389 0.012305283 

$R2
  Tau2_1_1   Tau2_2_2   Tau2_3_3   Tau2_4_4   Tau2_5_5   Tau2_6_6 
0.00000000 0.40749205 0.09738283 0.08267478 0.13932420 0.00000000

## Compare the models with and without the moderator
anova(fit1, fit0)

                    base   comparison ep  minus2LL  df       AIC   diffLL
1 Lag as moderator on Ax         <NA> 16 -323.6921 176 -675.6921       NA
2 Lag as moderator on Ax No moderator 12 -300.1701 180 -660.1701 23.52197
  diffdf            p
1     NA           NA
2      4 9.957566e-05

Using `Lag` as a moderator on the S matrix

Sx <- matrix(c(0,"0*data.Lag",0,0,
               "0*data.Lag",0,0,0,
               0,0,0,"0*data.Lag",
               0,0,"0*data.Lag",0),
             nrow=4, ncol=4, byrow=TRUE)
Sx

     [,1]         [,2]         [,3]         [,4]        
[1,] "0"          "0*data.Lag" "0"          "0"         
[2,] "0*data.Lag" "0"          "0"          "0"         
[3,] "0"          "0"          "0"          "0*data.Lag"
[4,] "0"          "0"          "0*data.Lag" "0"

M2 <- create.vechsR(A0=RAM1$A, S0=RAM1$S, Sx=Sx)

fit2 <- osmasem(model.name="Lag as moderator on Sx", Mmatrix=M2, Tmatrix=T0, data=my.df)
summary(fit2)

Summary of Lag as moderator on Sx 
 
free parameters:
         name  matrix row col     Estimate  Std.Error A     z value
1         w2w      A0  W2  W1  0.569234669 0.02189963    25.9928918
2         w2s      A0  S2  W1  0.085032455 0.02395176     3.5501553
3         s2w      A0  W2  S1  0.092336316 0.02405804     3.8380645
4         s2s      A0  S2  S1  0.584608194 0.02049270    28.5276275
5    w1WITHs1      S0  S1  W1  0.376986140 0.02230557    16.9009849
6    w2WITHs2      S0  S2  W2  0.165484768 0.02477818     6.6786498
7  w1WITHs1_1      S1  S1  W1 -0.053146784 0.01794365    -2.9618714
8  w2WITHs2_1      S1  S2  W2 -0.008169627 0.02172731    -0.3760074
9      Tau1_1 vecTau1   1   1 -2.169522558 0.14503073   -14.9590543
10     Tau1_2 vecTau1   2   1 -2.376926065 0.14470312   -16.4262248
11     Tau1_3 vecTau1   3   1 -2.524375473 0.15992211   -15.7850315
12     Tau1_4 vecTau1   4   1 -2.516292476 0.15999130   -15.7276834
13     Tau1_5 vecTau1   5   1 -2.464565231 0.14547983   -16.9409406
14     Tau1_6 vecTau1   6   1 -2.210554507 0.14170816   -15.5993453
       Pr(>|z|)
1  0.000000e+00
2  3.850039e-04
3  1.240079e-04
4  0.000000e+00
5  0.000000e+00
6  2.411538e-11
7  3.057755e-03
8  7.069114e-01
9  0.000000e+00
10 0.000000e+00
11 0.000000e+00
12 0.000000e+00
13 0.000000e+00
14 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             14                    178             -309.5338
   Saturated:             27                    165                    NA
Independence:             12                    180                    NA
Number of observations/statistics: 12906/192

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -665.5338              -281.5338                -281.5012
BIC:     -1994.3835              -177.0175                -221.5081
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:46 
Wall clock time: 0.7213342 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the estimated coefficients
coef(fit2)

         w2w          w2s          s2w          s2s     w1WITHs1 
 0.569234669  0.085032455  0.092336316  0.584608194  0.376986140 
    w2WITHs2   w1WITHs1_1   w2WITHs2_1 
 0.165484768 -0.053146784 -0.008169627

## Extract the residual heterogeneity variance-covariance matrix
VarCorr(fit2)

           Tau2_1      Tau2_2      Tau2_3     Tau2_4     Tau2_5     Tau2_6
Tau2_1 0.01304898 0.000000000 0.000000000 0.00000000 0.00000000 0.00000000
Tau2_2 0.00000000 0.008618432 0.000000000 0.00000000 0.00000000 0.00000000
Tau2_3 0.00000000 0.000000000 0.006417344 0.00000000 0.00000000 0.00000000
Tau2_4 0.00000000 0.000000000 0.000000000 0.00652193 0.00000000 0.00000000
Tau2_5 0.00000000 0.000000000 0.000000000 0.00000000 0.00723279 0.00000000
Tau2_6 0.00000000 0.000000000 0.000000000 0.00000000 0.00000000 0.01202089

## Calculate the R2
osmasemR2(fit2, fit0)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.013477716 0.008759246 0.007112291 0.007058491 0.007263349 0.012281315 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.013048983 0.008618432 0.006417344 0.006521930 0.007232790 0.012020894 

$R2
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.031810566 0.016076044 0.097710645 0.076016436 0.004207334 0.021204708

## Compare the models with and without the moderator
anova(fit2, fit0)

                    base   comparison ep  minus2LL  df       AIC   diffLL
1 Lag as moderator on Sx         <NA> 14 -309.5338 178 -665.5338       NA
2 Lag as moderator on Sx No moderator 12 -300.1701 180 -660.1701 9.363675
  diffdf           p
1     NA          NA
2      2 0.009261978

Using `Lag` as a moderator on both the A and S matrices

M3 <- create.vechsR(A0=RAM1$A, S0=RAM1$S, Ax=Ax, Sx=Sx)

fit3 <- osmasem(model.name="Lag as moderator on Ax and Sx", Mmatrix=M3, Tmatrix=T0, data=my.df)
summary(fit3)

Summary of Lag as moderator on Ax and Sx 
 
free parameters:
         name  matrix row col     Estimate  Std.Error A     z value
1         w2w      A0  W2  W1  0.574425649 0.01843827    31.1539808
2         w2s      A0  S2  W1  0.079662699 0.02387732     3.3363341
3         s2w      A0  W2  S1  0.083087394 0.02363557     3.5153539
4         s2s      A0  S2  S1  0.585858063 0.01964430    29.8233162
5    w1WITHs1      S0  S1  W1  0.381044097 0.02197101    17.3430435
6    w2WITHs2      S0  S2  W2  0.167582974 0.02478854     6.7605027
7       w2w_1      A1  W2  W1 -0.063025036 0.01656924    -3.8037371
8       w2s_1      A1  S2  W1 -0.013043445 0.02072889    -0.6292399
9       s2w_1      A1  W2  S1  0.006865329 0.02021928     0.3395438
10      s2s_1      A1  S2  S1 -0.031632460 0.01771820    -1.7853085
11 w1WITHs1_1      S1  S1  W1 -0.038859425 0.02182854    -1.7802120
12 w2WITHs2_1      S1  S2  W2  0.007050352 0.02335411     0.3018891
13     Tau1_1 vecTau1   1   1 -2.184627280 0.14427002   -15.1426279
14     Tau1_2 vecTau1   2   1 -2.633170158 0.16172682   -16.2815923
15     Tau1_3 vecTau1   3   1 -2.531969425 0.16055050   -15.7705481
16     Tau1_4 vecTau1   4   1 -2.534620289 0.16068831   -15.7735199
17     Tau1_5 vecTau1   5   1 -2.539156253 0.14724186   -17.2447990
18     Tau1_6 vecTau1   6   1 -2.205765057 0.14170872   -15.5654852
       Pr(>|z|)
1  0.000000e+00
2  8.489109e-04
3  4.391683e-04
4  0.000000e+00
5  0.000000e+00
6  1.375144e-11
7  1.425294e-04
8  5.291920e-01
9  7.342001e-01
10 7.421129e-02
11 7.504128e-02
12 7.627366e-01
13 0.000000e+00
14 0.000000e+00
15 0.000000e+00
16 0.000000e+00
17 0.000000e+00
18 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             18                    174             -327.0718
   Saturated:             27                    165                    NA
Independence:             12                    180                    NA
Number of observations/statistics: 12906/192

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -675.0718              -291.0718                -291.0187
BIC:     -1974.0596              -156.6937                -213.8959
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:47 
Wall clock time: 1.001056 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

coef(fit3)

         w2w          w2s          s2w          s2s     w1WITHs1 
 0.574425649  0.079662699  0.083087394  0.585858063  0.381044097 
    w2WITHs2        w2w_1        w2s_1        s2w_1        s2s_1 
 0.167582974 -0.063025036 -0.013043445  0.006865329 -0.031632460 
  w1WITHs1_1   w2WITHs2_1 
-0.038859425  0.007050352

VarCorr(fit3)

           Tau2_1      Tau2_2      Tau2_3      Tau2_4      Tau2_5
Tau2_1 0.01266067 0.000000000 0.000000000 0.000000000 0.000000000
Tau2_2 0.00000000 0.005162469 0.000000000 0.000000000 0.000000000
Tau2_3 0.00000000 0.000000000 0.006320614 0.000000000 0.000000000
Tau2_4 0.00000000 0.000000000 0.000000000 0.006287193 0.000000000
Tau2_5 0.00000000 0.000000000 0.000000000 0.000000000 0.006230414
Tau2_6 0.00000000 0.000000000 0.000000000 0.000000000 0.000000000
           Tau2_6
Tau2_1 0.00000000
Tau2_2 0.00000000
Tau2_3 0.00000000
Tau2_4 0.00000000
Tau2_5 0.00000000
Tau2_6 0.01213659

osmasemR2(fit3, fit0)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.013477716 0.008759246 0.007112291 0.007058491 0.007263349 0.012281315 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.012660675 0.005162469 0.006320614 0.006287193 0.006230414 0.012136594 

$R2
  Tau2_1_1   Tau2_2_2   Tau2_3_3   Tau2_4_4   Tau2_5_5   Tau2_6_6 
0.06062166 0.41062631 0.11131099 0.10927240 0.14221196 0.01178388

anova(fit3, fit0)

                           base   comparison ep  minus2LL  df       AIC
1 Lag as moderator on Ax and Sx         <NA> 18 -327.0718 174 -675.0718
2 Lag as moderator on Ax and Sx No moderator 12 -300.1701 180 -660.1701
    diffLL diffdf            p
1       NA     NA           NA
2 26.90165      6 0.0001510817

Roorda11

Data: Roorda, D. L., Koomen, H. M. Y., Spilt, J. L., & Oort, F. J. (2011). The influence of affective teacher-student relationships on students’ school engagement and achievement a meta-analytic approach. Review of Educational Research, 81(4), 493-529.

Data preparation

my.df <- Cor2DataFrame(Roorda11$data, Roorda11$n)

## Add centered SES as a moderator (standardizing SES helps to improve the stability of the results)
my.df$data <- data.frame(my.df$data, SES=scale(Roorda11$SES), check.names=FALSE)
head(my.df$data)

  neg_pos enga_pos achiev_pos enga_neg achiev_neg achiev_enga
1   -0.54       NA       0.18       NA      -0.29          NA
2      NA     0.64       0.29       NA         NA        0.23
3      NA     0.29         NA       NA         NA          NA
4      NA     0.29         NA       NA         NA          NA
5      NA       NA         NA       NA      -0.24          NA
6      NA     0.06      -0.09       NA         NA        0.20
  C(neg_pos neg_pos) C(enga_pos neg_pos) C(achiev_pos neg_pos)
1       0.0005989502       -0.0001515536         -0.0000907290
2       0.0018375277       -0.0004649532         -0.0002783482
3       0.0063790677       -0.0016141097         -0.0009663004
4       0.0118882522       -0.0030081036         -0.0018008299
5       0.0043833797       -0.0011091359         -0.0006639934
6       0.0084368390       -0.0021347978         -0.0012780133
  C(enga_neg neg_pos) C(achiev_neg neg_pos) C(achiev_enga neg_pos)
1        0.0001507646          0.0000388353          -2.214640e-05
2        0.0004625328          0.0001191431          -6.794299e-05
3        0.0016057053          0.0004136097          -2.358691e-04
4        0.0029924413          0.0007708176          -4.395722e-04
5        0.0011033613          0.0002842128          -1.620787e-04
6        0.0021236771          0.0005470320          -3.119582e-04
  C(enga_pos enga_pos) C(achiev_pos enga_pos) C(enga_neg enga_pos)
1          0.000639757           0.0001647860        -0.0001910375
2          0.001962720           0.0005055502        -0.0005860861
3          0.006813675           0.0017550374        -0.0020346272
4          0.012698200           0.0032707488        -0.0037917893
5          0.004682022           0.0012059754        -0.0013980968
6          0.009011640           0.0023211788        -0.0026909643
  C(achiev_neg enga_pos) C(achiev_enga enga_pos) C(achiev_pos achiev_pos)
1          -5.134723e-05            3.127064e-05             0.0007526511
2          -1.575289e-04            9.593586e-05             0.0023090689
3          -5.468674e-04            3.330398e-04             0.0080160396
4          -1.019154e-03            6.206713e-04             0.0149389681
5          -3.757800e-04            2.288500e-04             0.0055082274
6          -7.232801e-04            4.404741e-04             0.0106018642
  C(enga_neg achiev_pos) C(achiev_neg achiev_pos)
1          -0.0000639561            -0.0002463391
2          -0.0001962112            -0.0007557473
3          -0.0006811564            -0.0026236129
4          -0.0012694183            -0.0048894376
5          -0.0004680584            -0.0018028161
6          -0.0009008884            -0.0034699453
  C(achiev_enga achiev_pos) C(enga_neg enga_neg) C(achiev_neg enga_neg)
1              0.0001998712         0.0006389863           0.0001558888
2              0.0006131886         0.0019603555           0.0004782531
3              0.0021287085         0.0068054674           0.0016602796
4              0.0039671318         0.0126828943           0.0030941484
5              0.0014627436         0.0046763850           0.0011408628
6              0.0028153906         0.0090007824           0.0021958560
  C(achiev_enga enga_neg) C(achiev_neg achiev_neg)
1           -7.403034e-05             0.0007298082
2           -2.271193e-04             0.0022389906
3           -7.884556e-04             0.0077727610
4           -1.469385e-03             0.0144855703
5           -5.417899e-04             0.0053410557
6           -1.042798e-03             0.0102801097
  C(achiev_enga achiev_neg) C(achiev_enga achiev_enga)        SES
1             -0.0001921431               0.0006716397  0.4464188
2             -0.0005894798               0.0020605350  0.7145383
3             -0.0020464083               0.0071532436  0.8821129
4             -0.0038137375               0.0133310266 -0.8941783
5             -0.0014061892               0.0049153554 -0.9947231
6             -0.0027065420               0.0094607400 -0.5925439

## Display the pairwise no. of studies
pattern.na(my.df, show.na=FALSE, type="osmasem")

            neg_pos enga_pos achiev_pos enga_neg achiev_neg achiev_enga
neg_pos          15       30         28       17         20          24
enga_pos         30       21         37       23         35          24
achiev_pos       28       37         26       30         31          29
enga_neg         17       23         30        8         21          16
achiev_neg       20       35         31       21         18          26
achiev_enga      24       24         29       16         26          13

## Proposed model
model2 <- 'enga ~ b31*neg + b32*pos
           achiev ~ b41*neg + b42*pos + b43*enga
           pos ~~ p21*neg
           pos ~~ 1*pos
           neg ~~ 1*neg
           enga ~~ p33*enga
           achiev ~~ p44*achiev'

plot(model2, layout="spring", color="yellow")

## Convert the lavaan model to the RAM specification
RAM2 <- lavaan2RAM(model2, obs.variables=c("neg", "pos", "enga", "achiev"))
RAM2

$A
       neg     pos     enga    achiev
neg    "0"     "0"     "0"     "0"   
pos    "0"     "0"     "0"     "0"   
enga   "0*b31" "0*b32" "0"     "0"   
achiev "0*b41" "0*b42" "0*b43" "0"   

$S
       neg     pos     enga    achiev 
neg    "1"     "0*p21" "0"     "0"    
pos    "0*p21" "1"     "0"     "0"    
enga   "0"     "0"     "0*p33" "0"    
achiev "0"     "0"     "0"     "0*p44"

$F
       neg pos enga achiev
neg      1   0    0      0
pos      0   1    0      0
enga     0   0    1      0
achiev   0   0    0      1

$M
  neg pos enga achiev
1   0   0    0      0

Fitting a just-identified model (df=0)

## Create matrices with implicit diagonal constraints
M0 <- create.vechsR(A0=RAM2$A, S0=RAM2$S)

## Create heterogeneity variances
T0 <- create.Tau2(RAM=RAM2, RE.type="Diag")

mx.fit0a <- osmasem(model.name="Just identified model", Mmatrix=M0, Tmatrix=T0, data= my.df)
summary(mx.fit0a)

Summary of Just identified model 
 
free parameters:
     name  matrix    row  col    Estimate  Std.Error A    z value
1     b31      A0   enga  neg  0.25513472 0.04084654     6.246177
2     b41      A0 achiev  neg  0.04088992 0.02704882     1.511708
3     b32      A0   enga  pos -0.24326590 0.04795947    -5.072323
4     b42      A0 achiev  pos -0.09342501 0.03205929    -2.914132
5     b43      A0 achiev enga  0.23653110 0.04498218     5.258328
6     p21      S0    pos  neg -0.23871422 0.04044945    -5.901544
7  Tau1_1 vecTau1      1    1 -1.97354078 0.21976082    -8.980403
8  Tau1_2 vecTau1      2    1 -1.87778190 0.18152776   -10.344324
9  Tau1_3 vecTau1      3    1 -2.55758793 0.22104158   -11.570619
10 Tau1_4 vecTau1      4    1 -2.21995392 0.31525230    -7.041833
11 Tau1_5 vecTau1      5    1 -2.50426709 0.26585171    -9.419789
12 Tau1_6 vecTau1      6    1 -2.11922121 0.23882312    -8.873601
       Pr(>|z|)
1  4.206211e-10
2  1.306081e-01
3  3.929896e-07
4  3.566792e-03
5  1.453707e-07
6  3.601144e-09
7  0.000000e+00
8  0.000000e+00
9  0.000000e+00
10 1.897371e-12
11 0.000000e+00
12 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             12                     89             -127.1991
   Saturated:             27                     74                    NA
Independence:             12                     89                    NA
Number of observations/statistics: 29438/101

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -305.1991            -103.199115               -103.18851
BIC:     -1043.0128              -3.718615                -41.85445
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:50 
Wall clock time: 0.4687271 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

coef(mx.fit0a)

        b31         b41         b32         b42         b43         p21 
 0.25513472  0.04088992 -0.24326590 -0.09342501  0.23653110 -0.23871422

## Extract the variance component
VarCorr(mx.fit0a)

           Tau2_1     Tau2_2      Tau2_3     Tau2_4      Tau2_5     Tau2_6
Tau2_1 0.01931098 0.00000000 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_2 0.00000000 0.02338726 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_3 0.00000000 0.00000000 0.006004922 0.00000000 0.000000000 0.00000000
Tau2_4 0.00000000 0.00000000 0.000000000 0.01179703 0.000000000 0.00000000
Tau2_5 0.00000000 0.00000000 0.000000000 0.00000000 0.006680689 0.00000000
Tau2_6 0.00000000 0.00000000 0.000000000 0.00000000 0.000000000 0.01443005

Fitting an over-identified model with b41=b42=0 (df=2))

## Proposed model
model3 <- 'enga ~ b31*neg + b32*pos
           achiev ~ 0*neg + 0*pos + b43*enga
           pos ~~ p21*neg
           pos ~~ 1*pos
           neg ~~ 1*neg
           enga ~~ p33*enga
           achiev ~~ p44*achiev'

plot(model3, layout="spring", color="yellow")

RAM3 <- lavaan2RAM(model3, obs.variables=c("neg", "pos", "enga", "achiev"))
RAM3

$A
       neg     pos     enga    achiev
neg    "0"     "0"     "0"     "0"   
pos    "0"     "0"     "0"     "0"   
enga   "0*b31" "0*b32" "0"     "0"   
achiev "0"     "0"     "0*b43" "0"   

$S
       neg     pos     enga    achiev 
neg    "1"     "0*p21" "0"     "0"    
pos    "0*p21" "1"     "0"     "0"    
enga   "0"     "0"     "0*p33" "0"    
achiev "0"     "0"     "0"     "0*p44"

$F
       neg pos enga achiev
neg      1   0    0      0
pos      0   1    0      0
enga     0   0    1      0
achiev   0   0    0      1

$M
  neg pos enga achiev
1   0   0    0      0

## Create matrices with implicit diagonal constraints
M0b <- create.vechsR(A0=RAM3$A, S0=RAM3$S)

## Create heterogeneity variances
T0b <- create.Tau2(RAM=RAM3, RE.type="Diag")

mx.fit0b <- osmasem(model.name="Over identified model", Mmatrix=M0b, Tmatrix=T0b, data= my.df)

## Get the summary and test statistics
summary(mx.fit0b, Saturated=TRUE)

Summary of Over identified model 
 
free parameters:
     name  matrix    row  col   Estimate  Std.Error A    z value
1     b31      A0   enga  neg  0.2649761 0.03780500     7.009023
2     b32      A0   enga  pos -0.2964373 0.05474967    -5.414412
3     b43      A0 achiev enga  0.3500633 0.04052992     8.637157
4     p21      S0    pos  neg -0.2386957 0.04057830    -5.882347
5  Tau1_1 vecTau1      1    1 -1.9694733 0.21977212    -8.961434
6  Tau1_2 vecTau1      2    1 -1.8706614 0.18373398   -10.181358
7  Tau1_3 vecTau1      3    1 -2.5635352 0.22701110   -11.292554
8  Tau1_4 vecTau1      4    1 -2.0996828 0.34898880    -6.016476
9  Tau1_5 vecTau1      5    1 -2.3799827 0.27470090    -8.663906
10 Tau1_6 vecTau1      6    1 -1.9542601 0.26333240    -7.421267
       Pr(>|z|)
1  2.399858e-12
2  6.149033e-08
3  0.000000e+00
4  4.044888e-09
5  0.000000e+00
6  0.000000e+00
7  0.000000e+00
8  1.782546e-09
9  0.000000e+00
10 1.159073e-13

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             10                     91             -117.5580
   Saturated:             12                     89             -127.1991
Independence:             12                     89                    NA
Number of observations/statistics: 29438/101

chi-square:  χ² ( df=2 ) = 9.641115,  p = 0.00806229
Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:       -299.558              -97.55800                -97.55052
BIC:      -1053.952              -14.65758                -46.43744
CFI: NA 
TLI: NA   (also known as NNFI) 
RMSEA:  0.01139224  [95% CI (0.003446381, 0.02034632)]
Prob(RMSEA <= 0.05): 1
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:50 
Wall clock time: 0.4698486 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the SRMR
osmasemSRMR(mx.fit0b)

[1] 0.04402731

## Get the coefficients
coef(mx.fit0b)

       b31        b32        b43        p21 
 0.2649761 -0.2964373  0.3500633 -0.2386957

## Extract the variance component
VarCorr(mx.fit0b)

           Tau2_1    Tau2_2      Tau2_3     Tau2_4      Tau2_5     Tau2_6
Tau2_1 0.01946871 0.0000000 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_2 0.00000000 0.0237227 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_3 0.00000000 0.0000000 0.005933919 0.00000000 0.000000000 0.00000000
Tau2_4 0.00000000 0.0000000 0.000000000 0.01500509 0.000000000 0.00000000
Tau2_5 0.00000000 0.0000000 0.000000000 0.00000000 0.008565906 0.00000000
Tau2_6 0.00000000 0.0000000 0.000000000 0.00000000 0.000000000 0.02007018

Using `SES` as a moderator on the A matrix

## data.SES as a moderator
Ax <- matrix(c(0,0,0,0,
               0,0,0,0,
               "0*data.SES","0*data.SES",0,0,
               "0*data.SES","0*data.SES","0*data.SES",0),
             nrow=4, ncol=4, byrow=TRUE)
Ax

     [,1]         [,2]         [,3]         [,4]
[1,] "0"          "0"          "0"          "0" 
[2,] "0"          "0"          "0"          "0" 
[3,] "0*data.SES" "0*data.SES" "0"          "0" 
[4,] "0*data.SES" "0*data.SES" "0*data.SES" "0"

M1 <- create.vechsR(A0=RAM2$A, S0=RAM2$S)

mx.fit1 <- osmasem(model.name="Ax as moderator", Mmatrix=M1, Tmatrix=T0, data= my.df)
summary(mx.fit1)

Summary of Ax as moderator 
 
free parameters:
     name  matrix    row  col    Estimate  Std.Error A    z value
1     b31      A0   enga  neg  0.25513472 0.04084654     6.246177
2     b41      A0 achiev  neg  0.04088992 0.02704882     1.511708
3     b32      A0   enga  pos -0.24326590 0.04795947    -5.072323
4     b42      A0 achiev  pos -0.09342501 0.03205929    -2.914132
5     b43      A0 achiev enga  0.23653110 0.04498218     5.258328
6     p21      S0    pos  neg -0.23871422 0.04044945    -5.901544
7  Tau1_1 vecTau1      1    1 -1.97354078 0.21976082    -8.980403
8  Tau1_2 vecTau1      2    1 -1.87778190 0.18152776   -10.344324
9  Tau1_3 vecTau1      3    1 -2.55758793 0.22104158   -11.570619
10 Tau1_4 vecTau1      4    1 -2.21995392 0.31525230    -7.041833
11 Tau1_5 vecTau1      5    1 -2.50426709 0.26585171    -9.419789
12 Tau1_6 vecTau1      6    1 -2.11922121 0.23882312    -8.873601
       Pr(>|z|)
1  4.206211e-10
2  1.306081e-01
3  3.929896e-07
4  3.566792e-03
5  1.453707e-07
6  3.601144e-09
7  0.000000e+00
8  0.000000e+00
9  0.000000e+00
10 1.897371e-12
11 0.000000e+00
12 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             12                     89             -127.1991
   Saturated:             27                     74                    NA
Independence:             12                     89                    NA
Number of observations/statistics: 29438/101

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -305.1991            -103.199115               -103.18851
BIC:     -1043.0128              -3.718615                -41.85445
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:52 
Wall clock time: 0.4678843 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the coefficients
coef(mx.fit1)

        b31         b41         b32         b42         b43         p21 
 0.25513472  0.04088992 -0.24326590 -0.09342501  0.23653110 -0.23871422

## Get the R2
osmasemR2(mx.fit1, mx.fit0a)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.019310977 0.023387261 0.006004922 0.011797026 0.006680689 0.014430050 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.019310977 0.023387261 0.006004922 0.011797026 0.006680689 0.014430050 

$R2
Tau2_1_1 Tau2_2_2 Tau2_3_3 Tau2_4_4 Tau2_5_5 Tau2_6_6 
       0        0        0        0        0        0

## Extract the variance component
VarCorr(mx.fit1)

           Tau2_1     Tau2_2      Tau2_3     Tau2_4      Tau2_5     Tau2_6
Tau2_1 0.01931098 0.00000000 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_2 0.00000000 0.02338726 0.000000000 0.00000000 0.000000000 0.00000000
Tau2_3 0.00000000 0.00000000 0.006004922 0.00000000 0.000000000 0.00000000
Tau2_4 0.00000000 0.00000000 0.000000000 0.01179703 0.000000000 0.00000000
Tau2_5 0.00000000 0.00000000 0.000000000 0.00000000 0.006680689 0.00000000
Tau2_6 0.00000000 0.00000000 0.000000000 0.00000000 0.000000000 0.01443005

## Comparing with the model without the moderator
anova(mx.fit1, mx.fit0a)

             base            comparison ep  minus2LL df       AIC diffLL
1 Ax as moderator                  <NA> 12 -127.1991 89 -305.1991     NA
2 Ax as moderator Just identified model 12 -127.1991 89 -305.1991      0
  diffdf  p
1     NA NA
2      0 NA

Using `SES` as a moderator on the S matrix

Sx <- matrix(c(0,"0*data.SES",0,0,
               "0*data.SES",0,0,0,
               0,0,0,0,
               0,0,0,0),
             nrow=4, ncol=4, byrow=TRUE)
Sx

     [,1]         [,2]         [,3] [,4]
[1,] "0"          "0*data.SES" "0"  "0" 
[2,] "0*data.SES" "0"          "0"  "0" 
[3,] "0"          "0"          "0"  "0" 
[4,] "0"          "0"          "0"  "0"

M2 <- create.vechsR(A0=RAM2$A, S0=RAM2$S, Sx=Sx)

mx.fit2 <- osmasem(model.name="Sx as moderator", Mmatrix=M2, Tmatrix=T0, data= my.df)
summary(mx.fit2)

Summary of Sx as moderator 
 
free parameters:
     name  matrix    row  col    Estimate  Std.Error A    z value
1     b31      A0   enga  neg  0.25341805 0.04086685     6.201067
2     b41      A0 achiev  neg  0.04264173 0.02700073     1.579281
3     b32      A0   enga  pos -0.24065288 0.04559656    -5.277874
4     b42      A0 achiev  pos -0.09173707 0.03147241    -2.914841
5     b43      A0 achiev enga  0.23723933 0.04503765     5.267578
6     p21      S0    pos  neg -0.23700731 0.03856954    -6.144935
7   p21_1      S1    pos  neg -0.06195803 0.04074914    -1.520475
8  Tau1_1 vecTau1      1    1 -2.03486183 0.22008854    -9.245651
9  Tau1_2 vecTau1      2    1 -1.88594573 0.18130177   -10.402247
10 Tau1_3 vecTau1      3    1 -2.54500490 0.22122879   -11.503950
11 Tau1_4 vecTau1      4    1 -2.30296762 0.33650536    -6.843777
12 Tau1_5 vecTau1      5    1 -2.51530507 0.27003031    -9.314899
13 Tau1_6 vecTau1      6    1 -2.11483035 0.23847692    -8.868071
       Pr(>|z|)
1  5.608174e-10
2  1.142717e-01
3  1.306913e-07
4  3.558696e-03
5  1.382357e-07
6  7.999610e-10
7  1.283917e-01
8  0.000000e+00
9  0.000000e+00
10 0.000000e+00
11 7.713163e-12
12 0.000000e+00
13 0.000000e+00

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             13                     88             -129.4354
   Saturated:             27                     74                    NA
Independence:             12                     89                    NA
Number of observations/statistics: 29438/101

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -305.4354            -103.435449               -103.42308
BIC:     -1034.9591               4.335092                -36.97872
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:53 
Wall clock time: 1.674853 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the coefficients
coef(mx.fit2)

        b31         b41         b32         b42         b43         p21 
 0.25341805  0.04264173 -0.24065288 -0.09173707  0.23723933 -0.23700731 
      p21_1 
-0.06195803

## R2
osmasemR2(mx.fit2, mx.fit0a)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.019310977 0.023387261 0.006004922 0.011797026 0.006680689 0.014430050 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.017082108 0.023008503 0.006157960 0.009992352 0.006534822 0.014557329 

$R2
  Tau2_1_1   Tau2_2_2   Tau2_3_3   Tau2_4_4   Tau2_5_5   Tau2_6_6 
0.11541980 0.01619509 0.00000000 0.15297697 0.02183407 0.00000000

## Extract the variance component
VarCorr(mx.fit2)

           Tau2_1    Tau2_2     Tau2_3      Tau2_4      Tau2_5     Tau2_6
Tau2_1 0.01708211 0.0000000 0.00000000 0.000000000 0.000000000 0.00000000
Tau2_2 0.00000000 0.0230085 0.00000000 0.000000000 0.000000000 0.00000000
Tau2_3 0.00000000 0.0000000 0.00615796 0.000000000 0.000000000 0.00000000
Tau2_4 0.00000000 0.0000000 0.00000000 0.009992352 0.000000000 0.00000000
Tau2_5 0.00000000 0.0000000 0.00000000 0.000000000 0.006534822 0.00000000
Tau2_6 0.00000000 0.0000000 0.00000000 0.000000000 0.000000000 0.01455733

## Comparing with the model without the moderator
anova(mx.fit2, mx.fit0a)

             base            comparison ep  minus2LL df       AIC   diffLL
1 Sx as moderator                  <NA> 13 -129.4354 88 -305.4354       NA
2 Sx as moderator Just identified model 12 -127.1991 89 -305.1991 2.236335
  diffdf         p
1     NA        NA
2      1 0.1348002

Scalco17

Data: Scalco, A., Noventa, S., Sartori, R., & Ceschi, A. (2017). Predicting organic food consumption: A meta-analytic structural equation model based on the theory of planned behavior. Appetite, 112, 235-248.

Data preparation

# select studies without missing value on Age
data <- Scalco17$data[is.na(Scalco17$Age)==FALSE]
n <- Scalco17$n[is.na(Scalco17$Age)==FALSE]
Age <- Scalco17$Age[is.na(Scalco17$Age)==FALSE]
my.df <- Cor2DataFrame(data, n, acov = "weighted")

## Add the centered Age as a moderator
my.df$data <- data.frame(my.df$data, Age=scale(Age,  scale=FALSE), check.names=FALSE)
head(my.df$data)

                                      SN_ATT PBC_ATT BI_ATT BEH_ATT PBC_SN
Al-Swidi et al., 2014                  0.562    0.18  0.798      NA  0.314
Arvola et al., 2008 (Study A from IT)  0.690    0.44  0.730      NA  0.460
Arvola et al., 2008 (Study A from UK)  0.520    0.22  0.600      NA  0.280
Arvola et al., 2008 (Study A from FI)  0.570    0.40  0.670      NA  0.340
Arvola et al., 2008 (Study B from IT)  0.760    0.35  0.710      NA  0.360
Arvola et al., 2008 (Study B from UK)  0.460    0.03  0.550      NA  0.150
                                      BI_SN BEH_SN BI_PBC BEH_PBC BEH_BI
Al-Swidi et al., 2014                 0.696     NA  0.216      NA     NA
Arvola et al., 2008 (Study A from IT) 0.620     NA  0.410      NA     NA
Arvola et al., 2008 (Study A from UK) 0.560     NA  0.310      NA     NA
Arvola et al., 2008 (Study A from FI) 0.550     NA  0.360      NA     NA
Arvola et al., 2008 (Study B from IT) 0.640     NA  0.240      NA     NA
Arvola et al., 2008 (Study B from UK) 0.580     NA  0.100      NA     NA
                                      C(SN_ATT SN_ATT) C(PBC_ATT SN_ATT)
Al-Swidi et al., 2014                      0.003894925      0.0007859678
Arvola et al., 2008 (Study A from IT)      0.003547876      0.0007159569
Arvola et al., 2008 (Study A from UK)      0.002654344      0.0005356464
Arvola et al., 2008 (Study A from FI)      0.003583351      0.0007231084
Arvola et al., 2008 (Study B from IT)      0.003547865      0.0007159389
Arvola et al., 2008 (Study B from UK)      0.002654324      0.0005356252
                                      C(BI_ATT SN_ATT) C(BEH_ATT SN_ATT)
Al-Swidi et al., 2014                     0.0011346455      0.0012594637
Arvola et al., 2008 (Study A from IT)     0.0010335541      0.0011472581
Arvola et al., 2008 (Study A from UK)     0.0007732555      0.0008583232
Arvola et al., 2008 (Study A from FI)     0.0010438864      0.0011587247
Arvola et al., 2008 (Study B from IT)     0.0010335453      0.0011472441
Arvola et al., 2008 (Study B from UK)     0.0007732421      0.0008583047
                                      C(PBC_SN SN_ATT) C(BI_SN SN_ATT)
Al-Swidi et al., 2014                     0.0010444158     0.001906273
Arvola et al., 2008 (Study A from IT)     0.0009513749     0.001736428
Arvola et al., 2008 (Study A from UK)     0.0007117763     0.001299112
Arvola et al., 2008 (Study A from FI)     0.0009608795     0.001753788
Arvola et al., 2008 (Study B from IT)     0.0009513588     0.001736419
Arvola et al., 2008 (Study B from UK)     0.0007117527     0.001299095
                                      C(BEH_SN SN_ATT) C(BI_PBC SN_ATT)
Al-Swidi et al., 2014                      0.001564628     0.0005640135
Arvola et al., 2008 (Study A from IT)      0.001425230     0.0005137810
Arvola et al., 2008 (Study A from UK)      0.001066290     0.0003843903
Arvola et al., 2008 (Study A from FI)      0.001439479     0.0005189091
Arvola et al., 2008 (Study B from IT)      0.001425219     0.0005137618
Arvola et al., 2008 (Study B from UK)      0.001066271     0.0003843654
                                      C(BEH_PBC SN_ATT) C(BEH_BI SN_ATT)
Al-Swidi et al., 2014                      0.0003771623     0.0005704295
Arvola et al., 2008 (Study A from IT)      0.0003435754     0.0005196156
Arvola et al., 2008 (Study A from UK)      0.0002570495     0.0003887529
Arvola et al., 2008 (Study A from FI)      0.0003470017     0.0005248081
Arvola et al., 2008 (Study B from IT)      0.0003435598     0.0005196063
Arvola et al., 2008 (Study B from UK)      0.0002570310     0.0003887401
                                      C(PBC_ATT PBC_ATT) C(BI_ATT PBC_ATT)
Al-Swidi et al., 2014                        0.004582112      0.0006865784
Arvola et al., 2008 (Study A from IT)        0.004173823      0.0006254127
Arvola et al., 2008 (Study A from UK)        0.003122638      0.0004679025
Arvola et al., 2008 (Study A from FI)        0.004215553      0.0006316622
Arvola et al., 2008 (Study B from IT)        0.004173809      0.0006254021
Arvola et al., 2008 (Study B from UK)        0.003122622      0.0004678903
                                      C(BEH_ATT PBC_ATT) C(PBC_SN PBC_ATT)
Al-Swidi et al., 2014                        0.001476475       0.001698929
Arvola et al., 2008 (Study A from IT)        0.001344928       0.001547560
Arvola et al., 2008 (Study A from UK)        0.001006206       0.001157806
Arvola et al., 2008 (Study A from FI)        0.001358371       0.001563026
Arvola et al., 2008 (Study B from IT)        0.001344914       0.001547547
Arvola et al., 2008 (Study B from UK)        0.001006190       0.001157788
                                      C(BI_SN PBC_ATT) C(BEH_SN PBC_ATT)
Al-Swidi et al., 2014                     0.0004422728      0.0005909684
Arvola et al., 2008 (Study A from IT)     0.0004028844      0.0005383324
Arvola et al., 2008 (Study A from UK)     0.0003014209      0.0004027555
Arvola et al., 2008 (Study A from FI)     0.0004069057      0.0005437095
Arvola et al., 2008 (Study B from IT)     0.0004028716      0.0005383191
Arvola et al., 2008 (Study B from UK)     0.0003014040      0.0004027373
                                      C(BI_PBC PBC_ATT) C(BEH_PBC PBC_ATT)
Al-Swidi et al., 2014                       0.002780101        0.001778293
Arvola et al., 2008 (Study A from IT)       0.002532388        0.001619846
Arvola et al., 2008 (Study A from UK)       0.001894602        0.001211885
Arvola et al., 2008 (Study A from FI)       0.002557702        0.001636036
Arvola et al., 2008 (Study B from IT)       0.002532374        0.001619835
Arvola et al., 2008 (Study B from UK)       0.001894585        0.001211872
                                      C(BEH_BI PBC_ATT) C(BI_ATT BI_ATT)
Al-Swidi et al., 2014                      0.0006609428      0.001774441
Arvola et al., 2008 (Study A from IT)      0.0006020607      0.001616331
Arvola et al., 2008 (Study A from UK)      0.0004504317      0.001209257
Arvola et al., 2008 (Study A from FI)      0.0006080774      0.001632492
Arvola et al., 2008 (Study B from IT)      0.0006020518      0.001616326
Arvola et al., 2008 (Study B from UK)      0.0004504209      0.001209249
                                      C(BEH_ATT BI_ATT) C(PBC_SN BI_ATT)
Al-Swidi et al., 2014                      0.0013079546     0.0003808394
Arvola et al., 2008 (Study A from IT)      0.0011914175     0.0003469187
Arvola et al., 2008 (Study A from UK)      0.0008913587     0.0002595495
Arvola et al., 2008 (Study A from FI)      0.0012033287     0.0003503819
Arvola et al., 2008 (Study B from IT)      0.0011914093     0.0003469092
Arvola et al., 2008 (Study B from UK)      0.0008913478     0.0002595353
                                      C(BI_SN BI_ATT) C(BEH_SN BI_ATT)
Al-Swidi et al., 2014                    0.0004761844     0.0004869757
Arvola et al., 2008 (Study A from IT)    0.0004337657     0.0004435976
Arvola et al., 2008 (Study A from UK)    0.0003245244     0.0003318801
Arvola et al., 2008 (Study A from FI)    0.0004381001     0.0004480310
Arvola et al., 2008 (Study B from IT)    0.0004337592     0.0004435903
Arvola et al., 2008 (Study B from UK)    0.0003245133     0.0003318676
                                      C(BI_PBC BI_ATT) C(BEH_PBC BI_ATT)
Al-Swidi et al., 2014                     0.0004943946      0.0003008937
Arvola et al., 2008 (Study A from IT)     0.0004503545      0.0002740939
Arvola et al., 2008 (Study A from UK)     0.0003369337      0.0002050639
Arvola et al., 2008 (Study A from FI)     0.0004548520      0.0002768294
Arvola et al., 2008 (Study B from IT)     0.0004503432      0.0002740847
Arvola et al., 2008 (Study B from UK)     0.0003369194      0.0002050535
                                      C(BEH_BI BI_ATT) C(BEH_ATT BEH_ATT)
Al-Swidi et al., 2014                     0.0004709275        0.003023506
Arvola et al., 2008 (Study A from IT)     0.0004289729        0.002754103
Arvola et al., 2008 (Study A from UK)     0.0003209369        0.002060479
Arvola et al., 2008 (Study A from FI)     0.0004332608        0.002781640
Arvola et al., 2008 (Study B from IT)     0.0004289675        0.002754092
Arvola et al., 2008 (Study B from UK)     0.0003209292        0.002060465
                                      C(PBC_SN BEH_ATT) C(BI_SN BEH_ATT)
Al-Swidi et al., 2014                      0.0006150543     0.0006149720
Arvola et al., 2008 (Study A from IT)      0.0005602700     0.0005601933
Arvola et al., 2008 (Study A from UK)      0.0004191677     0.0004191112
Arvola et al., 2008 (Study A from FI)      0.0005658632     0.0005657893
Arvola et al., 2008 (Study B from IT)      0.0005602566     0.0005601831
Arvola et al., 2008 (Study B from UK)      0.0004191485     0.0004190960
                                      C(BEH_SN BEH_ATT) C(BI_PBC BEH_ATT)
Al-Swidi et al., 2014                      0.0009062377      0.0008205374
Arvola et al., 2008 (Study A from IT)      0.0008255075      0.0007474413
Arvola et al., 2008 (Study A from UK)      0.0006176054      0.0005591983
Arvola et al., 2008 (Study A from FI)      0.0008337577      0.0007549068
Arvola et al., 2008 (Study B from IT)      0.0008254966      0.0007474260
Arvola et al., 2008 (Study B from UK)      0.0006175885      0.0005591792
                                      C(BEH_PBC BEH_ATT) C(BEH_BI BEH_ATT)
Al-Swidi et al., 2014                       0.0005476760      0.0011377374
Arvola et al., 2008 (Study A from IT)       0.0004988910      0.0010363684
Arvola et al., 2008 (Study A from UK)       0.0003732451      0.0007753587
Arvola et al., 2008 (Study A from FI)       0.0005038719      0.0010467291
Arvola et al., 2008 (Study B from IT)       0.0004988785      0.0010363609
Arvola et al., 2008 (Study B from UK)       0.0003732311      0.0007753486
                                      C(PBC_SN PBC_SN) C(BI_SN PBC_SN)
Al-Swidi et al., 2014                      0.004822455    0.0009802695
Arvola et al., 2008 (Study A from IT)      0.004392751    0.0008929402
Arvola et al., 2008 (Study A from UK)      0.003286433    0.0006680572
Arvola et al., 2008 (Study A from FI)      0.004436669    0.0009018613
Arvola et al., 2008 (Study B from IT)      0.004392740    0.0008929291
Arvola et al., 2008 (Study B from UK)      0.003286413    0.0006680385
                                      C(BEH_SN PBC_SN) C(BI_PBC PBC_SN)
Al-Swidi et al., 2014                      0.001673764      0.002320225
Arvola et al., 2008 (Study A from IT)      0.001524641      0.002113491
Arvola et al., 2008 (Study A from UK)      0.001140662      0.001581208
Arvola et al., 2008 (Study A from FI)      0.001539879      0.002134616
Arvola et al., 2008 (Study B from IT)      0.001524629      0.002113479
Arvola et al., 2008 (Study B from UK)      0.001140642      0.001581188
                                      C(BEH_PBC PBC_SN) C(BEH_BI PBC_SN)
Al-Swidi et al., 2014                       0.001653818     0.0005719372
Arvola et al., 2008 (Study A from IT)       0.001506462     0.0005209869
Arvola et al., 2008 (Study A from UK)       0.001127060     0.0003897780
Arvola et al., 2008 (Study A from FI)       0.001521518     0.0005261909
Arvola et al., 2008 (Study B from IT)       0.001506453     0.0005209790
Arvola et al., 2008 (Study B from UK)       0.001127045     0.0003897652
                                      C(BI_SN BI_SN) C(BEH_SN BI_SN)
Al-Swidi et al., 2014                    0.002738110     0.001836711
Arvola et al., 2008 (Study A from IT)    0.002494136     0.001673064
Arvola et al., 2008 (Study A from UK)    0.001865989     0.001251705
Arvola et al., 2008 (Study A from FI)    0.002519074     0.001689791
Arvola et al., 2008 (Study B from IT)    0.002494130     0.001673056
Arvola et al., 2008 (Study B from UK)    0.001865975     0.001251690
                                      C(BI_PBC BI_SN) C(BEH_PBC BI_SN)
Al-Swidi et al., 2014                    0.0005233720     0.0003211968
Arvola et al., 2008 (Study A from IT)    0.0004767566     0.0002925934
Arvola et al., 2008 (Study A from UK)    0.0003566895     0.0002189066
Arvola et al., 2008 (Study A from FI)    0.0004815156     0.0002955110
Arvola et al., 2008 (Study B from IT)    0.0004767433     0.0002925824
Arvola et al., 2008 (Study B from UK)    0.0003566697     0.0002188920
                                      C(BEH_BI BI_SN) C(BEH_SN BEH_SN)
Al-Swidi et al., 2014                    0.0005686535      0.003414696
Arvola et al., 2008 (Study A from IT)    0.0005179953      0.003110439
Arvola et al., 2008 (Study A from UK)    0.0003875402      0.002327075
Arvola et al., 2008 (Study A from FI)    0.0005231716      0.003141539
Arvola et al., 2008 (Study B from IT)    0.0005179889      0.003110431
Arvola et al., 2008 (Study B from UK)    0.0003875298      0.002327058
                                      C(BI_PBC BEH_SN) C(BEH_PBC BEH_SN)
Al-Swidi et al., 2014                     0.0007000865      0.0004650804
Arvola et al., 2008 (Study A from IT)     0.0006377266      0.0004236574
Arvola et al., 2008 (Study A from UK)     0.0004771184      0.0003169616
Arvola et al., 2008 (Study A from FI)     0.0006440963      0.0004278865
Arvola et al., 2008 (Study B from IT)     0.0006377129      0.0004236460
Arvola et al., 2008 (Study B from UK)     0.0004770971      0.0003169459
                                      C(BEH_BI BEH_SN) C(BI_PBC BI_PBC)
Al-Swidi et al., 2014                     0.0009280224      0.004310889
Arvola et al., 2008 (Study A from IT)     0.0008453430      0.003926767
Arvola et al., 2008 (Study A from UK)     0.0006324446      0.002937806
Arvola et al., 2008 (Study A from FI)     0.0008537932      0.003966026
Arvola et al., 2008 (Study B from IT)     0.0008453364      0.003926756
Arvola et al., 2008 (Study B from UK)     0.0006324333      0.002937788
                                      C(BEH_PBC BI_PBC) C(BEH_BI BI_PBC)
Al-Swidi et al., 2014                       0.002432722     0.0009285300
Arvola et al., 2008 (Study A from IT)       0.002215958     0.0008458032
Arvola et al., 2008 (Study A from UK)       0.001657866     0.0006327880
Arvola et al., 2008 (Study A from FI)       0.002238110     0.0008542563
Arvola et al., 2008 (Study B from IT)       0.002215949     0.0008457942
Arvola et al., 2008 (Study B from UK)       0.001657853     0.0006327755
                                      C(BEH_PBC BEH_PBC) C(BEH_BI BEH_PBC)
Al-Swidi et al., 2014                        0.003390938      0.0004011002
Arvola et al., 2008 (Study A from IT)        0.003088784      0.0003653700
Arvola et al., 2008 (Study A from UK)        0.002310869      0.0002733521
Arvola et al., 2008 (Study A from FI)        0.003119666      0.0003690190
Arvola et al., 2008 (Study B from IT)        0.003088778      0.0003653626
Arvola et al., 2008 (Study B from UK)        0.002310860      0.0002733429
                                      C(BEH_BI BEH_BI)       Age
Al-Swidi et al., 2014                      0.001556498 -2.398421
Arvola et al., 2008 (Study A from IT)      0.001417809  2.871579
Arvola et al., 2008 (Study A from UK)      0.001060732  2.871579
Arvola et al., 2008 (Study A from FI)      0.001431985  2.871579
Arvola et al., 2008 (Study B from IT)      0.001417804  2.871579
Arvola et al., 2008 (Study B from UK)      0.001060726  2.871579

## Display the pairwise no. of studies
pattern.na(my.df, show.na=FALSE, type="osmasem")

        SN_ATT PBC_ATT BI_ATT BEH_ATT PBC_SN BI_SN BEH_SN BI_PBC BEH_PBC
SN_ATT      18      18     19      18     18    19     18     19      18
PBC_ATT     18      18     19      18     18    19     18     19      18
BI_ATT      19      19     19      19     19    19     19     19      19
BEH_ATT     18      18     19       4     18    19      4     19       4
PBC_SN      18      18     19      18     18    19     18     19      18
BI_SN       19      19     19      19     19    19     19     19      19
BEH_SN      18      18     19       4     18    19      4     19       4
BI_PBC      19      19     19      19     19    19     19     19      19
BEH_PBC     18      18     19       4     18    19      4     19       4
BEH_BI      18      18     19       4     18    19      4     19       4
        BEH_BI
SN_ATT      18
PBC_ATT     18
BI_ATT      19
BEH_ATT      4
PBC_SN      18
BI_SN       19
BEH_SN       4
BI_PBC      19
BEH_PBC      4
BEH_BI       4

## Proposed model
model4 <- 'BI ~ b41*ATT + b42*SN + b43*PBC
           BEH ~ b54*BI + b53*PBC 
           ATT ~~ p21*SN
           ATT ~~ p31*PBC
           SN ~~ p32*PBC
           ATT ~~ 1*ATT
           SN ~~ 1*SN
           PBC ~~ 1*PBC
           BI ~~ p44*BI
           BEH ~~ p44*BEH'

plot(model4, color="yellow")

RAM4 <- lavaan2RAM(model4, obs.variables=c("ATT", "SN", "PBC", "BI","BEH"))
RAM4

$A
    ATT     SN      PBC     BI      BEH
ATT "0"     "0"     "0"     "0"     "0"
SN  "0"     "0"     "0"     "0"     "0"
PBC "0"     "0"     "0"     "0"     "0"
BI  "0*b41" "0*b42" "0*b43" "0"     "0"
BEH "0"     "0"     "0*b53" "0*b54" "0"

$S
    ATT     SN      PBC     BI      BEH    
ATT "1"     "0*p21" "0*p31" "0"     "0"    
SN  "0*p21" "1"     "0*p32" "0"     "0"    
PBC "0*p31" "0*p32" "1"     "0"     "0"    
BI  "0"     "0"     "0"     "0*p44" "0"    
BEH "0"     "0"     "0"     "0"     "0*p44"

$F
    ATT SN PBC BI BEH
ATT   1  0   0  0   0
SN    0  1   0  0   0
PBC   0  0   1  0   0
BI    0  0   0  1   0
BEH   0  0   0  0   1

$M
  ATT SN PBC BI BEH
1   0  0   0  0   0

Fitting an over-identified model

## Create mean structure
M0 <- create.vechsR(A0=RAM4$A, S0=RAM4$S)

## Create heterogeneity variances
T0 <- create.Tau2(RAM=RAM4, RE.type="Diag")

mx.fit0 <- osmasem(model.name="No moderator", Mmatrix=M0, Tmatrix=T0, data=my.df)

## Get the chi-square statistic on the proposed model
summary(mx.fit0, Saturated=TRUE)

Summary of No moderator 
 
free parameters:
      name  matrix row col   Estimate  Std.Error A    z value     Pr(>|z|)
1      b41      A0  BI ATT  0.4835277 0.04170564    11.593820 0.000000e+00
2      b42      A0  BI  SN  0.2962538 0.06020033     4.921132 8.604506e-07
3      b43      A0  BI PBC  0.1027158 0.03693804     2.780760 5.423187e-03
4      b53      A0 BEH PBC  0.1740509 0.09367858     1.857959 6.317486e-02
5      b54      A0 BEH  BI  0.6262585 0.06625252     9.452598 0.000000e+00
6      p21      S0  SN ATT  0.4364109 0.04526608     9.641013 0.000000e+00
7      p31      S0 PBC ATT  0.2534395 0.03287804     7.708471 1.265654e-14
8      p32      S0 PBC  SN  0.2452304 0.02698189     9.088704 0.000000e+00
9   Tau1_1 vecTau1   1   1 -1.6849473 0.17884381    -9.421334 0.000000e+00
10  Tau1_2 vecTau1   2   1 -2.0449409 0.19199595   -10.650958 0.000000e+00
11  Tau1_3 vecTau1   3   1 -2.1316440 0.17348443   -12.287235 0.000000e+00
12  Tau1_4 vecTau1   4   1 -2.5231923 0.40426061    -6.241499 4.333964e-10
13  Tau1_5 vecTau1   5   1 -2.2968636 0.21839538   -10.516997 0.000000e+00
14  Tau1_6 vecTau1   6   1 -1.7114047 0.17600128    -9.723820 0.000000e+00
15  Tau1_7 vecTau1   7   1 -2.4037353 0.49631445    -4.843170 1.277838e-06
16  Tau1_8 vecTau1   8   1 -2.1618822 0.19340901   -11.177774 0.000000e+00
17  Tau1_9 vecTau1   9   1 -1.8628788 0.36552969    -5.096382 3.462070e-07
18 Tau1_10 vecTau1  10   1 -2.3005516 0.45855549    -5.016954 5.249723e-07

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             18                    109             -131.9660
   Saturated:             20                    107             -136.5611
Independence:             20                    107                    NA
Number of observations/statistics: 9507/127

chi-square:  χ² ( df=2 ) = 4.595131,  p = 0.1005032
Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:       -349.966              -95.96600                -95.89391
BIC:      -1130.382               32.91011                -24.29108
CFI: NA 
TLI: NA   (also known as NNFI) 
RMSEA:  0.01168269  [95% CI (0, 0.02850146)]
Prob(RMSEA <= 0.05): 0.9999995
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:55 
Wall clock time: 0.8048987 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

## Get the SRMR
osmasemSRMR(mx.fit0)

[1] 0.02889252

## Extract the variance component
VarCorr(mx.fit0)

            Tau2_1     Tau2_2     Tau2_3      Tau2_4     Tau2_5     Tau2_6
Tau2_1  0.03439326 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_2  0.00000000 0.01674121 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_3  0.00000000 0.00000000 0.01407595 0.000000000 0.00000000 0.00000000
Tau2_4  0.00000000 0.00000000 0.00000000 0.006432547 0.00000000 0.00000000
Tau2_5  0.00000000 0.00000000 0.00000000 0.000000000 0.01011509 0.00000000
Tau2_6  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.03262066
Tau2_7  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_8  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_9  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_10 0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
             Tau2_7     Tau2_8     Tau2_9    Tau2_10
Tau2_1  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_2  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_3  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_4  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_5  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_6  0.000000000 0.00000000 0.00000000 0.00000000
Tau2_7  0.008168495 0.00000000 0.00000000 0.00000000
Tau2_8  0.000000000 0.01324991 0.00000000 0.00000000
Tau2_9  0.000000000 0.00000000 0.02409484 0.00000000
Tau2_10 0.000000000 0.00000000 0.00000000 0.01004075

Using `Age` as a moderator on the A matrix

Ax <- matrix(c(0,0,0,0,0,
               0,0,0,0,0,
               0,0,0,0,0,
               "0*data.Age","0*data.Age","0*data.Age",0,0,
               0,0,"0*data.Age","0*data.Age",0),
             nrow=5, ncol=5, byrow=TRUE)
Ax

     [,1]         [,2]         [,3]         [,4]         [,5]
[1,] "0"          "0"          "0"          "0"          "0" 
[2,] "0"          "0"          "0"          "0"          "0" 
[3,] "0"          "0"          "0"          "0"          "0" 
[4,] "0*data.Age" "0*data.Age" "0*data.Age" "0"          "0" 
[5,] "0"          "0"          "0*data.Age" "0*data.Age" "0"

M1 <- create.vechsR(A0=RAM4$A, S0=RAM4$S, Ax=Ax)

mx.fit1 <- osmasem(model.name="Ax as moderator", Mmatrix=M1, Tmatrix=T0, data=my.df)
summary(mx.fit1)

Summary of Ax as moderator 
 
free parameters:
      name  matrix row col     Estimate   Std.Error A     z value
1      b41      A0  BI ATT  0.494226442 0.037079424    13.3288597
2      b42      A0  BI  SN  0.269821551 0.045638260     5.9121787
3      b43      A0  BI PBC  0.106890240 0.035049545     3.0496898
4      b53      A0 BEH PBC  0.226759954 0.214463900     1.0573339
5      b54      A0 BEH  BI  0.507277449 0.127673836     3.9732295
6      p21      S0  SN ATT  0.436647760 0.045332702     9.6320701
7      p31      S0 PBC ATT  0.254398080 0.033049678     7.6974450
8      p32      S0 PBC  SN  0.244896515 0.027105290     9.0350080
9    b41_1      A1  BI ATT -0.004220378 0.005032280    -0.8386612
10   b42_1      A1  BI  SN  0.018116307 0.005359145     3.3804476
11   b43_1      A1  BI PBC  0.001645497 0.004044292     0.4068690
12   b53_1      A1 BEH PBC -0.013210821 0.037494874    -0.3523367
13   b54_1      A1 BEH  BI  0.018242909 0.021048291     0.8667169
14  Tau1_1 vecTau1   1   1 -1.683167330 0.178727323    -9.4175155
15  Tau1_2 vecTau1   2   1 -2.039207734 0.191968015   -10.6226432
16  Tau1_3 vecTau1   3   1 -2.162536515 0.174254219   -12.4102390
17  Tau1_4 vecTau1   4   1 -2.507962191 0.407830034    -6.1495279
18  Tau1_5 vecTau1   5   1 -2.289117405 0.217645920   -10.5176215
19  Tau1_6 vecTau1   6   1 -2.040031928 0.179314151   -11.3768596
20  Tau1_7 vecTau1   7   1 -2.499064989 0.453627675    -5.5090664
21  Tau1_8 vecTau1   8   1 -2.224783563 0.196211391   -11.3387074
22  Tau1_9 vecTau1   9   1 -1.866924355 0.364290049    -5.1248294
23 Tau1_10 vecTau1  10   1 -2.623309449 0.461552744    -5.6836613
       Pr(>|z|)
1  0.000000e+00
2  3.376119e-09
3  2.290778e-03
4  2.903592e-01
5  7.090468e-05
6  0.000000e+00
7  1.398881e-14
8  0.000000e+00
9  4.016595e-01
10 7.236789e-04
11 6.841042e-01
12 7.245857e-01
13 3.860972e-01
14 0.000000e+00
15 0.000000e+00
16 0.000000e+00
17 7.771390e-10
18 0.000000e+00
19 0.000000e+00
20 3.607418e-08
21 0.000000e+00
22 2.978073e-07
23 1.318413e-08

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             23                    104             -149.2628
   Saturated:             65                     62                    NA
Independence:             20                    107                    NA
Number of observations/statistics: 9507/127

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -357.2628             -103.26284               -103.14642
BIC:     -1101.8803               61.41218                -11.67822
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:58:59 
Wall clock time: 2.530679 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

coef(mx.fit1)

         b41          b42          b43          b53          b54 
 0.494226442  0.269821551  0.106890240  0.226759954  0.507277449 
         p21          p31          p32        b41_1        b42_1 
 0.436647760  0.254398080  0.244896515 -0.004220378  0.018116307 
       b43_1        b53_1        b54_1 
 0.001645497 -0.013210821  0.018242909

osmasemR2(mx.fit1, mx.fit0)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.034393264 0.016741214 0.014075945 0.006432547 0.010115088 0.032620661 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.008168495 0.013249912 0.024094840 0.010040753 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.034515918 0.016934277 0.013232584 0.006631499 0.010273014 0.016906386 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.006750559 0.011683623 0.023900672 0.005265291 

$R2
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.000000000 0.000000000 0.059915106 0.000000000 0.000000000 0.481727668 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.173585943 0.118211273 0.008058499 0.475607951

## Extract the variance component
VarCorr(mx.fit1)

            Tau2_1     Tau2_2     Tau2_3      Tau2_4     Tau2_5     Tau2_6
Tau2_1  0.03451592 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_2  0.00000000 0.01693428 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_3  0.00000000 0.00000000 0.01323258 0.000000000 0.00000000 0.00000000
Tau2_4  0.00000000 0.00000000 0.00000000 0.006631499 0.00000000 0.00000000
Tau2_5  0.00000000 0.00000000 0.00000000 0.000000000 0.01027301 0.00000000
Tau2_6  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.01690639
Tau2_7  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_8  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_9  0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
Tau2_10 0.00000000 0.00000000 0.00000000 0.000000000 0.00000000 0.00000000
             Tau2_7     Tau2_8     Tau2_9     Tau2_10
Tau2_1  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_2  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_3  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_4  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_5  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_6  0.000000000 0.00000000 0.00000000 0.000000000
Tau2_7  0.006750559 0.00000000 0.00000000 0.000000000
Tau2_8  0.000000000 0.01168362 0.00000000 0.000000000
Tau2_9  0.000000000 0.00000000 0.02390067 0.000000000
Tau2_10 0.000000000 0.00000000 0.00000000 0.005265291

anova(mx.fit1, mx.fit0)

             base   comparison ep  minus2LL  df       AIC   diffLL diffdf
1 Ax as moderator         <NA> 23 -149.2628 104 -357.2628       NA     NA
2 Ax as moderator No moderator 18 -131.9660 109 -349.9660 17.29684      5
            p
1          NA
2 0.003969912

Using `Age` as a moderator on the S matrix

Sx <- matrix(c(0,"0*data.Age","0*data.Age",0,0,
               "0*data.Age",0,"0*data.Age",0,0,
               "0*data.Age","0*data.Age",0,0,0,
               0,0,0,0,0,
               0,0,0,0,0),
             nrow=5, ncol=5, byrow=TRUE)

M2 <- create.vechsR(A0=RAM4$A, S0=RAM4$S, Sx=Sx)

mx.fit2 <- osmasem(model.name="Sx as moderator", Mmatrix=M2, Tmatrix=T0, data=my.df)
summary(mx.fit2)

Summary of Sx as moderator 
 
free parameters:
      name  matrix row col     Estimate   Std.Error A    z value
1      b41      A0  BI ATT  0.502147813 0.036565017    13.733012
2      b42      A0  BI  SN  0.270704137 0.044927081     6.025411
3      b43      A0  BI PBC  0.112908747 0.034262416     3.295411
4      b53      A0 BEH PBC  0.158356978 0.094302440     1.679246
5      b54      A0 BEH  BI  0.608081905 0.058175035    10.452626
6      p21      S0  SN ATT  0.419449230 0.038352780    10.936606
7      p31      S0 PBC ATT  0.242380384 0.029557717     8.200240
8      p32      S0 PBC  SN  0.236297228 0.024784307     9.534147
9    p21_1      S1  SN ATT  0.020810720 0.004737072     4.393161
10   p31_1      S1 PBC ATT  0.009424315 0.003788768     2.487436
11   p32_1      S1 PBC  SN  0.008231119 0.003499737     2.351925
12  Tau1_1 vecTau1   1   1 -1.869040864 0.184899188   -10.108432
13  Tau1_2 vecTau1   2   1 -2.182834339 0.197486301   -11.053092
14  Tau1_3 vecTau1   3   1 -2.141432376 0.174540416   -12.268977
15  Tau1_4 vecTau1   4   1 -2.573068466 0.399609002    -6.438965
16  Tau1_5 vecTau1   5   1 -2.418158537 0.224005046   -10.795107
17  Tau1_6 vecTau1   6   1 -1.994278115 0.182161369   -10.947865
18  Tau1_7 vecTau1   7   1 -2.494803754 0.455839818    -5.472983
19  Tau1_8 vecTau1   8   1 -2.201416817 0.194577242   -11.313845
20  Tau1_9 vecTau1   9   1 -1.869557540 0.365239276    -5.118720
21 Tau1_10 vecTau1  10   1 -2.395656017 0.408378751    -5.866260
       Pr(>|z|)
1  0.000000e+00
2  1.686802e-09
3  9.827779e-04
4  9.310414e-02
5  0.000000e+00
6  0.000000e+00
7  2.220446e-16
8  0.000000e+00
9  1.117142e-05
10 1.286678e-02
11 1.867656e-02
12 0.000000e+00
13 0.000000e+00
14 0.000000e+00
15 1.202907e-10
16 0.000000e+00
17 0.000000e+00
18 4.425216e-08
19 0.000000e+00
20 3.076165e-07
21 4.457341e-09

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit (-2lnL units)
       Model:             21                    106             -161.3619
   Saturated:             65                     62                    NA
Independence:             20                    107                    NA
Number of observations/statistics: 9507/127

Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      -373.3619             -119.36187               -119.26446
BIC:     -1132.2989               30.99358                -35.74113
To get additional fit indices, see help(mxRefModels)
timestamp: 2018-10-18 09:59:02 
Wall clock time: 2.039036 secs 
optimizer:  SLSQP 
OpenMx version number: 2.11.5 
Need help?  See help(mxSummary)

coef(mx.fit2)

        b41         b42         b43         b53         b54         p21 
0.502147813 0.270704137 0.112908747 0.158356978 0.608081905 0.419449230 
        p31         p32       p21_1       p31_1       p32_1 
0.242380384 0.236297228 0.020810720 0.009424315 0.008231119

## The model may be misspecified as the R2 of Tau2_7_7 is -0.86.
osmasemR2(mx.fit2, mx.fit0)

$Tau2.0
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.034393264 0.016741214 0.014075945 0.006432547 0.010115088 0.032620661 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.008168495 0.013249912 0.024094840 0.010040753 

$Tau2.1
   Tau2_1_1    Tau2_2_2    Tau2_3_3    Tau2_4_4    Tau2_5_5    Tau2_6_6 
0.023799714 0.012706156 0.013803063 0.005821852 0.007936229 0.018526443 
   Tau2_7_7    Tau2_8_8    Tau2_9_9  Tau2_10_10 
0.006808336 0.012242600 0.023775133 0.008301558 

$R2
  Tau2_1_1   Tau2_2_2   Tau2_3_3   Tau2_4_4   Tau2_5_5   Tau2_6_6 
0.30801236 0.24102540 0.01938644 0.09493842 0.21540684 0.43206415 
  Tau2_7_7   Tau2_8_8   Tau2_9_9 Tau2_10_10 
0.16651275 0.07602411 0.01326870 0.17321354

## Extract the variance component
VarCorr(mx.fit2)

            Tau2_1     Tau2_2     Tau2_3      Tau2_4      Tau2_5
Tau2_1  0.02379971 0.00000000 0.00000000 0.000000000 0.000000000
Tau2_2  0.00000000 0.01270616 0.00000000 0.000000000 0.000000000
Tau2_3  0.00000000 0.00000000 0.01380306 0.000000000 0.000000000
Tau2_4  0.00000000 0.00000000 0.00000000 0.005821852 0.000000000
Tau2_5  0.00000000 0.00000000 0.00000000 0.000000000 0.007936229
Tau2_6  0.00000000 0.00000000 0.00000000 0.000000000 0.000000000
Tau2_7  0.00000000 0.00000000 0.00000000 0.000000000 0.000000000
Tau2_8  0.00000000 0.00000000 0.00000000 0.000000000 0.000000000
Tau2_9  0.00000000 0.00000000 0.00000000 0.000000000 0.000000000
Tau2_10 0.00000000 0.00000000 0.00000000 0.000000000 0.000000000
            Tau2_6      Tau2_7    Tau2_8     Tau2_9     Tau2_10
Tau2_1  0.00000000 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_2  0.00000000 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_3  0.00000000 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_4  0.00000000 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_5  0.00000000 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_6  0.01852644 0.000000000 0.0000000 0.00000000 0.000000000
Tau2_7  0.00000000 0.006808336 0.0000000 0.00000000 0.000000000
Tau2_8  0.00000000 0.000000000 0.0122426 0.00000000 0.000000000
Tau2_9  0.00000000 0.000000000 0.0000000 0.02377513 0.000000000
Tau2_10 0.00000000 0.000000000 0.0000000 0.00000000 0.008301558

anova(mx.fit2, mx.fit0)

             base   comparison ep  minus2LL  df       AIC   diffLL diffdf
1 Sx as moderator         <NA> 21 -161.3619 106 -373.3619       NA     NA
2 Sx as moderator No moderator 18 -131.9660 109 -349.9660 29.39587      3
             p
1           NA
2 1.848993e-06

Installation and help

First of all, you need R to run it. You can install the metaSEM package by running the following command inside an R session:

install.packages("metaSEM")

The developmental version can be installed from GitHub by running:

library(devtools)
install_github("mikewlcheung/metasem")

Potential issues with the OpenMx available at CRAN

The OpenMx available at CRAN includes the open source SLSQP and CSOLNP optimizers.
If the SLSQP optimizer (default in the metaSEM package) does not work well for you, e.g., there are many error codes, you may try to rerun the analysis. For example,

random1 <- meta(y=yi, v=vi, data=Hox02)
random1 <- rerun(random1)
summary(random1)

If you still prefer the non-free NPSOL optimizer, you may install it from the OpenMx website and call it by running:

mxOption(NULL, "Default optimizer", "NPSOL")

Help

If you need help, please send your questions to OpenMx discussion forum, a discussion forum for the metaSEM package in OpenMx. Please include information on the R session. It will be helpful if you can include a reproducible example. You may save a copy of your data, say my.df, and attach the content of myData.R in the post by using
```
sessionInfo()
dump(c("my.df"), file="myData.R")
```

The files are based on the following versions of R and R packages:

sessionInfo()

R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.1 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.utf8       LC_NUMERIC=C             
 [3] LC_TIME=en_US.utf8        LC_COLLATE=en_US.utf8    
 [5] LC_MONETARY=en_US.utf8    LC_MESSAGES=en_US.utf8   
 [7] LC_PAPER=en_US.utf8       LC_NAME=C                
 [9] LC_ADDRESS=C              LC_TELEPHONE=C           
[11] LC_MEASUREMENT=en_US.utf8 LC_IDENTIFICATION=C      

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] semPlot_1.1    metafor_2.0-0  Matrix_1.2-14  metaSEM_1.1.1 
[5] OpenMx_2.11.5  knitr_1.20     rmarkdown_1.10

loaded via a namespace (and not attached):
 [1] nlme_3.1-137         RColorBrewer_1.1-2   rprojroot_1.3-2     
 [4] mi_1.0               tools_3.5.1          backports_1.1.2     
 [7] R6_2.2.2             rpart_4.1-13         d3Network_0.5.2.1   
[10] Hmisc_4.1-1          lazyeval_0.2.1       colorspace_1.3-2    
[13] nnet_7.3-12          tidyselect_0.2.4     gridExtra_2.3       
[16] mnormt_1.5-5         compiler_3.5.1       qgraph_1.5          
[19] fdrtool_1.2.15       htmlTable_1.12       network_1.13.0.1    
[22] scales_1.0.0         checkmate_1.8.5      mvtnorm_1.0-8       
[25] psych_1.8.4          pbapply_1.3-4        sem_3.1-9           
[28] stringr_1.3.1        digest_0.6.17        pbivnorm_0.6.0      
[31] foreign_0.8-71       minqa_1.2.4          base64enc_0.1-3     
[34] jpeg_0.1-8           pkgconfig_2.0.2      htmltools_0.3.6     
[37] lme4_1.1-18-1        lisrelToR_0.1.4      htmlwidgets_1.2     
[40] rlang_0.2.2          rstudioapi_0.7       huge_1.2.7          
[43] bindr_0.1.1          gtools_3.8.1         statnet.common_4.1.4
[46] acepack_1.4.1        dplyr_0.7.6          magrittr_1.5        
[49] Formula_1.2-3        Rcpp_0.12.19         munsell_0.5.0       
[52] abind_1.4-5          rockchalk_1.8.117    stringi_1.2.4       
[55] whisker_0.3-2        yaml_2.2.0           carData_3.0-1       
[58] MASS_7.3-50          plyr_1.8.4           matrixcalc_1.0-3    
[61] lavaan_0.6-3         grid_3.5.1           parallel_3.5.1      
[64] crayon_1.3.4         lattice_0.20-35      splines_3.5.1       
[67] sna_2.4              pillar_1.3.0         igraph_1.2.2        
[70] boot_1.3-20          rjson_0.2.20         corpcor_1.6.9       
[73] BDgraph_2.51         reshape2_1.4.3       stats4_3.5.1        
[76] XML_3.98-1.16        glue_1.3.0           evaluate_0.11       
[79] latticeExtra_0.6-28  data.table_1.11.6    png_0.1-7           
[82] nloptr_1.0.4         gtable_0.2.0         purrr_0.2.5         
[85] assertthat_0.2.0     ggplot2_3.0.0        semTools_0.5-1      
[88] coda_0.19-1          survival_2.42-6      glasso_1.10         
[91] tibble_1.4.2         arm_1.10-1           ggm_2.3             
[94] ellipse_0.4.1        bindrcpp_0.2.2       cluster_2.0.7-1

The metaSEM package: Examples

Mike W.-L. Cheung

18 October, 2018

Introduction

Univariate meta-analysis

Random-effects model

Fixed-effects model

Mixed-effects model

Multivariate meta-analysis

Random-effects model

Fixed-effects model

Mixed-effects model

Three-level meta-analysis

Comparisons between two- and three-level models with Cooper et al.’s (2003) dataset

Inspecting the data

Two-level meta-analysis

Three-level meta-analysis

More complex 3-level meta-analyses with Bornmann et al.’s (2007) dataset

Inspecting the data

Model 0: Intercept

Model 1: Type as a covariate

Model 2: Year and Year^2 as covariates

Model 3: Discipline as a covariate

Model 4: Country as a covariate

Model 5: Type and Discipline as covariates

Model 6: Type and Country as covariates

Model 7: Discipline and Country as covariates

Model 8: Type, Discipline, and Country as covariates

Handling missing covariates with FIML

MCAR

MAR

Two-stage structural equation modeling (TSSEM)

Inspect the data: Digman (1997)

Fixed-effects TSSEM

Stage 1 analysis

Stage 2 analysis

Fixed-effects TSSEM with cluster

Stage 1 analysis

Stage 2 analysis

Random-effects TSSEM

Stage 1 analysis

Stage 2 analysis

Becker and Schram (1994)

Inspect the data

Fixed-effects TSSEM

Stage 1 analysis

Stage 2 analysis

Fixed-effects TSSEM with cluster

Stage 1 analysis

Stage 2 analysis

Random-effects TSSEM

Stage 1 analysis

Stage 2 analysis

One-stage meta-analytic structural equation modeling (OSMASEM)

Nohe15

Data preparation

Fitting a just-identified model (df=0)

Using Lag as a moderator on the A matrix

Using Lag as a moderator on the S matrix

Using Lag as a moderator on both the A and S matrices

Roorda11

Data preparation

Fitting a just-identified model (df=0)

Fitting an over-identified model with b41=b42=0 (df=2))

Using SES as a moderator on the A matrix

Using SES as a moderator on the S matrix

Scalco17

Data preparation

Fitting an over-identified model

Using Age as a moderator on the A matrix

Using Age as a moderator on the S matrix

Installation and help

Potential issues with the OpenMx available at CRAN

Help

Model 1: `Type` as a covariate

Model 2: `Year` and `Year^2` as covariates

Model 3: `Discipline` as a covariate

Model 4: `Country` as a covariate

Model 5: `Type` and `Discipline` as covariates

Model 6: `Type` and `Country` as covariates

Model 7: `Discipline` and `Country` as covariates

Model 8: `Type`, `Discipline`, and `Country` as covariates

Using `Lag` as a moderator on the A matrix

Using `Lag` as a moderator on the S matrix

Using `Lag` as a moderator on both the A and S matrices

Using `SES` as a moderator on the A matrix

Using `SES` as a moderator on the S matrix

Using `Age` as a moderator on the A matrix

Using `Age` as a moderator on the S matrix