The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Vignette 2. Psychometric meta-analysis with metaConvert (COSMIN framework)

Gosling CJ, Cortese S, Solmi M, Haza B, Vieta E, Delorme R, Fusar-Poli P, Radua J

2026-07-20

Introduction

The COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) framework provides a standardised taxonomy of measurement properties for evaluating patient-reported outcome measures (PROMs). Conducting a meta-analysis of measurement properties requires computing, transforming, and pooling a variety of effect sizes that are specific to psychometric studies.

The metaConvert package provides functions for several COSMIN measurement properties. The table below summarises the coverage:

COSMIN property	Coverage	metaConvert function(s)
Internal consistency	Full	`convert_df(measure = "alpha")`
Test-retest reliability	Full	`convert_df(measure = "icc")`
Measurement error	Full	`compute_sem()`, `compute_sdc()`
Criterion validity	Full	`convert_df(measure = "r")` + `es_disattenuate()`
Construct validity	Full	`convert_df(measure = "r")` + `es_disattenuate()`
Responsiveness	Full	`reliability_change_score()` + `es_disattenuate()`
Floor/ceiling effects	Full	`convert_df(measure = "prop")`
Structural validity	Not covered	Requires OSMASEM (metaSEM / OpenMx)
Diagnostic accuracy	Not covered	Requires bivariate GLMM (mada / lme4)
Cross-cultural validity	Partial	DIF log-odds via OR pipeline; invariance is descriptive
Interpretability (MIC)	Partial	Raw MIC via user-provided ES columns

Throughout this vignette, we use the df.psychom dataset distributed with metaConvert. It simulates a COSMIN systematic review of a fictional 9-item self-report PROM called the Mental Health Wellbeing Scale (MHWS), scored 0–27, validated across multiple populations and languages.

data(df.psychom)

str(df.psychom)

Each row represents one study reporting one measurement property. The outcome column identifies the COSMIN property assessed. Columns not relevant to a study’s property are NA, reflecting the typical sparsity of real-world data extraction sheets.

Data preparation

Before data extraction, you can generate a template extraction sheet for each psychometric measure. These templates list all column names that metaConvert recognises.

# Extraction sheet for Cronbach's alpha
data_extraction_sheet(measure = "alpha", extension = "data.frame")

# Extraction sheet for ICC
data_extraction_sheet(measure = "icc", extension = "data.frame")

# Extraction sheet for correlations (Pearson or Spearman)
data_extraction_sheet(measure = "r", extension = "data.frame")

# Extraction sheet for proportions
data_extraction_sheet(measure = "prop", extension = "data.frame")

The key psychometric columns are:

Column	Type	Used by
`cronbach_alpha`	numeric	`es_from_cronbach_alpha()`
`n_items`	numeric	`es_from_cronbach_alpha()`
`icc`	numeric	`es_from_icc()`
`n_measurements`	numeric	`es_from_icc()`
`icc_type`	character	`es_from_icc()` – `"agreement"` or `"consistency"`
`spearman_r`	numeric	`es_from_spearman_rho()`
`prop`	numeric	`es_from_prop_single_group()`
`n_cases`	numeric	`es_from_prop_single_group_counts()`

Internal consistency

Internal consistency is typically assessed using Cronbach’s alpha. For meta-analysis, the Bonett (2002) variance-stabilising transformation is recommended:

\[T(\alpha) = \ln(1 - \alpha), \quad \text{SE} = \sqrt{\frac{2k}{(k-1)(n-2)}}\]

where $k$ is the number of items and $n$ is the sample size (Bonett, 2002). This transformation normalises the sampling distribution and stabilises the variance, which is particularly important when alpha values are close to 1.

Effect size computation

# Subset internal consistency studies
dat_alpha <- df.psychom[df.psychom$outcome == "internal_consistency", ]

# Compute effect sizes with Bonett transformation (default)
res_alpha <- convert_df(dat_alpha, measure = "alpha",
                        verbose = FALSE, split_adjusted = FALSE)
summary_alpha <- summary(res_alpha, flags = TRUE, guidance = TRUE)
#> 
#> -- metaConvert summary --
#> Measure: Cronbach's alpha  |  8 studies
#> ES estimated: 8/8 (100%)
#> 
#> Methods selected:
#>   cronbach_alpha            8

summary_alpha[, c("author", "es", "se", "es_ci_lo", "es_ci_up", "info_used")]

The es column contains Bonett-transformed values $\ln(1 - \alpha)$, which are always negative (since $\alpha \in [0, 1)$). To back-transform for reporting:

\[\hat{\alpha} = 1 - \exp(T)\]

Meta-analysis

# Random-effects meta-analysis on Bonett-transformed scale
ma_alpha <- metafor::rma(yi = es, sei = se, data = summary_alpha,
                         method = "REML")

# Forest plot
metafor::forest(ma_alpha, slab = paste0(summary_alpha$author, " (", dat_alpha$year, ")"),
                xlab = "Bonett-transformed alpha: ln(1 - alpha)",
                header = TRUE)

# Back-transform pooled estimate and CI
pooled_alpha    <- 1 - exp(ma_alpha$beta[1])
pooled_alpha_lo <- 1 - exp(ma_alpha$ci.ub)
pooled_alpha_up <- 1 - exp(ma_alpha$ci.lb)

cat(sprintf("Pooled alpha = %.3f [%.3f, %.3f]\n",
            pooled_alpha, pooled_alpha_lo, pooled_alpha_up))
#> Pooled alpha = 0.867 [0.818, 0.902]

Note that the CI bounds are swapped when back-transforming because $1 - \exp(x)$ is a decreasing function of $x$.

The pooled reliability estimate can later serve as input for disattenuation corrections in criterion validity, construct validity, and responsiveness analyses (see below).

Alternative: raw alpha

An alternative is to meta-analyse raw Cronbach’s alpha values directly (alpha_to_es = "raw"), using the SE formula from Feldt et al. (1987). This is simpler to interpret but the sampling distribution may be skewed when alpha is close to 1. The Bonett transformation is generally recommended.

# Raw alpha (not recommended for meta-analysis)
res_alpha_raw <- convert_df(dat_alpha, measure = "alpha", alpha_to_es = "raw",
                            verbose = FALSE, split_adjusted = FALSE)
summary(res_alpha_raw)

Test-retest reliability

Test-retest reliability is assessed using the intraclass correlation coefficient (ICC). The same Bonett (2002) transformation is applied:

\[T(\text{ICC}) = \ln(1 - \text{ICC})\]

The SE of the transformed ICC is obtained by the delta method: the $(1-\rho)^2$ term from the raw ICC variance cancels with the $1/(1-\rho)^2$ from the transformation derivative. For a single-measure ICC, this leading-order variance is the same for the two-way absolute-agreement ICC(2,1) and the two-way consistency ICC(3,1):

ICC(2,1) – absolute agreement (icc_type = "agreement") and ICC(3,1) – consistency (icc_type = "consistency"): \[\text{SE}(T) = \sqrt{\frac{2(1+(k-1)\rho)^2}{k(k-1)(n-1)}}\]

where $k$ is the number of measurement occasions and $\rho$ is the ICC value. Deriving the ICC(3,1) variance directly from $F_0 = \text{MSR}/\text{MSE}$ (degrees of freedom $n-1$ and $(n-1)(k-1)$) yields $\left[(1+(k-1)\rho)/k\right]^2 \cdot 2k/((k-1)(n-1))$, which reduces to the same expression — so, contrary to a common simplification, the ICC(3,1) SE is not independent of $\rho$ (a Monte Carlo check confirms the $\rho$-dependent form; a $\rho$-free $\sqrt{2/((k-1)(n-1))}$ under-states the SE by up to ~2–3$\times$ in variance for high reliabilities).

Technical note on the ICC(2,1) formula: The SE formula labelled “ICC(2,1)” above is technically the analytical variance for the one-way random ICC(1,1) model (Shrout & Fleiss, 1979), as derived by Bonett (2002). The exact two-way random ICC(2,1) variance formula involves separate between-rater and residual variance components that are typically unavailable without the raw ANOVA mean squares. Using the one-way formula as a proxy for ICC(2,1) is standard practice in psychometric meta-analysis (see e.g., Bonett, 2002; Donner & Eliasziw, 1987) and provides a reasonable approximation.

Effect size computation

# Subset test-retest studies
dat_icc <- df.psychom[df.psychom$outcome == "test_retest", ]

# Compute effect sizes with Bonett transformation (default)
res_icc <- convert_df(dat_icc, measure = "icc",
                      verbose = FALSE, split_adjusted = FALSE)
summary_icc <- summary(res_icc, flags = TRUE)
#> 
#> -- metaConvert summary --
#> Measure: ICC  |  7 studies
#> ES estimated: 7/7 (100%)
#> 
#> Methods selected:
#>   icc                       7
#> 5 quality flag(s) raised

summary_icc[, c("author", "es", "se", "es_ci_lo", "es_ci_up",
                 "icc_type", "info_used")]

Note that studies using different ICC models (agreement vs. consistency) can coexist in the same analysis. Both types share the same leading-order SE formula: it is exact for consistency ICC(3,1), but for agreement ICC(2,1) it is a one-way approximation that assumes negligible between-rater variance – when raters differ systematically, the reported SE is anti-conservative, and summary() marks agreement-type rows with an informational flag (see ?es_from_icc). When pooling, icc_type can be examined as a moderator variable.

Meta-analysis

# Random-effects meta-analysis
ma_icc <- metafor::rma(yi = es, sei = se, data = summary_icc,
                       method = "REML")

# Forest plot
metafor::forest(ma_icc, slab = paste0(summary_icc$author, " (", dat_icc$year, ")"),
                xlab = "Bonett-transformed ICC: ln(1 - ICC)",
                header = TRUE)

# Back-transform pooled estimate
pooled_icc    <- 1 - exp(ma_icc$beta[1])
pooled_icc_lo <- 1 - exp(ma_icc$ci.ub)
pooled_icc_up <- 1 - exp(ma_icc$ci.lb)

cat(sprintf("Pooled ICC = %.3f [%.3f, %.3f]\n",
            pooled_icc, pooled_icc_lo, pooled_icc_up))
#> Pooled ICC = 0.825 [0.754, 0.875]

The pooled ICC will feed into the measurement error derivation (next section) and disattenuation corrections for validity and responsiveness.

Studies that report only Pearson correlations for test-retest reliability (rather than ICC) can be pooled separately using convert_df(..., measure = "r").

Measurement error

Measurement error is characterised by the standard error of measurement (SEM) and the smallest detectable change (SDC). These are not traditional effect sizes but are essential COSMIN measurement properties.

The SEM is derived from matched ICC and SD data:

\[\text{SEM} = \text{SD} \times \sqrt{1 - \text{ICC}}\]

The SDC is derived from the SEM:

\[\text{SDC} = 1.96 \times \sqrt{2} \times \text{SEM}\]

These are computed using standalone utility functions (not convert_df()), because they require combining reliability and variability from the same sample.

Study-specific SEM and SDC

# Use test-retest studies that report both ICC and SD
dat_me <- df.psychom[df.psychom$outcome == "test_retest" &
                     !is.na(df.psychom$sd_scores), ]

# Compute SEM for each study (delta-method SE included)
sem_results <- compute_sem(
  sd       = dat_me$sd_scores,
  icc      = dat_me$icc,
  n_sample = dat_me$n_sample
)

# Chain to SDC
sdc_results <- compute_sdc(
  sem    = sem_results$sem,
  sem_se = sem_results$sem_se
)

# Combined measurement error table
measurement_error <- data.frame(
  study  = dat_me$study_id,
  n      = dat_me$n_sample,
  sd     = dat_me$sd_scores,
  icc    = dat_me$icc,
  sem    = round(sem_results$sem, 3),
  sem_se = round(sem_results$sem_se, 3),
  sdc    = round(sdc_results$sdc, 3),
  sdc_se = round(sdc_results$sdc_se, 3)
)
measurement_error

Pooling SEM

The study-specific SEM values can be pooled using inverse-variance random-effects meta-analysis, with the delta-method SE providing the sampling variance:

ma_sem <- metafor::rma(yi = sem_results$sem, sei = sem_results$sem_se,
                       method = "REML")

metafor::forest(ma_sem,
                slab = paste0(dat_me$author, " (", dat_me$year, ")"),
                xlab = "Standard Error of Measurement (SEM)",
                header = TRUE)

cat(sprintf("Pooled SEM = %.3f [%.3f, %.3f]\n",
            ma_sem$beta[1], ma_sem$ci.lb, ma_sem$ci.ub))
#> Pooled SEM = 2.227 [2.044, 2.410]

# Derive pooled SDC from pooled SEM
pooled_sdc <- 1.96 * sqrt(2) * ma_sem$beta[1]
cat(sprintf("Pooled SDC = %.3f\n", pooled_sdc))
#> Pooled SDC = 6.172

For interpretability, the ratio of pooled SDC to pooled MIC (minimal important change) indicates whether the instrument can reliably detect changes that patients consider meaningful. A ratio $< 1$ (SDC < MIC) is desirable.

Important: SEM and SDC are expressed in the raw units of the measurement instrument. Pooling is only valid when all primary studies use the exact same version of the instrument with the same scoring range. If some studies use the original 0–27 MHWS and others use a modified 0–100 version, their SEMs are on different scales and cannot be pooled directly. In such cases, consider standardising by the score SD (computing the coefficient of variation SEM/SD) or pooling only within homogeneous subgroups.

Criterion and construct validity

Criterion and construct validity are assessed by pooling correlations between the MHWS and comparator instruments. Some studies report Pearson’s $r$, others report Spearman’s $\rho$. The metaConvert pipeline handles both: Spearman correlations are automatically converted to Pearson-equivalent values using the Rupinski & Dunlap (1996) formula:

\[r_{\text{Pearson}} = 2 \sin\!\left(\frac{\pi}{6} \times r_{\text{Spearman}}\right)\]

The SE is derived via the delta method, correctly propagating the uncertainty from the conversion.

Results should be reported three ways:

(a) Uncorrected: observed correlations pooled directly
(b) Individually corrected: study-specific disattenuation, then pooled
(c) Artifact-distribution corrected: using the distribution of reliabilities across studies (Hunter & Schmidt method; see the psychmeta package)

This vignette demonstrates approaches (a) and (b).

Effect size computation

# Subset criterion and construct validity studies
dat_validity <- df.psychom[df.psychom$outcome %in%
                           c("criterion_validity", "construct_validity"), ]

# convert_df automatically handles both Pearson r and Spearman rho
res_r <- convert_df(dat_validity, measure = "r",
                    verbose = FALSE, split_adjusted = FALSE)
summary_r <- summary(res_r, flags = TRUE)
#> 
#> -- metaConvert summary --
#> Measure: Pearson r  |  7 studies
#> ES estimated: 7/7 (100%)
#> 
#> Methods selected:
#>   pearson_r                 5
#>   spearman_r                2
#> 1 quality flag(s) raised

summary_r[, c("author", "es", "se", "info_used")]

The info_used column distinguishes between "pearson_r" and "spearman_r" sources, providing traceability for each study’s conversion pathway.

(a) Pooling uncorrected correlations

Meta-analysis of correlations is typically performed on the Fisher’s z scale, then back-transformed. Here, convert_df with measure = "r" returns correlations in the es column. The SE on the Fisher’s z scale is derived via the delta method: $\text{SE}(z) = \text{SE}(r) / (1 - r^2)$, which correctly propagates the uncertainty from any upstream conversion (e.g., Spearman to Pearson):

# Pool observed (uncorrected) correlations on Fisher's z scale
z_obs <- atanh(summary_r$es)
r_bounded <- pmin(pmax(summary_r$es, -0.9999), 0.9999)
z_se  <- summary_r$se / (1 - r_bounded^2)

ma_r_obs <- metafor::rma(yi = z_obs, sei = z_se, method = "REML")

# Forest plot on correlation scale
metafor::forest(ma_r_obs,
                slab = paste0(summary_r$author, " (", dat_validity$year, ")"),
                xlab = "Fisher's z (observed correlation)",
                header = TRUE)

# Back-transform to correlation scale
pooled_r_obs <- tanh(ma_r_obs$beta[1])
pooled_r_lo  <- tanh(ma_r_obs$ci.lb)
pooled_r_up  <- tanh(ma_r_obs$ci.ub)

cat(sprintf("(a) Pooled uncorrected r = %.3f [%.3f, %.3f]\n",
            pooled_r_obs, pooled_r_lo, pooled_r_up))
#> (a) Pooled uncorrected r = 0.555 [0.236, 0.767]

(b) Individual disattenuation then pooling

The es_disattenuate() function corrects each observed correlation for measurement error in both the target PROM and the comparator instrument:

\[r_{\text{corrected}} = \frac{r_{\text{observed}}}{\sqrt{\text{rel}_X \times \text{rel}_Y}}\]

# Disattenuate using study-specific reliabilities
corrected <- es_disattenuate(
  r             = summary_r$es,
  r_se          = summary_r$se,
  reliability_x = dat_validity$reliability_target,
  reliability_y = dat_validity$reliability_comparator,
  n_sample      = dat_validity$n_sample
)

# Comparison table
data.frame(
  study              = dat_validity$study_id,
  info_used          = summary_r$info_used,
  r_observed         = round(summary_r$es, 3),
  r_corrected        = round(corrected$r_corrected, 3),
  attenuation_factor = round(corrected$attenuation_factor, 3)
)

The corrected correlations are systematically larger in magnitude, reflecting the removal of attenuation due to measurement error.

Note on bounds: Because sampling error exists in both the observed correlation and the reliability estimates, r_corrected can occasionally exceed 1.0 in absolute value. When this happens, es_disattenuate() internally caps the corrected correlation at $$0.9999 before computing Fisher’s z, so the meta-analytic pipeline will not produce NaN values.

However, a corrected correlation exceeding 1.0 is a red flag that warrants investigation. Common causes include: (1) the reliability coefficient was estimated from a different population than the study sample; (2) the reliability was based on a very small sample and is itself imprecise; or (3) range restriction artefacts interact with the correction. We recommend running a sensitivity analysis excluding studies where $|r_c| > 1$ and comparing results. If multiple studies produce impossible values, an artifact-distribution method (Hunter & Schmidt, 2004; see the psychmeta package) may be more appropriate than individual study-level correction, as it uses the distribution of reliability estimates rather than point values.

# Pool corrected correlations on Fisher's z scale
ma_r_corr <- metafor::rma(yi = corrected$z_corrected,
                           sei = corrected$z_corrected_se,
                           method = "REML")

metafor::forest(ma_r_corr,
                slab = paste0(summary_r$author, " (", dat_validity$year, ")"),
                xlab = "Fisher's z (disattenuated correlation)",
                header = TRUE)

pooled_r_corr <- tanh(ma_r_corr$beta[1])
pooled_r_corr_lo <- tanh(ma_r_corr$ci.lb)
pooled_r_corr_up <- tanh(ma_r_corr$ci.ub)

cat(sprintf("(b) Pooled disattenuated r = %.3f [%.3f, %.3f]\n",
            pooled_r_corr, pooled_r_corr_lo, pooled_r_corr_up))
#> (b) Pooled disattenuated r = 0.675 [0.315, 0.865]

Using pooled reliability when study-specific data are unavailable

When study-specific reliability data are unavailable, the pooled reliability estimates from the internal consistency (Section 2) and test-retest (Section 3) meta-analyses can be used:

# Use pooled alpha and ICC from earlier sections as proxies
es_disattenuate(
  r = 0.65,
  r_se = 0.08,
  reliability_x = pooled_alpha,  # from Section 2
  reliability_y = 0.85,          # from external published data
  n_sample = 150
)

Responsiveness

Responsiveness is assessed by correlating change scores on the MHWS with change scores on a comparator instrument (e.g., CGI-Change, PGIC). Because change scores have lower reliability than single-occasion scores, disattenuation requires change-score reliability, not single-occasion reliability.

The Lord (1963) formula derives change-score reliability from single-occasion reliability and the pre-post correlation:

\[\text{rel}_{\text{change}} = \frac{\text{rel} - r_{\text{pre-post}}}{1 - r_{\text{pre-post}}}\]

Effect size computation with change-score correction

# Subset responsiveness studies
dat_resp <- df.psychom[df.psychom$outcome == "responsiveness", ]

# Step 1: Compute change-score reliability
change_rel <- reliability_change_score(
  reliability = dat_resp$single_occasion_reliability,
  r_pre_post  = dat_resp$r_pre_post
)

# Step 2: Get responsiveness correlations via convert_df
res_resp <- convert_df(dat_resp, measure = "r",
                       verbose = FALSE, split_adjusted = FALSE)
summary_resp <- summary(res_resp)
#> 
#> -- metaConvert summary --
#> Measure: Pearson r  |  4 studies
#> ES estimated: 4/4 (100%)
#> 
#> Methods selected:
#>   pearson_r                 4

# Step 3: Disattenuate using CHANGE-SCORE reliability
corrected_resp <- es_disattenuate(
  r             = summary_resp$es,
  r_se          = summary_resp$se,
  reliability_x = change_rel$rel_change,
  reliability_y = dat_resp$reliability_comparator,
  n_sample      = dat_resp$n_sample
)

data.frame(
  study        = dat_resp$study_id,
  r_observed   = round(summary_resp$es, 3),
  rel_single   = dat_resp$single_occasion_reliability,
  r_pre_post   = dat_resp$r_pre_post,
  rel_change   = round(change_rel$rel_change, 3),
  r_corrected  = round(corrected_resp$r_corrected, 3)
)

Note that change-score reliability (rel_change) is always lower than single-occasion reliability (rel_single), which leads to a larger disattenuation correction.

Meta-analysis

# Pool observed change-score correlations
z_resp     <- atanh(summary_resp$es)
r_resp_bounded <- pmin(pmax(summary_resp$es, -0.9999), 0.9999)
z_resp_se  <- summary_resp$se / (1 - r_resp_bounded^2)

ma_resp_obs <- metafor::rma(yi = z_resp, sei = z_resp_se, method = "REML")

cat(sprintf("Pooled observed r = %.3f [%.3f, %.3f]\n",
            tanh(ma_resp_obs$beta[1]),
            tanh(ma_resp_obs$ci.lb),
            tanh(ma_resp_obs$ci.ub)))
#> Pooled observed r = 0.502 [0.419, 0.577]

# Pool disattenuated change-score correlations
ma_resp_corr <- metafor::rma(yi = corrected_resp$z_corrected,
                              sei = corrected_resp$z_corrected_se,
                              method = "REML")

cat(sprintf("Pooled disattenuated r = %.3f [%.3f, %.3f]\n",
            tanh(ma_resp_corr$beta[1]),
            tanh(ma_resp_corr$ci.lb),
            tanh(ma_resp_corr$ci.ub)))
#> Pooled disattenuated r = 0.625 [0.483, 0.735]

Sensitivity analysis for unknown pre-post correlations

When pre-post correlations are not reported, a sensitivity analysis using plausible values illustrates the impact on the corrected estimates:

# Example: one study with r_observed = 0.50, rel_single = 0.85
r_pre_post_grid <- c(0.20, 0.40, 0.60, 0.80)

sensitivity <- data.frame(
  r_pre_post = r_pre_post_grid,
  rel_change = sapply(r_pre_post_grid, function(rpp) {
    reliability_change_score(reliability = 0.85, r_pre_post = rpp)$rel_change
  }),
  r_corrected = sapply(r_pre_post_grid, function(rpp) {
    rc <- reliability_change_score(reliability = 0.85, r_pre_post = rpp)$rel_change
    es_disattenuate(r = 0.50, r_se = 0.08, reliability_x = rc,
                    reliability_y = 0.80, n_sample = 100)$r_corrected
  })
)
#> Warning: Corrected r exceeds 0.999 in 1 row(s); the CI and Fisher's z outputs
#> are set to NA as unreliable. Check the reliability inputs (and the observed r)
#> - this often reflects a reliability value that is too low for the observed
#> correlation.
sensitivity
#>   r_pre_post rel_change r_corrected
#> 1        0.2     0.8125   0.6201737
#> 2        0.4     0.7500   0.6454972
#> 3        0.6     0.6250   0.7071068
#> 4        0.8     0.2500   1.1180340

As the pre-post correlation increases, change-score reliability decreases, and the disattenuation correction becomes larger. This highlights why reporting pre-post correlations in primary studies is essential.

Floor and ceiling effects

Floor and ceiling effects can be assessed by meta-analysing the proportion of respondents scoring at the minimum or maximum of the scale. Three transformations are available:

"raw" (default): raw proportion, SE = $\sqrt{p(1-p)/n}$. CIs bounded to [0, 1].
"logit": logit transform $\log(p/(1-p))$, SE = $\sqrt{1/(np(1-p))}$
"freeman_tukey" (recommended): Freeman-Tukey double arcsine. Handles zero proportions naturally and stabilises variance.

Effect size computation and pooling

# Subset floor/ceiling studies
dat_prop <- df.psychom[df.psychom$outcome == "floor_ceiling", ]

# Freeman-Tukey transformation (recommended)
res_prop <- convert_df(dat_prop, measure = "prop", prop_to_es = "freeman_tukey",
                       verbose = FALSE, split_adjusted = FALSE)
summary_prop <- summary(res_prop)
#> 
#> -- metaConvert summary --
#> Measure: Proportion  |  4 studies
#> ES estimated: 4/4 (100%)
#> 
#> Methods selected:
#>   prop_single_group         2
#>   prop_single_group_counts  2

ma_prop <- metafor::rma(yi = es, sei = se, data = summary_prop,
                        method = "REML")

metafor::forest(ma_prop,
                slab = paste0(summary_prop$author, " (", dat_prop$year, ")"),
                xlab = "Freeman-Tukey transformed proportion",
                header = TRUE)

# Approximate back-transformation (for the pooled estimate)
pooled_prop <- sin(ma_prop$beta[1])^2
cat(sprintf("Pooled proportion (approx.) = %.3f\n", pooled_prop))
#> Pooled proportion (approx.) = 0.088

Note on the Freeman-Tukey back-transformation: The sin(x)^2 formula is an approximation. Recent methodological work (Schwarzer et al., 2019, Research Synthesis Methods) has shown that the Freeman-Tukey double arcsine back-transformation can distort pooled results when sample sizes are heterogeneous across studies. For proportions near 0 or 1 (as is typical for floor/ceiling effects), a generalised linear mixed model (GLMM) with logit link — available via metafor::rma.glmm() — may be preferable as it avoids the transformation entirely and directly models the binomial counts.

Beyond metaConvert

Several COSMIN measurement properties require specialised methods not covered by metaConvert:

Structural validity requires meta-analytic structural equation modelling (OSMASEM; Jak & Cheung, 2020) to pool inter-item correlation matrices and fit confirmatory factor models. See the metaSEM and OpenMx packages.

Diagnostic and screening accuracy requires the bivariate generalised linear mixed model (Chu & Cole, 2006) for jointly modelling sensitivity and specificity. See the mada and lme4 packages, and the HSROC model for summary ROC curves.

Cross-cultural validity and measurement invariance is primarily a descriptive synthesis (tabulating configural, metric, scalar invariance levels). When DIF statistics (log-odds ratios) are available, they can be pooled using metaConvert’s existing OR pipeline via convert_df(measure = "or").

Interpretability (MIC) can be partially handled: if you have MIC estimates and their SEs, enter them as user-provided effect sizes via user_es_measure_crude, user_es_crude, and user_se_crude columns, then use convert_df() to organise them alongside other effect sizes.

Feasibility and acceptability are synthesised narratively, without quantitative pooling.

Summary

COSMIN property	metaConvert step	Key function(s)
Internal consistency	`convert_df(measure = "alpha")`	`es_from_cronbach_alpha()`
Test-retest reliability	`convert_df(measure = "icc")`	`es_from_icc()`
Measurement error	Standalone	`compute_sem()` + `compute_sdc()`
Criterion / construct validity	`convert_df(measure = "r")` then `es_disattenuate()`	`es_from_pearson_r()`, `es_from_spearman_rho()`, `es_disattenuate()`
Responsiveness	`reliability_change_score()` then `es_disattenuate()`	Full chain
Floor/ceiling effects	`convert_df(measure = "prop")`	`es_from_prop_single_group()`

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.