Was silence informative? The silence test, the sensor-gap test and the fatigue check

The question

A declared dm-graph says which mechanism is assumed (see vignette("dm-graphs")). The tests in this vignette ask the data whether that assumption is tenable. They exploit one fact: under every recoverable mechanism, the response indicator at prompt t carries no information about the states beyond what the observed history already carries. If skipped prompts turn out to predict what is observed afterwards, silence was informative.

Two tests and one descriptive check are available.

The two tests use person fixed effects (within-person demeaning), so stable between-person differences in level or in compliance cannot produce a spurious result. The fatigue check reports both a pooled contrast, which between-person differences in compliance inflate, and the mean of the person-specific contrasts, which they do not.

The silence test

Take every triple of consecutive scheduled prompts (t-1, t, t+1) in which the two outer prompts were answered. The middle prompt may or may not have been answered. Regress the state at t+1 on the state at t-1 and the response indicator at t, within person. If the middle skip is uninformative, its coefficient is zero.

sim2 <- simulate_ema(N = 60, n_prompts = 40, motifs = "M2", compliance = 0.7, delta = -1, seed = 1)
st <- silence_test(sim2$data, sim2$vars)
st
#> Silence test: X_{t+1} ~ X_{t-1} + R_t (within person), 1187 triples, 226 with a skipped middle prompt
#>  variable coef_R    se     z       p
#>      NegA -0.299 0.075 -4.00 0.00018
#>      PosA  0.161 0.076  2.13 0.03800
#>    Stress -0.147 0.070 -2.12 0.03800
#>   Fatigue  0.030 0.077  0.39 0.70000
#> Joint Wald test: statistic = 17.84 on 4 df, p = 0.00324   [SE: cluster; t / F with G - 1 df reference]
#> Interpretation: coefficients near zero are expected under the recoverable motifs; a negative coefficient on a variable means the states hidden by skips were higher.

The data were generated with self-censoring on negative affect (delta = -1: the probit propensity to respond falls by one unit per unit of negative affect). The coefficient of R_t for NegA is negative: after a skipped prompt (R_t = 0), negative affect at t+1 is higher than after an answered one, given the state at t-1. The states hidden by skips were higher than the reported ones, which is exactly what self-censoring produces. The joint Wald test over all four variables combines the evidence.

The result object is a data frame with attributes:

attr(st, "joint")
#>        chisq           df            p 
#> 17.841240732  4.000000000  0.003235468
c(triples = attr(st, "n_triples"), skipped_middle = attr(st, "n_skipped"))
#>        triples skipped_middle 
#>           1187            226

Under a recoverable mechanism the coefficients are near zero and the joint test does not reject:

sim1 <- simulate_ema(N = 60, n_prompts = 40, motifs = "M1", compliance = 0.7, seed = 1)
silence_test(sim1$data, sim1$vars)
#> Silence test: X_{t+1} ~ X_{t-1} + R_t (within person), 1147 triples, 276 with a skipped middle prompt
#>  variable coef_R    se     z     p
#>      NegA -0.038 0.072 -0.53 0.600
#>      PosA  0.138 0.070  1.96 0.054
#>    Stress  0.035 0.077  0.46 0.650
#>   Fatigue -0.063 0.081 -0.77 0.440
#> Joint Wald test: statistic = 4.82 on 4 df, p = 0.318   [SE: cluster; t / F with G - 1 df reference]
#> Interpretation: coefficients near zero are expected under the recoverable motifs; a negative coefficient on a variable means the states hidden by skips were higher.

one <- simulate_ema(N = 1, n_prompts = 200, motifs = "M2", compliance = 0.7, delta = -1, days = 8, seed = 5)
silence_test(one$data, one$vars, day = "day", B = 100, seed = 1)
#> Silence test: X_{t+1} ~ X_{t-1} + R_t (within person), 73 triples, 15 with a skipped middle prompt
#>  variable coef_R    se     z    p
#>      NegA -0.359 0.314 -1.14 0.25
#>      PosA  0.194 0.216  0.90 0.37
#>    Stress  0.145 0.346  0.42 0.68
#>   Fatigue -0.018 0.327 -0.05 0.96
#> Joint Wald test: statistic = 3.57 on 4 df, p = 0.467   [SE: dayblock; normal / chi-square reference]
#> Interpretation: coefficients near zero are expected under the recoverable motifs; a negative coefficient on a variable means the states hidden by skips were higher.

silence_test(sim1$data, sim1$vars, poly = 2)$p
#> [1] 0.63091580 0.05951182 0.61709757 0.41464900

sim13 <- simulate_ema(N = 300, n_prompts = 56, motifs = c("M1", "M3"), compliance = 0.7, kappa_R = 1, seed = 2)
silence_test(sim13$data, sim13$vars)
#> Silence test: X_{t+1} ~ X_{t-1} + R_t (within person), 8713 triples, 830 with a skipped middle prompt
#>  variable coef_R    se     z      p
#>      NegA  0.122 0.038  3.21 0.0015
#>      PosA -0.073 0.038 -1.94 0.0540
#>    Stress  0.029 0.035  0.82 0.4100
#>   Fatigue  0.011 0.040  0.28 0.7800
#> Joint Wald test: statistic = 12.37 on 4 df, p = 0.0162   [SE: cluster; t / F with G - 1 df reference]
#> Interpretation: coefficients near zero are expected under the recoverable motifs; a negative coefficient on a variable means the states hidden by skips were higher.
recoverability(dm_graph(c("M1", "M3")))$estimator[6]
#> [1] "null violated although the kernel is recoverable (collider at R_{t+1}); use the sensor-gap test or check for fatigue first"

The sensor-gap test

An always-observed channel that loads on the states (an accelerometer, heart rate, phone-usage features) turns the question around: instead of asking what happens after a skip, ask what the sensor recorded at the skipped prompt. sensor_gap_test() regresses the sensor at t on the state at t-1 and R_t, within person, over all prompts whose predecessor was answered. Nothing at t+1 is conditioned on, so the test is valid under M1 + M3 as well.

sim2s <- simulate_ema(N = 60, n_prompts = 40, motifs = "M2", compliance = 0.7, delta = -1,
                      sensor_cor = 0.6, seed = 3)
sg <- sensor_gap_test(sim2s$data, sim2s$vars, sensor = "S")
sg
#> Sensor-gap test: S_t ~ X_{t-1} + R_t (within person), 1640 prompts with answered predecessor
#>   coefficient of R_t: -0.611 (SE 0.052), z = -11.86, p = 2.92e-17   [SE: cluster]
#>   sensor loading on states (answered prompts): 0.571 -0.003 0.025 -0.011
attr(sg, "loading")
#>        NegA        PosA      Stress     Fatigue 
#>  0.57076631 -0.00251733  0.02456381 -0.01115139

The sensor gap is negative: at skipped prompts the sensor recorded higher values of what it measures. The loading attribute gives the within-person regression of the sensor on the states at answered prompts; the calibration in vignette("sensitivity-analysis") uses it to translate the gap into a sensitivity value.

Under a recoverable mechanism the gap is null:

sim1s <- simulate_ema(N = 60, n_prompts = 40, motifs = c("M1", "M3"), compliance = 0.7, sensor_cor = 0.6, seed = 3)
sensor_gap_test(sim1s$data, sim1s$vars, sensor = "S")
#> Sensor-gap test: S_t ~ X_{t-1} + R_t (within person), 1635 prompts with answered predecessor
#>   coefficient of R_t: -0.056 (SE 0.070), z = -0.80, p = 0.425   [SE: cluster]
#>   sensor loading on states (answered prompts): 0.585 0.001 0.029 0.025

The fatigue check

fatigue_check() is descriptive. It reports the response rate after an answered and after a skipped prompt, pooled and as the mean of the person-specific differences, and the response rate by study quarter. A large persistence contrast (the kappa_R term of the simulator) and, when compliance drifts over the study (a negative burden term), a declining rate over quarters are what burden (M3) produces; the example below uses kappa_R only, so the quarterly rates stay flat. Some serial dependence in R also arises under M1, M2 and M4 through the autocorrelated states and the person propensities, so the check does not discriminate M3 from the other motifs by itself.

sim3 <- simulate_ema(N = 60, n_prompts = 40, motifs = "M3", compliance = 0.7, kappa_R = 1, seed = 4)
fatigue_check(sim3$data)
#> Fatigue check over 2340 consecutive prompts
#>   response rate after an answered prompt: 0.808; after a skipped prompt: 0.450; difference 0.358 (person-mean difference 0.327)
#>   response rate by study quarter: 0.712 0.687 0.69 0.71
fatigue_check(sim2$data)
#> Fatigue check over 2340 consecutive prompts
#>   response rate after an answered prompt: 0.770; after a skipped prompt: 0.535; difference 0.235 (person-mean difference 0.111)
#>   response rate by study quarter: 0.702 0.692 0.707 0.7

Under burden (first output) the response rate after a skipped prompt is far below the rate after an answered one, pooled and within person. Under self-censoring alone (second output) the pooled contrast is smaller and the person-mean contrast smaller still: skips cluster in time because the states that cause them are autocorrelated, not because a skip itself lowers the next propensity.

Where burden is declared together with self-censoring, the observed persistence is the quantity that calibrate_delta(method = "postskip", burden = "fit") matches.

How often do the tests reject? A small Monte Carlo

The loop below repeats the silence test on new data sets under three mechanisms. It is deliberately small (the accompanying article uses 500 replications per cell and larger samples); the point is to show the pattern: size near the nominal level under M0 and M1, power under M2.

rej <- function(motifs, reps = 60, ...) {
  mean(vapply(seq_len(reps), function(r) {
    s <- simulate_ema(N = 60, n_prompts = 40, motifs = motifs, compliance = 0.7, seed = 1000 + r, ...)
    attr(silence_test(s$data, s$vars), "joint")["p"] < 0.05
  }, logical(1)))
}
c(M0 = rej("M0"), M1 = rej("M1"), M2 = rej("M2", delta = -1))
#>         M0         M1         M2 
#> 0.08333333 0.08333333 1.00000000

With 60 replications the Monte Carlo standard error of a rejection rate near .05 is about .03, so the first two numbers are compatible with the nominal level; the third shows the power the test has at this sample size and tilt. In the simulation studies of Yu (2026) (500 replications per cell, N = 100, T = 56) the rejection rate under the recoverable motifs stayed between 4% and 7% with two mild excesses (8.8% under burden at 70% compliance and 7.4% under lagged-state dependence at 85%), compatible with Monte Carlo error and the known anticonservatism of cluster-robust standard errors.

What to do with a rejection

A rejection says that the declared recoverable class is not tenable (or, under M1 + M3, that the collider is at work). It does not say which non-recoverable mechanism is present. The recommended continuation is

1. check for burden with fatigue_check(); if it is substantial and M1 is plausible, prefer the sensor-gap test;
2. if the sensor-gap test also rejects, or no sensor exists and burden is small, treat the skips as self-censoring and run the sensitivity analysis of vignette("sensitivity-analysis"), using the sign of the silence test as the sign of the tilt;
3. record both tests in the missingness_declaration().

A non-rejection is weaker evidence: at typical sample sizes the tests have limited power against small tilts, and the identified sets of the sensitivity analysis remain the honest summary when self-censoring is substantively plausible.

References

Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to cluster-robust inference. Journal of Human Resources, 50, 317-372. https://doi.org/10.3368/jhr.50.2.317

Yu, H.-T. (2026). What skipped prompts hide: Detecting, diagnosing, and correcting informative nonresponse in ecological momentary assessment. Manuscript under review.