Constrained Optimal Sample Allocation in Multilevel Regression Discontinuity Studies

Metin Bulus

February 5, 2017

Discontinuity at level 1

Block/Multisite Two-level Regression Discontinuity (BIRD2r1) Design

# power constrained - optimize n1 and n2
cosa.bird2r1(constrain = "power", power = .80,
             es = .20, rho2 = .20,  omega2 = .20,
             r21 = .20, r2t2 = .20, g2 = 1, p = .50,
             n1 = c(10, 40))
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##   <n1< n2 [p] cost  es 95%lcl 95%ucl [power]
##  15.75 96 0.5    0 0.2   0.06   0.34     0.8
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 0 
##  Level 1 control: 0 
##  Level 2: 0 
##  ----------------------------------------------- 
##  MDES = 0.2 (with power = 80) 
##  power = 0.8 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained
# cost constrained - optimize n1 and n2
cosa.bird2r1(constrain = "cost", cost = 2000,
             cn1 = c(5, 4), cn2 = 10,
             es = .20, rho2 = .20, omega2 = .20,
             r21 = .20, r2t2 = .20, g2 = 1, p = .50,
             n1 = c(10, 40))
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##    <n1< n2 [p] [cost]    es 95%lcl 95%ucl power
##  22.444 18 0.5   1998 0.414   0.12  0.708 0.274
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2: 10 
##  ----------------------------------------------- 
##  MDES = 0.414 (with power = 80) 
##  power = 0.274 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Block/Multisite Three-level Regression Discontinuity (BIRD3r1) Design

# power constrained - optimize n1 and n3
cosa.bird3r1(constrain = "power", power = .80,
             es = .20, rho2 = .20, rho3 = .10,  omega2 = .20, omega3 = .10,
             r21 = .20, r2t2 = .20, r2t3 = .20, g3 = 1, p = .50,
             n1 = c(10, 40), n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##    <n1< [n2] n3 [p] cost  es 95%lcl 95%ucl [power]
##  13.306    3 36 0.5    0 0.2  0.059  0.341     0.8
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 0 
##  Level 1 control: 0 
##  Level 2: 0 
##  Level 3: 0 
##  ----------------------------------------------- 
##  MDES = 0.2 (with power = 80) 
##  power = 0.8 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained
# cost constrained - optimize n1 and n3
cosa.bird3r1(constrain = "cost", cost = 5000,
             cn1 = c(5, 4), cn2 = 10, cn3 = 30,
             es = .20, rho2 = .20, rho3 = .10, omega2 = .20, omega3 = .10,
             r21 = .20, r2t2 = .20, r2t3 = .20, g3 = 1, p = .50,
             n1 = c(10, 40), n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##  <n1< [n2] n3 [p] [cost]    es 95%lcl 95%ucl power
##    22    3 14 0.5   4998 0.273  0.078  0.468 0.538
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2: 10 
##  Level 3: 30 
##  ----------------------------------------------- 
##  MDES = 0.273 (with power = 80) 
##  power = 0.538 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Block/Multisite Four-level Regression Discontinuity (BIRD4r1) Design

# cost constrained - optimize n1, and n3 
cosa.bird4r1(constrain = "cost", cost = 50000,
             cn1 = c(5, 4), cn2 = 10, cn3 = 30, cn4 = 50,
             es = .20, rho2 = .20, rho3 = .10, rho4 = .05,
             omega2 = .20, omega3 = .10, omega4 = .10,
             r21 = .20, r2t2 = .20, r2t3 = .20, r2t4 = .20,
             g4 = 1, p = .50,
             n1 = c(10, 40), n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##    <n1< [n2] n3 n4 [p] [cost]    es 95%lcl 95%ucl power
##  12.679    3 43  5 0.5  49951 0.166  0.039  0.293 0.901
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2: 10 
##  Level 3: 30 
##  Level 4: 50 
##  ----------------------------------------------- 
##  MDES = 0.166 (with power = 80) 
##  power = 0.901 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Discontinuity at level 2

Cluster-level Regression Discontinuity (CRD2r2) Design

# cost constrained - optimize n1 and n2
cosa.crd2r2(constrain = "cost", cost = 50000,
            cn1 = 5, cn2 = c(10, 8),
            es = .20, rho2 = .20, r21 = .20, r22 = .30,
            g2 = 1, p = .50)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##      n1  n2   [p] [cost]    es 95%lcl 95%ucl power
##  25.301 369 0.499  50000 0.197  0.059  0.335 0.811
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 5 
##  Level 2 treatment: 10 
##  Level 2 control: 8 
##  ----------------------------------------------- 
##  MDES = 0.197 (with power = 80) 
##  power = 0.811 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Block/Multisite Three-level Regression Discontinuity (BCRD3r2) Design

# cost constrained
cosa.bcrd3r2(constrain = "cost", cost = 50000,
             cn1 = c(5, 4), cn2 = 10, cn3 = 30,
             es = .20, rho2 = .20, rho3 = .10, omega3 = .10,
             r21 = .20, r22 = .20, r2t3 = .20,
             g3 = 1, p = .50, n1 = c(10, 40), n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##    <n1< [n2]  n3   [p]   [cost]   es 95%lcl 95%ucl power
##  13.623    3 205 0.501 50007.81 0.17  0.051  0.289  0.91
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2 treatment: 10 
##  Level 2 control: 10 
##  Level 3: 30 
##  ----------------------------------------------- 
##  MDES = 0.17 (with power = 80) 
##  power = 0.91 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Block/Multisite Four-level Regression Discontinuity (BCRD4r2) Design

# cost constrained
cosa.bcrd4r2(constrain = "cost", cost = 80000,
             cn1 = c(5, 4), cn2 = 10, cn3 = 30, cn4 = 50,
             es = .20, rho2 = .20, rho3 = .10, rho4 = .05,
             omega3 = .10, omega4 = .20,
             r21 = .20, r22 = .20, r2t3 = .20, r2t4 = .20,
             g4 = 1, p = .50,
             n1 = 10, n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##  [n1] [n2]     n3 n4 [p] [cost]    es 95%lcl 95%ucl power
##    10    3 68.167  6 0.5  80060 0.213  0.054  0.373 0.742
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2 treatment: 10 
##  Level 2 control: 10 
##  Level 3: 30 
##  Level 4: 50 
##  ----------------------------------------------- 
##  MDES = 0.213 (with power = 80) 
##  power = 0.742 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Discontinuity at level 3

Cluster-level Regression Discontinuity (CRD3r3) Design

# cost constrained - optimize n1 and n3
cosa.crd3r3(constrain = "cost", cost = 100000,
            cn1 = 5, cn2 = c(10, 5), cn3 = c(30, 10),
            es = .20, rho2 = .20, rho3 = .10,
            r21 = .20, r22 = .30, r23 = .40,
            g3 = 1, p = .50,
            n1 = c(10, 40), n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##    <n1< [n2]  n3 [p] [cost]    es 95%lcl 95%ucl power
##  19.689    3 296 0.5  1e+05 0.185  0.055  0.314 0.858
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 5 
##  Level 2 treatment: 10 
##  Level 2 control: 5 
##  Level 3 treatment: 30 
##  Level 3 control: 10 
##  ----------------------------------------------- 
##  MDES = 0.185 (with power = 80) 
##  power = 0.858 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Block/Multisite Four-level Regression Discontinuity (BCRD4r3) Design

# cost constrained - optimize n1, n3 and n4
cosa.bcrd4r3(constrain = "cost", cost = 100000,
             cn1 = c(5, 4), cn2 = 10, cn3 = 30, cn4 = 50,
             es = .20, rho2 = .20, rho3 = .10, rho4 = .05, omega4 = .20,
             r21 = .20, r22 = .20, r23 = .20, r2t4 = .20,
             g4 = 1, p = .50,
             n1 = 25, n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##  [n1] [n2]   n3 n4   [p] [cost]    es 95%lcl 95%ucl power
##    25    3 50.2  5 0.502 100060 0.367  0.086  0.648 0.351
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 4 
##  Level 2 treatment: 10 
##  Level 2 control: 10 
##  Level 3 treatment: 30 
##  Level 3 control: 30 
##  Level 4: 50 
##  ----------------------------------------------- 
##  MDES = 0.367 (with power = 80) 
##  power = 0.351 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained

Discontinuity at level 4

Cluster-level Regression Discontinuity (CRD4r4) Design

# cost constrained - optimize n1, n3 and n4
cosa.crd4r4(constrain = "cost", cost = 100000,
            cn1 = 5, cn2 = c(10, 5), cn3 = c(30, 10), cn4 = c(50,30),
            es = .20, rho2 = .20, rho3 = .10, rho4 = .05,
            r21 = .20, r22 = .30, r23 = .40, r24 = .50,
            g4 = 1, p = .50,
            n1 = 10, n2 = 3)
##  Solution converged with LBFGS 
## 
##  Rounded solution: 
##  ----------------------------------------------- 
##  [n1] [n2]     n3 n4 [p]   [cost]    es 95%lcl 95%ucl power
##    10    3 43.083 12 0.5 100002.5 0.503  0.141  0.865 0.202
##  ----------------------------------------------- 
##  Per unit marginal costs: 
##  Level 1 treatment: 5 
##  Level 1 control: 5 
##  Level 2 treatment: 10 
##  Level 2 control: 5 
##  Level 3 treatment: 30 
##  Level 3 control: 10 
##  Level 4 treatment: 50 
##  Level 4 control: 30 
##  ----------------------------------------------- 
##  MDES = 0.503 (with power = 80) 
##  power = 0.202 (for ES = 0.2) 
##  ----------------------------------------------- 
##  []: point constrained (fixed) 
##  <<: bound constrained