Testing k-fold CV

Copy/paste some functionality form other vignette.

Packages and source for reloading after edits.

library(riskRegression)
sessionInfo()

Registered S3 methods overwritten by 'ggplot2':
  method         from
  [.quosures     rlang
  c.quosures     rlang
  print.quosures rlang
riskRegression version 2019.06.13
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] riskRegression_2019.06.13

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1          mvtnorm_1.0-10      lattice_0.20-38
 [4] zoo_1.8-6           assertthat_0.2.1    digest_0.6.19
 [7] foreach_1.4.4       R6_2.4.0            plyr_1.8.4
[10] backports_1.1.4     acepack_1.4.1       MatrixModels_0.4-1
[13] ggplot2_3.1.1       pillar_1.4.1        rlang_0.3.4
[16] lazyeval_0.2.2      multcomp_1.4-10     rstudioapi_0.10
[19] data.table_1.12.2   SparseM_1.77        rpart_4.1-15
[22] Matrix_1.2-17       checkmate_1.9.3     splines_3.6.0
[25] stringr_1.4.0       foreign_0.8-71      htmlwidgets_1.3
[28] munsell_0.5.0       numDeriv_2016.8-1.1 compiler_3.6.0
[31] xfun_0.7            pkgconfig_2.0.2     base64enc_0.1-3
[34] htmltools_0.3.6     nnet_7.3-12         tidyselect_0.2.5
[37] tibble_2.1.3        gridExtra_2.3       htmlTable_1.13.1
[40] prodlim_2018.04.18  Hmisc_4.2-0         rms_5.1-3.1
[43] codetools_0.2-16    crayon_1.3.4        dplyr_0.8.1
[46] MASS_7.3-51.4       timereg_1.9.3       grid_3.6.0
[49] nlme_3.1-140        polspline_1.1.14    gtable_0.3.0
[52] magrittr_1.5        scales_1.0.0        stringi_1.4.3
[55] latticeExtra_0.6-28 sandwich_2.5-1      Formula_1.2-3
[58] TH.data_1.0-10      lava_1.6.5          RColorBrewer_1.1-2
[61] iterators_1.0.10    tools_3.6.0         cmprsk_2.2-8
[64] glue_1.3.1          purrr_0.3.2         survival_2.44-1.1
[67] colorspace_1.4-1    cluster_2.0.9       knitr_1.23
[70] quantreg_5.40

Setup data

set.seed(18)
astrain <- simActiveSurveillance(278)
astest <- simActiveSurveillance(208)
astrain[,Y1:=1*(event==1 & time<=1)]
astest[,Y1:=1*(event==1 & time<=1)]
lrfit.ex <- glm(Y1~age+lpsaden+ppb5+lmax+ct1+diaggs,data=astrain,family="binomial")
lrfit.inc <- glm(Y1~age+lpsaden+ppb5+lmax+ct1+diaggs+erg.status,data=astrain,family="binomial")
## Score(list("Exclusive ERG"=lrfit.ex,"Inclusive ERG"=lrfit.inc),data=astest,formula=Y1~1,se.fit=0L,metrics="brier",contrasts=FALSE)

Now also works with for bootcv= without errors, now also returning no-NA IPA. These are negativ, however, which I don't know if make sense.

X1 <- Score(list("Exclusive ERG"=lrfit.ex,"Inclusive ERG"=lrfit.inc),data=astest,
            formula=Y1~1,summary="ipa",se.fit=0L,metrics="brier",contrasts=FALSE,
            split.method = "bootcv", B=100)

X1

Metric Brier:

Results by model:

           model Brier     IPA
1:    Null model 0.157  0.0000
2: Exclusive ERG 0.169 -0.0781
3: Inclusive ERG 0.163 -0.0396

Bootstrap cross-validation based on 100 bootstrap samples (drawn with replacement) each of size 208.