Quick Modeling
When I first wrote the creditmodel package, its primary purpose was to provide a tool to make the development of binary classification models (machine learning based models as well as credit scorecard) simpler and faster. Therefore, I wrote the package to automatically build model. However, as the package grew in functionality, this choice was increasingly problematic.
Importantly, the creditmodel package now provides a set of complementary tools with different missions. Now, Let’s start with quick modeling.
library(creditmodel)
#> Registered S3 methods overwritten by 'ggplot2':
#> method from
#> [.quosures rlang
#> c.quosures rlang
#> print.quosures rlang
#> Package 'creditmodel' version 1.1.5
B_model = training_model(dat = UCICreditCard,
model_name = "UCICreditCard",
target = "default.payment.next.month",
x_list = NULL,
occur_time = "apply_date",
obs_id = "ID",
dat_test = NULL,
preproc = TRUE,
miss_values = c(-1, -2),
missing_proc = TRUE,
outlier_proc = TRUE,
trans_log = TRUE,
feature_filter = list(filter = c("IV", "PSI", "COR", "XGB"),
cv_folds = 1,
iv_cp = 0.02,
psi_cp = 0.2,
cor_cp = 0.95,
xgb_cp = 0,
hopper = TRUE),
vars_plot = FALSE,
algorithm = list("LR","XGB"),
breaks_list = NULL,
LR.params = lr_params(
iter = 2,
method = 'random_search',
tree_control = list(p = 0.02,
cp = c(0.00001, 0.00000001),
xval = 5,
maxdepth = c(10, 15)),
bins_control = list(bins_num = 10,
bins_pct = c(0.02, 0.03, 0.05),
b_chi = c(0.01, 0.02, 0.03),
b_odds = 0.1,
b_psi = c(0.02, 0.06),
b_or = c(.05, 0.1, 0.15, 0.2),
mono = c(0.1, 0.2, 0.4, 0.5),
odds_psi = c(0.1, 0.15, 0.2),
kc = 1),
f_eval = 'ks',
lasso = TRUE,
step_wise = FALSE),
XGB.params = xgb_params(
iter = 3,
method = 'random_search',
params = list(
max_depth = c(3:6),
eta = c(0.01, 0.05, 0.1, 0.2),
gamma = c(0.01, 0.05, 0.1),
min_child_weight = c(1, 5, 10, 20, 30, 40, 50),
subsample = c(0.8, 0.7, 0.6, 0.5),
colsample_bytree = c(0.8, 0.7, 0.6, 0.5),
scale_pos_weight = c(1, 2, 3)),
f_eval = 'auc'),
parallel = FALSE,
cores_num = NULL,
save_pmml = FALSE,
plot_show = TRUE,
model_path = tempdir(),
seed = 46)
#> -- Building ---------------------------------------------------------------------------------------- UCICreditCard --
#> -- Creating the model output file path ------------------------------------------------------------------------------
#> -- Seting model output file path:
#> * model : C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/model
#> * data : C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data
#> * variable : C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable
#> * performance: C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance
#> * predict : C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict
#> -- Checking datasets and target -------------------------------------------------------------------------------------
#> -- Cleansing & Prepocessing data ------------------------------------------------------------------------------------
#> -- Checking data and target format...
#> -- Cleansing data
#> -- Replacing null or blank or miss_values with NA
#> -- Deleting low variance variables
#> -- Processing NAs & special value rate is more than 0.999
#> -- Formating time variables
#> -- Transfering character variables which are actually numerical to numeric
#> -- Removing duplicated observations
#> -- Merging categories which percent is less than 0.001 or obs number is less than 20
#> -- Saving data_cleansing to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/data_cleansing.csv
#> -- Logarithmic transformation
#> -- Following variables are log transformed:
#> * LIMIT_BAL -> LIMIT_BAL_log
#> * PAY_0 -> PAY_0_log
#> * PAY_2 -> PAY_2_log
#> * PAY_AMT1 -> PAY_AMT1_log
#> * PAY_AMT2 -> PAY_AMT2_log
#> * PAY_AMT3 -> PAY_AMT3_log
#> * PAY_AMT4 -> PAY_AMT4_log
#> * PAY_AMT5 -> PAY_AMT5_log
#> * PAY_AMT6 -> PAY_AMT6_log
#> -- Spliting train & test --------------------------------------------------------------------------------------------
#> -- train_test_split:
#> * Total: 30000 (100%)
#> * Train: 20874 (70%)
#> * Test : 9126 (30%)
#> -- Processing outliers using Kmeans and LOF
#> * LIMIT_BAL_log 0% no_outlier
#> * AGE 0% no_outlier
#> * PAY_0_log 0% no_outlier
#> * PAY_2_log 0% no_outlier
#> * PAY_3 0% no_outlier
#> * PAY_4 0% no_outlier
#> * PAY_5 0% no_outlier
#> * PAY_6 0% no_outlier
#> * BILL_AMT1 0% no_outlier
#> * BILL_AMT2 0% no_outlier
#> * BILL_AMT3 0% no_outlier
#> * BILL_AMT4 0% no_outlier
#> * BILL_AMT5 0% no_outlier
#> * BILL_AMT6 0% no_outlier
#> * PAY_AMT1_log 0% no_outlier
#> * PAY_AMT2_log 0% no_outlier
#> * PAY_AMT3_log 0% no_outlier
#> * PAY_AMT4_log 0% no_outlier
#> * PAY_AMT5_log 0% no_outlier
#> * PAY_AMT6_log 0% no_outlier
#> -- Saving data_outlier_proc to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/data_outlier_proc.csv
#> -- Processing NAs
#> * MARRIAGE 0.1581% IM
#> * PAY_0_log 27.963% IM
#> * PAY_2_log 32.6579% IM
#> * PAY_3 33.5154% IM
#> * PAY_4 33.3573% IM
#> * PAY_5 33.5968% IM
#> * PAY_6 35.4939% IM
#> * BILL_AMT1 0.1102% IM
#> * BILL_AMT2 0.1246% IM
#> * BILL_AMT3 0.1389% IM
#> * BILL_AMT4 0.1533% IM
#> * BILL_AMT5 0.1198% IM
#> * BILL_AMT6 0.1533% IM
#> -- Saving data_missing_proc to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/data_missing_proc.csv
#> -- Filtering features -----------------------------------------------------------------------------------------------
#> -- Feature filtering by PSI
#> -- Feature filtering by IV
#> -- Selecting variables by XGboost
#> -- Feature filtering by Correlation
#> -- Saving feature_filter to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/feature_filter.csv
#> -- Saving feature_filter_table to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/feature_filter_table.csv
#> -- Training logistic regression model/scorecard ---------------------------------------------------------------------
#> -- Searching optimal binning & feature selection parameters ---------------------------------------------------------
#> [1] train_ks:0.4125 test_ks:0.389 psi:0.001
#> * tree_control:{ p:0.02, cp:0.00001, xval:5, maxdepth:10 }
#> * bins_control:{ bins_num:10, bins_pct:0.02, b_chi:0.01, b_odds:0.1, b_psi:0.02, b_or:0.05, mono:0.1, odds_psi:0.1, kc:1 }
#> * thresholds:{ cor_p:0.8, iv_i:0.02, psi_i:0.1, cos_i:0.5 }
#> [2] train_ks:0.4125 test_ks:0.389 psi:0.001
#> * tree_control:{ p:0.02, cp:0.00001, xval:5, maxdepth:10 }
#> * bins_control:{ bins_num:10, bins_pct:0.02, b_chi:0.01, b_odds:0.1, b_psi:0.02, b_or:0.05, mono:0.1, odds_psi:0.1, kc:1 }
#> * thresholds:{ cor_p:0.8, iv_i:0.02, psi_i:0.1, cos_i:0.5 }
#> -- [best iter] ------------------------------------------------------------------------------------------------------
#> [1] train_ks:0.4125 test_ks:0.389 psi:0.001
#> * tree_control:{ p:0.02, cp:0.00001, xval:5, maxdepth:10 }
#> * bins_control:{ bins_num:10, bins_pct:0.02, b_chi:0.01, b_odds:0.1, b_psi:0.02, b_or:0.05, mono:0.1, odds_psi:0.1, kc:1 }
#> * thresholds:{ cor_p:0.8, iv_i:0.02, psi_i:0.1, cos_i:0.5 }
#> -- Constrained optimal binning of varibles --------------------------------------------------------------------------
#> -- Getting optimal binning breaks
#> * PAY_0_log: -0.5,0.346573590279972,Inf
#> * PAY_2_log: -0.5,0.346573590279972,Inf
#> * PAY_3: -1,1,Inf
#> * PAY_4: -1,0,Inf
#> * PAY_5: -1,1,Inf
#> * PAY_AMT1_log: 3.06778244554087,7.60115239706291,Inf
#> -- Saving breaks_list.breaks_list to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/breaks_list.breaks_list.csv
#> -- Filtering variables by IV & PSI ----------------------------------------------------------------------------------
#> -- Selecting variables by PSI & IV
#> -- Calculating PSI
#> --PAY_0_log
#> * PSI: 0 --> Very stable
#> --PAY_2_log
#> * PSI: 0 --> Very stable
#> --PAY_3
#> * PSI: 0 --> Very stable
#> --PAY_4
#> * PSI: 0 --> Very stable
#> --PAY_5
#> * PSI: 0 --> Very stable
#> --PAY_AMT1_log
#> * PSI: 0 --> Very stable
#> -- Calculating IV
#> --PAY_0_log
#> * IV: 0.692 --> Very Strong
#> --PAY_2_log
#> * IV: 0.538 --> Very Strong
#> --PAY_3
#> * IV: 0.405 --> Very Strong
#> --PAY_4
#> * IV: 0.352 --> Very Strong
#> --PAY_5
#> * IV: 0.314 --> Very Strong
#> --PAY_AMT1_log
#> * IV: 0.148 --> Strong
#> -- Saving feature.IV_PSI to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/feature.IV_PSI.csv
#> -- Saving feature.PSI to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/feature.PSI.csv
#> -- Saving feature.IV to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/feature.IV.csv
#> -- Saving LR.IV_PSI_features to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/LR.IV_PSI_features.csv
#> -- Transforming WOE -------------------------------------------------------------------------------------------------
#> -- Transforming variables to woe
#> -- Saving lr_train.dat.woe to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/LR/lr_train.dat.woe.csv
#> -- Filtering variables by correlation -------------------------------------------------------------------------------
#> -- Processing bins table
#> * PAY_0_log IV: 0.692 PSI: 0
#> * PAY_2_log IV: 0.537 PSI: 0
#> * PAY_3 IV: 0.406 PSI: 0
#> * PAY_4 IV: 0.352 PSI: 0
#> * PAY_5 IV: 0.314 PSI: 0
#> * PAY_AMT1_log IV: 0.149 PSI: 0
#> -- Filtering variables by LASSO -------------------------------------------------------------------------------------
#> Saving 8 x 6.5 in image
#> -- Saving lr_premodel_features to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/lr_premodel_features.csv
#> -- Start training lr model ------------------------------------------------------------------------------------------
#> -- Saving lr_model_features to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/LR/lr_model_features.csv
#> -- Saving UCICreditCard.lr_coef to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance/LR/UCICreditCard.lr_coef.csv
#> -- Generating standard socrecard ------------------------------------------------------------------------------------
#> -- Using scorecard to predict the train and test
#> -- Saving lr_train_score to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/LR/lr_train_score.csv
#> -- Saving lr_test_score to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/LR/lr_test_score.csv
#> -- Saving lr_train_prob to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/LR/lr_train_prob.csv
#> -- Saving lr_test_prob to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/LR/lr_test_prob.csv
#> -- Producing plots that characterize performance of scorecard
#> Saving 12 x 6.5 in image
#> -- Saving UCICreditCard.LR.performance_table to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance/LR/UCICreditCard.LR.performance_table.csv
#> -- Saving LR.params to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance/LR/LR.params.csv
#> -- Training XGboost Model -------------------------------------------------------------------------------------------
#> -- Searching optimal parameters of XGboost --------------------------------------------------------------------------
#> [1] train_auc:0.768799 eval_auc:0.751964
#> * params:{max_depth:3, eta:0.01, gamma:0.01, min_child_weight:1, subsample:0.8, colsample_bytree:0.8, scale_pos_weight:1}
#> [2] train_auc:0.772677 eval_auc:0.751984
#> * params:{max_depth:3, eta:0.01, gamma:0.01, min_child_weight:1, subsample:0.8, colsample_bytree:0.8, scale_pos_weight:1}
#> [3] train_auc:0.769792 eval_auc:0.751976
#> * params:{max_depth:3, eta:0.01, gamma:0.01, min_child_weight:1, subsample:0.8, colsample_bytree:0.8, scale_pos_weight:1}
#> -- [best iter] ------------------------------------------------------------------------------------------------------
#> [2] train_auc:0.772677 eval_auc:0.751984
#> * params:{max_depth:3, eta:0.01, gamma:0.01, min_child_weight:1, subsample:0.8, colsample_bytree:0.8, scale_pos_weight:1}
#> -- Saving XGB.x_train to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/XGB/XGB.x_train.csv
#> -- Saving XGB.x_test to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/XGB/XGB.x_test.csv
#> -- Saving XGB.y_train to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/XGB/XGB.y_train.csv
#> -- Saving XGB.y_test to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/data/XGB/XGB.y_test.csv
#> [1] train-auc:0.709020 eval-auc:0.712786
#> Multiple eval metrics are present. Will use eval_auc for early stopping.
#> Will train until eval_auc hasn't improved in 100 rounds.
#>
#> [2] train-auc:0.747521 eval-auc:0.745226
#> [3] train-auc:0.747617 eval-auc:0.745381
#> [4] train-auc:0.756840 eval-auc:0.755324
#> [5] train-auc:0.757154 eval-auc:0.757429
#> [6] train-auc:0.758090 eval-auc:0.757886
#> [7] train-auc:0.758037 eval-auc:0.757704
#> [8] train-auc:0.757308 eval-auc:0.757175
#> [9] train-auc:0.758632 eval-auc:0.758537
#> [10] train-auc:0.758226 eval-auc:0.757628
#> [11] train-auc:0.758803 eval-auc:0.758058
#> [12] train-auc:0.758845 eval-auc:0.758036
#> [13] train-auc:0.758708 eval-auc:0.758519
#> [14] train-auc:0.759093 eval-auc:0.758778
#> [15] train-auc:0.760089 eval-auc:0.759870
#> [16] train-auc:0.760542 eval-auc:0.760058
#> [17] train-auc:0.760571 eval-auc:0.759975
#> [18] train-auc:0.760621 eval-auc:0.759956
#> [19] train-auc:0.760726 eval-auc:0.759699
#> [20] train-auc:0.760669 eval-auc:0.759714
#> [21] train-auc:0.760720 eval-auc:0.759621
#> [22] train-auc:0.760500 eval-auc:0.759595
#> [23] train-auc:0.760448 eval-auc:0.759457
#> [24] train-auc:0.760347 eval-auc:0.759224
#> [25] train-auc:0.760535 eval-auc:0.759467
#> [26] train-auc:0.760433 eval-auc:0.759268
#> [27] train-auc:0.760492 eval-auc:0.759368
#> [28] train-auc:0.760459 eval-auc:0.759314
#> [29] train-auc:0.760532 eval-auc:0.759474
#> [30] train-auc:0.760616 eval-auc:0.759489
#> [31] train-auc:0.760667 eval-auc:0.759546
#> [32] train-auc:0.761010 eval-auc:0.759613
#> [33] train-auc:0.760805 eval-auc:0.759576
#> [34] train-auc:0.760888 eval-auc:0.759696
#> [35] train-auc:0.761190 eval-auc:0.759652
#> [36] train-auc:0.761063 eval-auc:0.759825
#> [37] train-auc:0.761065 eval-auc:0.759823
#> [38] train-auc:0.761057 eval-auc:0.759915
#> [39] train-auc:0.761150 eval-auc:0.760020
#> [40] train-auc:0.761105 eval-auc:0.759977
#> [41] train-auc:0.761122 eval-auc:0.759953
#> [42] train-auc:0.761019 eval-auc:0.759911
#> [43] train-auc:0.761119 eval-auc:0.759977
#> [44] train-auc:0.761038 eval-auc:0.759915
#> [45] train-auc:0.760971 eval-auc:0.760060
#> [46] train-auc:0.761098 eval-auc:0.760065
#> [47] train-auc:0.761146 eval-auc:0.760093
#> [48] train-auc:0.761128 eval-auc:0.759962
#> [49] train-auc:0.761170 eval-auc:0.759930
#> [50] train-auc:0.761133 eval-auc:0.760027
#> [51] train-auc:0.761188 eval-auc:0.760016
#> [52] train-auc:0.761080 eval-auc:0.759963
#> [53] train-auc:0.761038 eval-auc:0.759860
#> [54] train-auc:0.761055 eval-auc:0.759916
#> [55] train-auc:0.761208 eval-auc:0.760020
#> [56] train-auc:0.761442 eval-auc:0.760171
#> [57] train-auc:0.761490 eval-auc:0.760191
#> [58] train-auc:0.761604 eval-auc:0.760362
#> [59] train-auc:0.761611 eval-auc:0.760247
#> [60] train-auc:0.761788 eval-auc:0.760395
#> [61] train-auc:0.761742 eval-auc:0.760294
#> [62] train-auc:0.761626 eval-auc:0.760282
#> [63] train-auc:0.761708 eval-auc:0.760361
#> [64] train-auc:0.761635 eval-auc:0.760311
#> [65] train-auc:0.761577 eval-auc:0.760264
#> [66] train-auc:0.761581 eval-auc:0.760306
#> [67] train-auc:0.761558 eval-auc:0.760311
#> [68] train-auc:0.761587 eval-auc:0.760317
#> [69] train-auc:0.761654 eval-auc:0.760420
#> [70] train-auc:0.761667 eval-auc:0.760559
#> [71] train-auc:0.761855 eval-auc:0.760611
#> [72] train-auc:0.761673 eval-auc:0.760532
#> [73] train-auc:0.761675 eval-auc:0.760600
#> [74] train-auc:0.761735 eval-auc:0.760669
#> [75] train-auc:0.761915 eval-auc:0.760806
#> [76] train-auc:0.761870 eval-auc:0.760799
#> [77] train-auc:0.761926 eval-auc:0.760855
#> [78] train-auc:0.762426 eval-auc:0.760984
#> [79] train-auc:0.762469 eval-auc:0.760988
#> [80] train-auc:0.762488 eval-auc:0.760938
#> [81] train-auc:0.762467 eval-auc:0.760945
#> [82] train-auc:0.762492 eval-auc:0.760911
#> [83] train-auc:0.762436 eval-auc:0.760898
#> [84] train-auc:0.762469 eval-auc:0.760912
#> [85] train-auc:0.762383 eval-auc:0.760888
#> [86] train-auc:0.762309 eval-auc:0.760873
#> [87] train-auc:0.762381 eval-auc:0.760921
#> [88] train-auc:0.762384 eval-auc:0.760956
#> [89] train-auc:0.762456 eval-auc:0.760896
#> [90] train-auc:0.762480 eval-auc:0.760893
#> [91] train-auc:0.762526 eval-auc:0.760923
#> [92] train-auc:0.762556 eval-auc:0.760955
#> [93] train-auc:0.762674 eval-auc:0.760867
#> [94] train-auc:0.762609 eval-auc:0.760923
#> [95] train-auc:0.762551 eval-auc:0.760970
#> [96] train-auc:0.762627 eval-auc:0.760947
#> [97] train-auc:0.762589 eval-auc:0.761013
#> [98] train-auc:0.762658 eval-auc:0.760956
#> [99] train-auc:0.762707 eval-auc:0.760961
#> [100] train-auc:0.762736 eval-auc:0.760982
#> [101] train-auc:0.762717 eval-auc:0.760998
#> [102] train-auc:0.762742 eval-auc:0.761004
#> [103] train-auc:0.762755 eval-auc:0.761060
#> [104] train-auc:0.762765 eval-auc:0.761040
#> [105] train-auc:0.762775 eval-auc:0.761125
#> [106] train-auc:0.762760 eval-auc:0.761040
#> [107] train-auc:0.762747 eval-auc:0.761049
#> [108] train-auc:0.763227 eval-auc:0.761298
#> [109] train-auc:0.763275 eval-auc:0.761377
#> [110] train-auc:0.763353 eval-auc:0.761194
#> [111] train-auc:0.763429 eval-auc:0.761158
#> [112] train-auc:0.763442 eval-auc:0.761147
#> [113] train-auc:0.763372 eval-auc:0.761158
#> [114] train-auc:0.763387 eval-auc:0.761155
#> [115] train-auc:0.763396 eval-auc:0.761221
#> [116] train-auc:0.763392 eval-auc:0.761133
#> [117] train-auc:0.763385 eval-auc:0.761175
#> [118] train-auc:0.763401 eval-auc:0.761186
#> [119] train-auc:0.763336 eval-auc:0.760945
#> [120] train-auc:0.763400 eval-auc:0.761190
#> [121] train-auc:0.763334 eval-auc:0.760949
#> [122] train-auc:0.763405 eval-auc:0.760905
#> [123] train-auc:0.763429 eval-auc:0.760871
#> [124] train-auc:0.763389 eval-auc:0.760943
#> [125] train-auc:0.763491 eval-auc:0.761199
#> [126] train-auc:0.763486 eval-auc:0.761237
#> [127] train-auc:0.763481 eval-auc:0.761198
#> [128] train-auc:0.763430 eval-auc:0.760977
#> [129] train-auc:0.763409 eval-auc:0.760986
#> [130] train-auc:0.763446 eval-auc:0.761245
#> [131] train-auc:0.763468 eval-auc:0.761217
#> [132] train-auc:0.763456 eval-auc:0.761232
#> [133] train-auc:0.763525 eval-auc:0.760957
#> [134] train-auc:0.763565 eval-auc:0.760914
#> [135] train-auc:0.763622 eval-auc:0.760990
#> [136] train-auc:0.763578 eval-auc:0.760913
#> [137] train-auc:0.763608 eval-auc:0.760907
#> [138] train-auc:0.763631 eval-auc:0.760935
#> [139] train-auc:0.763606 eval-auc:0.760911
#> [140] train-auc:0.763634 eval-auc:0.760923
#> [141] train-auc:0.763606 eval-auc:0.760908
#> [142] train-auc:0.763608 eval-auc:0.760904
#> [143] train-auc:0.763637 eval-auc:0.760911
#> [144] train-auc:0.763668 eval-auc:0.760954
#> [145] train-auc:0.763657 eval-auc:0.760921
#> [146] train-auc:0.763669 eval-auc:0.760976
#> [147] train-auc:0.763683 eval-auc:0.760991
#> [148] train-auc:0.763689 eval-auc:0.760980
#> [149] train-auc:0.763722 eval-auc:0.760936
#> [150] train-auc:0.763708 eval-auc:0.760906
#> [151] train-auc:0.763707 eval-auc:0.760907
#> [152] train-auc:0.763746 eval-auc:0.760906
#> [153] train-auc:0.763736 eval-auc:0.760912
#> [154] train-auc:0.763760 eval-auc:0.760890
#> [155] train-auc:0.763786 eval-auc:0.760896
#> [156] train-auc:0.764345 eval-auc:0.761730
#> [157] train-auc:0.764370 eval-auc:0.761795
#> [158] train-auc:0.764344 eval-auc:0.761742
#> [159] train-auc:0.764339 eval-auc:0.761752
#> [160] train-auc:0.764319 eval-auc:0.761790
#> [161] train-auc:0.764358 eval-auc:0.761756
#> [162] train-auc:0.764383 eval-auc:0.761777
#> [163] train-auc:0.764332 eval-auc:0.761741
#> [164] train-auc:0.764350 eval-auc:0.761774
#> [165] train-auc:0.764728 eval-auc:0.761771
#> [166] train-auc:0.764742 eval-auc:0.761746
#> [167] train-auc:0.764689 eval-auc:0.761693
#> [168] train-auc:0.764695 eval-auc:0.761546
#> [169] train-auc:0.764701 eval-auc:0.761573
#> [170] train-auc:0.764704 eval-auc:0.761562
#> [171] train-auc:0.764732 eval-auc:0.761567
#> [172] train-auc:0.764688 eval-auc:0.761500
#> [173] train-auc:0.764687 eval-auc:0.761477
#> [174] train-auc:0.764699 eval-auc:0.761499
#> [175] train-auc:0.764722 eval-auc:0.761506
#> [176] train-auc:0.764702 eval-auc:0.761490
#> [177] train-auc:0.764711 eval-auc:0.761474
#> [178] train-auc:0.764706 eval-auc:0.761473
#> [179] train-auc:0.764692 eval-auc:0.761465
#> [180] train-auc:0.764695 eval-auc:0.761438
#> [181] train-auc:0.764722 eval-auc:0.761472
#> [182] train-auc:0.764717 eval-auc:0.761491
#> [183] train-auc:0.763921 eval-auc:0.760319
#> [184] train-auc:0.763906 eval-auc:0.760302
#> [185] train-auc:0.763897 eval-auc:0.760292
#> [186] train-auc:0.763924 eval-auc:0.760313
#> [187] train-auc:0.763926 eval-auc:0.760301
#> [188] train-auc:0.763922 eval-auc:0.760349
#> [189] train-auc:0.763918 eval-auc:0.760297
#> [190] train-auc:0.763928 eval-auc:0.760292
#> [191] train-auc:0.763901 eval-auc:0.760244
#> [192] train-auc:0.763930 eval-auc:0.760262
#> [193] train-auc:0.763961 eval-auc:0.760340
#> [194] train-auc:0.763987 eval-auc:0.760320
#> [195] train-auc:0.763981 eval-auc:0.760279
#> [196] train-auc:0.764010 eval-auc:0.760278
#> [197] train-auc:0.764039 eval-auc:0.760334
#> [198] train-auc:0.764042 eval-auc:0.760347
#> [199] train-auc:0.764041 eval-auc:0.760354
#> [200] train-auc:0.764056 eval-auc:0.760321
#> [201] train-auc:0.764085 eval-auc:0.760294
#> [202] train-auc:0.764113 eval-auc:0.760321
#> [203] train-auc:0.764083 eval-auc:0.760242
#> [204] train-auc:0.764152 eval-auc:0.760321
#> [205] train-auc:0.764169 eval-auc:0.760321
#> [206] train-auc:0.764196 eval-auc:0.760326
#> [207] train-auc:0.764188 eval-auc:0.760358
#> [208] train-auc:0.764192 eval-auc:0.760404
#> [209] train-auc:0.764190 eval-auc:0.760402
#> [210] train-auc:0.764218 eval-auc:0.760465
#> [211] train-auc:0.764234 eval-auc:0.760504
#> [212] train-auc:0.764205 eval-auc:0.760488
#> [213] train-auc:0.764205 eval-auc:0.760530
#> [214] train-auc:0.764229 eval-auc:0.760518
#> [215] train-auc:0.764231 eval-auc:0.760504
#> [216] train-auc:0.764199 eval-auc:0.760507
#> [217] train-auc:0.764207 eval-auc:0.760490
#> [218] train-auc:0.764251 eval-auc:0.760507
#> [219] train-auc:0.764305 eval-auc:0.760543
#> [220] train-auc:0.764270 eval-auc:0.760525
#> [221] train-auc:0.764235 eval-auc:0.760521
#> [222] train-auc:0.764261 eval-auc:0.760532
#> [223] train-auc:0.764299 eval-auc:0.760558
#> [224] train-auc:0.764331 eval-auc:0.760589
#> [225] train-auc:0.764369 eval-auc:0.760615
#> [226] train-auc:0.764362 eval-auc:0.760651
#> [227] train-auc:0.764372 eval-auc:0.760658
#> [228] train-auc:0.764358 eval-auc:0.760628
#> [229] train-auc:0.764370 eval-auc:0.760759
#> [230] train-auc:0.764391 eval-auc:0.760726
#> [231] train-auc:0.764392 eval-auc:0.760653
#> [232] train-auc:0.764416 eval-auc:0.760671
#> [233] train-auc:0.764403 eval-auc:0.760679
#> [234] train-auc:0.764388 eval-auc:0.760674
#> [235] train-auc:0.764404 eval-auc:0.760702
#> [236] train-auc:0.764380 eval-auc:0.760696
#> [237] train-auc:0.765265 eval-auc:0.761974
#> [238] train-auc:0.765279 eval-auc:0.761972
#> [239] train-auc:0.765282 eval-auc:0.761999
#> [240] train-auc:0.765292 eval-auc:0.762065
#> [241] train-auc:0.765295 eval-auc:0.762073
#> [242] train-auc:0.765295 eval-auc:0.762067
#> [243] train-auc:0.765286 eval-auc:0.762078
#> [244] train-auc:0.765292 eval-auc:0.762073
#> [245] train-auc:0.765299 eval-auc:0.762044
#> [246] train-auc:0.765296 eval-auc:0.762002
#> [247] train-auc:0.764420 eval-auc:0.760781
#> [248] train-auc:0.764474 eval-auc:0.760779
#> [249] train-auc:0.764496 eval-auc:0.760778
#> [250] train-auc:0.764509 eval-auc:0.760782
#> [251] train-auc:0.764539 eval-auc:0.760869
#> [252] train-auc:0.764492 eval-auc:0.760860
#> [253] train-auc:0.764498 eval-auc:0.760831
#> [254] train-auc:0.764497 eval-auc:0.760814
#> [255] train-auc:0.764499 eval-auc:0.760813
#> [256] train-auc:0.764487 eval-auc:0.760811
#> [257] train-auc:0.764540 eval-auc:0.760871
#> [258] train-auc:0.764540 eval-auc:0.760888
#> [259] train-auc:0.764582 eval-auc:0.760861
#> [260] train-auc:0.764567 eval-auc:0.760768
#> [261] train-auc:0.764560 eval-auc:0.760736
#> [262] train-auc:0.764580 eval-auc:0.760751
#> [263] train-auc:0.765875 eval-auc:0.761472
#> [264] train-auc:0.765348 eval-auc:0.760970
#> [265] train-auc:0.765336 eval-auc:0.760943
#> [266] train-auc:0.765614 eval-auc:0.761168
#> [267] train-auc:0.765654 eval-auc:0.761191
#> [268] train-auc:0.765768 eval-auc:0.761308
#> [269] train-auc:0.765764 eval-auc:0.761299
#> [270] train-auc:0.765765 eval-auc:0.761309
#> [271] train-auc:0.766098 eval-auc:0.761902
#> [272] train-auc:0.766166 eval-auc:0.762343
#> [273] train-auc:0.766178 eval-auc:0.762343
#> [274] train-auc:0.766343 eval-auc:0.762628
#> [275] train-auc:0.766330 eval-auc:0.762635
#> [276] train-auc:0.766351 eval-auc:0.762618
#> [277] train-auc:0.766294 eval-auc:0.762633
#> [278] train-auc:0.766307 eval-auc:0.762625
#> [279] train-auc:0.766302 eval-auc:0.762619
#> [280] train-auc:0.766298 eval-auc:0.762613
#> [281] train-auc:0.766223 eval-auc:0.762811
#> [282] train-auc:0.766084 eval-auc:0.762705
#> [283] train-auc:0.766160 eval-auc:0.762758
#> [284] train-auc:0.766178 eval-auc:0.762770
#> [285] train-auc:0.766100 eval-auc:0.762675
#> [286] train-auc:0.766077 eval-auc:0.762666
#> [287] train-auc:0.766088 eval-auc:0.762670
#> [288] train-auc:0.766173 eval-auc:0.762710
#> [289] train-auc:0.766239 eval-auc:0.762957
#> [290] train-auc:0.766259 eval-auc:0.762964
#> [291] train-auc:0.766276 eval-auc:0.762949
#> [292] train-auc:0.766172 eval-auc:0.762820
#> [293] train-auc:0.766219 eval-auc:0.762808
#> [294] train-auc:0.766207 eval-auc:0.762787
#> [295] train-auc:0.766218 eval-auc:0.762790
#> [296] train-auc:0.766206 eval-auc:0.762778
#> [297] train-auc:0.766258 eval-auc:0.762862
#> [298] train-auc:0.766286 eval-auc:0.762866
#> [299] train-auc:0.766263 eval-auc:0.762845
#> [300] train-auc:0.766264 eval-auc:0.762857
#> [301] train-auc:0.766339 eval-auc:0.762831
#> [302] train-auc:0.766296 eval-auc:0.762869
#> [303] train-auc:0.766303 eval-auc:0.762894
#> [304] train-auc:0.766329 eval-auc:0.762991
#> [305] train-auc:0.766335 eval-auc:0.762996
#> [306] train-auc:0.766349 eval-auc:0.763000
#> [307] train-auc:0.766347 eval-auc:0.762972
#> [308] train-auc:0.766366 eval-auc:0.762931
#> [309] train-auc:0.766365 eval-auc:0.762918
#> [310] train-auc:0.766348 eval-auc:0.762912
#> [311] train-auc:0.766362 eval-auc:0.762907
#> [312] train-auc:0.766359 eval-auc:0.762930
#> [313] train-auc:0.766351 eval-auc:0.762936
#> [314] train-auc:0.766384 eval-auc:0.762941
#> [315] train-auc:0.766425 eval-auc:0.762952
#> [316] train-auc:0.766453 eval-auc:0.763005
#> [317] train-auc:0.766471 eval-auc:0.763041
#> [318] train-auc:0.766516 eval-auc:0.763020
#> [319] train-auc:0.766566 eval-auc:0.763068
#> [320] train-auc:0.766536 eval-auc:0.763104
#> [321] train-auc:0.766484 eval-auc:0.763086
#> [322] train-auc:0.766464 eval-auc:0.763213
#> [323] train-auc:0.766509 eval-auc:0.763217
#> [324] train-auc:0.766484 eval-auc:0.763209
#> [325] train-auc:0.766501 eval-auc:0.763222
#> [326] train-auc:0.766462 eval-auc:0.763182
#> [327] train-auc:0.766458 eval-auc:0.763160
#> [328] train-auc:0.766454 eval-auc:0.763119
#> [329] train-auc:0.766493 eval-auc:0.763180
#> [330] train-auc:0.766549 eval-auc:0.763170
#> [331] train-auc:0.766603 eval-auc:0.763207
#> [332] train-auc:0.766574 eval-auc:0.763220
#> [333] train-auc:0.766643 eval-auc:0.763192
#> [334] train-auc:0.766664 eval-auc:0.763189
#> [335] train-auc:0.766644 eval-auc:0.763191
#> [336] train-auc:0.766621 eval-auc:0.763198
#> [337] train-auc:0.766646 eval-auc:0.763194
#> [338] train-auc:0.766615 eval-auc:0.763112
#> [339] train-auc:0.766662 eval-auc:0.763102
#> [340] train-auc:0.766739 eval-auc:0.763149
#> [341] train-auc:0.766735 eval-auc:0.763143
#> [342] train-auc:0.766745 eval-auc:0.763175
#> [343] train-auc:0.766709 eval-auc:0.763181
#> [344] train-auc:0.766724 eval-auc:0.763172
#> [345] train-auc:0.766720 eval-auc:0.763170
#> [346] train-auc:0.766752 eval-auc:0.763166
#> [347] train-auc:0.766763 eval-auc:0.763187
#> [348] train-auc:0.766705 eval-auc:0.763224
#> [349] train-auc:0.766694 eval-auc:0.763195
#> [350] train-auc:0.766723 eval-auc:0.763205
#> [351] train-auc:0.766766 eval-auc:0.763180
#> [352] train-auc:0.766772 eval-auc:0.763159
#> [353] train-auc:0.766749 eval-auc:0.763213
#> [354] train-auc:0.766796 eval-auc:0.763232
#> [355] train-auc:0.766786 eval-auc:0.763228
#> [356] train-auc:0.766747 eval-auc:0.763235
#> [357] train-auc:0.766734 eval-auc:0.763214
#> [358] train-auc:0.766749 eval-auc:0.763187
#> [359] train-auc:0.766768 eval-auc:0.763221
#> [360] train-auc:0.766805 eval-auc:0.763214
#> [361] train-auc:0.766803 eval-auc:0.763210
#> [362] train-auc:0.766811 eval-auc:0.763202
#> [363] train-auc:0.766820 eval-auc:0.763195
#> [364] train-auc:0.766842 eval-auc:0.763174
#> [365] train-auc:0.766835 eval-auc:0.763170
#> [366] train-auc:0.766976 eval-auc:0.763309
#> [367] train-auc:0.767066 eval-auc:0.763342
#> [368] train-auc:0.767077 eval-auc:0.763372
#> [369] train-auc:0.767080 eval-auc:0.763356
#> [370] train-auc:0.767091 eval-auc:0.763369
#> [371] train-auc:0.767114 eval-auc:0.763261
#> [372] train-auc:0.767116 eval-auc:0.763281
#> [373] train-auc:0.767102 eval-auc:0.763279
#> [374] train-auc:0.767235 eval-auc:0.763281
#> [375] train-auc:0.767297 eval-auc:0.763325
#> [376] train-auc:0.767315 eval-auc:0.763340
#> [377] train-auc:0.767315 eval-auc:0.763326
#> [378] train-auc:0.767335 eval-auc:0.763328
#> [379] train-auc:0.767370 eval-auc:0.763286
#> [380] train-auc:0.767378 eval-auc:0.763294
#> [381] train-auc:0.767393 eval-auc:0.763298
#> [382] train-auc:0.767378 eval-auc:0.763316
#> [383] train-auc:0.767401 eval-auc:0.763316
#> [384] train-auc:0.767415 eval-auc:0.763322
#> [385] train-auc:0.767458 eval-auc:0.763295
#> [386] train-auc:0.767582 eval-auc:0.763405
#> [387] train-auc:0.767575 eval-auc:0.763398
#> [388] train-auc:0.767585 eval-auc:0.763452
#> [389] train-auc:0.767566 eval-auc:0.763475
#> [390] train-auc:0.767722 eval-auc:0.763527
#> [391] train-auc:0.767727 eval-auc:0.763499
#> [392] train-auc:0.767763 eval-auc:0.763486
#> [393] train-auc:0.767912 eval-auc:0.763690
#> [394] train-auc:0.767913 eval-auc:0.763713
#> [395] train-auc:0.767956 eval-auc:0.763678
#> [396] train-auc:0.767917 eval-auc:0.763693
#> [397] train-auc:0.767918 eval-auc:0.763688
#> [398] train-auc:0.767931 eval-auc:0.763653
#> [399] train-auc:0.767949 eval-auc:0.763667
#> [400] train-auc:0.768102 eval-auc:0.763722
#> [401] train-auc:0.768141 eval-auc:0.763682
#> [402] train-auc:0.768152 eval-auc:0.763668
#> [403] train-auc:0.768189 eval-auc:0.763755
#> [404] train-auc:0.768203 eval-auc:0.763736
#> [405] train-auc:0.768223 eval-auc:0.763792
#> [406] train-auc:0.768233 eval-auc:0.763855
#> [407] train-auc:0.768284 eval-auc:0.763830
#> [408] train-auc:0.768272 eval-auc:0.763881
#> [409] train-auc:0.768279 eval-auc:0.763910
#> [410] train-auc:0.768313 eval-auc:0.763926
#> [411] train-auc:0.768341 eval-auc:0.763945
#> [412] train-auc:0.768348 eval-auc:0.763962
#> [413] train-auc:0.768374 eval-auc:0.763904
#> [414] train-auc:0.768397 eval-auc:0.763898
#> [415] train-auc:0.768397 eval-auc:0.763907
#> [416] train-auc:0.768405 eval-auc:0.763885
#> [417] train-auc:0.768414 eval-auc:0.763887
#> [418] train-auc:0.768422 eval-auc:0.763908
#> [419] train-auc:0.768404 eval-auc:0.763945
#> [420] train-auc:0.768415 eval-auc:0.763923
#> [421] train-auc:0.768419 eval-auc:0.763955
#> [422] train-auc:0.768422 eval-auc:0.763935
#> [423] train-auc:0.768436 eval-auc:0.763915
#> [424] train-auc:0.768425 eval-auc:0.763900
#> [425] train-auc:0.768430 eval-auc:0.763877
#> [426] train-auc:0.768401 eval-auc:0.763843
#> [427] train-auc:0.768404 eval-auc:0.763848
#> [428] train-auc:0.768402 eval-auc:0.763846
#> [429] train-auc:0.768409 eval-auc:0.763806
#> [430] train-auc:0.768427 eval-auc:0.763795
#> [431] train-auc:0.768434 eval-auc:0.763820
#> [432] train-auc:0.768476 eval-auc:0.763828
#> [433] train-auc:0.768475 eval-auc:0.763789
#> [434] train-auc:0.768480 eval-auc:0.763828
#> [435] train-auc:0.768480 eval-auc:0.763840
#> [436] train-auc:0.768460 eval-auc:0.763869
#> [437] train-auc:0.768490 eval-auc:0.763860
#> [438] train-auc:0.768502 eval-auc:0.763886
#> [439] train-auc:0.768501 eval-auc:0.763874
#> [440] train-auc:0.768493 eval-auc:0.763887
#> [441] train-auc:0.768503 eval-auc:0.763875
#> [442] train-auc:0.768516 eval-auc:0.763863
#> [443] train-auc:0.768529 eval-auc:0.763873
#> [444] train-auc:0.768552 eval-auc:0.763864
#> [445] train-auc:0.768583 eval-auc:0.763848
#> [446] train-auc:0.768598 eval-auc:0.763835
#> [447] train-auc:0.768601 eval-auc:0.763843
#> [448] train-auc:0.768600 eval-auc:0.763831
#> [449] train-auc:0.768665 eval-auc:0.763861
#> [450] train-auc:0.768533 eval-auc:0.763701
#> [451] train-auc:0.768563 eval-auc:0.763641
#> [452] train-auc:0.768600 eval-auc:0.763669
#> [453] train-auc:0.768598 eval-auc:0.763684
#> [454] train-auc:0.768596 eval-auc:0.763678
#> [455] train-auc:0.768733 eval-auc:0.763595
#> [456] train-auc:0.768776 eval-auc:0.763617
#> [457] train-auc:0.768898 eval-auc:0.763669
#> [458] train-auc:0.768919 eval-auc:0.763652
#> [459] train-auc:0.768944 eval-auc:0.763638
#> [460] train-auc:0.768937 eval-auc:0.763608
#> [461] train-auc:0.768956 eval-auc:0.763601
#> [462] train-auc:0.768982 eval-auc:0.763632
#> [463] train-auc:0.769120 eval-auc:0.763807
#> [464] train-auc:0.769014 eval-auc:0.763659
#> [465] train-auc:0.769133 eval-auc:0.763808
#> [466] train-auc:0.769150 eval-auc:0.763817
#> [467] train-auc:0.769049 eval-auc:0.763644
#> [468] train-auc:0.769035 eval-auc:0.763692
#> [469] train-auc:0.769034 eval-auc:0.763657
#> [470] train-auc:0.769026 eval-auc:0.763672
#> [471] train-auc:0.769043 eval-auc:0.763670
#> [472] train-auc:0.769045 eval-auc:0.763671
#> [473] train-auc:0.769059 eval-auc:0.763662
#> [474] train-auc:0.769044 eval-auc:0.763669
#> [475] train-auc:0.769069 eval-auc:0.763674
#> [476] train-auc:0.769111 eval-auc:0.763697
#> [477] train-auc:0.769125 eval-auc:0.763696
#> [478] train-auc:0.769229 eval-auc:0.763842
#> [479] train-auc:0.769254 eval-auc:0.763913
#> [480] train-auc:0.769256 eval-auc:0.763898
#> [481] train-auc:0.769245 eval-auc:0.763892
#> [482] train-auc:0.769275 eval-auc:0.763903
#> [483] train-auc:0.769298 eval-auc:0.763903
#> [484] train-auc:0.769306 eval-auc:0.763900
#> [485] train-auc:0.769344 eval-auc:0.763916
#> [486] train-auc:0.769358 eval-auc:0.763964
#> [487] train-auc:0.769387 eval-auc:0.763946
#> [488] train-auc:0.769452 eval-auc:0.764000
#> [489] train-auc:0.769470 eval-auc:0.763986
#> [490] train-auc:0.769476 eval-auc:0.763982
#> [491] train-auc:0.769488 eval-auc:0.763987
#> [492] train-auc:0.769494 eval-auc:0.764011
#> [493] train-auc:0.769520 eval-auc:0.764012
#> [494] train-auc:0.769515 eval-auc:0.763988
#> [495] train-auc:0.769528 eval-auc:0.763982
#> [496] train-auc:0.769523 eval-auc:0.763919
#> [497] train-auc:0.769527 eval-auc:0.763906
#> [498] train-auc:0.769538 eval-auc:0.763817
#> [499] train-auc:0.769541 eval-auc:0.763776
#> [500] train-auc:0.769582 eval-auc:0.763785
#> [501] train-auc:0.769594 eval-auc:0.763783
#> [502] train-auc:0.769664 eval-auc:0.763832
#> [503] train-auc:0.769728 eval-auc:0.763858
#> [504] train-auc:0.769726 eval-auc:0.763886
#> [505] train-auc:0.769755 eval-auc:0.763911
#> [506] train-auc:0.769794 eval-auc:0.763916
#> [507] train-auc:0.769804 eval-auc:0.763906
#> [508] train-auc:0.769815 eval-auc:0.763900
#> [509] train-auc:0.769855 eval-auc:0.763897
#> [510] train-auc:0.769830 eval-auc:0.763901
#> [511] train-auc:0.769849 eval-auc:0.763918
#> [512] train-auc:0.769861 eval-auc:0.763894
#> [513] train-auc:0.769858 eval-auc:0.763843
#> [514] train-auc:0.769869 eval-auc:0.763838
#> [515] train-auc:0.769871 eval-auc:0.763850
#> [516] train-auc:0.769876 eval-auc:0.763839
#> [517] train-auc:0.769871 eval-auc:0.763840
#> [518] train-auc:0.769876 eval-auc:0.763835
#> [519] train-auc:0.769866 eval-auc:0.763824
#> [520] train-auc:0.769873 eval-auc:0.763806
#> [521] train-auc:0.769922 eval-auc:0.763774
#> [522] train-auc:0.769924 eval-auc:0.763773
#> [523] train-auc:0.769961 eval-auc:0.763832
#> [524] train-auc:0.769983 eval-auc:0.763823
#> [525] train-auc:0.769984 eval-auc:0.763819
#> [526] train-auc:0.769986 eval-auc:0.763858
#> [527] train-auc:0.769960 eval-auc:0.763780
#> [528] train-auc:0.769990 eval-auc:0.763783
#> [529] train-auc:0.770004 eval-auc:0.763792
#> [530] train-auc:0.769986 eval-auc:0.763805
#> [531] train-auc:0.769991 eval-auc:0.763782
#> [532] train-auc:0.770017 eval-auc:0.763795
#> [533] train-auc:0.770008 eval-auc:0.763785
#> [534] train-auc:0.770030 eval-auc:0.763800
#> [535] train-auc:0.770011 eval-auc:0.763787
#> [536] train-auc:0.770013 eval-auc:0.763785
#> [537] train-auc:0.770052 eval-auc:0.763810
#> [538] train-auc:0.770068 eval-auc:0.763800
#> [539] train-auc:0.770075 eval-auc:0.763820
#> [540] train-auc:0.770107 eval-auc:0.763811
#> [541] train-auc:0.770135 eval-auc:0.763813
#> [542] train-auc:0.770137 eval-auc:0.763808
#> [543] train-auc:0.770132 eval-auc:0.763799
#> [544] train-auc:0.770161 eval-auc:0.763830
#> [545] train-auc:0.770141 eval-auc:0.763834
#> [546] train-auc:0.770137 eval-auc:0.763831
#> [547] train-auc:0.770138 eval-auc:0.763830
#> [548] train-auc:0.770146 eval-auc:0.763833
#> [549] train-auc:0.770170 eval-auc:0.763834
#> [550] train-auc:0.770158 eval-auc:0.763827
#> [551] train-auc:0.770193 eval-auc:0.763841
#> [552] train-auc:0.770239 eval-auc:0.763858
#> [553] train-auc:0.770258 eval-auc:0.763849
#> [554] train-auc:0.770262 eval-auc:0.763838
#> [555] train-auc:0.770264 eval-auc:0.763838
#> [556] train-auc:0.770282 eval-auc:0.763819
#> [557] train-auc:0.770297 eval-auc:0.763820
#> [558] train-auc:0.770359 eval-auc:0.763937
#> [559] train-auc:0.770359 eval-auc:0.763936
#> [560] train-auc:0.770357 eval-auc:0.763929
#> [561] train-auc:0.770402 eval-auc:0.763870
#> [562] train-auc:0.770412 eval-auc:0.763878
#> [563] train-auc:0.770410 eval-auc:0.763871
#> [564] train-auc:0.770437 eval-auc:0.763861
#> [565] train-auc:0.770452 eval-auc:0.763872
#> [566] train-auc:0.770472 eval-auc:0.763890
#> [567] train-auc:0.770482 eval-auc:0.763882
#> [568] train-auc:0.770536 eval-auc:0.763878
#> [569] train-auc:0.770523 eval-auc:0.763864
#> [570] train-auc:0.770533 eval-auc:0.763921
#> [571] train-auc:0.770542 eval-auc:0.763936
#> [572] train-auc:0.770540 eval-auc:0.763939
#> [573] train-auc:0.770547 eval-auc:0.763951
#> [574] train-auc:0.770564 eval-auc:0.763969
#> [575] train-auc:0.770566 eval-auc:0.763981
#> [576] train-auc:0.770588 eval-auc:0.763927
#> [577] train-auc:0.770618 eval-auc:0.763924
#> [578] train-auc:0.770622 eval-auc:0.763939
#> [579] train-auc:0.770620 eval-auc:0.763945
#> [580] train-auc:0.770653 eval-auc:0.763966
#> [581] train-auc:0.770662 eval-auc:0.763992
#> [582] train-auc:0.770662 eval-auc:0.763992
#> [583] train-auc:0.770675 eval-auc:0.763975
#> [584] train-auc:0.770683 eval-auc:0.763908
#> [585] train-auc:0.770712 eval-auc:0.763907
#> [586] train-auc:0.770717 eval-auc:0.763910
#> [587] train-auc:0.770718 eval-auc:0.763905
#> [588] train-auc:0.770730 eval-auc:0.763912
#> [589] train-auc:0.770752 eval-auc:0.763900
#> [590] train-auc:0.770756 eval-auc:0.763884
#> [591] train-auc:0.770749 eval-auc:0.763897
#> [592] train-auc:0.770749 eval-auc:0.763897
#> [593] train-auc:0.770747 eval-auc:0.763874
#> Stopping. Best iteration:
#> [493] train-auc:0.769520 eval-auc:0.764012
#>
#> -- Saving UCICreditCard.XGB_input_vars to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/model/XGB/UCICreditCard.XGB_input_vars.csv
#> -- Saving XGB_feature_importance to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/variable/XGB/XGB_feature_importance.csv
#> -- Saving XGB.train_prob to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/XGB/XGB.train_prob.csv
#> -- Saving XGB.test_prob to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/predict/XGB/XGB.test_prob.csv
#> -- Producing plots that characterize the performance of XGboost
#> Saving 12 x 6.5 in image
#> -- Saving UCICreditCard.XGB.performance_table to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance/XGB/UCICreditCard.XGB.performance_table.csv
#> -- Saving XGB.params to:
#> * C:\Users\28142\AppData\Local\Temp\RtmpEHFLzU/UCICreditCard/performance/XGB/XGB.params.csvIn a few minutes, the program completed data cleaning and pretreatment, variable screening, scorecard, Xgboost, GBDT, RandomForest four models development and evaluation.