The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

uniml 快速上手(中文)

这个包解决什么问题

做二分类预测模型(比如院内死亡、术后并发症)时,标准流程是:切分数据 → 缺失值填补 → 标准化/哑变量 → 特征筛选 → 训练与调参 → 评价(区分度、校准度、临床获益)→ 画图 → 解释(SHAP)。每一步网上都有零散代码,但拼起来容易出现三个问题:

  1. 信息泄漏:在全集上做插补、标准化、特征筛选,再用交叉验证“验证”。
  2. 口径不一:A 模型的 AUC 用原始概率,B 模型用校准后概率,最后放一张图里比较。
  3. 结果不可追溯:跑完只有几张图,换台电脑复现不出来。

uniml 把这些步骤串成一条流水线,并强制区分「拟合用的数据」和「评价用的数据」。

数据角色

数据 用途 参与模型选择?
train 拟合预处理、筛选、模型、校准、阈值 是(内部交叉验证)
validation 模型比较与选择 是(默认)
test 只报告性能 否
external 外部验证,只报告泛化 否

一个最小例子

library(uniml)

development <- read.csv(
  system.file("extdata", "uniml-cvd-mortality.csv", package = "uniml")
)
set.seed(123)
development <- development[1:400, ]

# 1. 声明任务
spec <- uniml_spec(
  task_type = "binary",
  outcome = "died",
  positive = "yes",
  id = "id"
)

# 2. 预处理:中位数/众数填补 + z 分数标准化 + one-hot
spec <- set_preprocess(spec, impute = "median_mode", scale = "zscore", encode = "onehot")

# 3. 特征筛选:单因素 + LASSO 取交集
spec <- set_feature_selection(spec, methods = c("univariate", "lasso"), combine = "intersect")

# 4. 模型:logistic(glmnet)和决策树
spec <- set_learners(spec, c("lr", "cart"), tune = FALSE)

# 5. 评价设置:bootstrap 置信区间 + Platt 校准 + 用验证集选模型
spec <- set_evaluation(
  spec,
  ci = "bootstrap",
  calibration = "platt",
  select_on = "validation"
)

# 6. SHAP:Kernel SHAP 是工作流里最贵的一步,解释行数与背景样本量由规格控制,
#    这里取小值让 vignette 跑得快;要出论文级图时把两个数调大即可。
spec <- set_explain(spec, n_explain = 20, bg_n = 20)

# 7. 切分并跑通
split <- make_split(as_uniml_data(development, spec), prop = c(0.8, 0.2),
                    stratify = TRUE, seed = 123)

result <- run_workflow(
  spec,
  train = split$train$data,
  validation = split$validation$data,
  ci_R = 25
)
#> Warning: Fewer than 80% of bootstrap resamples were effective for: calib_slope,
#> calib_intercept, calib_large.
#> Warning: Fewer than 80% of bootstrap resamples were effective for: calib_slope,
#> calib_intercept, calib_large.
#> Warning: Fewer than 80% of bootstrap resamples were effective for: f1, f2,
#> calib_slope, calib_intercept, calib_large.
#> Warning: Fewer than 80% of bootstrap resamples were effective for: f1, f2,
#> calib_slope, calib_intercept, calib_large.
#> Warning: 1 workflow step(s) failed or were downgraded; see
#> `result$meta$failures`.

看结果

result$metrics[, c("model", "set", "probability", "auc", "brier", "accuracy")]
#>   model        set probability       auc      brier  accuracy
#> 1    lr      train         raw 0.8454917 0.04915465 0.8504673
#> 2    lr      train  calibrated 0.8454917 0.05063747 0.8504673
#> 3    lr validation         raw 0.8108108 0.05211526 0.7974684
#> 4    lr validation  calibrated 0.8108108 0.05047229 0.7974684
#> 5  cart      train         raw 0.6906186 0.04864703 0.9470405
#> 6  cart      train  calibrated 0.6906186 0.04980841 0.9470405
#> 7  cart validation         raw 0.5864865 0.06305127 0.9240506
#> 8  cart validation  calibrated 0.5864865 0.05865482 0.9240506
result$ranking[, c("model", "score", "rank", "set")]
#>   model     score rank        set
#> 1    lr 0.6238739    1 validation
#> 2  cart 0.4560517    2 validation
result$selection$selected_features
#> [1] "spo2"

result$metrics 里 probability 一列标明这一行用的是原始概率(raw)还是校准后概率 (calibrated),比较模型时不会混。

画图

plot_roc(result, set = "validation")
验证集 ROC 曲线
验证集 ROC 曲线
plot_calibration(result, set = "validation")
#> `geom_line()`: Each group consists of only one observation.
#> ℹ Do you need to adjust the group aesthetic?
#> `geom_line()`: Each group consists of only one observation.
#> ℹ Do you need to adjust the group aesthetic?
校准曲线(原始 vs 校准后)
校准曲线(原始 vs 校准后)
plot_dca(result)
决策曲线分析
决策曲线分析

图形参数可以统一控制:palette(ggsci 配色名、单色或颜色向量)、theme、title、 subtitle、caption。

解释模型

explained <- explain_shap(result, model = "best", n_explain = 25, bg_n = 15)
plot_shap_importance(explained)
最优模型的 SHAP 重要性
最优模型的 SHAP 重要性

n_explain 控制解释多少行(默认抽样 100 行),bg_n 控制背景样本量;两者共同决定 运行时间。

给新数据打分

scored <- predict_uniml(result, newdata = split$validation$data, model = "best")
head(scored[, c("model", "prob", "calibrated_prob", "probability", "predicted")])
#>   model        prob calibrated_prob probability predicted
#> 1    lr 0.345753789     0.278183612  calibrated         1
#> 2    lr 0.092485867     0.097278392  calibrated         0
#> 3    lr 0.004487839     0.009544945  calibrated         0
#> 4    lr 0.023532298     0.034083940  calibrated         0
#> 5    lr 0.037434486     0.048627179  calibrated         0
#> 6    lr 0.006265159     0.012340498  calibrated         0

predict_uniml() 复用已保存的预处理、特征筛选、校准器和阈值,不会重新拟合。

导出结果

out <- tempfile("uniml-zh-")
exported <- export_results(result, dir = out, include_plots = FALSE)
basename(exported$files)
#>  [1] "export_manifest.csv"          "result.rds"                  
#>  [3] "var_info.csv"                 "scaling_params.csv"          
#>  [5] "encoder_map.csv"              "selected_features.csv"       
#>  [7] "by_method_detail.csv"         "best_hyperparameters.csv"    
#>  [9] "cv_summary.csv"               "cv_detailed.csv"             
#> [11] "metrics_train.csv"            "metrics_validation.csv"      
#> [13] "metrics_test.csv"             "metrics_ci.csv"              
#> [15] "before_after_calibration.csv" "thresholds.csv"              
#> [17] "predictions.csv"              "leaderboard.csv"             
#> [19] "overall_rank.csv"             "params.csv"                  
#> [21] "ece_brier.csv"                "delong.csv"                  
#> [23] "nri_idi.csv"                  "dca_curve.csv"               
#> [25] "failures.csv"                 "tripod_ai.csv"               
#> [27] "importance.csv"               "values_matrix_std.csv"       
#> [29] "values_matrix_raw.csv"        "feature_values_std.csv"      
#> [31] "feature_values_raw.csv"       "final_lr.rds"                
#> [33] "final_cart.rds"

导出的 CSV 可以直接在 Excel、Python 或其它 R 脚本里用:预测概率、指标、阈值、 校准前后对比、TRIPOD+AI 自检表都在里面。

报告规范自检

checklist <- tripod_ai_check(result)
table(checklist$status)
#> 
#>      reported       partial  not_reported not_supported 
#>            12             0             2             4
subset(checklist, status != "reported", select = c("domain", "status"))
#>                        domain        status
#> 1     Data source and setting not_supported
#> 2                Participants not_supported
#> 10        External validation  not_reported
#> 12                Calibration  not_reported
#> 16     Fairness and subgroups not_supported
#> 18 Code and data availability not_supported

这个表只是自检工具:它告诉你结果对象里有哪些信息、缺哪些、哪些必须由作者自己写 (研究设计、亚组公平性、代码可得性)。它不会替你宣称“符合 TRIPOD+AI”。

更多文档

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.