vitals: Large Language Model Evaluation
A port of 'Inspect', a widely adopted 'Python' framework for large language model evaluation. Specifically aimed at 'ellmer' users who want to measure the effectiveness of their large language model-based products, the package supports prompt engineering, tool usage, multi-turn dialog, and model graded evaluations.
| Version: | 0.1.0 |
| Depends: | R (≥ 4.1) |
| Imports: | cli, dplyr, ellmer (≥ 0.2.1), glue, httpuv, jsonlite, purrr, R6, rlang, rstudioapi, S7, tibble, tidyr, withr |
| Suggests: | ggplot2, here, htmltools, knitr, ordinal, rmarkdown, testthat (≥ 3.0.0) |
| Published: | 2025-06-24 |
| DOI: | 10.32614/CRAN.package.vitals |
| Author: | Simon Couch  [aut,
cre],
Max Kuhn [ctb],
Hadley Wickham
[ctb],
Mine Cetinkaya-Rundel
[ctb],
Posit Software, PBC  [cph,
fnd] |
| Maintainer: | Simon Couch <simon.couch at posit.co> |
| BugReports: | https://github.com/tidyverse/vitals/issues |
| License: | MIT + file LICENSE |
| URL: | https://github.com/tidyverse/vitals, https://vitals.tidyverse.org |
| NeedsCompilation: | no |
| Materials: | README NEWS |
| CRAN checks: | vitals results |
Documentation:
| Reference manual: | vitals.pdf |
| Vignettes: |
Getting started with vitals (source, R code) Writing evals for your LLM product (source, R code) |
Downloads:
| Package source: | vitals_0.1.0.tar.gz |
| Windows binaries: | r-devel: vitals_0.1.0.zip, r-release: vitals_0.1.0.zip, r-oldrel: vitals_0.1.0.zip |
| macOS binaries: | r-release (arm64): vitals_0.1.0.tgz, r-oldrel (arm64): not available, r-release (x86_64): vitals_0.1.0.tgz, r-oldrel (x86_64): vitals_0.1.0.tgz |
Linking:
Please use the canonical form https://CRAN.R-project.org/package=vitals to link to this page.