quallmer.app: Interactive Validation App for 'quallmer'
Companion package to 'quallmer' providing an interactive 'shiny'
application for manual coding, reviewing large language model (LLM)
generated annotations, and computing inter-rater reliability metrics.
Supports three modes: blind manual coding, LLM output validation, and
agreement calculation. Computes standard reliability metrics including
Krippendorff's alpha (Krippendorff 2019 <doi:10.4135/9781071878781>),
Cohen's kappa, Fleiss' kappa (Fleiss 1971 <doi:10.1037/h0031619>),
intraclass correlation coefficient (ICC), and percent agreement for
nominal, ordinal, interval, and ratio data. Also computes gold-standard
validation metrics including accuracy, precision, recall, and F1 scores
following Sokolova and Lapalme (2009 <doi:10.1016/j.ipm.2009.03.002>).
| Version: |
0.1.0 |
| Depends: |
R (≥ 3.5) |
| Imports: |
quallmer (≥ 0.3.0), shiny, bslib, dplyr, tidyr, irr, htmltools, cli, stats, utils |
| Suggests: |
testthat (≥ 3.0.0) |
| Published: |
2026-03-08 |
| DOI: |
10.32614/CRAN.package.quallmer.app (may not be active yet) |
| Author: |
Seraphine F. Maerz
 [aut, cre],
Kenneth Benoit
[aut] |
| Maintainer: |
Seraphine F. Maerz <seraphine.maerz at unimelb.edu.au> |
| BugReports: |
https://github.com/quallmer/quallmer.app/issues |
| License: |
MIT + file LICENSE |
| URL: |
https://github.com/quallmer/quallmer.app |
| NeedsCompilation: |
no |
| Materials: |
README, NEWS |
| CRAN checks: |
quallmer.app results |
Documentation:
Downloads:
Linking:
Please use the canonical form
https://CRAN.R-project.org/package=quallmer.app
to link to this page.