The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
Gtheory4LLM provides exact balanced Gaussian generalizability-theory models, a bounded univariate Gaussian route for missing cells, and dense or sparse first-order Laplace models for categorical outcomes in LLM annotation and evaluation studies. Specify which evaluator, prompt, and repeated-run sources matter for your study, fit their variation, and compare projected reliability across alternative numbers of evaluators and prompts.
This is a research beta. Passing the package’s numerical acceptance checks does not establish parameter recovery, interval coverage, Laplace approximation quality, or a scientifically sufficient number of evaluators. Read docs/LIMITATIONS.md before quoting a coefficient, and docs/VALIDATION_SCOPE.md for what has actually been validated.
Source version: 1.0.0. For versioned archives, manuals and publication status, see the repository manifest and GitHub releases. These repository records are excluded from the package archive; this source version does not assert that a corresponding release has been published.
This version adds sparse categorical estimation and fixed call effects, optional information-based uncertainty, incomplete univariate Gaussian panels, shared categorical simulation, and explicitly named observed, decision and category-vector targets. Sparse categorical fits can use an opt-in analytic Laplace gradient, whose selected inverse and local contractions run in compiled code, and observed information from its differences. Cost planning can screen those simulated targets. Pilot planning assesses latent binary/ordinal interval precision or point-estimator behavior for the observed, vector and decision targets. The formula interface and historical release artifacts are retained.
Publication and qualification of the selected release archive are recorded in the repository manifest. Full-panel solver performance, approximation accuracy, interval coverage and cross-platform release qualification remain separate work; see development status.
R 4.5 or later and OpenMx are required. The bootstrap observation
identity uses tools::sha256sum(), introduced in R 4.5.0.
OpenMx 2.22.11 also calls a C entry point introduced in R 4.5.0 while
declaring only R (>= 3.5.0) itself; the package’s
declared floor prevents a later compilation failure. The same minimum
applies when sourcing load_functions.R directly. Older
R/OpenMx pairings reported for earlier source versions do not establish
compatibility with this implementation. This version contains C code, so
installing it from source, and source("load_functions.R"),
which compiles the same C file, need the C toolchain R itself uses: the
Xcode command-line tools on macOS, Rtools on Windows, or a C compiler on
Linux.
The current CRAN release, which can be older than this source version, installs with its dependencies:
install.packages("Gtheory4LLM")A specific release can also be installed from its versioned archive,
for example the v0.6.0
archive, Gtheory4LLM_0.6.0.tar.gz:
install.packages("OpenMx")
install.packages(
"https://github.com/Veronica0206/Gtheory4LLM/releases/download/v0.6.0/Gtheory4LLM_0.6.0.tar.gz",
repos = NULL, type = "source"
)
library(Gtheory4LLM)The repository keeps checksummed release files in artifacts/.
A 0.4.0 bundle, Gtheory4LLM_0.4.0.tar.gz, was prepared and
tagged but never published; 0.4.1 supersedes it. To install this source
version, run R CMD build ., then
R CMD INSTALL Gtheory4LLM_1.0.0.tar.gz, the archive it
creates. For the exact checked release archive, use the versioned GitHub
release and verify its accompanying manifest. Building the vignette
needs knitr, rmarkdown, and pandoc; using the installed package does
not. CRAN availability is separate from GitHub availability. The version
CRAN carries is recorded in the repository development
status.
The installed tutorial creates synthetic continuous judgments for 24 items, 4 evaluators, 3 prompts, and 2 runs within each evaluator/prompt pair. It completes design inspection, fitting, diagnostics, G/Phi calculation, and an 18-allocation decision study. It requires no API keys or external data.
source(system.file("doc", "LLM-workflow.R",
package = "Gtheory4LLM", mustWork = TRUE))The same workflow is installed as a vignette:
browseVignettes("Gtheory4LLM"), or
vignette("LLM-workflow", package = "Gtheory4LLM"). Its
source is vignettes/LLM-workflow.Rmd. The
main steps are:
# llm_data is created by the tutorial above.
llm_formula <- quality ~ item * (evaluator + prompt) + evaluator:prompt:run
gt_preflight(llm_formula, llm_data, measured = "item")
fit <- gt_fit(llm_formula, llm_data, measured = "item")
summary(fit)
gt_diagnostics(fit)$numerically_accepted
as.data.frame(gt_reliability(fit))
dstudy <- gt_dstudy(fit, expand.grid(evaluator = c(2, 4, 6),
prompt = c(2, 3, 4), run = c(1, 2)))
as.data.frame(dstudy)The formula names outcomes and random sources in one call. Use
gt_design() when a facet appears in no random-source term,
or when declaring within-cell replication or batches; those declarations
have no formula syntax. The study-planning vignette retains explicit
designs for these cases.
G describes relative comparisons between items. Phi additionally includes absolute shifts across the declared measurement conditions. Decision studies average over specified random facet populations and hold fitted source covariances fixed. More evaluators or prompts are meaningful projections only if those populations and covariance assumptions remain appropriate. These are point estimates with intervals only when uncertainty is available. For example, G = 0.80 describes the universe-score share of variance for relative item comparisons under the specified model; it is not 80% labeling accuracy or agreement with a human reference. Phi additionally penalizes absolute panel shifts and is no larger than G under this model.
The default Gaussian engine reports asymptotic standard errors and delta-method intervals where the likelihood curvature and boundary rules permit them. They use the restricted (REML) or profile (ML) likelihood Hessian:
summary(fit)$variances # Documented source-variance summary table.
subset(as.data.frame(gt_reliability(fit)), kind == "outcome")
logLik(fit); AIC(fit); gt_component_vcov(fit)These are asymptotic Wald quantities conditional on the declared
model and allocation. A source resting on a variance boundary has no
Wald standard error. For discrete fits,
gt_control(discrete = list(check_hessian = TRUE)) requests
observed information for nuisance and covariance parameters. A source at
a covariance boundary is held at its fitted matrix while the remaining
coordinates are assessed. When those information checks pass, the
resulting coefficient intervals explicitly name the held sources and are
conditional on them; the held covariance estimates themselves receive no
standard errors. All discrete intervals also depend on the Laplace
approximation, and their general finite-sample calibration remains
unestablished. gt_bootstrap(fit, replicates, seed)
resamples the objects instead, refitting each resample under the fit’s
own acceptance rule, and
gt_reliability(fit, bootstrap = boot) and
gt_dstudy() then report percentile or BCa intervals
conditional on the observed instrumentation levels. Their interpretation
and availability limits are described in docs/LIMITATIONS.md.
That document also explains the quadratic evaluation cost of discrete
information; full-panel point estimation does not qualify interval-pilot
work at the same size.
A facet is random when the study generalizes to a population of its
levels, and fixed when the universe is exactly the levels used. A chosen
temperature or prompt set may be fixed when inference is restricted to
those conditions; the choice follows the intended generalization, not
the facet name. Declare it with fixed, which applies the
mixed model of Brennan (2001): the object-by-fixed interaction is
averaged over that facet’s levels and joins universe-score variance, and
a source built only from fixed facets leaves the model.
gt_reliability(fit, fixed = "prompt")A fixed facet’s count cannot be changed, and a decision study may not project over it.
as.data.frame() exports reliability or D-study results
with allocation, coefficient scale, uncertainty status and extrapolation
recorded alongside each estimate. gt_dstudy_target()
identifies supplied allocations meeting an estimated target and
preserves ties for the fewest measurements among them. It does not
optimize beyond the supplied grid or guarantee future reliability.
reliability <- gt_reliability(fit)
as.data.frame(reliability)
plot(reliability, target = 0.80)Preflight plots show cell completeness and replication, random-source
dimensions, or category representation across design groups. Outcome
profiles include category counts and groups with no observed variation;
they do not certify adequate information. D-study plots show
coefficients against measurements per object, with optional target lines
and extrapolation markers. Figures use base R graphics, so standard
PNG/PDF devices can save them. The installed LLM-workflow
vignette demonstrates these views and CSV export; the
visualization guide explains their interpretation.
A portable report combines retained model context, diagnostics, result tables and figures without rerunning optimization:
report <- gt_report(fit, reliability = reliability)
# Writes a standalone offline HTML file; existing files are protected.
# gt_export_report(report, "analysis-report.html")Reports exclude observations and group-level identifiers, while retaining variable/source names and aggregate results. Review those names before sharing. Unavailable fitting provenance stays unavailable; the report-generation session is recorded separately. See the reporting guide.
Study planning separates three quantities: distinct pilot items provide estimation information, annotations per item define the final score, and items per request define shared context and request overhead. Increasing one does not substitute for increasing another. These workflows are included in the package since version 0.5.0; their statistical boundaries are described below.
The study-planning vignette runs synthetic examples of all three additions. Its monetary amounts are illustrative user inputs.
gt_batch_reliability() and
gt_batch_dstudy() project absolute item-score error,
explicitly named item differences and weighted aggregate error for
supported Gaussian shared-call fits. A call shift cancels from a
same-batch difference but remains in an absolute score. Targets
condition on the recorded fixed item grouping; all instrumentation
facets remain random. These are point projections, with no sampling
intervals, regrouping or batch-size optimization.gt_plan() evaluates a supplied bounded grid using
dated, user-supplied costs and constraints. It retains every candidate,
minimum-cost ties, the cost versus reliability Pareto set, and changes
in error contributions from a declared baseline. It screens a point
estimate or, separately, an available lower confidence bound. Missing
bounds stay unavailable. It does not choose evaluator identities or
change a facet from random to fixed to improve a result. Categorical
observed and decision targets use point screening and report Monte Carlo
integration error separately from parameter uncertainty.gt_simulate() draws Gaussian or categorical panels
under a fitted or explicit parameter scenario.
gt_pilot_plan() refits bounded Gaussian or binary/ordinal
pilots, including joint outcomes, to assess interval precision for
observed Gaussian or latent binary/ordinal coefficients. Pilot
item/facet counts may vary while target_counts holds the
eventual annotation protocol constant. The planner records every
attempt, refusal and unavailable interval, with Monte Carlo errors and
conditional interval-width summaries. This evaluates model-based
precision, not calibrated coverage or guaranteed study adequacy. For
categorical observed-score, category-vector or decision targets, a
separate point-estimator route reports acceptance, target availability,
bias and RMSE against an independently integrated scenario reference. It
includes nominal outcomes and eligible fixed calls, retains every failed
attempt, and reports reference and inner integration errors
separately.For example, after fitting an accepted model without a shared-call source:
plan <- gt_plan(fit,
candidates = list(evaluator = 2:4, prompt = 1:3),
cost = function(d) data.frame(annotation = 0.01 * d$total_annotations,
setup = 2 * d$allocation_evaluator),
currency = "USD", cost_date = "2026-10-09", study_items = 1000,
target = 0.80, coefficient = "Phi")
subset(as.data.frame(plan), minimum_cost %in% TRUE)The analytic cost-planning route refuses modelled Call
fits. The categorical target route supports the fitted equal fixed-call
layout, with unchanged item count and group membership; it does not
optimize batch size or regroup items. Use the separate batch projection
functions for Gaussian call targets. Supplying request costs for an
independent-item model does not account for shared-call error, and
adding human-review costs does not itself model a benefit from review.
Pilot interval-width planning refuses batch declarations and nominal
outcomes; their point-target route supplies no parameter intervals or
coverage. See the function manuals for the remaining scope and
limits.
A new shared-call fit retains compact item-to-batch labels even with
retain = list(data = FALSE): the layout is necessary for
named comparisons. Batch projection objects contain those labels and
supplied target weights. gt_report(fit) and its HTML export
exclude the labels; share a projection object only when its identifiers
are appropriate to disclose. Older fits lacking the layout must be
refitted rather than having their grouping guessed.
Facet names do not impose a hierarchy. Declare any nonempty subset of
the instrumentation facets to interact with the object, give other
facets explicit nesting parents, and choose higher-order interactions
separately. With four facets, all 15 nonempty crossed subsets are
supported by the design interface. An explicit random-source formula is
also available, and gt_fit() and
gt_preflight() accept a two-sided model formula such as
score ~ item * rater in place of the outcome names and the
design. gt_preflight() reports the requested and resolved
sources, dimensions, replication, resource limits, and downstream
operations before optimization; it does not certify statistical
identification or successful fitting.
| Outcome | Fitting | Coefficients and explicit projection targets |
|---|---|---|
| Gaussian | Exact balanced ML/REML, univariate or joint; bounded observed-row ML/REML for incomplete univariate panels | Observed-score G/Phi and explicit composites; incomplete fits require an intended balanced projection |
| Binary | Dense or sparse Bernoulli probit/logit, univariate or joint | Latent G/Phi; separate observed-score, category-vector and decision targets |
| Ordinal | Dense or sparse cumulative probit/logit, univariate or joint | Latent G/Phi; separate declared-score, category-vector and decision targets |
| Unordered categorical | Dense or sparse reference-category multinomial softmax, univariate or joint; unstructured, exchangeable or diagonal covariance | Latent G/Phi as the mean and range of the generalized eigenvalues over the category contrasts, invariant to the reference category (exchangeable or unstructured covariance); separate category-vector and decision targets; no arbitrary numeric-score ratio |
gt_family() records category order and reference
categories explicitly, beyond the distribution and link in a base R
family object. Named lists match these specifications to joint outcomes.
Gaussian and discrete outcomes cannot be combined in one joint fit.
Gaussian fits use the exact balanced likelihood through OpenMx by
default. For univariate Gaussian models,
gt_control(gaussian = list(engine = "lme4")) fits the same
sources through lme4. Acceptance requires completion, stationarity and
restart checks, plus agreement between the two deviance calculations at
the lme4 estimates. These checks do not prove global optimality. This
alternative does not supply source-variance standard errors or
delta-method coefficient intervals; for a complete panel without
declared batches, gt_bootstrap() supplies object-bootstrap
coefficient intervals.
To fit missing whole cells in a univariate Gaussian panel, explicitly
select
gt_control(gaussian = list(engine = "lme4", panel = "observed")).
The route checks an independent dense observed-row ML/REML likelihood
and covariance-kernel identification, with a default limit of 1,000
observed rows and a preparation-memory guard. It fills no cells and
assumes missingness is ignorable for the specified covariance model.
Ordinary reliability of that incomplete allocation remains unavailable.
gt_dstudy(..., projection = "balanced") instead requires
every intended facet count and projects to that complete future
allocation with all facets random; no coefficient intervals are
supplied.
Discrete designs may omit whole cells, including for joint outcomes, provided retained cells have equal declared replication and each retained row contains all requested outcomes. Partially observed outcome vectors are rejected. This does not model missingness or give analytic reliability for an irregular panel.
Discrete fits use a first-order Laplace likelihood. Select sparse
algebra with
gt_control(discrete = list(engine = "sparse")); dense
remains the default. Both routes support binary, ordinal and nominal
outcomes, joint categorical models and eligible equal fixed-call
layouts. Shared row, dimension and parameter guards still apply,
alongside the selected engine’s memory guard. Sparse algebra does not by
itself qualify a large model or improve the approximation.
gt_control(discrete = list(engine = "sparse", gradient = "analytic"))
gives the L-BFGS-B attempts the analytic gradient of the Laplace
objective in place of finite differences, for natural Cholesky,
log-Cholesky or fixed source covariances; with
check_hessian = TRUE the observed information then
differences that gradient. The route changes the search, not the
objective or the acceptance rules, and a refused gradient ends its
attempt without a fallback. An integrated external engine adapter for
these outcomes is not implemented. gt_laplace_check(fit)
compares its objective at the fitted parameters with numerical
quadrature or importance sampling. The latter uses a defensive mixture
of 90% Laplace Gaussian and 10% random-effect prior, with the full
mixture density in every weight. This guarantees finite weight moments
for the supported categorical probability likelihoods; it does not
guarantee adequate precision from a finite sample. The Monte Carlo
standard error and empirical effective sample size are diagnostics, and
the comparison does not estimate parameter bias.
Discrete observations follow their chosen response distributions; their latent random effects remain Gaussian. Gaussian assumptions therefore remain part of the model. Latent binary, ordinal or nominal reliability concerns an averaged latent response, not observed proportions, majority votes, or category agreement. Joint Gaussian-discrete models are not implemented.
gt_categorical_target() defines those other targets
explicitly using paired future panels that share object effects and
redraw instrumentation effects. Its score target is the absolute
repeated-panel variance ratio; ordinal scores must be declared. Its
category-vector target uses category proportions and is invariant to
relabeling unordered categories. Decision agreement uses plurality votes
with uniform tie-breaking and reports decision diversity: a constant
label can have perfect agreement and no discrimination. These targets
hold fitted parameters fixed and report simulation error, not parameter
confidence intervals. Weak positive object signal can cause frequent
unavailable ratios and upward bias in available-only summaries. Retain
all attempted seeds and refusals, and check precision using independent
panel pairs; more objects within one panel cannot remove
shared-instrumentation variation. The target help page describes budget
planning and its limits. They require no model refits.
gt_plan() can compare them using
coefficient = "observed_score",
"category_vector" or "decision_agreement",
with an explicit scale and seeded integration controls; see
help("gt_plan").
Traditional method-of-moments (MoM) G theory already accommodates random effects and crossed/nested designs. The package’s contribution is the common workflow for explicit source selection, joint covariance models, different response families, and supported allocation comparisons. It does not claim that random effects are novel or that likelihood is uniformly superior to MoM. See Brennan (2001) and Jiang et al. (2020) for G theory and earlier multivariate likelihood implementations.
At one observation per object-by-facets cell, a discrete fit cannot identify the object-by-all-facets source that the default design requests, and that source is never dropped silently. Declare the same design without it:
binary_design <- gt_design("item", "rater", full_cell = FALSE)full_cell = FALSE removes exactly that one term and
leaves every other requested source unchanged;
random = ~ item + rater writes the reduced model out in
full. Either way the omission is an explicit modeling choice.
A first discrete fit, start to finish — the guard above, preflight, binary and ordinal fits, diagnostics, latent reliability and a decision study — is installed as a runnable script:
source(system.file("examples", "discrete-first.R", package = "Gtheory4LLM"))system.file("examples", "discrete-boundary.R", package = "Gtheory4LLM")
shows the companion case: a constructed panel whose zero object variance
is a legitimate optimum rather than evidence about reliability.
gt_control() exposes named starting values, reproducible
additional fitting attempts, an optimizer held fixed across attempts,
the discrete resource limits, and which optional components a fit keeps.
Discrete fits retain rejected candidates and their reasons.
summary(fit) reports acceptance, selected attempt, boundary
status, and approximation status without printing model internals or
observations; detailed records remain available.
For reproducible fitting, retain the exact source archive and its
checksum, input data and row order, response coding, source
specification, controls, starting values and seeds, and the R,
dependency and numerical-library versions. Use an explicit Gaussian
retry_seed and a fixed thread count; simulation and Monte
Carlo checks need their own recorded seeds. A seed controls random
draws; it does not guarantee identical floating-point results across
environments.
In the same recorded environment, replay the same request and compare the estimates and numerical decisions with the saved run. Across platforms, compare matched, unrounded quantities under tolerances declared before seeing the new results. Numerical acceptance and uncertainty availability must also agree for a claimed successful replay. Record differences and refusals rather than widening tolerances after a failure. The installed native workflow below records its comparison policy; it does not promise bitwise-identical estimates or generalize its tolerances to other models.
The default Gaussian route and ordinary analytic coefficients require
a complete balanced coded panel. The explicitly selected observed-row
Gaussian route and intended balanced projection have separate guards
described above. Both discrete engines are bounded by row, dimension,
parameter and working-memory limits; sparse requests use the sparse
memory budget rather than the dense budget, and that budget is judged
against the design’s structural pattern (the exact count of the
conditional Hessian’s structural nonzeros, then one factorization of the
pattern, a planning estimate rather than a bound) rather than a dense
square. The exact Gaussian engine may recover once from a bounded
square-root search after an ordinary refusal of a multivariate
unstructured model; it consumes an existing attempt and decides nothing,
and initialization = "none" disables it.
gt_preflight() reports these structural and resource checks
before optimization.
Every limitation is listed once, in docs/LIMITATIONS.md — what the designs, the Gaussian engine and the discrete engine do and do not support, which combinations are not implemented, and what “supported” means here. Read it before quoting a coefficient. What has actually been checked, and what a future study still has to establish, is in docs/VALIDATION_SCOPE.md; the initial statistical pilots preserve their coverage, approximation and recovery results, including boundaries and unavailable comparisons.
The real-data workflow audits the native outcome types and full designs of all three panels, then demonstrates how to interpret unsupported requests and numerical failures.
gt_example() lists eight outcome sets from three
real LLM annotation panels: hate speech, mental health,
and drug reviews. Each contains 21,600 measurements of 100 items crossed
with four evaluators, three prompts, six temperatures, and three seeds.
These are alternative codings of three panels, not eight independent
datasets. The package includes design identifiers and modeled
annotations from the public OSF deposit. The
original source CSV checksums match that deposit. Raw texts, original
corpus reference labels, and API metadata are omitted from the package
tables. Use gt_example() rather than data() to
load an outcome set: it returns the chosen coding, its family
specification, and provenance together with the data.
The six discrete native outcome sets each contain 21,600 rows and
exceed the dense discrete engine limits by more than an order of
magnitude; raising the limits does not make the dense engine practical
at that size. With the sparse engine, the analytic gradient and
boundary_escape = "projected", the 25 October 2026 project
records report numerical acceptance on macOS and Linux for two
deliberately specified joint outcome sets: the efficacy and safety
aspects of drug_review_4aspect, and
mental_health_3group. Each uses all 21,600 rows and the ten
sources declared in its recorded model. These examples do not establish
a result for all four drug-review aspects or all six mental-health
flags. Ordinary information was unavailable in those original runs.
Subsequent macOS calculations at the saved estimates, without refitting,
produced conditional coefficient intervals by holding three drug-review
and five mental-health source matrices fixed. Those matrices are
singular and nonzero. The original point references come from
development source 2fc771b. On 28 October 2026, the 1.0.0
candidate built from 2b31e84 was refit on both panels on
macOS and Linux. All four fits were accepted and met both the point and
conditional-information comparison criteria. Its archive SHA-256 is
recorded in the scoped
evidence in development status. That result belongs to those
candidate bytes. The subsequent bootstrap data-association repair and
documentation changes require separate package checks; their presence
does not constitute another full-data refit. No interval-coverage study
is supplied by these runs. Numerical agreement alone does not establish
first-order Laplace accuracy or calibrated intervals.
The installed native workflow makes those two requests and their
reporting steps explicit. Sourcing the script defines the helper. Its
optional full mode performs new long fits using the current
conditional-information rule; the historical point estimates and later
uncertainty calculations remain distinct references. Follow the
one-thread launch and isolated-library setup in the
native workflow guide before choosing to run it:
source(system.file("examples", "native_1_0_workflow.R",
package = "Gtheory4LLM", mustWork = TRUE))
gt_native_workflow(mode = "full", output_dir = "native-1.0-results",
archive_path = "Gtheory4LLM_1.0.0.tar.gz")Use mode = "smoke" for a small synthetic workflow check;
its result is not a whole-data reproduction. The supplied archive hash
identifies the file passed to the helper; retain the installation record
to establish that the loaded package came from those bytes. Do not
remove sources or outcomes merely to obtain an accepted fit. The two
mental-health 7L/3L sets remain Gaussian working scores under both
coding options.
Temperature-specific seed agreement is a useful descriptive diagnostic. Lower agreement at higher temperatures may reflect changes in category probabilities, latent variances, or both; agreement alone does not identify which changed.
Use fixed = "temp" when the coefficient is intended to
average over exactly the observed temperatures. This changes the
reliability estimand after fitting; it cannot repair misspecification in
the fitted variance model. If pooling across temperatures is
questionable, a within-temperature analysis is one sensitivity analysis,
with its own conditional estimand. It is not the only possible model and
does not generalize across temperatures automatically. See
help("gtheory_datasets").
Other native codings preserve binary, ordinal, or unordered
categorical outcomes; coding = "numeric" reproduces the
seven historical Gaussian working-score codings. Mental-health 7L/3L
values are working scores, not established clinical severity scales. The
installed manual documents variables, category mappings, preprocessing,
source corpora, and the corresponding annotation studies. In particular,
the original mental-health preprocessing could default omitted items
within a parsed batch to NORMAL; these defaults cannot be distinguished
in the bundled tables. Interpret the labels with that limitation in
mind:
gt_example()
x <- gt_example("hate_speech")
help("gtheory_datasets", package = "Gtheory4LLM")
help("gtheory_hate_speech", package = "Gtheory4LLM")
help("gtheory_mental_health", package = "Gtheory4LLM")
help("gtheory_drug_review", package = "Gtheory4LLM")The task bibliography is
installed as
system.file("DATASET_REFERENCES.bib", package = "Gtheory4LLM").
Use citation("Gtheory4LLM") for the software citation.
Loading a table does not establish that every design or full joint
discrete fit is feasible.
A single release entrypoint checks sources and the distributed archive separately:
python3 scripts/run_validation.py --scope all --as-cranThe entry point selects the release manifest the same way the hosted
workflows do: a development or published source version keeps the
retained manifest, and a frozen, not yet tagged release version whose
retained manifest still records the previous release gets a rehearsal
bundle built outside the checkout, so the command works on such a
checkout without bypassing the identity checks. A staged rehearsal is
read by the source scope’s identity check only; the artifact scope
always checks the retained, committed bundle, and
--scope artifact stages nothing. Staging uses the run’s
interpreter, library and timeout, and is recorded as the
release_manifest_staging stage of the report. A version
that is already tagged cannot be rebuilt as a rehearsal; validate it
against its own bundle with --release-manifest PATH, or
after its bundle commit has been merged.
See validation instructions for the locked environment, minimum-R and Windows/macOS checks, source-only checks, and retained evidence. The numerical suite includes independent dense Gaussian likelihoods, binary/ordinal comparisons with lme4 and ordinal, likelihood/derivative identities, joint covariance estimation, and coefficient aggregation. These are specific numerical checks, not a substitute for simulation-based statistical validation. A configured workflow is not evidence of a passing run.
Report installation failures, unclear design declarations, and numerical problems through GitHub issues. Please include the package/R versions, a small synthetic example, the declared family/design, and relevant diagnostics. Remove private observations and credentials before posting. Maintainer: Jin Liu, Veronica.Liu0206@gmail.com.
Package code is licensed under GPL-3. The real annotation tables retain the CC BY 4.0 license stated in the public deposit’s data codebook; see data attribution and license. Synthetic tutorial and test examples are covered by the code license. Public release files are limited to the package, its documented examples, tests, and software release artifacts. Unpublished manuscripts and research archives are excluded.
Each document has one job, so that nothing has to be kept true in two places.
| Document | What it covers |
|---|---|
| This README | Install, a complete worked workflow, and what the package is for |
| vignettes/LLM-workflow.Rmd | The end-to-end reliability tutorial, installed and runnable |
| vignettes/study-planning.Rmd | Fixed-layout batch targets, monetary allocation search and Gaussian pilot precision |
| docs/LIMITATIONS.md | Everything the package does not do, listed once |
| docs/VALIDATION_SCOPE.md | What has been checked, how, and what that does not establish |
| docs/REAL_DATA_WORKFLOW.md | The bundled panels, their native outcomes, and their resource ceilings |
| docs/DEVELOPMENT_STATUS.md | The current release state and what is being worked on |
| docs/ROADMAP.md | Planned work beyond this release, in the order it is planned |
| docs/DISCRETE_BACKEND_CONTRACT.md | Private dense response, matrix and mode interfaces |
| NEWS.md | Version history |
| docs/CONTRIBUTING.md, docs/RELEASE_CHECKLIST.md, docs/REPOSITORY_POLICY.md, SECURITY.md | Working on the package itself |
See the contribution guide for setup and statistical change requirements, and the release checklist for version, tag, archive, and manual correspondence. Software checks and scientific validation are reported separately. The development status tracks completed work and remaining extensions.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.