The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
linreg() / inverse_noise() silently
returned support vertices in large samplesBoth functions defaulted the noise support to a fixed three-sigma
rule, v = c(-3s, 0, 3s) with s = sd(y).
Because the largest of T errors grows like
sigma * sqrt(2 log T), that support becomes
infeasible once T is more than a few
thousand: y can no longer be written as
X beta + e within the supports, the concentrated dual is
then unbounded below, the multipliers diverge, and the
estimates collapse to a vertex of the support (e.g.
beta = (3, -3, 3), or p = (0, 1, 0)) – while
optim() still reported convergence = 0 and no
warning was given.
Two changes fix this:
max(3, sqrt(2 log T)) * sd(y). On a 10,000-observation
regression the default fit now recovers the OLS coefficients (previously
it returned a support vertex). The signal support Z is
unchanged, since beta does not grow with T. An
explicitly supplied v is never modified.y - X beta - e = 0 at the optimum, report it as the new
foc_residual field, and warn when it is
violated – which catches every infeasible-support case, including a
too-narrow Z. linreg() additionally reports
converged = FALSE.Behaviour change: fits that relied on the default
v with more than about 90 observations will shift slightly,
since the default support is now a little wider (results were unchanged
to two decimals at n = 30-150 in testing).
linreg_iv()
now reports standard errorslinreg_iv() gains standard errors for the coefficients
via a new se_method argument: "sandwich"
(default, robust HC0), "delta" (classical homoskedastic),
"bootstrap" (pairs resampling), or "none".
\beta = Zp is a Z-estimator of the instrument moments, so
Var(beta) = J A^-1 Omega A^-1 J'. Validated against a
Monte-Carlo sampling SD and the 2SLS robust-SE limit (the sandwich
matches both; the classical delta assumes homoskedasticity). Adds a
vcov() method (type = "sandwich"/"delta") and
an Estimate | Std. Error | z | Pr(>|z|) table in
summary(). These are asymptotic SEs: the sampling
distribution of beta is support-bounded and can be skewed,
so Wald intervals are approximate with weak instruments or small
n.
linreg()
normalized entropy is now prior-relativelinreg()’s reported normalized entropy S
now uses the prior as the reference, S = H(p_hat) / H(p0)
(Golan 2008, Sec. 6.4), matching matrix_ce(),
markov_ce(), and inverse_noise(). For a
uniform prior this is unchanged (H(p0) = K log M, so GME
output is identical); for an informative p0 (GCE)
S is measured against that prior and can exceed 1 when the
data are less informative than the prior.
A second vignette, Generalized Maximum Entropy and Cross-Entropy
for the Linear Model, demonstrates linreg() and
linreg_iv(): GME versus OLS under collinearity, GCE with an
informative prior, the entropy weight nu, the
normalized-entropy / entropy-ratio / pseudo-R-squared and Wald
diagnostics, and an instrumental-variables example.
The package now focuses on the Generalized Maximum Entropy (GME) and
Generalized Cross-Entropy (GCE) estimator family. The following
functions were removed: me(),
gme(), me_mnl(), gme_mnl(),
gce_clogit(), gce_clogitWrap(), and
gce_table(). The retained estimators are
inverse_ce(), inverse_noise(),
linreg(), linreg_iv(),
panel_gce(), matrix_ce(),
matrix_gce(), markov_ce(),
markov_gce(), multinomial_gce(), and
mixed_gce().
The entropy.R module was trimmed to
shannon_entropy(); the divergence measures
kl_divergence(), renyi_entropy(),
renyi_divergence(), tsallis_divergence(),
cressie_read(), and normalized_entropy() were
removed. The deprecated inverse_pure() alias was also
dropped — use inverse_ce() directly.
inverse_ce() (was
inverse_pure())inverse_pure() has been renamed to
inverse_ce() (the method is cross-entropy: a
uniform prior gives ME, a non-uniform prior gives CE).inverse_ce() now reports information-matrix
standard errors Var(lambda) = I^-1
(se_lambda, vcov()) and delta-method
se_p, plus fano_bounds() for
the recovered p. These are curvature/identification
quantities (Golan 2008, sec 4.2), not sampling SEs;
NA/NULL when the information matrix is
singular.q to
p0 (the GME/GCE signal-prior name; for
inverse_ce it is a K-vector prior over the states).p_hat/lambda_hat/objective/
p0); dropped converged (the raw
convergence code is kept).inverse_noise()lambda_hat and
p_hat via se_method / vcov(type=)
/ summary(se_method=): "sandwich" (default,
robust HC0 — accurate), "delta" (naive H^-1;
overstates ~10x, for comparison only), and "bootstrap"
(residual bootstrap; stable aligned SEs for both parameters).
vcov()’s default changed from the raw
H^-1 to the sandwich, since H^-1 is not a
valid sampling covariance here (type = "delta" recovers
it).summary() now prints a p coefficient table
with standard errors and t-stats, reports
normalized entropy for both signal S(p) and noise
S(w), and adds a Fano line;
fano_bounds() is available for
p_hat.p_hat/lambda_hat/w_hat/
residuals/objective); dropped
converged (the raw convergence code is
kept).GME / GCE estimator (R/gme.R):
gme(formula, data, Z, p0, V, w0, nu, method, control) —
Generalized Maximum Entropy (GME) and Generalized Cross-Entropy (GCE)
estimation for linear regression, following Golan, Judge and Miller
(1996) and Golan (2008, Chapter 6). Uniform priors
(p0 = NULL, w0 = NULL) give GME; user-supplied
priors give GCE. One function, not two — the same relationship as
me() to ME/CE.Omega_k) and error (Psi_t) partition
functions use the log-sum-exp trick for numerical stability.print, summary,
coef, fitted, residuals,
vcov.S_p, overall
S_P, and pseudo_R2 (Golan 2008, Section
7.5).Golan, A., Judge, G. and Miller, D. (1996). Maximum Entropy Econometrics. Wiley.
Entropy measures (R/entropy.R):
shannon_entropy(p, base) — Shannon entropy H(p) = -Σ p
log p, with configurable base (nats, bits, hartleys). Convention
0·log(0) = 0 applied.kl_divergence(p, q, base) — Kullback-Leibler divergence
D(p‖q).renyi_entropy(p, alpha, base) — Rényi entropy of order
α; reduces to Shannon as α → 1.renyi_divergence(p, q, alpha, base) — Rényi
cross-entropy of order α (Golan 2008, Eq. 3.6).tsallis_divergence(p, q, alpha) — Tsallis cross-entropy
of order α (Golan 2008, Eq. 3.7).cressie_read(p, q, alpha) — Cressie-Read power
divergence, the unifying criterion for the IT estimator class (Golan
2008, Eq. 3.8). Special cases include KL (α → 0), Pearson χ² (α = 1), EL
(α → -1).normalized_entropy(p, q) — S(p̃) = H(p) / H(q), the
primary goodness-of-fit statistic for IT models (Golan 2008, Section
6.4).ME / CE estimator (R/me.R):
me(y, X, q, method, control) — Maximum Entropy (ME) and
Cross-Entropy (CE) estimation for under-determined systems. Both primal
(constrained, via nloptr) and dual (concentrated,
unconstrained, default) formulations are implemented.print, summary,
coef, fitted, residuals.Utilities (R/utils.R):
normalize_data(x, by) — rescales data to [0, 1] to
prevent overflow in partition function computations.make_support(half_range, M, center) — constructs
symmetric M-point support spaces for GME/GCE estimation.default_supports(y, X, ...) — constructs data-driven
default signal (Z) and error (V) support spaces from OLS estimates,
using the three-sigma rule for error bounds (Pukelsheim 1994).Golan, A. (2008). Information and Entropy Econometrics — A Review and Synthesis. Foundations and Trends in Econometrics, 2(1-2), 1-145.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.