The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

RenyiExtropy

RenyiExtropy provides a comprehensive collection of entropy and extropy measures for discrete probability distributions. All measures use the natural logarithm (results in nats).

Functions

Function	Description
`shannon_entropy()`	Shannon entropy H(P)
`renyi_entropy()`	Rényi entropy H_q(P) of order q
`tsallis_entropy()`	Tsallis entropy S_q(P) of order q
`normalized_entropy()`	Shannon entropy normalised to [0, 1]
`extropy()` / `shannon_extropy()`	Classical (Shannon) extropy J(P)
`renyi_extropy()`	Rényi extropy J_q(P) of order q
`max_renyi_extropy()`	Maximum Rényi extropy for n outcomes
`conditional_renyi_extropy()`	Conditional Rényi extropy J_q(Y\|X)
`joint_entropy()`	Joint entropy H(X, Y)
`conditional_entropy()`	Conditional entropy H(Y\|X)
`kl_divergence()`	Kullback–Leibler divergence D_KL(P ‖ Q)
`js_divergence()`	Jensen–Shannon divergence
`cross_entropy()`	Cross-entropy H(P, Q)

Installation

Install the released version from CRAN:

install.packages("RenyiExtropy")

Or install the development version from GitHub:

# install.packages("remotes")
remotes::install_github("itsmdivakaran/RenyiExtropy")

Quick start

library(RenyiExtropy)

p <- c(0.2, 0.5, 0.3)

# Entropy measures
shannon_entropy(p)        # 1.030 nats
renyi_entropy(p, q = 2)   # 0.968 nats
tsallis_entropy(p, q = 2) # 0.620
normalized_entropy(p)     # 0.937

# Extropy measures
extropy(p)                # 0.775 nats
renyi_extropy(p, q = 2)   # 0.742 nats
max_renyi_extropy(3)      # maximum over uniform 3-outcome distribution

# Divergences
kl_divergence(p, c(0.3, 0.4, 0.3))   # KL(P || Q)
js_divergence(p, c(0.3, 0.4, 0.3))   # symmetric, bounded by log(2)
cross_entropy(p, c(0.3, 0.4, 0.3))   # H(P) + KL(P || Q)

# Joint / conditional measures
Pxy <- matrix(c(0.2, 0.3, 0.1, 0.4), nrow = 2, byrow = TRUE)
joint_entropy(Pxy)                     # H(X, Y)
conditional_entropy(Pxy)               # H(Y | X)
conditional_renyi_extropy(Pxy, q = 2)  # J_q(Y | X)

Limit / continuity behaviour

Functions with order parameter q automatically return their q → 1 limit (Shannon entropy or classical extropy) when |q − 1| < 1e-8:

renyi_entropy(p, 1)        # equals shannon_entropy(p)
renyi_extropy(p, 1)        # equals extropy(p)
tsallis_entropy(p, 1)      # equals shannon_entropy(p)

Citation

If you use RenyiExtropy in your research, please cite:

Mahesh Divakaran, G Rajesh, Sreekumar Jayalekshmi (2024). RenyiExtropy: Entropy and Extropy Measures for Probability Distributions. R package version 0.4.0. https://CRAN.R-project.org/package=RenyiExtropy

License

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.