The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

BIDistances: Distance computation, theory-guided selection, and scalable weighted Minkowski distances in R

Overview

BIDistances provides a common interface for calculating, comparing, and examining distances, dissimilarities, divergences, and selected similarities. The package connects four tasks that are usually treated separately:

1. Access a broad collection of distance methods through DistanceMatrix().
2. Check whether a method is a metric, pseudometric, non-metric dissimilarity, or divergence with DistanceProperties().
3. Select a candidate for distance-based cluster analysis by examining distance distributions with DistanceDistributions().
4. Scale weighted Minkowski calculations from multicore CPUs to optional OpenCL devices while retaining an optimized weighted Euclidean route for p = 2.

Additional domain-specific functionality includes a Gene Ontology-derived TF-IDF distance, time-series distances, nearest-neighbour distances, toroidal Euclidean distances, and mixed-data constructions.

Main advantages

• A unified DistanceMatrix() interface for native BIDistances methods and all predefined methods supplied by parallelDist, with optional access to philentropy and manydist.
• 60 canonical named numerical-data routes in the core installation and 94 when philentropy is installed. This count excludes aliases, user-defined C++ functions, direct-only functions, and the configurable mixed-data constructions in manydist.
• Weighted Minkowski distance calculation through parallelDist, an internal RcppParallel multicore backend, or optional OpenCL kernels.
• A dedicated weighted Euclidean GPU implementation retained as the optimized p = 2 OpenCL route.
• Memory-aware full-matrix, output-batched, and input-blocked OpenCL execution.
• DistanceDistributions() for comparing candidate distance distributions and identifying multimodality under the goal of finding distance-based cluster structures. Distance vectors above 72,000 values use the integrated batch dip-test backend, while smaller vectors retain diptest::dip.test().
• Tfidf_Distance() for deriving distances between genes from a gene-by-Gene-Ontology-term feature matrix.
• DistanceProperties() for documenting the mathematical properties and conditions of native methods.

Method coverage in release 1.0.0

manydist adds preset-based and custom mixed-type distance constructions. These are not included in the total because their specifications are configurable rather than a fixed list of independent methods. Direct-only functions such as TWED_Distance() and ToroidalEuclidean_Distance() are also outside the dispatcher count.

Installation

From CRAN:

install.packages("BIDistances")

From GitHub:

install.packages("remotes")
remotes::install_github("Mthrun/BIDistances")

System requirements are R >= 3.5.0, GNU make, and a suitable C++ toolchain. An OpenCL library and the R package OpenCL are optional and are required only for GPU acceleration. Pandoc is required when rebuilding the vignette.

Quick start

Example 1: Several distances through one interface

library(BIDistances)

data("Hepta")
Data = Hepta$Data

D_euclidean = DistanceMatrix(
  Data,
  method = "euclidean"
)

D_minkowski = DistanceMatrix(
  Data,
  method = "minkowski",
  p = 3
)

D_cosine = DistanceMatrix(
  Data,
  method = "Cosine_Distance"
)

D_euclidean[1:5, 1:5]

DistanceMatrix() first checks for a native BIDistances route. Other method names are evaluated by parallelDist; methods unavailable there can be obtained from philentropy when it is installed. Mixed numerical and categorical data can be handled through manydist.

Example 2: Weighted Minkowski distances on three backends

For finite p >= 1, Minkowski_Distance() computes

\[ d_{p,w}(x,y) = \left(\sum_k w_k |x_k-y_k|^p\right)^{1/p}. \]

Data = as.matrix(iris[1:30, 1:4])
Weights = c(1, 2, 0.5, 1)

D_parallel = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "parallelDist",
  threads = 2
)

D_multicore = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "multicore",
  threads = 2
)

max(abs(D_parallel - D_multicore))

The optional OpenCL route uses separate kernels for general p and delegates p = 2 to the established weighted Euclidean GPU implementation:

# Requires an installed OpenCL implementation and a usable device.
D_opencl = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "opencl",
  Mem = 2
)

Use backend = "auto" to try OpenCL and fall back to the internal multicore backend with a warning. For p = Inf, unit weights are required and the maximum or Chebyshev distance is returned. Values 0 < p < 1 are available through Fractional_Distance() but are generally not metrics.

The same weighted route is available through DistanceMatrix():

D_dispatch = DistanceMatrix(
  Data,
  method = "Minkowski_Distance",
  p = 3,
  Weights = Weights,
  backend = "multicore",
  threads = 2
)

The memory planner can be inspected before a large OpenCL calculation:

MemoryPlan = calculateMemoryDemandGPU(
  n = 70000,
  d = 784,
  mem = 6
)
MemoryPlan

The Euclidean-related entry points have deliberately different roles:

Example 3: Theory-guided distance selection for clustering

library(BIDistances)

data("Hepta")
set.seed(42)

Selection = DistanceDistributions(
  Data = Hepta$Data,
  DistanceMethods = c(
    "euclidean",
    "manhattan",
    "minkowski",
    "chord"
  ),
  CosineNonParallel = TRUE,
  CorrelationDist = TRUE,
  PlotIt = FALSE
)

Selection$DistanceChoice
Selection$SelectionStatistics[, c(
  "Distance", "DipStatistic", "DipPValue", "BimodalityAmplitude", "Status"
)]
if (!is.null(Selection$DistanceMatrix)) {
  Selection$DistanceMatrix[1:5, 1:5]
}
Selection$ggobject

The workflow ranks candidate distributions using multimodality-related information under the explicit goal of finding distance-based cluster structures. SelectionStatistics reports both the Hartigan dip statistic and its p-value. For at most 72,000 pairwise values the established diptest::dip.test() path is retained; larger candidate columns are processed together by the integrated C++ batch with explicit asymptotic calibration. The result is exploratory: multimodality supports this structural goal, but it is not a universal proof of cluster validity or biological relevance.

Example 4: Gene Ontology-derived TF-IDF distances

library(BIDistances)

data("Hearingloss_N109")

Result = Tfidf_Distance(
  Hearingloss_N109$FeatureMatrix_Gene2Term,
  tf_fun = mean
)

Distance = Result$Distance
TfidfWeights = Result$TfidfWeights

Distance[1:5, 1:5]
TfidfWeights[1:5]

Rows represent genes and columns represent GO terms. The function returns pairwise absolute differences between scalar TF-IDF weights together with those weights. Because different annotation rows can receive the same scalar weight, the construction is a pseudometric on the original rows.

Example 5: Column-wise time-series distances

library(BIDistances)

TimeSeries = cbind(
  SeriesA = c(0, 1, 2, 1, 0),
  SeriesB = c(0, 1, 1.5, 1, 0),
  SeriesC = c(2, 1, 0, 1, 2)
)

D_msmd = DistanceMatrix(
  TimeSeries,
  method = "MSMD_Distance",
  ParameterC = 1
)

D_msmd

For MSMD_Distance and DTW_Distance, every column is one time series and row order is the temporal order. No separate time variable should be supplied. TWED_Distance() requires explicit timestamps and is therefore called directly rather than through DistanceMatrix().

Concepts

Data and Distance conventions

• Data: [1:n,1:d] numerical matrix with n cases, d variables
• Distance: [1:d,1:d] numerical matrix of distances

Most methods compare the rows of Data. The documented exceptions are:

• DTW_Distance and MSMD_Distance, which compare time series stored in columns.
• EndresSchindelin_Distance, which compares columns as samples of variables.
• TWED_Distance, which compares two time series directly and requires explicit time vectors.

Mathematical terminology

The word distance is used as a broad software convention. Not every returned quantity is a mathematical metric. The complete machine-readable classification is available through:

Properties = DistanceProperties()
Properties[, c(
  "Function",
  "Classification",
  "Conditions"
)]

Examples include:

• Ordinary Euclidean distance is a metric.
• Weighted Minkowski distance is a metric for p >= 1 with strictly positive weights and can become a pseudometric when zero weights remove distinguishing coordinates.
• Fractional_Distance() with 0 < p < 1 generally violates the triangle inequality.
• Mahalanobis_Distance() returns squared Mahalanobis dissimilarities; applying sqrt() yields the metric form.
• Cosine_Distance() returns 1 - cosine similarity and is not a metric in general.
• Tfidf_Distance() is a pseudometric on the original gene-annotation rows.
• Kullback_Leibler_div() returns a divergence, including when symmetrized.

See ?distance-conventions and ?DistanceProperties for details. The registry documents mathematical conditions; it is not a runtime proof for arbitrary user-supplied data.

Native DistanceMatrix() routes

Canonical package methods available through DistanceMatrix() are:

• Cosine_Distance
• EndresSchindelin_Distance
• Fractional_Distance
• Gini_Distance
• Jaccard_Distance
• Mahalanobis_Distance
• Minkowski_Distance
• SharedNeighbor_Distance
• Tfidf_Distance
• Wasserstein_Distance
• DTW_Distance
• MSMD_Distance

TWED_Distance() is intentionally excluded from the dispatcher because its explicit time vectors cannot be inferred from the standard matrix interface.

Optional functionality

Some functions require suggested packages only when they are used:

• OpenCL for the GPU backend.
• philentropy for additional probability-distribution methods.
• manydist for mixed numerical and categorical data.
• dtw and proxy for DTW_Distance().
• ineq for Gini_Distance().
• transport for Wasserstein_Distance().
• ScatterDensity for sample-based Endres-Schindelin calculations.

Manual and citation

The full manual is available at CRAN.

The package vignette can be opened from R with:

vignette("BIDistances")

Citation information is available with:

citation("BIDistances")

References

[Thrun, 2021] Thrun, M. C.: The Exploitation of Distance Distributions for Clustering, International Journal of Computational Intelligence and Applications, Vol. 20(3), 2150016, doi:10.1142/S1469026821500164, 2021.

[Stier & Thrun, 2023] Stier, Q., & Thrun, M. C.: Deriving Homogeneous Subsets from Gene Sets by Exploiting the Gene Ontology, Informatica, Vol. 34(2), pp. 357-386, doi:10.15388/23-INFOR517, 2023.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.