The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Get started with nestedscales

nestedscales provides a multi-level discrete representation of a data dimension and the operations that move data between its levels. This vignette introduces the representation and the main functions on a small industry classification. The dimension packages timescales and geoscales apply the same functions to calendars and region hierarchies. The package website includes a roadmap of the package’s scope.

library(nestedscales)

Representation

A nested scale is a table with one row per atom, the finest unit of the dimension, and one column per frame, a coarser level that groups the atoms. The frames are given coarsest first. Numeric columns that are not frames are weights.

industries <- data.frame(
  section  = c("C", "C", "C", "D", "D"),
  division = c("C10", "C10", "C11", "D35", "D35"),
  class    = c("C101", "C102", "C110", "D351", "D352"),
  gva      = c(120, 80, 45, 300, 150)
)

ind <- scale_from_leaftable(
  industries,
  frames = c("section", "division", "class"),
  name   = "nace"
)
ind
#> NestedScale: nace
#> Frames (3, coarsest first):
#>   - section (2)
#>     - division (3)
#>       - class (5)
#> Atoms: 5
#> Weights: gva (default: gva)

The scale records the frames, the units at each frame, the atom key and the weights:

scale_frames(ind)
#> [1] "section"  "division" "class"
scale_units(ind, "division")
#> [1] "C10" "C11" "D35"
scale_key(ind)
#> [1] "class"
scale_weights(ind)
#> [1] "gva"

Every frame partitions the atoms. Frames need not nest in each other: a unit at one frame may belong to several units at a coarser frame, and two frames may be independent, in which case the atoms are their combinations. scale_nests(), scale_crosses() and scale_is_uniform() report the relation a given pair of frames has; no relation is assumed.

scale_nests(ind, "section", "division")
#> [1] TRUE
scale_family(ind, "section", "division")
#>   parent_frame parent child_frame child
#> 1      section      C    division   C10
#> 2      section      C    division   C11
#> 3      section      D    division   D35

Transition rules

Data keyed at one frame is converted to another frame with recast_scale(). Each value column follows a rule from SCALE_RULES:

rule aggregation (fine to coarse) disaggregation (coarse to fine)
sum sum split in proportion to the weight
weighted_mean weighted mean copy
mean mean over atoms copy
copy the common value copy
sd standard deviation over atoms not defined
share value as a share of its parent’s total not defined

The rule is given per value column, so extensive and intensive quantities convert in one call:

x <- data.frame(
  class = c("C101", "C102", "C110", "D351", "D352"),
  jobs  = c(12, 8, 5, 30, 20),
  wage  = c(31, 29, 27, 45, 40)
)

recast_scale(x, ind,
  source = "class", target = "section",
  rule = c(jobs = "sum", wage = "weighted_mean")
)
#>   section jobs     wage
#> 1       C   25 29.61224
#> 2       D   50 43.33333

The same function converts in the other direction. A total at the section level is split over the classes in proportion to the weight:

y <- data.frame(section = c("C", "D"), investment = c(100, 400))

recast_scale(y, ind, source = "section", target = "class", rule = "sum")
#>   class investment
#> 1  C101   48.97959
#> 2  C102   32.65306
#> 3  C110   18.36735
#> 4  D351  266.66667
#> 5  D352  133.33333

Columns that are neither the key nor a value column are identifiers and are preserved as groups, so a panel of several years or scenarios converts in one call. A rule may also be registered once for a named value column with register_scale_rule(), after which rule = can be omitted for that column.

Labels

join_scale() attaches the scale’s labels to a dataset: the unit at the keyed frame, optionally the coarser frames (attach =) and the unit’s share and weight (meta =):

join_scale(x, ind, frame = "class", attach = TRUE)
#>   class jobs wage nace.section nace.division nace
#> 1  C101   12   31            C           C10 C101
#> 2  C102    8   29            C           C10 C102
#> 3  C110    5   27            C           C11 C110
#> 4  D351   30   45            D           D35 D351
#> 5  D352   20   40            D           D35 D352

Subsets

filter_scale() keeps the atoms under given units; the result records what fraction of each weight it retains. prune_scale() collapses a scale to a coarser atom level.

manufacturing <- filter_scale(ind, "section", "C")
scale_units(manufacturing)
#> [1] "C101" "C102" "C110"
scale_coverage(manufacturing)
#>      gva 
#> 0.352518

scale_frames(prune_scale(ind, "division"))
#> [1] "section"  "division"

Attributes and aliases

A scale may carry values that belong to its members but are not weights: a calorific value per fuel, a density, an emission factor. They live in an attribute table keyed by (frame, unit), at whatever level they are published, and scale_attributes() resolves one value per member of a frame, walking to the nearest coarser level when a member has none of its own. convert_scale() applies the resolved value row by row. Alternative labels live in an alias table, and scale_match() resolves labels against member names and aliases exactly, reporting ambiguity rather than guessing. The package ships one such scale, the energypal fuel taxonomy with the IPCC 2006 defaults:

carriers <- read_scale("carriers")
lhv <- scale_attributes(carriers, "lhv", frame = "fuel")
head(lhv[!is.na(lhv$value), c("unit", "value", "unit_of_measure")])
#>            unit value unit_of_measure
#> 1    Anthracite  26.7            GJ/t
#> 2    Bituminous  25.8            GJ/t
#> 3 SubBituminous  18.9            GJ/t
#> 4       Lignite  11.9            GJ/t
#> 5          Coke  28.2            GJ/t
#> 7         Crude  42.3            GJ/t

fuel_use <- data.frame(fuel = c("Lignite", "NaturalGas"), t = c(1000, 1000))
convert_scale(fuel_use, carriers, "lhv", from_unit = "t", to_unit = "GJ")
#>         fuel     t  lhv lhv.origin_frame
#> 1    Lignite 11900 11.9             fuel
#> 2 NaturalGas 48000 48.0             fuel

scale_match(carriers, c("GAS", "lignite", "Hydrogen"))
#>      label   frame      unit    method                      candidates
#> 1      GAS carrier FossilGas     alias                            <NA>
#> 2  lignite    fuel   Lignite     exact                            <NA>
#> 3 Hydrogen    <NA>      <NA> ambiguous carrier/Hydrogen; fuel/Hydrogen

vignette("attributes") has the full account.

Definition files

A scale is stored as a definition: a JSON header with its tables inline, or a bundle directory with Arrow or CSV tables, which the Python binding reads as well. write_scale() and read_scale() exchange them, and scale_catalog() lists the definitions installed packages ship:

path <- file.path(tempdir(), "nace.json")
write_scale(ind, path)
scale_frames(read_scale(path))
#> [1] "section"  "division" "class"
scale_catalog()[c("id", "name", "frames", "n_atoms", "package")]
#>         id     name             frames n_atoms      package
#> 1 carriers carriers group/carrier/fuel      71 nestedscales

Several dimensions

scale_product() combines scales into one multi-dimensional index without materialising the product of their atoms. recast_product(), join_product() and filter_product() apply the single-scale operations axis by axis.

p <- scale_product(a = scale_example(), b = scale_example2())
product_size(p)
#> $axes
#> a b 
#> 7 4 
#> 
#> $total
#> [1] 28

Further operations

All operations accept a data.frame, tibble, data.table, dtplyr table or arrow dataset and return the same class; lazy inputs return a query unless collect = TRUE.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.