Introduction to Cache

Eliot J. B. McIntire

November 15 2018

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations; however, it also uses fastdigest::fastdigest to make it substantially faster in many cases. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmppSwtOv/reproducible_examples/Cache"
ras <- raster(extent(0,1000,0,1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   1.955   0.142   2.101
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.501   0.409   3.060
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.099   0.012   0.111
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.021   0.000   0.021
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
# magrittr, if loaded, gives an error below
  try(detach("package:magrittr", unload = TRUE), silent = TRUE)

try(clearCache(tmpDir), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- rnorm(10, 16) %>% Cache(cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- rnorm(10, 6) %>% Cache(cacheRepo = tmpDir) # different

1.4 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 0a3e317ec67f1b4f16ad47c164cb772e         format
##  2: 0a3e317ec67f1b4f16ad47c164cb772e           name
##  3: 0a3e317ec67f1b4f16ad47c164cb772e          class
##  4: 0a3e317ec67f1b4f16ad47c164cb772e           date
##  5: 0a3e317ec67f1b4f16ad47c164cb772e        cacheId
##  6: 0a3e317ec67f1b4f16ad47c164cb772e     objectName
##  7: 0a3e317ec67f1b4f16ad47c164cb772e       function
##  8: 0a3e317ec67f1b4f16ad47c164cb772e    object.size
##  9: 0a3e317ec67f1b4f16ad47c164cb772e       accessed
## 10: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 11: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 12: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 13: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 14: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 15: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 16: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 17: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 18: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 19: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 20: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 21: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
## 22: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
## 23: 93a32cb588ccd5e88316f782a05ca5f5         format
## 24: 93a32cb588ccd5e88316f782a05ca5f5           name
## 25: 93a32cb588ccd5e88316f782a05ca5f5          class
## 26: 93a32cb588ccd5e88316f782a05ca5f5           date
## 27: 93a32cb588ccd5e88316f782a05ca5f5        cacheId
## 28: 93a32cb588ccd5e88316f782a05ca5f5     objectName
## 29: 93a32cb588ccd5e88316f782a05ca5f5       function
## 30: 93a32cb588ccd5e88316f782a05ca5f5    object.size
## 31: 93a32cb588ccd5e88316f782a05ca5f5       accessed
## 32: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 33: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 34: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 35: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 36: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 37: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 38: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 39: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 40: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 41: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 42: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 43: 93a32cb588ccd5e88316f782a05ca5f5      preDigest
## 44: 93a32cb588ccd5e88316f782a05ca5f5      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:25
##  2:      0a3e317ec67f1b4f16ad47c164cb772e 2018-11-15 13:47:25
##  3:                               numeric 2018-11-15 13:47:25
##  4:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:25
##  6:                                     a 2018-11-15 13:47:25
##  7:                                 rnorm 2018-11-15 13:47:25
##  8:                                  1024 2018-11-15 13:47:25
##  9:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 10:                      vweave_rmarkdown 2018-11-15 13:47:25
## 11:                          process_file 2018-11-15 13:47:25
## 12:                   withCallingHandlers 2018-11-15 13:47:25
## 13:                         process_group 2018-11-15 13:47:25
## 14:                   process_group.block 2018-11-15 13:47:25
## 15:                            call_block 2018-11-15 13:47:25
## 16:                            block_exec 2018-11-15 13:47:25
## 17:                                in_dir 2018-11-15 13:47:25
## 18:                             timing_fn 2018-11-15 13:47:25
## 19:                                handle 2018-11-15 13:47:25
## 20:                           withVisible 2018-11-15 13:47:25
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
## 23:                                   rda 2018-11-15 13:47:25
## 24:      93a32cb588ccd5e88316f782a05ca5f5 2018-11-15 13:47:25
## 25:                               numeric 2018-11-15 13:47:25
## 26:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 27:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:25
## 28:                                     b 2018-11-15 13:47:25
## 29:                                 runif 2018-11-15 13:47:25
## 30:                                  1024 2018-11-15 13:47:25
## 31:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 32:                      vweave_rmarkdown 2018-11-15 13:47:25
## 33:                          process_file 2018-11-15 13:47:25
## 34:                   withCallingHandlers 2018-11-15 13:47:25
## 35:                         process_group 2018-11-15 13:47:25
## 36:                   process_group.block 2018-11-15 13:47:25
## 37:                            call_block 2018-11-15 13:47:25
## 38:                            block_exec 2018-11-15 13:47:25
## 39:                                in_dir 2018-11-15 13:47:25
## 40:                             timing_fn 2018-11-15 13:47:25
## 41:                                handle 2018-11-15 13:47:25
## 42:                           withVisible 2018-11-15 13:47:25
## 43:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 44: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:25
##                                  tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 256 bytes
##                             artifact         tagKey
##  1: 0a3e317ec67f1b4f16ad47c164cb772e         format
##  2: 0a3e317ec67f1b4f16ad47c164cb772e           name
##  3: 0a3e317ec67f1b4f16ad47c164cb772e          class
##  4: 0a3e317ec67f1b4f16ad47c164cb772e           date
##  5: 0a3e317ec67f1b4f16ad47c164cb772e        cacheId
##  6: 0a3e317ec67f1b4f16ad47c164cb772e     objectName
##  7: 0a3e317ec67f1b4f16ad47c164cb772e       function
##  8: 0a3e317ec67f1b4f16ad47c164cb772e    object.size
##  9: 0a3e317ec67f1b4f16ad47c164cb772e       accessed
## 10: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 11: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 12: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 13: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 14: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 15: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 16: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 17: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 18: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 19: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 20: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 21: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
## 22: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:25
##  2:      0a3e317ec67f1b4f16ad47c164cb772e 2018-11-15 13:47:25
##  3:                               numeric 2018-11-15 13:47:25
##  4:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:25
##  6:                                     a 2018-11-15 13:47:25
##  7:                                 rnorm 2018-11-15 13:47:25
##  8:                                  1024 2018-11-15 13:47:25
##  9:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 10:                      vweave_rmarkdown 2018-11-15 13:47:25
## 11:                          process_file 2018-11-15 13:47:25
## 12:                   withCallingHandlers 2018-11-15 13:47:25
## 13:                         process_group 2018-11-15 13:47:25
## 14:                   process_group.block 2018-11-15 13:47:25
## 15:                            call_block 2018-11-15 13:47:25
## 16:                            block_exec 2018-11-15 13:47:25
## 17:                                in_dir 2018-11-15 13:47:25
## 18:                             timing_fn 2018-11-15 13:47:25
## 19:                                handle 2018-11-15 13:47:25
## 20:                           withVisible 2018-11-15 13:47:25
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
##                                  tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 256 bytes
##                             artifact         tagKey
##  1: 93a32cb588ccd5e88316f782a05ca5f5         format
##  2: 93a32cb588ccd5e88316f782a05ca5f5           name
##  3: 93a32cb588ccd5e88316f782a05ca5f5          class
##  4: 93a32cb588ccd5e88316f782a05ca5f5           date
##  5: 93a32cb588ccd5e88316f782a05ca5f5        cacheId
##  6: 93a32cb588ccd5e88316f782a05ca5f5     objectName
##  7: 93a32cb588ccd5e88316f782a05ca5f5       function
##  8: 93a32cb588ccd5e88316f782a05ca5f5    object.size
##  9: 93a32cb588ccd5e88316f782a05ca5f5       accessed
## 10: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 11: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 12: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 13: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 14: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 15: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 16: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 17: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 18: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 19: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 20: 93a32cb588ccd5e88316f782a05ca5f5 otherFunctions
## 21: 93a32cb588ccd5e88316f782a05ca5f5      preDigest
## 22: 93a32cb588ccd5e88316f782a05ca5f5      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:25
##  2:      93a32cb588ccd5e88316f782a05ca5f5 2018-11-15 13:47:25
##  3:                               numeric 2018-11-15 13:47:25
##  4:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##  5:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:25
##  6:                                     b 2018-11-15 13:47:25
##  7:                                 runif 2018-11-15 13:47:25
##  8:                                  1024 2018-11-15 13:47:25
##  9:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 10:                      vweave_rmarkdown 2018-11-15 13:47:25
## 11:                          process_file 2018-11-15 13:47:25
## 12:                   withCallingHandlers 2018-11-15 13:47:25
## 13:                         process_group 2018-11-15 13:47:25
## 14:                   process_group.block 2018-11-15 13:47:25
## 15:                            call_block 2018-11-15 13:47:25
## 16:                            block_exec 2018-11-15 13:47:25
## 17:                                in_dir 2018-11-15 13:47:25
## 18:                             timing_fn 2018-11-15 13:47:25
## 19:                                handle 2018-11-15 13:47:25
## 20:                           withVisible 2018-11-15 13:47:25
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 22: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:25
##                                  tagValue         createdDate
clearCache(tmpDir, userTags = c("runif")) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 256 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 816 bytes
##   Selected objects (not including Rasters): 816 bytes
##                             artifact         tagKey
##  1: 0a3e317ec67f1b4f16ad47c164cb772e         format
##  2: 0a3e317ec67f1b4f16ad47c164cb772e           name
##  3: 0a3e317ec67f1b4f16ad47c164cb772e          class
##  4: 0a3e317ec67f1b4f16ad47c164cb772e           date
##  5: 0a3e317ec67f1b4f16ad47c164cb772e        cacheId
##  6: 0a3e317ec67f1b4f16ad47c164cb772e     objectName
##  7: 0a3e317ec67f1b4f16ad47c164cb772e       function
##  8: 0a3e317ec67f1b4f16ad47c164cb772e    object.size
##  9: 0a3e317ec67f1b4f16ad47c164cb772e       accessed
## 10: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 11: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 12: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 13: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 14: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 15: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 16: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 17: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 18: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 19: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 20: 0a3e317ec67f1b4f16ad47c164cb772e otherFunctions
## 21: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
## 22: 0a3e317ec67f1b4f16ad47c164cb772e      preDigest
## 23: cf29dcc05a0db510200821c151a9288e         format
## 24: cf29dcc05a0db510200821c151a9288e           name
## 25: cf29dcc05a0db510200821c151a9288e          class
## 26: cf29dcc05a0db510200821c151a9288e           date
## 27: cf29dcc05a0db510200821c151a9288e        cacheId
## 28: cf29dcc05a0db510200821c151a9288e       function
## 29: cf29dcc05a0db510200821c151a9288e    object.size
## 30: cf29dcc05a0db510200821c151a9288e       accessed
## 31: cf29dcc05a0db510200821c151a9288e otherFunctions
## 32: cf29dcc05a0db510200821c151a9288e otherFunctions
## 33: cf29dcc05a0db510200821c151a9288e otherFunctions
## 34: cf29dcc05a0db510200821c151a9288e otherFunctions
## 35: cf29dcc05a0db510200821c151a9288e otherFunctions
## 36: cf29dcc05a0db510200821c151a9288e otherFunctions
## 37: cf29dcc05a0db510200821c151a9288e otherFunctions
## 38: cf29dcc05a0db510200821c151a9288e otherFunctions
## 39: cf29dcc05a0db510200821c151a9288e otherFunctions
## 40: cf29dcc05a0db510200821c151a9288e otherFunctions
## 41: cf29dcc05a0db510200821c151a9288e otherFunctions
## 42: cf29dcc05a0db510200821c151a9288e otherFunctions
## 43: cf29dcc05a0db510200821c151a9288e otherFunctions
## 44: cf29dcc05a0db510200821c151a9288e      preDigest
## 45: cf29dcc05a0db510200821c151a9288e      preDigest
## 46: cf29dcc05a0db510200821c151a9288e      preDigest
## 47: dd9e815f76c24f624c9b7a31f389e0c1         format
## 48: dd9e815f76c24f624c9b7a31f389e0c1           name
## 49: dd9e815f76c24f624c9b7a31f389e0c1          class
## 50: dd9e815f76c24f624c9b7a31f389e0c1           date
## 51: dd9e815f76c24f624c9b7a31f389e0c1        cacheId
## 52: dd9e815f76c24f624c9b7a31f389e0c1       function
## 53: dd9e815f76c24f624c9b7a31f389e0c1    object.size
## 54: dd9e815f76c24f624c9b7a31f389e0c1       accessed
## 55: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 56: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 57: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 58: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 59: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 60: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 61: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 62: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 63: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 64: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 65: dd9e815f76c24f624c9b7a31f389e0c1 otherFunctions
## 66: dd9e815f76c24f624c9b7a31f389e0c1      preDigest
## 67: dd9e815f76c24f624c9b7a31f389e0c1      preDigest
## 68: dd9e815f76c24f624c9b7a31f389e0c1      preDigest
## 69: dd9e815f76c24f624c9b7a31f389e0c1       accessed
## 70: dd9e815f76c24f624c9b7a31f389e0c1       accessed
## 71: dd9e815f76c24f624c9b7a31f389e0c1       accessed
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:25
##  2:      0a3e317ec67f1b4f16ad47c164cb772e 2018-11-15 13:47:25
##  3:                               numeric 2018-11-15 13:47:25
##  4:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:25
##  6:                                     a 2018-11-15 13:47:25
##  7:                                 rnorm 2018-11-15 13:47:25
##  8:                                  1024 2018-11-15 13:47:25
##  9:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 10:                      vweave_rmarkdown 2018-11-15 13:47:25
## 11:                          process_file 2018-11-15 13:47:25
## 12:                   withCallingHandlers 2018-11-15 13:47:25
## 13:                         process_group 2018-11-15 13:47:25
## 14:                   process_group.block 2018-11-15 13:47:25
## 15:                            call_block 2018-11-15 13:47:25
## 16:                            block_exec 2018-11-15 13:47:25
## 17:                                in_dir 2018-11-15 13:47:25
## 18:                             timing_fn 2018-11-15 13:47:25
## 19:                                handle 2018-11-15 13:47:25
## 20:                           withVisible 2018-11-15 13:47:25
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
## 23:                                   rda 2018-11-15 13:47:25
## 24:      cf29dcc05a0db510200821c151a9288e 2018-11-15 13:47:25
## 25:                               numeric 2018-11-15 13:47:25
## 26:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 27:      93a7b68b228b551669eb36211e857b39 2018-11-15 13:47:25
## 28:                   rnorm pipe sequence 2018-11-15 13:47:25
## 29:                                  1120 2018-11-15 13:47:25
## 30:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 31:                      vweave_rmarkdown 2018-11-15 13:47:25
## 32:                          process_file 2018-11-15 13:47:25
## 33:                   withCallingHandlers 2018-11-15 13:47:25
## 34:                         process_group 2018-11-15 13:47:25
## 35:                   process_group.block 2018-11-15 13:47:25
## 36:                            call_block 2018-11-15 13:47:25
## 37:                            block_exec 2018-11-15 13:47:25
## 38:                                in_dir 2018-11-15 13:47:25
## 39:                             timing_fn 2018-11-15 13:47:25
## 40:                                handle 2018-11-15 13:47:25
## 41:                           withVisible 2018-11-15 13:47:25
## 42:                                   %>% 2018-11-15 13:47:25
## 43:                               do.call 2018-11-15 13:47:25
## 44:    n:c192d58bc0a9ddf2b5f916d929bccc08 2018-11-15 13:47:25
## 45: mean:55835193261a0d71e70d137dfa31cd5b 2018-11-15 13:47:25
## 46: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
## 47:                                   rda 2018-11-15 13:47:25
## 48:      dd9e815f76c24f624c9b7a31f389e0c1 2018-11-15 13:47:25
## 49:                               numeric 2018-11-15 13:47:25
## 50:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 51:      c878d1fb026c0854fc0480553fa4cf24 2018-11-15 13:47:25
## 52:                                 rnorm 2018-11-15 13:47:25
## 53:                                  1120 2018-11-15 13:47:25
## 54:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 55:                      vweave_rmarkdown 2018-11-15 13:47:25
## 56:                          process_file 2018-11-15 13:47:25
## 57:                   withCallingHandlers 2018-11-15 13:47:25
## 58:                         process_group 2018-11-15 13:47:25
## 59:                   process_group.block 2018-11-15 13:47:25
## 60:                            call_block 2018-11-15 13:47:25
## 61:                            block_exec 2018-11-15 13:47:25
## 62:                                in_dir 2018-11-15 13:47:25
## 63:                             timing_fn 2018-11-15 13:47:25
## 64:                                handle 2018-11-15 13:47:25
## 65:                           withVisible 2018-11-15 13:47:25
## 66:    n:c192d58bc0a9ddf2b5f916d929bccc08 2018-11-15 13:47:25
## 67: mean:eab1c55b26658701ccc6e12b858d29e4 2018-11-15 13:47:25
## 68: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
## 69:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 70:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 71:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##                                  tagValue         createdDate
clearCache(tmpDir)

1.5 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 256 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 256 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 256 bytes
##   Selected objects (not including Rasters): 256 bytes
##                             artifact         tagKey
##  1: 3b0a4735072f1e14a4109993dd147be4         format
##  2: 3b0a4735072f1e14a4109993dd147be4           name
##  3: 3b0a4735072f1e14a4109993dd147be4          class
##  4: 3b0a4735072f1e14a4109993dd147be4           date
##  5: 3b0a4735072f1e14a4109993dd147be4        cacheId
##  6: 3b0a4735072f1e14a4109993dd147be4     objectName
##  7: 3b0a4735072f1e14a4109993dd147be4       function
##  8: 3b0a4735072f1e14a4109993dd147be4    object.size
##  9: 3b0a4735072f1e14a4109993dd147be4       accessed
## 10: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 11: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 12: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 13: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 14: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 15: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 16: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 17: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 18: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 19: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 20: 3b0a4735072f1e14a4109993dd147be4 otherFunctions
## 21: 3b0a4735072f1e14a4109993dd147be4      preDigest
## 22: 3b0a4735072f1e14a4109993dd147be4      preDigest
## 23: 3b0a4735072f1e14a4109993dd147be4       accessed
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:25
##  2:      3b0a4735072f1e14a4109993dd147be4 2018-11-15 13:47:25
##  3:                               numeric 2018-11-15 13:47:25
##  4:                   2018-11-15 13:47:25 2018-11-15 13:47:25
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:25
##  6:                                     a 2018-11-15 13:47:25
##  7:                                 rnorm 2018-11-15 13:47:25
##  8:                                  1024 2018-11-15 13:47:25
##  9:                   2018-11-15 13:47:25 2018-11-15 13:47:25
## 10:                      vweave_rmarkdown 2018-11-15 13:47:25
## 11:                          process_file 2018-11-15 13:47:25
## 12:                   withCallingHandlers 2018-11-15 13:47:25
## 13:                         process_group 2018-11-15 13:47:25
## 14:                   process_group.block 2018-11-15 13:47:25
## 15:                            call_block 2018-11-15 13:47:25
## 16:                            block_exec 2018-11-15 13:47:25
## 17:                                in_dir 2018-11-15 13:47:25
## 18:                             timing_fn 2018-11-15 13:47:25
## 19:                                handle 2018-11-15 13:47:25
## 20:                           withVisible 2018-11-15 13:47:25
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:25
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:25
## 23:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##                                  tagValue         createdDate
clearCache(tmpDir)

1.6 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay)
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 30e8c8926fa97a9a07f8069ee68654c7         format
##  2: 30e8c8926fa97a9a07f8069ee68654c7           name
##  3: 30e8c8926fa97a9a07f8069ee68654c7          class
##  4: 30e8c8926fa97a9a07f8069ee68654c7           date
##  5: 30e8c8926fa97a9a07f8069ee68654c7        cacheId
##  6: 30e8c8926fa97a9a07f8069ee68654c7     objectName
##  7: 30e8c8926fa97a9a07f8069ee68654c7       function
##  8: 30e8c8926fa97a9a07f8069ee68654c7    object.size
##  9: 30e8c8926fa97a9a07f8069ee68654c7       accessed
## 10: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 11: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 12: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 13: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 14: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 15: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 16: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 17: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 18: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 19: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 20: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 21: 30e8c8926fa97a9a07f8069ee68654c7      preDigest
## 22: 30e8c8926fa97a9a07f8069ee68654c7      preDigest
## 23: 7734759c8621a7598b61dc93471a4a72         format
## 24: 7734759c8621a7598b61dc93471a4a72           name
## 25: 7734759c8621a7598b61dc93471a4a72          class
## 26: 7734759c8621a7598b61dc93471a4a72           date
## 27: 7734759c8621a7598b61dc93471a4a72        cacheId
## 28: 7734759c8621a7598b61dc93471a4a72     objectName
## 29: 7734759c8621a7598b61dc93471a4a72       function
## 30: 7734759c8621a7598b61dc93471a4a72    object.size
## 31: 7734759c8621a7598b61dc93471a4a72       accessed
## 32: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 33: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 34: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 35: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 36: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 37: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 38: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 39: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 40: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 41: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 42: 7734759c8621a7598b61dc93471a4a72 otherFunctions
## 43: 7734759c8621a7598b61dc93471a4a72      preDigest
## 44: 7734759c8621a7598b61dc93471a4a72      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:26
##  2:      30e8c8926fa97a9a07f8069ee68654c7 2018-11-15 13:47:26
##  3:                               numeric 2018-11-15 13:47:26
##  4:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:26
##  6:                                     a 2018-11-15 13:47:26
##  7:                                 rnorm 2018-11-15 13:47:26
##  8:                                  1024 2018-11-15 13:47:26
##  9:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 10:                      vweave_rmarkdown 2018-11-15 13:47:26
## 11:                          process_file 2018-11-15 13:47:26
## 12:                   withCallingHandlers 2018-11-15 13:47:26
## 13:                         process_group 2018-11-15 13:47:26
## 14:                   process_group.block 2018-11-15 13:47:26
## 15:                            call_block 2018-11-15 13:47:26
## 16:                            block_exec 2018-11-15 13:47:26
## 17:                                in_dir 2018-11-15 13:47:26
## 18:                             timing_fn 2018-11-15 13:47:26
## 19:                                handle 2018-11-15 13:47:26
## 20:                           withVisible 2018-11-15 13:47:26
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:26
## 23:                                   rda 2018-11-15 13:47:26
## 24:      7734759c8621a7598b61dc93471a4a72 2018-11-15 13:47:26
## 25:                               numeric 2018-11-15 13:47:26
## 26:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 27:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:26
## 28:                                     b 2018-11-15 13:47:26
## 29:                                 runif 2018-11-15 13:47:26
## 30:                                  1024 2018-11-15 13:47:26
## 31:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 32:                      vweave_rmarkdown 2018-11-15 13:47:26
## 33:                          process_file 2018-11-15 13:47:26
## 34:                   withCallingHandlers 2018-11-15 13:47:26
## 35:                         process_group 2018-11-15 13:47:26
## 36:                   process_group.block 2018-11-15 13:47:26
## 37:                            call_block 2018-11-15 13:47:26
## 38:                            block_exec 2018-11-15 13:47:26
## 39:                                in_dir 2018-11-15 13:47:26
## 40:                             timing_fn 2018-11-15 13:47:26
## 41:                                handle 2018-11-15 13:47:26
## 42:                           withVisible 2018-11-15 13:47:26
## 43:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 44: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:26
##                                  tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 256 bytes
##                             artifact         tagKey
##  1: 30e8c8926fa97a9a07f8069ee68654c7         format
##  2: 30e8c8926fa97a9a07f8069ee68654c7           name
##  3: 30e8c8926fa97a9a07f8069ee68654c7          class
##  4: 30e8c8926fa97a9a07f8069ee68654c7           date
##  5: 30e8c8926fa97a9a07f8069ee68654c7        cacheId
##  6: 30e8c8926fa97a9a07f8069ee68654c7     objectName
##  7: 30e8c8926fa97a9a07f8069ee68654c7       function
##  8: 30e8c8926fa97a9a07f8069ee68654c7    object.size
##  9: 30e8c8926fa97a9a07f8069ee68654c7       accessed
## 10: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 11: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 12: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 13: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 14: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 15: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 16: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 17: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 18: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 19: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 20: 30e8c8926fa97a9a07f8069ee68654c7 otherFunctions
## 21: 30e8c8926fa97a9a07f8069ee68654c7      preDigest
## 22: 30e8c8926fa97a9a07f8069ee68654c7      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:26
##  2:      30e8c8926fa97a9a07f8069ee68654c7 2018-11-15 13:47:26
##  3:                               numeric 2018-11-15 13:47:26
##  4:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:26
##  6:                                     a 2018-11-15 13:47:26
##  7:                                 rnorm 2018-11-15 13:47:26
##  8:                                  1024 2018-11-15 13:47:26
##  9:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 10:                      vweave_rmarkdown 2018-11-15 13:47:26
## 11:                          process_file 2018-11-15 13:47:26
## 12:                   withCallingHandlers 2018-11-15 13:47:26
## 13:                         process_group 2018-11-15 13:47:26
## 14:                   process_group.block 2018-11-15 13:47:26
## 15:                            call_block 2018-11-15 13:47:26
## 16:                            block_exec 2018-11-15 13:47:26
## 17:                                in_dir 2018-11-15 13:47:26
## 18:                             timing_fn 2018-11-15 13:47:26
## 19:                                handle 2018-11-15 13:47:26
## 20:                           withVisible 2018-11-15 13:47:26
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:26
##                                  tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 256 bytes
##   Selected objects (not including Rasters): 256 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

1.7 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 3837336a2dc856472ded40b64e3b21c7         format
##  2: 3837336a2dc856472ded40b64e3b21c7           name
##  3: 3837336a2dc856472ded40b64e3b21c7          class
##  4: 3837336a2dc856472ded40b64e3b21c7           date
##  5: 3837336a2dc856472ded40b64e3b21c7        cacheId
##  6: 3837336a2dc856472ded40b64e3b21c7     objectName
##  7: 3837336a2dc856472ded40b64e3b21c7       function
##  8: 3837336a2dc856472ded40b64e3b21c7    object.size
##  9: 3837336a2dc856472ded40b64e3b21c7       accessed
## 10: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 11: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 12: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 13: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 14: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 15: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 16: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 17: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 18: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 19: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 20: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 21: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 22: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 23: aee722adef0ee2ac2649facff8fed28a         format
## 24: aee722adef0ee2ac2649facff8fed28a           name
## 25: aee722adef0ee2ac2649facff8fed28a          class
## 26: aee722adef0ee2ac2649facff8fed28a           date
## 27: aee722adef0ee2ac2649facff8fed28a        cacheId
## 28: aee722adef0ee2ac2649facff8fed28a     objectName
## 29: aee722adef0ee2ac2649facff8fed28a       function
## 30: aee722adef0ee2ac2649facff8fed28a    object.size
## 31: aee722adef0ee2ac2649facff8fed28a       accessed
## 32: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 33: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 34: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 35: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 36: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 37: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 38: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 39: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 40: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 41: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 42: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 43: aee722adef0ee2ac2649facff8fed28a      preDigest
## 44: aee722adef0ee2ac2649facff8fed28a      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:26
##  2:      3837336a2dc856472ded40b64e3b21c7 2018-11-15 13:47:26
##  3:                               numeric 2018-11-15 13:47:26
##  4:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:26
##  6:                                     b 2018-11-15 13:47:26
##  7:                                 rnorm 2018-11-15 13:47:26
##  8:                                  1024 2018-11-15 13:47:26
##  9:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 10:                      vweave_rmarkdown 2018-11-15 13:47:26
## 11:                          process_file 2018-11-15 13:47:26
## 12:                   withCallingHandlers 2018-11-15 13:47:26
## 13:                         process_group 2018-11-15 13:47:26
## 14:                   process_group.block 2018-11-15 13:47:26
## 15:                            call_block 2018-11-15 13:47:26
## 16:                            block_exec 2018-11-15 13:47:26
## 17:                                in_dir 2018-11-15 13:47:26
## 18:                             timing_fn 2018-11-15 13:47:26
## 19:                                handle 2018-11-15 13:47:26
## 20:                           withVisible 2018-11-15 13:47:26
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:26
## 23:                                   rda 2018-11-15 13:47:26
## 24:      aee722adef0ee2ac2649facff8fed28a 2018-11-15 13:47:26
## 25:                               numeric 2018-11-15 13:47:26
## 26:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 27:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:26
## 28:                                     a 2018-11-15 13:47:26
## 29:                                 runif 2018-11-15 13:47:26
## 30:                                  1024 2018-11-15 13:47:26
## 31:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 32:                      vweave_rmarkdown 2018-11-15 13:47:26
## 33:                          process_file 2018-11-15 13:47:26
## 34:                   withCallingHandlers 2018-11-15 13:47:26
## 35:                         process_group 2018-11-15 13:47:26
## 36:                   process_group.block 2018-11-15 13:47:26
## 37:                            call_block 2018-11-15 13:47:26
## 38:                            block_exec 2018-11-15 13:47:26
## 39:                                in_dir 2018-11-15 13:47:26
## 40:                             timing_fn 2018-11-15 13:47:26
## 41:                                handle 2018-11-15 13:47:26
## 42:                           withVisible 2018-11-15 13:47:26
## 43:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 44: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:26
##                                  tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 3837336a2dc856472ded40b64e3b21c7         format
##  2: 3837336a2dc856472ded40b64e3b21c7           name
##  3: 3837336a2dc856472ded40b64e3b21c7          class
##  4: 3837336a2dc856472ded40b64e3b21c7           date
##  5: 3837336a2dc856472ded40b64e3b21c7        cacheId
##  6: 3837336a2dc856472ded40b64e3b21c7     objectName
##  7: 3837336a2dc856472ded40b64e3b21c7       function
##  8: 3837336a2dc856472ded40b64e3b21c7    object.size
##  9: 3837336a2dc856472ded40b64e3b21c7       accessed
## 10: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 11: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 12: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 13: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 14: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 15: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 16: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 17: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 18: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 19: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 20: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 21: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 22: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 23: aee722adef0ee2ac2649facff8fed28a         format
## 24: aee722adef0ee2ac2649facff8fed28a           name
## 25: aee722adef0ee2ac2649facff8fed28a          class
## 26: aee722adef0ee2ac2649facff8fed28a           date
## 27: aee722adef0ee2ac2649facff8fed28a        cacheId
## 28: aee722adef0ee2ac2649facff8fed28a     objectName
## 29: aee722adef0ee2ac2649facff8fed28a       function
## 30: aee722adef0ee2ac2649facff8fed28a    object.size
## 31: aee722adef0ee2ac2649facff8fed28a       accessed
## 32: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 33: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 34: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 35: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 36: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 37: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 38: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 39: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 40: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 41: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 42: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 43: aee722adef0ee2ac2649facff8fed28a      preDigest
## 44: aee722adef0ee2ac2649facff8fed28a      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:26
##  2:      3837336a2dc856472ded40b64e3b21c7 2018-11-15 13:47:26
##  3:                               numeric 2018-11-15 13:47:26
##  4:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:26
##  6:                                     b 2018-11-15 13:47:26
##  7:                                 rnorm 2018-11-15 13:47:26
##  8:                                  1024 2018-11-15 13:47:26
##  9:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 10:                      vweave_rmarkdown 2018-11-15 13:47:26
## 11:                          process_file 2018-11-15 13:47:26
## 12:                   withCallingHandlers 2018-11-15 13:47:26
## 13:                         process_group 2018-11-15 13:47:26
## 14:                   process_group.block 2018-11-15 13:47:26
## 15:                            call_block 2018-11-15 13:47:26
## 16:                            block_exec 2018-11-15 13:47:26
## 17:                                in_dir 2018-11-15 13:47:26
## 18:                             timing_fn 2018-11-15 13:47:26
## 19:                                handle 2018-11-15 13:47:26
## 20:                           withVisible 2018-11-15 13:47:26
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:26
## 23:                                   rda 2018-11-15 13:47:26
## 24:      aee722adef0ee2ac2649facff8fed28a 2018-11-15 13:47:26
## 25:                               numeric 2018-11-15 13:47:26
## 26:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 27:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:26
## 28:                                     a 2018-11-15 13:47:26
## 29:                                 runif 2018-11-15 13:47:26
## 30:                                  1024 2018-11-15 13:47:26
## 31:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 32:                      vweave_rmarkdown 2018-11-15 13:47:26
## 33:                          process_file 2018-11-15 13:47:26
## 34:                   withCallingHandlers 2018-11-15 13:47:26
## 35:                         process_group 2018-11-15 13:47:26
## 36:                   process_group.block 2018-11-15 13:47:26
## 37:                            call_block 2018-11-15 13:47:26
## 38:                            block_exec 2018-11-15 13:47:26
## 39:                                in_dir 2018-11-15 13:47:26
## 40:                             timing_fn 2018-11-15 13:47:26
## 41:                                handle 2018-11-15 13:47:26
## 42:                           withVisible 2018-11-15 13:47:26
## 43:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 44: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:26
##                                  tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 3837336a2dc856472ded40b64e3b21c7         format
##  2: 3837336a2dc856472ded40b64e3b21c7           name
##  3: 3837336a2dc856472ded40b64e3b21c7          class
##  4: 3837336a2dc856472ded40b64e3b21c7           date
##  5: 3837336a2dc856472ded40b64e3b21c7        cacheId
##  6: 3837336a2dc856472ded40b64e3b21c7     objectName
##  7: 3837336a2dc856472ded40b64e3b21c7       function
##  8: 3837336a2dc856472ded40b64e3b21c7    object.size
##  9: 3837336a2dc856472ded40b64e3b21c7       accessed
## 10: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 11: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 12: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 13: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 14: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 15: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 16: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 17: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 18: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 19: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 20: 3837336a2dc856472ded40b64e3b21c7 otherFunctions
## 21: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 22: 3837336a2dc856472ded40b64e3b21c7      preDigest
## 23: aee722adef0ee2ac2649facff8fed28a         format
## 24: aee722adef0ee2ac2649facff8fed28a           name
## 25: aee722adef0ee2ac2649facff8fed28a          class
## 26: aee722adef0ee2ac2649facff8fed28a           date
## 27: aee722adef0ee2ac2649facff8fed28a        cacheId
## 28: aee722adef0ee2ac2649facff8fed28a     objectName
## 29: aee722adef0ee2ac2649facff8fed28a       function
## 30: aee722adef0ee2ac2649facff8fed28a    object.size
## 31: aee722adef0ee2ac2649facff8fed28a       accessed
## 32: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 33: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 34: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 35: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 36: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 37: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 38: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 39: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 40: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 41: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 42: aee722adef0ee2ac2649facff8fed28a otherFunctions
## 43: aee722adef0ee2ac2649facff8fed28a      preDigest
## 44: aee722adef0ee2ac2649facff8fed28a      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:26
##  2:      3837336a2dc856472ded40b64e3b21c7 2018-11-15 13:47:26
##  3:                               numeric 2018-11-15 13:47:26
##  4:                   2018-11-15 13:47:26 2018-11-15 13:47:26
##  5:      f55d44974d91fbf61b6cb524983b1ebc 2018-11-15 13:47:26
##  6:                                     b 2018-11-15 13:47:26
##  7:                                 rnorm 2018-11-15 13:47:26
##  8:                                  1024 2018-11-15 13:47:26
##  9:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 10:                      vweave_rmarkdown 2018-11-15 13:47:26
## 11:                          process_file 2018-11-15 13:47:26
## 12:                   withCallingHandlers 2018-11-15 13:47:26
## 13:                         process_group 2018-11-15 13:47:26
## 14:                   process_group.block 2018-11-15 13:47:26
## 15:                            call_block 2018-11-15 13:47:26
## 16:                            block_exec 2018-11-15 13:47:26
## 17:                                in_dir 2018-11-15 13:47:26
## 18:                             timing_fn 2018-11-15 13:47:26
## 19:                                handle 2018-11-15 13:47:26
## 20:                           withVisible 2018-11-15 13:47:26
## 21:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 22: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:26
## 23:                                   rda 2018-11-15 13:47:26
## 24:      aee722adef0ee2ac2649facff8fed28a 2018-11-15 13:47:26
## 25:                               numeric 2018-11-15 13:47:26
## 26:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 27:      7cb09aa387ba235db013863648303b34 2018-11-15 13:47:26
## 28:                                     a 2018-11-15 13:47:26
## 29:                                 runif 2018-11-15 13:47:26
## 30:                                  1024 2018-11-15 13:47:26
## 31:                   2018-11-15 13:47:26 2018-11-15 13:47:26
## 32:                      vweave_rmarkdown 2018-11-15 13:47:26
## 33:                          process_file 2018-11-15 13:47:26
## 34:                   withCallingHandlers 2018-11-15 13:47:26
## 35:                         process_group 2018-11-15 13:47:26
## 36:                   process_group.block 2018-11-15 13:47:26
## 37:                            call_block 2018-11-15 13:47:26
## 38:                            block_exec 2018-11-15 13:47:26
## 39:                                in_dir 2018-11-15 13:47:26
## 40:                             timing_fn 2018-11-15 13:47:26
## 41:                                handle 2018-11-15 13:47:26
## 42:                           withVisible 2018-11-15 13:47:26
## 43:    n:c1de94a5a8d2a5813e07152fb7ea2038 2018-11-15 13:47:26
## 44: .FUN:179092fbad3ec261e58bc89bc8a703c3 2018-11-15 13:47:26
##                                  tagValue         createdDate
clearCache(tmpDir)

1.8 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.9 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  0.2381528  0.2027992 -1.3193765
## attr(,"tags")
## [1] "cacheId:7360b42d0a2ddade58d480f1f9e9eef4"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 512 bytes
##   Selected objects (not including Rasters): 512 bytes
##                             artifact         tagKey
##  1: 30b042e0a8993c8737a06c71884afeb8         format
##  2: 30b042e0a8993c8737a06c71884afeb8           name
##  3: 30b042e0a8993c8737a06c71884afeb8          class
##  4: 30b042e0a8993c8737a06c71884afeb8           date
##  5: 30b042e0a8993c8737a06c71884afeb8        cacheId
##  6: 30b042e0a8993c8737a06c71884afeb8       function
##  7: 30b042e0a8993c8737a06c71884afeb8    object.size
##  8: 30b042e0a8993c8737a06c71884afeb8       accessed
##  9: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 10: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 11: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 12: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 13: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 14: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 15: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 16: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 17: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 18: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 19: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 20: 30b042e0a8993c8737a06c71884afeb8 otherFunctions
## 21: 30b042e0a8993c8737a06c71884afeb8      preDigest
## 22: 30b042e0a8993c8737a06c71884afeb8      preDigest
## 23: 30b042e0a8993c8737a06c71884afeb8      preDigest
## 24: f3004172044f8366e994ba47f6252057         format
## 25: f3004172044f8366e994ba47f6252057           name
## 26: f3004172044f8366e994ba47f6252057          class
## 27: f3004172044f8366e994ba47f6252057           date
## 28: f3004172044f8366e994ba47f6252057        cacheId
## 29: f3004172044f8366e994ba47f6252057       function
## 30: f3004172044f8366e994ba47f6252057    object.size
## 31: f3004172044f8366e994ba47f6252057       accessed
## 32: f3004172044f8366e994ba47f6252057 otherFunctions
## 33: f3004172044f8366e994ba47f6252057 otherFunctions
## 34: f3004172044f8366e994ba47f6252057 otherFunctions
## 35: f3004172044f8366e994ba47f6252057 otherFunctions
## 36: f3004172044f8366e994ba47f6252057 otherFunctions
## 37: f3004172044f8366e994ba47f6252057 otherFunctions
## 38: f3004172044f8366e994ba47f6252057 otherFunctions
## 39: f3004172044f8366e994ba47f6252057 otherFunctions
## 40: f3004172044f8366e994ba47f6252057 otherFunctions
## 41: f3004172044f8366e994ba47f6252057 otherFunctions
## 42: f3004172044f8366e994ba47f6252057 otherFunctions
## 43: f3004172044f8366e994ba47f6252057      preDigest
## 44: f3004172044f8366e994ba47f6252057      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:27
##  2:      30b042e0a8993c8737a06c71884afeb8 2018-11-15 13:47:27
##  3:                               numeric 2018-11-15 13:47:27
##  4:                   2018-11-15 13:47:27 2018-11-15 13:47:27
##  5:      696a8c27b0af4d8124a69d16123c2a9b 2018-11-15 13:47:27
##  6:                                 rnorm 2018-11-15 13:47:27
##  7:                                  1024 2018-11-15 13:47:27
##  8:                   2018-11-15 13:47:27 2018-11-15 13:47:27
##  9:                      vweave_rmarkdown 2018-11-15 13:47:27
## 10:                          process_file 2018-11-15 13:47:27
## 11:                   withCallingHandlers 2018-11-15 13:47:27
## 12:                         process_group 2018-11-15 13:47:27
## 13:                   process_group.block 2018-11-15 13:47:27
## 14:                            call_block 2018-11-15 13:47:27
## 15:                            block_exec 2018-11-15 13:47:27
## 16:                                in_dir 2018-11-15 13:47:27
## 17:                             timing_fn 2018-11-15 13:47:27
## 18:                                handle 2018-11-15 13:47:27
## 19:                           withVisible 2018-11-15 13:47:27
## 20:                               do.call 2018-11-15 13:47:27
## 21:    n:70822253d659facdcbfdb8a038cce013 2018-11-15 13:47:27
## 22: mean:e48ada84553a52f78ea2567bd2bc806a 2018-11-15 13:47:27
## 23: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:27
## 24:                                   rda 2018-11-15 13:47:27
## 25:      f3004172044f8366e994ba47f6252057 2018-11-15 13:47:27
## 26:                               numeric 2018-11-15 13:47:27
## 27:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 28:      7360b42d0a2ddade58d480f1f9e9eef4 2018-11-15 13:47:27
## 29:                                 outer 2018-11-15 13:47:27
## 30:                                  1024 2018-11-15 13:47:27
## 31:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 32:                      vweave_rmarkdown 2018-11-15 13:47:27
## 33:                          process_file 2018-11-15 13:47:27
## 34:                   withCallingHandlers 2018-11-15 13:47:27
## 35:                         process_group 2018-11-15 13:47:27
## 36:                   process_group.block 2018-11-15 13:47:27
## 37:                            call_block 2018-11-15 13:47:27
## 38:                            block_exec 2018-11-15 13:47:27
## 39:                                in_dir 2018-11-15 13:47:27
## 40:                             timing_fn 2018-11-15 13:47:27
## 41:                                handle 2018-11-15 13:47:27
## 42:                           withVisible 2018-11-15 13:47:27
## 43:    n:bfc3454880d65ea9fc34069951dbbc35 2018-11-15 13:47:27
## 44: .FUN:dfb4fe3bd3b241021e1799f8234bd28f 2018-11-15 13:47:27
##                                  tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 256 bytes
##   Selected objects (not including Rasters): 256 bytes
##                             artifact         tagKey
##  1: 01e9b86905e420bfa36375b8ee07ce36         format
##  2: 01e9b86905e420bfa36375b8ee07ce36           name
##  3: 01e9b86905e420bfa36375b8ee07ce36          class
##  4: 01e9b86905e420bfa36375b8ee07ce36           date
##  5: 01e9b86905e420bfa36375b8ee07ce36        cacheId
##  6: 01e9b86905e420bfa36375b8ee07ce36       function
##  7: 01e9b86905e420bfa36375b8ee07ce36    object.size
##  8: 01e9b86905e420bfa36375b8ee07ce36       accessed
##  9: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 10: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 11: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 12: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 13: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 14: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 15: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 16: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 17: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 18: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 19: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 20: 01e9b86905e420bfa36375b8ee07ce36 otherFunctions
## 21: 01e9b86905e420bfa36375b8ee07ce36      preDigest
## 22: 01e9b86905e420bfa36375b8ee07ce36      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:27
##  2:      01e9b86905e420bfa36375b8ee07ce36 2018-11-15 13:47:27
##  3:                               numeric 2018-11-15 13:47:27
##  4:                   2018-11-15 13:47:27 2018-11-15 13:47:27
##  5:      3d9566c2b0275c7478ee84c56177b51c 2018-11-15 13:47:27
##  6:                                 inner 2018-11-15 13:47:27
##  7:                                  1024 2018-11-15 13:47:27
##  8:                   2018-11-15 13:47:27 2018-11-15 13:47:27
##  9:                      vweave_rmarkdown 2018-11-15 13:47:27
## 10:                          process_file 2018-11-15 13:47:27
## 11:                   withCallingHandlers 2018-11-15 13:47:27
## 12:                         process_group 2018-11-15 13:47:27
## 13:                   process_group.block 2018-11-15 13:47:27
## 14:                            call_block 2018-11-15 13:47:27
## 15:                            block_exec 2018-11-15 13:47:27
## 16:                                in_dir 2018-11-15 13:47:27
## 17:                             timing_fn 2018-11-15 13:47:27
## 18:                                handle 2018-11-15 13:47:27
## 19:                           withVisible 2018-11-15 13:47:27
## 20:                               do.call 2018-11-15 13:47:27
## 21: mean:e48ada84553a52f78ea2567bd2bc806a 2018-11-15 13:47:27
## 22: .FUN:cf56d34c48d5d8c8a0ac5bd78262cdd9 2018-11-15 13:47:27
##                                  tagValue         createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 768 bytes
##   Selected objects (not including Rasters): 768 bytes
##                             artifact         tagKey
##  1: 7190146cea078acd6f41cfed5aa9c464         format
##  2: 7190146cea078acd6f41cfed5aa9c464           name
##  3: 7190146cea078acd6f41cfed5aa9c464          class
##  4: 7190146cea078acd6f41cfed5aa9c464           date
##  5: 7190146cea078acd6f41cfed5aa9c464        cacheId
##  6: 7190146cea078acd6f41cfed5aa9c464       outerTag
##  7: 7190146cea078acd6f41cfed5aa9c464       function
##  8: 7190146cea078acd6f41cfed5aa9c464    object.size
##  9: 7190146cea078acd6f41cfed5aa9c464       accessed
## 10: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 11: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 12: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 13: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 14: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 15: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 16: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 17: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 18: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 19: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 20: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 21: 7190146cea078acd6f41cfed5aa9c464 otherFunctions
## 22: 7190146cea078acd6f41cfed5aa9c464      preDigest
## 23: 7190146cea078acd6f41cfed5aa9c464      preDigest
## 24: 7a0346c751f4d364563e3aef29e4ff39         format
## 25: 7a0346c751f4d364563e3aef29e4ff39           name
## 26: 7a0346c751f4d364563e3aef29e4ff39          class
## 27: 7a0346c751f4d364563e3aef29e4ff39           date
## 28: 7a0346c751f4d364563e3aef29e4ff39        cacheId
## 29: 7a0346c751f4d364563e3aef29e4ff39       innerTag
## 30: 7a0346c751f4d364563e3aef29e4ff39       outerTag
## 31: 7a0346c751f4d364563e3aef29e4ff39       function
## 32: 7a0346c751f4d364563e3aef29e4ff39    object.size
## 33: 7a0346c751f4d364563e3aef29e4ff39       accessed
## 34: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 35: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 36: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 37: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 38: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 39: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 40: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 41: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 42: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 43: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 44: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 45: 7a0346c751f4d364563e3aef29e4ff39 otherFunctions
## 46: 7a0346c751f4d364563e3aef29e4ff39      preDigest
## 47: 7a0346c751f4d364563e3aef29e4ff39      preDigest
## 48: 7a0346c751f4d364563e3aef29e4ff39      preDigest
## 49: a59489f8f84debcd36a9119a12d169da         format
## 50: a59489f8f84debcd36a9119a12d169da           name
## 51: a59489f8f84debcd36a9119a12d169da          class
## 52: a59489f8f84debcd36a9119a12d169da           date
## 53: a59489f8f84debcd36a9119a12d169da        cacheId
## 54: a59489f8f84debcd36a9119a12d169da       outerTag
## 55: a59489f8f84debcd36a9119a12d169da       function
## 56: a59489f8f84debcd36a9119a12d169da    object.size
## 57: a59489f8f84debcd36a9119a12d169da       accessed
## 58: a59489f8f84debcd36a9119a12d169da otherFunctions
## 59: a59489f8f84debcd36a9119a12d169da otherFunctions
## 60: a59489f8f84debcd36a9119a12d169da otherFunctions
## 61: a59489f8f84debcd36a9119a12d169da otherFunctions
## 62: a59489f8f84debcd36a9119a12d169da otherFunctions
## 63: a59489f8f84debcd36a9119a12d169da otherFunctions
## 64: a59489f8f84debcd36a9119a12d169da otherFunctions
## 65: a59489f8f84debcd36a9119a12d169da otherFunctions
## 66: a59489f8f84debcd36a9119a12d169da otherFunctions
## 67: a59489f8f84debcd36a9119a12d169da otherFunctions
## 68: a59489f8f84debcd36a9119a12d169da otherFunctions
## 69: a59489f8f84debcd36a9119a12d169da      preDigest
## 70: a59489f8f84debcd36a9119a12d169da      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-11-15 13:47:27
##  2:      7190146cea078acd6f41cfed5aa9c464 2018-11-15 13:47:27
##  3:                               numeric 2018-11-15 13:47:27
##  4:                   2018-11-15 13:47:27 2018-11-15 13:47:27
##  5:      21818a9ce14c5526152af5f6bd649f1c 2018-11-15 13:47:27
##  6:                              outerTag 2018-11-15 13:47:27
##  7:                                 inner 2018-11-15 13:47:27
##  8:                                  1024 2018-11-15 13:47:27
##  9:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 10:                      vweave_rmarkdown 2018-11-15 13:47:27
## 11:                          process_file 2018-11-15 13:47:27
## 12:                   withCallingHandlers 2018-11-15 13:47:27
## 13:                         process_group 2018-11-15 13:47:27
## 14:                   process_group.block 2018-11-15 13:47:27
## 15:                            call_block 2018-11-15 13:47:27
## 16:                            block_exec 2018-11-15 13:47:27
## 17:                                in_dir 2018-11-15 13:47:27
## 18:                             timing_fn 2018-11-15 13:47:27
## 19:                                handle 2018-11-15 13:47:27
## 20:                           withVisible 2018-11-15 13:47:27
## 21:                               do.call 2018-11-15 13:47:27
## 22: mean:e48ada84553a52f78ea2567bd2bc806a 2018-11-15 13:47:27
## 23: .FUN:7371e13cb8e6f554d8e048c938825ead 2018-11-15 13:47:27
## 24:                                   rda 2018-11-15 13:47:27
## 25:      7a0346c751f4d364563e3aef29e4ff39 2018-11-15 13:47:27
## 26:                               numeric 2018-11-15 13:47:27
## 27:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 28:      696a8c27b0af4d8124a69d16123c2a9b 2018-11-15 13:47:27
## 29:                              innerTag 2018-11-15 13:47:27
## 30:                              outerTag 2018-11-15 13:47:27
## 31:                                 rnorm 2018-11-15 13:47:27
## 32:                                  1024 2018-11-15 13:47:27
## 33:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 34:                      vweave_rmarkdown 2018-11-15 13:47:27
## 35:                          process_file 2018-11-15 13:47:27
## 36:                   withCallingHandlers 2018-11-15 13:47:27
## 37:                         process_group 2018-11-15 13:47:27
## 38:                   process_group.block 2018-11-15 13:47:27
## 39:                            call_block 2018-11-15 13:47:27
## 40:                            block_exec 2018-11-15 13:47:27
## 41:                                in_dir 2018-11-15 13:47:27
## 42:                             timing_fn 2018-11-15 13:47:27
## 43:                                handle 2018-11-15 13:47:27
## 44:                           withVisible 2018-11-15 13:47:27
## 45:                               do.call 2018-11-15 13:47:27
## 46:    n:70822253d659facdcbfdb8a038cce013 2018-11-15 13:47:27
## 47: mean:e48ada84553a52f78ea2567bd2bc806a 2018-11-15 13:47:27
## 48: .FUN:a979d0700a6ef8c7c18736096ff1d522 2018-11-15 13:47:27
## 49:                                   rda 2018-11-15 13:47:27
## 50:      a59489f8f84debcd36a9119a12d169da 2018-11-15 13:47:27
## 51:                               numeric 2018-11-15 13:47:27
## 52:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 53:      527dd0f4adb50e543f78626baca662a7 2018-11-15 13:47:27
## 54:                              outerTag 2018-11-15 13:47:27
## 55:                                 outer 2018-11-15 13:47:27
## 56:                                  1024 2018-11-15 13:47:27
## 57:                   2018-11-15 13:47:27 2018-11-15 13:47:27
## 58:                      vweave_rmarkdown 2018-11-15 13:47:27
## 59:                          process_file 2018-11-15 13:47:27
## 60:                   withCallingHandlers 2018-11-15 13:47:27
## 61:                         process_group 2018-11-15 13:47:27
## 62:                   process_group.block 2018-11-15 13:47:27
## 63:                            call_block 2018-11-15 13:47:27
## 64:                            block_exec 2018-11-15 13:47:27
## 65:                                in_dir 2018-11-15 13:47:27
## 66:                             timing_fn 2018-11-15 13:47:27
## 67:                                handle 2018-11-15 13:47:27
## 68:                           withVisible 2018-11-15 13:47:27
## 69:    n:bfc3454880d65ea9fc34069951dbbc35 2018-11-15 13:47:27
## 70: .FUN:ccd653e1f8453c8e3dcd49c99f824ed9 2018-11-15 13:47:27
##                                  tagValue         createdDate

1.10 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,"tags")
## [1] "cacheId:a5fee427faceb81b71e68a2f834d377b"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

1.11 Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

2.1 Nested Caching

2.1.0.1 Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.