Introduction to Cache

Eliot J. B. McIntire

March 18 2019

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpqUrpyD/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.538   0.230   2.772
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.450   0.187   2.771
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.089   0.004   0.093
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.026   0.000   0.027
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

1.4 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 54c6831864905d1c1ce378906fa5aee2         format
##  2: 54c6831864905d1c1ce378906fa5aee2           name
##  3: 54c6831864905d1c1ce378906fa5aee2          class
##  4: 54c6831864905d1c1ce378906fa5aee2           date
##  5: 54c6831864905d1c1ce378906fa5aee2        cacheId
##  6: 54c6831864905d1c1ce378906fa5aee2     objectName
##  7: 54c6831864905d1c1ce378906fa5aee2       function
##  8: 54c6831864905d1c1ce378906fa5aee2    object.size
##  9: 54c6831864905d1c1ce378906fa5aee2       accessed
## 10: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 11: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 12: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 13: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 14: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 15: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 16: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 17: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 18: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 19: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 20: 54c6831864905d1c1ce378906fa5aee2      preDigest
## 21: 54c6831864905d1c1ce378906fa5aee2      preDigest
## 22: 9e3abba2bfbacd48314e82764730782a         format
## 23: 9e3abba2bfbacd48314e82764730782a           name
## 24: 9e3abba2bfbacd48314e82764730782a          class
## 25: 9e3abba2bfbacd48314e82764730782a           date
## 26: 9e3abba2bfbacd48314e82764730782a        cacheId
## 27: 9e3abba2bfbacd48314e82764730782a     objectName
## 28: 9e3abba2bfbacd48314e82764730782a       function
## 29: 9e3abba2bfbacd48314e82764730782a    object.size
## 30: 9e3abba2bfbacd48314e82764730782a       accessed
## 31: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 32: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 33: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 34: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 35: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 36: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 37: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 38: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 39: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 40: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 41: 9e3abba2bfbacd48314e82764730782a      preDigest
## 42: 9e3abba2bfbacd48314e82764730782a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:07
##  2: 54c6831864905d1c1ce378906fa5aee2 2019-03-18 10:09:07
##  3:                          numeric 2019-03-18 10:09:07
##  4:              2019-03-18 10:09:07 2019-03-18 10:09:07
##  5:                 3aef38d1fc02aee5 2019-03-18 10:09:07
##  6:                                b 2019-03-18 10:09:07
##  7:                            runif 2019-03-18 10:09:07
##  8:                             1008 2019-03-18 10:09:07
##  9:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 10:                 vweave_rmarkdown 2019-03-18 10:09:07
## 11:                     process_file 2019-03-18 10:09:07
## 12:                    process_group 2019-03-18 10:09:07
## 13:              process_group.block 2019-03-18 10:09:07
## 14:                       call_block 2019-03-18 10:09:07
## 15:                       block_exec 2019-03-18 10:09:07
## 16:                           in_dir 2019-03-18 10:09:07
## 17:                        timing_fn 2019-03-18 10:09:07
## 18:                           handle 2019-03-18 10:09:07
## 19:                      withVisible 2019-03-18 10:09:07
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:07
## 21:            .FUN:881ec847b7161f3c 2019-03-18 10:09:07
## 22:                              rda 2019-03-18 10:09:07
## 23: 9e3abba2bfbacd48314e82764730782a 2019-03-18 10:09:07
## 24:                          numeric 2019-03-18 10:09:07
## 25:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 26:                 f7bee22047b8d0c1 2019-03-18 10:09:07
## 27:                                a 2019-03-18 10:09:07
## 28:                            rnorm 2019-03-18 10:09:07
## 29:                             1008 2019-03-18 10:09:07
## 30:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 31:                 vweave_rmarkdown 2019-03-18 10:09:07
## 32:                     process_file 2019-03-18 10:09:07
## 33:                    process_group 2019-03-18 10:09:07
## 34:              process_group.block 2019-03-18 10:09:07
## 35:                       call_block 2019-03-18 10:09:07
## 36:                       block_exec 2019-03-18 10:09:07
## 37:                           in_dir 2019-03-18 10:09:07
## 38:                        timing_fn 2019-03-18 10:09:07
## 39:                           handle 2019-03-18 10:09:07
## 40:                      withVisible 2019-03-18 10:09:07
## 41:               n:7eef4eae85fd9229 2019-03-18 10:09:07
## 42:            .FUN:4f604aa46882b368 2019-03-18 10:09:07
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 9e3abba2bfbacd48314e82764730782a         format
##  2: 9e3abba2bfbacd48314e82764730782a           name
##  3: 9e3abba2bfbacd48314e82764730782a          class
##  4: 9e3abba2bfbacd48314e82764730782a           date
##  5: 9e3abba2bfbacd48314e82764730782a        cacheId
##  6: 9e3abba2bfbacd48314e82764730782a     objectName
##  7: 9e3abba2bfbacd48314e82764730782a       function
##  8: 9e3abba2bfbacd48314e82764730782a    object.size
##  9: 9e3abba2bfbacd48314e82764730782a       accessed
## 10: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 11: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 12: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 13: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 14: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 15: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 16: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 17: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 18: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 19: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 20: 9e3abba2bfbacd48314e82764730782a      preDigest
## 21: 9e3abba2bfbacd48314e82764730782a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:07
##  2: 9e3abba2bfbacd48314e82764730782a 2019-03-18 10:09:07
##  3:                          numeric 2019-03-18 10:09:07
##  4:              2019-03-18 10:09:07 2019-03-18 10:09:07
##  5:                 f7bee22047b8d0c1 2019-03-18 10:09:07
##  6:                                a 2019-03-18 10:09:07
##  7:                            rnorm 2019-03-18 10:09:07
##  8:                             1008 2019-03-18 10:09:07
##  9:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 10:                 vweave_rmarkdown 2019-03-18 10:09:07
## 11:                     process_file 2019-03-18 10:09:07
## 12:                    process_group 2019-03-18 10:09:07
## 13:              process_group.block 2019-03-18 10:09:07
## 14:                       call_block 2019-03-18 10:09:07
## 15:                       block_exec 2019-03-18 10:09:07
## 16:                           in_dir 2019-03-18 10:09:07
## 17:                        timing_fn 2019-03-18 10:09:07
## 18:                           handle 2019-03-18 10:09:07
## 19:                      withVisible 2019-03-18 10:09:07
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:07
## 21:            .FUN:4f604aa46882b368 2019-03-18 10:09:07
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 54c6831864905d1c1ce378906fa5aee2         format
##  2: 54c6831864905d1c1ce378906fa5aee2           name
##  3: 54c6831864905d1c1ce378906fa5aee2          class
##  4: 54c6831864905d1c1ce378906fa5aee2           date
##  5: 54c6831864905d1c1ce378906fa5aee2        cacheId
##  6: 54c6831864905d1c1ce378906fa5aee2     objectName
##  7: 54c6831864905d1c1ce378906fa5aee2       function
##  8: 54c6831864905d1c1ce378906fa5aee2    object.size
##  9: 54c6831864905d1c1ce378906fa5aee2       accessed
## 10: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 11: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 12: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 13: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 14: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 15: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 16: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 17: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 18: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 19: 54c6831864905d1c1ce378906fa5aee2 otherFunctions
## 20: 54c6831864905d1c1ce378906fa5aee2      preDigest
## 21: 54c6831864905d1c1ce378906fa5aee2      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:07
##  2: 54c6831864905d1c1ce378906fa5aee2 2019-03-18 10:09:07
##  3:                          numeric 2019-03-18 10:09:07
##  4:              2019-03-18 10:09:07 2019-03-18 10:09:07
##  5:                 3aef38d1fc02aee5 2019-03-18 10:09:07
##  6:                                b 2019-03-18 10:09:07
##  7:                            runif 2019-03-18 10:09:07
##  8:                             1008 2019-03-18 10:09:07
##  9:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 10:                 vweave_rmarkdown 2019-03-18 10:09:07
## 11:                     process_file 2019-03-18 10:09:07
## 12:                    process_group 2019-03-18 10:09:07
## 13:              process_group.block 2019-03-18 10:09:07
## 14:                       call_block 2019-03-18 10:09:07
## 15:                       block_exec 2019-03-18 10:09:07
## 16:                           in_dir 2019-03-18 10:09:07
## 17:                        timing_fn 2019-03-18 10:09:07
## 18:                           handle 2019-03-18 10:09:07
## 19:                      withVisible 2019-03-18 10:09:07
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:07
## 21:            .FUN:881ec847b7161f3c 2019-03-18 10:09:07
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 40afe0e54cb6b58b07089e62b71c52c6         format
##  2: 40afe0e54cb6b58b07089e62b71c52c6           name
##  3: 40afe0e54cb6b58b07089e62b71c52c6          class
##  4: 40afe0e54cb6b58b07089e62b71c52c6           date
##  5: 40afe0e54cb6b58b07089e62b71c52c6        cacheId
##  6: 40afe0e54cb6b58b07089e62b71c52c6       function
##  7: 40afe0e54cb6b58b07089e62b71c52c6    object.size
##  8: 40afe0e54cb6b58b07089e62b71c52c6       accessed
##  9: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 10: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 11: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 12: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 13: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 14: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 15: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 16: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 17: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 18: 40afe0e54cb6b58b07089e62b71c52c6 otherFunctions
## 19: 40afe0e54cb6b58b07089e62b71c52c6      preDigest
## 20: 40afe0e54cb6b58b07089e62b71c52c6      preDigest
## 21: 40afe0e54cb6b58b07089e62b71c52c6      preDigest
## 22: 40afe0e54cb6b58b07089e62b71c52c6       accessed
## 23: 40afe0e54cb6b58b07089e62b71c52c6       accessed
## 24: 40afe0e54cb6b58b07089e62b71c52c6       accessed
## 25: 9e3abba2bfbacd48314e82764730782a         format
## 26: 9e3abba2bfbacd48314e82764730782a           name
## 27: 9e3abba2bfbacd48314e82764730782a          class
## 28: 9e3abba2bfbacd48314e82764730782a           date
## 29: 9e3abba2bfbacd48314e82764730782a        cacheId
## 30: 9e3abba2bfbacd48314e82764730782a     objectName
## 31: 9e3abba2bfbacd48314e82764730782a       function
## 32: 9e3abba2bfbacd48314e82764730782a    object.size
## 33: 9e3abba2bfbacd48314e82764730782a       accessed
## 34: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 35: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 36: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 37: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 38: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 39: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 40: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 41: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 42: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 43: 9e3abba2bfbacd48314e82764730782a otherFunctions
## 44: 9e3abba2bfbacd48314e82764730782a      preDigest
## 45: 9e3abba2bfbacd48314e82764730782a      preDigest
## 46: f7385a3095e644dbec813a45bc5544e3         format
## 47: f7385a3095e644dbec813a45bc5544e3           name
## 48: f7385a3095e644dbec813a45bc5544e3          class
## 49: f7385a3095e644dbec813a45bc5544e3           date
## 50: f7385a3095e644dbec813a45bc5544e3        cacheId
## 51: f7385a3095e644dbec813a45bc5544e3       function
## 52: f7385a3095e644dbec813a45bc5544e3    object.size
## 53: f7385a3095e644dbec813a45bc5544e3       accessed
## 54: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 55: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 56: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 57: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 58: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 59: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 60: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 61: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 62: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 63: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 64: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 65: f7385a3095e644dbec813a45bc5544e3 otherFunctions
## 66: f7385a3095e644dbec813a45bc5544e3      preDigest
## 67: f7385a3095e644dbec813a45bc5544e3      preDigest
## 68: f7385a3095e644dbec813a45bc5544e3      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:07
##  2: 40afe0e54cb6b58b07089e62b71c52c6 2019-03-18 10:09:07
##  3:                          numeric 2019-03-18 10:09:07
##  4:              2019-03-18 10:09:07 2019-03-18 10:09:07
##  5:                 f11fb1a2880f8060 2019-03-18 10:09:07
##  6:                            rnorm 2019-03-18 10:09:07
##  7:                             1104 2019-03-18 10:09:07
##  8:              2019-03-18 10:09:07 2019-03-18 10:09:07
##  9:                 vweave_rmarkdown 2019-03-18 10:09:07
## 10:                     process_file 2019-03-18 10:09:07
## 11:                    process_group 2019-03-18 10:09:07
## 12:              process_group.block 2019-03-18 10:09:07
## 13:                       call_block 2019-03-18 10:09:07
## 14:                       block_exec 2019-03-18 10:09:07
## 15:                           in_dir 2019-03-18 10:09:07
## 16:                        timing_fn 2019-03-18 10:09:07
## 17:                           handle 2019-03-18 10:09:07
## 18:                      withVisible 2019-03-18 10:09:07
## 19:               n:c5775c3b366fb719 2019-03-18 10:09:07
## 20:            mean:15620f138033a66c 2019-03-18 10:09:07
## 21:            .FUN:4f604aa46882b368 2019-03-18 10:09:07
## 22:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 23:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 24:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 25:                              rda 2019-03-18 10:09:07
## 26: 9e3abba2bfbacd48314e82764730782a 2019-03-18 10:09:07
## 27:                          numeric 2019-03-18 10:09:07
## 28:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 29:                 f7bee22047b8d0c1 2019-03-18 10:09:07
## 30:                                a 2019-03-18 10:09:07
## 31:                            rnorm 2019-03-18 10:09:07
## 32:                             1008 2019-03-18 10:09:07
## 33:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 34:                 vweave_rmarkdown 2019-03-18 10:09:07
## 35:                     process_file 2019-03-18 10:09:07
## 36:                    process_group 2019-03-18 10:09:07
## 37:              process_group.block 2019-03-18 10:09:07
## 38:                       call_block 2019-03-18 10:09:07
## 39:                       block_exec 2019-03-18 10:09:07
## 40:                           in_dir 2019-03-18 10:09:07
## 41:                        timing_fn 2019-03-18 10:09:07
## 42:                           handle 2019-03-18 10:09:07
## 43:                      withVisible 2019-03-18 10:09:07
## 44:               n:7eef4eae85fd9229 2019-03-18 10:09:07
## 45:            .FUN:4f604aa46882b368 2019-03-18 10:09:07
## 46:                              rda 2019-03-18 10:09:07
## 47: f7385a3095e644dbec813a45bc5544e3 2019-03-18 10:09:07
## 48:                          numeric 2019-03-18 10:09:07
## 49:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 50:                 c16f0e2c57596069 2019-03-18 10:09:07
## 51:              rnorm pipe sequence 2019-03-18 10:09:07
## 52:                             1104 2019-03-18 10:09:07
## 53:              2019-03-18 10:09:07 2019-03-18 10:09:07
## 54:                 vweave_rmarkdown 2019-03-18 10:09:07
## 55:                     process_file 2019-03-18 10:09:07
## 56:                    process_group 2019-03-18 10:09:07
## 57:              process_group.block 2019-03-18 10:09:07
## 58:                       call_block 2019-03-18 10:09:07
## 59:                       block_exec 2019-03-18 10:09:07
## 60:                           in_dir 2019-03-18 10:09:07
## 61:                        timing_fn 2019-03-18 10:09:07
## 62:                           handle 2019-03-18 10:09:07
## 63:                      withVisible 2019-03-18 10:09:07
## 64:                              %C% 2019-03-18 10:09:07
## 65:                          do.call 2019-03-18 10:09:07
## 66:               n:c5775c3b366fb719 2019-03-18 10:09:07
## 67:            mean:152602b8ff81e5bb 2019-03-18 10:09:07
## 68:            .FUN:4f604aa46882b368 2019-03-18 10:09:07
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.5 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

1.6 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: abecc1438c88ccd756219cc5b0524302         format
##  2: abecc1438c88ccd756219cc5b0524302           name
##  3: abecc1438c88ccd756219cc5b0524302          class
##  4: abecc1438c88ccd756219cc5b0524302           date
##  5: abecc1438c88ccd756219cc5b0524302        cacheId
##  6: abecc1438c88ccd756219cc5b0524302     objectName
##  7: abecc1438c88ccd756219cc5b0524302       function
##  8: abecc1438c88ccd756219cc5b0524302    object.size
##  9: abecc1438c88ccd756219cc5b0524302       accessed
## 10: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 11: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 12: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 13: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 14: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 15: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 16: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 17: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 18: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 19: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 20: abecc1438c88ccd756219cc5b0524302      preDigest
## 21: abecc1438c88ccd756219cc5b0524302      preDigest
## 22: dd5031e15a8e4d3e2db9403fa66f9d41         format
## 23: dd5031e15a8e4d3e2db9403fa66f9d41           name
## 24: dd5031e15a8e4d3e2db9403fa66f9d41          class
## 25: dd5031e15a8e4d3e2db9403fa66f9d41           date
## 26: dd5031e15a8e4d3e2db9403fa66f9d41        cacheId
## 27: dd5031e15a8e4d3e2db9403fa66f9d41     objectName
## 28: dd5031e15a8e4d3e2db9403fa66f9d41       function
## 29: dd5031e15a8e4d3e2db9403fa66f9d41    object.size
## 30: dd5031e15a8e4d3e2db9403fa66f9d41       accessed
## 31: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 32: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 33: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 34: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 35: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 36: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 37: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 38: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 39: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 40: dd5031e15a8e4d3e2db9403fa66f9d41 otherFunctions
## 41: dd5031e15a8e4d3e2db9403fa66f9d41      preDigest
## 42: dd5031e15a8e4d3e2db9403fa66f9d41      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:08
##  2: abecc1438c88ccd756219cc5b0524302 2019-03-18 10:09:08
##  3:                          numeric 2019-03-18 10:09:08
##  4:              2019-03-18 10:09:08 2019-03-18 10:09:08
##  5:                 f7bee22047b8d0c1 2019-03-18 10:09:08
##  6:                                a 2019-03-18 10:09:08
##  7:                            rnorm 2019-03-18 10:09:08
##  8:                             1008 2019-03-18 10:09:08
##  9:              2019-03-18 10:09:08 2019-03-18 10:09:08
## 10:                 vweave_rmarkdown 2019-03-18 10:09:08
## 11:                     process_file 2019-03-18 10:09:08
## 12:                    process_group 2019-03-18 10:09:08
## 13:              process_group.block 2019-03-18 10:09:08
## 14:                       call_block 2019-03-18 10:09:08
## 15:                       block_exec 2019-03-18 10:09:08
## 16:                           in_dir 2019-03-18 10:09:08
## 17:                        timing_fn 2019-03-18 10:09:08
## 18:                           handle 2019-03-18 10:09:08
## 19:                      withVisible 2019-03-18 10:09:08
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:08
## 21:            .FUN:4f604aa46882b368 2019-03-18 10:09:08
## 22:                              rda 2019-03-18 10:09:08
## 23: dd5031e15a8e4d3e2db9403fa66f9d41 2019-03-18 10:09:08
## 24:                          numeric 2019-03-18 10:09:08
## 25:              2019-03-18 10:09:08 2019-03-18 10:09:08
## 26:                 3aef38d1fc02aee5 2019-03-18 10:09:08
## 27:                                b 2019-03-18 10:09:08
## 28:                            runif 2019-03-18 10:09:08
## 29:                             1008 2019-03-18 10:09:08
## 30:              2019-03-18 10:09:08 2019-03-18 10:09:08
## 31:                 vweave_rmarkdown 2019-03-18 10:09:08
## 32:                     process_file 2019-03-18 10:09:08
## 33:                    process_group 2019-03-18 10:09:08
## 34:              process_group.block 2019-03-18 10:09:08
## 35:                       call_block 2019-03-18 10:09:08
## 36:                       block_exec 2019-03-18 10:09:08
## 37:                           in_dir 2019-03-18 10:09:08
## 38:                        timing_fn 2019-03-18 10:09:08
## 39:                           handle 2019-03-18 10:09:08
## 40:                      withVisible 2019-03-18 10:09:08
## 41:               n:7eef4eae85fd9229 2019-03-18 10:09:08
## 42:            .FUN:881ec847b7161f3c 2019-03-18 10:09:08
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: abecc1438c88ccd756219cc5b0524302         format
##  2: abecc1438c88ccd756219cc5b0524302           name
##  3: abecc1438c88ccd756219cc5b0524302          class
##  4: abecc1438c88ccd756219cc5b0524302           date
##  5: abecc1438c88ccd756219cc5b0524302        cacheId
##  6: abecc1438c88ccd756219cc5b0524302     objectName
##  7: abecc1438c88ccd756219cc5b0524302       function
##  8: abecc1438c88ccd756219cc5b0524302    object.size
##  9: abecc1438c88ccd756219cc5b0524302       accessed
## 10: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 11: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 12: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 13: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 14: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 15: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 16: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 17: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 18: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 19: abecc1438c88ccd756219cc5b0524302 otherFunctions
## 20: abecc1438c88ccd756219cc5b0524302      preDigest
## 21: abecc1438c88ccd756219cc5b0524302      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:08
##  2: abecc1438c88ccd756219cc5b0524302 2019-03-18 10:09:08
##  3:                          numeric 2019-03-18 10:09:08
##  4:              2019-03-18 10:09:08 2019-03-18 10:09:08
##  5:                 f7bee22047b8d0c1 2019-03-18 10:09:08
##  6:                                a 2019-03-18 10:09:08
##  7:                            rnorm 2019-03-18 10:09:08
##  8:                             1008 2019-03-18 10:09:08
##  9:              2019-03-18 10:09:08 2019-03-18 10:09:08
## 10:                 vweave_rmarkdown 2019-03-18 10:09:08
## 11:                     process_file 2019-03-18 10:09:08
## 12:                    process_group 2019-03-18 10:09:08
## 13:              process_group.block 2019-03-18 10:09:08
## 14:                       call_block 2019-03-18 10:09:08
## 15:                       block_exec 2019-03-18 10:09:08
## 16:                           in_dir 2019-03-18 10:09:08
## 17:                        timing_fn 2019-03-18 10:09:08
## 18:                           handle 2019-03-18 10:09:08
## 19:                      withVisible 2019-03-18 10:09:08
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:08
## 21:            .FUN:4f604aa46882b368 2019-03-18 10:09:08
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

1.7 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: bef41a90b34bec8f0e8521cab176db7b         format
##  2: bef41a90b34bec8f0e8521cab176db7b           name
##  3: bef41a90b34bec8f0e8521cab176db7b          class
##  4: bef41a90b34bec8f0e8521cab176db7b           date
##  5: bef41a90b34bec8f0e8521cab176db7b        cacheId
##  6: bef41a90b34bec8f0e8521cab176db7b     objectName
##  7: bef41a90b34bec8f0e8521cab176db7b       function
##  8: bef41a90b34bec8f0e8521cab176db7b    object.size
##  9: bef41a90b34bec8f0e8521cab176db7b       accessed
## 10: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 11: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 12: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 13: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 14: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 15: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 16: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 17: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 18: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 19: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 20: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 21: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 22: c5ef60c8f6e14c46c83615fbdf9caa93         format
## 23: c5ef60c8f6e14c46c83615fbdf9caa93           name
## 24: c5ef60c8f6e14c46c83615fbdf9caa93          class
## 25: c5ef60c8f6e14c46c83615fbdf9caa93           date
## 26: c5ef60c8f6e14c46c83615fbdf9caa93        cacheId
## 27: c5ef60c8f6e14c46c83615fbdf9caa93     objectName
## 28: c5ef60c8f6e14c46c83615fbdf9caa93       function
## 29: c5ef60c8f6e14c46c83615fbdf9caa93    object.size
## 30: c5ef60c8f6e14c46c83615fbdf9caa93       accessed
## 31: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 32: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 33: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 34: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 35: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 36: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 37: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 38: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 39: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 40: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 41: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
## 42: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:09
##  2: bef41a90b34bec8f0e8521cab176db7b 2019-03-18 10:09:09
##  3:                          numeric 2019-03-18 10:09:09
##  4:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  5:                 3aef38d1fc02aee5 2019-03-18 10:09:09
##  6:                                a 2019-03-18 10:09:09
##  7:                            runif 2019-03-18 10:09:09
##  8:                             1008 2019-03-18 10:09:09
##  9:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 10:                 vweave_rmarkdown 2019-03-18 10:09:09
## 11:                     process_file 2019-03-18 10:09:09
## 12:                    process_group 2019-03-18 10:09:09
## 13:              process_group.block 2019-03-18 10:09:09
## 14:                       call_block 2019-03-18 10:09:09
## 15:                       block_exec 2019-03-18 10:09:09
## 16:                           in_dir 2019-03-18 10:09:09
## 17:                        timing_fn 2019-03-18 10:09:09
## 18:                           handle 2019-03-18 10:09:09
## 19:                      withVisible 2019-03-18 10:09:09
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 21:            .FUN:881ec847b7161f3c 2019-03-18 10:09:09
## 22:                              rda 2019-03-18 10:09:09
## 23: c5ef60c8f6e14c46c83615fbdf9caa93 2019-03-18 10:09:09
## 24:                          numeric 2019-03-18 10:09:09
## 25:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 26:                 f7bee22047b8d0c1 2019-03-18 10:09:09
## 27:                                b 2019-03-18 10:09:09
## 28:                            rnorm 2019-03-18 10:09:09
## 29:                             1008 2019-03-18 10:09:09
## 30:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 31:                 vweave_rmarkdown 2019-03-18 10:09:09
## 32:                     process_file 2019-03-18 10:09:09
## 33:                    process_group 2019-03-18 10:09:09
## 34:              process_group.block 2019-03-18 10:09:09
## 35:                       call_block 2019-03-18 10:09:09
## 36:                       block_exec 2019-03-18 10:09:09
## 37:                           in_dir 2019-03-18 10:09:09
## 38:                        timing_fn 2019-03-18 10:09:09
## 39:                           handle 2019-03-18 10:09:09
## 40:                      withVisible 2019-03-18 10:09:09
## 41:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 42:            .FUN:4f604aa46882b368 2019-03-18 10:09:09
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: bef41a90b34bec8f0e8521cab176db7b         format
##  2: bef41a90b34bec8f0e8521cab176db7b           name
##  3: bef41a90b34bec8f0e8521cab176db7b          class
##  4: bef41a90b34bec8f0e8521cab176db7b           date
##  5: bef41a90b34bec8f0e8521cab176db7b        cacheId
##  6: bef41a90b34bec8f0e8521cab176db7b     objectName
##  7: bef41a90b34bec8f0e8521cab176db7b       function
##  8: bef41a90b34bec8f0e8521cab176db7b    object.size
##  9: bef41a90b34bec8f0e8521cab176db7b       accessed
## 10: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 11: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 12: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 13: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 14: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 15: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 16: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 17: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 18: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 19: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 20: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 21: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 22: c5ef60c8f6e14c46c83615fbdf9caa93         format
## 23: c5ef60c8f6e14c46c83615fbdf9caa93           name
## 24: c5ef60c8f6e14c46c83615fbdf9caa93          class
## 25: c5ef60c8f6e14c46c83615fbdf9caa93           date
## 26: c5ef60c8f6e14c46c83615fbdf9caa93        cacheId
## 27: c5ef60c8f6e14c46c83615fbdf9caa93     objectName
## 28: c5ef60c8f6e14c46c83615fbdf9caa93       function
## 29: c5ef60c8f6e14c46c83615fbdf9caa93    object.size
## 30: c5ef60c8f6e14c46c83615fbdf9caa93       accessed
## 31: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 32: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 33: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 34: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 35: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 36: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 37: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 38: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 39: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 40: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 41: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
## 42: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:09
##  2: bef41a90b34bec8f0e8521cab176db7b 2019-03-18 10:09:09
##  3:                          numeric 2019-03-18 10:09:09
##  4:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  5:                 3aef38d1fc02aee5 2019-03-18 10:09:09
##  6:                                a 2019-03-18 10:09:09
##  7:                            runif 2019-03-18 10:09:09
##  8:                             1008 2019-03-18 10:09:09
##  9:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 10:                 vweave_rmarkdown 2019-03-18 10:09:09
## 11:                     process_file 2019-03-18 10:09:09
## 12:                    process_group 2019-03-18 10:09:09
## 13:              process_group.block 2019-03-18 10:09:09
## 14:                       call_block 2019-03-18 10:09:09
## 15:                       block_exec 2019-03-18 10:09:09
## 16:                           in_dir 2019-03-18 10:09:09
## 17:                        timing_fn 2019-03-18 10:09:09
## 18:                           handle 2019-03-18 10:09:09
## 19:                      withVisible 2019-03-18 10:09:09
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 21:            .FUN:881ec847b7161f3c 2019-03-18 10:09:09
## 22:                              rda 2019-03-18 10:09:09
## 23: c5ef60c8f6e14c46c83615fbdf9caa93 2019-03-18 10:09:09
## 24:                          numeric 2019-03-18 10:09:09
## 25:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 26:                 f7bee22047b8d0c1 2019-03-18 10:09:09
## 27:                                b 2019-03-18 10:09:09
## 28:                            rnorm 2019-03-18 10:09:09
## 29:                             1008 2019-03-18 10:09:09
## 30:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 31:                 vweave_rmarkdown 2019-03-18 10:09:09
## 32:                     process_file 2019-03-18 10:09:09
## 33:                    process_group 2019-03-18 10:09:09
## 34:              process_group.block 2019-03-18 10:09:09
## 35:                       call_block 2019-03-18 10:09:09
## 36:                       block_exec 2019-03-18 10:09:09
## 37:                           in_dir 2019-03-18 10:09:09
## 38:                        timing_fn 2019-03-18 10:09:09
## 39:                           handle 2019-03-18 10:09:09
## 40:                      withVisible 2019-03-18 10:09:09
## 41:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 42:            .FUN:4f604aa46882b368 2019-03-18 10:09:09
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: bef41a90b34bec8f0e8521cab176db7b         format
##  2: bef41a90b34bec8f0e8521cab176db7b           name
##  3: bef41a90b34bec8f0e8521cab176db7b          class
##  4: bef41a90b34bec8f0e8521cab176db7b           date
##  5: bef41a90b34bec8f0e8521cab176db7b        cacheId
##  6: bef41a90b34bec8f0e8521cab176db7b     objectName
##  7: bef41a90b34bec8f0e8521cab176db7b       function
##  8: bef41a90b34bec8f0e8521cab176db7b    object.size
##  9: bef41a90b34bec8f0e8521cab176db7b       accessed
## 10: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 11: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 12: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 13: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 14: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 15: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 16: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 17: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 18: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 19: bef41a90b34bec8f0e8521cab176db7b otherFunctions
## 20: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 21: bef41a90b34bec8f0e8521cab176db7b      preDigest
## 22: c5ef60c8f6e14c46c83615fbdf9caa93         format
## 23: c5ef60c8f6e14c46c83615fbdf9caa93           name
## 24: c5ef60c8f6e14c46c83615fbdf9caa93          class
## 25: c5ef60c8f6e14c46c83615fbdf9caa93           date
## 26: c5ef60c8f6e14c46c83615fbdf9caa93        cacheId
## 27: c5ef60c8f6e14c46c83615fbdf9caa93     objectName
## 28: c5ef60c8f6e14c46c83615fbdf9caa93       function
## 29: c5ef60c8f6e14c46c83615fbdf9caa93    object.size
## 30: c5ef60c8f6e14c46c83615fbdf9caa93       accessed
## 31: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 32: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 33: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 34: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 35: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 36: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 37: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 38: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 39: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 40: c5ef60c8f6e14c46c83615fbdf9caa93 otherFunctions
## 41: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
## 42: c5ef60c8f6e14c46c83615fbdf9caa93      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:09
##  2: bef41a90b34bec8f0e8521cab176db7b 2019-03-18 10:09:09
##  3:                          numeric 2019-03-18 10:09:09
##  4:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  5:                 3aef38d1fc02aee5 2019-03-18 10:09:09
##  6:                                a 2019-03-18 10:09:09
##  7:                            runif 2019-03-18 10:09:09
##  8:                             1008 2019-03-18 10:09:09
##  9:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 10:                 vweave_rmarkdown 2019-03-18 10:09:09
## 11:                     process_file 2019-03-18 10:09:09
## 12:                    process_group 2019-03-18 10:09:09
## 13:              process_group.block 2019-03-18 10:09:09
## 14:                       call_block 2019-03-18 10:09:09
## 15:                       block_exec 2019-03-18 10:09:09
## 16:                           in_dir 2019-03-18 10:09:09
## 17:                        timing_fn 2019-03-18 10:09:09
## 18:                           handle 2019-03-18 10:09:09
## 19:                      withVisible 2019-03-18 10:09:09
## 20:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 21:            .FUN:881ec847b7161f3c 2019-03-18 10:09:09
## 22:                              rda 2019-03-18 10:09:09
## 23: c5ef60c8f6e14c46c83615fbdf9caa93 2019-03-18 10:09:09
## 24:                          numeric 2019-03-18 10:09:09
## 25:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 26:                 f7bee22047b8d0c1 2019-03-18 10:09:09
## 27:                                b 2019-03-18 10:09:09
## 28:                            rnorm 2019-03-18 10:09:09
## 29:                             1008 2019-03-18 10:09:09
## 30:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 31:                 vweave_rmarkdown 2019-03-18 10:09:09
## 32:                     process_file 2019-03-18 10:09:09
## 33:                    process_group 2019-03-18 10:09:09
## 34:              process_group.block 2019-03-18 10:09:09
## 35:                       call_block 2019-03-18 10:09:09
## 36:                       block_exec 2019-03-18 10:09:09
## 37:                           in_dir 2019-03-18 10:09:09
## 38:                        timing_fn 2019-03-18 10:09:09
## 39:                           handle 2019-03-18 10:09:09
## 40:                      withVisible 2019-03-18 10:09:09
## 41:               n:7eef4eae85fd9229 2019-03-18 10:09:09
## 42:            .FUN:4f604aa46882b368 2019-03-18 10:09:09
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.8 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.9 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1] -0.75133638  0.04503018 -0.26304840
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 8f260a8d81a6516296ae665adec5ce31         format
##  2: 8f260a8d81a6516296ae665adec5ce31           name
##  3: 8f260a8d81a6516296ae665adec5ce31          class
##  4: 8f260a8d81a6516296ae665adec5ce31           date
##  5: 8f260a8d81a6516296ae665adec5ce31        cacheId
##  6: 8f260a8d81a6516296ae665adec5ce31       function
##  7: 8f260a8d81a6516296ae665adec5ce31    object.size
##  8: 8f260a8d81a6516296ae665adec5ce31       accessed
##  9: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 10: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 11: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 12: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 13: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 14: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 15: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 16: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 17: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 18: 8f260a8d81a6516296ae665adec5ce31 otherFunctions
## 19: 8f260a8d81a6516296ae665adec5ce31      preDigest
## 20: 8f260a8d81a6516296ae665adec5ce31      preDigest
## 21: 8f260a8d81a6516296ae665adec5ce31      preDigest
## 22: f096da0ebd9f23075e52d4226b81669c         format
## 23: f096da0ebd9f23075e52d4226b81669c           name
## 24: f096da0ebd9f23075e52d4226b81669c          class
## 25: f096da0ebd9f23075e52d4226b81669c           date
## 26: f096da0ebd9f23075e52d4226b81669c        cacheId
## 27: f096da0ebd9f23075e52d4226b81669c       function
## 28: f096da0ebd9f23075e52d4226b81669c    object.size
## 29: f096da0ebd9f23075e52d4226b81669c       accessed
## 30: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 31: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 32: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 33: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 34: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 35: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 36: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 37: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 38: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 39: f096da0ebd9f23075e52d4226b81669c otherFunctions
## 40: f096da0ebd9f23075e52d4226b81669c      preDigest
## 41: f096da0ebd9f23075e52d4226b81669c      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:09
##  2: 8f260a8d81a6516296ae665adec5ce31 2019-03-18 10:09:09
##  3:                          numeric 2019-03-18 10:09:09
##  4:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  5:                 4ac2b7c0f42d1e46 2019-03-18 10:09:09
##  6:                            rnorm 2019-03-18 10:09:09
##  7:                             1008 2019-03-18 10:09:09
##  8:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  9:                 vweave_rmarkdown 2019-03-18 10:09:09
## 10:                     process_file 2019-03-18 10:09:09
## 11:                    process_group 2019-03-18 10:09:09
## 12:              process_group.block 2019-03-18 10:09:09
## 13:                       call_block 2019-03-18 10:09:09
## 14:                       block_exec 2019-03-18 10:09:09
## 15:                           in_dir 2019-03-18 10:09:09
## 16:                        timing_fn 2019-03-18 10:09:09
## 17:                           handle 2019-03-18 10:09:09
## 18:                      withVisible 2019-03-18 10:09:09
## 19:               n:7f12988bd88a0fb8 2019-03-18 10:09:09
## 20:            mean:22413394efd9f6a3 2019-03-18 10:09:09
## 21:            .FUN:4f604aa46882b368 2019-03-18 10:09:09
## 22:                              rda 2019-03-18 10:09:09
## 23: f096da0ebd9f23075e52d4226b81669c 2019-03-18 10:09:09
## 24:                          numeric 2019-03-18 10:09:09
## 25:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 26:                 a7af5367c13aba8f 2019-03-18 10:09:09
## 27:                            outer 2019-03-18 10:09:09
## 28:                             1008 2019-03-18 10:09:09
## 29:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 30:                 vweave_rmarkdown 2019-03-18 10:09:09
## 31:                     process_file 2019-03-18 10:09:09
## 32:                    process_group 2019-03-18 10:09:09
## 33:              process_group.block 2019-03-18 10:09:09
## 34:                       call_block 2019-03-18 10:09:09
## 35:                       block_exec 2019-03-18 10:09:09
## 36:                           in_dir 2019-03-18 10:09:09
## 37:                        timing_fn 2019-03-18 10:09:09
## 38:                           handle 2019-03-18 10:09:09
## 39:                      withVisible 2019-03-18 10:09:09
## 40:               n:82dc709f2b91918a 2019-03-18 10:09:09
## 41:            .FUN:892a6afc47a63a90 2019-03-18 10:09:09
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 25d8938dcc1e28bc1877bfd21a01b80c         format
##  2: 25d8938dcc1e28bc1877bfd21a01b80c           name
##  3: 25d8938dcc1e28bc1877bfd21a01b80c          class
##  4: 25d8938dcc1e28bc1877bfd21a01b80c           date
##  5: 25d8938dcc1e28bc1877bfd21a01b80c        cacheId
##  6: 25d8938dcc1e28bc1877bfd21a01b80c       function
##  7: 25d8938dcc1e28bc1877bfd21a01b80c    object.size
##  8: 25d8938dcc1e28bc1877bfd21a01b80c       accessed
##  9: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 10: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 11: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 12: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 13: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 14: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 15: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 16: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 17: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 18: 25d8938dcc1e28bc1877bfd21a01b80c otherFunctions
## 19: 25d8938dcc1e28bc1877bfd21a01b80c      preDigest
## 20: 25d8938dcc1e28bc1877bfd21a01b80c      preDigest
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:09
##  2: 25d8938dcc1e28bc1877bfd21a01b80c 2019-03-18 10:09:09
##  3:                          numeric 2019-03-18 10:09:09
##  4:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  5:                 33ceb4fb525fd08f 2019-03-18 10:09:09
##  6:                            inner 2019-03-18 10:09:09
##  7:                             1008 2019-03-18 10:09:09
##  8:              2019-03-18 10:09:09 2019-03-18 10:09:09
##  9:                 vweave_rmarkdown 2019-03-18 10:09:09
## 10:                     process_file 2019-03-18 10:09:09
## 11:                    process_group 2019-03-18 10:09:09
## 12:              process_group.block 2019-03-18 10:09:09
## 13:                       call_block 2019-03-18 10:09:09
## 14:                       block_exec 2019-03-18 10:09:09
## 15:                           in_dir 2019-03-18 10:09:09
## 16:                        timing_fn 2019-03-18 10:09:09
## 17:                           handle 2019-03-18 10:09:09
## 18:                      withVisible 2019-03-18 10:09:09
## 19:            mean:22413394efd9f6a3 2019-03-18 10:09:09
## 20:            .FUN:87e2c30917a34d25 2019-03-18 10:09:09
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 0eeb592d7d32caa9bb100d48f20d9645         format
##  2: 0eeb592d7d32caa9bb100d48f20d9645           name
##  3: 0eeb592d7d32caa9bb100d48f20d9645          class
##  4: 0eeb592d7d32caa9bb100d48f20d9645           date
##  5: 0eeb592d7d32caa9bb100d48f20d9645        cacheId
##  6: 0eeb592d7d32caa9bb100d48f20d9645       outerTag
##  7: 0eeb592d7d32caa9bb100d48f20d9645       function
##  8: 0eeb592d7d32caa9bb100d48f20d9645    object.size
##  9: 0eeb592d7d32caa9bb100d48f20d9645       accessed
## 10: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 11: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 12: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 13: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 14: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 15: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 16: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 17: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 18: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 19: 0eeb592d7d32caa9bb100d48f20d9645 otherFunctions
## 20: 0eeb592d7d32caa9bb100d48f20d9645      preDigest
## 21: 0eeb592d7d32caa9bb100d48f20d9645      preDigest
## 22: 338c2e0c6df21a9d0979f4f2dfe3eb9f         format
## 23: 338c2e0c6df21a9d0979f4f2dfe3eb9f           name
## 24: 338c2e0c6df21a9d0979f4f2dfe3eb9f          class
## 25: 338c2e0c6df21a9d0979f4f2dfe3eb9f           date
## 26: 338c2e0c6df21a9d0979f4f2dfe3eb9f        cacheId
## 27: 338c2e0c6df21a9d0979f4f2dfe3eb9f       innerTag
## 28: 338c2e0c6df21a9d0979f4f2dfe3eb9f       outerTag
## 29: 338c2e0c6df21a9d0979f4f2dfe3eb9f       function
## 30: 338c2e0c6df21a9d0979f4f2dfe3eb9f    object.size
## 31: 338c2e0c6df21a9d0979f4f2dfe3eb9f       accessed
## 32: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 33: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 34: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 35: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 36: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 37: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 38: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 39: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 40: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 41: 338c2e0c6df21a9d0979f4f2dfe3eb9f otherFunctions
## 42: 338c2e0c6df21a9d0979f4f2dfe3eb9f      preDigest
## 43: 338c2e0c6df21a9d0979f4f2dfe3eb9f      preDigest
## 44: 338c2e0c6df21a9d0979f4f2dfe3eb9f      preDigest
## 45: 542378c53deeb0dcc1a068e2a8fe23eb         format
## 46: 542378c53deeb0dcc1a068e2a8fe23eb           name
## 47: 542378c53deeb0dcc1a068e2a8fe23eb          class
## 48: 542378c53deeb0dcc1a068e2a8fe23eb           date
## 49: 542378c53deeb0dcc1a068e2a8fe23eb        cacheId
## 50: 542378c53deeb0dcc1a068e2a8fe23eb       outerTag
## 51: 542378c53deeb0dcc1a068e2a8fe23eb       function
## 52: 542378c53deeb0dcc1a068e2a8fe23eb    object.size
## 53: 542378c53deeb0dcc1a068e2a8fe23eb       accessed
## 54: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 55: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 56: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 57: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 58: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 59: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 60: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 61: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 62: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 63: 542378c53deeb0dcc1a068e2a8fe23eb otherFunctions
## 64: 542378c53deeb0dcc1a068e2a8fe23eb      preDigest
## 65: 542378c53deeb0dcc1a068e2a8fe23eb      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:09:10
##  2: 0eeb592d7d32caa9bb100d48f20d9645 2019-03-18 10:09:10
##  3:                          numeric 2019-03-18 10:09:10
##  4:              2019-03-18 10:09:10 2019-03-18 10:09:10
##  5:                 88a34e1d033329e5 2019-03-18 10:09:10
##  6:                         outerTag 2019-03-18 10:09:10
##  7:                            outer 2019-03-18 10:09:10
##  8:                             1008 2019-03-18 10:09:10
##  9:              2019-03-18 10:09:10 2019-03-18 10:09:10
## 10:                 vweave_rmarkdown 2019-03-18 10:09:10
## 11:                     process_file 2019-03-18 10:09:10
## 12:                    process_group 2019-03-18 10:09:10
## 13:              process_group.block 2019-03-18 10:09:10
## 14:                       call_block 2019-03-18 10:09:10
## 15:                       block_exec 2019-03-18 10:09:10
## 16:                           in_dir 2019-03-18 10:09:10
## 17:                        timing_fn 2019-03-18 10:09:10
## 18:                           handle 2019-03-18 10:09:10
## 19:                      withVisible 2019-03-18 10:09:10
## 20:               n:82dc709f2b91918a 2019-03-18 10:09:10
## 21:            .FUN:5f06fb5fbffe9e3b 2019-03-18 10:09:10
## 22:                              rda 2019-03-18 10:09:09
## 23: 338c2e0c6df21a9d0979f4f2dfe3eb9f 2019-03-18 10:09:09
## 24:                          numeric 2019-03-18 10:09:09
## 25:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 26:                 4ac2b7c0f42d1e46 2019-03-18 10:09:09
## 27:                         innerTag 2019-03-18 10:09:09
## 28:                         outerTag 2019-03-18 10:09:09
## 29:                            rnorm 2019-03-18 10:09:09
## 30:                             1008 2019-03-18 10:09:09
## 31:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 32:                 vweave_rmarkdown 2019-03-18 10:09:09
## 33:                     process_file 2019-03-18 10:09:09
## 34:                    process_group 2019-03-18 10:09:09
## 35:              process_group.block 2019-03-18 10:09:09
## 36:                       call_block 2019-03-18 10:09:09
## 37:                       block_exec 2019-03-18 10:09:09
## 38:                           in_dir 2019-03-18 10:09:09
## 39:                        timing_fn 2019-03-18 10:09:09
## 40:                           handle 2019-03-18 10:09:09
## 41:                      withVisible 2019-03-18 10:09:09
## 42:               n:7f12988bd88a0fb8 2019-03-18 10:09:09
## 43:            mean:22413394efd9f6a3 2019-03-18 10:09:09
## 44:            .FUN:4f604aa46882b368 2019-03-18 10:09:09
## 45:                              rda 2019-03-18 10:09:09
## 46: 542378c53deeb0dcc1a068e2a8fe23eb 2019-03-18 10:09:09
## 47:                          numeric 2019-03-18 10:09:09
## 48:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 49:                 b06af03d5a73dc7d 2019-03-18 10:09:09
## 50:                         outerTag 2019-03-18 10:09:09
## 51:                            inner 2019-03-18 10:09:09
## 52:                             1008 2019-03-18 10:09:09
## 53:              2019-03-18 10:09:09 2019-03-18 10:09:09
## 54:                 vweave_rmarkdown 2019-03-18 10:09:09
## 55:                     process_file 2019-03-18 10:09:09
## 56:                    process_group 2019-03-18 10:09:09
## 57:              process_group.block 2019-03-18 10:09:09
## 58:                       call_block 2019-03-18 10:09:10
## 59:                       block_exec 2019-03-18 10:09:10
## 60:                           in_dir 2019-03-18 10:09:10
## 61:                        timing_fn 2019-03-18 10:09:10
## 62:                           handle 2019-03-18 10:09:10
## 63:                      withVisible 2019-03-18 10:09:10
## 64:            mean:22413394efd9f6a3 2019-03-18 10:09:10
## 65:            .FUN:7ad10bc1ae528d8c 2019-03-18 10:09:10
##                             tagValue         createdDate

1.10 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

1.11 Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

2.1 Nested Caching

2.1.0.1 Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.