Introduction to Cache

Eliot J. B. McIntire

March 06 2019

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpFx8Vzv/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.552   0.286   2.843
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.956   0.261   3.402
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.107   0.004   0.111
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.039   0.000   0.039
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

1.4 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 94672e30258fcf08bbb11f1eaa8fb288         format
##  2: 94672e30258fcf08bbb11f1eaa8fb288           name
##  3: 94672e30258fcf08bbb11f1eaa8fb288          class
##  4: 94672e30258fcf08bbb11f1eaa8fb288           date
##  5: 94672e30258fcf08bbb11f1eaa8fb288        cacheId
##  6: 94672e30258fcf08bbb11f1eaa8fb288     objectName
##  7: 94672e30258fcf08bbb11f1eaa8fb288       function
##  8: 94672e30258fcf08bbb11f1eaa8fb288    object.size
##  9: 94672e30258fcf08bbb11f1eaa8fb288       accessed
## 10: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 11: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 12: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 13: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 14: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 15: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 16: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 17: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 18: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 19: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 20: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
## 21: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
## 22: ff9e274d8fa460799fd252b73c784bf1         format
## 23: ff9e274d8fa460799fd252b73c784bf1           name
## 24: ff9e274d8fa460799fd252b73c784bf1          class
## 25: ff9e274d8fa460799fd252b73c784bf1           date
## 26: ff9e274d8fa460799fd252b73c784bf1        cacheId
## 27: ff9e274d8fa460799fd252b73c784bf1     objectName
## 28: ff9e274d8fa460799fd252b73c784bf1       function
## 29: ff9e274d8fa460799fd252b73c784bf1    object.size
## 30: ff9e274d8fa460799fd252b73c784bf1       accessed
## 31: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 32: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 33: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 34: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 35: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 36: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 37: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 38: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 39: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 40: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 41: ff9e274d8fa460799fd252b73c784bf1      preDigest
## 42: ff9e274d8fa460799fd252b73c784bf1      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:51
##  2: 94672e30258fcf08bbb11f1eaa8fb288 2019-03-06 08:11:51
##  3:                          numeric 2019-03-06 08:11:51
##  4:              2019-03-06 08:11:51 2019-03-06 08:11:51
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:51
##  6:                                a 2019-03-06 08:11:51
##  7:                            rnorm 2019-03-06 08:11:51
##  8:                             1008 2019-03-06 08:11:51
##  9:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 10:                 vweave_rmarkdown 2019-03-06 08:11:51
## 11:                     process_file 2019-03-06 08:11:51
## 12:                    process_group 2019-03-06 08:11:51
## 13:              process_group.block 2019-03-06 08:11:51
## 14:                       call_block 2019-03-06 08:11:51
## 15:                       block_exec 2019-03-06 08:11:51
## 16:                           in_dir 2019-03-06 08:11:51
## 17:                        timing_fn 2019-03-06 08:11:51
## 18:                           handle 2019-03-06 08:11:51
## 19:                      withVisible 2019-03-06 08:11:51
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:51
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:51
## 22:                              rda 2019-03-06 08:11:51
## 23: ff9e274d8fa460799fd252b73c784bf1 2019-03-06 08:11:51
## 24:                          numeric 2019-03-06 08:11:51
## 25:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 26:                 3aef38d1fc02aee5 2019-03-06 08:11:51
## 27:                                b 2019-03-06 08:11:51
## 28:                            runif 2019-03-06 08:11:51
## 29:                             1008 2019-03-06 08:11:51
## 30:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 31:                 vweave_rmarkdown 2019-03-06 08:11:51
## 32:                     process_file 2019-03-06 08:11:51
## 33:                    process_group 2019-03-06 08:11:51
## 34:              process_group.block 2019-03-06 08:11:51
## 35:                       call_block 2019-03-06 08:11:51
## 36:                       block_exec 2019-03-06 08:11:51
## 37:                           in_dir 2019-03-06 08:11:51
## 38:                        timing_fn 2019-03-06 08:11:51
## 39:                           handle 2019-03-06 08:11:51
## 40:                      withVisible 2019-03-06 08:11:51
## 41:               n:7eef4eae85fd9229 2019-03-06 08:11:51
## 42:            .FUN:881ec847b7161f3c 2019-03-06 08:11:51
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 94672e30258fcf08bbb11f1eaa8fb288         format
##  2: 94672e30258fcf08bbb11f1eaa8fb288           name
##  3: 94672e30258fcf08bbb11f1eaa8fb288          class
##  4: 94672e30258fcf08bbb11f1eaa8fb288           date
##  5: 94672e30258fcf08bbb11f1eaa8fb288        cacheId
##  6: 94672e30258fcf08bbb11f1eaa8fb288     objectName
##  7: 94672e30258fcf08bbb11f1eaa8fb288       function
##  8: 94672e30258fcf08bbb11f1eaa8fb288    object.size
##  9: 94672e30258fcf08bbb11f1eaa8fb288       accessed
## 10: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 11: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 12: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 13: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 14: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 15: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 16: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 17: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 18: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 19: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 20: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
## 21: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:51
##  2: 94672e30258fcf08bbb11f1eaa8fb288 2019-03-06 08:11:51
##  3:                          numeric 2019-03-06 08:11:51
##  4:              2019-03-06 08:11:51 2019-03-06 08:11:51
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:51
##  6:                                a 2019-03-06 08:11:51
##  7:                            rnorm 2019-03-06 08:11:51
##  8:                             1008 2019-03-06 08:11:51
##  9:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 10:                 vweave_rmarkdown 2019-03-06 08:11:51
## 11:                     process_file 2019-03-06 08:11:51
## 12:                    process_group 2019-03-06 08:11:51
## 13:              process_group.block 2019-03-06 08:11:51
## 14:                       call_block 2019-03-06 08:11:51
## 15:                       block_exec 2019-03-06 08:11:51
## 16:                           in_dir 2019-03-06 08:11:51
## 17:                        timing_fn 2019-03-06 08:11:51
## 18:                           handle 2019-03-06 08:11:51
## 19:                      withVisible 2019-03-06 08:11:51
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:51
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:51
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: ff9e274d8fa460799fd252b73c784bf1         format
##  2: ff9e274d8fa460799fd252b73c784bf1           name
##  3: ff9e274d8fa460799fd252b73c784bf1          class
##  4: ff9e274d8fa460799fd252b73c784bf1           date
##  5: ff9e274d8fa460799fd252b73c784bf1        cacheId
##  6: ff9e274d8fa460799fd252b73c784bf1     objectName
##  7: ff9e274d8fa460799fd252b73c784bf1       function
##  8: ff9e274d8fa460799fd252b73c784bf1    object.size
##  9: ff9e274d8fa460799fd252b73c784bf1       accessed
## 10: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 11: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 12: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 13: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 14: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 15: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 16: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 17: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 18: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 19: ff9e274d8fa460799fd252b73c784bf1 otherFunctions
## 20: ff9e274d8fa460799fd252b73c784bf1      preDigest
## 21: ff9e274d8fa460799fd252b73c784bf1      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:51
##  2: ff9e274d8fa460799fd252b73c784bf1 2019-03-06 08:11:51
##  3:                          numeric 2019-03-06 08:11:51
##  4:              2019-03-06 08:11:51 2019-03-06 08:11:51
##  5:                 3aef38d1fc02aee5 2019-03-06 08:11:51
##  6:                                b 2019-03-06 08:11:51
##  7:                            runif 2019-03-06 08:11:51
##  8:                             1008 2019-03-06 08:11:51
##  9:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 10:                 vweave_rmarkdown 2019-03-06 08:11:51
## 11:                     process_file 2019-03-06 08:11:51
## 12:                    process_group 2019-03-06 08:11:51
## 13:              process_group.block 2019-03-06 08:11:51
## 14:                       call_block 2019-03-06 08:11:51
## 15:                       block_exec 2019-03-06 08:11:51
## 16:                           in_dir 2019-03-06 08:11:51
## 17:                        timing_fn 2019-03-06 08:11:51
## 18:                           handle 2019-03-06 08:11:51
## 19:                      withVisible 2019-03-06 08:11:51
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:51
## 21:            .FUN:881ec847b7161f3c 2019-03-06 08:11:51
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 94672e30258fcf08bbb11f1eaa8fb288         format
##  2: 94672e30258fcf08bbb11f1eaa8fb288           name
##  3: 94672e30258fcf08bbb11f1eaa8fb288          class
##  4: 94672e30258fcf08bbb11f1eaa8fb288           date
##  5: 94672e30258fcf08bbb11f1eaa8fb288        cacheId
##  6: 94672e30258fcf08bbb11f1eaa8fb288     objectName
##  7: 94672e30258fcf08bbb11f1eaa8fb288       function
##  8: 94672e30258fcf08bbb11f1eaa8fb288    object.size
##  9: 94672e30258fcf08bbb11f1eaa8fb288       accessed
## 10: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 11: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 12: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 13: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 14: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 15: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 16: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 17: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 18: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 19: 94672e30258fcf08bbb11f1eaa8fb288 otherFunctions
## 20: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
## 21: 94672e30258fcf08bbb11f1eaa8fb288      preDigest
## 22: a6d0f9d152800840ededdcb85bbfad86         format
## 23: a6d0f9d152800840ededdcb85bbfad86           name
## 24: a6d0f9d152800840ededdcb85bbfad86          class
## 25: a6d0f9d152800840ededdcb85bbfad86           date
## 26: a6d0f9d152800840ededdcb85bbfad86        cacheId
## 27: a6d0f9d152800840ededdcb85bbfad86       function
## 28: a6d0f9d152800840ededdcb85bbfad86    object.size
## 29: a6d0f9d152800840ededdcb85bbfad86       accessed
## 30: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 31: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 32: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 33: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 34: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 35: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 36: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 37: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 38: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 39: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 40: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 41: a6d0f9d152800840ededdcb85bbfad86 otherFunctions
## 42: a6d0f9d152800840ededdcb85bbfad86      preDigest
## 43: a6d0f9d152800840ededdcb85bbfad86      preDigest
## 44: a6d0f9d152800840ededdcb85bbfad86      preDigest
## 45: a722f6db988bbd97fe186bb1b06ef160         format
## 46: a722f6db988bbd97fe186bb1b06ef160           name
## 47: a722f6db988bbd97fe186bb1b06ef160          class
## 48: a722f6db988bbd97fe186bb1b06ef160           date
## 49: a722f6db988bbd97fe186bb1b06ef160        cacheId
## 50: a722f6db988bbd97fe186bb1b06ef160       function
## 51: a722f6db988bbd97fe186bb1b06ef160    object.size
## 52: a722f6db988bbd97fe186bb1b06ef160       accessed
## 53: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 54: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 55: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 56: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 57: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 58: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 59: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 60: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 61: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 62: a722f6db988bbd97fe186bb1b06ef160 otherFunctions
## 63: a722f6db988bbd97fe186bb1b06ef160      preDigest
## 64: a722f6db988bbd97fe186bb1b06ef160      preDigest
## 65: a722f6db988bbd97fe186bb1b06ef160      preDigest
## 66: a722f6db988bbd97fe186bb1b06ef160       accessed
## 67: a722f6db988bbd97fe186bb1b06ef160       accessed
## 68: a722f6db988bbd97fe186bb1b06ef160       accessed
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:51
##  2: 94672e30258fcf08bbb11f1eaa8fb288 2019-03-06 08:11:51
##  3:                          numeric 2019-03-06 08:11:51
##  4:              2019-03-06 08:11:51 2019-03-06 08:11:51
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:51
##  6:                                a 2019-03-06 08:11:51
##  7:                            rnorm 2019-03-06 08:11:51
##  8:                             1008 2019-03-06 08:11:51
##  9:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 10:                 vweave_rmarkdown 2019-03-06 08:11:51
## 11:                     process_file 2019-03-06 08:11:51
## 12:                    process_group 2019-03-06 08:11:51
## 13:              process_group.block 2019-03-06 08:11:51
## 14:                       call_block 2019-03-06 08:11:51
## 15:                       block_exec 2019-03-06 08:11:51
## 16:                           in_dir 2019-03-06 08:11:51
## 17:                        timing_fn 2019-03-06 08:11:51
## 18:                           handle 2019-03-06 08:11:51
## 19:                      withVisible 2019-03-06 08:11:51
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:51
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:51
## 22:                              rda 2019-03-06 08:11:51
## 23: a6d0f9d152800840ededdcb85bbfad86 2019-03-06 08:11:51
## 24:                          numeric 2019-03-06 08:11:51
## 25:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 26:                 c16f0e2c57596069 2019-03-06 08:11:51
## 27:              rnorm pipe sequence 2019-03-06 08:11:51
## 28:                             1104 2019-03-06 08:11:51
## 29:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 30:                 vweave_rmarkdown 2019-03-06 08:11:51
## 31:                     process_file 2019-03-06 08:11:51
## 32:                    process_group 2019-03-06 08:11:51
## 33:              process_group.block 2019-03-06 08:11:51
## 34:                       call_block 2019-03-06 08:11:51
## 35:                       block_exec 2019-03-06 08:11:51
## 36:                           in_dir 2019-03-06 08:11:51
## 37:                        timing_fn 2019-03-06 08:11:51
## 38:                           handle 2019-03-06 08:11:51
## 39:                      withVisible 2019-03-06 08:11:51
## 40:                              %C% 2019-03-06 08:11:51
## 41:                          do.call 2019-03-06 08:11:51
## 42:               n:c5775c3b366fb719 2019-03-06 08:11:51
## 43:            mean:152602b8ff81e5bb 2019-03-06 08:11:51
## 44:            .FUN:4f604aa46882b368 2019-03-06 08:11:51
## 45:                              rda 2019-03-06 08:11:50
## 46: a722f6db988bbd97fe186bb1b06ef160 2019-03-06 08:11:50
## 47:                          numeric 2019-03-06 08:11:50
## 48:              2019-03-06 08:11:50 2019-03-06 08:11:50
## 49:                 f11fb1a2880f8060 2019-03-06 08:11:50
## 50:                            rnorm 2019-03-06 08:11:50
## 51:                             1104 2019-03-06 08:11:50
## 52:              2019-03-06 08:11:50 2019-03-06 08:11:50
## 53:                 vweave_rmarkdown 2019-03-06 08:11:50
## 54:                     process_file 2019-03-06 08:11:50
## 55:                    process_group 2019-03-06 08:11:50
## 56:              process_group.block 2019-03-06 08:11:50
## 57:                       call_block 2019-03-06 08:11:51
## 58:                       block_exec 2019-03-06 08:11:51
## 59:                           in_dir 2019-03-06 08:11:51
## 60:                        timing_fn 2019-03-06 08:11:51
## 61:                           handle 2019-03-06 08:11:51
## 62:                      withVisible 2019-03-06 08:11:51
## 63:               n:c5775c3b366fb719 2019-03-06 08:11:51
## 64:            mean:15620f138033a66c 2019-03-06 08:11:51
## 65:            .FUN:4f604aa46882b368 2019-03-06 08:11:51
## 66:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 67:              2019-03-06 08:11:51 2019-03-06 08:11:51
## 68:              2019-03-06 08:11:51 2019-03-06 08:11:51
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.5 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

1.6 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 440551f39de4efc0bdba3d28b5f43e01         format
##  2: 440551f39de4efc0bdba3d28b5f43e01           name
##  3: 440551f39de4efc0bdba3d28b5f43e01          class
##  4: 440551f39de4efc0bdba3d28b5f43e01           date
##  5: 440551f39de4efc0bdba3d28b5f43e01        cacheId
##  6: 440551f39de4efc0bdba3d28b5f43e01     objectName
##  7: 440551f39de4efc0bdba3d28b5f43e01       function
##  8: 440551f39de4efc0bdba3d28b5f43e01    object.size
##  9: 440551f39de4efc0bdba3d28b5f43e01       accessed
## 10: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 11: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 12: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 13: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 14: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 15: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 16: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 17: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 18: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 19: 440551f39de4efc0bdba3d28b5f43e01 otherFunctions
## 20: 440551f39de4efc0bdba3d28b5f43e01      preDigest
## 21: 440551f39de4efc0bdba3d28b5f43e01      preDigest
## 22: 769ce0cefb8b1efd10519de6ab0c7cf2         format
## 23: 769ce0cefb8b1efd10519de6ab0c7cf2           name
## 24: 769ce0cefb8b1efd10519de6ab0c7cf2          class
## 25: 769ce0cefb8b1efd10519de6ab0c7cf2           date
## 26: 769ce0cefb8b1efd10519de6ab0c7cf2        cacheId
## 27: 769ce0cefb8b1efd10519de6ab0c7cf2     objectName
## 28: 769ce0cefb8b1efd10519de6ab0c7cf2       function
## 29: 769ce0cefb8b1efd10519de6ab0c7cf2    object.size
## 30: 769ce0cefb8b1efd10519de6ab0c7cf2       accessed
## 31: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 32: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 33: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 34: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 35: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 36: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 37: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 38: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 39: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 40: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 41: 769ce0cefb8b1efd10519de6ab0c7cf2      preDigest
## 42: 769ce0cefb8b1efd10519de6ab0c7cf2      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:53
##  2: 440551f39de4efc0bdba3d28b5f43e01 2019-03-06 08:11:53
##  3:                          numeric 2019-03-06 08:11:53
##  4:              2019-03-06 08:11:53 2019-03-06 08:11:53
##  5:                 3aef38d1fc02aee5 2019-03-06 08:11:53
##  6:                                b 2019-03-06 08:11:53
##  7:                            runif 2019-03-06 08:11:53
##  8:                             1008 2019-03-06 08:11:53
##  9:              2019-03-06 08:11:53 2019-03-06 08:11:53
## 10:                 vweave_rmarkdown 2019-03-06 08:11:53
## 11:                     process_file 2019-03-06 08:11:53
## 12:                    process_group 2019-03-06 08:11:53
## 13:              process_group.block 2019-03-06 08:11:53
## 14:                       call_block 2019-03-06 08:11:53
## 15:                       block_exec 2019-03-06 08:11:53
## 16:                           in_dir 2019-03-06 08:11:53
## 17:                        timing_fn 2019-03-06 08:11:53
## 18:                           handle 2019-03-06 08:11:53
## 19:                      withVisible 2019-03-06 08:11:53
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:53
## 21:            .FUN:881ec847b7161f3c 2019-03-06 08:11:53
## 22:                              rda 2019-03-06 08:11:52
## 23: 769ce0cefb8b1efd10519de6ab0c7cf2 2019-03-06 08:11:52
## 24:                          numeric 2019-03-06 08:11:53
## 25:              2019-03-06 08:11:52 2019-03-06 08:11:53
## 26:                 f7bee22047b8d0c1 2019-03-06 08:11:53
## 27:                                a 2019-03-06 08:11:53
## 28:                            rnorm 2019-03-06 08:11:53
## 29:                             1008 2019-03-06 08:11:53
## 30:              2019-03-06 08:11:52 2019-03-06 08:11:53
## 31:                 vweave_rmarkdown 2019-03-06 08:11:53
## 32:                     process_file 2019-03-06 08:11:53
## 33:                    process_group 2019-03-06 08:11:53
## 34:              process_group.block 2019-03-06 08:11:53
## 35:                       call_block 2019-03-06 08:11:53
## 36:                       block_exec 2019-03-06 08:11:53
## 37:                           in_dir 2019-03-06 08:11:53
## 38:                        timing_fn 2019-03-06 08:11:53
## 39:                           handle 2019-03-06 08:11:53
## 40:                      withVisible 2019-03-06 08:11:53
## 41:               n:7eef4eae85fd9229 2019-03-06 08:11:53
## 42:            .FUN:4f604aa46882b368 2019-03-06 08:11:53
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 769ce0cefb8b1efd10519de6ab0c7cf2         format
##  2: 769ce0cefb8b1efd10519de6ab0c7cf2           name
##  3: 769ce0cefb8b1efd10519de6ab0c7cf2          class
##  4: 769ce0cefb8b1efd10519de6ab0c7cf2           date
##  5: 769ce0cefb8b1efd10519de6ab0c7cf2        cacheId
##  6: 769ce0cefb8b1efd10519de6ab0c7cf2     objectName
##  7: 769ce0cefb8b1efd10519de6ab0c7cf2       function
##  8: 769ce0cefb8b1efd10519de6ab0c7cf2    object.size
##  9: 769ce0cefb8b1efd10519de6ab0c7cf2       accessed
## 10: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 11: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 12: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 13: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 14: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 15: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 16: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 17: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 18: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 19: 769ce0cefb8b1efd10519de6ab0c7cf2 otherFunctions
## 20: 769ce0cefb8b1efd10519de6ab0c7cf2      preDigest
## 21: 769ce0cefb8b1efd10519de6ab0c7cf2      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:52
##  2: 769ce0cefb8b1efd10519de6ab0c7cf2 2019-03-06 08:11:52
##  3:                          numeric 2019-03-06 08:11:53
##  4:              2019-03-06 08:11:52 2019-03-06 08:11:53
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:53
##  6:                                a 2019-03-06 08:11:53
##  7:                            rnorm 2019-03-06 08:11:53
##  8:                             1008 2019-03-06 08:11:53
##  9:              2019-03-06 08:11:52 2019-03-06 08:11:53
## 10:                 vweave_rmarkdown 2019-03-06 08:11:53
## 11:                     process_file 2019-03-06 08:11:53
## 12:                    process_group 2019-03-06 08:11:53
## 13:              process_group.block 2019-03-06 08:11:53
## 14:                       call_block 2019-03-06 08:11:53
## 15:                       block_exec 2019-03-06 08:11:53
## 16:                           in_dir 2019-03-06 08:11:53
## 17:                        timing_fn 2019-03-06 08:11:53
## 18:                           handle 2019-03-06 08:11:53
## 19:                      withVisible 2019-03-06 08:11:53
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:53
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:53
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

1.7 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 45db70ddf39f63e6faf37f3bf825e8ee         format
##  2: 45db70ddf39f63e6faf37f3bf825e8ee           name
##  3: 45db70ddf39f63e6faf37f3bf825e8ee          class
##  4: 45db70ddf39f63e6faf37f3bf825e8ee           date
##  5: 45db70ddf39f63e6faf37f3bf825e8ee        cacheId
##  6: 45db70ddf39f63e6faf37f3bf825e8ee     objectName
##  7: 45db70ddf39f63e6faf37f3bf825e8ee       function
##  8: 45db70ddf39f63e6faf37f3bf825e8ee    object.size
##  9: 45db70ddf39f63e6faf37f3bf825e8ee       accessed
## 10: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 11: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 12: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 13: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 14: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 15: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 16: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 17: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 18: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 19: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 20: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 21: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 22: fa08a4894ab5972bc8741c148b688318         format
## 23: fa08a4894ab5972bc8741c148b688318           name
## 24: fa08a4894ab5972bc8741c148b688318          class
## 25: fa08a4894ab5972bc8741c148b688318           date
## 26: fa08a4894ab5972bc8741c148b688318        cacheId
## 27: fa08a4894ab5972bc8741c148b688318     objectName
## 28: fa08a4894ab5972bc8741c148b688318       function
## 29: fa08a4894ab5972bc8741c148b688318    object.size
## 30: fa08a4894ab5972bc8741c148b688318       accessed
## 31: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 32: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 33: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 34: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 35: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 36: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 37: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 38: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 39: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 40: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 41: fa08a4894ab5972bc8741c148b688318      preDigest
## 42: fa08a4894ab5972bc8741c148b688318      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:54
##  2: 45db70ddf39f63e6faf37f3bf825e8ee 2019-03-06 08:11:54
##  3:                          numeric 2019-03-06 08:11:54
##  4:              2019-03-06 08:11:54 2019-03-06 08:11:54
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:54
##  6:                                b 2019-03-06 08:11:54
##  7:                            rnorm 2019-03-06 08:11:54
##  8:                             1008 2019-03-06 08:11:54
##  9:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 10:                 vweave_rmarkdown 2019-03-06 08:11:54
## 11:                     process_file 2019-03-06 08:11:54
## 12:                    process_group 2019-03-06 08:11:54
## 13:              process_group.block 2019-03-06 08:11:54
## 14:                       call_block 2019-03-06 08:11:54
## 15:                       block_exec 2019-03-06 08:11:54
## 16:                           in_dir 2019-03-06 08:11:54
## 17:                        timing_fn 2019-03-06 08:11:54
## 18:                           handle 2019-03-06 08:11:54
## 19:                      withVisible 2019-03-06 08:11:54
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:54
## 22:                              rda 2019-03-06 08:11:54
## 23: fa08a4894ab5972bc8741c148b688318 2019-03-06 08:11:54
## 24:                          numeric 2019-03-06 08:11:54
## 25:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 26:                 3aef38d1fc02aee5 2019-03-06 08:11:54
## 27:                                a 2019-03-06 08:11:54
## 28:                            runif 2019-03-06 08:11:54
## 29:                             1008 2019-03-06 08:11:54
## 30:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 31:                 vweave_rmarkdown 2019-03-06 08:11:54
## 32:                     process_file 2019-03-06 08:11:54
## 33:                    process_group 2019-03-06 08:11:54
## 34:              process_group.block 2019-03-06 08:11:54
## 35:                       call_block 2019-03-06 08:11:54
## 36:                       block_exec 2019-03-06 08:11:54
## 37:                           in_dir 2019-03-06 08:11:54
## 38:                        timing_fn 2019-03-06 08:11:54
## 39:                           handle 2019-03-06 08:11:54
## 40:                      withVisible 2019-03-06 08:11:54
## 41:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 42:            .FUN:881ec847b7161f3c 2019-03-06 08:11:54
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 45db70ddf39f63e6faf37f3bf825e8ee         format
##  2: 45db70ddf39f63e6faf37f3bf825e8ee           name
##  3: 45db70ddf39f63e6faf37f3bf825e8ee          class
##  4: 45db70ddf39f63e6faf37f3bf825e8ee           date
##  5: 45db70ddf39f63e6faf37f3bf825e8ee        cacheId
##  6: 45db70ddf39f63e6faf37f3bf825e8ee     objectName
##  7: 45db70ddf39f63e6faf37f3bf825e8ee       function
##  8: 45db70ddf39f63e6faf37f3bf825e8ee    object.size
##  9: 45db70ddf39f63e6faf37f3bf825e8ee       accessed
## 10: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 11: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 12: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 13: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 14: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 15: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 16: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 17: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 18: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 19: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 20: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 21: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 22: fa08a4894ab5972bc8741c148b688318         format
## 23: fa08a4894ab5972bc8741c148b688318           name
## 24: fa08a4894ab5972bc8741c148b688318          class
## 25: fa08a4894ab5972bc8741c148b688318           date
## 26: fa08a4894ab5972bc8741c148b688318        cacheId
## 27: fa08a4894ab5972bc8741c148b688318     objectName
## 28: fa08a4894ab5972bc8741c148b688318       function
## 29: fa08a4894ab5972bc8741c148b688318    object.size
## 30: fa08a4894ab5972bc8741c148b688318       accessed
## 31: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 32: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 33: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 34: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 35: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 36: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 37: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 38: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 39: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 40: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 41: fa08a4894ab5972bc8741c148b688318      preDigest
## 42: fa08a4894ab5972bc8741c148b688318      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:54
##  2: 45db70ddf39f63e6faf37f3bf825e8ee 2019-03-06 08:11:54
##  3:                          numeric 2019-03-06 08:11:54
##  4:              2019-03-06 08:11:54 2019-03-06 08:11:54
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:54
##  6:                                b 2019-03-06 08:11:54
##  7:                            rnorm 2019-03-06 08:11:54
##  8:                             1008 2019-03-06 08:11:54
##  9:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 10:                 vweave_rmarkdown 2019-03-06 08:11:54
## 11:                     process_file 2019-03-06 08:11:54
## 12:                    process_group 2019-03-06 08:11:54
## 13:              process_group.block 2019-03-06 08:11:54
## 14:                       call_block 2019-03-06 08:11:54
## 15:                       block_exec 2019-03-06 08:11:54
## 16:                           in_dir 2019-03-06 08:11:54
## 17:                        timing_fn 2019-03-06 08:11:54
## 18:                           handle 2019-03-06 08:11:54
## 19:                      withVisible 2019-03-06 08:11:54
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:54
## 22:                              rda 2019-03-06 08:11:54
## 23: fa08a4894ab5972bc8741c148b688318 2019-03-06 08:11:54
## 24:                          numeric 2019-03-06 08:11:54
## 25:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 26:                 3aef38d1fc02aee5 2019-03-06 08:11:54
## 27:                                a 2019-03-06 08:11:54
## 28:                            runif 2019-03-06 08:11:54
## 29:                             1008 2019-03-06 08:11:54
## 30:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 31:                 vweave_rmarkdown 2019-03-06 08:11:54
## 32:                     process_file 2019-03-06 08:11:54
## 33:                    process_group 2019-03-06 08:11:54
## 34:              process_group.block 2019-03-06 08:11:54
## 35:                       call_block 2019-03-06 08:11:54
## 36:                       block_exec 2019-03-06 08:11:54
## 37:                           in_dir 2019-03-06 08:11:54
## 38:                        timing_fn 2019-03-06 08:11:54
## 39:                           handle 2019-03-06 08:11:54
## 40:                      withVisible 2019-03-06 08:11:54
## 41:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 42:            .FUN:881ec847b7161f3c 2019-03-06 08:11:54
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 45db70ddf39f63e6faf37f3bf825e8ee         format
##  2: 45db70ddf39f63e6faf37f3bf825e8ee           name
##  3: 45db70ddf39f63e6faf37f3bf825e8ee          class
##  4: 45db70ddf39f63e6faf37f3bf825e8ee           date
##  5: 45db70ddf39f63e6faf37f3bf825e8ee        cacheId
##  6: 45db70ddf39f63e6faf37f3bf825e8ee     objectName
##  7: 45db70ddf39f63e6faf37f3bf825e8ee       function
##  8: 45db70ddf39f63e6faf37f3bf825e8ee    object.size
##  9: 45db70ddf39f63e6faf37f3bf825e8ee       accessed
## 10: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 11: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 12: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 13: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 14: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 15: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 16: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 17: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 18: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 19: 45db70ddf39f63e6faf37f3bf825e8ee otherFunctions
## 20: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 21: 45db70ddf39f63e6faf37f3bf825e8ee      preDigest
## 22: fa08a4894ab5972bc8741c148b688318         format
## 23: fa08a4894ab5972bc8741c148b688318           name
## 24: fa08a4894ab5972bc8741c148b688318          class
## 25: fa08a4894ab5972bc8741c148b688318           date
## 26: fa08a4894ab5972bc8741c148b688318        cacheId
## 27: fa08a4894ab5972bc8741c148b688318     objectName
## 28: fa08a4894ab5972bc8741c148b688318       function
## 29: fa08a4894ab5972bc8741c148b688318    object.size
## 30: fa08a4894ab5972bc8741c148b688318       accessed
## 31: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 32: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 33: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 34: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 35: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 36: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 37: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 38: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 39: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 40: fa08a4894ab5972bc8741c148b688318 otherFunctions
## 41: fa08a4894ab5972bc8741c148b688318      preDigest
## 42: fa08a4894ab5972bc8741c148b688318      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:54
##  2: 45db70ddf39f63e6faf37f3bf825e8ee 2019-03-06 08:11:54
##  3:                          numeric 2019-03-06 08:11:54
##  4:              2019-03-06 08:11:54 2019-03-06 08:11:54
##  5:                 f7bee22047b8d0c1 2019-03-06 08:11:54
##  6:                                b 2019-03-06 08:11:54
##  7:                            rnorm 2019-03-06 08:11:54
##  8:                             1008 2019-03-06 08:11:54
##  9:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 10:                 vweave_rmarkdown 2019-03-06 08:11:54
## 11:                     process_file 2019-03-06 08:11:54
## 12:                    process_group 2019-03-06 08:11:54
## 13:              process_group.block 2019-03-06 08:11:54
## 14:                       call_block 2019-03-06 08:11:54
## 15:                       block_exec 2019-03-06 08:11:54
## 16:                           in_dir 2019-03-06 08:11:54
## 17:                        timing_fn 2019-03-06 08:11:54
## 18:                           handle 2019-03-06 08:11:54
## 19:                      withVisible 2019-03-06 08:11:54
## 20:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 21:            .FUN:4f604aa46882b368 2019-03-06 08:11:54
## 22:                              rda 2019-03-06 08:11:54
## 23: fa08a4894ab5972bc8741c148b688318 2019-03-06 08:11:54
## 24:                          numeric 2019-03-06 08:11:54
## 25:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 26:                 3aef38d1fc02aee5 2019-03-06 08:11:54
## 27:                                a 2019-03-06 08:11:54
## 28:                            runif 2019-03-06 08:11:54
## 29:                             1008 2019-03-06 08:11:54
## 30:              2019-03-06 08:11:54 2019-03-06 08:11:54
## 31:                 vweave_rmarkdown 2019-03-06 08:11:54
## 32:                     process_file 2019-03-06 08:11:54
## 33:                    process_group 2019-03-06 08:11:54
## 34:              process_group.block 2019-03-06 08:11:54
## 35:                       call_block 2019-03-06 08:11:54
## 36:                       block_exec 2019-03-06 08:11:54
## 37:                           in_dir 2019-03-06 08:11:54
## 38:                        timing_fn 2019-03-06 08:11:54
## 39:                           handle 2019-03-06 08:11:54
## 40:                      withVisible 2019-03-06 08:11:54
## 41:               n:7eef4eae85fd9229 2019-03-06 08:11:54
## 42:            .FUN:881ec847b7161f3c 2019-03-06 08:11:54
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.8 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.9 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  0.2691780 -0.3862321  0.2322681
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 542422f12983e1a91d0953ee346d81af         format
##  2: 542422f12983e1a91d0953ee346d81af           name
##  3: 542422f12983e1a91d0953ee346d81af          class
##  4: 542422f12983e1a91d0953ee346d81af           date
##  5: 542422f12983e1a91d0953ee346d81af        cacheId
##  6: 542422f12983e1a91d0953ee346d81af       function
##  7: 542422f12983e1a91d0953ee346d81af    object.size
##  8: 542422f12983e1a91d0953ee346d81af       accessed
##  9: 542422f12983e1a91d0953ee346d81af otherFunctions
## 10: 542422f12983e1a91d0953ee346d81af otherFunctions
## 11: 542422f12983e1a91d0953ee346d81af otherFunctions
## 12: 542422f12983e1a91d0953ee346d81af otherFunctions
## 13: 542422f12983e1a91d0953ee346d81af otherFunctions
## 14: 542422f12983e1a91d0953ee346d81af otherFunctions
## 15: 542422f12983e1a91d0953ee346d81af otherFunctions
## 16: 542422f12983e1a91d0953ee346d81af otherFunctions
## 17: 542422f12983e1a91d0953ee346d81af otherFunctions
## 18: 542422f12983e1a91d0953ee346d81af otherFunctions
## 19: 542422f12983e1a91d0953ee346d81af      preDigest
## 20: 542422f12983e1a91d0953ee346d81af      preDigest
## 21: f7e729e282e7fb45185efca4e8c461b7         format
## 22: f7e729e282e7fb45185efca4e8c461b7           name
## 23: f7e729e282e7fb45185efca4e8c461b7          class
## 24: f7e729e282e7fb45185efca4e8c461b7           date
## 25: f7e729e282e7fb45185efca4e8c461b7        cacheId
## 26: f7e729e282e7fb45185efca4e8c461b7       function
## 27: f7e729e282e7fb45185efca4e8c461b7    object.size
## 28: f7e729e282e7fb45185efca4e8c461b7       accessed
## 29: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 30: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 31: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 32: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 33: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 34: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 35: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 36: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 37: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 38: f7e729e282e7fb45185efca4e8c461b7 otherFunctions
## 39: f7e729e282e7fb45185efca4e8c461b7      preDigest
## 40: f7e729e282e7fb45185efca4e8c461b7      preDigest
## 41: f7e729e282e7fb45185efca4e8c461b7      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:55
##  2: 542422f12983e1a91d0953ee346d81af 2019-03-06 08:11:55
##  3:                          numeric 2019-03-06 08:11:55
##  4:              2019-03-06 08:11:55 2019-03-06 08:11:55
##  5:                 a7af5367c13aba8f 2019-03-06 08:11:55
##  6:                            outer 2019-03-06 08:11:55
##  7:                             1008 2019-03-06 08:11:55
##  8:              2019-03-06 08:11:55 2019-03-06 08:11:55
##  9:                 vweave_rmarkdown 2019-03-06 08:11:55
## 10:                     process_file 2019-03-06 08:11:55
## 11:                    process_group 2019-03-06 08:11:55
## 12:              process_group.block 2019-03-06 08:11:55
## 13:                       call_block 2019-03-06 08:11:55
## 14:                       block_exec 2019-03-06 08:11:55
## 15:                           in_dir 2019-03-06 08:11:55
## 16:                        timing_fn 2019-03-06 08:11:55
## 17:                           handle 2019-03-06 08:11:55
## 18:                      withVisible 2019-03-06 08:11:55
## 19:               n:82dc709f2b91918a 2019-03-06 08:11:55
## 20:            .FUN:892a6afc47a63a90 2019-03-06 08:11:55
## 21:                              rda 2019-03-06 08:11:55
## 22: f7e729e282e7fb45185efca4e8c461b7 2019-03-06 08:11:55
## 23:                          numeric 2019-03-06 08:11:55
## 24:              2019-03-06 08:11:55 2019-03-06 08:11:55
## 25:                 4ac2b7c0f42d1e46 2019-03-06 08:11:55
## 26:                            rnorm 2019-03-06 08:11:55
## 27:                             1008 2019-03-06 08:11:55
## 28:              2019-03-06 08:11:55 2019-03-06 08:11:55
## 29:                 vweave_rmarkdown 2019-03-06 08:11:55
## 30:                     process_file 2019-03-06 08:11:55
## 31:                    process_group 2019-03-06 08:11:55
## 32:              process_group.block 2019-03-06 08:11:55
## 33:                       call_block 2019-03-06 08:11:55
## 34:                       block_exec 2019-03-06 08:11:55
## 35:                           in_dir 2019-03-06 08:11:55
## 36:                        timing_fn 2019-03-06 08:11:55
## 37:                           handle 2019-03-06 08:11:55
## 38:                      withVisible 2019-03-06 08:11:55
## 39:               n:7f12988bd88a0fb8 2019-03-06 08:11:55
## 40:            mean:22413394efd9f6a3 2019-03-06 08:11:55
## 41:            .FUN:4f604aa46882b368 2019-03-06 08:11:55
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 2b6e1f8dd0ef02c961ecada479b6e5f8         format
##  2: 2b6e1f8dd0ef02c961ecada479b6e5f8           name
##  3: 2b6e1f8dd0ef02c961ecada479b6e5f8          class
##  4: 2b6e1f8dd0ef02c961ecada479b6e5f8           date
##  5: 2b6e1f8dd0ef02c961ecada479b6e5f8        cacheId
##  6: 2b6e1f8dd0ef02c961ecada479b6e5f8       function
##  7: 2b6e1f8dd0ef02c961ecada479b6e5f8    object.size
##  8: 2b6e1f8dd0ef02c961ecada479b6e5f8       accessed
##  9: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 10: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 11: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 12: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 13: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 14: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 15: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 16: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 17: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 18: 2b6e1f8dd0ef02c961ecada479b6e5f8 otherFunctions
## 19: 2b6e1f8dd0ef02c961ecada479b6e5f8      preDigest
## 20: 2b6e1f8dd0ef02c961ecada479b6e5f8      preDigest
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:55
##  2: 2b6e1f8dd0ef02c961ecada479b6e5f8 2019-03-06 08:11:55
##  3:                          numeric 2019-03-06 08:11:55
##  4:              2019-03-06 08:11:55 2019-03-06 08:11:55
##  5:                 33ceb4fb525fd08f 2019-03-06 08:11:55
##  6:                            inner 2019-03-06 08:11:55
##  7:                             1008 2019-03-06 08:11:55
##  8:              2019-03-06 08:11:55 2019-03-06 08:11:55
##  9:                 vweave_rmarkdown 2019-03-06 08:11:55
## 10:                     process_file 2019-03-06 08:11:55
## 11:                    process_group 2019-03-06 08:11:55
## 12:              process_group.block 2019-03-06 08:11:55
## 13:                       call_block 2019-03-06 08:11:55
## 14:                       block_exec 2019-03-06 08:11:55
## 15:                           in_dir 2019-03-06 08:11:55
## 16:                        timing_fn 2019-03-06 08:11:55
## 17:                           handle 2019-03-06 08:11:55
## 18:                      withVisible 2019-03-06 08:11:55
## 19:            mean:22413394efd9f6a3 2019-03-06 08:11:55
## 20:            .FUN:87e2c30917a34d25 2019-03-06 08:11:55
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 6cb9ea27e56eb41581a6dd2ba9cbe3e9         format
##  2: 6cb9ea27e56eb41581a6dd2ba9cbe3e9           name
##  3: 6cb9ea27e56eb41581a6dd2ba9cbe3e9          class
##  4: 6cb9ea27e56eb41581a6dd2ba9cbe3e9           date
##  5: 6cb9ea27e56eb41581a6dd2ba9cbe3e9        cacheId
##  6: 6cb9ea27e56eb41581a6dd2ba9cbe3e9       outerTag
##  7: 6cb9ea27e56eb41581a6dd2ba9cbe3e9       function
##  8: 6cb9ea27e56eb41581a6dd2ba9cbe3e9    object.size
##  9: 6cb9ea27e56eb41581a6dd2ba9cbe3e9       accessed
## 10: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 11: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 12: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 13: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 14: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 15: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 16: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 17: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 18: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 19: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 otherFunctions
## 20: 6cb9ea27e56eb41581a6dd2ba9cbe3e9      preDigest
## 21: 6cb9ea27e56eb41581a6dd2ba9cbe3e9      preDigest
## 22: ca2c959635df557ca0aee50f33d5f2d8         format
## 23: ca2c959635df557ca0aee50f33d5f2d8           name
## 24: ca2c959635df557ca0aee50f33d5f2d8          class
## 25: ca2c959635df557ca0aee50f33d5f2d8           date
## 26: ca2c959635df557ca0aee50f33d5f2d8        cacheId
## 27: ca2c959635df557ca0aee50f33d5f2d8       innerTag
## 28: ca2c959635df557ca0aee50f33d5f2d8       outerTag
## 29: ca2c959635df557ca0aee50f33d5f2d8       function
## 30: ca2c959635df557ca0aee50f33d5f2d8    object.size
## 31: ca2c959635df557ca0aee50f33d5f2d8       accessed
## 32: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 33: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 34: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 35: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 36: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 37: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 38: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 39: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 40: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 41: ca2c959635df557ca0aee50f33d5f2d8 otherFunctions
## 42: ca2c959635df557ca0aee50f33d5f2d8      preDigest
## 43: ca2c959635df557ca0aee50f33d5f2d8      preDigest
## 44: ca2c959635df557ca0aee50f33d5f2d8      preDigest
## 45: e5fd27dec663c914c6c529dd263c5ed7         format
## 46: e5fd27dec663c914c6c529dd263c5ed7           name
## 47: e5fd27dec663c914c6c529dd263c5ed7          class
## 48: e5fd27dec663c914c6c529dd263c5ed7           date
## 49: e5fd27dec663c914c6c529dd263c5ed7        cacheId
## 50: e5fd27dec663c914c6c529dd263c5ed7       outerTag
## 51: e5fd27dec663c914c6c529dd263c5ed7       function
## 52: e5fd27dec663c914c6c529dd263c5ed7    object.size
## 53: e5fd27dec663c914c6c529dd263c5ed7       accessed
## 54: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 55: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 56: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 57: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 58: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 59: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 60: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 61: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 62: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 63: e5fd27dec663c914c6c529dd263c5ed7 otherFunctions
## 64: e5fd27dec663c914c6c529dd263c5ed7      preDigest
## 65: e5fd27dec663c914c6c529dd263c5ed7      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-06 08:11:56
##  2: 6cb9ea27e56eb41581a6dd2ba9cbe3e9 2019-03-06 08:11:56
##  3:                          numeric 2019-03-06 08:11:56
##  4:              2019-03-06 08:11:56 2019-03-06 08:11:56
##  5:                 88a34e1d033329e5 2019-03-06 08:11:56
##  6:                         outerTag 2019-03-06 08:11:56
##  7:                            outer 2019-03-06 08:11:56
##  8:                             1008 2019-03-06 08:11:56
##  9:              2019-03-06 08:11:56 2019-03-06 08:11:56
## 10:                 vweave_rmarkdown 2019-03-06 08:11:56
## 11:                     process_file 2019-03-06 08:11:56
## 12:                    process_group 2019-03-06 08:11:56
## 13:              process_group.block 2019-03-06 08:11:56
## 14:                       call_block 2019-03-06 08:11:56
## 15:                       block_exec 2019-03-06 08:11:56
## 16:                           in_dir 2019-03-06 08:11:56
## 17:                        timing_fn 2019-03-06 08:11:56
## 18:                           handle 2019-03-06 08:11:56
## 19:                      withVisible 2019-03-06 08:11:56
## 20:               n:82dc709f2b91918a 2019-03-06 08:11:56
## 21:            .FUN:5f06fb5fbffe9e3b 2019-03-06 08:11:56
## 22:                              rda 2019-03-06 08:11:56
## 23: ca2c959635df557ca0aee50f33d5f2d8 2019-03-06 08:11:56
## 24:                          numeric 2019-03-06 08:11:56
## 25:              2019-03-06 08:11:56 2019-03-06 08:11:56
## 26:                 4ac2b7c0f42d1e46 2019-03-06 08:11:56
## 27:                         innerTag 2019-03-06 08:11:56
## 28:                         outerTag 2019-03-06 08:11:56
## 29:                            rnorm 2019-03-06 08:11:56
## 30:                             1008 2019-03-06 08:11:56
## 31:              2019-03-06 08:11:56 2019-03-06 08:11:56
## 32:                 vweave_rmarkdown 2019-03-06 08:11:56
## 33:                     process_file 2019-03-06 08:11:56
## 34:                    process_group 2019-03-06 08:11:56
## 35:              process_group.block 2019-03-06 08:11:56
## 36:                       call_block 2019-03-06 08:11:56
## 37:                       block_exec 2019-03-06 08:11:56
## 38:                           in_dir 2019-03-06 08:11:56
## 39:                        timing_fn 2019-03-06 08:11:56
## 40:                           handle 2019-03-06 08:11:56
## 41:                      withVisible 2019-03-06 08:11:56
## 42:               n:7f12988bd88a0fb8 2019-03-06 08:11:56
## 43:            mean:22413394efd9f6a3 2019-03-06 08:11:56
## 44:            .FUN:4f604aa46882b368 2019-03-06 08:11:56
## 45:                              rda 2019-03-06 08:11:56
## 46: e5fd27dec663c914c6c529dd263c5ed7 2019-03-06 08:11:56
## 47:                          numeric 2019-03-06 08:11:56
## 48:              2019-03-06 08:11:56 2019-03-06 08:11:56
## 49:                 b06af03d5a73dc7d 2019-03-06 08:11:56
## 50:                         outerTag 2019-03-06 08:11:56
## 51:                            inner 2019-03-06 08:11:56
## 52:                             1008 2019-03-06 08:11:56
## 53:              2019-03-06 08:11:56 2019-03-06 08:11:56
## 54:                 vweave_rmarkdown 2019-03-06 08:11:56
## 55:                     process_file 2019-03-06 08:11:56
## 56:                    process_group 2019-03-06 08:11:56
## 57:              process_group.block 2019-03-06 08:11:56
## 58:                       call_block 2019-03-06 08:11:56
## 59:                       block_exec 2019-03-06 08:11:56
## 60:                           in_dir 2019-03-06 08:11:56
## 61:                        timing_fn 2019-03-06 08:11:56
## 62:                           handle 2019-03-06 08:11:56
## 63:                      withVisible 2019-03-06 08:11:56
## 64:            mean:22413394efd9f6a3 2019-03-06 08:11:56
## 65:            .FUN:7ad10bc1ae528d8c 2019-03-06 08:11:56
##                             tagValue         createdDate

1.10 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

1.11 Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

2.1 Nested Caching

2.1.0.1 Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.