The recordr package collects information about R script executions (also refered to as “runs”). The recorded information includes the files that were read and written by the R script, and details of the execution environment, such as the operating system version, R packages loaded, etc.
The recorded information for a script constitutes data provenance for the data products and analysis outputs (graphs, .csv files, etc) generated by a script execution, by providing information to describe how the data products were created.
The record() method takes an R script filename as an argument and sources it, recording files that were read and written by R functions that are registered with recordr. It is not necessary to modify an R script in order to use record.
The following example runs a sample script that is included with the recordr package:
library(recordr)
suppressWarnings(require(ggplot2))
rc <- new("Recordr")
sampleScript <- system.file("extdata/EmCoverage.R", package="recordr")
firstRunId <- record(rc, sampleScript , tag="first recordr run")Information about the script execution is stored in the recordr cache (~/.recordr). recordr provides methods to search and view items stored in the cache. It is not recommended that files or directories be manually edited or deleted from the cache directories, with the exception of the items mentioned in this document.
Script runs that have been recorded can be listed using the listRuns() method. The listing can be filtered by the tag value specified when a run was recorded. Runs can also be filtered by run start time, run end time, the text of error messages for a run and by a sequence number, which is an integer value assigned to each run to assist in easily specifying a particular run for listing or viewing.
In this example, all runs with a tag containing the string “first” are listed. Because recordr has only run once in this demo, only one run is listed:
listRuns(rc, tag="first")## Seq Script Tag Start Time End Time Run Id Published Time Error Message
## 5 /private/var/...a/EmCoverage.R first recordr run 2016-08-31 09:44:13 PDT 2016-08-31 09:44:14 PDT ...34350dfdc9 NA NAIf no search parameters are specified to listRuns, then all recorded runs are listed.
recordr can also collect information during an R console session using the startRecord() and endRecord() methods. When startRecord() is typed in the R console, information capture begins. Information will be captured for any function registered with recordr, while all other console input will not cause any information capture. Information capture is terminated when endRecord() is entered in the console, and execution information is written to the recordr cache.
startRecord(rc, tag="first console run")
df <- read.csv(file = system.file("./extdata/coverages_2001-2010.csv", package="recordr"))
endocladia_coverage <- df[df$final_classification=="endocladia muricata",]
myDir <- tempdir()
csvOutFile <- sprintf("%s/Endocladia_muricata.csv", myDir)
write.csv(endocladia_coverage, file = csvOutFile)
endRecord(rc)The history of all statements typed during this recorded console session is saved in the recordr cache.
More detailed information can be retrieved and viewed for a run or set of runs using the viewRuns() method, for example:
viewRuns(rc, id=firstRunId, sections=c("details", "used", "generated"))## [details]: Run details
## ----------------------
## "/private/var/...a/EmCoverage.R" was executed on 2016-08-31 09:44:13 PDT
## Tag: "first recordr run"
## Run sequence #: 5
## Publish date: Not published
## Published to: NA
## Published Id: NA
## View at: NA
## Run by user: slaughter
## Account subject: NA
## Run Id: urn:uuid:cb6cdb2a-5416-4b39-8fba-8a34350dfdc9
## Data package Id: urn:uuid:95a48d72-7171-43ff-ab85-54cf5cadb6fa
## HostId: dhcp177.nceas.ucsb.edu
## Operating system: x86_64-apple-darwin13.4.0
## R version: R version 3.3.1 (2016-06-21)
## Dependencies: stats, graphics, grDevices, utils, datasets, methods, base, hash_2.2.6, Rcpp_0.12.6, knitr_1.14, magrittr_1.5, roxygen2_5.0.1, munsell_0.4.3, uuid_0.1-2, colorspace_1.2-6, stringr_1.1.0, plyr_1.8.4, tools_3.3.1, redland_1.0.17-8, grid_3.3.1, gtable_0.2.0, DBI_0.5, htmltools_0.3.5, yaml_2.1.13, digest_0.6.10, formatR_1.4, datapack_1.0.1, RSQLite_1.0.0, evaluate_0.9, rmarkdown_1.0, stringi_1.1.1, scales_0.4.0, XML_3.98-1.4, ggplot2_2.1.0, recordr_1.0.3
## Run start time: 2016-08-31 09:44:13 PDT
## Run end time: 2016-08-31 09:44:14 PDT
## Error message from this run: NA
##
## [used]: 1 items used by this run
## -----------------------------------
## Location Size (kb) Modified time
## /private/var/folders/zb/y107...tdata/coverages_2001-2010.csv 138365 2016-08-31 09:44:10
##
## [generated]: 2 items generated by this run
## -----------------------------------------
## Location Size (kb) Modified time
## /private/var/folders/zb/y107...cOUut/Endocladia_muricata.csv 8052 2016-08-31 09:44:13
## /private/var/folders/zb/y107...n/T/RtmpscOUut/emCoverage.png 60960 2016-08-31 09:44:14Information for all matching runs is retrieved and displayed, The output displayed by viewRuns is divided into the sections “details”, “used” and “generated”, which can be selectively displayed using the sections parameter.
The record() method will currently record information for the following methods:
| package | function |
|---|---|
| dataone | getObject |
| dataone | create |
| dataone | updateObject |
| utils | read.csv |
| utils | write.csv |
| ggplot2 | ggsave |
| base | readLines |
| base | writeLines |
| png | readPNG |
| png | writePNG |
| base | scan |
Other information about the execution environment is also recorded, such as the R packages that were loaded, the operating system, system user name.
The following description is provided for informational purposes only and is not required to use the recordr package.
The recordr package can record execution information for the commonly used R functions mentioned in the previous section by using wrapper functions that are called before a requested function is called. This overriding of functions is only in effect when the record() function is running. This overridding is accomplished by temporarily adding an entry to the R search path so that the recordr wrapper functions are first in the search path. For example, if a script that is run with the record() function calls the following function:
df <- read.csv(file = ("/usr/smith/data/coverages_2001-2010.csv", package="recordr"))then the wrapper function recordr_read.csv is first called because record() has temporarily bound recordr_read.csv to the function name read.csv in the temporary environment named “.recordr” that is attached to the search path, so that the overridden function appears first in the search path, regardless of package load order. The function recordr_read.csv records that the file /usr/smith/data/coverages_2001-2010.csv was read. Then this wrapper function searches for the next function read.csv in the search path, which is the function that would have been run if record() was not active.
When the script has completed, the record() function unattaches the “.recordr” environment from the search path, thereby restoring the R search path to it’s previous state, as it was before record() was called.
Note that this mechanism that record() used to override functions doesn’t work for function calls that are fully qualified, i.e. the package name is included in the call. For example, the following function call would not be recorded:
df <- utils::read.csv(file = ("/usr/smith/data/coverages_2001-2010.csv", package="recordr"))Also, the record() function currently cannot record information for input or output files that are opened as a connection, for example, the following call to writeLines would not be recorded:
# Write out to a file using a connection
sbuf <- paste(LETTERS, collapse="")
tfile <- sprintf("%s/letters.dat", tempdir())
fcon <- file(description=tfile, open="w")
writeLines(sbuf, fcon)
close(fcon)This problem will be addressed in the next feature release of recordr.