Ignoring case

For example, to ignore case in variable names (so that Arm and arm are considered the same), pass tol.vars = "case".

You can do this using comparedf.control()

summary(comparedf(mockstudy, mockstudy2, by = "case", control = comparedf.control(tol.vars = "case")))

or pass it through the ... arguments.

summary(comparedf(mockstudy, mockstudy2, by = "case", tol.vars = "case"))

Treating dots and underscores the same (equivalence classes)

It is possible to treat certain characters or sets of characters as the same by passing a character vector of equivalence classes to the tol.vars= argument.

In short, each string in the vector is split into single characters, and the resulting set of characters is replaced by the first character in the string. For example, passing c("._") would replace all underscores with dots in the column names of both datasets. Similarly, passing c("aA", "BbCc") would replace all instances of "A" with "a" and all instances of "b", "C", or "c" with "B". This is one way to ignore case for certain letters. Otherwise, it’s possible to combine the equivalence classes with ignoring case, by passing (e.g.) c("._", "case").

Passing a single character as an element this vector will replace that character with the empty string. For example, passing c(" “,”.") would remove all spaces and dots from the column names.

For mockstudy, let’s treat dots, underscores, and spaces as the same, and ignore case:

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case") # dots=underscores=spaces, ignore case
))

Manually specifying columns to match together

If you pass a named vector to the tol.vars= argument, comparedf() will line up the names of that vector to the column names of x and the values of that vector to the column names of y. In this way, you can manually specify which non-identically-named columns to compare.

For mockstudy, let’s specify our variables manually in this way:

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c(arm = "Arm", fu.stat = "fu stat", fu.time = "fu_time")
))

Logical tolerance

Use the tol.logical= argument to change how logicals are compared. By default, they’re expected to be equal to each other.

Numeric tolerance

To allow numeric differences of a certain tolerance, use the tol.num= and tol.num.val= options. tol.num.val= determines the maximum (unsigned) difference tolerated if tol.num="absolute" (default), and determines the maximum (unsigned) percent difference tolerated if tol.num="percent".

Also note the option int.as.num=, which determines whether integers and numerics should be compared despite their class difference. If TRUE, the integers are coerced to numeric. Note that mockstudy$ast is integer, while mockstudy2$ast is numeric:

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE            # compare integers and numerics
))

Suppose a tolerance of up to 10 is allowed for ast:

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE,           # compare integers and numerics
                tol.num.val = 10             # allow absolute differences <= 10
))

Factor tolerance

By default, factors are compared to each other based on both the labels and the underlying numeric levels. Set tol.factor="levels" to match only the numeric levels, or set tol.factor="labels" to match only the labels.

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE,           # compare integers and numerics
                tol.num.val = 10,            # allow absolute differences <= 10
                tol.factor = "labels"        # match only factor labels
))

Also note the option factor.as.char=, which determines whether factors and characters should be compared despite their class difference. If TRUE, the factors are coerced to characters. Note that mockstudy$race is a character, while mockstudy2$race is a factor:

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE,           # compare integers and numerics
                tol.num.val = 10,            # allow absolute differences <= 10
                tol.factor = "labels",       # match only factor labels
                factor.as.char = TRUE        # compare factors and characters
))

Character tolerance

Use the tol.char= argument to change how character variables are compared. By default, they are compared as-is, but they can be compared after ignoring case or trimming whitespace or both.

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE,           # compare integers and numerics
                tol.num.val = 10,            # allow absolute differences <= 10
                tol.factor = "labels",       # match only factor labels
                factor.as.char = TRUE,       # compare factors and characters
                tol.char = "case"            # ignore case in character vectors
))

Date tolerance

Use the tol.date= argument to change how dates are compared. By default, they’re expected to be equal to each other.

Other data type tolerances

Use the tol.other= argument to change how other objects are compared. By default, they’re expected to be identical().

Specifying tolerances for each variable

You can also provide a list of tolerance functions to comparedf():

comparedf.control(tol.char = list(
  "none",      # the default
  x1 = "case", # be case-insensitive for the variable "x1"
  x2 = function(x, y) tol.NA(x, y, x != y | y == "NA") # a custom-defined tolerance
))

User-defined tolerance functions

my.tol <- function(x, y, tol)
{
  tol.NA(x, y, x > y)
}

date.df1 <- data.frame(dt = as.Date(c("2017-09-07", "2017-08-08", "2017-07-09", NA)))
date.df2 <- data.frame(dt = as.Date(c("2017-10-01", "2017-08-08", "2017-07-10", "2017-01-01")))
n.diffs(comparedf(date.df1, date.df2)) # default finds any differences

[1] 3

n.diffs(comparedf(date.df1, date.df2, tol.date = my.tol)) # our function identifies only the NA as different...

[1] 1

n.diffs(comparedf(date.df2, date.df1, tol.date = my.tol)) # ... until we change the argument order

[1] 3

tol.minus9 <- function(x, y, tol)
{
  idx1 <- is.na(x) & !is.na(y) & y == -9
  idx2 <- tol.num.absolute(x, y, tol) # find other absolute differences
  return(!idx1 & idx2)
}

summary(comparedf(mockstudy, mockstudy2, by = "case",
                tol.vars = c("._ ", "case"), # dots=underscores=spaces, ignore case
                int.as.num = TRUE,           # compare integers and numerics
                tol.num.val = 10,            # allow absolute differences <= 10
                tol.factor = "labels",       # match only factor labels
                factor.as.char = TRUE,       # compare factors and characters
                tol.char = "case",           # ignore case in character vectors
                tol.num = tol.minus9         # ignore NA -> -9 changes
))

Stucture of the Object

(This section is just as much for my use as for yours!)

obj <- comparedf(mockstudy, mockstudy2, by = "case")

There are two main objects in the "comparedf" object, each with its own print method.

The frame.summary contains:

• the substituted-deparsed arguments

• information about the number of columns and rows in each dataset

• the by-variables for each dataset (which may not be the same)

• the attributes for each dataset (which get counted in the print method)

• a data.frame of by-variables and row numbers of observations not shared between datasets

• the number of shared observations

print(obj$frame.summary)

  version        arg ncol nrow   by        attrs       unique n.shared
1       x  mockstudy   14 1499 case 3 attributes 4 unique obs     1495
2       y mockstudy2   13 1495 case 3 attributes 0 unique obs     1495

The vars.summary contains:

• variable name, column number, and class vector (with possibly more than one element) for each x and y. These are all NA if there isn’t a match in both datasets.

• values, a list-column of the text string "by-variable" for the by-variables, NULL for columns that aren’t compared, or a data.frame containing:

  • The by-variables for differences found

  • The values which are different for x and y

  • The row numbers for differences found

• attrs, a list-column of NULL if there are no attributes, or a data.frame containing:

  • The name of the attributes

  • The attributes for x and y, set to NA if non-existant

  • The actual attributes (if show.attr=TRUE).

print(obj$vars.summary)

         var.x pos.x         class.x       var.y pos.y         class.y           values        attrs
1         case     1         integer        case     1         integer      by-variable 0 attributes
2          sex     4          factor         sex     2          factor 1495 differences 2 attributes
3         race     5       character        race     3          factor     Not compared 3 attributes
4           ps     8         integer          ps     4         integer    1 differences 0 attributes
5          hgb     9         numeric         hgb     5         numeric  266 differences 0 attributes
6          bmi    10         numeric         bmi     6         numeric    0 differences 1 attributes
7     alk.phos    11         integer    alk.phos     7         integer    0 differences 0 attributes
8          ast    12         integer         ast     8         numeric     Not compared 0 attributes
9  mdquality.s    13         integer mdquality.s     9         integer    0 differences 0 attributes
10     age.ord    14 ordered, factor     age.ord    10 ordered, factor    0 differences 0 attributes
11         age     2         integer        <NA>    NA              NA     Not compared 0 attributes
12         arm     3       character        <NA>    NA              NA     Not compared 0 attributes
13     fu.time     6         integer        <NA>    NA              NA     Not compared 0 attributes
14     fu.stat     7         integer        <NA>    NA              NA     Not compared 0 attributes
15        <NA>    NA              NA     fu_time    11         integer     Not compared 0 attributes
16        <NA>    NA              NA     fu stat    12         integer     Not compared 0 attributes
17        <NA>    NA              NA         Arm    13       character     Not compared 0 attributes