Overview of supported structures

The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

The idea of tibblify() is to make it easier and more robust to convert lists of lists into tibbles. This is a typical task after receiving API responses in JSON format. The following provides an overview of which kinds of R objects are supported and the JSON to which they correspond.

Scalars

There are 4 basic types of scalars in JSON: boolean, integer, float, string. In R there are not really scalars but only vectors of length 1.

true
1
1.5
"a"

TRUE
1
1.5
"a"

Other R vectors without JSON equivalent are also supported as long as they:

are a vector in the vctrs definition and
have size one, i.e. vctrs::vec_size(x) is 1.

Examples are Date or POSIXct.

In general a scalar can be parsed with tib_scalar(). There are some special functions for common types:

tib_lgl()
tib_int()
tib_dbl()
tib_chr()
tib_date()
tib_chr_date() to parse dates encoded as string.

Vectors

A homogeneous JSON array is an array of scalars where each scalar has the same type. In R they correspond to a logical(), integer(), double() or character() vector.

[true, null, false]
[1, null, 3]
[1.5, null, 3.5]
["a", null, "c"]

c(TRUE, NA, FALSE)
c(1L, NA, 2L)
c(1.5, NA, 2.5)
c("a", NA, "c")

As with scalars, other vector types are also supported as long as they are a vector in the vctrs definition.

They can be parsed with tib_vector(). As with scalars, there are shortcuts for some common types, e.g. tib_lgl_vec().

Empty lists

Empty lists list() are a special case. They might appear when parsing an empty JSON array.

x_json <- '[
  {"a": [1, 2]},
  {"a": []}
]'

x <- jsonlite::fromJSON(x_json, simplifyDataFrame = FALSE)
str(x)
#> List of 2
#>  $ :List of 1
#>   ..$ a: int [1:2] 1 2
#>  $ :List of 1
#>   ..$ a: list()

By default they are not supported but produce an error.

tibblify(x, tspec_df(tib_int_vec("a")))
#> Error in `tibblify()`:
#> ! Problem while tibblifying `x[[2]]$a`
#> Caused by error:
#> ! Can't convert `<list>` <list> to <integer>.

Use vector_allows_empty_list = TRUE in tspec_*() so that they are converted to an empty vector instead.

tibblify(x, tspec_df(tib_int_vec("a"), .vector_allows_empty_list = TRUE))$a
#> <list_of<integer>[2]>
#> [[1]]
#> [1] 1 2
#> 
#> [[2]]
#> integer(0)

Homogeneous R lists of scalars

When using jsonlite::fromJSON(simplifyVector = FALSE) to parse JSON to an R object one does not get R vectors but homogeneous lists of scalars.

x_json <- '[
  {"a": [1, 2]},
  {"a": [1, 2, 3]}
]'

x <- jsonlite::fromJSON(x_json, simplifyVector = FALSE)
str(x)
#> List of 2
#>  $ :List of 1
#>   ..$ a:List of 2
#>   .. ..$ : int 1
#>   .. ..$ : int 2
#>  $ :List of 1
#>   ..$ a:List of 3
#>   .. ..$ : int 1
#>   .. ..$ : int 2
#>   .. ..$ : int 3

By default they cannot be parsed with tib_vector().

tibblify(x, tspec_df(tib_int_vec("a")))
#> Error in `tibblify()`:
#> ! Problem while tibblifying `x[[1]]$a`
#> Caused by error:
#> ! Can't convert `<list>` <list> to <integer>.

Use .input_form = "scalar_list" in tib_vector() to parse them:

tibblify(x, tspec_df(tib_int_vec("a", .input_form = "scalar_list")))$a
#> <list_of<integer>[2]>
#> [[1]]
#> [1] 1 2
#> 
#> [[2]]
#> [1] 1 2 3

Homogeneous JSON objects of scalars

Sometimes vectors are encoded as objects in JSON.

x_json <- '[
  {"a": {"x": 1, "y": 2}},
  {"a": {"a": 1, "b": 2, "b": 3}}
]'

x <- jsonlite::fromJSON(x_json, simplifyVector = FALSE)
str(x)
#> List of 2
#>  $ :List of 1
#>   ..$ a:List of 2
#>   .. ..$ x: int 1
#>   .. ..$ y: int 2
#>  $ :List of 1
#>   ..$ a:List of 3
#>   .. ..$ a: int 1
#>   .. ..$ b: int 2
#>   .. ..$ b: int 3

Use .input_form = "object" in tib_vector() to parse them. To store the names use the .names_to and .values_to arguments.

spec <- tspec_df(
  tib_int_vec(
    "a",
    .input_form = "object",
    .names_to = "name",
    .values_to = "value"
  )
)

tibblify(x, spec)$a[[1]]
#> # A tibble: 2 × 2
#>   name  value
#>   <chr> <int>
#> 1 x         1
#> 2 y         2
tibblify(x, spec)$a[[2]]
#> # A tibble: 3 × 2
#>   name  value
#>   <chr> <int>
#> 1 a         1
#> 2 b         2
#> 3 b         3

Varying

JSON also has lists where elements do not have a common type, but instead vary. For example:

[1, "a", true]

list(1, "a", TRUE)

Such lists can be parsed with tib_variant().

Object

The R equivalent to a JSON object is a named list where the names fulfill the requirements of vctrs::vec_as_names(repair = "check_unique").

{
  "a": 1,
  "b": true
}

x <- list(
  a = 1,
  b = TRUE
)

They can be parsed with tib_row(). For example:

x <- list(
  list(row = list(a = 1, b = TRUE)),
  list(row = list(a = 2, b = FALSE))
)

spec <- tspec_df(
  tib_row(
    "row",
    tib_int("a"),
    tib_lgl("b")
  )
)

tibblify(x, spec)
#> # A tibble: 2 × 1
#>   row$a $b   
#>   <int> <lgl>
#> 1     1 TRUE 
#> 2     2 FALSE

Data Frames

JSON can also store lists of objects.

[
  {"a": 1, "b": true},
  {"b": 2, "b": false}
]

x <- list(
  list(a = 1, b = TRUE),
  list(a = 2, b = FALSE)
)

They can be parsed with tib_df().

Object of objects

JSON can also store named lists of objects. In JSON they are represented as objects where each element is an object.

{
  "object1": {"a": 1, "b": true},
  "object2": {"b": 2, "b": false}
}

x <- list(
  object1 = list(a = 1, b = TRUE),
  object2 = list(a = 2, b = FALSE)
)

They are also parsed with tib_df(), but you can parse the names into an extra column via the .names_to argument:

x_json <- '[
{
  "df": {
    "object1": {"a": 1, "b": true},
    "object2": {"a": 2, "b": false}
  }
}]'

x <- jsonlite::fromJSON(x_json, simplifyDataFrame = FALSE)

spec <- tspec_df(
  tib_df(
    "df",
    tib_int("a"),
    tib_lgl("b"),
    .names_to = "name"
  )
)

tibblify(x, spec)$df
#> <list_of<
#>   tbl_df<
#>     name: character
#>     a   : integer
#>     b   : logical
#>   >
#> >[1]>
#> [[1]]
#> # A tibble: 2 × 3
#>   name        a b    
#>   <chr>   <int> <lgl>
#> 1 object1     1 TRUE 
#> 2 object2     2 FALSE

Column-major format

The column-major format is also supported.

{
  "a": [1, 2],
  "b": [true, false]
}

x <- list(
  a = c(1, 2),
  b = c(TRUE, FALSE)
)

Parse this using .input_form = "colmajor" in tspec_*().

df_spec <- tspec_df(
  tib_int("a"),
  tib_lgl("b"),
  .input_form = "colmajor"
)

tibblify(x, df_spec)
#> # A tibble: 2 × 2
#>       a b    
#>   <int> <lgl>
#> 1     1 TRUE 
#> 2     2 FALSE

This is roughly equivalent to tibble::as_tibble(x).

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.