2020-01-31

Getting started with JSON and jsonlite

The jsonlite package is a JSON parser/generator optimized for the web. Its main strength is that it implements a bidirectional mapping between JSON data and the most important R data types. Thereby we can convert between R objects and JSON without loss of type or information, and without the need for any manual data munging. This is ideal for interacting with web APIs, or to build pipelines where data structures seamlessly flow in and out of R using JSON.

library(jsonlite)
all.equal(mtcars, fromJSON(toJSON(mtcars)))

[1] TRUE

This vignette introduces basic concepts to get started with jsonlite. For a more detailed outline and motivation of the mapping, see: arXiv:1403.2805.

Simplification

Simplification is the process where JSON arrays automatically get converted from a list into a more specific R class. The fromJSON function has 3 arguments which control the simplification process: simplifyVector, simplifyDataFrame and simplifyMatrix. Each one is enabled by default.

JSON structure	Example JSON data	Simplifies to R class	Argument in fromJSON
Array of primitives	["Amsterdam", "Rotterdam", "Utrecht", "Den Haag"]	Atomic Vector	simplifyVector
Array of objects	[{"name":"Erik", "age":43}, {"name":"Anna", "age":32}]	Data Frame	simplifyDataFrame
Array of arrays	[ [1, 2, 3], [4, 5, 6] ]	Matrix	simplifyMatrix

Atomic Vectors

When simplifyVector is enabled, JSON arrays containing primitives (strings, numbers, booleans or null) simplify into an atomic vector:

# A JSON array of primitives
json <- '["Mario", "Peach", null, "Bowser"]'

# Simplifies into an atomic vector
fromJSON(json)

[1] "Mario"  "Peach"  NA       "Bowser"

Without simplification, any JSON array turns into a list:

# No simplification:
fromJSON(json, simplifyVector = FALSE)

[[1]]
[1] "Mario"

[[2]]
[1] "Peach"

[[3]]
NULL

[[4]]
[1] "Bowser"

Data Frames

When simplifyDataFrame is enabled, JSON arrays containing objects (key-value pairs) simplify into a data frame:

json <-
'[
  {"Name" : "Mario", "Age" : 32, "Occupation" : "Plumber"}, 
  {"Name" : "Peach", "Age" : 21, "Occupation" : "Princess"},
  {},
  {"Name" : "Bowser", "Occupation" : "Koopa"}
]'
mydf <- fromJSON(json)
mydf

    Name Age Occupation
1  Mario  32    Plumber
2  Peach  21   Princess
3   <NA>  NA       <NA>
4 Bowser  NA      Koopa

The data frame gets converted back into the original JSON structure by toJSON (whitespace and line breaks are ignorable in JSON).

mydf$Ranking <- c(3, 1, 2, 4)
toJSON(mydf, pretty=TRUE)

[
  {
    "Name": "Mario",
    "Age": 32,
    "Occupation": "Plumber",
    "Ranking": 3
  },
  {
    "Name": "Peach",
    "Age": 21,
    "Occupation": "Princess",
    "Ranking": 1
  },
  {
    "Ranking": 2
  },
  {
    "Name": "Bowser",
    "Occupation": "Koopa",
    "Ranking": 4
  }
] 

Hence you can go back and forth between dataframes and JSON, without any manual data restructuring.

Matrices and Arrays

When simplifyMatrix is enabled, JSON arrays containing equal-length sub-arrays simplify into a matrix (or higher order R array):

json <- '[
  [1, 2, 3, 4],
  [5, 6, 7, 8],
  [9, 10, 11, 12]
]'
mymatrix <- fromJSON(json)
mymatrix

     [,1] [,2] [,3] [,4]
[1,]    1    2    3    4
[2,]    5    6    7    8
[3,]    9   10   11   12

Again, we can use toJSON to convert the matrix or array back into the original JSON structure:

toJSON(mymatrix, pretty = TRUE)

[
  [1, 2, 3, 4],
  [5, 6, 7, 8],
  [9, 10, 11, 12]
] 

The simplification works for arrays of arbitrary dimensionality, as long as the dimensions match (R does not support ragged arrays).

json <- '[
   [[1, 2], 
    [3, 4]],
   [[5, 6], 
    [7, 8]],
   [[9, 10],
    [11, 12]]
]'
myarray <- fromJSON(json)
myarray[1, , ]

     [,1] [,2]
[1,]    1    2
[2,]    3    4

myarray[ , ,1]

     [,1] [,2]
[1,]    1    3
[2,]    5    7
[3,]    9   11

This is all there is to it! For a more detailed outline and motivation of the mapping, see: arXiv:1403.2805.