stream_in.RdThe stream_in and stream_out functions implement line-by-line processing
of JSON data over a connection, such as a socket, url, file or pipe. JSON
streaming requires the ndjson format, which slightly differs
from fromJSON and toJSON, see details.
stream_in(con, handler = NULL, pagesize = 500, verbose = TRUE, ...) stream_out(x, con = stdout(), pagesize = 500, verbose = TRUE, prefix = "", ...)
| con | a |
|---|---|
| handler | a custom function that is called on each page of JSON data. If not specified,
the default handler stores all pages and binds them into a single data frame that will be
returned by |
| pagesize | number of lines to read/write from/to the connection per iteration. |
| verbose | print some information on what is going on. |
| ... | arguments for |
| x | object to be streamed out. Currently only data frames are supported. |
| prefix | string to write before each line (use |
The stream_out function always returns NULL.
When no custom handler is specified, stream_in returns a data frame of all pages binded together.
When a custom handler function is specified, stream_in always returns NULL.
Because parsing huge JSON strings is difficult and inefficient, JSON streaming is done
using lines of minified JSON records, a.k.a. ndjson.
This is pretty standard: JSON databases such as dat
or MongoDB use the same format to import/export datasets. Note that this means that the
total stream combined is not valid JSON itself; only the individual lines are. Also note
that because line-breaks are used as separators, prettified JSON is not permitted: the
JSON lines must be minified. In this respect, the format is a bit different from
fromJSON and toJSON where all lines are part of a single JSON
structure with optional line breaks.
The handler is a callback function which is called for each page (batch) of
JSON data with exactly one argument (usually a data frame with pagesize rows).
If handler is missing or NULL, a default handler is used which stores all
intermediate pages of data, and at the very end binds all pages together into one single
data frame that is returned by stream_in. When a custom handler function
is specified, stream_in does not store any intermediate results and always returns
NULL. It is then up to the handler to process or store data pages.
A handler function that does not store intermediate results in memory (for
example by writing output to another connection) results in a pipeline that can process an
unlimited amount of data. See example.
If a connection is not opened yet, stream_in and stream_out
will automatically open and later close the connection. Because R destroys connections
when they are closed, they cannot be reused. To use a single connection for multiple
calls to stream_in or stream_out, it needs to be opened
beforehand. See example.
MongoDB export format: http://docs.mongodb.org/manual/reference/program/mongoexport/#cmdoption--query
Documentation for the JSON Lines text file format: http://jsonlines.org/
#> [{"Sepal.Length":5.1,"Sepal.Width":3.5,"Petal.Length":1.4,"Petal.Width":0.2,"Species":"setosa"},{"Sepal.Length":4.9,"Sepal.Width":3,"Petal.Length":1.4,"Petal.Width":0.2,"Species":"setosa"},{"Sepal.Length":4.7,"Sepal.Width":3.2,"Petal.Length":1.3,"Petal.Width":0.2,"Species":"setosa"}]stream_out(x)#> {"Sepal.Length":5.1,"Sepal.Width":3.5,"Petal.Length":1.4,"Petal.Width":0.2,"Species":"setosa"} #> {"Sepal.Length":4.9,"Sepal.Width":3,"Petal.Length":1.4,"Petal.Width":0.2,"Species":"setosa"} #> {"Sepal.Length":4.7,"Sepal.Width":3.2,"Petal.Length":1.3,"Petal.Width":0.2,"Species":"setosa"} #> Complete! Processed total of 3 rows.#>#> Found 100 records... Imported 100 records. Simplifying...#>#> Error in library(nycflights13): there is no package called ‘nycflights13’#>#> Error in is.data.frame(x): object 'flights' not found#>#>#> Imported 0 records. Simplifying...#>#> Error in as.data.frame(flights): object 'flights' not found#>#> Found 500 records... Found 1000 records... Found 1500 records... Found 2000 records... Found 2500 records... Found 3000 records... Found 3500 records... Found 4000 records... Found 4500 records... Found 5000 records... Found 5500 records... Found 6000 records... Found 6500 records... Found 7000 records... Found 7500 records... Found 8000 records... Found 8500 records... Found 9000 records... Found 9500 records... Found 10000 records... Found 10500 records... Found 11000 records... Found 11500 records... Found 12000 records... Found 12500 records... Found 13000 records... Found 13500 records... Found 14000 records... Found 14500 records... Found 15000 records... Found 15500 records... Found 16000 records... Found 16500 records... Found 17000 records... Found 17500 records... Found 18000 records... Found 18500 records... Found 19000 records... Found 19500 records... Found 20000 records... Found 20500 records... Found 21000 records... Found 21500 records... Found 22000 records... Found 22500 records... Found 23000 records... Found 23500 records... Found 24000 records... Found 24500 records... Found 25000 records... Found 25500 records... Found 26000 records... Found 26500 records... Found 27000 records... Found 27500 records... Found 28000 records... Found 28500 records... Found 29000 records... Found 29500 records... Found 30000 records... Found 30500 records... Found 31000 records... Found 31500 records... Found 32000 records... Found 32500 records... Found 33000 records... Found 33500 records... Found 34000 records... Found 34500 records... Found 35000 records... Found 35500 records... Found 36000 records... Found 36500 records... Found 37000 records... Found 37500 records... Found 38000 records... Found 38500 records... Found 39000 records... Found 39500 records... Found 40000 records... Found 40500 records... Found 41000 records... Found 41500 records... Found 42000 records... Found 42500 records... Found 43000 records... Found 43500 records... Found 44000 records... Found 44500 records... Found 45000 records... Found 45500 records... Found 46000 records... Found 46500 records... Found 47000 records... Found 47500 records... Found 48000 records... Found 48500 records... Found 49000 records... Found 49500 records... Found 50000 records... Found 50500 records... Found 51000 records... Found 51500 records... Found 52000 records... Found 52500 records... Found 53000 records... Found 53500 records... Found 53940 records... Imported 53940 records. Simplifying...#># stream over HTTP with gzip compression flights3 <- stream_in(gzcon(url("http://jeroen.github.io/data/nycflights13.json.gz")))#>#> Found 500 records... Found 1000 records... Found 1500 records... Found 2000 records... Found 2500 records... Found 3000 records... Found 3500 records... Found 4000 records... Found 4500 records... Found 5000 records... Found 5500 records... Found 6000 records... Found 6500 records... Found 7000 records... Found 7500 records... Found 8000 records... Found 8500 records... Found 9000 records... Found 9500 records... Found 10000 records... Found 10500 records... Found 11000 records... Found 11500 records... Found 12000 records... Found 12500 records... Found 13000 records... Found 13500 records... Found 14000 records... Found 14500 records... Found 15000 records... Found 15500 records... Found 16000 records... Found 16500 records... Found 17000 records... Found 17500 records... Found 18000 records... Found 18500 records... Found 19000 records... Found 19500 records... Found 20000 records... Found 20500 records... Found 21000 records... Found 21500 records... Found 22000 records... Found 22500 records... Found 23000 records... Found 23500 records... Found 24000 records... Found 24500 records... Found 25000 records... Found 25500 records... Found 26000 records... Found 26500 records... Found 27000 records... Found 27500 records... Found 28000 records... Found 28500 records... Found 29000 records... Found 29500 records... Found 30000 records... Found 30500 records... Found 31000 records... Found 31500 records... Found 32000 records... Found 32500 records... Found 33000 records... Found 33500 records... Found 34000 records... Found 34500 records... Found 35000 records... Found 35500 records... Found 36000 records... Found 36500 records... Found 37000 records... Found 37500 records... Found 38000 records... Found 38500 records... Found 39000 records... Found 39500 records... Found 40000 records... Found 40500 records... Found 41000 records... Found 41500 records... Found 42000 records... Found 42500 records... Found 43000 records... Found 43500 records... Found 44000 records... Found 44500 records... Found 45000 records... Found 45500 records... Found 46000 records... Found 46500 records... Found 47000 records... Found 47500 records... Found 48000 records... Found 48500 records... Found 49000 records... Found 49500 records... Found 50000 records... Found 50500 records... Found 51000 records... Found 51500 records... Found 52000 records... Found 52500 records... Found 53000 records... Found 53500 records... Found 54000 records... Found 54500 records... Found 55000 records... Found 55500 records... Found 56000 records... Found 56500 records... Found 57000 records... Found 57500 records... Found 58000 records... Found 58500 records... Found 59000 records... Found 59500 records... Found 60000 records... Found 60500 records... Found 61000 records... Found 61500 records... Found 62000 records... Found 62500 records... Found 63000 records... Found 63500 records... Found 64000 records... Found 64500 records... Found 65000 records... Found 65500 records... Found 66000 records... Found 66500 records... Found 67000 records... Found 67500 records... Found 68000 records... Found 68500 records... Found 69000 records... Found 69500 records... Found 70000 records... Found 70500 records... Found 71000 records... Found 71500 records... Found 72000 records... Found 72500 records... Found 73000 records... Found 73500 records... Found 74000 records... Found 74500 records... Found 75000 records... Found 75500 records... Found 76000 records... Found 76500 records... Found 77000 records... Found 77500 records... Found 78000 records... Found 78500 records... Found 79000 records... Found 79500 records... Found 80000 records... Found 80500 records... Found 81000 records... Found 81500 records... Found 82000 records... Found 82500 records... Found 83000 records... Found 83500 records... Found 84000 records... Found 84500 records... Found 85000 records... Found 85500 records... Found 86000 records... Found 86500 records... Found 87000 records... Found 87500 records... Found 88000 records... Found 88500 records... Found 89000 records... Found 89500 records... Found 90000 records... Found 90500 records... Found 91000 records... Found 91500 records... Found 92000 records... Found 92500 records... Found 93000 records... Found 93500 records... Found 94000 records... Found 94500 records... Found 95000 records... Found 95500 records... Found 96000 records... Found 96500 records... Found 97000 records... Found 97500 records... Found 98000 records... Found 98500 records... Found 99000 records... Found 99500 records... Found 1e+05 records... Found 100500 records... Found 101000 records... Found 101500 records... Found 102000 records... Found 102500 records... Found 103000 records... Found 103500 records... Found 104000 records... Found 104500 records... Found 105000 records... Found 105500 records... Found 106000 records... Found 106500 records... Found 107000 records... Found 107500 records... Found 108000 records... Found 108500 records... Found 109000 records... Found 109500 records... Found 110000 records... Found 110500 records... Found 111000 records... Found 111500 records... Found 112000 records... Found 112500 records... Found 113000 records... Found 113500 records... Found 114000 records... Found 114500 records... Found 115000 records... Found 115500 records... Found 116000 records... Found 116500 records... Found 117000 records... Found 117500 records... Found 118000 records... Found 118500 records... Found 119000 records... Found 119500 records... Found 120000 records... Found 120500 records... Found 121000 records... Found 121500 records... Found 122000 records... Found 122500 records... Found 123000 records... Found 123500 records... Found 124000 records... Found 124500 records... Found 125000 records... Found 125500 records... Found 126000 records... Found 126500 records... Found 127000 records... Found 127500 records... Found 128000 records... Found 128500 records... Found 129000 records... Found 129500 records... Found 130000 records... Found 130500 records... Found 131000 records... Found 131500 records... Found 132000 records... Found 132500 records... Found 133000 records... Found 133500 records... Found 134000 records... Found 134500 records... Found 135000 records... Found 135500 records... Found 136000 records... Found 136500 records... Found 137000 records... Found 137500 records... Found 138000 records... Found 138500 records... Found 139000 records... Found 139500 records... Found 140000 records... Found 140500 records... Found 141000 records... Found 141500 records... Found 142000 records... Found 142500 records... Found 143000 records... Found 143500 records... Found 144000 records... Found 144500 records... Found 145000 records... Found 145500 records... Found 146000 records... Found 146500 records... Found 147000 records... Found 147500 records... Found 148000 records... Found 148500 records... Found 149000 records... Found 149500 records... Found 150000 records... Found 150500 records... Found 151000 records... Found 151500 records... Found 152000 records... Found 152500 records... Found 153000 records... Found 153500 records... Found 154000 records... Found 154500 records... Found 155000 records... Found 155500 records... Found 156000 records... Found 156500 records... Found 157000 records... Found 157500 records... Found 158000 records... Found 158500 records... Found 159000 records... Found 159500 records... Found 160000 records... Found 160500 records... Found 161000 records... Found 161500 records... Found 162000 records... Found 162500 records... Found 163000 records... Found 163500 records... Found 164000 records... Found 164500 records... Found 165000 records... Found 165500 records... Found 166000 records... Found 166500 records... Found 167000 records... Found 167500 records... Found 168000 records... Found 168500 records... Found 169000 records... Found 169500 records... Found 170000 records... Found 170500 records... Found 171000 records... Found 171500 records... Found 172000 records... Found 172500 records... Found 173000 records... Found 173500 records... Found 174000 records... Found 174500 records... Found 175000 records... Found 175500 records... Found 176000 records... Found 176500 records... Found 177000 records... Found 177500 records... Found 178000 records... Found 178500 records... Found 179000 records... Found 179500 records... Found 180000 records... Found 180500 records... Found 181000 records... Found 181500 records... Found 182000 records... Found 182500 records... Found 183000 records... Found 183500 records... Found 184000 records... Found 184500 records... Found 185000 records... Found 185500 records... Found 186000 records... Found 186500 records... Found 187000 records... Found 187500 records... Found 188000 records... Found 188500 records... Found 189000 records... Found 189500 records... Found 190000 records... Found 190500 records... Found 191000 records... Found 191500 records... Found 192000 records... Found 192500 records... Found 193000 records... Found 193500 records... Found 194000 records... Found 194500 records... Found 195000 records... Found 195500 records... Found 196000 records... Found 196500 records... Found 197000 records... Found 197500 records... Found 198000 records... Found 198500 records... Found 199000 records... Found 199500 records... Found 2e+05 records... Found 200500 records... Found 201000 records... Found 201500 records... Found 202000 records... Found 202500 records... Found 203000 records... Found 203500 records... Found 204000 records... Found 204500 records... Found 205000 records... Found 205500 records... Found 206000 records... Found 206500 records... Found 207000 records... Found 207500 records... Found 208000 records... Found 208500 records... Found 209000 records... Found 209500 records... Found 210000 records... Found 210500 records... Found 211000 records... Found 211500 records... Found 212000 records... Found 212500 records... Found 213000 records... Found 213500 records... Found 214000 records... Found 214500 records... Found 215000 records... Found 215500 records... Found 216000 records... Found 216500 records... Found 217000 records... Found 217500 records... Found 218000 records... Found 218500 records... Found 219000 records... Found 219500 records... Found 220000 records... Found 220500 records... Found 221000 records... Found 221500 records... Found 222000 records... Found 222500 records... Found 223000 records... Found 223500 records... Found 224000 records... Found 224500 records... Found 225000 records... Found 225500 records... Found 226000 records... Found 226500 records... Found 227000 records... Found 227500 records... Found 228000 records... Found 228500 records... Found 229000 records... Found 229500 records... Found 230000 records... Found 230500 records... Found 231000 records... Found 231500 records... Found 232000 records... Found 232500 records... Found 233000 records... Found 233500 records... Found 234000 records... Found 234500 records... Found 235000 records... Found 235500 records... Found 236000 records... Found 236500 records... Found 237000 records... Found 237500 records... Found 238000 records... Found 238500 records... Found 239000 records... Found 239500 records... Found 240000 records... Found 240500 records... Found 241000 records... Found 241500 records... Found 242000 records... Found 242500 records... Found 243000 records... Found 243500 records... Found 244000 records... Found 244500 records... Found 245000 records... Found 245500 records... Found 246000 records... Found 246500 records... Found 247000 records... Found 247500 records... Found 248000 records... Found 248500 records... Found 249000 records... Found 249500 records... Found 250000 records... Found 250500 records... Found 251000 records... Found 251500 records... Found 252000 records... Found 252500 records... Found 253000 records... Found 253500 records... Found 254000 records... Found 254500 records... Found 255000 records... Found 255500 records... Found 256000 records... Found 256500 records... Found 257000 records... Found 257500 records... Found 258000 records... Found 258500 records... Found 259000 records... Found 259500 records... Found 260000 records... Found 260500 records... Found 261000 records... Found 261500 records... Found 262000 records... Found 262500 records... Found 263000 records... Found 263500 records... Found 264000 records... Found 264500 records... Found 265000 records... Found 265500 records... Found 266000 records... Found 266500 records... Found 267000 records... Found 267500 records... Found 268000 records... Found 268500 records... Found 269000 records... Found 269500 records... Found 270000 records... Found 270500 records... Found 271000 records... Found 271500 records... Found 272000 records... Found 272500 records... Found 273000 records... Found 273500 records... Found 274000 records... Found 274500 records... Found 275000 records... Found 275500 records... Found 276000 records... Found 276500 records... Found 277000 records... Found 277500 records... Found 278000 records... Found 278500 records... Found 279000 records... Found 279500 records... Found 280000 records... Found 280500 records... Found 281000 records... Found 281500 records... Found 282000 records... Found 282500 records... Found 283000 records... Found 283500 records... Found 284000 records... Found 284500 records... Found 285000 records... Found 285500 records... Found 286000 records... Found 286500 records... Found 287000 records... Found 287500 records... Found 288000 records... Found 288500 records... Found 289000 records... Found 289500 records... Found 290000 records... Found 290500 records... Found 291000 records... Found 291500 records... Found 292000 records... Found 292500 records... Found 293000 records... Found 293500 records... Found 294000 records... Found 294500 records... Found 295000 records... Found 295500 records... Found 296000 records... Found 296500 records... Found 297000 records... Found 297500 records... Found 298000 records... Found 298500 records... Found 299000 records... Found 299500 records... Found 3e+05 records... Found 300500 records... Found 301000 records... Found 301500 records... Found 302000 records... Found 302500 records... Found 303000 records... Found 303500 records... Found 304000 records... Found 304500 records... Found 305000 records... Found 305500 records... Found 306000 records... Found 306500 records... Found 307000 records... Found 307500 records... Found 308000 records... Found 308500 records... Found 309000 records... Found 309500 records... Found 310000 records... Found 310500 records... Found 311000 records... Found 311500 records... Found 312000 records... Found 312500 records... Found 313000 records... Found 313500 records... Found 314000 records... Found 314500 records... Found 315000 records... Found 315500 records... Found 316000 records... Found 316500 records... Found 317000 records... Found 317500 records... Found 318000 records... Found 318500 records... Found 319000 records... Found 319500 records... Found 320000 records... Found 320500 records... Found 321000 records... Found 321500 records... Found 322000 records... Found 322500 records... Found 323000 records... Found 323500 records... Found 324000 records... Found 324500 records... Found 325000 records... Found 325500 records... Found 326000 records... Found 326500 records... Found 327000 records... Found 327500 records... Found 328000 records... Found 328500 records... Found 329000 records... Found 329500 records... Found 330000 records... Found 330500 records... Found 331000 records... Found 331500 records... Found 332000 records... Found 332500 records... Found 333000 records... Found 333500 records... Found 334000 records... Found 334500 records... Found 335000 records... Found 335500 records... Found 336000 records... Found 336500 records... Found 336776 records... Imported 336776 records. Simplifying...#>#> Error in as.data.frame(flights): object 'flights' not found# stream over HTTPS (HTTP+SSL) via curl library(curl) flights4 <- stream_in(gzcon(curl("https://jeroen.github.io/data/nycflights13.json.gz")))#>#> Found 500 records... Found 1000 records... Found 1500 records... Found 2000 records... Found 2500 records... Found 3000 records... Found 3500 records... Found 4000 records... Found 4500 records... Found 5000 records... Found 5500 records... Found 6000 records... Found 6500 records... Found 7000 records... Found 7500 records... Found 8000 records... Found 8500 records... Found 9000 records... Found 9500 records... Found 10000 records... Found 10500 records... Found 11000 records... Found 11500 records... Found 12000 records... Found 12500 records... Found 13000 records... Found 13500 records... Found 14000 records... Found 14500 records... Found 15000 records... Found 15500 records... Found 16000 records... Found 16500 records... Found 17000 records... Found 17500 records... Found 18000 records... Found 18500 records... Found 19000 records... Found 19500 records... Found 20000 records... Found 20500 records... Found 21000 records... Found 21500 records... Found 22000 records... Found 22500 records... Found 23000 records... Found 23500 records... Found 24000 records... Found 24500 records... Found 25000 records... Found 25500 records... Found 26000 records... Found 26500 records... Found 27000 records... Found 27500 records... Found 28000 records... Found 28500 records... Found 29000 records... Found 29500 records... Found 30000 records... Found 30500 records... Found 31000 records... Found 31500 records... Found 32000 records... Found 32500 records... Found 33000 records... Found 33500 records... Found 34000 records... Found 34500 records... Found 35000 records... Found 35500 records... Found 36000 records... Found 36500 records... Found 37000 records... Found 37500 records... Found 38000 records... Found 38500 records... Found 39000 records... Found 39500 records... Found 40000 records... Found 40500 records... Found 41000 records... Found 41500 records... Found 42000 records... Found 42500 records... Found 43000 records... Found 43500 records... Found 44000 records... Found 44500 records... Found 45000 records... Found 45500 records... Found 46000 records... Found 46500 records... Found 47000 records... Found 47500 records... Found 48000 records... Found 48500 records... Found 49000 records... Found 49500 records... Found 50000 records... Found 50500 records... Found 51000 records... Found 51500 records... Found 52000 records... Found 52500 records... Found 53000 records... Found 53500 records... Found 54000 records... Found 54500 records... Found 55000 records... Found 55500 records... Found 56000 records... Found 56500 records... Found 57000 records... Found 57500 records... Found 58000 records... Found 58500 records... Found 59000 records... Found 59500 records... Found 60000 records... Found 60500 records... Found 61000 records... Found 61500 records... Found 62000 records... Found 62500 records... Found 63000 records... Found 63500 records... Found 64000 records... Found 64500 records... Found 65000 records... Found 65500 records... Found 66000 records... Found 66500 records... Found 67000 records... Found 67500 records... Found 68000 records... Found 68500 records... Found 69000 records... Found 69500 records... Found 70000 records... Found 70500 records... Found 71000 records... Found 71500 records... Found 72000 records... Found 72500 records... Found 73000 records... Found 73500 records... Found 74000 records... Found 74500 records... Found 75000 records... Found 75500 records... Found 76000 records... Found 76500 records... Found 77000 records... Found 77500 records... Found 78000 records... Found 78500 records... Found 79000 records... Found 79500 records... Found 80000 records... Found 80500 records... Found 81000 records... Found 81500 records... Found 82000 records... Found 82500 records... Found 83000 records... Found 83500 records... Found 84000 records... Found 84500 records... Found 85000 records... Found 85500 records... Found 86000 records... Found 86500 records... Found 87000 records... Found 87500 records... Found 88000 records... Found 88500 records... Found 89000 records... Found 89500 records... Found 90000 records... Found 90500 records... Found 91000 records... Found 91500 records... Found 92000 records... Found 92500 records... Found 93000 records... Found 93500 records... Found 94000 records... Found 94500 records... Found 95000 records... Found 95500 records... Found 96000 records... Found 96500 records... Found 97000 records... Found 97500 records... Found 98000 records... Found 98500 records... Found 99000 records... Found 99500 records... Found 1e+05 records... Found 100500 records... Found 101000 records... Found 101500 records... Found 102000 records... Found 102500 records... Found 103000 records... Found 103500 records... Found 104000 records... Found 104500 records... Found 105000 records... Found 105500 records... Found 106000 records... Found 106500 records... Found 107000 records... Found 107500 records... Found 108000 records... Found 108500 records... Found 109000 records... Found 109500 records... Found 110000 records... Found 110500 records... Found 111000 records... Found 111500 records... Found 112000 records... Found 112500 records... Found 113000 records... Found 113500 records... Found 114000 records... Found 114500 records... Found 115000 records... Found 115500 records... Found 116000 records... Found 116500 records... Found 117000 records... Found 117500 records... Found 118000 records... Found 118500 records... Found 119000 records... Found 119500 records... Found 120000 records... Found 120500 records... Found 121000 records... Found 121500 records... Found 122000 records... Found 122500 records... Found 123000 records... Found 123500 records... Found 124000 records... Found 124500 records... Found 125000 records... Found 125500 records... Found 126000 records... Found 126500 records... Found 127000 records... Found 127500 records... Found 128000 records... Found 128500 records... Found 129000 records... Found 129500 records... Found 130000 records... Found 130500 records... Found 131000 records... Found 131500 records... Found 132000 records... Found 132500 records... Found 133000 records... Found 133500 records... Found 134000 records... Found 134500 records... Found 135000 records... Found 135500 records... Found 136000 records... Found 136500 records... Found 137000 records... Found 137500 records... Found 138000 records... Found 138500 records... Found 139000 records... Found 139500 records... Found 140000 records... Found 140500 records... Found 141000 records... Found 141500 records... Found 142000 records... Found 142500 records... Found 143000 records... Found 143500 records... Found 144000 records... Found 144500 records... Found 145000 records... Found 145500 records... Found 146000 records... Found 146500 records... Found 147000 records... Found 147500 records... Found 148000 records... Found 148500 records... Found 149000 records... Found 149500 records... Found 150000 records... Found 150500 records... Found 151000 records... Found 151500 records... Found 152000 records... Found 152500 records... Found 153000 records... Found 153500 records... Found 154000 records... Found 154500 records... Found 155000 records... Found 155500 records... Found 156000 records... Found 156500 records... Found 157000 records... Found 157500 records... Found 158000 records... Found 158500 records... Found 159000 records... Found 159500 records... Found 160000 records... Found 160500 records... Found 161000 records... Found 161500 records... Found 162000 records... Found 162500 records... Found 163000 records... Found 163500 records... Found 164000 records... Found 164500 records... Found 165000 records... Found 165500 records... Found 166000 records... Found 166500 records... Found 167000 records... Found 167500 records... Found 168000 records... Found 168500 records... Found 169000 records... Found 169500 records... Found 170000 records... Found 170500 records... Found 171000 records... Found 171500 records... Found 172000 records... Found 172500 records... Found 173000 records... Found 173500 records... Found 174000 records... Found 174500 records... Found 175000 records... Found 175500 records... Found 176000 records... Found 176500 records... Found 177000 records... Found 177500 records... Found 178000 records... Found 178500 records... Found 179000 records... Found 179500 records... Found 180000 records... Found 180500 records... Found 181000 records... Found 181500 records... Found 182000 records... Found 182500 records... Found 183000 records... Found 183500 records... Found 184000 records... Found 184500 records... Found 185000 records... Found 185500 records... Found 186000 records... Found 186500 records... Found 187000 records... Found 187500 records... Found 188000 records... Found 188500 records... Found 189000 records... Found 189500 records... Found 190000 records... Found 190500 records... Found 191000 records... Found 191500 records... Found 192000 records... Found 192500 records... Found 193000 records... Found 193500 records... Found 194000 records... Found 194500 records... Found 195000 records... Found 195500 records... Found 196000 records... Found 196500 records... Found 197000 records... Found 197500 records... Found 198000 records... Found 198500 records... Found 199000 records... Found 199500 records... Found 2e+05 records... Found 200500 records... Found 201000 records... Found 201500 records... Found 202000 records... Found 202500 records... Found 203000 records... Found 203500 records... Found 204000 records... Found 204500 records... Found 205000 records... Found 205500 records... Found 206000 records... Found 206500 records... Found 207000 records... Found 207500 records... Found 208000 records... Found 208500 records... Found 209000 records... Found 209500 records... Found 210000 records... Found 210500 records... Found 211000 records... Found 211500 records... Found 212000 records... Found 212500 records... Found 213000 records... Found 213500 records... Found 214000 records... Found 214500 records... Found 215000 records... Found 215500 records... Found 216000 records... Found 216500 records... Found 217000 records... Found 217500 records... Found 218000 records... Found 218500 records... Found 219000 records... Found 219500 records... Found 220000 records... Found 220500 records... Found 221000 records... Found 221500 records... Found 222000 records... Found 222500 records... Found 223000 records... Found 223500 records... Found 224000 records... Found 224500 records... Found 225000 records... Found 225500 records... Found 226000 records... Found 226500 records... Found 227000 records... Found 227500 records... Found 228000 records... Found 228500 records... Found 229000 records... Found 229500 records... Found 230000 records... Found 230500 records... Found 231000 records... Found 231500 records... Found 232000 records... Found 232500 records... Found 233000 records... Found 233500 records... Found 234000 records... Found 234500 records... Found 235000 records... Found 235500 records... Found 236000 records... Found 236500 records... Found 237000 records... Found 237500 records... Found 238000 records... Found 238500 records... Found 239000 records... Found 239500 records... Found 240000 records... Found 240500 records... Found 241000 records... Found 241500 records... Found 242000 records... Found 242500 records... Found 243000 records... Found 243500 records... Found 244000 records... Found 244500 records... Found 245000 records... Found 245500 records... Found 246000 records... Found 246500 records... Found 247000 records... Found 247500 records... Found 248000 records... Found 248500 records... Found 249000 records... Found 249500 records... Found 250000 records... Found 250500 records... Found 251000 records... Found 251500 records... Found 252000 records... Found 252500 records... Found 253000 records... Found 253500 records... Found 254000 records... Found 254500 records... Found 255000 records... Found 255500 records... Found 256000 records... Found 256500 records... Found 257000 records... Found 257500 records... Found 258000 records... Found 258500 records... Found 259000 records... Found 259500 records... Found 260000 records... Found 260500 records... Found 261000 records... Found 261500 records... Found 262000 records... Found 262500 records... Found 263000 records... Found 263500 records... Found 264000 records... Found 264500 records... Found 265000 records... Found 265500 records... Found 266000 records... Found 266500 records... Found 267000 records... Found 267500 records... Found 268000 records... Found 268500 records... Found 269000 records... Found 269500 records... Found 270000 records... Found 270500 records... Found 271000 records... Found 271500 records... Found 272000 records... Found 272500 records... Found 273000 records... Found 273500 records... Found 274000 records... Found 274500 records... Found 275000 records... Found 275500 records... Found 276000 records... Found 276500 records... Found 277000 records... Found 277500 records... Found 278000 records... Found 278500 records... Found 279000 records... Found 279500 records... Found 280000 records... Found 280500 records... Found 281000 records... Found 281500 records... Found 282000 records... Found 282500 records... Found 283000 records... Found 283500 records... Found 284000 records... Found 284500 records... Found 285000 records... Found 285500 records... Found 286000 records... Found 286500 records... Found 287000 records... Found 287500 records... Found 288000 records... Found 288500 records... Found 289000 records... Found 289500 records... Found 290000 records... Found 290500 records... Found 291000 records... Found 291500 records... Found 292000 records... Found 292500 records... Found 293000 records... Found 293500 records... Found 294000 records... Found 294500 records... Found 295000 records... Found 295500 records... Found 296000 records... Found 296500 records... Found 297000 records... Found 297500 records... Found 298000 records... Found 298500 records... Found 299000 records... Found 299500 records... Found 3e+05 records... Found 300500 records... Found 301000 records... Found 301500 records... Found 302000 records... Found 302500 records... Found 303000 records... Found 303500 records... Found 304000 records... Found 304500 records... Found 305000 records... Found 305500 records... Found 306000 records... Found 306500 records... Found 307000 records... Found 307500 records... Found 308000 records... Found 308500 records... Found 309000 records... Found 309500 records... Found 310000 records... Found 310500 records... Found 311000 records... Found 311500 records... Found 312000 records... Found 312500 records... Found 313000 records... Found 313500 records... Found 314000 records... Found 314500 records... Found 315000 records... Found 315500 records... Found 316000 records... Found 316500 records... Found 317000 records... Found 317500 records... Found 318000 records... Found 318500 records... Found 319000 records... Found 319500 records... Found 320000 records... Found 320500 records... Found 321000 records... Found 321500 records... Found 322000 records... Found 322500 records... Found 323000 records... Found 323500 records... Found 324000 records... Found 324500 records... Found 325000 records... Found 325500 records... Found 326000 records... Found 326500 records... Found 327000 records... Found 327500 records... Found 328000 records... Found 328500 records... Found 329000 records... Found 329500 records... Found 330000 records... Found 330500 records... Found 331000 records... Found 331500 records... Found 332000 records... Found 332500 records... Found 333000 records... Found 333500 records... Found 334000 records... Found 334500 records... Found 335000 records... Found 335500 records... Found 336000 records... Found 336500 records... Found 336776 records... Imported 336776 records. Simplifying...#>#> Error in as.data.frame(flights): object 'flights' not found# or alternatively: flights5 <- stream_in(gzcon(pipe("curl https://jeroen.github.io/data/nycflights13.json.gz")))#>#> Found 500 records... Found 1000 records... Found 1500 records... Found 2000 records... Found 2500 records... Found 3000 records... Found 3500 records... Found 4000 records... Found 4500 records... Found 5000 records... Found 5500 records... Found 6000 records... Found 6500 records... Found 7000 records... Found 7500 records... Found 8000 records... Found 8500 records... Found 9000 records... Found 9500 records... Found 10000 records... Found 10500 records... Found 11000 records... Found 11500 records... Found 12000 records... Found 12500 records... Found 13000 records... Found 13500 records... Found 14000 records... Found 14500 records... Found 15000 records... Found 15500 records... Found 16000 records... Found 16500 records... Found 17000 records... Found 17500 records... Found 18000 records... Found 18500 records... Found 19000 records... Found 19500 records... Found 20000 records... Found 20500 records... Found 21000 records... Found 21500 records... Found 22000 records... Found 22500 records... Found 23000 records... Found 23500 records... Found 24000 records... Found 24500 records... Found 25000 records... Found 25500 records... Found 26000 records... Found 26500 records... Found 27000 records... Found 27500 records... Found 28000 records... Found 28500 records... Found 29000 records... Found 29500 records... Found 30000 records... Found 30500 records... Found 31000 records... Found 31500 records... Found 32000 records... Found 32500 records... Found 33000 records... Found 33500 records... Found 34000 records... Found 34500 records... Found 35000 records... Found 35500 records... Found 36000 records... Found 36500 records... Found 37000 records... Found 37500 records... Found 38000 records... Found 38500 records... Found 39000 records... Found 39500 records... Found 40000 records... Found 40500 records... Found 41000 records... Found 41500 records... Found 42000 records... Found 42500 records... Found 43000 records... Found 43500 records... Found 44000 records... Found 44500 records... Found 45000 records... Found 45500 records... Found 46000 records... Found 46500 records... Found 47000 records... Found 47500 records... Found 48000 records... Found 48500 records... Found 49000 records... Found 49500 records... Found 50000 records... Found 50500 records... Found 51000 records... Found 51500 records... Found 52000 records... Found 52500 records... Found 53000 records... Found 53500 records... Found 54000 records... Found 54500 records... Found 55000 records... Found 55500 records... Found 56000 records... Found 56500 records... Found 57000 records... Found 57500 records... Found 58000 records... Found 58500 records... Found 59000 records... Found 59500 records... Found 60000 records... Found 60500 records... Found 61000 records... Found 61500 records... Found 62000 records... Found 62500 records... Found 63000 records... Found 63500 records... Found 64000 records... Found 64500 records... Found 65000 records... Found 65500 records... Found 66000 records... Found 66500 records... Found 67000 records... Found 67500 records... Found 68000 records... Found 68500 records... Found 69000 records... Found 69500 records... Found 70000 records... Found 70500 records... Found 71000 records... Found 71500 records... Found 72000 records... Found 72500 records... Found 73000 records... Found 73500 records... Found 74000 records... Found 74500 records... Found 75000 records... Found 75500 records... Found 76000 records... Found 76500 records... Found 77000 records... Found 77500 records... Found 78000 records... Found 78500 records... Found 79000 records... Found 79500 records... Found 80000 records... Found 80500 records... Found 81000 records... Found 81500 records... Found 82000 records... Found 82500 records... Found 83000 records... Found 83500 records... Found 84000 records... Found 84500 records... Found 85000 records... Found 85500 records... Found 86000 records... Found 86500 records... Found 87000 records... Found 87500 records... Found 88000 records... Found 88500 records... Found 89000 records... Found 89500 records... Found 90000 records... Found 90500 records... Found 91000 records... Found 91500 records... Found 92000 records... Found 92500 records... Found 93000 records... Found 93500 records... Found 94000 records... Found 94500 records... Found 95000 records... Found 95500 records... Found 96000 records... Found 96500 records... Found 97000 records... Found 97500 records... Found 98000 records... Found 98500 records... Found 99000 records... Found 99500 records... Found 1e+05 records... Found 100500 records... Found 101000 records... Found 101500 records... Found 102000 records... Found 102500 records... Found 103000 records... Found 103500 records... Found 104000 records... Found 104500 records... Found 105000 records... Found 105500 records... Found 106000 records... Found 106500 records... Found 107000 records... Found 107500 records... Found 108000 records... Found 108500 records... Found 109000 records... Found 109500 records... Found 110000 records... Found 110500 records... Found 111000 records... Found 111500 records... Found 112000 records... Found 112500 records... Found 113000 records... Found 113500 records... Found 114000 records... Found 114500 records... Found 115000 records... Found 115500 records... Found 116000 records... Found 116500 records... Found 117000 records... Found 117500 records... Found 118000 records... Found 118500 records... Found 119000 records... Found 119500 records... Found 120000 records... Found 120500 records... Found 121000 records... Found 121500 records... Found 122000 records... Found 122500 records... Found 123000 records... Found 123500 records... Found 124000 records... Found 124500 records... Found 125000 records... Found 125500 records... Found 126000 records... Found 126500 records... Found 127000 records... Found 127500 records... Found 128000 records... Found 128500 records... Found 129000 records... Found 129500 records... Found 130000 records... Found 130500 records... Found 131000 records... Found 131500 records... Found 132000 records... Found 132500 records... Found 133000 records... Found 133500 records... Found 134000 records... Found 134500 records... Found 135000 records... Found 135500 records... Found 136000 records... Found 136500 records... Found 137000 records... Found 137500 records... Found 138000 records... Found 138500 records... Found 139000 records... Found 139500 records... Found 140000 records... Found 140500 records... Found 141000 records... Found 141500 records... Found 142000 records... Found 142500 records... Found 143000 records... Found 143500 records... Found 144000 records... Found 144500 records... Found 145000 records... Found 145500 records... Found 146000 records... Found 146500 records... Found 147000 records... Found 147500 records... Found 148000 records... Found 148500 records... Found 149000 records... Found 149500 records... Found 150000 records... Found 150500 records... Found 151000 records... Found 151500 records... Found 152000 records... Found 152500 records... Found 153000 records... Found 153500 records... Found 154000 records... Found 154500 records... Found 155000 records... Found 155500 records... Found 156000 records... Found 156500 records... Found 157000 records... Found 157500 records... Found 158000 records... Found 158500 records... Found 159000 records... Found 159500 records... Found 160000 records... Found 160500 records... Found 161000 records... Found 161500 records... Found 162000 records... Found 162500 records... Found 163000 records... Found 163500 records... Found 164000 records... Found 164500 records... Found 165000 records... Found 165500 records... Found 166000 records... Found 166500 records... Found 167000 records... Found 167500 records... Found 168000 records... Found 168500 records... Found 169000 records... Found 169500 records... Found 170000 records... Found 170500 records... Found 171000 records... Found 171500 records... Found 172000 records... Found 172500 records... Found 173000 records... Found 173500 records... Found 174000 records... Found 174500 records... Found 175000 records... Found 175500 records... Found 176000 records... Found 176500 records... Found 177000 records... Found 177500 records... Found 178000 records... Found 178500 records... Found 179000 records... Found 179500 records... Found 180000 records... Found 180500 records... Found 181000 records... Found 181500 records... Found 182000 records... Found 182500 records... Found 183000 records... Found 183500 records... Found 184000 records... Found 184500 records... Found 185000 records... Found 185500 records... Found 186000 records... Found 186500 records... Found 187000 records... Found 187500 records... Found 188000 records... Found 188500 records... Found 189000 records... Found 189500 records... Found 190000 records... Found 190500 records... Found 191000 records... Found 191500 records... Found 192000 records... Found 192500 records... Found 193000 records... Found 193500 records... Found 194000 records... Found 194500 records... Found 195000 records... Found 195500 records... Found 196000 records... Found 196500 records... Found 197000 records... Found 197500 records... Found 198000 records... Found 198500 records... Found 199000 records... Found 199500 records... Found 2e+05 records... Found 200500 records... Found 201000 records... Found 201500 records... Found 202000 records... Found 202500 records... Found 203000 records... Found 203500 records... Found 204000 records... Found 204500 records... Found 205000 records... Found 205500 records... Found 206000 records... Found 206500 records... Found 207000 records... Found 207500 records... Found 208000 records... Found 208500 records... Found 209000 records... Found 209500 records... Found 210000 records... Found 210500 records... Found 211000 records... Found 211500 records... Found 212000 records... Found 212500 records... Found 213000 records... Found 213500 records... Found 214000 records... Found 214500 records... Found 215000 records... Found 215500 records... Found 216000 records... Found 216500 records... Found 217000 records... Found 217500 records... Found 218000 records... Found 218500 records... Found 219000 records... Found 219500 records... Found 220000 records... Found 220500 records... Found 221000 records... Found 221500 records... Found 222000 records... Found 222500 records... Found 223000 records... Found 223500 records... Found 224000 records... Found 224500 records... Found 225000 records... Found 225500 records... Found 226000 records... Found 226500 records... Found 227000 records... Found 227500 records... Found 228000 records... Found 228500 records... Found 229000 records... Found 229500 records... Found 230000 records... Found 230500 records... Found 231000 records... Found 231500 records... Found 232000 records... Found 232500 records... Found 233000 records... Found 233500 records... Found 234000 records... Found 234500 records... Found 235000 records... Found 235500 records... Found 236000 records... Found 236500 records... Found 237000 records... Found 237500 records... Found 238000 records... Found 238500 records... Found 239000 records... Found 239500 records... Found 240000 records... Found 240500 records... Found 241000 records... Found 241500 records... Found 242000 records... Found 242500 records... Found 243000 records... Found 243500 records... Found 244000 records... Found 244500 records... Found 245000 records... Found 245500 records... Found 246000 records... Found 246500 records... Found 247000 records... Found 247500 records... Found 248000 records... Found 248500 records... Found 249000 records... Found 249500 records... Found 250000 records... Found 250500 records... Found 251000 records... Found 251500 records... Found 252000 records... Found 252500 records... Found 253000 records... Found 253500 records... Found 254000 records... Found 254500 records... Found 255000 records... Found 255500 records... Found 256000 records... Found 256500 records... Found 257000 records... Found 257500 records... Found 258000 records... Found 258500 records... Found 259000 records... Found 259500 records... Found 260000 records... Found 260500 records... Found 261000 records... Found 261500 records... Found 262000 records... Found 262500 records... Found 263000 records... Found 263500 records... Found 264000 records... Found 264500 records... Found 265000 records... Found 265500 records... Found 266000 records... Found 266500 records... Found 267000 records... Found 267500 records... Found 268000 records... Found 268500 records... Found 269000 records... Found 269500 records... Found 270000 records... Found 270500 records... Found 271000 records... Found 271500 records... Found 272000 records... Found 272500 records... Found 273000 records... Found 273500 records... Found 274000 records... Found 274500 records... Found 275000 records... Found 275500 records... Found 276000 records... Found 276500 records... Found 277000 records... Found 277500 records... Found 278000 records... Found 278500 records... Found 279000 records... Found 279500 records... Found 280000 records... Found 280500 records... Found 281000 records... Found 281500 records... Found 282000 records... Found 282500 records... Found 283000 records... Found 283500 records... Found 284000 records... Found 284500 records... Found 285000 records... Found 285500 records... Found 286000 records... Found 286500 records... Found 287000 records... Found 287500 records... Found 288000 records... Found 288500 records... Found 289000 records... Found 289500 records... Found 290000 records... Found 290500 records... Found 291000 records... Found 291500 records... Found 292000 records... Found 292500 records... Found 293000 records... Found 293500 records... Found 294000 records... Found 294500 records... Found 295000 records... Found 295500 records... Found 296000 records... Found 296500 records... Found 297000 records... Found 297500 records... Found 298000 records... Found 298500 records... Found 299000 records... Found 299500 records... Found 3e+05 records... Found 300500 records... Found 301000 records... Found 301500 records... Found 302000 records... Found 302500 records... Found 303000 records... Found 303500 records... Found 304000 records... Found 304500 records... Found 305000 records... Found 305500 records... Found 306000 records... Found 306500 records... Found 307000 records... Found 307500 records... Found 308000 records... Found 308500 records... Found 309000 records... Found 309500 records... Found 310000 records... Found 310500 records... Found 311000 records... Found 311500 records... Found 312000 records... Found 312500 records... Found 313000 records... Found 313500 records... Found 314000 records... Found 314500 records... Found 315000 records... Found 315500 records... Found 316000 records... Found 316500 records... Found 317000 records... Found 317500 records... Found 318000 records... Found 318500 records... Found 319000 records... Found 319500 records... Found 320000 records... Found 320500 records... Found 321000 records... Found 321500 records... Found 322000 records... Found 322500 records... Found 323000 records... Found 323500 records... Found 324000 records... Found 324500 records... Found 325000 records... Found 325500 records... Found 326000 records... Found 326500 records... Found 327000 records... Found 327500 records... Found 328000 records... Found 328500 records... Found 329000 records... Found 329500 records... Found 330000 records... Found 330500 records... Found 331000 records... Found 331500 records... Found 332000 records... Found 332500 records... Found 333000 records... Found 333500 records... Found 334000 records... Found 334500 records... Found 335000 records... Found 335500 records... Found 336000 records... Found 336500 records... Found 336776 records... Imported 336776 records. Simplifying...#>#> Error in as.data.frame(flights): object 'flights' not found# Full JSON IO stream from URL to file connection. # Calculate delays for flights over 1000 miles in batches of 5k library(dplyr)#> #>#> #> #>#> #> #>con_in <- gzcon(url("http://jeroen.github.io/data/nycflights13.json.gz")) con_out <- file(tmp <- tempfile(), open = "wb") stream_in(con_in, handler = function(df){ df <- dplyr::filter(df, distance > 1000) df <- dplyr::mutate(df, delta = dep_delay - arr_delay) stream_out(df, con_out, pagesize = 1000) }, pagesize = 5000)#>#>#> Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2322 rows. #> Found 5000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2115 rows. #> Found 10000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2108 rows. #> Found 15000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2144 rows. #> Found 20000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2141 rows. #> Found 25000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2067 rows. #> Found 30000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2145 rows. #> Found 35000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2163 rows. #> Found 40000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2144 rows. #> Found 45000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2094 rows. #> Found 50000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2150 rows. #> Found 55000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2187 rows. #> Found 60000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2184 rows. #> Found 65000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2160 rows. #> Found 70000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2199 rows. #> Found 75000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2267 rows. #> Found 80000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2282 rows. #> Found 85000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2216 rows. #> Found 90000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2144 rows. #> Found 95000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2231 rows. #> Found 1e+05 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2400 rows. #> Found 105000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2492 rows. #> Found 110000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2258 rows. #> Found 115000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2127 rows. #> Found 120000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2125 rows. #> Found 125000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2188 rows. #> Found 130000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2196 rows. #> Found 135000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2189 rows. #> Found 140000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2183 rows. #> Found 145000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2161 rows. #> Found 150000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2201 rows. #> Found 155000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2206 rows. #> Found 160000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2213 rows. #> Found 165000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2187 rows. #> Found 170000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2249 rows. #> Found 175000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2206 rows. #> Found 180000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2221 rows. #> Found 185000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2172 rows. #> Found 190000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2162 rows. #> Found 195000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2153 rows. #> Found 2e+05 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2153 rows. #> Found 205000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2128 rows. #> Found 210000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2144 rows. #> Found 215000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2181 rows. #> Found 220000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2168 rows. #> Found 225000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2183 rows. #> Found 230000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2151 rows. #> Found 235000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2188 rows. #> Found 240000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2214 rows. #> Found 245000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2193 rows. #> Found 250000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2210 rows. #> Found 255000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2193 rows. #> Found 260000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2190 rows. #> Found 265000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2218 rows. #> Found 270000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2151 rows. #> Found 275000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2201 rows. #> Found 280000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2201 rows. #> Found 285000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2192 rows. #> Found 290000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2164 rows. #> Found 295000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2216 rows. #> Found 3e+05 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2209 rows. #> Found 305000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2180 rows. #> Found 310000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2137 rows. #> Found 315000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2136 rows. #> Found 320000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2072 rows. #> Found 325000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2117 rows. #> Found 330000 records... Processed 1000 rows... Processed 2000 rows... Complete! Processed total of 2125 rows. #> Found 335000 records... Complete! Processed total of 738 rows. #> Found 336776 records...#>#>#> Found 500 records... Found 1000 records... Found 1500 records... Found 2000 records... Found 2500 records... Found 3000 records... Found 3500 records... Found 4000 records... Found 4500 records... Found 5000 records... Found 5500 records... Found 6000 records... Found 6500 records... Found 7000 records... Found 7500 records... Found 8000 records... Found 8500 records... Found 9000 records... Found 9500 records... Found 10000 records... Found 10500 records... Found 11000 records... Found 11500 records... Found 12000 records... Found 12500 records... Found 13000 records... Found 13500 records... Found 14000 records... Found 14500 records... Found 15000 records... Found 15500 records... Found 16000 records... Found 16500 records... Found 17000 records... Found 17500 records... Found 18000 records... Found 18500 records... Found 19000 records... Found 19500 records... Found 20000 records... Found 20500 records... Found 21000 records... Found 21500 records... Found 22000 records... Found 22500 records... Found 23000 records... Found 23500 records... Found 24000 records... Found 24500 records... Found 25000 records... Found 25500 records... Found 26000 records... Found 26500 records... Found 27000 records... Found 27500 records... Found 28000 records... Found 28500 records... Found 29000 records... Found 29500 records... Found 30000 records... Found 30500 records... Found 31000 records... Found 31500 records... Found 32000 records... Found 32500 records... Found 33000 records... Found 33500 records... Found 34000 records... Found 34500 records... Found 35000 records... Found 35500 records... Found 36000 records... Found 36500 records... Found 37000 records... Found 37500 records... Found 38000 records... Found 38500 records... Found 39000 records... Found 39500 records... Found 40000 records... Found 40500 records... Found 41000 records... Found 41500 records... Found 42000 records... Found 42500 records... Found 43000 records... Found 43500 records... Found 44000 records... Found 44500 records... Found 45000 records... Found 45500 records... Found 46000 records... Found 46500 records... Found 47000 records... Found 47500 records... Found 48000 records... Found 48500 records... Found 49000 records... Found 49500 records... Found 50000 records... Found 50500 records... Found 51000 records... Found 51500 records... Found 52000 records... Found 52500 records... Found 53000 records... Found 53500 records... Found 54000 records... Found 54500 records... Found 55000 records... Found 55500 records... Found 56000 records... Found 56500 records... Found 57000 records... Found 57500 records... Found 58000 records... Found 58500 records... Found 59000 records... Found 59500 records... Found 60000 records... Found 60500 records... Found 61000 records... Found 61500 records... Found 62000 records... Found 62500 records... Found 63000 records... Found 63500 records... Found 64000 records... Found 64500 records... Found 65000 records... Found 65500 records... Found 66000 records... Found 66500 records... Found 67000 records... Found 67500 records... Found 68000 records... Found 68500 records... Found 69000 records... Found 69500 records... Found 70000 records... Found 70500 records... Found 71000 records... Found 71500 records... Found 72000 records... Found 72500 records... Found 73000 records... Found 73500 records... Found 74000 records... Found 74500 records... Found 75000 records... Found 75500 records... Found 76000 records... Found 76500 records... Found 77000 records... Found 77500 records... Found 78000 records... Found 78500 records... Found 79000 records... Found 79500 records... Found 80000 records... Found 80500 records... Found 81000 records... Found 81500 records... Found 82000 records... Found 82500 records... Found 83000 records... Found 83500 records... Found 84000 records... Found 84500 records... Found 85000 records... Found 85500 records... Found 86000 records... Found 86500 records... Found 87000 records... Found 87500 records... Found 88000 records... Found 88500 records... Found 89000 records... Found 89500 records... Found 90000 records... Found 90500 records... Found 91000 records... Found 91500 records... Found 92000 records... Found 92500 records... Found 93000 records... Found 93500 records... Found 94000 records... Found 94500 records... Found 95000 records... Found 95500 records... Found 96000 records... Found 96500 records... Found 97000 records... Found 97500 records... Found 98000 records... Found 98500 records... Found 99000 records... Found 99500 records... Found 1e+05 records... Found 100500 records... Found 101000 records... Found 101500 records... Found 102000 records... Found 102500 records... Found 103000 records... Found 103500 records... Found 104000 records... Found 104500 records... Found 105000 records... Found 105500 records... Found 106000 records... Found 106500 records... Found 107000 records... Found 107500 records... Found 108000 records... Found 108500 records... Found 109000 records... Found 109500 records... Found 110000 records... Found 110500 records... Found 111000 records... Found 111500 records... Found 112000 records... Found 112500 records... Found 113000 records... Found 113500 records... Found 114000 records... Found 114500 records... Found 115000 records... Found 115500 records... Found 116000 records... Found 116500 records... Found 117000 records... Found 117500 records... Found 118000 records... Found 118500 records... Found 119000 records... Found 119500 records... Found 120000 records... Found 120500 records... Found 121000 records... Found 121500 records... Found 122000 records... Found 122500 records... Found 123000 records... Found 123500 records... Found 124000 records... Found 124500 records... Found 125000 records... Found 125500 records... Found 126000 records... Found 126500 records... Found 127000 records... Found 127500 records... Found 128000 records... Found 128500 records... Found 129000 records... Found 129500 records... Found 130000 records... Found 130500 records... Found 131000 records... Found 131500 records... Found 132000 records... Found 132500 records... Found 133000 records... Found 133500 records... Found 134000 records... Found 134500 records... Found 135000 records... Found 135500 records... Found 136000 records... Found 136500 records... Found 137000 records... Found 137500 records... Found 138000 records... Found 138500 records... Found 139000 records... Found 139500 records... Found 140000 records... Found 140500 records... Found 141000 records... Found 141500 records... Found 142000 records... Found 142500 records... Found 143000 records... Found 143500 records... Found 144000 records... Found 144500 records... Found 145000 records... Found 145500 records... Found 146000 records... Found 146500 records... Found 147000 records... Found 147105 records... Imported 147105 records. Simplifying...#>nrow(mydata)#> [1] 147105unlink(tmp) # Data from http://openweathermap.org/current#bulk # Each row contains a nested data frame. daily14 <- stream_in(gzcon(url("http://78.46.48.103/sample/daily_14.json.gz")), pagesize=50)#>#> Found 50 records... Found 100 records... Found 150 records... Found 200 records... Found 250 records... Found 300 records... Found 350 records... Found 400 records... Found 450 records... Found 500 records... Found 550 records... Found 600 records... Found 650 records... Found 700 records... Found 750 records... Found 800 records... Found 850 records... Found 900 records... Found 950 records... Found 1000 records... Found 1050 records... Found 1100 records... Found 1150 records... Found 1200 records... Found 1250 records... Found 1300 records... Found 1350 records... Found 1400 records... Found 1450 records... Found 1500 records... Found 1550 records... Found 1600 records... Found 1650 records... Found 1700 records... Found 1750 records... Found 1800 records... Found 1850 records... Found 1900 records... Found 1950 records... Found 2000 records... Found 2050 records... Found 2100 records... Found 2150 records... Found 2200 records... Found 2250 records... Found 2300 records... Found 2350 records... Found 2400 records... Found 2450 records... Found 2500 records... Found 2550 records... Found 2600 records... Found 2650 records... Found 2700 records... Found 2750 records... Found 2800 records... Found 2850 records... Found 2900 records... Found 2950 records... Found 3000 records... Found 3050 records... Found 3100 records... Found 3150 records... Found 3200 records... Found 3250 records... Found 3300 records... Found 3350 records... Found 3400 records... Found 3450 records... Found 3500 records... Found 3550 records... Found 3600 records... Found 3650 records... Found 3700 records... Found 3750 records... Found 3800 records... Found 3850 records... Found 3900 records... Found 3950 records... Found 4000 records... Found 4050 records... Found 4100 records... Found 4150 records... Found 4200 records... Found 4250 records... Found 4300 records... Found 4350 records... Found 4400 records... Found 4450 records... Found 4500 records... Found 4550 records... Found 4600 records... Found 4650 records... Found 4700 records... Found 4750 records... Found 4800 records... Found 4850 records... Found 4900 records... Found 4950 records... Found 5000 records... Found 5050 records... Found 5100 records... Found 5150 records... Found 5200 records... Found 5250 records... Found 5300 records... Found 5350 records... Found 5400 records... Found 5450 records... Found 5500 records... Found 5550 records... Found 5600 records... Found 5650 records... Found 5700 records... Found 5750 records... Found 5800 records... Found 5850 records... Found 5900 records... Found 5950 records... Found 6000 records... Found 6050 records... Found 6100 records... Found 6150 records... Found 6200 records... Found 6250 records... Found 6300 records... Found 6350 records... Found 6400 records... Found 6450 records... Found 6500 records... Found 6550 records... Found 6600 records... Found 6650 records... Found 6700 records... Found 6750 records... Found 6800 records... Found 6850 records... Found 6900 records... Found 6950 records... Found 7000 records... Found 7050 records... Found 7100 records... Found 7150 records... Found 7200 records... Found 7250 records... Found 7300 records... Found 7350 records... Found 7400 records... Found 7450 records... Found 7500 records... Found 7550 records... Found 7600 records... Found 7650 records... Found 7700 records... Found 7750 records... Found 7800 records... Found 7850 records... Found 7900 records... Found 7950 records... Found 8000 records... Found 8050 records... Found 8100 records... Found 8150 records... Found 8200 records... Found 8250 records... Found 8300 records... Found 8350 records... Found 8400 records... Found 8450 records... Found 8500 records... Found 8550 records... Found 8600 records... Found 8650 records... Found 8700 records... Found 8750 records... Found 8800 records... Found 8850 records... Found 8900 records... Found 8950 records... Found 9000 records... Found 9050 records... Found 9100 records... Found 9150 records... Found 9200 records... Found 9250 records... Found 9300 records... Found 9350 records... Found 9400 records... Found 9450 records... Found 9500 records... Found 9550 records... Found 9600 records... Found 9650 records... Found 9700 records... Found 9750 records... Found 9800 records... Found 9850 records... Found 9900 records... Found 9950 records... Found 10000 records... Found 10050 records... Found 10100 records... Found 10150 records... Found 10200 records... Found 10250 records... Found 10300 records... Found 10350 records... Found 10400 records... Found 10450 records... Found 10500 records... Found 10550 records... Found 10600 records... Found 10650 records... Found 10700 records... Found 10750 records... Found 10800 records... Found 10850 records... Found 10900 records... Found 10950 records... Found 11000 records... Found 11050 records... Found 11100 records... Found 11150 records... Found 11200 records... Found 11250 records... Found 11300 records... Found 11350 records... Found 11400 records... Found 11450 records... Found 11500 records... Found 11550 records... Found 11600 records... Found 11650 records... Found 11700 records... Found 11750 records... Found 11800 records... Found 11850 records... Found 11900 records... Found 11950 records... Found 12000 records... Found 12050 records... Found 12100 records... Found 12150 records... Found 12200 records... Found 12250 records... Found 12300 records... Found 12350 records... Found 12400 records... Found 12450 records... Found 12500 records... Found 12550 records... Found 12600 records... Found 12650 records... Found 12700 records... Found 12750 records... Found 12800 records... Found 12850 records... Found 12900 records... Found 12950 records... Found 13000 records... Found 13050 records... Found 13100 records... Found 13150 records... Found 13200 records... Found 13250 records... Found 13300 records... Found 13350 records... Found 13400 records... Found 13450 records... Found 13500 records... Found 13550 records... Found 13600 records... Found 13650 records... Found 13700 records... Found 13750 records... Found 13800 records... Found 13850 records... Found 13900 records... Found 13950 records... Found 14000 records... Found 14050 records... Found 14100 records... Found 14150 records... Found 14200 records... Found 14250 records... Found 14300 records... Found 14350 records... Found 14400 records... Found 14450 records... Found 14500 records... Found 14550 records... Found 14600 records... Found 14650 records... Found 14700 records... Found 14750 records... Found 14800 records... Found 14850 records... Found 14900 records... Found 14950 records... Found 15000 records... Found 15050 records... Found 15100 records... Found 15150 records... Found 15200 records... Found 15250 records... Found 15300 records... Found 15350 records... Found 15400 records... Found 15450 records... Found 15500 records... Found 15550 records... Found 15600 records... Found 15650 records... Found 15700 records... Found 15750 records... Found 15800 records... Found 15850 records... Found 15900 records... Found 15950 records... Found 16000 records... Found 16050 records... Found 16100 records... Found 16150 records... Found 16200 records... Found 16250 records... Found 16300 records... Found 16350 records... Found 16400 records... Found 16450 records... Found 16500 records... Found 16550 records... Found 16600 records... Found 16650 records... Found 16700 records... Found 16750 records... Found 16800 records... Found 16850 records... Found 16900 records... Found 16950 records... Found 17000 records... Found 17050 records... Found 17100 records... Found 17150 records... Found 17200 records... Found 17250 records... Found 17300 records... Found 17350 records... Found 17400 records... Found 17450 records... Found 17500 records... Found 17550 records... Found 17600 records... Found 17650 records... Found 17700 records... Found 17750 records... Found 17800 records... Found 17850 records... Found 17900 records... Found 17950 records... Found 18000 records... Found 18050 records... Found 18100 records... Found 18150 records... Found 18200 records... Found 18250 records... Found 18300 records... Found 18350 records... Found 18400 records... Found 18450 records... Found 18500 records... Found 18550 records... Found 18600 records... Found 18650 records... Found 18700 records... Found 18750 records... Found 18800 records... Found 18850 records... Found 18900 records... Found 18950 records... Found 19000 records... Found 19050 records... Found 19100 records... Found 19150 records... Found 19200 records... Found 19250 records... Found 19300 records... Found 19350 records... Found 19400 records... Found 19450 records... Found 19500 records... Found 19550 records... Found 19600 records... Found 19650 records... Found 19700 records... Found 19750 records... Found 19800 records... Found 19850 records... Found 19900 records... Found 19950 records... Found 20000 records... Found 20050 records... Found 20100 records... Found 20150 records... Found 20200 records... Found 20250 records... Found 20300 records... Found 20350 records... Found 20400 records... Found 20450 records... Found 20500 records... Found 20550 records... Found 20600 records... Found 20650 records... Found 20700 records... Found 20750 records... Found 20800 records... Found 20850 records... Found 20900 records... Found 20950 records... Found 21000 records... Found 21050 records... Found 21100 records... Found 21150 records... Found 21200 records... Found 21250 records... Found 21300 records... Found 21350 records... Found 21400 records... Found 21450 records... Found 21500 records... Found 21550 records... Found 21600 records... Found 21650 records... Found 21700 records... Found 21750 records... Found 21800 records... Found 21850 records... Found 21900 records... Found 21950 records... Found 22000 records... Found 22050 records... Found 22100 records... Found 22150 records... Found 22200 records... Found 22250 records... Found 22300 records... Found 22350 records... Found 22400 records... Found 22450 records... Found 22500 records... Found 22550 records... Found 22600 records... Found 22635 records... Imported 22635 records. Simplifying...#>#> dt temp.day temp.min temp.max temp.night temp.eve temp.morn pressure #> 1 1489402800 280.98 271.47 280.98 273.71 277.89 271.47 1032.38 #> 2 1489489200 281.82 274.94 283.16 280.38 281.51 274.94 1035.70 #> 3 1489575600 283.10 277.72 283.23 277.72 280.76 280.69 1035.13 #> 4 1489662000 283.32 275.93 284.03 277.25 280.86 275.93 1033.85 #> 5 1489748400 282.55 279.71 282.55 279.71 281.11 280.18 1019.56 #> 6 1489834800 283.20 280.82 283.20 282.94 283.06 280.82 1004.91 #> 7 1489921200 281.82 277.83 281.82 277.83 279.85 281.75 996.66 #> 8 1490007600 280.77 276.65 280.77 278.55 280.35 276.65 1013.97 #> 9 1490094000 280.12 273.52 280.12 273.52 277.74 277.01 1017.81 #> 10 1490180400 281.10 272.59 281.10 275.45 278.06 272.59 1035.30 #> 11 1490266800 280.26 273.43 280.26 274.00 277.47 273.43 1038.20 #> 12 1490353200 279.84 273.41 279.84 275.21 277.81 273.41 1031.64 #> 13 1490439600 280.25 273.66 280.25 277.34 279.52 273.66 1020.69 #> 14 1490526000 282.67 275.82 282.67 277.65 280.88 275.82 1016.05 #> 15 1490612400 282.05 276.25 282.05 278.89 279.51 276.25 1014.98 #> 16 1490698800 282.48 279.95 282.48 279.95 280.60 280.95 1017.14 #> 17 1490785200 279.95 279.95 279.95 279.95 279.95 279.95 1022.25 #> humidity weather speed deg clouds uvi rain #> 1 0 500, Rain, light rain, 10d 1.82 161 23 1.90 NA #> 2 89 500, Rain, light rain, 10d 5.21 269 64 1.94 NA #> 3 77 800, Clear, sky is clear, 01d 7.86 295 0 2.18 NA #> 4 82 800, Clear, sky is clear, 01d 4.10 288 0 2.28 NA #> 5 0 500, Rain, light rain, 10d 7.48 269 46 2.38 0.50 #> 6 0 501, Rain, moderate rain, 10d 8.69 257 100 2.38 7.96 #> 7 0 501, Rain, moderate rain, 10d 9.81 248 59 1.91 10.29 #> 8 0 500, Rain, light rain, 10d 7.11 271 42 2.42 1.78 #> 9 0 500, Rain, light rain, 10d 6.61 338 2 2.42 NA #> 10 0 800, Clear, sky is clear, 01d 7.27 67 0 2.42 NA #> 11 0 800, Clear, sky is clear, 01d 6.70 79 0 2.42 NA #> 12 0 500, Rain, light rain, 10d 6.22 72 0 2.42 0.49 #> 13 0 500, Rain, light rain, 10d 8.70 101 17 2.42 NA #> 14 0 800, Clear, sky is clear, 01d 7.42 88 0 2.42 NA #> 15 0 500, Rain, light rain, 10d 10.47 66 1 2.42 0.77 #> 16 0 501, Rain, moderate rain, 10d 11.05 61 94 2.42 6.62 #> 17 0 500, Rain, light rain, 10d 9.86 52 100 2.42 0.61# Or with dplyr: library(dplyr) daily14f <- flatten(daily14) filter(daily14f, city.name == "Berlin")$data[[1]]#> dt temp.day temp.min temp.max temp.night temp.eve temp.morn pressure #> 1 1489402800 280.98 271.47 280.98 273.71 277.89 271.47 1032.38 #> 2 1489489200 281.82 274.94 283.16 280.38 281.51 274.94 1035.70 #> 3 1489575600 283.10 277.72 283.23 277.72 280.76 280.69 1035.13 #> 4 1489662000 283.32 275.93 284.03 277.25 280.86 275.93 1033.85 #> 5 1489748400 282.55 279.71 282.55 279.71 281.11 280.18 1019.56 #> 6 1489834800 283.20 280.82 283.20 282.94 283.06 280.82 1004.91 #> 7 1489921200 281.82 277.83 281.82 277.83 279.85 281.75 996.66 #> 8 1490007600 280.77 276.65 280.77 278.55 280.35 276.65 1013.97 #> 9 1490094000 280.12 273.52 280.12 273.52 277.74 277.01 1017.81 #> 10 1490180400 281.10 272.59 281.10 275.45 278.06 272.59 1035.30 #> 11 1490266800 280.26 273.43 280.26 274.00 277.47 273.43 1038.20 #> 12 1490353200 279.84 273.41 279.84 275.21 277.81 273.41 1031.64 #> 13 1490439600 280.25 273.66 280.25 277.34 279.52 273.66 1020.69 #> 14 1490526000 282.67 275.82 282.67 277.65 280.88 275.82 1016.05 #> 15 1490612400 282.05 276.25 282.05 278.89 279.51 276.25 1014.98 #> 16 1490698800 282.48 279.95 282.48 279.95 280.60 280.95 1017.14 #> 17 1490785200 279.95 279.95 279.95 279.95 279.95 279.95 1022.25 #> humidity weather speed deg clouds uvi rain #> 1 0 500, Rain, light rain, 10d 1.82 161 23 1.90 NA #> 2 89 500, Rain, light rain, 10d 5.21 269 64 1.94 NA #> 3 77 800, Clear, sky is clear, 01d 7.86 295 0 2.18 NA #> 4 82 800, Clear, sky is clear, 01d 4.10 288 0 2.28 NA #> 5 0 500, Rain, light rain, 10d 7.48 269 46 2.38 0.50 #> 6 0 501, Rain, moderate rain, 10d 8.69 257 100 2.38 7.96 #> 7 0 501, Rain, moderate rain, 10d 9.81 248 59 1.91 10.29 #> 8 0 500, Rain, light rain, 10d 7.11 271 42 2.42 1.78 #> 9 0 500, Rain, light rain, 10d 6.61 338 2 2.42 NA #> 10 0 800, Clear, sky is clear, 01d 7.27 67 0 2.42 NA #> 11 0 800, Clear, sky is clear, 01d 6.70 79 0 2.42 NA #> 12 0 500, Rain, light rain, 10d 6.22 72 0 2.42 0.49 #> 13 0 500, Rain, light rain, 10d 8.70 101 17 2.42 NA #> 14 0 800, Clear, sky is clear, 01d 7.42 88 0 2.42 NA #> 15 0 500, Rain, light rain, 10d 10.47 66 1 2.42 0.77 #> 16 0 501, Rain, moderate rain, 10d 11.05 61 94 2.42 6.62 #> 17 0 500, Rain, light rain, 10d 9.86 52 100 2.42 0.61# Stream import large data from zip file tmp <- tempfile() download.file("http://jsonstudio.com/wp-content/uploads/2014/02/companies.zip", tmp)#> Warning: cannot open URL 'http://jsonstudio.com/wp-content/uploads/2014/02/companies.zip': HTTP status was '404 Not Found'#> Error in download.file("http://jsonstudio.com/wp-content/uploads/2014/02/companies.zip", tmp): cannot open URL 'http://jsonstudio.com/wp-content/uploads/2014/02/companies.zip'#>#> Warning: cannot open zip file '/var/folders/l5/gk7g18z52876d4lbk9tjhr0c0000gn/T//Rtmpy97g5O/filee3e664a962d7'#> Error in open.connection(con, "rb"): cannot open the connection# }