The curl() and curl_download() functions provide highly configurable drop-in replacements for base url() and download.file() with better performance, support for encryption (https, ftps), gzip compression, authentication, and other libcurl goodies. The core of the package implements a framework for performing fully customized requests where data can be processed either in memory, on disk, or streaming via the callback or connection interfaces. Some knowledge of libcurl is recommended; for a more-user-friendly web client see the ‘httr’ package which builds on this package with http specific tools and logic.
Note that you cannot use
devtools::install_github() because it uses
About the R package:
There are three download interfaces (memory, disk and streaming). Always start by setting up a request handle:
library(curl) h <- new_handle(copypostfields = "moo=moomooo") handle_setheaders(h, "Content-Type" = "text/moo", "Cache-Control" = "no-cache", "User-Agent" = "A cow" )
Perform request and download response in memory:
# Perform the request req <- curl_fetch_memory("http://httpbin.org/post", handle = h) # Show some outputs parse_headers(req$headers) cat(rawToChar(req$content)) str(req)
Or alternatively, write response to disk:
Or stream response via Connection interface:
Binary packages for OS-X or Windows can be installed directly from CRAN:
sudo apt-get install -y libcurl-dev
On Fedora, CentOS or RHEL use libcurl-devel:
sudo yum install libcurl-devel
On OS-X libcurl is included with the system so nothing extra is needed. However if you want to build against the most recent version of libcurl, install and force-link curl from homebrew:
brew install curl brew link --force curl
Note that on OS-X you must recompile the R package from source after force-linking curl, otherwise you get a version conflict with the system version of libcurl.
httr depend on
curl, installing with
install_github does not work well. The easiest way to install the development version of
curl is a clean R session:
Of course windows users need Rtools to compile from source.
To use HTTP/2 the
libcurl library has to be built
--with-nghttp2 which is usually not the default. Use
curl_version() in R to check if it is enabled. To enable it on OSX use:
brew install curl-openssl export PKG_CONFIG_PATH="/usr/local/opt/curl-openssl/lib/pkgconfig" R
And then reinstall the
curl package from source:
curl_version() to see if HTTP/2 is enabled now.