Public Interface

Requests

HTTP.request — Function

HTTP.request(method, url [, headers [, body]]; <keyword arguments>]) -> HTTP.Response

Send a HTTP Request Message and receive a HTTP Response Message.

e.g.

r = HTTP.request("GET", "http://httpbin.org/ip")
println(r.status)
println(String(r.body))

headers can be any collection where [string(k) => string(v) for (k,v) in headers] yields Vector{Pair}. e.g. a Dict(), a Vector{Tuple}, a Vector{Pair} or an iterator.

body can take a number of forms:

a String, a Vector{UInt8} or any T accepted by write(::IO, ::T)
a collection of String or AbstractVector{UInt8} or IO streams or items of any type T accepted by write(::IO, ::T...)
a readable IO stream or any IO-like type T for which eof(T) and readavailable(T) are defined.

The HTTP.Response struct contains:

status::Int16 e.g. 200
headers::Vector{Pair{String,String}} e.g. ["Server" => "Apache", "Content-Type" => "text/html"]
body::Vector{UInt8}, the Response Body bytes (empty if a response_stream was specified in the request).

Functions HTTP.get, HTTP.put, HTTP.post and HTTP.head are defined as shorthand for HTTP.request("GET", ...), etc.

HTTP.request and HTTP.open also accept optional keyword parameters.

e.g.

HTTP.request("GET", "http://httpbin.org/ip"; retries=4, cookies=true)

HTTP.get("http://s3.us-east-1.amazonaws.com/"; aws_authorization=true)

conf = (readtimeout = 10,
        pipeline_limit = 4,
        retry = false,
        redirect = false)

HTTP.get("http://httpbin.org/ip"; conf...)
HTTP.put("http://httpbin.org/put", [], "Hello"; conf...)

URL options

query = nothing, replaces the query part of url.

Streaming options

response_stream = nothing, a writeable IO stream or any IO-like type T for which write(T, AbstractVector{UInt8}) is defined.
verbose = 0, set to 1 or 2 for extra message logging.

Connection Pool options

connect_timeout = 0, close the connection after this many seconds if it is still attempting to connect. Use connect_timeout = 0 to disable.
connection_limit = 8, number of concurrent connections to each host:port.
pipeline_limit = 16, number of concurrent requests per connection.
reuse_limit = nolimit, number of times a connection is reused after the first request.
socket_type = TCPSocket

Timeout options

readtimeout = 0, close the connection if no data is received for this many seconds. Use readtimeout = 0 to disable.

Retry options

retry = true, retry idempotent requests in case of error.
retries = 4, number of times to retry.
retry_non_idempotent = false, retry non-idempotent requests too. e.g. POST.

Redirect options

redirect = true, follow 3xx redirect responses.
redirect_limit = 3, number of times to redirect.
forwardheaders = true, forward original headers on redirect.

Status Exception options

status_exception = true, throw HTTP.StatusError for response status >= 300.

SSLContext options

require_ssl_verification = NetworkOptions.verify_host(host), pass MBEDTLS_SSL_VERIFY_REQUIRED to the mbed TLS library. "... peer must present a valid certificate, handshake is aborted if verification failed."
sslconfig = SSLConfig(require_ssl_verification)

Basic Authentication options

Basic authentication is detected automatically from the provided url's userinfo (in the form scheme://user:password@host) and adds the Authorization: Basic header

AWS Authentication options

aws_authorization = false, enable AWS4 Authentication.
aws_service = split(url.host, ".")[1]
aws_region = split(url.host, ".")[2]
aws_access_key_id = ENV["AWS_ACCESS_KEY_ID"]
aws_secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
aws_session_token = get(ENV, "AWS_SESSION_TOKEN", "")
body_sha256 = digest(MD_SHA256, body),
body_md5 = digest(MD_MD5, body),

Cookie options

cookies::Union{Bool, Dict{<:AbstractString, <:AbstractString}} = false, enable cookies, or alternatively, pass a Dict{AbstractString, AbstractString} of name-value pairs to manually pass cookies
cookiejar::Dict{String, Set{Cookie}}=default_cookiejar,

Canonicalization options

canonicalize_headers = false, rewrite request and response headers in Canonical-Camel-Dash-Format.

Proxy options

proxy = proxyurl, pass request through a proxy given as a url

Alternatively, HTTP.jl also respects the http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY, and no_proxy environment variables; if set, they will be used automatically when making requests.

Request Body Examples

String body:

HTTP.request("POST", "http://httpbin.org/post", [], "post body data")

Stream body from file:

io = open("post_data.txt", "r")
HTTP.request("POST", "http://httpbin.org/post", [], io)

Generator body:

chunks = ("chunk$i" for i in 1:1000)
HTTP.request("POST", "http://httpbin.org/post", [], chunks)

Collection body:

chunks = [preamble_chunk, data_chunk, checksum(data_chunk)]
HTTP.request("POST", "http://httpbin.org/post", [], chunks)

open() do io body:

HTTP.open("POST", "http://httpbin.org/post") do io
    write(io, preamble_chunk)
    write(io, data_chunk)
    write(io, checksum(data_chunk))
end

Response Body Examples

String body:

r = HTTP.request("GET", "http://httpbin.org/get")
println(String(r.body))

Stream body to file:

io = open("get_data.txt", "w")
r = HTTP.request("GET", "http://httpbin.org/get", response_stream=io)
close(io)
println(read("get_data.txt"))

Stream body through buffer:

io = Base.BufferStream()
@async while !eof(io)
    bytes = readavailable(io)
    println("GET data: $bytes")
end
r = HTTP.request("GET", "http://httpbin.org/get", response_stream=io)
close(io)

Stream body through open() do io:

r = HTTP.open("GET", "http://httpbin.org/stream/10") do io
   while !eof(io)
       println(String(readavailable(io)))
   end
end

using HTTP.IOExtras

HTTP.open("GET", "https://tinyurl.com/bach-cello-suite-1-ogg") do http
    n = 0
    r = startread(http)
    l = parse(Int, HTTP.header(r, "Content-Length"))
    open(`vlc -q --play-and-exit --intf dummy -`, "w") do vlc
        while !eof(http)
            bytes = readavailable(http)
            write(vlc, bytes)
            n += length(bytes)
            println("streamed $n-bytes $((100*n)÷l)%\u1b[1A")
        end
    end
end

Request and Response Body Examples

String bodies:

r = HTTP.request("POST", "http://httpbin.org/post", [], "post body data")
println(String(r.body))

Interfacing with RESTful JSON APIs:

using JSON
params = Dict("user"=>"RAO...tjN", "token"=>"NzU...Wnp", "message"=>"Hello!")
base_url = "http://api.domain.com"
endpoint = "/1/messages.json"
url = base_url * endpoint
r = HTTP.request("POST", url,
             ["Content-Type" => "application/json"],
             JSON.json(params))
println(JSON.parse(String(r.body)))

Stream bodies from and to files:

in = open("foo.png", "r")
out = open("foo.jpg", "w")
HTTP.request("POST", "http://convert.com/png2jpg", [], in, response_stream=out)

Stream bodies through: open() do io:

using HTTP.IOExtras

HTTP.open("POST", "http://music.com/play") do io
    write(io, JSON.json([
        "auth" => "12345XXXX",
        "song_id" => 7,
    ]))
    r = startread(io)
    @show r.status
    while !eof(io)
        bytes = readavailable(io)
        play_audio(bytes)
    end
end