Client API

Transport(; tls_config=nothing, proxy=nothing, retry_bucket=RetryBucket(),
          max_idle_per_host=2, max_idle_total=64, max_conns_per_host=0,
          idle_timeout_ns=90_000_000_000, max_header_bytes=1024*1024,
          max_line_bytes=64*1024, local_addr=nothing)

The connection pool of a Client, with the settings for opening connections.

• tls_config: a Reseau.TLS.Config for https:// connections.
• proxy: a ProxyConfig, a proxy URL, or nothing to connect directly. A Client created without a transport uses ProxyFromEnvironment instead.
• retry_bucket: the RetryBucket that limits retries, or nothing.
• max_idle_per_host, max_idle_total: how many idle HTTP/1 connections to keep for one host and in all.
• max_conns_per_host: the most HTTP/1 connections to one host, counting idle, busy, and opening ones; further requests wait for one. 0 means no limit.
• idle_timeout_ns: how long, in nanoseconds, an idle HTTP/1 or HTTP/2 connection stays reusable; 0 means no limit. Keep it below the idle timeout of NATs and load balancers on the path, often a few minutes, so that requests are not sent on connections they dropped. A response header Keep-Alive: timeout=N shortens it to N - 1 seconds for that connection.
• max_line_bytes, max_header_bytes: the longest HTTP/1 response line and the largest response head (and chunked trailers); a longer one fails the request with a ProtocolError. max_line_bytes defaults to the smaller of 64 KiB and max_header_bytes. Raise it for servers that send very long header lines, such as large Content-Security-Policy values.
• local_addr: the source IP address of connections, as a string or a Reseau.TCP.SocketAddrV4/SocketAddrV6.

Client(; transport=Transport(proxy=ProxyFromEnvironment()), cookiejar=CookieJar(),
       max_redirects=10, check_redirect=nothing, prefer_http2=true,
       http2_settings=HTTP2Settings(), default_headers=[], default_query=nothing,
       default_basicauth=nothing, connect_timeout=0, request_timeout=0,
       response_header_timeout=0, read_idle_timeout=0, write_idle_timeout=0,
       local_addr=nothing)

An HTTP client: a connection pool plus the settings and cookies that requests sent through it share. Pass it to request, the verb helpers, or open as the first argument or as the client keyword. Calls without a client use a shared default Client. close(client) closes its connections; using a closed client throws ArgumentError.

• transport: the Transport that owns the connections. The default one takes its proxy from the environment.
• cookiejar: the CookieJar for this client, or nothing for no cookies.
• max_redirects: the most redirects one request follows.
• check_redirect: a function (response, request, location) -> Bool; return false to stop at a redirect and return it.
• prefer_http2: try HTTP/2 for https:// URLs. When an origin turns out to speak only HTTP/1.1, the client remembers that until close_idle_connections!. protocol = :h2 on a call always tries HTTP/2.
• http2_settings: the HTTP2Settings for HTTP/2 connections.
• default_headers: headers added to each request that does not set them.
• default_query: query parameters added to each URL; the call's query replaces those with the same key.
• default_basicauth: user => password used unless the call passes basicauth or an Authorization header.
• connect_timeout, request_timeout, response_header_timeout, read_idle_timeout, write_idle_timeout: timeouts in seconds, used when the call passes no value or 0. 0 here means no default.
• local_addr: the source IP address for connections, as a string or a Reseau.TCP.SocketAddrV4/SocketAddrV6. With your own transport, set it on the Transport instead.

RetryBucket(; backoff_scale_factor_ms=25, max_backoff_secs=20, capacity=500)

A retry budget shared by the requests of a Transport, so that a burst of failures does not become a burst of retries. Each host has its own budget of capacity. A retry reserves part of it; the reservation comes back when the retried attempt gets a response that is not worth retrying, and is kept, in whole or in part, when the attempt fails again. Each request that succeeds without a retry earns back one unit. When a host's budget runs out, failures are not retried, and a RetrySkippedEvent with reason :retry_bucket reports it.

Retries wait a random time of up to backoff_scale_factor_ms * 2^(attempt - 1) milliseconds, and at most max_backoff_secs seconds.

RequestRetryError(err)

What a retry_if function receives as err when an attempt failed without a response. err.err is the exception.

retry_attempts(request) -> Int
retry_attempts(response) -> Int

Return how many times the client retried the request, 0 when the first attempt was the only one. The count is also stored as request.context[:retryattempt].

isrecoverable(err::Exception) -> Bool

Return true when err is a transient failure that the client's default retry rules retry: a connection that was reset or closed early, a socket error, a malformed response, a connect or TLS handshake timeout, or an HTTP/2 stream the server refused. An expired deadline and other errors return false. err may also be the RequestRetryError that a retry_if function receives.

roundtrip!(transport, address, request; secure=false, server_name=nothing) -> Response

Send request to address ("host:port") over one HTTP/1 connection from transport, without redirects, cookies, or the client's retry rules. secure = true uses TLS, with server_name for SNI and certificate checks. The response body is a stream; read it with body_read! and close it with body_close!.

request(method, url, headers=[], body=nothing; kwargs...) -> Response
request(trace, method, url, headers=[], body=nothing; kwargs...) -> Response

Send an HTTP request and return the Response, with the whole body in response.body as a Vector{UInt8}.

url is a string or URI. headers is anything Headers accepts. body may be a String, a byte vector, an IO, a Dict or NamedTuple (sent as a URL-encoded form), a Form, an iterable of string or byte chunks, or an AbstractBody. headers and body can also be passed as keywords. Unless the request sets them, HTTP.jl sends User-Agent: HTTP.jl/<version> and Accept-Encoding: gzip, deflate.

A response with status 300 or above throws a StatusError, except for redirect responses (301, 302, 303, 307, and 308). A timeout throws a TimeoutError, and a failure to connect throws an error such as ConnectError or DNSError.

Keywords

Request and response:

• query: a Dict, NamedTuple, vector of pairs, or string to add to the URL's query string.
• basicauth: credentials as user => password or (user, password). An Authorization header takes precedence; credentials in the URL are used when neither is given.
• status_exception = true: throw StatusError for a failing response.
• response_stream: an IO or byte vector that receives the body instead of response.body, which is then nothing for an IO and the filled vector or view otherwise. A byte vector must be large enough for the body, and is resized to it if it is a Vector{UInt8}. A response with status 300 or above, including a redirect that is not followed, leaves a byte vector untouched and puts up to 1 MiB of its body (or max_decompressed_size, if smaller) in response.body.
• decompress = true: decompress gzip and deflate responses. false returns the body as sent and does not send Accept-Encoding.
• max_decompressed_size = 64 MiB: the largest decompressed body; a larger one throws DecompressionLimitError. 0 means no limit.
• sse_callback: for a server-sent event stream, call f(event) or f(stream, event) with each SSEEvent instead of reading the body. max_sse_line_bytes = 1 MiB and max_sse_event_bytes = 16 MiB limit the size of a line and an event; 0 means no limit.
• copyheaders = true: false lets the request use an HTTP.Headers collection without copying it. Do not touch it until the call returns; its contents afterwards are unspecified.
• canonicalize_headers = false: true sends header names in canonical_header_key form. Otherwise names are sent as given, and HTTP/2 sends them in lowercase.

Redirects:

• redirect = true: follow redirects.
• redirect_limit: the most redirects to follow; defaults to the client's max_redirects, 10 by default. With 0, the redirect response is returned.
• redirect_method: the method for following a 301 or 302 redirect, or :same to keep the original method. By default GET is used, except that HEAD and QUERY are kept. A 303 always uses GET, and 307 and 308 keep the method.
• forwardheaders = true: send the request's headers again when following a redirect. Credentials are dropped when the redirect goes to another origin.

Retries (see the client guide for the default rules):

• retry = true: retry failed attempts. An idempotent request that fails on a reused connection the server already closed is resent either way.
• retries = 4: the most retries after the first attempt.
• retry_non_idempotent = false: also retry methods other than GET, HEAD, OPTIONS, TRACE, QUERY, PUT, and DELETE.
• retry_if: a function (attempt, err, req, resp) that returns true to retry, false to stop, or nothing for the default rules. For a failure without a response, err is a RequestRetryError and resp is nothing; for a response, err is nothing. A body that cannot be sent again is never retried.
• respect_retry_after = true: wait as long as a 429 or 503 response's Retry-After header asks.
• retry_bucket = true: use the transport's RetryBucket to limit retries; pass false to skip it, or another RetryBucket.

Timeouts, in seconds:

• connect_timeout: DNS lookup, TCP connect, proxy handshake, TLS handshake, and HTTP/2 setup. Defaults to the client's connect_timeout, or 30.
• request_timeout: the whole call.
• response_header_timeout: the wait for the response headers.
• read_idle_timeout: the time between reads, including the wait for the response headers.
• write_idle_timeout: the time between writes.
• expect_continue_timeout: how long a request with an Expect: 100-continue header waits for 100 Continue before it sends the body anyway, 1 second by default. 0 sends the body without waiting.
• context: a RequestContext for cancellation and an absolute deadline.

Connections:

• client: the Client to use; defaults to a shared client.
• proxy: a proxy URL, a ProxyConfig, or nothing to connect directly. Cannot be combined with client; set the proxy on its Transport.
• cookies = true: send and store cookies with the cookie jar. false turns cookies off, and a Dict adds name/value pairs. Cookies from the jar are merged into a Cookie header set on the request.
• cookiejar: the CookieJar to use; defaults to the client's jar, or HTTP.COOKIEJAR for calls without a client.
• require_ssl_verification = true: false skips certificate checks, for testing only. Cannot be combined with client.
• protocol = :auto: :h1 or :h2 to force HTTP/1.1 or HTTP/2.

Debugging:

• verbose: 1 (or true) prints a line for each attempt, response, retry, redirect, and the result; 2 also prints the request and response heads.
• trace: a function called with each RequestEvent, ResponseHeadEvent, RetryEvent, RetrySkippedEvent, RedirectEvent, and DoneEvent.

Extended help

These HTTP.jl 1.x keywords are accepted but have no effect, and log a warning: pool, retry_delays, retry_check, sslconfig, socket_type_tls, detect_content_type, observelayers, logerrors, and logtag. The deprecated readtimeout sets read_idle_timeout.

get(headers, key, default)

Dict-style get on Headers. Returns the first value for key, including an empty value, or default if the header is absent.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

get(url, headers=[]; kwargs...) -> Response
head(url, headers=[]; kwargs...) -> Response
options(url, headers=[]; kwargs...) -> Response
post(url, [headers], [body]; kwargs...) -> Response
put(url, [headers], [body]; kwargs...) -> Response
patch(url, [headers], [body]; kwargs...) -> Response
delete(url, [headers], [body]; kwargs...) -> Response
query(url, [headers], [body]; kwargs...) -> Response

Send a request with the method in the function's name: HTTP.get(url) is HTTP.request("GET", url). Each takes the keywords of request, including headers and body, and also accepts a Client as its first argument.

open(method, url, headers=[]; kwargs...) -> Stream
open(f, method, url, headers=[]; status_exception=true, kwargs...) -> Response

Send a request and stream its body and response through a Stream. Write the request body to the stream, call startread to finish the request and get the response status and headers, then read the response body from the stream like any IO.

With a function f, open calls f(stream), finishes the exchange, and returns the final Response, not the value of f. It throws a StatusError for a failing response unless status_exception = false. Without f, close the stream with closewrite and closeread when you are done.

open follows redirects, retries, decompresses, and applies timeouts like request, and takes its keywords except body, response_stream, sse_callback, trace, verbose, and the size limits.

buf = IOBuffer()
response = HTTP.open("GET", "https://example.com") do stream
    write(buf, stream)  # copy the body as it arrives
end

do!(client, address, request; secure=false, server_name=nothing, protocol=:auto,
    redirect_limit=nothing, redirect_method=nothing, forwardheaders=true,
    cookies=true, cookiejar=nothing, context=nothing) -> Response

Send a prepared request to address ("host:port") through client, following redirects and using cookies like request, but without retries or status errors. secure = true connects with TLS.

The response body is a stream: read it with body_read! and close it with body_close!.

get!(client, address, target; secure=false, protocol=:auto, kwargs...) -> Response

Send a GET request for target to address with do!.

HTTP.@client request_middleware
HTTP.@client request_middleware stream_middleware

Define request, the verb helpers (get, post, ...), and open in the calling module so that they pass through middleware before reaching HTTP.jl.

A middleware is a function mw(next) -> f. Request middleware returns f(method, url, headers, body; kwargs...), which handles request and the verb helpers. Stream middleware returns f(method, url, headers; kwargs...), which handles open. Each position takes one middleware or a tuple of them; in (outer, inner), outer runs first.

close_idle_connections!()
close_idle_connections!(client::HTTP.Client)
close_idle_connections!(transport::HTTP.Transport)

Close the idle connections in a pool; requests in progress are not affected. Without an argument, it uses the shared client of calls that pass no client.

The Client and no-argument forms also close idle HTTP/2 connections, and forget which origins answered with HTTP/1.1, so later requests try HTTP/2 with them again. The Transport form covers only its HTTP/1 connections.

idle_connection_count(transport; key=nothing) -> Int

Return the number of idle HTTP/1 connections in transport, or only those for key, a scheme and address such as "https://example.com:443".

isaborted(stream) -> Bool

Return true when the response on a client stream ended in an error, so its connection cannot be reused.

RequestEvent

Trace event emitted immediately before a high-level request attempt is sent.

Fields:

• request: request head/body metadata for the attempt
• url: absolute request URL for the attempt
• attempt: 1-based request attempt number
• redirect_count: number of redirects already followed before this attempt
• protocol: :h1 or :h2 for the selected wire protocol

ResponseHeadEvent

Trace event emitted after response headers are available for a successful request attempt and before the body is fully consumed.

Fields:

• response: response head metadata for the attempt
• url: absolute request URL for the attempt
• attempt: 1-based request attempt number
• redirect_count: number of redirects followed before this response

RetryEvent

Trace event emitted when the high-level request path schedules another attempt.

Fields:

• request: request metadata that will be retried
• url: absolute request URL for the attempt being retried
• attempt: current 1-based attempt number
• next_attempt: next 1-based attempt number
• redirect_count: redirects already followed when the retry decision was made
• delay_ns: delay before the next attempt in nanoseconds
• response: retry-triggering response, or nothing for request-path failures
• err: retry-triggering exception, or nothing for response-based retries

RetrySkippedEvent

Trace event emitted when the retry policy wanted to retry an attempt but the retry was not armed, so the attempt's failure becomes the request's outcome. Without this event a denied retry is indistinguishable from a non-retryable failure.

Fields:

• request: request metadata for the attempt that will not be retried
• url: absolute request URL for the attempt
• attempt: current 1-based attempt number
• redirect_count: redirects already followed when the retry decision was made
• reason: :retry_bucket when the transport's RetryBucket denied retry capacity; :deadline when the request deadline preempted the retry backoff
• response: retry-triggering response, or nothing for request-path failures
• err: retry-triggering exception, or nothing for response-based retries

RedirectEvent

Trace event emitted when a redirect response is accepted and a follow-up request is about to be issued.

Fields:

• request: original request metadata that produced the redirect response
• response: redirect response head
• from_url: original absolute request URL
• to_url: resolved redirect target URL
• redirect_count: 1-based redirect count after accepting this redirect

DoneEvent

Trace event emitted when a high-level request call finishes, either with a final response or an exception.

Fields:

• response: final response, or nothing if the request failed before one was produced
• err: terminal exception, or nothing when the request completed successfully
• url: absolute request URL for the overall call