Cowboy will set the socket's buffer size dynamically to
better fit the current workload. When the incoming data
is small, a low buffer size reduces the memory footprint
and improves responsiveness and therefore performance.
When the incoming data is large, such as large HTTP
request bodies, a larger buffer size helps us avoid
doing too many binary appends and related allocations.
Setting a large buffer size for all use cases is
sub-optimal because allocating more than needed
necessarily results in a performance hit (not just
increased memory usage).
By default Cowboy starts with a buffer size of 8192 bytes.
It then doubles or halves the buffer size depending on
the size of the data it receives from the socket. It
stops decreasing at 8192 and increasing at 131072 by
default.
To keep track of the size of the incoming data Cowboy
maintains a moving average. It allows Cowboy to avoid
changing the buffer too often but still react quickly
when necessary. Cowboy will increase the buffer size
when the moving average is above 90% of the current
buffer size, and decrease when the moving average is
below 40% of the current buffer size.
The current buffer size and moving average are
propagated when switching protocols. The dynamic buffer
is implemented in HTTP/1, HTTP/2 and HTTP/1 Websocket.
HTTP/2 Websocket has it disabled because it doesn't
interact directly with the socket; in that case it
is HTTP/2 that has a dynamic buffer.
The dynamic buffer provides a very large performance improvement
in many scenarios, at minimal cost for others. Because it largely
depend on the underlying protocol the improvements are no all equal.
TLS and compression also impact the results.
The improvement when reading a large request body, with the
requests repeated in a fast loop are:
* HTTP: 6x to 20x faster
* HTTPS: 2x to 6x faster
* H2: 4x to 5x faster
* H2C: 20x to 40x faster
I am not sure why H2C's performance was so bad, especially compared
to H2, when using default buffer sizes. Dynamic buffers make H2C a
lot more viable with default settings.
The performance impact on "hello world" type requests is minimal,
it goes from -5% to +5% roughly.
Websocket improvements vary again depending on the protocol, but
also depending on whether compression is enabled:
* HTTP echo: roughly 2x faster
* HTTP send: roughly 4x faster
* H2C echo: roughly 2x faster
* H2C send: 3x to 4x faster
In the echo test we reply back, and Gun doesn't have the dynamic
buffer optimisation, so that probably explains the x2 difference.
With compression however there isn't much improvement. The results
are roughly within -10% to +10% of each other. Zlib compression
seems to be a bottleneck, or at least to modify the performance
profile to such an extent that the size of the buffer does not
matter. This happens to randomly generated binary data as well
so it is probably not caused by the test data.
This includes Websocket over HTTP/3.
Since quicer, which provides the QUIC implementation,
is a NIF, Cowboy cannot depend directly on it. In order
to enable QUIC and HTTP/3, users have to set the
COWBOY_QUICER environment variable:
export COWBOY_QUICER=1
In order to run the test suites, the same must be done
for Gun:
export GUN_QUICER=1
HTTP/3 support is currently not available on Windows
due to compilation issues of quicer which have yet to
be looked at or resolved.
HTTP/3 support is also unavailable on the upcoming
OTP-27 due to compilation errors in quicer dependencies.
Once resolved HTTP/3 should work on OTP-27.
Because of how QUIC currently works, it's possible
that streams that get reset after sending a response
do not receive that response. The test suite was
modified to accomodate for that. A future extension
to QUIC will allow us to gracefully reset streams.
This also updates Erlang.mk.
It is now tested both via cowboy_req:read_body and
via cowboy_req:cast.
Removes a bad example from the guide of body reading
with period of infinity, which does not work.
While we are identified as a supervisor in the tree,
we no longer manage children processes at that point,
so do not need to trap exit signals. Users can still
enable trap_exit if they prefer to.
In the cowboy_compress_h stream handler.
Otherwise this could cause issues with caching, with the
etag being the same for compressed/uncompressed content.
Users that wish to send etags AND compress will have to
do it manually for the time being.
The read buffer was changed into an iovec to avoid doing
too many binary concatenations and allocations.
Decompression happens transparently: when decoding gzip,
the content-encoding header is removed (we only decode
when "gzip" is the only encoding so nothing remains).
We always add a content_decoded key to the Req object.
This key contains a list of codings that were decoded,
in the reverse order in which they were. Currently it
can only be empty or contain <<"gzip">> but future
improvements or user handlers may see it contain more
values.
The option to disable decompression was renamed to
decompress_enabled and defaults to true.
It is no longer possible to enable/disable decompression
in the middle of reading the body: this ensures that the
data we pass forward is always valid.
Various smaller improvements were made to the code,
tests and manual pages.
A new option reset_idle_timeout_on_send has been added.
When set to 'true', the idle timeout is reset not only
when data is received, but also when data is sent.
This allows sending large responses without having to
worry about timeouts triggering.
The default is currently unchanged but might change in
a future release.
LH: Greatly reworked the implementation so that the
timeout gets reset on almost all socket writes.
This essentially completely supersets the original
work. Tests are mostly the same although I
refactored a bit to avoid test code duplication.
This commit also changes HTTP/2 behavior a little when
data is received: Cowboy will not attempt to update the
window before running stream handler commands to avoid
sending WINDOW_UPDATE frames twice. Now it has some
small heuristic to ensure they can only be sent once
at most.
Doing so will let us notice when the connection is gone instead
of waiting for timeouts, at least in the cases where the remote
socket was closed properly. Timeouts are still needed in case
of TCP half-open problems.
This change means that the order of stream handler commands is
more important than before because socket errors may occur
during the processing of commands.
Note: This commit makes cowboy depend on cowlib master.
Graceful shutdown for HTTP/2:
1. A GOAWAY frame with the last stream id set to 2^31-1 is sent and a
timer is started (goaway_initial_timeout, default 1000ms), to wait
for any in-flight requests sent by the client, and the status is set
to 'closing_initiated'. If the client responds with GOAWAY and closes
the connection, we're done.
2. A second GOAWAY frame is sent with the actual last stream id and the
status is set to 'closing'. If no streams exist, the connection
terminates. Otherwise a second timer (goaway_complete_timeout,
default 3000ms) is started, to wait for the streams to complete. New
streams are not accepted when status is 'closing'.
3. If all streams haven't completed after the second timeout, the
connection is forcefully terminated.
Graceful shutdown for HTTP/1.x:
1. If a request is currently being handled, it is waited for and the
response is sent back to the client with the header "Connection:
close". Then, the connection is closed.
2. If the current request handler is not finished within the time
configured in transport option 'shutdown' (default 5000ms), the
connection process is killed by its supervisor (ranch).
Implemented for HTTP/1.x and HTTP/2 in the following scenarios:
* When receiving exit signal 'shutdown' from the supervisor (e.g. when
cowboy:stop_listener/3 is called).
* When a connection process is requested to terminate using
sys:terminate/2,3.
LH: Edited tests a bit and added todos for useful tests to add.
When calling cowboy_req:reply/4 with a body a crash will occur
resulting in a 500 response. When calling cowboy_req:stream_reply/2,3
and then attempting to send a body a crash will occur.
This resulted in a badarith error due to the current flow being
set to infinity when the body has been fully read. A test case
has been added reproducing the issue.
This allows giving custom metadata to the metrics stream handler.
This can be useful to for example provide the name of the
module handling the request which is only known after routing.
But any user data is allowed.
When called multiple times the user data maps are merged.
This should limit the amount of memory that Cowboy is using
when a handler is sending data much faster than the network.
The new max_stream_buffer_size is a soft limit and only has
an effect when the cowboy_stream_h handler is used.
A bug was fixed in cowboy_rest where when content_types_provided
returned a media type with a wildcard as first in the list, and
a request comes in without an accept header, then the media_type
value in the Req object would contain '*' instead of [] for the
parameters.
It allows disabling the chunked transfer-encoding. It
can also be disabled on a per-request basis, although
it will be ignored for responses that are not streamed.
