Over cleartext TCP the `protocols` option lists the enabled
protocols. The default is to allow both HTTP/1.1 and HTTP/2.
Over TLS the default protocol to use when ALPN is not used
can now be configured via the `alpn_default_protocol` option.
Performing an HTTP/1.1 upgrade to HTTP/2 over TLS is now
rejected with an error as connecting to HTTP/2 over TLS
requires the use of ALPN (or that HTTP/2 be the default
when connecting over TLS).
When enabled the connection process will automatically hibernate.
Because hibernation triggers GC, this can be used as a way to
keep memory usage lower, at the cost of performance.
The problem was that when a request immediately following another
request with a large enough body, the data for the new request
(including headers) would be buffered waiting for more data,
instead of being processed immediately. If no more data came
in on the socket the request_timeout would eventually trigger.
Now the buffer is processed immediately.
In rare cases it was possible for active mode to be disabled
when there were no streams pipelined. This resulted in the
dropping of the connection due to timeouts as no data could
be received.
We now enable active mode when necessary even if there are
no streams pipelined.
This was found while benchmarking and I have not been able
to extract a test case.
Cowboy will set the socket's buffer size dynamically to
better fit the current workload. When the incoming data
is small, a low buffer size reduces the memory footprint
and improves responsiveness and therefore performance.
When the incoming data is large, such as large HTTP
request bodies, a larger buffer size helps us avoid
doing too many binary appends and related allocations.
Setting a large buffer size for all use cases is
sub-optimal because allocating more than needed
necessarily results in a performance hit (not just
increased memory usage).
By default Cowboy starts with a buffer size of 8192 bytes.
It then doubles or halves the buffer size depending on
the size of the data it receives from the socket. It
stops decreasing at 8192 and increasing at 131072 by
default.
To keep track of the size of the incoming data Cowboy
maintains a moving average. It allows Cowboy to avoid
changing the buffer too often but still react quickly
when necessary. Cowboy will increase the buffer size
when the moving average is above 90% of the current
buffer size, and decrease when the moving average is
below 40% of the current buffer size.
The current buffer size and moving average are
propagated when switching protocols. The dynamic buffer
is implemented in HTTP/1, HTTP/2 and HTTP/1 Websocket.
HTTP/2 Websocket has it disabled because it doesn't
interact directly with the socket; in that case it
is HTTP/2 that has a dynamic buffer.
The dynamic buffer provides a very large performance improvement
in many scenarios, at minimal cost for others. Because it largely
depend on the underlying protocol the improvements are no all equal.
TLS and compression also impact the results.
The improvement when reading a large request body, with the
requests repeated in a fast loop are:
* HTTP: 6x to 20x faster
* HTTPS: 2x to 6x faster
* H2: 4x to 5x faster
* H2C: 20x to 40x faster
I am not sure why H2C's performance was so bad, especially compared
to H2, when using default buffer sizes. Dynamic buffers make H2C a
lot more viable with default settings.
The performance impact on "hello world" type requests is minimal,
it goes from -5% to +5% roughly.
Websocket improvements vary again depending on the protocol, but
also depending on whether compression is enabled:
* HTTP echo: roughly 2x faster
* HTTP send: roughly 4x faster
* H2C echo: roughly 2x faster
* H2C send: 3x to 4x faster
In the echo test we reply back, and Gun doesn't have the dynamic
buffer optimisation, so that probably explains the x2 difference.
With compression however there isn't much improvement. The results
are roughly within -10% to +10% of each other. Zlib compression
seems to be a bottleneck, or at least to modify the performance
profile to such an extent that the size of the buffer does not
matter. This happens to randomly generated binary data as well
so it is probably not caused by the test data.
This includes Websocket over HTTP/3.
Since quicer, which provides the QUIC implementation,
is a NIF, Cowboy cannot depend directly on it. In order
to enable QUIC and HTTP/3, users have to set the
COWBOY_QUICER environment variable:
export COWBOY_QUICER=1
In order to run the test suites, the same must be done
for Gun:
export GUN_QUICER=1
HTTP/3 support is currently not available on Windows
due to compilation issues of quicer which have yet to
be looked at or resolved.
HTTP/3 support is also unavailable on the upcoming
OTP-27 due to compilation errors in quicer dependencies.
Once resolved HTTP/3 should work on OTP-27.
Because of how QUIC currently works, it's possible
that streams that get reset after sending a response
do not receive that response. The test suite was
modified to accomodate for that. A future extension
to QUIC will allow us to gracefully reset streams.
This also updates Erlang.mk.
While we are identified as a supervisor in the tree,
we no longer manage children processes at that point,
so do not need to trap exit signals. Users can still
enable trap_exit if they prefer to.
The previous behavior was to accept them and drop the
content-length header as per the RFC recommendation.
But since this behavior is not normal it is safer to
just reject such requests than risk security issues.
A new option reset_idle_timeout_on_send has been added.
When set to 'true', the idle timeout is reset not only
when data is received, but also when data is sent.
This allows sending large responses without having to
worry about timeouts triggering.
The default is currently unchanged but might change in
a future release.
LH: Greatly reworked the implementation so that the
timeout gets reset on almost all socket writes.
This essentially completely supersets the original
work. Tests are mostly the same although I
refactored a bit to avoid test code duplication.
This commit also changes HTTP/2 behavior a little when
data is received: Cowboy will not attempt to update the
window before running stream handler commands to avoid
sending WINDOW_UPDATE frames twice. Now it has some
small heuristic to ensure they can only be sent once
at most.
Sending extra response prevented by terminating all streams except
the one currently executing.
LH: Reworded some variables to make what happens more obvious.
Doing so will let us notice when the connection is gone instead
of waiting for timeouts, at least in the cases where the remote
socket was closed properly. Timeouts are still needed in case
of TCP half-open problems.
This change means that the order of stream handler commands is
more important than before because socket errors may occur
during the processing of commands.
Note: This commit makes cowboy depend on cowlib master.
Graceful shutdown for HTTP/2:
1. A GOAWAY frame with the last stream id set to 2^31-1 is sent and a
timer is started (goaway_initial_timeout, default 1000ms), to wait
for any in-flight requests sent by the client, and the status is set
to 'closing_initiated'. If the client responds with GOAWAY and closes
the connection, we're done.
2. A second GOAWAY frame is sent with the actual last stream id and the
status is set to 'closing'. If no streams exist, the connection
terminates. Otherwise a second timer (goaway_complete_timeout,
default 3000ms) is started, to wait for the streams to complete. New
streams are not accepted when status is 'closing'.
3. If all streams haven't completed after the second timeout, the
connection is forcefully terminated.
Graceful shutdown for HTTP/1.x:
1. If a request is currently being handled, it is waited for and the
response is sent back to the client with the header "Connection:
close". Then, the connection is closed.
2. If the current request handler is not finished within the time
configured in transport option 'shutdown' (default 5000ms), the
connection process is killed by its supervisor (ranch).
Implemented for HTTP/1.x and HTTP/2 in the following scenarios:
* When receiving exit signal 'shutdown' from the supervisor (e.g. when
cowboy:stop_listener/3 is called).
* When a connection process is requested to terminate using
sys:terminate/2,3.
LH: Edited tests a bit and added todos for useful tests to add.
When calling cowboy_req:reply/4 with a body a crash will occur
resulting in a 500 response. When calling cowboy_req:stream_reply/2,3
and then attempting to send a body a crash will occur.
This resulted in a badarith error due to the current flow being
set to infinity when the body has been fully read. A test case
has been added reproducing the issue.
The flow control is now only set to infinity when we are
skipping the request body of the stream that is being
terminated. This fixes a bug where it was set to infinity
while reading a subsequent request's body, leading to a
crash.
The timeout is no longer reset on stream termination.
Timeout handling is already done when receiving data
from the socket and doing a reset on stream termination
was leading to the wrong timeout being set or the right
timeout being reset needlessly.
This reduces the number of times we need to ask for more packets,
and as a result we get a fairly large boost in performance,
especially with HTTP/1.1.
Unfortunately this makes Cowboy require at least Erlang/OTP 21.3+
because the ssl application did not have active,N. For simplicity
the version required will be Erlang/OTP 22+.
In addition this change improves hibernate handling in
cowboy_websocket. Hibernate will now work for HTTP/2 transport
as well, and stray or unrelated messages will no longer cancel
hibernate (the process will handle the message and go back into
hibernation).
Thanks go to Stressgrid for benchmarking an early version of this
commit: https://stressgrid.com/blog/cowboy_performance_part_2/
We now stop reading from the socket unless asked to,
when we reach the request body. The option
initial_stream_flow_size controls how much data
we read without being asked, as an optimization.
We may also have received additional data along
with the request headers.
This commit also reworks the timeout handling for HTTP/1.1
because the stray timeout message was easily reproducible
after implementing the flow control. The issue should be
gone for good this time.
While the protocol does not allow sending data before
receiving a successful Websocket upgrade response, we
do not want to discard that data if it does come in.
Now both HTTP/1.1 and HTTP/2 follow the documented format.
HTTP/1.1 was including an extra element containing the
StreamID before, which was unnecessary because it is also
given as argument to the callback.
HTTP/2 early_error will now include headers in its PartialReq.
Fix a case where Cowboy was waiting for more data that simply
did not come. Now Cowboy will generate an error immediately
when a header line has no colon separator.
These test cases come from known request smuggling attack
vectors. Cowboy was not vulnerable to any of them.
It allows disabling the chunked transfer-encoding. It
can also be disabled on a per-request basis, although
it will be ignored for responses that are not streamed.
It is now possible to stream one or more sendfile tuples.
A simple example of what can now be done would be for
example to build a tar file on the fly using the sendfile
syscall for sending the files, or to support Range requests
with more than one range with the sendfile syscall.
When using cowboy_compress_h unfortunately we have to read
the file in order to send it. More options will be added
at a later time to make sure users don't read too much
into memory. This is a new feature however so existing
code is not affected.
Also rework cowboy_http's data sending to be flatter.
Currently the compression threshold is set to 300 and hardcoded in the
codebase. There are cases where it make sense to allow this to be
configured, for instance when you want to enforce all responses to be
compressed regarldess of their size.
