2016-01-14 13:35:25 +01:00
|
|
|
[[multipart]]
|
|
|
|
== Multipart requests
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2014-09-30 20:12:13 +03:00
|
|
|
Multipart originates from MIME, an Internet standard that
|
2016-09-02 12:59:45 +02:00
|
|
|
extends the format of emails.
|
2014-09-30 20:12:13 +03:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
A multipart message is a list of parts. A part contains
|
|
|
|
headers and a body. The body of the parts may be
|
|
|
|
of any media type, and contain text or binary data.
|
|
|
|
It is possible for parts to contain a multipart media
|
|
|
|
type.
|
2014-09-30 20:12:13 +03:00
|
|
|
|
|
|
|
In the context of HTTP, multipart is most often used
|
2016-09-02 12:59:45 +02:00
|
|
|
with the `multipart/form-data` media type. It is what
|
|
|
|
browsers use to upload files through HTML forms.
|
2014-09-30 20:12:13 +03:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
The `multipart/byteranges` is also common. It is the
|
|
|
|
media type used to send arbitrary bytes from a resource,
|
|
|
|
enabling clients to resume downloads.
|
2014-09-30 20:12:13 +03:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
=== Form-data
|
2014-09-30 20:12:13 +03:00
|
|
|
|
|
|
|
In the normal case, when a form is submitted, the
|
|
|
|
browser will use the `application/x-www-form-urlencoded`
|
|
|
|
content-type. This type is just a list of keys and
|
|
|
|
values and is therefore not fit for uploading files.
|
|
|
|
|
|
|
|
That's where the `multipart/form-data` content-type
|
|
|
|
comes in. When the form is configured to use this
|
2016-09-02 12:59:45 +02:00
|
|
|
content-type, the browser will create a multipart
|
|
|
|
message where each part corresponds to a field on
|
|
|
|
the form. For files, it also adds some metadata in
|
|
|
|
the part headers, like the file name.
|
2014-09-30 20:12:13 +03:00
|
|
|
|
|
|
|
A form with a text input, a file input and a select
|
|
|
|
choice box will result in a multipart message with
|
|
|
|
three parts, one for each field.
|
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
The browser does its best to determine the media type
|
2014-09-30 20:12:13 +03:00
|
|
|
of the files it sends this way, but you should not
|
|
|
|
rely on it for determining the contents of the file.
|
|
|
|
Proper investigation of the contents is recommended.
|
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
=== Checking for multipart messages
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
The content-type header indicates the presence of
|
|
|
|
a multipart message:
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
[source,erlang]
|
|
|
|
----
|
2014-09-23 16:43:29 +03:00
|
|
|
{<<"multipart">>, <<"form-data">>, _}
|
2014-02-06 19:36:25 +01:00
|
|
|
= cowboy_req:parse_header(<<"content-type">>, Req).
|
2016-01-14 13:35:25 +01:00
|
|
|
----
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
=== Reading a multipart message
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
Cowboy provides two sets of functions for reading
|
|
|
|
request bodies as multipart messages.
|
|
|
|
|
|
|
|
The `cowboy_req:read_part/1,2` functions return the
|
|
|
|
next part's headers, if any.
|
|
|
|
|
|
|
|
The `cowboy_req:read_part_body/1,2` functions return
|
|
|
|
the current part's body. For large bodies you may
|
|
|
|
need to call the function multiple times.
|
|
|
|
|
|
|
|
To read a multipart message you need to iterate over
|
|
|
|
all its parts:
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
[source,erlang]
|
|
|
|
----
|
2016-09-02 12:59:45 +02:00
|
|
|
multipart(Req0) ->
|
|
|
|
case cowboy_req:read_part(Req0) of
|
|
|
|
{ok, _Headers, Req1} ->
|
|
|
|
{ok, _Body, Req} = cowboy_req:read_part_body(Req1),
|
|
|
|
multipart(Req);
|
|
|
|
{done, Req} ->
|
|
|
|
Req
|
2014-02-06 19:36:25 +01:00
|
|
|
end.
|
2016-01-14 13:35:25 +01:00
|
|
|
----
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
When part bodies are too large, Cowboy will return
|
|
|
|
a `more` tuple, and allow you to loop until the part
|
|
|
|
body has been fully read.
|
2014-02-06 19:36:25 +01:00
|
|
|
|
|
|
|
The function `cow_multipart:form_data/1` can be used
|
|
|
|
to quickly obtain information about a part from a
|
2016-09-02 12:59:45 +02:00
|
|
|
`multipart/form-data` message. The function returns
|
|
|
|
a `data` or a `file` tuple depending on whether this
|
|
|
|
is a normal field or a file being uploaded.
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
The following snippet will use this function and
|
|
|
|
use different strategies depending on whether the
|
|
|
|
part is a file:
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
[source,erlang]
|
|
|
|
----
|
2016-09-02 12:59:45 +02:00
|
|
|
multipart(Req0) ->
|
|
|
|
case cowboy_req:read_part(Req0) of
|
|
|
|
{ok, Headers, Req1} ->
|
|
|
|
Req = case cow_multipart:form_data(Headers) of
|
2014-02-06 19:36:25 +01:00
|
|
|
{data, _FieldName} ->
|
2016-09-02 12:59:45 +02:00
|
|
|
{ok, _Body, Req2} = cowboy_req:read_part_body(Req1),
|
|
|
|
Req2;
|
2017-06-09 16:57:11 +02:00
|
|
|
{file, _FieldName, _Filename, _CType} ->
|
2016-09-02 12:59:45 +02:00
|
|
|
stream_file(Req1)
|
2014-02-06 19:36:25 +01:00
|
|
|
end,
|
2016-09-02 12:59:45 +02:00
|
|
|
multipart(Req);
|
|
|
|
{done, Req} ->
|
|
|
|
Req
|
2014-02-06 19:36:25 +01:00
|
|
|
end.
|
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
stream_file(Req0) ->
|
|
|
|
case cowboy_req:read_part_body(Req0) of
|
2017-09-27 18:54:51 +02:00
|
|
|
{ok, _LastBodyChunk, Req} ->
|
2016-09-02 12:59:45 +02:00
|
|
|
Req;
|
2017-09-27 18:54:51 +02:00
|
|
|
{more, _BodyChunk, Req} ->
|
2016-09-02 12:59:45 +02:00
|
|
|
stream_file(Req)
|
2014-02-06 19:36:25 +01:00
|
|
|
end.
|
2016-01-14 13:35:25 +01:00
|
|
|
----
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
Both the part header and body reading functions can take
|
|
|
|
options that will be given to the request body reading
|
|
|
|
functions. By default, `cowboy_req:read_part/1` reads
|
|
|
|
up to 64KB for up to 5 seconds. `cowboy_req:read_part_body/1`
|
|
|
|
has the same defaults as `cowboy_req:read_body/1`.
|
|
|
|
|
|
|
|
To change the defaults for part headers:
|
|
|
|
|
|
|
|
[source,erlang]
|
|
|
|
cowboy_req:read_part(Req, #{length => 128000}).
|
|
|
|
|
|
|
|
And for part bodies:
|
|
|
|
|
|
|
|
[source,erlang]
|
|
|
|
cowboy_req:read_part_body(Req, #{length => 1000000, period => 7000}).
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
=== Skipping unwanted parts
|
2014-02-06 19:36:25 +01:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
Part bodies do not have to be read. Cowboy will automatically
|
|
|
|
skip it when you request the next part's body.
|
2014-02-06 19:36:25 +01:00
|
|
|
|
|
|
|
The following snippet reads all part headers and skips
|
|
|
|
all bodies:
|
|
|
|
|
2016-01-14 13:35:25 +01:00
|
|
|
[source,erlang]
|
|
|
|
----
|
2016-09-02 12:59:45 +02:00
|
|
|
multipart(Req0) ->
|
2017-01-02 14:20:15 +01:00
|
|
|
case cowboy_req:read_part(Req0) of
|
2016-09-02 12:59:45 +02:00
|
|
|
{ok, _Headers, Req} ->
|
|
|
|
multipart(Req);
|
|
|
|
{done, Req} ->
|
|
|
|
Req
|
2014-02-06 19:36:25 +01:00
|
|
|
end.
|
2016-01-14 13:35:25 +01:00
|
|
|
----
|
2014-02-06 19:36:25 +01:00
|
|
|
|
|
|
|
Similarly, if you start reading the body and it ends up
|
2016-09-02 12:59:45 +02:00
|
|
|
being too big, you can simply continue with the next part.
|
2014-02-06 19:36:25 +01:00
|
|
|
Cowboy will automatically skip what remains.
|
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
While Cowboy can skip part bodies automatically, the read
|
|
|
|
rate is not configurable. Depending on your application
|
|
|
|
you may want to skip manually, in particular if you observe
|
|
|
|
poor performance while skipping.
|
|
|
|
|
|
|
|
You do not have to read all parts either. You can stop
|
|
|
|
reading as soon as you find the data you need.
|
Add request body reading options
The options were added to allow developers to fix timeout
issues when reading large bodies. It is also a cleaner and
easier to extend interface.
This commit deprecates the functions init_stream, stream_body
and skip_body which are no longer needed. They will be removed
in 1.0.
The body function can now take an additional argument that is a
list of options. The body_qs, part and part_body functions can
too and simply pass this argument down to the body call.
There are options for disabling the automatic continue reply,
setting a maximum length to be returned (soft limit), setting
the read length and read timeout, and setting the transfer and
content decode functions.
The return value of the body and body_qs have changed slightly.
The body function now works similarly to the part_body function,
in that it returns either an ok or a more tuple depending on
whether there is additional data to be read. The body_qs function
can return a badlength tuple if the body is too big. The default
size has been increased from 16KB to 64KB.
The default read length and timeout have been tweaked and vary
depending on the function called.
The body function will now adequately process chunked bodies,
which means that the body_qs function will too. But this means
that the behavior has changed slightly and your code should be
tested properly when updating your code.
The body and body_qs still accept a length as first argument
for compatibility purpose with older code. Note that this form
is deprecated and will be removed in 1.0. The part and part_body
function, being new and never having been in a release yet, have
this form completely removed in this commit.
Again, while most code should work as-is, you should make sure
that it actually does before pushing this to production.
2014-06-02 23:09:43 +02:00
|
|
|
|
2016-09-02 12:59:45 +02:00
|
|
|
// @todo Cover the building of multipart messages.
|