boostorg/beast
1
0
Fork 1
mirror of https://github.com/boostorg/beast.git synced 2025-07-30 04:47:29 +02:00
Code Issues Packages Projects Releases Wiki Activity

Documentation work

Browse Source
This commit is contained in:
Vinnie Falco
2017-05-26 20:46:43 -07:00
parent c6cddc83c4
commit c29974eec9
7 changed files with 1313 additions and 604 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -5,6 +5,7 @@ Version 45
* Better test::enable_yield_to
* Add test::pipe
* Fix header::reason
* Documentation work
--------------------------------------------------------------------------------

602
doc/design.qbk
View File

@ -9,9 +9,10 @@
[block '''
<informaltable frame="all"><tgroup cols="1"><colspec colname="a"/><tbody><row><entry valign="top"><simplelist>
<member><link linkend="beast.design.http">HTTP FAQ</link></member>
<member><link linkend="beast.design.websocket">WebSocket FAQ</link></member>
<member><link linkend="beast.design.websocketpp">Comparison to Zaphoyd Studios WebSocket++</link></member>
<member><link linkend="beast.design.http_message">HTTP Message Container</link></member>
<member><link linkend="beast.design.http_comparison">HTTP Comparison to Other Libraries</link></member>
<member><link linkend="beast.design.websocket_zaphoyd">Comparison to Zaphoyd Studios WebSocket++</link></member>
<member><link linkend="beast.design.review">Boost Formal Review FAQ</link></member>
</simplelist></entry></row></tbody></tgroup></informaltable>
''']
@ -46,27 +47,6 @@ of other packages offering similar functionality. In this section and the
FAQs that follow we attempt to answer those questions that are also applicable
to Beast.
[variablelist
[[
"I would also like to see instances of this library being used
in production. That would give some evidence that the design
works in practice."
][
Beast.HTTP and Beast.WebSocket are production ready and currently
running on public servers receiving traffic and handling millions of
dollars worth of financial transactions daily. The servers run [*rippled],
open source software ([@https://github.com/ripple/rippled repository])
implementing the
[@https://ripple.com/files/ripple_consensus_whitepaper.pdf [*Ripple Consensus Protocol]],
technology provided by [@http://ripple.com Ripple].
]]
]
[section:http HTTP FAQ]
For HTTP we model the message to maximize flexibility of implementation
strategies while allowing familiar verbs such as [*`read`] and [*`write`].
The HTTP interface is further driven by the needs of the WebSocket module,
@ -79,575 +59,9 @@ start. Other design goals:
* Allow for customizations, if the user needs it.
[variablelist
[[
"Some more advanced examples, e.g. including TLS with client/server
certificates would help."
][
The HTTP interface doesn't try to reinvent the wheel, it just uses
the `boost::asio::ip::tcp::socket` or `boost::asio::ssl::stream` that
you set up beforehand. Callers use the interfaces already existing
on those objects to make outgoing connections, accept incoming connections,
or establish TLS sessions with certificates. We find the available Asio
examples for performing these tasks sufficient.
]]
[[
"A built-in HTTP router?"
][
We presume this means a facility to match expressions against the URI
in HTTP requests, and dispatch them to calling code. The authors feel
that this is a responsibility of higher level code. Beast does
not try to offer a web server.
]]
[[
"HTTP Cookies? Forms/File Uploads?"
][
Cookies, or managing these types of HTTP headers in general, is the
responsibility of higher levels. Beast.HTTP just tries to get complete
messages to and from the calling code. It deals in the HTTP headers just
enough to process the message body and leaves the rest to callers. However,
for forms and file uploads the symmetric interface of the message class
allows HTTP requests to include arbitrary body types including those needed
to upload a file or fill out a form.
]]
[[
"...supporting TLS (is this a feature? If not this would be a show-stopper),
etc."
][
Beast does not provide direct facilities for implementing TLS connections;
however, the interfaces already existing on the `boost::asio::ssl::stream`
are available and can be used to establish secure connections. Then,
functions like `http::read` or `http::async_write` can work with those
encrypted connections with no problem.
]]
[[
"There should also be more examples of how to integrate the http service
with getting files from the file system, generating responses CGI-style"
][
The design goal for the library is to not try to invent a web server.
We feel that there is a strong need for a basic implementation that
models the HTTP message and provides functions to send and receive them
over Asio. Such an implementation should serve as a building block upon
which higher abstractions such as the aforementioned HTTP service or
cgi-gateway can be built.
One of the example programs implements a simple HTTP server that
delivers files from the filesystem.
]]
[[
"You should send a 100-continue to ask for the rest of the body if required."
][
The Beast interface supporst this functionality (by allowing this
special case of partial message parsing and serialization). Specifically,
it lets callers read the request up to just before the body,
and let callers write the request up to just before the body. However,
making use of this behavior is up to callers (since Beast is low level).
]]
[[
"What about HTTP/2?"
][
Many reviewers feel that HTTP/2 support is an essential feature of
a HTTP library. The authors agree that HTTP/2 is important but also
feel that the most sensible implementation is one that does not re-use
the same network reading and writing interface for 2 as that for 1.0
and 1.1.
The Beast HTTP message model is suitable for HTTP/2 and can be re-used.
The IETF HTTP Working Group adopted message compatiblity with HTTP/1.x
as an explicit goal. A parser can simply emit full headers after
decoding the compressed HTTP/2 headers. The stream ID is not logically
part of the message but rather message metadata and should be
communicated out-of-band (see below). HTTP/2 sessions begin with a
traditional HTTP/1.1 Upgrade similar in fashion to the WebSocket
upgrade. An HTTP/2 implementation can use existing Beast.HTTP primitives
to perform this handshake.
Free functions for HTTP/2 sessions are not possible because of the
requirement to maintain per-session state. For example, to decode the
compressed headers. Or to remember and respect the remote peer's window
settings. The authors propose that a HTTP/2 implementation be written
as a separate class template, similar to the `websocket::stream` but with
additional interfaces to support version 2 features. We feel that
Beast.HTTP offers enough useful functionality to justify inclusion,
so that developers can take advantage of it right away instead of
waiting.
]]
]
[endsect]
[section:websocket WebSocket FAQ]
[variablelist
[[
What about message compression?
][
Beast WebSocket supports the permessage-deflate extension described in
[@https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-00 draft-ietf-hybi-permessage-compression-00].
The library comes with a header-only, C++11 port of ZLib's "deflate" codec
used in the implementation of the permessage-deflate extension.
]]
[[
Where is the TLS/SSL interface?
][
The `websocket::stream` wraps the socket or stream that you provide
(for example, a `boost::asio::ip::tcp::socket` or a
`boost::asio::ssl::stream`). You establish your TLS connection using the
interface on `ssl::stream` like shown in all of the Asio examples, then
construct your `websocket::stream` around it. It works perfectly fine;
Beast.WebSocket doesn't try to reinvent the wheel or put a fresh coat of
interface paint on the `ssl::stream`.
The WebSocket implementation [*does] provide support for shutting down
the TLS connection through the use of the ADL compile-time virtual functions
[link beast.ref.websocket__teardown `teardown`] and
[link beast.ref.websocket__async_teardown `async_teardown`]. These will
properly close the connection as per rfc6455 and overloads are available
for TLS streams. Callers may provide their own overloads of these functions
for user-defined next layer types.
]]
]
[endsect]
[section:websocketpp Comparison to Zaphoyd Studios WebSocket++]
[variablelist
[[
How does this compare to [@https://www.zaphoyd.com/websocketpp websocketpp],
an alternate header-only WebSocket implementation?
][
[variablelist
[[1. Synchronous Interface][
Beast offers full support for WebSockets using a synchronous interface. It
uses the same style of interfaces found in Boost.Asio: versions that throw
exceptions, or versions that return the error code in a reference parameter:
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L774 Beast]]
[websocketpp]
][
[```
template<class DynamicBuffer>
void
read(opcode& op, DynamicBuffer& dynabuf)
```]
[
/<not available>/
]
]]]]
[[2. Connection Model][
websocketpp supports multiple transports by utilizing a trait, the `config::transport_type`
([@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/asio/connection.hpp#L60 asio transport example])
To get an idea of the complexity involved with implementing a transport,
compare the asio transport to the
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/iostream/connection.hpp#L59 `iostream` transport]
(a layer that allows websocket communication over a `std::iostream`).
In contrast, Beast abstracts the transport by defining just one [*`NextLayer`]
template argument The type requirements for [*`NextLayer`] are
already familiar to users as they are documented in Asio:
__AsyncReadStream__, __AsyncWriteStream__, __SyncReadStream__, __SyncWriteStream__.
The type requirements for instantiating `beast::websocket::stream` versus
`websocketpp::connection` with user defined types are vastly reduced
(18 functions versus 2). Note that websocketpp connections are passed by
`shared_ptr`. Beast does not use `shared_ptr` anywhere in its public interface.
A `beast::websocket::stream` is constructible and movable in a manner identical
to a `boost::asio::ip::tcp::socket`. Callers can put such objects in a
`shared_ptr` if they want to, but there is no requirement to do so.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L234 websocketpp]]
][
[```
template<class NextLayer>
class stream
{
NextLayer next_layer_;
...
}
```]
[```
template <typename config>
class connection
: public config::transport_type::transport_con_type
, public config::connection_base
{
public:
typedef lib::shared_ptr<type> ptr;
...
}
```]
]]]]
[[3. Client and Server Role][
websocketpp provides multi-role support through a hierarchy of
different classes. A `beast::websocket::stream` is role-agnostic, it
offers member functions to perform both client and server handshakes
in the same class. The same types are used for client and server
streams.
[table
[
[Beast]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/roles/server_endpoint.hpp#L39 websocketpp],
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/roles/client_endpoint.hpp#L42 also]]
][
[
/<not needed>/
]
[```
template <typename config>
class client : public endpoint<connection<config>,config>;
template <typename config>
class server : public endpoint<connection<config>,config>;
```]
]]]]
[[4. Thread Safety][
websocketpp uses mutexes to protect shared data from concurrent
access. In contrast, Beast does not use mutexes anywhere in its
implementation. Instead, it follows the Asio pattern. Calls to
asynchronous initiation functions use the same method to invoke
intermediate handlers as the method used to invoke the final handler,
through the __asio_handler_invoke__ mechanism.
The only requirement in Beast is that calls to asynchronous initiation
functions are made from the same implicit or explicit strand. For
example, if the `io_service` associated with a `beast::websocket::stream`
is single threaded, this counts as an implicit strand and no performance
costs associated with mutexes are incurred.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/impl/read_frame_op.ipp#L118 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/iostream/connection.hpp#L706 websocketpp]]
][
[```
template <class Function>
friend
void asio_handler_invoke(Function&& f, read_frame_op* op)
{
return boost_asio_handler_invoke_helpers::invoke(f, op->d_->h);
}
```]
[```
mutex_type m_read_mutex;
```]
]]]]
[[5. Callback Model][
websocketpp requires a one-time call to set the handler for each event
in its interface (for example, upon message receipt). The handler is
represented by a `std::function` equivalent. Its important to recognize
that the websocketpp interface performs type-erasure on this handler.
In comparison, Beast handlers are specified in a manner identical to
Boost.Asio. They are function objects which can be copied or moved but
most importantly they are not type erased. The compiler can see
through the type directly to the implementation, permitting
optimization. Furthermore, Beast follows the Asio rules for treatment
of handlers. It respects any allocation, continuation, or invocation
customizations associated with the handler through the use of argument
dependent lookup overloads of functions such as `asio_handler_allocate`.
The Beast completion handler is provided at the call site. For each
call to an asynchronous initiation function, it is guaranteed that
there will be exactly one final call to the handler. This functions
exactly the same way as the asynchronous initiation functions found in
Boost.Asio, allowing the composition of higher level abstractions.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L834 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L281 websocketpp],
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L473 also]]
][
[```
template<class DynamicBuffer, class ReadHandler>
typename async_completion<ReadHandler, void(error_code)>::result_type
async_read(opcode& op, DynamicBuffer& dynabuf, ReadHandler&& handler);
```]
[```
typedef lib::function<void(connection_hdl,message_ptr)> message_handler;
void set_message_handler(message_handler h);
```]
]]]]
[[6. Extensible Asynchronous Model][
Beast fully supports the
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf Extensible Asynchronous Model]
developed by Christopher Kohlhoff, author of Boost.Asio (see Section 8).
Beast websocket asynchronous interfaces may be used seamlessly with
`std::future` stackful/stackless coroutines, or user defined customizations.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/impl/stream.ipp#L378 Beast]]
[websocketpp]
][
[```
beast::async_completion<ReadHandler, void(error_code)> completion(handler);
read_op<DynamicBuffer, decltype(completion.handler)>{
completion.handler, *this, op, buffer};
return completion.result.get();
```]
[
/<not available>/
]
]]]]
[[7. Message Buffering][
websocketpp defines a message buffer, passed in arguments by
`shared_ptr`, and an associated message manager which permits
aggregation and reuse of memory. The implementation of
`websocketpp::message` uses a `std::string` to hold the payload. If an
incoming message is broken up into multiple frames, the string may be
reallocated for each continuation frame. The `std::string` always uses
the standard allocator, it is not possible to customize the choice of
allocator.
Beast allows callers to specify the object for receiving the message
or frame data, which is of any type meeting the requirements of
__DynamicBuffer__ (modeled after `boost::asio::streambuf`).
Beast comes with the class __basic_multi_buffer__, an efficient
implementation of the __DynamicBuffer__ concept which makes use of multiple
allocated octet arrays. If an incoming message is broken up into
multiple pieces, no reallocation occurs. Instead, new allocations are
appended to the sequence when existing allocations are filled. Beast
does not impose any particular memory management model on callers. The
__basic_multi_buffer__ provided by beast supports standard allocators through
a template argument. Use the __DynamicBuffer__ that comes with beast,
customize the allocator if you desire, or provide your own type that
meets the requirements.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L774 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/message_buffer/message.hpp#L78 websocketpp]]
][
[```
template<class DynamicBuffer>
read(opcode& op, DynamicBuffer& dynabuf);
```]
[```
template <template<class> class con_msg_manager>
class message {
public:
typedef lib::shared_ptr<message> ptr;
...
std::string m_payload;
...
};
```]
]]]]
[[8. Sending Messages][
When sending a message, websocketpp requires that the payload is
packaged in a `websocketpp::message` object using `std::string` as the
storage, or it requires a copy of the caller provided buffer by
constructing a new message object. Messages are placed onto an
outgoing queue. An asynchronous write operation runs in the background
to clear the queue. No user facing handler can be registered to be
notified when messages or frames have completed sending.
Beast doesn't allocate or make copies of buffers when sending data. The
caller's buffers are sent in-place. You can use any object meeting the
requirements of
[@http://www.boost.org/doc/html/boost_asio/reference/ConstBufferSequence.html ConstBufferSequence],
permitting efficient scatter-gather I/O.
The [*ConstBufferSequence] interface allows callers to send data from
memory-mapped regions (not possible in websocketpp). Callers can also
use the same buffers to send data to multiple streams, for example
broadcasting common subscription data to many clients at once. For
each call to `async_write` the completion handler is called once when
the data finishes sending, in a manner identical to `boost::asio::async_write`.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L1048 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L672 websocketpp]]
][
[```
template<class ConstBufferSequence>
void
write(ConstBufferSequence const& buffers);
```]
[```
lib::error_code send(std::string const & payload,
frame::opcode::value op = frame::opcode::text);
...
lib::error_code send(message_ptr msg);
```]
]]]]
[[9. Streaming Messages][
websocketpp requires that the entire message fit into memory, and that
the size is known ahead of time.
Beast allows callers to compose messages in individual frames. This is
useful when the size of the data is not known ahead of time or if it
is not desired to buffer the entire message in memory at once before
sending it. For example, sending periodic output of a database query
running on a coroutine. Or sending the contents of a file in pieces,
without bringing it all into memory.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L1151 Beast]]
[websocketpp]
][
[```
template<class ConstBufferSequence>
void
write_frame(bool fin,
ConstBufferSequence const& buffers);
```]
[
/<not available>/
]
]]]]
[[10. Flow Control][
The websocketpp read implementation continuously reads asynchronously
from the network and buffers message data. To prevent unbounded growth
and leverage TCP/IP's flow control mechanism, callers can periodically
turn this 'read pump' off and back on.
In contrast a `beast::websocket::stream` does not independently begin
background activity, nor does it buffer messages. It receives data only
when there is a call to an asynchronous initiation function (for
example `beast::websocket::stream::async_read`) with an associated handler.
Applications do not need to implement explicit logic to regulate the
flow of data. Instead, they follow the traditional model of issuing a
read, receiving a read completion, processing the message, then
issuing a new read and repeating the process.
[table
[
[Beast]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L728 websocketpp]]
][
[
/<implicit>/
]
[```
lib::error_code pause_reading();
lib::error_code resume_reading();
```]
]]]]
[[11. Connection Establishment][
websocketpp offers the `endpoint` class which can handle binding and
listening to a port, and spawning connection objects.
Beast does not reinvent the wheel here, callers use the interfaces
already in `boost::asio` for receiving incoming connections resolving
host names, or establishing outgoing connections. After the socket (or
`boost::asio::ssl::stream`) is connected, the `beast::websocket::stream`
is constructed around it and the WebSocket handshake can be performed.
Beast users are free to implement their own "connection manager", but
there is no requirement to do so.
[table
[
[[@http://www.boost.org/doc/html/boost_asio/reference/async_connect.html Beast],
[@http://www.boost.org/doc/html/boost_asio/reference/basic_socket_acceptor/async_accept.html also]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/asio/endpoint.hpp#L52 websocketpp]]
][
[```
#include <boost/asio.hpp>
```]
[```
template <typename config>
class endpoint : public config::socket_type;
```]
]]]]
[[12. WebSocket Handshaking][
Callers invoke `beast::websocket::accept` to perform the WebSocket
handshake, but there is no requirement to use this function. Advanced
users can perform the WebSocket handshake themselves. Beast WebSocket
provides the tools for composing the request or response, and the
Beast HTTP interface provides the container and algorithms for sending
and receiving HTTP/1 messages including the necessary HTTP Upgrade
request for establishing the WebSocket session.
Beast allows the caller to pass the incoming HTTP Upgrade request for
the cases where the caller has already received an HTTP message.
This flexibility permits novel and robust implementations. For example,
a listening socket that can handshake in multiple protocols on the
same port.
Sometimes callers want to read some bytes on the socket before reading
the WebSocket HTTP Upgrade request. Beast allows these already-received
bytes to be supplied to an overload of the accepting function to permit
sophisticated features. For example, a listening socket that can
accept both regular WebSocket and Secure WebSocket (SSL) connections.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L501 Beast],
[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L401 also]]
[websocketpp]
][
[```
template<class ConstBufferSequence>
void
accept(ConstBufferSequence const& buffers);
template<class Body, class Headers>
void
accept(http::request_v1<Body, Headers> const& request);
```]
[
/<not available>/
]
]]]]
]
]]
]
[endsect]
[include design/http_message.qbk]
[include design/http_comparison.qbk]
[include design/websocket_zaphoyd.qbk]
[include design/review.qbk]
[endsect]

454
doc/design/http_comparison.qbk Normal file
View File

@ -0,0 +1,454 @@
[/
Copyright (c) 2013-2017 Vinnie Falco (vinnie dot falco at gmail dot com)
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
]
[section:http_comparison HTTP Comparison to Other Libraries]
There are a few C++ published libraries which implement some of the HTTP
protocol. We analyze the message model chosen by those libraries and discuss
the advantages and disadvantages relative to Beast.
The general strategy used by the author to evaluate external libraries is
as follows:
* Review the message model. Can it represent a complete request or
response? What level of allocator support is present? How much
customization is possible?
* Review the stream abstraction. This is the type of object, such as
a socket, which may be used to parse or serialize (i.e. read and write).
Can user defined types be specified? What's the level of conformance to
to Asio or Networking-TS concepts?
* Check treatment of buffers. Does the library manage the buffers or
or can users provide their own buffers?
* How does the library handle corner cases such as trailers,
Expect: 100-continue, or deferred commitment of the body type?
[note
Declarations examples from external libraries have been edited:
portions have been removed for simplification.
]
[heading cpp-netlib]
[@https://github.com/cpp-netlib/cpp-netlib/tree/092cd570fb179d029d1865aade9f25aae90d97b9 [*cpp-netlib]]
is a network programming library previously intended for Boost but not
having gone through formal review. As of this writing it still uses the
Boost name, namespace, and directory structure although the project states
that Boost acceptance is no longer a goal. The library is based on Boost.Asio
and bills itself as ['"a collection of network related routines/implementations
geared towards providing a robust cross-platform networking library"]. It
cites ['"Common Message Type"] as a feature. As of the branch previous
linked, it uses these declarations:
```
template <class Tag>
struct basic_message {
public:
typedef Tag tag;
typedef typename headers_container<Tag>::type headers_container_type;
typedef typename headers_container_type::value_type header_type;
typedef typename string<Tag>::type string_type;
headers_container_type& headers() { return headers_; }
headers_container_type const& headers() const { return headers_; }
string_type& body() { return body_; }
string_type const& body() const { return body_; }
string_type& source() { return source_; }
string_type const& source() const { return source_; }
string_type& destination() { return destination_; }
string_type const& destination() const { return destination_; }
private:
friend struct detail::directive_base<Tag>;
friend struct detail::wrapper_base<Tag, basic_message<Tag> >;
mutable headers_container_type headers_;
mutable string_type body_;
mutable string_type source_;
mutable string_type destination_;
};
```
This container is the base class template used to represent HTTP messages.
It uses a "tag" type style specializations for a variety of trait classes,
allowing for customization of the various parts of the message. For example,
a user specializes `headers_container<T>` to determine what container type
holds the header fields. We note some problems with the container declaration:
* The header and body containers may only be default-constructed.
* No stateful allocator support.
* There is no way to defer the commitment of the type for `body_` to
after the headers are read in.
* The message model includes a "source" and "destination." This is
extraneous metadata associated with the connection which is not part
of the HTTP protocol specification and belongs elsewhere.
* The use of `string_type` (a customization point) for source,
destination, and body suggests that `string_type` models a
[*ForwardRange] whose `value_type` is `char`. This representation
is less than ideal, considering that the library is built on
Boost.Asio. Adapting a __DynamicBuffer__ to the required forward
range destroys information conveyed by the __ConstBufferSequence__
and __MutableBufferSequence__ used in dynamic buffers. The consequence
is that cpp-netlib implementations will be less efficient than an
equivalent __N4588__ conforming implementation.
* The library uses specializatons of `string<Tag>` to change the type
of string used everywhere, including the body, field name and value
pairs, and extraneous metadata such as source and destination. The
user may only choose a single type: field name, field values, and
the body container will all use the same string type. This limits
utility of the customization point. The library's use of the string
trait is limited to selecting between `std::string` and `std::wstring`.
We do not find this use-case compelling given the limitations.
* The specialized trait classes generate a proliferation of small
additional framework types. To specialize traits, users need to exit
their namespace and intrude into the `boost::network::http` namespace.
The way the traits are used in the library limits the usefulness
of the traits to trivial purpose.
* The `string<Tag> customization point constrains user defined body types
to few possible strategies. There is no way to represent an HTTP message
body as a filename with accompanying algorithms to store or retrieve data
from the file system.
The design of the message container in this library is cumbersome
with its system of customization using trait specializations. The
use of these customizations is extremely limited due to the way they
are used in the container declaration, making the design overly
complex without corresponding benefit.
[heading Boost.HTTP]
[@https://github.com/BoostGSoC14/boost.http/tree/45fc1aa828a9e3810b8d87e669b7f60ec100bff4 [*boost.http]]
is a library resulting from the 2014 Google Summer of Code. It was submitted
for a Boost formal review and rejected in 2015. It is based on Boost.Asio,
and development on the library has continued to the present. As of the branch
previously linked, it uses these message declarations:
```
template<class Headers, class Body>
struct basic_message
{
typedef Headers headers_type;
typedef Body body_type;
headers_type &headers();
const headers_type &headers() const;
body_type &body();
const body_type &body() const;
headers_type &trailers();
const headers_type &trailers() const;
private:
headers_type headers_;
body_type body_;
headers_type trailers_;
};
typedef basic_message<boost::http::headers, std::vector<std::uint8_t>> message;
template<class Headers, class Body>
struct is_message<basic_message<Headers, Body>>: public std::true_type {};
```
* This container cannot model a complete message. The ['start-line] items
(method and target for requests, reason-phrase for responses) are
communicated out of band, as is the ['http-version]. A function that
operates on the message including the start line requires additional
parameters. This is evident in one of the
[@https://github.com/BoostGSoC14/boost.http/blob/45fc1aa828a9e3810b8d87e669b7f60ec100bff4/example/basic_router.cpp#L81 example programs].
The `500` and `"OK"` arguments represent the response ['status-code] and
['reason-phrase] respectively:
```
...
http::message reply;
...
self->socket.async_write_response(500, string_ref("OK"), reply, yield);
```
* `headers_`, `body_`, and `trailers_` may only be default-constructed,
since there are no explicitly declared constructors.
* There is no way to defer the commitment of the [*Body] type to after
the headers are read in. This is related to the previous limitation
on default-construction.
* No stateful allocator support. This follows from the previous limitation
on default-construction. Buffers for start-line strings must be
managed externally from the message object since they are not members.
* The trailers are stored in a separate object. Aside from the combinatorial
explosion of the number of additional constructors necessary to fully
support arbitrary forwarded parameter lists for each of the headers, body,
and trailers members, the requirement to know in advance whether a
particular HTTP field will be located in the headers or the trailers
poses an unnecessary complication for general purpose functions that
operate on messages.
* The declarations imply that `std::vector` is a model of [*Body].
More formally, that a body is represented by the [*ForwardRange]
concept whose `value_type` is an 8-bit integer. This representation
is less than ideal, considering that the library is built on
Boost.Asio. Adapting a __DynamicBuffer__ to the required forward range
destroys information conveyed by the __ConstBufferSequence__ and
__MutableBufferSequence__ used in dynamic buffers. The consequence is
that Boost.HTTP implementations will be less efficient when dealing
with body containers than an equivalent __N4588__ conforming
implementation.
* The [*Body] customization point constrains user defined types to
very limited implementation strategies. For example, there is no way
to represent an HTTP message body as a filename with accompanying
algorithms to store or retrieve data from the file system.
This representation addresses a narrow range of use cases. It has
limited potential for customization and performance. It is more difficult
to use because it excludes the start line fields from the model.
[heading C++ REST SDK (cpprestsdk)]
[@https://github.com/Microsoft/cpprestsdk/tree/381f5aa92d0dfb59e37c0c47b4d3771d8024e09a [*cpprestsdk]]
is a Microsoft project which ['"...aims to help C++ developers connect to and
interact with services"]. It offers the most functionality of the libraries
reviewed here, including support for Websocket services using its websocket++
dependency. It can use native APIs such as HTTP.SYS when building Windows
based applications, and it can use Boost.Asio. The WebSocket module uses
Boost.Asio exclusively.
As cpprestsdk is developed by a large corporation, it contains quite a bit
of functionality and necessarily has more interfaces. We will break down
the interfaces used to model messages into more manageable pieces. This
is the container used to store the HTTP header fields:
```
class http_headers
{
public:
...
private:
std::map<utility::string_t, utility::string_t, _case_insensitive_cmp> m_headers;
};
```
This declaration is quite bare-bones. We note the typical problems of
most field containers:
* The container may only be default-constructed.
* No support for allocators, stateful or otherwise.
* There are no customization points at all.
Now we analyze the structure of
the larger message container. The library uses a handle/body idiom. There
are two public message container interfaces, one for requests (`http_request`)
and one for responses (`http_response`). Each interface maintains a private
shared pointer to an implementation class. Public member function calls
are routed to the internal implementation. This is the first implementation
class, which forms the base class for both the request and response
implementations:
```
namespace details {
class http_msg_base
{
public:
http_headers &headers() { return m_headers; }
_ASYNCRTIMP void set_body(const concurrency::streams::istream &instream, const utf8string &contentType);
/// Set the stream through which the message body could be read
void set_instream(const concurrency::streams::istream &instream) { m_inStream = instream; }
/// Set the stream through which the message body could be written
void set_outstream(const concurrency::streams::ostream &outstream, bool is_default) { m_outStream = outstream; m_default_outstream = is_default; }
const pplx::task_completion_event<utility::size64_t> & _get_data_available() const { return m_data_available; }
protected:
/// Stream to read the message body.
concurrency::streams::istream m_inStream;
/// stream to write the msg body
concurrency::streams::ostream m_outStream;
http_headers m_headers;
bool m_default_outstream;
/// <summary> The TCE is used to signal the availability of the message body. </summary>
pplx::task_completion_event<utility::size64_t> m_data_available;
};
```
To understand these declarations we need to first understand that cpprestsdk
uses the asynchronous model defined by Microsoft's
[@https://msdn.microsoft.com/en-us/library/dd504870.aspx [*Concurrency Runtime]].
Identifiers from the [@https://msdn.microsoft.com/en-us/library/jj987780.aspx [*`pplx` namespace]]
define common asynchronous patterns such as tasks and events. The
`concurrency::streams::istream` parameter and `m_data_available` data member
indicates a lack of separation of concerns. The representation of HTTP messages
should not be conflated with the asynchronous model used to serialize or
parse those messages in the message declarations.
The next declaration forms the complete implementation class referenced by the
handle in the public interface (which follows after):
```
/// Internal representation of an HTTP request message.
class _http_request final : public http::details::http_msg_base, public std::enable_shared_from_this<_http_request>
{
public:
_ASYNCRTIMP _http_request(http::method mtd);
_ASYNCRTIMP _http_request(std::unique_ptr<http::details::_http_server_context> server_context);
http::method &method() { return m_method; }
const pplx::cancellation_token &cancellation_token() const { return m_cancellationToken; }
_ASYNCRTIMP pplx::task<void> reply(const http_response &response);
private:
// Actual initiates sending the response, without checking if a response has already been sent.
pplx::task<void> _reply_impl(http_response response);
http::method m_method;
std::shared_ptr<progress_handler> m_progress_handler;
};
} // namespace details
```
As before, we note that the implementation class for HTTP requests concerns
itself more with the mechanics of sending the message asynchronously than
it does with actually modeling the HTTP message as described in __rfc7230__:
* The constructor accepting `std::unique_ptr<http::details::_http_server_context`
breaks encapsulation and separation of concerns. This cannot be extended
for user defined server contexts.
* The "cancellation token" is stored inside the message. This breaks the
separation of concerns.
* The `_reply_impl` function implies that the message implementation also
shares responsibility for the means of sending back an HTTP reply. This
would be better if it was completely separate from the message container.
Finally, here is the public class which represents an HTTP request:
```
class http_request
{
public:
const http::method &method() const { return _m_impl->method(); }
void set_method(const http::method &method) const { _m_impl->method() = method; }
/// Extract the body of the request message as a string value, checking that the content type is a MIME text type.
/// A body can only be extracted once because in some cases an optimization is made where the data is 'moved' out.
pplx::task<utility::string_t> extract_string(bool ignore_content_type = false)
{
auto impl = _m_impl;
return pplx::create_task(_m_impl->_get_data_available()).then([impl, ignore_content_type](utility::size64_t) { return impl->extract_string(ignore_content_type); });
}
/// Extracts the body of the request message into a json value, checking that the content type is application/json.
/// A body can only be extracted once because in some cases an optimization is made where the data is 'moved' out.
pplx::task<json::value> extract_json(bool ignore_content_type = false) const
{
auto impl = _m_impl;
return pplx::create_task(_m_impl->_get_data_available()).then([impl, ignore_content_type](utility::size64_t) { return impl->_extract_json(ignore_content_type); });
}
/// Sets the body of the message to the contents of a byte vector. If the 'Content-Type'
void set_body(const std::vector<unsigned char> &body_data);
/// Defines a stream that will be relied on to provide the body of the HTTP message when it is
/// sent.
void set_body(const concurrency::streams::istream &stream, const utility::string_t &content_type = _XPLATSTR("application/octet-stream"));
/// Defines a stream that will be relied on to hold the body of the HTTP response message that
/// results from the request.
void set_response_stream(const concurrency::streams::ostream &stream);
{
return _m_impl->set_response_stream(stream);
}
/// Defines a callback function that will be invoked for every chunk of data uploaded or downloaded
/// as part of the request.
void set_progress_handler(const progress_handler &handler);
private:
friend class http::details::_http_request;
friend class http::client::http_client;
std::shared_ptr<http::details::_http_request> _m_impl;
};
```
It is clear from this declaration that the goal of the message model in
this library is driven by its use-case (interacting with REST servers)
and not to model HTTP messages generally. We note problems similar to
the other declarations:
* There are no compile-time customization points at all. The only
customization is in the `concurrency::streams::istream` and
`concurrency::streams::ostream` reference parameters. Presumably,
these are abstract interfaces which may be subclassed by users
to achieve custom behaviors.
* The extraction of the body is conflated with the asynchronous model.
* No way to define an allocator for the container used when extracting
the body.
* A body can only be extracted once, limiting the use of this container
when using a functional programming style.
* Setting the body requires either a vector or a `concurrency::streams::istream`.
No user defined types are possible.
* The HTTP request container conflates HTTP response behavior (see the
`set_response_stream` member). Again this is likely purpose-driven but
the lack of separation of concerns limits this library to only the
uses explicitly envisioned by the authors.
The general theme of the HTTP message model in cpprestsdk is "no user
definable customizations". There is no allocator support, and no
separation of concerns. It is designed to perform a specific set of
behaviors. In other words, it does not follow the open/closed principle.
Tasks in the Concurrency Runtime operate in a fashion similar to
`std::future`, but with some improvements such as continuations which
are not yet in the C++ standard. The costs of using a task based
asynchronous interface instead of completion handlers is well
documented: synchronization points along the call chain of composed
task operations which cannot be optimized away. See:
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3747.pdf
[*A Universal Model for Asynchronous Operations]] (Kohlhoff).
[endsect]

284
doc/design/http_message.qbk Normal file
View File

@ -0,0 +1,284 @@
[/
Copyright (c) 2013-2017 Vinnie Falco (vinnie dot falco at gmail dot com)
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
]
[section:http_message HTTP Message Container]
In this section we describe the problem of modeling HTTP messages and explain
how the library arrived at its solution, with a discussion of the benefits
and drawbacks of the design choices. The goal for creating a message model
is to create a container with value semantics, possibly movable and/or
copyable, that completely describes the message. More formally, the container
has all of the information necessary for the serialization algorithms to
represent the entire message as a series of octets, and that the parsing
algorithms store all of the captured information about an entire message
in the container.
In addition to capturing the entirety of a message, we would like the message
container to contain enough customization points to permit the following:
allocator support, user-defined containers to represent header fields, and
user-defined containers to represent the body. And finally, because requests
and responses have slightly different information in the ['start-line], we
would like the containers for requests and responses to be represented by
different types.
Here is our first attempt at declaring some message containers:
[table
[[
```
/// An HTTP request
template<class Fields, class Body>
struct request
{
int version;
std::string method;
std::string target;
Fields fields;
typename Body::value_type body;
};
```
][
```
/// An HTTP response
template<class Fields, class Body>
struct response
{
int version;
int status;
std::string reason;
Fields fields;
typename Body::value_type body;
};
```
]]
]
Here we have accomplished a few things. Request and response objects are
different types. The user can choose the container used to represent the
fields. And the user can choose the [*Body] type, which is a concept
defining not only the type of `body` member but also the algorithms used
to transfer information in and out of that member when performing
serialization and parsing.
However, a problem arises. How do we write a function which can accept
an object that is either a request or a response? As written, the only
ovious solution is to make the message a template type. Additional traits
classes would then be needed to make sure that the passed object has a
valid type which meets the requirements. We can avoid those complexities
by renaming the containers and making them partial specializations of a
single class, like this:
```
/// An HTTP message
template<bool isRequest, class Fields, class Body>
struct message;
/// An HTTP request
template<class Fields, class Body>
struct message<true, Fields, Body>
{
int version;
std::string method;
std::string target;
Fields fields;
typename Body::value_type body;
};
/// An HTTP response
template<bool isRequest, class Fields, class Body>
struct response<false, Fields, Body>
{
int version;
int status;
std::string reason;
Fields fields;
typename Body::value_type body;
};
```
Now we can declare a function which takes any message as a parameter:
```
template<bool isRequest, class Fields, class Body>
void f(message<isRequest, Fields, Body>& msg);
```
This function can manipulate the fields common to requests and responses.
If it needs to access the other fields, it can do so using tag dispatch
with an object of type `std::integral_constant<bool, isRequest>`.
Often, in non-trivial HTTP applications, we want to read the HTTP header
and examine its contents before choosing a top for [*Body]. To accomplish
this, there needs to be a way to model the header portion of a message.
And we'd like to do this in a way that allows functions which take the
header as a parameter, to also accept a type representing the whole
message (the function will see just the header part). This suggests
inheritance:
```
/// An HTTP message header
template<bool isRequest, class Fields>
struct header;
/// An HTTP request header
template<class Fields>
struct header<true, Fields>
{
int version;
std::string method;
std::string target;
Fields fields;
};
/// An HTTP response header
template<class Fields>
struct header<false, Fields>
{
int version;
int status;
std::string reason;
Fields fields;
};
/// An HTTP message
template<bool isRequest, class Fields, class Body>
struct message : header<isRequest, Fields>
{
typename Body::value_type body;
/// Construct from a `header`
message(header<isRequest, Fields>&& h);
};
```
Note that the `message` class now has a constructor allowing messages
to be constructed from a similarly typed `header`. This handles the case
where the user already has the header and wants to make a commitment to the
type for [*Body]. This also lets us declare a function accepting any header:
```
template<bool isRequest, class Fields>
void f(header<isRequest, Fields>& msg);
```
Until now we have not given significant consideration to the constructors
of the `message` class. But to achieve all our goals we will need to make
sure that there are enough constructor overloads to not only provide for
the special copy and move members if the instantiated types support it,
but also allow the fields container and body container to be constructed
with arbitrary variadic lists of parameters. This allows those members
to properly support allocators.
The solution used in the library is to treat the message like a `std::pair`
for the purposes of construction, except that instead of `first` and `last`
we have `fields` and `body`. This means that single-argument constructors
for those fields should be accessible as they are with `std::pair`, and
that a mechanism identical to the pair's use of `std::piecewise_construct`
should be provided. Those constructors are too complex to repeat here, but
interested readers can view the declarations in the corresponding header
file.
There is now significant progress with our message container but a stumbling
block remains. The presence of `std::string` members is an obstacle to our
goal of making messages allocator-aware. One obvious solution is to add an
allocator to the template parameter list of the header and message classes,
and allow the string based members to construct with instances of those
allocators. This is unsatisfying because of the combinatorial explosion of
constructor variations needed to support the scheme. It also means that
request messages could have [*four] different allocators: two for the fields
and body, and two for the method and target strings. A better solution is
needed.
To make allocators workable, we make a simple change to the interface and
then engineer a clever concession. First, the interface change:
```
/// An HTTP request header
template<class Fields>
struct header<true, Fields>
{
int version;
string_view method() const;
void method(string_view);
string_view target(); const;
void target(string_view);
Fields fields;
};
/// An HTTP response header
template<class Fields>
struct header<false, Fields>
{
int version;
int status;
string_view reason() const;
void reason(string_view);
Fields fields;
};
```
This approach replaces public data members with traditional accessors
using non-owning references to string buffers. Now we make a concession:
management of the corresponding string is delegated to the [*Fields]
container, which can already be allocator aware and constructed with the
necessary allocator parameter via the provided constructor overloads for
`message`. The delegation implementation looks like this (only the
response header specialization is shown):
```
/// An HTTP response header
template<class Fields>
struct header<false, Fields>
{
int version;
int status;
auto
reason() const -> decltype(std::declval<Fields>().reason()) const
{
return fields.reason();
}
template<class Value>
void
reason(Value&& value)
{
fields.reason(std::forward<Value>(value));
}
Fields fields;
```
An advantage of this technique is that user-provided implementations of
[*Fields] may use arbitrary representation strategies. For example, instead
of explicitly storing the method as a string, store it as an enumerated
integer with a lookup table of static strings for known message types.
Now that we've accomplished our initial goals and more, there is one small
quality of life improvement to make. Users will choose different types for
`Body` far more often than they will for `Fields`. Thus, we swap the order
of these types and provide a default:
```
/// An HTTP header
template<bool isRequest, class Body, class Fields = fields>
struct header;
/// An HTTP message
template<bool isRequest, class Body, class Fields = fields>
struct message;
```
This container is also capable of representing complete HTTP/2 messages.
Not because it was explicitly designed for, but because the IETF wanted to
preserve message compatibility with HTTP/1. Aside from version specific
fields such as Connection, the contents of HTTP/1 and HTTP/2 messages are
identical even though their serialized representation is considerably
different. The message model presented in this library is ready for HTTP/2.
In conclusion, this representation for the message container is well thought
out, provides comprehensive flexibility, and avoids the necessity of defining
additional traits classes. User declarations of functions that accept headers
or messages as parameters are easy to write in a variety of ways to accomplish
different results, without forcing cumbersome SFINAE declarations everywhere.
[endsect]

135
doc/review.qbk → doc/design/review.qbk
View File

@ -5,7 +5,7 @@
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
]
[section:review For Reviewers]
[section:review Boost Formal Review FAQ]
To set realistic expectations and prevent a litany of duplicate review
statements, these notes address the most common questions and comments
@ -58,6 +58,12 @@ about Beast and other HTTP libraries that have gone through formal review.
[[
"There's no HTTP/2 support yet!"
][
Many reviewers feel that HTTP/2 support is an essential feature of
a HTTP library. The authors agree that HTTP/2 is important but also
feel that the most sensible implementation is one that does not re-use
the same network reading and writing interface for 2 as that for 1.0
and 1.1.
The Beast HTTP message model was designed with the new protocol
in mind and should be evaluated in that context. There are plans
to add HTTP/2 in the future, but there is no rush to do so.
@ -66,6 +72,16 @@ about Beast and other HTTP libraries that have gone through formal review.
It is the author's position that there is sufficient value in
Beast's HTTP/1-only implementation that the lack of HTTP/2
should not be a barrier to acceptance.
The Beast HTTP message model is suitable for HTTP/2 and can be re-used.
The IETF HTTP Working Group adopted message compatiblity with HTTP/1.x
as an explicit goal. A parser can simply emit full headers after
decoding the compressed HTTP/2 headers. The stream ID is not logically
part of the message but rather message metadata and should be
communicated out-of-band (see below). HTTP/2 sessions begin with a
traditional HTTP/1.1 Upgrade similar in fashion to the WebSocket
upgrade. An HTTP/2 implementation can use existing Beast.HTTP primitives
to perform this handshake.
]]
[[
"This should work with standalone-Asio!"
@ -106,7 +122,7 @@ about Beast and other HTTP libraries that have gone through formal review.
is lifted from another extremely popular project which has performance
as a design goal (see https://github.com/h2o/picohttpparser).
From: http://www.boost.org/development/requirements.html
From: [@http://www.boost.org/development/requirements.html]
"Aim first for clarity and correctness; optimization should
be only a secondary concern in most Boost libraries."
@ -143,6 +159,121 @@ about Beast and other HTTP libraries that have gone through formal review.
[[
"Some more advanced examples, e.g. including TLS with client/server
certificates would help."
][
The HTTP interface doesn't try to reinvent the wheel, it just uses
the `boost::asio::ip::tcp::socket` or `boost::asio::ssl::stream` that
you set up beforehand. Callers use the interfaces already existing
on those objects to make outgoing connections, accept incoming connections,
or establish TLS sessions with certificates. We find the available Asio
examples for performing these tasks sufficient.
]]
[[
"A built-in HTTP router?"
][
We presume this means a facility to match expressions against the URI
in HTTP requests, and dispatch them to calling code. The authors feel
that this is a responsibility of higher level code. Beast does
not try to offer a web server.
]]
[[
"HTTP Cookies? Forms/File Uploads?"
][
Cookies, or managing these types of HTTP headers in general, is the
responsibility of higher levels. Beast.HTTP just tries to get complete
messages to and from the calling code. It deals in the HTTP headers just
enough to process the message body and leaves the rest to callers. However,
for forms and file uploads the symmetric interface of the message class
allows HTTP requests to include arbitrary body types including those needed
to upload a file or fill out a form.
]]
[[
"...supporting TLS (is this a feature? If not this would be a show-stopper),
etc."
][
Beast does not provide direct facilities for implementing TLS connections;
however, the interfaces already existing on the `boost::asio::ssl::stream`
are available and can be used to establish secure connections. Then,
functions like `http::read` or `http::async_write` can work with those
encrypted connections with no problem.
]]
[[
"There should also be more examples of how to integrate the http service
with getting files from the file system, generating responses CGI-style"
][
The design goal for the library is to not try to invent a web server.
We feel that there is a strong need for a basic implementation that
models the HTTP message and provides functions to send and receive them
over Asio. Such an implementation should serve as a building block upon
which higher abstractions such as the aforementioned HTTP service or
cgi-gateway can be built.
One of the example programs implements a simple HTTP server that
delivers files from the filesystem.
]]
[[
"You should send a 100-continue to ask for the rest of the body if required."
][
The Beast interface supporst this functionality (by allowing this
special case of "split" message parsing and serialization). Specifically,
it lets callers read the request up to just before the body,
and let callers write the request up to just before the body. However,
making use of this behavior is up to callers (since Beast is low level).
]]
[[
"I would also like to see instances of this library being used
in production. That would give some evidence that the design
works in practice."
][
Beast.HTTP and Beast.WebSocket are production ready and currently
running on public servers receiving traffic and handling millions of
dollars worth of financial transactions daily. The servers run [*rippled],
open source software ([@https://github.com/ripple/rippled repository])
implementing the
[@https://ripple.com/files/ripple_consensus_whitepaper.pdf [*Ripple Consensus Protocol]],
technology provided by [@http://ripple.com Ripple].
]]
[[
What about WebSocket message compression?
][
Beast WebSocket supports the permessage-deflate extension described in
[@https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-00 draft-ietf-hybi-permessage-compression-00].
The library comes with a header-only, C++11 port of ZLib's "deflate" codec
used in the implementation of the permessage-deflate extension.
]]
[[
Where is the WebSocket TLS/SSL interface?
][
The `websocket::stream` wraps the socket or stream that you provide
(for example, a `boost::asio::ip::tcp::socket` or a
`boost::asio::ssl::stream`). You establish your TLS connection using the
interface on `ssl::stream` like shown in all of the Asio examples, then
construct your `websocket::stream` around it. It works perfectly fine;
Beast.WebSocket doesn't try to reinvent the wheel or put a fresh coat of
interface paint on the `ssl::stream`.
The WebSocket implementation [*does] provide support for shutting down
the TLS connection through the use of the ADL compile-time virtual functions
[link beast.ref.websocket__teardown `teardown`] and
[link beast.ref.websocket__async_teardown `async_teardown`]. These will
properly close the connection as per rfc6455 and overloads are available
for TLS streams. Callers may provide their own overloads of these functions
for user-defined next layer types.
]]
]
[endsect]

431
doc/design/websocket_zaphoyd.qbk Normal file
View File

@ -0,0 +1,431 @@
[/
Copyright (c) 2013-2017 Vinnie Falco (vinnie dot falco at gmail dot com)
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
]
[section:websocket_zaphoyd Comparison to Zaphoyd Studios WebSocket++]
[variablelist
[[
How does this compare to [@https://www.zaphoyd.com/websocketpp websocketpp],
an alternate header-only WebSocket implementation?
][
[variablelist
[[1. Synchronous Interface][
Beast offers full support for WebSockets using a synchronous interface. It
uses the same style of interfaces found in Boost.Asio: versions that throw
exceptions, or versions that return the error code in a reference parameter:
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L774 Beast]]
[websocketpp]
][
[```
template<class DynamicBuffer>
void
read(opcode& op, DynamicBuffer& dynabuf)
```]
[
/<not available>/
]
]]]]
[[2. Connection Model][
websocketpp supports multiple transports by utilizing a trait, the `config::transport_type`
([@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/asio/connection.hpp#L60 asio transport example])
To get an idea of the complexity involved with implementing a transport,
compare the asio transport to the
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/iostream/connection.hpp#L59 `iostream` transport]
(a layer that allows websocket communication over a `std::iostream`).
In contrast, Beast abstracts the transport by defining just one [*`NextLayer`]
template argument The type requirements for [*`NextLayer`] are
already familiar to users as they are documented in Asio:
__AsyncReadStream__, __AsyncWriteStream__, __SyncReadStream__, __SyncWriteStream__.
The type requirements for instantiating `beast::websocket::stream` versus
`websocketpp::connection` with user defined types are vastly reduced
(18 functions versus 2). Note that websocketpp connections are passed by
`shared_ptr`. Beast does not use `shared_ptr` anywhere in its public interface.
A `beast::websocket::stream` is constructible and movable in a manner identical
to a `boost::asio::ip::tcp::socket`. Callers can put such objects in a
`shared_ptr` if they want to, but there is no requirement to do so.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L234 websocketpp]]
][
[```
template<class NextLayer>
class stream
{
NextLayer next_layer_;
...
}
```]
[```
template <typename config>
class connection
: public config::transport_type::transport_con_type
, public config::connection_base
{
public:
typedef lib::shared_ptr<type> ptr;
...
}
```]
]]]]
[[3. Client and Server Role][
websocketpp provides multi-role support through a hierarchy of
different classes. A `beast::websocket::stream` is role-agnostic, it
offers member functions to perform both client and server handshakes
in the same class. The same types are used for client and server
streams.
[table
[
[Beast]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/roles/server_endpoint.hpp#L39 websocketpp],
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/roles/client_endpoint.hpp#L42 also]]
][
[
/<not needed>/
]
[```
template <typename config>
class client : public endpoint<connection<config>,config>;
template <typename config>
class server : public endpoint<connection<config>,config>;
```]
]]]]
[[4. Thread Safety][
websocketpp uses mutexes to protect shared data from concurrent
access. In contrast, Beast does not use mutexes anywhere in its
implementation. Instead, it follows the Asio pattern. Calls to
asynchronous initiation functions use the same method to invoke
intermediate handlers as the method used to invoke the final handler,
through the __asio_handler_invoke__ mechanism.
The only requirement in Beast is that calls to asynchronous initiation
functions are made from the same implicit or explicit strand. For
example, if the `io_service` associated with a `beast::websocket::stream`
is single threaded, this counts as an implicit strand and no performance
costs associated with mutexes are incurred.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/impl/read_frame_op.ipp#L118 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/iostream/connection.hpp#L706 websocketpp]]
][
[```
template <class Function>
friend
void asio_handler_invoke(Function&& f, read_frame_op* op)
{
return boost_asio_handler_invoke_helpers::invoke(f, op->d_->h);
}
```]
[```
mutex_type m_read_mutex;
```]
]]]]
[[5. Callback Model][
websocketpp requires a one-time call to set the handler for each event
in its interface (for example, upon message receipt). The handler is
represented by a `std::function` equivalent. Its important to recognize
that the websocketpp interface performs type-erasure on this handler.
In comparison, Beast handlers are specified in a manner identical to
Boost.Asio. They are function objects which can be copied or moved but
most importantly they are not type erased. The compiler can see
through the type directly to the implementation, permitting
optimization. Furthermore, Beast follows the Asio rules for treatment
of handlers. It respects any allocation, continuation, or invocation
customizations associated with the handler through the use of argument
dependent lookup overloads of functions such as `asio_handler_allocate`.
The Beast completion handler is provided at the call site. For each
call to an asynchronous initiation function, it is guaranteed that
there will be exactly one final call to the handler. This functions
exactly the same way as the asynchronous initiation functions found in
Boost.Asio, allowing the composition of higher level abstractions.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L834 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L281 websocketpp],
[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L473 also]]
][
[```
template<class DynamicBuffer, class ReadHandler>
typename async_completion<ReadHandler, void(error_code)>::result_type
async_read(opcode& op, DynamicBuffer& dynabuf, ReadHandler&& handler);
```]
[```
typedef lib::function<void(connection_hdl,message_ptr)> message_handler;
void set_message_handler(message_handler h);
```]
]]]]
[[6. Extensible Asynchronous Model][
Beast fully supports the
[@http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf Extensible Asynchronous Model]
developed by Christopher Kohlhoff, author of Boost.Asio (see Section 8).
Beast websocket asynchronous interfaces may be used seamlessly with
`std::future` stackful/stackless coroutines, or user defined customizations.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/impl/stream.ipp#L378 Beast]]
[websocketpp]
][
[```
beast::async_completion<ReadHandler, void(error_code)> completion(handler);
read_op<DynamicBuffer, decltype(completion.handler)>{
completion.handler, *this, op, buffer};
return completion.result.get();
```]
[
/<not available>/
]
]]]]
[[7. Message Buffering][
websocketpp defines a message buffer, passed in arguments by
`shared_ptr`, and an associated message manager which permits
aggregation and reuse of memory. The implementation of
`websocketpp::message` uses a `std::string` to hold the payload. If an
incoming message is broken up into multiple frames, the string may be
reallocated for each continuation frame. The `std::string` always uses
the standard allocator, it is not possible to customize the choice of
allocator.
Beast allows callers to specify the object for receiving the message
or frame data, which is of any type meeting the requirements of
__DynamicBuffer__ (modeled after `boost::asio::streambuf`).
Beast comes with the class __basic_multi_buffer__, an efficient
implementation of the __DynamicBuffer__ concept which makes use of multiple
allocated octet arrays. If an incoming message is broken up into
multiple pieces, no reallocation occurs. Instead, new allocations are
appended to the sequence when existing allocations are filled. Beast
does not impose any particular memory management model on callers. The
__basic_multi_buffer__ provided by beast supports standard allocators through
a template argument. Use the __DynamicBuffer__ that comes with beast,
customize the allocator if you desire, or provide your own type that
meets the requirements.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L774 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/message_buffer/message.hpp#L78 websocketpp]]
][
[```
template<class DynamicBuffer>
read(opcode& op, DynamicBuffer& dynabuf);
```]
[```
template <template<class> class con_msg_manager>
class message {
public:
typedef lib::shared_ptr<message> ptr;
...
std::string m_payload;
...
};
```]
]]]]
[[8. Sending Messages][
When sending a message, websocketpp requires that the payload is
packaged in a `websocketpp::message` object using `std::string` as the
storage, or it requires a copy of the caller provided buffer by
constructing a new message object. Messages are placed onto an
outgoing queue. An asynchronous write operation runs in the background
to clear the queue. No user facing handler can be registered to be
notified when messages or frames have completed sending.
Beast doesn't allocate or make copies of buffers when sending data. The
caller's buffers are sent in-place. You can use any object meeting the
requirements of
[@http://www.boost.org/doc/html/boost_asio/reference/ConstBufferSequence.html ConstBufferSequence],
permitting efficient scatter-gather I/O.
The [*ConstBufferSequence] interface allows callers to send data from
memory-mapped regions (not possible in websocketpp). Callers can also
use the same buffers to send data to multiple streams, for example
broadcasting common subscription data to many clients at once. For
each call to `async_write` the completion handler is called once when
the data finishes sending, in a manner identical to `boost::asio::async_write`.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L1048 Beast]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L672 websocketpp]]
][
[```
template<class ConstBufferSequence>
void
write(ConstBufferSequence const& buffers);
```]
[```
lib::error_code send(std::string const & payload,
frame::opcode::value op = frame::opcode::text);
...
lib::error_code send(message_ptr msg);
```]
]]]]
[[9. Streaming Messages][
websocketpp requires that the entire message fit into memory, and that
the size is known ahead of time.
Beast allows callers to compose messages in individual frames. This is
useful when the size of the data is not known ahead of time or if it
is not desired to buffer the entire message in memory at once before
sending it. For example, sending periodic output of a database query
running on a coroutine. Or sending the contents of a file in pieces,
without bringing it all into memory.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L1151 Beast]]
[websocketpp]
][
[```
template<class ConstBufferSequence>
void
write_frame(bool fin,
ConstBufferSequence const& buffers);
```]
[
/<not available>/
]
]]]]
[[10. Flow Control][
The websocketpp read implementation continuously reads asynchronously
from the network and buffers message data. To prevent unbounded growth
and leverage TCP/IP's flow control mechanism, callers can periodically
turn this 'read pump' off and back on.
In contrast a `beast::websocket::stream` does not independently begin
background activity, nor does it buffer messages. It receives data only
when there is a call to an asynchronous initiation function (for
example `beast::websocket::stream::async_read`) with an associated handler.
Applications do not need to implement explicit logic to regulate the
flow of data. Instead, they follow the traditional model of issuing a
read, receiving a read completion, processing the message, then
issuing a new read and repeating the process.
[table
[
[Beast]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/connection.hpp#L728 websocketpp]]
][
[
/<implicit>/
]
[```
lib::error_code pause_reading();
lib::error_code resume_reading();
```]
]]]]
[[11. Connection Establishment][
websocketpp offers the `endpoint` class which can handle binding and
listening to a port, and spawning connection objects.
Beast does not reinvent the wheel here, callers use the interfaces
already in `boost::asio` for receiving incoming connections resolving
host names, or establishing outgoing connections. After the socket (or
`boost::asio::ssl::stream`) is connected, the `beast::websocket::stream`
is constructed around it and the WebSocket handshake can be performed.
Beast users are free to implement their own "connection manager", but
there is no requirement to do so.
[table
[
[[@http://www.boost.org/doc/html/boost_asio/reference/async_connect.html Beast],
[@http://www.boost.org/doc/html/boost_asio/reference/basic_socket_acceptor/async_accept.html also]]
[[@https://github.com/zaphoyd/websocketpp/blob/378437aecdcb1dfe62096ffd5d944bf1f640ccc3/websocketpp/transport/asio/endpoint.hpp#L52 websocketpp]]
][
[```
#include <boost/asio.hpp>
```]
[```
template <typename config>
class endpoint : public config::socket_type;
```]
]]]]
[[12. WebSocket Handshaking][
Callers invoke `beast::websocket::accept` to perform the WebSocket
handshake, but there is no requirement to use this function. Advanced
users can perform the WebSocket handshake themselves. Beast WebSocket
provides the tools for composing the request or response, and the
Beast HTTP interface provides the container and algorithms for sending
and receiving HTTP/1 messages including the necessary HTTP Upgrade
request for establishing the WebSocket session.
Beast allows the caller to pass the incoming HTTP Upgrade request for
the cases where the caller has already received an HTTP message.
This flexibility permits novel and robust implementations. For example,
a listening socket that can handshake in multiple protocols on the
same port.
Sometimes callers want to read some bytes on the socket before reading
the WebSocket HTTP Upgrade request. Beast allows these already-received
bytes to be supplied to an overload of the accepting function to permit
sophisticated features. For example, a listening socket that can
accept both regular WebSocket and Secure WebSocket (SSL) connections.
[table
[
[[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L501 Beast],
[@https://github.com/vinniefalco/Beast/blob/6c8b4b2f8dde72b01507e4ac7fde4ffea57ebc99/include/beast/websocket/stream.hpp#L401 also]]
[websocketpp]
][
[```
template<class ConstBufferSequence>
void
accept(ConstBufferSequence const& buffers);
template<class Body, class Headers>
void
accept(http::request_v1<Body, Headers> const& request);
```]
[
/<not available>/
]
]]]]
]
]]
]
[endsect]

10
doc/master.qbk
View File

@ -88,13 +88,8 @@ provides implementations of the HTTP and WebSocket protocols.
[[
[link beast.design Design]
][
Design rationale, answers to questions, and
other library comparisons.
]]
[[
[link beast.review For Reviewers]
][
Participants in Beast's formal review should read this first.
Design rationale, answers to questions, library comparisons,
and special considerations for Boost Formal Review participants.
]]
[[
[link beast.ref Reference]
@ -114,7 +109,6 @@ provides implementations of the HTTP and WebSocket protocols.
[include websocket.qbk]
[include examples.qbk]
[include design.qbk]
[include review.qbk]
[section:ref Reference]
[xinclude quickref.xml]