Parser concept, fixes:

A new concept Parser is introduced with routines to read from a stream
into the parser. This solves a problem with the old read interface where
messages must be default constructible and move assignable.

Parser fixes:

* Fix detect invalid reason-phrase octets
* Fix write_eof to set the 'complete' state on success
* Fix consider parse complete if eof received on empty body

WebSocket:

* Increase coverage

TODO.txt

@@ -30,7 +30,8 @@ WebSocket:
 * optimized versions of key/masking, choose prepared_key size
 * invokable unit test
 * Don't try to read requests into empty_body
-
+* Give callers control over the http request/response used during handshake
+* Investigate poor autobahn results in Debug builds
 
 HTTP:
 * Define Parser concept in HTTP
@@ -43,6 +44,7 @@ HTTP:
 * More fine grained parser errors
 * HTTP parser size limit with test (configurable?)
 * HTTP parser trailers with test
+* Decode chunk encoding parameters
 * URL parser, strong URL character checking in HTTP parser
 * Update for rfc7230
 * Consider rename to MessageBody concept

doc/beast.qbk

@@ -40,11 +40,11 @@
 
 [section:intro Introduction]
 
-Beast is a cross-platform C++ library built on Boost, containing two modules
+Beast is a cross-platform C++ library built on Boost.Asio and Boost, containing
-implementing widely used network protocols. Beast.HTTP offers a universal
+two modules implementing widely used network protocols. Beast.HTTP offers a
-model for describing, sending, and receiving HTTP messages while Beast.WebSocket
+universal model for describing, sending, and receiving HTTP messages while
-provides a complete implementation of the WebSocket protocol. Their design
+Beast.WebSocket provides a complete implementation of the WebSocket protocol.
-achieves these goals:
+Their design achieves these goals:
 
 * [*Symmetry.] Interfaces are role-agnostic; the same interfaces can be
 used to build clients, servers, or both.
@@ -60,7 +60,7 @@ strategies; important decisions such as buffer or thread management are
 left to users of the library.
 
 * [*Performance.] The implementation performs competitively, making it a
-realistic choice for building a high performance network server.
+realistic choice for building high performance network servers.
 
 * [*Scalability.] Development of network applications that scale to thousands
 of concurrent connections is possible with the implementation.
@@ -168,12 +168,16 @@ int main()
 
 [section:credits Credits]
 
+Boost.Asio is the inspiration behind which all of the interfaces and
+implementation strategies are built. Some parts of the documentation are
+written to closely resemble the wording and presentation of Boost.Asio
+documentation. Credit goes to Christopher Kohloff for the wonderful
+Asio library and the ideas upon which Beast is built.
+
 Beast would not be possible without the considerable time and patience
 contributed by David Schwartz, Edward Hennis, Howard Hinnant, Miguel Portilla,
 Nikolaos Bougalis, Scott Determan, Scott Schurr, and Ripple Labs for
-supporting its development. Thanks also to Christopher Kohloff, whose Asio
+supporting its development.
-C++ library is the inspiration behind which much of the design and
-documentation is based.
 
 [endsect]
 

doc/http.qbk

@@ -12,7 +12,7 @@ their associated operations including synchronous and asynchronous reading and
 writing of messages in the HTTP/1 wire format using Boost.Asio.
 
 The HTTP protocol is described fully in
-[@https://tools.ietf.org/html/rfc2616 rfc2616]
+[@https://tools.ietf.org/html/rfc7230 rfc7230]
 
 
 

extras/beast/unit_test/print.hpp

@@ -23,7 +23,7 @@ namespace unit_test {
 /** @{ */
 template <class = void>
 void
-print (results const& r, beast::detail::abstract_ostream& stream)
+print (results const& r, abstract_ostream& stream)
 {
     for (auto const& s : r)
     {

include/beast/core/impl/basic_streambuf.ipp

@@ -855,8 +855,11 @@ std::size_t
 read_size_helper(basic_streambuf<
     Allocator> const& streambuf, std::size_t max_size)
 {
-    return std::min<std::size_t>(max_size,
+    auto const avail = streambuf.prepare_size();
-        std::max<std::size_t>(512, streambuf.prepare_size()));
+    if(avail == 0)
+        return std::min(max_size,
+            std::max<std::size_t>(512, streambuf.alloc_size_));
+    return std::min(max_size, avail);
 }
 
 template<class Alloc, class T>

include/beast/http/basic_parser_v1.hpp

@@ -36,11 +36,26 @@ enum values
 };
 } // parse_flag
 
-/** Base class for parsing HTTP/1 requests and responses.
+/** A parser for decoding HTTP/1 wire format messages.
 
-    During parsing, callbacks will be made to the derived class
+    This parser is designed to efficiently parse messages in the
-    if those members are present (detected through SFINAE). The
+    HTTP/1 wire format. It allocates no memory and uses minimal
-    signatures which can be present in the derived class are:<br>
+    state. It will handle chunked encoding and it understands the
+    semantics of the Connection and Content-Length header fields.
+
+    The interface uses CRTP (Curiously Recurring Template Pattern).
+    To use this class, derive from basic_parser. When bytes are
+    presented, the implementation will make a series of zero or
+    more calls to derived class members functions (referred to as
+    "callbacks" from here on) matching a specific signature.
+
+    Callbacks are detected through SFINAE. The derived class may
+    implement as few or as many of the members as needed.
+    These are the signatures of the callbacks:<br>
+
+    @li `void on_start(error_code&)`
+
+        Called when the first valid octet of a new message is received
 
     @li `void on_method(boost::string_ref const&, error_code&)`
 
@@ -106,6 +121,9 @@ enum values
 
     If a callback sets an error, parsing stops at the current octet
     and the error is returned to the caller.
+
+    @tparam isRequest A `bool` indicating whether the parser will be
+    presented with request or response message.
 */
 template<bool isRequest, class Derived>
 class basic_parser_v1
@@ -188,7 +206,8 @@ private:
         s_chunk_data_done,
 
         s_complete,
-        s_restart
+        s_restart,
+        s_closed_complete
     };
 
     enum field_state : std::uint8_t
@@ -341,7 +360,7 @@ public:
     bool
     complete() const
     {
-        return s_ == s_restart;
+        return s_ == s_restart || s_ == s_closed_complete;
     }
 
     /** Write a sequence of buffers to the parser.
@@ -411,6 +430,24 @@ private:
     bool
     needs_eof(std::false_type) const;
 
+    template<class C>
+    class has_on_start_t
+    {
+        template<class T, class R =
+            decltype(std::declval<T>().on_start(
+                std::declval<error_code&>()),
+                    std::true_type{})>
+        static R check(int);
+        template <class>
+        static std::false_type check(...);
+        using type = decltype(check<C>(0));
+    public:
+        static bool const value = type::value;
+    };
+    template<class C>
+    using has_on_start =
+        std::integral_constant<bool, has_on_start_t<C>::value>;
+
     template<class C>
     class has_on_method_t
     {
@@ -596,6 +633,20 @@ private:
     using has_on_complete =
         std::integral_constant<bool, has_on_complete_t<C>::value>;
 
+    void call_on_start(error_code& ec, std::true_type)
+    {
+        impl().on_start(ec);
+    }
+
+    void call_on_start(error_code& ec, std::false_type)
+    {
+    }
+
+    void call_on_start(error_code& ec)
+    {
+        call_on_start(ec, has_on_start<Derived>{});
+    }
+
     void call_on_method(error_code& ec,
         boost::string_ref const& s, std::true_type)
     {

include/beast/http/detail/basic_parser_v1.hpp

@@ -132,7 +132,7 @@ to_value_char(char c)
 }
 
 inline
-std::uint8_t
+std::int8_t
 unhex(char c)
 {
     static std::array<std::int8_t, 256> constexpr tab = {{

include/beast/http/impl/basic_parser_v1.ipp

@@ -9,6 +9,7 @@
 #define BEAST_HTTP_IMPL_BASIC_PARSER_V1_IPP
 
 #include <beast/core/buffer_concepts.hpp>
+#include <cassert>
 
 namespace beast {
 namespace http {
@@ -88,6 +89,11 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         s_ = s_closed;
         return used();
     };
+    auto errc = [&]
+    {
+        s_ = s_closed;
+        return used();
+    };
     auto piece = [&]
     {
         return boost::string_ref{
@@ -113,6 +119,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         switch(s_)
         {
         case s_closed:
+        case s_closed_complete:
             return err(parse_error::connection_closed);
             break;
 
@@ -126,6 +133,9 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         case s_req_method_start:
             if(! is_token(ch))
                 return err(parse_error::bad_method);
+            call_on_start(ec);
+            if(ec)
+                return errc();
             cb_ = &self::call_on_method;
             s_ = s_req_method;
             break;
@@ -134,7 +144,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             if(! is_token(ch))
             {
                 if(cb(nullptr))
-                    return used();
+                    return errc();
                 s_ = s_req_space_before_url;
                 goto redo;
             }
@@ -147,21 +157,23 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             break;
 
         case s_req_url_start:
+        {
             if(ch == ' ')
                 return err(parse_error::bad_uri);
             // VFALCO TODO Better checking for valid URL characters
             if(! is_text(ch))
                 return err(parse_error::bad_uri);
-            if(cb(&self::call_on_uri))
+            assert(! cb_);
-                return used();
+            cb(&self::call_on_uri);
             s_ = s_req_url;
             break;
+        }
 
         case s_req_url:
             if(ch == ' ')
             {
                 if(cb(nullptr))
-                    return used();
+                    return errc();
                 s_ = s_req_http_start;
                 break;
             }
@@ -245,7 +257,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 return err(parse_error::bad_crlf);
             call_on_request(ec);
             if(ec)
-                return used();
+                return errc();
             s_ = s_header_field_start;
             break;
 
@@ -257,7 +269,14 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             content_length_ = no_content_length;
             switch(ch)
             {
-            case 'H': s_ = s_res_H; break;
+            case 'H':
+                call_on_start(ec);
+                if(ec)
+                    return errc();
+                s_ = s_res_H;
+                break;
+            // VFALCO NOTE this allows whitespace at the beginning,
+            //        need to check rfc7230
             case '\r':
             case '\n':
                 break;
@@ -365,13 +384,16 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 s_ = s_res_line_almost_done;
                 break;
             }
+            // VFALCO Is this up to spec?
             if(ch == '\n')
             {
                 s_ = s_header_field_start;
                 break;
             }
+            if(! is_text(ch))
+                return err(parse_error::bad_status);
             if(cb(&self::call_on_reason))
-                return used();
+                return errc();
             pos_ = 0;
             s_ = s_res_status;
             break;
@@ -380,17 +402,19 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             if(ch == '\r')
             {
                 if(cb(nullptr))
-                    return used();
+                    return errc();
                 s_ = s_res_line_almost_done;
                 break;
             }
             if(ch == '\n')
             {
                 if(cb(nullptr))
-                    return used();
+                    return errc();
                 s_ = s_header_field_start;
                 break;
             }
+            if(! is_text(ch))
+                return err(parse_error::bad_status);
             break;
 
         case s_res_line_almost_done:
@@ -402,7 +426,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         case s_res_line_done:
             call_on_response(ec);
             if(ec)
-                return used();
+                return errc();
             s_ = s_header_field_start;
             goto redo;
 
@@ -431,8 +455,8 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 fs_ = h_general;
                 break;
             }
-            if(cb(&self::call_on_field))
+            assert(! cb_);
-                return used();
+            cb(&self::call_on_field);
             s_ = s_header_field;
             break;
         }
@@ -529,7 +553,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             if(ch == ':')
             {
                 if(cb(nullptr))
-                    return used();
+                    return errc();
                 s_ = s_header_value_start;
                 break;
             }
@@ -579,7 +603,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             }
             call_on_value(ec, boost::string_ref{"", 0});
             if(ec)
-                return used();
+                return errc();
             s_ = s_header_field_start;
             goto redo;
 
@@ -629,7 +653,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             }
             pos_ = 0;
             if(cb(&self::call_on_value))
-                return used();
+                return errc();
             s_ = s_header_value_text;
             break;
         }
@@ -641,7 +665,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 if(ch == '\r')
                 {
                     if(cb(nullptr))
-                        return used();
+                        return errc();
                     s_ = s_header_value_discard_lWs;
                     break;
                 }
@@ -775,9 +799,9 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 return err(parse_error::bad_value);
             call_on_value(ec, boost::string_ref(" ", 1));
             if(ec)
-                return used();
+                return errc();
             if(cb(&self::call_on_value))
-                return used();
+                return errc();
             s_ = s_header_value_text;
             break;
 
@@ -811,7 +835,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 return err(parse_error::bad_crlf);
             if(flags_ & parse_flag::trailing)
             {
-                //if(cb(&self::call_on_chunk_complete)) return used();
+                //if(cb(&self::call_on_chunk_complete)) return errc();
                 s_ = s_complete;
                 goto redo;
             }
@@ -821,7 +845,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 (parse_flag::upgrade | parse_flag::connection_upgrade)) /*|| method == "connect"*/;
             auto const maybe_skip = call_on_headers(ec);
             if(ec)
-                return used();
+                return errc();
             switch(maybe_skip)
             {
             case 0: break;
@@ -839,7 +863,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             assert(! cb_);
             call_on_headers(ec);
             if(ec)
-                return used();
+                return errc();
             bool const hasBody =
                 (flags_ & parse_flag::chunked) || (content_length_ > 0 &&
                     content_length_ != no_content_length);
@@ -878,8 +902,8 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         }
 
         case s_body_identity0:
-            if(cb(&self::call_on_body))
+            assert(! cb_);
-                return used();
+            cb(&self::call_on_body);
             s_ = s_body_identity;
             goto redo; // VFALCO fall through?
 
@@ -903,8 +927,8 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         }
 
         case s_body_identity_eof0:
-            if(cb(&self::call_on_body))
+            assert(! cb_);
-                return used();
+            cb(&self::call_on_body);
             s_ = s_body_identity_eof;
             goto redo; // VFALCO fall through?
 
@@ -963,13 +987,13 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
                 s_ = s_header_field_start;
                 break;
             }
-            //call_chunk_header(ec); if(ec) return used();
+            //call_chunk_header(ec); if(ec) return errc();
             s_ = s_chunk_data_start;
             break;
 
         case s_chunk_data_start:
-            if(cb(&self::call_on_body))
+            assert(! cb_);
-                return used();
+            cb(&self::call_on_body);
             s_ = s_chunk_data;
             goto redo; // VFALCO fall through?
 
@@ -991,7 +1015,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
             if(ch != '\r')
                 return err(parse_error::bad_crlf);
             if(cb(nullptr))
-                return used();
+                return errc();
             s_ = s_chunk_data_done;
             break;
 
@@ -1005,10 +1029,10 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
         case s_complete:
             ++p;
             if(cb(nullptr))
-                return used();
+                return errc();
             call_on_complete(ec);
             if(ec)
-                return used();
+                return errc();
             s_ = s_restart;
             return used();
 
@@ -1024,7 +1048,7 @@ write(boost::asio::const_buffer const& buffer, error_code& ec)
     {
         (this->*cb_)(ec, piece());
         if(ec)
-            return used();
+            return errc();
     }
     return used();
 }
@@ -1036,17 +1060,31 @@ write_eof(error_code& ec)
 {
     switch(s_)
     {
+    case s_restart:
+        s_ = s_closed_complete;
+        break;
+
+    case s_closed:
+    case s_closed_complete:
+        break;
+
+    case s_body_identity_eof0:
     case s_body_identity_eof:
         cb_ = nullptr;
         call_on_complete(ec);
         if(ec)
-            return;
+        {
-        return;
+            s_ = s_closed;
-    default:
             break;
         }
-    ec = parse_error::short_read;
+        s_ = s_closed_complete;
+        break;
+
+    default:
         s_ = s_closed;
+        ec = parse_error::short_read;
+        break;
+    }
 }
 
 template<bool isRequest, class Derived>

include/beast/http/impl/read.ipp

@@ -9,6 +9,7 @@
 #define BEAST_HTTP_IMPL_READ_IPP_HPP
 
 #include <beast/http/parser_v1.hpp>
+#include <beast/http/type_check.hpp>
 #include <beast/core/bind_handler.hpp>
 #include <beast/core/handler_alloc.hpp>
 #include <beast/core/stream_concepts.hpp>
@@ -19,6 +20,185 @@ namespace http {
 
 namespace detail {
 
+template<class Stream,
+    class Streambuf, class Parser, class Handler>
+class parse_op
+{
+    using alloc_type =
+        handler_alloc<char, Handler>;
+
+    struct data
+    {
+        Stream& s;
+        Streambuf& sb;
+        Parser& p;
+        Handler h;
+        bool started = false;
+        bool cont;
+        int state = 0;
+
+        template<class DeducedHandler>
+        data(DeducedHandler&& h_, Stream& s_,
+                Streambuf& sb_, Parser& p_)
+            : s(s_)
+            , sb(sb_)
+            , p(p_)
+            , h(std::forward<DeducedHandler>(h_))
+            , cont(boost_asio_handler_cont_helpers::
+                is_continuation(h))
+        {
+        }
+    };
+
+    std::shared_ptr<data> d_;
+
+public:
+    parse_op(parse_op&&) = default;
+    parse_op(parse_op const&) = default;
+
+    template<class DeducedHandler, class... Args>
+    parse_op(DeducedHandler&& h, Stream& s, Args&&... args)
+        : d_(std::allocate_shared<data>(alloc_type{h},
+            std::forward<DeducedHandler>(h), s,
+                std::forward<Args>(args)...))
+    {
+        (*this)(error_code{}, 0, false);
+    }
+
+    void
+    operator()(error_code ec,
+        std::size_t bytes_transferred, bool again = true);
+
+    friend
+    void* asio_handler_allocate(
+        std::size_t size, parse_op* op)
+    {
+        return boost_asio_handler_alloc_helpers::
+            allocate(size, op->d_->h);
+    }
+
+    friend
+    void asio_handler_deallocate(
+        void* p, std::size_t size, parse_op* op)
+    {
+        return boost_asio_handler_alloc_helpers::
+            deallocate(p, size, op->d_->h);
+    }
+
+    friend
+    bool asio_handler_is_continuation(parse_op* op)
+    {
+        return op->d_->cont;
+    }
+
+    template <class Function>
+    friend
+    void asio_handler_invoke(Function&& f, parse_op* op)
+    {
+        return boost_asio_handler_invoke_helpers::
+            invoke(f, op->d_->h);
+    }
+};
+
+template<class Stream,
+    class Streambuf, class Parser, class Handler>
+void
+parse_op<Stream, Streambuf, Parser, Handler>::
+operator()(error_code ec, std::size_t bytes_transferred, bool again)
+{
+    auto& d = *d_;
+    d.cont = d.cont || again;
+    while(d.state != 99)
+    {
+        switch(d.state)
+        {
+        case 0:
+        {
+            auto const used =
+                d.p.write(d.sb.data(), ec);
+            if(ec)
+            {
+                // call handler
+                d.state = 99;
+                d.s.get_io_service().post(
+                    bind_handler(std::move(*this), ec, 0));
+                return;
+            }
+            if(used > 0)
+                d.started = true;
+            d.sb.consume(used);
+            if(d.p.complete())
+            {
+                // call handler
+                d.state = 99;
+                d.s.get_io_service().post(
+                    bind_handler(std::move(*this), ec, 0));
+                return;
+            }
+            d.state = 1;
+            break;
+        }
+
+        case 1:
+            // read
+            d.state = 2;
+            d.s.async_read_some(d.sb.prepare(
+                read_size_helper(d.sb, 65536)),
+                    std::move(*this));
+            return;
+
+        // got data
+        case 2:
+        {
+            if(ec == boost::asio::error::eof)
+            {
+                if(! d.started)
+                {
+                    // call handler
+                    d.state = 99;
+                    break;
+                }
+                // Caller will see eof on next read.
+                ec = {};
+                d.p.write_eof(ec);
+                assert(ec || d.p.complete());
+                // call handler
+                d.state = 99;
+                break;
+            }
+            if(ec)
+            {
+                // call handler
+                d.state = 99;
+                break;
+            }
+            d.sb.commit(bytes_transferred);
+            auto const used = d.p.write(d.sb.data(), ec);
+            if(ec)
+            {
+                // call handler
+                d.state = 99;
+                break;
+            }
+            if(used > 0)
+                d.started = true;
+            d.sb.consume(used);
+            if(d.p.complete())
+            {
+                // call handler
+                d.state = 99;
+                break;
+            }
+            d.state = 1;
+            break;
+        }
+        }
+    }
+    d.h(ec);
+}
+
+//------------------------------------------------------------------------------
+
 template<class Stream, class Streambuf,
     bool isRequest, class Body, class Headers,
         class Handler>
@@ -69,12 +249,11 @@ public:
             std::forward<DeducedHandler>(h), s,
                 std::forward<Args>(args)...))
     {
-        (*this)(error_code{}, 0, false);
+        (*this)(error_code{}, false);
     }
 
     void
-    operator()(error_code ec,
+    operator()(error_code ec, bool again = true);
-        std::size_t bytes_transferred, bool again = true);
 
     friend
     void* asio_handler_allocate(
@@ -112,98 +291,25 @@ template<class Stream, class Streambuf,
         class Handler>
 void
 read_op<Stream, Streambuf, isRequest, Body, Headers, Handler>::
-operator()(error_code ec, std::size_t bytes_transferred, bool again)
+operator()(error_code ec, bool again)
 {
     auto& d = *d_;
     d.cont = d.cont || again;
-    while(d.state != 99)
+    while(! ec && d.state != 99)
     {
         switch(d.state)
         {
         case 0:
-        {
-            auto const used =
-                d.p.write(d.sb.data(), ec);
-            if(ec)
-            {
-                // call handler
-                d.state = 99;
-                d.s.get_io_service().post(
-                    bind_handler(std::move(*this), ec, 0));
-                return;
-            }
-            if(used > 0)
-                d.started = true;
-            d.sb.consume(used);
-            if(d.p.complete())
-            {
-                // call handler
-                d.state = 99;
-                d.m = d.p.release();
-                d.s.get_io_service().post(
-                    bind_handler(std::move(*this), ec, 0));
-                return;
-            }
             d.state = 1;
-            break;
+            async_parse(d.s, d.sb, d.p, std::move(*this));
-        }
+            return;
 
         case 1:
-            // read
-            d.state = 2;
-            d.s.async_read_some(d.sb.prepare(
-                read_size_helper(d.sb, 65536)),
-                    std::move(*this));
-            return;
-
-        // got data
-        case 2:
-        {
-            if(ec == boost::asio::error::eof)
-            {
-                if(! d.started)
-                {
-                    // call handler
-                    d.state = 99;
-                    break;
-                }
-                // Caller will see eof on next read.
-                ec = {};
-                d.p.write_eof(ec);
-                if(! ec)
-                {
-                    assert(d.p.complete());
-                    d.m = d.p.release();
-                }
-                // call handler
-                d.state = 99;
-                break;
-            }
-            if(ec)
-            {
-                // call handler
-                d.state = 99;
-                break;
-            }
-            d.sb.commit(bytes_transferred);
-            d.sb.consume(d.p.write(d.sb.data(), ec));
-            if(ec)
-            {
-                // call handler
-                d.state = 99;
-                break;
-            }
-            if(d.p.complete())
-            {
             // call handler
             d.state = 99;
             d.m = d.p.release();
             break;
         }
-            d.state = 1;
-            break;
-        }
-        }
     }
     d.h(ec);
 }
@@ -212,12 +318,91 @@ operator()(error_code ec, std::size_t bytes_transferred, bool again)
 
 //------------------------------------------------------------------------------
 
+template<class SyncReadStream, class Streambuf, class Parser>
+void
+parse(SyncReadStream& stream,
+    Streambuf& streambuf, Parser& parser)
+{
+    error_code ec;
+    parse(stream, streambuf, parser, ec);
+    if(ec)
+        throw boost::system::system_error{ec};
+}
+
+template<class SyncReadStream, class Streambuf, class Parser>
+void
+parse(SyncReadStream& stream, Streambuf& streambuf,
+    Parser& parser, error_code& ec)
+{
+    static_assert(is_SyncReadStream<SyncReadStream>::value,
+        "SyncReadStream requirements not met");
+    static_assert(is_Streambuf<Streambuf>::value,
+        "Streambuf requirements not met");
+    static_assert(is_Parser<Parser>::value,
+        "Parser requirements not met");
+    bool started = false;
+    for(;;)
+    {
+        auto used =
+            parser.write(streambuf.data(), ec);
+        if(ec)
+            return;
+        streambuf.consume(used);
+        if(used > 0)
+            started = true;
+        if(parser.complete())
+            break;
+        streambuf.commit(stream.read_some(
+            streambuf.prepare(read_size_helper(
+                streambuf, 65536)), ec));
+        if(ec && ec != boost::asio::error::eof)
+            return;
+        if(ec == boost::asio::error::eof)
+        {
+            if(! started)
+                return;
+            // Caller will see eof on next read.
+            ec = {};
+            parser.write_eof(ec);
+            if(ec)
+                return;
+            assert(parser.complete());
+            break;
+        }
+    }
+}
+
+template<class AsyncReadStream,
+    class Streambuf, class Parser, class ReadHandler>
+typename async_completion<
+    ReadHandler, void(error_code)>::result_type
+async_parse(AsyncReadStream& stream,
+    Streambuf& streambuf, Parser& parser, ReadHandler&& handler)
+{
+    static_assert(is_AsyncReadStream<AsyncReadStream>::value,
+        "AsyncReadStream requirements not met");
+    static_assert(is_Streambuf<Streambuf>::value,
+        "Streambuf requirements not met");
+    static_assert(is_Parser<Parser>::value,
+        "Parser requirements not met");
+    beast::async_completion<ReadHandler,
+        void(error_code)> completion(handler);
+    detail::parse_op<AsyncReadStream, Streambuf,
+        Parser, decltype(completion.handler)>{
+            completion.handler, stream, streambuf, parser};
+    return completion.result.get();
+}
+
 template<class SyncReadStream, class Streambuf,
     bool isRequest, class Body, class Headers>
 void
 read(SyncReadStream& stream, Streambuf& streambuf,
     message_v1<isRequest, Body, Headers>& msg)
 {
+    static_assert(is_SyncReadStream<SyncReadStream>::value,
+        "SyncReadStream requirements not met");
+    static_assert(is_Streambuf<Streambuf>::value,
+        "Streambuf requirements not met");
     error_code ec;
     read(stream, streambuf, msg, ec);
     if(ec)
@@ -236,40 +421,11 @@ read(SyncReadStream& stream, Streambuf& streambuf,
     static_assert(is_Streambuf<Streambuf>::value,
         "Streambuf requirements not met");
     parser_v1<isRequest, Body, Headers> p;
-    bool started = false;
+    parse(stream, streambuf, p, ec);
-    for(;;)
-    {
-        auto used =
-            p.write(streambuf.data(), ec);
-        if(ec)
-            return;
-        streambuf.consume(used);
-        if(used > 0)
-            started = true;
-        if(p.complete())
-        {
-            m = p.release();
-            break;
-        }
-        streambuf.commit(stream.read_some(
-            streambuf.prepare(read_size_helper(
-                streambuf, 65536)), ec));
-        if(ec && ec != boost::asio::error::eof)
-            return;
-        if(ec == boost::asio::error::eof)
-        {
-            if(! started)
-                return;
-            // Caller will see eof on next read.
-            ec = {};
-            p.write_eof(ec);
     if(ec)
         return;
     assert(p.complete());
     m = p.release();
-            break;
-        }
-    }
 }
 
 template<class AsyncReadStream, class Streambuf,

include/beast/http/parser_v1.hpp

@@ -11,7 +11,6 @@
 #include <beast/http/basic_parser_v1.hpp>
 #include <beast/http/message_v1.hpp>
 #include <beast/core/error.hpp>
-#include <boost/optional.hpp>
 #include <functional>
 #include <string>
 #include <type_traits>
@@ -39,6 +38,8 @@ struct parser_response
 
     This class uses the basic HTTP/1 wire format parser to convert
     a series of octets into a `message_v1`.
+
+    @note A new instance of the parser is required for each message.
 */
 template<bool isRequest, class Body, class Headers>
 class parser_v1
@@ -47,9 +48,12 @@ class parser_v1
     , private std::conditional<isRequest,
         detail::parser_request, detail::parser_response>::type
 {
+public:
+    /// The type of message this parser produces.
     using message_type =
         message_v1<isRequest, Body, Headers>;
 
+private:
     std::string field_;
     std::string value_;
     message_type m_;
@@ -57,15 +61,55 @@ class parser_v1
 
 public:
     parser_v1(parser_v1&&) = default;
+    parser_v1(parser_v1 const&) = delete;
+    parser_v1& operator=(parser_v1&&) = delete;
+    parser_v1& operator=(parser_v1 const&) = delete;
 
-    parser_v1()
+    /** Construct the parser.
-        : r_(m_)
+
+        @param args A list of arguments forwarded to the message constructor.
+    */
+    template<class... Args>
+    explicit
+    parser_v1(Args&&... args)
+        : m_(std::forward<Args>(args)...)
+        , r_(m_)
     {
     }
 
+    /** Returns the parsed message.
+
+        Only valid if `complete()` would return `true`.
+    */
+    message_type const&
+    get() const
+    {
+        return m_;
+    }
+
+    /** Returns the parsed message.
+
+        Only valid if `complete()` would return `true`.
+    */
+    message_type&
+    get()
+    {
+        return m_;
+    }
+
+    /** Returns the parsed message.
+
+        Ownership is transferred to the caller.
+        Only valid if `complete()` would return `true`.
+
+        Requires:
+            `message<isRequest, Body, Headers>` is MoveConstructible
+    */
     message_type
     release()
     {
+        static_assert(std::is_move_constructible<decltype(m_)>::value,
+            "MoveConstructible requirements not met");
         return std::move(m_);
     }
 
@@ -84,6 +128,10 @@ private:
         }
     }
 
+    void on_start(error_code&)
+    {
+    }
+
     void on_method(boost::string_ref const& s, error_code&)
     {
         this->method_.append(s.data(), s.size());

include/beast/http/read.hpp

@@ -17,6 +17,127 @@
 namespace beast {
 namespace http {
 
+/** Parse a HTTP/1 message from a stream.
+
+    This function synchronously reads from a stream and passes
+    data to the specified parser. The call will block until one
+    of the following conditions are met:
+
+    @li A complete message is read in.
+
+    @li An error occurs in the stream or parser.
+
+    This function is implemented in terms of one or more calls
+    to the stream's `read_some` function. The implementation may
+    read additional octets that lie past the end of the message
+    being parsed. This additional data is stored in the stream
+    buffer, which may be used in subsequent calls.
+
+    @param stream The stream from which the data is to be read.
+    The type must support the @b `SyncReadStream` concept.
+
+    @param streambuf A `Streambuf` holding additional bytes
+    read by the implementation from the stream. This is both
+    an input and an output parameter; on entry, any data in the
+    stream buffer's input sequence will be given to the parser
+    first.
+
+    @param parser An object meeting the requirements of Parser
+    which will receive the data.
+
+    @throws boost::system::system_error on failure.
+*/
+template<class SyncReadStream, class Streambuf, class Parser>
+void
+parse(SyncReadStream& stream,
+    Streambuf& streambuf, Parser& parser);
+
+/** Parse a HTTP/1 message from a stream.
+
+    This function synchronously reads from a stream and passes
+    data to the specified parser. The call will block until one
+    of the following conditions are met:
+
+    @li A complete message is read in.
+
+    @li An error occurs in the stream or parser.
+
+    This function is implemented in terms of one or more calls
+    to the stream's `read_some` function. The implementation may
+    read additional octets that lie past the end of the message
+    being parsed. This additional data is stored in the stream
+    buffer, which may be used in subsequent calls.
+
+    @param stream The stream from which the data is to be read.
+    The type must support the @b `SyncReadStream` concept.
+
+    @param streambuf A `Streambuf` holding additional bytes
+    read by the implementation from the stream. This is both
+    an input and an output parameter; on entry, any data in the
+    stream buffer's input sequence will be given to the parser
+    first.
+
+    @param parser An object meeting the requirements of `Parser`
+    which will receive the data.
+
+    @param ec Set to the error, if any occurred.
+*/
+template<class SyncReadStream, class Streambuf, class Parser>
+void
+parse(SyncReadStream& stream,
+    Streambuf& streambuf, Parser& parser, error_code& ec);
+
+/** Start an asynchronous operation to parse a HTTP/1 message from a stream.
+
+    This function is used to asynchronously read from a stream and
+    pass the data to the specified parser. The function call always
+    returns immediately. The asynchronous operation will continue
+    until one of the following conditions is true:
+
+    @li A complete message is read in.
+
+    @li An error occurs in the stream or parser.
+
+    This operation is implemented in terms of one or more calls to
+    the next layer's `async_read_some` function, and is known as a
+    <em>composed operation</em>. The program must ensure that the
+    stream performs no other operations until this operation completes.
+
+    @param stream The stream from which the data is to be read.
+    The type must support the @b `AsyncReadStream` concept.
+
+    @param streambuf A `Streambuf` holding additional bytes
+    read by the implementation from the stream. This is both
+    an input and an output parameter; on entry, any data in the
+    stream buffer's input sequence will be given to the parser
+    first.
+
+    @param parser An object meeting the requirements of `Parser`
+    which will receive the data. This object must remain valid
+    until the completion handler is invoked.
+
+    @param handler The handler to be called when the request completes.
+    Copies will be made of the handler as required. The equivalent
+    function signature of the handler must be:
+    @code void handler(
+        error_code const& error // result of operation
+    ); @endcode
+    Regardless of whether the asynchronous operation completes
+    immediately or not, the handler will not be invoked from within
+    this function. Invocation of the handler will be performed in a
+    manner equivalent to using `boost::asio::io_service::post`.
+*/
+template<class AsyncReadStream,
+    class Streambuf, class Parser, class ReadHandler>
+#if GENERATING_DOCS
+void_or_deduced
+#else
+typename async_completion<
+    ReadHandler, void(error_code)>::result_type
+#endif
+async_parse(AsyncReadStream& stream, Streambuf& streambuf,
+    Parser& parser, ReadHandler&& handler);
+
 /** Read a HTTP/1 message from a stream.
 
     This function is used to synchronously read a message from
@@ -25,18 +146,22 @@ namespace http {
 
     @li A complete message is read in.
 
-    @li An error occurs on the stream.
+    @li An error occurs in the stream or parser.
 
-    This function is implemented in terms of one or more calls to the
+    This function is implemented in terms of one or more calls
-    stream's `read_some` function.
+    to the stream's `read_some` function. The implementation may
+    read additional octets that lie past the end of the message
+    being parsed. This additional data is stored in the stream
+    buffer, which may be used in subsequent calls.
 
-    @param stream The stream to which the data is to be written.
+    @param stream The stream from which the data is to be read.
     The type must support the @b `SyncReadStream` concept.
 
-    @param streambuf An object meeting the @b `Streambuf` type requirements
+    @param streambuf A `Streambuf` holding additional bytes
-    used to hold unread bytes. The implementation may read past the end of
+    read by the implementation from the stream. This is both
-    the message. The extra bytes are stored here, to be presented in a
+    an input and an output parameter; on entry, any data in the
-    subsequent call to @ref read.
+    stream buffer's input sequence will be given to the parser
+    first.
 
     @param msg An object used to store the message. Any
     contents will be overwritten.
@@ -57,21 +182,25 @@ read(SyncReadStream& stream, Streambuf& streambuf,
 
     @li A complete message is read in.
 
-    @li An error occurs on the stream.
+    @li An error occurs in the stream or parser.
 
-    This function is implemented in terms of one or more calls to the
+    This function is implemented in terms of one or more calls
-    stream's `read_some` function.
+    to the stream's `read_some` function. The implementation may
+    read additional octets that lie past the end of the message
+    being parsed. This additional data is stored in the stream
+    buffer, which may be used in subsequent calls.
 
-    @param stream The stream to which the data is to be written.
+    @param stream The stream from which the data is to be read.
     The type must support the @b `SyncReadStream` concept.
 
-    @param streambuf An object meeting the @b `Streambuf` type requirements
+    @param streambuf A `Streambuf` holding additional bytes
-    used to hold unread bytes. The implementation may read past the end of
+    read by the implementation from the stream. This is both
-    the message. The extra bytes are stored here, to be presented in a
+    an input and an output parameter; on entry, any data in the
-    subsequent call to @ref read.
+    stream buffer's input sequence will be given to the parser
+    first.
 
-    @param msg An object used to store the message. Any contents
+    @param msg An object used to store the message. Any
-    will be overwritten.
+    contents will be overwritten.
 
     @param ec Set to the error, if any occurred.
 */
@@ -90,7 +219,7 @@ read(SyncReadStream& stream, Streambuf& streambuf,
 
     @li A complete message is read in.
 
-    @li An error occurs on the stream.
+    @li An error occurs in the stream or parser.
 
     This operation is implemented in terms of one or more calls to the
     next layer's `async_read_some` function, and is known as a
@@ -100,10 +229,11 @@ read(SyncReadStream& stream, Streambuf& streambuf,
     @param stream The stream to read the message from.
     The type must support the @b `AsyncReadStream` concept.
 
-    @param streambuf A Streambuf used to hold unread bytes. The
+    @param streambuf A `Streambuf` holding additional bytes
-    implementation may read past the end of the message. The extra
+    read by the implementation from the stream. This is both
-    bytes are stored here, to be presented in a subsequent call to
+    an input and an output parameter; on entry, any data in the
-    @ref async_read.
+    stream buffer's input sequence will be given to the parser
+    first.
 
     @param msg An object used to store the message. Any contents
     will be overwritten.

include/beast/http/type_check.hpp

@@ -40,11 +40,9 @@ class is_Parser
     static std::false_type check2(...);
     using type2 = decltype(check2<T>(0));
 
-    template<class U, class R =
+    template<class U, class R = decltype(
-        std::is_convertible<decltype(
+        std::declval<U>().write_eof(std::declval<error_code&>()),
-            std::declval<U>().write_eof(
+            std::true_type{})>
-                std::declval<error_code&>())),
-            std::size_t>>
     static R check3(int);
     template<class>
     static std::false_type check3(...);

test/Jamfile

@@ -75,9 +75,9 @@ unit-test websocket-tests :
     websocket/rfc6455.cpp
     websocket/stream.cpp
     websocket/teardown.cpp
-    websocket/utf8_checker.cpp
     websocket/detail/frame.cpp
     websocket/detail/mask.cpp
+    websocket/detail/utf8_checker.cpp
     ;
 
 exe websocket-echo :

test/core/CMakeLists.txt

@@ -7,6 +7,7 @@ GroupSources(test/core "/")
 add_executable (core-tests
     ${BEAST_INCLUDES}
     ../../extras/beast/unit_test/main.cpp
+    buffer_test.hpp
     async_completion.cpp
     basic_streambuf.cpp
     bind_handler.cpp

test/core/basic_streambuf.cpp

@@ -8,6 +8,7 @@
 // Test that header file is self-contained.
 #include <beast/core/basic_streambuf.hpp>
 
+#include "buffer_test.hpp"
 #include <beast/core/streambuf.hpp>
 #include <beast/core/to_string.hpp>
 #include <beast/unit_test/suite.hpp>
@@ -148,6 +149,24 @@ public:
         return to_string(sb1.data()) == to_string(sb2.data());
     }
 
+    template<class ConstBufferSequence>
+    void
+    expect_size(std::size_t n, ConstBufferSequence const& buffers)
+    {
+        expect(test::size_pre(buffers) == n);
+        expect(test::size_post(buffers) == n);
+        expect(test::size_rev_pre(buffers) == n);
+        expect(test::size_rev_post(buffers) == n);
+    }
+
+    template<class U, class V>
+    static
+    void
+    self_assign(U& u, V&& v)
+    {
+        u = std::forward<V>(v);
+    }
+
     void testSpecialMembers()
     {
         using boost::asio::buffer;
@@ -177,19 +196,31 @@ public:
             {
                 streambuf sb2(std::move(sb));
                 expect(to_string(sb2.data()) == s);
-                expect(buffer_size(sb.data()) == 0);
+                expect_size(0, sb.data());
                 sb = std::move(sb2);
                 expect(to_string(sb.data()) == s);
-                expect(buffer_size(sb2.data()) == 0);
+                expect_size(0, sb2.data());
             }
-            sb = sb;
+            self_assign(sb, sb);
-            sb = std::move(sb);
+            expect(to_string(sb.data()) == s);
+            self_assign(sb, std::move(sb));
+            expect(to_string(sb.data()) == s);
         }
         }}}
+        try
+        {
+            streambuf sb0(0);
+            fail();
+        }
+        catch(std::exception const&)
+        {
+            pass();
+        }
     }
 
     void testAllocator()
     {
+        // VFALCO This needs work
         {
             using alloc_type =
                 test_allocator<char, false, false, false, false>;
@@ -206,7 +237,6 @@ public:
             sb_type sb2(sb);
             expect(sb2.get_allocator().id() == 2);
             sb_type sb3(sb, alloc_type{});
-            //expect(sb3.get_allocator().id() == 3);
         }
     }
 
@@ -223,21 +253,9 @@ public:
         {
             streambuf sb(2);
             sb.prepare(2);
-            {
+            expect(test::buffer_count(sb.prepare(5)) == 2);
-                auto const bs = sb.prepare(5);
+            expect(test::buffer_count(sb.prepare(8)) == 3);
-                expect(std::distance(
+            expect(test::buffer_count(sb.prepare(4)) == 2);
-                    bs.begin(), bs.end()) == 2);
-            }
-            {
-                auto const bs = sb.prepare(8);
-                expect(std::distance(
-                    bs.begin(), bs.end()) == 3);
-            }
-            {
-                auto const bs = sb.prepare(4);
-                expect(std::distance(
-                    bs.begin(), bs.end()) == 2);
-            }
         }
     }
 
@@ -248,10 +266,21 @@ public:
         sb.prepare(2);
         sb.prepare(5);
         sb.commit(1);
-        expect(buffer_size(sb.data()) == 1);
+        expect_size(1, sb.data());
     }
 
-    void testStreambuf()
+    void testConsume()
+    {
+        using boost::asio::buffer_size;
+        streambuf sb(1);
+        expect_size(5, sb.prepare(5));
+        sb.commit(3);
+        expect_size(3, sb.data());
+        sb.consume(1);
+        expect_size(2, sb.data());
+    }
+
+    void testMatrix()
     {
         using boost::asio::buffer;
         using boost::asio::buffer_cast;
@@ -354,41 +383,11 @@ public:
         sb.commit(1);
         sb.prepare(2);
         sb.commit(2);
-        expect(buffer_size(sb.data()) == 3);
+        expect_size(3, sb.data());
         sb.prepare(1);
-        expect(buffer_size(sb.prepare(3)) == 3);
+        expect_size(3, sb.prepare(3));
-        expect(read_size_helper(sb, 3) == 3);
         sb.commit(2);
-        try
+        expect(test::buffer_count(sb.data()) == 4);
-        {
-            streambuf sb0(0);
-            fail();
-        }
-        catch(std::exception const&)
-        {
-            pass();
-        }
-        std::size_t n;
-        n = 0;
-        for(auto it = sb.data().begin();
-                it != sb.data().end(); it++)
-            ++n;
-        expect(n == 4);
-        n = 0;
-        for(auto it = sb.data().begin();
-                it != sb.data().end(); ++it)
-            ++n;
-        expect(n == 4);
-        n = 0;
-        for(auto it = sb.data().end();
-                it != sb.data().begin(); it--)
-            ++n;
-        expect(n == 4);
-        n = 0;
-        for(auto it = sb.data().end();
-                it != sb.data().begin(); --it)
-            ++n;
-        expect(n == 4);
     }
 
     void testOutputStream()
@@ -398,15 +397,63 @@ public:
         expect(to_string(sb.data()) == "x");
     }
 
+    void testReadSizeHelper()
+    {
+        using boost::asio::buffer_size;
+        {
+            streambuf sb(10);
+            expect(read_size_helper(sb, 0) == 0);
+            expect(read_size_helper(sb, 1) == 1);
+            expect(read_size_helper(sb, 10) == 10);
+            expect(read_size_helper(sb, 20) == 20);
+            expect(read_size_helper(sb, 1000) == 512);
+            sb.prepare(3);
+            sb.commit(3);
+            expect(read_size_helper(sb, 10) == 7);
+            expect(read_size_helper(sb, 1000) == 7);
+        }
+        {
+            streambuf sb(1000);
+            expect(read_size_helper(sb, 0) == 0);
+            expect(read_size_helper(sb, 1) == 1);
+            expect(read_size_helper(sb, 1000) == 1000);
+            expect(read_size_helper(sb, 2000) == 1000);
+            sb.prepare(3);
+            expect(read_size_helper(sb, 0) == 0);
+            expect(read_size_helper(sb, 1) == 1);
+            expect(read_size_helper(sb, 1000) == 1000);
+            expect(read_size_helper(sb, 2000) == 1000);
+            sb.commit(3);
+            expect(read_size_helper(sb, 0) == 0);
+            expect(read_size_helper(sb, 1) == 1);
+            expect(read_size_helper(sb, 1000) == 997);
+            expect(read_size_helper(sb, 2000) == 997);
+            sb.consume(2);
+            expect(read_size_helper(sb, 0) == 0);
+            expect(read_size_helper(sb, 1) == 1);
+            expect(read_size_helper(sb, 1000) == 997);
+            expect(read_size_helper(sb, 2000) == 997);
+        }
+        {
+            streambuf sb(2);
+            expect(test::buffer_count(sb.prepare(2)) == 1);
+            expect(test::buffer_count(sb.prepare(3)) == 2);
+            expect(buffer_size(sb.prepare(5)) == 5);
+            expect(read_size_helper(sb, 10) == 6);
+        }
+    }
+
     void run() override
     {
         testSpecialMembers();
         testAllocator();
         testPrepare();
         testCommit();
-        testStreambuf();
+        testConsume();
+        testMatrix();
         testIterators();
         testOutputStream();
+        testReadSizeHelper();
     }
 };
 

test/core/buffer_cat.cpp

@@ -116,7 +116,8 @@ public:
 
         try
         {
-            expect((buffer_size(*bs.end()) == 0, false));
+            buffer_size(*bs.end());
+            fail();
         }
         catch(std::exception const&)
         {

test/core/buffer_test.hpp

@@ -0,0 +1,89 @@
+//
+// Copyright (c) 2013-2016 Vinnie Falco (vinnie dot falco at gmail dot com)
+//
+// Distributed under the Boost Software License, Version 1.0. (See accompanying
+// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+//
+
+#ifndef BEAST_TEST_BUFFER_TEST_HPP
+#define BEAST_TEST_BUFFER_TEST_HPP
+
+#include <beast/core/buffer_concepts.hpp>
+#include <boost/asio/buffer.hpp>
+#include <algorithm>
+#include <type_traits>
+
+namespace beast {
+namespace test {
+
+template<class ConstBufferSequence>
+typename std::enable_if<
+    is_ConstBufferSequence<ConstBufferSequence>::value,
+        std::size_t>::type
+buffer_count(ConstBufferSequence const& buffers)
+{
+    return std::distance(buffers.begin(), buffers.end());
+}
+
+template<class ConstBufferSequence>
+typename std::enable_if<
+    is_ConstBufferSequence<ConstBufferSequence>::value,
+        std::size_t>::type
+size_pre(ConstBufferSequence const& buffers)
+{
+    std::size_t n = 0;
+    for(auto it = buffers.begin(); it != buffers.end(); ++it)
+    {
+        typename ConstBufferSequence::const_iterator it0(std::move(it));
+        typename ConstBufferSequence::const_iterator it1(it0);
+        typename ConstBufferSequence::const_iterator it2;
+        it2 = it1;
+        n += boost::asio::buffer_size(*it2);
+        it = std::move(it2);
+    }
+    return n;
+}
+
+template<class ConstBufferSequence>
+typename std::enable_if<
+    is_ConstBufferSequence<ConstBufferSequence>::value,
+        std::size_t>::type
+size_post(ConstBufferSequence const& buffers)
+{
+    std::size_t n = 0;
+    for(auto it = buffers.begin(); it != buffers.end(); it++)
+        n += boost::asio::buffer_size(*it);
+    return n;
+}
+
+template<class ConstBufferSequence>
+typename std::enable_if<
+    is_ConstBufferSequence<ConstBufferSequence>::value,
+        std::size_t>::type
+size_rev_pre(ConstBufferSequence const& buffers)
+{
+    std::size_t n = 0;
+    for(auto it = buffers.end(); it != buffers.begin();)
+        n += boost::asio::buffer_size(*--it);
+    return n;
+}
+
+template<class ConstBufferSequence>
+typename std::enable_if<
+    is_ConstBufferSequence<ConstBufferSequence>::value,
+        std::size_t>::type
+size_rev_post(ConstBufferSequence const& buffers)
+{
+    std::size_t n = 0;
+    for(auto it = buffers.end(); it != buffers.begin();)
+    {
+        it--;
+        n += boost::asio::buffer_size(*it);
+    }
+    return n;
+}
+
+} // test
+} // beast
+
+#endif

test/core/consuming_buffers.cpp

@@ -8,6 +8,8 @@
 // Test that header file is self-contained.
 #include <beast/core/consuming_buffers.hpp>
 
+#include "buffer_test.hpp"
+#include <beast/core/to_string.hpp>
 #include <beast/unit_test/suite.hpp>
 #include <boost/asio/buffer.hpp>
 #include <string>
@@ -17,22 +19,25 @@ namespace beast {
 class consuming_buffers_test : public beast::unit_test::suite
 {
 public:
-    template<class ConstBufferSequence>
+    template<class Buffers1, class Buffers2>
     static
-    std::string
+    bool
-    to_string(ConstBufferSequence const& bs)
+    eq(Buffers1 const& lhs, Buffers2 const& rhs)
     {
-        using boost::asio::buffer_cast;
+        return to_string(lhs) == to_string(rhs);
-        using boost::asio::buffer_size;
-        std::string s;
-        s.reserve(buffer_size(bs));
-        for(auto const& b : bs)
-            s.append(buffer_cast<char const*>(b),
-                buffer_size(b));
-        return s;
     }
 
-    void testBuffers()
+    template<class ConstBufferSequence>
+    void
+    expect_size(std::size_t n, ConstBufferSequence const& buffers)
+    {
+        expect(test::size_pre(buffers) == n);
+        expect(test::size_post(buffers) == n);
+        expect(test::size_rev_pre(buffers) == n);
+        expect(test::size_rev_post(buffers) == n);
+    }
+
+    void testMatrix()
     {
         using boost::asio::buffer;
         using boost::asio::const_buffer;
@@ -54,16 +59,23 @@ public:
                 const_buffer{&buf[i+j], k}}};
             consuming_buffers<decltype(bs)> cb(bs);
             expect(to_string(cb) == s);
+            expect_size(s.size(), cb);
             cb.consume(0);
+            expect(eq(cb, consumed_buffers(bs, 0)));
             expect(to_string(cb) == s);
+            expect_size(s.size(), cb);
             cb.consume(x);
             expect(to_string(cb) == s.substr(x));
+            expect(eq(cb, consumed_buffers(bs, x)));
             cb.consume(y);
             expect(to_string(cb) == s.substr(x+y));
+            expect(eq(cb, consumed_buffers(bs, x+y)));
             cb.consume(z);
             expect(to_string(cb) == "");
+            expect(eq(cb, consumed_buffers(bs, x+y+z)));
             cb.consume(1);
             expect(to_string(cb) == "");
+            expect(eq(cb, consumed_buffers(bs, x+y+z)));
         }
         }}}}
     }
@@ -94,7 +106,7 @@ public:
 
     void run() override
     {
-        testBuffers();
+        testMatrix();
         testNullBuffers();
         testIterator();
     }

test/core/prepare_buffers.cpp

@@ -33,20 +33,20 @@ public:
         return s;
     }
 
-    void testBuffers()
+    template<class BufferType>
+    void testMatrix()
     {
         using boost::asio::buffer_size;
-        using boost::asio::const_buffer;
+        std::string s = "Hello, world";
-        std::string const s = "Hello, world";
         expect(s.size() == 12);
         for(std::size_t x = 1; x < 4; ++x) {
         for(std::size_t y = 1; y < 4; ++y) {
         std::size_t z = s.size() - (x + y);
         {
-            std::array<const_buffer, 3> bs{{
+            std::array<BufferType, 3> bs{{
-                const_buffer{&s[0], x},
+                BufferType{&s[0], x},
-                const_buffer{&s[x], y},
+                BufferType{&s[x], y},
-                const_buffer{&s[x+y], z}}};
+                BufferType{&s[x+y], z}}};
             for(std::size_t i = 0; i <= s.size() + 1; ++i)
             {
                 auto pb = prepare_buffers(i, bs);
@@ -104,7 +104,8 @@ public:
 
     void run() override
     {
-        testBuffers();
+        testMatrix<boost::asio::const_buffer>();
+        testMatrix<boost::asio::mutable_buffer>();
         testNullBuffers();
         testIterator();
     }

test/http/basic_headers.cpp

@@ -30,6 +30,14 @@ public:
             h.insert(std::to_string(i), i);
     }
 
+    template<class U, class V>
+    static
+    void
+    self_assign(U& u, V&& v)
+    {
+        u = std::forward<V>(v);
+    }
+
     void testHeaders()
     {
         bh h1;
@@ -47,12 +55,23 @@ public:
         bh h3(std::move(h1));
         expect(h3.size() == 2);
         expect(h1.size() == 0);
-        h2 = std::move(h2);
+        self_assign(h3, std::move(h3));
+        expect(h3.size() == 2);
+        expect(h2.erase("Not-Present") == 0);
+    }
+
+    void testRFC2616()
+    {
+        bh h;
+        h.insert("a", "x");
+        h.insert("a", "y");
+        expect(h["a"] == "x,y");
     }
 
     void run() override
     {
         testHeaders();
+        testRFC2616();
     }
 };
 

test/http/basic_parser_v1.cpp

@@ -49,6 +49,7 @@ public:
             cb_req_checker, cb_res_checker>::type
 
     {
+        bool start = false;
         bool field = false;
         bool value = false;
         bool headers = false;
@@ -58,6 +59,10 @@ public:
     private:
         friend class basic_parser_v1<isRequest, cb_checker<isRequest>>;
 
+        void on_start(error_code&)
+        {
+            this->start = true;
+        }
         void on_method(boost::string_ref const&, error_code&)
         {
             this->method = true;
@@ -101,68 +106,6 @@ public:
         }
     };
 
-    template<bool isRequest>
-    struct cb_fail
-        : public basic_parser_v1<isRequest, cb_fail<isRequest>>
-
-    {
-        std::size_t n_;
-
-        void fail(error_code& ec)
-        {
-            if(n_ > 0)
-                --n_;
-            if(! n_)
-                ec = boost::system::errc::make_error_code(
-                    boost::system::errc::invalid_argument);
-        }
-
-    private:
-        friend class basic_parser_v1<isRequest, cb_checker<isRequest>>;
-
-        void on_method(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_uri(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_reason(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_request(error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_response(error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_field(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_value(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        int on_headers(error_code& ec)
-        {
-            fail(ec);
-            return 0;
-        }
-        void on_body(boost::string_ref const&, error_code& ec)
-        {
-            fail(ec);
-        }
-        void on_complete(error_code& ec)
-        {
-            fail(ec);
-        }
-    };
-
     //--------------------------------------------------------------------------
 
     static
@@ -238,99 +181,6 @@ public:
         }
     };
 
-    void
-    testFail()
-    {
-        using boost::asio::buffer;
-        {
-            std::string const s =
-                "GET / HTTP/1.1\r\n"
-                "User-Agent: test\r\n"
-                "Content-Length: 1\r\n"
-                "\r\n"
-                "*";
-            static std::size_t constexpr limit = 100;
-            std::size_t n = 1;
-            for(; n < limit; ++n)
-            {
-                error_code ec;
-                basic_parser_v1<true, cb_fail<true>> p;
-                p.write(buffer(s), ec);
-                if(! ec)
-                    break;
-            }
-            expect(n < limit);
-        }
-        {
-            std::string const s =
-                "HTTP/1.1 200 OK\r\n"
-                "Server: test\r\n"
-                "Content-Length: 1\r\n"
-                "\r\n"
-                "*";
-            static std::size_t constexpr limit = 100;
-            std::size_t n = 1;
-            for(; n < limit; ++n)
-            {
-                error_code ec;
-                basic_parser_v1<false, cb_fail<false>> p;
-                p.write(buffer(s), ec);
-                if(! ec)
-                    break;
-            }
-            expect(n < limit);
-        }
-    }
-
-    void
-    testCallbacks()
-    {
-        using boost::asio::buffer;
-        {
-            cb_checker<true> p;
-            error_code ec;
-            std::string const s =
-                "GET / HTTP/1.1\r\n"
-                "User-Agent: test\r\n"
-                "Content-Length: 1\r\n"
-                "\r\n"
-                "*";
-            p.write(buffer(s), ec);
-            if( expect(! ec))
-            {
-                expect(p.method);
-                expect(p.uri);
-                expect(p.request);
-                expect(p.field);
-                expect(p.value);
-                expect(p.headers);
-                expect(p.body);
-                expect(p.complete);
-            }
-        }
-        {
-            cb_checker<false> p;
-            error_code ec;
-            std::string const s =
-                "HTTP/1.1 200 OK\r\n"
-                "Server: test\r\n"
-                "Content-Length: 1\r\n"
-                "\r\n"
-                "*";
-            p.write(buffer(s), ec);
-            if( expect(! ec))
-            {
-                expect(p.reason);
-                expect(p.response);
-                expect(p.field);
-                expect(p.value);
-                expect(p.headers);
-                expect(p.body);
-                expect(p.complete);
-            }
-        }
-    }
-
     // Parse the entire input buffer as a valid message,
     // then parse in two pieces of all possible lengths.
     //
@@ -339,13 +189,22 @@ public:
     parse(boost::string_ref const& m, F&& f)
     {
         using boost::asio::buffer;
+        for(;;)
         {
             error_code ec;
             Parser p;
             p.write(buffer(m.data(), m.size()), ec);
+            if(! expect(! ec, ec.message()))
+                break;
+            if(p.needs_eof())
+            {
+                p.write_eof(ec);
+                if(! expect(! ec, ec.message()))
+                    break;
+            }
             if(expect(p.complete()))
-                if(expect(! ec, ec.message()))
                 f(p);
+            break;
         }
         for(std::size_t i = 1; i < m.size() - 1; ++i)
         {
@@ -354,18 +213,21 @@ public:
             p.write(buffer(&m[0], i), ec);
             if(! expect(! ec, ec.message()))
                 continue;
-            if(p.complete())
+            if(! p.complete())
-            {
-                f(p);
-            }
-            else
             {
                 p.write(buffer(&m[i], m.size() - i), ec);
                 if(! expect(! ec, ec.message()))
                     continue;
-                expect(p.complete());
-                f(p);
             }
+            if(! p.complete() && p.needs_eof())
+            {
+                p.write_eof(ec);
+                if(! expect(! ec, ec.message()))
+                    continue;
+            }
+            if(! expect(p.complete()))
+                continue;
+            f(p);
         }
     }
 
@@ -403,8 +265,6 @@ public:
         }
     }
 
-    //--------------------------------------------------------------------------
-
     // Parse a valid message with expected version
     //
     template<bool isRequest>
@@ -432,24 +292,89 @@ public:
             });
     }
 
+    //--------------------------------------------------------------------------
+
+    // Check all callbacks invoked
+    void
+    testCallbacks()
+    {
+        using boost::asio::buffer;
+        {
+            cb_checker<true> p;
+            error_code ec;
+            std::string const s =
+                "GET / HTTP/1.1\r\n"
+                "User-Agent: test\r\n"
+                "Content-Length: 1\r\n"
+                "\r\n"
+                "*";
+            p.write(buffer(s), ec);
+            if(expect(! ec))
+            {
+                expect(p.start);
+                expect(p.method);
+                expect(p.uri);
+                expect(p.request);
+                expect(p.field);
+                expect(p.value);
+                expect(p.headers);
+                expect(p.body);
+                expect(p.complete);
+            }
+        }
+        {
+            cb_checker<false> p;
+            error_code ec;
+            std::string const s =
+                "HTTP/1.1 200 OK\r\n"
+                "Server: test\r\n"
+                "Content-Length: 1\r\n"
+                "\r\n"
+                "*";
+            p.write(buffer(s), ec);
+            if(expect(! ec))
+            {
+                expect(p.start);
+                expect(p.reason);
+                expect(p.response);
+                expect(p.field);
+                expect(p.value);
+                expect(p.headers);
+                expect(p.body);
+                expect(p.complete);
+            }
+        }
+    }
+
     void
     testVersion()
     {
-        version<true>("GET / HTTP/0.0\r\n\r\n", 0, 0);
+        version <true>("GET / HTTP/0.0\r\n\r\n", 0, 0);
-        version<true>("GET / HTTP/0.1\r\n\r\n", 0, 1);
+        version <true>("GET / HTTP/0.1\r\n\r\n", 0, 1);
-        version<true>("GET / HTTP/0.9\r\n\r\n", 0, 9);
+        version <true>("GET / HTTP/0.9\r\n\r\n", 0, 9);
-        version<true>("GET / HTTP/1.0\r\n\r\n", 1, 0);
+        version <true>("GET / HTTP/1.0\r\n\r\n", 1, 0);
-        version<true>("GET / HTTP/1.1\r\n\r\n", 1, 1);
+        version <true>("GET / HTTP/1.1\r\n\r\n", 1, 1);
-        version<true>("GET / HTTP/9.9\r\n\r\n", 9, 9);
+        version <true>("GET / HTTP/9.9\r\n\r\n", 9, 9);
-        version<true>("GET / HTTP/999.999\r\n\r\n", 999, 999);
+        version <true>("GET / HTTP/999.999\r\n\r\n", 999, 999);
         parse_ev<true>("GET / HTTP/1000.0\r\n\r\n", parse_error::bad_version);
         parse_ev<true>("GET / HTTP/0.1000\r\n\r\n", parse_error::bad_version);
         parse_ev<true>("GET / HTTP/99999999999999999999.0\r\n\r\n", parse_error::bad_version);
         parse_ev<true>("GET / HTTP/0.99999999999999999999\r\n\r\n", parse_error::bad_version);
+
+        version <false>("HTTP/0.0 200 OK\r\n\r\n", 0, 0);
+        version <false>("HTTP/0.1 200 OK\r\n\r\n", 0, 1);
+        version <false>("HTTP/0.9 200 OK\r\n\r\n", 0, 9);
+        version <false>("HTTP/1.0 200 OK\r\n\r\n", 1, 0);
+        version <false>("HTTP/1.1 200 OK\r\n\r\n", 1, 1);
+        version <false>("HTTP/9.9 200 OK\r\n\r\n", 9, 9);
+        version <false>("HTTP/999.999 200 OK\r\n\r\n", 999, 999);
+        parse_ev<false>("HTTP/1000.0 200 OK\r\n\r\n", parse_error::bad_version);
+        parse_ev<false>("HTTP/0.1000 200 OK\r\n\r\n", parse_error::bad_version);
+        parse_ev<false>("HTTP/99999999999999999999.0 200 OK\r\n\r\n", parse_error::bad_version);
+        parse_ev<false>("HTTP/0.99999999999999999999 200 OK\r\n\r\n", parse_error::bad_version);
     }
 
-    void
+    void testConnection(std::string const& token,
-    testConnection(std::string const& token,
         std::uint8_t flag)
     {
         checkf("GET / HTTP/1.1\r\nConnection:" + token + "\r\n\r\n", flag);
@@ -472,8 +397,7 @@ public:
         checkf("GET / HTTP/1.1\r\nConnection: X," + token + "\t\r\n\r\n", flag);
     }
 
-    void
+    void testContentLength()
-    testContentLength()
     {
         std::size_t const length = 0;
         std::string const length_s =
@@ -493,8 +417,7 @@ public:
         checkf("GET / HTTP/1.1\r\nContent-Length:\t\r\n" "\t"+ length_s + "\r\n\r\n", parse_flag::contentlength);
     }
 
-    void
+    void testTransferEncoding()
-    testTransferEncoding()
     {
         checkf("GET / HTTP/1.1\r\nTransfer-Encoding:chunked\r\n\r\n0\r\n\r\n", parse_flag::chunked);
         checkf("GET / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n0\r\n\r\n", parse_flag::chunked);
@@ -510,17 +433,15 @@ public:
         checkf("GET / HTTP/1.1\r\nTransfer-Encoding:\t\r\n" "\tchunked\r\n\r\n0\r\n\r\n", parse_flag::chunked );
     }
 
-    void
+    void testFlags()
-    testFlags()
     {
-        testConnection("keep-alive",
+        testConnection("keep-alive", parse_flag::connection_keep_alive);
-            parse_flag::connection_keep_alive);
+        testConnection("close", parse_flag::connection_close);
+        testConnection("upgrade", parse_flag::connection_upgrade);
 
-        testConnection("close",
+        checkf("GET / HTTP/1.1\r\nConnection: close, win\r\n\r\n", parse_flag::connection_close);
-            parse_flag::connection_close);
+        checkf("GET / HTTP/1.1\r\nConnection: keep-alive, win\r\n\r\n", parse_flag::connection_keep_alive);
-
+        checkf("GET / HTTP/1.1\r\nConnection: upgrade, win\r\n\r\n", parse_flag::connection_upgrade);
-        testConnection("upgrade",
-            parse_flag::connection_upgrade);
 
         testContentLength();
 
@@ -537,11 +458,46 @@ public:
             "GET / HTTP/1.1\r\n"
             "Transfer-Encoding:chunked\r\n"
             "Content-Length: 0\r\n"
+            "Proxy-Connection: close\r\n"
             "\r\n", parse_error::illegal_content_length);
     }
 
-    void
+    void testHeaders()
-    testUpgrade()
+    {
+        parse<null_parser<true>>(
+            "GET / HTTP/1.0\r\n"
+            "Conniving: yes\r\n"
+            "Content-Lengthening: yes\r\n"
+            "Transfer-Encoding: deflate\r\n"
+            "Connection: sweep\r\n"
+            "\r\n",
+            [](null_parser<true> const&)
+            {
+            });
+        
+        parse_ev<true>(
+            "GET / HTTP/1.0\r\n"
+            "Content-Length: 1\r\n"
+            "Content-Length: 2\r\n"
+            "\r\n",
+                parse_error::bad_content_length);
+        
+        parse_ev<true>(
+            "GET / HTTP/1.0\r\n"
+            "Transfer-Encoding: chunked\r\n"
+            "\r\n"
+            "fffffffffffffffff\r\n"
+            "0\r\n\r\n",
+                parse_error::bad_content_length);
+        
+        parse_ev<true>("GET / HTTP/1.0\r\nContent-Length: 1e9\r\n\r\n",
+            parse_error::bad_content_length);
+
+        parse_ev<true>("GET / HTTP/1.0\r\nContent-Length: 99999999999999999999999\r\n\r\n",
+            parse_error::bad_content_length);
+    }
+
+    void testUpgrade()
     {
         using boost::asio::buffer;
         null_parser<true> p;
@@ -576,11 +532,10 @@ public:
     void testInvalidMatrix()
     {
         using boost::asio::buffer;
-        static std::size_t constexpr limit = 200;
         std::string s;
         std::size_t n;
 
-        for(n = 0; n < limit; ++n)
+        for(n = 0;; ++n)
         {
             // Create a request and set one octet to an invalid char
             s =
@@ -590,11 +545,8 @@ public:
                 "Content-Length: 00\r\n"
                 "\r\n";
             auto const len = s.size();
-            if(n >= s.size())
+            if(n < len)
             {
-                pass();
-                break;
-            }
                 s[n] = 0;
                 for(std::size_t m = 1; m < len - 1; ++m)
                 {
@@ -610,29 +562,32 @@ public:
                     expect(ec);
                 }
             }
-        expect(n < limit);
+            else
+            {
+                null_parser<true> p;
+                error_code ec;
+                p.write(buffer(s.data(), s.size()), ec);
+                expect(! ec, ec.message());
+                break;
+            }
+        }
 
-        for(n = 0; n < limit; ++n)
+        for(n = 0;; ++n)
         {
             // Create a response and set one octet to an invalid char
             s =
                 "HTTP/1.1 200 OK\r\n"
                 "Server: test\r\n"
-                "Transer-Encoding: chunked\r\n"
+                "Transfer-Encoding: chunked\r\n"
                 "\r\n"
-                "10\r\n"
-                "****************\r\n"
                 "0\r\n\r\n";
             auto const len = s.size();
-            if(n >= s.size())
+            if(n < len)
             {
-                pass();
-                break;
-            }
                 s[n] = 0;
                 for(std::size_t m = 1; m < len - 1; ++m)
                 {
-                null_parser<true> p;
+                    null_parser<false> p;
                     error_code ec;
                     p.write(buffer(s.data(), m), ec);
                     if(ec)
@@ -644,7 +599,15 @@ public:
                     expect(ec);
                 }
             }
-        expect(n < limit);
+            else
+            {
+                null_parser<false> p;
+                error_code ec;
+                p.write(buffer(s.data(), s.size()), ec);
+                expect(! ec, ec.message());
+                break;
+            }
+        }
     }
 
     void
@@ -754,24 +717,30 @@ public:
                         expect(p.body == body);
                     };
             };
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nContent-Length: 1\r\n\r\n123", match("1"));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nContent-Length: 3\r\n\r\n123", match("123"));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nContent-Length: 0\r\n\r\n", match(""));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n"
             "1\r\n"
             "a\r\n"
             "0\r\n"
             "\r\n", match("a"));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n"
             "2\r\n"
             "ab\r\n"
             "0\r\n"
             "\r\n", match("ab"));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n"
             "2\r\n"
@@ -780,6 +749,7 @@ public:
             "c\r\n"
             "0\r\n"
             "\r\n", match("abc"));
+
         parse<test_parser<true>>(
             "GET / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n"
             "10\r\n"
@@ -790,10 +760,10 @@ public:
 
     void run() override
     {
-        testFail();
         testCallbacks();
         testVersion();
         testFlags();
+        testHeaders();
         testUpgrade();
         testBad();
         testInvalidMatrix();

test/http/message_v1.cpp

@@ -9,126 +9,34 @@
 #include <beast/http/message_v1.hpp>
 
 #include <beast/unit_test/suite.hpp>
-#include <beast/unit_test/thread.hpp>
+#include <beast/http/empty_body.hpp>
-#include <beast/core/placeholders.hpp>
-#include <beast/core/streambuf.hpp>
-#include <beast/http.hpp>
-#include <boost/asio.hpp>
 
 namespace beast {
 namespace http {
 
-class sync_echo_http_server
-{
-public:
-    using endpoint_type = boost::asio::ip::tcp::endpoint;
-    using address_type = boost::asio::ip::address;
-    using socket_type = boost::asio::ip::tcp::socket;
-
-private:
-    unit_test::suite& suite_;
-    boost::asio::io_service ios_;
-    socket_type sock_;
-    boost::asio::ip::tcp::acceptor acceptor_;
-    unit_test::thread thread_;
-
-public:
-    sync_echo_http_server(
-            endpoint_type ep, unit_test::suite& suite)
-        : suite_(suite)
-        , sock_(ios_)
-        , acceptor_(ios_)
-    {
-        error_code ec;
-        acceptor_.open(ep.protocol(), ec);
-        maybe_throw(ec, "open");
-        acceptor_.bind(ep, ec);
-        maybe_throw(ec, "bind");
-        acceptor_.listen(
-            boost::asio::socket_base::max_connections, ec);
-        maybe_throw(ec, "listen");
-        acceptor_.async_accept(sock_,
-            std::bind(&sync_echo_http_server::on_accept, this,
-                beast::asio::placeholders::error));
-        thread_ = unit_test::thread(suite_,
-            [&]
-            {
-                ios_.run();
-            });
-    }
-
-    ~sync_echo_http_server()
-    {
-        error_code ec;
-        ios_.dispatch(
-            [&]{ acceptor_.close(ec); });
-        thread_.join();
-    }
-
-private:
-    void
-    fail(error_code ec, std::string what)
-    {
-        suite_.log <<
-            what << ": " << ec.message();
-    }
-
-    void
-    maybe_throw(error_code ec, std::string what)
-    {
-        if(ec &&
-            ec != boost::asio::error::operation_aborted)
-        {
-            fail(ec, what);
-            throw ec;
-        }
-    }
-
-    void
-    on_accept(error_code ec)
-    {
-        if(ec == boost::asio::error::operation_aborted)
-            return;
-        maybe_throw(ec, "accept");
-        std::thread{&sync_echo_http_server::do_client, this,
-            std::move(sock_), boost::asio::io_service::work{
-                sock_.get_io_service()}}.detach();
-        acceptor_.async_accept(sock_,
-            std::bind(&sync_echo_http_server::on_accept, this,
-                beast::asio::placeholders::error));
-    }
-
-    void
-    do_client(socket_type sock, boost::asio::io_service::work)
-    {
-        error_code ec;
-        streambuf rb;
-        for(;;)
-        {
-            request_v1<string_body> req;
-            read(sock, rb, req, ec);
-            if(ec)
-                break;
-            response_v1<string_body> resp;
-            resp.status = 100;
-            resp.reason = "OK";
-            resp.version = req.version;
-            resp.body = "Completed successfully.";
-            write(sock, resp, ec);
-            if(ec)
-                break;
-        }
-    }
-};
-
 class message_v1_test : public beast::unit_test::suite
 {
 public:
-    using endpoint_type = boost::asio::ip::tcp::endpoint;
+    void testFreeFunctions()
-    using address_type = boost::asio::ip::address;
+    {
-    using socket_type = boost::asio::ip::tcp::socket;
+        {
+            request_v1<empty_body> m;
+            m.method = "GET";
+            m.url = "/";
+            m.version = 11;
+            m.headers.insert("Upgrade", "test");
+            expect(! is_upgrade(m));
 
-    void testFunctions()
+            prepare(m, connection::upgrade);
+            expect(is_upgrade(m));
+            expect(m.headers["Connection"] == "upgrade");
+
+            m.version = 10;
+            expect(! is_upgrade(m));
+        }
+    }
+
+    void testPrepare()
     {
         request_v1<empty_body> m;
         m.version = 10;
@@ -164,45 +72,10 @@ public:
         }
     }
 
-    void
-    syncEcho(endpoint_type ep)
-    {
-        boost::asio::io_service ios;
-        socket_type sock(ios);
-        sock.connect(ep);
-
-        streambuf rb;
-        {
-            request_v1<string_body> req;
-            req.method = "GET";
-            req.url = "/";
-            req.version = 11;
-            req.body = "Beast.HTTP";
-            req.headers.replace("Host",
-                ep.address().to_string() + ":" +
-                    std::to_string(ep.port()));
-            write(sock, req);
-        }
-        {
-            response_v1<string_body> m;
-            read(sock, rb, m);
-        }
-    }
-
-    void
-    testAsio()
-    {
-        endpoint_type ep{
-            address_type::from_string("127.0.0.1"), 6000};
-        sync_echo_http_server s(ep, *this);
-        syncEcho(ep);
-    }
-
     void run() override
     {
-        testFunctions();
+        testFreeFunctions();
-        testAsio();
+        testPrepare();
-        pass();
     }
 };
 

test/http/read.cpp

@@ -8,6 +8,7 @@
 // Test that header file is self-contained.
 #include <beast/http/read.hpp>
 
+#include <beast/http/headers.hpp>
 #include <beast/http/streambuf_body.hpp>
 #include <beast/test/fail_stream.hpp>
 #include <beast/test/string_stream.hpp>
@@ -23,6 +24,227 @@ class read_test
     , public test::enable_yield_to
 {
 public:
+    template<bool isRequest>
+    class fail_parser
+        : public basic_parser_v1<isRequest, fail_parser<isRequest>>
+    {
+        test::fail_counter& fc_;
+
+    public:
+        template<class... Args>
+        explicit
+        fail_parser(test::fail_counter& fc, Args&&... args)
+            : fc_(fc)
+        {
+        }
+
+        void on_start(error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_method(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_uri(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_reason(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_request(error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_response(error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_field(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_value(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        int on_headers(error_code& ec)
+        {
+            fc_.fail(ec);
+            return 0;
+        }
+        
+        void on_body(boost::string_ref const&, error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+        
+        void on_complete(error_code& ec)
+        {
+            fc_.fail(ec);
+        }
+    };
+
+    template<bool isRequest>
+    void failMatrix(const char* s, yield_context do_yield)
+    {
+        using boost::asio::buffer;
+        using boost::asio::buffer_copy;
+        static std::size_t constexpr limit = 100;
+        std::size_t n;
+        auto const len = strlen(s);
+        for(n = 0; n < limit; ++n)
+        {
+            streambuf sb;
+            sb.commit(buffer_copy(
+                sb.prepare(len), buffer(s, len)));
+            test::fail_counter fc(n);
+            test::fail_stream<
+                test::string_stream> fs{fc, ios_, ""};
+            fail_parser<isRequest> p(fc);
+            error_code ec;
+            parse(fs, sb, p, ec);
+            if(! ec)
+                break;
+        }
+        expect(n < limit);
+        for(n = 0; n < limit; ++n)
+        {
+            static std::size_t constexpr pre = 10;
+            streambuf sb;
+            sb.commit(buffer_copy(
+                sb.prepare(pre), buffer(s, pre)));
+            test::fail_counter fc(n);
+            test::fail_stream<test::string_stream> fs{
+                fc, ios_, std::string{s + pre, len - pre}};
+            fail_parser<isRequest> p(fc);
+            error_code ec;
+            parse(fs, sb, p, ec);
+            if(! ec)
+                break;
+        }
+        expect(n < limit);
+        for(n = 0; n < limit; ++n)
+        {
+            streambuf sb;
+            sb.commit(buffer_copy(
+                sb.prepare(len), buffer(s, len)));
+            test::fail_counter fc(n);
+            test::fail_stream<
+                test::string_stream> fs{fc, ios_, ""};
+            fail_parser<isRequest> p(fc);
+            error_code ec;
+            async_parse(fs, sb, p, do_yield[ec]);
+            if(! ec)
+                break;
+        }
+        expect(n < limit);
+        for(n = 0; n < limit; ++n)
+        {
+            static std::size_t constexpr pre = 10;
+            streambuf sb;
+            sb.commit(buffer_copy(
+                sb.prepare(pre), buffer(s, pre)));
+            test::fail_counter fc(n);
+            test::fail_stream<test::string_stream> fs{
+                fc, ios_, std::string{s + pre, len - pre}};
+            fail_parser<isRequest> p(fc);
+            error_code ec;
+            async_parse(fs, sb, p, do_yield[ec]);
+            if(! ec)
+                break;
+        }
+        expect(n < limit);
+    }
+
+    void testThrow()
+    {
+        try
+        {
+            streambuf sb;
+            test::string_stream ss(ios_, "GET / X");
+            parser_v1<true, streambuf_body, headers> p;
+            parse(ss, sb, p);
+            fail();
+        }
+        catch(std::exception const&)
+        {
+            pass();
+        }
+    }
+
+    void testFailures(yield_context do_yield)
+    {
+        char const* req[] = {
+            "GET / HTTP/1.0\r\n"
+            "Host: localhost\r\n"
+            "User-Agent: test\r\n"
+            "Empty:\r\n"
+            "\r\n"
+            ,
+            "GET / HTTP/1.1\r\n"
+            "Host: localhost\r\n"
+            "User-Agent: test\r\n"
+            "Content-Length: 2\r\n"
+            "\r\n"
+            "**"
+            ,
+            "GET / HTTP/1.1\r\n"
+            "Host: localhost\r\n"
+            "User-Agent: test\r\n"
+            "Transfer-Encoding: chunked\r\n"
+            "\r\n"
+            "10\r\n"
+            "****************\r\n"
+            "0\r\n\r\n"
+            ,
+            nullptr
+        };
+
+        char const* res[] = {
+            "HTTP/1.0 200 OK\r\n"
+            "Server: test\r\n"
+            "\r\n"
+            ,
+            "HTTP/1.0 200 OK\r\n"
+            "Server: test\r\n"
+            "\r\n"
+            "***"
+            ,
+            "HTTP/1.1 200 OK\r\n"
+            "Server: test\r\n"
+            "Content-Length: 3\r\n"
+            "\r\n"
+            "***"
+            ,
+            "HTTP/1.1 200 OK\r\n"
+            "Server: test\r\n"
+            "Transfer-Encoding: chunked\r\n"
+            "\r\n"
+            "10\r\n"
+            "****************\r\n"
+            "0\r\n\r\n"
+            ,
+            nullptr
+        };
+
+        for(std::size_t i = 0; req[i]; ++i)
+            failMatrix<true>(req[i], do_yield);
+
+        for(std::size_t i = 0; res[i]; ++i)
+            failMatrix<false>(res[i], do_yield);
+    }
+
     void testRead(yield_context do_yield)
     {
         static std::size_t constexpr limit = 100;
@@ -30,7 +252,6 @@ public:
 
         for(n = 0; n < limit; ++n)
         {
-            streambuf sb;
             test::fail_stream<test::string_stream> fs(n, ios_,
                 "GET / HTTP/1.1\r\n"
                 "Host: localhost\r\n"
@@ -41,6 +262,7 @@ public:
             request_v1<streambuf_body> m;
             try
             {
+                streambuf sb;
                 read(fs, sb, m);
                 break;
             }
@@ -52,7 +274,6 @@ public:
 
         for(n = 0; n < limit; ++n)
         {
-            streambuf sb;
             test::fail_stream<test::string_stream> fs(n, ios_,
                 "GET / HTTP/1.1\r\n"
                 "Host: localhost\r\n"
@@ -62,6 +283,7 @@ public:
             );
             request_v1<streambuf_body> m;
             error_code ec;
+            streambuf sb;
             read(fs, sb, m, ec);
             if(! ec)
                 break;
@@ -70,7 +292,6 @@ public:
 
         for(n = 0; n < limit; ++n)
         {
-            streambuf sb;
             test::fail_stream<test::string_stream> fs(n, ios_,
                 "GET / HTTP/1.1\r\n"
                 "Host: localhost\r\n"
@@ -80,6 +301,7 @@ public:
             );
             request_v1<streambuf_body> m;
             error_code ec;
+            streambuf sb;
             async_read(fs, sb, m, do_yield[ec]);
             if(! ec)
                 break;
@@ -87,10 +309,38 @@ public:
         expect(n < limit);
     }
 
+    void testEof(yield_context do_yield)
+    {
+        {
+            streambuf sb;
+            test::string_stream ss(ios_, "");
+            parser_v1<true, streambuf_body, headers> p;
+            error_code ec;
+            parse(ss, sb, p, ec);
+            expect(ec == boost::asio::error::eof);
+        }
+        {
+            streambuf sb;
+            test::string_stream ss(ios_, "");
+            parser_v1<true, streambuf_body, headers> p;
+            error_code ec;
+            async_parse(ss, sb, p, do_yield[ec]);
+            expect(ec == boost::asio::error::eof);
+        }
+    }
+
     void run() override
     {
+        testThrow();
+
+        yield_to(std::bind(&read_test::testFailures,
+            this, std::placeholders::_1));
+
         yield_to(std::bind(&read_test::testRead,
             this, std::placeholders::_1));
+
+        yield_to(std::bind(&read_test::testEof,
+            this, std::placeholders::_1));
     }
 };
 

test/http/streambuf_body.cpp

@@ -7,3 +7,38 @@
 
 // Test that header file is self-contained.
 #include <beast/http/streambuf_body.hpp>
+
+#include <beast/core/to_string.hpp>
+#include <beast/http/headers.hpp>
+#include <beast/http/parser_v1.hpp>
+#include <beast/http/read.hpp>
+#include <beast/test/string_stream.hpp>
+#include <beast/unit_test/suite.hpp>
+
+namespace beast {
+namespace http {
+
+class streambuf_body_test : public beast::unit_test::suite
+{
+    boost::asio::io_service ios_;
+
+public:
+    void run() override
+    {
+        test::string_stream ss(ios_,
+            "HTTP/1.1 200 OK\r\n"
+            "Server: test\r\n"
+            "Content-Length: 3\r\n"
+            "\r\n"
+            "xyz");
+        parser_v1<false, streambuf_body, headers> p;
+        streambuf sb;
+        parse(ss, sb, p);
+        expect(to_string(p.get().body.data()) == "xyz");
+    }
+};
+
+BEAST_DEFINE_TESTSUITE(streambuf_body,http,beast);
+
+} // http
+} // beast

test/websocket/CMakeLists.txt

@@ -14,9 +14,9 @@ add_executable (websocket-tests
     rfc6455.cpp
     stream.cpp
     teardown.cpp
-    utf8_checker.cpp
     detail/frame.cpp
     detail/mask.cpp
+    detail/utf8_checker.cpp
 )
 
 if (NOT WIN32)

test/websocket/detail/frame.cpp

@@ -32,7 +32,7 @@ operator==(frame_header const& lhs, frame_header const& rhs)
 class frame_test : public beast::unit_test::suite
 {
 public:
-    void testValidOpcode()
+    void testCloseCodes()
     {
         expect(! is_valid(0));
         expect(! is_valid(1));
@@ -43,6 +43,8 @@ public:
         expect(! is_valid(1016));
         expect(! is_valid(2000));
         expect(! is_valid(2999));
+        expect(is_valid(1000));
+        expect(is_valid(1002));
         expect(is_valid(3000));
         expect(is_valid(4000));
         expect(is_valid(5000));
@@ -220,7 +222,7 @@ public:
 
     void run() override
     {
-        testValidOpcode();
+        testCloseCodes();
         testFrameHeader();
         testBadFrameHeaders();
         pass();

test/websocket/detail/utf8_checker.cpp

@@ -8,12 +8,14 @@
 // Test that header file is self-contained.
 #include <beast/websocket/detail/utf8_checker.hpp>
 
+#include <beast/core/consuming_buffers.hpp>
 #include <beast/core/streambuf.hpp>
 #include <beast/unit_test/suite.hpp>
 #include <array>
 
 namespace beast {
 namespace websocket {
+namespace detail {
 
 class utf8_checker_test : public beast::unit_test::suite
 {
@@ -21,7 +23,7 @@ public:
     void
     testOneByteSequence()
     {
-        detail::utf8_checker utf8;
+        utf8_checker utf8;
         std::array<std::uint8_t, 256> const buf =
             ([]()
             {
@@ -50,7 +52,7 @@ public:
     void
     testTwoByteSequence()
     {
-        detail::utf8_checker utf8;
+        utf8_checker utf8;
         std::uint8_t buf[2];
         for(auto i = 194; i <= 223; ++i)
         {
@@ -84,7 +86,7 @@ public:
     void
     testThreeByteSequence()
     {
-        detail::utf8_checker utf8;
+        utf8_checker utf8;
         std::uint8_t buf[3];
         for (auto i = 224; i <= 239; ++i)
         {
@@ -140,7 +142,8 @@ public:
     void
     testFourByteSequence()
     {
-        detail::utf8_checker utf8;
+        using boost::asio::const_buffers_1;
+        utf8_checker utf8;
         std::uint8_t buf[4];
         for (auto i = 240; i <= 244; ++i)
         {
@@ -163,7 +166,7 @@ public:
                     {
                         // Fourth byte valid range 128-191
                         buf[3] = static_cast<std::uint8_t>(n);
-                        expect(utf8.write(buf, 4));
+                        expect(utf8.write(const_buffers_1{buf, 4}));
                         expect(utf8.finish());
                     }
 
@@ -171,7 +174,7 @@ public:
                     {
                         // Fourth byte invalid range 0-127
                         buf[3] = static_cast<std::uint8_t>(n);
-                        expect(! utf8.write(buf, 4));
+                        expect(! utf8.write(const_buffers_1{buf, 4}));
                     }
 
                     for (auto n = 192; n <= 255; ++n)
@@ -217,12 +220,14 @@ public:
     testWithStreamBuffer()
     {
         using namespace boost::asio;
+        {
             // Valid UTF8 encoded text
-        std::vector<std::vector<std::uint8_t>> const data{
+            std::vector<std::vector<std::uint8_t>> const data{{
-            {0x48,0x65,0x69,0x7A,0xC3,0xB6,0x6C,0x72,0xC3,0xBC,0x63,0x6B,
+                    0x48,0x65,0x69,0x7A,0xC3,0xB6,0x6C,0x72,0xC3,0xBC,0x63,0x6B,
                     0x73,0x74,0x6F,0xC3,0x9F,0x61,0x62,0x64,0xC3,0xA4,0x6D,0x70,
-            0x66,0x75,0x6E,0x67},
+                    0x66,0x75,0x6E,0x67
-            {0xCE,0x93,0xCE,0xB1,0xCE,0xB6,0xCE,0xAD,0xCE,0xB5,0xCF,0x82,
+                }, {
+                    0xCE,0x93,0xCE,0xB1,0xCE,0xB6,0xCE,0xAD,0xCE,0xB5,0xCF,0x82,
                     0x20,0xCE,0xBA,0xCE,0xB1,0xE1,0xBD,0xB6,0x20,0xCE,0xBC,0xCF,
                     0x85,0xCF,0x81,0xCF,0x84,0xCE,0xB9,0xE1,0xBD,0xB2,0xCF,0x82,
                     0x20,0xCE,0xB4,0xE1,0xBD,0xB2,0xCE,0xBD,0x20,0xCE,0xB8,0xE1,
@@ -230,23 +235,34 @@ public:
                     0x80,0xCE,0xB9,0xE1,0xBD,0xB0,0x20,0xCF,0x83,0xCF,0x84,0xE1,
                     0xBD,0xB8,0x20,0xCF,0x87,0xCF,0x81,0xCF,0x85,0xCF,0x83,0xCE,
                     0xB1,0xCF,0x86,0xE1,0xBD,0xB6,0x20,0xCE,0xBE,0xCE,0xAD,0xCF,
-            0x86,0xCF,0x89,0xCF,0x84,0xCE,0xBF},
+                    0x86,0xCF,0x89,0xCF,0x84,0xCE,0xBF
-            {0xC3,0x81,0x72,0x76,0xC3,0xAD,0x7A,0x74,0xC5,0xB1,0x72,0xC5,
+                }, {
+                    0xC3,0x81,0x72,0x76,0xC3,0xAD,0x7A,0x74,0xC5,0xB1,0x72,0xC5,
                     0x91,0x20,0x74,0xC3,0xBC,0x6B,0xC3,0xB6,0x72,0x66,0xC3,0xBA,
-            0x72,0xC3,0xB3,0x67,0xC3,0xA9,0x70}
+                    0x72,0xC3,0xB3,0x67,0xC3,0xA9,0x70
+                }
             };
-        detail::utf8_checker utf8;
+            utf8_checker utf8;
             for(auto const& s : data)
             {
-            beast::streambuf sb(
+                static std::size_t constexpr size = 8;
-                s.size() / 4); // Force split across blocks
+                std::size_t n = s.size();
-            sb.commit(buffer_copy(
+                auto cb = consumed_buffers(
-                sb.prepare(s.size()),
+                    boost::asio::const_buffers_1(
-                    const_buffer(s.data(), s.size())));
+                        s.data(), n), 0);
+                streambuf sb(size);
+                while(n)
+                {
+                    auto const amount = std::min(n, size);
+                    sb.commit(buffer_copy(sb.prepare(amount), cb));
+                    cb.consume(amount);
+                    n -= amount;
+                }
                 expect(utf8.write(sb.data()));
                 expect(utf8.finish());
             }
         }
+    }
 
     void run() override
     {
@@ -260,5 +276,6 @@ public:
 
 BEAST_DEFINE_TESTSUITE(utf8_checker,websocket,beast);
 
+} // detail
 } // websocket
 } // beast

test/websocket/stream.cpp

@@ -477,6 +477,88 @@ public:
         }
     }
 
+    struct con
+    {
+        stream<socket_type> ws;
+
+        explicit
+        con(endpoint_type const& ep, boost::asio::io_service& ios)
+            : ws(ios)
+        {
+            ws.next_layer().connect(ep);
+            ws.handshake("localhost", "/");
+        }
+    };
+
+    template<std::size_t N>
+    class cbuf_helper
+    {
+        std::array<std::uint8_t, N> v_;
+        boost::asio::const_buffer cb_;
+
+    public:
+        using value_type = decltype(cb_);
+        using const_iterator = value_type const*;
+
+        template<class... Vn>
+        explicit
+        cbuf_helper(Vn... vn)
+            : v_({{ static_cast<std::uint8_t>(vn)... }})
+            , cb_(v_.data(), v_.size())
+        {
+        }
+
+        const_iterator
+        begin() const
+        {
+            return &cb_;
+        }
+
+        const_iterator
+        end() const
+        {
+            return begin()+1;
+        }
+    };
+
+    template<class... Vn>
+    cbuf_helper<sizeof...(Vn)>
+    cbuf(Vn... vn)
+    {
+        return cbuf_helper<sizeof...(Vn)>(vn...);
+    }
+
+    void testClose(endpoint_type const& ep, yield_context do_yield)
+    {
+        using boost::asio::buffer;
+        {
+            // payload length 1
+            con c(ep, ios_);
+            boost::asio::write(c.ws.next_layer(),
+                cbuf(0x88, 0x81, 0xff, 0xff, 0xff, 0xff, 0x00));
+        }
+        {
+            // invalid close code 1005
+            con c(ep, ios_);
+            boost::asio::write(c.ws.next_layer(),
+                cbuf(0x88, 0x82, 0xff, 0xff, 0xff, 0xff, 0xfc, 0x12));
+        }
+        {
+            // invalid utf8
+            con c(ep, ios_);
+            boost::asio::write(c.ws.next_layer(),
+                cbuf(0x88, 0x86, 0xff, 0xff, 0xff, 0xff, 0xfc, 0x15,
+                    0x0f, 0xd7, 0x73, 0x43));
+        }
+        {
+            // good utf8
+            con c(ep, ios_);
+            boost::asio::write(c.ws.next_layer(),
+                cbuf(0x88, 0x86, 0xff, 0xff, 0xff, 0xff, 0xfc, 0x15,
+                    'u', 't', 'f', '8'));
+        }
+    }
+
     void testWriteFrame(endpoint_type const& ep)
     {
         for(;;)
@@ -528,6 +610,9 @@ public:
             yield_to(std::bind(&stream_test::testMask,
                 this, ep, std::placeholders::_1));
 
+            yield_to(std::bind(&stream_test::testClose,
+                this, ep, std::placeholders::_1));
+
             testWriteFrame(ep);
         }
         {
@@ -542,6 +627,9 @@ public:
 
             yield_to(std::bind(&stream_test::testMask,
                 this, ep, std::placeholders::_1));
+
+            yield_to(std::bind(&stream_test::testClose,
+                this, ep, std::placeholders::_1));
         }
 
         pass();