Documentation work

doc/3_3_streams.qbk

@@ -84,6 +84,25 @@ code indicating the result is printed:
         });
 ```
 
+If a read stream algorithm cannot complete its operation without exceeding
+the maximum specified size of the dynamic buffer provided, the error
+[link beast.ref.http__error `buffer_overflow`]
+is returned. This may be used to impose a limit on the maximum size of an
+HTTP message header for protection from buffer overflow attacks. The following
+code will generate an error:
+```
+    // This buffer is too small for much of anything
+    flat_buffer buffer{10};
+
+    // Try to read a request
+    error_code ec;
+    request<string_body> req;
+    read(sock, buffer, req, ec);
+    if(ec == error::buffer_overflow)
+        std::cerr << "Buffer limit exceeded!" << std::endl;
+
+```
+
 [heading Writing]
 
 A set of free functions allow serialization of an entire HTTP message to

doc/3_6_serializer_buffers.qbk

@@ -10,8 +10,7 @@
 In extreme cases, users may wish to create an instance of __serializer__
 and invoke its methods directly instead of using the provided stream
 algorithms. This could be useful for implementing algorithms on streams
-whose asynchronous interface does not conform to __AsyncStream__. For
+whose interface does not conform to __Stream__. For example, a
-example, a
 [@https://github.com/libuv/libuv *libuv* socket].
 
 The serializer interface is interactive; the caller invokes it repeatedly to
@@ -30,8 +29,8 @@ takes an error code parameter and invokes a visitor argument with the
 error code and buffer of unspecified type. In C++14 this is easily
 expressed with a generic lambda. The function
 [link beast.ref.http__serializer.is_done `serializer::is_done`]
-will return `true` when all the buffers have been produced. This example
+will return `true` when all the buffers have been produced. This C++14
-prints the buffers to standard output:
+example prints the buffers to standard output:
 ```
 template<bool isRequest, class Body, class Fields>
 void print(message<isRequest, Body, Fields> const& m)
@@ -55,8 +54,8 @@ void print(message<isRequest, Body, Fields> const& m)
 }
 ```
 
-Generic lambda expressions are not available in C++11, so a functor with
+Generic lambda expressions are only available in C++14 or later. A functor
-a templated function call operator is necessary:
+with a templated function call operator is necessary to use C++11 as shown:
 ```
 template<class Serializer>
 struct lambda

doc/3_7_parser_buffers.qbk

@@ -7,4 +7,130 @@
 
 [section:parser_buffers Buffer-Oriented Parsing]
 
+In extreme cases, users may wish to create an instance of __message_parser__,
+__header_parser__, or a user-defined type derived from __basic_parser__ and
+invoke its methods directly instead of using the provided stream algorithms.
+This could be useful for implementing algorithms on streams whose interface
+does not conform to any __Stream__. For example, a
+[@http://zeromq.org/ *ZeroMQ* socket].
+
+The basic parser interface is interactive; the caller invokes the function
+[link beast.ref.http__basic_parser.put `basic_parser::put`]
+repeatedly with buffers until an error occurs or the parsing is done. The
+function
+[link beast.ref.http__basic_parser.put_eof `basic_parser::put_eof`]
+Is used when the caller knows that there will never be more data (for example,
+if the underlying connection is closed), 
+
+[heading Split Parsing]
+
+[heading Eager Parsing]
+
+[heading Example: Parsing from a std::istream]
+
+The standard library provides the type `std::istream` for performing high
+level operations on character streams. The variable `std::cin` is based
+on this input stream. In this example, we build a stream operation which
+parses an HTTP message from a `std::istream`:
+```
+/** Parse an HTTP/1 message from a `std::istream`.
+
+    This function attempts to parse a complete message from the stream.
+
+    @param is The `std::istream` to read from.
+
+    @param buffer The buffer to use.
+
+    @param msg The message to store the result.
+
+    @param ec Set to the error, if any occurred.
+*/
+template<
+    class Allocator,
+    bool isRequest,
+    class Body,
+    class Fields>
+void
+parse_istream(
+    std::istream& is,
+    basic_flat_buffer<Allocator>& buffer,
+    message<isRequest, Body, Fields>& msg,
+    error_code& ec)
+{
+    // Create the message parser
+    message_parser<isRequest, Body, Fields> parser;
+
+    do
+    {
+        // Extract whatever characters are presently available in the istream
+        if(is.rdbuf()->in_avail() > 0)
+        {
+            // Get a mutable buffer sequence for writing
+            auto const mb = buffer.prepare(is.rdbuf()->in_avail());
+
+            // Now get everything we can from the istream
+            buffer.commit(is.readsome(
+                boost::asio::buffer_cast<char*>(mb),
+                boost::asio::buffer_size(mb)));
+        }
+        else if(buffer.size() == 0)
+        {
+            // Our buffer is empty and we need more characters, 
+            // see if we've reached the end of file on the istream
+            if(! is.eof())
+            {
+                // Get a mutable buffer sequence for writing
+                auto const mb = buffer.prepare(1024);
+
+                // Try to get more from the istream. This might block.
+                is.read(
+                    boost::asio::buffer_cast<char*>(mb),
+                    boost::asio::buffer_size(mb));
+
+                // If an error occurs on the istream then return it to the caller.
+                if(is.fail() && ! is.eof())
+                {
+                    // We'll just re-use io_error since std::istream has no error_code interface.
+                    ec = make_error_code(errc::io_error);
+                    return;
+                }
+
+                // Commit the characters we got to the buffer.
+                buffer.commit(is.gcount());
+            }
+            else
+            {
+                // Inform the parser that we've reached the end of the istream.
+                parser.put_eof(ec);
+                if(ec)
+                    return;
+                break;
+            }
+        }
+
+        // Write the data to the parser
+        auto const bytes_used = parser.put(buffer.data(), ec);
+
+        // This error means that the parser needs additional octets.
+        if(ec == error::need_more)
+            ec = {};
+        if(ec)
+            return;
+
+        // Consume the buffer octets that were actually parsed.
+        buffer.consume(bytes_used);
+    }
+    while(! parser.is_done());
+
+    // Transfer ownership of the message container in the parser to the caller.
+    msg = parser.release();
+}
+```
+[tip
+    Parsing from a `std::istream` could be implemented using an alternate
+    strategy: adapt the `std::istream` interface to a __SyncReadStream__.
+    This would allow it to work with the library's existing algorithms.
+    We leave this as an exercise for the reader.
+]
+
 [endsect]

test/http/design.cpp

@@ -16,6 +16,7 @@
 #include <beast/unit_test/suite.hpp>
 #include <boost/asio/read.hpp>
 #include <boost/asio/write.hpp>
+#include <sstream>
 
 namespace beast {
 namespace http {
@@ -502,6 +503,121 @@ public:
             upstream.server.str(), req.body));
     }
 
+    //--------------------------------------------------------------------------
+    //
+    // Example: Parse from std::istream
+    //
+    //--------------------------------------------------------------------------
+
+    /** Parse an HTTP/1 message from a `std::istream`.
+
+        This function attempts to parse a complete message from the stream.
+
+        @param is The `std::istream` to read from.
+
+        @param buffer The buffer to use.
+
+        @param msg The message to store the result.
+
+        @param ec Set to the error, if any occurred.
+    */
+    template<
+        class Allocator,
+        bool isRequest,
+        class Body,
+        class Fields>
+    void
+    parse_istream(
+        std::istream& is,
+        basic_flat_buffer<Allocator>& buffer,
+        message<isRequest, Body, Fields>& msg,
+        error_code& ec)
+    {
+        // Create the message parser
+        message_parser<isRequest, Body, Fields> parser;
+
+        do
+        {
+            // Extract whatever characters are presently available in the istream
+            if(is.rdbuf()->in_avail() > 0)
+            {
+                // Get a mutable buffer sequence for writing
+                auto const mb = buffer.prepare(is.rdbuf()->in_avail());
+
+                // Now get everything we can from the istream
+                buffer.commit(is.readsome(
+                    boost::asio::buffer_cast<char*>(mb),
+                    boost::asio::buffer_size(mb)));
+            }
+            else if(buffer.size() == 0)
+            {
+                // Our buffer is empty and we need more characters, 
+                // see if we've reached the end of file on the istream
+                if(! is.eof())
+                {
+                    // Get a mutable buffer sequence for writing
+                    auto const mb = buffer.prepare(1024);
+
+                    // Try to get more from the istream. This might block.
+                    is.read(
+                        boost::asio::buffer_cast<char*>(mb),
+                        boost::asio::buffer_size(mb));
+
+                    // If an error occurs on the istream then return it to the caller.
+                    if(is.fail() && ! is.eof())
+                    {
+                        // We'll just re-use io_error since std::istream has no error_code interface.
+                        ec = make_error_code(errc::io_error);
+                        return;
+                    }
+
+                    // Commit the characters we got to the buffer.
+                    buffer.commit(is.gcount());
+                }
+                else
+                {
+                    // Inform the parser that we've reached the end of the istream.
+                    parser.put_eof(ec);
+                    if(ec)
+                        return;
+                    break;
+                }
+            }
+
+            // Write the data to the parser
+            auto const bytes_used = parser.put(buffer.data(), ec);
+
+            // This error means that the parser needs additional octets.
+            if(ec == error::need_more)
+                ec = {};
+            if(ec)
+                return;
+
+            // Consume the buffer octets that were actually parsed.
+            buffer.consume(bytes_used);
+        }
+        while(! parser.is_done());
+
+        // Transfer ownership of the message container in the parser to the caller.
+        msg = parser.release();
+    }
+
+    void
+    doParseStdStream()
+    {
+        std::istringstream is(
+            "HTTP/1.0 200 OK\r\n"
+            "User-Agent: test\r\n"
+            "\r\n"
+            "Hello, world!"
+        );
+        error_code ec;
+        flat_buffer buffer;
+        response<string_body> res;
+        parse_istream(is, buffer, res, ec);
+        BEAST_EXPECTS(! ec, ec.message());
+    }
+
     //--------------------------------------------------------------------------
     //
     // Deferred Body type commitment
@@ -671,6 +787,7 @@ public:
         doExpect100Continue();
         doCgiResponse();
         doRelay();
+        doParseStdStream();
         doDeferredBody();
     }
 };