Files
beast/examples/http_crawl.cpp
Vinnie Falco e8be3fd7d3 New HTTP interfaces (API Change):
fix #123
fix #154
fix #265

This completely replaces the HTTP parser used to read and
parse incoming messages. The new version is faster than
the old one, and written in fewer lines. A summary of
changes:

* parse and async_parse renamed to read and async_read

* basic_parser is optimized for linear buffers,
  use flat_streambuf for the best performance with these
  functions:

  - http::read
  - http::read_some
  - http::async_read
  - http::async_read_some

* The overloads of read() and async_read() which take
  just a header have been removed, since they would
  throw away important parse metadata.

* The derived class callbacks for basic_parser have
  been streamlined. All strings passed to callbacks
  are presented in their entirety, instead of being
  provided in pieces.

These changes allow use-cases that were previously
difficult or impossible, such as:

- Efficient relaying

- Late body type commitment

- Expect: 100-continue handling
2017-07-20 08:12:15 -07:00

63 lines
1.7 KiB
C++

//
// Copyright (c) 2013-2017 Vinnie Falco (vinnie dot falco at gmail dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//
#include "urls_large_data.hpp"
#include <beast/core/streambuf.hpp>
#include <beast/http.hpp>
#include <boost/asio.hpp>
#include <boost/lexical_cast.hpp>
#include <iostream>
using namespace beast::http;
using namespace boost::asio;
template<class String>
void
err(beast::error_code const& ec, String const& what)
{
std::cerr << what << ": " << ec.message() << std::endl;
}
int main(int, char const*[])
{
io_service ios;
for(auto const& host : urls_large_data())
{
try
{
ip::tcp::resolver r(ios);
auto it = r.resolve(
ip::tcp::resolver::query{host, "http"});
ip::tcp::socket sock(ios);
connect(sock, it);
auto ep = sock.remote_endpoint();
request<string_body> req;
req.method = "GET";
req.url = "/";
req.version = 11;
req.fields.insert("Host", host + std::string(":") +
boost::lexical_cast<std::string>(ep.port()));
req.fields.insert("User-Agent", "beast/http");
prepare(req);
write(sock, req);
response<string_body> res;
streambuf sb;
beast::http::read(sock, sb, res);
std::cout << res;
}
catch(beast::system_error const& ec)
{
std::cerr << host << ": " << ec.what();
}
catch(...)
{
std::cerr << host << ": unknown exception" << std::endl;
}
}
}