Add a chapter on optimising communication (multiflight, packet queing)

Summary: related to T12804

Reviewers: ivica

Reviewed By: ivica

Subscribers: miljen, iljazovic

Differential Revision: https://repo.mireo.local/D28440

doc/qbk/00_main.qbk

@@ -113,6 +113,8 @@
 [include 02_configuring_the_client.qbk]
 [include 03_auto_reconnect.qbk]
 [include 04_maintaining_a_stable_connection.qbk]
+[include 05_optimising_communication.qbk]
+
 
 [include 10_examples.qbk]
 

doc/qbk/05_optimising_communication.qbk

@@ -0,0 +1,76 @@
+[section:optimising_communication Optimising communication]
+[nochunk]
+
+This chapter provides a detailed breakdown of how __Client__ optimises its communications with the Broker
+with multiflight mode for simultanious message handling and strategies for efficient bandwidth usage.
+These techniques are key to getting the most out of MQTT in scenarios demanding fast and
+dependable message delivery, all while meeting the protocol's quality of service requirements and network efficiency standards.
+
+[section:multiflight The multiflight mode]
+
+The __Self__ library introduces a multiflight feature.
+This allows the initiation of multiple asynchronous requests simultaneously, without waiting for the completion of the previous requests.
+With this feature, you can repeatedly call [refmem mqtt_client async_publish] or any similar `async_xxx` function
+without waiting for the handler invocation of the previous `async_xxx` calls.
+
+This feature is particulary helpful when using __Client__ with callbacks,
+as it allows you to quickly dispatch multiple requests one after the other,
+instead of nesting them in callbacks.
+
+Consider the example below, [refmem mqtt_client async_publish] with QoS 2 is called `5` times in a `for` loop.
+QoS level 2 ensures that each message is delivered exactly once and involves a four-step communication process:
+sending a __PUBLISH__ packet, receiving a __PUBREC__ acknowledgement from the Broker,
+transmitting a __PUBREL__ packet, and finally receiving a __PUBCOMP__ packet, confirming successful message delivery.
+
+Despite the complexity of initiating several such message exchange sequences consecutively,
+the __Client__ will manage all intermediate packet exchange between the __Client__ and the Broker correctly and complete the message delivery.
+
+It is important to note that there is no guarantee that the final handlers will be invoked
+in the same order as the corresponding `async_xxx` calls were initiated.
+
+[import ../../example/multiflight_client.cpp]
+Source: [link async_mqtt5.multiflight_client multiflight_client.cpp]
+[multiflight_client]
+
+[endsect] [/multiflight]
+
+[section:packet_queing Efficient bandwidth usage with packet queing]
+
+The __Client__ employs a strategic queuing mechanism crucial in optimising network usage and performance for the user's requests.
+This mechanism bundles multiple MQTT packets for transmission within a single TCP packet whenever feasible 
+[footnote Requests are queued and bundled whenever the __Client__ is in progress of writing previous request(s) to the transport.].
+This significantly reduces performance overhead, enhances data output, and reduces the latency associated with individual packet transmissions.
+This results in fast performance and efficient use of network resources.
+
+Additionally, the queuing mechanism ensures that __Client__ complies with the `Receive Maximum` value set by the Broker.
+This value is used to implement a send quota to restrict the number of __PUBLISH__ packets with QoS > 0 that have not received an acknowledgement
+(__PUBACK__ for QoS 1 and __PUBCOMP__ for QoS 2) (see [mqttlink 3901251 `Flow Control`]).
+When [refmem mqtt_client async_publish] with QoS > 0 is invoked,
+__Client__ evaluates the current count of unacknowledged __PUBLISH__ packets against the Broker's `Receive Maximum` threshold.
+If the count is below this threshold, __Client__ dispatches the __PUBLISH__ packet.
+Otherwise, it remains in the queue until the count decreases below the threshold.
+
+As a result, in the [link async_mqtt5.multiflight_client multiflight_client.cpp] example,
+the __Client__ will transmit all `5` __PUBLISH__ packets in a single TCP packet
+if possible [footnote The Broker's `Receive Maximum` is equal to or greater than `5`.].
+
+[endsect] [/packet_queing]
+
+[section:packet_ordering Packet ordering]
+
+The __Client__ uses a packet ordering mechanism to manage the queued packets pending dispatch to the Broker.
+The most important ordering rules are:
+
+- The __PUBLISH__ packets are transmitted in the order they were initiated through [refmem mqtt_client async_publish] calls.
+This sequential integrity is preserved even in instances requiring packet retransmission, ensuring consistency in message delivery order.
+However, it is important to note that sequentiality is not preserved between QoS 0 and QoS > 0 packets
+when the Broker sets up the `Receive Maximum` value.
+A Broker can set this value to limit the number of simultaneous QoS > 0 messages they can process,
+potentially causing QoS 0 messages to be transmitted ahead of QoS > 0 messages in the delivery order.
+
+- The __DISCONNECT__ packet is sent *in a single TCP packet before any other packets* in the queue.
+[/ TODO: reference to the disconnect chapter]
+
+[endsect] [/packet_ordering]
+
+[endsect] [/optimising_communication]

doc/qbk/10_examples.qbk

@@ -28,6 +28,10 @@ The following list contains all the examples that showcase how to use the __Clie
 		[[link async_mqtt5.hello_world_over_websocket_tls hello_world_over_websocket_tls.cpp]]
 		[Publishes a "Hello World" message via Websocket/TLS.]
 	]
+	[
+		[[link async_mqtt5.multiflight_client multiflight_client.cpp]]
+		[Shows how to use the __Client__ to simultaneously dispatch multiple requests.]
+	]
 ]
 
 [endsect][/examples]

doc/qbk/examples/Examples.qbk

@@ -40,4 +40,12 @@ The __Client__ will use TCP to connect to the Broker and __USE_AWAITABLE__ as th
 [receiver]
 [endsect]
 
+[section:multiflight_client The multiflight Client]
+This example will show how to use __Client__ to simultaneously dispatch multiple
+requests.
+
+[import ../../../example/multiflight_client.cpp]
+[multiflight_client]
+[endsect]
+
 [block'''</part>''']

example/multiflight_client.cpp

@@ -0,0 +1,40 @@
+//[multiflight_client
+#include <iostream>
+
+#include <boost/asio/io_context.hpp>
+#include <boost/asio/detached.hpp>
+#include <boost/asio/ip/tcp.hpp>
+#include <boost/asio/signal_set.hpp>
+
+#include <async_mqtt5.hpp>
+
+int main() {
+	boost::asio::io_context ioc;
+
+	async_mqtt5::mqtt_client<boost::asio::ip::tcp::socket> client(ioc);
+
+	client.brokers("<your-mqtt-broker>", 1883)
+		.async_run(boost::asio::detached);
+
+	// Publish with QoS 2 five times in a row without waiting for the handler
+	// of the previous async_publish call to be invoked.
+	for (auto i = 1; i <= 5; ++i)
+		client.async_publish<async_mqtt5::qos_e::exactly_once>(
+			"<topic>", "Hello world!",
+			async_mqtt5::retain_e::no, async_mqtt5::publish_props {},
+			[i](async_mqtt5::error_code ec, async_mqtt5::reason_code rc, async_mqtt5::pubcomp_props) {
+				std::cout << "Publish number " << i << " completed with: " << std::endl;
+				std::cout << "\t ec: " << ec.message() << std::endl;
+				std::cout << "\t rc: " << rc.message() << std::endl;
+			}
+		);
+
+	// We can stop the Client by using signals.
+	boost::asio::signal_set signals(ioc, SIGINT, SIGTERM);
+	signals.async_wait([&client](async_mqtt5::error_code, int) {
+		client.async_disconnect(boost::asio::detached);
+	});
+
+	ioc.run();
+}
+//]