forked from boostorg/endian
Initial commit and merge from develop.
This commit is contained in:
Beman
412
choosing_approach.html
Normal file
412
choosing_approach.html
Normal file
@@ -0,0 +1,412 @@
|
|||||||
|
<html>
|
||||||
|
|
||||||
|
<head>
|
||||||
|
<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
|
||||||
|
<meta name="ProgId" content="FrontPage.Editor.Document">
|
||||||
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
|
||||||
|
<title>Choosing Approach</title>
|
||||||
|
<link href="styles.css" rel="stylesheet">
|
||||||
|
</head>
|
||||||
|
|
||||||
|
<body>
|
||||||
|
|
||||||
|
<table border="0" cellpadding="5" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%">
|
||||||
|
<tr>
|
||||||
|
<td width="339">
|
||||||
|
<a href="../../../index.html">
|
||||||
|
<img src="../../../boost.png" alt="Boost logo" align="middle" border="0" width="277" height="86"></a></td>
|
||||||
|
<td align="middle" width="1253">
|
||||||
|
<font size="6"><b>Choosing the Approach</b></font></td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
<table border="0" cellpadding="5" cellspacing="0" style="border-collapse: collapse"
|
||||||
|
bordercolor="#111111" bgcolor="#D7EEFF" width="100%">
|
||||||
|
<tr>
|
||||||
|
<td><b>
|
||||||
|
<a href="index.html">Endian Home</a>
|
||||||
|
<a href="conversion.html">Conversion Functions</a>
|
||||||
|
<a href="arithmetic.html">Arithmetic Types</a>
|
||||||
|
<a href="buffers.html">Buffer Types</a>
|
||||||
|
<a href="choosing_approach.html">Choosing Approach</a></b></td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
<p></p>
|
||||||
|
|
||||||
|
<table border="1" cellpadding="5" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" align="right">
|
||||||
|
<tr>
|
||||||
|
<td width="100%" bgcolor="#D7EEFF" align="center">
|
||||||
|
<i><b>Contents</b></i></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td width="100%" bgcolor="#E8F5FF">
|
||||||
|
<a href="#Introduction">Introduction</a><br>
|
||||||
|
<a href="#Choosing">Choosing between conversion functions,</a><br>
|
||||||
|
<a href="#Choosing">buffer types, and arithmetic types</a><br>
|
||||||
|
<a href="#Characteristics">Characteristics</a><br>
|
||||||
|
<a href="#Endianness-invariants">Endianness invariants</a><br>
|
||||||
|
<a href="#Conversion-explicitness">Conversion explicitness</a><br>
|
||||||
|
<a href="#Arithmetic-operations">Arithmetic operations</a><br>
|
||||||
|
<a href="#Sizes">Sizes</a><br>
|
||||||
|
<a href="#Alignments">Alignments</a><br>
|
||||||
|
<a href="#Design-patterns">Design patterns</a><br>
|
||||||
|
<a href="#As-needed">Convert only as needed (i.e. lazy)</a><br>
|
||||||
|
<a href="#Anticipating-need">Convert in anticipation of need</a><br>
|
||||||
|
<a href="#Convert-generally-as-needed-locally-in-anticipation">Generally
|
||||||
|
as needed, locally in anticipation</a><br>
|
||||||
|
<a href="#Use-cases">Use case examples</a><br>
|
||||||
|
<a href="#Porting-endian-unaware-codebase">Porting endian unaware codebase</a><br>
|
||||||
|
<a href="#Porting-endian-aware-codebase">Porting endian aware codebase</a><br>
|
||||||
|
<a href="#Reliability-arithmetic-speed">Reliability and arithmetic-speed</a><br>
|
||||||
|
<a href="#Reliability-ease-of-use">Reliability and ease-of-use</a></td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
<h2><a name="Introduction">Introduction</a></h2>
|
||||||
|
|
||||||
|
<p>Deciding which is the best endianness approach (conversion functions, buffer
|
||||||
|
types, or arithmetic types) for a particular application involves complex
|
||||||
|
engineering trade-offs. It is hard to assess those trade-offs without some
|
||||||
|
understanding of the different interfaces, so you might want to read the
|
||||||
|
<a href="conversion.html">conversion functions</a>, <a href="buffers.html">
|
||||||
|
buffer types</a>, and <a href="arithmetic.html">arithmetic types</a> pages
|
||||||
|
before diving into this page.</p>
|
||||||
|
|
||||||
|
<h2><a name="Choosing">Choosing</a> between conversion functions, buffer types,
|
||||||
|
and arithmetic types</h2>
|
||||||
|
|
||||||
|
<p>The best approach to endianness for a particular application depends on the interaction between
|
||||||
|
the application's needs and the characteristics of each of the three approaches.</p>
|
||||||
|
|
||||||
|
<p><b>Recommendation:</b> If you are new to endianness, uncertain, or don't want to invest
|
||||||
|
the time to
|
||||||
|
study
|
||||||
|
engineering trade-offs, use <a href="arithmetic.html">endian arithmetic types</a>. They are safe, easy
|
||||||
|
to use, and easy to maintain. Use the
|
||||||
|
<a href="#Anticipating-need"> <i>
|
||||||
|
anticipating need</i></a> design pattern locally around performance hot spots
|
||||||
|
like lengthy loops, if needed.</p>
|
||||||
|
|
||||||
|
<h3><a name="Background">Background</a> </h3>
|
||||||
|
|
||||||
|
<p>A dealing with endianness usually implies a program portability or a data
|
||||||
|
portability requirement, and often both. That means real programs dealing with
|
||||||
|
endianness are usually complex, so the examples shown here would really be
|
||||||
|
written as multiple functions spread across multiple translation units. They
|
||||||
|
would involve interfaces that can not be altered as they are supplied by
|
||||||
|
third-parties or the standard library. </p>
|
||||||
|
|
||||||
|
<h3><a name="Characteristics">Characteristics</a></h3>
|
||||||
|
|
||||||
|
<p>The characteristics that differentiate the three approaches to endianness are the endianness
|
||||||
|
invariants, conversion explicitness, arithmetic operations, sizes available, and
|
||||||
|
alignment requirements.</p>
|
||||||
|
|
||||||
|
<h4><a name="Endianness-invariants">Endianness invariants</a></h4>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p><b>Endian conversion functions</b> use objects of the ordinary C++ arithmetic
|
||||||
|
types like <code>int</code> or <code>unsigned short</code> to hold values. That
|
||||||
|
breaks the implicit invariant that the C++ language rules apply. The usual
|
||||||
|
language rules only apply if the endianness of the object is currently set to the native endianness for the platform. That can
|
||||||
|
make it very hard to reason about logic flow, and result in difficult to
|
||||||
|
find bugs.</p>
|
||||||
|
|
||||||
|
<p>For example:</p>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
<pre>struct data_t // big endian
|
||||||
|
{
|
||||||
|
int32_t v1; // description ...
|
||||||
|
int32_t v2; // description ...
|
||||||
|
... additional character data members (i.e. non-endian)
|
||||||
|
int32_t v3; // description ...
|
||||||
|
};
|
||||||
|
|
||||||
|
data_t data;
|
||||||
|
|
||||||
|
read(data);
|
||||||
|
big_to_native_inplace(data.v1);
|
||||||
|
big_to_native_inplace(data.v2);
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
++v1;
|
||||||
|
third_party::func(data.v2);
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
native_to_big_inplace(data.v1);
|
||||||
|
native_to_big_inplace(data.v2);
|
||||||
|
write(data);
|
||||||
|
</pre>
|
||||||
|
<p>The programmer didn't bother to convert <code>data.v3</code> to native
|
||||||
|
endianness because that member isn't used. A later maintainer needs to pass
|
||||||
|
<code>data.v3</code> to the third-party function, so adds <code>third_party::func(data.v3);</code>
|
||||||
|
somewhere deep in the code. This causes a silent failure because the usual
|
||||||
|
invariant that an object of type <code>int32_t</code> holds a value as
|
||||||
|
described by the C++ core language does not apply.</p>
|
||||||
|
</blockquote>
|
||||||
|
<p><b>Endian buffer and arithmetic types</b> hold values internally as arrays of
|
||||||
|
characters with an invariant that the endianness of the array never changes.
|
||||||
|
That makes these types easier to use and programs easier to maintain. </p>
|
||||||
|
<p>Here is the same example, using an endian arithmetic type:</p>
|
||||||
|
<blockquote>
|
||||||
|
<pre>struct data_t
|
||||||
|
{
|
||||||
|
big_int32_t v1; // description ...
|
||||||
|
big_int32_t v2; // description ...
|
||||||
|
... additional character data members (i.e. non-endian)
|
||||||
|
big_int32_t v3; // description ...
|
||||||
|
};
|
||||||
|
|
||||||
|
data_t data;
|
||||||
|
|
||||||
|
read(data);
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
++v1;
|
||||||
|
third_party::func(data.v2);
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
write(data);
|
||||||
|
</pre>
|
||||||
|
<p>A later maintainer can add <code>third_party::func(data.v3)</code>and it
|
||||||
|
will just-work.</p>
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h4><a name="Conversion-explicitness">Conversion explicitness</a></h4>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p><b>Endian conversion functions</b> and <b>buffer types</b> never perform
|
||||||
|
implicit conversions. This gives users explicit control of when conversion
|
||||||
|
occurs, and may help avoid unnecessary conversions.</p>
|
||||||
|
|
||||||
|
<p><b>Endian arithmetic types</b> perform conversion implicitly. That makes
|
||||||
|
these types very easy to use, but can result in unnecessary conversions. Failure
|
||||||
|
to hoist conversions out of inner loops can bring a performance penalty.</p>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h4><a name="Arithmetic-operations">Arithmetic operations</a></h4>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p><b>Endian conversion functions</b> do not supply arithmetic
|
||||||
|
operations, but this is not a concern since this approach uses ordinary C++
|
||||||
|
arithmetic types to hold values.</p>
|
||||||
|
|
||||||
|
<p><b>Endian buffer types</b> do not supply arithmetic operations. Although this
|
||||||
|
approach avoids unnecessary conversions, it can result in the introduction of
|
||||||
|
additional variables and confuse maintenance programmers.</p>
|
||||||
|
|
||||||
|
<p><b>Endian</b> <b>arithmetic types</b> do supply arithmetic operations. They
|
||||||
|
are very easy to use if lots of arithmetic is involved. </p>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h4><a name="Sizes">Sizes</a></h4>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p><b>Endianness conversion functions</b> only support 1, 2, 4, and 8 byte
|
||||||
|
integers. That's sufficient for many applications.</p>
|
||||||
|
|
||||||
|
<p><b>Endian buffer and arithmetic types</b> support 1, 2, 3, 4, 5, 6, 7, and 8
|
||||||
|
byte integers. For an application where memory use or I/O speed is the limiting
|
||||||
|
factor, using sizes tailored to application needs can be useful.</p>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h4><a name="Alignments">Alignments</a></h4>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p><b>Endianness conversion functions</b> only support aligned integer and
|
||||||
|
floating-point types. That's sufficient for most applications.</p>
|
||||||
|
|
||||||
|
<p><b>Endian buffer and arithmetic types</b> support both aligned and unaligned
|
||||||
|
integer and floating-point types. Unaligned types are rarely needed, but when
|
||||||
|
needed they are often very useful and workarounds are painful. For example,</p>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
<p>Non-portable code like this:<blockquote>
|
||||||
|
<pre>struct S {
|
||||||
|
uint16_t a; // big endian
|
||||||
|
uint32_t b; // big endian
|
||||||
|
} __attribute__ ((packed));</pre>
|
||||||
|
</blockquote>
|
||||||
|
<p>Can be replaced with portable code like this:</p>
|
||||||
|
<blockquote>
|
||||||
|
<pre>struct S {
|
||||||
|
big_uint16_ut a;
|
||||||
|
big_uint32_ut b;
|
||||||
|
};</pre>
|
||||||
|
</blockquote>
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h3><a name="Design-patterns">Design patterns</a></h3>
|
||||||
|
|
||||||
|
<p>Applications often traffic in endian data as records or packets containing
|
||||||
|
multiple endian data elements. For simplicity, we will just call them records.</p>
|
||||||
|
|
||||||
|
<p>If desired endianness differs from native endianness, a conversion has to be
|
||||||
|
performed. When should that conversion occur? Three design patterns have
|
||||||
|
evolved.</p>
|
||||||
|
|
||||||
|
<h4><a name="As-needed">Convert only as needed</a> (i.e. lazy)</h4>
|
||||||
|
|
||||||
|
<p>This pattern defers conversion to the point in the code where the data
|
||||||
|
element is actually used.</p>
|
||||||
|
|
||||||
|
<p>This pattern is appropriate when which endian element is actually used varies
|
||||||
|
greatly according to record content or other circumstances</p>
|
||||||
|
|
||||||
|
<h4><a name="Anticipating-need">Convert in anticipation of need</a></h4>
|
||||||
|
|
||||||
|
<p>This pattern performs conversion to native endianness in anticipation of use,
|
||||||
|
such as immediately after reading records. If needed, conversion to the output
|
||||||
|
endianness is performed after all possible needs have passed, such as just
|
||||||
|
before writing records.</p>
|
||||||
|
|
||||||
|
<p>One implementation of this pattern is to create a proxy record with
|
||||||
|
endianness converted to native in a read function, and expose only that proxy to
|
||||||
|
the rest of the implementation. If a write function, if needed, handles the
|
||||||
|
conversion from native to the desired output endianness.</p>
|
||||||
|
|
||||||
|
<p>This pattern is appropriate when all endian elements in a record are
|
||||||
|
typically used regardless of record content or other circumstances</p>
|
||||||
|
|
||||||
|
<h4><a name="Convert-generally-as-needed-locally-in-anticipation">Convert
|
||||||
|
only as needed, except locally in anticipation of need</a></h4>
|
||||||
|
|
||||||
|
<p>This pattern in general defers conversion but for specific local needs does
|
||||||
|
anticipatory conversion. Although particularly appropriate when coupled with the endian buffer
|
||||||
|
or arithmetic types, it also works well with the conversion functions.</p>
|
||||||
|
|
||||||
|
<p>Example:</p>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
<pre>struct data_t
|
||||||
|
{
|
||||||
|
big_int32_t v1;
|
||||||
|
big_int32_t v2;
|
||||||
|
big_int32_t v3;
|
||||||
|
};
|
||||||
|
|
||||||
|
data_t data;
|
||||||
|
|
||||||
|
read(data);
|
||||||
|
|
||||||
|
...
|
||||||
|
++v1;
|
||||||
|
...
|
||||||
|
|
||||||
|
int32_t v3_temp = data.v3; // hoist conversion out of loop
|
||||||
|
|
||||||
|
for (int32_t i = 0; i < <i><b>large-number</b></i>; ++i)
|
||||||
|
{
|
||||||
|
... <i><b>lengthy computation that accesses </b></i>v3_temp<i><b> many times</b></i> ...
|
||||||
|
}
|
||||||
|
data.v3 = v3_temp;
|
||||||
|
|
||||||
|
write(data);
|
||||||
|
</pre>
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<p dir="ltr">In general the above pseudo-code leaves conversion up to the endian
|
||||||
|
arithmetic type <code>big_int32_t</code>. But to avoid conversion inside the
|
||||||
|
loop, a temporary is created before the loop is entered, and then used to set
|
||||||
|
the new value of <code>data.v3</code> after the loop is complete.</p>
|
||||||
|
|
||||||
|
<blockquote>
|
||||||
|
|
||||||
|
<p dir="ltr">Question: Won't the compiler's optimizer hoist the conversion out
|
||||||
|
of the loop anyhow?</p>
|
||||||
|
|
||||||
|
<p dir="ltr">Answer: VC++ 2015 Preview, and probably others, does not, even for
|
||||||
|
a toy test program. Although the savings is small (two register <code>
|
||||||
|
<span style="font-size: 85%">bswap</span></code> instructions), the cost might
|
||||||
|
be significant if the loop is repeated enough times. On the other hand, the
|
||||||
|
program may be so dominated by I/O time that even a lengthy loop will be
|
||||||
|
immaterial.</p>
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
|
||||||
|
<h3><a name="Use-cases">Use case examples</a></h3>
|
||||||
|
|
||||||
|
<h4><a name="Porting-endian-unaware-codebase">Porting endian unaware codebase</a></h4>
|
||||||
|
|
||||||
|
<p>An existing codebase runs on big endian systems. It does not
|
||||||
|
currently deal with endianness. The codebase needs to be modified so it can run
|
||||||
|
on little endian systems under various operating systems. To ease
|
||||||
|
transition and protect value of existing files, external data will continue to
|
||||||
|
be maintained as big endian.</p>
|
||||||
|
|
||||||
|
<p dir="ltr">The <a href="arithmetic.html">endian
|
||||||
|
arithmetic approach</a> is recommended to meet these needs. A relatively small
|
||||||
|
number of header files dealing with binary I/O layouts need to change types. For
|
||||||
|
example,
|
||||||
|
<code>short</code> or <code>int16_t</code> would change to <code>big_int16_t</code>. No
|
||||||
|
changes are required for <code>.cpp</code> files.</p>
|
||||||
|
|
||||||
|
<h4><a name="Porting-endian-aware-codebase">Porting endian aware codebase</a></h4>
|
||||||
|
|
||||||
|
<p>An existing codebase runs on little-endian Linux systems. It already
|
||||||
|
deals with endianness via
|
||||||
|
<a href="http://man7.org/linux/man-pages/man3/endian.3.html">Linux provided
|
||||||
|
functions</a>. Because of a business merger, the codebase has to be quickly
|
||||||
|
modified for Windows and possibly other operating systems, while still
|
||||||
|
supporting Linux. The codebase is reliable and the programmers are all
|
||||||
|
well-aware of endian issues. </p>
|
||||||
|
|
||||||
|
<p dir="ltr">These factors all argue for an <a href="conversion.html">endian conversion
|
||||||
|
approach</a> that just mechanically changes the calls to <code>htobe32</code>,
|
||||||
|
etc. to <code>boost::endian::native_to_big</code>, etc. and replaces <code><endian.h></code>
|
||||||
|
with <code><boost/endian/conversion.hpp></code>.</p>
|
||||||
|
|
||||||
|
<h4><a name="Reliability-arithmetic-speed">Reliability and arithmetic-speed</a></h4>
|
||||||
|
|
||||||
|
<p>A new, complex, multi-threaded application is to be developed that must run
|
||||||
|
on little endian machines, but do big endian network I/O. The developers believe
|
||||||
|
computational speed for endian variable is critical but have seen numerous bugs
|
||||||
|
result from inability to reason about endian conversion state. They are also
|
||||||
|
worried that future maintenance changes could inadvertently introduce a lot of
|
||||||
|
slow conversions if full-blown endian arithmetic types are used.</p>
|
||||||
|
|
||||||
|
<p>The <a href="buffers.html">endian buffers</a> approach is made-to-order for
|
||||||
|
this use case.</p>
|
||||||
|
|
||||||
|
<h4><a name="Reliability-ease-of-use">Reliability and ease-of-use</a></h4>
|
||||||
|
|
||||||
|
<p>A new, complex, multi-threaded application is to be developed that must run
|
||||||
|
on little endian machines, but do big endian network I/O. The developers believe
|
||||||
|
computational speed for endian variables is <b>not critical</b> but have seen
|
||||||
|
numerous bugs result from inability to reason about endian conversion state.
|
||||||
|
They are also concerned about ease-of-use both during development and long-term
|
||||||
|
maintenance.</p>
|
||||||
|
|
||||||
|
<p>Removing concern about conversion speed and adding concern about ease-of-use
|
||||||
|
tips the balance strongly in favor the <a href="arithmetic.html">endian
|
||||||
|
arithmetic approach</a>.</p>
|
||||||
|
|
||||||
|
<hr>
|
||||||
|
<p>Last revised:
|
||||||
|
<!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->19 January, 2015<!--webbot bot="Timestamp" endspan i-checksum="38903" --></p>
|
||||||
|
<p>© Copyright Beman Dawes, 2011, 2013, 2014</p>
|
||||||
|
<p>Distributed under the Boost Software License, Version 1.0. See
|
||||||
|
<a href="http://www.boost.org/LICENSE_1_0.txt">www.boost.org/ LICENSE_1_0.txt</a></p>
|
||||||
|
|
||||||
|
<p> </p>
|
||||||
|
|
||||||
|
</body>
|
||||||
|
|
||||||
|
</html>
|
||||||
Reference in New Issue
Block a user