diff --git a/arithmetic.html b/arithmetic.html index 848bf3c..2d72bd3 100644 --- a/arithmetic.html +++ b/arithmetic.html @@ -1,4 +1,4 @@ - + @@ -17,7 +17,7 @@ -Boost logo +Boost logo Endian Arithmetic Types @@ -27,11 +27,12 @@ - + Buffer Types     + Choosing Approach
Boost Home     + Endian Home     Conversion Functions     Arithmetic Types     - Buffer Types
@@ -64,17 +65,7 @@ Acknowledgements - - - Headers - - - - <boost/endian/conversion.hpp>
- <boost/endian/buffers.hpp>
- <boost/endian/arithmetic.hpp> - - +

Introduction

Header boost/endian/arithmetic.hpp provides integer and floating point binary types with control over @@ -118,10 +109,11 @@ using namespace boost::endian; namespace { - // This is an extract from a very widely used GIS file format. It seems odd - // to mix big and little endians in the same file - but this is a real-world - // format and users wishing to write low level code manipulating these files - // must deal with the mixed endianness. + // This is an extract from a very widely used GIS file format. + // Why the designer decided to mix big and little endians in + // the same file is not known. But this is a real-world format + // and users wishing to write low level code manipulating these + // files have to deal with the mixed endianness. struct header { @@ -145,11 +137,13 @@ int main(int, char* []) h.version = 1; h.shape_type = 0x01020304; - // Low-level I/O such as POSIX read/write or <cstdio> fread/fwrite is sometimes - // used for binary file operations when ultimate efficiency is important. Such - // I/O is often performed in some C++ wrapper class, but to drive home the - // point that endian integers are often used in fairly low-level code that does - // bulk I/O operations, <cstdio> fopen/fwrite is used for I/O in this example. + // Low-level I/O such as POSIX read/write or <cstdio> + // fread/fwrite is sometimes used for binary file operations + // when ultimate efficiency is important. Such I/O is often + // performed in some C++ wrapper class, but to drive home the + // point that endian integers are often used in fairly + // low-level code that does bulk I/O operations, <cstdio> + // fopen/fwrite is used for I/O in this example. std::FILE* fi = std::fopen(filename, "wb"); // MUST BE BINARY @@ -214,8 +208,9 @@ enum class align {no, yes};

One class template is provided:

-
template <order Order, typename T, std::size_t n_bits, align A = align::no>
-  class endian_arithmetic;
+  
template <order Order, typename T, std::size_t n_bits,
+  align Align = align::no>
+class endian_arithmetic;
 

Typedefs, such as big_int32_t, provide convenient naming @@ -375,9 +370,10 @@ usual operations on integers are supplied.

enum class align {no, yes}; - template <order Order, class T, std::size_t n_bits, align A = align::no> + template <order Order, class T, std::size_t n_bits, + align Align = align::no> class endian_arithmetic - : public endian_buffer<Order, T, n_bits, A> + : public endian_buffer<Order, T, n_bits, Align> { public: typedef T value_type; @@ -389,8 +385,8 @@ usual operations on integers are supplied.

endian_arithmetic& operator=(T v) noexcept; operator value_type() const noexcept; - value_type value() const noexcept; // exposition only; see endian_buffer - const char* data() const noexcept; // exposition only; see endian_buffer + value_type value() const noexcept; // for exposition; see endian_buffer + const char* data() const noexcept; // for exposition; see endian_buffer // arithmetic operations // note that additional operations are provided by the value_type @@ -426,35 +422,35 @@ usual operations on integers are supplied.

// typedefs // aligned big endian floating point types - typedef endian<order::big, float, 32, align::yes> big_float32_t; - typedef endian<order::big, double, 64, align::yes> big_float64_t; + typedef endian<order::big, float, 32, align::yes> big_float32_t; + typedef endian<order::big, double, 64, align::yes> big_float64_t; // aligned little endian floating point types - typedef endian<order::little, float, 32, align::yes> little_float32_t; - typedef endian<order::little, double, 64, align::yes> little_float64_t; + typedef endian<order::little, float, 32, align::yes> little_float32_t; + typedef endian<order::little, double, 64, align::yes> little_float64_t; // unaligned big endian floating point types - typedef endian<order::big, float, 32, align::no> big_float32un_t; - typedef endian<order::big, double, 64, align::no> big_float64un_t; + typedef endian<order::big, float, 32, align::no> big_float32_ut; + typedef endian<order::big, double, 64, align::no> big_float64_ut; // unaligned little endian floating point types - typedef endian<order::little, float, 32, align::no> little_float32un_t; - typedef endian<order::little, double, 64, align::no> little_float64un_t; + typedef endian<order::little, float, 32, align::no> little_float32_ut; + typedef endian<order::little, double, 64, align::no> little_float64_ut; // aligned big endian signed integer types - typedef endian<order::big, int16_t, 16, align::yes> big_int16_t; - typedef endian<order::big, int32_t, 32, align::yes> big_int32_t; - typedef endian<order::big, int64_t, 64, align::yes> big_int64_t; + typedef endian<order::big, int16_t, 16, align::yes> big_int16_t; + typedef endian<order::big, int32_t, 32, align::yes> big_int32_t; + typedef endian<order::big, int64_t, 64, align::yes> big_int64_t; // aligned big endian unsigned integer types - typedef endian<order::big, uint16_t, 16, align::yes> big_uint16_t; - typedef endian<order::big, uint32_t, 32, align::yes> big_uint32_t; - typedef endian<order::big, uint64_t, 64, align::yes> big_uint64_t; + typedef endian<order::big, uint16_t, 16, align::yes> big_uint16_t; + typedef endian<order::big, uint32_t, 32, align::yes> big_uint32_t; + typedef endian<order::big, uint64_t, 64, align::yes> big_uint64_t; // aligned little endian signed integer types - typedef endian<order::little, int16_t, 16, align::yes> little_int16_t; - typedef endian<order::little, int32_t, 32, align::yes> little_int32_t; - typedef endian<order::little, int64_t, 64, align::yes> little_int64_t; + typedef endian<order::little, int16_t, 16, align::yes> little_int16_t; + typedef endian<order::little, int32_t, 32, align::yes> little_int32_t; + typedef endian<order::little, int64_t, 64, align::yes> little_int64_t; // aligned little endian unsigned integer types typedef endian<order::little, uint16_t, 16, align::yes> little_uint16_t; @@ -681,7 +677,7 @@ differs from endian representation size. Vicente Botet and other reviewers suggested supporting floating point types.


Last revised: -05 December, 2014

+19 January, 2015

© Copyright Beman Dawes, 2006-2009, 2013

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

diff --git a/buffers.html b/buffers.html index e171fbb..594b941 100644 --- a/buffers.html +++ b/buffers.html @@ -1,4 +1,4 @@ - + @@ -17,7 +17,7 @@ -Boost logo +Boost logo Endian Buffer Types @@ -27,11 +27,12 @@ - + Buffer Types     + Choosing Approach
Boost Home     + Endian Home     Conversion Functions     Arithmetic Types     - Buffer Types
@@ -53,30 +54,20 @@     Synopsis
    Members
+    Non-Members
FAQ
Design
C++11
Compilation - - - Headers - - - -

- <boost/endian/conversion.hpp>
- <boost/endian/buffers.hpp>
- <boost/endian/arithmetic.hpp> - - +

Introduction

The internal byte order of arithmetic types is traditionally called endianness. See the Wikipedia for a full exploration of endianness, including definitions of big endian and little endian.

-

Header boost/endian/buffers.hpp +

Header boost/endian/buffers.hpp provides endian_buffer, a portable endian integer and floating-point binary buffer class template with control over byte order, value type, size, and alignment independent of the platform's native @@ -93,22 +84,23 @@ base class for the endian_arithmetic class template, which is aimed at users who wish fully automatic endianness conversion and direct support for all normal arithmetic operations.

Example

-

The endian_example.cpp program writes a +

The example/endian_example.cpp program writes a binary file containing four byte big-endian and little-endian integers:

#include <iostream>
 #include <cstdio>
-#include <boost/endian/buffers.hpp>
+#include <boost/endian/buffers.hpp>  // see Synopsis below
 #include <boost/static_assert.hpp>
 
 using namespace boost::endian;
 
 namespace 
 {
-  //  This is an extract from a very widely used GIS file format. Why the designer
-  //  decided to mix big and little endians in the same file is not known. But
-  //  this is a real-world format and users wishing to write low level code
-  //  manipulating these files have to deal with the mixed endianness.
+  //  This is an extract from a very widely used GIS file format.
+  //  Why the designer decided to mix big and little endians in
+  //  the same file is not known. But this is a real-world format
+  //  and users wishing to write low level code manipulating these
+  //  files have to deal with the mixed endianness.
 
   struct header
   {
@@ -132,11 +124,13 @@ int main(int, char* [])
   h.version     = 1;
   h.shape_type  = 0x01020304;
 
-  //  Low-level I/O such as POSIX read/write or <cstdio> fread/fwrite is sometimes
-  //  used for binary file operations when ultimate efficiency is important. Such
-  //  I/O is often performed in some C++ wrapper class, but to drive home the
-  //  point that endian integers are often used in fairly low-level code that does
-  //  bulk I/O operations, <cstdio> fopen/fwrite is used for I/O in this example.
+  //  Low-level I/O such as POSIX read/write or <cstdio>
+  //  fread/fwrite is sometimes used for binary file operations
+  //  when ultimate efficiency is important. Such I/O is often
+  //  performed in some C++ wrapper class, but to drive home the
+  //  point that endian integers are often used in fairly
+  //  low-level code that does bulk I/O operations, <cstdio>
+  //  fopen/fwrite is used for I/O in this example.
 
   std::FILE* fi = std::fopen(filename, "wb");  // MUST BE BINARY
   
@@ -160,7 +154,7 @@ int main(int, char* [])
 }
 
-

After compiling and executing endian_example.cpp, +

After compiling and executing example/endian_example.cpp, a hex dump of test.dat shows:

01020304 00000010 01000000 04030201
@@ -201,8 +195,9 @@ enum class align {no, yes};

One class template is provided:

-
template <order Order, typename T, std::size_t Nbits, align A = align::no>
-  class endian_buffer;
+  
template <order Order, typename T, std::size_t Nbits,
+  align Align = align::no>
+class endian_buffer;
 

Typedefs, such as big_int32_buf_t, provide convenient naming @@ -339,7 +334,7 @@ requirements vary between hardware architectures and because alignment may be affected by compiler switches or pragmas. For example, alignment of an 64-bit integer may be to a 32-bit boundary on a 32-bit machine. Furthermore, aligned types are only available on architectures with 16, 32, and 64-bit integer types.

-

Note: One-byte types +

Note: One-byte big and little buffer types have identical functionality. They are provided to improve code readability and searchability.

Class template endian_buffer

@@ -362,7 +357,8 @@ usual operations on integers are supplied.

enum class align {no, yes}; - template <order Order, class T, std::size_t Nbits, align Align = align::no> + template <order Order, class T, std::size_t Nbits, + align Align = align::no> class endian_buffer { public: @@ -377,6 +373,20 @@ usual operations on integers are supplied.

protected: implementaton-defined endian_value; // for exposition only }; + + // stream inserter + template <class charT, class traits, order Order, class T, + std::size_t n_bits, align Align> + std::basic_ostream<charT, traits>& + operator<<(std::basic_ostream<charT, traits>& os, + const endian_buffer<Order, T, n_bits, Align>& x); + + // stream extractor + template <class charT, class traits, order Order, class T, + std::size_t n_bits, align A> + std::basic_istream<charT, traits>& + operator>>(std::basic_istream<charT, traits>& is, + endian_buffer<Order, T, n_bits, Align>& x); // typedefs @@ -479,7 +489,6 @@ usual operations on integers are supplied.

typedef implementation-defined_uint56_buf_ut native_uint56_buf_ut; typedef implementation-defined_uint64_buf_ut native_uint64_buf_ut; - } // namespace endian } // namespace boost

The implementation-defined text in typedefs above is either @@ -539,6 +548,30 @@ boost::endian::endian_reverse.

Returns: A pointer to the first byte of endian_value.

+

Non-member functions

+
template <class charT, class traits, order Order, class T,
+  std::size_t n_bits, align Align>
+std::basic_ostream<charT, traits>& operator<<(std::basic_ostream<charT, traits>& os,
+  const endian_buffer<Order, T, n_bits, Align>& x);
+
+
+

Returns: os << x.value().

+
+
template <class charT, class traits, order Order, class T,
+  std::size_t n_bits, align A>
+std::basic_istream<charT, traits>& operator>>(std::basic_istream<charT, traits>& is,
+  endian_buffer<Order, T, n_bits, Align>& x);
+
+
+

Effects: As if:

+
+
T i;
+if (is >> i)
+  x = i;
+
+
+

Returns: is.

+

FAQ

See the Endian home page FAQ for a library-wide @@ -617,7 +650,7 @@ any Boost object libraries.


Last revised: -06 December, 2014

+19 January, 2015

© Copyright Beman Dawes, 2006-2009, 2013

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

diff --git a/conversion.html b/conversion.html index 5ea86b5..c23617b 100644 --- a/conversion.html +++ b/conversion.html @@ -1,4 +1,4 @@ - + @@ -15,7 +15,7 @@ -Boost logo +Boost logo Endian Conversion Functions @@ -24,11 +24,12 @@ - + Buffer Types     + Choosing Approach
Boost Home     + Endian Home     Conversion Functions     Arithmetic Types     - Buffer Types
@@ -52,17 +53,7 @@ FAQ
Acknowledgements - - - Headers - - - - <boost/endian/conversion.hpp>
- <boost/endian/buffers.hpp>
- <boost/endian/arithmetic.hpp> - - +

Introduction

@@ -232,8 +223,7 @@ call to endian_reverse().

EndianReversibleInplace are required to perform reversal of endianness if needed by making an unqualified call to endian_reverse_inplace().

-

See -udt_conversion_example.cpp for an example user-defined type.

+

See example/udt_conversion_example.cpp for an example user-defined type.

Functions

int8_t   endian_reverse(int8_t x) noexcept;
@@ -371,12 +361,13 @@ portability for both programs and data.

Acknowledgements

Tomas Puverle was instrumental in identifying and articulating the need to -support endian conversion as separate from endian integer types. Phil Endecott suggested the form of the value returning signatures. Vicente Botet and other reviewers suggested supporting floating point types and user defined types. General reverse template implementation approach using std::reverse suggested by Mathias Gaunard. Portable implementation approach for 16, 32, and 64-bit integers suggested by tymofey, with avoidance of undefined behavior as suggested by Giovanni Piero Deretta, and a further refinement suggested by Pyry Jahkola. Intrinsic builtins implementation approach for 16, 32, and 64-bit integers suggested by several reviewers, and by David Stone, who provided his Boost licensed macro implementation that became the starting point for boost/endian/detail/intrinsic.hpp. +support endian conversion as separate from endian integer types. Phil Endecott suggested the form of the value returning signatures. Vicente Botet and other reviewers suggested supporting floating point types and user defined types. General reverse template implementation approach using std::reverse suggested by Mathias Gaunard. Portable implementation approach for 16, 32, and 64-bit integers suggested by tymofey, with avoidance of undefined behavior as suggested by Giovanni Piero Deretta, and a further refinement suggested by Pyry Jahkola. Intrinsic builtins implementation approach for 16, 32, and 64-bit integers suggested by several reviewers, and by David Stone, who provided his Boost licensed macro implementation that became the starting point for +boost/endian/detail/intrinsic.hpp. Pierre Talbot provided the int8_t endian_reverse() and templated endian_reverse_inplace() implementations.


Last revised: -16 December, 2014

+19 January, 2015

© Copyright Beman Dawes, 2011, 2013

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

diff --git a/index.html b/index.html index 0ccac4e..27e0b10 100644 --- a/index.html +++ b/index.html @@ -15,23 +15,26 @@ -Boost logo +Boost logo Endian Library - +
- + Buffer Types     + Choosing Approach
Boost Home     + Endian Home     Conversion Functions     Arithmetic Types     - Buffer Types

+ - - - - - - -
@@ -42,40 +45,22 @@ Abstract
Introduction to endianness
Introduction to the Boost.Endian library
-Choosing between conversion, buffer types,
and arithmetic types
-   Characteristics
-      Endianness invariants
-      Conversion explicitness
-      Arithmetic operations
-      Sizes
-      Alignments
-   Use cases
-      Porting endian unaware codebase
-      Porting endian aware codebase
-      Reliability and arithmetic-speed
-      Reliability and ease-of-use
+Choosing between conversion functions,
+   buffer types, and arithmetic types
Built-in support for Intrinsics
Performance
-   Timings for Example 2
-   Conclusions
+   Timings
Overall FAQ
Release history
-   Changes since formal review
+   Changes +requested by formal review
+   Other changes since +formal review
+Compatibility with interim releases
Acknowledgements
- Headers
- <boost/endian/conversion.hpp>
- <boost/endian/buffers.hpp>
- <boost/endian/arithmetic.hpp>
+

Abstract

@@ -83,7 +68,11 @@ endianness of integers, floating point numbers, and user-defined types.

-

 

-

 

-

Introduction to endianness

Consider the following code:

@@ -181,189 +164,35 @@ integers. The types may be aligned.

-

Boost Endian is a header-only library.

+

Boost Endian is a header-only library. C++03 compilers are supported. C++11 features +affecting interfaces, such as noexcept, are used only if available.

Choosing between conversion functions, buffer types, and arithmetic types

-

The best approach to endianness for a particular use case depends on interactions between -the approach characteristics and -the application needs.

- -

Recommendation: If you are uncertain, new to endianness, concerned -about security, concerned about reliability, or don't want to invest time making -engineering trade-offs, use the endian arithmetic types. They are safer, easier -to use, and your code will be easier to maintain.

- -

Characteristics

- -

The characteristics that differentiate the three approaches to endianness are the endianness -invariants, conversion explicitness, arithmetic operations, sizes available, and -alignment requirements.

- -

Endianness invariants

- -
- -

Endian conversion functions use objects of the ordinary C++ arithmetic -types like int or unsigned short to hold values. That -breaks the implicit invariant that the C++ language rules apply. The usual -language rules only apply if the endianness of the object is currently set by -the conversion functions to the native endianness for the platform. That can -make it very hard to reason about complex logic flow, and result in difficult to -find bugs.

- -

Endian buffer and arithmetic types hold values internally as arrays of -characters with an invariant that the endianness of the array never changes. -That makes these types easier to use and programs easier to maintain.

- -
- -

Conversion explicitness

- -
- -

Endian conversion functions and buffer types never perform -implicit conversions. This gives users explicit control of when conversion -occurs, and may help avoid unnecessary conversions.

- -

Endian arithmetic types perform conversion implicitly. That makes -these types very easy to use, but can result in unnecessary conversions. Failure -to hoist conversions out of inner loops can bring a performance penalty.

- -
- -

Arithmetic operations

- -
- -

Endian conversion functions do not supply arithmetic -operations, but this is not a concern since this approach uses ordinary C++ -arithmetic types to hold values.

- -

Endian buffer types do not supply arithmetic operations. Although this -approach avoids unnecessary conversions, it can result in the introduction of -additional variables and confuse maintenance programmers.

- -

Endian arithmetic types do supply arithmetic operations. They -are very easy to use if lots of arithmetic is involved.

- -
- -

Sizes

- -
- -

Endianness conversion functions only support 1, 2, 4, and 8 byte -integers. That's sufficient for many applications.

- -

Endian buffer and arithmetic types support 1, 2, 3, 4, 5, 6, 7, and 8 -byte integers. For an application where memory use or I/O speed is the limiting -factor, using sizes tailored to application needs can be useful.

- -
- -

Alignments

- -
- -

Endianness conversion functions only support aligned integer and -floating-point types. That's sufficient for most applications.

- -

Endian buffer and arithmetic types support both aligned and unaligned -integer and floating-point types. Unaligned types are rarely needed, but when -needed they are often very useful and workarounds are painful. For example,

- -
-

Non-portable code like this:

-

struct S {
-   uint16_t a;  // big endian
-   uint32_t b;  // big endian
- } __attribute__ ((packed));
-

-

Can be replaced with portable code like this:

-
-

struct S {
-   big_uint16_ut a;
-   big_uint32_ut b;
- };
-

-
- -
- -

Use cases

- -

Porting endian unaware codebase

- -

An existing codebase runs on big endian systems. It does not -currently deal with endianness. The codebase needs to be modified so it can run -on  little endian systems under various operating systems. To ease -transition and protect value of existing files, external data will continue to -be maintained as big endian.

- -

The endian -arithmetic approach is recommended to meet these needs. A relatively small -number of header files dealing with binary I/O layouts need to change types like -short or int16_t to big_int16_t, and -int or int32_t to bif_int32_t. No -changes are required for .cpp files.

- -

Porting endian aware codebase

- -

An existing codebase runs on little-endian Linux systems. It already -deals with endianness via -Linux provided -functions. Because of a business merger, the codebase has to be quickly -modified for Windows and possibly other operating systems, while still -supporting Linux. The codebase is reliable and the programmers are all -well-aware of endian issues.

- -

These factors all argue for an endian conversion -approach that just mechanically changes the calls to htobe32, -etc. to boost::endian::native_to_big, etc. and replaces <endian.h> -with <boost/endian/conversion.hpp>.

- -

Reliability and arithmetic-speed

- -

A new, complex, multi-threaded application is to be developed that must run -on little endian machines, but do big endian network I/O. The developers believe -computational speed for endian variable is critical but have seen numerous bugs -result from inability to reason about endian conversion state. They are also -worried that future maintenance changes could inadvertently introduce a lot of -slow conversions if full-blown endian arithmetic types are used.

- -

The endian buffers approach is made-to-order for -this use case.

- -

Reliability and ease-of-use

- -

A new, complex, multi-threaded application is to be developed that must run -on little endian machines, but do big endian network I/O. The developers believe -computational speed for endian variables is not critical but have seen -numerous bugs result from inability to reason about endian conversion state. -They are also concerned about ease-of-use both during development and long-term -maintenance.

- -

Removing concern about conversion speed and adding concern about ease-of-use -tips the balance strongly in favor the endian -arithmetic approach.

+

This section has been moved to its own +Choosing the Approach page.

Built-in support for Intrinsics

-

Supply compilers, including GCC, Clang, and Visual C++, supply built-in support for byte swapping intrinsics. -The library uses these intrinsics when available since they may result in smaller and faster generated code, particularly for release +

Most compilers, including GCC, Clang, and Visual C++, supply built-in support for byte swapping intrinsics. +The Endian library uses these intrinsics when available since they may result in smaller and faster generated code, particularly for release builds.

-

Defining BOOST_ENDIAN_NO_INTRINSICS will suppress use +

Defining the macro BOOST_ENDIAN_NO_INTRINSICS will suppress use of the intrinsics. Useful when intrinsic headers such as -byteswap.h are not being found on your platform.

+byteswap.h are not being found by your compiler, perhaps because it +is an older release or has very limited supporting libraries.

The macro BOOST_ENDIAN_INTRINSIC_MSG is defined as either "no byte swap intrinsics" or a string describing the -particular set of intrinsics being used.

+particular set of intrinsics being used. Useful for eliminating missing +intrinsics as a source of performance issues.

Performance

Consider this problem:

+
+
+
@@ -404,15 +233,20 @@ native_to_big_inplace(x);
+
+
+

There will be no performance difference between the two approaches in release builds, -regardless of the native endianness of the machine. That's because optimizing compilers will likely -generate exactly the same code for each. That conclusion was confirmed by +regardless of the native endianness of the machine. That's because optimizing compilers will generate exactly the same code for each. That conclusion was confirmed by studying the generated assembly code for GCC and Visual C++. Furthermore, time spent doing I/O will determine the speed of this application.

Now consider a slightly different problem: 

+
+
+
@@ -456,230 +290,63 @@ native_to_big_inplace(x);
+
+
+

With the Endian arithmetic approach, on little endian platforms an implicit conversion from and then back to big endian is done inside the loop. With the Endian conversion function approach, the user has ensured the conversions are done outside the loop, so the code may run more quickly on little endian platforms.

-

Timings for Example 2 (conversion functions hoisted -out of loop)

+

Timings

These tests were run against release builds on a circa 2012 4-core little endian X64 Intel Core i5-3570K CPU @ 3.40GHz under Windows 7.

Caveat emptor: The Windows CPU timer has very high granularity. Repeated runs of the same tests often yield considerably different results.

-

See loop_time_test.cpp and -Jamfile.v2 for the actual code and build -setup. -(For GCC 4.7, there are no 16-bit intrinsics, so they are emulated by using -32-bit intrinsics.)

+

See test/loop_time_test.cpp for the actual code and benchmark/Jamfile.v2 for the build +setup.

+ +
+
- - - - - + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + +
GNU C++ version 4.7.0
Iterations: 1000000000, Intrinsics: __builtin_bswap16, etc.
Test CaseEndian
arithmetic
Endian
conversion
function
GNU C++ version 4.8.2 on Linux virtual + machine
Iterations: 10'000'000'000, Intrinsics: __builtin_bswap16, etc.
Test CaseEndian
arithmetic
type
Endian
conversion
function
16-bit aligned big endian1.37 s0.81 s
16-bit aligned little endian0.83 s0.81 s
16-bit unaligned big endian1.09 s0.83 s
16-bit unaligned little endian1.09 s0.81 s
32-bit aligned big endian0.98 s0.27 s
32-bit aligned little endian0.28 s0.27 s
32-bit unaligned big endian3.82 s0.27 s
32-bit unaligned little endian3.82 s0.27 s
64-bit aligned big endian1.65 s0.41 s
64-bit aligned little endian0.41 s0.41 s
64-bit unaligned big endian17.53 s0.41 s
64-bit unaligned little endian17.52 s0.41 s
Iterations: 1000000000, Intrinsics: no byte swap intrinsics
Test CaseEndian
arithmetic
Endian
conversion
function
16-bit aligned big endian1.95 s0.81 s
16-bit aligned little endian0.83 s0.81 s
16-bit unaligned big endian1.19 s0.81 s
16-bit unaligned little endian1.20 s0.81 s
32-bit aligned big endian0.97 s0.28 s
32-bit aligned little endian0.27 s0.28 s
32-bit unaligned big endian4.10 s0.27 s
32-bit unaligned little endian4.10 s0.27 s
64-bit aligned big endian1.64 s0.42 s
64-bit aligned little endian0.41 s0.41 s
64-bit unaligned big endian17.52 s0.42 s
64-bit unaligned little endian17.52 s0.41 s
16-bit aligned big endian8.46 s5.28 s
16-bit aligned little endian5.28 s5.22 s
32-bit aligned big endian8.40 s2.11 s
32-bit aligned little endian2.11 s2.10 s
64-bit aligned big endian14.02 s3.10 s
64-bit aligned little endian3.00 s3.03 s
+
+
+

-

Comment: Note that the 32-bit aligned big endian -timings are the same with or without intrinsics turned on. Presumably the -optimizer is recognizing the byte swapping and applying the intrinsics itself.

- +
- - - - - + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + +
Microsoft Visual C++ version 11.0
Iterations: 1000000000, Intrinsics: cstdlib _byteswap_ushort, etc.
Test CaseEndian
type
Endian
conversion
function
Microsoft Visual C++ version 14.0
Iterations: 10'000'000'000, Intrinsics: cstdlib _byteswap_ushort, etc.
Test CaseEndian
arithmetic
type
Endian
conversion
function
16-bit aligned big endian0.83 s0.51 s
16-bit aligned little endian0.51 s0.50 s
16-bit unaligned big endian1.37 s0.51 s
16-bit unaligned little endian1.37 s0.50 s
32-bit aligned big endian0.81 s0.50 s
32-bit aligned little endian0.51 s0.51 s
32-bit unaligned big endian2.98 s0.53 s
32-bit unaligned little endian3.00 s0.51 s
64-bit aligned big endian1.33 s0.33 s
64-bit aligned little endian0.34 s0.27 s
64-bit unaligned big endian7.05 s0.33 s
64-bit unaligned little endian7.11 s0.31 s
Iterations: 1000000000, Intrinsics: no byte swap intrinsics
Test CaseEndian
type
Endian
conversion
function
16-bit aligned big endian0.83 s0.51 s
16-bit aligned little endian0.51 s0.51 s
16-bit unaligned big endian1.36 s0.51 s
16-bit unaligned little endian1.37 s0.51 s
32-bit aligned big endian3.42 s0.50 s
32-bit aligned little endian0.51 s0.51 s
32-bit unaligned big endian2.93 s0.50 s
32-bit unaligned little endian2.95 s0.50 s
64-bit aligned big endian5.99 s0.33 s
64-bit aligned little endian0.33 s0.33 s
64-bit unaligned big endian7.02 s0.27 s
64-bit unaligned little endian7.02 s0.27 s
16-bit aligned big endian8.27 s5.26 s
16-bit aligned little endian5.29 s5.32 s
32-bit aligned big endian8.36 s5.24 s
32-bit aligned little endian5.24 s5.24 s
64-bit aligned big endian13.65 s3.34 s
64-bit aligned little endian3.35 s2.73 s
- - -

Conclusions

- -

When program logic dictates many more conversions for the Endian arithmetic -approach than the Endian conversion function approach (example -2):

- -
- -

There may be a considerable performance difference. If machine endianness differs from the -desired endianness, the Endian arithmetic approach must do the byte reversal many -times while the Endian conversion approach only does the reversal once. But if -the endianness is the same, there is no conversion with either approach and no -conversion code is generated for typical release builds.

- -

Whether or not compiler byte swap intrinsics are explicitly available has little -impact on GCC but a lot of impact on Visual C++, for the tested compiler -versions. Yet another example of why actual timing tests are needed to -determine if some coding technique has significant impact on performance.

- -

Unaligned types are much slower that aligned types, regardless of -endianness considerations. Instead of single instruction register loads and -stores, multiple instructions are required on common platforms.

- -
+

Overall FAQ

@@ -692,6 +359,14 @@ stores, multiple instructions are required on common platforms.

+

Are C++03 compilers supported?

+ +
+ +

Yes.

+ +
+

Does the implementation use compiler intrinsic built-in byte swapping?

@@ -708,7 +383,7 @@ stores, multiple instructions are required on common platforms.

I/O formats?

Using the unaligned integer types to save internal or external -memory space is a minor secondary use case.

+memory space is a minor secondary use.

Why bother with binary I/O? Why not just use C++ Standard Library stream inserters and extractors?

@@ -764,68 +439,71 @@ and 16, 32, and 64-bit aligned integers.

Release history

-

Changes since formal review

+

Changes requested by formal review

+

The library was reworked from top to bottom to accommodate changes requested +during the formal review. See Mini-Review +page for details.

+

Other changes since formal +review

+

Compatibility with interim releases

+ +

Prior to the official Boost release, class template +endian_arithmetic has been used for a decade or more with the same +functionality but under the name endian. Other names also changed +in the official release. If the macro BOOST_ENDIAN_DEPRECATED_NAMES +is defined, those old now deprecated names are still supported. However, the +class template endian name is only provided for compilers +supporting C++11 template aliases. For C++03 compilers, the name will have to be +changed to endian_arithmetic.

+ +

To support backward header compatibility, deprecated header boost/endian/endian.hpp +forwards to boost/endian/arithmetic.hpp. It requires +BOOST_ENDIAN_DEPRECATED_NAMES be defined. It should only be used while +transitioning to the official Boost release of the library as it will be removed +in some future release.

+

Acknowledgements

Comments and suggestions were received from Adder, Benaka Moorthi, Christopher Kohlhoff, Cliff Green, Daniel James, Gennaro Proto, Giovanni Piero -Deretta, Gordon Woodhull, dizzy, Hartmut Kaiser, Jeff Flinn, John Filo, John +Deretta, Gordon Woodhull, dizzy, Hartmut Kaiser, Jeff Flinn, Jeremy Maitin-Shepard, John Filo, John Maddock, Kim Barrett, Marsh Ray, Martin Bonner, Mathias Gaunard, Matias Capeletto, Neil Mayhew, Paul Bristow, Pierre Talbot, Phil Endecott, Pyry Jahkola, Rene Rivera, Robert Stewart, Roland Schwarz, Scott McMurray, Sebastian Redl, Tim Blechmann, Tim Moore, tymofey, Tomas Puverle, Vincente Botet, Yuval Ronen and -Vitaly Budovski,.

+Vitaly Budovsk. Apologies if anyone has been missed.


Last revised: -17 December, 2014

+19 January, 2015

© Copyright Beman Dawes, 2011, 2013

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

diff --git a/mini_review_topics.html b/mini_review_topics.html index 340baed..d7a7f3c 100644 --- a/mini_review_topics.html +++ b/mini_review_topics.html @@ -9,34 +9,60 @@ -

Here is what needs to be done to get the library ready for a mini-review: + + + + + +
+ +Boost logo + + Endian Mini-Review
+ + + + +
+ Endian Home     + Conversion Functions     + Arithmetic Types     + Buffer Types     + Choosing Approach
+

+

The results of the Boost.Endian formal review included a list of issues to be +resolved before a mini-review.

-

+

The issues are shown in bold below, with the resolution indicated.

-

Common use case scenarios should be developed.

+

1. Common use case scenarios should be developed.

-

Done. See Use cases.

+

Done. The documentation have been refactored. A page is now devoted to + Choosing the Approach to endianness. See + Use cases for use case + scenarios.

-

Example programs should be developed for the common use case scenarios.

+

2. Example programs should be developed for the common use case scenarios.

-

 

+

Done. See Choosing the Approach. + Example code has been added throughout.

-

Documentation should illuminate the differences between endian +

3. Documentation should illuminate the differences between endian integer/float type and endian conversion approaches to the common use case scenarios, and provide guidelines for choosing the most appropriate approach in user's applications.

-

Done. See Choosing between endian conversion - functions, endian buffer types, and endian arithmetic types.

+

Done. See Choosing the Approach.

-

Conversion functions supplying results via return should be provided.

+

4 .Conversion functions supplying results via return should be provided.

-

Done. See conversion docs.

+

Done. See ConversionFunctions.

-

Platform specific performance enhancements such as use of compiler intrinsics or relaxed alignment requirements should be supported.

+

5. Platform specific performance enhancements such as use of compiler intrinsics or relaxed alignment requirements should be supported.

Done. Compiler (Clang, GCC, VisualC++, etc.) intrinsics and built-in functions are used in the implementation where appropriate, as requested. See @@ -44,7 +70,7 @@ approach in user's applications.

Timings for Example 2 to gauge the impact of intrinsics.

-

Endian integer (and floating) types should be implemented via the +

6. Endian integer (and floating) types should be implemented via the conversion functions. If that can't be done efficiently, consideration should be given to expanding the conversion function signatures to resolve the inefficiencies.

@@ -52,7 +78,7 @@ resolve the inefficiencies.

Done. For the endian types, the implementation uses the endian conversion functions, and thus the intrinsics, as requested.

-

Benchmarks that measure performance should be provided. It should be +

7. Benchmarks that measure performance should be provided. It should be possible to compare platform specific performance enhancements against portable base implementations, and to compare endian integer approaches against endian conversion approaches for the common use case scenarios.

@@ -60,7 +86,7 @@ against endian conversion approaches for the common use case scenarios.

Done. See Timings for Example 2. The endian/test directory also contains several additional benchmark and speed test programs.

-

Float (32-bits) and double (64-bits) should be supported. IEEE 754 is +

8. Float (32-bits) and double (64-bits) should be supported. IEEE 754 is the primary use case.

Done. The endian buffer types,  @@ -68,12 +94,13 @@ the primary use case.

endian conversion functions now support 32-bit (float) and 64-bit (double) floating point, as requested.

-

Support for user defined types (UDTs) is desirable, and should be +

9. Support for user defined types (UDTs) is desirable, and should be provided where there would be no conflict with the other concerns.

-

Done. See conversion requirements.

+

Done. See Customization + points for user-defined types (UDTs).

-

There is some concern that endian integer/float arithmetic operations +

10. There is some concern that endian integer/float arithmetic operations might used inadvertently or inappropriately. The impact of adding an endian_buffer class without arithmetic operations should be investigated.

@@ -83,23 +110,37 @@ might used inadvertently or inappropriately. The impact of adding an endian_buff endian_buffer is a public base class for endian_arithmetic, and can also be used by users as a stand-alone class.

-

Stream insertion and extraction of the endian integer/float types should +

11. Stream insertion and extraction of the endian integer/float types should be documented and included in the test coverage.

-

Done. See Stream inserter and - Stream extractor.

+

Done. See Stream inserter and + Stream extractor.

-

Binary I/O support that was investigated during development of the Endian +

12. Binary I/O support that was investigated during development of the Endian library should be put up for mini-review for inclusion in the Boost I/O library.

-

 

+

Not done yet. Will be handled as a separate min-review soon after the + Endian mini-review.

+
+

13. Other requested changes.

+ +
+
    +
  • In addition to the named-endianness conversion functions, functions that perform + compile-time (via template) and run-time (via function argument) dispatch + are now provided.
  • +
  • order::native is now a synonym for order::big + or order::little according to the endianness of the platform. This reduces the number of template specializations required.
  • +
  • Headers have been reorganized to make them easier to read, + with a synopsis at the front and implementation following.
  • +

Last revised: -17 December, 2014

+19 January, 2015

© Copyright Beman Dawes, 2014

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

diff --git a/styles.css b/styles.css index d1b8958..f761bcb 100644 --- a/styles.css +++ b/styles.css @@ -3,6 +3,7 @@ body { font-family: sans-serif; max-width: 6.5in; + margin: 0px auto; font-size: 85%; } ins {background-color: #CCFFCC;} @@ -12,7 +13,7 @@ body table{font-size: 100%;} /* - © Copyright Beman Dawes, 2014 + © Copyright Beman Dawes, 2014 Distributed under the Boost Software License, Version 1.0. See www.boost.org/LICENSE_1_0.txt */ diff --git a/todo_list.html b/todo_list.html index b4f3241..fed67d3 100644 --- a/todo_list.html +++ b/todo_list.html @@ -12,23 +12,9 @@

Endian Library TODO List

-

Last revised: -17 December, 2014

- -

August 12, 2014: The many items that have been completed should be -removed, after verifying that they are in fact taken care of.

To Do

-
    -
  • Develop the use-cases example programs, using plain integers and UDT's.
  • -
  • Review UDT examples.
  • -
  • Review buffer stream extractors and inserters, and how they work for - arithmetic types. TEST. Make sure nothing got dropped on the floor during - buffer decomposition.
  • -
  • Run inspect. -
  • -

Format Review Comments

-

Votes

+

Interesting

  • John Filo - "Absolutely. I'd like to see support for float and @@ -52,26 +38,6 @@ removed, after verifying that they are in fact taken care of.

    Spirit with the reviewed version. All of Spirits regression tests still
    pass. "
     
  • -
  • -

    Robert Stewart - "There are issues that keep me from saying yes - at this time.  There are too many suggested variations and ideas under - consideration to accept the library in its present state.  However, a - mini-review should be sufficient to evaluate the final form, once Beman - determines a course of action, and determine whether to accept it or not."

  • -
  • -

    Tim Blechmann - "the library should be accepted, if
    -
    - (a) the interface of the conversion functions is changed
    - (b) the performance can be improved
    - (c) the documentation integrates better with the rest of the boost
    -    documentation."

  • -
  • -

    Vicente J. Botet Escriba - "No in its current state.
    - Once the library takes in account the requested modification (that Beman has - already accepted) a mini-review will be necessary to improve the library - before release."

Executive summary

    @@ -109,37 +75,6 @@ removed, after verifying that they are in fact taken care of.

Docs

    -
  • Document use of endian integers with stream inserters and extractors.
  • -
  • Conversion in note mention similarity to htonl() , etc.
  • -
  • Conversion: add discussion of alignment, packing, etc. Bottom line; use at - your own risk. Use Phil's example:
    - struct S {
    -  uint16_t a;
    -  uint32_t b;
    - } __attribute__ ((packed));
  • -
  • Requirements for template parameters.
  • -
  • UDTs
      -
    • Integers
    • -
    • Conversion
    • -
    -
  • -
  • Distinguish use cases and recommendations for which approach (integers vs - conversion) is appropriate. -
  • -
  • > section `experience': this section gives no insights for people who use - or
    - > read the code. it mainly tells: "we are not the first and the domain of the
    - > library is important.". imo this section can be removed (maybe the part that
    - > it is not related to any c library can go to the `design considerations'
    - >
    - > section "motivating use cases": this is more a marketing blurb/testimonial.
    - > maybe this could be changed to a section about possible use cases, listing
    - > `communicating between different devices' and `reading/writing of binary - file
    - > formats'.
  • one other point ... the help file seems to directly link to the c++ headers.
    this should be changed:
    @@ -157,75 +92,10 @@ removed, after verifying that they are in fact taken care of.


    so i'd suggest to completely remove the links to the c++ headers.
     
  • -
  • -
    - > explaining the other builting types are not considered. Why only - big/little
    - > endianness has been taken in account?
    -

    I'll add FAQ and/or add more entries to the final docs.
    -
    - Only big/little endianness is taken into account because these are the
    - only endian schemes that have any practical value. All the others are
    - just historical curiosities.

-

Code

-

Also change docs if applicable.

-
    -
  • Beman: TODO: - Google - "unaligned integer", look at various entries. For example, http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka3544.html
  • -
  • Beman: Some platforms (compiler/processor taken together) don't require - alignment for the conversion functions if code is inlined, although speed may - suffer. (test to verify this assertion). On those platforms, conversion - functions (perhaps in-place) can be used to implement unaligned integers. -
  • -
  • Beman: Does the use of the unrolling templates get in the way of processor - specific optimization code?
  • -
  • Test use of endian integers with stream inserters and extractors.
  • -
  • Continue work on benchmarks and timings. Consider using use-case example - programs as benchmarks.
  • -
  • -
    - > The library should provide in addition endian conversion functions that - have
    - > the endiannes as template parameters to make possible generic functions.
    -

    Compile time dispatch on an endianness enum was also requested in
    - another review. That's fine with me, but I haven't had a chance to
    - figure out the interface details.

  • -
-

Infrastructure

-
    -
  • -
    - > endian_operations_test.cpp and endian_in_union_test.cpp ... maybe rename - from
    - > _test.cpp to _compile_test.cpp? they don't seem to do any run-time tests. - they
    - > also should not include <cassert> since no assertion statement is needed, - this
    - > might speed up the compilation time of the testsuite by something like - 50ms ;)
    -

    Will do.

  • -
  • Make the bin() functionality available
  • -
- -

* I'm only willing to provide conversion.hpp FP support. Providing
-types that mimic FP types is far beyond my knowledge of how to deal
-with floating point's notorious arithmetic issues.

-

Support IEEE754 format (32 bit, 64 bit) only.


Last revised: -17 December, 2014

+17 January, 2015

© Copyright Beman Dawes, 2012

Distributed under the Boost Software License, Version 1.0. See www.boost.org/LICENSE_1_0.txt