From 0427a954597fb2708435239c1c3e03ae2e778c75 Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 10:28:32 -0400 Subject: [PATCH 01/12] Convert Endian documentation to asciidoc --- doc/.gitignore | 2 + doc/Jamfile | 29 +++ doc/endian-docinfo-footer.html | 4 + doc/endian.adoc | 30 +++ doc/endian/overview.adoc | 443 +++++++++++++++++++++++++++++++++ 5 files changed, 508 insertions(+) create mode 100644 doc/.gitignore create mode 100644 doc/Jamfile create mode 100644 doc/endian-docinfo-footer.html create mode 100644 doc/endian.adoc create mode 100644 doc/endian/overview.adoc diff --git a/doc/.gitignore b/doc/.gitignore new file mode 100644 index 0000000..0972e2d --- /dev/null +++ b/doc/.gitignore @@ -0,0 +1,2 @@ +/html/ +/pdf/ diff --git a/doc/Jamfile b/doc/Jamfile new file mode 100644 index 0000000..1e3b0cf --- /dev/null +++ b/doc/Jamfile @@ -0,0 +1,29 @@ +# Copyright 2019 Glen Joseph Fernandes +# (glenjofe@gmail.com) +# +# Distributed under the Boost Software License, Version 1.0. +# (http://www.boost.org/LICENSE_1_0.txt) + +project doc/endian ; + +import asciidoctor ; + +html endian.html : endian.adoc ; + +install html_ : endian.html : html ; + +pdf endian.pdf : endian.adoc ; + +explicit endian.pdf ; + +install pdf_ : endian.pdf : pdf ; + +explicit pdf_ ; + +alias boostdoc ; + +explicit boostdoc ; + +alias boostrelease : html_ ; + +explicit boostrelease ; diff --git a/doc/endian-docinfo-footer.html b/doc/endian-docinfo-footer.html new file mode 100644 index 0000000..bb13138 --- /dev/null +++ b/doc/endian-docinfo-footer.html @@ -0,0 +1,4 @@ + diff --git a/doc/endian.adoc b/doc/endian.adoc new file mode 100644 index 0000000..021d31a --- /dev/null +++ b/doc/endian.adoc @@ -0,0 +1,30 @@ +//// +Copyright 2019 Glen Joseph Fernandes +(glenjofe@gmail.com) + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +# Boost.Endian: The Boost Endian Library +Beman Dawes +:toc: left +:toclevels: 2 +:idprefix: +:listing-caption: Code Example +:docinfo: private-footer + +:leveloffset: +1 + +include::endian/overview.adoc[] + +:leveloffset: -1 + +[appendix] +## Copyright and License + +This documentation is + +* Copyright 2011-2016 Beman Dawes + +and is distributed under the http://www.boost.org/LICENSE_1_0.txt[Boost Software License, Version 1.0]. diff --git a/doc/endian/overview.adoc b/doc/endian/overview.adoc new file mode 100644 index 0000000..26fd2bd --- /dev/null +++ b/doc/endian/overview.adoc @@ -0,0 +1,443 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#overview] +# Overview + +## Abstract + +Boost.Endian provides facilities to manipulate the +<> of integers and user-defined types. + +* Three approaches to endianness are supported. Each has a long history of +successful use, and each approach has use cases where it is preferred over the +other approaches. +* Primary uses: +** Data portability. The Endian library supports binary data exchange, via +either external media or network transmission, regardless of platform +endianness. +** Program portability. POSIX-based and Windows-based operating systems +traditionally supply libraries with non-portable functions to perform endian +conversion. There are at least four incompatible sets of functions in common +use. The Endian library is portable across all {cpp} platforms. +* Secondary use: Minimizing data size via sizes and/or alignments not supported +by the standard {cpp} integer types. + +[#overview_endianness] +## Introduction to endianness + +Consider the following code: + +``` +int16_t i = 0x0102; +FILE * file = fopen("test.bin", "wb"); // binary file! +fwrite(&i, sizeof(int16_t), 1, file); +fclose(file); +``` + +On OS X, Linux, or Windows systems with an Intel CPU, a hex dump of the +"test.bin" output file produces: + +``` +0201 +``` + +On OS X systems with a PowerPC CPU, or Solaris systems with a SPARC CPU, a hex +dump of the "test.bin" output file produces: + +``` +0102 +``` + +What's happening here is that Intel CPUs order the bytes of an integer with the +least-significant byte first, while SPARC CPUs place the most-significant byte +first. Some CPUs, such as the PowerPC, allow the operating system to choose +which ordering applies. + +Most-significant-byte-first ordering is traditionally called "big endian" +ordering and least-significant-byte-first is traditionally called +"little-endian" ordering. The names are derived from +http://en.wikipedia.org/wiki/Jonathan_Swift[Jonathan Swift]'s satirical novel +_http://en.wikipedia.org/wiki/Gulliver's_Travels[Gulliver's Travels]_, where +rival kingdoms opened their soft-boiled eggs at different ends. + +See Wikipedia's http://en.wikipedia.org/wiki/Endianness[Endianness] article for +an extensive discussion of endianness. + +Programmers can usually ignore endianness, except when reading a core dump on +little-endian systems. But programmers have to deal with endianness when +exchanging binary integers and binary floating point values between computer +systems with differing endianness, whether by physical file transfer or over a +network. And programmers may also want to use the library when minimizing either +internal or external data sizes is advantageous. + +[#overview_introduction] +## Introduction to the Boost.Endian library + +Boost.Endian provides three different approaches to dealing with endianness. All +three approaches support integers and user-define types (UDTs). + +Each approach has a long history of successful use, and each approach has use +cases where it is preferred to the other approaches. + +<>:: +The application uses the built-in integer types to hold values, and calls the +provided conversion functions to convert byte ordering as needed. Both mutating +and non-mutating conversions are supplied, and each comes in unconditional and +conditional variants. + +<>:: +The application uses the provided endian buffer types to hold values, and +explicitly converts to and from the built-in integer types. Buffer sizes of 8, +16, 24, 32, 40, 48, 56, and 64 bits (i.e. 1, 2, 3, 4, 5, 6, 7, and 8 bytes) are +provided. Unaligned integer buffer types are provided for all sizes, and aligned +buffer types are provided for 16, 32, and 64-bit sizes. The provided specific +types are typedefs for a generic class template that may be used directly for +less common use cases. + +<>:: +The application uses the provided endian arithmetic types, which supply the same +operations as the built-in {cpp} arithmetic types. All conversions are implicit. +Arithmetic sizes of 8, 16, 24, 32, 40, 48, 56, and 64 bits (i.e. 1, 2, 3, 4, 5, +6, 7, and 8 bytes) are provided. Unaligned integer types are provided for all +sizes and aligned arithmetic types are provided for 16, 32, and 64-bit sizes. +The provided specific types are typedefs for a generic class template that may +be used directly in generic code of for less common use cases. + +Boost Endian is a header-only library. {cpp}11 features affecting interfaces, +such as `noexcept`, are used only if available. See +<> for details. + +## Choosing between conversion functions, buffer types, and arithmetic types + +This section has been moved to its own <> page. + +[#overview_intrinsic] +## Built-in support for Intrinsics + +Most compilers, including GCC, Clang, and Visual {cpp}, supply built-in support +for byte swapping intrinsics. The Endian library uses these intrinsics when +available since they may result in smaller and faster generated code, +particularly for optimized builds. + +Defining the macro `BOOST_ENDIAN_NO_INTRINSICS` will suppress use of the +intrinsics. This is useful when a compiler has no intrinsic support or fails to +locate the appropriate header, perhaps because it is an older release or has +very limited supporting libraries. + +The macro `BOOST_ENDIAN_INTRINSIC_MSG` is defined as either +`"no byte swap intrinsics"` or a string describing the particular set of +intrinsics being used. This is useful for eliminating missing intrinsics as a +source of performance issues. + +## Performance + +Consider this problem: + +### Example 1 +Add 100 to a big endian value in a file, then write the result to a file +[%header,cols=2*] +|=== +|Endian arithmetic type approach |Endian conversion function approach +a| +---- +big_int32_at x; + +... read into x from a file ... + +x += 100; + +... write x to a file ... +---- +a| +---- +int32_t x; + +... read into x from a file ... + +big_to_native_inplace(x); +x += 100; +native_to_big_inplace(x); + +... write x to a file ... +---- +|=== + +*There will be no performance difference between the two approaches in optimized +builds, regardless of the native endianness of the machine.* That's because +optimizing compilers will generate exactly the same code for each. That +conclusion was confirmed by studying the generated assembly code for GCC and +Visual {cpp}. Furthermore, time spent doing I/O will determine the speed of this +application. + +Now consider a slightly different problem: + +### Example 2 +Add a million values to a big endian value in a file, then write the result to a +file +[%header,cols=2*] +|=== +|Endian arithmetic type approach |Endian conversion function approach +a| +---- +big_int32_at x; + +... read into x from a file ... + +for (int32_t i = 0; i < 1000000; ++i) + x += i; + +... write x to a file ... +---- +a| +---- +int32_t x; + +... read into x from a file ... + +big_to_native_inplace(x); + +for (int32_t i = 0; i < 1000000; ++i) + x += i; + +native_to_big_inplace(x); + +... write x to a file ... +---- +|=== + +With the Endian arithmetic approach, on little endian platforms an implicit +conversion from and then back to big endian is done inside the loop. With the +Endian conversion function approach, the user has ensured the conversions are +done outside the loop, so the code may run more quickly on little endian +platforms. + +### Timings + +These tests were run against release builds on a circa 2012 4-core little endian +X64 Intel Core i5-3570K CPU @ 3.40GHz under Windows 7. + +CAUTION: The Windows CPU timer has very high granularity. Repeated runs of the +same tests often yield considerably different results. + +See `test/loop_time_test.cpp` for the actual code and `benchmark/Jamfile.v2` for +the build setup. + +#### GNU C++ version 4.8.2 on Linux virtual machine +Iterations: 10'000'000'000, Intrinsics: __builtin_bswap16, etc. +[%header,cols=3*] +|=== +|Test Case |Endian arithmetic type |Endian conversion function +|16-bit aligned big endian |8.46 s |5.28 s +|16-bit aligned little endian |5.28 s |5.22 s +|32-bit aligned big endian |8.40 s |2.11 s +|32-bit aligned little endian |2.11 s |2.10 s +|64-bit aligned big endian |14.02 s |3.10 s +|64-bit aligned little endian |3.00 s |3.03 s +|=== + +#### Microsoft Visual C++ version 14.0 +Iterations: 10'000'000'000, Intrinsics: cstdlib _byteswap_ushort, etc. +[%header,cols=3*] +|=== +|Test Case |Endian arithmetic type |Endian conversion function +|16-bit aligned big endian |8.27 s |5.26 s +|16-bit aligned little endian |5.29 s |5.32 s +|32-bit aligned big endian |8.36 s |5.24 s +|32-bit aligned little endian |5.24 s |5.24 s +|64-bit aligned big endian |13.65 s |3.34 s +|64-bit aligned little endian |3.35 s |2.73 s +|=== + +## Overall FAQ + +Is the implementation header only?:: +Yes. + +Are {cpp}03 compilers supported?:: +Yes. + +Does the implementation use compiler intrinsic built-in byte swapping?:: +Yes, if available. See <>. + +Why bother with endianness?:: +Binary data portability is the primary use case. + +Does endianness have any uses outside of portable binary file or network I/O formats?:: +Using the unaligned integer types with a size tailored to the application's +needs is a minor secondary use that saves internal or external memory space. For +example, using `big_int40_buf_t` or `big_int40_t` in a large array saves a lot +of space compared to one of the 64-bit types. + +Why bother with binary I/O? Why not just use {cpp} Standard Library stream inserters and extractors?:: +* Data interchange formats often specify binary integer data. Binary integer +data is smaller and therefore I/O is faster and file sizes are smaller. Transfer +between systems is less expensive. +* Furthermore, binary integer data is of fixed size, and so fixed-size disk +records are possible without padding, easing sorting and allowing random access. +* Disadvantages, such as the inability to use text utilities on the resulting +files, limit usefulness to applications where the binary I/O advantages are +paramount. + +Which is better, big-endian or little-endian?:: +Big-endian tends to be preferred in a networking environment and is a bit more +of an industry standard, but little-endian may be preferred for applications +that run primarily on x86, x86-64, and other little-endian CPU's. The +http://en.wikipedia.org/wiki/Endian[Wikipedia] article gives more pros and cons. + +Why are only big and little native endianness supported?:: +These are the only endian schemes that have any practical value today. PDP-11 +and the other middle endian approaches are interesting curiosities but have no +relevance for today's {cpp} developers. The same is true for architectures that +allow runtime endianness switching. The +<> has +been carefully crafted to allow support for such orderings in the future, should +the need arise. Thanks to Howard Hinnant for suggesting this. + +Why do both the buffer and arithmetic types exist?:: +Conversions in the buffer types are explicit. Conversions in the arithmetic +types are implicit. This fundamental difference is a deliberate design feature +that would be lost if the inheritance hierarchy were collapsed. +The original design provided only arithmetic types. Buffer types were requested +during formal review by those wishing total control over when conversion occurs. +They also felt that buffer types would be less likely to be misused by +maintenance programmers not familiar with the implications of performing a lot +of integer operations on the endian arithmetic integer types. + +What is gained by using the buffer types rather than always just using the arithmetic types?:: +Assurance that hidden conversions are not performed. This is of overriding +importance to users concerned about achieving the ultimate in terms of speed. +"Always just using the arithmetic types" is fine for other users. When the +ultimate in speed needs to be ensured, the arithmetic types can be used in the +same design patterns or idioms that would be used for buffer types, resulting in +the same code being generated for either types. + +What are the limitations of integer support?:: +Tests have only been performed on machines that use two's complement +arithmetic. The Endian conversion functions only support 16, 32, and 64-bit +aligned integers. The endian types only support 8, 16, 24, 32, 40, 48, 56, and +64-bit unaligned integers, and 8, 16, 32, and 64-bit aligned integers. + +Why is there no floating point support?:: +An attempt was made to support four-byte ``float``s and eight-byte +``double``s, limited to +http://en.wikipedia.org/wiki/IEEE_floating_point[IEEE 754] (also known as +ISO/IEC/IEEE 60559) floating point and further limited to systems where floating +point endianness does not differ from integer endianness. Even with those +limitations, support for floating point types was not reliable and was removed. +For example, simply reversing the endianness of a floating point number can +result in a signaling-NAN. For all practical purposes, binary serialization and +endianness for integers are one and the same problem. That is not true for +floating point numbers, so binary serialization interfaces and formats for +floating point does not fit well in an endian-based library. + + +## Release history + +### Changes requested by formal review + +The library was reworked from top to bottom to accommodate changes requested +during the formal review. See <> +page for details. + +### Other changes since formal review + +* Header `boost/endian/endian.hpp` has been renamed to +`boost/endian/arithmetic.hpp`. Headers +`boost/endian/conversion.hpp` and `boost/endian/buffers.hpp` have been added. +Infrastructure file names were changed accordingly. +* The endian arithmetic type aliases have been renamed, using a naming pattern +that is consistent for both integer and floating point, and a consistent set of +aliases supplied for the endian buffer types. +* The unaligned-type alias names still have the `_t` suffix, but the +aligned-type alias names now have an `_at` suffix. +* `endian_reverse()` overloads for `int8_t` and `uint8_t` have been added for +improved generality. (Pierre Talbot) +* Overloads of `endian_reverse_inplace()` have been replaced with a single +`endian_reverse_inplace()` template. (Pierre Talbot) +* For X86 and X64 architectures, which permit unaligned loads and stores, +unaligned little endian buffer and arithmetic types use regular loads and +stores when the size is exact. This makes unaligned little endian buffer and +arithmetic types significantly more efficient on these architectures. (Jeremy +Maitin-Shepard) +* {cpp}11 features affecting interfaces, such as `noexcept`, are now used. +{cpp}03 compilers are still supported. +* Acknowledgements have been updated. + +## Compatibility with interim releases + +Prior to the official Boost release, class template `endian_arithmetic` has been +used for a decade or more with the same functionality but under the name +`endian`. Other names also changed in the official release. If the macro +`BOOST_ENDIAN_DEPRECATED_NAMES` is defined, those old now deprecated names are +still supported. However, the class template `endian` name is only provided for +compilers supporting {cpp}11 template aliases. For {cpp}03 compilers, the name +will have to be changed to `endian_arithmetic`. + +To support backward header compatibility, deprecated header +`boost/endian/endian.hpp` forwards to `boost/endian/arithmetic.hpp`. It requires +`BOOST_ENDIAN_DEPRECATED_NAMES` be defined. It should only be used while +transitioning to the official Boost release of the library as it will be removed +in some future release. + +## {cpp}03 support for {cpp}11 features + +[%header,cols=2*] +|=== +|{cpp}11 Feature +|Action with {cpp}03 Compilers +|Scoped enums +|Uses header +http://www.boost.org/libs/core/doc/html/core/scoped_enum.html[boost/core/scoped_enum.hpp] +to emulate {cpp}11 scoped enums. +|`noexcept` +|Uses `BOOST_NOEXCEPT` macro, which is defined as null for compilers not +supporting this {cpp}11 feature. +|{cpp}11 PODs +(http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2342.htm[N2342]) +|Takes advantage of {cpp}03 compilers that relax {cpp}03 POD rules, but see +Limitations <> and <>. +Also see macros for explicit POD control <> and +<> +|=== + +## Future directions + +Standardization.:: +The plan is to submit Boost.Endian to the {cpp} standards committee for possible +inclusion in a Technical Specification or the {cpp} standard itself. + +Specializations for `numeric_limits`.:: +Roger Leigh requested that all `boost::endian` types provide `numeric_limits` +specializations. +See https://github.com/boostorg/endian/issues/4[GitHub issue 4]. + +Character buffer support.:: +Peter Dimov pointed out during the mini-review that getting and setting basic +arithmetic types (or `` equivalents) from/to an offset into an array of +unsigned char is a common need. See +http://lists.boost.org/Archives/boost/2015/01/219574.php[Boost.Endian +mini-review posting]. + +Out-of-range detection.:: +Peter Dimov pointed suggested during the mini-review that throwing an exception +on buffer values being out-of-range might be desirable. See the end of +http://lists.boost.org/Archives/boost/2015/01/219659.php[this posting] and +subsequent replies. + +## Acknowledgements + +Comments and suggestions were received from Adder, Benaka Moorthi, Christopher +Kohlhoff, Cliff Green, Daniel James, Dave Handley, Gennaro Proto, Giovanni Piero +Deretta, Gordon Woodhull, dizzy, Hartmut Kaiser, Howard Hinnant, Jason Newton, +Jeff Flinn, Jeremy Maitin-Shepard, John Filo, John Maddock, Kim Barrett, Marsh +Ray, Martin Bonner, Mathias Gaunard, Matias Capeletto, Neil Mayhew, Nevin Liber, +Olaf van der Spek, Paul Bristow, Peter Dimov, Pierre Talbot, Phil Endecott, +Philip Bennefall, Pyry Jahkola, Rene Rivera, Robert Stewart, Roger Leigh, Roland +Schwarz, Scott McMurray, Sebastian Redl, Tim Blechmann, Tim Moore, tymofey, +Tomas Puverle, Vincente Botet, Yuval Ronen and Vitaly Budovsk. Apologies if +anyone has been missed. From 01278f951d21255b7dca3e67aa86680888f833d0 Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 14:22:36 -0400 Subject: [PATCH 02/12] Convert Endian mini review to asciidoc --- doc/endian.adoc | 2 + doc/endian/mini_review_topics.adoc | 96 ++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+) create mode 100644 doc/endian/mini_review_topics.adoc diff --git a/doc/endian.adoc b/doc/endian.adoc index 021d31a..24dd548 100644 --- a/doc/endian.adoc +++ b/doc/endian.adoc @@ -18,6 +18,8 @@ Beman Dawes include::endian/overview.adoc[] +include::endian/mini_review_topics.adoc[] + :leveloffset: -1 [appendix] diff --git a/doc/endian/mini_review_topics.adoc b/doc/endian/mini_review_topics.adoc new file mode 100644 index 0000000..c42604f --- /dev/null +++ b/doc/endian/mini_review_topics.adoc @@ -0,0 +1,96 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#appendix_mini_review_topics] +[appendix] +# Endian Mini-Review + +The results of the Boost.Endian formal review included a list of issues to be +resolved before a mini-review. + +The issues are shown in *bold* below, with the resolution indicated. + +1. *Common use case scenarios should be developed.* +* Done. The documentation have been refactored. A page is now devoted to +<> to endianness. See +<> for use case scenarios. + +2. *Example programs should be developed for the common use case scenarios.* +* Done. See <>. Example code has been added +throughout. + +3. *Documentation should illuminate the differences between endian +integer/float type and endian conversion approaches to the common use case +scenarios, and provide guidelines for choosing the most appropriate approach in +user's applications.* +* Done. See <>. + +4. *Conversion functions supplying results via return should be provided.* +* Done. See <>. + +5. *Platform specific performance enhancements such as use of compiler +intrinsics or relaxed alignment requirements should be supported.* +* Done. Compiler (Clang, GCC, Visual{cpp}, etc.) intrinsics and built-in +functions are used in the implementation where appropriate, as requested. See +<>. See +<> to gauge the impact of intrinsics. + +6. *Endian integer (and floating) types should be implemented via the +conversion functions. If that can't be done efficiently, consideration should +be given to expanding the conversion function signatures to resolve the +inefficiencies.* +* Done. For the endian types, the implementation uses the endian conversion +functions, and thus the intrinsics, as requested. + +7. *Benchmarks that measure performance should be provided. It should be +possible to compare platform specific performance enhancements against portable +base implementations, and to compare endian integer approaches against endian +conversion approaches for the common use case scenarios.* +* Done. See <>. The `endian/test` +directory also contains several additional benchmark and speed test programs. + +8. *Float (32-bits) and double (64-bits) should be supported. IEEE 754 is +the primary use case.* +* Done. The <>, +<> and +<> now support 32-bit `(float)` +and 64-bit `(double)` floating point, as requested. + +9. *Support for user defined types (UDTs) is desirable, and should be +provided where there would be no conflict with the other concerns.* +* Done. See <>. + +10. *There is some concern that endian integer/float arithmetic operations +might used inadvertently or inappropriately. The impact of adding an +endian_buffer class without arithmetic operations should be investigated.* +* Done. The endian types have been decomposed into class template +`<>` and class template +`<>`. Class `endian_buffer` is a public base +class for `endian_arithmetic`, and can also be used by users as a stand-alone +class. + +11. *Stream insertion and extraction of the endian integer/float types should +be documented and included in the test coverage.* +* Done. See <> and +<>. + +12. *Binary I/O support that was investigated during development of the +Endian library should be put up for mini-review for inclusion in the Boost I/O +library.* +* Not done yet. Will be handled as a separate min-review soon after the Endian +mini-review. + +13. *Other requested changes.* +* In addition to the named-endianness conversion functions, functions that +perform compile-time (via template) and run-time (via function argument) +dispatch are now provided. +* `order*native` is now a synonym for `order*big` or `order*little` according +to the endianness of the platform. This reduces the number of template +specializations required. +* Headers have been reorganized to make them easier to read, with a synopsis +at the front and implementation following.s From cd17067afbb9536a258a199a5c697cde457a712b Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 15:44:14 -0400 Subject: [PATCH 03/12] Convert Choosing approach to asiidoc --- doc/endian.adoc | 2 + doc/endian/choosing_approach.adoc | 317 ++++++++++++++++++++++++++++++ 2 files changed, 319 insertions(+) create mode 100644 doc/endian/choosing_approach.adoc diff --git a/doc/endian.adoc b/doc/endian.adoc index 24dd548..1f4911c 100644 --- a/doc/endian.adoc +++ b/doc/endian.adoc @@ -18,6 +18,8 @@ Beman Dawes include::endian/overview.adoc[] +include::endian/choosing_approach.adoc[] + include::endian/mini_review_topics.adoc[] :leveloffset: -1 diff --git a/doc/endian/choosing_approach.adoc b/doc/endian/choosing_approach.adoc new file mode 100644 index 0000000..d9d0dc7 --- /dev/null +++ b/doc/endian/choosing_approach.adoc @@ -0,0 +1,317 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#choosing] +# Choosing Approach + +## Introduction + +Deciding which is the best endianness approach (conversion functions, buffer +types, or arithmetic types) for a particular application involves complex +engineering trade-offs. It is hard to assess those trade-offs without some +understanding of the different interfaces, so you might want to read the +<>, <>, and +<> pages before diving into this page. + +## Choosing between conversion functions, buffer types, and arithmetic types + +The best approach to endianness for a particular application depends on the +interaction between the application's needs and the characteristics of each of +the three approaches. + +*Recommendation:* If you are new to endianness, uncertain, or don't want to +invest the time to study engineering trade-offs, use +<>. They are safe, easy to use, and easy to +maintain. Use the _<>_ design +pattern locally around performance hot spots like lengthy loops, if needed. + +### Background + +A dealing with endianness usually implies a program portability or a data +portability requirement, and often both. That means real programs dealing with +endianness are usually complex, so the examples shown here would really be +written as multiple functions spread across multiple translation units. They +would involve interfaces that can not be altered as they are supplied by +third-parties or the standard library. + +### Characteristics + +The characteristics that differentiate the three approaches to endianness are +the endianness invariants, conversion explicitness, arithmetic operations, sizes +available, and alignment requirements. + +#### Endianness invariants + +*Endian conversion functions* use objects of the ordinary {cpp} arithmetic types +like `int` or `unsigned short` to hold values. That breaks the implicit +invariant that the {cpp} language rules apply. The usual language rules only apply +if the endianness of the object is currently set to the native endianness for +the platform. That can make it very hard to reason about logic flow, and result +in difficult to find bugs. + +For example: + +``` +struct data_t // big endian +{ + int32_t v1; // description ... + int32_t v2; // description ... + ... additional character data members (i.e. non-endian) + int32_t v3; // description ... +}; + +data_t data; + +read(data); +big_to_native_inplace(data.v1); +big_to_native_inplace(data.v2); + +... + +++v1; +third_party::func(data.v2); + +... + +native_to_big_inplace(data.v1); +native_to_big_inplace(data.v2); +write(data); +``` + +The programmer didn't bother to convert `data.v3` to native endianness because +that member isn't used. A later maintainer needs to pass `data.v3` to the +third-party function, so adds `third_party::func(data.v3);` somewhere deep in +the code. This causes a silent failure because the usual invariant that an +object of type `int32_t` holds a value as described by the {cpp} core language +does not apply. + +*Endian buffer and arithmetic types* hold values internally as arrays of +characters with an invariant that the endianness of the array never changes. +That makes these types easier to use and programs easier to maintain. + +Here is the same example, using an endian arithmetic type: + +``` +struct data_t +{ + big_int32_t v1; // description ... + big_int32_t v2; // description ... + ... additional character data members (i.e. non-endian) + big_int32_t v3; // description ... +}; + +data_t data; + +read(data); + +... + +++v1; +third_party::func(data.v2); + +... + +write(data); +``` + +A later maintainer can add `third_party::func(data.v3)` and it will just-work. + +#### Conversion explicitness + +*Endian conversion functions* and *buffer types* never perform implicit +conversions. This gives users explicit control of when conversion occurs, and +may help avoid unnecessary conversions. + +*Endian arithmetic types* perform conversion implicitly. That makes these types +very easy to use, but can result in unnecessary conversions. Failure to hoist +conversions out of inner loops can bring a performance penalty. + +#### Arithmetic operations + +*Endian conversion functions* do not supply arithmetic operations, but this is +not a concern since this approach uses ordinary {cpp} arithmetic types to hold +values. + +*Endian buffer types* do not supply arithmetic operations. Although this +approach avoids unnecessary conversions, it can result in the introduction of +additional variables and confuse maintenance programmers. + +*Endian arithmetic types* do supply arithmetic operations. They are very easy to +use if lots of arithmetic is involved. + +### Sizes + +*Endianness conversion functions* only support 1, 2, 4, and 8 byte integers. +That's sufficient for many applications. + +*Endian buffer and arithmetic types* support 1, 2, 3, 4, 5, 6, 7, and 8 byte +integers. For an application where memory use or I/O speed is the limiting +factor, using sizes tailored to application needs can be useful. + +#### Alignments + +*Endianness conversion functions* only support aligned integer and +floating-point types. That's sufficient for most applications. + +*Endian buffer and arithmetic types* support both aligned and unaligned +integer and floating-point types. Unaligned types are rarely needed, but when +needed they are often very useful and workarounds are painful. For example: + +Non-portable code like this: + +``` +struct S { + uint16_t a; // big endian + uint32_t b; // big endian +} __attribute__ ((packed)); +``` + +Can be replaced with portable code like this: + +``` +struct S { + big_uint16_ut a; + big_uint32_ut b; +}; +``` + +### Design patterns + +Applications often traffic in endian data as records or packets containing +multiple endian data elements. For simplicity, we will just call them records. + +If desired endianness differs from native endianness, a conversion has to be +performed. When should that conversion occur? Three design patterns have +evolved. + +#### Convert only as needed (i.e. lazy) + +This pattern defers conversion to the point in the code where the data +element is actually used. + +This pattern is appropriate when which endian element is actually used varies +greatly according to record content or other circumstances + +[#choosing_anticipating_need] +#### Convert in anticipation of need + +This pattern performs conversion to native endianness in anticipation of use, +such as immediately after reading records. If needed, conversion to the output +endianness is performed after all possible needs have passed, such as just +before writing records. + +One implementation of this pattern is to create a proxy record with endianness +converted to native in a read function, and expose only that proxy to the rest +of the implementation. If a write function, if needed, handles the conversion +from native to the desired output endianness. + +This pattern is appropriate when all endian elements in a record are typically +used regardless of record content or other circumstances. + +#### Convert only as needed, except locally in anticipation of need + +This pattern in general defers conversion but for specific local needs does +anticipatory conversion. Although particularly appropriate when coupled with the +endian buffer or arithmetic types, it also works well with the conversion +functions. + +Example: + +[subs=+quotes] +``` +struct data_t +{ + big_int32_t v1; + big_int32_t v2; + big_int32_t v3; +}; + +data_t data; + +read(data); + +... +++v1; +... + +int32_t v3_temp = data.v3; // hoist conversion out of loop + +for (int32_t i = 0; i < `large-number`; ++i) +{ + ... `lengthy computation that accesses v3_temp` ... +} +data.v3 = v3_temp; + +write(data); +``` + +In general the above pseudo-code leaves conversion up to the endian arithmetic +type `big_int32_t`. But to avoid conversion inside the loop, a temporary is +created before the loop is entered, and then used to set the new value of +`data.v3` after the loop is complete. + +Question: Won't the compiler's optimizer hoist the conversion out of the loop +anyhow? + +Answer: V{cpp} 2015 Preview, and probably others, does not, even for a toy test +program. Although the savings is small (two register `bswap` instructions), the +cost might be significant if the loop is repeated enough times. On the other +hand, the program may be so dominated by I/O time that even a lengthy loop will +be immaterial. + +### Use case examples + +#### Porting endian unaware codebase + +An existing codebase runs on big endian systems. It does not currently deal +with endianness. The codebase needs to be modified so it can run on little +endian systems under various operating systems. To ease transition and protect +value of existing files, external data will continue to be maintained as big +endian. + +The <> is recommended to meet these +needs. A relatively small number of header files dealing with binary I/O layouts +need to change types. For example, `short` or `int16_t` would change to +`big_int16_t`. No changes are required for `.cpp` files. + +#### Porting endian aware codebase + +An existing codebase runs on little-endian Linux systems. It already deals with +endianness via +http://man7.org/linux/man-pages/man3/endian.3.html[Linux provided functions]. +Because of a business merger, the codebase has to be quickly modified for +Windows and possibly other operating systems, while still supporting Linux. The +codebase is reliable and the programmers are all well-aware of endian issues. + +These factors all argue for an <> that +just mechanically changes the calls to `htobe32`, etc. to +`boost::endian::native_to_big`, etc. and replaces `` with +``. + +#### Reliability and arithmetic-speed + +A new, complex, multi-threaded application is to be developed that must run +on little endian machines, but do big endian network I/O. The developers believe +computational speed for endian variable is critical but have seen numerous bugs +result from inability to reason about endian conversion state. They are also +worried that future maintenance changes could inadvertently introduce a lot of +slow conversions if full-blown endian arithmetic types are used. + +The <> approach is made-to-order for this use case. + +#### Reliability and ease-of-use + +A new, complex, multi-threaded application is to be developed that must run on +little endian machines, but do big endian network I/O. The developers believe +computational speed for endian variables is *not critical* but have seen +numerous bugs result from inability to reason about endian conversion state. +They are also concerned about ease-of-use both during development and long-term +maintenance. + +Removing concern about conversion speed and adding concern about ease-of-use +tips the balance strongly in favor the +<>. From 607cc7da73a0cd467366b529963cd31dfc6fe8e5 Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 19:38:16 -0400 Subject: [PATCH 04/12] Convert Endian Conversion Functions to asciidoc --- doc/endian.adoc | 2 + doc/endian/conversion.adoc | 341 +++++++++++++++++++++++++++++++++++++ 2 files changed, 343 insertions(+) create mode 100644 doc/endian/conversion.adoc diff --git a/doc/endian.adoc b/doc/endian.adoc index 1f4911c..8f09fd2 100644 --- a/doc/endian.adoc +++ b/doc/endian.adoc @@ -18,6 +18,8 @@ Beman Dawes include::endian/overview.adoc[] +include::endian/conversion.adoc[] + include::endian/choosing_approach.adoc[] include::endian/mini_review_topics.adoc[] diff --git a/doc/endian/conversion.adoc b/doc/endian/conversion.adoc new file mode 100644 index 0000000..30a955a --- /dev/null +++ b/doc/endian/conversion.adoc @@ -0,0 +1,341 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#conversion] +# Endian Conversion Functions + +## Introduction + +Header `boost/endian/conversion.hpp` provides byte order reversal and conversion +functions that convert objects of the built-in integer types between native, +big, or little endian byte ordering. User defined types are also supported. + +## Reference + +Functions are implemented `inline` if appropriate. For {cpp}03 compilers, +`noexcept` is elided. Boost scoped enum emulation is used so that the library +still works for compilers that do not support scoped enums. + +### Definitions + +*Endianness* refers to the ordering of bytes within internal or external +integers and other arithmetic data. Most-significant byte first is called +*big endian* ordering. Least-significant byte first is called +*little endian* ordering. Other orderings are possible and some CPU +architectures support both big and little ordering. + +NOTE: The names are derived from +http://en.wikipedia.org/wiki/Jonathan_Swift[Jonathan Swift]'s satirical novel +_http://en.wikipedia.org/wiki/Gulliver's_Travels[Gulliver's Travels]_, where +rival kingdoms opened their soft-boiled eggs at different ends. Wikipedia has an +extensive description of https://en.wikipedia.org/wiki/Endianness[Endianness]. + +The standard integral types ({cpp}std 3.9.1) except `bool` are collectively +called the *endian types*. + +### Header `` Synopsis + +[subs=+quotes] +``` +#define BOOST_ENDIAN_INTRINSIC_MSG \ + "`message describing presence or absence of intrinsics`" + +namespace boost +{ +namespace endian +{ + enum class order + { + native = `see below`, + big = `see below`, + little = `see below`, + }; + + int8_t endian_reverse(int8_t x) noexcept; + int16_t endian_reverse(int16_t x) noexcept; + int32_t endian_reverse(int32_t x) noexcept; + int64_t endian_reverse(int64_t x) noexcept; + uint8_t endian_reverse(uint8_t x) noexcept; + uint16_t endian_reverse(uint16_t x) noexcept; + uint32_t endian_reverse(uint32_t x) noexcept; + uint64_t endian_reverse(uint64_t x) noexcept; + + template + EndianReversible big_to_native(EndianReversible x) noexcept; + template + EndianReversible native_to_big(EndianReversible x) noexcept; + template + EndianReversible little_to_native(EndianReversible x) noexcept; + template + EndianReversible native_to_little(EndianReversible x) noexcept; + template + EndianReversible conditional_reverse(EndianReversible x) noexcept; + template + EndianReversible conditional_reverse(EndianReversible x, + order order1, order order2) noexcept; + + template + void endian_reverse_inplace(EndianReversible& x) noexcept; + + template + void big_to_native_inplace(EndianReversibleInplace& x) noexcept; + template + void native_to_big_inplace(EndianReversibleInplace& x) noexcept; + template + void little_to_native_inplace(EndianReversibleInplace& x) noexcept; + template + void native_to_little_inplace(EndianReversibleInplace& x) noexcept; + template + void conditional_reverse_inplace(EndianReversibleInplace& x) noexcept; + template + void conditional_reverse_inplace(EndianReversibleInplace& x, + order order1, order order2) noexcept; + +} // namespace endian +} // namespace boost +``` + +The values of `order::little` and `order::big` shall not be equal to one +another. + +The value of `order::native` shall be: + +* equal to `order::big` if the execution environment is big endian, otherwise +* equal to `order::little` if the execution environment is little endian, +otherwise +* unequal to both `order::little` and `order::big`. + +### Requirements + +#### Template argument requirements + +The template definitions in the `boost/endian/conversion.hpp` header refer to +various named requirements whose details are set out in the tables in this +subsection. In these tables, `T` is an object or reference type to be supplied +by a {cpp} program instantiating a template; `x` is a value of type (possibly +`const`) `T`; `mlx` is a modifiable lvalue of type `T`. + +[#conversion_endianreversible] +##### EndianReversible requirements (in addition to `CopyConstructible`) + +[%header,cols=3*] +|=== +|Expression |Return |Requirements +|`endian_reverse(x)` |`T` +a|`T` is an endian type or a class type. + +If `T` is an endian type, returns the value of `x` with the order of bytes +reversed. + +If `T` is a class type, the function: + +* Returns the value of `x` with the order of bytes reversed for all data members +of types or arrays of types that meet the `EndianReversible` requirements, and; +* Is a non-member function in the same namespace as `T` that can be found by +argument dependent lookup (ADL). +|=== + +[#conversion_endianreversibleinplace] +##### EndianReversibleInplace requirements (in addition to `CopyConstructible`) + +[%header,cols=2*] +|=== +|Expression |Requirements +|`endian_reverse_inplace(mlx)` +a|`T` is an endian type or a class type. + +If `T` is an endian type, reverses the order of bytes in `mlx`. + +If `T` is a class type, the function: + +* Reverses the order of bytes of all data members of `mlx` that have types or +arrays of types that meet the `EndianReversible` or `EndianReversibleInplace` +requirements, and; +* Is a non-member function in the same namespace as `T` that can be found by +argument dependent lookup (ADL). +|=== + +NOTE: Because there is a function template for `endian_reverse_inplace` that +calls `endian_reverse`, only `endian_reverse` is required for a user-defined +type to meet the `EndianReversibleInplace` requirements. Although User-defined +types are not required to supply an `endian_reverse_inplace` function, doing so +may improve efficiency. + +#### Customization points for user-defined types (UDTs) + +This subsection describes requirements on the Endian library's implementation. + +The library's function templates requiring +`<>` are required to perform +reversal of endianness if needed by making an unqualified call to +`endian_reverse()`. + +The library's function templates requiring +`<>` are required to +perform reversal of endianness if needed by making an unqualified call to +`endian_reverse_inplace()`. + +See `example/udt_conversion_example.cpp` for an example user-defined type. + +### Functions + +``` +int8_t endian_reverse(int8_t x) noexcept; +int16_t endian_reverse(int16_t x) noexcept; +int32_t endian_reverse(int32_t x) noexcept; +int64_t endian_reverse(int64_t x) noexcept; +uint8_t endian_reverse(uint8_t x) noexcept; +uint16_t endian_reverse(uint16_t x) noexcept; +uint32_t endian_reverse(uint32_t x) noexcept; +uint64_t endian_reverse(uint64_t x) noexcept; +``` +[horizontal] +Returns:: `x`, with the order of its constituent bytes reversed. +Remarks:: The type of `x` meets the `EndianReversible` requirements. + +NOTE: The Boost.Endian library does not provide overloads for the {cpp} standard +library supplied types. + +``` +template +EndianReversible big_to_native(EndianReversible x) noexcept; +``` +[horizontal] +Returns:: `conditional_reverse(x)`. + +``` +template +EndianReversible native_to_big(EndianReversible x) noexcept; +``` +[horizontal] +Returns:: `conditional_reverse(x)`. + +``` +template +EndianReversible little_to_native(EndianReversible x) noexcept; +``` +[horizontal] +Returns:: `conditional_reverse(x)`. + +``` +template +EndianReversible native_to_little(EndianReversible x) noexcept; +``` +[horizontal] +Returns:: `conditional_reverse(x)`. + +``` +template +EndianReversible conditional_reverse(EndianReversible x) noexcept; +``` +[horizontal] +Returns:: `x` if `O1 == O2,` otherwise `endian_reverse(x)`. +Remarks:: Whether `x` or `endian_reverse(x)` is to be returned shall be +determined at compile time. + + +``` +template +EndianReversible conditional_reverse(EndianReversible x, + order order1, order order2) noexcept; +``` +[horizontal] +Returns:: `order1 == order2 ? x : endian_reverse(x)`. + +``` +template +void endian_reverse_inplace(EndianReversible& x) noexcept; +``` +[horizontal] +Effects:: `x` `= endian_reverse(x)`. + +``` +template +void big_to_native_inplace(EndianReversibleInplace& x) noexcept; +``` +[horizontal] +Effects:: `conditional_reverse_inplace(x)`. + +``` +template +void native_to_big_inplace(EndianReversibleInplace& x) noexcept; +``` +[horizontal] +Effects:: `conditional_reverse_inplace(x)`. + +``` +template +void little_to_native_inplace(EndianReversibleInplace& x) noexcept; +``` +[horizontal] +Effects:: `conditional_reverse_inplace(x)`. + +``` +template +void native_to_little_inplace(EndianReversibleInplace& x) noexcept; +``` +[horizontal] +Effects:: `conditional_reverse_inplace(x)`. + +``` +template +void conditional_reverse_inplace(EndianReversibleInplace& x) noexcept; +``` +[horizontal] +Effects:: None if `O1 == O2,` otherwise `endian_reverse_inplace(x)`. +Remarks:: Which effect applies shall be determined at compile time. + +``` +template +void conditional_reverse_inplace(EndianReversibleInplace& x, + order order1, order order2) noexcept; +``` +[horizontal] +Effects:: If `order1 == order2` then `endian_reverse_inplace(x)`. + +## FAQ + +See the <> FAQ for a library-wide FAQ. + +*Why are both value returning and modify-in-place functions provided?* + +* Returning the result by value is the standard C and {cpp} idiom for functions +that compute a value from an argument. Modify-in-place functions allow cleaner +code in many real-world endian use cases and are more efficient for user-defined +types that have members such as string data that do not need to be reversed. +Thus both forms are provided. + +*Why are exact-length 8, 16, 32, and 64-bit integers supported rather than the +built-in char, short, int, long, long long, etc?* + +* The primary use case, portable file or network data, needs these de facto +standard sizes. Using types that vary with the platform would greatly limit +portability for both programs and data. + +*Why not use the Linux names (htobe16, htole16, be16toh, le16toh, etc.) ?* + +* Those names are non-standard and vary even between POSIX-like operating +systems. A {cpp} library TS was going to use those names, but found they were +sometimes implemented as macros. Since macros do not respect scoping and +namespace rules, to use them would be very error prone. + +## Acknowledgements + +Tomas Puverle was instrumental in identifying and articulating the need to +support endian conversion as separate from endian integer types. Phil Endecott +suggested the form of the value returning signatures. Vicente Botet and other +reviewers suggested supporting user defined types. General reverse template +implementation approach using `std::reverse` suggested by Mathias Gaunard. +Portable implementation approach for 16, 32, and 64-bit integers suggested by +tymofey, with avoidance of undefined behavior as suggested by Giovanni Piero +Deretta, and a further refinement suggested by Pyry Jahkola. Intrinsic builtins +implementation approach for 16, 32, and 64-bit integers suggested by several +reviewers, and by David Stone, who provided his Boost licensed macro +implementation that became the starting point for +`boost/endian/detail/intrinsic.hpp`. Pierre Talbot provided the +`int8_t endian_reverse()` and templated `endian_reverse_inplace()` +implementations. From 0e349f3d5b387bc53d15dd4ea4a8898719ab4d3e Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 20:50:28 -0400 Subject: [PATCH 05/12] Convert Endian Conversion Types to asciidoc --- doc/endian.adoc | 2 + doc/endian/arithmetic.adoc | 561 +++++++++++++++++++++++++++++++++++++ 2 files changed, 563 insertions(+) create mode 100644 doc/endian/arithmetic.adoc diff --git a/doc/endian.adoc b/doc/endian.adoc index 8f09fd2..3f48d5a 100644 --- a/doc/endian.adoc +++ b/doc/endian.adoc @@ -20,6 +20,8 @@ include::endian/overview.adoc[] include::endian/conversion.adoc[] +include::endian/arithmetic.adoc[] + include::endian/choosing_approach.adoc[] include::endian/mini_review_topics.adoc[] diff --git a/doc/endian/arithmetic.adoc b/doc/endian/arithmetic.adoc new file mode 100644 index 0000000..42d7eea --- /dev/null +++ b/doc/endian/arithmetic.adoc @@ -0,0 +1,561 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#arithmetic] +# Endian Arithmetic Types + +## Introduction + +Header `boost/endian/arithmetic.hpp` provides integer binary types with +control over byte order, value type, size, and alignment. Typedefs provide +easy-to-use names for common configurations. + +These types provide portable byte-holders for integer data, independent of +particular computer architectures. Use cases almost always involve I/O, either +via files or network connections. Although data portability is the primary +motivation, these integer byte-holders may also be used to reduce memory use, +file size, or network activity since they provide binary integer sizes not +otherwise available. + +Such integer byte-holder types are traditionally called *endian* types. See the +http://en.wikipedia.org/wiki/Endian[Wikipedia] for a full exploration of +*endianness*, including definitions of *big endian* and *little endian*. + +Boost endian integers provide the same full set of {cpp} assignment, arithmetic, +and relational operators as {cpp} standard integral types, with the standard +semantics. + +Unary arithmetic operators are `+`, `-`, `~`, `!`, plus both prefix and postfix +`--` and `++`. Binary arithmetic operators are `+`, `+=`, `-`, `-=`, `\*`, +``*=``, `/`, `/=`, `&`, `&=`, `|`, `|=`, `^`, `^=`, `<<`, `<\<=`, `>>`, and +`>>=`. Binary relational operators are `==`, `!=`, `<`, `<=`, `>`, and `>=`. + +Implicit conversion to the underlying value type is provided. An implicit +constructor converting from the underlying value type is provided. + +## Example +The `endian_example.cpp` program writes a binary file containing four-byte, +big-endian and little-endian integers: + +``` +#include +#include +#include +#include + +using namespace boost::endian; + +namespace +{ + // This is an extract from a very widely used GIS file format. + // Why the designer decided to mix big and little endians in + // the same file is not known. But this is a real-world format + // and users wishing to write low level code manipulating these + // files have to deal with the mixed endianness. + + struct header + { + big_int32_t file_code; + big_int32_t file_length; + little_int32_t version; + little_int32_t shape_type; + }; + + const char* filename = "test.dat"; +} + +int main(int, char* []) +{ + header h; + + BOOST_STATIC_ASSERT(sizeof(h) == 16U); // reality check + + h.file_code = 0x01020304; + h.file_length = sizeof(header); + h.version = 1; + h.shape_type = 0x01020304; + + // Low-level I/O such as POSIX read/write or + // fread/fwrite is sometimes used for binary file operations + // when ultimate efficiency is important. Such I/O is often + // performed in some {cpp} wrapper class, but to drive home the + // point that endian integers are often used in fairly + // low-level code that does bulk I/O operations, + // fopen/fwrite is used for I/O in this example. + + std::FILE* fi = std::fopen(filename, "wb"); // MUST BE BINARY + + if (!fi) + { + std::cout << "could not open " << filename << '\n'; + return 1; + } + + if (std::fwrite(&h, sizeof(header), 1, fi)!= 1) + { + std::cout << "write failure for " << filename << '\n'; + return 1; + } + + std::fclose(fi); + + std::cout << "created file " << filename << '\n'; + + return 0; +} +``` + +After compiling and executing `endian_example.cpp`, a hex dump of `test.dat` +shows: + +``` +01020304 00000010 01000000 04030201 +``` + +Notice that the first two 32-bit integers are big endian while the second two +are little endian, even though the machine this was compiled and run on was +little endian. + +## Limitations + +Requires ``, `CHAR_BIT == 8`. If `CHAR_BIT` is some other value, +compilation will result in an `#error`. This restriction is in place because the +design, implementation, testing, and documentation has only considered issues +related to 8-bit bytes, and there have been no real-world use cases presented +for other sizes. + +In {cpp}03, `endian_arithmetic` does not meet the requirements for POD types +because it has constructors, private data members, and a base class. This means +that common use cases are relying on unspecified behavior in that the {cpp} +Standard does not guarantee memory layout for non-POD types. This has not been a +problem in practice since all known {cpp} compilers lay out memory as if +`endian` were a POD type. In {cpp}11, it is possible to specify the default +constructor as trivial, and private data members and base classes no longer +disqualify a type from being a POD type. Thus under {cpp}11, `endian_arithmetic` +will no longer be relying on unspecified behavior. + +## Feature set + +* Big endian| little endian | native endian byte ordering. +* Signed | unsigned +* Unaligned | aligned +* 1-8 byte (unaligned) | 1, 2, 4, 8 byte (aligned) +* Choice of value type + +## Enums and typedefs + +Two scoped enums are provided: + +``` +enum class order {big, little, native}; + +enum class align {no, yes}; +``` + +One class template is provided: + +``` +template +class endian_arithmetic; +``` + +Typedefs, such as `big_int32_t`, provide convenient naming conventions for +common use cases: + +[%header,cols=5*] +|=== +|Name |Alignment |Endianness |Sign |Sizes in bits (n) +|big_intn_t |no |big |signed |8,16,24,32,40,48,56,64 +|big_uintn_t |no |big |unsigned |8,16,24,32,40,48,56,64 +|little_intn_t |no |little |signed |8,16,24,32,40,48,56,64 +|little_uintn_t |no |little |unsigned |8,16,24,32,40,48,56,64 +|native_intn_t |no |native |signed |8,16,24,32,40,48,56,64 +|native_uintn_t |no |native |unsigned |8,16,24,32,40,48,56,64 +|big_intn_at |yes |big |signed |8,16,32,64 +|big_uintn_at |yes |big |unsigned |8,16,32,64 +|little_intn_at |yes |little |signed |8,16,32,64 +|little_uintn_at |yes |little |unsigned |8,16,32,64 +|=== + +The unaligned types do not cause compilers to insert padding bytes in classes +and structs. This is an important characteristic that can be exploited to +minimize wasted space in memory, files, and network transmissions. + +CAUTION: Code that uses aligned types is possibly non-portable because +alignment requirements vary between hardware architectures and because +alignment may be affected by compiler switches or pragmas. For example, +alignment of an 64-bit integer may be to a 32-bit boundary on a 32-bit machine. +Furthermore, aligned types are only available on architectures with 8, 16, 32, +and 64-bit integer types. + +TIP: Prefer unaligned arithmetic types. + +TIP: Protect yourself against alignment ills. For example: + +``` +static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong"); +``` + +NOTE: One-byte arithmetic types have identical layout on all platforms, so they +never actually reverse endianness. They are provided to enable generic code, +and to improve code readability and searchability. + +## Class template `endian_arithmetic` + +An `endian_integer` is an integer byte-holder with user-specified +<>, value type, size, and +<>. The usual operations on arithmetic types +are supplied. + +### Synopsis + +``` +#include +#include + +namespace boost +{ + namespace endian + { + // {cpp}11 features emulated if not available + + enum class align {no, yes}; + + template + class endian_arithmetic + : public endian_buffer + { + public: + typedef T value_type; + + // if BOOST_ENDIAN_FORCE_PODNESS is defined && {cpp}11 PODs are not + // available then these two constructors will not be present + endian_arithmetic() noexcept = default; + endian_arithmetic(T v) noexcept; + + endian_arithmetic& operator=(T v) noexcept; + operator value_type() const noexcept; + value_type value() const noexcept; // for exposition; see endian_buffer + const char* data() const noexcept; // for exposition; see endian_buffer + + // arithmetic operations + // note that additional operations are provided by the value_type + value_type operator+(const endian& x) noexcept; + endian& operator+=(endian& x, value_type y) noexcept; + endian& operator-=(endian& x, value_type y) noexcept; + endian& operator*=(endian& x, value_type y) noexcept; + endian& operator/=(endian& x, value_type y) noexcept; + endian& operator%=(endian& x, value_type y) noexcept; + endian& operator&=(endian& x, value_type y) noexcept; + endian& operator|=(endian& x, value_type y) noexcept; + endian& operator^=(endian& x, value_type y) noexcept; + endian& operator<<=(endian& x, value_type y) noexcept; + endian& operator>>=(endian& x, value_type y noexcept; + value_type operator<<(const endian& x, value_type y) noexcept; + value_type operator>>(const endian& x, value_type y) noexcept; + endian& operator++(endian& x) noexcept; + endian& operator--(endian& x) noexcept; + endian operator++(endian& x, int) noexcept; + endian operator--(endian& x, int) noexcept; + + // Stream inserter + template + friend std::basic_ostream& + operator<<(std::basic_ostream& os, const T& x); + + // Stream extractor + template + friend std::basic_istream& + operator>>(std::basic_istream& is, T& x); + }; + + // typedefs + + // unaligned big endian signed integer types + typedef endian big_int8_t; + typedef endian big_int16_t; + typedef endian big_int24_t; + typedef endian big_int32_t; + typedef endian big_int40_t; + typedef endian big_int48_t; + typedef endian big_int56_t; + typedef endian big_int64_t; + + // unaligned big endian unsigned integer types + typedef endian big_uint8_t; + typedef endian big_uint16_t; + typedef endian big_uint24_t; + typedef endian big_uint32_t; + typedef endian big_uint40_t; + typedef endian big_uint48_t; + typedef endian big_uint56_t; + typedef endian big_uint64_t; + + // unaligned little endian signed integer types + typedef endian little_int8_t; + typedef endian little_int16_t; + typedef endian little_int24_t; + typedef endian little_int32_t; + typedef endian little_int40_t; + typedef endian little_int48_t; + typedef endian little_int56_t; + typedef endian little_int64_t; + + // unaligned little endian unsigned integer types + typedef endian little_uint8_t; + typedef endian little_uint16_t; + typedef endian little_uint24_t; + typedef endian little_uint32_t; + typedef endian little_uint40_t; + typedef endian little_uint48_t; + typedef endian little_uint56_t; + typedef endian little_uint64_t; + + // unaligned native endian signed integer types + typedef implementation-defined_int8_t native_int8_t; + typedef implementation-defined_int16_t native_int16_t; + typedef implementation-defined_int24_t native_int24_t; + typedef implementation-defined_int32_t native_int32_t; + typedef implementation-defined_int40_t native_int40_t; + typedef implementation-defined_int48_t native_int48_t; + typedef implementation-defined_int56_t native_int56_t; + typedef implementation-defined_int64_t native_int64_t; + + // unaligned native endian unsigned integer types + typedef implementation-defined_uint8_t native_uint8_t; + typedef implementation-defined_uint16_t native_uint16_t; + typedef implementation-defined_uint24_t native_uint24_t; + typedef implementation-defined_uint32_t native_uint32_t; + typedef implementation-defined_uint40_t native_uint40_t; + typedef implementation-defined_uint48_t native_uint48_t; + typedef implementation-defined_uint56_t native_uint56_t; + typedef implementation-defined_uint64_t native_uint64_t; + + // aligned big endian signed integer types + typedef endian big_int8_at; + typedef endian big_int16_at; + typedef endian big_int32_at; + typedef endian big_int64_at; + + // aligned big endian unsigned integer types + typedef endian big_uint8_at; + typedef endian big_uint16_at; + typedef endian big_uint32_at; + typedef endian big_uint64_at; + + // aligned little endian signed integer types + typedef endian little_int8_at; + typedef endian little_int16_at; + typedef endian little_int32_at; + typedef endian little_int64_at; + + // aligned little endian unsigned integer types + typedef endian little_uint8_at; + typedef endian little_uint16_at; + typedef endian little_uint32_at; + typedef endian little_uint64_at; + + // aligned native endian typedefs are not provided because + // types are superior for that use case + + } // namespace endian +} // namespace boost +``` + +The `implementation-defined` text above is either `big` or `little` according +to the endianness of the platform. + +### Members + +``` +endian() = default; // {cpp}03: endian(){} +``` +[horizontal] +Effects:: Constructs an uninitialized object of type +`endian_arithmetic`. + +``` +endian(T v); +``` +[horizontal] +Effects:: Constructs an object of type `endian_arithmetic`. +Postcondition:: `x == v,` where `x` is the constructed object. + +``` +endian& operator=(T v); +``` +[horizontal] +Postcondition:: `x == v,` where `x` is the constructed object. +Returns:: `*this`. + +``` +operator T() const; +``` +[horizontal] +Returns:: The current value stored in `*this`, converted to `value_type`. + +``` +const char* data() const; +``` +[horizontal] +Returns:: A pointer to the first byte of the endian binary value stored in +`*this`. + +### Other operators + +Other operators on endian objects are forwarded to the equivalent operator on +`value_type`. + +### Stream inserter + +``` +template +friend std::basic_ostream& + operator<<(std::basic_ostream& os, const T& x); + +``` +[horizontal] +Returns:: `os << +x`.

+ +### Stream extractor + +``` +template +friend std::basic_istream& + operator>>(std::basic_istream& is, T& x); +``` +[horizontal] +Effects:: As if: +``` +T i; +if (is >> i) + x = i; +``` +[horizontal] +Returns:: `is`. + +## FAQ + +See the <> FAQ for a library-wide FAQ. + +### Why not just use Boost.Serialization? + +Serialization involves a conversion for every object involved in I/O. Endian +integers require no conversion or copying. They are already in the desired +format for binary I/O. Thus they can be read or written in bulk. + +### Are endian types PODs? + +Yes for {cpp}11. No for {cpp}03, although several +<> are available to force PODness in all cases. + +### What are the implications of endian integer types not being PODs with {cpp}03 +compilers? + +They can't be used in unions. Also, compilers aren't required to align or lay +out storage in portable ways, although this potential problem hasn't prevented +use of Boost.Endian with real compilers. + +### What good is native endianness? + +It provides alignment and size guarantees not available from the built-in +types. It eases generic programming. + +### Why bother with the aligned endian types? + +Aligned integer operations may be faster (as much as 10 to 20 times faster) +if the endianness and alignment of the type matches the endianness and +alignment requirements of the machine. The code, however, will be somewhat less +portable than with the unaligned types. + +### Why provide the arithmetic operations? + +Providing a full set of operations reduces program clutter and makes code +both easier to write and to read. Consider incrementing a variable in a record. +It is very convenient to write: +``` +++record.foo; +``` +Rather than: +``` +int temp(record.foo); +++temp; +record.foo = temp; +``` + +## Design considerations for Boost.Endian types + +* Must be suitable for I/O - in other words, must be memcpyable. +* Must provide exactly the size and internal byte ordering specified. +* Must work correctly when the internal integer representation has more bits +that the sum of the bits in the external byte representation. Sign extension +must work correctly when the internal integer representation type has more +bits than the sum of the bits in the external bytes. For example, using +a 64-bit integer internally to represent 40-bit (5 byte) numbers must work for +both positive and negative values. +* Must work correctly (including using the same defined external +representation) regardless of whether a compiler treats char as signed or +unsigned. +* Unaligned types must not cause compilers to insert padding bytes. +* The implementation should supply optimizations with great care. Experience +has shown that optimizations of endian integers often become pessimizations +when changing machines or compilers. Pessimizations can also happen when +changing compiler switches, compiler versions, or CPU models of the same +architecture. + +## Experience + +Classes with similar functionality have been independently developed by +several Boost programmers and used very successful in high-value, high-use +applications for many years. These independently developed endian libraries +often evolved from C libraries that were also widely used. Endian types have +proven widely useful across a wide range of computer architectures and +applications. + +## Motivating use cases + +Neil Mayhew writes: "I can also provide a meaningful use-case for this +library: reading TrueType font files from disk and processing the contents. The +data format has fixed endianness (big) and has unaligned values in various +places. Using Boost.Endian simplifies and cleans the code wonderfully." + +## {cpp}11 + +The availability of the {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions] feature is detected automatically, and will be used if present to +ensure that objects of `class endian_arithmetic` are trivial, and thus PODs. + +## Compilation +Boost.Endian is implemented entirely within headers, with no need to link to any +Boost object libraries. + +Several macros allow user control over features: + +* BOOST_ENDIAN_NO_CTORS causes `class endian_arithmetic` to have no +constructors. The intended use is for compiling user code that must be portable +between compilers regardless of {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions] support. Use of constructors will always fail, +* BOOST_ENDIAN_FORCE_PODNESS causes BOOST_ENDIAN_NO_CTORS to be defined if +the compiler does not support {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions]. This is ensures that objects of `class endian_arithmetic` are PODs, +and so can be used in {cpp}03 unions. In {cpp}11, `class endian_arithmetic` +objects are PODs, even though they have constructors, so can always be used in +unions. + +## Acknowledgements + +Original design developed by Darin Adler based on classes developed by Mark +Borgerding. Four original class templates combined into a single +`endian_arithmetic` class template by Beman Dawes, who put the library together, +provided documentation, added the typedefs, and also added the +`unrolled_byte_loops` sign partial specialization to correctly extend the sign +when cover integer size differs from endian representation size. From 9ffe32f740ae7e97d32ed167e55c38d47baba8bd Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Sun, 24 Mar 2019 22:38:54 -0400 Subject: [PATCH 06/12] Convert Endian buffer types to asciidoc --- doc/endian.adoc | 2 + doc/endian/arithmetic.adoc | 36 +- doc/endian/buffers.adoc | 506 +++++++++++++++++++++++++++++ doc/endian/mini_review_topics.adoc | 78 +++-- 4 files changed, 577 insertions(+), 45 deletions(-) create mode 100644 doc/endian/buffers.adoc diff --git a/doc/endian.adoc b/doc/endian.adoc index 3f48d5a..1b7c06d 100644 --- a/doc/endian.adoc +++ b/doc/endian.adoc @@ -22,6 +22,8 @@ include::endian/conversion.adoc[] include::endian/arithmetic.adoc[] +include::endian/buffers.adoc[] + include::endian/choosing_approach.adoc[] include::endian/mini_review_topics.adoc[] diff --git a/doc/endian/arithmetic.adoc b/doc/endian/arithmetic.adoc index 42d7eea..68e9096 100644 --- a/doc/endian/arithmetic.adoc +++ b/doc/endian/arithmetic.adoc @@ -82,7 +82,7 @@ int main(int, char* []) // Low-level I/O such as POSIX read/write or // fread/fwrite is sometimes used for binary file operations // when ultimate efficiency is important. Such I/O is often - // performed in some {cpp} wrapper class, but to drive home the + // performed in some C++ wrapper class, but to drive home the // point that endian integers are often used in fairly // low-level code that does bulk I/O operations, // fopen/fwrite is used for I/O in this example. @@ -196,7 +196,9 @@ and 64-bit integer types. TIP: Prefer unaligned arithmetic types. TIP: Protect yourself against alignment ills. For example: - +[none] +{blank}:: ++ ``` static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong"); ``` @@ -222,7 +224,7 @@ namespace boost { namespace endian { - // {cpp}11 features emulated if not available + // C++11 features emulated if not available enum class align {no, yes}; @@ -234,7 +236,7 @@ namespace boost public: typedef T value_type; - // if BOOST_ENDIAN_FORCE_PODNESS is defined && {cpp}11 PODs are not + // if BOOST_ENDIAN_FORCE_PODNESS is defined && C++11 PODs are not // available then these two constructors will not be present endian_arithmetic() noexcept = default; endian_arithmetic(T v) noexcept; @@ -374,7 +376,7 @@ to the endianness of the platform. ### Members ``` -endian() = default; // {cpp}03: endian(){} +endian() = default; // C++03: endian(){} ``` [horizontal] Effects:: Constructs an uninitialized object of type @@ -432,57 +434,53 @@ friend std::basic_istream& ``` [horizontal] Effects:: As if: ++ ``` T i; if (is >> i) x = i; ``` -[horizontal] Returns:: `is`. ## FAQ See the <> FAQ for a library-wide FAQ. -### Why not just use Boost.Serialization? - +Why not just use Boost.Serialization?:: Serialization involves a conversion for every object involved in I/O. Endian integers require no conversion or copying. They are already in the desired format for binary I/O. Thus they can be read or written in bulk. -### Are endian types PODs? - +Are endian types PODs?:: Yes for {cpp}11. No for {cpp}03, although several <> are available to force PODness in all cases. -### What are the implications of endian integer types not being PODs with {cpp}03 -compilers? - +What are the implications of endian integer types not being PODs with {cpp}03 compilers?:: They can't be used in unions. Also, compilers aren't required to align or lay out storage in portable ways, although this potential problem hasn't prevented use of Boost.Endian with real compilers. -### What good is native endianness? - +What good is native endianness?:: It provides alignment and size guarantees not available from the built-in types. It eases generic programming. -### Why bother with the aligned endian types? - +Why bother with the aligned endian types?:: Aligned integer operations may be faster (as much as 10 to 20 times faster) if the endianness and alignment of the type matches the endianness and alignment requirements of the machine. The code, however, will be somewhat less portable than with the unaligned types. -### Why provide the arithmetic operations? - +Why provide the arithmetic operations?:: Providing a full set of operations reduces program clutter and makes code both easier to write and to read. Consider incrementing a variable in a record. It is very convenient to write: ++ ``` ++record.foo; ``` ++ Rather than: ++ ``` int temp(record.foo); ++temp; diff --git a/doc/endian/buffers.adoc b/doc/endian/buffers.adoc new file mode 100644 index 0000000..cd85669 --- /dev/null +++ b/doc/endian/buffers.adoc @@ -0,0 +1,506 @@ +//// +Copyright 2011-2016 Beman Dawes + +Distributed under the Boost Software License, Version 1.0. +(http://www.boost.org/LICENSE_1_0.txt) +//// + +[#buffers] +# Endian Buffer Types + +## Introduction + +The internal byte order of arithmetic types is traditionally called +*endianness*. See the http://en.wikipedia.org/wiki/Endian[Wikipedia] for a full +exploration of *endianness*, including definitions of *big endian* and *little +endian*. + +Header `boost/endian/buffers.hpp` provides `endian_buffer`, a portable endian +integer binary buffer class template with control over byte order, value type, +size, and alignment independent of the platform's native endianness. Typedefs +provide easy-to-use names for common configurations. + +Use cases primarily involve data portability, either via files or network +connections, but these byte-holders may also be used to reduce memory use, file +size, or network activity since they provide binary numeric sizes not otherwise +available. + +Class `endian_buffer` is aimed at users who wish explicit control over when +endianness conversions occur. It also serves as the base class for the +<> class template, which is aimed at users who +wish fully automatic endianness conversion and direct support for all normal +arithmetic operations. + +## Example + +The `example/endian_example.cpp` program writes a binary file containing +four-byte, big-endian and little-endian integers: + +``` +#include +#include +#include // see Synopsis below +#include + +using namespace boost::endian; + +namespace +{ + // This is an extract from a very widely used GIS file format. + // Why the designer decided to mix big and little endians in + // the same file is not known. But this is a real-world format + // and users wishing to write low level code manipulating these + // files have to deal with the mixed endianness. + + struct header + { + big_int32_`buf_`t file_code; + big_int32_`buf_`t file_length; + little_int32_`buf_`t version; + little_int32_`buf_`t shape_type; + }; + + const char* filename = "test.dat"; +} + +int main(int, char* []) +{ + header h; + + BOOST_STATIC_ASSERT(sizeof(h) == 16U); // reality check + + h.file_code = 0x01020304; + h.file_length = sizeof(header); + h.version = 1; + h.shape_type = 0x01020304; + + // Low-level I/O such as POSIX read/write or + // fread/fwrite is sometimes used for binary file operations + // when ultimate efficiency is important. Such I/O is often + // performed in some {cpp} wrapper class, but to drive home the + // point that endian integers are often used in fairly + // low-level code that does bulk I/O operations, + // fopen/fwrite is used for I/O in this example. + + std::FILE* fi = std::fopen(filename, "wb"); // MUST BE BINARY + + if (!fi) + { + std::cout << "could not open " << filename << '\n'; + return 1; + } + + if (std::fwrite(&h, sizeof(header), 1, fi)!= 1) + { + std::cout << "write failure for " << filename << '\n'; + return 1; + } + + std::fclose(fi); + + std::cout << "created file " << filename << '\n'; + + return 0; +} +``` + +After compiling and executing `example/endian_example.cpp`, a hex dump of +`test.dat` shows: + +``` +01020304 00000010 01000000 04030201 +``` + +Notice that the first two 32-bit integers are big endian while the second two +are little endian, even though the machine this was compiled and run on was +little endian. + +## Limitations + +Requires ``, `CHAR_BIT == 8`. If `CHAR_BIT` is some other value, +compilation will result in an `#error`. This restriction is in place because the +design, implementation, testing, and documentation has only considered issues +related to 8-bit bytes, and there have been no real-world use cases presented +for other sizes. + +In {cpp}03, `endian_buffer` does not meet the requirements for POD types because +it has constructors, private data members, and a base class. This means that +common use cases are relying on unspecified behavior in that the {cpp} Standard +does not guarantee memory layout for non-POD types. This has not been a problem +in practice since all known {cpp} compilers lay out memory as if `endian` were +a POD type. In {cpp}11, it is possible to specify the default constructor as +trivial, and private data members and base classes no longer disqualify a type +from being a POD type. Thus under {cpp}11, `endian_buffer` will no longer be +relying on unspecified behavior. + +## Feature set + +* Big endian| little endian | native endian byte ordering. +* Signed | unsigned +* Unaligned | aligned +* 1-8 byte (unaligned) | 1, 2, 4, 8 byte (aligned) +* Choice of value type + +## Enums and typedefs + +Two scoped enums are provided: + +``` +enum class order {big, little, native}; + +enum class align {no, yes}; +``` + +One class template is provided: + +``` +template +class endian_buffer; +``` + +Typedefs, such as `big_int32_buf_t`, provide convenient naming conventions for +common use cases: + +[%header,cols=5*] +|=== +|Name |Alignment |Endianness |Sign |Sizes in bits (n) +|big_intn_buf_t |no |big |signed |8,16,24,32,40,48,56,64 +|big_uintn_buf_t |no |big |unsigned |8,16,24,32,40,48,56,64 +|little_intn_buf_t |no |little |signed |8,16,24,32,40,48,56,64 +|little_uintn_buf_t |no |little |unsigned |8,16,24,32,40,48,56,64 +|native_intn_buf_t |no |native |signed |8,16,24,32,40,48,56,64 +|native_uintn_buf_t |no |native |unsigned |8,16,24,32,40,48,56,64 +|big_intn_buf_at |yes |big |signed |8,16,32,64 +|big_uintn_buf_at |yes |big |unsigned |8,16,32,64 +|little_intn_buf_at |yes |little |signed |8,16,32,64 +|little_uintn_buf_at |yes |little |unsigned |8,16,32,64 +|=== + +The unaligned types do not cause compilers to insert padding bytes in classes +and structs. This is an important characteristic that can be exploited to +minimize wasted space in memory, files, and network transmissions. + +CAUTION: Code that uses aligned types is possibly non-portable because alignment +requirements vary between hardware architectures and because alignment may be +affected by compiler switches or pragmas. For example, alignment of an 64-bit +integer may be to a 32-bit boundary on a 32-bit machine and to a 64-bit boundary +on a 64-bit machine. Furthermore, aligned types are only available on +architectures with 8, 16, 32, and 64-bit integer types. + +TIP: Prefer unaligned buffer types. + +TIP: Protect yourself against alignment ills. For example: +[none] +{blank}:: ++ +``` +static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong"); +``` + +Note: One-byte big and little buffer types have identical layout on all +platforms, so they never actually reverse endianness. They are provided to +enable generic code, and to improve code readability and searchability. + +## Class template `endian_buffer` + +An `endian_buffer` is a byte-holder for arithmetic types with +user-specified <>, value type, size, and +<>. + +### Synopsis + +``` +#include + class endian_buffer + { + public: + typedef T value_type; + + endian_buffer() noexcept = default; + explicit endian_buffer(T v) noexcept; + + endian_buffer& operator=(T v) noexcept; + value_type value() const noexcept; + const char* data() const noexcept; + protected: + implementaton-defined endian_value; // for exposition only + }; + + // stream inserter + template + std::basic_ostream& + operator<<(std::basic_ostream& os, + const endian_buffer& x); + + // stream extractor + template + std::basic_istream& + operator>>(std::basic_istream& is, + endian_buffer& x); + + // typedefs + + // unaligned big endian signed integer buffers + typedef endian_buffer big_int8_buf_t; + typedef endian_buffer big_int16_buf_t; + typedef endian_buffer big_int24_buf_t; + typedef endian_buffer big_int32_buf_t; + typedef endian_buffer big_int40_buf_t; + typedef endian_buffer big_int48_buf_t; + typedef endian_buffer big_int56_buf_t; + typedef endian_buffer big_int64_buf_t; + + // unaligned big endian unsigned integer buffers + typedef endian_buffer big_uint8_buf_t; + typedef endian_buffer big_uint16_buf_t; + typedef endian_buffer big_uint24_buf_t; + typedef endian_buffer big_uint32_buf_t; + typedef endian_buffer big_uint40_buf_t; + typedef endian_buffer big_uint48_buf_t; + typedef endian_buffer big_uint56_buf_t; + typedef endian_buffer big_uint64_buf_t; + + // unaligned little endian signed integer buffers + typedef endian_buffer little_int8_buf_t; + typedef endian_buffer little_int16_buf_t; + typedef endian_buffer little_int24_buf_t; + typedef endian_buffer little_int32_buf_t; + typedef endian_buffer little_int40_buf_t; + typedef endian_buffer little_int48_buf_t; + typedef endian_buffer little_int56_buf_t; + typedef endian_buffer little_int64_buf_t; + + // unaligned little endian unsigned integer buffers + typedef endian_buffer little_uint8_buf_t; + typedef endian_buffer little_uint16_buf_t; + typedef endian_buffer little_uint24_buf_t; + typedef endian_buffer little_uint32_buf_t; + typedef endian_buffer little_uint40_buf_t; + typedef endian_buffer little_uint48_buf_t; + typedef endian_buffer little_uint56_buf_t; + typedef endian_buffer little_uint64_buf_t; + + // unaligned native endian signed integer types + typedef implementation-defined_int8_buf_t native_int8_buf_t; + typedef implementation-defined_int16_buf_t native_int16_buf_t; + typedef implementation-defined_int24_buf_t native_int24_buf_t; + typedef implementation-defined_int32_buf_t native_int32_buf_t; + typedef implementation-defined_int40_buf_t native_int40_buf_t; + typedef implementation-defined_int48_buf_t native_int48_buf_t; + typedef implementation-defined_int56_buf_t native_int56_buf_t; + typedef implementation-defined_int64_buf_t native_int64_buf_t; + + // unaligned native endian unsigned integer types + typedef implementation-defined_uint8_buf_t native_uint8_buf_t; + typedef implementation-defined_uint16_buf_t native_uint16_buf_t; + typedef implementation-defined_uint24_buf_t native_uint24_buf_t; + typedef implementation-defined_uint32_buf_t native_uint32_buf_t; + typedef implementation-defined_uint40_buf_t native_uint40_buf_t; + typedef implementation-defined_uint48_buf_t native_uint48_buf_t; + typedef implementation-defined_uint56_buf_t native_uint56_buf_t; + typedef implementation-defined_uint64_buf_t native_uint64_buf_t; + + // aligned big endian signed integer buffers + typedef endian_buffer big_int8_buf_at; + typedef endian_buffer big_int16_buf_at; + typedef endian_buffer big_int32_buf_at; + typedef endian_buffer big_int64_buf_at; + + // aligned big endian unsigned integer buffers + typedef endian_buffer big_uint8_buf_at; + typedef endian_buffer big_uint16_buf_at; + typedef endian_buffer big_uint32_buf_at; + typedef endian_buffer big_uint64_buf_at; + + // aligned little endian signed integer buffers + typedef endian_buffer little_int8_buf_at; + typedef endian_buffer little_int16_buf_at; + typedef endian_buffer little_int32_buf_at; + typedef endian_buffer little_int64_buf_at; + + // aligned little endian unsigned integer buffers + typedef endian_buffer little_uint8_buf_at; + typedef endian_buffer little_uint16_buf_at; + typedef endian_buffer little_uint32_buf_at; + typedef endian_buffer little_uint64_buf_at; + + // aligned native endian typedefs are not provided because + // types are superior for this use case + + } // namespace endian +} // namespace boost +``` + +The `implementation-defined` text in typedefs above is either `big` or `little` +according to the native endianness of the platform. + +The expository data member `endian_value` stores the current value of an +`endian_value` object as a sequence of bytes ordered as specified by the +`Order` template parameter. The `implementation-defined` type of +`endian_value` is a type such as `char[Nbits/CHAR_BIT]` or `T` that meets the +requirements imposed by the `Nbits` and `Align` template parameters. The +`CHAR_BIT` macro is defined in ``. The only value of +`CHAR_BIT` that is required to be supported is 8. + +Template parameter `T` is required to be a standard integer type ({cpp}std, +3.9.1) and `sizeof(T)*CHAR_BIT` is required to be greater or equal to `Nbits`. + +### Members + +``` +endian_buffer() noexcept = default; +``` +[horizontal] +Effects:: Constructs an uninitialized object of type `endian_buffer`. + +``` +explicit endian_buffer(T v) noexcept; +``` +[horizontal] +Effects:: Constructs an object of type `endian_buffer`. +Postcondition:: `value() == v & mask`, where `mask` is a constant of type +`value_type` with `Nbits` low-order bits set to one. +Remarks:: If `Align` is `align::yes` then endianness conversion, if required, +is performed by `boost::endian::endian_reverse`. + +``` +endian_buffer& operator=(T v) noexcept; +``` +[horizontal] +Postcondition:: `value() == v & mask`, where `mask` is a constant of type +`value_type` with `Nbits` low-order bits set to one. +Returns:: `*this`. +Remarks:: If `Align` is `align::yes` then endianness conversion, if required, is +performed by `boost::endian::endian_reverse`. + +``` +value_type value() const noexcept; +``` +[horizontal] +Returns:: `endian_value`, converted to `value_type`, if required, and having the +endianness of the native platform. +Remarks:: If `Align` is `align::yes` then endianness conversion, if required, is +performed by `boost::endian::endian_reverse`. + +``` +const char* data() const noexcept; +``` +[horizontal] +Returns:: A pointer to the first byte of `endian_value`. + +### Non-member functions + +``` +template +std::basic_ostream& operator<<(std::basic_ostream& os, + const endian_buffer& x); +``` +[horizontal] +Returns:: `os << x.value()`. + +``` +template +std::basic_istream& operator>>(std::basic_istream& is, + endian_buffer& x); +``` +[horizontal] +Effects:: As if: ++ +``` +T i; +if (is >> i) + x = i; +``` +Returns:: `is`. + +## FAQ + +See the <> FAQ for a library-wide FAQ. + +Why not just use Boost.Serialization?:: +Serialization involves a conversion for every object involved in I/O. Endian +integers require no conversion or copying. They are already in the desired +format for binary I/O. Thus they can be read or written in bulk. + +Are endian types PODs?:: +Yes for {cpp}11. No for {cpp}03, although several +<> are available to force PODness in all cases. + +What are the implications of endian integer types not being PODs with {cpp}03 compilers?:: +They can't be used in unions. Also, compilers aren't required to align or lay +out storage in portable ways, although this potential problem hasn't prevented +use of Boost.Endian with real compilers. + +What good is native endianness?:: +It provides alignment and size guarantees not available from the built-in +types. It eases generic programming. + +Why bother with the aligned endian types?:: +Aligned integer operations may be faster (as much as 10 to 20 times faster) if +the endianness and alignment of the type matches the endianness and alignment +requirements of the machine. The code, however, is likely to be somewhat less +portable than with the unaligned types. + +## Design considerations for Boost.Endian buffers + +* Must be suitable for I/O - in other words, must be memcpyable. +* Must provide exactly the size and internal byte ordering specified. +* Must work correctly when the internal integer representation has more bits +that the sum of the bits in the external byte representation. Sign extension +must work correctly when the internal integer representation type has more +bits than the sum of the bits in the external bytes. For example, using +a 64-bit integer internally to represent 40-bit (5 byte) numbers must work for +both positive and negative values. +* Must work correctly (including using the same defined external +representation) regardless of whether a compiler treats char as signed or +unsigned. +* Unaligned types must not cause compilers to insert padding bytes. +* The implementation should supply optimizations with great care. Experience +has shown that optimizations of endian integers often become pessimizations +when changing machines or compilers. Pessimizations can also happen when +changing compiler switches, compiler versions, or CPU models of the same +architecture. + +## {cpp}11 + +The availability of the {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions] feature is detected automatically, and will be used if present to +ensure that objects of `class endian_buffer` are trivial, and thus +PODs. + +## Compilation + +Boost.Endian is implemented entirely within headers, with no need to link to +any Boost object libraries. + +Several macros allow user control over features: + +* BOOST_ENDIAN_NO_CTORS causes `class endian_buffer` to have no +constructors. The intended use is for compiling user code that must be +portable between compilers regardless of {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions] support. Use of constructors will always fail, +* BOOST_ENDIAN_FORCE_PODNESS causes BOOST_ENDIAN_NO_CTORS to be defined if the +compiler does not support {cpp}11 +http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted +Functions]. This is ensures that objects of `class endian_buffer` are PODs, and +so can be used in {cpp}03 unions. In {cpp}11, `class endian_buffer` objects are +PODs, even though they have constructors, so can always be used in unions. diff --git a/doc/endian/mini_review_topics.adoc b/doc/endian/mini_review_topics.adoc index c42604f..987a868 100644 --- a/doc/endian/mini_review_topics.adoc +++ b/doc/endian/mini_review_topics.adoc @@ -14,78 +14,104 @@ resolved before a mini-review. The issues are shown in *bold* below, with the resolution indicated. -1. *Common use case scenarios should be developed.* -* Done. The documentation have been refactored. A page is now devoted to +*Common use case scenarios should be developed.* + +{blank}:: +Done. The documentation have been refactored. A page is now devoted to <> to endianness. See <> for use case scenarios. -2. *Example programs should be developed for the common use case scenarios.* -* Done. See <>. Example code has been added +*Example programs should be developed for the common use case scenarios.* + +{blank}:: +Done. See <>. Example code has been added throughout. -3. *Documentation should illuminate the differences between endian +*Documentation should illuminate the differences between endian integer/float type and endian conversion approaches to the common use case scenarios, and provide guidelines for choosing the most appropriate approach in user's applications.* -* Done. See <>. -4. *Conversion functions supplying results via return should be provided.* -* Done. See <>. +{blank}:: +Done. See <>. -5. *Platform specific performance enhancements such as use of compiler +*Conversion functions supplying results via return should be provided.* + +{blank}:: +Done. See <>. + +*Platform specific performance enhancements such as use of compiler intrinsics or relaxed alignment requirements should be supported.* -* Done. Compiler (Clang, GCC, Visual{cpp}, etc.) intrinsics and built-in + +{blank}:: +Done. Compiler (Clang, GCC, Visual{cpp}, etc.) intrinsics and built-in functions are used in the implementation where appropriate, as requested. See <>. See <> to gauge the impact of intrinsics. -6. *Endian integer (and floating) types should be implemented via the +*Endian integer (and floating) types should be implemented via the conversion functions. If that can't be done efficiently, consideration should be given to expanding the conversion function signatures to resolve the inefficiencies.* -* Done. For the endian types, the implementation uses the endian conversion + +{blank}:: +Done. For the endian types, the implementation uses the endian conversion functions, and thus the intrinsics, as requested. -7. *Benchmarks that measure performance should be provided. It should be +*Benchmarks that measure performance should be provided. It should be possible to compare platform specific performance enhancements against portable base implementations, and to compare endian integer approaches against endian conversion approaches for the common use case scenarios.* -* Done. See <>. The `endian/test` + +{blank}:: +Done. See <>. The `endian/test` directory also contains several additional benchmark and speed test programs. -8. *Float (32-bits) and double (64-bits) should be supported. IEEE 754 is +*Float (32-bits) and double (64-bits) should be supported. IEEE 754 is the primary use case.* -* Done. The <>, + +{blank}:: +Done. The <>, <> and <> now support 32-bit `(float)` and 64-bit `(double)` floating point, as requested. -9. *Support for user defined types (UDTs) is desirable, and should be +*Support for user defined types (UDTs) is desirable, and should be provided where there would be no conflict with the other concerns.* -* Done. See <>. -10. *There is some concern that endian integer/float arithmetic operations +*There is some concern that endian integer/float arithmetic operations might used inadvertently or inappropriately. The impact of adding an endian_buffer class without arithmetic operations should be investigated.* -* Done. The endian types have been decomposed into class template + +{blank}:: +Done. The endian types have been decomposed into class template `<>` and class template `<>`. Class `endian_buffer` is a public base class for `endian_arithmetic`, and can also be used by users as a stand-alone class. -11. *Stream insertion and extraction of the endian integer/float types should +*Stream insertion and extraction of the endian integer/float types should be documented and included in the test coverage.* -* Done. See <> and + +{blank}:: +Done. See <> and <>. -12. *Binary I/O support that was investigated during development of the +*Binary I/O support that was investigated during development of the Endian library should be put up for mini-review for inclusion in the Boost I/O library.* -* Not done yet. Will be handled as a separate min-review soon after the Endian + +{blank}:: +Not done yet. Will be handled as a separate min-review soon after the Endian mini-review. -13. *Other requested changes.* +*Other requested changes.* + +{blank}:: * In addition to the named-endianness conversion functions, functions that perform compile-time (via template) and run-time (via function argument) dispatch are now provided. @@ -93,4 +119,4 @@ dispatch are now provided. to the endianness of the platform. This reduces the number of template specializations required. * Headers have been reorganized to make them easier to read, with a synopsis -at the front and implementation following.s +at the front and implementation following. From a6aaec46f5edb1ddeaf67e048b91bfedd135b543 Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Mon, 25 Mar 2019 09:32:54 -0400 Subject: [PATCH 07/12] Minor formatting changes in documentation --- doc/endian/arithmetic.adoc | 31 ++++++++---- doc/endian/buffers.adoc | 29 +++++++++--- doc/endian/conversion.adoc | 60 +++++++++++++++++------- doc/endian/mini_review_topics.adoc | 75 +++++++----------------------- 4 files changed, 105 insertions(+), 90 deletions(-) diff --git a/doc/endian/arithmetic.adoc b/doc/endian/arithmetic.adoc index 68e9096..65769f0 100644 --- a/doc/endian/arithmetic.adoc +++ b/doc/endian/arithmetic.adoc @@ -378,34 +378,44 @@ to the endianness of the platform. ``` endian() = default; // C++03: endian(){} ``` -[horizontal] +[none] +* {blank} ++ Effects:: Constructs an uninitialized object of type `endian_arithmetic`. ``` endian(T v); ``` -[horizontal] +[none] +* {blank} ++ Effects:: Constructs an object of type `endian_arithmetic`. Postcondition:: `x == v,` where `x` is the constructed object. ``` endian& operator=(T v); ``` -[horizontal] +[none] +* {blank} ++ Postcondition:: `x == v,` where `x` is the constructed object. Returns:: `*this`. ``` operator T() const; ``` -[horizontal] +[none] +* {blank} ++ Returns:: The current value stored in `*this`, converted to `value_type`. ``` const char* data() const; ``` -[horizontal] +[none] +* {blank} ++ Returns:: A pointer to the first byte of the endian binary value stored in `*this`. @@ -422,8 +432,11 @@ friend std::basic_ostream& operator<<(std::basic_ostream& os, const T& x); ``` -[horizontal] -Returns:: `os << +x`.

+[none] +* {blank} ++ +Returns:: `os << +x`. +[none] ### Stream extractor @@ -432,7 +445,9 @@ template friend std::basic_istream& operator>>(std::basic_istream& is, T& x); ``` -[horizontal] +[none] +* {blank} ++ Effects:: As if: + ``` diff --git a/doc/endian/buffers.adoc b/doc/endian/buffers.adoc index cd85669..df67699 100644 --- a/doc/endian/buffers.adoc +++ b/doc/endian/buffers.adoc @@ -364,14 +364,18 @@ Template parameter `T` is required to be a standard integer type ({cpp}std, ``` endian_buffer() noexcept = default; ``` -[horizontal] +[none] +* {blank} ++ Effects:: Constructs an uninitialized object of type `endian_buffer`. ``` explicit endian_buffer(T v) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: Constructs an object of type `endian_buffer`. Postcondition:: `value() == v & mask`, where `mask` is a constant of type `value_type` with `Nbits` low-order bits set to one. @@ -381,7 +385,9 @@ is performed by `boost::endian::endian_reverse`. ``` endian_buffer& operator=(T v) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Postcondition:: `value() == v & mask`, where `mask` is a constant of type `value_type` with `Nbits` low-order bits set to one. Returns:: `*this`. @@ -391,7 +397,9 @@ performed by `boost::endian::endian_reverse`. ``` value_type value() const noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `endian_value`, converted to `value_type`, if required, and having the endianness of the native platform. Remarks:: If `Align` is `align::yes` then endianness conversion, if required, is @@ -400,8 +408,11 @@ performed by `boost::endian::endian_reverse`. ``` const char* data() const noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: A pointer to the first byte of `endian_value`. +[none] ### Non-member functions @@ -411,7 +422,9 @@ template & operator<<(std::basic_ostream& os, const endian_buffer& x); ``` -[horizontal] +[none] +* {blank} ++ Returns:: `os << x.value()`. ``` @@ -420,7 +433,9 @@ template & operator>>(std::basic_istream& is, endian_buffer& x); ``` -[horizontal] +[none] +* {blank} ++ Effects:: As if: + ``` diff --git a/doc/endian/conversion.adoc b/doc/endian/conversion.adoc index 30a955a..b3348c1 100644 --- a/doc/endian/conversion.adoc +++ b/doc/endian/conversion.adoc @@ -193,10 +193,12 @@ uint16_t endian_reverse(uint16_t x) noexcept; uint32_t endian_reverse(uint32_t x) noexcept; uint64_t endian_reverse(uint64_t x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `x`, with the order of its constituent bytes reversed. Remarks:: The type of `x` meets the `EndianReversible` requirements. - ++ NOTE: The Boost.Endian library does not provide overloads for the {cpp} standard library supplied types. @@ -204,88 +206,111 @@ library supplied types. template EndianReversible big_to_native(EndianReversible x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `conditional_reverse(x)`. ``` template EndianReversible native_to_big(EndianReversible x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `conditional_reverse(x)`. ``` template EndianReversible little_to_native(EndianReversible x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `conditional_reverse(x)`. ``` template EndianReversible native_to_little(EndianReversible x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `conditional_reverse(x)`. ``` template EndianReversible conditional_reverse(EndianReversible x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `x` if `O1 == O2,` otherwise `endian_reverse(x)`. Remarks:: Whether `x` or `endian_reverse(x)` is to be returned shall be determined at compile time. - ``` template EndianReversible conditional_reverse(EndianReversible x, order order1, order order2) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Returns:: `order1 == order2 ? x : endian_reverse(x)`. ``` template void endian_reverse_inplace(EndianReversible& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: `x` `= endian_reverse(x)`. ``` template void big_to_native_inplace(EndianReversibleInplace& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: `conditional_reverse_inplace(x)`. ``` template void native_to_big_inplace(EndianReversibleInplace& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: `conditional_reverse_inplace(x)`. ``` template void little_to_native_inplace(EndianReversibleInplace& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: `conditional_reverse_inplace(x)`. ``` template void native_to_little_inplace(EndianReversibleInplace& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: `conditional_reverse_inplace(x)`. ``` template void conditional_reverse_inplace(EndianReversibleInplace& x) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: None if `O1 == O2,` otherwise `endian_reverse_inplace(x)`. Remarks:: Which effect applies shall be determined at compile time. @@ -294,8 +319,11 @@ template void conditional_reverse_inplace(EndianReversibleInplace& x, order order1, order order2) noexcept; ``` -[horizontal] +[none] +* {blank} ++ Effects:: If `order1 == order2` then `endian_reverse_inplace(x)`. +[none] ## FAQ diff --git a/doc/endian/mini_review_topics.adoc b/doc/endian/mini_review_topics.adoc index 987a868..a614d16 100644 --- a/doc/endian/mini_review_topics.adoc +++ b/doc/endian/mini_review_topics.adoc @@ -14,109 +14,66 @@ resolved before a mini-review. The issues are shown in *bold* below, with the resolution indicated. -*Common use case scenarios should be developed.* - -{blank}:: +Common use case scenarios should be developed.:: Done. The documentation have been refactored. A page is now devoted to <> to endianness. See <> for use case scenarios. -*Example programs should be developed for the common use case scenarios.* - -{blank}:: +Example programs should be developed for the common use case scenarios.:: Done. See <>. Example code has been added throughout. -*Documentation should illuminate the differences between endian -integer/float type and endian conversion approaches to the common use case -scenarios, and provide guidelines for choosing the most appropriate approach in -user's applications.* - -{blank}:: +Documentation should illuminate the differences between endian integer/float type and endian conversion approaches to the common use case scenarios, and provide guidelines for choosing the most appropriate approach in user's applications.:: Done. See <>. -*Conversion functions supplying results via return should be provided.* - -{blank}:: +Conversion functions supplying results via return should be provided.:: Done. See <>. -*Platform specific performance enhancements such as use of compiler -intrinsics or relaxed alignment requirements should be supported.* - -{blank}:: +Platform specific performance enhancements such as use of compiler intrinsics or relaxed alignment requirements should be supported.:: Done. Compiler (Clang, GCC, Visual{cpp}, etc.) intrinsics and built-in functions are used in the implementation where appropriate, as requested. See <>. See <> to gauge the impact of intrinsics. -*Endian integer (and floating) types should be implemented via the -conversion functions. If that can't be done efficiently, consideration should -be given to expanding the conversion function signatures to resolve the -inefficiencies.* - -{blank}:: +Endian integer (and floating) types should be implemented via the conversion functions. If that can't be done efficiently, consideration should be given to expanding the conversion function signatures to resolve the inefficiencies.:: Done. For the endian types, the implementation uses the endian conversion functions, and thus the intrinsics, as requested. -*Benchmarks that measure performance should be provided. It should be -possible to compare platform specific performance enhancements against portable -base implementations, and to compare endian integer approaches against endian -conversion approaches for the common use case scenarios.* - -{blank}:: +Benchmarks that measure performance should be provided. It should be possible to compare platform specific performance enhancements against portable base implementations, and to compare endian integer approaches against endian conversion approaches for the common use case scenarios.:: Done. See <>. The `endian/test` directory also contains several additional benchmark and speed test programs. -*Float (32-bits) and double (64-bits) should be supported. IEEE 754 is -the primary use case.* - -{blank}:: +Float (32-bits) and double (64-bits) should be supported. IEEE 754 is the primary use case.:: Done. The <>, <> and <> now support 32-bit `(float)` and 64-bit `(double)` floating point, as requested. -*Support for user defined types (UDTs) is desirable, and should be -provided where there would be no conflict with the other concerns.* - -{blank}:: +Support for user defined types (UDTs) is desirable, and should be provided where there would be no conflict with the other concerns.:: Done. See <>. -*There is some concern that endian integer/float arithmetic operations -might used inadvertently or inappropriately. The impact of adding an -endian_buffer class without arithmetic operations should be investigated.* - -{blank}:: +There is some concern that endian integer/float arithmetic operations might used inadvertently or inappropriately. The impact of adding an endian_buffer class without arithmetic operations should be investigated.:: Done. The endian types have been decomposed into class template `<>` and class template `<>`. Class `endian_buffer` is a public base class for `endian_arithmetic`, and can also be used by users as a stand-alone class. -*Stream insertion and extraction of the endian integer/float types should -be documented and included in the test coverage.* - -{blank}:: +Stream insertion and extraction of the endian integer/float types should be documented and included in the test coverage.:: Done. See <> and <>. -*Binary I/O support that was investigated during development of the -Endian library should be put up for mini-review for inclusion in the Boost I/O -library.* - -{blank}:: +Binary I/O support that was investigated during development of the Endian library should be put up for mini-review for inclusion in the Boost I/O library.:: Not done yet. Will be handled as a separate min-review soon after the Endian mini-review. -*Other requested changes.* - -{blank}:: -* In addition to the named-endianness conversion functions, functions that +Other requested changes.:: + In addition to the named-endianness conversion functions, functions that perform compile-time (via template) and run-time (via function argument) dispatch are now provided. -* `order*native` is now a synonym for `order*big` or `order*little` according + `order*native` is now a synonym for `order*big` or `order*little` according to the endianness of the platform. This reduces the number of template specializations required. -* Headers have been reorganized to make them easier to read, with a synopsis + Headers have been reorganized to make them easier to read, with a synopsis at the front and implementation following. From 59b4062507311810cc0892fea70f4424b33b73e1 Mon Sep 17 00:00:00 2001 From: Glen Fernandes Date: Mon, 25 Mar 2019 09:35:59 -0400 Subject: [PATCH 08/12] Update index.html to point to new documentation --- index.html | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/index.html b/index.html index 3ebbb5a..7f4ed4d 100644 --- a/index.html +++ b/index.html @@ -1,14 +1,18 @@ + - + +

Automatic redirection failed, please go to -doc/index.html. 

-

© Copyright Beman Dawes, 2001

-

Distributed under the Boost Software License, Version 1.0. -(See www.boost.org/LICENSE_1_0.txt) - +doc/html/endian.html.

- \ No newline at end of file + + \ No newline at end of file From cdfe2eb3cb66cfcc4aa74d3f724f4f11a3d10506 Mon Sep 17 00:00:00 2001 From: Peter Dimov Date: Mon, 25 Mar 2019 15:45:34 +0200 Subject: [PATCH 09/12] Delete original .html files --- doc/arithmetic.html | 639 ------------------------------------ doc/bikeshed.txt | 79 ----- doc/buffers.html | 600 --------------------------------- doc/choosing_approach.html | 412 ----------------------- doc/conversion.html | 405 ----------------------- doc/fp_concerns.html | 97 ------ doc/index.html | 606 ---------------------------------- doc/mini_review_topics.html | 150 --------- doc/msvc_readme.txt | 31 -- doc/p0803r0.html | 134 -------- doc/publish.bat | 8 - doc/std-rfc.html | 140 -------- doc/styles.css | 20 -- doc/todo_list.html | 106 ------ 14 files changed, 3427 deletions(-) delete mode 100644 doc/arithmetic.html delete mode 100644 doc/bikeshed.txt delete mode 100644 doc/buffers.html delete mode 100644 doc/choosing_approach.html delete mode 100644 doc/conversion.html delete mode 100644 doc/fp_concerns.html delete mode 100644 doc/index.html delete mode 100644 doc/mini_review_topics.html delete mode 100644 doc/msvc_readme.txt delete mode 100644 doc/p0803r0.html delete mode 100644 doc/publish.bat delete mode 100644 doc/std-rfc.html delete mode 100644 doc/styles.css delete mode 100644 doc/todo_list.html diff --git a/doc/arithmetic.html b/doc/arithmetic.html deleted file mode 100644 index 7c091c9..0000000 --- a/doc/arithmetic.html +++ /dev/null @@ -1,639 +0,0 @@ - - - - - - - -Endian Arithmetic Types - - - - - - - - - - - - -
- -Boost logo - - Endian Arithmetic Types -
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
- -

- - - - - - - - -
- Contents
- Introduction
- Example
- Limitations
- Feature set
- Enums and typedefs
- Class template endian
-     - Synopsis
-     Members
-    Stream inserter
-    Stream extractor
- FAQ
- Design
- Experience
- Motivating use cases
- C++11
- Compilation
- Acknowledgements -
-

Introduction

-

Header boost/endian/arithmetic.hpp -provides integer binary types with control over -byte order, value type, size, and alignment. Typedefs provide easy-to-use names -for common configurations.

-

These types provide portable byte-holders for integer data, independent of -particular computer architectures. Use cases almost always involve I/O, either via files or -network connections. Although data portability is the primary motivation, these -integer byte-holders may -also be used to reduce memory use, file size, or network activity since they -provide binary integer sizes not otherwise available.

-

Such integer byte-holder types are traditionally called -endian types. See the -Wikipedia for -a full -exploration of endianness, including definitions of big -endian and little endian.

-

Boost endian integers provide the same full set of C++ assignment, -arithmetic, and relational operators as C++ standard integral types, with -the standard semantics.

-

Unary arithmetic operators are +, - -, ~, -!, plus both prefix and postfix -- and ++. Binary -arithmetic operators are +, +=, -, - --=, *, *=, /, - /=, &, &=, - |, |=, -^, ^=, <<, <<=, ->>, and ->>=. Binary relational operators are ==, - !=, -<, <=, >, -and >=.

-

Implicit conversion to the underlying value type is provided. An implicit -constructor converting from the underlying value type is provided.

-

Example

-

The endian_example.cpp program writes a -binary file containing four-byte, big-endian and little-endian integers:

-
- 
#include <iostream>
-#include <cstdio>
-#include <boost/endian/arithmetic.hpp>
-#include <boost/static_assert.hpp>
-
-using namespace boost::endian;
-
-namespace 
-{
-  //  This is an extract from a very widely used GIS file format.
-  //  Why the designer decided to mix big and little endians in
-  //  the same file is not known. But this is a real-world format
-  //  and users wishing to write low level code manipulating these
-  //  files have to deal with the mixed endianness.
-
-  struct header
-  {
-    big_int32_t     file_code;
-    big_int32_t     file_length;
-    little_int32_t  version;
-    little_int32_t  shape_type;
-  };
-
-  const char* filename = "test.dat";
-}
-
-int main(int, char* [])
-{
-  header h;
-
-  BOOST_STATIC_ASSERT(sizeof(h) == 16U);  // reality check
-  
-  h.file_code   = 0x01020304;
-  h.file_length = sizeof(header);
-  h.version     = 1;
-  h.shape_type  = 0x01020304;
-
-  //  Low-level I/O such as POSIX read/write or <cstdio>
-  //  fread/fwrite is sometimes used for binary file operations
-  //  when ultimate efficiency is important. Such I/O is often
-  //  performed in some C++ wrapper class, but to drive home the
-  //  point that endian integers are often used in fairly
-  //  low-level code that does bulk I/O operations, <cstdio>
-  //  fopen/fwrite is used for I/O in this example.
-
-  std::FILE* fi = std::fopen(filename, "wb");  // MUST BE BINARY
-  
-  if (!fi)
-  {
-    std::cout << "could not open " << filename << '\n';
-    return 1;
-  }
-
-  if (std::fwrite(&h, sizeof(header), 1, fi)!= 1)
-  {
-    std::cout << "write failure for " << filename << '\n';
-    return 1;
-  }
-
-  std::fclose(fi);
-
-  std::cout << "created file " << filename << '\n';
-
-  return 0;
-}
-
-
-

After compiling and executing endian_example.cpp, -a hex dump of test.dat shows:

-
- 
01020304 00000010 01000000 04030201
-
-

Notice that the first two 32-bit integers are big endian while the second two -are little endian, even though the machine this was compiled and run on was -little endian.

-

Limitations

-

Requires <climits> CHAR_BIT == 8. If CHAR_BIT -is some other value, compilation will result in an #error. This -restriction is in place because the design, implementation, testing, and -documentation has only considered issues related to 8-bit bytes, and there have -been no real-world use cases presented for other sizes.

-

In C++03, endian_arithmetic does not meet the requirements for POD types -because it has constructors, private data members, and a base class. This means -that common use cases are relying on unspecified behavior in that the C++ -Standard does not guarantee memory layout for non-POD types. This has not been a -problem in practice since all known C++ compilers lay out memory as if -endian were a POD type. In C++11, it is possible to specify the -default constructor as trivial, and private data members and base classes no longer disqualify a type from being a POD -type. Thus under C++11, endian_arithmetic -will no longer be relying on unspecified behavior.

-

Feature set

-
    -
  • Big endian| little endian | native endian byte ordering.
    • -
  • Signed | unsigned
    • -
  • Unaligned | aligned
    • -
  • 1-8 byte (unaligned) | 1, 2, 4, 8 byte (aligned)
    • -
  • Choice of value type
    • -
-

Enums and typedefs

-

Two scoped enums are provided:

-
- 
enum class order {big, little, native};
-
-enum class align {no, yes};
-
-

One class template is provided:

-
- 
template <order Order, typename T, std::size_t n_bits,
-  align Align = align::no>
-class endian_arithmetic;
-
-
-

Typedefs, such as big_int32_t, provide convenient naming -conventions for common use cases:

-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NameAlignmentEndiannessSignSizes in bits (n)
big_intn_tnobigsigned8,16,24,32,40,48,56,64
big_uintn_tnobigunsigned8,16,24,32,40,48,56,64
little_intn_tnolittlesigned8,16,24,32,40,48,56,64
little_uintn_tnolittleunsigned8,16,24,32,40,48,56,64
native_intn_tnonativesigned8,16,24,32,40,48,56,64
native_uintn_tnonativeunsigned8,16,24,32,40,48,56,64
big_intn_atyesbigsigned8,16,32,64
big_uintn_atyesbigunsigned8,16,32,64
little_intn_atyeslittlesigned8,16,32,64
little_uintn_atyeslittleunsigned8,16,32,64
-
-

The unaligned types do not cause compilers to insert padding bytes in classes -and structs. This is an important characteristic that can be exploited to minimize wasted space in -memory, files, and network transmissions.

-

Warning: -Code that uses aligned types is possibly non-portable because alignment -requirements vary between hardware architectures and because alignment may be -affected by compiler switches or pragmas. For example, alignment of an 64-bit -integer may be to a 32-bit boundary on a 32-bit machine. Furthermore, aligned types -are only available on architectures with 8, 16, 32, and 64-bit integer types.

-

Recommendation: Prefer unaligned arithmetic types.

-

Recommendation: Protect yourself against alignment ills. For -example:

-
- 
static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong");
-
-

Note: Note: One-byte arithmetic types -have identical layout on all platforms, so they never actually reverse endianness. They are provided to enable generic code, and -to improve code readability and searchability.

-

Class template endian_arithmetic

-

An endian_integer is an integer byte-holder with user-specified -endianness, value type, size, and alignment. The -usual operations on arithmetic types are supplied.

-

Synopsis

-
#include <boost/endian/conversion.hpp>
-#include <boost/endian/buffers.hpp>
-
-namespace boost
-{
-  namespace endian
-  {
-    //  C++11 features emulated if not available
-   
-    enum class align {no, yes};            
-
-    template <order Order, class T, std::size_t n_bits,
-      align Align = align::no>
-    class endian_arithmetic
-      : public endian_buffer<Order, T, n_bits, Align>
-    {
-    public:
-      typedef T value_type;
-
-      // if BOOST_ENDIAN_FORCE_PODNESS is defined && C++11 PODs are not
-      // available then these two constructors will not be present
-      endian_arithmetic() noexcept = default;
-      endian_arithmetic(T v) noexcept;
-
-      endian_arithmetic& operator=(T v) noexcept;
-      operator value_type() const noexcept;
-      value_type value() const noexcept; // for exposition; see endian_buffer
-      const char* data() const noexcept; // for exposition; see endian_buffer
-
-      // arithmetic operations
-      //   note that additional operations are provided by the value_type 
-      value_type operator+(const endian& x) noexcept;
-      endian& operator+=(endian& x, value_type y) noexcept;
-      endian& operator-=(endian& x, value_type y) noexcept;
-      endian& operator*=(endian& x, value_type y) noexcept;
-      endian& operator/=(endian& x, value_type y) noexcept;
-      endian& operator%=(endian& x, value_type y) noexcept;
-      endian& operator&=(endian& x, value_type y) noexcept;
-      endian& operator|=(endian& x, value_type y) noexcept;
-      endian& operator^=(endian& x, value_type y) noexcept;
-      endian& operator<<=(endian& x, value_type y) noexcept;
-      endian& operator>>=(endian& x, value_type y noexcept;
-      value_type operator<<(const endian& x, value_type y) noexcept;
-      value_type operator>>(const endian& x, value_type y) noexcept;
-      endian& operator++(endian& x) noexcept;
-      endian& operator--(endian& x) noexcept;
-      endian operator++(endian& x, int) noexcept;
-      endian operator--(endian& x, int) noexcept;
-
-      // Stream inserter
-      template <class charT, class traits>
-      friend std::basic_ostream<charT, traits>&
-        operator<<(std::basic_ostream<charT, traits>& os, const T& x);
-
-      // Stream extractor 
-      template <class charT, class traits>
-      friend std::basic_istream<charT, traits>&
-        operator>>(std::basic_istream<charT, traits>& is, T& x);
-    };
-
-    // typedefs  
-
-    // unaligned big endian signed integer types
-    typedef endian<order::big, int_least8_t, 8>        big_int8_t;
-    typedef endian<order::big, int_least16_t, 16>      big_int16_t;
-    typedef endian<order::big, int_least32_t, 24>      big_int24_t;
-    typedef endian<order::big, int_least32_t, 32>      big_int32_t;
-    typedef endian<order::big, int_least64_t, 40>      big_int40_t;
-    typedef endian<order::big, int_least64_t, 48>      big_int48_t;
-    typedef endian<order::big, int_least64_t, 56>      big_int56_t;
-    typedef endian<order::big, int_least64_t, 64>      big_int64_t;
-  
-    // unaligned big endian unsigned integer types
-    typedef endian<order::big, uint_least8_t, 8>       big_uint8_t;
-    typedef endian<order::big, uint_least16_t, 16>     big_uint16_t;
-    typedef endian<order::big, uint_least32_t, 24>     big_uint24_t;
-    typedef endian<order::big, uint_least32_t, 32>     big_uint32_t;
-    typedef endian<order::big, uint_least64_t, 40>     big_uint40_t;
-    typedef endian<order::big, uint_least64_t, 48>     big_uint48_t;
-    typedef endian<order::big, uint_least64_t, 56>     big_uint56_t;
-    typedef endian<order::big, uint_least64_t, 64>     big_uint64_t;
-
-    // unaligned little endian signed integer types
-    typedef endian<order::little, int_least8_t, 8>     little_int8_t;
-    typedef endian<order::little, int_least16_t, 16>   little_int16_t;
-    typedef endian<order::little, int_least32_t, 24>   little_int24_t;
-    typedef endian<order::little, int_least32_t, 32>   little_int32_t;
-    typedef endian<order::little, int_least64_t, 40>   little_int40_t;
-    typedef endian<order::little, int_least64_t, 48>   little_int48_t;
-    typedef endian<order::little, int_least64_t, 56>   little_int56_t;
-    typedef endian<order::little, int_least64_t, 64>   little_int64_t;
-  
-    // unaligned little endian unsigned integer types
-    typedef endian<order::little, uint_least8_t, 8>    little_uint8_t;
-    typedef endian<order::little, uint_least16_t, 16>  little_uint16_t;
-    typedef endian<order::little, uint_least32_t, 24>  little_uint24_t;
-    typedef endian<order::little, uint_least32_t, 32>  little_uint32_t;
-    typedef endian<order::little, uint_least64_t, 40>  little_uint40_t;
-    typedef endian<order::little, uint_least64_t, 48>  little_uint48_t;
-    typedef endian<order::little, uint_least64_t, 56>  little_uint56_t;
-    typedef endian<order::little, uint_least64_t, 64>  little_uint64_t;
-
-    // unaligned native endian signed integer types
-    typedef implementation-defined_int8_t   native_int8_t;
-    typedef implementation-defined_int16_t  native_int16_t;
-    typedef implementation-defined_int24_t  native_int24_t;
-    typedef implementation-defined_int32_t  native_int32_t;
-    typedef implementation-defined_int40_t  native_int40_t;
-    typedef implementation-defined_int48_t  native_int48_t;
-    typedef implementation-defined_int56_t  native_int56_t;
-    typedef implementation-defined_int64_t  native_int64_t;
-
-    // unaligned native endian unsigned integer types
-    typedef implementation-defined_uint8_t   native_uint8_t;
-    typedef implementation-defined_uint16_t  native_uint16_t;
-    typedef implementation-defined_uint24_t  native_uint24_t;
-    typedef implementation-defined_uint32_t  native_uint32_t;
-    typedef implementation-defined_uint40_t  native_uint40_t;
-    typedef implementation-defined_uint48_t  native_uint48_t;
-    typedef implementation-defined_uint56_t  native_uint56_t;
-    typedef implementation-defined_uint64_t  native_uint64_t;
-    
-    // aligned big endian signed integer types
-    typedef endian<order::big, int8_t, 8, align::yes>       big_int8_at;
-    typedef endian<order::big, int16_t, 16, align::yes>     big_int16_at;
-    typedef endian<order::big, int32_t, 32, align::yes>     big_int32_at;
-    typedef endian<order::big, int64_t, 64, align::yes>     big_int64_at;
-
-    // aligned big endian unsigned integer types
-    typedef endian<order::big, uint8_t, 8, align::yes>      big_uint8_at;
-    typedef endian<order::big, uint16_t, 16, align::yes>    big_uint16_at;
-    typedef endian<order::big, uint32_t, 32, align::yes>    big_uint32_at;
-    typedef endian<order::big, uint64_t, 64, align::yes>    big_uint64_at;
-
-    // aligned little endian signed integer types
-    typedef endian<order::little, int8_t, 8, align::yes>    little_int8_at;
-    typedef endian<order::little, int16_t, 16, align::yes>  little_int16_at;
-    typedef endian<order::little, int32_t, 32, align::yes>  little_int32_at;
-    typedef endian<order::little, int64_t, 64, align::yes>  little_int64_at;
-
-    // aligned little endian unsigned integer types
-    typedef endian<order::little, uint8_t, 8, align::yes>   little_uint8_at;
-    typedef endian<order::little, uint16_t, 16, align::yes> little_uint16_at;
-    typedef endian<order::little, uint32_t, 32, align::yes> little_uint32_at;
-    typedef endian<order::little, uint64_t, 64, align::yes> little_uint64_at;
-
-    // aligned native endian typedefs are not provided because
-    // <cstdint> types are superior for that use case
-  
-  } // namespace endian
-} // namespace boost
-

The implementation-defined text above is either -big or little according to the endianness of the -platform.

-

Members

-
- 
endian() = default;  // C++03: endian(){}
-
-
-

Effects: Constructs an uninitialized object of type endian_arithmetic<E, T, n_bits, A>.

-
-
endian(T v);
-
-

Effects: Constructs an object of type endian_arithmetic<E, T, n_bits, A>.

-

Postcondition: x == v, where x is the -constructed object.

-
-
endian& operator=(T v);
-
-

Postcondition: x == v, where x is the - constructed object.

-

Returns: *this.

-
-
operator T() const;
-
-

Returns: The current value stored in *this, converted to -value_type.

-
-
const char* data() const;
-
-

Returns: A pointer to the first byte of the endian binary value stored -in *this.

-
-

Other operators

-

Other operators on endian objects are forwarded to the equivalent -operator on value_type.

-

Stream inserter

-
template <class charT, class traits>
-friend std::basic_ostream<charT, traits>&
-  operator<<(std::basic_ostream<charT, traits>& os, const T& x);
-
-
-

Returns: os << +x.

-
-

Stream extractor

-
template <class charT, class traits>
-friend std::basic_istream<charT, traits>&
-  operator>>(std::basic_istream<charT, traits>& is, T& x);
-
-
-

Effects: As if:

-
- 
T i;
-if (is >> i)
-  x = i;
-
-
-

Returns: is.

-
-

FAQ

- -

See the Endian home page FAQ for a library-wide -FAQ.

- -

Why not just use Boost.Serialization? Serialization involves a -conversion for every object involved in I/O. Endian integers require no -conversion or copying. They are already in the desired format for binary I/O. -Thus they can be read or written in bulk.

-

Are endian types PODs? Yes for C++11. No for C++03, although several -macros are available to force PODness in all cases.

-

What are the implications of endian integer types not being PODs with C++03 -compilers? They -can't be used in unions. Also, compilers aren't required to align or lay -out storage in portable ways, although this potential problem hasn't prevented -use of Boost.Endian with -real compilers.

-

What good is native endianness? It provides alignment and -size guarantees not available from the built-in types. It eases generic -programming.

-

Why bother with the aligned endian types? Aligned integer operations -may be faster (as much as 10 to 20 times faster) if the endianness and alignment of -the type matches the endianness and alignment requirements of the machine. The code, -however, will be somewhat less portable than with the unaligned types.

-

Why provide the arithmetic operations? Providing a full set of operations reduces program -clutter and makes code both easier to write and to read. Consider -incrementing a variable in a record. It is very convenient to write:

-
    ++record.foo;
-

Rather than:

-
    int temp(record.foo);
-    ++temp;
-    record.foo = temp;
-

Design considerations for Boost.Endian types

-
    -
  • Must be suitable for I/O - in other words, must be memcpyable.
    • -
  • Must provide exactly the size and internal byte ordering specified.
    • -
  • Must work correctly when the internal integer representation has more bits - that the sum of the bits in the external byte representation. Sign extension - must work correctly when the internal integer representation type has more - bits than the sum of the bits in the external bytes. For example, using - a 64-bit integer internally to represent 40-bit (5 byte) numbers must work for - both positive and negative values.
    • -
  • Must work correctly (including using the same defined external - representation) regardless of whether a compiler treats char as signed or - unsigned.
    • -
  • Unaligned types must not cause compilers to insert padding bytes.
    • -
  • The implementation should supply optimizations with great care. Experience has shown that optimizations of endian - integers often become pessimizations when changing - machines or compilers. Pessimizations can also happen when changing compiler switches, - compiler versions, or CPU models of the same architecture.
    • -
-

Experience

-

Classes with similar functionality have been independently developed by -several Boost programmers and used very successful in high-value, high-use -applications for many years. These independently developed endian libraries -often evolved from C libraries that were also widely used. Endian types have proven widely useful across a wide -range of computer architectures and applications.

-

Motivating use cases

-

Neil Mayhew writes: "I can also provide a meaningful use-case for this -library: reading TrueType font files from disk and processing the contents. The -data format has fixed endianness (big) and has unaligned values in various -places. Using Boost.Endian simplifies and cleans the code wonderfully."

-

C++11

-

The availability of the C++11 - -Defaulted Functions feature is detected automatically, and will be used if -present to ensure that objects of class endian_arithmetic are trivial, and -thus PODs.

-

Compilation

-

Boost.Endian is implemented entirely within headers, with no need to link to -any Boost object libraries.

-

Several macros allow user control over features:

-
    -
  • BOOST_ENDIAN_NO_CTORS causes class endian_arithmetic to have no - constructors. The intended use is for compiling user code that must be - portable between compilers regardless of C++11 - - Defaulted Functions support. Use of constructors will always fail,
    - 
    • -
  • BOOST_ENDIAN_FORCE_PODNESS causes BOOST_ENDIAN_NO_CTORS to be defined if - the compiler does not support C++11 - - Defaulted Functions. This is ensures that objects of class endian_arithmetic - are PODs, and so can be used in C++03 unions. - In C++11, class endian_arithmetic objects are PODs, even though they have - constructors, so can always be used in unions.
    • -
-

Acknowledgements

-

Original design developed by Darin Adler based on classes developed by Mark -Borgerding. Four original class templates combined into a single endian_arithmetic -class template by Beman Dawes, who put the library together, provided -documentation, added the typedefs, and also added the unrolled_byte_loops -sign partial specialization to correctly extend the sign when cover integer size -differs from endian representation size.

-
-

Last revised: -14 October, 2015

-

© Copyright Beman Dawes, 2006-2009, 2013

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/ LICENSE_1_0.txt

- - - - \ No newline at end of file diff --git a/doc/bikeshed.txt b/doc/bikeshed.txt deleted file mode 100644 index da17001..0000000 --- a/doc/bikeshed.txt +++ /dev/null @@ -1,79 +0,0 @@ -Conversion function naming bikeshed - -return-by-value modify-argument ------------------- --------------- - -reverse_endianness reverse_endianness_in_place - " reverse_endianness_arg -endian_reverse endian_reverse_in_place - " endian_reverse_inplace - " endian_reverse_replace - " endian_reverse_in_situ - " endian_reverse_here - " endian_reverse_this - " endian_reverse_self - " endian_reverse_arg - " endian_reverse_in - -reverse reverse_in_place -reverse_endian reverse_endian_in_place - -swap_endianness swap_endianness_in_place -swap_endian swap_endian_in_place -endian_swap endian_swap_this - -flip_endianness flip_endianness_in_place -flip_endian flip_endian_in_place -endian_flip endian_flip_in_place - - -reverse_order reverse_order_in_place - - -Key points: - -* The above names are defined in a user namespace as customization points to be found by - ADL, and so cannot depend on the enclosing namespace name to signal readers that they - are related to endianness. -* The above functions are rarely called directly by user code, which is more likely to use - the various conditional functions instead. So explicitness is more important than - brevity. - -Conditional names - -big_to_native native_to_big little_to_native native_to_little - -big_to_host host_to_big - -be_to_ne ne_to_be - -from_big, to_big - -big_to_native big_to_native -native_to_big native_to_big - -conditional_reverse runtime_conditional_reverse -conditional_reverse conditional_reverse <------ - -merriam-webster.com/dictionary - -reverse [1] (adjective): opposite or contrary to a previous or normal condition -reverse [2] (verb) : to change (something) to an opposite state or condition - -swap (verb) : to give something to someone and receive something in return : to trade or exchange (things) - -flip (verb) - -: to turn (something) over by throwing it up in the air with a quick movement - -: to cause (something) to turn or turn over quickly - -: to move (something) with a quick light movement - --------------------------------------------------- - -Copyright Beman Dawes, 2014 -Distributed under the Boost Software License, Version 1.0. -See www.boost.org/LICENSE_1_0.txt - - \ No newline at end of file diff --git a/doc/buffers.html b/doc/buffers.html deleted file mode 100644 index 8968459..0000000 --- a/doc/buffers.html +++ /dev/null @@ -1,600 +0,0 @@ - - - - - - - -Endian Buffer Types - - - - - - - - - - - - -
- -Boost logo - - Endian Buffer Types -
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
- -

- - - - - - - - -
- Contents
- Introduction
- Example
- Limitations
- Feature set
- Enums and typedefs
- Class template endian_buffer
-     - Synopsis
-     Members
-    Non-Members
- FAQ
- Design
- C++11
- Compilation
-

Introduction

-

The internal byte order of arithmetic types is traditionally called endianness. See -the -Wikipedia for -a full -exploration of endianness, including definitions of big -endian and little endian.

-

Header boost/endian/buffers.hpp -provides endian_buffer, a portable endian integer binary buffer -class template with control over -byte order, value type, size, and alignment independent of the platform's native -endianness. Typedefs provide easy-to-use names -for common configurations.

-

Use cases primarily involve data portability, either via files or network -connections, but these byte-holders may -also be used to reduce memory use, file size, or network activity since they - -provide binary numeric sizes not otherwise available.

-

Class endian_buffer is aimed at users who wish -explicit control over when endianness conversions occur. It also serves as the -base class for the endian_arithmetic -class template, which is aimed at users who wish fully automatic endianness -conversion and direct support for all normal arithmetic operations.

-

Example

-

The example/endian_example.cpp program writes a -binary file containing four-byte, big-endian and little-endian integers:

-
- 
#include <iostream>
-#include <cstdio>
-#include <boost/endian/buffers.hpp>  // see Synopsis below
-#include <boost/static_assert.hpp>
-
-using namespace boost::endian;
-
-namespace 
-{
-  //  This is an extract from a very widely used GIS file format.
-  //  Why the designer decided to mix big and little endians in
-  //  the same file is not known. But this is a real-world format
-  //  and users wishing to write low level code manipulating these
-  //  files have to deal with the mixed endianness.
-
-  struct header
-  {
-    big_int32_buf_t     file_code;
-    big_int32_buf_t     file_length;
-    little_int32_buf_t  version;
-    little_int32_buf_t  shape_type;
-  };
-
-  const char* filename = "test.dat";
-}
-
-int main(int, char* [])
-{
-  header h;
-
-  BOOST_STATIC_ASSERT(sizeof(h) == 16U);  // reality check
-  
-  h.file_code   = 0x01020304;
-  h.file_length = sizeof(header);
-  h.version     = 1;
-  h.shape_type  = 0x01020304;
-
-  //  Low-level I/O such as POSIX read/write or <cstdio>
-  //  fread/fwrite is sometimes used for binary file operations
-  //  when ultimate efficiency is important. Such I/O is often
-  //  performed in some C++ wrapper class, but to drive home the
-  //  point that endian integers are often used in fairly
-  //  low-level code that does bulk I/O operations, <cstdio>
-  //  fopen/fwrite is used for I/O in this example.
-
-  std::FILE* fi = std::fopen(filename, "wb");  // MUST BE BINARY
-  
-  if (!fi)
-  {
-    std::cout << "could not open " << filename << '\n';
-    return 1;
-  }
-
-  if (std::fwrite(&h, sizeof(header), 1, fi)!= 1)
-  {
-    std::cout << "write failure for " << filename << '\n';
-    return 1;
-  }
-
-  std::fclose(fi);
-
-  std::cout << "created file " << filename << '\n';
-
-  return 0;
-}
-
-
-

After compiling and executing example/endian_example.cpp, -a hex dump of test.dat shows:

-
- 
01020304 00000010 01000000 04030201
-
-

Notice that the first two 32-bit integers are big endian while the second two -are little endian, even though the machine this was compiled and run on was -little endian.

-

Limitations

-

Requires <climits> CHAR_BIT == 8. If CHAR_BIT -is some other value, compilation will result in an #error. This -restriction is in place because the design, implementation, testing, and -documentation has only considered issues related to 8-bit bytes, and there have -been no real-world use cases presented for other sizes.

-

In C++03, endian_buffer does not meet the requirements for POD types -because it has constructors, private data members, and a base class. This means -that common use cases are relying on unspecified behavior in that the C++ -Standard does not guarantee memory layout for non-POD types. This has not been a -problem in practice since all known C++ compilers lay out memory as if -endian were a POD type. In C++11, it is possible to specify the -default constructor as trivial, and private data members and base classes no longer disqualify a type from being a POD -type. Thus under C++11, endian_buffer -will no longer be relying on unspecified behavior.

-

Feature set

-
    -
  • Big endian| little endian | native endian byte ordering.
    • -
  • Signed | unsigned
    • -
  • Unaligned | aligned
    • -
  • 1-8 byte (unaligned) | 1, 2, 4, 8 byte (aligned)
    • -
  • Choice of value type
    • -
-

Enums and typedefs

-

Two scoped enums are provided:

-
- 
enum class order {big, little, native};
-
-enum class align {no, yes};
-
-

One class template is provided:

-
- 
template <order Order, typename T, std::size_t Nbits,
-  align Align = align::no>
-class endian_buffer;
-
-
-

Typedefs, such as big_int32_buf_t, provide convenient naming -conventions for common use cases:

-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NameAlignmentEndiannessSignSizes in bits (n)
big_intn_buf_tnobigsigned8,16,24,32,40,48,56,64
big_uintn_buf_tnobigunsigned8,16,24,32,40,48,56,64
little_intn_buf_tnolittlesigned8,16,24,32,40,48,56,64
little_uintn_buf_tnolittleunsigned8,16,24,32,40,48,56,64
native_intn_buf_tnonativesigned8,16,24,32,40,48,56,64
native_uintn_buf_tnonativeunsigned8,16,24,32,40,48,56,64
big_intn_buf_atyesbigsigned8,16,32,64
big_uintn_buf_atyesbigunsigned8,16,32,64
little_intn_buf_atyeslittlesigned8,16,32,64
little_uintn_buf_atyeslittleunsigned8,16,32,64
-
-

The unaligned types do not cause compilers to insert padding bytes in classes -and structs. This is an important characteristic that can be exploited to minimize wasted space in -memory, files, and network transmissions.

-

Warning: -Code that uses aligned types is possibly non-portable because alignment -requirements vary between hardware architectures and because alignment may be -affected by compiler switches or pragmas. For example, alignment of an 64-bit -integer may be to a 32-bit boundary on a 32-bit machine and to a 64-bit boundary -on a 64-bit machine. Furthermore, aligned types -are only available on architectures with 8, 16, 32, and 64-bit integer types.

-

Recommendation: Prefer unaligned buffer types.

-

Recommendation: Protect yourself against alignment ills. For -example:

-
- 
static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong");
-
-

Note: One-byte big and little buffer types -have identical layout on all platforms, so they never actually reverse endianness. They are provided to enable generic code, and -to improve code readability and searchability.

-

Class template endian_buffer

-

An endian_buffer is a byte-holder for arithmetic types with user-specified -endianness, value type, size, and alignment.

-

Synopsis

-
#include <boost/endian/conversion.hpp
-
-namespace boost
-{
-  namespace endian
-  {
-    //  C++11 features emulated if not available
-   
-    enum class align {no, yes};            
-
-    template <order Order, class T, std::size_t Nbits,
-      align Align = align::no>
-    class endian_buffer
-    {
-    public:
-      typedef T value_type;
-
-      endian_buffer() noexcept = default;
-      explicit endian_buffer(T v) noexcept;
-
-      endian_buffer& operator=(T v) noexcept;
-      value_type     value() const noexcept;
-      const char*    data() const noexcept;
-    protected:
-      implementaton-defined  endian_value;  // for exposition only
-    };
-    
-    //  stream inserter
-    template <class charT, class traits, order Order, class T,
-      std::size_t n_bits, align Align>
-    std::basic_ostream<charT, traits>&
-      operator<<(std::basic_ostream<charT, traits>& os,
-        const endian_buffer<Order, T, n_bits, Align>& x);
-
-    //  stream extractor 
-    template <class charT, class traits, order Order, class T,
-      std::size_t n_bits, align A>
-    std::basic_istream<charT, traits>&
-      operator>>(std::basic_istream<charT, traits>& is,
-        endian_buffer<Order, T, n_bits, Align>& x);
-
-    // typedefs  
-
-    // unaligned big endian signed integer buffers
-    typedef endian_buffer<order::big, int_least8_t, 8>        big_int8_buf_t;
-    typedef endian_buffer<order::big, int_least16_t, 16>      big_int16_buf_t;
-    typedef endian_buffer<order::big, int_least32_t, 24>      big_int24_buf_t;
-    typedef endian_buffer<order::big, int_least32_t, 32>      big_int32_buf_t;
-    typedef endian_buffer<order::big, int_least64_t, 40>      big_int40_buf_t;
-    typedef endian_buffer<order::big, int_least64_t, 48>      big_int48_buf_t;
-    typedef endian_buffer<order::big, int_least64_t, 56>      big_int56_buf_t;
-    typedef endian_buffer<order::big, int_least64_t, 64>      big_int64_buf_t;
-  
-    // unaligned big endian unsigned integer buffers
-    typedef endian_buffer<order::big, uint_least8_t, 8>       big_uint8_buf_t;
-    typedef endian_buffer<order::big, uint_least16_t, 16>     big_uint16_buf_t;
-    typedef endian_buffer<order::big, uint_least32_t, 24>     big_uint24_buf_t;
-    typedef endian_buffer<order::big, uint_least32_t, 32>     big_uint32_buf_t;
-    typedef endian_buffer<order::big, uint_least64_t, 40>     big_uint40_buf_t;
-    typedef endian_buffer<order::big, uint_least64_t, 48>     big_uint48_buf_t;
-    typedef endian_buffer<order::big, uint_least64_t, 56>     big_uint56_buf_t;
-    typedef endian_buffer<order::big, uint_least64_t, 64>     big_uint64_buf_t;
-  
-    // unaligned little endian signed integer buffers
-    typedef endian_buffer<order::little, int_least8_t, 8>     little_int8_buf_t;
-    typedef endian_buffer<order::little, int_least16_t, 16>   little_int16_buf_t;
-    typedef endian_buffer<order::little, int_least32_t, 24>   little_int24_buf_t;
-    typedef endian_buffer<order::little, int_least32_t, 32>   little_int32_buf_t;
-    typedef endian_buffer<order::little, int_least64_t, 40>   little_int40_buf_t;
-    typedef endian_buffer<order::little, int_least64_t, 48>   little_int48_buf_t;
-    typedef endian_buffer<order::little, int_least64_t, 56>   little_int56_buf_t;
-    typedef endian_buffer<order::little, int_least64_t, 64>   little_int64_buf_t;
-  
-    // unaligned little endian unsigned integer buffers
-    typedef endian_buffer<order::little, uint_least8_t, 8>    little_uint8_buf_t;
-    typedef endian_buffer<order::little, uint_least16_t, 16>  little_uint16_buf_t;
-    typedef endian_buffer<order::little, uint_least32_t, 24>  little_uint24_buf_t;
-    typedef endian_buffer<order::little, uint_least32_t, 32>  little_uint32_buf_t;
-    typedef endian_buffer<order::little, uint_least64_t, 40>  little_uint40_buf_t;
-    typedef endian_buffer<order::little, uint_least64_t, 48>  little_uint48_buf_t;
-    typedef endian_buffer<order::little, uint_least64_t, 56>  little_uint56_buf_t;
-    typedef endian_buffer<order::little, uint_least64_t, 64>  little_uint64_buf_t;
-  
-    // unaligned native endian signed integer types
-    typedef implementation-defined_int8_buf_t   native_int8_buf_t;
-    typedef implementation-defined_int16_buf_t  native_int16_buf_t;
-    typedef implementation-defined_int24_buf_t  native_int24_buf_t;
-    typedef implementation-defined_int32_buf_t  native_int32_buf_t;
-    typedef implementation-defined_int40_buf_t  native_int40_buf_t;
-    typedef implementation-defined_int48_buf_t  native_int48_buf_t;
-    typedef implementation-defined_int56_buf_t  native_int56_buf_t;
-    typedef implementation-defined_int64_buf_t  native_int64_buf_t;
-
-    // unaligned native endian unsigned integer types
-    typedef implementation-defined_uint8_buf_t   native_uint8_buf_t;
-    typedef implementation-defined_uint16_buf_t  native_uint16_buf_t;
-    typedef implementation-defined_uint24_buf_t  native_uint24_buf_t;
-    typedef implementation-defined_uint32_buf_t  native_uint32_buf_t;
-    typedef implementation-defined_uint40_buf_t  native_uint40_buf_t;
-    typedef implementation-defined_uint48_buf_t  native_uint48_buf_t;
-    typedef implementation-defined_uint56_buf_t  native_uint56_buf_t;
-    typedef implementation-defined_uint64_buf_t  native_uint64_buf_t;
-    
-    // aligned big endian signed integer buffers
-    typedef endian_buffer<order::big, int8_t, 8, align::yes>       big_int8_buf_at;
-    typedef endian_buffer<order::big, int16_t, 16, align::yes>     big_int16_buf_at;
-    typedef endian_buffer<order::big, int32_t, 32, align::yes>     big_int32_buf_at;
-    typedef endian_buffer<order::big, int64_t, 64, align::yes>     big_int64_buf_at;
-  
-    // aligned big endian unsigned integer buffers
-    typedef endian_buffer<order::big, uint8_t, 8, align::yes>      big_uint8_buf_at;
-    typedef endian_buffer<order::big, uint16_t, 16, align::yes>    big_uint16_buf_at;
-    typedef endian_buffer<order::big, uint32_t, 32, align::yes>    big_uint32_buf_at;
-    typedef endian_buffer<order::big, uint64_t, 64, align::yes>    big_uint64_buf_at;
-  
-    // aligned little endian signed integer buffers
-    typedef endian_buffer<order::little, int8_t, 8, align::yes>    little_int8_buf_at;
-    typedef endian_buffer<order::little, int16_t, 16, align::yes>  little_int16_buf_at;
-    typedef endian_buffer<order::little, int32_t, 32, align::yes>  little_int32_buf_at;
-    typedef endian_buffer<order::little, int64_t, 64, align::yes>  little_int64_buf_at;
-  
-    // aligned little endian unsigned integer buffers
-    typedef endian_buffer<order::little, uint8_t, 8, align::yes>   little_uint8_buf_at;
-    typedef endian_buffer<order::little, uint16_t, 16, align::yes> little_uint16_buf_at;
-    typedef endian_buffer<order::little, uint32_t, 32, align::yes> little_uint32_buf_at;
-    typedef endian_buffer<order::little, uint64_t, 64, align::yes> little_uint64_buf_at;
-
-    // aligned native endian typedefs are not provided because
-    // <cstdint> types are superior for this use case
-  
-  } // namespace endian
-} // namespace boost
-

The implementation-defined text in typedefs above is either -big or little according to the native endianness of the -platform.

-

The expository data member endian_value stores the current value -of an endian_value object as a sequence of bytes ordered as -specified by the Order template parameter.  The -implementation-defined type of endian_value is a -type such as char[Nbits/CHAR_BIT] -or T that meets the -requirements imposed by the Nbits and Align template -parameters.  The CHAR_BIT -macro is defined in <climits>. -The only value of CHAR_BIT that -is required to be supported is 8.

-

Template parameter T is -required to be a standard integer type (C++std, 3.9.1) and -sizeof(T)*CHAR_BIT is required to be -greater or equal to Nbits.

-

Members

- 
endian_buffer() noexcept = default;
-
-

Effects: Constructs an uninitialized object of type endian_buffer<Order, T, -Nbits, Align>.

-
-
explicit endian_buffer(T v) noexcept;
-
-

Effects: Constructs an object of type endian_buffer<Order, T, -Nbits, Align>.

-

Postcondition: value() == v & mask, where mask -is a constant of type value_type with Nbits low-order -bits set to one.

-

Remarks: If Align is align::yes then -endianness conversion, if required, is performed by -boost::endian::endian_reverse.

-
-
endian_buffer& operator=(T v) noexcept;
-
-

Postcondition: value() == v & mask, where mask - is a constant of type value_type with Nbits - low-order bits set to one.

-

Returns: *this.

-

Remarks: If Align is align::yes then -endianness conversion, if required, is performed by -boost::endian::endian_reverse.

-
-
value_type value() const noexcept;
-
-

Returns: endian_value, converted to value_type, -if required, and having the endianness of the native platform.

-

Remarks: If Align is align::yes then -endianness conversion, if required, is performed by -boost::endian::endian_reverse.

-
-
const char* data() const noexcept;
-
-

Returns: A pointer to the first byte of endian_value.

-
-

Non-member functions

-
template <class charT, class traits, order Order, class T,
-  std::size_t n_bits, align Align>
-std::basic_ostream<charT, traits>& operator<<(std::basic_ostream<charT, traits>& os,
-  const endian_buffer<Order, T, n_bits, Align>& x);
-
-
-

Returns: os << x.value().

-
-
template <class charT, class traits, order Order, class T,
-  std::size_t n_bits, align A>
-std::basic_istream<charT, traits>& operator>>(std::basic_istream<charT, traits>& is,
-  endian_buffer<Order, T, n_bits, Align>& x);
-
-
-

Effects: As if:

-
- 
T i;
-if (is >> i)
-  x = i;
-
-
-

Returns: is.

-
-

FAQ

- -

See the Endian home page FAQ for a library-wide -FAQ.

- -

Why not just use Boost.Serialization? Serialization involves a -conversion for every object involved in I/O. Endian integers require no -conversion or copying. They are already in the desired format for binary I/O. -Thus they can be read or written in bulk.

-

Are endian types PODs? Yes for C++11. No for C++03, although several -macros are available to force PODness in all cases.

-

What are the implications of endian integer types not being PODs with C++03 -compilers? They -can't be used in unions. Also, compilers aren't required to align or lay -out storage in portable ways, although this potential problem hasn't prevented -use of Boost.Endian with -real compilers.

-

What good is native endianness? It provides alignment and -size guarantees not available from the built-in types. It eases generic -programming.

-

Why bother with the aligned endian types? Aligned integer operations -may be faster (as much as 10 to 20 times faster) if the endianness and alignment of -the type matches the endianness and alignment requirements of the machine. The code, -however, is -likely to be somewhat less portable than with the unaligned types.

-

Design considerations for Boost.Endian buffers

-
    -
  • Must be suitable for I/O - in other words, must be memcpyable.
    • -
  • Must provide exactly the size and internal byte ordering specified.
    • -
  • Must work correctly when the internal integer representation has more bits - that the sum of the bits in the external byte representation. Sign extension - must work correctly when the internal integer representation type has more - bits than the sum of the bits in the external bytes. For example, using - a 64-bit integer internally to represent 40-bit (5 byte) numbers must work for - both positive and negative values.
    • -
  • Must work correctly (including using the same defined external - representation) regardless of whether a compiler treats char as signed or - unsigned.
    • -
  • Unaligned types must not cause compilers to insert padding bytes.
    • -
  • The implementation should supply optimizations with great care. Experience has shown that optimizations of endian - integers often become pessimizations when changing - machines or compilers. Pessimizations can also happen when changing compiler switches, - compiler versions, or CPU models of the same architecture.
    • -
-

C++11

-

The availability of the C++11 - -Defaulted Functions feature is detected automatically, and will be used if -present to ensure that objects of class endian_buffer are trivial, and -thus PODs.

-

Compilation

-

Boost.Endian is implemented entirely within headers, with no need to link to -any Boost object libraries.

-

Several macros allow user control over features:

-
    -
  • BOOST_ENDIAN_NO_CTORS causes class endian_buffer to have no - constructors. The intended use is for compiling user code that must be - portable between compilers regardless of C++11 - - Defaulted Functions support. Use of constructors will always fail,
    - 
    • -
  • BOOST_ENDIAN_FORCE_PODNESS causes BOOST_ENDIAN_NO_CTORS to be defined if - the compiler does not support C++11 - - Defaulted Functions. This is ensures that objects of class endian_buffer - are PODs, and so can be used in C++03 unions. - In C++11, class endian_buffer objects are PODs, even though they have - constructors, so can always be used in unions.
    • -
-
-

Last revised: -14 October, 2015

-

© Copyright Beman Dawes, 2006-2009, 2013

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/ LICENSE_1_0.txt

- - - - \ No newline at end of file diff --git a/doc/choosing_approach.html b/doc/choosing_approach.html deleted file mode 100644 index effb79f..0000000 --- a/doc/choosing_approach.html +++ /dev/null @@ -1,412 +0,0 @@ - - - - - - -Choosing Approach - - - - - - - - - - -
- -Boost logo - Choosing the Approach
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
-

- - - - - - - - -
- Contents
-Introduction
-Choosing between conversion functions,
-   buffer types, and arithmetic types
-   Characteristics
-      Endianness invariants
-      Conversion explicitness
-      Arithmetic operations
-      Sizes
-      Alignments
-   Design patterns
-      Convert only as needed (i.e. lazy)
-      Convert in anticipation of need
-      Generally -as needed, locally in anticipation
-   Use case examples
-      Porting endian unaware codebase
-      Porting endian aware codebase
-      Reliability and arithmetic-speed
-      Reliability and ease-of-use
- -

Introduction

- -

Deciding which is the best endianness approach (conversion functions, buffer -types, or arithmetic types) for a particular application involves complex -engineering trade-offs. It is hard to assess those trade-offs without some -understanding of the different interfaces, so you might want to read the -conversion functions, -buffer types, and arithmetic types pages -before diving into this page.

- -

Choosing between conversion functions, buffer types, -and arithmetic types

- -

The best approach to endianness for a particular application depends on the interaction between -the application's needs and the characteristics of each of the three approaches.

- -

Recommendation: If you are new to endianness, uncertain, or don't want to invest -the time to -study -engineering trade-offs, use endian arithmetic types. They are safe, easy -to use, and easy to maintain. Use the - -anticipating need design pattern locally around performance hot spots -like lengthy loops, if needed.

- -

Background

- -

A dealing with endianness usually implies a program portability or a data -portability requirement, and often both. That means real programs dealing with -endianness are usually complex, so the examples shown here would really be -written as multiple functions spread across multiple translation units. They -would involve interfaces that can not be altered as they are supplied by -third-parties or the standard library.

- -

Characteristics

- -

The characteristics that differentiate the three approaches to endianness are the endianness -invariants, conversion explicitness, arithmetic operations, sizes available, and -alignment requirements.

- -

Endianness invariants

- -
- -

Endian conversion functions use objects of the ordinary C++ arithmetic -types like int or unsigned short to hold values. That -breaks the implicit invariant that the C++ language rules apply. The usual -language rules only apply if the endianness of the object is currently set to the native endianness for the platform. That can -make it very hard to reason about logic flow, and result in difficult to -find bugs.

- -

For example:

- -
- 
struct data_t  // big endian
-{
-  int32_t   v1;  // description ...
-  int32_t   v2;  // description ...
-  ... additional character data members (i.e. non-endian)
-  int32_t   v3;  // description ...
-};
-
-data_t data;
-
-read(data);
-big_to_native_inplace(data.v1);
-big_to_native_inplace(data.v2);
-
-... 
-
-++v1;
-third_party::func(data.v2);
-
-... 
-
-native_to_big_inplace(data.v1);
-native_to_big_inplace(data.v2);
-write(data);
-
-

The programmer didn't bother to convert data.v3 to native - endianness because that member isn't used. A later maintainer needs to pass - data.v3 to the third-party function, so adds third_party::func(data.v3); - somewhere deep in the code. This causes a silent failure because the usual - invariant that an object of type int32_t holds a value as - described by the C++ core language does not apply.

-
-

Endian buffer and arithmetic types hold values internally as arrays of -characters with an invariant that the endianness of the array never changes. -That makes these types easier to use and programs easier to maintain.

-

Here is the same example, using an endian arithmetic type:

-
- 
struct data_t
-{
-  big_int32_t   v1;  // description ...
-  big_int32_t   v2;  // description ...
-  ... additional character data members (i.e. non-endian)
-  big_int32_t   v3;  // description ...
-};
-
-data_t data;
-
-read(data);
-
-... 
-
-++v1;
-third_party::func(data.v2);
-
-... 
-
-write(data);
-
-

A later maintainer can add third_party::func(data.v3)and it - will just-work.

-
- -
- -

Conversion explicitness

- -
- -

Endian conversion functions and buffer types never perform -implicit conversions. This gives users explicit control of when conversion -occurs, and may help avoid unnecessary conversions.

- -

Endian arithmetic types perform conversion implicitly. That makes -these types very easy to use, but can result in unnecessary conversions. Failure -to hoist conversions out of inner loops can bring a performance penalty.

- -
- -

Arithmetic operations

- -
- -

Endian conversion functions do not supply arithmetic -operations, but this is not a concern since this approach uses ordinary C++ -arithmetic types to hold values.

- -

Endian buffer types do not supply arithmetic operations. Although this -approach avoids unnecessary conversions, it can result in the introduction of -additional variables and confuse maintenance programmers.

- -

Endian arithmetic types do supply arithmetic operations. They -are very easy to use if lots of arithmetic is involved.

- -
- -

Sizes

- -
- -

Endianness conversion functions only support 1, 2, 4, and 8 byte -integers. That's sufficient for many applications.

- -

Endian buffer and arithmetic types support 1, 2, 3, 4, 5, 6, 7, and 8 -byte integers. For an application where memory use or I/O speed is the limiting -factor, using sizes tailored to application needs can be useful.

- -
- -

Alignments

- -
- -

Endianness conversion functions only support aligned integer and -floating-point types. That's sufficient for most applications.

- -

Endian buffer and arithmetic types support both aligned and unaligned -integer and floating-point types. Unaligned types are rarely needed, but when -needed they are often very useful and workarounds are painful. For example,

- -
-

Non-portable code like this:

- 
struct S {
-  uint16_t a;  // big endian
-  uint32_t b;  // big endian
-} __attribute__ ((packed));
-
-

Can be replaced with portable code like this:

-
- 
struct S {
-  big_uint16_ut a;
-  big_uint32_ut b;
-};
-
-
- -
- -

Design patterns

- -

Applications often traffic in endian data as records or packets containing -multiple endian data elements. For simplicity, we will just call them records.

- -

If desired endianness differs from native endianness, a conversion has to be -performed. When should that conversion occur? Three design patterns have -evolved.

- -

Convert only as needed (i.e. lazy)

- -

This pattern defers conversion to the point in the code where the data -element is actually used.

- -

This pattern is appropriate when which endian element is actually used varies -greatly according to record content or other circumstances

- -

Convert in anticipation of need

- -

This pattern performs conversion to native endianness in anticipation of use, -such as immediately after reading records. If needed, conversion to the output -endianness is performed after all possible needs have passed, such as just -before writing records.

- -

One implementation of this pattern is to create a proxy record with -endianness converted to native in a read function, and expose only that proxy to -the rest of the implementation. If a write function, if needed, handles the -conversion from native to the desired output endianness.

- -

This pattern is appropriate when all endian elements in a record are -typically used regardless of record content or other circumstances

- -

Convert -only as needed, except locally in anticipation of need

- -

This pattern in general defers conversion but for specific local needs does -anticipatory conversion. Although particularly appropriate when coupled with the endian buffer -or arithmetic types, it also works well with the conversion functions.

- -

Example:

- -
- 
struct data_t
-{
-  big_int32_t   v1;
-  big_int32_t   v2;
-  big_int32_t   v3;
-};
-
-data_t data;
-
-read(data);
-
-...
-++v1;
-...
-
-int32_t v3_temp = data.v3;  // hoist conversion out of loop
-
-for (int32_t i = 0; i < large-number; ++i)
-{
-  ... lengthy computation that accesses v3_temp many times ...
-}
-data.v3 = v3_temp; 
-
-write(data);
-
-
- -

In general the above pseudo-code leaves conversion up to the endian -arithmetic type big_int32_t. But to avoid conversion inside the -loop, a temporary is created before the loop is entered, and then used to set -the new value of data.v3 after the loop is complete.

- -
- -

Question: Won't the compiler's optimizer hoist the conversion out -of the loop anyhow?

- -

Answer: VC++ 2015 Preview, and probably others, does not, even for -a toy test program. Although the savings is small (two register -bswap instructions), the cost might -be significant if the loop is repeated enough times. On the other hand, the -program may be so dominated by I/O time that even a lengthy loop will be -immaterial.

- -
- -

Use case examples

- -

Porting endian unaware codebase

- -

An existing codebase runs on big endian systems. It does not -currently deal with endianness. The codebase needs to be modified so it can run -on  little endian systems under various operating systems. To ease -transition and protect value of existing files, external data will continue to -be maintained as big endian.

- -

The endian -arithmetic approach is recommended to meet these needs. A relatively small -number of header files dealing with binary I/O layouts need to change types. For -example,  -short or int16_t would change to big_int16_t. No -changes are required for .cpp files.

- -

Porting endian aware codebase

- -

An existing codebase runs on little-endian Linux systems. It already -deals with endianness via -Linux provided -functions. Because of a business merger, the codebase has to be quickly -modified for Windows and possibly other operating systems, while still -supporting Linux. The codebase is reliable and the programmers are all -well-aware of endian issues.

- -

These factors all argue for an endian conversion -approach that just mechanically changes the calls to htobe32, -etc. to boost::endian::native_to_big, etc. and replaces <endian.h> -with <boost/endian/conversion.hpp>.

- -

Reliability and arithmetic-speed

- -

A new, complex, multi-threaded application is to be developed that must run -on little endian machines, but do big endian network I/O. The developers believe -computational speed for endian variable is critical but have seen numerous bugs -result from inability to reason about endian conversion state. They are also -worried that future maintenance changes could inadvertently introduce a lot of -slow conversions if full-blown endian arithmetic types are used.

- -

The endian buffers approach is made-to-order for -this use case.

- -

Reliability and ease-of-use

- -

A new, complex, multi-threaded application is to be developed that must run -on little endian machines, but do big endian network I/O. The developers believe -computational speed for endian variables is not critical but have seen -numerous bugs result from inability to reason about endian conversion state. -They are also concerned about ease-of-use both during development and long-term -maintenance.

- -

Removing concern about conversion speed and adding concern about ease-of-use -tips the balance strongly in favor the endian -arithmetic approach.

- -
-

Last revised: -19 January, 2015

-

© Copyright Beman Dawes, 2011, 2013, 2014

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/ LICENSE_1_0.txt

- -

 

- - - - \ No newline at end of file diff --git a/doc/conversion.html b/doc/conversion.html deleted file mode 100644 index 0e86598..0000000 --- a/doc/conversion.html +++ /dev/null @@ -1,405 +0,0 @@ - - - - - - - -Endian Conversion Functions - - - - - - - - - - -
- -Boost logo - - Endian Conversion Functions
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
- -

- - - - - - - - -
- Contents
- Introduction
- Reference
-    Synopsis
-    Requirements
-      EndianReversible
-      Customization for - UDTs
- Functions
- FAQ
- Acknowledgements
- -

Introduction

- -

Header boost/endian/conversion.hpp -provides byte order reversal and conversion functions that convert objects of -the built-in -integer types -between native, big, or little endian byte -ordering. User defined types are also supported.

- -

Reference

- -

Functions are implemented inline if appropriate. -For C++03 compilers, noexcept is -elided. -Boost scoped enum emulation is used so that the library still works for compilers that do not support scoped enums.

- -

Definitions

-

Endianness refers to the ordering of bytes within internal or -external integers and other arithmetic data. Most-significant byte first is -called big endian ordering. Least-significant byte first is called -little endian ordering. Other orderings are possible and some CPU -architectures support both big and little ordering.

-
-

[Note: The names are derived from - -Jonathan Swift's satirical novel - -Gulliver’s Travels, where rival kingdoms opened their soft-boiled eggs -at different ends. Wikipedia has an extensive description of -Endianness. —end note]

-
-

The standard integral types (C++std 3.9.1) except bool -are collectively called the endian types.

- -

-Header <boost/endian/conversion.hpp> -Synopsis

- -
#define BOOST_ENDIAN_INTRINSIC_MSG \
-   "message describing presence or absence of intrinsics"
-
-namespace boost
-{
-namespace endian
-{
-  enum class order
-  {
-    native = see below,
-    big    = see below,
-    little = see below,
-   };
-
-  int8_t   endian_reverse(int8_t x) noexcept;
-  int16_t  endian_reverse(int16_t x) noexcept;
-  int32_t  endian_reverse(int32_t x) noexcept;
-  int64_t  endian_reverse(int64_t x) noexcept;
-  uint8_t  endian_reverse(uint8_t x) noexcept;
-  uint16_t endian_reverse(uint16_t x) noexcept;
-  uint32_t endian_reverse(uint32_t x) noexcept;
-  uint64_t endian_reverse(uint64_t x) noexcept;
-
-  template <class EndianReversible>
-    EndianReversible big_to_native(EndianReversible x) noexcept;
-  template <class EndianReversible>
-    EndianReversible native_to_big(EndianReversible x) noexcept;
-  template <class EndianReversible>
-    EndianReversible little_to_native(EndianReversible x) noexcept;
-  template <class EndianReversible>
-    EndianReversible native_to_little(EndianReversible x) noexcept;
-  template <order O1, order O2, class EndianReversible>
-    EndianReversible conditional_reverse(EndianReversible x) noexcept;
-  template <class EndianReversible>
-    EndianReversible conditional_reverse(EndianReversible x,
-      order order1, order order2) noexcept;
-      
-  template <class EndianReversible>
-    void endian_reverse_inplace(EndianReversible& x) noexcept;
-
-  template <class EndianReversibleInplace>
-    void big_to_native_inplace(EndianReversibleInplace& x) noexcept;
-  template <class EndianReversibleInplace>
-    void native_to_big_inplace(EndianReversibleInplace& x) noexcept;
-  template <class EndianReversibleInplace>
-    void little_to_native_inplace(EndianReversibleInplace& x) noexcept;
-  template <class EndianReversibleInplace>
-    void native_to_little_inplace(EndianReversibleInplace& x) noexcept;
-  template <order O1, order O2, class EndianReversibleInplace>
-    void conditional_reverse_inplace(EndianReversibleInplace& x) noexcept; 
-  template <class EndianReversibleInplace>
-   void conditional_reverse_inplace(EndianReversibleInplace& x,
-     order order1, order order2) noexcept;
-
-} // namespace endian
-} // namespace boost
-

The values of order::little and order::big shall -not be equal to one another.

-

The value of order::native -shall be:

-
    -
  • equal to order::big if the execution environment is big - endian, otherwise
    • -
  • equal to order::little if the execution environment is little - endian, otherwise
    • -
  • unequal to both order::little and order::big.
    • -
-

Requirements

-

Template argument requirements

-

The template definitions in the boost/endian/conversion.hpp -header refer to various named requirements whose details are set out in the -tables in this subsection. In these tables, T is an object or -reference type to be supplied by a C++ program instantiating a template; x -is a value of type (possibly const) T; mlx is a -modifiable lvalue of type T.

- - - - - - - - - - - - - - -
EndianReversible - requirements (in addition to CopyConstructible)
ExpressionReturn
- type		Requirements
endian_reverse(x)T T is an endian type or a class type.

If T is - an endian type, returns the value of x with the order of bytes - reversed.

-

If T is a class type, the function:

-
    -
  • Returns the value of x - with the order of bytes reversed for all data members of types or arrays of - types that meet the EndianReversible requirements, and;
    • -
  • Is a non-member function in the same namespace as T that -can be found by argument dependent lookup (ADL).
    • -
-
-

 

- - - - - - - - - - - - -
EndianReversibleInplace - requirements (in addition to CopyConstructible)
ExpressionRequirements
endian_reverse_inplace(mlx) T is an endian type or a class type.

If T is - an endian type, reverses the order of bytes in mlx.

-

If T is a class type, the function:

-
    -
  • Reverses the order of bytes of all data members of mlx - that have types or arrays of - types that meet the EndianReversible or EndianReversibleInplace - requirements, and;
    • -
  • Is a non-member function in the same namespace as T that -can be found by argument dependent lookup (ADL). 
    • -
-
- -

[Note: Because there is a function template for endian_reverse_inplace -that calls endian_reverse, only endian_reverse -is required for a user-defined type to meet the EndianReversibleInplace -requirements. Although User-defined types are not required to supply an endian_reverse_inplace -function, doing so may improve efficiency.  —end note]

- -

Customization points for user-defined types (UDTs)

- -

This subsection describes requirements on the Endian library's implementation.

- -

The library's function templates requiring -EndianReversible are -required to perform reversal of endianness if needed by making an unqualified -call to endian_reverse().

- -

The library's function templates requiring -EndianReversibleInplace are required to perform reversal of endianness if needed by making an -unqualified call to endian_reverse_inplace().

- -

See example/udt_conversion_example.cpp for an example user-defined type.

- -

Functions

-
int8_t   endian_reverse(int8_t x) noexcept;
-int16_t  endian_reverse(int16_t x) noexcept;
-int32_t  endian_reverse(int32_t x) noexcept;
-int64_t  endian_reverse(int64_t x) noexcept;
-uint8_t  endian_reverse(uint8_t x) noexcept;
-uint16_t endian_reverse(uint16_t x) noexcept;
-uint32_t endian_reverse(uint32_t x) noexcept;
-uint64_t endian_reverse(uint64_t x) noexcept;
-
-

Returns: x, with the order of its - constituent bytes reversed.

-

Remarks: The type of x meets the EndianReversible requirements.

-

[Note: The Boost.Endian library does not provide overloads for the C++ standard library - supplied types. —end note]

-
- -
template <class EndianReversible>
-EndianReversible big_to_native(EndianReversible x) noexcept;
-
-

- Returns: conditional_reverse<order::big, order::native>(x).

-
-
template <class EndianReversible>
-EndianReversible native_to_big(EndianReversible x) noexcept;
-
-

Returns: conditional_reverse<order::native, order::big>(x).

-
-
template <class EndianReversible>
-EndianReversible little_to_native(EndianReversible x) noexcept;
-
-

Returns: conditional_reverse<order::little, order::native>(x).

-
-
template <class EndianReversible>
-EndianReversible native_to_little(EndianReversible x) noexcept;
-
-

Returns: conditional_reverse<order::native, order::little>(x).

-
-
template <order O1, order O2, class EndianReversible>
-EndianReversible conditional_reverse(EndianReversible x) noexcept;
-
-

Returns: x if O1 == O2, otherwise endian_reverse(x).

-

Remarks: Whether x or endian_reverse(x) is to be returned shall be determined at compile time.

-
-
template <class EndianReversible>
-EndianReversible conditional_reverse(EndianReversible x,
-      order order1, order order2) noexcept;
-
-

Returns: order1 == order2 ? x : endian_reverse(x).

-
- -
template <class EndianReversible>
-void endian_reverse_inplace(EndianReversible& x) noexcept;
- -
-

Effects: x = endian_reverse(x).

-
- -
template <class EndianReversibleInplace>
-void big_to_native_inplace(EndianReversibleInplace& x) noexcept;
-
-

- Effects: conditional_reverse_inplace<order::big, order::native>(x).

-
-
template <class EndianReversibleInplace>
-void native_to_big_inplace(EndianReversibleInplace& x) noexcept;
-
-

- Effects: conditional_reverse_inplace<order::native, order::big>(x).

-
-
template <class EndianReversibleInplace>
-void little_to_native_inplace(EndianReversibleInplace& x) noexcept;
-
-

- Effects: conditional_reverse_inplace<order::little, order::native>(x).

-
-
template <class EndianReversibleInplace>
-void native_to_little_inplace(EndianReversibleInplace& x) noexcept;
-
-

- Effects: conditional_reverse_inplace<order::native, order::little>(x).

-
-
template <order O1, order O2, class EndianReversibleInplace>
-void conditional_reverse_inplace(EndianReversibleInplace& x) noexcept;
-
-

Effects: None if O1 == O2, otherwise endian_reverse_inplace(x).

-

Remarks: Which effect applies shall be determined at compile time.

-
-
template <class EndianReversibleInplace>
-void conditional_reverse_inplace(EndianReversibleInplace& x,
-     order order1, order order2) noexcept;
- - -
-

Effects: If order1 == order2 then endian_reverse_inplace(x).

-
- - -

FAQ

- -

See the Endian home page FAQ for a library-wide -FAQ.

- -

Why are both value returning and modify-in-place functions provided?

- -
- -

Returning the result by value is the standard C and C++ idiom for functions that compute a -value from an argument. Modify-in-place functions allow cleaner code in many real-world -endian use cases and are more efficient for user-defined types that have -members such as string data that do not need to be reversed. Thus both forms are -provided.

- -
- -

Why are exact-length 8, 16, 32, and 64-bit integers supported rather than the built-in -char, short, int, long, long long, etc?

- -
- -

The primary use case, portable file or network data, needs these de facto -standard sizes. Using types that vary with the platform would greatly limit -portability for both programs and data.

- -
- -

Why not use the Linux names (htobe16, htole16, be16toh, le16toh, etc.) ?

- -
- -

Those names are non-standard and vary even between POSIX-like operating -systems. A C++ library TS was going to use those names, but found they were -sometimes implemented as macros. Since macros do not respect scoping and -namespace rules, to use them would be very error prone.

- -
- -

Acknowledgements

Tomas Puverle was instrumental -in identifying and articulating the need to support endian conversion as separate from -endian integer types. Phil Endecott suggested the form of the value returning signatures. -Vicente Botet and other reviewers suggested supporting user defined types. -General reverse template implementation approach using std::reverse suggested by Mathias Gaunard. -Portable implementation approach for 16, 32, and 64-bit integers suggested by tymofey, -with avoidance of undefined behavior as suggested by Giovanni Piero Deretta, -and a further refinement suggested by Pyry Jahkola. -Intrinsic builtins implementation approach for 16, 32, and 64-bit integers suggested by -several reviewers, and by David Stone, who provided his Boost licensed macro implementation -that became the starting point for boost/endian/detail/intrinsic.hpp. -Pierre Talbot provided the int8_t endian_reverse() and templated -endian_reverse_inplace() implementations.

-
-

Last revised: -14 October, 2015

-

© Copyright Beman Dawes, 2011, 2013

-

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

- - - - \ No newline at end of file diff --git a/doc/fp_concerns.html b/doc/fp_concerns.html deleted file mode 100644 index 5c5f67c..0000000 --- a/doc/fp_concerns.html +++ /dev/null @@ -1,97 +0,0 @@ - - - - - - - -Floating Point Concerns - - - - - -

Floating Point Concerns

- -

Consider this simple implementation of endian_reverse:

- -
- 
template <class T>
-inline T endian_reverse(T x) BOOST_NOEXCEPT
-{
-  std::reverse(reinterpret_cast<unsigned char*>(&x),
-    reinterpret_cast<unsigned char*>(&x) + sizeof(T));
-  return x;
-}
-
-

Under what conditions with this code fail?

-

It will fail if an object of type T has one or more -bit patterns that cause a failure. Failures usually occur when  an invalid -or otherwise special bit pattern is loaded into or saved from a hardware -register.

-

The problem could in theory occur with both integers and floating -point numbers, but the -two's complement integers ubiquitous in modern computer architectures do not -have any invalid or otherwise special bit patterns that cause failure when -byte-wise reversed.

-

But floating point numbers are a different story. Even if we limit -discussion to IEEE 754 (aka ISO/IEC/IEEE 60559) binary representations of 4 and -8 byte sizes, several problems are easy to demonstrate:

-
    -
  • -

    ...

    • -
-

Safe interfaces and possible reference implementations

-

In-place interface

-
- 
template <class T>
-inline void endian_reverse_inplace(T& x)
-{
-  std::reverse(reinterpret_cast<unsigned char*>(&x),
-    reinterpret_cast<unsigned char*>(&x) + sizeof(T));
-}
-
-

This is the same as the current (i.e integer) customization point -interface, so there is no need for any change.

-

Warning: Even thought x may have had a valid -value on the originating platform, after calling this function the value of -x may differ or be invalid on this platform.

-

Copy interface

-
- 
template <class T>
-inline void endian_reverse_copy(const T& from, T& to)
-{
-  std::reverse_copy(reinterpret_cast<const unsigned char*>(&from),
-    reinterpret_cast<const unsigned char*>(&from) + sizeof(T),
-    reinterpret_cast<unsigned char*>(&to));
-}
-
-

Warning: Even thought from may have been a valid value on -the originating platform, after calling this function the value of to -may differ or be invalid on this platform.

-

Return-by-value interface

-
- 
template <class T>
-inline T endian_reverse_to_native(const T& x) BOOST_NOEXCEPT
-{
-  T tmp;
-  std::reverse_copy(reinterpret_cast<const unsigned char*>(&x),
-    reinterpret_cast<const unsigned char*>(&x) + sizeof(T),
-    reinterpret_cast<unsigned char*>(&tmp));
-  return tmp;
-}
-
-

Warning: Even thought x may have had a valid value on the -originating platform, the value of returned by this function may differ or be -invalid on this platform.

-

Acknowledgements

-
-

Last revised: 27 March, 2015

-

© Copyright Beman Dawes, 2015

-

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt

- -

 

- - - - \ No newline at end of file diff --git a/doc/index.html b/doc/index.html deleted file mode 100644 index 0338f70..0000000 --- a/doc/index.html +++ /dev/null @@ -1,606 +0,0 @@ - - - - - - - -Endian Library - - - - - - - - - - -
- -Boost logo - - Endian Library
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
-

- - - - - - - - -
- Contents
-Abstract
-Introduction to endianness
-Introduction to the Boost.Endian library
-Choosing between conversion functions,
-   buffer types, and arithmetic types
-Built-in support for Intrinsics
-Performance
-   Timings
-Overall FAQ
-Release history
-   Changes -requested by formal review
-   Other changes since -formal review
-Compatibility with interim releases
-C++03 support for C++11 features
-Future directions
-Acknowledgements
-
- -

Abstract

- -

Boost.Endian provides facilities to manipulate the -endianness of integers and user-defined types.

-
    -
  • Three approaches to endianness are supported. Each has a - long history of successful use, and each approach has use cases where it is - preferred over the other approaches.
    - 
    • -
  • Primary uses:
    - 
      -
    • Data portability. The Endian library supports binary data exchange, via either external media or network transmission, - regardless of platform endianness.
      - 
      • -
    • Program portability. POSIX-based and - Windows-based operating systems traditionally supply libraries with - non-portable functions to perform endian conversion. There are at least four - incompatible sets of functions in common use. The Endian library is - portable across all C++ platforms.
      - 
      • -
    - -
    • -
  • Secondary use: Minimizing data size via sizes and/or alignments not supported by the - standard C++ integer types.
    • -
- -

Introduction to endianness

- -

Consider the following code:

- -
- 
int16_t i = 0x0102;
-FILE * file = fopen("test.bin", "wb");   // binary file!
-fwrite(&i, sizeof(int16_t), 1, file);
-fclose(file);
-
-

On OS X, Linux, or Windows systems with an Intel CPU, a hex dump -of the "test.bin" output file produces:

-
-

0201

-
-

On OS X systems with a PowerPC CPU, or Solaris systems with a SPARC CPU, a hex dump of the "test.bin" -output file produces:

-
-

0102

-
-

What's happening here is that Intel CPUs order the bytes of an integer with -the least-significant byte first, while SPARC CPUs place the most-significant -byte first. Some CPUs, such as the PowerPC, allow the operating system to -choose which ordering applies.

-

Most-significant-byte-first ordering is traditionally called "big endian" -ordering and least-significant-byte-first is traditionally called -"little-endian" ordering. The names are derived from - -Jonathan Swift's satirical novel - -Gulliver’s Travels, where rival kingdoms opened their soft-boiled eggs -at different ends.

-

See Wikipedia's -Endianness article for an -extensive discussion of endianness.

-

Programmers can usually ignore endianness, except when reading a core -dump on little-endian systems. But programmers have to deal with endianness when exchanging binary integers and binary floating point -values between computer systems with differing endianness, whether by physical file transfer or over a network. -And programmers may also want to use the library when minimizing either internal or -external data sizes is advantageous.

-

Introduction to the Boost.Endian library

- -

Boost.Endian provides three different approaches to dealing with - -endianness. All three approaches support integers and user-define types (UDTs).

- -

Each approach has a long history of successful use, and each approach has use -cases where it is preferred to the other approaches.

- -
- -

Endian conversion functions - The -application uses the built-in integer types to hold values, and calls the -provided conversion functions to convert byte ordering as needed. Both mutating -and non-mutating conversions are supplied, and each comes in unconditional and -conditional variants.

- -

Endian buffer types - The application uses the provided endian -buffer types -to hold values, and explicitly converts to and from the built-in integer types. Buffer sizes of 8, 16, 24, 32, 40, 48, 56, and 64 bits (i.e. -1, 2, 3, 4, 5, 6, 7, and 8 bytes) are provided. Unaligned integer buffer types -are provided for all sizes, and aligned buffer types are provided for 16, 32, and -64-bit sizes. The provided specific types are typedefs for a generic class -template that may be used directly for less common use cases.

- -

Endian arithmetic types - The -application uses the provided endian arithmetic types, which supply the same -operations as the built-in C++ arithmetic types. All conversions are implicit. -Arithmetic sizes of 8, 16, 24, 32, 40, 48, 56, and 64 bits (i.e. 1, 2, 3, 4, 5, -6, 7, and 8 bytes) are provided. Unaligned integer types are provided for all -sizes and aligned -arithmetic types are provided for 16, 32, and 64-bit sizes. The provided -specific types are typedefs for a generic class template that may be used -directly in generic code of for less common use cases.

- -
- -

Boost Endian is a header-only library. C++11 features -affecting interfaces, such as noexcept, are used only if available. -See C++03 support for C++11 features for details.

- -

Choosing between conversion functions, buffer types, -and arithmetic types

- -

This section has been moved to its own -Choosing the Approach page.

- -

Built-in support for Intrinsics

-

Most compilers, including GCC, Clang, and Visual C++, supply built-in support for byte swapping intrinsics. -The Endian library uses these intrinsics when available since they may result in smaller and faster generated code, particularly for -optimized -builds.

-

Defining the macro BOOST_ENDIAN_NO_INTRINSICS will suppress use -of the intrinsics. This is useful when a compiler has no intrinsic support or -fails to locate the appropriate header, perhaps because it -is an older release or has very limited supporting libraries.

-

The macro BOOST_ENDIAN_INTRINSIC_MSG is defined as -either "no byte swap intrinsics" or a string describing the -particular set of intrinsics being used. This is useful for eliminating missing -intrinsics as a source of performance issues.

- -

Performance

- -

Consider this problem:

- -
-
- - - - - - - - - - - - - - - - -
-

Example 1

Add 100 to a big endian value in a file, then write the - result to a file
Endian arithmetic type approachEndian conversion function approach
- 
big_int32_at x;
-
-... read into x from a file ...
-
-x += 100;
-
-... write x to a file ...
-
- 		- 
  
-int32_t x;
-
-... read into x from a file ...
-
-big_to_native_inplace(x);
-x += 100;
-native_to_big_inplace(x);
-
-... write x to a file ...
-
-
- -
-
- -

There will be no performance difference between the two approaches in -optimized builds, -regardless of the native endianness of the machine. That's because optimizing compilers will generate exactly the same code for each. That conclusion was confirmed by -studying the generated assembly code for GCC and Visual C++. Furthermore, time -spent doing I/O will determine the speed of this application.

- -

Now consider a slightly different problem: 

- -
-
- - - - - - - - - - - - - - - - -
-

Example 2

Add a million values to a big endian value in a file, then write the - result to a file
Endian arithmetic type approachEndian conversion function approach
- 
big_int32_at x;
-
-... read into x from a file ...
-
-for (int32_t i = 0; i < 1000000; ++i)
-  x += i;
-
-... write x to a file ...
-
- 		- 
int32_t x;
-
-... read into x from a file ...
-
-big_to_native_inplace(x);
-
-for (int32_t i = 0; i < 1000000; ++i)
-  x += i;
-
-native_to_big_inplace(x);
-
-... write x to a file ...
-
-
- -
-
- -

With the Endian arithmetic approach, on little endian platforms an implicit conversion from and then back to -big endian is done inside the loop. With the Endian conversion function -approach, the user has ensured the conversions are done outside the loop, so the -code may run more quickly on little endian platforms.

- -

Timings

-

These tests were run against release builds on a circa 2012 4-core little endian X64 Intel Core i5-3570K -CPU @ 3.40GHz under Windows 7.

- -

Caveat emptor: The Windows CPU timer has very high granularity. Repeated -runs of the same tests often yield considerably different results.

- -

See test/loop_time_test.cpp for the actual code and benchmark/Jamfile.v2 for the build -setup.

- - -
-
- - - - - - - - - - - - - - -
GNU C++ version 4.8.2 on Linux virtual - machine
Iterations: 10'000'000'000, Intrinsics: __builtin_bswap16, etc.
Test CaseEndian
arithmetic
type		Endian
conversion
function
16-bit aligned big endian8.46 s5.28 s
16-bit aligned little endian5.28 s5.22 s
32-bit aligned big endian8.40 s2.11 s
32-bit aligned little endian2.11 s2.10 s
64-bit aligned big endian14.02 s3.10 s
64-bit aligned little endian3.00 s3.03 s
-
-
-

- -
- - - - - - - - - - - - - -
Microsoft Visual C++ version 14.0
Iterations: 10'000'000'000, Intrinsics: cstdlib _byteswap_ushort, etc.
Test CaseEndian
arithmetic
type		Endian
conversion
function
16-bit aligned big endian8.27 s5.26 s
16-bit aligned little endian5.29 s5.32 s
32-bit aligned big endian8.36 s5.24 s
32-bit aligned little endian5.24 s5.24 s
64-bit aligned big endian13.65 s3.34 s
64-bit aligned little endian3.35 s2.73 s
-
- - -

Overall FAQ

- -

Is the implementation header only?

- -
- -

Yes.

- -
- -

Are C++03 compilers supported?

- -
- -

Yes.

- -
- -

Does the implementation use compiler intrinsic built-in byte swapping?

- -
- -

Yes, if available. See Intrinsic built-in support.

- -
- -

Why bother with endianness?

-
-

Binary data portability is the primary use case.

-
-

Does endianness have any uses outside of portable binary file or network -I/O formats?

-
-

Using the unaligned integer types with a size tailored to the application's -needs is a minor secondary use that saves internal or external memory space. For -example, using big_int40_buf_t or big_int40_t in a -large array saves a lot of space compared to one of the 64-bit types.

-
-

Why bother with binary I/O? Why not just use C++ Standard Library stream -inserters and extractors?

-
-

Data interchange formats often specify binary integer data.

-

Binary integer data is smaller and therefore I/O is faster and file sizes -are smaller. Transfer between systems is less expensive.

-

Furthermore, binary integer data is of fixed size, and so fixed-size disk -records are possible without padding, easing sorting and allowing random access.

-

Disadvantages, such as the inability to use text utilities on the -resulting files, limit usefulness to applications where the binary I/O -advantages are paramount.

-
- -

Which is better, big-endian or little-endian?

-
-

Big-endian tends to be preferred in a networking environment and is a bit -more of an industry standard, but little-endian may be preferred for -applications that run primarily on x86, x86-64, and other little-endian -CPU's. The Wikipedia article -gives more pros and cons.

-
- -

Why are only big and little native endianness supported?

-
-

These are the only endian schemes that have any practical value today. PDP-11 -and the other middle endian approaches are interesting curiosities -but have no relevance for today's C++ developers. The same is true for -architectures that allow runtime endianness switching. The -specification for native -ordering has been carefully crafted to allow support for such orderings in -the future, should the need arise. Thanks to Howard Hinnant for suggesting this.

-
- -

Why do both the buffer and arithmetic types exist?

-
-

Conversions in the buffer types are explicit. Conversions in the arithmetic -types are implicit. This fundamental difference is a deliberate design feature -that would be lost if the inheritance hierarchy were collapsed.

-

The original design provided only arithmetic types. Buffer types were -requested during formal review by those wishing total control over when -conversion occurs. They also felt that buffer types would be less likely to be -misused by maintenance programmers not familiar with the implications of -performing a lot of integer operations on the endian arithmetic integer types.

-
-

What is gained by using the buffer types rather than always just using the -arithmetic types?

-
-

Assurance that hidden conversions are not performed. This is of overriding -importance to users concerned about achieving the ultimate in terms of speed.

-

"Always just using the arithmetic types" is fine for other users. When the -ultimate in speed needs to be ensured, the arithmetic types can be used in the -same design patterns or idioms that would be used for buffer types, resulting in -the same code being generated for either types.

-
- -

What are the limitations of integer support?

- -
- -

Tests have only been -performed on machines that use two's complement arithmetic. The Endian -conversion functions only support 16, 32, and 64-bit aligned integers. The -endian types only support 8, 16, 24, 32, 40, 48, 56, and 64-bit unaligned integers, -and 8, 16, 32, and 64-bit aligned integers.

- -
- -

Why is there no floating point support?

- -
- -

An attempt was made to support four-byte floats and eight-byte -doubles, limited to -IEEE 754 (also -know as ISO/IEC/IEEE 60559) floating point and further limited to systems where -floating point endianness does not differ from integer -endianness.

- -

Even with those limitations, support for floating point types was not -reliable and was removed. For example, simply reversing the endianness of a -floating point number can result in a signaling-NAN. For all practical purposes, -binary serialization and endianness for integers are one and the same problem. -That is not true for floating point numbers, so binary serialization interfaces -and formats for floating point does not fit well in an endian-based library.

- -
- -

Release history

-

Changes requested by formal review

-

The library was reworked from top to bottom to accommodate changes requested -during the formal review. See Mini-Review -page for details.

-

Other changes since formal -review

-
    -
  • Header boost/endian/endian.hpp has been renamed to - boost/endian/arithmetic.hpp. Headers - boost/endian/conversion.hpp and boost/endian/buffers.hpp have been - added. - Infrastructure file names were changed accordingly.
    • -
  • The endian arithmetic type aliases have been renamed, - using a naming pattern that is consistent for both integer and floating point, - and a consistent set of aliases supplied for the endian buffer types.
    • -
  • The unaligned-type alias names still have the _t suffix, but - the aligned-type alias names now have an _at suffix..
    • -
  • endian_reverse() overloads for int8_t and - uint8_t have been added for improved generality. (Pierre Talbot)
    • -
  • Overloads of endian_reverse_inplace() have been replaced with a single - endian_reverse_inplace() template. (Pierre Talbot)
    • -
  • For X86 and X64 architectures, which permit unaligned loads and stores, - unaligned little endian buffer and arithmetic types use regular loads and - stores when the size is exact. This makes unaligned little endian buffer and - arithmetic types significantly more efficient on these architectures. (Jeremy - Maitin-Shepard)
    • -
  • C++11 features affecting interfaces, such as noexcept, are now used. - C++03 compilers are still - supported.
    • -
  • Acknowledgements have been updated.
    • -
- -

Compatibility with interim releases

- -

Prior to the official Boost release, class template -endian_arithmetic has been used for a decade or more with the same -functionality but under the name endian. Other names also changed -in the official release. If the macro BOOST_ENDIAN_DEPRECATED_NAMES -is defined, those old now deprecated names are still supported. However, the -class template endian name is only provided for compilers -supporting C++11 template aliases. For C++03 compilers, the name will have to be -changed to endian_arithmetic.

- -

To support backward header compatibility, deprecated header boost/endian/endian.hpp -forwards to boost/endian/arithmetic.hpp. It requires -BOOST_ENDIAN_DEPRECATED_NAMES be defined. It should only be used while -transitioning to the official Boost release of the library as it will be removed -in some future release.

- -

C++03 support for C++11 features

- - - - - - - - - - - - - - - - - - -
C++11 FeatureAction with C++03 Compilers
Scoped enums Uses header - - boost/core/scoped_enum.hpp - to emulate C++11 scoped enums.
noexceptUses BOOST_NOEXCEPT macro, which is defined as - null for compilers not supporting this C++11 feature.
C++11 PODs (N2342)Takes advantage of C++03 compilers that - relax C++03 POD rules, but see Limitations - here and - here. Also see macros for explicit - POD control here and - here.
- -

Future directions

- -

Standardization. The plan is to submit Boost.Endian to the C++ -standards committee for possible inclusion in a Technical Specification or the -C++ standard itself.

- -

Specializations for numeric_limits. Roger Leigh -requested that all boost::endian types provide numeric_limits -specializations. See -GitHub issue 4.

- -

Character buffer support. Peter Dimov pointed out during the -mini-review that getting and setting basic arithmetic types (or <cstdint> -equivalents) from/to an offset into an array of unsigned char is a common need. -See -Boost.Endian mini-review posting.

- -

Out-of-range detection. Peter Dimov pointed suggested during the -mini-review that throwing an exception on buffer values being out-of-range might -be desirable. See the end of -this posting -and subsequent replies.

- -

Acknowledgements

-

Comments and suggestions were received from Adder, Benaka Moorthi, -Christopher Kohlhoff, Cliff Green, Daniel James, Dave Handley, Gennaro Proto, Giovanni Piero -Deretta, Gordon Woodhull, dizzy, Hartmut Kaiser, Howard Hinnant, Jason Newton, Jeff Flinn, Jeremy Maitin-Shepard, John Filo, John -Maddock, Kim Barrett, Marsh Ray, Martin Bonner, Mathias Gaunard, Matias -Capeletto, Neil Mayhew, Nevin Liber, -Olaf van der Spek, Paul Bristow, Peter Dimov, Pierre Talbot, Phil Endecott, -Philip Bennefall, Pyry Jahkola, -Rene Rivera, Robert Stewart, Roger Leigh, Roland Schwarz, Scott McMurray, Sebastian Redl, Tim -Blechmann, Tim Moore, tymofey, Tomas Puverle, Vincente Botet, Yuval Ronen and -Vitaly Budovsk. Apologies if anyone has been missed.

-
-

Last revised: -05 April, 2016

-

© Copyright Beman Dawes, 2011, 2013

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/ LICENSE_1_0.txt

- - - - \ No newline at end of file diff --git a/doc/mini_review_topics.html b/doc/mini_review_topics.html deleted file mode 100644 index d7a7f3c..0000000 --- a/doc/mini_review_topics.html +++ /dev/null @@ -1,150 +0,0 @@ - - - - - - -Endian Mini-Review - - - - - - - - - -
- -Boost logo - - Endian Mini-Review
- - - - - -
- Endian Home     - Conversion Functions     - Arithmetic Types     - Buffer Types     - Choosing Approach
-

- -

The results of the Boost.Endian formal review included a list of issues to be -resolved before a mini-review.

- -

The issues are shown in bold below, with the resolution indicated.

- -

1. Common use case scenarios should be developed.

-
- -

Done. The documentation have been refactored. A page is now devoted to - Choosing the Approach to endianness. See - Use cases for use case - scenarios.

-
-

2. Example programs should be developed for the common use case scenarios.

-
-

Done. See Choosing the Approach. - Example code has been added throughout.

-
-

3. Documentation should illuminate the differences between endian -integer/float type and endian conversion approaches to the common use -case scenarios, and provide guidelines for choosing the most appropriate -approach in user's applications.

-
-

Done. See Choosing the Approach.

-
-

4 .Conversion functions supplying results via return should be provided.

-
-

Done. See ConversionFunctions.

-
-

5. Platform specific performance enhancements such as use of compiler intrinsics or relaxed alignment requirements should be supported.

-
-

Done. Compiler (Clang, GCC, VisualC++, etc.) intrinsics and built-in - functions are used in the implementation where appropriate, as requested. See - Built-in support for Intrinsics. See - Timings for Example 2 to gauge the impact of - intrinsics.

-
-

6. Endian integer (and floating) types should be implemented via the -conversion functions. If that can't be done efficiently, consideration -should be given to expanding the conversion function signatures to -resolve the inefficiencies.

-
-

Done. For the endian types, the implementation uses the endian conversion - functions, and thus the intrinsics, as requested.

-
-

7. Benchmarks that measure performance should be provided. It should be -possible to compare platform specific performance enhancements against -portable base implementations, and to compare endian integer approaches -against endian conversion approaches for the common use case scenarios.

-
-

Done. See Timings for Example 2. The endian/test directory - also contains several additional benchmark and speed test programs.

-
-

8. Float (32-bits) and double (64-bits) should be supported. IEEE 754 is -the primary use case.

-
-

Done. The endian buffer types,  - endian arithmetic types and - endian conversion functions now support 32-bit (float) - and 64-bit (double) floating point, as requested.

-
-

9. Support for user defined types (UDTs) is desirable, and should be -provided where there would be no conflict with the other concerns.

-
-

Done. See Customization - points for user-defined types (UDTs).

-
-

10. There is some concern that endian integer/float arithmetic operations -might used inadvertently or inappropriately. The impact of adding an endian_buffer - class without arithmetic operations should be investigated.

-
-

Done. The endian types have been decomposed into class template - endian_buffer and class template - endian_arithmetic. Class - endian_buffer is a public base class for endian_arithmetic, - and can also be used by users as a stand-alone class.

-
-

11. Stream insertion and extraction of the endian integer/float types should - be documented and included in the test coverage.

-
-

Done. See Stream inserter and - Stream extractor.

-
-

12. Binary I/O support that was investigated during development of the Endian - library should be put up for mini-review for inclusion in the Boost I/O - library.

- -
-

Not done yet. Will be handled as a separate min-review soon after the - Endian mini-review.

- -
-

13. Other requested changes.

- -
-
    -
  • In addition to the named-endianness conversion functions, functions that perform - compile-time (via template) and run-time (via function argument) dispatch - are now provided.
    • -
  • order::native is now a synonym for order::big - or order::little according to the endianness of the platform. This reduces the number of template specializations required.
    • -
  • Headers have been reorganized to make them easier to read, - with a synopsis at the front and implementation following.
    • -
-
-
-

Last revised: -19 January, 2015

-

© Copyright Beman Dawes, 2014

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/ LICENSE_1_0.txt

- - - - \ No newline at end of file diff --git a/doc/msvc_readme.txt b/doc/msvc_readme.txt deleted file mode 100644 index 88ea31c..0000000 --- a/doc/msvc_readme.txt +++ /dev/null @@ -1,31 +0,0 @@ -Windows -------- - -Prerequisites - -Boost libraries available in boost-root\stage\lib. Example: - - cd boost-root - .\bootstrap - .\b2 --with-system --with-chrono --with-timer link=shared stage - -The provided Visual Studio solution (endian/test/msvc/endian.sln) has a property page -(endian/test/msvc/common.prop) with these Common Properties set (do not include the -double quotes): - - VC++ Directores|Executable Directories: prefix default value with "..\..\..\..\..\stage\lib;" - (Click "Inherit from parent or project defaults" if not checked) - - C/C++|General|Additional Include Directories: prefix default value with "..\..\..\..\..\stage\lib;" - - Linker|General|Additional Library Directories: prefix default value with "..\..\..\..\..\stage\lib;" - - C/C++|Preprocessor: prefix default value with "BOOST_ALL_DYN_LINK;" - -IMPORTANT: If Preprocessor macros are supplied via a common property page, - must be set for each project! - ------------------------------------------------------------------------------------------- -Copyright Beman Dawes, 2013 -Distributed under the Boost Software License, Version 1.0. -See http://www.boost.org/LICENSE_1_0.txt diff --git a/doc/p0803r0.html b/doc/p0803r0.html deleted file mode 100644 index 9aa3677..0000000 --- a/doc/p0803r0.html +++ /dev/null @@ -1,134 +0,0 @@ - - - - - - - -Endian RFC - - - - - - - - - - - - - - - - - - - -
Doc. no.:P0803R0
Date: - 2017-10-15
Reply to:Beman Dawes <bdawes at acm dot org> -
Audience:Library Evolution
- - - -

Endian Library Request for Comments (R0)

- - -

Abstract

-

The Boost Endian library manipulates the -endianness of integers -and user-defined types. It provides three approaches to dealing with endianness: -conversion functions, buffer classes, and arithmetic classes. Each approach -has use cases where it is preferred over the other approaches. The purpose of -this RFC is to determine interest in adding all or any of these components to -the C++ standard library or a library TS.

-

Introduction

-

The Boost Endian documentation (www.boost.org/libs/endian/doc) -provides an overview -and detailed technical specifications for the -conversion -functions, -arithmetic types, and -buffer types. -

-

A separate discussion covers -choosing -between the three approaches provided by the library.

-

For those not interested in plowing through the entire documentation, the -Overall FAQ, -Conversion -FAQ, -Arithmetic FAQ, and -Buffers FAQ -answer a lot of question likely to arise during standardization.

-

Motivation for standardization

-
    -
  • Lack of any one de facto standard endianness capability in either C or - C++. While most platforms provides some way to handle endianness in C, the - mechanism varies from platform to platform.
    • -
  • Commonly used endian handling functions are extremely error prone. While - the Boost Endian conversion functions have the same problems, the buffer and - arithmetic classes do not.
    • -
  • Surprisingly difficult to write and test yourself in a portable yet - efficient (i.e. using intrinsics) way.
    • -
  • Explicit requests for standardization from Boost Endian users.
    • -
  • The Boost library is existing practice with years of both implementation - and end-user experience.
    • -
  • The committee has already failed once to standardize endian conversion - functions; perhaps now is time to do it right.
    • -
-

Impact on the standard library

-

All three proposed approaches to endianness would be a pure additions to the -standard library and would break no existing user code (modulo the usual -namespace caveats).

-

Proposed wording

-

The Boost Endian reference documentation for - -Conversion, - -Arithmetic, and -Buffer -approaches follows the standard library's Method of description -[description], so provides a fairly close approximation of what proposed wording -would look like. Actual proposed wording will be provided in a proposal document -for any portions of the library of interest to the committee. Names will need -the usual bikeshedding.

-

Questions for the Library Evolution Group

-
    -
  • How would you likely vote on a fully-worded and reviewed proposal to add a - standardized version of the Boost Endian conversion functions to the standard - library or a library TS?
    - 
    • -
  • How would you likely vote on a fully-worded and reviewed proposal to add a - standardized version of the Boost Endian buffer classes to the standard - library or a library TS?
    - 
    • -
  • How would you likely vote on a fully-worded and reviewed proposal to add a - standardized version of the Boost Endian arithmetic classes to the standard - library or a library TS?
    • -
-

Acknowledgements

-

The original design for the arithmetic classes was developed by Darin Adler -based on work by Mark Borgerding. The Boost documentation acknowledges -45+ -other people.

-
-

 

- - - - - \ No newline at end of file diff --git a/doc/publish.bat b/doc/publish.bat deleted file mode 100644 index a8fdb91..0000000 --- a/doc/publish.bat +++ /dev/null @@ -1,8 +0,0 @@ -copy /y c:\boost\develop\libs\endian\doc\* d:\boost\endian-gh-pages -pushd d:\boost\endian-gh-pages -git commit -a -m "copy from develop" -git push -popd -rem Copyright Beman Dawes, 2014 -rem Distributed under the Boost Software License, Version 1.0. -rem See www.boost.org/LICENSE_1_0.txt \ No newline at end of file diff --git a/doc/std-rfc.html b/doc/std-rfc.html deleted file mode 100644 index 7ce39cf..0000000 --- a/doc/std-rfc.html +++ /dev/null @@ -1,140 +0,0 @@ - - - - - - - -Endian RFC - - - - - - - - - - - - - - - - - - - -
Doc. no.:D0803R1
Date: - 2017-11-10
Reply to:Beman Dawes <bdawes at acm dot org> -
Audience:Library Evolution
- - - -

Endian Library Request for Comments (R1)

- - -

Abstract

-

The Boost Endian library manipulates the -endianness of integers -and user-defined types. It provides three approaches to dealing with endianness: -conversion functions, buffer classes, and arithmetic classes. Each approach -has use cases where it is preferred by users over the other approaches, but -adding all three approaches to the standard library would likely be too much of -a good thing. The purpose of -this RFC is to determine interest in adding the Boost conversion functions alone to -the C++ standard library or a library TS.

-

Introduction

-

The Boost Endian documentation (www.boost.org/libs/endian/doc) -provides an overview -and detailed technical specifications for the -conversion -functions, -arithmetic types, and -buffer types. -

-

A separate discussion covers -choosing -between the three approaches provided by the library.

-

For those not interested in plowing through the entire documentation, the -Overall FAQ, -Conversion -FAQ, -Arithmetic FAQ, and -Buffers FAQ -answer a lot of question likely to arise during standardization.

-

Motivation for standardization

-Lack of any one de facto standard endianness capability in either C or - C++. While most platforms provides some way to handle endianness in C, the - mechanism varies from platform to platform.
    -
  • Commonly used C endian handling functions are extremely error prone. While - the Boost Endian conversion functions have some of the same problems, the buffer and - arithmetic classes do not.
    • -
  • Surprisingly difficult to write and test yourself in a portable yet - efficient (i.e. using intrinsics) way.
    • -
  • Explicit requests for standardization from Boost Endian users.
    • -
  • The Boost library is existing practice with years of both implementation - and end-user experience.
    • -
  • The committee has already started to standardize endian conversion - functions once, but had to pull the proposal at the last minute because of - name clashes with widely used macros.
    • -
-

Motivation for standardizing only the conversion functions

-
    -
  • KISS; Providing three different solutions to the same problem space - regularly confuses to Boost users and makes endianness solutions harder to - teach.
    • -
  • The conversion functions are what end users would need to build their own - higher level abstractions.
    • -
  • The safe use patterns for the conversion functions are already familiar to - users of platform specific C endian conversion functions.
    • -
  • The Boost buffer and arithmetic types are specified and implemented in - terms of the conversion functions, so are easy to add later if desired.
    • -
-

Impact on the standard library

-

Endianness conversion functions would be pure additions to the -standard library and would break no existing user code (modulo the usual -namespace std caveats).

-

Proposed wording

-

The Boost Endian reference documentation for - -Conversion follows the standard library's Method of description -[description], so provides a fairly close approximation of what proposed wording -would look like. Actual proposed wording will be provided in a proposal document -if the library is of interest to the committee. Names will need -the usual review.

-

Revision history

-

R1: Limits proposal to the conversion functions. KISS.

-

R0: Considers adding all three endianness approaches to the standard library.

-

Question for the Library Evolution Group

-Assuming a fully-worded and reviewed proposal to add a - standardized version of the Boost Endian conversion functions to the standard - library or a library TS, how would you likely vote for it?

Target?

-

It is premature to discuss whether to target a TS or the IS. We need to have -an acceptable proposal in hand and see where we are in the release cycle before -deciding on the target.

-

Acknowledgements

-

The original design for the arithmetic classes was developed by Darin Adler -based on work by Mark Borgerding. The Boost documentation acknowledges -45+ -other people.

-
-

 

- - - - - \ No newline at end of file diff --git a/doc/styles.css b/doc/styles.css deleted file mode 100644 index 2018ec0..0000000 --- a/doc/styles.css +++ /dev/null @@ -1,20 +0,0 @@ - -body -{ - font-family: arial, sans-serif; - max-width: 6.5in; - margin: 0px auto; - font-size: 85%; -} - ins {background-color: #CCFFCC;} - del {background-color: #FFCACA;} - pre {background-color: #D7EEFF; font-size: 95%; font-family: "courier new", courier, serif;} - code {font-size: 110%; font-family: "courier new", courier, serif;} - table {font-size: 100%;} - - /* - © Copyright Beman Dawes, 2014 - Distributed under the Boost Software License, Version 1.0. - See www.boost.org/LICENSE_1_0.txt - */ - \ No newline at end of file diff --git a/doc/todo_list.html b/doc/todo_list.html deleted file mode 100644 index fed67d3..0000000 --- a/doc/todo_list.html +++ /dev/null @@ -1,106 +0,0 @@ - - - - - - - -Endian Library Do List - - - - -

Endian Library TODO List

- -

To Do

-

Format Review Comments

-

Interesting

-
    -
  • -

    John Filo - "Absolutely. I'd like to see support for float and - double, but
    - even without those additions, I still vote yes." "For those who deal with - non-native endian data, this library is
    - extremely useful. It automatically eliminates a whole class of common
    - programming errors when dealing with such data."
    - 

    • -
  • -

    Hartmut Kaiser - "Even if this is not a full review, I would like - to vote YES to include this
    - library into Boost. -

    Boost.Spirit is using (and shipping) with an older version of this library -
    - for several years now and we never had any problems with its usage in
    - Spirit. It is used as the underlying framework for the binary parsers and
    - generators and it is functioning as advertised.

    -

    As a quick test I replaced the internal (older) version of Boost.Endian in -
    - Spirit with the reviewed version. All of Spirits regression tests still
    - pass. "
    - 

    • -
-

Executive summary

-
    -
  • Common use case scenarios should be developed.
    • -
  • Example programs should be developed for the common use case scenarios.
    • -
  • Documentation should illuminate the differences between endian - integer/float type and endian conversion approaches to the common use case - scenarios, and provide guidelines for choosing the most appropriate approach - for user's applications.
    • -
  • Conversion functions supplying results via return should be - provided.
    • -
  • Platform specific performance enhancements such as use of compiler - intrinsics or relaxed alignment requirements should be supported.
    • -
  • Endian integer (and floating) types should be implemented via the - conversion functions. If that can't be done efficiently, consideration should - be given to expanding the conversion function signatures to resolve the - inefficiencies.
    • -
  • Benchmarks that measure performance should be provided. It should be - possible to compare platform specific performance enhancements against - portable base implementations, and to compare endian integer approaches - against endian conversion approaches for the common use case scenarios.
    • -
  • Float (32-bits) and double (64-bits) should be supported. IEEE 754 is the - primary use case.
    • -
  • Support for user defined types (UDTs) is desirable, and should be - supported where there would be no conflict with the other concerns.
    • -
  • There is some concern that endian integer/float arithmetic operations - might used - inadvertently or inappropriately. The impact of adding an endian_buffer class without arithmetic - operations should be investigated.
    • -
  • Stream insertion and extraction of the endian integer/float types should - be documented and included in the test coverage.
    • -
  • Binary I/O support that was investigated during development of the Endian - library should be put up for min-review for inclusion in the Boost I/O - library.
    • -
-

Docs

-
    -
  • one other point ... the help file seems to directly link to the c++ - headers.
    - this should be changed:
    -
    - * some browsers (at least chromium) will not display the header when clicking
    -  the link, but will save them on disk.
    -
    - * providing a direct link to the source code from the docs implies that the
    -  user will get some information that are necessary to use the library by
    -  reading the sources. imo, this is not the case for using boost.endian.
    -
    - * if a user opens integer.hpp, the first 60 lines just contain copyright, some
    -  historical notes, compiler-specific stuff, includes and ifdefs. imo, this is
    -  the implementation part, which should not be exposed to a user.
    -
    - so i'd suggest to completely remove the links to the c++ headers.
    - 
    • -
-
-

Last revised: -17 January, 2015

-

© Copyright Beman Dawes, 2012

-

Distributed under the Boost Software License, Version 1.0. See -www.boost.org/LICENSE_1_0.txt

-

 

- - - - \ No newline at end of file From f4f3c433dd420b28e5b5ab10186badd4e313fbfa Mon Sep 17 00:00:00 2001 From: Peter Dimov Date: Mon, 25 Mar 2019 15:45:52 +0200 Subject: [PATCH 10/12] Fix index.html --- index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/index.html b/index.html index 3ebbb5a..fe5eb44 100644 --- a/index.html +++ b/index.html @@ -1,10 +1,10 @@ - + Automatic redirection failed, please go to -doc/index.html. 
+doc/html/endian.html. 

© Copyright Beman Dawes, 2001

Distributed under the Boost Software License, Version 1.0. (See www.boost.org/LICENSE_1_0.txt) From ac96e4ac24a6e9cca974e3f3678f30aa898d3d23 Mon Sep 17 00:00:00 2001 From: Peter Dimov Date: Mon, 25 Mar 2019 19:23:25 +0200 Subject: [PATCH 11/12] Remove project-id from doc/Jamfile --- doc/Jamfile | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/Jamfile b/doc/Jamfile index 1e3b0cf..fb4caba 100644 --- a/doc/Jamfile +++ b/doc/Jamfile @@ -4,8 +4,6 @@ # Distributed under the Boost Software License, Version 1.0. # (http://www.boost.org/LICENSE_1_0.txt) -project doc/endian ; - import asciidoctor ; html endian.html : endian.adoc ; From 7c35802f6a4b37d906cbe4f0a1c2795fa764782e Mon Sep 17 00:00:00 2001 From: Peter Dimov Date: Sun, 14 Apr 2019 18:41:28 +0300 Subject: [PATCH 12/12] Switch Appveyor to 2015 image --- appveyor.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/appveyor.yml b/appveyor.yml index 41c9c01..f52a371 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -14,28 +14,28 @@ branches: environment: matrix: - - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013 - TOOLSET: msvc-9.0,msvc-10.0,msvc-11.0,msvc-12.0 - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 - TOOLSET: msvc-14.0 + TOOLSET: msvc-9.0,msvc-10.0,msvc-11.0 + - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 + TOOLSET: msvc-12.0,msvc-14.0 ADDRMD: 32,64 - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017 TOOLSET: msvc-14.1 CXXSTD: 14,17 ADDRMD: 32,64 - - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013 + - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 ADDPATH: C:\cygwin\bin; TOOLSET: gcc CXXSTD: 03,11,14,1z - - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013 + - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 ADDPATH: C:\cygwin64\bin; TOOLSET: gcc CXXSTD: 03,11,14,1z - - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013 + - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 ADDPATH: C:\mingw\bin; TOOLSET: gcc CXXSTD: 03,11,14,1z - - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2013 + - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015 ADDPATH: C:\mingw-w64\x86_64-6.3.0-posix-seh-rt_v5-rev1\mingw64\bin; TOOLSET: gcc CXXSTD: 03,11,14,1z