Boost Endian Library

Endian Library

Boost Home Endian Home Conversion Functions Endian Types Tutorial

Contents

Abstract
Introduction to endianness
Introduction to the Boost.Endian library
Acknowledgements

Headers

Abstract

Boost.Endian provides two facilities to manipulate the byte ordering of integers.

The primary use case is binary I/O of integers for portable exchange with other systems, via either file or network transmission.
A secondary use case is minimizing storage size via integers of sizes and/or alignments not supported by the built-in types. Integers 1, 2, 3, 4, 5, 6, 7, and 8 bytes in length are supported.
Two distinct approaches to byte ordering are provided. Each approach has a long history of successful use, and each approach has use cases where it is superior to the other approach.

Introduction to endianness

Consider the following code:

int16_t i = 0x0102;
FILE * file = fopen("test.bin", "wb");   // MUST BE BINARY
fwrite(&i, sizeof(int16_t), 1, file);
fclose(file);

On OS X, Linux, or Windows systems with an Intel CPU, a hex dump of the "test.bin" output file produces:

0201

On OS X systems with a PowerPC CPU, or Solaris systems with a SPARC CPU, a hex dump of the "test.bin" output file produces:

0102

What's happening here is that Intel CPU's order the bytes of an integer with the least-significant byte first, while SPARC CPU's place the most-significant byte first. Some CPU's, such as the PowerPC, allow the operating system to choose which ordering applies.

Most-significant-byte-first ordering is traditionally called "big endian" ordering and the least-significant-byte-first is traditionally called "little-endian" ordering. The names are derived from Jonathan Swift's satirical novel Gulliver’s Travels, where rival kingdoms opened their soft-boiled eggs at different ends.

See the Wikipedia's Endianness article for an extensive discussion of endianness.

Except for reading a core dump on little-endian systems, most programmers can ignore endianness. But when exchanging binary integers and binary floating point values between computer systems with differing endianness, whether by physical file transfer or over a network, programmers have to deal with endianness in their code.

Introduction to the Boost.Endian library

The Boost.Endian library provides two facilities for dealing with endianness.

The library provides two approaches to dealing with integer endianness:

Endian conversions - The application uses the built-in integer and floating point types, and calls the provided conversion functions to convert byte ordering as needed. Both mutating and non-mutating conversions are supplied, and each comes in unconditional and conditional variants. Type long double is not currently supported.

Endian types - The application uses the provided endian types which mimic the built-in integer types. For example, big32_t or little64_t. Types with lengths of 1-8 bytes are supported, rather than just 2, 4, and 8 bytes. There are no alignment requirements. Floating point types are not currently supported.

Boost Endian is a header-only library.

Choosing between endian types and endian conversion functions

Which approach is best for dealing with endianness depends on the application.

Advantages

Advantages
Endian types	Endian conversion functions
Mimic the built-in types. This can simplify use and eliminate logic errors since there is no need to reason about the current endianness of variables. 1, 2, 3, 4, 5, 6, 7, and 8 byte integers are supported. Use of 3, 5, 6, or 7 byte integers can reduce internal and external space usage and save I/O time. Alignment is not required. This can eliminate padding bytes in structures, reducing internal and external space usage and saving I/O time.	Already familiar to developers who have been using C conversion functions for years. Uses less CPU time, particularly if each variable is used many times relative to I/O of the variable. Easier to pass structures to third-party libraries expecting a specific structure format. Supports `float` and `double`.

Endian types

Endian conversion functions

Mimic the built-in types. This can simplify use and eliminate logic errors since there is no need to reason about the current endianness of variables.
1, 2, 3, 4, 5, 6, 7, and 8 byte integers are supported. Use of 3, 5, 6, or 7 byte integers can reduce internal and external space usage and save I/O time.
Alignment is not required. This can eliminate padding bytes in structures, reducing internal and external space usage and saving I/O time.

Already familiar to developers who have been using C conversion functions for years.
Uses less CPU time, particularly if each variable is used many times relative to I/O of the variable.
Easier to pass structures to third-party libraries expecting a specific structure format.
Supports float and double.

Overall FAQ

Why bother with endianness?

Binary data portability is the primary use case.

Does endianness have any uses outside of portable binary file or network I/O formats?

Using the unaligned integer types to save internal or external memory space is a minor secondary use case.

Why bother with binary I/O? Why not just use C++ Standard Library stream inserters and extractors?

Binary arithmetic data is smaller and therefore I/O is faster and file sizes are smaller. Transfer between systems is less expensive. Standard interchange formats often specify binary arithmetic data.

Furthermore, binary arithmetic data is of fixed size, and so fixed-size disk records are possible without padding, easing sorting and allowing direct access. Disadvantages, such as the inability to use text utilities on the resulting files, limit usefulness to applications where the binary I/O advantages are paramount.

Why is only big, little, and native endianness supported?

These are the only endian schemes that have any practical value today. PDP-11 and the other middle endian approaches are interesting historical curiosities but have no relevance to C++ developers.

Acknowledgements

Comments and suggestions were received from Adder, Benaka Moorthi, Christopher Kohlhoff, Cliff Green,Daniel James, Gennaro Proto, Giovanni Piero Deretta, Gordon Woodhull, dizzy, Hartmut Kaiser, Jeff Flinn, John Filo, John Maddock, Kim Barrett, Marsh Ray, Martin Bonner, Mathias Gaunard, Matias Capeletto, Neil Mayhew, Paul Bristow, Phil Endecott, Pyry Jahkola, Rene Rivera, Robert Stewart, Roland Schwarz, Scott McMurray, Sebastian Redl, Tim Blechmann, Tim Moore, tymofey, Tomas Puverle, Vincente Botet, Yuval Ronen and Vitaly Budovski,.

Last revised: 22 May, 2013

Distributed under the Boost Software License, Version 1.0. See www.boost.org/ LICENSE_1_0.txt