Files
mpl/doc/paper/mpl_paper.html
Dave Abrahams 01d61922bc Added libs/mpl
[SVN r15384]
2002-09-16 19:25:33 +00:00

1958 lines
100 KiB
HTML

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>The Boost C++ Metaprogramming Library</title>
<link rel="stylesheet" href="article.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.50.0">
<meta name="keywords" content="template metaprogramming, generic programming, programming languages, C++, STL, type systems, polymorphism, compile-time">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<div class="article">
<div class="titlepage">
<div>
<div class="articletitle">
<h1 class="title"><a name="idafawr"></a>The Boost C++ Metaprogramming Library</h1>
</div>
</div>
<div>
<div class="authorgroup">
<h4>Aleksey Gurtovoy<sup>i</sup> and David Abrahams<sup>ii</sup></h4>
<div class="affiliation"><sup>i</sup> <span class="orgname">MetaCommunications,</span> <a href="mailto:agurtovoy@meta-comm.com">agurtovoy@meta-comm.com</a></div>
<div class="affiliation"><sup>ii</sup> <span class="orgname">Boost Consulting,</span> <a href="mailto:david.abrahams@rcn.com">david.abrahams@rcn.com</a></div>
</div>
</div>
<div>
<div class="abstract">
<p class="title"><b>Abstract</b></p>
<p>This paper describes the <tt>Boost</tt> C++ template metaprogramming library (MPL), an extensible compile-time framework of algorithms, sequences and metafunction classes. The library brings together important abstractions from the generic and functional programming worlds to build a powerful and easy-to-use toolset which makes template metaprogramming practical enough for the real-world environments. The MPL is heavily influenced by its run-time equivalent - the Standard Template Library (STL), a part of the C++ standard library <span class="citation">[<a class="interlink" href="#ref.stl94" title="[stl94]">STL94</a>]</span>, <span class="citation">[<a class="interlink" href="#ref.iso98" title="[iso98]">ISO98</a>]</span>. Like the STL, it defines an open conceptual and implementation framework which can serve as a foundation for future contributions in the domain. The library's fundamental concepts and idioms enable the user to focus on solutions without navigating the universe of possible ad-hoc approaches to a given metaprogramming problem, even if no actual MPL code is used. The library also provides a compile-time lambda expression facility enabling arbitrary currying and composition of class templates, a feature whose runtime counterpart is often cited as missing from the STL. This paper explains the motivation, usage, design, and implementation of the MPL with examples of its real-life applications, and offers some lessons learned about C++ template metaprogramming.</p>
</div>
</div>
<div>
<div class="keywords">
<p><b>Keywords:</b> template metaprogramming, generic programming, programming languages, C++, STL, type systems, polymorphism, compile-time</p>
</div>
</div>
</div>
<div class="toc">
<p><b>Table of Contents</b></p>
<dl>
<dt>1. <a href="#intro">Introduction</a></dt>
<dd>
<dl>
<dt>1.1. <a href="#intro.native">Native language metaprogramming</a></dt>
<dt>1.2. <a href="#intro.cxx">Metaprogramming in C++</a></dt>
<dd>
<dl>
<dt>1.2.1. <a href="#intro.cxx.numeric">Numeric computations</a></dt>
<dt>1.2.2. <a href="#intro.cxx.type">Type computations</a></dt>
<dt>1.2.3. <a href="#intro.cxx.seq">Type sequences</a></dt>
</dl>
</dd>
<dt>1.3. <a href="#intro.whymetaprog">Why metaprogramming?</a></dt>
<dt>1.4. <a href="#intro.whylibrary">Why a metaprogramming library?</a></dt>
</dl>
</dd>
<dt>2. <a href="#usage">Basic usage</a></dt>
<dd>
<dl>
<dt>2.1. <a href="#typeselection">Conditional type selection</a></dt>
<dd>
<dl>
<dt>2.1.1. <a href="#delayedeval">Delayed evaluation</a></dt>
</dl>
</dd>
<dt>2.2. <a href="#metafunctions">Metafunctions</a></dt>
<dd>
<dl>
<dt>2.2.1. <a href="#metafunctions.simple">The simple form</a></dt>
<dt>2.2.2. <a href="#metafunctions.higherorder">Higher-order metafunctions</a></dt>
<dt>2.2.3. <a href="#metafunctions.classes">Metafunction classes</a></dt>
<dt>2.2.4. <a href="#metafunctions.onesize">One size fits all?</a></dt>
<dt>2.2.5. <a href="#metafunctions.lambda">From metafunction to metafunction class</a></dt>
</dl>
</dd>
<dt>2.3. <a href="#sequences">Sequences, algorithms, and iterators</a></dt>
<dd>
<dl>
<dt>2.3.1. <a href="#sequences.intro">Introduction</a></dt>
<dt>2.3.2. <a href="#sequences.algo">Algorithms and sequences</a></dt>
<dt>2.3.3. <a href="#sequences.concepts">Sequence concepts</a></dt>
<dt>2.3.4. <a href="#sequences.revisited">Ad hoc example revisited</a></dt>
<dt>2.3.5. <a href="#sequences.iterfold">iter_fold as the main iteration algorithm</a></dt>
<dt>2.3.6. <a href="#sequences.numbers">Sequences of numbers</a></dt>
<dt>2.3.7. <a href="#sequences.variety">A variety of sequences</a></dt>
<dt>2.3.8. <a href="#sequences.unrolling">Loop/recursion unrolling</a></dt>
</dl>
</dd>
</dl>
</dd>
<dt>3. <a href="#lambda">Lambda facility</a></dt>
<dt>4. <a href="#codegeneration">Code generation facilities</a></dt>
<dt>5. <a href="#example">Example: a compile-time FSM generator</a></dt>
<dd>
<dl>
<dt>5.1. <a href="#example.impl">Implementation</a></dt>
<dt>5.2. <a href="#example.relatedwork">Related work</a></dt>
</dl>
</dd>
<dt>6. <a href="#acknowl">Acknowledgements</a></dt>
<dt><a href="#refs">References</a></dt>
</dl>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="intro"></a>1. Introduction</h2>
</div>
</div>
<p>Metaprogramming is usually defined as the creation of programs which generate other programs. Parser generators such as YACC <span class="citation">[<a class="interlink" href="#ref.joh79" title="[joh79]">Joh79</a>]</span> are examples of one kind of program-generating program. The input language to YACC is a context-free grammar in Extended Backus-Naur Form <span class="citation">[<a class="interlink" href="#ref.ebnf" title="[ebnf]">EBNF</a>]</span>, and its output is a program which parses that grammar. Note that in this case the metaprogram (YACC) is written in a language (C) which does not directly support the description of generated programs. These specifications, which we'll call <span class="emphasis"><em>metadata</em></span>, are not written in C, but in a <span class="emphasis"><em>meta-language</em></span>. Because the the rest of the user's program typically requires a general-purpose programming system and must interact with the generated parser, the metadata is translated into C, which is then compiled and linked together with the rest of the system. The metadata thus undergoes two translation steps, and the user is always very conscious of the boundary between her metadata and the rest of her program.</p>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="intro.native"></a>1.1. Native language metaprogramming</h3>
</div>
</div>
<p>A more interesting form of metaprogramming is available in languages such as Scheme <span class="citation">[<a class="interlink" href="#ref.ss75" title="[ss75]">SS75</a>]</span>, where the generated program specification is given in the same language as the metaprogram itself. The metaprogrammer defines her meta-language as a subset of the expressible forms of the underlying language, and program generation can take place in the same translation step used to process the rest of the user's program. This allows users to switch transparently between ordinary programming, generated program specification, and metaprogramming, often without being aware of the transition.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="intro.cxx"></a>1.2. Metaprogramming in C++</h3>
</div>
</div>
<p>In C++, it was discovered almost by accident <span class="citation">[<a class="interlink" href="#ref.unr" title="[unr]">Unr</a>]</span>, <span class="citation">[<a class="interlink" href="#ref.vel95a" title="[vel95a]">Vel95a</a>]</span> that the template mechanism provides a rich facility for computation at compile-time. In this section, we'll explore the basic mechanisms and some common idioms used for metaprogramming in C++.</p>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="intro.cxx.numeric"></a>1.2.1. Numeric computations</h4>
</div>
</div>
<p>The availability of <span class="emphasis"><em>non-type template parameters</em></span> makes it possible to perform integer computations at compile-time. For example, the following template computes the factorial of its argument:</p>
<pre class="programlisting">
template&lt; unsigned n &gt;
struct factorial
{
static const unsigned value = n * factorial&lt;n-1&gt;::value;
};
template&lt;&gt;
struct factorial&lt;0&gt;
{
static const unsigned value = 1;
};
</pre>
<p>The program fragment above is called a <span class="emphasis"><em>metafunction</em></span>, and it is easy to see its relationship to a function designed to be evaluated at runtime: the &ldquo;metafunction argument&rdquo; is passed as a template parameter, and its &ldquo;return value&rdquo; is defined as a nested static constant. Because of the hard line between the expression of compile-time and runtime computation in C++, metaprograms look different from their runtime counterparts. Thus, although as in Scheme the C++ metaprogrammer writes her code in the same language as the ordinary program, only a subset of the full C++ language is available to her: those expressions which can be evaluated at compile-time. Compare the above with a straightforward runtime definition of the factorial function:</p>
<pre class="programlisting">
unsigned factorial(unsigned N)
{
return N == 0 ? 1 : N * factorial(N - 1);
}
</pre>
<p>While it is easy to see the analogy between the two recursive definitions, recursion is in general more important to C++ metaprograms than it is to runtime C++. In contrast to languages such as Lisp where recursion is idiomatic, C++ programmers will typically avoid recursion when possible. This is done not only for efficiency reasons, but also because of &ldquo;cultural momentum&rdquo;: recursive programs are simply harder (for C++ programmers) to think about. Like pure Lisp, though, the C++ template mechanism is a <span class="emphasis"><em>functional</em></span> programming language: as such it rules out the use of data mutation required to maintain loop variables.</p>
<p>A key difference between the runtime and compile-time factorial functions is the expression of the termination condition: our meta-factorial uses template specialization as a kind of <span class="emphasis"><em>pattern-matching</em></span> mechanism to describe the behavior when <tt>N</tt> is zero. The syntactic analogue in the runtime world would require two separate definitions of the same function. In this case the impact of the second definition is minimal, but in large metaprograms the cost of maintaining and understanding the terminating definitions can become significant.</p>
<p>Note also that a C++ metafunction's return value must be <span class="emphasis"><em>named</em></span>. The name chosen here, <tt>value</tt>, is the same one used for all numeric returns in the MPL. As we'll see, establishing a consistent naming convention for metafunction returns is crucial to the power of the library.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="intro.cxx.type"></a>1.2.2. Type computations</h4>
</div>
</div>
<p>How could we apply our <tt>factorial</tt> metafunction? We might, for example, produce an array type of an appropriate size to hold all permutations of instances of another type:</p>
<pre class="programlisting">
// permutation_holder&lt;T&gt;::type is an array type which can contain
// all permutations of a given T.
// unspecialized template for scalars
template&lt; typename T &gt;
struct permutation_holder
{
typedef T type[1][1];
};
// specialization for array types
template&lt; typename T, unsigned N &gt;
struct permutation_holder&lt;T[N]&gt;
{
typedef T type[factorial&lt;N&gt;::value][N];
};
</pre>
<p>Here we have introduced the notion of a <span class="emphasis"><em>type computation</em></span>. Like <tt>factorial</tt> above, <tt>permutation_holder</tt> template is a metafunction. However, where <tt>factorial</tt> manipulates unsigned integer values, <tt>permutation_holder</tt> accepts and &ldquo;returns&rdquo; a type (as the nested typedef <tt>type</tt>). Because the C++ type system provides a much richer set of expressions than anything we can use as a nontype template argument (e.g. the integers), C++ metaprograms tend to be composed mostly of type computations.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="intro.cxx.seq"></a>1.2.3. Type sequences</h4>
</div>
</div>
<p>The ability to programmatically manipulate collections of types is a central tool of most interesting C++ metaprograms. Because this capability is so well-supported by the MPL, we'll provide just a brief introduction to the basics here. Later on, we'll revisit the example below to show how it can be implemented using MPL.</p>
<p>First, we'd need a way to represent the collection. One idea might be to store the types in a structure:</p>
<pre class="programlisting">
struct types
{
int t1;
long t2;
std::vector&lt;double&gt; t3;
};
</pre>
<p>Unfortunately, this arrangement is not susceptible to the compile-time type introspection power that C++ gives us: there's no way to find out what the names of the members are, and even if we assume that they're named according to some convention as above, there's no way to know how many members there are. The key to solving this problem is to increase the uniformity of the representation. If we have a consistent way to get the first type of any sequence and the rest of the sequence, we can easily access all members:</p>
<pre class="programlisting">
template&lt; typename First, typename Rest &gt;
struct cons
{
typedef First first;
typedef Rest rest;
};
struct nil {};
typedef
cons&lt;int
, cons&lt;long
, cons&lt;std::vector&lt;double&gt;
, nil
&gt; &gt; &gt; my_types;
</pre>
<p>The structure described by <tt>types</tt> above is the compile-time analogue of a singly-linked list; it has been first introduced by Czarnecki and Eisenecker in <span class="citation">[<a class="interlink" href="#ref.ce98" title="[ce98]">CE98</a>]</span>. Now that we've adjusted the structure so that the C++ template machinery can &ldquo;peel it apart&rdquo;, let's examine a simple metafunction which does so. Suppose a user wished to find the largest of an arbitrary collection of types. We can apply the recursive metafunction formula which should by now be familiar:</p>
<div class="example"><a name="example.largest"></a>
<p class="title"><b>Example 1. 'largest' metafunction</b></p>
<pre class="programlisting">
// choose the larger of two types
template&lt;
typename T1
, typename T2
, bool choose1 = (sizeof(T1) &gt; sizeof(T2)) // hands off!
&gt;
struct choose_larger
{
typedef T1 type;
};
// specialization for the case where sizeof(T2) &gt;= sizeof(T1)
template&lt; typename T1, typename T2 &gt;
struct choose_larger&lt; T1,T2,false &gt;
{
typedef T2 type;
};
// get the largest of a cons-list
template&lt; typename T &gt; struct largest;
// specialization to peel apart the cons list
template&lt; typename First, typename Rest &gt;
struct largest&lt; cons&lt;First,Rest&gt; &gt;
: choose_larger&lt; First, typename largest&lt;Rest&gt;::type &gt;
{
// type inherited from base
};
// specialization for loop termination
template&lt; typename First &gt;
struct largest&lt; cons&lt;First,nil&gt; &gt;
{
typedef First type;
};
int main()
{
// print the name of the largest of my_types
std::cout
&lt;&lt; typeid(largest&lt;my_types&gt;::type).name()
&lt;&lt; std::endl
;
}
</pre>
</div>
<p>There are several things worth noticing about this code:</p>
<div class="itemizedlist">
<ul type="box">
<li style="list-style-type: square">
<p>It uses a few ad-hoc, esoteric techniques, or &ldquo;hacks&rdquo;. The default template argument <tt>choose1</tt> (labeled &ldquo;hands off!&rdquo;) is one example. Without it, we would have needed yet another template to provide the implementation of <tt>choose_larger</tt>, or we would have had to provide the computation explicitly as a parameter to the template - perhaps not bad for this example, but it would make <tt>choose_larger</tt> much less useful and more error-prone. The other hack is the derivation of a specialization of <tt>largest</tt> from <tt>choose_larger</tt>. This is a code-saving device which allows the programmer to avoid writing &ldquo;<tt>typedef typename</tt> ...<tt>::type type</tt>&rdquo; in the template body.</p>
</li>
<li style="list-style-type: square">
<p>Even this simple metaprogram uses three separate partial specializations. The <tt>largest</tt> metafunction uses <span class="emphasis"><em>two</em></span> specializations. One might expect that this indicates there are two termination conditions, but there are not: one specialization is needed simply to deal with access to the sequence elements. These specializations make the code difficult to read by spreading the definition of a single metafunction over several C++ template definitions. Also, because they are <span class="emphasis"><em>partial</em></span> specializations, they make the code unusable for a large community of C++ programmers whose compilers don't support that feature.</p>
</li>
</ul>
</div>
<p>While these techniques are, of course, a valuable part of the arsenal of any good C++ metaprogrammer, their use tends to make programs written in what is already an unusual style harder-to-read and harder-to-write. By encapsulating commonly-used structures and dealing with loop terminations internally, the MPL reduces the need for both tricky hacks and for template specializations.</p>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="intro.whymetaprog"></a>1.3. Why metaprogramming?</h3>
</div>
</div>
<p>It's worth asking why anyone would want to do this. After all, even a simple toy example like the factorial metafunction is somewhat esoteric. To show how the type computation can be put to work, let's examine a simple example. The following code produces an array containing all possible permutations of another array:</p>
<pre class="programlisting">
// can't return an array in C++, so we need this wrapper
template&lt; typename T &gt;
struct wrapper
{
T x;
};
// return an array of the N! permutations of 'in'
template&lt; typename T &gt;
wrapper&lt; typename permutation_holder&lt;T&gt;::type &gt;
all_permutations(T const&amp; in)
{
wrapper&lt;typename permutation_holder&lt;T&gt;::type&gt; result;
// copy the unpermutated array to the first result element
unsigned const N = sizeof(T) / sizeof(**result.x);
std::copy(&amp;*in, &amp;*in + N, result.x[0]);
// enumerate the permutations
unsigned const result_size = sizeof(result.x) / sizeof(T);
for (T* dst = result.x + 1; dst != result.x + result_size; ++dst)
{
T* src = dst - 1;
std::copy(*src, *src + N, *dst);
std::next_permutation(*dst, *dst + N);
}
return result;
}
</pre>
<p>The runtime definition of <tt>factorial</tt> would be useless in <tt>all_permutations</tt> above, since in C++ the sizes of array members must be computed at compile-time. However, there are alternative approaches; how could we avoid metaprogramming, and what would the consequences be?</p>
<div class="orderedlist">
<ol type="1">
<li>
<p>We could write programs to interpret the metadata directly. In our factorial example, the array size could have been a runtime quantity; then we'd have been able to use the straightforward factorial function. However, that would imply the use of dynamic allocation, which is often expensive.</p>
<p>To carry this further, YACC might be rewritten to accept a pointer-to-function returning tokens from the stream to be parsed, and a string containing the grammar description. This approach, however, would impose unacceptable runtime costs for most applications: either the parser would have to treat the grammar nondeterministically, exploring the grammar for each parse, or it would have to begin by replicating at runtime the substantial table-generation and optimization work of the existing YACC for each input grammar.</p>
</li>
<li>
<p>We could replace the compile-time computation with our own analysis. After all, the size of arrays passed to <tt>all_permutations</tt> are always known at compile-time, and thus can be known to its user. We could ask the user to supply the result type explicitly:</p>
<pre class="programlisting">
template&lt; typename Result, typename T &gt;
Result all_permutations(T const&amp; input);
</pre>
<p>The costs to this approach are obvious: we give up expressivity (by requiring the user to explicitly specify implementation details), and correctness (by allowing the user to specify them incorrectly). Anyone who has had to write parser tables by hand will tell you that the impracticality of this approach is the very reason of YACC's existence.</p>
<p>In a language such as C++, where the metadata can be expressed in the same language as the rest of the user's program, expressivity is further enhanced: the user can invoke metaprograms directly, without learning a foreign syntax or interrupting the flow of her code.</p>
</li>
</ol>
</div>
<p>So, the motivation for metaprogramming comes down to the combination of three factors: efficiency, expressivity, and correctness. While in classical programming there is always a tension between expressivity and correctness on one hand and efficiency on the other, in the metaprogramming world we wield new power: we can move the computation required for expressivity from runtime to compile-time.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="intro.whylibrary"></a>1.4. Why a metaprogramming <span class="emphasis"><em>library</em></span>?</h3>
</div>
</div>
<p>One might just as well ask why we need any generic library:</p>
<div class="itemizedlist">
<ul type="box">
<li style="list-style-type: square">
<p>Quality. Code that is appropriate for a general-purpose library is usually incidental to the purpose of its users. To a library developer, it is the central mission. On average, the containers and algorithms provided by any given C++ standard library implementation are more-flexible and better-implemented than the project-specific implementations which abound, because library development was treated as an end in itself rather than a task incidental to the development of some other application. With a centralized implementation for any given function, optimizations and improvements are more likely to have been applied.</p>
</li>
<li style="list-style-type: square">
<p>Re-use. More important even than the re-use of code which all libraries provide, a well-designed generic library establishes a <span class="emphasis"><em>framework of concepts and idioms</em></span> which establishes a reusable mental model for approaching problems. Just as the C++ Standard Template Library gave us iterator concepts and a function object protocol, the Boost Metaprogramming Library provides type-iterators and metafunction class protocol. A well-considered framework of idioms saves the metaprogrammer from considering irrelevant implementation details and allows her to concentrate on the problem at hand.</p>
</li>
<li style="list-style-type: square">
<p>Portability. A good library can smooth over the ugly realities of platform differences. While in theory a metaprogramming library is fully generic and shouldn't be concerned with these issues, in practice support for templates remains inconsistent even four years after standardization. This should perhaps not be surprising: C++ templates are the language's furthest-reaching and most complicated feature, which largely accounts for the power of metaprogramming in C++.</p>
</li>
<li style="list-style-type: square">
<p>Fun. Repeating the same idioms over and over is <span class="emphasis"><em>tedious</em></span>. It makes programmers tired and reduces productivity. Furthermore, when programmers get bored they get sloppy, and buggy code is even more costly than slowly-written code. Often the most useful libraries are simply patterns that have been &ldquo;plucked&rdquo; by an astute programmer from a sea of repetition. The MPL helps to reduce boredom by eliminating the need for the most commonly-repeated boilerplate coding patterns.</p>
</li>
</ul>
</div>
<p>As one can see, the MPL's development is motivated primarily by the same practical, real-world considerations that justify the development of any other library. Perhaps this is an indication that template metaprogramming is finally ready to leave the realm of the esoteric and enter the lingua franca of every day programmers.</p>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="usage"></a>2. Basic usage</h2>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="typeselection"></a>2.1. Conditional type selection</h3>
</div>
</div>
<p>Conditional type selection is the simplest basic construct of C++ template metaprogramming. Veldhuizen <span class="citation">[<a class="interlink" href="#ref.vel95a" title="[vel95a]">Vel95a</a>]</span> was the first to show how to implement it, and Czarnecki and Eisenecker <span class="citation">[<a class="interlink" href="#ref.ce00" title="[ce00]">CE00</a>]</span> first presented it as a standalone library primitive. The MPL defines the corresponding facility as follows:</p>
<pre class="programlisting">
template&lt;
typename Condition
, typename T1
, typename T2
&gt;
struct if_
{
typedef /*unspecified*/ type;
};
</pre>
<p>Note that the first template parameter of the template is a type.</p>
<pre class="programlisting">
// usage/semantics
typedef mpl::if_&lt;mpl::true_c,char,long&gt;::type t1;
typedef mpl::if_&lt;mpl::false_c,char,long&gt;::type t2;
BOOST_MPL_ASSERT_IS_SAME(t1, char);
BOOST_MPL_ASSERT_IS_SAME(t2, long);
</pre>
<p>The construct is important because template metaprograms often contain a lot of decision-making code, and, as we will show, spelling it manually every time via (partial) class template specialization quickly becomes impractical. The template is also important from the point of encapsulating the compiler workarounds.</p>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="delayedeval"></a>2.1.1. Delayed evaluation</h4>
</div>
</div>
<p>The way the C++ template instantiation mechanism works imposes some subtle limitations on applicability of the type selection primitive (<tt>if_</tt>), compared to a manually implemented equivalent of the selection code. For example, suppose we are implementing a <tt>pointed_type</tt> traits template such that <tt>pointed_type&lt;T&gt;::type</tt> instantiated for a <tt>T</tt> that is either a plain pointer (<tt>U*</tt>), <tt>std::auto_ptr&lt;U&gt;</tt>, or any of the Boost smart pointers <span class="citation">[<a class="interlink" href="#ref.spl" title="[spl]">SPL</a>]</span>, e.g. <tt>boost::scoped_ptr&lt;U&gt;</tt>, will give us the pointed type (<tt>U</tt>):</p>
<pre class="programlisting">
BOOST_MPL_ASSERT_IS_SAME(pointed_type&lt;my*&gt;::type, my);
BOOST_MPL_ASSERT_IS_SAME(pointed_type&lt; std::auto_ptr&lt;my&gt; &gt;::type, my);
BOOST_MPL_ASSERT_IS_SAME(pointed_type&lt; boost::scoped_ptr&lt;my&gt; &gt;::type, my);
</pre>
<p>Unfortunately, the straightforward application of <tt>if_</tt> to this problem does not work: <sup><a name="note.pointedtype" href="#ftn.note.pointedtype">1</a></sup></p>
<pre class="programlisting">
template&lt; typename T &gt;
struct pointed_type
: mpl::if_&lt;
boost::is_pointer&lt;T&gt;
, typename boost::remove_pointer&lt;T&gt;::type
, typename T::element_type // #1
&gt;
{
};
// the following code causes compilation error in line #1:
// name followed by "::" must be a class or namespace name
typedef pointed_type&lt;char*&gt;::type result;
</pre>
<p>Clearly, the expression <tt>typename T::element_type</tt> is not valid in the case of <tt>T == char*</tt>, and that's what the compiler is complaining about. Implementing the selection code manually solves the problem:</p>
<pre class="programlisting">
namespace aux {
// general case
template&lt; typename T, bool is_pointer = false &gt;
struct select_pointed_type
{
typedef typename T::element_type type;
};
// specialization for plain pointers
template&lt; typename T &gt;
struct select_pointed_type&lt;T,true&gt;
{
typedef typename boost::remove_pointer&lt;T&gt;::type type;
};
}
template&lt; typename T &gt;
struct pointed_type
: aux::select_pointed_type&lt;
T, boost::is_pointer&lt;T&gt;::value
&gt;
{
};
</pre>
<p>But this quickly becomes awkward if needs to be done repeatedly, and this awkwardness is compounded when partial specialization is not available. We can try to work around the problem as follows:</p>
<pre class="programlisting">
namespace aux {
template&lt; typename T &gt;
struct element_type
{
typedef typename T::element_type type;
};
}
template&lt; typename T &gt;
struct pointed_type
{
typedef typename mpl::if_&lt;
boost::is_pointer&lt;T&gt;
, typename boost::remove_pointer&lt;T&gt;::type
, typename aux::element_type&lt;T&gt;::type
&gt;::type type;
};
</pre>
<p>but this doesn't work either - the access to the <tt>aux::element_type&lt;T&gt;</tt>'s nested <tt>type</tt> member still forces the compiler to instantiate <tt>element_type&lt;T&gt;</tt> with <tt>T == char*</tt>, and that instantiation is, of course, invalid. Also, although in our case this does not lead to a compile error, the <tt>boost::remove_pointer&lt;T&gt;</tt> template always gets instantiated as well, and for the same reason (because we are accessing its nested <tt>type</tt> member). Unnecessary instantiation that is not fatal may or may be not a problem, depending on the &ldquo;weight&rdquo; of the template (how much the instantiation taxes the compiler), but a general rule of thumb would be to avoid such code.</p>
<p>Returning to our error, to make the above code compile, we need to factor the act of &ldquo;asking&rdquo; <tt>aux::element_type&lt;T&gt;</tt> for its nested <tt>type</tt> out of the <tt>if_</tt> invocation. The fact that both the <tt>boost::remove_pointer&lt;T&gt;</tt> trait template and <tt>aux::element_type&lt;T&gt;</tt> use the same naming convention for their result types makes the refactoring easier:</p>
<pre class="programlisting">
template&lt; typename T &gt;
struct pointed_type
{
private:
typedef typename mpl::if_&lt;
boost::is_pointer&lt;T&gt;
, boost::remove_pointer&lt;T&gt;
, aux::element_type&lt;T&gt;
&gt;::type func_;
public:
typedef typename func_::type type;
};
</pre>
<p>Now the compiler is guaranteed not to instantiate both <tt>boost::remove_pointer&lt;T&gt;</tt> and <tt>aux::element_type&lt;T&gt;</tt>, even although they are used as actual parameters to the <tt>if_</tt> template, so we are allowed to get away with <tt>aux::element_type&lt;char*&gt;</tt> so long as it won't end up being selected as <tt>func_</tt>.</p>
<p>The above technique is so common in template metaprograms, that it even makes sense to facilitate the selection of a nested <tt>type</tt> member by introducing a high-level equivalent to <tt>if_</tt> - the one that will do the <tt>func_::type</tt> operation (that is called [nullary] metafunction class application) as a part of its invocation. The MPL provides such template - it's called <tt>apply_if</tt>. Using it, we can re-write the above code as simple as:</p>
<pre class="programlisting">
template&lt; typename T &gt;
struct pointed_type
{
typedef typename mpl::apply_if&lt;
boost::is_pointer&lt;T&gt;
, boost::remove_pointer&lt;T&gt;
, aux::element_type&lt;T&gt;
&gt;::type type;
};
</pre>
<p>To make our techniques review complete, let's consider a slightly different example - suppose we want to define a high-level wrapper around <tt>boost::remove_pointer</tt> traits template <span class="citation">[<a class="interlink" href="#ref.ttl" title="[ttl]">TTL</a>]</span>, which will strip the pointer qualification conditionally. We will call it <tt>remove_pointer_if</tt>:</p>
<pre class="programlisting">
template&lt;
typename Condition
, typename T
&gt;
struct remove_pointer_if
{
typedef typename mpl::if_&lt;
Condition
, typename boost::remove_pointer&lt;T&gt;::type
, T
&gt;::type type;
};
</pre>
<p>Now the above works the first time, but it suffers from the problem we mentioned earlier - <tt>boost::remove_pointer&lt;T&gt;</tt> gets instantiated even if its result is never used. In the metaprogramming world compilation time is an important resource <span class="citation">[<a class="interlink" href="#ref.abr01" title="[abr01]">Abr01</a>]</span>, and it is wasted by unnecessary template instantiations. We've just seen how to deal with the problem when both arguments to <tt>if_</tt> are the results of nullary metafunction class applications, but in this example one of the arguments (<tt>T</tt>) is just a simple type, so the refactoring just doesn't seem possible.</p>
<p>The easiest way out of this situation would be to pass to <tt>if_</tt> a real nullary metafunction instead of <tt>T</tt> - the one that returns <tt>T</tt> on its invocation. The MPL provides a simple way to do it - we just substitute <tt>identity&lt;T&gt;</tt> and <tt>apply_if</tt> for <tt>T</tt> and <tt>if_</tt>:</p>
<pre class="programlisting">
template&lt;
typename Condition
, typename T
&gt;
struct remove_pointer_if
{
typedef typename mpl::apply_if&lt;
Condition
, boost::remove_pointer&lt;T&gt;
, mpl::identity&lt;T&gt;
&gt;::type type;
};
</pre>
<p>which gives us exactly what we wanted.</p>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="metafunctions"></a>2.2. Metafunctions</h3>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="metafunctions.simple"></a>2.2.1. The simple form</h4>
</div>
</div>
<p>In C++, the basic underlying language construct which allows parameterized compile-time computation is the <i>class template</i> (<span class="citation">[<a class="interlink" href="#ref.iso98" title="[iso98]">ISO98</a>]</span>, section 14.5.1 [temp.class]). A bare class template is the simplest possible model we could choose for metafunctions: it can take types and/or non-type arguments as actual template parameters, and instantiation &ldquo;returns&rdquo; a new type. For example, the following produces a type derived from its arguments:</p>
<pre class="programlisting">
template&lt; typename T1, typename T2 &gt;
struct derive : T1, T2
{
};
</pre>
<p>However, this model is far too limiting: it restricts the metafunction result not only to class types, but to instantiations of a given class template, to say nothing of the fact that every metafunction invocation introduces an additional level of template nesting. While that might be acceptable for this particular metafunction, any model which prevented us from &ldquo;returning&rdquo;, say, <tt>int</tt> is obviously not general enough. To meet this basic requirement, we must rely on a nested type to provide our return value:</p>
<pre class="programlisting">
template&lt; typename T1, typename T2 &gt;
struct derive
{
struct type : N1, N2 {};
};
// silly specialization, but demonstrates "returning" int
template&lt;&gt;
struct derive&lt;void,void&gt;
{
typedef int type;
};
</pre>
<p>Veldhuizen <span class="citation">[<a class="interlink" href="#ref.vel95a" title="[vel95a]">Vel95a</a>]</span> was first to talk about class templates of this form as &ldquo;compile-time functions&rdquo;, and Czarnecki and Eisenecker <span class="citation">[<a class="interlink" href="#ref.ce00" title="[ce00]">CE00</a>]</span> have introduced &ldquo;template metafunction&rdquo; as an equivalent term (they also use the simpler term &ldquo;metafunction&rdquo;, as do we). Czarnecki and Eisenecker have also recognized the limitations of the simple metafunction representation and suggested the form that we discuss in <a class="interlink" href="#metafunctions.classes" title="2.2.3. metafunction classes">Section 2.2.3</a>.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="metafunctions.higherorder"></a>2.2.2. Higher-order metafunctions</h4>
</div>
</div>
<p>While syntactically simple, the simple template metafunction form does not always interact optimally with the rest of C++. In particular, the simple metafunction form makes it unnecessarily awkward and tedious to define and work with higher-order metafunctions (metafunctions that operate on other metafunctions). In order to pass a simple metafunction to another template, we need to use <i>template template parameters</i>:</p>
<pre class="programlisting">
// returns F(T1,F(T2,T3))
template&lt;
template&lt;typename&gt; class F
, typename T1
, typename T2
, typename T3
&gt;
struct apply_twice
{
typedef typename F&lt;
T1
, typename F&lt;T2,T3&gt;::type
&gt;::type type;
};
// a new metafunction returning a type derived from T1, T2, and T3
template&lt;
typename T1
, typename T2
, typename T3
&gt;
struct derive3
: apply_twice&lt;derive,T1,T2,T3&gt;
{
};
</pre>
<p>This looks different, but it seems to work. <sup><a name="note.higherorder" href="#ftn.note.higherorder">2</a></sup> However, things begin to break down noticeably when we want to &ldquo;return&rdquo; a metafunction from our metafunction:</p>
<pre class="programlisting">
// returns G s.t. G(T1,T2,T3) == F(T1,F(T2,T3))
template&lt; template&lt;typename&gt; class F &gt;
struct compose_self
{
template&lt;
typename T1
, typename T2
, typename T3
&gt;
struct type
: apply_twice&lt;F,T1,T2,T3&gt;
{
};
};
</pre>
<p>The first and most obvious problem is that the result of applying <tt>compose_self</tt> is not itself a type, but a template, so it can't be passed in the usual ways to other metafunctions. A more subtle issue, however, is that the metafunction &ldquo;returned&rdquo; is not exactly what we intended. Although it acts just like <tt>apply_twice</tt>, it differs in one important respect: its identity. In the C++ type system, <tt>compose_self&lt;F&gt;::template type&lt;T,U,V&gt;</tt> is not a synonym for <tt>apply_twice&lt;F,T,U,V&gt;</tt>, and any metaprogram which compared metafunctions would discover that fact.</p>
<p>Because C++ makes a strict distinction between type and class template template parameters, reliance on simple metafunctions creates a &ldquo;wall&rdquo; between metafunctions and metadata, relegating metafunctions to the status of second-class citizens. For example, recalling our introduction to type sequences, there's no way to make a <tt>cons</tt> list of metafunctions:</p>
<pre class="programlisting">
typedef cons&lt;derive, cons&lt;derive3, nil&gt; &gt; derive_functions; // error!
</pre>
<p>We might consider redefining our <tt>cons</tt> cell so we can pass <tt>derive</tt> as the head element:</p>
<pre class="programlisting">
template &lt;
template&lt; template&lt;typename T, typename U&gt; class F
, typename Tail
&gt;
struct cons;
</pre>
<p>However, now we have another problem: C++ templates are polymorphic with respect to their type arguments, but not with respect to template template parameters. The arity (number of parameters) of any template template parameter is strictly enforced, so we <span class="emphasis"><em>still</em></span> can't embed <tt>derive3</tt> in a <tt>cons</tt> list. Moreover, polymorphism <span class="emphasis"><em>between</em></span> types and metafunctions is not supported (the compiler expects one or the other), and as we've seen, the syntax and semantics of &ldquo;returned&rdquo; metafunctions is different from that of returned types. Trying to accomplish everything with the simple template metafunction form would seriously limit the applicability of higher-order metafunctions and would have an overall negative effect on the both conceptual and implementation clarity, simplicity and size of the library.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="metafunctions.classes"></a>2.2.3. Metafunction classes</h4>
</div>
</div>
<p>Fortunately, the truism that &ldquo;there is no problem in software which can't be solved by adding yet another level of indirection&rdquo; applies here. To elevate metafunctions to the status of first-class objects, the MPL introduces the concept of a &ldquo;metafunction class&rdquo;:</p>
<pre class="programlisting">
// metafunction class form of derive
struct derive
{
template&lt; typename N1, typename N2 &gt;
struct apply
{
struct type : N1, N2 {};
};
};
</pre>
<p>This form should look familiar to anyone acquainted with function objects in STL, with the nested <tt>apply</tt> template taking the same role as the runtime function-call operator. In fact, compile-time metafunction classes have the same relationship to metafunctions that runtime function objects have to functions:</p>
<pre class="programlisting">
// function form of add
template&lt; typename T &gt; T add(T x, T y) { return x + y; }
// function object form of add
struct add
{
template&lt; typename T &gt;
T operator()(T x, T y) { return x + y; }
};
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="metafunctions.onesize"></a>2.2.4. One size fits all?</h4>
</div>
</div>
<p>The metafunction class form solves all the problems with ordinary template metafunction mentioned earlier: since it is a regular class, it can be placed in compile-time metadata sequences and manipulated by other metafunctions using the same protocols as for any other metadata. We thereby avoid the code-duplication needed to provide versions of each library component to operate on ordinary metadata and on metafunctions with each distinct supported arity.</p>
<p>On the other hand, it seems that accepting metafunction classes as <span class="emphasis"><em>the</em></span> representation for compile-time function entities imposes code duplication danger as well: if the library's own primitives, algorithms, etc. are represented as class templates, that means that one either cannot reuse these algorithms in the context of higher-order functions, or she have to duplicate all algorithms in the second form, so, for instance, there would be two versions of <tt>find</tt>:</p>
<pre class="programlisting">
// user-friendly form
template&lt;
typename Sequence
, typename T
&gt;
struct find
{
typedef /* ... */ type;
};
// "metafunction class" form
struct find_func
{
template&lt; typename Sequence, typename T &gt;
struct apply
{
typedef /* ... */ type;
};
};
</pre>
<p>Of course, the third option is to eliminate &ldquo;user-friendly form&rdquo; completely so one would always have to write:</p>
<pre class="programlisting">
typedef mpl::find::apply&lt;list,long&gt;::type iter;
// or, if one prefers,
// typedef mpl::apply&lt; mpl::find,list,long &gt;::type iter;
</pre>
<p>instead of</p>
<pre class="programlisting">
typedef mpl::find&lt;list,long&gt;::type iter;
</pre>
<p>That too would hurt usability, considering that the direct invocations of library's algorithms are far more often-used than passing algorithms as arguments to other algorithms/metafunctions.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="metafunctions.lambda"></a>2.2.5. From metafunction to metafunction class</h4>
</div>
</div>
<p>The MPL's answer to this dilemma is <i>lambda expressions</i>. Lambda is the mechanism that enables the library to curry metafunctions and convert them into metafunction classes, so when one wants to pass the <tt>find</tt> algorithm as an argument to a higher-order metafunction, she just write:</p>
<pre class="programlisting">
using namespace mpl::placeholder;
typedef mpl::apply&lt; my_f, mpl::find&lt;_1,_2&gt; &gt;::type result;
</pre>
<p>where <tt>_1</tt> and <tt>_2</tt> are placeholders for the first and second arguments to the resulting metafunction class. This preserves the intuitive syntax below for when the user wants to use <tt>find</tt> directly in her code:</p>
<pre class="programlisting">
typedef mpl::find&lt;list,long&gt;::type iter;
</pre>
<p>Lambda facility is described in more details in <a class="interlink" href="#lambda" title="3. lambda facility">Section 3</a>.</p>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="sequences"></a>2.3. Sequences, algorithms, and iterators</h3>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.intro"></a>2.3.1. Introduction</h4>
</div>
</div>
<p>Compile-time iteration over a sequence (of types) is one of the basic concepts of template metaprogramming. Differences in types of objects being manipulated is the most common point of variability of similar but not identical code/design, and such designs are the direct target for some metaprogramming. Templates were originally designed to solve this exact problem (e.g. <tt>std::vector</tt>). However, without predefined abstractions/constructs for manipulating/iterating over <span class="emphasis"><em>sequences</em></span> of types (as opposed to standalone types), and without known techniques for emulating these constructs using the current language facilities, their effect on helping high-level metaprogramming happen has been limited.</p>
<p>Czarnecki and Eisenecker <span class="citation">[<a class="interlink" href="#ref.ce98" title="[ce98]">CE98</a>]</span>, <span class="citation">[<a class="interlink" href="#ref.ce00" title="[ce00]">CE00</a>]</span> were the first to introduce compile-time sequences of types and some simple algorithms on them, although the idea of representing common data structures like trees, lists, etc. at compile time, using class template composition has been around for a while (e.g. most of the expression template libraries build such trees as a part of their expression "parsing" process <span class="citation">[<a class="interlink" href="#ref.vel95b" title="[vel95b]">Vel95b</a>]</span>). Alexandrescu <span class="citation">[<a class="interlink" href="#ref.ale01" title="[ale01]">Ale01</a>]</span> used lists of types and some algorithms on them to implement several design patterns; the accompanying code is known as the Loki library <span class="citation">[<a class="interlink" href="#ref.loki" title="[loki]">Loki</a>]</span>.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.algo"></a>2.3.2. Algorithms and sequences</h4>
</div>
</div>
<p>Most of the algorithms in the Boost Metaprogramming Library operate on sequences. For example, searching for a type in a list looks like this:</p>
<pre class="programlisting">
typedef mpl::list&lt;char,short,int,long,float,double&gt; types;
typedef mpl::find&lt;types,long&gt;::type iter;
</pre>
<p>Here, <tt>find</tt> accepts two parameters - a sequence to search (<tt>types</tt>) and the type to search for (<tt>long</tt>) - and returns an iterator <tt>iter</tt> pointing to the first element of the sequence such that <tt>iter::type</tt> is identical to <tt>long</tt>. If no such element exists, <tt>iter</tt> is identical to <tt>end&lt;types&gt;::type</tt>. Basically, this is how one would search for a value in a <tt>std::list</tt> or <tt>std::vector</tt>, except that <tt>mpl::find</tt> accepts the sequence as a single parameter, while <tt>std::find</tt> takes two iterators. Everything else is pretty much the same - the names are the same, the semantics are very close, there are iterators, and one can search not only by type, but also by using a predicate:</p>
<pre class="programlisting">
typedef mpl::find_if&lt; types,boost::is_float&lt;_&gt; &gt;::type iter;
</pre>
<p>This conceptual/syntactical similarity with the STL is not coincidental. Reusing the conceptual framework of the STL in the compile-time world allows us to apply familiar and sound approaches for dealing with sequential data structures. The algorithms and idioms which programmers already know from the STL can be applied again at compile-time. We consider this to be one of MPL's greatest strengths, distinguishing it from earlier attempts to build a template metaprogramming library.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.concepts"></a>2.3.3. Sequence concepts</h4>
</div>
</div>
<p>In the <tt>find</tt> example above, we searched for the type in a sequence built using the <tt>mpl::list</tt> template; but <tt>list</tt> is not the only sequence that the library provides. Neither is <tt>mpl::find</tt> or any other algorithm hard-coded to work only with <tt>list</tt> sequences. <tt>list</tt> is just one model of MPL's <span class="concept">Forward Sequence</span> concept, and <tt>find</tt> works with anything that satisfies this concept's requirements. The hierarchy of sequence concepts in MPL is quite simple - a <span class="concept">Sequence</span> is any compile-time entity for which <tt>begin&lt;&gt;</tt> and <tt>end&lt;&gt;</tt> produce iterators to the range of its elements; a <span class="concept">Forward Sequence</span> is a Sequence whose iterators satisfy <span class="concept">Forward Iterator</span> requirements; a <span class="concept">Bidirectional Sequence</span> is a Forward Sequence whose iterators satisfy <span class="concept">Bidirectional Iterator</span> requirements; finally, a <span class="concept">Random Access Sequence</span> is a Bidirectional Sequence whose iterators satisfy <span class="concept">Random Access Iterator</span> requirements. <sup><a name="note.seqconcepts" href="#ftn.note.seqconcepts">3</a></sup></p>
<p>Decoupling algorithms from particular sequence implementations (through iterators) allows a metaprogrammer to create her own sequence types and to retain the rest of the library at her disposal. For example, one can define a <tt>tiny_list</tt> for dealing with sequences of three types as follows:</p>
<pre class="programlisting">
template&lt; typename TinyList, long Pos &gt;
struct tiny_list_item;
template&lt; typename TinyList, long Pos &gt;
struct tiny_list_iterator
{
typedef typename tiny_list_item&lt;TinyList,Pos&gt;::type type;
typedef tiny_list_iterator&lt;TinyList, Pos-1&gt; prior;
typedef tiny_list_iterator&lt;TinyList, Pos+1&gt; next;
};
template&lt; typename T0, typename T1, typename T2 &gt;
struct tiny_list
{
typedef tiny_list_iterator&lt;tiny_list, 0&gt; begin;
typedef tiny_list_iterator&lt;tiny_list, 3&gt; end;
typedef T0 type0;
typedef T1 type1;
typedef T2 type2;
};
template&lt; typename TinyList &gt;
struct tiny_list_item&lt;TinyList,0&gt;
{
typedef typename TinyList::type0 type;
};
template&lt; typename TinyList &gt;
struct tiny_list_item&lt;TinyList,1&gt;
{
typedef typename TinyList::type1 type;
};
template&lt; typename TinyList &gt;
struct tiny_list_item&lt;TinyList,2&gt;
{
typedef typename TinyList::type2 type;
};
</pre>
<p>and then use it with any of the library algorithms as if it were <tt>mpl::list</tt>:</p>
<pre class="programlisting">
typedef tiny_list&lt; char,short,int &gt; types;
typedef mpl::transform&lt;
types
, boost::add_pointer&lt;_1&gt;
&gt;::type pointers;
</pre>
<p>Note that <tt>tiny_list</tt> is a model of Bidirectional Sequence; it would be a Random Access Sequence if we added <tt>advance</tt> and <tt>distance</tt> members to <tt>tiny_list_iterator</tt>:</p>
<pre class="programlisting">
template&lt; typename TinyList, long Pos &gt;
struct tiny_list_iterator
{
static long const position = Pos;
typedef typename tiny_list_item&lt;TinyList,Pos&gt;::type type;
typedef tiny_list_iterator&lt;TinyList, Pos-1&gt; prior;
typedef tiny_list_iterator&lt;TinyList, Pos+1&gt; next;
template&lt; typename N &gt; struct advance
{
typedef tiny_list_iterator&lt;
TinyList
, Pos + N::value
&gt; type;
};
template&lt; typename Other &gt; struct distance
{
typedef mpl::integral_c&lt;
long
, Other::position - position
&gt; type;
};
};
</pre>
<p>While the <tt>tiny_list</tt> itself might be not that interesting (after all, it can hold only three elements), if the technique above could be automated so we would be able to define not-so-tiny sequences (with five, ten, twenty, etc. elements), it would be very valuable. <sup><a name="note.tinylist" href="#ftn.note.tinylist">4</a></sup></p>
<p>External code generation is an option, but there exists a solution within the language. However, it is not a template metaprogramming, but rather <span class="emphasis"><em>preprocessor metaprogramming</em></span>. In fact, MPL's <tt>vector</tt> - a fixed-size type sequence that provides random-access iterators - is implemented very much like the above <tt>tiny_list</tt> - using the Boost Preprocessor library <span class="citation">[<a class="interlink" href="#ref.pre" title="[pre]">PRE</a>]</span>.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.revisited"></a>2.3.4. Ad hoc example revisited</h4>
</div>
</div>
<p>So, the library provides its users with almost complete compile-time equivalent of the STL framework. Does it help them to solve their metaprogramming tasks? Let's return to our earlier <a class="interlink" href="#example.largest" title="example 1. 'largest' metafunction"><tt>largest</tt></a> example to see if we can rewrite it in a better way with what MPL has to offer. Well, actually, there is not much to look at, because the MPL implementation is a one-liner (we'll spread it out here for readability) <sup><a name="note.maxelement" href="#ftn.note.maxelement">5</a></sup> :</p>
<pre class="programlisting">
template&lt; typename Sequence &gt;
struct largest
{
typedef typename mpl::max_element&lt;
Sequence
mpl::less&lt;
mpl::size_of&lt;_1&gt;
, mpl::size_of&lt;_2&gt;
&gt;
&gt;::type iter;
typedef typename iter::type type;
};
</pre>
<p>There are no more termination conditions with tricky pattern matching, no more partial specializations; and even more importantly, it's <span class="emphasis"><em>obvious</em></span> what the above code does - even although it's all templates - something that one could not say about the original version.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.iterfold"></a>2.3.5. iter_fold as the main iteration algorithm</h4>
</div>
</div>
<p>For the purpose of examining a little bit more of the library's internal structure, let's look at how <tt>max_element</tt> from the above example is implemented. One might expect that <span class="emphasis"><em>now</em></span> we will again see all these awkward partial specializations, esoteric pattern matching, etc. Well, let's see:</p>
<pre class="programlisting">
template&lt;
typename Sequence
, typename Predicate
&gt;
struct max_element
{
typedef typename mpl::iter_fold&lt;
Sequence
, typename mpl::begin&lt;Sequence&gt;::type
, if_&lt; less&lt; deref&lt;_1&gt;,deref&lt;_2&gt; &gt;, _2, _1 &gt;
&gt;::type type;
};
</pre>
<p>The first thing to notice here is that this algorithm is implemented in terms of another one: <tt>iter_fold</tt>. In fact, this is probably the most important point of the example, because nearly all other generic sequence algorithms in the library are implemented in terms of <tt>iter_fold</tt>. If a user should ever need to implement her own sequence algorithm, she'll almost certainly be able to do so using this primitive, which means she won't have to resort to implementing hand-crafted iteration, pattern matching of special cases for loop termination, or workarounds for lack of partial specialization. It also means that her algorithm will automatically benefit from any optimizations the library has implemented, (e.g. recursion unrolling), and that it will work with any sequence that is a model of ForwardSequence, because <tt>iter_fold</tt> does not require anything more of its sequence argument.</p>
<p><tt>iter_fold</tt> algorithm is basically a compile-time equivalent of the <tt>fold</tt> or <tt>reduce</tt> functions that comprise the basic and well-known primitives of many functional programming languages. An analogy more familiar to a C++ programmer would be the <tt>std::accumulate</tt> algorithm from the C++ standard library (<span class="citation">[<a class="interlink" href="#ref.iso98" title="[iso98]">ISO98</a>]</span>, section 26.4.1 [lib.accumulate]). However, <tt>iter_fold</tt> is designed to take advantage of the natural characteristics of recursive traversal: it accepts <span class="emphasis"><em>two</em></span> metafunction class arguments, the first of which is applied to the state "on the way in" and the second of which is applied "on the way out".</p>
<p>The interface to <tt>iter_fold</tt> is defined in MPL as follows:</p>
<pre class="programlisting">
template&lt;
typename Sequence
, typename InitialState
, typename ForwardOp
, typename BackwardOp = _1
&gt;
struct iter_fold
{
typedef /*unspecified*/ type;
};
</pre>
<p>The algorithm &ldquo;returns&rdquo; the result of two-way successive applications of binary <tt>ForwardOp</tt> and <tt>BackwardOp</tt> operations to iterators in range [<tt>begin&lt;Sequence&gt;::type</tt>, <tt>end&lt;Sequence&gt;::type</tt>) and previous result of an operation; the <tt>InitialState</tt> is logically placed before the sequence and included in the forward traversal. The result <tt>type</tt> is identical to <tt>InitialState</tt> if the sequence is empty.</p>
<p>The library also provides <tt>iter_fold_backward</tt>, <tt>fold</tt>, and <tt>fold_backward</tt> algorithms which wrap <tt>iter_fold</tt> to accommodate its most common usage patterns.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.numbers"></a>2.3.6. Sequences of numbers</h4>
</div>
</div>
<p>What we've seen so far were sequences (and algorithms on sequences) of types. It is both possible and easy to manipulate compile-time <span class="emphasis"><em>values</em></span> using the library as well. The only thing to remember is that in C++, class template non-type template parameters give us one more example of non-polymorphic behavior. In other words, if one declared a metafunction to take a non-type template parameter (e.g. <tt>long</tt>) it's not possible to pass anything besides compile-time integral constants to it:</p>
<pre class="programlisting">
template&lt; long N1, long N2 &gt;
struct equal_to
{
static bool const value = (N1 == N2);
};
equal_to&lt;5,5&gt;::value; // ok
equal_to&lt;int,int&gt;::value; // error!
</pre>
<p>And of course this doesn't work the other way around either:</p>
<pre class="programlisting">
typedef mpl::list&lt;1,2,3,4,5&gt; numbers; // error!
</pre>
<p>While this may be an obvious limitation, it imposes yet another dilemma on the library design: on the one hand, we don't want to restrict users to type manipulations only, and on the other hand, full support for integral manipulations would require at least duplication of most of the library facilities <sup><a name="note.nontype" href="#ftn.note.nontype">6</a></sup> - the same situation as we would have if we had chosen to represent metafunctions as ordinary class templates. The solution for this issue is the same as well: we represent integral values by wrapping them in types <sup><a name="note.valuewrapping" href="#ftn.note.valuewrapping">7</a></sup> . For example, to create a list of numbers one can write:</p>
<pre class="programlisting">
typedef mpl::list&lt;
mpl::int_c&lt;1&gt;
, mpl::int_c&lt;2&gt;
, mpl::int_c&lt;3&gt;
, mpl::int_c&lt;4&gt;
, mpl::int_c&lt;5&gt;
&gt; numbers;
</pre>
<p>Wrapping integral constants into types to make them first-class citizens is important well inside metaprograms, where one often doesn't know (and doesn't care) if the metafunctions she is using operate on types, integral values, other metafunctions, or something else, like fixed-point or rational numbers (<tt>mpl::fixed_c</tt> and <tt>mpl::rational_c</tt>).</p>
<p>But, from the user's perspective, the above example is much more verbose than the shorter, incorrect one. Thus, for the purpose of convenience, the library does provide users with a template that takes non-type template parameters, but offers a more compact notation:</p>
<pre class="programlisting">
typedef mpl::list_c&lt;long,1,2,3,4,5&gt; numbers;
</pre>
<p>There is a similar <tt>vector</tt> counterpart as well:</p>
<pre class="programlisting">
typedef mpl::vector_c&lt;long,1,2,3,4,5&gt; numbers;
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.variety"></a>2.3.7. A variety of sequences</h4>
</div>
</div>
<p>Previous efforts to provide generalized metaprogramming facilities for C++ have always concentrated on <tt>cons</tt>-style type lists and a few core algorithms like <tt>size</tt> and <tt>at</tt>, which are tied to the specific sequence implementation. Such systems have an elegant simplicity reminiscent of the analogous functionality in pure functional Lisp. It is much more time-consuming to implement even a basic set of the sequence algorithms provided by equivalent run-time libraries (the STL in particular), but if we have learned anything from the STL, it is that tying those algorithms' implementations to a specific sequence implementation is a misguided effort!</p>
<p>The truth is that there is no single &ldquo;best&rdquo; type sequence implementation for the same reasons that there will never be a single &ldquo;best&rdquo; runtime sequence implementation. Furthermore, there are <span class="emphasis"><em>already</em></span> quite a number of type list implementations in use today; and just as the STL algorithms can operate on sequences which don't come from STL containers, so the MPL algorithms are designed to work with foreign type sequences.</p>
<p>It may be an eye-opening fact for some that type lists are not the only useful compile-time sequence. Again, the need for a variety of compile-time containers arises for the same reasons that we have lists, vectors, deques, and sets in the C++ standard library - different containers have different functional and performance characteristics which determine not only applicability and efficiency of particular algorithms, but also the expressiveness or verbosity of the code that uses them. While runtime performance is not an issue for C++ metaprograms, compilation speed is often a significant bottleneck to advanced C++ software development <span class="citation">[<a class="interlink" href="#ref.abr01" title="[abr01]">Abr01</a>]</span>.</p>
<p>The MPL provides five built-in sequences: <tt>list</tt>, <tt>list_c</tt> (really just a <tt>list</tt> of value wrappers), <tt>vector</tt>, a randomly-accessible sequence of fixed maximum size, <tt>vector_c</tt>, and <tt>range_c</tt>, a randomly-accessible sequence of consecutive integral values. More important, however, is its ability to adapt to arbitrary sequence types. The only core operations that a sequence is required to provide in order to be used with the library algorithms are <tt>begin&lt;&gt;</tt> and <tt>end&lt;&gt;</tt> metafunctions which "return" iterators into the sequence. As with the STL, it is the iterators which are used to implement most of the general-purpose sequence algorithms the library provides. Also, as with the STL, algorithm specialization is used to take advantage of implementation knowledge about particular sequences: many of the "basic" sequence operations such as <tt>back&lt;&gt;</tt>, <tt>front&lt;&gt;</tt>, <tt>size&lt;&gt;</tt>, and <tt>at&lt;&gt;</tt> are specialized on sequence type to provide a more efficient implementation than the fully generic version.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h4 class="title"><a name="sequences.unrolling"></a>2.3.8. Loop/recursion unrolling</h4>
</div>
</div>
<p>Almost coincidentally, loop unrolling can be as important to compile-time iterative algorithms as it is to runtime algorithms. To see why, one must first remember that all "loops" in C++ metaprograms, are in fact, implemented with recursion, and that the template instantiation depth can be a valuable resource in a compiler implementation. In fact, Annex B of the C++ standard (<span class="citation">[<a class="interlink" href="#ref.iso98" title="[iso98]">ISO98</a>]</span>, annex B [limits]) <span class="emphasis"><em>recommends</em></span> a minimum depth of 17 recursively nested template instantiations; but this is far too low for many serious metaprograms, some of which easily exceed the hard-coded instantiation limits of some otherwise excellent compilers. To see how this works in action, let's examine a straightforward implementation of the <tt>fold</tt> metafunction, which combines some algorithm state with each element of a sequence:</p>
<pre class="programlisting">
namespace aux {
// unspecialized version combines the initial state and first element
// and recurses to process the rest
template&lt;
typename Start
, typename Finish
, typename State
, typename BinaryFunction
&gt;
struct fold_impl
: fold_impl&lt;
typename Start::next
, Finish
, typename apply&lt;
BinaryFunction
, State
, typename Start::type
&gt;::type
, BinaryFunction
&gt;
{
};
// specialization for loop termination
template&lt;
typename Finish
, typename State
, typename BinaryFunction
&gt;
struct fold_impl&lt;Finish,Finish,State,BinaryFunction&gt;
{
typedef State type;
};
} // namespace aux
// public interface
template&lt;
typename Sequence
, typename State
, typename ForwardOp
&gt;
struct fold
: aux::fold_impl&lt;
, typename begin&lt;Sequence&gt;::type
, typename end&lt;Sequence&gt;::type
, State
, typename lambda&lt;ForwardOp&gt;::type
&gt;
{
};
</pre>
<p>Although simple and elegant, this implementation will always incur at least as many levels of recursive template instantiation as there are elements in the input sequence. <sup><a name="note.unrolling1" href="#ftn.note.unrolling1">8</a></sup> The library addresses this problem by explicitly "unrolling" the recursion. To apply the technique to our <tt>fold</tt> example, we begin by factoring out a single step of the algorithm. Our <tt>fold_impl_step</tt> metafunction has two results: <tt>type</tt> (the next state), and <tt>iterator</tt> (the next sequence position).</p>
<pre class="programlisting">
template&lt;
typename BinaryFunction
, typename State
, typename Start
, typename Finish
&gt;
struct fold_impl_step
{
typedef typename apply&lt;
BinaryFunction
, State
, typename Start::type
&gt;::type type;
typedef typename Start::next iterator;
};
</pre>
<p>As with our main algorithm implementation, we specialize for the loop termination condition so that the step becomes a no-op:</p>
<pre class="programlisting">
template&lt;
typename BinaryFunction
, typename State
, typename Finish
&gt;
struct fold_impl_step&lt;BinaryFunction,State,Finish,Finish&gt;
{
typedef State type;
typedef Finish iterator;
};
</pre>
<p>Now we can now reduce <tt>fold</tt>'s instantiation depth by any constant factor N simply by inserting N invocations of <tt>fold_impl_step</tt>. Here we've chosen a factor of 4:</p>
<pre class="programlisting">
template&lt;
typename Start
, typename Finish
, typename State
, typename BinaryFunction
&gt;
struct fold_impl
{
private:
typedef fold_impl_step&lt;
BinaryFunction
, State
, Start
, Finish
&gt; next1;
typedef fold_impl_step&lt;
BinaryFunction
, typename next1::type
, typename next1::iterator
, Finish
&gt; next2;
typedef fold_impl_step&lt;
BinaryFunction
, typename next2::type
, typename next2::iterator
, Finish
&gt; next3;
typedef fold_impl_step&lt;
BinaryFunction
, typename next3::type
, typename next3::iterator
, Finish
&gt; next4;
typedef fold_impl_step&lt;
typename next4::iterator
, Finish
, typename next4::type
, BinaryFunction
&gt; recursion;
public:
typedef typename recursion::type type;
};
</pre>
<p>The MPL applies this unrolling technique across all algorithms with an unrolling factor tuned according to the demands of the C++ implementation in use, and with an option for the user to override the value. <sup><a name="note.unrolling2" href="#ftn.note.unrolling2">9</a></sup> This fact enables users to push beyond the metaprogramming limits they would usually encounter with more naive algorithm implementations. Experiments also show a small (up to 10%) increase in metaprogram instantiation speed on some compilers when loop unrolling is used.</p>
</div>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="lambda"></a>3. Lambda facility</h2>
</div>
</div>
<p>The MPL's lambda facility allows the <i>inline composition</i> of class templates into &ldquo;lambda expressions&rdquo;, which are classes and can therefore be passed around as ordinary metafunction classes, or transformed into metafunction classes before application using the expression:</p>
<pre class="programlisting">
typedef mpl::lambda&lt;expr&gt;::type func;
</pre>
<p>For example, <tt>boost::remove_const</tt> traits template from Boost <tt>type_traits</tt> library <span class="citation">[<a class="interlink" href="#ref.ttl" title="[ttl]">TTL</a>]</span> is a class template (obviously), or a <a class="interlink" href="#metafunctions" title="2.2. metafunctions">metafunction</a> in MPL terminology. The simplest example of an &ldquo;inline composition&rdquo; of it would be something like:</p>
<pre class="programlisting">
typedef boost::remove_const&lt;_1&gt; expr;
</pre>
<p>This forms a so called &ldquo;lambda expression&rdquo;, which is neither a metafunction class, nor a metafunction, yet can be passed around everywhere because it's an ordinary C++ class, because all MPL facilities are polymorphic with respect to their arguments. Now, that lambda expression can be <span class="emphasis"><em>transformed</em></span> into a metafunction class using the MPL's <tt>lambda</tt> facility:</p>
<pre class="programlisting">
typedef boost::remove_const&lt;_1&gt; expr;
typedef mpl::lambda&lt;expr&gt;::type func;
</pre>
<p>The <tt>func</tt> is a unary metafunction class and can be used as such. In particular, it can be pass around or invoked (applied):</p>
<pre class="programlisting">
typedef mpl::apply&lt;func,int const&gt;::type res;
BOOST_MPL_ASSERT_IS_SAME(res, int);
</pre>
<p>or even</p>
<pre class="programlisting">
typedef func::apply&lt;int const&gt;::type res;
BOOST_MPL_ASSERT_IS_SAME(res, int);
</pre>
<p>Inline composition is very appealing syntactically when one deals with metafunctions, because it makes the expression obvious:</p>
<pre class="programlisting">
typedef mpl::logical_or&lt;
mpl::less&lt; mpl::size_of&lt;_1&gt;, mpl::int_c&lt;16&gt; &gt;
, boost::is_same&lt;_1,_2&gt;
&gt; expr;
typedef mpl::lambda&lt;expr&gt;::type func;
</pre>
<p>And one does not have to specify the last part (<tt>typedef lambda&lt;expr&gt;::type func</tt>), because all the algorithms do this to any of their metafunction class operands internally (a <tt>lambda&lt;T&gt;::type</tt> expression applied to a metafunction class gives back the same metafunction class, so it's safe to apply the expression unconditionally).</p>
<p>The alternative way to write an equivalent of the above metafunction class would be:</p>
<pre class="programlisting">
typedef bind&lt;
mpl::meta_fun2&lt;mpl::logical_or&gt;
, mpl::bind&lt; mpl::meta_fun2&lt;mpl::less&gt;
, mpl::bind&lt; mpl::meta_fun1&lt;mpl::size_of&gt;,_1 &gt;
, mpl::int_c&lt;16&gt;
&gt;
, mpl::bind&lt; mpl::meta_fun2&lt;boost::is_same&gt;,_1,_2 &gt;
&gt; func;
</pre>
<p>Or to use <tt>mpl::compose_</tt> family of templates in a similar way. Here, we use <tt>mpl::meta_fun</tt> templates to convert metafunctions into metafunction classes and then combine them using <tt>mpl::bind</tt>. The transformation from this form to the above inline lambda expression and vice-versa is mechanical, and that is essentially what the <tt>typedef mpl::lambda&lt;expr&gt;::type</tt> expression does.</p>
<p>For its own metafunctions (algorithms, primitives, etc.), MPL enables one to write the above in a less cumbersome way:</p>
<pre class="programlisting">
typedef mpl::bind&lt;
mpl::logical_or&lt;&gt;
, mpl::bind&lt; mpl::less&lt;&gt;, mpl::bind&lt;mpl::size_of&lt;&gt;,_1&gt;, mpl::int_c&lt;16&gt; &gt;
, mpl::bind&lt; mpl::make_f2&lt;boost::is_same&gt;, _1,_2 &gt;
&gt; func;
</pre>
<p>Note that we still have to wrap <tt>is_same</tt> into <tt>make_f2</tt>, because it's a foreign template.</p>
<p>Now, about combining class template metafunctions and metafunction classes in the single lambda expression - it can be done like this:</p>
<pre class="programlisting">
struct my_predicate
{
template&lt; typename T1, typename T2 &gt; struct apply
{
//...
};
};
typedef mpl::logical_or&lt;
mpl::less&lt; mpl::size_of&lt;_&gt;,mpl::int_c&lt;16&gt; &gt;
, mpl::bind&lt; my_predicate,_,_ &gt; // here
&gt; expr;
</pre>
<p>To bind something to one of its arguments (or change the order of parameters), then use either:</p>
<pre class="programlisting">
typedef mpl::logical_or&lt;
mpl::less&lt; mpl::size_of&lt;_&gt;,mpl::int_c&lt;16&gt; &gt;
, mpl::bind&lt;my_predicate,int,_&gt;::type // here
&gt; expr;
</pre>
<p>or</p>
<pre class="programlisting">
typedef mpl::logical_or&lt;
mpl::less&lt; mpl::size_of&lt;_&gt;,mpl::int_c&lt;16&gt; &gt;
, my_predicate::apply&lt;int,_&gt; // here
&gt; expr;
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="codegeneration"></a>4. Code generation facilities</h2>
</div>
</div>
<p>There are cases, especially in the domain of numeric computation, when one wants to perform some part of the calculations at compile-time, and then pass the results to a run-time part of the program for further processing. For example, suppose one has implemented a complex compile-time algorithm that works with fixed-point arithmetic:</p>
<pre class="programlisting">
// fixed-point algorithm input
typedef mpl::vector&lt;
mpl::fixed_c&lt;-1,2345678&gt;
, mpl::fixed_c&lt;9,0001&gt;
// ..
, mpl::fixed_c&lt;3,14159&gt;
&gt; input_data;
/*
complex compile-time algorithm
*/
typedef /*...*/ result_data;
</pre>
<p>Suppose the <tt>result_data</tt> here is a sequence of <tt>mpl::fixed_c</tt> types that keeps the results of the algorithm, and now one wishes to feed that result to the run-time part of the algorithm. With MPL she can do this:</p>
<pre class="programlisting">
double my_algorithm()
{
// passing the results to the run-time part of the program
std::vector&lt;double&gt; results;
results.reserve(mpl::size&lt;result_data&gt;::value);
mpl::for_each&lt;numbers,_&gt;(
boost::bind(&amp;std::vector&lt;double&gt;::push_back, &amp;results, _1)
);
// ...
}
</pre>
<p>The <tt>for_each&lt;numbers,_&gt;(...)</tt> call is what actually transfers the compile-time <tt>result_data</tt> into run-time <tt>results</tt>. <tt>for_each</tt> is a function template declared as:</p>
<pre class="programlisting">
template&lt;
typename Seq
, typename TransformOp
, typename F
&gt;
void for_each(F f)
{
// ...
}
</pre>
<p>To call the function, one is required to explicitly provide two actual template parameters, a compile-time sequence <tt>Seq</tt> and a unary transformation metafunction <tt>TransformOp</tt>, plus a run-time function argument <tt>f</tt> (in our example, <tt>numbers</tt>, <tt>_</tt>, and <tt>boost::bind(...)</tt> correspondingly). <tt>f</tt> is a function object which <tt>operator()</tt> is called for every element in the <tt>Seq</tt> tranfromed by <tt>TransformOp</tt>.</p>
<p>Applying this to our example, the</p>
<pre class="programlisting">
mpl::for_each&lt;numbers,_&gt;(
boost::bind(&amp;std::vector&lt;double&gt;::push_back, &amp;results, _1)
);
</pre>
<p>call is roughly equivalent to this:</p>
<pre class="programlisting">
f(mpl::apply&lt; _,mpl::at_c&lt;result_data,0&gt;::type &gt;::type());
f(mpl::apply&lt; _,mpl::at_c&lt;result_data,1&gt;::type &gt;::type());
// ...
f(mpl::apply&lt; _,mpl::at_c&lt;result_data,n&gt;::type &gt;::type());
</pre>
<p>where <tt>n == mpl::size&lt;result_data&gt;::type::value</tt>.</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="example"></a>5. Example: a compile-time FSM generator</h2>
</div>
</div>
<p>Finite state machines (FSMs) are an important tool for describing and implementing program behavior <span class="citation">[<a class="interlink" href="#ref.hu79" title="[hu79]">HU79</a>]</span>, <span class="citation">[<a class="interlink" href="#ref.mar98" title="[mar98]">Mar98</a>]</span>. They also are a good example of a domain in which metaprogramming can be applied to reduce the amount of repetitive and boilerplate operations one must perform in order to implement these simple mathematical models in code. Below we present a simple state machine generator that has been implemented using Boost Metaprogramming Library facilities. The generator takes a compile-time automata description, and converts it into C++ code that implements the FSM at run-time.</p>
<p>The FSM description is basically a combination of states and events plus a state transition table (STT), which ties them all together. The generator walks through the table and generates the state machine's <tt>process_event</tt> method that is the essence of an FSM.</p>
<p>Suppose we want to implement a simple music player using a finite state machine model. The state transition table for the FSM is shown in <a class="interlink" href="#example.fsm.stt" title="table 1. player's state transition table with actions">Table 1</a>. The STT format reflects the way one usually describes the behavior of an FSM in plain English. For example, the first line of the table can be read as follows: &ldquo;If the model is in the <tt>stopped</tt> state and the <tt>play_event</tt> is received, then the <tt>do_play</tt> transition function is called, and the model transitions to the <tt>playing</tt> state&rdquo;.</p>
<div class="table"><a name="example.fsm.stt"></a>
<table summary="Player's state transition table with actions" border="0">
<colgroup>
<col>
<col>
<col>
<col></colgroup>
<thead>
<tr>
<th align="left">State</th>
<th align="left">Event</th>
<th align="left">Next state</th>
<th align="left">Transition function</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><tt>stopped</tt></td>
<td align="left"><tt>play_event</tt></td>
<td align="left"><tt>playing</tt></td>
<td align="left"><tt>do_play</tt></td>
</tr>
<tr>
<td align="left"><tt>playing</tt></td>
<td align="left"><tt>stop_event</tt></td>
<td align="left"><tt>stopped</tt></td>
<td align="left"><tt>do_stop</tt></td>
</tr>
<tr>
<td align="left"><tt>playing</tt></td>
<td align="left"><tt>pause_event</tt></td>
<td align="left"><tt>paused</tt></td>
<td align="left"><tt>do_pause</tt></td>
</tr>
<tr>
<td align="left"><tt>paused</tt></td>
<td align="left"><tt>play_event</tt></td>
<td align="left"><tt>playing</tt></td>
<td align="left"><tt>do_resume</tt></td>
</tr>
<tr>
<td align="left"><tt>paused</tt></td>
<td align="left"><tt>stop_event</tt></td>
<td align="left"><tt>stopped</tt></td>
<td align="left"><tt>do_stop</tt></td>
</tr>
</tbody>
</table>
<p class="title"><b>Table 1. Player's state transition table with actions</b></p>
</div>
<p>The transition table provides us with a complete formal definition of the target FSM, and there are several ways to transform that definition into code. For instance, if we define states as members of an enumeration type, and events as classes derived from some base <tt>event</tt> class <sup><a name="note.fsm" href="#ftn.note.fsm">10</a></sup> , like so:</p>
<pre class="programlisting">
class player
{
public:
// event declarations
struct event;
struct play_event;
struct stop_event;
struct pause_event;
// "input" function
void process_event(event const&amp;); // throws
private:
// states
enum state_t { stopped, playing, paused };
// transition functions
void do_play(play_event const&amp;);
void do_stop(stop_event const&amp;);
void do_pause(pause_event const&amp;);
void do_resume(play_event const&amp;);
private:
state_t m_state;
};
</pre>
<p>then the most straightforward way to derive the FSM implementation from the above table would be something like this:</p>
<pre class="programlisting">
void player::process_event(event const&amp; e)
{
if (m_state == stopped)
{
if (typeid(e) == typeid(play_event))
{
do_play(static_cast&lt;play_event const&amp;&gt;(e));
m_state = playing;
return;
}
}
else if (m_state == playing)
{
if (typeid(e) == typeid(stop_event))
{
do_stop(static_cast&lt;stop_event const&amp;&gt;(e));
m_state = stopped;
return;
}
if (typeid(e) == typeid(pause_event))
{
do_pause(static_cast&lt;pause_event const&amp;&gt;(e));
m_state = paused;
return;
}
}
else if (m_state == paused)
{
if (typeid(e) == typeid(stop_event))
{
do_stop(static_cast&lt;stop_event const&amp;&gt;(e));
m_state = stopped;
return;
}
if (typeid(e) == typeid(play_event))
{
do_play(static_cast&lt;play_event const&amp;&gt;(e));
m_state = playing;
return;
}
}
else
{
throw logic_error(
boost::format("unknown state: %d")
% static_cast&lt;int&gt;(m_state)
);
}
throw std::logic_error(
"unexpected event: " + typeid(e).name()
);
}
</pre>
<p>Although there is nothing particularly wrong with implementing an FSM's structure using nested <tt>if</tt> (or <tt>switch-case</tt>) statements, the obvious weakness of this approach is that most of the above code is boilerplate. What one tends to do with boilerplate code is to copy and paste it, then change names etc. to adjust it to its new location; and that's where the errors are most likely to creep in. Since all the lines of event processing look alike (structurally), it's very easy to overlook or forget something that needs to be changed, and many such errors won't appear until the runtime.</p>
<p>The transition table of our FSM is just five lines long; ideally, we would like the skeleton implementation of the automata's controlling logic to be equally short (or, at least, to look equally short, i.e. to be encapsulated in some form so we never worry about it).</p>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="example.impl"></a>5.1. Implementation</h3>
</div>
</div>
<p>To represent the STT in a C++ program, we define a <tt>transition</tt> class template that represents a single line of the table. Then the table itself can be represented as a sequence of such lines:</p>
<pre class="programlisting">
typedef mpl::list&lt;
transition&lt;stopped, play_event, playing, &amp;player::do_play&gt;
, transition&lt;playing, stop_event, stopped, &amp;player::do_stop&gt;
, transition&lt;playing, pause_event, paused, &amp;player::do_pause&gt;
, transition&lt;paused, play_event, playing, &amp;player::do_resume&gt;
, transition&lt;paused, stop_event, stopped, &amp;player::do_stop&gt;
&gt;::type transition_table;
</pre>
<p>Now, the complete FSM will look like this:</p>
<pre class="programlisting">
class player
: state_machine&lt;player&gt;
{
private:
typedef player self_t;
// state invariants
void stopped_state_invariant();
void playing_state_invariant();
void paused_state_invariant();
// states (invariants are passed as non-type template arguments,
// and are called then the FSM enters the corresponding state)
typedef state&lt;0, &amp;self_t::stopped_state_invariant&gt; stopped;
typedef state&lt;1, &amp;self_t::playing_state_invariant&gt; playing;
typedef state&lt;2, &amp;self_t::paused_state_invariant&gt; paused;
private:
// event declarations; events are represented as types,
// and can carry a specific data for each event;
// but it's not needed for generator, so we define them later
struct play_event;
struct stop_event;
struct pause_event;
// transition functions
void do_play(play_event const&amp;);
void do_stop(stop_event const&amp;);
void do_pause(pause_event const&amp;);
void do_resume(play_event const&amp;);
// STT
friend class state_machine&lt;player&gt;;
typedef mpl::list&lt;
transition&lt;stopped, play_event, playing, &amp;player::do_play&gt;
, transition&lt;playing, stop_event, stopped, &amp;player::do_stop&gt;
, transition&lt;playing, pause_event, paused, &amp;player::do_pause&gt;
, transition&lt;paused, play_event, playing, &amp;player::do_resume&gt;
, transition&lt;paused, stop_event, stopped, &amp;player::do_stop&gt;
&gt;::type transition_table;
};
</pre>
<p>That's all - the above will generate a complete FSM implementation according to our specification. The only thing we need before using it is the definition of the event types (that were just forward declared before):</p>
<pre class="programlisting">
// event definitions
struct player::play_event
: player::event
{
};
// ...
</pre>
<p>The usage is simple as well:</p>
<pre class="programlisting">
int main()
{
// usage example
player p;
p.process_event(player::play_event());
p.process_event(player::pause_event());
p.process_event(player::play_event());
p.process_event(player::stop_event());
return 0;
}
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<h3 class="title"><a name="example.relatedwork"></a>5.2. Related work</h3>
</div>
</div>
<p>A notable prior work in the field of automation of general-purpose state machine implementation in C++ is the Robert Martin's <span class="emphasis"><em>State Machine Compiler</em></span> <span class="citation">[<a class="interlink" href="#ref.smc" title="[smc]">SMC</a>]</span>. The SMC takes an ASCII description of the machine's state transition table and produces C++ code that implements the FSM using a variation of State design pattern <span class="citation">[<a class="interlink" href="#ref.hun91" title="[hun91]">Hun91</a>]</span>, <span class="citation">[<a class="interlink" href="#ref.ghj95" title="[ghj+95]">GHJ+95</a>]</span>. Lafreniere <span class="citation">[<a class="interlink" href="#ref.laf00" title="[laf00]">Laf00</a>]</span> presents another approach, where no external tools are used, and the FSMs are table driven.</p>
</div>
</div>
<div class="section">
<div class="titlepage">
<div>
<h2 class="title" style="clear: both"><a name="acknowl"></a>6. Acknowledgements</h2>
</div>
</div>
<p>Peter Dimov contributed the <tt>bind</tt> functionality without which compile-time lambda expressions wouldn't have been possible. The MPL implementation would have been much more difficult without Vesa Karvonen's wonderful Boost Preprocessor Metaprogramming Library. Authors are also greatly indebted to David B. Held who kindly volunteered to thoroughly edit this document. Of course, any remaining errors are exclusively ours.</p>
</div>
<div class="bibliography">
<div class="titlepage">
<div>
<h1 class="title"><a name="refs"></a>References</h1>
</div>
</div>
<div class="biblioentry"><a name="ref.abr01"></a>
<p>[Abr01] <span class="authorgroup">David Abrahams and Carlos Pinto Coelho,</span> <span class="title"><i><a href="http://users.rcn.com/abrahams/instantiation_speed/index.html" target="_top">Effects of Metaprogramming Style on Compilation Time</a></i>,</span> <span class="date">2001</span></p>
</div>
<div class="biblioentry"><a name="ref.ale01"></a>
<p>[Ale01] <span class="author">Andrei Alexandrescu,</span> <span class="title"><i>Modern C++ Design: Generic Programming and Design Patterns Applied</i>,</span> <span class="publishername">Addison-Wesley,</span> <span class="isbn">ISBN 0-201-70431-5,</span> <span class="date">2001</span></p>
</div>
<div class="biblioentry"><a name="ref.ce98"></a>
<p>[CE98] <span class="authorgroup">Krzysztof Czarnecki and Ulrich Eisenecker,</span> <span class="title"><i>Metalisp</i>,</span> <span class="bibliomisc"><a href="http://home.t-online.de/home/Ulrich.Eisenecker/meta.htm" target="_top">http://home.t-online.de/home/Ulrich.Eisenecker/meta.htm</a></span></p>
</div>
<div class="biblioentry"><a name="ref.ce00"></a>
<p>[CE00] <span class="authorgroup">Krzysztof Czarnecki and Ulrich Eisenecker,</span> <span class="title"><i>Generative Programming: Methods, Tools, and Applications</i>,</span> <span class="publishername">Addison-Wesley,</span> <span class="isbn">ISBN 0-201-30977-7,</span> <span class="date">2000</span></p>
</div>
<div class="biblioentry"><a name="ref.ebnf"></a>
<p>[EBNF] <span class="title"><i>ISO/IEC 14977:1996(E), Information technology &mdash; Syntactic metalanguage &mdash; Extended BNF</i>,</span> <span class="orgname">ISO/IEC,</span> <span class="date">1996</span></p>
</div>
<div class="biblioentry"><a name="ref.ghj95"></a>
<p>[GHJ+95] <span class="authorgroup">Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides,</span> <span class="title"><i>Design Patterns, Elements of Reusable Object-Oriented Software</i>,</span> <span class="publishername">Addison-Wesley,</span> <span class="isbn">ISBN 0-201-63361-2,</span> <span class="date">1995</span></p>
</div>
<div class="biblioentry"><a name="ref.hu79"></a>
<p>[HU79] <span class="authorgroup">Hopcroft and Ullman,</span> <span class="title"><i>Introduction to automata theory, languages and computations</i>,</span> <span class="publishername">Addison-Wesley,</span> <span class="date">1979</span></p>
</div>
<div class="biblioentry"><a name="ref.hud89"></a>
<p>[Hud89] <span class="author">Paul Hudak,</span> <span class="title"><i>Conception, Evolution, and Application of Functional Programming Languages</i>,</span> <span class="biblioset"><i>ACM Computing Surveys</i>, <span class="publishername">Association for Computing Machinery (ACM),</span> <span class="issn">ISSN 0360-0300,</span></span> <span class="volumenum">Vol. 21,</span> <span class="issuenum">No. 3,</span> <span class="pagenums">pp. 359-411,</span> <span class="date">September, 1989</span></p>
</div>
<div class="biblioentry"><a name="ref.hun91"></a>
<p>[Hun91] <span class="author">Immo Huneke,</span> <span class="title"><i>Finite State Machines: A Model of Behavior for C++</i>,</span> <span class="biblioset"><i>C++ Report</i>, <span class="publishername">SIGS Publications Inc.,</span> <span class="issn">ISSN 1040-6042,</span></span> <span class="date">1991</span></p>
</div>
<div class="biblioentry"><a name="ref.iso98"></a>
<p>[ISO98] <span class="title"><i>ISO/IEC 14882:1998(E), Programming languages &mdash; C++</i>,</span> <span class="orgname">ISO/IEC,</span> <span class="date">1998</span></p>
</div>
<div class="biblioentry"><a name="ref.joh79"></a>
<p>[Joh79] <span class="author">Stephen C. Johnson,</span> <span class="title"><i><a href="http://dinosaur.compilertools.net/yacc/index.html" target="_top">Yacc: Yet Another Compiler Compiler</a></i>,</span> <span class="publishername">UNIX Programmer's Manual,</span> <span class="volumenum">Vol. 2b,</span> <span class="pagenums">pp. 353-387,</span> <span class="date">1979</span></p>
</div>
<div class="biblioentry"><a name="ref.laf00"></a>
<p>[Laf00] <span class="author">David Lafreniere,</span> <span class="title"><i><a href="http://www.cuj.com/articles/2000/0005/0005f/0005f.htm?topic=articles" target="_top">State Machine Design in C++</a></i>,</span> <span class="biblioset"><i>C/C++ User Journal</i>, <span class="publishername">CMP Media LCC,</span> <span class="issn">ISSN 1075-2838,</span></span> <span class="volumenum">Vol. 18,</span> <span class="issuenum">No. 5,</span> <span class="date">May 1998</span></p>
</div>
<div class="biblioentry"><a name="ref.loki"></a>
<p>[Loki] <span class="title"><i>The Loki library</i>,</span> <span class="bibliomisc"><a href="http://sourceforge.net/projects/loki-lib/" target="_top">http://sourceforge.net/projects/loki-lib/</a></span></p>
</div>
<div class="biblioentry"><a name="ref.mar98"></a>
<p>[Mar98] <span class="author">Robert C. Martin,</span> <span class="title"><i><a href="http://www.objectmentor.com/resources/articles/umlfsm.pdf" target="_top">UML Tutorial: Finite State Machines</a></i>,</span> <span class="biblioset"><i>C++ Report</i>, <span class="publishername">SIGS Publications Inc.,</span> <span class="issn">ISSN 1040-6042,</span></span> <span class="date">June 1998</span></p>
</div>
<div class="biblioentry"><a name="ref.mplr"></a>
<p>[MPLR] <span class="title"><i>Boost MPL Library Reference Documentation</i>,</span> <span class="bibliomisc"><a href="http://www.mywikinet.com/mpl/ref/Table_of_Content.html" target="_top">http://www.mywikinet.com/mpl/ref/Table_of_Content.html</a></span></p>
</div>
<div class="biblioentry"><a name="ref.pre"></a>
<p>[PRE] <span class="author">Vesa Karvonen,</span> <span class="title"><i>Boost Preprocessor Metaprogramming library</i>,</span> <span class="bibliomisc"><a href="http://www.boost.org/libs/preprocessor/doc/" target="_top">http://www.boost.org/libs/preprocessor/doc/</a></span></p>
</div>
<div class="biblioentry"><a name="ref.smc"></a>
<p>[SMC] <span class="author">Robert C. Martin,</span> <span class="title"><i>SMC - Finite State Machine Compiler (C++)</i>,</span> <span class="bibliomisc"><a href="http://www.objectmentor.com/resources/downloads/index" target="_top">http://www.objectmentor.com/resources/downloads/index</a></span></p>
</div>
<div class="biblioentry"><a name="ref.stl94"></a>
<p>[STL94] <span class="authorgroup">A. A. Stepanov and M. Lee,</span> <span class="title"><i>The Standard Template Library</i>,</span> <span class="orgname">Hewlett-Packard Laboratories,</span> <span class="date">1994</span></p>
</div>
<div class="biblioentry"><a name="ref.spl"></a>
<p>[SPL] <span class="title"><i>Boost Smart Pointer library</i>,</span> <span class="bibliomisc"><a href="http://www.boost.org/libs/smart_ptr/" target="_top">http://www.boost.org/libs/smart_ptr/</a></span></p>
</div>
<div class="biblioentry"><a name="ref.ss75"></a>
<p>[SS75] <span class="authorgroup">Gerald&nbsp;J. Sussman and Guy&nbsp;L. Steele Jr.,</span> <span class="title"><i>Scheme: An interpreter for extended lambda calculus</i>,</span> <span class="biblioset"><i>MIT AI Memo 349</i>, <span class="publishername">Massachusetts Institute of Technology,</span></span> <span class="date">May 1975</span></p>
</div>
<div class="biblioentry"><a name="ref.ttl"></a>
<p>[TTL] <span class="title"><i>Boost Type Traits library</i>,</span> <span class="bibliomisc"><a href="http://www.boost.org/libs/type_traits/" target="_top">http://www.boost.org/libs/type_traits/</a></span></p>
</div>
<div class="biblioentry"><a name="ref.vel95a"></a>
<p>[Vel95a] <span class="author">Todd Veldhuizen,</span> <span class="title"><i><a href="http://osl.iu.edu/~tveldhui/papers/Template-Metaprograms/meta-art.html" target="_top">Using C++ template metaprograms</a></i>,</span> <span class="biblioset"><i>C++ Report</i>, <span class="publishername">SIGS Publications Inc.,</span> <span class="issn">ISSN 1040-6042,</span></span> <span class="volumenum">Vol. 7,</span> <span class="issuenum">No. 4,</span> <span class="pagenums">pp. 36-43,</span> <span class="date">May 1995</span></p>
</div>
<div class="biblioentry"><a name="ref.vel95b"></a>
<p>[Vel95b] <span class="author">Todd Veldhuizen,</span> <span class="title"><i><a href="http://osl.iu.edu/~tveldhui/papers/Expression-Templates/exprtmpl.html" target="_top">Expression templates</a></i>,</span> <span class="biblioset"><i>C++ Report</i>, <span class="publishername">SIGS Publications Inc.,</span> <span class="issn">ISSN 1040-6042,</span></span> <span class="volumenum">Vol. 7,</span> <span class="issuenum">No. 5,</span> <span class="pagenums">pp. 26-31,</span> <span class="date">Jun 1995</span></p>
</div>
<div class="biblioentry"><a name="ref.unr"></a>
<p>[Unr] <span class="author">Erwin Unruh,</span> <span class="title"><i>Prime number computation</i>,</span> <span class="publishername">ANSI X3J16-94-0075/ISO WG21-462</span></p>
</div>
</div>
<div class="footnotes"><br>
<hr width="100" align="left">
<div class="footnote">
<p><sup><a name="ftn.note.pointedtype" href="#note.pointedtype">1</a></sup> Although it would be easy to implement <tt>pointed_type</tt> using partial specialization to distinguish the case where <tt>T</tt> is a pointer, <tt>if_</tt> is likely to be the right tool for dealing with more complex conditionals. For the purposes of exposition, please suspend disbelief!</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.higherorder" href="#note.higherorder">2</a></sup> In fact it's already broken: <tt>apply_twice</tt> doesn't even fit the metafunction concept since it requires a template (rather than a type) as its first parameter, which breaks the metafunction protocol.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.seqconcepts" href="#note.seqconcepts">3</a></sup> A more precise definition of these concepts can be found in the library reference documentation <span class="citation">[<a class="interlink" href="#ref.mplr" title="[mplr]">MPLR</a>]</span>.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.tinylist" href="#note.tinylist">4</a></sup> Random access is almost as important at compile-time as it is at run-time. For example, searching for an item in a sorted random-access sequence using <tt>lower_bound</tt> can be much faster than performing the same operation on a forward-access-only <tt>list</tt>.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.maxelement" href="#note.maxelement">5</a></sup> Here is another, even more elegant implementation:</p>
<pre class="programlisting">
template&lt; typename Sequence &gt;
struct largest
{
typedef typename mpl::max_element&lt;
mpl::transform_view&lt;
Sequence
, mpl::size_of&lt;_&gt;
&gt;
&gt;::type type;
};
</pre>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.nontype" href="#note.nontype">6</a></sup> Ideally, if going this route, all the templates should be re-implemented for every integral type - <tt>char</tt>, <tt>int</tt>, <tt>short</tt>, <tt>long</tt>, etc.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.valuewrapping" href="#note.valuewrapping">7</a></sup> The same technique was suggested by Czarnecki and Eisenecker in <span class="citation">[<a class="interlink" href="#ref.ce00" title="[ce00]">CE00</a>]</span>.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.unrolling1" href="#note.unrolling1">8</a></sup> It could be much more, depending on the complexity of the <tt>apply&lt;...&gt;</tt> expression, whose depth is added to the overall recursion depth.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.unrolling2" href="#note.unrolling2">9</a></sup> This implementation detail is made relatively painless through heavy reliance on the Boost Preprocessor Library, so only one copy of the code needs to be maintained.</p>
</div>
<div class="footnote">
<p><sup><a name="ftn.note.fsm" href="#note.fsm">10</a></sup> The events need to be passed to action functions, as they may contain some event-specific information for an action.</p>
</div>
</div>
</div>
</body>
</html>