Boost.Preprocessor

Tutorial

Motivation
Preprocessor Metaprogramming Techniques Use a Local Macro to avoid small scale repetition Use BOOST_PP_EMPTY as an unused parameter in Local Macro instantiations Use BOOST_PP_CAT instead of ## when necessary Use BOOST_PP_STRINGIZE instead of # whenever necessary Avoid O(N) repetition on lists in general Use a Conditional Define to enable user configuration of code repetition Use Token Look-Up Function to eliminate categorical repetition Use BOOST_PP_REPEAT to avoid O(N*N) repetition Use BOOST_PP_IF to implement special case for the first element Use arithmetic, logical and comparison operations when necessary

Motivation

The C++ function and template parameter lists are special syntactic constructs and it is impossible to directly manipulate or generate them using C++ constructs. This leads to unnecessary code repetition.

Consider the implementation of the is_function<> metafunction in Boost. The implementation uses an overloaded is_function_tester() function that is used for testing if a type is convertible to pointer to a function. Because of the special treatment of parameter lists, it is not possible to directly match a function with an arbitrary parameter list. Instead, the is_function_tester() must be overloaded for every distinct number of parameters that is to be supported. Example:

template <class R>
yes_type is_function_tester(R (*)());
template <class R, class A0>
yes_type is_function_tester(R (*)(A0));
template <class R, class A0, A1>
yes_type is_function_tester(R (*)(A0, A1));
template <class R, class A0, A1, A2>
yes_type is_function_tester(R (*)(A0, A1, A2));

// ...

The need for this kind of repetition occurs particularly frequently while implementing generic components or metaprogramming facilities, but the need also manifests itself in many far simpler situations.

Typical solutions

Typically the repetition is done manually. Manual code repetition is highly unproductive, but sometimes more readable to the untrained eye.

Another solution is to write an external program for generating the repeated code or use some other extra linquistic means such as a smart editor. Unfortunately, using external code generators has many disadvantages:

writing the generator takes time (this could be helped by using a standard generator)
it is no longer productive to manipulate C++ code directly
invoking the generator may be difficult
automating the invocation of the generator can be difficult in certain environments (automatic invocation is desirable for active libraries)
porting and distributing the generator may be difficult or simply takes precious time

What about the preprocessor?

Because C++ comes with a preprocessor, one would assume that it would support these kind of needs directly. Using the preprocessor in this case is highly desirable because:

preprocessor is highly portable
preprocessor is automatically invoked as part of the compiling process
preprocessor metacode can be directly embedded into the C++ source code
Compilers generally allow to view or output the preprocessed code, which can be used for debugging or to copy-paste the generated code.

Most unfortunately, the preprocessor is a very low level preprocessor that specifically does not support repetition or recursive macros. Library support is needed!

For detailed information on the capabilities and limitations of the preprocessor, please refer to the C++ standard [Std].

The motivation example revisited

Using the primitives of the PREPROCESSOR library, the is_function_tester()s could be implemented like this:

#define IS_FUNCTION_TESTER(N,_)\
  template<class R BOOST_PP_COMMA_IF(N) BOOST_PP_ENUM_PARAMS(N, class A)>\
  yes_type is_function_tester(R (*)(BOOST_PP_ENUM_PARAMS(N,A)));

BOOST_PP_REPEAT(BOOST_PP_INC(MAX_IS_FUNCTION_TESTER_PARAMS),IS_FUNCTION_TESTER,_)
#undef IS_FUNCTION_TESTER

In order to change the maximum number of function parameters supported, you now simply change the MAX_IS_FUNCTION_TESTER_PARAMS definition and recompile.

Preprocessor Metaprogramming Techniques

The preprocessor metaprogramming techniques are presented in example format.

EXAMPLE: Use a Local Macro to avoid small scale repetition

#define BOOST_PP_DEF(OP)   \
  template<class T, int n> \
  vec<T,n>&                \
    operator OP##=         \
    ( vec<T,n>&            \
        lhs                \
    , const vec<T,n>&      \
        rhs                \
    )                      \
  { for (int i=0; i<n; ++i)\
      lhs(i) OP##= rhs(i); \
    return lhs;            \
  }

BOOST_PP_DEF(+)
BOOST_PP_DEF(-)
BOOST_PP_DEF(*)
BOOST_PP_DEF(/)
#undef BOOST_PP_DEF

TIP: It is usually okay to use a standard macro name like BOOST_PP_DEF for this kind of code, because the macro is both defined and undefined in the immediate site of its use.

TIP: It is easier to verify proper use of the line continuation operator when they are aligned.

NOTES: You can extend this example by defining more and different kinds of operators. Before doing so, consider using the Algebraic Categories technique introduced in [Barton] or a Layered Architecture (see for instance [Czarnecki]). However, at some point you must type the operator tokens *, /, +, -, ..., because it is impossible to generate them using templates. The resulting Categorical Repetition of tokens can be eliminated by using preprocessor metaprogramming.

EXAMPLE: Use BOOST_PP_EMPTY as an unused parameter in Local Macro instantiations

#define BOOST_PP_DEF(CV)  \
  template<class base>    \
  CV() typename implement_subscript_using_begin_subscript<base>::value_type&\
    implement_subscript_using_begin_subscript<base>::operator[]\
    ( index_type          \
        i                 \
    ) CV()                \
{ return base::begin()[i];\
}

BOOST_PP_DEF(BOOST_PP_EMPTY)
BOOST_PP_DEF(const BOOST_PP_EMPTY)
#undef BOOST_PP_DEF

HOW: BOOST_PP_EMPTY() expands to nothing and can be used as an unused parameter.

NOTE: BOOST_PP_EMPTY without the () never gets expanded. The () is necessary to invoke a function-like macro.

CAVEAT: You can not safely use concatenation while using BOOST_PP_EMPTY().

TIP: Occasionally one or two lines are considerably longer than the rest. It can often save some work to not align all of the line continuation operators without making the code too unreadable.

TIP: Use syntax highlighting on preprocessor metaprogramming macro and parameter identifiers such as

BOOST_PP_DEF,
BOOST_PP_EMPTY,
BOOST_PP_REPEAT,
OP,
CV,
...

It can greatly improve readability.

EXAMPLE: Use BOOST_PP_CAT instead of ## when necessary

#define STATIC_ASSERT(EXPR)\
  enum\
  { BOOST_PP_CAT(static_check_,__LINE__) = (EXPR) ? 1 : -1\
  };\
  typedef char\
    BOOST_PP_CAT(static_assert_,__LINE__)\
    [ BOOST_PP_CAT(static_check_,__LINE__)\
    ]

// ...

STATIC_ASSERT(sizeof(int) <= sizeof(long));

WHY: Macro expansion proceeds recursively in "layers". Token pasting prevents the preprocessor from performing macro expansion, therefore it is often necessary to delay token concatenation.

EXAMPLE: Use BOOST_PP_STRINGIZE instead of # whenever necessary

#define NOTE(STR)\
  message(__FILE__ "(" BOOST_PP_STRINGIZE(__LINE__) ") : " STR)

// ...

#pragma NOTE("TBD!")

WHY: Macro expansion proceeds recursively in "layers". Stringization prevents the preprocessor from performing macro expansion, therefore it is often necessary to delay stringization.

EXAMPLE: Use:

BOOST_PP_ENUM_PARAMS,
BOOST_PP_ENUM_PARAMS_WITH_A_DEFAULT,
BOOST_PP_ENUM_PARAMS_WITH_DEFAULTS,
BOOST_PP_ENUM_SHIFTED_PARAMS, or
BOOST_PP_REPEAT, and
BOOST_PP_COMMA_IF

to avoid O(N) repetition on lists in general

struct make_type_list_end;

template
< BOOST_PP_ENUM_PARAMS_WITH_A_DEFAULT
  ( MAKE_TYPE_LIST_MAX_LENGTH
  , class T
  , make_type_list_end
  )
>
struct make_type_list
{
private:
  enum
  { end = is_same<T0,make_type_list_end>::value
  };

public:
  typedef typename
    type_if
    < end
    , type_cons_empty
    , type_cons
      < T0
      , typename
        type_inner_if
        < end
        , type_identity<end>
        , make_type_list
          < BOOST_PP_ENUM_SHIFTED_PARAMS
            ( MAKE_TYPE_LIST_MAX_LENGTH
            , T
            )
          >
        >::type
      >
    >::type type;
};

HOW: BOOST_PP_REPEAT uses simulated recursion (pseudo code):

#define BOOST_PP_REPEAT(N,M,P) BOOST_PP_REPEAT##N(M,P)
#define BOOST_PP_REPEAT0(M,P)
#define BOOST_PP_REPEAT1(M,P) M(0,P)
#define BOOST_PP_REPEAT2(M,P) M(0,P) M(1,P)
#define BOOST_PP_REPEAT3(M,P) BOOST_PP_REPEAT2(M,P) M(2,P)
#define BOOST_PP_REPEAT4(M,P) BOOST_PP_REPEAT3(M,P) M(3,P)
// ...

BOOST_PP_ENUM_PARAMS variations use BOOST_PP_REPEAT

BOOST_PP_COMMA_IF(I) expands to a comma if I != 0.

BOOST_PP_INC(I) expands essentially to "I+1" and BOOST_PP_DEC(I) expands essentially to "I-1".

EXAMPLE: Use a Conditional Define to enable user configuration of code repetition based on need rather than some "reasonable" upper limit

#ifndef MAKE_TYPE_LIST_MAX_LENGTH
#define MAKE_TYPE_LIST_MAX_LENGTH 8
#endif

Now the user can configure the make_type_list primitive without modifying library code.

EXAMPLE: Use BOOST_PP_REPEAT and a Token Look-Up Function to eliminate categorical repetition

// CAVEAT: My compiler is not standard on arithmetic types.
#define ARITHMETIC_TYPE(I)  ARITHMETIC_TYPE##I
#define ARITHMETIC_TYPE0    bool
#define ARITHMETIC_TYPE1    char
#define ARITHMETIC_TYPE2    signed char
#define ARITHMETIC_TYPE3    unsigned char
#define ARITHMETIC_TYPE4    short
#define ARITHMETIC_TYPE5    unsigned short
#define ARITHMETIC_TYPE6    int
#define ARITHMETIC_TYPE7    unsigned int
#define ARITHMETIC_TYPE8    long
#define ARITHMETIC_TYPE9    unsigned long
#define ARITHMETIC_TYPE10   float
#define ARITHMETIC_TYPE11   double
#define ARITHMETIC_TYPE12   long double
#define ARITHMETIC_TYPE_CNT 13

//  ...

#define BOOST_PP_DEF(I,_)\
catch (ARITHMETIC_TYPE(I) t)\
{ report_typeid(t);\
  report_value(t);\
}
BOOST_PP_REPEAT
( ARITHMETIC_TYPE_CNT
, BOOST_PP_DEF
, _
)
#undef BOOST_PP_DEF

// ...

NOTE: The repetition of the above example can be eliminated using template metaprogramming [Czarnecki] as well. However categorical repetition of operator tokens can not be completely eliminated by using template metaprogramming.

EXAMPLE: Use BOOST_PP_REPEAT to avoid O(N*N) repetition

#ifndef MAX_VEC_ARG_CNT
#define MAX_VEC_ARG_CNT 8
#endif

//  ...

#define ARG_FUN(I,_) BOOST_PP_COMMA_IF(I) T a##I
#define ASSIGN_FUN(I,_) (*this)[I] = a##I;

#define DEF_VEC_CTOR_FUN(I,_)\
vec( BOOST_PP_REPEAT(I,ARG_FUN,_) )\
{ BOOST_PP_REPEAT(I,ASSIGN_FUN,_)\
}

BOOST_PP_REPEAT
( BOOST_PP_INC(MAX_VEC_ARG_CNT)
, DEF_VEC_CTOR_FUN
, _
)

#undef ARG_FUN
#undef ASSIGN_FUN
#undef DEF_VEC_CTOR_FUN

// ...

HOW: BOOST_PP_REPEAT is implemented in a special way to enable automatic recursion.

EXAMPLE: Use BOOST_PP_IF to implement special case for the first element

#define BOOST_PP_COMMA_IF(C)\
  BOOST_PP_IF(C,BOOST_PP_COMMA,BOOST_PP_EMPTY)()

BOOST_PP_IF(0,true,false) == false;
BOOST_PP_IF(1,true,false) == true;

BOOST_PP_IF enables convenient generation of lists using BOOST_PP_REPEAT.

NOTE: THEN and ELSE don't have to be macros. However, if at least one of them is a function-like macro and you want it to be expanded conditionally, you have to make the other parameter a function-like macro, too. This can often be done using BOOST_PP_IDENTITY. Consider the following example (by Aleksey Gurtovoy):

#define NUMBERED_EXPRESSION(I,X)\
  BOOST_PP_IF                   \
  ( I                           \
  , BOOST_PP_IDENTITY(X##I)     \
  , BOOST_PP_EMPTY              \
  )()

NOTE: Like in the above implementation of COMMA_IF, the result of IF is often invoked and not the THEN and ELSE parameters. If the parameters were invoked, the code would not expand correctly, because the EMPTY parameter would get expanded to nothing before the IF would be properly expanded.

HOW: BOOST_PP_IF is defined for the entire repeat range (pseudo code):

#define BOOST_PP_IF(C,THEN,ELSE) BOOST_PP_IF##C(THEN,ELSE)
#define BOOST_PP_IF0(THEN,ELSE) ELSE
#define BOOST_PP_IF1(THEN,ELSE) THEN
#define BOOST_PP_IF2(THEN,ELSE) THEN
// ...

EXAMPLE: Use arithmetic, logical and comparison operations when necessary

The PREPROCESSOR library supports saturated arithmetic, logical and comparison operations on decimal integer literals in the range [0,BOOST_PP_LIMIT_MAG].

Suppose that you want to generate a numbered lists with a special element inserted at a desired position. For example: E0, E1, S, E2. Consider the following example:

#define SPECIAL_NUMBERED_LIST(N,I,ELEM,SPECIAL)\
  BOOST_PP_ASSERT_MSG(BOOST_PP_LESS(I,N),BAD PARAMS FOR SPECIAL_NUMBERED_LIST!)\
  BOOST_PP_ENUM_PARAMS(I,ELEM)\
  BOOST_PP_COMMA_IF(I) SPECIAL\
  BOOST_PP_REPEAT(BOOST_PP_SUB(\
    BOOST_PP_DEC(N),I),SPECIAL_NUMBERED_LIST_HELPER,(ELEM,I))
#define SPECIAL_NUMBERED_LIST_HELPER(I,ELEM_BASE)\
  ,\
  BOOST_PP_CAT\
  ( BOOST_PP_TUPLE_ELEM(2,0,ELEM_BASE)\
  , BOOST_PP_ADD\
    ( I\
    , BOOST_PP_TUPLE_ELEM(2,1,ELEM_BASE)\
    )\
  )

SPECIAL_NUMBERED_LIST(3,0,E,S)
SPECIAL_NUMBERED_LIST(3,1,E,S)
SPECIAL_NUMBERED_LIST(3,2,E,S)
SPECIAL_NUMBERED_LIST(3,3,E,S)

Revised

Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.