Reentrancy
Macro expansion in the preprocessor is entirely functional. Therefore,
there is no iteration. Unfortunately, the preprocessor also disallows
recursion. This means that the library must fake iteration or recursion
by defining sets of macros that are implemented similarly.
To illustrate, here is a simple concatenation macro:
#define CONCAT(a, b) CONCAT_D(a, b)
#define CONCAT_D(a, b) a ## b
CONCAT(a, CONCAT(b, c)) // abc
This is fine for a simple case like the above, but what happens in a scenario
like the following:
#define AB(x, y) CONCAT(x, y)
CONCAT(A, B(p, q)) // CONCAT(p, q)
Because there is no recursion, the example above expands to CONCAT(p, q)
rather than pq.
There are only two ways to "fix" the above. First, it can be documented
that AB uses CONCAT and disallow usage similar to the
above. Second, multiple concatenation macros can be provided....
#define CONCAT_1(a, b) CONCAT_1_D(a, b)
#define CONCAT_1_D(a, b) a ## b
#define CONCAT_2(a, b) CONCAT_2_D(a, b)
#define CONCAT_2_D(a, b) a ## b
#define AB(x, y) CONCAT_2(x, y)
CONCAT_1(A, B(p, q)) // pq
This solves the problem. However, it is now necessary to know that AB
uses, not only a concatenation macro, but CONCAT_2 specifically.
A better solution is to abstract which concatenation macro is used....
#define AB(c, x, y) CONCAT_ ## c(x, y)
CONCAT_1(A, B(2, p, q)) // pq
This is an example of generic reentrance, in this case, into a fictional
set of concatenation macros. The c parameter represents the
"state" of the concatenation construct, and as long as the user keeps track of
this state, AB can be used inside of a concatenation macro.
The library has the same choices. It either has to disallow a construct
being inside itself or provide multiple, equivalent definitions of a construct
and provide a uniform way to reenter that construct. There are
several contructs that require recursion (such as BOOST_PP_WHILE).
Consequently, the library chooses to provide several sets of macros with
mechanisms to reenter the set at a macro that has not already been used.
In particular, the library must provide reentrance for BOOST_PP_FOR, BOOST_PP_REPEAT,
and BOOST_PP_WHILE. There are two mechanisms that are used to
accomplish this: state parameters (like the above concatenation example)
and automatic recursion.
State Parameters
Each of the above constructs (BOOST_PP_FOR, BOOST_PP_REPEAT, and BOOST_PP_WHILE)
has an associated state. This state provides the means to reenter the
respective construct.
Several user-defined macros are passed to each of these constructs (for use as
predicates, operations, etc.). Every time a user-defined macro is
invoked, it is passed the current state of the construct that invoked it so
that the macro can reenter the respective set if necessary.
These states are used in one of two ways--either by concatenating to or passing
to another macro.
There are three types of macros that use these state parameters. First,
the set itself which is reentered through concatenation. Second,
corresponding sets that act like they are a part of the the primary set.
These are also reentered through concatenation. And third, macros that
internally use the first or second type of macro. These macros take the
state as an additional argument.
The state of BOOST_PP_WHILE is symbolized by the letter D.
Two user-defined macros are passed to BOOST_PP_WHILE--a predicate and an
operation. When BOOST_PP_WHILE expands these macros, it passes
along its state so that these macros can reenter the BOOST_PP_WHILE set.
Consider the following multiplication implementation that illustrates this
technique:
// The addition interface macro.
// The _D signifies that it reenters
// BOOST_PP_WHILE with concatenation.
#define ADD_D(d, x, y) \
BOOST_PP_TUPLE_ELEM( \
2, 0, \
BOOST_PP_WHILE_ ## d(ADD_P, ADD_O, (x, y)) \
) \
/**/
// The predicate that is passed to BOOST_PP_WHILE.
// It returns "true" until "y" becomes zero.
#define ADD_P(d, xy) BOOST_PP_TUPLE_ELEM(2, 1, xy)
// The operation that is passed to BOOST_PP_WHILE.
// It increments "x" and decrements "y" which will
// eventually cause "y" to equal zero and therefore
// cause the predicate to return "false."
#define ADD_O(d, xy) \
( \
BOOST_PP_INC( \
BOOST_PP_TUPLE_ELEM(2, 0, xy) \
), \
BOOST_PP_DEC( \
BOOST_PP_TUPLE_ELEM(2, 1, xy) \
) \
) \
/**/
// The multiplication interface macro.
#define MUL(x, y) \
BOOST_PP_TUPLE_ELEM( \
3, 0, \
BOOST_PP_WHILE(MUL_P, MUL_O, (0, x, y)) \
) \
/**/
// The predicate that is passed to BOOST_PP_WHILE.
// It returns "true" until "y" becomes zero.
#define MUL_P(d, rxy) BOOST_PP_TUPLE_ELEM(3, 2, rxy)
// The operation that is passed to BOOST_PP_WHILE.
// It adds "x" to "r" and decrements "y" which will
// eventually cause "y" to equal zero and therefore
// cause the predicate to return "false."
#define MUL_O(d, rxy) \
( \
ADD_D( \
d, /* pass the state on to ADD_D */ \
BOOST_PP_TUPLE_ELEM(3, 0, rxy), \
BOOST_PP_TUPLE_ELEM(3, 1, rxy) \
), \
BOOST_PP_TUPLE_ELEM(3, 1, rxy), \
BOOST_PP_DEC( \
BOOST_PP_TUPLE_ELEM(3, 2, rxy) \
) \
) \
/**/
MUL(3, 2) // expands to 6
There are a couple things to note in the above implementation. First,
note how ADD_D reenters BOOST_PP_WHILE using the d state
parameter. Second, note how MUL's operation, which is
expanded by BOOST_PP_WHILE, passes the state on to ADD_D.
This illustrates state reentrance by both argument and concatenation.
For every macro in the library that uses BOOST_PP_WHILE, there is a
state reentrant variant. If that variant uses an argument rather than
concatenation, it is suffixed by _D to symbolize its method of
reentrance. Examples or this include the library's own BOOST_PP_ADD_D
and BOOST_PP_MUL_D. If the variant uses concatenation, it is
suffixed by an underscore. It is completed by concatenation of the
state. This includes BOOST_PP_WHILE itself with BOOST_PP_WHILE_
## d and, for example, BOOST_PP_LIST_FOLD_LEFT with BOOST_PP_LIST_FOLD_LEFT_
## d.
The same set of conventions are used for BOOST_PP_FOR and BOOST_PP_REPEAT,
but with the letters R and Z, respectively, to symbolize their
states.
Also note that the above MUL implementation, though not
immediately obvious, is using all three types of reentrance. Not
only is it using both types of state reentrance, it is also using automatic
recursion....
Automatic Recursion
Automatic recursion is a technique that vastly simplifies the use of reentrant
constructs. It is used by simply not using any state parameters at
all.
The MUL example above uses automatic recursion when it uses BOOST_PP_WHILE
by itself. In other words, MUL can still be used
inside BOOST_PP_WHILE even though it doesn't reenter BOOST_PP_WHILE
by concatenating the state to BOOST_PP_WHILE_.
To accomplish this, the library uses a "trick." Despite what it looks
like, the macro BOOST_PP_WHILE does not take three arguments. In
fact, it takes no arguments at all. Instead, the BOOST_PP_WHILE macro
expands to a macro that takes three arguments. It simply detects
what the next available BOOST_PP_WHILE_ ## d macro is and returns
it. This detection process is somewhat involved, so I won't go into how
it works here, but suffice to say it does work.
Using automatic recursion to reenter various sets of macros is obviously much
simpler. It completely hides the underlying implementation details.
So, if it is so much easier to use, why do the state parameters still
exist? The reason is simple as well. When state parameters are
used, the state is known at all times. This is not the case when
automatic recursion is used. The automatic recursion mechanism has to deduce
the state at each point that it is used. This implies a cost in macro
complexity that in some situations--notably at deep macro depths--will slow
some preprocessors to a crawl.
Conclusion
It is really a tradeoff whether to use state parameters or automatic recursion
for reentrancy. The strengths of automatic recursion are ease of use and
implementation encapsulation. These come at a performance cost on some
preprocessors in some situations. The primary strength of state
parameters, on the other hand, is efficiency. Use of the state parameters
is the only way to achieve maximum efficiency. This efficiency
comes at the cost of both code complexity and exposition of implementation.
See Also
- Paul Mensonides