Boost C++ Libraries

PrevUpHomeNext

Extending library settings support

Adding support for user-defined types to the formatter parser
Adding support for user-defined types to the filter parser
Adding support for user-defined sinks

If you write your own logging sinks or use your own types in attributes, you may want to add support for these components to the settings parser provided by the library. Without doing this, the library will not be aware of your types and thus will not be able to use them when parsing settings.

#include <boost/log/utility/setup/formatter_parser.hpp>

In order to add support for user-defined types to the formatter parser, one has to register a formatter factory. The factory is basically an object that derives from formatter_factory interface. The factory mainly implements the single create_formatter method which, when called, will construct a formatter for the particular attribute value.

When the user-defined type supports putting to a stream with operator<< and this operator behavior is suitable for logging, one can use a simple generic formatter factory provided by the library out of the box. For example, let's assume we have the following user-defined type that we want to use as an attribute value:

struct point
{
    float m_x, m_y;

    point() : m_x(0.0f), m_y(0.0f) {}
    point(float x, float y) : m_x(x), m_y(y) {}
};

template< typename CharT, typename TraitsT >
std::basic_ostream< CharT, TraitsT >& operator<< (std::basic_ostream< CharT, TraitsT >& strm, point const& p)
{
    strm << "(" << p.m_x << ", " << p.m_y << ")";
    return strm;
}

Then, in order to register this type with the simple formatter factory, a single call to register_simple_formatter_factory will suffice:

void init_factories()
{
    logging::register_simple_formatter_factory< point, char >("Coordinates");
}

[Note] Note

The operator<< for the attribute value stored type must be visible from the point of this call.

The function takes the stored attribute value type (point, in our case) and the target character type used by formatters as template parameters. From the point of this call, whenever the formatter parser encounters a reference to the "Coordinates" attribute in the format string, it will invoke the formatter factory, which will construct the formatter that calls our operator<< for class point.

[Tip] Tip

It is typically a good idea to register all formatter factories at an early stage of the application initialization, before any other library initialization, such as reading config files.

From the formatter parser description it is known that the parser supports passing additional parameters from the format string to the formatter factory. We can use these parameters to customize the output generated by the formatter.

For example, let's implement customizable formatting of our point objects, so that the following format string works as expected:

%TimeStamp% %Coordinates(format="{%0.3f; %0.3f}")% %Message%

The simple formatter factory ignores all additional parameters from the format string, so we have to implement our own factory instead. Custom factories are registered with the register_formatter_factory function, which is similar to register_simple_formatter_factory but accepts a pointer to the factory instead of the explicit template parameters.

// Custom point formatter
class point_formatter
{
public:
    typedef void result_type;

public:
    explicit point_formatter(std::string const& fmt) : m_format(fmt)
    {
    }

    void operator() (logging::formatting_ostream& strm, logging::value_ref< point > const& value) const
    {
        if (value)
        {
            point const& p = value.get();
            m_format % p.m_x % p.m_y;
            strm << m_format;
            m_format.clear();
        }
    }

private:
    mutable boost::format m_format;
};

// Custom point formatter factory
class point_formatter_factory :
    public logging::basic_formatter_factory< char, point >
{
public:
    formatter_type create_formatter(logging::attribute_name const& name, args_map const& args)
    {
        args_map::const_iterator it = args.find("format");
        if (it != args.end())
            return boost::phoenix::bind(point_formatter(it->second), expr::stream, expr::attr< point >(name));
        else
            return expr::stream << expr::attr< point >(name);
    }
};

void init_factories()
{
    logging::register_formatter_factory("Coordinates", boost::make_shared< point_formatter_factory >());
}

Let's walk through this code sample. Our point_formatter_factory class derives from the basic_formatter_factory base class provided by the library. This class derives from the base formatter_factory interface and defines a few useful types, such as formatter_type and args_map that we use. The only thing left to do in our factory is to define the create_formatter method. The method analyzes the parameters from the format string which are passed as the args argument, which is basically std::map of string keys (parameter names) to string values (the parameter values). We seek for the format parameter and expect it to contain a Boost.Format-compatible format string for our point objects. If the parameter is found we create a formatter that invokes point_formatter for the attribute values. Otherwise we create a default formatter that simply uses the operator<<, like the simple formatter factory does. Note that we use the name argument of create_formatter to identify the attribute so that the same factory can be used for different attributes.

The point_formatter is our custom formatter based on Boost.Format. With help of Boost.Phoenix and expression placeholders we can construct a formatter that will extract the attribute value and pass it along with the target stream to the point_formatter function object. Note that the formatter accepts the attribute value wrapped into the value_ref wrapper which can be empty if the value is not present.

Lastly, the call to register_formatter_factory creates the factory and adds it to the library.

You can find the complete code of this example here.

#include <boost/log/utility/setup/filter_parser.hpp>

You can extend filter parser in the similar way you can extend the formatter parser - by registering filter factories for your attribute values into the library. However, since it takes a considerably more complex syntax to describe filters, a filter factory typically implements several generator functions.

Like with formatter parser extension, you can avoid spelling out the filter factory and register a simple factory provided by the library:

void init_factories()
{
    logging::register_simple_filter_factory< point, char >("Coordinates");
}

In order this to work the user's type should fulfill these requirements:

  1. Support reading from an input stream with operator>>.
  2. Support the complete set of comparison and ordering operators.

Naturally, all these operators must be visible from the point of the register_simple_filter_factory call. Note that unlike the simple formatter factory, the filter factory requires the user's type to support reading from a stream. This is so because the filter factory would have to parse the argument of the filter relation from a string.

But we won't get away with a simple filter factory, because our point class doesn't have a sensible ordering semantics and thus we cannot define the complete set of operators. We'll have to implement our own filter factory instead. Filter factories derive from the filter_factory interface. This base class declares a number of virtual functions that will be called in order to create filters, according to the filter expression. If some functions are not overridden by the factory, the corresponding operations are considered to be not supported by the attribute value. But before we define the filter factory we have to improve our point class slightly:

struct point
{
    float m_x, m_y;

    point() : m_x(0.0f), m_y(0.0f) {}
    point(float x, float y) : m_x(x), m_y(y) {}
};

bool operator== (point const& left, point const& right);
bool operator!= (point const& left, point const& right);

template< typename CharT, typename TraitsT >
std::basic_ostream< CharT, TraitsT >& operator<< (std::basic_ostream< CharT, TraitsT >& strm, point const& p);
template< typename CharT, typename TraitsT >
std::basic_istream< CharT, TraitsT >& operator>> (std::basic_istream< CharT, TraitsT >& strm, point& p);

We have added comparison and input operators for the point class. The output operator is still used by formatters and not required by the filter factory. Now we can define and register the filter factory:

// Custom point filter factory
class point_filter_factory :
    public logging::filter_factory< char >
{
public:
    logging::filter on_exists_test(logging::attribute_name const& name)
    {
        return expr::has_attr< point >(name);
    }

    logging::filter on_equality_relation(logging::attribute_name const& name, string_type const& arg)
    {
        return expr::attr< point >(name) == parse_argument(arg);
    }

    logging::filter on_inequality_relation(logging::attribute_name const& name, string_type const& arg)
    {
        return expr::attr< point >(name) != parse_argument(arg);
    }

private:
    static point parse_argument(string_type const& arg)
    {
        std::istringstream strm(arg);
        point val;
        strm >> val;
        if (strm.fail() || strm.bad())
            throw std::runtime_error("Failed to parse point from \"" + arg + "\"");
        return val;
    }
};

void init_factories()
{
    logging::register_filter_factory("Coordinates", boost::make_shared< point_filter_factory >());
}

Having called the register_filter_factory function, whenever the filter parser encounters the "Coordinates" attribute mentioned in the filter, it will use the point_filter_factory object to construct the appropriate filter. For example, in the case of the following filter

%Coordinates% = "(10, 10)"

the on_equality_relation method will be called with name argument being "Coordinates" and arg being "(10, 10)".

[Note] Note

The quotes around the parenthesis are necessary because the filter parser should interpret the point coordinates as a single string. Also, round brackets are already used to group subexpressions of the filter expression. Whenever there is need to pass several parameters to the relation (like in this case - a number of components of the point class) the parameters should be encoded into a quoted string. The string may include C-style escape sequences that will be unfolded upon parsing.

The constructed filter will use the corresponding comparison operators for the point class. Ordering operations, like ">" or "<=", will not be supported for attributes named "Coordinates", and this is exactly the way we want it, because the point class does not support them either. The complete example is available here.

The library allows not only adding support for new types, but also associating new relations with them. For instance, we can create a new relation "is_in_rectangle" that will yield positive if the coordinates fit into a rectangle denoted with two points. The filter might look like this:

%Coordinates% is_in_rectangle "{(10, 10) - (20, 20)}"

First, let's define our rectangle class:

struct rectangle
{
    point m_top_left, m_bottom_right;
};

template< typename CharT, typename TraitsT >
std::basic_ostream< CharT, TraitsT >& operator<< (std::basic_ostream< CharT, TraitsT >& strm, rectangle const& r);
template< typename CharT, typename TraitsT >
std::basic_istream< CharT, TraitsT >& operator>> (std::basic_istream< CharT, TraitsT >& strm, rectangle& r);

As it was said, the rectangle is described by two points - the top left and the bottom right corners of the rectangle area. Now let's extend our filter factory with the on_custom_relation method:

// The function checks if the point is inside the rectangle
bool is_in_rectangle(logging::value_ref< point > const& p, rectangle const& r)
{
    if (p)
    {
        return p->m_x >= r.m_top_left.m_x && p->m_x <= r.m_bottom_right.m_x &&
               p->m_y >= r.m_top_left.m_y && p->m_y <= r.m_bottom_right.m_y;
    }
    return false;
}

// Custom point filter factory
class point_filter_factory :
    public logging::filter_factory< char >
{
public:
    logging::filter on_exists_test(logging::attribute_name const& name)
    {
        return expr::has_attr< point >(name);
    }

    logging::filter on_equality_relation(logging::attribute_name const& name, string_type const& arg)
    {
        return expr::attr< point >(name) == parse_argument< point >(arg);
    }

    logging::filter on_inequality_relation(logging::attribute_name const& name, string_type const& arg)
    {
        return expr::attr< point >(name) != parse_argument< point >(arg);
    }

    logging::filter on_custom_relation(logging::attribute_name const& name, string_type const& rel, string_type const& arg)
    {
        if (rel == "is_in_rectangle")
        {
            return boost::phoenix::bind(&is_in_rectangle, expr::attr< point >(name), parse_argument< rectangle >(arg));
        }
        throw std::runtime_error("Unsupported filter relation: " + rel);
    }

private:
    template< typename ArgumentT >
    static ArgumentT parse_argument(string_type const& arg)
    {
        std::istringstream strm(arg);
        ArgumentT val;
        strm >> val;
        if (strm.fail() || strm.bad())
            throw std::runtime_error("Failed to parse argument from \"" + arg + "\"");
        return val;
    }
};

void init_factories()
{
    logging::register_filter_factory("Coordinates", boost::make_shared< point_filter_factory >());
}

The on_custom_relation method is called with the relation name (the "is_in_rectangle" string in our case) and the right-hand argument for the relation (the rectangle description). All we have to do is to construct the filter, which is implemented by our is_in_rectangle function. We use bind from Boost.Phoenix to compose the filter from the function and the attribute placeholder. You can find the complete code of this example here.

#include <boost/log/utility/setup/from_settings.hpp>
#include <boost/log/utility/setup/from_stream.hpp>

The library provides mechanism of extending support for sinks similar to the formatter and filter parsers. In order to be able to mention user-defined sinks in a settings file, the user has to register a sink factory, which essentially contains the create_sink method that receives a settings subsection and returns a pointer to the initialized sink. The factory is registered for a specific destination (see the settings file description), so whenever a sink with the specified destination is mentioned in the settings file, the factory gets called.

For example, let's register the stat_collector sink we described before in the library. First, let's remember the sink definition:

// The backend collects statistical information about network activity of the application
class stat_collector :
    public sinks::basic_sink_backend<
        sinks::combine_requirements<
            sinks::synchronized_feeding,
            sinks::flushing
        >::type
    >
{
private:
    // The file to write the collected information to
    std::ofstream m_csv_file;

    // Here goes the data collected so far:
    // Active connections
    unsigned int m_active_connections;
    // Sent bytes
    unsigned int m_sent_bytes;
    // Received bytes
    unsigned int m_received_bytes;

    // The number of collected records since the last write to the file
    unsigned int m_collected_count;
    // The time when the collected data has been written to the file last time
    boost::posix_time::ptime m_last_store_time;
    // The collected data writing interval
    boost::posix_time::time_duration m_write_interval;

public:
    // The constructor initializes the internal data
    stat_collector(const char* file_name, boost::posix_time::time_duration write_interval);

    // The function consumes the log records that come from the frontend
    void consume(logging::record_view const& rec);
    // The function flushes the file
    void flush();

private:
    // The function resets statistical accumulators to initial values
    void reset_accumulators();
    // The function writes the collected data to the file
    void write_data();
};

Compared to the earlier definition we added the write_interval constructor parameter so we can set the statistical information flush interval in the settings file. The implementation of the sink stays pretty much the same as before. Now we have to define the factory:

// Factory for the stat_collector sink
class stat_collector_factory :
    public logging::sink_factory< char >
{
public:
    // Creates the sink with the provided parameters
    boost::shared_ptr< sinks::sink > create_sink(settings_section const& settings)
    {
        // Read sink parameters
        std::string file_name;
        if (boost::optional< std::string > param = settings["FileName"])
            file_name = param.get();
        else
            throw std::runtime_error("No target file name specified in settings");

        boost::posix_time::time_duration write_interval = boost::posix_time::minutes(1);
        if (boost::optional< std::string > param = settings["WriteInterval"])
        {
            unsigned long sec = std::stoul(param.get());
            write_interval = boost::posix_time::seconds(sec);
        }

        // Create the sink
        boost::shared_ptr< stat_collector > backend = boost::make_shared< stat_collector >(file_name.c_str(), write_interval);
        boost::shared_ptr< sinks::synchronous_sink< stat_collector > > sink = boost::make_shared< sinks::synchronous_sink< stat_collector > >(backend);

        if (boost::optional< std::string > param = settings["Filter"])
        {
            sink->set_filter(logging::parse_filter(param.get()));
        }

        return sink;
    }
};

void init_factories()
{
    logging::register_sink_factory("StatCollector", boost::make_shared< stat_collector_factory >());
}

As you can see, we read parameters from settings and simply create our sink with them as a result of create_sink method. Generally, users are free to name parameters of their sinks the way they like, as long as settings file format is adhered. However, it is a good idea to follow the pattern established by the library and reuse parameter names with the same meaning. That is, it should be obvious that the parameter "Filter" means the same for both the library-provided "TextFile" sink and out custom "StatCollector" sink.

After defining the factory we only have to register it with the register_sink_factory call. The first argument is the new value of the "Destination" parameter in the settings. Whenever the library finds sink description with destination "StatCollector", our factory will be invoked to create the sink. It is also possible to override library-provided destination types with user-defined factories, however it is not possible to restore the default factories afterwards.

[Note] Note

As the "Destination" parameter is used to determine the sink factory, this parameter is reserved and cannot be used by sink factories for their own purposes.

Now that the factory is registered, we can use it when initializing from files or settings. For example, this is what the settings file could look like:

[Sinks.MyStat]

Destination=StatCollector
FileName=stat.csv
WriteInterval=30

The complete code of the example in this section can be found here.


PrevUpHomeNext