Chuck Knight's Blog

I’ve been using Google’s unit test framework (gtest) for a few years now and while it has been very useful, I have to admit that it’s been a pain trying to make the libraries available to each and every project. I’ve thought about adding gtest to our standard build tools area but didn’t look forward to having to manage yet another tool.

While looking at Boost for my last post, I was pleasantly surprised to find that Boost also has a unit test framework. The Boost unit test framework (boost_unit_test_framework) is functionally equivalent to Google’s. Coupled with the fact that we already use Boost, it presents an opportunity to reduce the number of dependencies our projects have on external libraries.

The discussion that follows provides some simple guidance on how to start using the Boost unit test framework to test your software.

Using the Boost Unit Test Framework

Basic Structure

Similar to Google test, each compilation unit in your group of tests is meant to test a specific set of functionally. This basic unit is the test module. Each test module is composed of one or more test suites which may contain several test cases. Here is a basic skeleton of a test module:

The Boost unit test framework provides its own main entry point (main). In order to do this, Boost must know if you are using a dynamic or static version of the Boost libraries. If you are using a dynamic library version of the Boost, you must define the BOOST_TEST_DYN_LINK macro. If you are using a static library version, remove this definition since it will cause multiple definitions of the main entry point during linking.

The test module name is defined using the BOOST_TEST_MODULE macro and must precede the inclusion of the Boost unit test headers. This macro will taylor the Boost unit test definitions to your test module and generate the appropriate code.

Include the Boost unit test header (unit_test.hpp) and define the relevant namespaces (boost and boost::unit_test) to make using the unit test framework easier to use.

The Boost unit test suite name is defined by the macro BOOST_AUTO_TEST_SUITE. This macro identifies the test cases between it and the BOOST_AUTO_TEST_SUITE_END macro as being associated with the the test suite. Be aware that these macros generate code that defines a scope and must be paired to compile correctly.

The test fixture (named TestFixture above) defines an object which is used to initialize and later cleanup any external testing requirements. At the beginning of each test case a test fixture is created within the test case scope. This creation invokes the object’s constructor giving you an opportunity to acquire any resources your test case needs. At the end of each test case the test fixture will go out of scope allowing the destructor to clean up any resources you have acquired. Note that any public methods or instance variables within the test fixture will be accessible within your test case.

Finally, each test case is defined using the macro BOOST_FIXTURE_TEST_CASE. This macro takes two arguments. The first argument defines the test case name and should be unique. The second argument identifies the type name of the test fixture to be used during test case initialization.

As a final point, I have to point out that the Boost unit test framework can be very sensitive to name collisions. Make sure that each module, suite and test case use a unique name.

Test Predicates

Test cases may contain any number of test predicates. Here is an example of a valid test case:


BOOST_FIXTURE_TEST_CASE(MyTest, TestFixture)
{
     // Verify i is not 0
     BOOST_REQUIRE_NO_THROW(i = 4/i);
     // Verify i is equal to 3
     BOOST_CHECK(i == 3);
}


Test predicates are used within a test case to determine if the test requirements have been met. These predicates function like asserts and depending on their name modifiers issue warnings, test failures or fatal errors. Here is a list of the basic test predicates and an explanation of each:

BOOST_WARN(boolean expression)

This predicate evaluates the boolean statement provided and issues a warning if the statement is false.

BOOST_CHECK(boolean expression)

This predicate evaluates the boolean statement provided and issues a test failure if the statement is false.

BOOST_REQUIRE(boolean expression)

This predicate evaluates the boolean statement provided and issues a test error if the statement is false.

BOOST_WARN_MESSAGE(boolean expression, message)

This predicate evaluates the boolean statement provided and issues a warning with the specified message if the statement is false.

BOOST_CHECK_MESSAGE(boolean expression, message)

This predicate evaluates the boolean statement provided and issues a test failure with the specified message if the statement is false.

BOOST_REQUIRE_MESSAGE(boolean expression, message)

This predicate evaluates the boolean statement provided and issues a test error with the specified message if the statement is false.

BOOST_WARN_THROW(statement, exception)

This predicate evaluates the statement and issues a warning if the statement does not throw the expected exception (specified in the second parameter).

BOOST_CHECK_THROW(statement, exception)

This predicate evaluates the statement and issues a test failure if the statement does not throw the expected exception (specified in the second parameter).

BOOST_REQUIRE_THROW(statement, exception)

This predicate evaluates the statement and issues a test error if the statement does not throw the expected exception (specified in the second parameter).

BOOST_WARN_NO_THROW(statement)

This predicate evaluates the statement and issues a warning if the statement throws an exception.

BOOST_CHECK_NO_THROW(statement)

This predicate evaluates the statement and issues a test failure if the statement throws an exception.

BOOST_REQUIRE_NO_THROW(statement)

This predicate evaluates the statement and issues a test error if the statement throws an exception.

Running A Test Module

Running a test module is straight forward, just run the linked executable. Like Google test, you can control which tests are run and what level of detail the test framework will emit. Here are some simple examples:

example.test –help

This command runs the test executable and displays a command line argument help message.

example.test –log_level=warning

This command runs the test executable and requests that it emit only warning (and error) messages. Valid values for the log_level argument are: all, success, test_suite, message, warning, error, cpp_exception, system_error, fatal_error and nothing.

example.test –run_test=ExampleTestSuite/BoostWarnTest

This command runs the BoostWarnTest test case within the ExampleTestSuite test suite. Note that wildcards are acceptable within the test specification.

Putting it all together

Here is a full listing of the makefile used to create the test module:

# Define comple related values
CXX = clang++
CXXFLAGS = -O2 -g -Wall -std=c++11 -fmessage-length=0
INCLUDES := -I ~/Documents/Addons/boost

# Define linker related values
LD := clang++
LIBS := -L ~/Documents/Addons/boost/lib -lboost_unit_test_framework

# Create lists of the elements of our build
SOURCES := $(shell find . -depth 1 -name ‘*.cpp’ -print | sort)
OBJECTS := $(SOURCES:.cpp=.o)
TARGETS := $(SOURCES:.cpp=.test)

%.o:%.cpp
$(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o $@

# Define the default stanza
all: $(TARGETS)

# Deine a default stanza for linking all test executables
%.test : %.o
$(LD) $(LIBS) $< -o $@
chmod 755 $@

# Run all tests
run_tests: $(TARGETS)
@for target in $(TARGETS) ; do \
echo Running $$target ; \
$$target ; \
echo ; \
done

# Clean up deleting all temporary files
clean:
rm -f $(TARGETS) $(OBJECTS)

Here is a full listing of a working test module:

// The following two defines are required by the Boost unit test framework
// to create the necessary testing support. These defines must be placed
// before the inclusion of the boost headers.
//
// The first of these defines is used to indicate that we are building a
// unit test module that will link dynamically with Boost. If you are using
// a static library version of Boost, this define must be deleted. (or
// in this case commented out)
//
// The second define provides a name for our Boost test module.
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE ExampleTestModule
#include <boost/test/unit_test.hpp>

#include
#include

// The two relevant Boost namespaces for the unit test framework are:
using namespace boost;
using namespace boost::unit_test;

// Provide a name for our suite of tests. This statement is used to bracket
// our test cases.
BOOST_AUTO_TEST_SUITE(ExampleTestSuite)

// The structure below allows us to pass a test initialization object to
// each test case. Note the use of struct to default all methods and member
// variables to public access.
struct TestFixture
{
TestFixture()
: i( 0 )
{
// Put test initialization here, the constructor will be called
// prior to the execution of each test case.
printf(“Initialize test\n”);
}
~TestFixture()
{
// Put test cleanup here, the destructor will automatically be
// invoked at the end of each test case.
printf(“Cleanup test\n”);
}

// Public test fixture variables are automatically available to all test
// cases. Don’t forget to initialize these variables in the constructors
// to avoid initialized variable errors.
int i;
};

// Define a test case. The first argument specifies the name of the test.
// Take some care in naming your tests. Do not reuse names or accidentally use
// the same name for a test as specified for the module test suite name.
//
// The second argument provides a test build-up/tear-down object that is
// responsible for creating and destroying any resources needed by the
// unit test
BOOST_FIXTURE_TEST_CASE(BoostWarnTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and will not issue a warning.
BOOST_WARN(true);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will issue a warning.
BOOST_WARN(false);
}

BOOST_FIXTURE_TEST_CASE(BoostCheckTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and will not indicate a failed test.
BOOST_CHECK(true);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will indicate a test failure.
BOOST_CHECK(false);
}

BOOST_FIXTURE_TEST_CASE(BoostRequireTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and will not indicate a fatal test error.
BOOST_REQUIRE(true);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will indicate a fatal test error.
BOOST_REQUIRE(false);
}

BOOST_FIXTURE_TEST_CASE(BoostWarnMessageTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and will not issue a warning with
// message.
BOOST_WARN_MESSAGE(true, “This should pass”);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will issue a warning with message.
BOOST_WARN_MESSAGE(false, “This should fail”);
}

BOOST_FIXTURE_TEST_CASE(BoostCheckMessageTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and not will indicate a failed test
// with message.
BOOST_CHECK_MESSAGE(true, “This should pass”);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will indicate a test failure
// with message.
BOOST_CHECK_MESSAGE(false, “This should fail”);
}

BOOST_FIXTURE_TEST_CASE(BoostRequireMessageTest, TestFixture)
{
// Provide a test predicate (i.e. a conditional statement) that evaluates
// to true to allow the test to pass and will not indicate a fatal test
// error with message.
BOOST_REQUIRE_MESSAGE(true, “This should pass”);

// Provide a test predicate (i.e. a conditional statement) that evaluates
// to false to force the test to fail and will indicate a fatal test error
// with message.
BOOST_REQUIRE_MESSAGE(false, “This should fail”);
}

BOOST_FIXTURE_TEST_CASE(BoostWarnThrowTest, TestFixture)
{
// Provide a statement that is expected to throw with a specific exception.
// Provide the expected exception as the second argument. Since the
// exception types match, the test will pass and will not issue a warning.
BOOST_WARN_THROW(
throw std::range_error(“This should pass”),
std::range_error);

// Provide a statement that is expected to throw with a specific exception.
// Provide the expected exception as the second argument. Since the
// exception types do not match, the test will not pass and will issue a
// warning.
BOOST_WARN_THROW(
throw std::range_error(“This should fail”),
std::underflow_error);
}

BOOST_FIXTURE_TEST_CASE(BoostCheckThrowTest, TestFixture)
{
// Provide a statement that is expected to throw with a specific
// exception. Provide the expected exception as the second argument.
// Since the exception types match, the test will pass and will
// not issue a test failure.
BOOST_CHECK_THROW(
throw std::range_error(“This should pass”),
std::range_error);

// Provide a statement that is expected to throw with a specific
// exception. Provide the expected exception as the second argument.
// Since the exception types do not match, the test will not pass and
// will issue a test failure.
BOOST_CHECK_THROW(
throw std::range_error(“This should fail”),
std::underflow_error);
}

BOOST_FIXTURE_TEST_CASE(BoostRequireThrowTest, TestFixture)
{
// Provide a statement that is expected to throw with a specific
// exception. Provide the expected exception as the second argument.
// Since the exception types match, the test will pass and will not
// issue a fatal test error.
BOOST_REQUIRE_THROW(
throw std::range_error(“This should pass”),
std::range_error);

// Provide a statement that is expected to throw with a specific
// exception. Provide the expected exception as the second argument.
// Since the exception types do not match, the test will not pass
// and will issue a fatal test error.
BOOST_REQUIRE_THROW(
throw std::range_error(“This should fail”),
std::underflow_error);
}

BOOST_FIXTURE_TEST_CASE(BoostWarnNoThrowTest, TestFixture)
{
// Provide a statement that is not expected to throw. If the statement
// does not throw, the test will pass and not issue a warning.
BOOST_WARN_NO_THROW(i = 1);

// Provide a statement that is expected to not throw. If the statement
// does throw, the test will fail and issue a warning.
BOOST_WARN_NO_THROW(
throw std::range_error(“This should fail”));
}

BOOST_FIXTURE_TEST_CASE(BoostCheckNoThrowTest, TestFixture)
{
// Provide a statement that is not expected to throw. If the statement
// does not throw, the test will pass and not issue a test failure.
BOOST_CHECK_NO_THROW(i = 2);

// Provide a statement that is expected to not throw. If the statement
// does throw, the test will fail and issue a test failure.
BOOST_CHECK_NO_THROW(
throw std::range_error(“This should fail”));
}

BOOST_FIXTURE_TEST_CASE(BoostRequireNoThrowTest, TestFixture)
{
// Provide a statement that is not expected to throw. If the statement
// does not throw, the test will pass and not issue a test failure.
BOOST_REQUIRE_NO_THROW(i = 3);

// Provide a statement that is expected to not throw. If the statement
// does throw, the test will fail and issue a test failure.
BOOST_REQUIRE_NO_THROW(
throw std::range_error(“This should fail”));
}

// This must match the BOOST_AUTO_TEST_SUITE(ExampleTestSuite) statement
// above and is used to bracket our test cases.
BOOST_AUTO_TEST_SUITE_END()

For more information consult the Boost unit test framework user’s guide located at:

http://www.boost.org/doc/libs/1_46_1/libs/test/doc/html/utf/user-guide/