TUT is a small and portable unit test framework for C++.

• TUT is very portable: no matter what compiler or OS do you use.
• TUT consists of header files only. No libraries required.
• Custom reporter interface allows to integrate TUT with virtually any IDE or tool in the world.
• Tests are organised into named test groups.
• Regression (all tests in the application), one-group or one-test execution.
• Pure C++, no macros!
• Support for multi-process testing.
• TUT is free and distributed under a BSD-like license.

TUT tests are easy to read and maintain. Here's the simplest test file possible:

#include <tut.h>

namespace tut
{
  struct basic{};
  typedef test_group<basic> factory;
  typedef factory::object object;
}

namespace
{
  tut::factory tf("basic test");
}

namespace tut
{
  template<> template<>

  void object::test<1>()
  {
    ensure_equals("2+2=?",2+2,4);
  }
}

An example testgroup which tests a std::auto_ptr implementation. Of course, those tests are far from complete.

#include <tut.h>
#include <set>
#include <memory>

namespace tut
{
  /**
   * Struct which may contain test data members.
   * Test object (class that contains test methods)
   * will inherite from it, so each test method can
   * access members directly.
   *
   * Additionally, for each test, test object is re-created
   * using defaut constructor. Thus, any prepare work can be put
   * into default constructor.
   *
   * Finally, after each test, test object is destroyed independently
   * of test result, so any cleanup work should be located in destructor.
   */
  struct auto_ptr_data
  { 
    /**
     * Type used to check scope lifetime of auto_ptr object.
     * Sets extern boolean value into true at constructor, and
     * to false at destructor.
     */
    bool exists;
    struct existing
    {
      bool& s_;
      existing(bool& s) : s_(s){ s_ = true; };
      ~existing(){ s_ = false; };
    };
  };

  /**
   * This group of declarations is just to register
   * test group in test-application-wide singleton.
   * Name of test group object (auto_ptr_group) shall
   * be unique in tut:: namespace. Alternatively, you
   * you may put it into anonymous namespace.
   */
  typedef test_group<auto_ptr_data> tf;
  typedef tf::object object;
  tf auto_ptr_group("std::auto_ptr");

  /**
   * Checks default constructor.
   */
  template<>

  template<>
  void object::test<1>()
  {
    std::auto_ptr<existing> ap;
    ensure(ap.get()==0);
    ensure(ap.operator->()==0);
  }

  /**
   * Checks constructor with object
   */
  template<>
  template<>
  void object::test<2>()
  {
    {
      std::auto_ptr<existing> ap(new existing(exists));
      ensure("get",ap.get()!=0);
      ensure_equals("constructed",exists,true);
    }
    // ptr left scope
    ensure_equals("destructed",exists,false);
  }

  /**
   * Checks operator -> and get()
   */
  template<>

  template<>
  void object::test<3>()
  {
    std::auto_ptr<existing> ap(new existing(exists));
    existing* p1 = ap.get();
    existing* p2 = ap.operator->();
    ensure("get equiv ->",p1==p2);
    // ensure no losing ownership
    p1 = ap.get();
    ensure("still owner",p1==p2);
  }

  /**
   * Checks release()
   */
  template<>
  template<>
  void object::test<4>()
  {
    {
      std::auto_ptr<existing> ap(new existing(exists));
      existing* p1 = ap.get();
      std::auto_ptr<existing> ap2(ap.release());
      ensure("same pointer",p1==ap2.get());
      ensure("lost ownership",ap.get()==0);
    }
    ensure("destructed",exists==false);
  }

  /**
   * Checks assignment.
   */
  template<>

  template<>
  void object::test<5>()
  {
    {
      std::auto_ptr<existing> ap(new existing(exists));
      existing* p1 = ap.get();
      std::auto_ptr<existing> ap2;
      ap2 = ap;
      ensure("same pointer",p1==ap2.get());
      ensure("lost ownership",ap.get()==0);
    }
    ensure("destructed",exists==false);
  }

  /**
   * Checks copy constructor.
   */
  template<>
  template<>

  void object::test<6>()
  {
    {
      std::auto_ptr<existing> ap(new existing(exists));
      existing* p1 = ap.get();
      std::auto_ptr<existing> ap2(ap);
      ensure("same pointer",p1==ap2.get());
      ensure("lost ownership",ap.get()==0);
    }
    ensure("destructed",exists==false);
  }
}

Tests organization

Tests are compiled into a single binary (a test application).

The test application contains tests organized into test groups. Every test group has a human-readable name (such as "payroll"). Normally every group is located in it's own source file. A group can contain an unlimited number of tests. Every test has an unique number as test identifier ( C++ templates do not support specialization by a string value).

Tests

A test is a function (method) implementing some specific scenario and checking if the code (unit) works as required. Every test usually checks only one specific bit of functionality. In every test we have a preparation phase, execution phase and verification phase. For instance, if we need to test our container's clear() method, we need to:

• create a container instance and fill it with some data (preparation phase)
• call clear() method (execution phase)
• ensure that size() now returns zero (verification phase)

Let's implement this scenario as a example TUT test.

Creating a test group

To begin we need to create a new source file for our test group. Let's call it test.cpp.

 
// test.cpp 
#include <tut.h> 
namespace tut 
{ 
	struct data // (1)
	{ 
	};
	
	typedef	test_group<data> tg; // (2)
	tg test_group("my first test");    // (3)
}; 

Let's outline what we've just done.

• We included TUT header file (an obvious step).
• We defined test data structure. Any data that should be available to every test could be located in this data structure. An instance of the structure is created before every test starts and is destroyed right after the test ends, so you may use the structure's constructor and destructor for initialization and cleanup purposes.
• We created an instance of the test group with the name "my first test". Behind the scene the instance registers itself with a global test runner, so the name shall be unique within the test application.

Creating a test

In TUT all tests have an unique numbers, not names, within their test group. You can provide an optional name to the test by calling set_test_name() method.

 
#include <my_set.h> 	
	
#include <tut.h> 
namespace tut 
{ 
	struct data //
	{ 
	};
	
	typedef	test_group<data> tg;
	tg test_group("my first test");
	
	typedef tg::object testobject;
	
	template<> 
	template<> 
	void testobject::test<1>() 
	{ 
		set_test_name("clear");

		my::set s('a','b','c');
		ensure_equals("has elements",s.size(),3);

		s.clear();
		ensure_equals("has no elements",s.size(),0);	
	}	
}; 

Our test is completed. Let walk through the code:

• For the sake of brevity we had created yet another typedef for test object.
• We defined our test as a test number 1. We also gave it a human-readable name "clear".
• We created an instance of our proprietary set class with three elements in it.
• We checked that the size() method in fact returns 3.
• We cleared the set.
• We verified that the size() returns zero now.

Finally, the main module of the test application

#include <tut.h>
#include <tut_reporter.h>
#include <iostream>

using std::exception;
using std::cerr;
using std::endl;

namespace tut
{
    test_runner_singleton runner;
}

int main()
{
    tut::reporter reporter;
    tut::runner.get().set_callback(&reporter);

    tut::runner.get().run_tests();

    return !reporter.all_ok();
}

Here we see the minimum required implementation of the main module. It contains the instance of the runner (a singleton, as class name suggests), where test groups register themselves. It creates an instance of a reporter (which can be extended or replaced to support virtually any target and style of the report. And finally, it runs all the tests in one batch.

Runner support a few more precise methods to run tests as well, such as "run all tests within a group" or even "run a specific test within a group".

We're done.

Tip: data structure constructor/destructor as setUp/tearDown methods

To aquire resources before the test and to release them right after the test use constructor and destructor of the data structure:

 
	...
	struct complex_data 
	{ 
		connection* con;
		complex_data() { con = db_pool.get_connection(); } 
		~complex_data() { db_pool.release_connection(con); } 
	};

	template<> 
	template<> 
	void testobject::test<1>() 
	{ 
		...
		con->commit();
	}

Each test in the group from now on will have the connection initialized by constructor and released in destructor.

What would happen if the constructor throws an exception? TUT will treat it as if test itself failed with an exception. The test body won't be executed and it will be reported as failed with exception. But the destructor of the data structure will be executed anyway!

An exception in the destructor is threated differently though. Reaching destruction phase means that the test itself has finished successfuly. In this case TUT marks the test with a warning status.

Tip: more than 50 tests per group

A newly created group has a predefined set of dummy tests (i.e. test placeholders). By default, there are 50 tests in a group. To create a test group with a higher volume (e.g. when tests are generated by a script) provide a higher border of test group size when it is instantiated:

 
// test group with maximum 500 tests 
typedef test_group<huge_test_data,500> testgroup;

Note that your compiler would possibly need a command-line switch or pragma to enlarge the recursive instantiation depth. For g++, for example, you should specify --ftemplate-depth-501 to make example above compile. Please consult your compiler's documentation.

TUT is a small and portable unit test framework for C++. It's so small that it fits into one header file. It's so portable that it could be used on almost any C++ platform, including Windows, MacOS, Unices and embedded devices.

No and yes. C++ template engine doesn't support usage of run-time objects (and string is a run-time object) for specialization. Compile-time constants is the only way.

On the other hand, there is a method set_test_name("a name") which you can call in the beginning of a test to make it look prettier in the failed build report. <grin>

Alas.

First, from a pure theoretical POV, testing private methods is considered harmful as it exposes class internals, while traditional testing focuses on public interface only.

Second, it's just plain impossible in C++ without making TUT a friend of tested class (which can be done, I guess, I just never tried).

C++Unit, boost::test and other frameworks has similar goals.

But there are some issues with many of them:

• they require to use a library
• tests depend on preprocessor macros
• they often overloaded with features

TUT, in contrast, is located in a single header file (20K). All you should do is to include it into the test module. No linking at all.

TUT uses C++ template engine to dispatch calls to test methods. Therefore you shouldn't even register methods as tests; template will do it for you automatically. As a result, the test code is more readable.

And finally, TUT is a minimal software. It only does what it's designed for. It doesn't integrate with MSDN or control production processes. It just runs tests.

Most modern compilers are supported. Some outdated compilers are unable to handle templates in TUT, alas.

Supported:

• GNU GCC 2.95
• GNU GCC 3.x (both under unix and MinGW)
• Borland 5.6 (Borland C++ Builder 6)
• Intel C++ Compiler 6.x
• Intel C++ Compiler 8.1
• Sun Studio 9: C++ 5.6 Compiler
• Microsoft VC7 (Microsoft VS.NET 2003 and later)
• Sun WorkShop 6 update 2 C++ 5.3 (probably, previous versions as well)

Unsupported:

• Borland 5.5.x (Borland C++ Builder 5)
• Microsoft VC6 (including SP5)
• Microsoft VC7 (Microsoft VS.NET 2002)
• C for AIX Compiler, Version 6
• KAI C++ Compiler
• Portland Group Compiler, Version 5.2-4

If you use TUT with any other compiler or environment please let me know.

Some broken compiler/linker/platform combinations require to make methods ensure(), ensure_equals and fail() to be inline, and not in anonymous namespace. Try to change tut.h accordingly if you see "duplicate symbol ensure" or "ensure is not found" during linking stage.

Self tests are very special beasties, and actually you've seen two(!) TUT frameworks running one under control of another. The case is quite extreme. Regular TUT tests are very simple to read.

One or both types you've provided to ensure_equals() have no operator << at all. Since the diagnostic message is built using std::stringstream, compiler needs the operator to format your objects. Either add the operator or use ensure() method (which doesn't tell you the exact values the objects had, just the fact they were not equal).

C++ Standard doesn't specify what happens if the code references wrong address. Thus, segmentation fault processing is system and compiler dependent, and shall be handled differently for each system/compiler pair.

If you want TUT to react correctly to tests failures caused by segfaults, you must somehow convert hardware faults into C++ exceptions.

For Win32 TUT uses SEH. You need to specify -DTUT_USE_SEH at the test build time.

For unixes there is no standard way to convert SIGSEGV into an exception. Consult your platform/compiler documentation for possible ways to do that.

You may also use restartable wrapper defined in optional header tut_restartable.h. It runs the tests the same way as regular runner does, but also logs the progress. If a test crashes the test application, and then test application is started again, the wrapper will load last log record, and continue test execution from position after the crashed one.

Of course, this requires helper script that runs test application until all tests will be runned. The script might be is as simple as

while true
do
  ./restartable && exit 0
done

Directory examples/restartable contains a simple restartable test application.

This approach can be extended to support deadlocks in code. The script must be modified to automatically kill test application after specified period of time.

It allows spawning child processes on platforms that support it. This is often needed when testing client/server applications. Using TUT, one can fork the main process and use ensure_* functions in both processes. Moreover, one can ensure that spawned processes exited, were killed etc.

TUT takes care of all necessary cleanup: all spawned processes are stopped after current test case finishes.

TUT has a small forum on SourceForge. Feel free to post your comments, questions and bug-reports there.

This software (TUT) was designed and written by Vladimir Dyuzhev, who currently lives in Toronto, Canada. His personal site is vladimir-dyuzhev.net.

In recent years TUT was part-time maintained by Denis Kononenko.

Please post your comments and bug-reports to SourceForge TUT forum

www.armaties.com/testfram.htm - Kent Beck's paper on Smalltalk unit test framework xUnit "Simple Smalltalk Testing:With Patterns".

cppUnit - Another popular C++ unit test framework.

junit.org - Most popular framework for Java: JUnit.

maxkir.com - Kir & Sashka Maximov's site: people who are translating a lot of Xtreme Programming books and articles into Russian. A must for any Russian who're interested in Agile development. Kir was also my coach in Java, and Sashka translated most of this site into English. Thank you both!

stlport.com Industrial-quality free STL implementation: STLport.

Copyright 2002-2006 Vladimir Dyuzhev
Copyright 2007 Denis Kononenko
Copyright 2008 Michał Rzechonek

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Latest versions of TUT:

TUT-2008-11-30

Changes:

Introduced a whole new feature of multi-process testing. This includes forking, reporting failed tests in children, controlling their exit status and cleanup code:

pid_t childPid = fork();
if(childPid == 0)
{
	ensure("Child failed", false);
}

// this is going to fail
ensure_child_exit(childPid, 0);

Added ensure_errno test for meaningful reporting of failed system calls.

Both features are available only on *nix platforms.

Download:

• TUT-2008-11-30.tar.gz
  
• TUT-2008-11-30.zip
  

TUT-2007-07-06

Changes:

Introduced new header include style. There are two ways you can include the TUT headers:

#include <tut.h>

or:

#include <tut/tut.hpp>

The second option is preferred method whereas the first option is to provide backward compatibility with clients of the previous releases.

Minor fixes for Borland C++ Builder 6 and Microsoft Visual Studio 2005.

Download:

• TUT-2007-07-06.tar.gz
  
• TUT-2007-07-06.zip
  

TUT-2007-03-19

Changes:

Introduced a new exception tut_error as base for all TUT exceptions. I have two reasons to do it:

• To avoid interference with std::logic_error leading to annoyed pitfalls in inaccurate test code. For example:

  
    // function to test
    void foo(bool cond)
    {
        if (!cond) 
        {
            throw logic_error("condition is not set");
        }
    }
  	
    // inside a test
    try
    {
        ensure("something", some_expression); // can throw logic_error
        foo(false);
    }
    catch (const logic_error&)
    {
        // ok or pitfall?
    }
  

  Howewer, because of that tut_error is derived from std::exception, you should avoid catching std::exception in your test code without appropriate checks.
• Some implementations of Standard C++ Library restrict size of a message passed into a standard exception (from <stdexcept>) within narrow limits (usually, 256 bytes). Sometimes, it leads to incomplete messages in TUT reports.

Minors:

• actual and expected values are quoted to increase failure messages readability;
• if ensure_distance fails it will output a range in round brackets because range borders are excluded from range (thanks to Koolin Timofey).

New function added: ensure_not. I found that ensure_not(condition) is more readable than ensure(!condition).

Download:

• TUT-2007-03-19.tar.gz
  
• TUT-2007-03-19.zip
  

TUT-2007-02-03

Changes:

Microsoft Visual C++ 2005 is supported. This version of compiler supports standard C++ exception handling in accordance with the C++ standard. It means that only synchronous C++ exceptions thrown with a throw statement will be caught in a catch block.

TUT uses two models of exception: handling SEH and standard C++ exception handling. TUT expects that if any structured exception is raised it will be handled by nearer C++ catch handler. It was default behavior of the previous version of compiler (option /EHs). But for the new version I have to turn on asynchronous exception handling (option /EHa).

Minors: Some polish work.

Note: I consider to stop maintain a lot of Makefiles due to lack of time and support only Jamfile.

Download:

• TUT-2007-02-03.tar.gz
  
• TUT-2007-02-03.zip
  

TUT-2006-11-04

Changes:

Fix: lost changes from version TUT-2006-03-29 are restored.

Fix: errors appeared while compiling restartable example are fixed. But there are a lot of outstanding works to support completely the last changes.

Jamfiles for examples added.

Download:

• TUT-2006-11-04.tar.gz
  
• TUT-2006-11-04.zip
  

TUT-2006-10-31

Changes:

Today's update is the work of a new TUT's contributor, Denis Kononenko.

Jamfile added to build TUT Framework using Boost Build System V2. It builds TUT itself and automatically executes the selftest. Further enchancements are coming soon.

New functionality is added: now we can optionally specify the test name right from inside the test.

    template < > template < > 
    void my_object_tests::test < 1 > () 
    { 
        set_test_name("test basic scenario"); 
        ... 
    }

If the test fails the test name will appear in the test report, e.g.:

    ---> group: my_object_tests, test: test<1> : test basic scenario 
         problem: assertion failed 
         failed assertion: "not implemented"

Also a custom reporter can retrieve the test name using tut::test_result::name member.

Minor fix: TUT selftest didn't exit with code 0 if there have been failed tests.

Download:

• TUT-2006-10-31.tar.gz
  
• TUT-2006-10-31.zip
  

TUT-2006-03-29

Changes:

New callback events: group_started() and group_completed(). Thanks to Mateusz Loskot and Bartlomiej Kalinowski for pointing out to the incompleteness of the interface.

Download:

• TUT-2006-03-29.zip
  

TUT-2005-06-22

Changes:

• ensure() now is a template and thus accepts both const char* and std::string
• optimization level is reduced to -g for GCC3 in samples since otherwise I have a sigsegv

Download:

• TUT-2005-06-22.tar.gz
  
• TUT-2005-06-22.zip
  

TUT-2005-05-19

Changes:

Bugfixing: Win32 doesn't allow to mix try/catch and _try/_catch. Mea culpa.

Download:

• TUT-2005-05-19.tar.gz
  
• TUT-2005-05-19.zip
  

TUT-2005-05-16

Changes:

Bugfixing release (thanks to Denis Kononenko).

An exception in test group constructor was causing all tests in the group to be marked as [F]: failed, even including dummy methods, i.e. those not written by the developer. The result is usually a list of 50 "errors" only few of them are actual tests.

Fix:

• An exception in test group ctor terminates the whole group. No tests in this group are executed in this run.
• First failed test has reason set to ex_ctor, and default reporter shows it as [1=C].

Also, unix makefiles are converted to 0xA again (CVS converts them to 0xD 0xA when it checks out to Windows machine; now I marked them binary, so it shouldn't happen again).

Download:

• TUT-2005-05-16.tar.gz
  
• TUT-2005-05-16.zip
  

TUT-2004-03-26

Changes:

Minor, really minor change: dropped extra ';' from end of declaration which prevented TUT from being used with Sun C++ compiler (thanks to Andrey Sidorenko)

Download:

• TUT-2004-03-26.tar.gz
  
• TUT-2004-03-26.zip
  

TUT-2003-08-17

Changes:

Usablility changes (thanks to Justin Sampson):

• Class tut::reporter has new method

          bool all_ok() const;
          

  to check if everything goes/gone fine.
• Template methods ensure(), ensure_equals(), ensure_distance() and fail() can now be called not only inside test object methods, but from any code location.

Download:

• TUT-2003-08-17.tar.gz
  
• TUT-2003-08-17.zip
  

TUT-2003-07-04

Changes:

• TUT now works under VC7 2003. VC7 2002 is not supported, and will not be supported due to lack of standard features.
• Tests are now should be specialized more standard-way, i.e. with using two template<> statements:

  
        template<>
        template<>
        void object::test<1>()
        {
        }
      

  All compilers except for VC allow to specify only one template line, but it is not standard, and going to change soon.
• Since VC forbids to use __try and try in same context, minor changes are made in SEH code to replace throw with good old return.
• Makefiles organization is refactored.

Download:

• TUT-2003-07-04.tar.gz
  
• TUT-2003-07-04.zip
  

TUT-2003-06-25

Changes:

• Default results callback handler is moved into separate optional header tut_reporter.h and refactored to increase verbosity level. There are also a few more self-tests to cover the reporter. Usage:

          #include <tut.h>
          #include <tut_reporter.h>
  
          namespace tut
          {
            test_runner_singleton runner;
          }
   
          int main()
          {
            tut::reporter reporter;
            tut::runner.get().set_callback(&reporter);
            tut::runner.get().run_tests();          
            ...
          }
          

• Makefile structure for selftest is refactored finally to reuse common dependencies Makefile.common. Shame on me not done it early.
• SCO platforms are no longer supported. SCO is no more a software company, it produces only sues.

Download:

• TUT-2003-06-25.tar.gz
  
• TUT-2003-06-25.zip
  

TUT-2003-04-28

Changes:

• Tar.gz is now built using real TAR and GZIP applications, not 7ZIP.
• Fixed missed STL includes required for some platforms.
• Removed outdated dependencies in Makefiles.
• Supressed useless and mistaken compiler warnings.

Download:

• TUT-2003-04-28.tar.gz
  
• TUT-2003-04-28.zip