unit testing - How to write runnable tests of static_assert?

Question

I am writing a unit-test suite for a source code library that contains static_asserts. I want to provide assurance these static_asserts do no more and no less than they are desired to do, in design terms. So I would like to be able to test them.

I could of course add uncompilable unit tests of the interface that cause the static asserts to be violated by a comprehensive variety of means, and comment or #if 0 them all out, with my personal assurance to the user that if any of them are un-commented then they will observe that the library does not compile.

But that would be rather ridiculous. Instead, I would like to have some apparatus that would, in the context of the unit-test suite, replace a static_assert with an equivalently provoked runtime exception, that the test framework could catch and report in effect: This code would have static_asserted in a real build.

Am I overlooking some glaring reason why this would be a daft idea?

If not, how might it be done? Macro apparatus is an obvious approach and I don't rule it out. But maybe also, and preferably, with a template specialization or SFINAE approach?

Answer 1

As I seem to be a lone crank in my interest in this question I have cranked out an answer for myself, with a header file essentially like this:

exceptionalized_static_assert.h

#ifndef TEST__EXCEPTIONALIZE_STATIC_ASSERT_H
#define TEST__EXCEPTIONALIZE_STATIC_ASSERT_H

/* Conditionally compilable apparatus for replacing `static_assert`
    with a runtime exception of type `exceptionalized_static_assert`
    within (portions of) a test suite.
*/
#if TEST__EXCEPTIONALIZE_STATIC_ASSERT == 1

#include <string>
#include <stdexcept>

namespace test {

struct exceptionalized_static_assert : std::logic_error
{
    exceptionalized_static_assert(char const *what)
    : std::logic_error(what){};
    virtual ~exceptionalized_static_assert() noexcept {}
};

template<bool Cond>
struct exceptionalize_static_assert;

template<>
struct exceptionalize_static_assert<true>
{
    explicit exceptionalize_static_assert(char const * reason) {
        (void)reason;
    }
};


template<>
struct exceptionalize_static_assert<false>
{
    explicit exceptionalize_static_assert(char const * reason) {
        std::string s("static_assert would fail with reason: ");
        s += reason;
        throw exceptionalized_static_assert(s.c_str());
    }
};

} // namespace test

// A macro redefinition of `static_assert`
#define static_assert(cond,gripe) 
    struct _1_test 
    : test::exceptionalize_static_assert<cond> 
    {   _1_test() : 
        test::exceptionalize_static_assert<cond>(gripe){}; 
    }; 
    _1_test _2_test

#endif // TEST__EXCEPTIONALIZE_STATIC_ASSERT == 1

#endif // EOF

This header is for inclusion only in a test suite, and then it will make visible the macro redefinition of static_assert visible only when the test suite is built with

`-DTEST__EXCEPTIONALIZE_STATIC_ASSERT=1`    

The use of this apparatus can be sketched with a toy template library:

my_template.h

#ifndef MY_TEMPLATE_H
#define MY_TEMPLATE_H

#include <type_traits>

template<typename T>
struct my_template
{
    static_assert(std::is_pod<T>::value,"T must be POD in my_template<T>");

    explicit my_template(T const & t = T())
    : _t(t){}
    // ...
    template<int U>
    static int increase(int i) {
        static_assert(U != 0,"I cannot be 0 in my_template<T>::increase<I>");
        return i + U;
    }
    template<int U>
    static constexpr int decrease(int i) {
        static_assert(U != 0,"I cannot be 0 in my_template<T>::decrease<I>");
        return i - U;
    }
    // ...
    T _t;
    // ...  
};

#endif // EOF

Try to imagine that the code is sufficiently large and complex that you cannot at the drop of a hat just survey it and pick out the static_asserts and satisfy yourself that you know why they are there and that they fulfil their design purposes. You put your trust in regression testing.

Here then is a toy regression test suite for my_template.h:

test.cpp

#include "exceptionalized_static_assert.h"
#include "my_template.h"
#include <iostream>

template<typename T, int I>
struct a_test_template
{
    a_test_template(){};
    my_template<T> _specimen;
    //...
    bool pass = true;
};

template<typename T, int I>
struct another_test_template
{
    another_test_template(int i) {
        my_template<T> specimen;
        auto j = specimen.template increase<I>(i);
        //...
        (void)j;
    }
    bool pass = true;
};

template<typename T, int I>
struct yet_another_test_template
{
    yet_another_test_template(int i) {
        my_template<T> specimen;
        auto j = specimen.template decrease<I>(i);
        //...
        (void)j;
    }
    bool pass = true;
};

using namespace std;

int main()
{
    unsigned tests = 0;
    unsigned passes = 0;

    cout << "Test: " << ++tests << endl;    
    a_test_template<int,0> t0;
    passes += t0.pass;
    cout << "Test: " << ++tests << endl;    
    another_test_template<int,1> t1(1);
    passes += t1.pass;
    cout << "Test: " << ++tests << endl;    
    yet_another_test_template<int,1> t2(1);
    passes += t2.pass;
#if TEST__EXCEPTIONALIZE_STATIC_ASSERT == 1
    try {
        // Cannot instantiate my_template<T> with non-POD T
        using type = a_test_template<int,0>;
        cout << "Test: " << ++tests << endl;
        a_test_template<type,0> specimen;

    }
    catch(test::exceptionalized_static_assert const & esa) {
        ++passes;
        cout << esa.what() << endl;
    }
    try {
        // Cannot call my_template<T>::increase<I> with I == 0
        cout << "Test: " << ++tests << endl;
        another_test_template<int,0>(1);
    }
    catch(test::exceptionalized_static_assert const & esa) {
        ++passes;
        cout << esa.what() << endl;
    }
    try {
        // Cannot call my_template<T>::decrease<I> with I == 0
        cout << "Test: " << ++tests << endl;
        yet_another_test_template<int,0>(1);
    }
    catch(test::exceptionalized_static_assert const & esa) {
        ++passes;
        cout << esa.what() << endl;
    }
#endif // TEST__EXCEPTIONALIZE_STATIC_ASSERT == 1
    cout << "Passed " << passes << " out of " << tests << " tests" << endl;
    cout << (passes == tests ? "*** Success :)" : "*** Failure :(") << endl; 
    return 0;
}

// EOF

You can compile test.cpp with at least gcc 6.1, clang 3.8 and option -std=c++14, or VC++ 19.10.24631.0 and option /std:c++latest. Do so first without defining TEST__EXCEPTIONALIZE_STATIC_ASSERT (or defining it = 0). Then run and the the output should be:

Test: 1
Test: 2
Test: 3
Passed 3 out of 3 tests
*** Success :)

If you then repeat, but compile with -DTEST__EXCEPTIONALIZE_STATIC_ASSERT=1,

Test: 1
Test: 2
Test: 3
Test: 4
static_assert would fail with reason: T must be POD in my_template<T>
Test: 5
static_assert would fail with reason: I cannot be 0 in my_template<T>::increase<I>
Test: 6
static_assert would fail with reason: I cannot be 0 in my_template<T>::decrease<I>
Passed 6 out of 6 tests
*** Success :)

Clearly the repetitious coding of try/catch blocks in the static-assert test cases is tedious, but in the setting of a real and respectable unit-test framework one would expect it to package exception-testing apparatus to generate such stuff out of your sight. In googletest, for example, you are able to write the like of:

TYPED_TEST(t_my_template,insist_non_zero_increase)
{
    ASSERT_THROW(TypeParam::template increase<0>(1),
        exceptionalized_static_assert);
}

Now I can get back to my calculations of the date of Armageddon :)