Re: Unit testing with asserts: Why is assertHandler required to throw?

[Lutger]

On 01/31/2010 09:39 PM, Trip Volpe wrote:
I recently began porting an existing C++ project of mine (a 
compiler/interpreter for a dynamic language) to D. In the process I found that 
the built-in unit testing support, while an awesome concept, was a little bit 
sparse. In particular, assert() is fairly useless for unit tests, since it 
throws on a failure and prevents all subsequent tests from running.
Unfortunately, it's the only facility that seems to give you access to the 
current file and line number, very important bits of info for unit tests in a 
any non-trivial project. I was contemplating just abandoning D and sticking 
with C++ and Googletest, but after doing a bit of digging through the source, I 
found setAssertHandler() in core.exception. Perfect! I could use my own assert 
handler that just records the error and allows future asserts to still be 
reached. So I wrote it up and gave it a try, but pretty quickly started getting 
access violation errors.

I did a bit of searching, and found a bug report on the issue:

http://d.puremagic.com/issues/show_bug.cgi?id=3208

In the comments it's indicated that this is intentional, that the compiler 
expects the assertion handler to throw. So I guess I have a few questions:

1. Why is this the expected behavior? It seems to me that there are relatively 
few useful things you can do with a custom assert handler unless it is possible 
to refrain from throwing.

2. Is this going to be fixed any time soon? It seems fairly important to me; it 
really kills the value of built-in unit tests to be forced to choose between 
not having any line number information and being able to recognize only one 
assert failure per test run(!!!).

3. Are there any current workarounds for the problem? I could just use my own assertion 
function, but 1) I can't call it "assert" because that's a reserved word, and 
2) most importantly, it won't have any way of indicating in its diagnostic output which 
source line the failure occurred on. In C++, of course, I could use preprocessor macros 
to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems 
like a case where D hasn't provided replacement functionality.

But maybe there's a sneaky fix I haven't thought of? I also found 
Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but 
provides an _almost_ useful improvement: I can catch the AssertErrors on a 
module-by-module basis, which would at least allow the unit tests for all 
modules to be run regardless of a failure in one of them. However, a single 
module could easily have dozens of tests in it, each of which could contain 
many individual asserts. Stopping the test early because one assert failed 
makes no sense.

It would be _almost_ acceptable to at least be sure of running all the tests in 
each module; is there any way to poke inside or otherwise override ModuleInfo's 
unitTest() function?


Thanks in advance for any advice or comments! D is a completely awesome 
language, but I found this issue a bit strange.


You can use line and file info with default arguments, this works 
(special case). I just hack around the default unittest system, 
something like this:

void test(string testName)(void delegate () testClosure,
                           int line = __LINE__,
                           string file = __FILE__)
{
    // register test start, run testClosure, end test
}

void expectEquals(A, B)(A a, B b,
                        int line = __LINE__,
                        string file = __FILE__)
{
    // record outcome of assertion
}

unittest
{
    test!"truth of it"( {
        expectEquals(true, true);
    } );
}

or slightly abusive:

unittest
{
    test!"truth of it" =  {
        expectEquals(true, true);
    };
}