Re: C/C++ White-Box Unit Testing and Test::More

[Adrian Howard]

On 24 Jun 2004, at 19:59, Andrew Pimlott wrote:
[snip]
- You don't have much control (correct me if I'm wrong) about the order
  of tests, or the relationship between tests, eg you can't say "if 
this
  test fails, skip these others".  This is straightforward in
  Test::More's simple procedural style.
[snip]
This is probably due to the same test/assertion confusion - but the 
whole /point/ of xUnit is that the test order shouldn't matter :-)

Test isolation == good!
If I need isolation, why can't I just ask for it directly?
    with_test_setup {
        run_tests();
    }
You can.
But then the test writer/reader has to code/understand the 
with_test_setup() code for each test script, and the structure of each 
of those routines is pretty much the same.

So you might decide to generalise it and have a standard mechanism for 
you to plug in your start up code.

Then you discover that some of your tests need to clean up after 
themselves so you add another slot to run after the tests.

Then you might find that you often have common sets of tests that are 
run in different situations, so it would be nice if you could package 
them up and use them as a unit. So you add some infrastructure to 
support it.

Well done - you've implemented an xUnit framework!
(or rather provided the few elements of an xUnit framework not already 
supplied by Test::Harness, Test::Builder and friends).

These sorts of requirements are exactly what made me build Test::Class 
- adding the bits of xUnit not already in the Perl testing framework.

I didn't start out to write an xUnit framework. What I did was refactor 
several large and slow test suites and the very xUnit-ish Test::Class 
just fell out.

I don't really think of xUnit as a competitor to Perl's normal testing 
infrastructure - it's more of a superset. The fact that it's so easy to 
layer the missing bits of xUnit on top of Perl's standard mechanisms 
shows that.

[snip]
Even better would be to put Test::Builder in "skip" mode, where it 
skips
automatically whenever a test fails:

    skip_mode {
        ok(something);
        is(this, that);
    }
sub skip_mode (&) {
    my $test_sub = shift;
    my $old_ok = \&Test::Builder::ok;
    my $test_passed=1;
    local $Test::Builder::Level = $Test::Builder::Level + 1;
    no warnings;
    local *Test::Builder::ok = sub {
        die unless $test_passed = $old_ok->(@_);
    };
    eval { $test_sub->() };
    die $@ if $@ && $test_passed;
};
:-)
[snip]
Every time I hear about xUnit, I figure there must be something other
than "setup and teardown" in its favor.  If that's all there is, I'm 
not
sold.
You have to remember that xUnit isn't just setup/teardown routines. 
It's all the rest too - assertions, a test runner, test suites, OO 
reuse framework, etc. Most of which we already have with Test::Harness, 
Test:Builder and friends.

When people speak about the advantages of xUnit, they're mostly talking 
about the advantages of having a standard testing infrastructure.

Cheers,
Adrian