On Nov 19, 2007, at 1:36 AM, Ovid wrote:
----- Original Message ----
From: Matisse Enzer <[EMAIL PROTECTED]>
This reminds me - I was wondering what it would take to implement a
"BAIL_ON_FAIL" approach to running a test suite
...
Should be fairly easy to implement with the new Test::Harness. What
syntax would be desired? Also, it seems to me that you've two
different cases. Have the entire suite BAIL_ON_FAIL or have an
individual test program halt on failure. The latter seems much more
useful to me.
I had not thought of limiting BAIL_ON_FAIL to a single test program,
can you say more about this?
Personally, I would set the entire test run to BAIL_ON_FAIL - I want
the whole test run to stop as soon as a single unexpected (non-TODO)
failure occurs.
As to the interface, it should be something that can be passed through
from Module::Build and/or MakeMaker,
make test BAIL_ON_FAIL=1
or
./Build test --bail_on_fail=1
maybe with an interface in Test::Harness like this:
# BAIL_OUT on 1st failure anywhere in suite
$harness->bail_on_fail( suite => 1);
$harness->bail_on_fail( suite => 3); # BAIL on 3rd failure
# bail on first failure in foo.t or 5th failure in bar.t
$harness->bail_on_fail( programs => { 'foo.t' => 1, 'bar.t' => 5 } );
-------------------------------------------------------
Matisse Enzer <[EMAIL PROTECTED]>
http://www.matisse.net/ - http://www.eigenstate.net/
