Eric Wilhelm wrote:
# from Geoffrey Young
# on Tuesday 15 January 2008 11:49:
what I think many are asking for is for Test::Builder to be able to
issue the appropriate TAP on $condition.
Where $condition is a run-time option set at the command-line, not
hardcoded into the test.
Writing "or BAIL_OUT(...)" in the test is a different thing than
saying "I'm going to start a test run, but I want it to bail if
anything fails".
The former is in effect for both the developer and the end-user/
installer. One typically wants a full report from an end-user, so you
don't want to ship it with bail everywhere. This is not a command-line
conditional bail, but rather "the module won't even load for some
reason and thus testing anything else is pointless".
The latter is for developer convenience while adding some tests or
changing some code. The error is often an obvious typo or something
that can be fixed with only the data from the first fail. This is not
really a "bail out" so much as a "ok, I get it. Everything should stop
now."
Well, possibly.
The bail outs you'd keep long term are in big test suites where you have
tests that run early in the order (assuming non-shuffle) that suffer a
failure that violates the integrity of the test suite itself.
For example, if your compile test can't compile your mail module, then
running the rest of the test suite is normally pointless.
If the environment of configuration test can't locate configuration
resources that MUST exist to run sanely (configuration management is a
big deal in very large applications) then just forget about running.
Or if the configuration resources exist, but the tests for your
error-checking configuration module find that the config is invalid, etc
etc.
Adam K
