On Monday 14 January 2008 15:42:49 Adam Kennedy wrote: > Test::Builder should just do what it's told. > > If the code that IS responsible for testing suites tells it to bail on > fail, that's exactly what it should do.
Let me rephrase this in an amusing way which demonstrates how silly this idea is. We have a program which runs suites of tests. It does this by spawning new processes which produce TAP somehow. Some of those processes produce TAP by using Perl modules which eventually use Test::Builder. Some do not. They don't have to, because there's a separation between producing TAP and consuming TAP, by design. This is why we have a program which runs suites of tests. Now some people say "If anything which generates TAP generates a TAP failure, I don't want to run the rest of the test suite." They have plenty of potential reasons, and those reasons are reasonable reasons. We do have a program which runs suites of tests. Actually, we can have several, because one of the goals of writing TAP::Parser which is now much of the guts of Test::Harness 3.0 is to allow people to write programs which run suites of TAP producers and consume and aggregate the resulting TAP. The point is customized behavior. That's a digression though, because as I understand it the goal of putting this OH NOES SOMETHING FALLDED!!! logic in Test::Builder is so that the currently running test file (necessarily Perl, necessarily using Test::Builder, so see the caveats about neither necessarily being true for TAP generators) can stop running when it encounters a failure. Of course, that won't stop a whole test suite from continuing to run. Dot dot dot. ... unless you *could* figure out some way of writing a temporary file somewhere that Test::Builder could check to see if it's supposed to abort subsequent test files in the suite (caveats again applying about how they have to be Perl programs which somehow use Test::Builder) and just not run the tests. That's a little tricky though, if you use parallel test runs, which has become one of the other goals of TAP::Parser, and I'm all for that. I tried to think of a way to use environment variables set by one test process to communicate with another process, but then you've limited this to working only in certain shells on Windows for Perl programs which somehow use Test::Builder. At least you can get away from the nasty file locking race condition autoflush problem of using temporary files to communicate "Hey I launched the process but ha ha just fooling not really why don't you die now thanks?" to child processes. It's not as if THAT test failed. Some other test did. Or, you know, stop launching new processes to generate TAP from the test suite runner, which itself knows a few things about launching new processes and is customizable, thanks to one of the goals of TAP::Parser. This is why making Test::Builder bail out the whole test suite is a Flies Through Space* solution. If you want to stop running the test suite, stop the thing that runs the test suite from running the test suite. Bridge on the captain, -- c * http://www.mail-archive.com/perl-qa@perl.org/msg07433.html
