On Wednesday 25 June 2003 20:15, Adrian Howard wrote:Add an explicit "test script finished" footer?
But how does the footer-adder know that the correct number of tests ran. You
would need to declare a plan to run x additional extensions at which point
you're doing sub-plans.
The footer doesn't indicates that the correct number of tests ran (that's the plan's job), it just shows that a test script completed without error.
So a noplan test might look like
ok 1 ok 2 1..2 # fin
a planned test with an extension might look like
1..2 ok 1 ok 2 3..3 ok 3 # fin
and a test with a failed extension might look like
1..2
ok 1
ok 2
# some ghastly error that caused an early exit and no error code on exit
and Test::Harness would pick up the lack of a "# fin" as indicating a premature exit. Most early exits would be caught via the bogus exit value anyway. Having an explicit "script complete" footer would only catch a few odd cases.
Of course if we carry on much longer we're going to end up with T::H and T::B having to do nasty version checks to make sure they're talking the same protocol :-)
I suppose I'm thinking of things from the I messed up my test script point of
view rather than my test script died unexpectedly. Exit codes should catch
the died unexpectedly stuff. A correct plan is good for catching the
programmer errors but is a pain to maintain.
To be honest, I'm not sure that having a global plan at the test script level gives you that much benefit.
I like planned tests, but a manually maintained global plan is a maintainence nightmare if you have a large number of tests, or if the number of tests is runtime dependant.
I've not noticed any problems now that I use Test::Class for most of my test code. I still plan the number of tests executed - I just do it at the method-level rather than the test-script level.
That's why I liked the previous suggestions about being able to extend plans. It seemed a nice simple way to get the security of being able to have micro-plans without having to worry about the global plan count.
(not that we couldn't do it your way too).
I have some vague thoughts that there are some potentially cute things
that you could do with extended plans that would be hard to do with
sub-blocks/plans (e.g. any sort of test that cross-cuts several test
scripts in an AOPish sort of way - checking that test data has returned
to a state of grace for example).
I guess that can be solved by controlling the timing of the output. I don't
think there's a need for the results of test 1 to be printed before carrying
out test 2, except maybe when the tests are being run by the developer.
That would work, but you face the problem of having an evil internal-error causing your script to exit early without getting any output from earlier tests. You also lose user feedback for long test sequences (you only find out that test 2 failed after all 276000 test run).
Being able to extend the plan seems a simpler solution to me.
Adrian
