On 9/14/06, Ovid <[EMAIL PROTECTED]> wrote:
Here's we have a test where the plan is 1..3 but we've run 7 tests.TAPx-Parser $ prove -v t/sample-tests/too_many t/sample-tests/too_many...1..3 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 dubious Test returned status 4 (wstat 1024, 0x400) DIED. FAILED tests 4-7 Failed 4/3 tests, -33.33% okay Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- t/sample-tests/too_many 4 1024 3 4 4-7 Failed 1/1 test scripts. -4/3 subtests failed. Files=1, Tests=3, 0 wallclock secs ( 0.01 cusr + 0.01 csys = 0.02 CPU) Failed 1/1 test programs. -4/3 subtests failed. The last three tests have passed, but Test::Harness says they've failed. My TAPx::Parser reports that they've passed and the only real way to know if there's a problem is to test the $parser->good_plan method. I've added this as a parse error, but why are the passing tests listed as failing?
Well, I dont know that I can say it authoritatively, but treating a passing test you have specifically said wasnt going to occur as a fail seems to reasonable behaviour to me. How else are you going to deal with "you've run more tests than you said you were"? If you accept the results and assume the count is wrong how do you know what the program didn't silently die part way through and in fact you are dealing with a catastrophic failure in the middle of more tests than were run? But a nice message like "you ran 128 tests, but you said you were going to run 123 tests, probably you should change your test count" would be useful. (And if i recall is produced by Test::Harness?) Cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
