The following line is giving me pause:
ok 9 Elegy 9B # TOdO
That's an 'unexpectedly succeeded' test ('bonus', in the Test::Harness world).
Right now, if that read 'not ok # TODO', TAPx::Parser would have this:
passed true
actual_passed false
todo_failed false
But since it reads 'ok', I've reversed the sense:
passed false
actual_passed true
todo_failed true
In this case, Test::Harness and friends report that 'ok 9 # todo' is passing,
not failing, but I'm reporting the opposite result. I think my behavior is
more correct because I'm trying to write things so that someone who forgets
writes a bad harness will still see what's going on. For example, let's say
someone only wants to see failing tests:
# the 'source' key is new. You no longer have to manually create a stream
my $parser = TAPx::Parser->new( { source => $test_file } );
while ( my $result = $parser->next ) {
print $result->as_string if ! $parser->passed;
}
With that, they'll never notice that 'ok # TODO' has unexpectedly succeeded if
I adopt the current behavior of Test::Harness unless they explicitly remember
to check the $parser->todo_failed method.
I propose that 'ok # TODO' tests be reported as failures. I think it would be
good if Test::Harness also adopted this strategy because right now, if you see
that tests unexpectedly succeeded, you don't know which tests they are and you
have to try and grep through the TAP output manually.
Thoughts? Is this going to break a lot of stuff (I suspect it might).
Cheers,
Ovid
--
Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
