On Thursday 15 February 2007, Ovid wrote:
> --- Shlomi Fish <[EMAIL PROTECTED]> wrote:
> > > Test::Run works fine with TAPx-Parser 0.41, but it breaks with the
> > > TAPx-Parser from the trunk:
>
> I've figured it out and that's because TAPx::Parser 0.41 was buggy and
> the latest TAPx::Parser has found a point where your code differs in
> interpretation from Test::Harness.
I find it unlikely that my code differs in interpreation from Test::Harness in
this regard. After all, my code is derived from Test::Harness, and I don't
recall changing the tests inside t/test-harness.t. Furthermore, the results
specification for the test for 'bignum' is the same in both places.
>
> > Replying to myself I'd like to note that from my analysis at least
> > the
> > bignum test failed because of the following code in the new
> > TAPx::Parser:
> >
> > <<<<<<<<<
> > sub is_ok {
> > my $self = shift;
> >
> > return if $self->is_unplanned; # <-----------
> >
> > # TODO directives reverse the sense of a test.
> > return $self->has_todo ? 1 : $self->ok !~ /not/;
> > }
>
> You noted that the 'bignum' test failed. Well, let's look at that one:
>
> print <<DUMMY;
> 1..2
> ok 1
> ok 2
> ok 136211425
> ok 136211426
> DUMMY
>
> If you try and run that through 'prove', you'll get a strange "Can't
> detailize, too big" error, so let's keep it simple, with normal tests
> but a bad plan:
>
> $ cat badplan
> print <<DUMMY;
> 1..2
> ok 1
> ok 2
> ok 3
> ok 4
> DUMMY
>
> And then use 'prove':
>
> $ prove badplan
> badplan...FAILED tests 3-4
>
> Failed 2/2 tests, 0.00% okay
> Failed Test Stat Wstat Total Fail List of Failed
> ------------------------------------------------------------
> badplan 2 2 3-4
> Failed 1/1 test scripts. -2/2 subtests failed.
> Files=1, Tests=2, 0 wallclock secs ( 0.01 cusr + 0.01 csys = 0.02
> CPU)
> Failed 1/1 test programs. -2/2 subtests failed.
>
> Hmm, 2/2 tests failed with 0.00% ok? Later we get -2/2 subtests
> failed?
I'm getting:
<<<<<<<<<<<<<<
shlomi:$trunk/modules/Test-Shlomif-Harness$ prove badplan
badplan...FAILED tests 3-4
Failed 2/2 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
badplan 2 2 100.00% 3-4
Failed 1/1 test scripts, 0.00% okay. -2/2 subtests failed, 200.00% okay.
>>>>>>>>>>>>>>
And with runprove (the Test::Run-based prove equivalent):
<<<<<<<<<<<<
badplan...FAILED test 3-4
Failed 2/2 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
badplan 2 2 100.00% 3-4
Failed 1/1 test scripts, 0.00% okay. -2/2 subtests failed, 200.00% okay.
>>>>>>>>>>>>
Which seem exactly identical to me.
>
> What does 'runtests' say? ('runtests' will handle the 'bignum' test
> above, too):
>
> $ runtests badplan
> badplan.... All 2 subtests passed
>
> Test Summary Report
> -------------------
> badplan (Wstat: 0 Tests: 4 Failed: 2)
> Failed tests: 3-4
> Parse errors: Bad plan. You planned 2 tests but ran 4.
> Files=1, Tests=4, 0 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00
> CPU)
>
> That's much clearer. The problem happens because you've run two more
> tests than you planned, so the third and fourth tests are 'unplanned'.
> As a result, the harness has no way of knowing if you have a bad plan
> or if you have spurious tests being run. Hence, running more tests
> than you have planned is an error.
>
> Note that 'is_unplanned' is documented to always return false with a
> trailing plan because you can't know until you reach the plan (or
> never, with an infinite stream) if any tests were unplanned.
>
OK.
> I'm also about to commit more tests to guarantee this behavior since it
> is documented but not tested well enough.
>
> I hope that explains things.
>
It does, but I still don't know how to emulate the Test::Harness behaviour
using the new TAPx::Parser API. And furthermore, I'm not sure I'll be able to
rely on the new API until a stable version of TAPx::Parser 0.5x is released,
in which case, I'll have to require this particular version in my "requires"
field. (Just as I'm requiring version 0.33 now).
Regards,
Shlomi Fish
---------------------------------------------------------------------
Shlomi Fish [EMAIL PROTECTED]
Homepage: http://www.shlomifish.org/
Chuck Norris wrote a complete Perl 6 implementation in a day but then
destroyed all evidence with his bare hands, so no one will know his secrets.
