Ovid wrote:
> From: Michael G Schwern <[EMAIL PROTECTED]>
>
>> I will now break your solution.
>>
>> use Test;
>> plan tests => 1;
>> ok 23, 42;
>>
>> PHEAR MY MAD HAX0R SKILLZ!
>>
>> :P
>
> Hmm, interesting. My parser handled that just fine.
Yes, that is interesting. How did it capture the diagnostics going to STDERR?
Wasn't that the whole problem, tests printing to both STDOUT and STDERR?
Test.pm does that. Are you sure it processed the diagnostics and it didn't
just pass straight through to the terminal and *look* like it processed it?
To illustrate the problem a little more directly, this is effectively what the
above is doing:
print "1..1\n";
print "not ok 1\n";
print STDERR "# Your shit is broke.\n";
How will TAPx::Parser deal with that? Test::Builder is not the only game in
town. You can't patch the world's tests for your convenience.
Here's another thing which will break:
use Test::More tests => 2;
is( "foo", "bar" );
TODO: {
local $TODO = "my shit is broke";
is(23, 42);
}
The user should not see the diagnostics from that TODO failure. If you smash
STDOUT and STDERR into one stream you have to guess at what diagnostics to
display and what to suppress. Guessing is a Bad Thing.
Another really bad bad no no no awful thing about this scheme is the parser is
changing the behavior of the tests. I run it in the parser and I get one
behavior. I run it outside the parser and I get another behavior. This
creates heisenbugs. This is a Very Bad Thing.
These are all Very Bad Things to be putting in so early in the design. They're
not solutions, they're hacks. The parser should not change the behavior of the
tests. The parser should have no knowledge of the internals of the tests. If
you're starting out by breaking these two your design is in trouble.
> The point isn't that the output comes before or after the test result. The
> point is that
> the parser should not merely discard the comments. The harness is free to
> discard those
> comments, which, in fact, is what Test::Harness does.
While I agree it would be nice, I don't agree that this an urgent OMG MUST FIX
WITH 80% HAX problem.
> Don't get me wrong, I understand your points about not relying on this output
> and I'm not
> trying to be difficult! The reason I have a sense of urgency about this is
> twofold:
>
> 1. I'm working on this *now* and I don't want to lose momentum.
> 2. So many people have contacted me to let me know they want to use
> TAPx::Parser.
#1, well, sometimes you work on something and find out it can't easily fixed.
That doesn't mean you pile on duct tape until it does. Sometimes you just have
to shelve it and come back later.
#2... How does not capturing STDERR stop people from using TAPx::Parser? If it
did we couldn't use Test::Harness. This is what I don't understand. Where's
the urgency here? I mean, its a nice feature but how is it a show stopper?
Finally, I'm really not convinced the sync'd two stream issue can't be solved
or that it hasn't been solved. Give it a little time. And again, has anyone
talked to the CPANPLUS folks? I remember Jos going through just this sort of
thing.
