Re: Proposal Test::TAPv13
> > From: Michael G Schwern >I think we don't have to guess at that. Structured diagnostics are only >associated with test results (and possibly start/end of test info). They're >not used like diag() is now as both a warning mechanism and a test result >information system. Just a minor comment: yes, we also would want structured diagnostics at the start and end of each test (and subtest). There are numerous reasons. Anyone who uses Test::Class extensively would be comfortable with this when they thing about startup/setup/teardown/shutdown test control methods. Cheers, Ovid -- Live and work overseas - http://www.overseas-exile.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter- http://twitter.com/OvidPerl/
Re: Proposal Test::TAPv13
On 2012.7.11 2:39 PM, Daniel Perrett wrote:
> I may have missed the point here, but would it be sufficient to have
> two flavours of diagnostic calls, distinguishing between pre-test
> diagnostics (`forewarn()`) and post-test diagnostics (`reflect()`), or
> is the problem that ok() must be a single function call?
Rather than tell you what I think the solution should be, let me frame the
problem. I'm going to use a fictional XML format for illustration, but the
problem applies to a broad class of formats which are different than TAP.
Consider this test...
#line 12
ok( 0, "something" );
and the output in our fictional XML format might look like this...
fail
something
12
The important thing to note is that the result has a clear start and end.
Now, if all information about a result has to go inside the tag, how
does this code...
#line 12
ok( 0, "something" );
info( { have => "this", want => "that" } );
Know to produce this output...
fail
something
12
this
that
...and yet still work when there is no tap_diag() call after the ok()?
There are a number of alternative interfaces to solve the problem. Making
each assert a single function call is the "best" from the point of view of
ease of formatting, but requires every assert to take an extra argument. I
feel this is getting bulky from an interface perspective, but maybe it's ok.
ok( 0, "something", {
have => "this", want => "that"
});
Another one would be to turn results into objects and only format when they go
out of scope...
ok( 0, "something" )->info({ have => "this", want => "that" });
...but that goes pear shaped if somebody does this.
$results{$test} = is( $this, $that );
You could do some clever tricks where the formatter waits until the next
result to display a result, in case extra info comes in.
# store the result.
ok( 0, "something" );
# attach this to the previous result.
info( { have => "this", want => "that" } );
# format and display the previous result and info.
# store this result.
ok( 0, "something else" );
# format and display the previous result with no extra info.
# store this result.
ok( 0, "something more" );
# format and display the previous result.
# end the test.
done_testing();
...but that significantly complicates the formatters and puts streamed output
on a delay. Very confusing if you're stepping through the code.
Right now the "all in one" version seems best. It's explicit. It's simple.
> If that's possible, maybe we can extend it to guess at what the
> various messages mean: most diags can be assumed to be of the
> `reflect()` type, as that's typical behaviour, e.g. Test::More, while
> more perlish warnings is most likely to be of the `forewarn()`
> variety.
I think we don't have to guess at that. Structured diagnostics are only
associated with test results (and possibly start/end of test info). They're
not used like diag() is now as both a warning mechanism and a test result
information system.
Since this is a new interface, we can deal with that by making different
functions for each different intent. One for "this goes with the previous
result". One for "this is general test information". And one for "this is
just a diagnostic dump for and not associated with anything".
--
You know what the chain of command is? It's the chain I go get and beat you
with 'til you understand who's in ruttin' command here.
-- Jayne Cobb, "Firefly"
Re: Proposal Test::TAPv13
I may have missed the point here, but would it be sufficient to have
two flavours of diagnostic calls, distinguishing between pre-test
diagnostics (`forewarn()`) and post-test diagnostics (`reflect()`), or
is the problem that ok() must be a single function call?
If that's possible, maybe we can extend it to guess at what the
various messages mean: most diags can be assumed to be of the
`reflect()` type, as that's typical behaviour, e.g. Test::More, while
more perlish warnings is most likely to be of the `forewarn()`
variety.
Daniel
On 10 July 2012 19:11, Michael G Schwern wrote:
> On 2012.6.1 5:40 AM, Steffen Schwigon wrote:
>> I am about to upload Test::TAPv13. I also did an prepan entry for that.
>>
>> PrePAN: http://prepan.org/module/429En4oFbn .
>> URL: https://github.com/renormalist/Test-TAPv13
>> Synopsis:
>>
>>use Test::TAPv13 ':all'; # must come before Test::More
>>use Test::More tests => 2;
>>
>>my $data = { affe => { tiger => 111,
>> birne => "amazing",
>> loewe => [ qw( 1 two three) ],
>> },
>> zomtec => "here's another one",
>> "DrWho" => undef,
>> };
>>
>>ok(1, "hot stuff");
>>tap13_yaml($data);
>>tap13_pragma "+strict";
>>ok(1, "more hot stuff");
>>
>> Does it make sense?
>> Did I overlook an existing way to generate TAP v13?
>
> I'm just seeing this now. The output looks correct and I'm happy to see
> somebody playing with the structured diagnostics for reals! There's a few
> code nits, but I'll note them on github.
>
> Test::Builder1.5 can generate TAP v13, in fact it does so by default, but is
> currently lacking the structured diagnostics part. Part of this is just
> time/effort, but the larger part is with how tests are written...
>
> The problem with...
>
> ok( ... );
> diagnostics( ... );
>
> Is that there's no reliable way to know that diagnostics() goes with the
> previous ok(). This is ok for TAP, where the test result and diagnostics are
> on separate lines and can be printed separately, but other formats need to
> print out the results and diagnostics together. Like anything where the
> diagnostics information is inside a tag, for example.
>
> That pattern has to be rethought. This was one of the goals of Test::Builder2
> (the new class to replace Test::Builder) but that is on hold. Any thoughts?
>
> Also, when did we add the pragma thing?
>
>
> --
> The interface should be as clean as newly fallen snow and its behavior
> as explicit as Japanese eel porn.
Re: Proposal Test::TAPv13
On 2012.6.1 5:40 AM, Steffen Schwigon wrote:
> I am about to upload Test::TAPv13. I also did an prepan entry for that.
>
> PrePAN: http://prepan.org/module/429En4oFbn .
> URL: https://github.com/renormalist/Test-TAPv13
> Synopsis:
>
>use Test::TAPv13 ':all'; # must come before Test::More
>use Test::More tests => 2;
>
>my $data = { affe => { tiger => 111,
> birne => "amazing",
> loewe => [ qw( 1 two three) ],
> },
> zomtec => "here's another one",
> "DrWho" => undef,
> };
>
>ok(1, "hot stuff");
>tap13_yaml($data);
>tap13_pragma "+strict";
>ok(1, "more hot stuff");
>
> Does it make sense?
> Did I overlook an existing way to generate TAP v13?
I'm just seeing this now. The output looks correct and I'm happy to see
somebody playing with the structured diagnostics for reals! There's a few
code nits, but I'll note them on github.
Test::Builder1.5 can generate TAP v13, in fact it does so by default, but is
currently lacking the structured diagnostics part. Part of this is just
time/effort, but the larger part is with how tests are written...
The problem with...
ok( ... );
diagnostics( ... );
Is that there's no reliable way to know that diagnostics() goes with the
previous ok(). This is ok for TAP, where the test result and diagnostics are
on separate lines and can be printed separately, but other formats need to
print out the results and diagnostics together. Like anything where the
diagnostics information is inside a tag, for example.
That pattern has to be rethought. This was one of the goals of Test::Builder2
(the new class to replace Test::Builder) but that is on hold. Any thoughts?
Also, when did we add the pragma thing?
--
The interface should be as clean as newly fallen snow and its behavior
as explicit as Japanese eel porn.
Proposal Test::TAPv13
Hi!
I am about to upload Test::TAPv13. I also did an prepan entry for that.
PrePAN: http://prepan.org/module/429En4oFbn .
URL: https://github.com/renormalist/Test-TAPv13
Synopsis:
use Test::TAPv13 ':all'; # must come before Test::More
use Test::More tests => 2;
my $data = { affe => { tiger => 111,
birne => "amazing",
loewe => [ qw( 1 two three) ],
},
zomtec => "here's another one",
"DrWho" => undef,
};
ok(1, "hot stuff");
tap13_yaml($data);
tap13_pragma "+strict";
ok(1, "more hot stuff");
Does it make sense?
Did I overlook an existing way to generate TAP v13?
Thanks!
Kind regards,
Steffen
--
Steffen Schwigon
Dresden Perl Mongers
