Re: getting more details from a test script
Ovid writes: > If it's the former, we need structured diagnostics in TAP to be > formalised and implemented. Which we already have. It does not need to be formalised further; it's all there. I use TAP v13 with embedded yaml and lots of diagnostics information in there. If I want to evaluate it I use TAP::DOM and search through it using Data::DPath or App::DPath. Kind regards, Steffen -- Steffen Schwigon
Re: getting more details from a test script
On Mon, Apr 5, 2010 at 1:58 PM, Joe McMahon wrote:
> On Mon, Apr 5, 2010 at 1:34 AM, Gabor Szabo wrote:
>> I could write this:
>>
>> $mech->content_like(qr{regex}) or diag $mech->content;
>>
>> but then I get all the content in the TAP stream making it quite unreadable.
> Yep. That's why we implemented the snapshot plugin for
> WWW::Mechanize::Pluggable. It automatically takes snapshots to a
> directory you set up at the start of the test script and diags() out a
> message saying where the snapshot is.
>
> It would be possible to set this up so that the snapshots were enabled
> only if (say) and environment variable was set.
>
I'll take a look at this.
Gabor
Re: getting more details from a test script
On Mon, Apr 5, 2010 at 12:17 PM, Ovid wrote:
> --- On Mon, 5/4/10, Gabor Szabo wrote:
>
>> From: Gabor Szabo
>
>> Maybe I need something like this:
>>
>> $mech->content_like(qr{regex}) or do {
>> my $filename = 'some_filename';
>> if (open my $fh, '>', $filename) {
>> print $fh $mech->content;
>> diag "File: $filename";
>> }
>> };
>>
>> and then parse the TAP output for 'File:' *after* a test
>> failure.
>>
>> Is there a better way to do this?
>
> The problem, I think, is that everyone wants subtly different things from
> tests outside of ok/not ok. The question I'm wondering is what you mean by
> "this" in "is there a better way to do this?".
hmm, looking at again I think I used 'this' to mean that I don't know
what needs to be improved.
>
> Are you wanting a better way of presenting the filename to test
> authors/runners? Are you wanting a better way to store the file contents?
>
> If it's the former, we need structured diagnostics in TAP to be formalised
> and implemented. If it's the latter, I would recommend writing your own
> "output to file" function and then instead of using "Test::More" and your own
> test utilities, bundle all of them with Test::Kit so you can just do this:
>
> use My::Custom::Test::More tests => $test_count;
>
> The advantage here is that you have your own custom test behaviours nicely
> controlled by one module and if you need to change them, you can do so in one
> spot.
I certainly don't want to repeated the do { ... } part for every call
that needs saving and as a
second thought maybe it would be better and simpler to save the
content of the web page
every time, regardless of success or failure and then let the user
drill down if she wants but I'd
also prefer an already formalized way to communicate the filenames so
I don't need to invent
my own way.
> Or maybe you meant something else by "this" entirely :)
Yeah, possibly.
Maybe I also meant that I'd like a more general solution that would work for
other Test::* modules as well and not only Test::WWW::Mechanize but I
am not sure
any more :-)
Maybe all that I need here is a callsave_data($var, 'title')
that would save the content of $var to a newly generated file and print
a diag("File: 'title' path/to/file"). The test runner would then collect the
files and zip them together with the TAP output. The tool that displays
them (e.g. Smolder) would then be able to add links to these files.
This save_data would work similarly to explain an recognize when $var is a
reference and call Dumper on it before saving.
Gabor
Re: getting more details from a test script
--- On Mon, 5/4/10, Gabor Szabo wrote:
> From: Gabor Szabo
> Maybe I need something like this:
>
> $mech->content_like(qr{regex}) or do {
> my $filename = 'some_filename';
> if (open my $fh, '>', $filename) {
> print $fh $mech->content;
> diag "File: $filename";
> }
> };
>
> and then parse the TAP output for 'File:' *after* a test
> failure.
>
> Is there a better way to do this?
The problem, I think, is that everyone wants subtly different things from tests
outside of ok/not ok. The question I'm wondering is what you mean by "this" in
"is there a better way to do this?".
Are you wanting a better way of presenting the filename to test
authors/runners? Are you wanting a better way to store the file contents?
If it's the former, we need structured diagnostics in TAP to be formalised and
implemented. If it's the latter, I would recommend writing your own "output to
file" function and then instead of using "Test::More" and your own test
utilities, bundle all of them with Test::Kit so you can just do this:
use My::Custom::Test::More tests => $test_count;
The advantage here is that you have your own custom test behaviours nicely
controlled by one module and if you need to change them, you can do so in one
spot.
Or maybe you meant something else by "this" entirely :)
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://blogs.perl.org/users/ovid/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
getting more details from a test script
Hi,
when I am writing a test script for a perl module or even
an application I am writing, the output I get from TAP seems enough.
When I write a test for an application where the author is someone
else and sometime the person running the test is a 3rd one, in those
case we usually need more details to drill down.
It is not unique to it but let me give an example with Test::WWW::Mechanize.
When this call fails
$mech->content_like(qr{regex});
it gives me a part of the content of the page. That's
ok in the TAP stream but I'd like to be able to save
the actual content and later let the user see it.
I could write this:
$mech->content_like(qr{regex}) or diag $mech->content;
but then I get all the content in the TAP stream making it quite unreadable.
Maybe I need something like this:
$mech->content_like(qr{regex}) or do {
my $filename = 'some_filename';
if (open my $fh, '>', $filename) {
print $fh $mech->content;
diag "File: $filename";
}
};
and then parse the TAP output for 'File:' *after* a test failure.
Is there a better way to do this?
regards
Gabor
--
Gabor Szabo
http://szabgab.com/
