Re: done_testing()
David E. Wheeler wrote: > On Feb 21, 2009, at 5:44 PM, Michael G Schwern wrote: > >>> Yeah, I'm suggesting this more for a new version of TAP. >> >> It won't work because it's not backwards compatible. > > I care less and less about backwards compatibility every day. Also, > chromatic. There is Perl 5 style backwards compatibility where you never, ever break anything for years and years and years and even for code that you're not sure even exists. That's what chromatic is on about. And then there's backwards compatibility where you try not to clearly break everyone's well-written code! I think that second one has value. That said, just a couple days ago I sent along a proposal to the TAP list to change the way the TAP version works to allow incompatible changes to the protocol. Then we can work on sub-plans without having to bend over backwards to compatibility. PS The signature choice was entirely random. Eerie. -- The past has a vote, but not a veto. -- Mordecai M. Kaplan
Re: done_testing()
On Feb 21, 2009, at 5:44 PM, Michael G Schwern wrote: Yeah, I'm suggesting this more for a new version of TAP. It won't work because it's not backwards compatible. I care less and less about backwards compatibility every day. Also, chromatic. Technically "ok 1.1" should be read as an unnumbered test with the description of "1.1". TAP::Parser reads "ok 1.1" as test #1 with the description of ".1" because it doesn't require spaces. In Oslo we tried to make this one work and be backwards compatible but never quite got it to work. http://testanything.org/wiki/index.php/Test_Blocks I don't feel like dragging out the details at the moment. I don't think you can do it without breaking compatibility. Best, David
Re: done_testing()
David E. Wheeler wrote: > On Feb 21, 2009, at 11:35 AM, Michael G Schwern wrote: > >> David E. Wheeler wrote: >>> Dot notation? >>> >>> ok 1.1 >>> ok 1.2 >>> ok 2.1 >>> 1..2 >> >> If you don't want any existing TAP parser to be able to read it and delay >> release until they do, sure! >> >> I am totally not waiting for TAP to work out sub-plan syntax. > > Yeah, I'm suggesting this more for a new version of TAP. It won't work because it's not backwards compatible. Technically "ok 1.1" should be read as an unnumbered test with the description of "1.1". TAP::Parser reads "ok 1.1" as test #1 with the description of ".1" because it doesn't require spaces. In Oslo we tried to make this one work and be backwards compatible but never quite got it to work. http://testanything.org/wiki/index.php/Test_Blocks I don't feel like dragging out the details at the moment. -- s7ank: i want to be one of those guys that types "s/j&jd//.^$ueu*///djsls/sm." and it's a perl script that turns dog crap into gold.
Re: done_testing()
On Feb 21, 2009, at 11:35 AM, Michael G Schwern wrote: David E. Wheeler wrote: Dot notation? ok 1.1 ok 1.2 ok 2.1 1..2 If you don't want any existing TAP parser to be able to read it and delay release until they do, sure! I am totally not waiting for TAP to work out sub-plan syntax. Yeah, I'm suggesting this more for a new version of TAP. David
Re: done_testing()
David E. Wheeler wrote: > Dot notation? > > ok 1.1 > ok 1.2 > ok 2.1 > 1..2 If you don't want any existing TAP parser to be able to read it and delay release until they do, sure! I am totally not waiting for TAP to work out sub-plan syntax. -- 185. My name is not a killing word. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/
Re: Testing scripts with expected STDOUT and STDERR in external files
On Sat, Feb 21, 2009 at 5:24 AM, Gabor Szabo wrote:
>>> I wonder if there are modules out there that already do this?
>>> I could not find any that would fit my needs.
Test::Cmd? It's not built on Test::Builder but could be
adapted/wrapped. It uses shell redirection to capture output, which
should be OK as long as you don't need interactivity with the programs
being run.
I have something I wrote a long time ago for repetitive testing of a
command line program. I've considered putting it on CPAN but never
got around to writing much documentation for it. It works more or
less like this:
my $program = Test::CLI->new($path_to_program);
$program->stdin( $input );
$program->runs_ok( @args );
$program->stdout_like( qr/$expected/ );
$stderr -> $program->stderr; # for custom examination
Right now, it assumes the "program" is invoked via perl, but that
could be stripped out. It also supports default arguments that are
added to every run. So you might use it like this:
my $interp = Test::CLI->new($path_to_interpreter, @default_params);
for my $c (@cases) {
$interp->stdin( slurp $c->{in_file} );
$interp->runs_ok( $c->{utility} );
$interp->stdout_like( $c->{stdout} );
$interp->stderr_is( $c->{stderr} );
}
If you want, I'll stick it in some boilerplate and publish it to CPAN
and github and then you're welcome to document it or improve it. Just
let me know if that would be helpful.
Or, if it would need a lot of adaptation for what you're looking to
do, you're welcome to browse it and take whatever is useful. It's
t/CLI.pm in the Pod::WikiDoc repo:
http://github.com/dagolden/pod-wikidoc/
> Capture::Tiny is cute.
More than cute -- it works and works pretty universally for all
situations. I have a few more edge cases to code (e.g. if STDOUT or
STDERR are closed before it starts capturing), but my goal is to have
it replace the 20 or so other capturing modules out there. Then
things like Test::Output can be re-written around it and will become
much more powerful.
-- David
Re: done_testing()
Aristotle Pagaltzis wrote: > * Michael G Schwern [2009-02-20 23:35]: >> And we come back to the beginning: it's all going to be ad hoc >> anyway until TAP formalizes it. Fine for eyeballing. If someone >> wants to scrape the information out they can do it from the >> description (with the usual caveats about scraping). > > What I am saying is that just because they are… *wrinkles nose* > …scraping [yuck, I feel dirty] doesn’t mean we shouldn’t try to > make it as easy as possible. > > Something about cow paths… so do we want those established by the > scraping plebes or not? No, their stuff will just break when the formatting changes. Such is the death of all screen scraping. >> I plan on doing this: > > So you see no way of doing it as a test numbering scheme? I would > still prefer that… Let's say we do. How does this let you programmatically detect sub-plans? What you write: use Test::More; subplan(2, "First thing"); pass(); pass(); subplan(1, "Second thing"); pass(); What it might output: # subplan "First thing" ok 1 ok 2 # subplan "Second thing" ok 3 1..3 There's even LESS formal information there than if the subplan() calls are tests themselves. Worse than scraping test descriptions, you're scraping comments. -- Being faith-based doesn't trump reality. -- Bruce Sterling
Re: Testing scripts with expected STDOUT and STDERR in external files
On Fri, Feb 20, 2009 at 11:30 PM, David E. Wheeler wrote: > On Feb 20, 2009, at 1:23 PM, Gabor Szabo wrote: > >> I wonder if there are modules out there that already do this? >> I could not find any that would fit my needs. > > Test::Output? > > http://search.cpan.org/perldoc?Test::Output > > If it doesn't capture output from other programs, have a look at > Capture::Tiny. I looked at both Test::Output and Test::Trap but for both of them I need to do lots of other things. Capture::Tiny is cute. Maybe I need to put together something using some of those though I think I'll start just by putting what I have now in a module. What should I call though ? Test::Exec ? Test::Execute ? Test::External ? Maybe I should extend Text::Cmd or Text::Commands. Test::Commands ? Gabor
