Re: a safer way to use no_plan?
Eric Wilhelm wrote:
At the bottom of a test file:
{my $finish = 1; END {$finish or die \n unplanned exit}};
Yeah, you have to remember to put it at the end of the file, but it may
be easier than counting tests. Thoughts? Maybe an 'until_done'
directive and 'tests_done' function? Ways to check that there is no
code after tests_done()?
*sigh* Nobody ever looks at the ticket tracker.
http://rt.cpan.org/Public/Bug/Display.html?id=20959
Sun Nov 05 13:07:03 2006 MSCHWERN - Correspondence added
I think I like this. I'm just always hesitant to add more functions to
Test::More so I need some convincing on the name.
Yes, this sort of thing is a good idea and there's already a patch for it.
The only real issue is the naming of the plan type and the I'm done
function. Discuss.
Additionally, the patch supplied a plan_at_end where you can supply a
fixed number but don't have to do it before you run any tests. This is
handy for those tricky situations where the number of tests run is
non-deterministic.
Finally, I wouldn't mind an easy way to say I'm going to run two more
tests to the plan_at_end variant.
Re: a safer way to use no_plan?
Andy Lester wrote: On Mar 3, 2007, at 8:15 PM, Eric Wilhelm wrote: Yeah, you have to remember to put it at the end of the file, but it may be easier than counting tests. Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! I don't know how many times I've used that approach and shipped code with the wrong count in the SKIP blocks. See, they don't get run on my machine so your empirical approach does not work. Followed closely by a bug report from a Windows user because the test count is wrong, assurances from me that there's not a real bug and everything's ok and then a new release the next day. Bleh, such work and flapping around. All of this flapping about how maybe we should come up with some mechanisms to do test counting and blah blah blah is like buying a computer to store your recipes on instead of grandma's 3x5 index cards. They ain't sexy but they WORK. If the computer is free and easy to use then its sensible. Test::Builder is free, we're just making it a little easier to use. Its O(1) vs O(n). We do the work once to make no plan safer, everybody benefits for happily ever after. Your argument sounds like the things I had to deal with when I introduced Test::More. Why should we load a module with all these functions when I can just write 'print ...my test... ? ok 1\n : not ok 1\n;'? Anyhow, not to argue by fiat, but I like the no plan concept and I prefer providing flexibility in a test framework and I control Test::Builder so no plan is here to stay. Patches welcome.
Re: a safer way to use no_plan?
Dominique Quatravaux wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! And what if you are running a variable number of tests depending on stuff such as compile-time options, maintainer mode enabled or not, or whatever? What about it? Go look at the PPI regression test scripts, those have a block at the top before a plan is declared that look for matching pairs of .code and .dump files, multiply it by 7 or something and add 5 and that's the number of tests. If anything goes wrong, it dies, the test ends without a plan declaration and the test script is thus seen to have failed. There's no reason that you can't calculate the number of tests in advance in cases where skip doesn't quite get you enough functionality. Adam K
Re: a safer way to use no_plan?
On Sunday 04 March 2007, chromatic wrote:
On Saturday 03 March 2007 18:18, Andy Lester wrote:
Good Lord do I get frustrated at the handwringing over test
counting. Look, it's simple. You write your tests. You run it
through prove. You see how many tests it reports. You add it at the
top of the file. Voila!
But Andy, what if I modify the test file? I don't want to have to modify
the test file to tell my test harness that I've modified the test file!
If only we had something that flies through space
My solution to this is:
http://search.cpan.org/dist/Test-Count/
It's not a perfect solution but it Works For Me.tm
What I do is add comments in a small domain-specific language that allow me to
count how many tests were added. These comments are usually placed close to
the tests themselves, so they can be updated at the same time. Examples for
such comments are:
# TEST
is ($mystring, Hello, String is Hello);
# TEST*3
for my $i (0 .. 2)
{
ok ()
}
Regards,
Shlomi Fish
-
Shlomi Fish [EMAIL PROTECTED]
Homepage:http://www.shlomifish.org/
Chuck Norris wrote a complete Perl 6 implementation in a day but then
destroyed all evidence with his bare hands, so no one will know his secrets.
Re: a safer way to use no_plan?
On 4 Mar 2007, at 03:40, Matisse Enzer wrote: A tangential comment: The xUnit approach avoids this question of counting altogether - you create one or more subroutines whose names begin with 'test', each of which contain one or more assertions such as is_deeply($got, $expected); and the test runner runs all the test* methods. It counts the number of test* methods for you. See Test::Class Test::Group :-) Adrian
Re: a safer way to use no_plan?
On Sunday 04 March 2007 09:40, Shlomi Fish wrote:
My solution to this is:
...
What I do is add comments in a small domain-specific language ...
...
# TEST
is ($mystring, Hello, String is Hello);
# TEST*3
for my $i (0 .. 2)
{
ok ()
}
Wouldn't Test::Block be better than a home brewed system? Test::Block is not
perfect either (but what module is?); I specially miss the possibility to
have blocks inside block.
My 2 cents, Nadim.
Re: a safer way to use no_plan?
On Sunday 04 March 2007 09:40, Shlomi Fish wrote: My solution to this is: Of late, one solution I've used is just Ctrl-A and Ctrl-X in Vim, with some shortcuts: map silent ,at m` :silent /plan tests =CR3wC-A,/`` map silent ,rt m` :silent /plan tests =CR3wC-X,/`` This works well enough for simple test files. Note -- I always use plan() explicitly rather than specifying a plan as part of use Test::More. More often, I find myself creating some data structure to define test cases and then: plan tests = $fixed + $tests_per_case * @cases; Once I've defined the basic tests, I'm usually adding cases rather than changing the per-case test count, so the total test count takes care of itself. David
Re: a safer way to use no_plan?
David Golden wrote:
More often, I find myself creating some data structure to define test
cases and then:
plan tests = $fixed + $tests_per_case * @cases;
Once I've defined the basic tests, I'm usually adding cases rather
than changing the per-case test count, so the total test count takes
care of itself.
I often use this solution, and, for the test scripts that don't fit
this, I usually count tests by small blocs, letting Perl calculate
the total:
use Test::More;
my $tests;
plan tests = $tests;
BEGIN { $tests += n }
# paragraph of code with n tests
BEGIN { $tests += m }
# paragraph of code with m tests
# etc
That's very simple and when adding new tests, I only have to update
the number of tests in the BEGIN that immediately precedes the
corresponding code.
--
Sébastien Aperghis-Tramoni
Close the world, txEn eht nepO.
Re: a safer way to use no_plan?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! And what if you are running a variable number of tests depending on stuff such as compile-time options, maintainer mode enabled or not, or whatever? - -- Tout n'y est pas parfait, mais on y honore certainement les jardiniers Dominique Quatravaux [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBReq8gfTYH7KfeIIFAQJJXgQAhUB16veolu17/SBExM8a5VnR1KkB/lW6 w0jJ1fyejFlbNXG+NQbdarbReRub0uX0VUzV2OnHp9dXn4n9gifeMFTGFIKPq3mv uS+aR3twnAg76HDCf6xAv8A7cBLNEROSCG/M3wpLFSzDlck9eTX3qmWvqQLVA16P s6kyIZ9pJss= =oP4M -END PGP SIGNATURE-
Re: a safer way to use no_plan?
In article [EMAIL PROTECTED], Eric Wilhelm
[EMAIL PROTECTED] wrote:
At the bottom of a test file:
{my $finish = 1; END {$finish or die \n unplanned exit}};
Yeah, you have to remember to put it at the end of the file, but it may
be easier than counting tests. Thoughts?
You don't have to remember to put it at the bottom of the file if you
start with a template. :)
I run into problems where a loop runs fewer iterations than it should
but the test script otherwise runs to completion normally.
That's also a problem for the just run it and see argument. Sure, it
works for simple things, but when you get into hundreds or thousands of
tests, it's difficult to know if the humber you ran is the number you
were supposed to run. (This often is the case when I'm running
something that has several boundary conditions and I'm running every
combination of those. The factorials get big quick :)
Re: a safer way to use no_plan?
# from Dominique Quatravaux # on Sunday 04 March 2007 04:33 am: And what if you are running a variable number of tests depending on stuff such as compile-time options, maintainer mode enabled or not, or whatever? Even under no_plan, I would say you should use skip() there. --Eric -- Time flies like an arrow, but fruit flies like a banana. --Groucho Marx --- http://scratchcomputing.com ---
Re: a safer way to use no_plan?
# from Sébastien Aperghis-Tramoni
# on Sunday 04 March 2007 06:19 am:
use Test::More;
my $tests;
plan tests = $tests;
BEGIN { $tests += n }
# paragraph of code with n tests
BEGIN { $tests += m }
# paragraph of code with m tests
Interesting. What if...
use More::Tests;
use More::Tests n;
# paragraph of code with n tests
use More::Tests m;
# paragraph of code with m tests
no More::Tests; # :-D
--Eric
--
If the above message is encrypted and you have lost your pgp key, please
send a self-addressed, stamped lead box to the address below.
---
http://scratchcomputing.com
---
Re: a safer way to use no_plan?
# from brian d foy
# on Sunday 04 March 2007 10:02 am:
I run into problems where a loop runs fewer iterations than it should
but the test script otherwise runs to completion normally.
I typically treat that as a test case.
my $counter = 0;
for(things(@foos)) {
ok($_);
$counter++;
}
is($counter, scalar(@foo), 'enough things');
I suppose a run-time goal adjustment and/or a for_ok([EMAIL PROTECTED], $sub)
would
be useful there?
I guess what I'm getting at is a goal-based plan. Aside from the (and I
do love to watch it spin) pretty progress output, the only real need
for a plan is to make sure that every test ran. I think that can
easily be done at run-time, which would make my life easier.
--Eric
--
I arise in the morning torn between a desire to improve the world and a
desire to enjoy the world. This makes it hard to plan the day.
--E.B. White
---
http://scratchcomputing.com
---
Re: a safer way to use no_plan?
Dominique Quatravaux wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! And what if you are running a variable number of tests depending on stuff such as compile-time options, maintainer mode enabled or not, or whatever? So what? Just skip the tests you can't execute under these conditions, it's precisely here to address such issues. -- Sébastien Aperghis-Tramoni Close the world, txEn eht nepO.
Re: a safer way to use no_plan?
On Mar 4, 2007, at 6:33 AM, Dominique Quatravaux wrote:
And what if you are running a variable number of tests depending on
stuff such as compile-time options, maintainer mode enabled or not, or
whatever?
It's really not that hard. It's OK to declare a plan at runtime as
long as you declare your plan before you run your first test. If
your test program dies before that plan is emitted, the harness will
notice that zero tests ran and will complain accordingly.
For example, in CAM::PDF I have a number of test PDF documents that I
use to exercise the code, but I only ship a subset of them to CPAN
(the rest are too big or are license-incompatible). So, I do roughly
like so:
use Test::More; # plan declared at runtime below
my @test_docs = grep { -f $_ } 't/test1.pdf', 't/test2.pdf', ...;
plan tests = 12 + @test_docs * 20;
... run basic tests ...
for my $pdf_file (@test_docs) {
... run per-doc tests ...
}
Chris
--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: a safer way to use no_plan?
* Eric Wilhelm [EMAIL PROTECTED] [2007-03-04 08:20]:
It's a substitute for
use Test::More (0 ? (no_plan) : (tests = 202));
... mostly because I don't like the number of parens in that.
Uh?
use Test::More 0 ? 'no_plan' : ( tests = 202 );
--
*AUTOLOAD=*_;sub _{s/(.*)::(.*)/print$2,(,$\/, )[defined wantarray]/e;$1}
Just-another-Perl-hack;
#Aristotle
Re: a safer way to use no_plan?
# from A. Pagaltzis
# on Sunday 04 March 2007 12:42 pm:
* Eric Wilhelm [EMAIL PROTECTED] [2007-03-04 08:20]:
It's a substitute for
use Test::More (0 ? 'no_plan' : (tests = 202));
... mostly because I don't like the number of parens in that.
Uh?
use Test::More 0 ? 'no_plan' : ( tests = 202 );
Search pattern not terminated at uh.pl line 1.
I think you meant:
use Test::More 0 0 ? (tests = 202 ) : 'no_plan', {
q ( o ) ='
\_/ '
}
;
Then, to turn off the plan, I'll just do:
use constant X = 1;
use Test::More 0 X ? (tests = 202 ) : 'no_plan', {
q ( o ) ='
\_/ '
}
;
--Eric
--
use warnings;
use strict;
use constant X = 1;
use Test::More 0 X ? (tests = 42 ) : join('', 'n', map
{split} q ( o ) = qw/_ p l a n/), q(
\_/careful or you'll plan your eye out );
a safer way to use no_plan?
At the bottom of a test file:
{my $finish = 1; END {$finish or die \n unplanned exit}};
Yeah, you have to remember to put it at the end of the file, but it may
be easier than counting tests. Thoughts? Maybe an 'until_done'
directive and 'tests_done' function? Ways to check that there is no
code after tests_done()?
--Eric
--
The opinions expressed in this e-mail were randomly generated by
the computer and do not necessarily reflect the views of its owner.
--Management
---
http://scratchcomputing.com
---
Re: a safer way to use no_plan?
On Mar 3, 2007, at 8:15 PM, Eric Wilhelm wrote: Yeah, you have to remember to put it at the end of the file, but it may be easier than counting tests. Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! All of this flapping about how maybe we should come up with some mechanisms to do test counting and blah blah blah is like buying a computer to store your recipes on instead of grandma's 3x5 index cards. They ain't sexy but they WORK. -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: a safer way to use no_plan?
On Saturday 03 March 2007 18:18, Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! But Andy, what if I modify the test file? I don't want to have to modify the test file to tell my test harness that I've modified the test file! If only we had something that flies through space -- c
Re: a safer way to use no_plan?
chromatic wrote: On Saturday 03 March 2007 18:18, Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! But Andy, what if I modify the test file? I don't want to have to modify the test file to tell my test harness that I've modified the test file! If only we had something that flies through space -- c Can't we use UNIVERSAL to work around these edge cases and throw lots of warnings? :-P -=Chris signature.asc Description: OpenPGP digital signature
Re: a safer way to use no_plan?
# from Andy Lester # on Saturday 03 March 2007 06:18 pm: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! I'm not wringing my hands, I just don't like step 3 of what could otherwise be a 2 step process. Further, it equates to steps 1 and 4 when adding more tests! Is it bad that I want it to work in a way that encourages me to add tests rather than punishing me for it? Do you comment out the tests = foo and uncomment the no_plan line whenever you edit tests, or do you use a ternary and the r0/r1 vim idiom? Or do you just delete the 'tests' option and type no_plan longhand? Seems like that would lead to at least some wristwringing. The ternary also leads to eyewringing in my case. In fact, so much eyewringing that I've taken to this lately: use inc::testplan(0, + 3 # use + 199 # those others ); ... done; --Eric -- To a database person, every nail looks like a thumb. --Jamie Zawinski --- http://scratchcomputing.com ---
Re: a safer way to use no_plan?
On Mar 3, 2007, at 8:59 PM, Eric Wilhelm wrote: Do you comment out the tests = foo and uncomment the no_plan line whenever you edit tests, or do you use a ternary and the r0/r1 vim idiom? Or do you just delete the 'tests' option and type no_plan longhand? I never use no_plan, except in the rare cases when a test can't reasonably know how much data it will be processing, as in a test of an arbitrary number of links on a web page. That means I don't use no_plan when editing tests. When I add a test, I go and add the tests, keeping a mental note of how many I've added, and then I go to the top of the file, and increment the number. I've added three tests? I increment the number by three. Sometimes I make a mistake. I'll add four tests but only add three to the count. I run prove, and oh no, the total is wrong. I svn diff to look to see that yes, I've actually added four, and I update the count by one more. I run prove again, and all is well. Commit. Repeat. It's really not that tough, and I don't mind the iteration. I think as programmers we get far too hung up on the idea of doing everything perfectly once. -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: a safer way to use no_plan?
* Eric Wilhelm [EMAIL PROTECTED] [2007-03-03T21:59:32]
In fact, so much eyewringing that I've taken to this lately:
use inc::testplan(0,
+ 3 # use
+ 199 # those others
);
What is that ... for?
I often do this:
use Test::More;
my @test_data = load_data('t/whatever.dat');
plan tests = 3 + @test_data * 3;
I'm not sure if your snippet is doing something else.
--
rjbs
Re: a safer way to use no_plan?
A tangential comment: The xUnit approach avoids this question of counting altogether - you create one or more subroutines whose names begin with 'test', each of which contain one or more assertions such as is_deeply($got, $expected); and the test runner runs all the test* methods. It counts the number of test* methods for you. -M --- Matisse Enzer [EMAIL PROTECTED] http://www.matisse.net/ - http://www.eigenstate.net/
Re: a safer way to use no_plan?
# from Ricardo SIGNES # on Saturday 03 March 2007 07:11 pm: use inc::testplan(0, + 3 # use + 199 # those others ); What is that ... for? It's a substitute for use Test::More (0 ? (no_plan) : (tests = 202)); ... mostly because I don't like the number of parens in that. The args are: use inc::testplan (haveplan, numtests); where haveplan is the battle-mode switch which is quite handily flipped with vim's 'r0' or 'r1' idiom (or Ctrl+a/Ctrl+x if you prefer.) In your case, you could: inc::testplan-import(1, 3 + @test_data * 3 ); I didn't bother wrapping plan() or adding much other interface as it is currently just an experiment in planning. Actually, if my test could figure out how many tests there will be so I got to watch that neato x/y progress output, I would never plan. http://svn.dotreader.com/svn/dotreader/trunk/inc/testplan.pm --Eric -- Entia non sunt multiplicanda praeter necessitatem. --Occam's Razor --- http://scratchcomputing.com ---
