Re: The relation between CPAN Testers and quality (or why CPAN Testers sucks if you don't need it)
On Fri, 5 Sep 2008, Aristotle Pagaltzis wrote: * chromatic <[EMAIL PROTECTED]> [2008-09-05 19:50]: If I could see somehow that my distribution implicitly runs on Perl 5.001 (or explicitly runs only on 5.11.0), or that it has no Makefile.PL or Build.PL, or any of the other dozens of packaging quirks that can cause problems, I could fix them before uploading and before triggering a wave of testing. That’s CPANTS, the useful part of it anyway… except that the feedback cycle is even *much* slower there than with the Testers. Is not! Check out Test::Kwalitee on cpan. I've taken to adding that to my maintainer-only tests for my distros. -dave /*== VegGuide.Org Your guide to all that's veg ==*/
Re: Ignoring Non-Failures
On Wed, 3 Sep 2008, David E. Wheeler wrote: http://cpantesters.perl.org/author/DWHEELER.rss This makes it easy for me to sift through things. The only thing that would make it better is if I could get it to display only FAILs. To whom should a feature request be sent (I thought I sent a patch to acme at one time…)? I just wrote a comment on use Perl in dagolden's journal with the exact same request. This would be fantastically more useful to me than the existing feed, which is usually 20-80 entries at a time, of which maybe 0-4 are of interest. -dave /*== VegGuide.Org Your guide to all that's veg ==*/
Re: Why should package declaration match filename?
On Fri, 14 Mar 2008, Matisse Enzer wrote: I'm discussing some potential refactorings at $work at wanted to give an articulate explanation of the benefits of having package declarations match file names, so that: # file is Foo/bar.pm package Foo::Bar; One reason is so that when you see a "package" statement, you know what the corresponding "use" statement would be, and when you see a "use" statement, you know what the corresponding package is, and have a good clue about the path path of the file you are importing. What are other good reasons to have package declarations match file paths? Because all the other Perl hackers do it that way? It always amazes me how isolated people or small groups come up with some completely ass backwards way of doing something, and then can't be talked out of it. There's a lot of value in following the existing best practices of the Perl community as a whole. For one thing, it means you can hire people with "Perl experience" and they can bring that experience to bear on your application. If you insist on reinventing every wheel, you've basically created your own in-house Perl-like dialect. It _looks_ sorta like Perl, but it's not Perl. This argument goes for _any_ language or tool for which there is a substantial community and history of shared practice. -dave /*== VegGuide.Org Your guide to all that's veg ==*/
Re: Auto: Your message 'FAIL IO-AIO-2.51 i386-freebsd-thread-multi 6.2-release' has NOT been received
On Sat, 22 Dec 2007, Andreas J. Koenig wrote:
On Sat, 22 Dec 2007 14:22:05 +0100, [EMAIL PROTECTED] (Andreas J. Koenig) said:
> 1. You get a fail report with an error message that doesn't tell you
>exactly what went wrong.
> 2. You rewrite your test in a way that it does tell you more.
> 3. Release.
> 4. If you now understand the problem, fix it, else goto 1.
Sorry I didn't notice before I posted that this can be refactored into
a while loop:
while (You don't understand the output of your own test script){
Rewrite your test script;
Release;
}
If only it were that easy.
Sometimes the test report is something mysterious to me, that I think just
can't happen. Other times I understand the output ("got X instead of Y")
but I can't think of any reason _why_ that should happen.
It's as simple as it can get. And I probably do not have to explain
why I'm backing the position of imacat. The developer never tells the
tester what to do. The tester is a device and you can use it IFF you
want to fix your bugs. Otherwise, don't.
If the tester were a device it shouldn't be telling the developer what to
do either, now should it?
Several people in this thread have interpreted the back-and-forth with
imacat I linked to as some sort of "my position vs hers (his?)". That's
not the case at all. Simply put, the initial bug report/request _DID NOT
MAKE SENSE_ to me. I tried to get clarification and got a fight instead.
Should testers pre-install Module::Build? I don't freaking care if they
do. OTOH, if the passthrough Module::Build installer bits don't work, that
is worth fixing, and is worthy of a bug report _for Module::Build_.
OTOH, if tester configure their systems to _intentionally_ not follow the
passthrough and install Module::Build, yeah, some stuff will just not
work, and that's not my problem. If you don't want to install the prereqs
as part of testing, then don't test.
-dave
/*===
VegGuide.Orgwww.BookIRead.com
Your guide to all that's veg. My book blog
===*/
Re: Auto: Your message 'FAIL IO-AIO-2.51 i386-freebsd-thread-multi 6.2-release' has NOT been received
On Thu, 20 Dec 2007, David Golden wrote: On Dec 20, 2007 1:19 PM, Dave Rolsky <[EMAIL PROTECTED]> wrote: It's generally pretty rare that the failure report includes enough information for me to do anything about it, so without an engaged party on the other end, it really is just noise. With CPAN::Reporter, I've been trying to add additional context (within reason) to assist with problem diagnosis. What kind of information would improve the reports? (Not to say that this obviates the need for a responsive tester, but every little bit helps.) Getting the full TAP as Michael P mentioned would be good. However, usually I end up needing to investigate aspects of the testers platform, often by having them run snippets of Perl code from the shell, or asking them to try a patch. There's not much you can do to automate that. As chromatic mentioned, failures often happen on platforms to which I as a module author don't have ready access, so I need some assistance from the user of that platform. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Auto: Your message 'FAIL IO-AIO-2.51 i386-freebsd-thread-multi 6.2-release' has NOT been received
On Thu, 20 Dec 2007, Ovid wrote: To summarize what I think Imacat's point is: some CPAN testers quite deliberately have only core modules installed in order to catch issues just like this. As a result, anything which assumes a non-core module (like your Makefile.PL previously assuming Module::Build was available) is simply going to fail. Actually, that wasn't the problem. I was using a standard "passthrough" Makefile.PL, which basically just tries to install Module::Build and exec Build.PL after doing so. It does prompt the user but it also defaults to "yes, install MB" so it should work in an automated environment without hanging. This seemed to have some sort of weird interaction with CPANPLUS and/or CPANPLUS's unfortunate decision to package Module::Build. Honestly, I never understood what the problem was. The real problem in that conversation was that I couldn't get a freaking answer to my question "don't you need Module::Build pre-installed to test all of CPAN?" I didn't necessarily object to using a standard Makefile.PL (in fact, I've done it before at others' request, but the request _made sense_). I just wanted some clarification on why she was making that request. In fact, with Module::Build, it's even more problematic than with other modules. I know there are many people who simply won't have anything to do with Module::Build and won't install it. While the specific reasons listed in http://perlmonks.org/?node_id=458282 have gone away, there are still other general reasons (including "religious" ones) which keep people from adopting Module::Build or installing it. Well, good for them, but that's going to make it pretty freaking hard to smoke test a lot of CPAN modules, and that's not _my_ problem. Insisting that no one else use Module::Build because you don't like it is exactly the sort of "religiousness" that Marc Lehmann was objecting to amongst CPAN testers. So honestly, don't let a language barrier get in the way. Germans can seem incredibly rude to Americans when the speak English while we sometimes seem incredibly obsequious due to how we approach asking questions. If you don't know the cultural differences shaping the language choices, it can seem horrifying. Sheesh, you'd think I wasn't married to a woman from Taiwan, or hadn't been there 5 times already. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Auto: Your message 'FAIL IO-AIO-2.51 i386-freebsd-thread-multi 6.2-release' has NOT been received
On Wed, 19 Dec 2007, Andy Armstrong wrote: His view: cpan-testers are incompetent, ego tripping, quasi-religious nuisances. I think there's some truth to this view. For support I submit this bug ticket - http://rt.cpan.org/Ticket/Display.html?id=27208 On the other hand, that was an exception, though a really annoying one. My other big annoyance with smoke testers/reporters is that a lot of folks simply do not respond to requests for more information. It's generally pretty rare that the failure report includes enough information for me to do anything about it, so without an engaged party on the other end, it really is just noise. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Milton Keynes PM coding collaboration
On Tue, 11 Dec 2007, Edwardson, Tony wrote: Anyone written any CPAN modules for which the testing coverage needs to be improved ? Oh, yes. Want someone else to sort this out for you ? Yes. Milton Keynes PM want to start working together to contribute to the Perl development effort and as we need to do something that beginners and experts can both work on, we thought that improving the testing coverage of popular CPAN Modules would be a good initial project. So, we are looking for popular CPAN modules which could do with a better test suite and the Author is willing to let others work in it on their behalf. Log::Dispatch might be a possibility. The tests are old and crufty, and probably don't cover lots of corner cases and features. I'd be happy to see the test suite refactored a bit into multiple files. It could also use better mocking so we could test things like syslog and email. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
ANNOUNCE: SmokeRunner::Multi 0.1
This is a combo library/script that is designed to do automated smoke tests on multiple "test sets" (in my case, branches of a code base). It has hooks for pushing reports into Smolder, and it has infrastructure in place so someone could write something to send emails of test results as well. We're using this at LiveText, and I wrote it on company time (thanks, LiveText), but it's released under the standard Perl license. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Installing Tests
On Mon, 11 Sep 2006, David Golden wrote: * test.pl vs t/*.t * Custom Makefile.PL or Build.PL that affects test runs * build_requires modules bundled in inc/ And don't forget that some tests may include tests data that is needed to run the tests. If we're lucky, it's under t/ -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Adding more kwalitee tests
On Mon, 5 Sep 2005, Thomas Klausner wrote: has_perl_dependency: In the META.yml (assuming it exists) there is a dependency on the version of perl required to install the dist. The goal of this is to make life a little easier on installers and CPAN testers and a few other things. Many many modules should pass this test already. The main goal however, is to help enable the NEXT kwalitee test to have better coverage. Hmm. I never added this of any to my dists. It might also be annoying for people if authors add a newer version of Perl as a prereq than they have installed and some installer then goes off to fetch & install a new version of Perl. (And it's a lot of hassle to find the exact lowest version of Perl you need). I'm pretty sure that CPAN and CPANPLUS will explicitly _not_ do this. The "perl" dependency is handled specially. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Test run of the new Phalanx 100
On Mon, 17 Jan 2005, Andy Lester wrote: Msql-Mysql-modules This is just the old distro that contained DBD::mysql and DBD::msql. I don't think the latter is maintained, and DBD::mysql is already on the list. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Refactoring a test program: advice sought
On Mon, 17 Nov 2003, darren chamberlain wrote: > * Andrew Savige [2003-11-15 14:51]: > > I took a quick look a mod_perl and Template Toolkit (TT). > > TT has a: > > lib/Template/Test.pm > > which looks wrong to me (should that not be under t/lib instead?). > > Template::Test contains a bunch of (pre-Test::More) testing methods that > are geared towards writing TT tests. Some of my TT plugins use > Template::Test instead of Test::More. And Mason includes a module caled HTML::Mason::Tests in its distro. I've used that module for my MasonX::Request::WithApacheSession module, and I expect it to be installed as part of the Mason core code. This seems useful for things like templating tools and other things which create their own environment, like mod_perl, POE, etc. -dave /*=== House Absolute Consulting www.houseabsolute.com ===*/
Re: Taint mode testing and project Phalanx
On Tue, 21 Oct 2003, Michael G Schwern wrote: > On Tue, Oct 21, 2003 at 12:34:44PM -0500, Dave Rolsky wrote: > > Anyway, my taint mode experience has been that random things break in very > > weird ways when using it. > > All the more reason to test with it on. :) At this point I've become rather disgusted with it. When taint mode breaks pos(), and as a result your regex-based parser blows up in weird ways, and you spend many, many hours figuring out what exactly is happening, and then can't reduce it to a simple test case, you tend to get a little peeved. Tim's #ifdef idea for testing taint mode seems like a really good idea. Once I know it's well tested in the core, I'll be happy to test my own modules with it. -dave /*=== House Absolute Consulting www.houseabsolute.com ===*/
Re: Taint mode testing and project Phalanx
On Mon, 20 Oct 2003, Michael G Schwern wrote:
> On Tue, Oct 21, 2003 at 12:24:03AM -0500, Dave Rolsky wrote:
> > On Mon, 20 Oct 2003, Andrew Savige wrote:
> > > I noticed in Test::Tutorial:
> > > "Taint mode is a funny thing. It's the globalest of all global features.
> > > Once you turn it on it effects all code in your program and all modules
> > > used (and all the modules they use). If a single piece of code isn't
> > > taint clean, the whole thing explodes. With that in mind, it's very
> > > important to ensure your module works under taint mode."
> >
> > Not to mention that it's buggy as hell. For example, in various versions
> > of Perl I've used there have been rather serious bugs in the regex engine
> > when taint mode is on, even when dealing with untainted variables!
>
> I've never hit anything like this. Do you have examples?
Well, one example comes from my Params::Validate module, where I have this
little bit of XS:
while (he = hv_iternext(p)) {
/* This may be related to bug #7387 on bugs.perl.org */
#if (PERL_VERSION == 5)
if (! PL_tainting)
#endif
SvGETMAGIC(HeVAL(he));
Whee, a random taint related bug.
Then there was the time I found that pos() didn't get updated inside
s/\G...//gc matches when taint mode was on, for certain versions of
Perl working with some strings (but not others). I don't think this
bug exists in the current version any more.
I could never reproduce this in a concise example, unfortunately.
Anyway, my taint mode experience has been that random things break in very
weird ways when using it.
-dave
/*===
House Absolute Consulting
www.houseabsolute.com
===*/
Re: Taint mode testing and project Phalanx
On Mon, 20 Oct 2003, Andrew Savige wrote: > I noticed in Test::Tutorial: > "Taint mode is a funny thing. It's the globalest of all global features. > Once you turn it on it effects all code in your program and all modules > used (and all the modules they use). If a single piece of code isn't > taint clean, the whole thing explodes. With that in mind, it's very > important to ensure your module works under taint mode." Not to mention that it's buggy as hell. For example, in various versions of Perl I've used there have been rather serious bugs in the regex engine when taint mode is on, even when dealing with untainted variables! I for one think taint mode is rather hopeless until taint mode itself gets better testing in the Perl core. > What are general opinions re taint tests for CPAN modules? > (i.e. a test with #!perl -Tw as the first line of the test). See above. -dave /*=== House Absolute Consulting www.houseabsolute.com ===*/
Re: Add Test::Harness 2.x Prerequisite to Test::Simple?
On Mon, 26 Aug 2002, Adrian Howard wrote: > Can anybody give me a definitive version of when TODO tests started > working in Test::Harness? From the Changes file I'm currently assuming > everything after Test::Harness 2.03 inclusive should be okay. > > Personally, I would tend towards leaving the prerequisites to > Test::Builder alone. I think I'm right in saying that everything > except TODO tests work fine with the earlier Test::Harness, so it > seems unhelpful to prevent people using the rest of the distribution. > > Maybe throw out a warning that TODO tests won't work on the > Test::Builder install if the latest Test::Harness isn't installed? Assuming Test::Harness 2.03+ is a drop-in replacement for earlier Test::Harness's (which it mostly is, I think), there's no reason for Test::More to not require it. It implements a core feature of Test::More/Builder. Having every module author who uses todo tests require Test::Harness 2.03 is rather silly. -dave /*== www.urth.org we await the New Sun ==*/
Re: [PATCH MakeMaker.pm] Add documentation for ExtUtils::MakeMaker::prompt()
On Tue, 20 Aug 2002, Michael G Schwern wrote: > +=head2 Other Handy Functions > + > +=over 4 > + > +=item prompt > + > +my $value = prompt($message); > +my $value = prompt($message, $default); > + > +The C function provides an easy way to request user input > +used to write a makefile. It displays the $message as a prompt for > +input. If a $default is provided it will be used as a default. The > +function returns the $value selected by the user. > + > +If C detects that it is not running in interactively (say, > +if it is running from a CPAN shell), or if the PERL_MM_USE_DEFAULT > +environment variable is set to true, the $default will be used without > +prompting. This prevents automated processes from blocking on user > +input. In my experience, it still prompts under the CPAN shell, but not CPANPLUS. I've actually come to rely on the former behavior and was surprised that CPANPLUS didn't work the same way, causing it to be basically imposssible to do a fresh install of Alzabo via CPANPLUS. How about a way to _force_ it to prompt or give up if that's really impossible? -dave /*== www.urth.org we await the New Sun ==*/
Test::Builder & forks
Run this little script:
use Test::More tests => 10;
use strict;
my $pid;
if ($pid = fork)
{
wait;
ok(1) for 1..10;
}
else
{
exit;
}
And note the annoying "# No tests run!" message that gets printed before
the first "ok" message.
I can't think of any really elegant solutions other than adding something
like:
Test::Builder->reset
and calling that in child processes.
-dave
/*==
www.urth.org
we await the New Sun
==*/
Re: Test::Builder bug
On Sat, 30 Mar 2002, Michael G Schwern wrote:
> Doesn't seem to have any effect on the code above, but it does have an
> effect on this:
>
> $ perl -Ilib -MTest::Builder -e '$tb=Test::Builder->new; $tb->plan('no_plan');
>$tb->current_test(10)'
> Modification of non-creatable array value attempted, subscript -1 at
>lib/Test/Builder.pm line 948.
> WHOA! Somehow you got a different number of results than tests ran!
> This should never happen! Please contact the author immediately!
> END failed--call queue aborted.
>
> which will result in this in 0.43:
>
> $ perl -Ilib -MTest::Builder -e '$tb=Test::Builder->new; $tb->plan('no_plan');
>$tb->current_test(10)'
> 1..10
Right, that's what I was trying to show. I just forgot all the context ;)
-dave
/*==
www.urth.org
we await the New Sun
==*/
Test::Builder bug
The changelog says this:
0.42 Wed Mar 6 15:00:24 EST 2002
- Setting Test::Builder->current_test() now works (see what happens
when you forget to test things?)
But if I do this:
perl -MTest::Builder -e 'Test::Builder->new->current_test(10)'
it still blows chunks.
Patch below.
houseabsolute:/usr/local/share/perl/5.6.1/Test> diff -u
/root/.cpan/build/Test-Simple-0.42/lib/Test/Builder.pm Builder.pm
--- /root/.cpan/build/Test-Simple-0.42/lib/Test/Builder.pm Wed Mar 6
14:14:58 2002
+++ Builder.pm Sat Mar 30 17:29:42 2002
@@ -935,7 +935,8 @@
if( defined $num ) {
$Curr_Test = $num;
if( $num > @Test_Results ) {
-for ($#Test_Results..$num-1) {
+my $start = @Test_Results ? $#Test_Results : 0;
+for ($start..$num-1) {
$Test_Results[$_] = 1;
}
}
Re: Untested modules update: There's more than we thought
On Sat, 15 Dec 2001, Kurt D. Starsinic wrote:
> > my $foo = Foo->new;
> > ok( defined $foo && $foo->isa('Foo') );
> >
> > which rapidly gets tiresome.
>
> Or ok(UNIVERSAL::isa(Foo->new, 'Foo'));
Under 5.005 it spits out warnings if Foo->new returns undef. Plus you
still don't get any useful diagnostics. You just know it failed and then
you'll have to go back and add debugging print statements or use the
debugger to figure out _why_ it failed (did Foo->new return undef,
something blessed into a different class, an unblessed thing, etc).
I recently converted a rather large test suite (for Alzabo, 600+ unique
tests or so) to use Test::More. Doing so helped me do several things:
1. Break down what I was testing into smaller units. Where I used to
have things like:
ok( @foo && defined $foo[0] && $foo[0]->name eq 'bob' );
I now have:
is( @foo, 2, "expect two elements from bar() call" );
isa_ok( $foo[0], "Frobnigator" );
is( $foo[0]->name, 'bob', "the frobnigator should be named 'bob'" );
The presence of these similar but slightly different utility functions
helped me really think about exactly what I should be testing.
2. Diagnosing new failures. The diagnostics are such that I rarely have
to go back and add print statements or use the debugger. I know _why_
things are failing right away, which is better than simply knowing that
they failed for some reason or other.
3. Removed liable to break support code bits that were being repeated
everywhere.
So in the end I have more tests, which are easier to diagnose, with less
support code that I have to maintain. This is good because Alzabo is very
complicated as it is and the tests have enough weird life support to do
things like test against different RDBMS backends and such. Anything I
don't have to write makes me happier.
And if there's a problem with the testing code, I can report the bug to
Schwern rather than fixing it in my five test suites where I copied the
code around.
Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
see why people seem to object to the use of Test::More in the core Perl
tests. I can't see how it couldn't help improve the quality of the tests
while providing a standardized way to do the things that all the tests do
anyway.
-dave
/*==
www.urth.org
We await the New Sun
==*/
RE: Make Schwern Poor before 5.8.0
On Tue, 20 Nov 2001, Tels wrote: > > If you would prefer to handle the > > ExtUtils::MakeMaker and ExtUtils::MM_* changes yourself let me know. > > You can do it. I'll wait until the dust settles (I wrote tests for routines > that now get removed, so I am a bit conservative with starting another heap > of work ;) Ok, I'll write up another patch to do this if my previous MM-> cleanup patch is accepted (I just want to make sure I'm on an acceptable path here). -dave /*== www.urth.org We await the New Sun ==*/
RE: Make Schwern Poor before 5.8.0
On Mon, 19 Nov 2001, Tels wrote: > * There are a couple of functions I have tests for, BUT: > > =item canonpath > > No physical check on the filesystem, but a logical cleanup of a > path. On UNIX eliminated successive slashes and successive "/.". > > =cut > > [snip] > =item catdir > > Concatenate two or more directory names to form a complete path > ending > [snip] > > etc etc. > > Shouldn't all these path functions be already somewhere else, f.i. in > File::Spec and shouldn't it use File::Spec, simply? Or am I something > missing? Yes, this should be using File::Spec (unless the code in ExtUtils for doing these differs in a way that needs to be preserved). I know there are some modules out there using ExtUtils::MakeMaker for file manipulation stuff. I just patched ExtUtils::Instal to use File::Spec instead. I was planning to hunt through the core for the rest (maybe this weekend) and patch them. Once that was done it'd be a simple matter to remove the code implementing these method from ExtUtils::MakeMaker. If you would prefer to handle the ExtUtils::MakeMaker and ExtUtils::MM_* changes yourself let me know. -dave /*== www.urth.org We await the New Sun ==*/
Re: Supressing "make test" jibber-jabber
On Tue, 2 Oct 2001, Michael G Schwern wrote: > You capture the output of 'make test' and manipuate that line > directly?? Eek! No, I add extra Perl code to that line to set environment variables and such. Maybe there's another way to do it but its not exactly like these things are well documented. > What are you doing with it, and can you instead do it by defining > MY::test_via_harness() in your Makefile.PL? I define MY::test() -dave /*== www.urth.org We await the New Sun ==*/
Re: Supressing "make test" jibber-jabber
On Tue, 2 Oct 2001, Michael G Schwern wrote: > Any particularly good reason we're showing all that "PERL_DL_NONLAZY" > noise as part of "make test"? What about just: I do some weird stuff with that piece to pass data around to my tests. Sometimes its useful to see it. How about only showing it if TEST_VERBOSE is true? -dave /*== www.urth.org We await the New Sun ==*/
Re: Starting with a test > 1
On Sun, 30 Sep 2001, Michael G Schwern wrote:
> OH! I thought you were literally using fork(), not running another
> program. Now that makes a little more sense.
Yeah, I realized I should've been clearer after a few exchanges (I'm
slow).
> I suppose the idea here is something like:
>
> system("some_test") for 1..3;
>
> so you test that you can run the same code simultaneously and they
> won't trip on each other?
Something like that. So I know what order the tests are in. I know how
many should be run each time. But when the 'some_test' code runs, it
shouldn't print a header (cause its in the middle of a larger batch) and
the analysis in the END{} block should be skipped cause its not really
done yet. But I _do_ want the Test::More functions. They're quite handy.
Actually, to make a testimonial. I just finished rewriting all the Alzabo
tests to use Test::More. In the process, I clarified the test suite,
added at least 100 or so new tests (mostly by breaking up old ones into
smaller units) and caught a few false positives. I think that a generic
ok() function makes it easier to have brain farts in your tests. Having
to use a different function to test different things helped me find lots
of errors.
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: Starting with a test > 1
On Sun, 30 Sep 2001, Michael G Schwern wrote:
> On Sat, Sep 29, 2001 at 09:12:01PM -0500, Dave Rolsky wrote:
> > > use Test::More tests => 10; # or whatever.
> >
> > Except this needs to printed just once in the parent _before_ any tests
> > are run.
>
> Why not just do it before you do any forking?
I could, but then Test::Builder gets confused cause the tests run via
system don't change the internal variables Test::Builder uses.
> Could I see a copy of the idealized representation of your test
> (ie. what you'd like to write)?
in parent
use Test::More qw(I know what the fuck I'm doing!);
print "1..944";
system ("run_some_tests");
in run_some_tests
---
use Test::More qw(I know what the fuck I'm doing!);
Test::Builder->new->current_test($something_passed_from_parent);
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: Starting with a test > 1
On Sat, 29 Sep 2001, Dave Rolsky wrote: > A. To be able to use Test::More's functions without ever printing a 1..foo > test header. And it turns out I also need to suppress all the stuff in Test::Builder::_ending (I simply bailed at line 838). So basically, I need an "I know what the fork I'm doing" option that will still create a builder object but that skips the test header and skips the messages in _ending. I implemented this as a 'no_header' option to Test::More (and corresponding method in Test::Builder) but there's probably a better term for this. Maybe 'minimal_support' or something. -dave /*== www.urth.org We await the New Sun ==*/
Re: Starting with a test > 1
On Sat, 29 Sep 2001, Michael G Schwern wrote: > Have you tried the Test::Builder thing yet? I think you should be > able to solve your forking problem like so: Working on it > use Test::More tests => 10; # or whatever. Except this needs to printed just once in the parent _before_ any tests are run. > $builder = Test::Builder->new; # get the underlying T::B object for > # fine control. > > $builder->use_numbers(0); I was thinking that I could just do $builder->current_test( $some_num_greater_than_1 ) and it should work. > So the output should look something like: > > 1..10 > ok 1 - normal test > ok 2 - another normal test > ok - forked test 1 (PID 3245) > ok - forked test 2 (PID 3245) > ok - forked test 1 (PID 3246) > ok - forked test 2 (PID 3246) > ok - forked test 1 (PID 3247) > ok - forked test 2 (PID 3247) > ok 9 - last two > ok 10 - normal tests I don't want to give up the numbers. The order of testing is predictable so that's not the problem. Right now I'm stuck on several things. 1. I want to grab the Test::More exported stuff in the child after a fork without printing a 1..foo header, either before (by setting an explicit number of tests) or after (no_plan). Here's what I _really_ want (my list of demands are thusly). A. To be able to use Test::More's functions without ever printing a 1..foo test header. B. To be able to arbitrarily set the current test number. Something like this'd be nice: use Test::More qw(functions_only); # or qw(no_header) That's probably all I need to get it going. -dave /*== www.urth.org We await the New Sun ==*/
Re: Starting with a test > 1
On 29 Sep 2001, Piers Cawley wrote:
> If it's not a dumb question, why are you doing it that way? If you
> were using PerlUnit, aka Test::Unit::TestCase, you could do something
> like:
Why am I doing what? Forking and starting tests at weird offsets?
The forking is cause I need to load different modules representing the
same interface each time. The starting at a different spot is so I can
track the number of tests over the whole run.
> package ParentTest;
>
> use base 'Test::Unit::TestCase';
>
> sub setup {
> ...
> }
>
>
> sub test_foo {
> ...
> }
I looked at Test::Unit at it was just too complicated for me back then.
The way I was doing it before was simply to use my own little ok() sub and
that was mostly good enough. Test::More is nice cause of all the
convenient functions like isa_ok. This has helped me improve and clarify
the test suite. But I don't see the need for even more functionality yet.
> Thinking about it, you could probably move the module loads into the
> test scripts and have them just run the basic test, which would, in
> turn, probably mean you could get away with just using Test::More/Simple
But I like Test::More! Its just the numbering issue that's the problem.
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: Starting with a test > 1
On Wed, 26 Sep 2001, Michael G Schwern wrote: > Uggg. No, there's no good way to handle this now unless there's some > way $Num_Tests can make itself shared between forks. Well, I do keep track of it in my code and pass it around between forks. What I need is a way to set it. > In the general sense, there is a good solution to this problem. > First, there's the 'no_plan' feature of Test::More, so you can run an > indeterminate number of tests. Second, Test::Harness allows you to > use unnumbered tests, however, there's currently no support for > this in Test::Simple/More. I don't want to use the no_plan feature. That would be a step backwards for my test suite (I _know_ how many tests are in each fork). > It's on the TODORSN list. Just tell me how you want the interface to look and I'll gladly write it. I want to use Test::More in the next release of Alzabo so I'm motivated. -dave /*== www.urth.org We await the New Sun ==*/
Starting with a test > 1
For my Alzabo tests, I do some weird stuff involving forking off processes during tests and running the same (basically same) set of tests multiple times, each time with different modules loaded. The parent process calculates how many tests will be involved in each run and how many runs will occur to print the '1..?' stuff at the beginning. As part of this, I need to be able to tell the forked process to start its test count at a certain number (greater than 1). There's no way to do this with Test::Simple currently. What I need is to be able to set $Planned_Tests and $Num_Tests without print the "1..$x" bit again. Either that or set a $Offset so that when it prints the test number it prints $Num_Tests + $Offset. Thoughts? -dave /*== www.urth.org We await the New Sun ==*/
Re: [PATCH] Test::More isa_ok function
On Tue, 25 Sep 2001, chromatic wrote:
> Combining the two gives us a test and a very contrived test failure output:
>
> isa_ok($foo, 'Alzabo::Foo', 'Return value from $bar->foreign_keys');
>
> not ok 1 - $foo->isa('bar')
> # Failed test (foo.plx at line 3)
> # Return value from $bar->foreign_keys isn't a 'Alzabo::Foo'
>
>
> The syntax is a little different, but it's *really* close.
I like it.
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: [PATCH] Test::More isa_ok function
On Tue, 25 Sep 2001, Michael G Schwern wrote:
> A, ok. How about this:
>
> my $yarrow = Bar->new;
> isa_ok($yarrow, "Bar", 'yarrow');
>
>
> not ok 1 - yarrow->isa('bar')
> # Failed test (foo.plx at line 3)
> # yarrow isn't a 'Bar'
That's better, I guess. But I'm mostly doing:
isa_ok($foo, 'Alzabo::Foo', 'Return value from $bar->foreign_keys should be
"Alzabo::Foo"');
That way if it fails I know exactly what was going on. I'm big on
verbosity here ;)
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: [PATCH] More Test::More stuff
On Tue, 25 Sep 2001, Kirrily Robert wrote: > But it does! It says something like: > > not ok 23 > # Failed test 1 (eval.t at line 69) > # got: 'blah blah blah' > # expected: '' Oops, that's what I get for not actually trying it out. I guess that's good enough, though I still like the eval_ok thing. I guess I can implement it easily enough just for the Alzabo tests. -dave /*== www.urth.org We await the New Sun ==*/
Re: [PATCH] More Test::More stuff
On Mon, 24 Sep 2001, Michael G Schwern wrote:
> On Mon, Sep 24, 2001 at 06:42:55PM -0500, Dave Rolsky wrote:
> > +sub eval_ok (&$) {
> > +my ($code, $name) = @_;
> > +
> > +eval { $code->() };
> > +if ($@) {
> > + ok( 0, "$name - $@" );
> > +} else {
> > + ok( 1, $name );
> > +}
> > +}
>
> The unfortunate problem is this has adverse effects on any code in the
> block that uses caller(). I don't want tests to introduce
> side-effects. Unfortunately, Sub::Uplevel doesn't work on anything
> before 5.6.0.
>
> However, you're not the first person that's wanted this.
>
> Maybe we can get rid of this problem with documentation. I tend to
> use this idiom.
>
> eval { ...code... };
> is( $@, '' );
Yeah, except that doesn't print out $@ in case of failure. If I'm
checking that no exception occurs I want to know what the exception is
when it happens.
I don't know that the caller() issue is a big deal. Document it and then
for code where that's an issue (a small fraction at best, I'd think)
people will know not to use it. Just cause it doesn't work for all
possible uses doesn't make it useless.
-dave
/*==
www.urth.org
We await the New Sun
==*/
Re: [PATCH] Test::More isa_ok function
On Mon, 24 Sep 2001, Michael G Schwern wrote: > On Mon, Sep 24, 2001 at 06:23:58PM -0500, Dave Rolsky wrote: > > The patch below allows you to supply your own test name for the isa_ok > > function (I find the default insufficiently descriptive). I'd like to do > > the same for can_ok but I don't think that could be done without breaking > > backwards compatibility. > > What sort of names do you tend to throw in there? Just something more descriptive than what it generates by default. Something like "Check that object X isa Blah". The key being that I want to say _what_ object I'm checking. -dave /*== www.urth.org We await the New Sun ==*/
[PATCH] More Test::More stuff
Ok, forget the last patch. This one incorporates that plus more.
This one also adds an eval_ok function. The idea here is that sometimes
you simply want to try something to see if it works or not. If it fails
it will append the error ($@) after the name of the test.
The reason for these patches is that I'm trying to use Test::More for
Alzabo and these are some changes that would make it more useful for me.
-dave
--- More.t.old Mon Sep 24 18:35:39 2001
+++ More.t Mon Sep 24 18:39:35 2001
@@ -1,4 +1,4 @@
-use Test::More tests => 22;
+use Test::More tests => 25;
use_ok('Text::Soundex');
require_ok('Test::More');
@@ -20,7 +20,10 @@
can_ok pass fail eq_array eq_hash
eq_set));
isa_ok(bless([], "Foo"), "Foo");
+isa_ok(bless([], "Foo"), "Foo", "Optional description to isa_ok");
+eval_ok( sub { 1 }, "eval_ok 1" );
+eval_ok { 1 } "eval_ok 2";
pass('pass() passed');
--- More.pm.old Mon Sep 24 18:35:27 2001
+++ More.pm Mon Sep 24 18:39:14 2001
@@ -25,6 +25,7 @@
$TODO
plan
can_ok isa_ok
+ eval_ok
);
@@ -447,7 +448,7 @@
=item B
- isa_ok($object, $class);
+ isa_ok($object, $class, $optional_name);
Checks to see if the given $object->isa($class). Also checks to make
sure the object was defined in the first place. Handy for this sort
@@ -465,11 +466,11 @@
=cut
-sub isa_ok ($$) {
-my($object, $class) = @_;
+sub isa_ok ($$;$) {
+my($object, $class, $name) = @_;
my $diag;
-my $name = "object->isa('$class')";
+$name ||= "object->isa('$class')";
if( !defined $object ) {
$diag = "The object isn't defined";
}
@@ -491,6 +492,35 @@
}
}
+=item B
+
+ eval_ok( sub { code ... }, $name );
+
+or
+
+ eval_ok { code ... } $name;
+
+Checks to see that a code block can be called in an C without
+throwing an exception.
+
+If it fails it will print the exception after the name as part of the
+test status.
+
+This is useful if you are calling a block of code simply to make sure
+it doesn't die.
+
+=cut
+
+sub eval_ok (&$) {
+my ($code, $name) = @_;
+
+eval { $code->() };
+if ($@) {
+ ok( 0, "$name - $@" );
+} else {
+ ok( 1, $name );
+}
+}
[PATCH] Test::More isa_ok function
The patch below allows you to supply your own test name for the isa_ok
function (I find the default insufficiently descriptive). I'd like to do
the same for can_ok but I don't think that could be done without breaking
backwards compatibility.
-dave
--- More.t~ Wed Sep 5 19:23:24 2001
+++ More.t Mon Sep 24 18:24:15 2001
@@ -1,4 +1,4 @@
-use Test::More tests => 22;
+use Test::More tests => 23;
use_ok('Text::Soundex');
require_ok('Test::More');
@@ -20,6 +20,7 @@
can_ok pass fail eq_array eq_hash
eq_set));
isa_ok(bless([], "Foo"), "Foo");
+isa_ok(bless([], "Foo"), "Foo", "Optional description to isa_ok");
--- More.pm~Tue Sep 18 16:54:50 2001
+++ More.pm Mon Sep 24 18:25:45 2001
@@ -447,7 +447,7 @@
=item B
- isa_ok($object, $class);
+ isa_ok($object, $class, $optional_name);
Checks to see if the given $object->isa($class). Also checks to make
sure the object was defined in the first place. Handy for this sort
@@ -465,11 +465,11 @@
=cut
-sub isa_ok ($$) {
-my($object, $class) = @_;
+sub isa_ok ($$;$) {
+my($object, $class, $name) = @_;
my $diag;
-my $name = "object->isa('$class')";
+$name ||= "object->isa('$class')";
if( !defined $object ) {
$diag = "The object isn't defined";
}
Re: named tests, do_all_tests(), use autotest/selftest/testpod?
On Thu, 15 Feb 2001, barries wrote:
> What do folks think of adding something like the following to Test.pm:
>
>my $tname ; ## ok(), skip(), todo() can get the test name from here
>my $is_todo ; ## ok() could look at this and adjust it's output
>
>sub do_all_tests {
> plan tests => scalar( grep ref eq "CODE", @_ ) ;
>
> while ( @_ ) {
>$tname = ref $_[0] eq "CODE" ? undef : shift ;
>$is_todo = defined $tname && $tname eq "todo" ;
>$name = undef if $is_todo ;
>if ( defined $tname && $tname eq "skip" ) {
> skip( 1, 0, 0 ) ;
> shift ;
> next ;
>}
>shift->( $ntest ) ;
> }
>}
This is exactly what came to mind when I saw your other email describing
how you use the Test module.
I like it.
-dave
