Re: Test::Exception... comments?

[Michael G Schwern]

On Wed, Apr 10, 2002 at 10:24:32AM +0900, Curt Sampson wrote:
> On Tue, 9 Apr 2002, Adrian Howard wrote:
> 
> > lives_ok BLOCK TEST_NAME
> > Tests to see that BLOCK exits normally, and doesn't die.
> 
> I'm not sure exactly what the purpose of this is; your test will
> still fail if it dies even when not in a lives_ok block, right?

It'll fail and take the whole rest of the test program with it.  Some
testing systems like to abort the test script on failure.  Perl's
doesn't.  Aegis, in particular, pretty much requires tests to always
exit gracefully, passing or not.

It's not uncommon to do this:

ok( eval { ...some code...;  1; } );

which would be really all that lives_ok() would be.


> Not that I have any real objection to this, but I can't see where
> I'd ever use it.
> 
> The rest looks great, however. Ideally it should end up in Test::More
> at some point, I would think.

Nope.  No more functions in Test::More, too much in there already.

I want to encourage the idea of having more than one Test::* module in
use in a test.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
HA HA HA  You're all so ridiculous!  But thanks for the money!

Re: Test::Exception... comments?

[Michael G Schwern]

On Wed, Apr 10, 2002 at 11:03:27AM +0100, Mark Fowler wrote:
> On Tue, 9 Apr 2002, Adrian Howard wrote:
> 
> > Any comments before I throw it at CPAN? Sound vaguely sane?
> 
> throws_ok can take either a class name or a Regexp ref as an argument to 
> compare against.  With earlier versions of perl you can't use the qr// 
> operator, so you can't pass regex.  Maybe it would be possible to take a 
> leaf out of Test::Builder's book for this:
> 
># Check if it looks like '/foo/'
> elsif( my($re, $opts) = $regex =~ m{^ /(.*)/ (\w*) $ }sx ) {
> $usable_regex = "(?$opts)$re";
> }

You could even tear that out of Test::Builder->like and into it's own
Test::Builder->may_be_regex method so you don't have to duplicate
code.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Hey you kids, come back here!  I know who your parents are!  You wouldn't
do this if Nixon was in the White House!  Come on, I'm the Walrus, damnit!

Re: Comparing Data Structures Slopply

[Michael G Schwern]

On Wed, Apr 10, 2002 at 08:57:30PM +0900, Curt Sampson wrote:
> On Wed, 10 Apr 2002, Mark Fowler wrote:
> 
> > On Wed, 10 Apr 2002, Curt Sampson wrote:
> >
> > eq_set() is really bag comparison.
> 
> Well, my point was, it *is* a set comparison if you pass it sets.
> The problem, in my view, is that perl lets you pass it something
> which is not a set. Thus, it seems perfectly fair to me for it to
> produce undefined behaviour under such circumstances.

No, Mark is right.  It's poorly named because the author doesn't
really understand set theory.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
The prefered method of receiving a paste enema is to go to St. Mark's
Place, stand in the middle of the street, and shout "I NEED A PASTE
ENEMA!"  You'll get something, anyway.

Re: Comparing Data Structures Slopply

[Michael G Schwern]

On Wed, Apr 10, 2002 at 02:03:51PM +0100, Nicholas Clark wrote:
> On Wed, Apr 10, 2002 at 08:57:30PM +0900, Curt Sampson wrote:
> > On Wed, 10 Apr 2002, Mark Fowler wrote:
> > 
> > > On Wed, 10 Apr 2002, Curt Sampson wrote:
> > >
> > > eq_set() is really bag comparison.
> > 
> > Well, my point was, it *is* a set comparison if you pass it sets.
> > The problem, in my view, is that perl lets you pass it something
> > which is not a set. Thus, it seems perfectly fair to me for it to
> > produce undefined behaviour under such circumstances.
> 
> The API doesn't define which side is "expected" and which side "got",
> does it?

eq_set() is not a test function.  It doesn't produce "ok/not ok" and
shows no diagnostics.  It just returns true if they're equal, false
otherwise.  So there's no concept of which is "expected" and which is
"got" anymore than:

$got eq $expected
$expected eq $got


Test::More's eq_set() is just a bad name.  Don't fixate on it, write
Test module with better set handling.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It's Flypaper Licking time!

Re: Comparing Data Structures Slopply

[Michael G Schwern]

On Wed, Apr 10, 2002 at 11:32:04AM +0100, Mark Fowler wrote:
> On Wed, 10 Apr 2002, Curt Sampson wrote:
> 
> > >  b) Doing proper set comparison (not bags)
> > 
> > This I'm not sure about. You demonstrated some set comparison along
> > these lines a while back, did you not?
> 
> eq_set() is really bag comparison.  I want a test which returns true iff
> for every item in two lists there exists an item in the other list that
> is identical.
> 
> > but that you want to be using some sort of Set object and you're trying
> > to use an array instead.
> 
> Probably ;-)  I was thinking of making both lists hashkeys and then
> checking that !(%a + %b), or keys %a == keys %b and that grep { $a{$_} } 
> keys %b and grep { $b{$_} } keys %a. 

Just to rain on your parade, don't forget about sets containing
references.


PS While we're on the subject of bad names and deep comparisons.  I've
been told by some folks that actually have CS degrees that is_deeply()
is actually a "shallow" comparison.  A deep comparison would check
that the references themselves are equal.  A shallow comparison checks
that the values referenced are equal.  Go figure.

Either way, you usually want to be shallow for testing.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
If you'll mount me, I'll let you bomb Canada until they swoon.

Re: Test::Exception... comments?

[Michael G Schwern]

On Wed, Apr 10, 2002 at 10:57:09PM +0100, Adrian Howard wrote:
> Hmmm... All seems sensible. Patch attached for Builder.pm that adds a
> _may_be_regex method. Okay?

Except it should be public (if Test::Exception wants to use it) and
documented and tested. :)


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
7.  It is always something
7a. Good, Fast, Cheap: Pick any two (you can't have all three).
 -- RFC 1925

[ANNOUNCE] Test::Simple/More/Builder 0.43

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Simple-0.43.tar.gz

0.43  Thu Apr 11 22:55:23 EDT 2002
- Adrian Howard added TB->maybe_regex()
- Adding Mark Fowler's suggestion to make diag() return
  false.
- TB->current_test() still not working when no tests were run via
  TB itself.  Fixed by Dave Rolsky.

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
The prefered method of receiving a paste enema is to go to St. Mark's
Place, stand in the middle of the street, and shout "I NEED A PASTE
ENEMA!"  You'll get something, anyway.

Re: Test::Exception... comments?

[Michael G Schwern]

On Thu, Apr 11, 2002 at 10:37:10AM +0900, Curt Sampson wrote:
> On Tue, 9 Apr 2002, Michael G Schwern wrote:
> 
> > > I'm not sure exactly what the purpose of this is; your test will
> > > still fail if it dies even when not in a lives_ok block, right?
> >
> > It'll fail and take the whole rest of the test program with it.  Some
> > testing systems like to abort the test script on failure.  Perl's
> > doesn't.
> 
> I'm not entirely sure I buy this, since the framework seems perfectly
> happy to tell me that something is wrong whether I complete all
> the tests in a script or not. But it's hardly a point worth arguing.

Well, yes.  But you'd like to see the results of the rest of the rest
of the tests.  Not simply up to where your first failure was.

Testing Doctrine says that any tests after the first failure are
suspect because if one interface is failing, anything derived from
that will also be wrong.  In practice this is rarely the case and
there's plenty of useful data to be gotten from the full run.

In practice if your test cases are getting so large that any of
this matters you have Other Problems.


> > Aegis, in particular, pretty much requires tests to always
> > exit gracefully, passing or not.
> 
> As perl does: when it dies, it exits "gracefully" with error code
> 255 indicating that the test failed? Or am I misunderstanding how
> Aegis works?

Aegis works by reading the exit code of the test.  0 is success, 1 is
failure, anything is "something went horribly wrong".  The thing to
keep in mind is Aegis was designed by a pre-ANSI C programmer, so any
run-time errors that cause the program to abort are an indication that
something is Very Wrong.

The reason this becomes important is Aegis has two modes of testing.
The first is your normal "everything should pass".  But the second is
interesting.  When your change in Aegis is fixing a bug Aegis requires
that the new tests against the old code *fail*, and that the new tests
against the new code succeed.  

It has to fail gracefully, exiting in a predictable manner, in order
to be reasonably sure that new tests will catch the bug.  If the test
simply dies you don't know if that was because it caught a SIGINT or
it segfaulted or what.


Which becomes a Royal Pain In The Ass when fixing the bug involves
adding new methods and then you have to do things like:

if( Some::Class->can('this_new_method') ) {
...some tests involving this_new_method()...
}
else {
fail('this_new_method not defined');
}

which becomes Rapidly Tiresome.  This is why SKIP and TODO blocks work
the way they do (even though they're not directly applicable) and why
Test::More will exit with the number of tests which failed (makes it
much easier to write a simple Aegis perl testing hook).



> (And yes, it's not a co-incidence that this looks a lot like Beck
> and Gamma's unit test frameworks for Java and Smalltalk.)
> 
> Thoughts?

See also Test::Unit and Test::Inline.  Test::Unit because it's XUnit
in Perl.  Test::Inline because each block of embedded tests runs in
it's own lexical scope.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Obscenity is the last resort of the illiterate, Mother Fucker
-- KAL

Re: Comparing Data Structures Slopply

[Michael G Schwern]

On Thu, Apr 11, 2002 at 09:54:56AM +0900, Curt Sampson wrote:
> On Wed, 10 Apr 2002, Nicholas Clark wrote:
> 
> > The API doesn't define which side is "expected" and which side "got",
> > does it?
> 
> I believe it's defined (though perhaps not explicitly) as the first
> argument being the "got," and the second being the "expected." This
> is how the error messages print it, IIRC.

I can tell you quite authoritatively that it is not defined for
eq_set, eq_array or eq_hash.  You might be confused with is() or
is_deeply().

Again, they are *NOT TEST FUNCTIONS*.  They produce no diagnostics.
They are just normal comparision functions and becoming more and more
out of place in Test::More.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
O you fat bastard   
anus clogged (library paste)
you're not laughing now
-- Halfjack

Re: Comparing Data Structures Slopply

[Michael G Schwern]

On Thu, Apr 11, 2002 at 10:02:59AM +0900, Curt Sampson wrote:

> The whole problem could never have arisen if sets were objects
> here, rather than, pretending that an array really is a set. That's
> the point I'm trying to make.

What must be remembered is these are common testing modules for
testing common Perl data structures in common Perl programs written by
common Perl programmers.

A common Perl program will likely use a hash to represent a set.  Or
maybe they'll use one of the 16 different Set::* modules on CPAN.  Or
maybe they'll have their own set class.  Who knows?  It's not the job
of the testing module to decide.  You have to be flexible enough to
handle all those eventualities.  The most flexible way is to have your
tests accept the simplest way to represent a set in Perl: a hash.


If we had a single set class from which everyone derived set objects
we'd be Smalltalk.  TMTOWTDI.  Live it.  Love it.  Work with it.  Send
lamentations to the contrary to comp.lang.perl.misc. :)


(Really the whole problem could never had arisen if you hadn't let an
ECE dropout redesign Perl's testing infrastructure ;) )

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Death?  Its like being on holiday with a group of Germans.

Re: Starting with a fresh sandbox

[Michael G Schwern]

On Thu, Apr 11, 2002 at 12:16:27PM -0500, Andy Lester wrote:
> 
> I'm using Test::More for doing infrastructure testing throughout my
> project.  One of the tests I'm doing is making sure that every .pm file is
> able to load on its own, since I've had problems where the .pm had
> dependencies on other modules, but so long as the calling program included
> those modules first, the problem would remain hidden.

One simple way around this is to use SelfTest or Test::Inline and
throw in at the top of each of your modules:

=for testing
use_ok('My::Self');

and then just run the tests in each self-testing module in turn.


> So I have a test where I do a use_ok( 'Foo::Bar' ).  All fine and good.
> What I'd like is a way to do use_ok( 'Foo' ) and a use_ok( 'Bar' ), but
> somehow make sure that the order of checking these not matter.  Even in an
> eval {}, my namespace is going to be polluted after the first use_ok(),
> sin't it?

What you want, basically, is the ability to quickly and easily run
perl code in a seperate process.  If you're on Unix you can just
fork().  Otherwise you have to make a system() call to Perl.

In the Perl core there's a file t/test.pl.  This reimplements most of
Test::More's features, but in a much more conservative way to avoid
using the very features that you're testing.  It also contains a
handful of interesting new functions above and beyond Test::More.  One
of these is fresh_perl() which runs a given chunk of code in a fresh
Perl process in a cross-platform manner.  It's usually used to test
code that will cause a segfault, but's useful for what you want, too.

I've been meaning to make a CPAN version of this code, but I'm out of
tuits.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
11. Every old idea will be proposed again with a different name and
a different presentation, regardless of whether it works.
 -- RFC 1925

[ANNOUNCE] Test::Inline 0.15

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Inline-0.15.tar.gz

One bug fix, one hack.

First, the line numbers reported by the tests are now correct.  So
when it says the test failed on line 198, the test failed on line 198
and not 197.  The way I did it is sort of nasty, but it works.

Second, I'm having to hack in a way to support 'open(STDOUT,
">somefile")' in the code being tested.  Currently it just pukes
because STDOUT is tied to a class with no OPEN method.  With this at
least it doesn't puke, but it also doesn't do the right thing.


So the question is, what is the right thing?  I believe it's...

print "foo";# trapped and sent to $_STDOUT_.

open(SAVEOUT, ">&STDOUT");
open(STDOUT, ">somefile");
print "foo";# sent to 'somefile' as normal

open(STDOUT, ">&SAVEOUT");
print "foo";# trapped and sent to $_STDOUT_.

on the theory that you only want to capture stuff which would go to
the screen, otherwise you'll be changing the behavior of the program.

Now I just have to figure out how to implement that.


0.15  Thu Apr 11 15:45:53 EDT 2002
- Hacked in a solution for when code tries to redirect STDOUT or
  STDERR.  The behavior when redirecting STDOUT & STDERR is
  currently *UNDEFINED* but at least it doesn't blow up.
* Off-by-some line number bugs now fixed.  Embedded tests should
  report correct line numbers.  __LINE__ should now be correct.

0.14  Thu Feb 28 12:38:53 EST 2002
* pod2test now provides an $Original_File
* Fixed handling of "print STDERR ..." in tests
* Fixed $_STDERR_ and $_STDOUT_ so they clear themselves
  between test blocks (bug reported by Wolfgang Weisselberg)
- Some point between Test::More 0.30 and 0.33 it became unsafe
  to redirect STDOUT/STDERR in tests.  This broke pod2test.
  The minimum version of Test::More has been uped (again, thanks
  Wolfgang)


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
And God was pleased.
And Dog was happy and wagged his tail.
And Adam was greatly improved.
And Cat did not care one way or the other.
-- http://www.catsarefrommars.com/creationist.htm

Re: Starting with a fresh sandbox

[Michael G Schwern]

On Thu, Apr 11, 2002 at 02:49:30PM -0500, Andy Lester wrote:
> > One simple way around this is to use SelfTest or Test::Inline and
> > throw in at the top of each of your modules:
> >
> > =for testing
> > use_ok('My::Self');
> >
> > and then just run the tests in each self-testing module in turn.
> 
> The bummer is that I'm not the only programmer, and I've had problems with
> .pm files being created that don't compile on their own.

Teach the other folks how to test. :)


> I want an automated test to File::Find thru the source tree and load
> up each .pm as it gets found, making sure it'll load on its own.

Ok:

sub test {
return unless /\.pm$/;
print "# compiling $_\n";
system(qq{$^X -e 'print eval { require "$_"; 1 } ? "ok\n" : "not ok\n"' });
}

use File::Find;
find(\&test , shift || '.');

adjust as necessary.  Doesn't have to be fancy.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
sun readdened wheat stalks
bowing down in autumn sun
my mind wanders far
-- stimps

Re: [PATCH] Make Test::More::can_ok() use objects, not just classes

[Michael G Schwern]

On Sat, Apr 20, 2002 at 11:51:30AM -0700, chromatic wrote:
> Attached is a patch to make Test::More do the right thing (as I see it) in this
> case.  Previously, it called can() on the class name, which obviously doesn't
> work here.

You're right.

> I suspect something similar should be done for isa_ok().

Piers already made sure isa_ok() calls isa() directly on the object and
doesn't use UNIVERSAL::isa.  So that's all good.


> The included test fails with vanilla 0.43 and passes with the attached patch.
> 
> Maybe it would be better to bless a scalar,

I've used a simplified version of your patch and also tested that isa_ok()
honors isa() overrides.  Thanks.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
4 WHEREAS, the siren song of payola issuing from the discordant calliopes
of these gambling vessels has led thousands of Kentucky citizens to vast
disappointment and woe;
-- Kentucky Legislature, HR 256

Re: [PATCH lib/Test/Builder.pm t/output.t] "Escape" newlines in test output

[Michael G Schwern]

On Sat, Apr 13, 2002 at 04:18:24PM -0700, chromatic wrote:
> I'm not dead or pining for the fjords.
> 
> This was discussed at the end of January on p5p, and Benjamin Goldberg
> suggested the regexp solution that appears here.

For future reference, TieOut.pm is sitting in t/lib/

Here's what I've applied.

--- lib/Test/Builder.pm 12 Apr 2002 03:00:27 -  1.18
+++ lib/Test/Builder.pm 23 Apr 2002 02:16:28 -
@@ -850,6 +850,15 @@
 
 local($\, $", $,) = (undef, ' ', '');
 my $fh = $self->output;
+
+# Escape each line after the first with a # so we don't
+# confuse Test::Harness.
+foreach (@msgs) {
+s/\n(.)/\n# $1/sg;
+}
+
+push @msgs, "\n" unless $msgs[-1] =~ /\n\Z/;
+
 print $fh @msgs;
 }
 
--- t/output.t  11 Jan 2002 00:59:07 -  1.3
+++ t/output.t  23 Apr 2002 02:18:05 -
@@ -3,12 +3,15 @@
 BEGIN {
 if( $ENV{PERL_CORE} ) {
 chdir 't';
-@INC = '../lib';
+@INC = ('../lib', 'lib');
+}
+else {
+unshift @INC, 't/lib';
 }
 }
 
 # Can't use Test.pm, that's a 5.005 thing.
-print "1..3\n";
+print "1..4\n";
 
 my $test_num = 1;
 # Utility testing functions.
@@ -21,8 +24,11 @@
 $ok .= "\n";
 print $ok;
 $test_num++;
+
+return $test;
 }
 
+use TieOut;
 use Test::Builder;
 my $Test = Test::Builder->new();
 
@@ -55,3 +61,31 @@
 ok($lines[1] =~ /Hello!/);
 
 unlink('foo');
+
+
+# Ensure stray newline in name escaping works.
+$out = tie *FAKEOUT, 'TieOut';
+$Test->output(\*FAKEOUT);
+$Test->no_ending(1);
+$Test->plan(tests => 5);
+
+$Test->ok(1, "ok");
+$Test->ok(1, "ok\n");
+$Test->ok(1, "ok, like\nok");
+$Test->skip("wibble\nmoof");
+$Test->todo_skip("todo\nskip\n");
+
+my $output = $out->read;
+ok( $output eq  Kwalitee Is Job One
Woah, like, did anybody see my watch?

Re: [PATCH] use_ok() with pragmas in 5.005_03

[Michael G Schwern]

On Tue, Apr 23, 2002 at 03:28:12AM +0900, Tatsuhiko Miyagawa wrote:
> it seems that calling pragma->import() raises a warning in
> perl5.005_03. It may be a lexer bug in 5.005_03 ...
> 
>   % perl5.00503 -we 'require base; base->import("CGI")'
>   Unquoted string "base" may clash with future reserved word at -e line 1.  
> 
> P.S. Mike, the same thing applies with your mixin.pm

gotcha


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
let me think it over while Cheese beats you with a baseball bat.

Core test formatting nits need work

[Michael G Schwern]

It's been noticed that there's a lot of this going on in the core tests.

.../ext/Sys/Syslog/syslog..ok, 3/6 skipped: assuming syslog doesn't accept 
inet connections

and it's unsightly for most people's 80 column terminals.

To properly fix this would require ripping apart Test::Harness's formatting
code, not something I want to do this close to 5.8.  But if anyone has some
extra energy lying around, they could sweep through the skip messages in the
core tests and just shorten them a bit, that will hold things together for a
while.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I'm not actually Kevin Lenzo, but I play him on TV.

Test::Harness skip reason report change?

[Michael G Schwern]

Sarathy's convinced me that we need a better way to format skip reasons so
they don't just run off the end of the screen.  After a lot of hemming and
hawing on my part, we may have found an easy fix.

So here's an example of a rather busy report with 2.03.

t/sample-tests/descriptive.ok
t/sample-tests/with_comments...ok, 2/5 unexpectedly succeeded
t/sample-tests/combinedFAILED tests 3, 9 
Failed 2/10 tests, 80.00% okay (-1 skipped test: 7 okay, 70.00%)
t/sample-tests/skipok, 1/5 skipped: rain delay   
t/sample-tests/skip_no_msg.ok, 1/1 skipped: [no reason given]
t/sample-tests/skip_allskipped: rope
t/sample-tests/todook, 1/5 unexpectedly succeeded
t/sample-tests/todo_inline.ok, 1/3 unexpectedly succeeded
t/sample-tests/simple..ok
Failed Test Stat Wstat Total Fail  Failed  List of Failed
---
t/sample-tests/combined   102  20.00%  3 9
 (5 subtests UNEXPECTEDLY SUCCEEDED), 1 test and 3 subtests skipped.
Failed 1/9 test scripts, 88.89% okay. 2/39 subtests failed, 94.87% okay.


and how here's the same battery of tests run with a few simple formatting
changes.

t/sample-tests/descriptive.ok
t/sample-tests/with_comments...ok
2/5 unexpectedly succeeded
t/sample-tests/combinedFAILED tests 3, 9 
Failed 2/10 tests, 80.00% okay (less 1 skipped test: 7 okay, 70.00%)
t/sample-tests/skipok
1/5 skipped: rain delay
t/sample-tests/skip_no_msg.ok
1/1 skipped: [no reason given]
t/sample-tests/skip_allskipped
skipped: rope
t/sample-tests/todook
1/5 unexpectedly succeeded
t/sample-tests/todo_inline.ok
1/3 unexpectedly succeeded
t/sample-tests/simple..ok
Failed Test Stat Wstat Total Fail  Failed  List of Failed
---
t/sample-tests/combined   102  20.00%  3 9
 (5 subtests UNEXPECTEDLY SUCCEEDED), 1 test and 3 subtests skipped.
Failed 1/9 test scripts, 88.89% okay. 2/39 subtests failed, 94.87% okay.


It makes a noticable difference when the file path to the test and the skip
reason is long.

.../ext/I18N/Langinfo/Langinfo.skipped: I18N::Langinfo or POSIX unavail
able

vs

.../ext/I18N/Langinfo/Langinfo.skipped
skipped: I18N::Langinfo or POSIX unavailable


What do people think?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Kids - don't try this at--oh, hell, go ahead, give it a whirl...

Re: Test::Harness skip reason report change?

[Michael G Schwern]

On Fri, Apr 26, 2002 at 11:39:26AM +0400, Ilya Martynov wrote:
> Looks nice. But I'm curious if it will be possible to see skip reasons
> with new formatting if one test had several different skip
> reasons. Currently it just emits 'skipped: various reasons'.

Yes, that's a good idea (the particulars of getting the output format would
be interesting), but it will require a lot of internal rejiggering.

With the exception of one feature (HARNESS_TODOFAIL), Test::Harness is
feature frozen for 5.8.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
You're smoother than a tunnel of shining sorrow.

Re: Test::Builder & forks

[Michael G Schwern]

On Sun, Apr 28, 2002 at 02:23:11PM -0500, Dave Rolsky wrote:
> I can't think of any really elegant solutions other than adding something
> like:
> 
>   Test::Builder->reset
> 
> and calling that in child processes.

Test::Builder->no_ending(1);

use Test::More tests => 10;
my $TB = Test::More->builder;

use strict;

my $pid;
if ($pid = fork)
{
wait;
pass for 1..10;
}
else
{
$TB->no_ending(1);
exit;
}

Not particulary imaginative, but it works.  I don't deal much with forking
so my brane isn't wired to think that way.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
"Let's face it," said bearded Rusty Simmons, opening a can after the
race.  "This is a good excuse to drink some beer."  At 10:30 in the
morning?  "Well, it's past noon in Dublin," said teammate Mike
[Joseph] Schwern.  "It's our duty."
-- "Sure, and It's a Great Day for Irish Runners" 
   Newsday, Sunday, March 20, 1988

[ANNOUNCE] Test::Harness 2.04

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.04.tar.gz

This release contains the new output style for skipped tests.  We'll see how
it goes.

2.04  Tue Apr 30 00:54:49 EDT 2002
* Changing the output format of skips
- Taking into account VMS's special exit codes in the tests.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I'm exploring my nipples.

Re: Test::MockObject 0.03

[Michael G Schwern]

On Sun, Apr 28, 2002 at 01:06:18AM -0700, chromatic wrote:
> I've written a module to make it easier to do very strict unit testing.  It
> lets you fake an existing interface somewhat more easily.  For example, if
> you're testing a web app, you can mock up CGI.pm instead of having to specify
> query information manually.

We've been doing a lot of this at Mitel lately.  Below is an example of an
embedded test for method which downloads an RPM over rsync after first
requesting how to download it over an RPC session.

The obvious problem is how do you test this without having a machine for it
to actually connect to and download from.  In this case we've created a mock
RPC session object with a single method, sendRequest(), which simply traps
the user's RPC request and returns a canned response.  I also made a small
modification to the code WRT the rsync call.  Previously it was calling
rsync directly via system().  I moved that off into a function which I can
then override in the test so I can trap and examine the arguments.  Finally,
I override the identity_file() method so I can change where it's putting
it's ssh key.

So on the whole, a lot of fun glassbox testing going on here.

chromatic, it might be a nice exercise for you to work through if/how this
would be done with MockObject?



=item I

  $pkg->download($session);

Downloads a copy of the RPM this $pkg represents from the $session and
places it into $pkg->abspath.

Throws E_DOWNLOAD if there is a problem.

=begin testing

{
package Fake::Session::DL;

sub sendRequest {
my($self) = shift;
$self->{_request} = shift;
return {
response_data => {
  user  => 'someuser',
  host  => 'somehost',
  session_key => 'CkR3t k33',
 }
   };
}
}

my $session = bless {}, 'Fake::Session::DL';

no warnings 'once';
local $esmith::Blades::Package::LICENSEDIR = '30e-smith-blades';
local $esmith::Blades::Package::PKGDIR = '30e-smith-blades';

no warnings 'redefine';
local *esmith::Blades::Package::identity_file = sub {
my $self = shift;
return "30e-smith-blades/ident_".$self->pkgString
};

my @rsync_args;
local *esmith::Blades::Package::rsync_ssh = sub {
@rsync_args = @_;
return 1;
};

my $pkg = $CLASS->new('foo-bar-1.2.3-01.i386');
ok( $pkg->download($session) );
is( $session->{_request}{request_type}, 'DownloadPackageKey' );
is( $session->{_request}{request_data}{package}, $pkg->pkgString );

is( $rsync_args[0], 'somehost:' );
is( $rsync_args[1], $pkg->abspath );
is( $rsync_args[2], 'someuser' );
is( $rsync_args[3], $pkg->identity_file );
isa_ok( $rsync_args[4], 'ARRAY' );

ok( open(IDENT, $pkg->identity_file) );
is( join('', ), 'CkR3t k33' );
close IDENT;

unlink $pkg->identity_file;


=end testing

=cut

use esmith::util::system qw(rsync_ssh);
sub download
{
my $self = shift;
my $sessionObj = shift;

my $request = {
  request_type => 'DownloadPackageKey',
  request_data => { package => $self->pkgString },
};

# XXX: catch exceptions
my $response = $sessionObj->sendRequest($request);

my $dataRef = $response->{response_data};
my $remoteUser = $dataRef->{user};
my $remoteHost = $dataRef->{host};
my $sshKey = $dataRef->{session_key};
my $identity = $self->identity_file;

# save download key
open (KEY, ">$identity")
or die "Could not open $identity: $!\n";
print KEY $sshKey;
close KEY;
chmod 0600, $identity;

# ok, now we should be able to start the rsync download
my @rsync_options = (
     '-p',
 '--partial',
 '--size-only',
);

my $source = $remoteHost.':';
my $dest = $self->abspath;

rsync_ssh($source, $dest, $remoteUser, $identity, \@rsync_options)
  or throw esmith::Blades::Package::Error('E_DOWNLOAD');
}


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Let's leave my ass out of this, shall we?

[ANNOUNCE] Test::Harness 2.20

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.20.tar.gz

Version jump!  What's happened is the last of the vestigal parsing code in
Test::Harness has been eliminated.  Test::Harness is now just a big wrapper
around Test::Harness::Straps->analyze_file().  This is good.  No more
duplication of parsing code.

Well, almost the last of it.  The big, ugly end-of-test figuring code is
still there.  And I've left the old %test and %tot structures in for minimum
impact, else I would have had to simultaneously rewrite t/test-harness.t and
that would have made confirming it worked much more difficult.

This also means Test::Harness::Straps will begin to mature.  Right now, T::H
and T::H::S are very chummy.  I plan to slowly pull them apart, improving
the public Strap interface.  Test::Harness will be abstracted into just
another client.


2.20  Sat May  4 22:31:20 EDT 2002
* An almost complete conversion of the Test::Harness test parsing
  to use Test::Harness::Straps.

2.04  Tue Apr 30 00:54:49 EDT 2002
* Changing the output format of skips
- Taking into account VMS's special exit codes in the tests.

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
How can I stoop so low?  Years of practise, that's how. It's been hard
going but now I can stoop lower than a pygmy limbo dancer.
-- BOFH

[ANNOUNCE] Test::Harness 2.21 verbose broken by 2.20

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.21.tar.gz

Test::Harness 2.20 broke verbose.  This release fixes it.

2.21  Mon May  6 00:43:22 EDT 2002
- removed a bunch of dead code left over after 2.20's gutting.
- The fix for the $^X "bug" added in 2.02 has been removed.  It
  caused more trouble than the old bug (I'd never seen a problem
  before anyway)
- 2.20 broke $verbose

2.20  Sat May  4 22:31:20 EDT 2002
* An almost complete conversion of the Test::Harness test parsing
  to use Test::Harness::Straps.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One

[ANNOUNCE] Test::Harness 2.22

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.22.tar.gz

Little bug fix release.  These are old bugs that only recently became
exposed.

2.22  Fri May 17 19:01:35 EDT 2002
- Fixed parsing of #!/usr/bin/perl-current to not see a -t.
  (RT #574)
- Fixed exit codes on MPE/iX

2.21  Mon May  6 00:43:22 EDT 2002
- removed a bunch of dead code left over after 2.20's gutting.
- The fix for the $^X "bug" added in 2.02 has been removed.  It
  caused more trouble than the old bug (I'd never seen a problem
  before anyway)
- 2.20 broke $verbose

[ANNOUNCE] Test::Harness 2.23

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.23.tar.gz

One big bug, and the rest are VMS and MPE/iX fixes.  Basically, you can't
know what the wait status will look like, you can only pull it appart with
POSIX macros.

Mark, could you confirm?


2.23  Wed May 22 12:59:47 EDT 2002
- reason for all skip wasn't being displayed.  Broken in 2.20.
- Changed the wait status tests to conform with POSIX standards.
- Quieted some SYSTEM$ABORT noise leaking out from dying test tests
  on VMS.

2.22  Fri May 17 19:01:35 EDT 2002
- Fixed parsing of #!/usr/bin/perl-current to not see a -t.
  (RT #574)
- Fixed exit codes on MPE/iX


-- 
This sig file temporarily out of order.

What ever happened to Test::Set?

[Michael G Schwern]

There was a big discussion about writing a module to make it easier to test
sets and common complex data structures and superceed the paltry eq_*
functions in Test::More.
http:[EMAIL PROTECTED]/msg01369.html

Anyone working towards that?

-- 
This sig file temporarily out of order.

[ANNOUNCE] Test::Harness 2.24

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.24.tar.gz

Two bugs.

2.24  Wed May 29 19:02:18 EDT 2002
* Nikola Knezevic found a bug when tests are completely skipped
  but no reason is given it was considered a failure.
* Made Test::Harness::Straps->analyze_file & Test::Harness a bit
  more graceful when the test doesn't exist.

2.23  Wed May 22 12:59:47 EDT 2002
- reason for all skip wasn't being displayed.  Broken in 2.20.
- Changed the wait status tests to conform with POSIX standards.
- Quieted some SYSTEM$ABORT noise leaking out from dying test tests
  on VMS.

-- 
This sig file temporarily out of order.

Re: Pondering Test::Depend

[Michael G Schwern]

On Sat, Jun 08, 2002 at 10:45:50AM -0700, chromatic wrote:
> Taking an average test suite as an example, we could have 'foo.t' and 'bar.t',
> testing Foo.pm and Bar.pm respectively.  Bar depends on Foo, and bar.t mocks a
> couple of methods of Foo.
> 
> If bar.t uses Test::Depend, it'll pick up several things:
>   - the module being tested (assumed from a use_ok() or require_ok() call)
>   - the module being mocked (assumed from an entry in %INC)
> 
> If there's no dependency information, it'll use the current version of Foo.pm,
> the last modified timestamp of the test file, and the test status of foo.t (if
> it also uses Test::Depend) to keep track of any changes to Foo.pm.
> 
> If this dependency information changes, it'll fail a test (or maybe warn)
> because there's a potential interface change that Bar.pm may need to know.

It looks interesting up to this point.  Basically, everytime Bar.pm is
touched, edited or upgraded, the test will fail.  Every doc patch, typo fix
and minor upgrade.  This will produce orders of magnitude more false
negatives than real failures, which will sap the credibility of the test
causing it to be ignored.

If Perl had strong interfaces, like Java, this could be done a bit more
intellegently.  But it doesn't.

In reality, if Bar.pm changes in an incompatible way Foo.pm's own tests
should fail and that should cover you.


> Like I said earlier, I'm aiming for an 80% solution.  There are some
> assumptions and some fragilities here, but it represents a great improvement
> over the current system, in my mind.


-- 
This sig file temporarily out of order.

Re: Pondering Test::Depend

[Michael G Schwern]

On Sat, Jun 08, 2002 at 11:12:40AM -0700, chromatic wrote:
> > > If this dependency information changes, it'll fail a test (or maybe warn)
> > > because there's a potential interface change that Bar.pm may need to
> > > know.
> 
> > It looks interesting up to this point.  Basically, everytime Bar.pm is
> > touched, edited or upgraded, the test will fail.  Every doc patch, typo fix
> > and minor upgrade.  This will produce orders of magnitude more false
> > negatives than real failures, which will sap the credibility of the test
> > causing it to be ignored.
> 
> If these cause a version number bump, yes, that'll be a problem.  I was 
> unclear, though.  My plan is to use the test for the unit *providing* the 
> dependency as the most accurate sort of information.  The heuristic might be, 
> in order of ascending importance:
> 
>  - version number changes in the dependency module may mark a change
>  - timestamp changes of the test file probably mark a change
>  - a change in the number of tests (expected/passing/failed/skipped) marks a 
> change
> 
> That gives me three levels of certainty.  The important thing is that it uses 
> the depending module's tests to be more sure if something has changed.

It gives you three levels of uncertainy.  If Bar.pm is being actively
developed, the tests and version number will be changing constantly and the
dependency check will also be constantly "failing", causing it to be
ignored, or only given a cursory glance.  

If Bar.pm is relatively static, ie. from CPAN, failures will be rare but
when it does fail, it will still be false orders of magnitude more often
than it will be true.  Since the developer will have to go in and check if
the API has been broken *by hand* they will rapidly tire of it and begin
ignoring it, or only giving it a glance.  Realistically, the best you could
expect is for the developer to look at the module's change log and see if
there's any flashing "THE API HAS CHANGED" lights.


> > In reality, if Bar.pm changes in an incompatible way Foo.pm's own tests
> > should fail and that should cover you.
> 
> Not in a strict isolation testing environment, where I don't want to use the 
> real Bar.pm when testing Foo.pm.

I presume you're refering to things like IO::Socket and friends that are
frequently mocked?


> Maybe I'm being too strict about my isolation, but it's had good results for 
> me so far.

-- 
This sig file temporarily out of order.

Re: Thread-safe Test::Builder

[Michael G Schwern]

On Sat, Jun 15, 2002 at 06:00:44PM +0100, Mark Fowler wrote:
> I was, out of curiousity, wondering what happened to these changes as they 
> don't seem be in 5.8.0RC1

My time machine's in the shop awaiting a shipment of sky hooks.
(Translation: Arthur's patch came after RC1.)

As for why it's not in the CPAN version yet:  I'm lazy.


-- 
This sig file temporarily out of order.

[ANNOUCE] Test::Harness 2.25

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Harness-2.25.tar.gz

Some bleadperl fixes for the tests under MacPerl were integrated, but in a
slightly different form than the original patches.  They've been simplified
which eliminates the need for VMS specific code to fix the MacOS specific
code.

The Test::Harness::Straps part of 16938 has been defered until Pudge gets
back to be with some more data.  The patch only addresses the symptoms, the
real problem is likely elsewhere.
http:[EMAIL PROTECTED]/msg00224.html

Because of all the MacPerl fixes, Test::Harness now requires File::Spec.  So
pre-5.004_05 users will have to install File::Spec, which is likely the
least of their worries.


And Andy finally gets his global $Strap so he can begin toying with
Test::Harness::Straps subclasses.


2.25  Sun Jun 16 03:00:33 EDT 2002
- $Strap is now a global to allow Test::Harness::Straps
  experimentation.
- Little spelling nit in a diagnostic.
- Chris Richmond noted that the runtests() docs were wrong.  It will
  die, not return false, when any tests fail.  This is silly, but
  historically necessary for 'make test'.  Docs corrected.
- MacPerl test fixes from Pudge. (mutation of bleadperl@16989)
- Undef warning introduced in 2.24 on skipped tests with no reasons 
  fixed.
* Test::Harness now depends on File::Spec

2.24  Wed May 29 19:02:18 EDT 2002
* Nikola Knezevic found a bug when tests are completely skipped
  but no reason is given it was considered a failure.
* Made Test::Harness::Straps->analyze_file & Test::Harness a bit
  more graceful when the test doesn't exist.


-- 
This sig file temporarily out of order.

Re: Test::Class... comments?

[Michael G Schwern]

On Sun, Jun 16, 2002 at 12:14:59AM +0100, Adrian Howard wrote:
> 0)  Do other people find this vaguely sane? Even  possibly useful?

Yes!  xUnit with a Perl spin is something I've wanted for a long time.


> 1)  Is Test::Class the right name?

If you ignore the "it doesn't say it's xUnit" problem, it's a good name.

Since you mention xUnit in the NAME part of the docs, it should be found.


> 2)  You can have multiple setup/teardown methods. This fell out of the
> implementation, rather than being a deliberate design decision, but it
> has proved surprisingly useful (e.g. have one teardown method to clean
> up resources, another to check that class invarients still hold). Is
> there some reason why this is evil?

As long as the order is deterministic (which it is) it sounds ok.


> 3)  The fact that num_method_tests() uses the class it was called from,
> rather that the one it was applied to, seems kind of gnarly. However,
> it did seem the neatest solution to the inheritance problem mentioned
> in the documentation. How offensive do people find this, and is there
> a better solution?

I don't quite follow the problem.  Could you clarify?


-- 
This sig file temporarily out of order.

Re: [ANNOUCE] Test::Harness 2.25

[Michael G Schwern]

On Sun, Jun 16, 2002 at 03:12:17AM -0400, Michael G Schwern wrote:
> The Test::Harness::Straps part of 16938 has been defered until Pudge gets
> back to be with some more data.  The patch only addresses the symptoms, the
> real problem is likely elsewhere.
> http:[EMAIL PROTECTED]/msg00224.html

Pudge got back to me.  Chris found the real problem: MacPerl doesn't have a
working putenv.  So even though PERL5LIB is being correctly set the test
subprocess won't see it.

So I'll put 16938 in.

-- 
This sig file temporarily out of order.

Re: Fwd: Test::Class... comments?

[Michael G Schwern]

On Sun, Jun 16, 2002 at 08:28:26PM +0100, Adrian Howard wrote:
> >Now, the way I do this at the moment is have num_method_tests() walk up
> >the callstack until it finds a package that also isa Test::Class (in 
> >this
> >case Base::Test) and then assume that is the class containing the method
> >whose # tests it should alter.

What happens when you've got a subclass of a subclass of Test::Class, and
they're each overriding a method and adding a test?


> >While this works, it just seems a trifle evil. When I coded it my
> >thoughts went something like "Gosh. That's a greasy hack... must fix
> >that later". When later eventually came around the other solutions I
> >came up with seemed even worse.
> >
> >I just have a sneaking feeling that there's a more elegant (for some
> >definition of "elegant") way of handling changing the number of tests
> >for test methods that will cope with inheritance nicely.

Make no_plan the default?  Works for Test::Inline.


-- 
This sig file temporarily out of order.

Re: use_ok w/version numbers

[Michael G Schwern]

On Mon, Jul 01, 2002 at 09:57:24AM -0500, Andy Lester wrote:
> I can do this:
> 
> use PHP::Session 0.10;
> 
> to ensure a version number, but I can't do this:
> 
> use_ok( 'PHP::Session', '0.10' );
> 
> because I get this error:
> 
> alester@flr4[~/tw/Dev]$ more Modules.t
> #!/usr/bin/perl -w
> 
> use strict;
> use Test::More tests => 1;
> 
> use_ok( 'PHP::Session', '0.10' );
> 
> 
> alester@flr4[~/tw/Dev]$ perl Modules.t
> 1..1
> not ok 1 - use PHP::Session;
> # Failed test (Modules.t at line 6)
> # Tried to use 'PHP::Session'.
> # Error:  Can't locate object method "require_version" via package 
>"PHP::Session" (perhaps you forgot to load "PHP::Session"?) at 
>/usr/local/lib/perl5/5.6.1/Exporter/Heavy.pm line 105.
> # Looks like you failed 1 tests of 1.
> 
> 
> 
> Before I go digging into a patch, is this something that we even want to
> allow?  I know I'd like to allow it, because I'd like to have one .t
> file that ensures that I have proper module prereqs for my entire source
> tree.

This is due to a quirk in the way "use Foo VERSION" is handled.

 use PHP::Session 0.10;

this is handled directly by Perl itself and uses UNIVERSAL::VERSION to get
$PHP::Session::VERSION.

 require PHP::Session;
 PHP::Session->import(0.10);

In this case, PHP::Session::import would be used, or an inherited method,
usually Exporter::import.  But PHP::Session doesn't inherit from Exporter,
yet we're winding up in there anyway.

The problem is, using PHP::Session is defining a UNIVERSAL::import somehow.

  DB<1> x defined &UNIVERSAL::import
0  ''
  DB<2> use PHP::Session

  DB<3> x defined &UNIVERSAL::import
0  1

PHP::Session uses UNIVERSAL::require which has to use UNIVERSAL.pm for an
obscure ambiguity warning [1].  UNIVERSAL.pm does

require Exporter;
*import = \&Exporter::import;

so all classes suddenly have an import method, but they never inherited from
Exporter!  So Bad Things happen.  The fact that UNIVERSAL.pm has an import
method is a historical accident.

It's only limited to situations when UNIVERSAL.pm is loaded.  Since "use
UNIVERSAL" is rather pointless, I'm going to change UNIVERSAL::require so it
no longer pre-loads UNIVERSAL.pm and tell the Email::Valid author to remove
his "use UNIVERSAL" line which is what started the problem in the first
place.


[1] To see this bug, download UNIVERSAL::require 0.02 and do:

$ cd UNIVERSAL-exports-0.02
$ perl -Ilib -MUNIVERSAL::require -MUNIVERSAL -wle 1
Ambiguous call resolved as CORE::require(), qualify as such or use & at 
/usr/share/perl/5.6.1/UNIVERSAL.pm line 6.


-- 
This sig file temporarily out of order.

Re: [PATCH Test::More] Allow isa_ok( $classname, $parent )

[Michael G Schwern]

On Sun, Jun 30, 2002 at 05:37:14PM -0700, chromatic wrote:
> Here's a patch to the Test::Simple 0.45 distribution to make isa_ok() work with
> class names, not just objects.  It tries to respect custom isa() methods, as
> well.

The purpose of isa_ok() is two fold:

Check that a scalar contains an object
Check that object is of the right class

and it only exists because it's a very common test and you have to do the
above in several steps to get good diagnostics:

ok( defined $obj,   'object is defined' );
ok( ref $obj,   'object is a reference' );
ok( $obj->isa('My::Thing'), 'object isa My::Thing' );

Whereas checking a class is much simpler

ok( $class->isa('Whatever'), "$class isa Whatever" );

Worse, it adds an ambiguity into isa_ok() with the small possiblity that
$obj might contain a class string and cause a false positive.

So I'm not convinced.  Why did you find it useful?


-- 
This sig file temporarily out of order.

Re: Test::Simple fails "output" subtest 2 on Cygwin/2000

[Michael G Schwern]

On Fri, Jun 28, 2002 at 02:34:48PM -0500, Danny Faught wrote:
> I just tried to install Test::More 0.45 under Cygwin on Windows 2000. 
> It fails the output test.  Same results on two different machines, and
> also on version 0.44.  I'm running perl 5.6.1.  All other tests pass. 
> All tests, including output, pass on Cygwin on Windows NT 4.0 SP6a.
> 
> The subtest is expecting the text "hi!", but it actually gets "hi!\r" (a
> carriage return on the end).
> 
> Anyone see this before?  Is chomp not dealing with text-mode files
> properly?

Never seen it in 5.6.1, but it sounds like an odd chomp() mistake.  Where
did you get this perl?


> Also, I see a few tests are skipped because they say they need
> Test::Harness version 1.20 or 1.23.  But I just installed 1.26 - CPAN
> listed it as a prerequisite.

Did you install it or "install" it?  ie. Did you run 'make install' (or
nmake install) or just copy the .pm files somewhere?  The normal
installation process should overwrite the old Test::Harness.

PS  Test::Harness is way beyond 1.26.

-- 
This sig file temporarily out of order.

Re: RFC: Test::Warn

[Michael G Schwern]

On Thu, Jun 27, 2002 at 04:32:31PM +0200, Janek Schleicher wrote:
> I couldn't find a module doing this job on CPAN,
> so I'm ready to write a Test::Warn module.
> 
> I thought that two methods should be implemented:
> warns_ok BLOCK  REGEX, TEST_NAME   (regex and test_name are optional)
> no_warns_ok  BLOCK  TEST_NAME  (test_name is optional)
> 
> (similar to throws_ok and lives_ok from Test::Exception)

You have to be extra careful about preserving the call stack, since the
BLOCK is called inside warns_ok() as a subroutine.  Consider:

  # line 5 "foo.pl"
  warns_ok { warn "Foo" } /^Foo in foo.pl at line 5$/;
  
Gotta make sure this comes out ok.  Test::Exception does this using
Sub::Uplevel.

Rather than restricting it to a regex, you might want to write functions
analgous to Test::More:

 warn_isBLOCK  STRING, [TEST_NAME]
 warn_like  BLOCK  REGEX,  [TEST_NAME]

this also eliminates the awkward "no_warns_ok".  Instead, you just do:

 warn_is { foo() }  '',  'no warnings from foo';


-- 
This sig file temporarily out of order.

Re: [PATCH Test::More] Allow isa_ok( $classname, $parent )

[Michael G Schwern]

On Sat, Jul 06, 2002 at 06:57:27PM -0700, chromatic wrote:
> My test was:
> 
>   use_ok( 'Child::Class' );
>   isa_ok( 'Child::Class', 'Parent::Class' );
> 
> I could just as easily check @Child::Class::ISA or use UNIVERSAL::isa().

Child::Class->isa('Parent::Class')


-- 
This sig file temporarily out of order.

Re: [PATCH Test::Builder] Add diagnostic messages to details

[Michael G Schwern]

On Sat, Jul 27, 2002 at 09:44:03AM -0700, chromatic wrote:
> This patch captures messages sent through diag() and stores them in the
> diagnostic array.  Now all of the information the tests generate is available
> for later inspection.

Dude, where's my patch?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Me? A robot? That's rediculous! For one thing, that doesn't compute at all!

Re: Help spreading Test

[Michael G Schwern]

On Sat, Jul 27, 2002 at 09:55:38AM -0700, chromatic wrote:
> > One of the problems I have with using Test::Builder is that I want to
> > distribute packages to systems that do not (necessarily) have a decent version
> > of Test::* installed. Now it is easy to include a copy of a suitable version
> > of Test::Builder with the package (provided it is not too big). Would it be a
> > good idea to add a provision to Test::Builder that can be called, from the
> > Makefile.PL, to display a message like this:
> > 
> >   The verificiation tests for this package require the Test::Builder package
> >   of at least version X.Y. You do not seem to have this installed. I have
> >   included a copy of Test::Builder in this distribution that I can use for
> >   testing. Do you want me to install this version of Test::Builder as well?
> > 
> > This would help spreading the good stuff.

I'm not sure of the context here, but at this point of complexity it's
probably easier to just make it a normal prerequisite.

Why?

o If you're not using a CPAN shell, every module installation involving any
  prerequisites is going to be a chore.  Adding one more to that pile isn't
  really going to make much difference.

o At least for Test::Simple/More/Builder, so many modules already have
  dependencies on them (including CPAN, CPANPLUS and Storable) that you
  probably already have them installed.  There's a slide in the
  Test::Tutorial talk with a list current as of a month ago.

o If you don't already have them installed, you only need to do it once.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
5. It is always possible to aglutenate multiple separate problems
   into a single complex interdependent solution. In most cases
   this is a bad idea.
-- RFC 1925

Slides from OSCON

[Michael G Schwern]

Figured folks might be interested in slides from talks at OSCON:

http://www.pobox.com/~schwern/talks/How_To_Be_Lazy_Redux/
http://www.pobox.com/~schwern/talks/Test_Tutorial/
http://www.pobox.com/~schwern/talks/Writing_A_Test_Library/

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Home of da bomb

Re: [perl #15479] perl 5.8.0 segfault

[Michael G Schwern]

On Wed, Jul 31, 2002 at 12:30:12PM +0100, Nicholas Clark wrote:
> On Wed, Jul 31, 2002 at 01:20:25PM +0200, Rafael Garcia-Suarez wrote:
> > Wasn't there a .t file to run separate perl interpreters
> > and test for core dumps ?

We're dissolving that.  Use fresh_perl_is() and fresh_perl_like() from
t/test.pl and place the test in the appropriate group.  So somewhere with
other filehandle tests.


> I keep forgetting that I need to remember to ask this. Is there a FAQ
> for regression test writing? Well, an guide to "so I want to write a
> regression test, explaining how to do it, how perl5's tests are structured
> to reduce interdependencies, use Test::More; when Test::More is not
> appropriate.."

Porting/patching.pod

About the only thing that's missing is docs for t/test.pl.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
We have returned to claim the pyramids.

Thoughts from TPC: Testing perlguts with Inline::C

[Michael G Schwern]

More chronicles of brainstorming from TPC.

Last year at TPC, Hugo asked for ideas about testing the internal C
functions of Perl.  Testing perl itself is ok, but if we can test at an even
finer grained level of the C API many bugs can be more easily caught.
Additionally the documentation will be improved in the process (can't test
it if you don't know what it's supposed to do) and the XS mechanism might
also be improved as more people have to touch it to test it.

At the time, there was nothing to do this except write tests in C, and that
sucks.  It's hard enough to find people to write tests in Perl.

Then Ingy comes along with Inline::C.  One of the tricks he pulled off at
YAPC::Europe was writing a simple Inline module which exposed all the
internal C functions as perl functions.  AH HA!  Now we can test C functions
just like any other Perl function.

So, this is a request to place Inline::C into 5.9 to facilitate easy testing
of the C API.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
IMHO bugs in Perl 5 shouldn't carry over to Perl 6. (Unless, of course,
we *like* the bugs... ;)
-- Ken Fox in <[EMAIL PROTECTED]>

Re: [PATCH MakeMaker.pm] Add documentation for ExtUtils::MakeMaker::prompt()

[Michael G Schwern]

On Sun, Aug 18, 2002 at 06:48:24PM -0700, chromatic wrote:
> Here's a rough cut at documentation for the hitherto rumored prompt() 
> function.  Suggestions welcome.

Something like so.

--- MakeMaker.pm18 Jul 2002 17:46:50 -  1.64
+++ MakeMaker.pm21 Aug 2002 05:39:38 -
@@ -2224,9 +2224,34 @@
 dependency in a CPAN::Bundle, but the functionality is supported by
 different means on the current architecture).
 
+=head2 Other Handy Functions
+
+=over 4
+
+=item prompt
+
+my $value = prompt($message);
+my $value = prompt($message, $default);
+
+The C function provides an easy way to request user input
+used to write a makefile.  It displays the $message as a prompt for
+input.  If a $default is provided it will be used as a default.  The
+function returns the $value selected by the user.
+
+If C detects that it is not running in interactively (say,
+if it is running from a CPAN shell), or if the PERL_MM_USE_DEFAULT
+environment variable is set to true, the $default will be used without
+prompting.  This prevents automated processes from blocking on user
+input.
+
+If no $default is provided an empty string will be used instead.
+
+=back



-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Tasty, yet morally ambiguous.

Re: [PATCH] Test::Builder holds object inside

[Michael G Schwern]

On Thu, Aug 22, 2002 at 10:29:25PM +0900, Tatsuhiko Miyagawa wrote:
> Here's a patch for Test::(Simple|More) to allow idioms like
> 
>   my $obj = Foo->new;
>   ok($obj, 'object is defined');
> 
> Without this patch, passed $obj was referenced in Test::Builder, so
> REFCNT doesn't come to 0 before global cleanup.

Whoa, nice catch!  And thanks for catching the missing t/has_plan's in the
MANIFEST. :)


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
gigaconway: a hypothetical unit of mind expansion, so disturbing it
  is likely to change the fundemental nature of the universe.  Some
  contend that gigaconway events, while rare, are much cheaper to produce
  than antiprotons, nuclear weapons or even XML specifications, and start
  at US$60,000 each.  If you believe gigaconway events are indeed possible,
  please send your tax deductable contributions to:

The Conway Fund,
c/o Yet Another Society
http://www.yetanother.org/
-- Ziggy  

Re: Add Test::Harness 2.x Prerequisite to Test::Simple?

[Michael G Schwern]

On Sat, Aug 24, 2002 at 03:08:08PM -0700, chromatic wrote:
> I've been using Test::Exception on a project and am very glad to have it.  I
> ran into a small issue trying to install it, though: it has TODO tests, but
> those failed as the existing version of Test::Harness (1.26 or so) did not
> understand them.
> 
> Since TODO tests are a feature of Test::More, perhaps it would be correct to
> mark Test::Harness 2.x as a prerequisite for the Test::Simple bundle.  I almost
> sent Adrian a patch doing this for Test::Exception, but it seems that
> Test::Simple is the best place to handle this.

Test::Harness 2.03 is a prerequsite of Test::More.

Either it didn't get installed for some reason or your old version is
shadowing.  T::H somewhere around 1.2x installed itself into site_perl, not
the core, so you might have a leftover.  Try a "make install UNINST=1".


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
"In my day, we didn't have none of yer new-fangled token-pasting
operators!  We stuck the tokens together with our BARE HANDS and we
LIKED IT."
--  Mark-Jason Dominus fondly remembers perl 1.0
in <[EMAIL PROTECTED]>

Re: [PATCH MakeMaker.pm] Add documentation for ExtUtils::MakeMaker::prompt()

[Michael G Schwern]

On Thu, Aug 22, 2002 at 06:55:46AM -0500, Dave Rolsky wrote:
> > +If C detects that it is not running in interactively (say,
> > +if it is running from a CPAN shell), or if the PERL_MM_USE_DEFAULT
> > +environment variable is set to true, the $default will be used without
> > +prompting.  This prevents automated processes from blocking on user
> > +input.
> 
> In my experience, it still prompts under the CPAN shell, but not CPANPLUS.

Ok, I'll just pull out the reference to the CPAN shells and just say "if
it's not running interactively".  Let the CPAN shell authors figure it out.
They should probably have a configuration option.


> I've actually come to rely on the former behavior and was surprised that
> CPANPLUS didn't work the same way, causing it to be basically imposssible
> to do a fresh install of Alzabo via CPANPLUS.  How about a way to _force_
> it to prompt or give up if that's really impossible?

A timeout argument to prompt() might be worthwhile.  And an accompanying
environment variable.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I've just gone through a lung-crushing breakup with my blender and I don't 
think I should screw my forehead alone tonight.

Re: Add Test::Harness 2.x Prerequisite to Test::Simple?

[Michael G Schwern]

On Mon, Aug 26, 2002 at 12:48:45AM +0100, Adrian Howard wrote:
> I've added a patch to Test::Exception (should have spotted that... *bad*
> Adrian :-). It should be hitting CPAN in the next few minutes.

You shouldn't need it, but it can't hurt.


> Can anybody give me a definitive version of when TODO tests started
> working in Test::Harness? From the Changes file I'm currently assuming
> everything after Test::Harness 2.03 inclusive should be okay.

Old-style TODO tests were added long ago, somewhere between 5.4.4 and 5.4.5.
New-style TODO tests, as we know them, were added somewhere between 1.19 and
1.22.  In 1.23 they worked reasonably well.

2.03 is the last version in which a bug in TODO parsing was fixed.  That's
the minimum version you should be requiring for TODO test support.


> Personally, I would tend towards leaving the prerequisites to
> Test::Builder alone. I think I'm right in saying that everything
> except TODO tests work fine with the earlier Test::Harness, so it
> seems unhelpful to prevent people using the rest of the distribution.

I'm all for leaving them alone since they already contain a prereq on
Test::Harness.  The problem is the installation is almost always screwed up,
an old version of T::H winds up shadowing.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It's Airplane Glue sniffing time!

[ANNOUNCE] Test::Simple 0.47

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Simple-0.47.tar.gz

Two major bugs were added in 0.46.  Objects would accidentally be stored
internally causing destructors not to fire when people expected them to.
Also ithread support was broken.  These have both been fixed.


0.47  Mon Aug 26 03:54:22 PDT 2002 
* Tatsuhiko Miyagawa noticed Test::Builder was accidentally storing 
  objects passed into test functions causing problems with tests 
  relying on object destruction.
- Added example of calculating the number of tests to Test::Tutorial
- Peter Scott made the ending logic not fire on child processes when
  forking.
* Test::Builder is once again ithread safe.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I do have a cause though.  It is obscenity.  I'm for it.
-- Tom Lehrer "Smut"

Re: Testing POSIX locale support

[Michael G Schwern]

On Mon, Aug 26, 2002 at 04:05:26PM -0400, John Peacock wrote:
> Specifically, in order to test that the locale stuff is working, I need to 
> have two different locales installed to switch between.  Currently, I am 
> using en_US and en_GB, but obviously that ignores most of the planet.
> 
> I was hoping to get some suggestion for how I can go about being sensitive 
> about how I do these tests, so that I get as few misleading failures as 
> possible.  One thought I had was to test for a series of locales during the 
> Makefile.PL processing, then update the t/test.t file for some combination 
> of the installed locales.  The problem I have already identified is that 
> `locale -a` may report erroneous information (partially installed locales), 
> so I am going to have to poke around some more.

Perhaps you can run a small suite of checks against things you find in the
locale list to be reasonably sure the locale is valid and fully installed
and pick two which seem to be so.


> In particular, I know I am going to have to skip that testing entirely on 
> platforms without any locale support (like CygWin), but I am still going to 
> have some issues where I may only have a single locale fully installed to 
> test.  I need something like the opposite of TODO, so that I can have 
> failures but warn that those failures are acceptable with caveats.  I 
> really need 'ok', 'not ok', and 'not ok but not to worry too much' so I can 
> handle all the various possibilities.

If you're going to have a test which suggests you simply ignore the failure
then just don't bother running the test.  A success tells you it's working
but a failure doesn't tell you it's broken, so it's not worth much.

What you should probably do is emit further diagnostic information on what
the test failure might mean like:  

is($this, $that) || diag( Kwalitee Is Job One
Monkey tennis

Fwd: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]

[Michael G Schwern]

There's already the beginnings of something like this in t/TestInit.pm, but
it wasn't updated for the PERL_CORE trick.  Nor do I know if it can be
entirely trusted.

Also, Nick's example is a little odd.  You usually don't want '.' (ie. t/)
in your @INC.  It's more like this:

BEGIN {
if($ENV{PERL_CORE}) {
chdir 't';
@INC = '../lib';
}
}

but in some cases you need to include something more.  For example,
MakeMaker does this:

BEGIN {
if($ENV{PERL_CORE}) {
chdir 't';
@INC = ('../lib', 'lib');
}
else {
unshift @INC, 't/lib';
}
}

so it can see specialty helper modules in t/lib/.


- Forwarded message from Ken Williams <[EMAIL PROTECTED]> -

From: Ken Williams <[EMAIL PROTECTED]>
Date: Thu, 29 Aug 2002 14:08:00 +1000
To: Nicholas Clark <[EMAIL PROTECTED]>
Subject: Re: [Inline 0.43] insecure dependency when tainting

On Thursday, August 29, 2002, at 06:51 AM, Nicholas Clark wrote:
>You'll often see regression tests in the core start like this:
>
>sub BEGIN {
>if ($ENV{PERL_CORE}){
>   chdir('t') if -d 't';
>   @INC = ('.', '../lib');
>} else {
>   unshift @INC, 't';
>}
>
>
>if $ENV{PERL_CORE} is set, then the test knows that it's being 
>run as part
>of a core build, and so it should force @INC to only find uninstalled
>modules in the core tree.

Seems like it would be nicer to set that externally, rather than 
in every test file.  Maybe some code like this could be added to 
blib.pm or something?  Then one could do something like this:

 perl -Mblib=only t/foo.t


>Sorry if I've wasted your time by telling you something you 
>already knew,
>or had inferred from Hugo's reply.

I might have inferred it, but this helped me understand various 
implications, actually.

 -Ken




- End forwarded message -

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Try explaining that to my brain which is currently ON FIRE!
http://www.goats.com/archive/010704.html

Re: Help spreading Test

[Michael G Schwern]

On Wed, Aug 28, 2002 at 09:32:10PM -0700, chromatic wrote:
> Perhaps we are looking at this from the wrong direction.  Instead of 
> installing bundled modules, perhaps a shell should be able to specify that 
> certain dependencies are available only for testing.  That way, users 
> wouldn't necessarily have to install Test::Builder and Test::Builder::Tester 
> and so forth in the public directories.

I think David hit the nail right on the head.

  Given what the Test::* modules are for, I think that this is common. And
  since they're small and won't waste much in the way of resources once 
  they've been installed, I personally think it's no big deal to install 
  them.

Just install the things.  They're small and you only have to do it once.

A CPAN shell will automate the process so it's rather moot.  And if you're
not using a CPAN shell every module install is going to be a chore anyway.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I blame myself.  AND SATAN.

Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]

[Michael G Schwern]

On Thu, Aug 29, 2002 at 06:23:26PM +1000, Ken Williams wrote:
> I admit that I was a little surprised to see 'lib' stuff here 
> and not 'blib' stuff.  Don't blibs get created when building 
> modules in the core?

Everything winds up in lib/ before the tests are run.  All non-XS modules
are never really "built", they're already in lib/.  All XS modules are built
and their results put into lib/ and then the tests are run.

So, effectively, the core "installs" every modules into lib/ then runs all
their tests.  This is so module A's tests can find module B.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Help, help!  A Heffalump, a Horrible Heffalump!  Help, help, a Herrible 
Hoffalump!  Hoff, Hoff, a Hellible Horralump!

Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])

[Michael G Schwern]

On Fri, Aug 30, 2002 at 05:54:15PM +1000, Ken Williams wrote:
> Oh, one big lib/, not several different ones?  So then why can't it be 
> run with
> 
>  perl -Mblib=lib t/foo/bar.t
> 
> ?

Because tests all try to run from t/.  That's what the chdir 't' is for.
t/TEST does this for you and so do the tests themselves which makes it much
more forgiving.  Otherwise it would be very picky and this:

$ cd /path/to/src/perl
$ ./perl -Ilib t/foo/bar.t

wouldn't work when you ran a test manually.  You'd have to do:

$ cd /path/to/src/perl/t
$ ./perl -I../lib foo/bar.t

and always remember to run from t/.


> I'm just trying to push the decision into a more appropriate place, 
> though I guess in the end it's not such a big deal.

This is already effectively done by the TestInit module but nobody trusts
it.  t/TEST will run each test with -MTestInit.  t/harness does not so that
will have to be somehow fixed.

I usually do exactly that when I run them manually:

./perl -Ilib lib/Foo/whatever.t.

but that -Ilib isn't really necessary since the test does it.  If we remove
the BEGIN block code and place that elsewhere things get more complicated
when trying to run an individual test:

./perl -It -MTestInit lib/Foo/whatever.t

or

./perl -I. -MTestInit ../lib/Foo/whatever.t


What it boils down to is how to you remove the need for the cargo-cult BEGIN
block in each test while still making it easy to run individual tests by
hand?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Tasty, yet morally ambiguous.

Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]

[Michael G Schwern]

On Fri, Aug 30, 2002 at 06:25:27PM +1000, Ken Williams wrote:
> This is because I've been frustrated a number of times by modules that 
> died on CPAN when they got integrated into the core, but needed bugs to 
> be fixed, but wouldn't see new versions until perl got maintenance 
> releases...

All that really requires is someone to pick up the torch and release new
versions to CPAN.  And keep the CPAN versions and perl versions in sync. As
much as they're supposed to, p5p doesn't treat the CPAN version as the
canonical one and always forgets to report patches back to the author.

Keeping them in sync with p5p's patches is the hardest part.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
There is a disurbing lack of PASTE ENEMA on the internet.

Re: Thought

[Michael G Schwern]

On Fri, Aug 30, 2002 at 10:30:22AM +0200, H.Merijn Brand wrote:
> I was thinking about highjacking a standard function: read ()

Giving read() semantics completely unrelated to reading a filehandle would
be a bad choice of syntax.  So would making it a keyword.  Could you do it
(mostly) via XS?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Do you have a map? Because I keep getting lost in your armpits.

Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])

[Michael G Schwern]

On Fri, Aug 30, 2002 at 11:57:16AM +0100, Nick Ing-Simmons wrote:
> Michael G Schwern <[EMAIL PROTECTED]> writes:
> >On Fri, Aug 30, 2002 at 05:54:15PM +1000, Ken Williams wrote:
> >> Oh, one big lib/, not several different ones?  So then why can't it be 
> >> run with
> >> 
> >>  perl -Mblib=lib t/foo/bar.t
> >> 
> >> ?
> 
> blib was invented to allow just such usage.
> 
> >
> >Because tests all try to run from t/.  That's what the chdir 't' is for.
> >t/TEST does this for you and so do the tests themselves which makes it much
> >more forgiving.  Otherwise it would be very picky and this:
> 
> If you look at it you will see it tries to hunt down the right things
> allowing for fact you may have chdir'ed into t (or even deeper).

It's not just getting @INC right.  The tests must be run from t/ (ie. a
canonical directory they're always run from) so that they can find certain
files properly and so that they only scribble scratch files inside t/.

For example, lib/Test/Simple/t/exit.t has to find some sample programs to
test against in t/lib/Test/Simple/sample_tests.  How can it find it if it
doesn't know where it was run from?  You have to get into things like
looking at $0 and trying to walk your way up to the top level source
directory.  Icky.

The existing t/TestInit.pm takes this into account by simply chdir'ing to
t/.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I do have a cause though.  It is obscenity.  I'm for it.
-- Tom Lehrer "Smut"

Re: Test::More::can_ok when using AutoLoader

[Michael G Schwern]

On Fri, Aug 30, 2002 at 04:01:11PM +0200, Elizabeth Mattijsen wrote:
> Is there an easy way to check methods, whose loading is deferred with 
> AutoLoader, with the can_ok() check?

Nope.  You have to create stubs.  The AutoLoader module should do this
automatically for you assuming you've gone through the full build process
and AutoSplit has had a chance to do it's job.

$ Makefile.PL
Writing Makefile for AutoExample

$ make
cp lib/AutoExample.pm blib/lib/AutoExample.pm
AutoSplitting blib/lib/AutoExample.pm (blib/lib/auto/AutoExample)

$ cat blib/lib/auto/AutoExample/autosplit.ix 
# Index created by AutoSplit for blib/lib/AutoExample.pm
#(file acts as timestamp)
package AutoExample;
sub foo  ;
1;

$ perl -Mblib -wle 'use AutoExample;  print "Yes" if AutoExample->can("foo")'
Using /home/schwern/tmp/AutoExample/blib
Yes

$ perl -Ilib  -wle 'use AutoExample;  print "Yes" if AutoExample->can("foo")'
Can't locate auto/AutoExample/autosplit.ix in @INC (@INC contains: lib 
/usr/local/lib/perl/5.6.1 /usr/local/share/perl/5.6.1 /usr/lib/perl5 /usr/share/perl5 
/usr/lib/perl/5.6.1 /usr/share/perl/5.6.1 /usr/local/lib/site_perl .) at 
/usr/share/perl/5.6.1/AutoLoader.pm line 147.
 at lib/AutoExample.pm line 5


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It's Mainline Heroin time!

Re: Test::More::can_ok when using AutoLoader

[Michael G Schwern]

On Sat, Aug 31, 2002 at 02:05:53PM +0200, Elizabeth Mattijsen wrote:
> >$ perl -Mblib -wle 'use AutoExample;  print "Yes" if 
> >AutoExample->can("foo")'
> >Using /home/schwern/tmp/AutoExample/blib
> >Yes
> 
> Hmmm... I'm doing BEGIN { use_ok( 'Thread::Pool' ) }...  Maybe there is a 
> difference there...

Hmmm.  Shouldn't be a difference.

> Ok, so it _should_ work.  I'll see if I can boil this down and create a 
> bugreport if I can.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Out of ammunition.  God save the King.

Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])

[Michael G Schwern]

On Mon, Sep 02, 2002 at 08:13:06AM +0100, Nick Ing-Simmons wrote:
> I understand all that. My point was that while test itself may care 
> where it is run, blib.pm does not mind as much. Also blib.pm's job 
> is to make running an un-installed module "easy" which is what you 
> want to do for a pre-install test. So if there is boiler-plate stuff 
> to do blib.pm is an appropriate place to do it.
> 
> Thus is you need to be in t this might suit
> 
>   cd t; perl -Mblib=lib foo/bar.t
> 
> would work, and I would be more than happy it it looked for a t directory 
> as well.

There's a chicken/egg problem here.  blib.pm is going to find the lib/
directory so we can load uninstalled modules.  So... how do you find
blib.pm?

I believe the above worked for you because you happen to have an already
installed perl in the same spot as the new one is going to go.  If you move
that out of the way, no good.

Other problems include that we can't assume blib.pm works.  Don't want the
whole basic test suite falling apart because of a module failure.

Finally, since you have to run from t/ anyway, blib.pm is overkill.  This
works just as well:

cd t;  ./perl -I../lib foo/bar.t

Anyhow, t/TestInit.pm *already does this*.

schwern@blackrider:/usr/local/src/perl-current$ ./perl -It -MTestInit -wle 'print 
@INC;  use Cwd;  print cwd;'
.../lib
/usr/local/src/perl-current/t

While the result is still ugly, it means we can expand and alter the
requirements for running a core test.  For example, the PERL_CORE
environment variable should be set (t/TestInit.pm currently doesn't).  So
the full command is really something like this:

cd t;
PERL_CORE=1 ./perl -I../lib path/to/test.t

Which could be reduced somewhat to:

./perl -It -MTestInit t/path/to/test.t

and perhaps reduced a little further by moving/linking TestInit.pm into the
top level directory.

./perl -MTestInit t/path/to/test.t

but that will cause trouble when running tests with -T (because . is no
longer in the path).

When tests are run using t/TEST (ie. "make test") are run with TestInit.  So
strictly speaking the BEGIN block is redundant.  t/harness (ie. "make
test_harness") is currently not using TestInit.  There's currently a bug
where $Test::Harness::switches is not honored.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
MERV GRIFFIN!

Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])

[Michael G Schwern]

On Tue, Sep 03, 2002 at 11:26:01AM +1000, Ken Williams wrote:
> >>Thus is you need to be in t this might suit
> >>
> >>  cd t; perl -Mblib=lib foo/bar.t
> 
> I'd be happier if that 'cd t;' happened inside blib.pm as 'chdir "t"'.

It already happens inside TestInit.


> I'd be happier still if tests didn't run from t/, since that's 
> not how most CPAN modules work.

The trouble here is we don't want tests running from the top level.  Too
much chance of them overwriting something they shouldn't be, or writing a
file and forgetting to clean it up.  Or touching the wrong directory, etc...
So t/ is used as a safer location.  It's also used just because that's what
all the other tests do, so there's some cargo-cult/needless tradition.

So they can't run from the top level directory, and at that point you're
already different than CPAN.  However, we can chdir for them, that's not
hard.


> >There's a chicken/egg problem here.  blib.pm is going to find the lib/
> >directory so we can load uninstalled modules.  So... how do you find
> >blib.pm?
> 
> This doesn't seem like a real problem.  The test suite can find 
> it because it knows where it's being run from.  It can set up 
> @INC however it wants, to make sure blib.pm is found.  Then 
> blib.pm sets things up for the rest of the tests.

I thought the whole idea was to run blib.pm to set @INC.  If the test suite
is setting up @INC we don't need blib.  And if we're not using blib to setup
@INC there's no point in shoving extra functionality unrelated to finding
modules into it.  Put it into something else.  Like TestInit.pm, which
already has all this.


> >Other problems include that we can't assume blib.pm works.  
> >Don't want the
> >whole basic test suite falling apart because of a module failure.
> 
> For core perl, we can indeed assume blib.pm works.  If it 
> doesn't, we want to know about it right away.

You want to be able to continue with basic testing even in the event of a
catastrophic failure.  There was a big discussion along these lines about at
what point in the tests we can start relying on modules, this was about
using things like Test and Test::More in basic core tests.  The result was
that 'make test' should rely on as little as possible and I agree with that.

Worse, blib relies on Cwd.pm and File::Spec.  If any of these fail the test
suite doesn't work at all.  No good.


> >Finally, since you have to run from t/ anyway, blib.pm is 
> >overkill.  This
> >works just as well:
> >
> >cd t;  ./perl -I../lib foo/bar.t
> 
> I have to wonder why it's preferable to put code on command 
> lines rather than in modules.

As mentioned below, it's not.  That's why it's already in TestInit.pm (are
you sensing a theme here?) I was just spelling out the equivalent command to
show blib wasn't buying us anything.


> Really, it would be nice if you could tell the module about 
> certain situations, like -MTestInit=core or whatever, and that 
> would do whatever's necessary for a core test run (including 
> chdir, @INC adjustments, etc.).

TestInit is a module only available and useful in the core for doing the
special little things necessary to run a CPAN-style test as a core test.
There wouldn't be anything to do otherwise outside the core.


> Heck: with a smart enough TestInit, it could even adjust @ARGV 
> so that you could specify all *.t paths in Unix format, if that 
> would be helpful.

Hmmm.  I can't think of a use.


> >and perhaps reduced a little further by moving/linking 
> >TestInit.pm into the
> >top level directory.
> >
> >./perl -MTestInit t/path/to/test.t
> >
> >but that will cause trouble when running tests with -T (because . is no
> >longer in the path).
> 
> '.' won't necessarily be in the path anyway.

Sorry, I ment perl's @INC.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Not king yet

Re: Looking for tool like pydoc to document perl cgi scripts

[Michael G Schwern]

On Mon, Sep 02, 2002 at 01:01:24PM -0400, Jeff Kowalczyk wrote:
> Is there an application for perl cgi scripts that will generate formatted
> HTML documentation similar to what I can get with pydoc?
> http://web.lfw.org/python/pydoc.html

POD & perldoc.  See the perldoc and perlpod man pages.  It can produce HTML
and lots of other formats.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Still not king

Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])

[Michael G Schwern]

On Tue, Sep 03, 2002 at 03:34:39PM +1000, Ken Williams wrote:
> >I thought the whole idea was to run blib.pm to set @INC.  If 
> >the test suite
> >is setting up @INC we don't need blib.
> 
> blib.pm only adds to @INC.  Seems like the core requires 
> something to strip @INC down to a small set, not add to it.

@INC only contains paths to where the libraries will be installed.  Until
installation happens, perl can't find it's own not-yet-installed libraries
without help.

We remove all other entries from @INC so that previously installed modules
don't interfere with testing the new versions.


> Anyway, what I meant was that @INC needs to be one value to find 
> blib.pm, and then blib.pm changes @INC to be another value, so 
> there's no chicken/egg problem.

There is no other value to be changed to.  All you need to do is find lib/


> >>'.' won't necessarily be in the path anyway.
> >
> >Sorry, I ment perl's @INC.
> 
> That's what I meant too.  People can build Perl without '.' in 
> @INC, right?

I don't think so.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
If your module passes test
You're one of the very best
Don't fuck up the MANIFEST
Burma-Shave
- ignatz

Re: TEST_VERBOSE

[Michael G Schwern]

On Tue, Sep 03, 2002 at 07:08:25PM -0700, David Wheeler wrote:
> In fact, other than the Test::Class method names, TEST_VERBOSE adds 
> nothing to the output -- not even diag messages. So I've added this to 
> the script that runs my test scripts through Test::Harness:
> 
>   use Test::Harness qw(runtests $verbose);
>   $verbose = $ENV{TEST_VERBOSE} ? 1 : 0;
> 
> But what am I missing? I know that a lot of people on the cpan-testers 
> list ask for tests to be re-run with TEST_VERBOSE=1, but I can't see 
> that it actually makes much difference. Should it?

TEST_VERBOSE, to Test::Harness, is not an environment variable.  It's a make
attribute used by MakeMaker to set $Test::Harness::verbose like you have
above.  

make test TEST_VERBOSE=1

That's what the folks on cpan-testers are asking for.

HARNESS_VERBOSE is the environment variable Test::Harness responds
to.

Test::Class uses the TEST_VERBOSE environment variable to print out the name
of each test before it's run.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It's Tobacco time!

Re: Test::More

[Michael G Schwern]

On Tue, Sep 24, 2002 at 05:57:41PM +0200, H.Merijn Brand wrote:
> 1. use_ok should have an entry in the manual for minimal version
> 
>   use_ok ("Test::More", 0.47);

This currently doesn't work quite right.  Observe...

$ perl -MTest::More -wle 'plan tests => 1;  use_ok("Text::Soundex", 0.20)'
1..1
not ok 1 - use Text::Soundex;
# Failed test (-e at line 1)
# Tried to use 'Text::Soundex'.
# Error:  Import directive '0.2' was not recognized at (eval 1) line 3
# Looks like you failed 1 tests of 1.

The trouble is use_ok does this:

require "Text::Soundex";
Text::Soundex->import(0.20);

this is different than "use Text::Soundex 0.20".  In the latter, Perl
handles the version check itself.  But in the former, Text::Soundex's import
routine does the work.  Normally, Exporter will emulate the version check
but if you're not using Exporter, as Text::Soundex does not, anything could
happen.

For similar reasons, Test::More can't check itself.

$ perl -MTest::More -wle 'plan tests => 1;  use_ok("Test::More", 0.40)'
1..1
not ok 1 - use Test::More;
# Failed test (-e at line 1)
# Tried to use 'Test::More'.
# Error:  You tried to plan twice!  Second plan at (eval 1) line 3
# Looks like you failed 1 tests of 1.
schwern@blackrider:~/src/devel/Test-Simple$ 


I'll have to put in some special case code to make it work right.


> 2. I'm testing conversions to and from Unicode

Unicode... wasn't he the bad guy in the Transformers Movie?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Kids - don't try this at--oh, hell, go ahead, give it a whirl...

Re: Test::More

[Michael G Schwern]

On Mon, Sep 30, 2002 at 09:12:14AM +0200, H.Merijn Brand wrote:
> use_ok ("Test::More");
> is ($Test::More::VERSION, 0.47, "Test::More version check");
> 
> ain't quite the same either. 

> Some of my tests indeed *require* a certain version of some::module.
> Someting to add to Test::More?

Requiring a specific version of a module is rare.  Write a Test::Version if
you like.  BTW  You want to check using UNIVERSAL::VERSION.
Test::More->VERSION.


> > > 2. I'm testing conversions to and from Unicode
> > 
> > Unicode... wasn't he the bad guy in the Transformers Movie?
> 
> \N{INVERTED QUESTION MARK LEFT}

http://www.unicron.com/mainpage/unicron/unicron.htm

In "The Transformers: The Movie" Unicron was this robotic planet that ate
other planets.  This wouldn't be interesting except his voice was played by
Orson Wells in one of his last performances.

Anyhow, I know more about Unicron than Unicode.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Perl_croak(aTHX_ "Believe me, you don't want to use \"-u\" on a Macintosh");
-- toke.c

Re: [ Memory ] Re: Thought

[Michael G Schwern]

On Thu, Oct 03, 2002 at 11:39:42AM -0400, Green, Paul wrote:
> H.Merijn Brand [mailto:[EMAIL PROTECTED]] writes:
> > So far, all I got was criticism. I asked for it. But no-one said it was
> useful.
> > (Or I didn't read between the lines enough).
> 
> In my experience, the phrase "I really liked your idea and I hope you won't
> mind if I share a few ideas for improving it" needs to be inserted before
> almost anything critical that an engineer says. I've just taken to assuming
> that the fact that people are giving narrow criticisms is a sign they like
> the overall picture. And my life is now better for realizing these two
> truths.   :-)

We're Perl programmers.  Praise is implicit, like everything else in the
language. :)


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
The desired effect is what you get when you improve your interplanetary 
funksmanship.

Re: [ Memory ] Re: Thought

[Michael G Schwern]

On Thu, Oct 03, 2002 at 10:43:43AM +0100, Nicholas Clark wrote:
> The reason I'm saying it might not be much use to people unfamiliar with
> the internals of unix is precisely because it does return a precisely defined
> but not directly useful value - the highest address on the heap.
> If you read it, call perl code that creates a known perl structure, and read
> it again, doesn't go up directly and exactly related to the amount of memory
> that structure needed. Depending on how much other memory there is free, it
> may not go up at all, or it may go up more if the allocation system grabs a
> big block of free space.

Ok, now I'm really confused.

- sbrk() returns, effectively, the size of the heap.
- the heap may not go up when you create a perl structure

but MemUsed() is shown to be used like this:

use Devel::Internals;

my @array = (1..1);
MemUsed "creation of array"; 
my @dup = @array;
MemUsed "duplication of array";

and MemUsed, in scalar context, returns the sbrk value, the size of the
heap.  So I presume in the above code it simply returns the difference
between the sbrk value between the two calls.  But Nick just said this isn't
actually the memory used, it's just the different in heap size.  But the
function is called MemUsed!

Brane hurt!

A lot of the problem here is the expectations from the names.

- MemUsed() implies we're going to be able to figure out the amount of
memory used by a hunk of Perl code without knowing a lot of magic.  Instead,
we get something about heap sizes.

- sbrk() implies dark memory magic.  Most people know it's just some memory
thing beyond free() and malloc().  When they see it, they get scared, and
rightfully so apparently.

The problem is when you put those two next to each other, one promising a
friendly interface, one a bare-metal interface, it confuses the intent of
the module.  Is it for Joe End User or is it for Joe Core Hacker?

It would seem this module is for Joe Core Hacker, and it's all about sbrk().
So let's make that clear in the names:

 NAME
 
 Devel::sbrk - Access sbrk values

 SYNOPSIS

use Devel::sbrk;
my $end = sbrk (); 
my @array = (1..1);
print "The creation of this array used ",
sbrk () - $end, "bytes\n"; 

# Easier
use Devel::sbrk;

my @array = (1..1);
sbrkChange "creation of array"; 
my @dup = @array;
sbrkChange "duplication of array";


Ironically, I'm now advocating sbrk() as a name. :)  Nick is right, if
sbrk() is something special, call it that.  With this interface, nobody's
going to complain that sbrkChange() didn't actually return the amount of
memory used by "my @dup = @array;"

You might want to generalize the concept a bit to a more general "heap size"
module.  That way on systems where you don't have sbrk but can query the
heap size it will still make sense.  I don't know if they are quite the same
thing, so I just leave it as a suggestion.


 NAME
 
 Devel::Memory::Heap - Access the size of the memory heap

 SYNOPSIS

use Devel::Memory::Heap;
my $end = HeapSize ();   
my @array = (1..1);
print "The creation of this array used ",
HeapSize () - $end, "bytes\n"; 

# Easier
use Devel::Memory::Heap;

my @array = (1..1);
HeapChange "creation of array"; 
my @dup = @array;
HeapChange "duplication of array";

A more broad Devel::Internals or a more user-friendly Devel::Memory can be
left for later, and the functionality of Devel::Memory::Heap can be put into
it if it makes sense.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Here's some scholarly-ass opinions...

Re: [ Memory ] Re: Thought

[Michael G Schwern]

On Wed, Oct 02, 2002 at 01:50:56PM +0200, H.Merijn Brand wrote:
>  SYNOPSIS
> use Devel::Internals;

A little broad.  Perhaps Devel::Memory?


> my $end = sbrk ();
> 
> my @array = (1..1);
> print "The creation of this array used ",
> sbrk () - $end, "bytes\n";
> 
> # Easier
> use Devel::Internals;
> 
> my @array = (1..1);
> MemUsed "creation of array";
> 
> my @dup = @array;
> MemUsed "duplication of array";
> 
>  DESCRIPTION
>   sbrk
>   returns the current memory top. See sbrk(2) manual page

I realize that sbrk() is a familiar concept to deep C programmers on
BSD-style systems, but in order for this to be useful to Perl programmers,
or even users not familiar with BSD, you're going to have to name it
something more descriptive than sbrk() and explain better what it does.

It can still do the same thing as sbrk(), and you might even want to leave
in an alias to sbrk() so BSD core hackers can feel all warm and fuzzy.


>   MemUsed ($msg)
>   Used in void context reports the number of bytes used since the
>   previous call to MemUsed with the message passed

Global state is a Bad Thing.  Consider when you have two different,
unrelated subroutines both calling MemUsed() in the same program.  foo()
calls MemUsed() then bar() then foo() calls MemUsed() again expecting to get
the memory used since it's last call but it's actually getting the memory
used since bar() last called it.

If you're going to keep state like this, better just have an object to
represent various views on the memory state.

I can see two interfaces for this, possibly parallel.  One is similar to
your MemUsed() except the state is stored perl object.  It simply returns
the memory used since the last check on that object.

  package A;
  my $mem_state = Devel::Memory->new;

  sub foo {
  printf "%d has been used since the last call to foo()\n",
 $mem_state->used_since_last;
  }

The other one allows you to more easily track the memory usage of a block of
code by saying "track from this point to this point".  Here's an example of
tracking the memory used inside a subroutine.

  # meanwhile, in another file
  package B;
  my $mem_state = Devel::Memory->new;

  sub bar {
  $mem_state->start;
  ...do some stuff...
  printf "%d was used inside bar()\n", $mem_state->stop;
  }

For convinence, stop() returns the amount of memory used so you can do it in
two steps (start, stop/report) rather than three (start, stop, report).

The other nice part about having an object is you can then hang lots of
methods off it without polluting the caller's namespace.  So you can have
your methods which print neatly formatted diagnostics seperate from your
methods which simply return values.


>   Used in scalar context returns the current sbrk value

Shouldn't sbrk() do that?  Is this the same value as MemUsed() would print
in void context?


>   Used in list context returns the values saved on every call

?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
And God was pleased.
And Dog was happy and wagged his tail.
And Adam was greatly improved.
And Cat did not care one way or the other.
-- http://www.catsarefrommars.com/creationist.htm

Re: [ Memory ] Re: Thought

[Michael G Schwern]

On Wed, Oct 02, 2002 at 10:26:32PM +0200, H.Merijn Brand wrote:
> > >  SYNOPSIS
> > > use Devel::Internals;
> > 
> > A little broad.  Perhaps Devel::Memory?
> 
> My intent was to gather more internal states. Whatever useful.

That's a potentially very large number of exported functions and a very
large interface.  Best to partition it off.

Besides, you can always write Devel::Internals later and plug Devel::Memory
into it.

Also, there's already an Internals.pm


> > >  DESCRIPTION
> > >   sbrk
> > >   returns the current memory top. See sbrk(2) manual page
> > 
> > I realize that sbrk() is a familiar concept to deep C programmers on
> > BSD-style systems, but in order for this to be useful to Perl programmers,
> > or even users not familiar with BSD, you're going to have to name it
> > something more descriptive than sbrk() and explain better what it does.
> 
> That's why there /is/ an Internals.pm: to hide the gory details and
> implementation specifics to the user.

But you're not.  You're just exposing sbrk(), which is a gory detail. My
sbrk man page describes sbrk as being used to find "the current location of
the program break" which means nothing to me.  Nor does "returns the current
memory top".

It'll be even worse on an OS which doesn't have sbrk, which means no sbrk(2)
man page to look at.

Does sbrk() just return the current number of bytes allocated to the
program?


> > unrelated subroutines both calling MemUsed() in the same program.  foo()
> > calls MemUsed() then bar() then foo() calls MemUsed() again expecting to get
> > the memory used since it's last call but it's actually getting the memory
> > used since bar() last called it.
> 
> Bad luck. we don't have any tool to separate.

You should be able to seperate it easily, unless I really don't understand
sbrk() (which is entirely possible).  Here's a sketch implementation of
start(), stop() and used_since_last().

sub start {
my $self = shift;
$self->{start} = $self->sbrk;
return;
}

sub stop {
my $self = shift;
return $self->sbrk - $self->{start};
}

sub used_since_last {
my $self = shift;
my $sbrk = $self->sbrk;
my $used = $sbrk - $self->{last};
$self->{last} = $sbrk;
return $used;
}

Should it be anything significantly more complicated than that?


> > If you're going to keep state like this, better just have an object to
> > represent various views on the memory state.
> 
> No way. This was meant to be a *low* level interface to help you find memory
> hogs and other possible snakes. I see no use whatsoever in any production
> environment. That's why I went for Devel:: namespace

It may not be turned on when the code is run in production, but end-users
will want to use it in their normal day-to-day code.  For that you'll need
something a bit less cryptic and prone to error than the current MemUsed()
implementation with a global state.

If this is just a tool for p5p core hackers, then please don't eat the broad
name Devel::Internals.


> > >   Used in scalar context returns the current sbrk value
> > 
> > Shouldn't sbrk() do that?  Is this the same value as MemUsed() would print
> > in void context?
> 
> Complaints enough already about sbrk (). MemUsed () should hide the
> implementation. That's why.

The MemUsed() docs simply say it returns what sbrk() returns.  You're just
shuffling the complexity around, MemUsed() users still have to understand
what sbrk() does.


> MemUsed ("blah"); # void context
> 
> should print a message to STDERR
> 
> while
> 
> my $mem = MemUsed ("now");# Scalar context
> 
> should print nothing, but return the sbrk value, hiding the sbrk call to the
> user, but still putting it on the history stack.

Functions which print something in one context and return something in
another are a very bad idea.  You're conflating form and functionality and
mixing in context magic just to confuse the issue some more.


> > >   Used in list context returns the values saved on every call
> > 
> > ?
> 
> See another answer in this thread proposing to return a list of lists. I like
> that.

Seperate out the three different pieces of functionality, returning the
current memory state, printing a memory diagnostic and returning the memory
history, into three different functions.  They are operating on the same set
of data, but doing three different things with it.

Smashing the three together makes the function more complicated, the
documentation more complicated, the testing more complicated and the use
more prone to subtle bugs such as forgetting that foo(MemUsed); should
probably be foo(scalar MemUsed);


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One

Re: Test::Class - comments wanted

[Michael G Schwern]

On Sun, Oct 13, 2002 at 04:01:10PM -0700, David Wheeler wrote:
> On Sunday, October 13, 2002, at 10:05  AM, Tony Bowden wrote:
> >>  Makes it simpler for people who prefer the 'no_plan' style of
> >>  testing
> >
> >Maybe this is what I just don't get. I'm not one of those people, so I
> >don't really understand why people might prefer it. Especially here
> >where there's such a natural way to specify them, and you're only
> >counting them per method, rather than over the entire test.
> 
> I have to agree with Tony. I think it's important to explicitly 
> indicate the number of tests that a given method runs, and to be 
> explicit about saying when you're not sure how many tests there will 
> be.

The reason I went with no_plan in Test::Inline was that unlike a dedicated
test script, a T::I test is cobbled together from a series of seperated
blocks of tests and it's more difficult than usual to count them all and add
them up.  Adds another step to writing more tests.

Test::Class shares that cobbled together nature, but if it has a way to
specify them on a per-block basis, that will work without much trouble.


OTOH, my thinking recently is that the explicit plan has become obsolescent.
[1]

The explicit plan protects against:

1. Your test dying.
2. Your test not printing tests to STDOUT
3. Exiting early via exit().

#1 and #2 are protected by other implicit mechanisms in Test::Builder.  #3
will be protected against by overriding CORE::exit in a future version.

For the negatives to the explicit plan:

1. Makes writing SKIP blocks more difficult
2. Makes writing complex, iterative/looping tests more difficult

Of course, no_plan requires Test::Harness to be upgraded which has been
causing some problems lately.  Something I'm moving to fix by making the
Test::Harness upgrade non-optional when you install Test::More.  But that's
an implementation issue and will eventually fade away.


[1] This thinking makes me nervous, so I'm open to someone convincing me
otherwise.

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Here's hoping you don't become a robot!

Some upcoming changes to Test::Builder

[Michael G Schwern]

I'm working on Test::Builder 0.48 and here's some relatively major changes
I'm putting in.  Some advance warning and a chance for folks to convince me
otherwise.


* Test::Harness will no longer be optional.

Currently, the tests do not require a Test::Harness upgrade.  They simply
skip over anything that requries T::H 2.0.  This has caused a lot of
problems lately with botched and shadowed T::H installs and people
installing Test::More without realizing T::H wasn't really upgraded and
tests which have 'no_plan' not working.

The real reason I made the T::H upgrade optional was it made it easier to
test against older perls.  I could just run the test suite against, for
example, a clean 5.5.3 without having to dirty the installation by
installing T::H 2.0.  I've figured a way around this with MakeMaker hackery.

A future version of Test::Harness will include a post-install check to make
sure it actually got installed properly and isn't being shadowed, so it will
help the problem from that end.


* threads will no longer be loaded by default.

Currently, Test::Builder loads threads.pm if available.  It was done this
way because otherwise one would have to remember to load threads before
Test::More (or other T::B based module) in order for it to work properly.

Unfortunately, this makes it impossible to test unthreaded code with
Test::More when perl is built with threads as I recently found out while
trying to test threads::shared in its disabled state.  As I'm going for
maximum flexibility, and loading threads.pm by default makes a whole class
of tests impossible, threads.pm will no longer be loaded by default.

5.8.0's threads are giving me serious headaches.  When 5.8.1 comes out I
might drop support for 5.8.0's threads just so I can remove a large volume
of work-around code.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
List context isn't dangerous.  Misquoting Gibson is dangerous.
-- Ziggy

Re: Some upcoming changes to Test::Builder

[Michael G Schwern]

On Tue, Oct 15, 2002 at 10:34:26AM +0100, Nicholas Clark wrote:
> On Mon, Oct 14, 2002 at 09:00:42PM -0400, Michael G Schwern wrote:
> > 5.8.0's threads are giving me serious headaches.  When 5.8.1 comes out I
> > might drop support for 5.8.0's threads just so I can remove a large volume
> > of work-around code.
> 
> Leaving support for 5.005 threads in? I'm confused.

Test::Builder has never supported 5.005 threads.  At least, I haven't done
anything special to make it work.


> Or do you mean dropping all automatic threading support?

What I mean is in order to make Test::Builder work with 5.8.0's ithreads I
had to hack around a whole lot of bugs and write some really awful code.
Most of it having to do with automatic array/hash extensions not being
shared.  For example:

# 5.8.0 threads bug.  Shared arrays will not be auto-extended 
# by a slice.  Worse, we have to fill in every entry else
# we'll get an "Invalid value for shared scalar" error
for my $idx ($#Test_Results..$Expected_Tests-1) {
my %empty_result = ();
share(%empty_result);
$Test_Results[$idx] = \%empty_result
  unless defined $Test_Results[$idx];
}

That whole block is just a work around for a 5.8.0 ithreads bug.  There's
more like it to make Test::Builder work transparently with ithreads.  Here's
another:

my %result;
share(%result);
%result = (
'ok'  => 1,
actual_ok => 1,
name  => '',
type  => 'skip',
reason=> $why,
);
$Test_Results[$Curr_Test-1] = \%result;

which was originally written:

$Test_Results[$Curr_Test-1] = {
'ok'  => 1,
actual_ok => 1,
name  => '',
type  => 'skip',
reason=> $why,
};

and could not be written:

$Test_Results[$Curr_Test-1] = &share {
'ok'  => 1,
actual_ok => 1,
name  => '',
type  => 'skip',
reason=> $why,
};

because threads::shared::share() is broken when threads are disabled.

Really irritating stuff.  Every release since 0.45 has added more ithread
work arounds.

So when 5.8.1 comes out and ithreads work better I plan on dropping support
for 5.8.0's still broken ithreads like a hot potato so I can remove all that
hackery.  5.8.1's less buggy ithreads will be supported, hopefully with a
minimum of hackery.  Users will simply have to upgrade, which they really
should do anyway if they're using threads.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I'm spanking my yacht.

Re: callbacks at the end of Test::More ?

[Michael G Schwern]

On Sat, Oct 26, 2002 at 04:22:49PM +0100, Nicholas Clark wrote:
> However, I'd like to be able to cleanly print out my random number seed to
> STDERR (or whatever Test::Builder's correct name for this is) if it believes
> that any tests have failed, and I can't see a clean way to do this.

In an END block, grab the T::B object and check the summary().

  #!/usr/bin/perl -w

  use Test::More tests => 1;

  pass('foo');

  END { 
  my $failed = grep!$_, Test::More->builder->summary;
  diag $failed ? "Failed\n" : "Passed\n";
  }

The only trouble there is if the test as a whole fails during the ending,
like because the wrong number of tests were run or the exit code was odd,
the above logic is too simple to know that.  You have to go through the
sort of logic _ending() does.

Not sure to add in an ending callback or an is_passing() method.  Or both.


BTW  How does one get the current srand if you use the one set by Perl?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
I'm not actually Kevin Lenzo, but I play him on TV.

Test::Builder->level

[Michael G Schwern]

At YAPC::Europe there was some discussion about Test::Builder->level,
$Test::Builder::Level and the fact that they don't really work well as
implemented.  I know we reached some sort of consensus about how to do it
better, but I've forgotten it.

Anyone remember?

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
We're talkin' to you, weaselnuts.
http://www.goats.com/archive/000831.html

Upcoming change to Test::Simple/Builder/More wrt threads

[Michael G Schwern]

In 0.48 the behavior of Test::Simple/More/Builder with regard to threads
will change.

Previously, if you were using a perl >= 5.8.0 and have ithreads compiled,
loading Test::More would load threads and threads::shared.  This was to
avoid the problem of a user doing this:

  use Test::More;
  use threads;

and then wondering why the test output was confused.  Test::More must know
threads are enabled when it compiles so it can share the appropriate
variables.

Unfortunately, this makes it impossible to use Test::More or any
Test::Builder derived module to explicitly test code with threads off on an
ithread enabled Perl.  I discovered this problem when I tried to test
threads::shared without threads.pm loaded.  Couldn't do it.  Had to use
Test.pm.

As the current attempt at DWIM leaves Test::Builder unable to test a certain
class of code, it must be changed.  It will be necessary to load threads.pm
before Test::More or Test::Builder if you wish to use threads in your tests.

   use threads;
   use Test::More;


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
GOD made us funky!

And for the mod_perl folks... Test::Builder->reset

[Michael G Schwern]

Coming soon in Test::Builder, reset()!  This will allow a Test::Builder user
to reset the internal state of the test making it much easier to run tests
in persistent environments.  mod_perl users have Alex Francis to thank for
this feature.

Also, for those who aren't happy with the fact that Test::Builder is a hard
singleton with its state held in a bunch of file-scoped lexicals (hard to
debug) reset() made me collect together all those state variables in one
point in the source making it much easier someone to convert them into
object data, if they feel so inclined.  I'm not yet convinced its a good
idea.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Home of da bomb

[ANNOUNCE] Test::Simple/More/Builder 0.48_01

[Michael G Schwern]

http://www.pobox.com/~schwern/src/Test-Simple-0.48_01.tar.gz

Since there's a lot of internal and external change going on in this
release, I decided to put out an alpha first.

The external API changes:

* The Test::Harness upgrade is no longer optional.
* threads.pm is no longer automatically loaded

And new features.

* Test::Builder->reset
* use_ok() now does proper version checks
* 'no_diag' switch for Test::More

Did I miss anything?  I had a lot of patches to catch up on.

I'm punting on Nick's ending callback idea until next release.


0.48_01  Mon Nov 11 02:36:43 EST 2002
- Mention Test::Class in Test::More's SEE ALSO
* use_ok() now DWIM for version checks
- More problems with ithreads fixed.
* Test::Harness upgrade no longer optional.  It was causing too
  many problems when the T::H upgrade didn't work.
* Drew Taylor added a 'no_diag' option to Test::More to switch
  off all diag() statements.
* Test::Builder/More no longer automatically loads threads.pm
  when threads are enabled.  The user must now do this manually.
* Alex Francis added reset() reset the state of Test::Builder in 
  persistent environments.
- David Hand noted that Test::Builder/More exit code behavior was
  not documented.  Only Test::Simple.

0.47  Mon Aug 26 03:54:22 PDT 2002 
* Tatsuhiko Miyagawa noticed Test::Builder was accidentally storing 
  objects passed into test functions causing problems with tests 
  relying on object destruction.
- Added example of calculating the number of tests to Test::Tutorial
- Peter Scott made the ending logic not fire on child processes when
  forking.
* Test::Builder is once again ithread safe.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
kiloconway: unit of extreme mind expansion.  Equal to 1024 conways,
  one kiloconway gives the sensation that all of the quantuum particles
  in your brain have been spontaneously converted into Mars bars (IN
  CONSTANT TIME!).  The presumed existance of kiloconway scale events
  has led many to believe that higher forms of intelligence do in
  fact exist, and are living quite happily in Rural Australia.
-- Ziggy

Re: And for the mod_perl folks... Test::Builder->reset

[Michael G Schwern]

On Mon, Nov 11, 2002 at 09:05:13AM -0800, chromatic wrote:
> > Also, for those who aren't happy with the fact that Test::Builder is a hard
> > singleton with its state held in a bunch of file-scoped lexicals (hard to
> > debug) reset() made me collect together all those state variables in one
> > point in the source making it much easier someone to convert them into
> > object data, if they feel so inclined.  I'm not yet convinced its a good
> > idea.
> 
> We *could* add a method called really_create_a_new_builder() that doesn't have
> the singleton properties, but what problem does that solve?  As long as we're
> stuck with test numbers, we have to try not to confuse Test::Harness.

Little known fact: test numbers are optional.


I know people have grumbled before about there being no way around the
singleton property, I just can't remember why.  I'm doing that a lot lately.
Maybe I should check my hospital receipt, make sure all they took out was my
appendix.  What's this scar on the back of my head...?


Ok, here's one from me.

It would make Test::Builder easier to test itself for one.  There's lots of
T::B tests which deliberately throw it off into some bizarre state or cause
lots of failures.  These results must be pipped off somewhere and analyzed
seperately.  For those tests, we're back to using a hand-rolled ok()
function.

With two T::B objects, we could beat on one object while leaving another in
a normal state to perform the tests.


The real reason why I put all the data into lexicals rather than a hash is
because its easier to type $Have_Plan than $self->{Have_Plan}.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
11. Every old idea will be proposed again with a different name and
a different presentation, regardless of whether it works.
 -- RFC 1925

Re: And for the mod_perl folks... Test::Builder->reset

[Michael G Schwern]

On Mon, Nov 11, 2002 at 02:51:00PM -0800, chromatic wrote:
> On Monday 11 November 2002 14:40, Michael G Schwern wrote:
> 
> > > We *could* add a method called really_create_a_new_builder() that doesn't
> > > have the singleton properties, but what problem does that solve?  As long
> > > as we're stuck with test numbers, we have to try not to confuse
> > > Test::Harness.
> 
> > Little known fact: test numbers are optional.
> 
> With the mandatory Test::Harness upgrade, yes.

No, they've always been optional.  Don't try to pin this one on me! :)

$ cat ~/tmp/nonums.t
#!/usr/bin/perl -w

print <<'TEST';
1..3
ok
ok
ok
TEST

$ perl5.4.0 -MTest::Harness -e 'runtests @ARGV' nonums.t
nonums..ok
All tests successful.
Files=1,  Tests=3,  0 secs ( 0.00 cusr  0.02 csys =  0.02 cpu)


> My point is that mixing.  Test::Builder outputs is bad juju:

Test::Builder->new would remain as a singleton.  We'd just provide an
alternate constructor to provide a second object if someone really wants it.


> > I know people have grumbled before about there being no way around the
> > singleton property, I just can't remember why.  I'm doing that a lot
> > lately. Maybe I should check my hospital receipt, make sure all they took
> > out was my appendix.  What's this scar on the back of my head...?
> 
> And why are you beeping?

I get Comedy Central if I stand on my head.


> > Ok, here's one from me.
> >
> > It would make Test::Builder easier to test itself for one.  There's lots of
> > T::B tests which deliberately throw it off into some bizarre state or cause
> > lots of failures.  These results must be pipped off somewhere and analyzed
> > seperately.  For those tests, we're back to using a hand-rolled ok()
> > function.
> >
> > With two T::B objects, we could beat on one object while leaving another in
> > a normal state to perform the tests.
> 
> Simplifying the tests there is a *good* reason.  They scare me.
> 
> I suppose we could also log only certain kinds of tests, too.  That might make
> some of my fiendish plans easier.  (Log only can_ok(), in this T::B object.)

Hmmm... can_ok() isn't a T::B method.  Ooops.

In theory you could write a T::B subclass which calls each method in a
wrapper to change the output.  Something like:

package Test::Builder::Log::like;

sub like {
my($self) = shift;

my $old_output_fh = $self->output();
my $old_fail_fh   = $self->failure_output();
my $old_todo_fh   = $self->todo_output();

$self->output('some_output.log');
$self->failure_output('some_output.log');
$self->todo_ouptut('some_output.log');

local $Test::Builder::Level = $Test::Builder::Level + 1;
my $ret = $self->SUPER::like(@_);

$self->output($old_output_fh);
$self->failure_output($old_fail_fh);
$self->todo_output($old_todo_fh);

return $ret;
}

Something like that, obviously made more generic .

Hmmm.  All those *output calls are really irritating.  I think I'll change
it to something like this instead:

package Test::Builder::Log::like;

my %outputs = map { ($_ => 'some_output.log') } qw(test failure todo);
sub like {
my($self) = shift;

# Add outputs() to combine all three *output methods
# together.  Also, have it return the former set of
# filehandles.
my %old_fhs = $self->outputs(%outputs);

local $Test::Builder::Level = $Test::Builder::Level + 1;
my $ret = $self->SUPER::like(@_);

$self->outputs(%old_fhs);

return $ret;
}


> Maybe we're too coarsely grained on the singletonness.  It's only the test 
> counters and outputs that really need to be single, right?  We could make 
> Test::Builder::Counter and Test::Builder::Output as singletons and let 
> Test::Builder use those by default.  For the tests, we could mock them or 
> override them, or whatever we find necessary.

I think its more complicated than that.  Depends on how its being used.
Right now my use would be to effectively have two independent tests running
in parallel in the same process.  One test being the beating which is
testing Test::Builder and the results are trapped, the other test doing the
testing of the first.

In this case, you don't even want the counter and output as singletons.  You
want complete seperation.  Two tests running in parall

Re: And for the mod_perl folks... Test::Builder->reset

[Michael G Schwern]

On Tue, Nov 12, 2002 at 01:31:43AM +, Mark Fowler wrote:
> > Test::Builder->new would remain as a singleton.  We'd just provide an
> > alternate constructor to provide a second object if someone really wants it.
> 
> You know, that would at the very least tidy up Test::Builder::Tester 
> somewhat.  It's currently just a bit of a hack...(sssh, I didn't really 
> say that)

That's two votes.


> > > > With two T::B objects, we could beat on one object while leaving
> > > > another in a normal state to perform the tests.
> 
> One day (maybe not this rewrite, maybe not the next) I'd like to see you
> be able to write layers of tests.  That is to say that you have some kind
> of Test::Harness like system running actually inside a Test::Builder test
> that runs another set of tests and returns "ok" or "not ok" if all *its*
> tests returned "ok" or not.  And then you could have subtests within that,
> down to an arbitary level of abstraction.  Does this make sense?

Test::Harness::Straps->analyze* already do that if I understand you
correctly.


> Not that I'm one to break OO boundaries, but by allowing of breaking the
> OO boundaries just a little bit by skipping the accessors you could just
> do this, which means you are less likey to shoot yourself in the foot with 
> forgeting to reset the outputs back again.
> 
>package Test::Builder::Log::like;   
>
>my %outputs = map { ($_ => 'some_output.log') } qw(test failure todo);
>sub like {
>  my ($self) = shift;
> 
>  local $self->{outputs} = \%outputs;
>  local $Test::Builder::Level = $Test::Builder::Level + 1;
> 
>  return $self->SUPER::like(@_)
>}

Oh, to have real end-of-scope conditions.


> > > > The real reason why I put all the data into lexicals rather than a hash is
> > > > because its easier to type $Have_Plan than $self->{Have_Plan}.
> 
> http://magnonel.guild.net/~schwern/talks/Refactoring/slides/slide015.html
> 
> ;-)

The nice part about refactorings is they're reversable.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
That which stirs me, stirs everything.
-- Squonk Opera, "Spoon"

Re: Removing CPAN's dependency on Test::More

[Michael G Schwern]

On Wed, Nov 13, 2002 at 11:32:21AM +1100, Ken Williams wrote:
> So why does CPAN.pm depend on Test::More anyway?  It only uses 
> it during its 'make test' phase, not during runtime operations.
> And when you can't install modules very easily because you don't 
> have CPAN.pm working properly, it's, well, not very easy to 
> install other modules. ;-)
> 
> Seems like one of the following would be better:
> 
>  1) Include a copy of Test::More and Test::Harness in a t/lib/ 
> directory and use that

This is easiest.  You should only have to throw Test::More/Builder in there
and slap a "use lib 't/lib'" in mirroredby.t and Nox.t.  The tests aren't
using any features which require a Test::Harness upgrade.


>  2) Rework the two Test::More-using tests (mirroredby.t and 
> Nox.t) to just use Test.pm
>
> Or is this a sneaky way to get more people to install Test::More 
> on their machines?

CPAN::MakeMaker?


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Ooops, fatal mutation in the test script.

Re: Test::Builder->level

[Michael G Schwern]

On Tue, Nov 12, 2002 at 04:21:38PM +, Adrian Howard wrote:
> >At YAPC::Europe there was some discussion about Test::Builder->level,
> >$Test::Builder::Level and the fact that they don't really work well as
> >implemented.  I know we reached some sort of consensus about how to do it
> >better, but I've forgotten it.
> 
> Yet another suggestion to add to the list of possible options.
> 
> Most of the time when I need to mess with level, the code that does
> the test is sitting in a different package from the one that is
> calling the test. How about having it walk up the call stack until it
> find a package that differs from the one it was called in?
> 
> No nasty messing with numbers and you don't have to fiddle with it
> every time you add/remove a sub in the call chain.

ETOOMAGICAL and it makes it impossible to have something like this:

# Test code
use Test::Foo 'test_func';
test_func($stuff);


# Test/Foo.pm
package Test::Foo;

use Test::Foo::Bar 'real_test_func';

sub test_func {
...
real_test_func(@_);
...
}


# Test/Foo/Bar.pm
package Test::Foo::Bar
use Test::Builder;
my $TB = Test::Builder->new;

sub real_test_func {
...
$TB->ok($test);
...
}

a practical example would be an Exporter/Exporter::Heavy poor-man's
autoloader setup.

Of course, there's nothing stopping you from overriding level() to be
magical, once I implement your ideas about being able to change the default
object and stack them.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
You'll do.

Changing T::B behavior (was Re: And for the mod_perl folks... Test::Builder->reset)

[Michael G Schwern]

On Tue, Nov 12, 2002 at 11:04:53PM +, Adrian Howard wrote:
> O! Just had an idea prompted by reset(). How about having
> 
>   $builder->push_state;
>   $builder->pop_state;
> 
> that would store and restore the complete state of the builder object
> in a stack? Would make it easier to isolate a set of tests from the main 
> sequence.

I think I'll merge your $Test::Builder::default idea with this.  Have a
stack of Test::Builder objects rather than just the states.  This way you
can do something like the above.  If you add a $tb->copy method which
returns a carbon-copy of $tb then...

$tb->push_state;  # calls $tb->copy and pushes it into the stack
  # the copy will now be used by default
$tb->pop_state;   # pops the stack and tosses it, restoring the
  # original.

but what if you want to use your own subclass?  Well if you also add a
$tb->state method which return the full internal state it makes it possible
for subclasses to copy each other.  So...

$tb->push_state(My::Test::Builder->copy($tb));
...my test code...
$tb->pop_state;

The problem there is the case where you want to override behaviors but still
keep state between the two objects.  So things like the test counter and
test details would have to be preserved.  I guess this is what chromatic was
talking about when he suggested having only certain parts of the internal
state be a singleton.

So looking at the stuff in reset():

sub reset {
my ($self) = @_;

$Test_Died = 0;
$Have_Plan = 0;
$No_Plan   = 0;
$Curr_Test = 0;
$Level = 1;
$Original_Pid = $$;
@Test_Results = ();
@Test_Details = ();

$Exported_To= undef;
$Expected_Tests = 0;

$Skip_All = 0;

$Use_Nums = 1;

($No_Header, $No_Ending) = (0,0);

$self->_dup_stdhandles unless $^C;

return undef;
}

We can seperate them out into Test::Builder configuration and test state.

This is test state and would be shared amongst the T::B objects in the
default stack

$Test_Died
$Have_Plan
$No_Plan
$Curr_Test
$Original_Pid
@Test_Results
@Test_Details
$Expected_Tests
$Skip_All

and these are T::B configuration values and wouldn't be shared, though they
would be initially copied.

$Level
$Exported_To
$Use_Nums
$No_Header
$No_Ending
the output filehandles

So I guess we need several constructors.

real constructor  - a "normal" constructor
singleton constructor - returns a single, default object.  This object
is actually a wrapper around the default object
stack.
copy  - copies the state of one object to a new one
copy & share state- like copy, but test state is shared between the
original and the copy

with those plus the push/pop_stack methods you can pick and choose what sort
of state sharing you want.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
My enormous capacity for love is being WASTED on YOU guys
-- http://www.angryflower.com/497day.gif

Re: Changing T::B behavior

[Michael G Schwern]

a new Test::Builder object, plug in a
default Test::Builder::State object and... not sure what to do about the
config.  Share it?  Copy it?  From where?


> >copy   - copies the state of one object to a new one
> 
> use Storable qw(dclone);
> my $clone_tb = Test::Builder->create(state => dclone($tb->state));

Module dependency.

Besides, that wouldn't even work.  Consider how the existing T::B stores
things.  Better to have T::B::State implement its own clone method.


> >copy & share state- like copy, but test state is shared between the
> > original and the copy
> 
> my $shared_tb = Test::Builder->create(state => $tb->state);

You're right, this one isn't necessary given the *state methods sketched out
above.


> >with those plus the push/pop_stack methods you can pick and choose what 
> >sort
> >of state sharing you want.
> 
> If we have explicit access to the state - is there any advantage in
> having a stack of objects?
> 
> Localising $Test::Builder::Default, or saving/restoring T::B->default
> would seem to give you all the functionality necessary. (You could
> always subclass T::B if you really wanted it :-)

Its more convenient.


> I really don't want to think about how all this will interact with 
> threading :-/

(Famous last words) There shouldn't be a problem.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It's Tobacco time!

Re: Test::Builder->level

[Michael G Schwern]

On Tue, Nov 12, 2002 at 04:21:38PM +, Adrian Howard wrote:
> Most of the time when I need to mess with level, the code that does
> the test is sitting in a different package from the one that is
> calling the test. How about having it walk up the call stack until it
> find a package that differs from the one it was called in?

People suggest this so often it needs its own FAQ.

   package Test::Foo::Inside;
   
   my $tb = Test::Builder->new;
   
   sub something {
   $tb->ok(@_);
   }
   
   
   package Test::Foo;
   use Test::Foo::Inside;

   sub wiffle {
   something(@_);
   }


   ...and in your test code...
   use Test::Foo;
   wiffle();


Ooops.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Death was thought to be fatal.
-- Craig A. Berry in 

Re: Binary-wise is()

[Michael G Schwern]

On Wed, Nov 27, 2002 at 03:24:51PM +0100, Rafael Garcia-Suarez wrote:
> I'd like to have a custom version of is(), say binary_is(), that
> reports 'strings 1 and 2 differ at byte 635, got 0x92, expected 0x42'
> or 'strings 1 differ in length, got 3874, expected 3875'.
>
> Is there already some module that does this ?

Not that I know of.

> If not, what's the best way to implement this myself ? (my tests using
> Test::More.)

Write Test::Binary or something using Test::Builder.  Have it export
binary_is() and then you can just do:

use Test::More tests => 42;
use Test::Binary;

is(...);
binary_is(...);

See http://www.pobox.com/~schwern/talks/Writing_A_Test_Library/full_slides/
for details about using Test::Builder.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Stupid am I?  Stupid like a fox!

Wiki Wiki Working

[Michael G Schwern]

After shuffling some files around and doing some permissions tricks, the
Perl QA Wiki is working again!  Everything is now editable.

http://www.pobox.com/~schwern/cgi-bin/perl-qa-wiki.cgi


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
 Contrary to popular belief, Perl is not developed by a team of 
  independently wealthy mad scientists.
 (qw/independent wealthy mad scientist/)[rand(3)]; # choose one
 the reason it's not rand(4) is because perl isn't a science :p

Re: testing with stubs

[Michael G Schwern]

On Thu, Dec 12, 2002 at 03:27:32PM -0600, Danny Faught wrote:
> I have a legacy Perl script, not object-oriented but at least ported to 
> Perl 5.  I want to use a real-world example, rather than new code like 
> what Kent Beck uses in the book Test-Driven Development.  So I thought 
> I'd implement a unit test for one of the functions in this old script. 
> I'm using Test::Unit::Procedural.
> 
> I'm pulling my hair out trying to stub out the functions that the 
> function under test is calling.  Here's the function (complete with an 
> error that will be corrected as part of the exercise):
> 
> sub abort_handler {
> my ($event) = @_;
> print STDERR "stress_driver: aborted by SIG$event->data()\n";
> &log ("stress_driver: aborted by SIG$event->data()");
> exit (&cleanup);
> }
> 
> I want to stub out (replace) the &log and &cleanup functions and the 
> builtin exit function.

In your test, do something like this:

BEGIN {
# Override with itself so it can be overridden at run-time later.
*CORE::GLOBAL::exit = sub {
CORE::exit @_;
}
}

{
 my $exit_code;
 no warnings 'redefine';
 local *CORE::GLOBAL::exit = sub {
 $exit_code = shift;
 goto EXIT;
 };
 
 my @logs = (); 
 local *Foo::log = sub {
 push @logs, @_;
 }
 
 my $cleanup_called = 0;
 local *Foo::cleanup = sub {
 $cleanup_called++
 return something;
 }
 
 ...do code which calls abort handler...
 EXIT: ..tests here...

}


> I'll probably also want to redirect STDERR to capture that output as well

That's easy, tie it.  Look at t/lib/TieOut.pm in the Test-Simple tarball.


>  But I can't figure out how to turn unit test mode on and off.

Scripts are hard to test.  Libraries are easy.  So...

Step 1:  pull all the subroutines out of the script and into a seperate
library.

Step 2:  test that library.

That leaves a lot less to try and test in the script.

Beyond that the real problem is that scripts must be called as seperate
processes which makes much of the subroutine overriding tricks above
difficult.  I have a trick which simulates running a perl script, but all
its really doing is eval'ing the code in the current process.  This means
the tricks above will work.

It can be found here: 
http://www.pobox.com/~schwern/tmp/esmith-TestUtils.pm

Adapt as you like.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
conway: unit of mind expansion.  One Conway == ~20 lines of Perl code
  found in $CPAN/authors/id/D/DC/DCONWAY, which gives the sensation
  of your brain being wrapped around a brick, with kiwi juice squeezed
  on top.
-- Ziggy

Re: ANNOUNCE: Test::XML

[Michael G Schwern]

On Mon, Mar 31, 2003 at 12:45:14PM +0100, Dominic Mitchell wrote:
> I've just uploaded a new module to CPAN[1], Test::XML.  It contains a 
> couple of functions for examining XML output of functions in a slightly 
> saner way than is().
> 
> http://search.cpan.org/author/SEMANTICO/Test-XML-0.03/

   is_xml( '', '' );   # PASS

I like this.


-- 
You see, in this world there's two kinds of people.  Those with loaded
guns, and those who dig.  Dig.
-- Blondie, "The Good, The Bad And The Ugly"

Re: Using environment variables to control long running tests

[Michael G Schwern]

On Thu, Jun 05, 2003 at 10:55:29AM +0100, Mark Fowler wrote:
> On Thu, 5 Jun 2003, [iso-8859-1] Andrew Savige wrote:
> 
> > While a standard name is not required, having one would allow
> > automated CPAN test harnesses to run the longer tests.
> 
> While we're on it, it'd be nice to have a standard enviromental variable
> that turned off all interactivty.  Make the module accept all it's
> defaults, skip tests where it requires an interactive question...etc.
> 
> I know this is covered to a certain extent by prompt() being
> semi-inteligent, but it'd be nice if I could override all cases (even in
> test suites) where questions are asked, and I could be sure that my
> install script won't fail unless a test actually fails.

I've hopped in my time machine and added PERL_MM_USE_DEFAULT to prompt()
and documented it in ExtUtils::MakeMaker 6.04.


-- 
You're more radiant than a memory of breathtaking ecstasy.

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Mon, Jun 23, 2003 at 05:49:06PM +0100, Fergal Daly wrote:
>   I'm looking for comment or suggestions about this new module. It's 
> independent of and complementary to Test::Warn. It tests that your test 
> script didn't emit any warnings. Just add
> 
> use Test::More::None;
> 
> to the top your test script, update your plan (if you've got one) and that's 
> it. You'll get an extra test, executed after your script ends checking 
> whether there were any warnings. If there were you'll get a full run down of 
> them including a stack trace.

Good idea.  Too bad about the plan calculation hackery necesssary. :(


-- 
Me? A robot? That's rediculous! For one thing, that doesn't compute at all!

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Tue, Jun 24, 2003 at 10:59:43AM +0200, Tels wrote:
> > On Mon, Jun 23, 2003 at 05:49:06PM +0100, Fergal Daly wrote:
> >>  I'm looking for comment or suggestions about this new module. It's 
> >> independent of and complementary to Test::Warn. It tests that your test 
> >> script didn't emit any warnings. Just add
> >> 
> >> use Test::More::None;

Typo?

> >> to the top your test script, update your plan (if you've got one) and
> >> that's 
> >> it. You'll get an extra test, executed after your script ends checking 
> >> whether there were any warnings. If there were you'll get a full run
> >> down of 
> >> them including a stack trace.
> > 
> > Good idea.  Too bad about the plan calculation hackery necesssary. :(
> 
> 
> Can't nowarings() call Test::More::plan_add(1) or something like this?
> 

Consider the following.

use Test::More;
use Test::Warn::None;
plan tests => 42;

To make this work I'd have to overhaul the internal Test::Builder planning
system to allow Test::Warn::None to say "I'm going to add an extra test,
please remember this fact".  I can be convinced its worth it.


-- 
Life is like a sewer.  What you get out of it depends on what you put into it.
- Tom Lehrer

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Tue, Jun 24, 2003 at 12:07:19PM +0100, Fergal Daly wrote:
> > Consider the following.
> > 
> > use Test::More;
> > use Test::Warn::None;
> > plan tests => 42;
> > 
> > To make this work I'd have to overhaul the internal Test::Builder planning
> > system to allow Test::Warn::None to say "I'm going to add an extra test,
> > please remember this fact".  I can be convinced its worth it.
> 
> It seems like a good idea to me. Are there any other modules that would use 
> it?

Potentially, but I can't think of anything at the moment.


-- 
Loon.

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Tue, Jun 24, 2003 at 12:37:36PM +0100, Fergal Daly wrote:
> Also how about calling it Test::Warn::Auto? I'm not particularly happy with 
> None,

Test::Warn::Auto doesn't say anything about its main purpose: to ensure
that you have no warnings.  Instead it documents an implementation detail,
that the check for no warnings is automatic.

I like Test::Warn::None or some variation on it.  Or even Test::NoWarnings.
Doesn't have to sit in the Test::Warn namespace.


-- 
You're the sickest teenager I've ever set my wallet on.

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Tue, Jun 24, 2003 at 01:36:52PM +0200, Rafael Garcia-Suarez wrote:
> BTW, what about modules that define their own category of warnings
> (via warnings::register) ? It'd be useful to have a module to ease
> testing for warnings presence/absence on certain conditions.

There's Test::Warn, but I don't think it does anything special with
registered warning classes.


> (Avoiding to span perl interpreters at each test would be a bonus.)

I don't quite understsand what "spanning perl interpreters" means.


-- 
It's Yellowing Laudanum time!

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Tue, Jun 24, 2003 at 02:04:25PM -0500, Andy Lester wrote:
> All this "make sure no warnings fired" is good thinking.  But why not 
> roll it into Test::Harness, and make it switch selectable?  It's 
> really T::H that we want keeping an eye on this, right?

No, its definately a test feature.  Much easier to set up (no MakeMaker
muckery), finer granularity (can be done on a per test file basis rather
than a whole suite) and can distinguish between warnings and diagnostic
output (ie. warn() vs diag() vs print STDERR).

Besides, Test::Harness can't portably trap STDERR.  If you can figure out
a way to do it that would solve lots of problems.

If you want to do it to a whole test suite, PERL5OPT=-MTest::Warn::None comes
to mind.


-- 
"Congratulations, you're a thieving bastard... Now give me back my pants."
"That's MISTER Pants!"
-- Ian's Adventures In Morrowind
   http://www.machall.com/morrowind/page3.html

Re: [ANNOUNCE] Test::Warn::None 0.02

[Michael G Schwern]

On Wed, Jun 25, 2003 at 07:09:26PM +0100, Fergal Daly wrote:
> I just thought of a big problem with plan extensions. If the script silently 
> eat's itself just before you extend the plan, then you don't know that 
> anything went wrong.

It would have to also exit normally.  That is rare.


-- 
Is there an airport nearby or is that just my tae-kwon-do taking off?