Re: Untested modules update: There's more than we thought

[Piers Cawley]

"Kurt D. Starsinic" <[EMAIL PROTECTED]> writes:

> On Dec 15, Dave Rolsky wrote:
>> Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
>> see why people seem to object to the use of Test::More in the core Perl
>> tests.  I can't see how it couldn't help improve the quality of the tests
>> while providing a standardized way to do the things that all the tests do
>> anyway.
>
> I, for one, don't object to the use of Test::More in the core Perl
> tests.  I'm just being responsibly skeptical about the promotion of many
> different test harness modules with many different functions.  It runs
> the risk of _seeming_ like the People in Charge don't know what testing
> is for, and of not providing effective guidance to those people writing
> their first tests.
>
> I also think that we have a real testing problem, which is that it
> isn't _trivially_ easy to run a huge test suite (like Alzabo's, or Perl's)
> and to find out at the end which tests failed and what their output was.
> 3000 lines and 20 minutes later, a message that 3 out of 1 tests
> failed is not maximally useful.  Greater variety of test functions is not
> (IMNSHO) nearly as important as addressing that.

This is one of the things that the PerlUnit test framework does
reasonably well, but I definitely wouldn't recommend it for the perl
test suite as it does some vaguely hairy stuff with OO and is
definitely designed for testing rather higher level constructs than
perl ops etc.

Note to self. Polish the docs for PerlUnit and make a new release
already. 

-- 
Piers

   "It is a truth universally acknowledged that a language in
possession of a rich syntax must be in need of a rewrite."
 -- Jane Austen?

Re: Untested modules update: There's more than we thought

[Piers Cawley]

Michael G Schwern <[EMAIL PROTECTED]> writes:

> On Mon, Dec 17, 2001 at 08:12:43AM +, Piers Cawley wrote:
>> >> What's wrong with 
>> >> 
>> >> ok ( eval { $foo->isa('Foo') } );
>> >>
>> >> or even:
>> >> 
>> >> ok (eval { ref($foo) && $foo->isa('Foo') });
>> >
>> > As Kurt already pointed out, you can do:
>> >
>> > ok( UNIVERSAL::isa($foo, 'Foo') );
>> >
>> > but if it fails you have no idea what $foo was.
>> 
>> No you can't. Not if you've overridden isa anywhere. (Which is
>> perfectly possible.)
>
> Ooooh, hadn't thought about that.  I've done it myself a few times (to
> make a delegation look like inheritance).  Now I have to go fix
> Test::More to deal with it.  Ya know, I was literally just about to
> release 0.40 and I HAD to go read my email first.
>
> Either way, your above ok() solutions still give you no failure
> diagnostics, which is the whole point.

Well, yes. At least the point about overridden isa didn't get lost... 

-- 
Piers

   "It is a truth universally acknowledged that a language in
possession of a rich syntax must be in need of a rewrite."
 -- Jane Austen?

Re: Untested modules update: There's more than we thought

[Gerrit P. Haase]

Hallo chromatic,

Am 2001-12-16 um 19:09 schriebst du:

> On Sunday 16 December 2001 02:10, Gerrit P. Haase wrote:

> Thanks for the report.

>> ../lib/ExtUtils/MM_Cygwin.# Failed test
>> (../lib/ExtUtils/MM_Cygwin.t at line 73) #   undef
>> # doesn't match '(?-xism:could not locate your pod2man)'
>> # Failed test (../lib/ExtUtils/MM_Cygwin.t at line 75)
>> #   'POD2MAN_EXE = /usr/bin/pod2man
>> # POD2MAN = $(PERL) -we '%m=@ARGV;for (keys %m){' \
>> # -e 'next if -e $$m{$$_} && -M $$m{$$_} < -M $$_ && -M $$m{$$_} < -M
>> "";' \ # -e 'print "Manifying $$m{$$_}\n"; $$m{$$_} =~ s/::/./g;' \
>> # -e 'system(qq[$$^X ].q["-I$(PERL_ARCHLIB)" "-I$(PERL_LIB)"
>> $(POD2MAN_EXE) ].qq[$$_>$$m{$$_}])==0 or warn "Couldn\047t install
>> $$m{$$_}\n";' \ # -e 'chmod(oct($(PERM_RW))), $$m{$$_} or warn "chmod
>> $(PERM_RW) $$m{$$_}: $$!\n";}' #
>> manifypods : pure_all foo
>> #   noecho$(POD2MAN) \
>> #   foo \
>> #   1'
>> # doesn't match '(?-xism:POD2MAN_EXE = -S pod2man)'
>> # Looks like you failed 2 tests of 17.

> What happens if you apply this patch and run it?  It seems to me it'll 
> generate a more reliable failure, which is what the test is supposed to catch.

> -- c

> --- lib/ExtUtils/~MM_Cygwin.t   Sun Dec 16 11:02:04 2001
> +++ lib/ExtUtils/MM_Cygwin.tSun Dec 16 11:07:31 2001
> @@ -70,11 +70,16 @@
>  $args->{MAN3PODS} = { foo => 1 };
>  my $out = tie *STDOUT, 'FakeOut';
>  my $res = $args->manifypods();
> -like( $$out, qr/could not locate your pod2man/,
> -   '... should warn if pod2man cannot be located' );
> -like( $res, qr/POD2MAN_EXE = -S pod2man/,
> -   '... should use default pod2man target' );
> -like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
> +
> +{
> +   # try to trip warning on line 53
> +   local *MM::perl_script = sub { return };
> +   like( $$out, qr/could not locate your pod2man/,
> +   '... should warn if pod2man cannot be located' );
> +   like( $res, qr/POD2MAN_EXE = -S pod2man/,
> +   '... should use default pod2man target' );
> +   like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
> +}
 
>  $args->{PERL_SRC} = File::Spec->updir;
>  $args->{MAN1PODS} = { bar => 1 };

.../lib/ExtUtils/MM_Cygwin.ok 9/17# Failed test 
(../lib/ExtUtils/MM_Cygwin.t at line 73)
#   undef
# doesn't match '(?-xism:could not locate your pod2man)'
# Failed test (../lib/ExtUtils/MM_Cygwin.t at line 75)
#   'POD2MAN_EXE = /usr/bin/pod2man
# POD2MAN = $(PERL) -we '%m=@ARGV;for (keys %m){' \
# -e 'next if -e $$m{$$_} && -M $$m{$$_} < -M $$_ && -M $$m{$$_} < -M "";' \
# -e 'print "Manifying $$m{$$_}\n"; $$m{$$_} =~ s/::/./g;' \
# -e 'system(qq[$$^X ].q["-I$(PERL_ARCHLIB)" "-I$(PERL_LIB)" $(POD2MAN_EXE) 
].qq[$$_>$$m{$$_}])==0 or warn "Couldn\047t install $$m{$$_}\n";' \
# -e 'chmod(oct($(PERM_RW))), $$m{$$_} or warn "chmod $(PERM_RW) $$m{$$_}: 
$$!\n";}'
# 
manifypods : pure_all foo
#   noecho$(POD2MAN) \
#   foo \
#   1'
# doesn't match '(?-xism:POD2MAN_EXE = -S pod2man)'
# Looks like you failed 2 tests of 17.
.../lib/ExtUtils/MM_Cygwin.dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 10-11
Failed 2/17 tests, 88.24% okay


-- 
=^..^=mailto:[EMAIL PROTECTED]

Re: Untested modules update: There's more than we thought

[Gerrit P. Haase]

Hallo Michael,

Am 2001-12-16 um 21:20 schriebst du:

> On Sun, Dec 16, 2001 at 11:09:29AM -0700, chromatic wrote:
>> + like( $$out, qr/could not locate your pod2man/,
>> + '... should warn if pod2man cannot be located' );

> Gerrit, do you already have a perl installed in the spot you're about
> to put this new one?  If so, it might be seeing that pod2man.

I have 5.6.1 installed in /usr and pod2man in /usr/bin.
Hmm, up to now there never was an influence on building bleadperl (I guess),
after I renamed /bin/pod2man it works now without error, but ...

Gerrit
-- 
=^..^=mailto:[EMAIL PROTECTED]

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Mon, Dec 17, 2001 at 08:12:43AM +, Piers Cawley wrote:
> >> What's wrong with 
> >> 
> >> ok ( eval { $foo->isa('Foo') } );
> >>
> >> or even:
> >> 
> >> ok (eval { ref($foo) && $foo->isa('Foo') });
> >
> > As Kurt already pointed out, you can do:
> >
> > ok( UNIVERSAL::isa($foo, 'Foo') );
> >
> > but if it fails you have no idea what $foo was.
> 
> No you can't. Not if you've overridden isa anywhere. (Which is
> perfectly possible.)

Ooooh, hadn't thought about that.  I've done it myself a few times (to
make a delegation look like inheritance).  Now I have to go fix
Test::More to deal with it.  Ya know, I was literally just about to
release 0.40 and I HAD to go read my email first.

Either way, your above ok() solutions still give you no failure
diagnostics, which is the whole point.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
pleasing symmetry
the paste goes in at one end
comes out the other.
-- mjd

Re: Untested modules update: There's more than we thought

[Piers Cawley]

Michael G Schwern <[EMAIL PROTECTED]> writes:
> On Sun, Dec 16, 2001 at 02:41:31PM +, Piers Cawley wrote:
>> > The equivalent code without isa_ok() would be:
>> >
>> > my $foo = Foo->new;
>> > ok( $foo->isa('Foo') );
>> >
>> > except should $foo be unblessed or undef that will explode.  You have
>> > to start doing things like:
>> >
>> > my $foo = Foo->new;
>> > ok( defined $foo && $foo->isa('Foo') );
>> >
>> > which rapidly gets tiresome.  It also provides you little information
>> > about what value $foo held that caused it to fail.
>> 
>> What's wrong with 
>> 
>> ok ( eval { $foo->isa('Foo') } );
>>
>> or even:
>> 
>> ok (eval { ref($foo) && $foo->isa('Foo') });
>
> As Kurt already pointed out, you can do:
>
> ok( UNIVERSAL::isa($foo, 'Foo') );
>
> but if it fails you have no idea what $foo was.

No you can't. Not if you've overridden isa anywhere. (Which is
perfectly possible.)

-- 
Piers

   "It is a truth universally acknowledged that a language in
possession of a rich syntax must be in need of a rewrite."
 -- Jane Austen?

Re: Untested modules update: There's more than we thought

[chromatic]

Hello again Gerrit,

You know, I didn't put the MOST important line in the block.  Here's a better 
patch.  I blame Jeffrey Friedl.  :)

Any better?

-- c



--- lib/ExtUtils/~MM_Cygwin.t	Sun Dec 16 11:02:04 2001
+++ lib/ExtUtils/MM_Cygwin.t	Sun Dec 16 19:59:44 2001
@@ -69,12 +69,17 @@
 
 $args->{MAN3PODS} = { foo => 1 };
 my $out = tie *STDOUT, 'FakeOut';
-my $res = $args->manifypods();
-like( $$out, qr/could not locate your pod2man/,
-	'... should warn if pod2man cannot be located' );
-like( $res, qr/POD2MAN_EXE = -S pod2man/,
-	'... should use default pod2man target' );
-like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
+
+{
+	# try to trip warning on line 53
+	local *MM::perl_script = sub { return };
+	my $res = $args->manifypods();
+	like( $$out, qr/could not locate your pod2man/,
+		'... should warn if pod2man cannot be located' );
+	like( $res, qr/POD2MAN_EXE = -S pod2man/,
+		'... should use default pod2man target' );
+	like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
+}
 
 $args->{PERL_SRC} = File::Spec->updir;
 $args->{MAN1PODS} = { bar => 1 };

Re: Untested modules update: There's more than we thought

[chromatic]

On Sun, 16 Dec 2001 19:30:18 -0700, Benjamin Goldberg wrote:

> I think that if "all" we know about the returned type is that it is supposed
> to provide some specific interface, it would be more robust to test that the
> returned thing actually *does* provide the interface.
 
Agreed.  You have my permission not to use isa_ok() in that case.  :)

It did come in handy for the ExtUtils::Installed test, though.  The constructor
returns an object with a Perl member, which itself has a packlist member that's
supposed to be an ExtUtils::Packlist object.  This gives me the nice and easy
test:

isa_ok( $realei->{Perl}{packlist}, 'ExtUtils::Packlist' );

If someone subclasses ExtUtils::Packlist and modifies ExtUtils::Installed to
use the subclass, the test won't break.  If they write something with a similar
interface but don't derive from ExtUtils::Packlist, the test will break.

I'm not worried about that.  With a change that large, the test will probably
need to be modified anyway.

-- c

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sun, Dec 16, 2001 at 09:30:18PM -0500, Benjamin Goldberg wrote:
> Suppose we have RandomThing->new which randomly returns an instance of
> one of a few dozen different classes, which have no relation at all with
> each other except a common interface.

In such an odd case, don't use isa_ok().  Simp.


> I think that if "all" we know about the returned type is that it is
> supposed to provide some specific interface, it would be more robust to
> test that the returned thing actually *does* provide the interface.

As the ratio of constructors which return objects of random classes to
those which produce something a bit more predictable is tiny, I'm not
going to start assuming every constructor produces weird objects.

Its more robust to do both.  Check you got the right type and check
the interface.

On the off chance that the internals change to start producing
randomly typed, yet interface conforming, objects, just change the
tests.


PS  Do you know of anything that does return unpredictably typed
objects?

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
36 haikus
would scare the shit out of me.
Thank goodness for paste!
-- Schwern

Re: Untested modules update: There's more than we thought

[Benjamin Goldberg]

Michael G Schwern wrote:
> 
> On Sun, Dec 16, 2001 at 02:41:31PM +, Piers Cawley wrote:
> > Nothing wrong with an adaptor/factory returning something that isn't
> > a Foo, so long as it has the same interface.
> 
> That's why its isa_ok() and not ref_ok().
> 
> On the off chance Foo->new is supposed to return something that bears
> no relation to Foo at all, then just don't use isa_ok.  Or, check that
> its that other thing:
> 
> my $foo = Foo->new;
> isa_ok( $foo, 'Bar' );

Suppose we have RandomThing->new which randomly returns an instance of
one of a few dozen different classes, which have no relation at all with
each other except a common interface.  It wouldn't be a good idea to
test the return value of new for being a specific type, or even if that
thing is one of the 'known' types, since something new might be added in
a future version.

I think that if "all" we know about the returned type is that it is
supposed to provide some specific interface, it would be more robust to
test that the returned thing actually *does* provide the interface.

my $foo = Foo->new;
for my $method (qw/bar baz quux quuux/) {
ok( $foo->can($method),
"Foo->new should return something with a $method method"
);
}

-- 
Klein bottle for rent - inquire within.

Re: lib.t (was Re: Untested modules update: There's more than we thought)

[Michael G Schwern]

On Sun, Dec 16, 2001 at 07:30:37PM +, Nicholas Clark wrote:
> > I just thought of a better way.  Since all we're testing is that
> > lib.pm does the right things to @INC, we can presume that if one of
> > require(), do() or use() works, the rest will work.
> 
> Can't we just test what @INC now contains by directly reading it?
> I'm assuming that it's safe for the test to assume that do/require/use
> work with @INC correctly.

Technically, yeah.  But Brent's already gone that extra mile.

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Any sufficiently encapsulated hack is no longer a hack.

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sun, Dec 16, 2001 at 02:41:31PM +, Piers Cawley wrote:
> Nothing wrong with an adaptor/factory returning something that isn't a
> Foo, so long as it has the same interface.

That's why its isa_ok() and not ref_ok().

On the off chance Foo->new is supposed to return something that bears
no relation to Foo at all, then just don't use isa_ok.  Or, check that
its that other thing:

my $foo = Foo->new;
isa_ok( $foo, 'Bar' );


> > The equivalent code without isa_ok() would be:
> >
> > my $foo = Foo->new;
> > ok( $foo->isa('Foo') );
> >
> > except should $foo be unblessed or undef that will explode.  You have
> > to start doing things like:
> >
> > my $foo = Foo->new;
> > ok( defined $foo && $foo->isa('Foo') );
> >
> > which rapidly gets tiresome.  It also provides you little information
> > about what value $foo held that caused it to fail.
> 
> What's wrong with 
> 
> ok ( eval { $foo->isa('Foo') } );
>
> or even:
> 
> ok (eval { ref($foo) && $foo->isa('Foo') });

As Kurt already pointed out, you can do:

ok( UNIVERSAL::isa($foo, 'Foo') );

but if it fails you have no idea what $foo was.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
First day in Prison
Back behind the shower door
Adhesive applied
-- ignatz

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sun, Dec 16, 2001 at 11:09:29AM -0700, chromatic wrote:
> + like( $$out, qr/could not locate your pod2man/,
> + '... should warn if pod2man cannot be located' );

Gerrit, do you already have a perl installed in the spot you're about
to put this new one?  If so, it might be seeing that pod2man.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Pancakes is the better part of valor.
http://www.goats.com/archive/971202.html

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sun, Dec 16, 2001 at 04:20:33PM +, Nicholas Clark wrote:
> Does that mean that it should have its own tests placed in t/lib, so that
> they run before ext/ and lib/?

Yes, it would be nice if they ran before the rest, but I really don't
want to start shuffling tests around this close to 5.8.  Anyhow, its
usually Really Obvious when Test::More starts failing.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
This is my sig file.  Is it not nify?  Worship the sig file.
http://www.sluggy.com

Re: Untested modules update: There's more than we thought

[chromatic]

On Sunday 16 December 2001 02:10, Gerrit P. Haase wrote:

Thanks for the report.

> ../lib/ExtUtils/MM_Cygwin.# Failed test
> (../lib/ExtUtils/MM_Cygwin.t at line 73) #   undef
> # doesn't match '(?-xism:could not locate your pod2man)'
> # Failed test (../lib/ExtUtils/MM_Cygwin.t at line 75)
> #   'POD2MAN_EXE = /usr/bin/pod2man
> # POD2MAN = $(PERL) -we '%m=@ARGV;for (keys %m){' \
> # -e 'next if -e $$m{$$_} && -M $$m{$$_} < -M $$_ && -M $$m{$$_} < -M
> "";' \ # -e 'print "Manifying $$m{$$_}\n"; $$m{$$_} =~ s/::/./g;' \
> # -e 'system(qq[$$^X ].q["-I$(PERL_ARCHLIB)" "-I$(PERL_LIB)"
> $(POD2MAN_EXE) ].qq[$$_>$$m{$$_}])==0 or warn "Couldn\047t install
> $$m{$$_}\n";' \ # -e 'chmod(oct($(PERM_RW))), $$m{$$_} or warn "chmod
> $(PERM_RW) $$m{$$_}: $$!\n";}' #
> manifypods : pure_all foo
> #   noecho$(POD2MAN) \
> #   foo \
> #   1'
> # doesn't match '(?-xism:POD2MAN_EXE = -S pod2man)'
> # Looks like you failed 2 tests of 17.

What happens if you apply this patch and run it?  It seems to me it'll 
generate a more reliable failure, which is what the test is supposed to catch.

-- c

--- lib/ExtUtils/~MM_Cygwin.t   Sun Dec 16 11:02:04 2001
+++ lib/ExtUtils/MM_Cygwin.tSun Dec 16 11:07:31 2001
@@ -70,11 +70,16 @@
 $args->{MAN3PODS} = { foo => 1 };
 my $out = tie *STDOUT, 'FakeOut';
 my $res = $args->manifypods();
-like( $$out, qr/could not locate your pod2man/,
-   '... should warn if pod2man cannot be located' );
-like( $res, qr/POD2MAN_EXE = -S pod2man/,
-   '... should use default pod2man target' );
-like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
+
+{
+   # try to trip warning on line 53
+   local *MM::perl_script = sub { return };
+   like( $$out, qr/could not locate your pod2man/,
+   '... should warn if pod2man cannot be located' );
+   like( $res, qr/POD2MAN_EXE = -S pod2man/,
+   '... should use default pod2man target' );
+   like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
+}
 
 $args->{PERL_SRC} = File::Spec->updir;
 $args->{MAN1PODS} = { bar => 1 };

Re: Untested modules update: There's more than we thought

[Piers Cawley]

Michael G Schwern <[EMAIL PROTECTED]> writes:

> On Sat, Dec 15, 2001 at 04:09:19AM +0200, Jarkko Hietaniemi wrote:
>> I don't get all the different (well, only VMS and Cygwin, so far, but
>> I like to nip buds) MM_XXX test suites: won't they be testing pretty
>> much the same things?  
>
> Yes, but with different values and, I'm sure, lots and lots of
> uniquely odd life support code.
>
> ExtUtils::MM_* is going to be very tricky to test even as individual
> modules.  Trying to unify them at the same time will make life a lot
> more difficult, made exponentially harder by the fact that the
> behaviors of the various modules have drifted apart somewhat over the
> years.
>
> Once the tests are in place I'm going to start detangling common
> methods and functionality out into a top level ExtUtils::MM_Any to try
> and oust ExtUtils::MM_Unix from its position at the top of the
> hiearchy.  When that's underway we can think about unifying the tests.
>
> About the only thing I could see unifying at this point in the MM
> tests is that they all respond to the same basic methods.  Basicly a
> foreach loop around a shitload of can_ok() tests.
>
>
>> And frankly, the current MM_VMS.t is hardly
>> worth the bits it's printed on.
>
> Yeah, its still on the untested list because of that.
>
>
>> - What good is use_ok(), really?  How is that better than... use?
>>   Gives you a prettier output?
>
> Yep, it fails gracefully instead of simply exploding, and sometimes a
> module use failing doesn't mean the test can't continue.  The old h2xs
> generated tests had a whole lot of confusing "# black magic" code to
> do exactly that, so I figured it was worthwhile.
>
>
>> - Ditto for isa_ok(). I find quite comical to do
>> 
>>  $foo = Foo->new;
>> 
>>  isa_ok($foo, 'Foo)';
>>
>>   Have we suddenly stopped trusting the basic OO? 
>
> We trust that Perl's method calls work, but we don't trust that the
> guy who wrote Foo got his constructors right.  Possibilities include
> straight failure (ie. you get nothing), forgetting to bless() (its
> happened) or, if the constructor is some sort of adaptor/factory,
> returning an object which isn't a subclass of Foo.

Nothing wrong with an adaptor/factory returning something that isn't a
Foo, so long as it has the same interface. Actually, that can be one
of Perl's nicer aspects in that it becomes almost trivial to set up
mock objects (at work we use the PerlUnit harness and for some of the
more complicated classes we test it's been really handy to be able to
pass the test object (masquerading as, say, a message object) as an
argument without having to worry about inheritance issues (we cheat
and delegate sometimes...).

> Constructors fail often enough that I felt it worthwhile to write a
> test function for them.
>
> You could leave off the test entirely and just trust that you use $foo
> later on down in the test and it'll explode there, but its better to
> put the point of failure as close to where the bug is as possible.
> And you get no diagnostic information (see below).

Die early and die often eh?

>
> The equivalent code without isa_ok() would be:
>
> my $foo = Foo->new;
> ok( $foo->isa('Foo') );
>
> except should $foo be unblessed or undef that will explode.  You have
> to start doing things like:
>
> my $foo = Foo->new;
> ok( defined $foo && $foo->isa('Foo') );
>
> which rapidly gets tiresome.  It also provides you little information
> about what value $foo held that caused it to fail.

What's wrong with 

ok ( eval { $foo->isa('Foo') } );

or even:

ok (eval { ref($foo) && $foo->isa('Foo') });

?

-- 
Piers

   "It is a truth universally acknowledged that a language in
possession of a rich syntax must be in need of a rewrite."
 -- Jane Austen?

Re: Untested modules update: There's more than we thought

[Gerrit P. Haase]

Hallo Jarkko,

Am 2001-12-15 um 20:47 schriebst du:

> On Fri, Dec 14, 2001 at 05:20:24PM -0700, chromatic wrote:
>> On Fri, 14 Dec 2001 15:18:43 -0700, Michael G Schwern wrote:
>> 
>> > ExtUtils::MM_Cygwin
>> 
>> Here's a test for that.  It could use someone on Cygwin testing it.  Tels said
>> he would, but apparently got busy.

> Thanks, applied.


.../lib/ExtUtils/MM_Cygwin.# Failed test (../lib/ExtUtils/MM_Cygwin.t 
at line 73)
#   undef
# doesn't match '(?-xism:could not locate your pod2man)'
# Failed test (../lib/ExtUtils/MM_Cygwin.t at line 75)
#   'POD2MAN_EXE = /usr/bin/pod2man
# POD2MAN = $(PERL) -we '%m=@ARGV;for (keys %m){' \
# -e 'next if -e $$m{$$_} && -M $$m{$$_} < -M $$_ && -M $$m{$$_} < -M "";' \
# -e 'print "Manifying $$m{$$_}\n"; $$m{$$_} =~ s/::/./g;' \
# -e 'system(qq[$$^X ].q["-I$(PERL_ARCHLIB)" "-I$(PERL_LIB)" $(POD2MAN_EXE) 
].qq[$$_>$$m{$$_}])==0 or warn "Couldn\047t install $$m{$$_}\n";' \
# -e 'chmod(oct($(PERM_RW))), $$m{$$_} or warn "chmod $(PERM_RW) $$m{$$_}: 
$$!\n";}'
# 
manifypods : pure_all foo
#   noecho$(POD2MAN) \
#   foo \
#   1'
# doesn't match '(?-xism:POD2MAN_EXE = -S pod2man)'
# Looks like you failed 2 tests of 17.
dubious
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 10-11
Failed 2/17 tests, 88.24% okay

Gerrit
-- 
=^..^=mailto:[EMAIL PROTECTED]

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

Thinks like use_ok, require_ok, isa_ok, can_ok, etc... are all
functions that I threw into Test::More because I found myself needing
them and suspected others did, too.  If you prefer to use ok() instead
of one of the above special purpose functions, I'm not going to jump
up and down and fling poo at you (as enjoyable as that is).

They're nice to have, which reflects the "More" nature of Test::More.
If they didn't exist, I could see worrying about their utility weight
against the effort involved in implementing them.  But some crazy test
monkey already implemented them, so there they are.

Ooop ooop.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
my anus yearns now
warm paste fills me happily
saturday morning
-- imploded

Re: Untested modules update: There's more than we thought

[Jarkko Hietaniemi]

> Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
> see why people seem to object to the use of Test::More in the core Perl
> tests.  I can't see how it couldn't help improve the quality of the tests
> while providing a standardized way to do the things that all the tests do
> anyway.

Mainly it's a question of bootstrapping.  Test::More is quite fancy
and unless all the Perl it needs is working, you are not testing
what you think you are testing.  Requiring poetry before you know
how to spell is unfair.  Try meditating on the word "core".

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: Untested modules update: There's more than we thought

[Kurt D. Starsinic]

On Dec 15, Dave Rolsky wrote:
> Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
> see why people seem to object to the use of Test::More in the core Perl
> tests.  I can't see how it couldn't help improve the quality of the tests
> while providing a standardized way to do the things that all the tests do
> anyway.

I, for one, don't object to the use of Test::More in the core Perl
tests.  I'm just being responsibly skeptical about the promotion of many
different test harness modules with many different functions.  It runs
the risk of _seeming_ like the People in Charge don't know what testing
is for, and of not providing effective guidance to those people writing
their first tests.

I also think that we have a real testing problem, which is that it
isn't _trivially_ easy to run a huge test suite (like Alzabo's, or Perl's)
and to find out at the end which tests failed and what their output was.
3000 lines and 20 minutes later, a message that 3 out of 1 tests
failed is not maximally useful.  Greater variety of test functions is not
(IMNSHO) nearly as important as addressing that.

And I only chimed in here to point out that one can use isa() on undef,
which Schwern claimed couldn't be done.

- Kurt

P.S.  Schwern, not to deter you from doing what's important to you.  I
know that patches are welcome.  YMMV.

Re: Untested modules update: There's more than we thought

[Dave Rolsky]

On Sat, 15 Dec 2001, Kurt D. Starsinic wrote:

> > my $foo = Foo->new;
> > ok( defined $foo && $foo->isa('Foo') );
> >
> > which rapidly gets tiresome.
>
> Or ok(UNIVERSAL::isa(Foo->new, 'Foo'));

Under 5.005 it spits out warnings if Foo->new returns undef.  Plus you
still don't get any useful diagnostics.  You just know it failed and then
you'll have to go back and add debugging print statements or use the
debugger to figure out _why_ it failed (did Foo->new return undef,
something blessed into a different class, an unblessed thing, etc).

I recently converted a rather large test suite (for Alzabo, 600+ unique
tests or so) to use Test::More.  Doing so helped me do several things:

1.  Break down what I was testing into smaller units.  Where I used to
have things like:

 ok( @foo && defined $foo[0] && $foo[0]->name eq 'bob' );

I now have:

 is( @foo, 2, "expect two elements from bar() call" );
 isa_ok( $foo[0], "Frobnigator" );
 is( $foo[0]->name, 'bob', "the frobnigator should be named 'bob'" );

The presence of these similar but slightly different utility functions
helped me really think about exactly what I should be testing.

2.  Diagnosing new failures.  The diagnostics are such that I rarely have
to go back and add print statements or use the debugger.  I know _why_
things are failing right away, which is better than simply knowing that
they failed for some reason or other.

3.  Removed liable to break support code bits that were being repeated
everywhere.

So in the end I have more tests, which are easier to diagnose, with less
support code that I have to maintain.  This is good because Alzabo is very
complicated as it is and the tests have enough weird life support to do
things like test against different RDBMS backends and such.  Anything I
don't have to write makes me happier.

And if there's a problem with the testing code, I can report the bug to
Schwern rather than fixing it in my five test suites where I copied the
code around.

Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
see why people seem to object to the use of Test::More in the core Perl
tests.  I can't see how it couldn't help improve the quality of the tests
while providing a standardized way to do the things that all the tests do
anyway.


-dave

/*==
www.urth.org
We await the New Sun
==*/

Re: Untested modules update: There's more than we thought

[Kurt D. Starsinic]

On Dec 14, Michael Schwern wrote:
> Constructors fail often enough that I felt it worthwhile to write a
> test function for them.
> 
> You could leave off the test entirely and just trust that you use $foo
> later on down in the test and it'll explode there, but its better to
> put the point of failure as close to where the bug is as possible.
> And you get no diagnostic information (see below).
> 
> The equivalent code without isa_ok() would be:
> 
> my $foo = Foo->new;
> ok( $foo->isa('Foo') );
> 
> except should $foo be unblessed or undef that will explode.  You have
> to start doing things like:
> 
> my $foo = Foo->new;
> ok( defined $foo && $foo->isa('Foo') );
> 
> which rapidly gets tiresome.

Or ok(UNIVERSAL::isa(Foo->new, 'Foo'));

- Kurt

Re: Untested modules update: There's more than we thought

[Tels]

-BEGIN PGP SIGNED MESSAGE-

Moin,

On 15-Dec-01 Jarkko Hietaniemi tried to scribble about:
>> Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
>> see why people seem to object to the use of Test::More in the core Perl
>> tests.  I can't see how it couldn't help improve the quality of the
>> tests
>> while providing a standardized way to do the things that all the tests
>> do
>> anyway.
> 
> Mainly it's a question of bootstrapping.  Test::More is quite fancy
> and unless all the Perl it needs is working, you are not testing
> what you think you are testing.  Requiring poetry before you know
> how to spell is unfair.  Try meditating on the word "core".

But it should be okay for additional modules, like Math::BigInt. One day I
need to switch to Test::More, since the "add debugging print statements"
get's on my nerves.

Cheers,

Te"One day Math::Big* will have more tests than Perl;)"ls

- -- 
 perl -MDev::Bollocks -e'print Dev::Bollocks->rand(),"\n"'
 confidentially innovate market-driven customers

 http://bloodgate.com/perl   My current Perl projects
 PGP key available on http://bloodgate.com/tels.asc or via email 

-BEGIN PGP SIGNATURE-
Version: 2.6.3i
Charset: latin1

iQEVAwUBPBvTNncLPEOTuEwVAQGW0wf7BafFa25z0gTcGQ89DRh140yevRcuxmFg
4x43kdy9NiBtal7k2RMrkkyglhTJCd6ytIihE0Bsg6/tufjm5nHRyHqT+Anx1L+9
0BX7tPzfcTJ68oPV24enczbtLv3hsjcCDipMPrz/gcE/Ua6I2Q/kitagxV44hh4p
Tl8sDVHeuHNskkooOAPobo99URwjBdGckfiMn1Q9kSg8T7wuyUG8+bvdkshM0w5v
tQMt8pwarzmHhR+dzrSOxIaEfhr28ldC/anWbhPsH9rzjsty7cjnIzLm/LRDLH8s
moirvu4UFFj5aToqHqSiEwtQyKzYfbX3TIUQkeKz5yL/qKGibpoT1g==
=TeV5
-END PGP SIGNATURE-

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sat, Dec 15, 2001 at 11:48:29PM +0100, Tels wrote:
> Te"One day Math::Big* will have more tests than Perl;)"ls

One day Perl will have more tests than Bit::Vector.  67255 at last count.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Is there an airport nearby or is that just my tae-kwon-do taking off?

Re: Untested modules update: There's more than we thought

[Paul Johnson]

On Sat, Dec 15, 2001 at 11:46:29PM +, Nicholas Clark wrote:
> On Sat, Dec 15, 2001 at 05:59:22PM -0500, Michael G Schwern wrote:
> > On Sat, Dec 15, 2001 at 11:48:29PM +0100, Tels wrote:
> > > Te"One day Math::Big* will have more tests than Perl;)"ls
> > 
> > One day Perl will have more tests than Bit::Vector.  67255 at last count.
> 
> Ilya chose not to attempt that for 5.6, by make op/numconvert.t only report
> 1 ok (or not) for a cluster of 15 numbers passed through the same conversions.
> We could always change op/numconvert.t to report 15 times as many tests
> (about another 20,000 I think)
> 
> But that's cheating, isn't it?

My Gedcom module has 7671 tests.  Most of these are from checking a 1476
line file five times.  I could easily make that file bigger ...

Although really I'd prefer to make it smaller.  It used to take about 15
mins to run on my old Dec.  Nowadays that's not such a priority :-)

> More on topic - I like Test::More.
> I like test.pl in the core.
> But I fear that we aren't bootstrapping our tests carefully enough.
> In that I'm pretty sure we shouldn't be using Test::More until we're
> sure we've tested all the ops, and we shouldn't be using test.pl until
> we've tested all the ops it uses (and be careful to limit the ops
> and the complexity of ops it uses)
> 
> TEST starts:
> 
> # This is written in a peculiar style, since we're trying to avoid
> # most of the constructs we'll be testing for.
> 
> Maybe it should print that out on each run, to remind everyone :-)

I agree.  That is, I share your fears.  I also wonder about TEST.  Last
time I did a patch for it I was surprised at just what constructs it did
include.

Maybe all the constructs it uses are tested earlier, but after all the
fingers that have been in the pie, it's probably time for someone to
take a good look at what gets tested where to make sure that the various
test programs do only use previously tested constructs as far as
possible.

-- 
Paul Johnson - [EMAIL PROTECTED]
http://www.pjcj.net

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sat, Dec 15, 2001 at 02:19:21PM -0800, Kurt D. Starsinic wrote:
> On Dec 15, Dave Rolsky wrote:
> > Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't
> > see why people seem to object to the use of Test::More in the core Perl
> > tests.  I can't see how it couldn't help improve the quality of the tests
> > while providing a standardized way to do the things that all the tests do
> > anyway.
> 
> I, for one, don't object to the use of Test::More in the core Perl
> tests.  I'm just being responsibly skeptical about the promotion of many
> different test harness modules with many different functions.  It runs
> the risk of _seeming_ like the People in Charge don't know what testing
> is for, and of not providing effective guidance to those people writing
> their first tests.

We have Test::Simple, Test::Tutorial and eventually Test::FAQ for such
guidance.  Work on those if you're worried about providing effective
guidance.  Beyond that, I'm not going to cut down my toolbox to only a
hammer just because it looks tidy.  (And its PERL!  Of course you're
going to have 25 different types of bevels in the toolbox.)

Its also been "discovered" just recently that its now possible to
inter-mix several different Test::* modules in the same program, as
long as they all use the Test::Builder backend (and Barrie found a way
to cheat with Test.pm).
http:[EMAIL PROTECTED]/msg01091.html

This means its no longer necessary to write monolithic test modules.
You can write special purpose modules for special purpose things and
combine them as you please.  After the next release, which will
complete the interface with cmp_ok() (aka. is_op), unlike() and
eval_ok(), Test::More's growth is pretty much over.  I'll leave the
rest to seperate modules.


> I also think that we have a real testing problem, which is that it
> isn't _trivially_ easy to run a huge test suite (like Alzabo's, or Perl's)
> and to find out at the end which tests failed and what their output was.
> 3000 lines and 20 minutes later, a message that 3 out of 1 tests
> failed is not maximally useful.  Greater variety of test functions is not
> (IMNSHO) nearly as important as addressing that.

Test::More outputs its diagnostics to STDERR, so they appear as you
run 'make test'.

PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -e 'use Test::Harness 
qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
t/Module-InfoNOK 3# Failed test (t/Module-Info.t at line 15) 
#  got: 'foo'
# expected: 'bar'
t/Module-Infook 73/74# Looks like you planned 74 tests but ran 1 extra.  
t/Module-Infodubious 
Test returned status 2 (wstat 512, 0x200)
DIED. FAILED test 3
Failed 1/74 tests, 98.65% okay
Failed Test Stat Wstat Total Fail  Failed  List of Failed
---
t/Module-Info.t2   512741   1.35%  3
Failed 1/1 test scripts, 0.00% okay. 0/74 subtests failed, 100.00% okay.

Even in the core:

$ ./perl TEST ../lib/foo.t
t/../lib/foo# Failed test (../lib/foo.t at line 10)
#  got: 'foo'
# expected: 'bar'
# Looks like you failed 1 tests of 1.
FAILED at test 1
Failed 1 test script out of 1, 0.00% okay.

Its a little messy, but it works.  I'd welcome help making it look
nicer.

However t/test.pl (the low-fat Test::More substitute which almost
everything in t/ uses) puts its diagnostics on STDOUT which gets
sucked up by Test::Harness.  That could be changed.

As for providing further information at the end of a Perl core test
run, that requires modifying t/TEST which has its own set of mental
health problems.


Finally, if you want to see real movement in improving the state of
'make test' output, help out finishing up Test::Harness 2.00
http://www.pobox.com/~schwern/src/Test-Harness-2.00_01.tar.gz and
finishing the ExtUtils::* tests so we can work on that safely (the
lives of MakeMaker and Test::Harness being intertwined).


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
An official "I want James Earl Jones' cock up my ass" t-shirt.
http://www.goats/com/archive/010303.html

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sat, Dec 15, 2001 at 11:46:29PM +, Nicholas Clark wrote:
> On Sat, Dec 15, 2001 at 05:59:22PM -0500, Michael G Schwern wrote:
> > On Sat, Dec 15, 2001 at 11:48:29PM +0100, Tels wrote:
> > > Te"One day Math::Big* will have more tests than Perl;)"ls
> > 
> > One day Perl will have more tests than Bit::Vector.  67255 at last count.
> 
> Ilya chose not to attempt that for 5.6, by make op/numconvert.t only report
> 1 ok (or not) for a cluster of 15 numbers passed through the same conversions.
> We could always change op/numconvert.t to report 15 times as many tests
> (about another 20,000 I think)

Finer granularity is often better than lumping them all together,
makes it easier to figure out which one is failing.  numconvert.t gets
around this by the way it reports its failures with the big blat of
diagnostics.

In the interests of myself never wanting to touch numconvert.t for
fear of it getting angry, digitizing and being thrown onto the game
grid, do whatever you feel is best with it. :)


> But that's cheating, isn't it?

It may be useless, but its fun to see who can pee the farthest. ;)


> More on topic - I like Test::More.
> I like test.pl in the core.
> But I fear that we aren't bootstrapping our tests carefully enough.
> In that I'm pretty sure we shouldn't be using Test::More until we're
> sure we've tested all the ops

Already done.  By the Treaty Of t/, Test::More's domain is limited to
ext/ and lib/.  And Andy's fears about its widespread use leading to a
cascade failure has come true, as it collapsed on djgpp and took a
good chunk of lib/ with it.  Still working out my djgpp environment to
look at that.


> and we shouldn't be using test.pl until we've tested all the ops it
> uses (and be careful to limit the ops and the complexity of ops it
> uses)

Sort of.  t/test.pl is written to avoid the more twitchy things which
Test::More plays with (like duping filehandles) and does keep the
"peculiar style" axiom in mind.  However, trying to limit the use of
t/test.pl to only things which it doesn't use at all can get
complicated and somewhat tedious as it leaves us without a test
library for good chunk of the ops.  It also gets into the tricky
problem of finely ordering the tests to they run in some sort of
dependency order, and I really don't want to get into that right now.

t/test.pl using, say, caller() and it also being used in t/op/caller.t
(which, umm... I can't seem to find) is fine as long as t/test.pl
limits itself to the basic, rock-solid functionality of caller.  If
t/test.pl started using things like the more obscure bits of caller()
that's a bit more likely to break (say, the $hints) I'd be worried.


> TEST starts:
> 
> # This is written in a peculiar style, since we're trying to avoid
> # most of the constructs we'll be testing for.

Which is sort of a white lie these days, but its still quite a bit
simpler than Test::Harness.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
It sure is fun masturbating.
http://www.unamerican.com/

Re: Untested modules update: There's more than we thought

[Jarkko Hietaniemi]

On Fri, Dec 14, 2001 at 05:20:24PM -0700, chromatic wrote:
> On Fri, 14 Dec 2001 15:18:43 -0700, Michael G Schwern wrote:
> 
> > ExtUtils::MM_Cygwin
> 
> Here's a test for that.  It could use someone on Cygwin testing it.  Tels said
> he would, but apparently got busy.

Thanks, applied.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Fri, Dec 14, 2001 at 05:20:24PM -0700, chromatic wrote:
> On Fri, 14 Dec 2001 15:18:43 -0700, Michael G Schwern wrote:
> > ExtUtils::MM_Cygwin
> 
> Here's a test for that.  It could use someone on Cygwin testing it.
> Tels said he would, but apparently got busy.

Checks out ok on Cygwin98.


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
The beatings will continue until your style improves.

Re: Untested modules update: There's more than we thought

[chromatic]

On Fri, 14 Dec 2001 15:18:43 -0700, Michael G Schwern wrote:

> ExtUtils::MM_Cygwin

Here's a test for that.  It could use someone on Cygwin testing it.  Tels said
he would, but apparently got busy.

-- c

--- ~MANIFEST   Sun Nov 25 19:50:46 2001
+++ MANIFESTSun Nov 25 21:08:01 2001
@@ -930,6 +930,7 @@
 lib/ExtUtils/Mkbootstrap.t See if ExtUtils::Mkbootstrap works
 lib/ExtUtils/Mksymlists.pm Writes a linker options file for extensions
 lib/ExtUtils/MM_Cygwin.pm  MakeMaker methods for Cygwin
+lib/ExtUtils/MM_Cygwin.t   See if ExtUtils::MM_Cygwin works
 lib/ExtUtils/MM_NW5.pm MakeMaker methods for NetWare
 lib/ExtUtils/MM_OS2.pm MakeMaker methods for OS/2
 lib/ExtUtils/MM_Unix.pmMakeMaker base class for Unix
--- /dev/null   Thu Aug 30 03:54:37 2001
+++ lib/ExtUtils/MM_Cygwin.tMon Nov 26 17:56:44 2001
@@ -0,0 +1,105 @@
+#!./perl
+
+BEGIN {
+   chdir 't' if -d 't';
+   @INC = '../lib';
+}
+
+use Test::More;
+
+BEGIN {
+   if ($^O =~ /cygwin/i) {
+   plan tests => 17;
+   } else {
+   plan skip_all => 'Test irrelevant outside of Cygwin';
+   }
+}
+
+use Config;
+use File::Spec;
+
+# MM package faked up by messy MI entanglement
+@MM::ISA = qw( ExtUtils::MM_Unix ExtUtils::Liblist::Kid ExtUtils::MakeMaker );
+
+use_ok( 'ExtUtils::MM_Cygwin' );
+
+# test canonpath
+my $path = File::Spec->canonpath('/a/../../c');
+is( MM->canonpath('/a/../../c'), $path,
+   'canonpath() method should work just like the one in File::Spec' );
+
+# test cflags, with the fake package below
+my $args = bless({
+   CFLAGS  => 'fakeflags',
+   CCFLAGS => '',
+}, MM);
+
+# with CFLAGS set, it should be returned
+is( $args->cflags(), 'fakeflags',
+   'cflags() should return CFLAGS member data, if set' );
+
+delete $args->{CFLAGS};
+
+# ExtUtils::MM_Cygwin::cflags() calls this, fake the output
+*ExtUtils::MM_Unix::cflags = sub { return $_[1] };
+
+# respects the config setting, should ignore whitespace around equal sign
+my $ccflags = $Config{useshrplib} eq 'true' ? ' -DUSEIMPORTLIB' : '';
+$args->cflags(<{CFLAGS}, qr/OPTIMIZE = opt/, '... should set OPTIMIZE' );
+like( $args->{CFLAGS}, qr/PERLTYPE = pt/, '... should set PERLTYPE' );
+like( $args->{CFLAGS}, qr/LARGE = lg/, '... should set LARGE' );
+like( $args->{CFLAGS}, qr/SPLIT = split/, '... should set SPLIT' );
+like( $args->{CFLAGS}, qr/CCFLAGS = $ccflags/, '... should set CCFLAGS' );
+
+# test manifypods
+$args = bless({
+   NOECHO => 'noecho',
+   MAN3PODS => {},
+   MAN1PODS => {},
+}, 'MM');
+like( $args->manifypods(), qr/pure_all\n\tnoecho/,
+   'manifypods() should return without PODS values set' );
+
+$args->{MAN3PODS} = { foo => 1 };
+my $out = tie *STDOUT, 'FakeOut';
+my $res = $args->manifypods();
+like( $$out, qr/could not locate your pod2man/,
+   '... should warn if pod2man cannot be located' );
+like( $res, qr/POD2MAN_EXE = -S pod2man/,
+   '... should use default pod2man target' );
+like( $res, qr/pure_all.+foo/, '... should add MAN3PODS targets' );
+
+$args->{PERL_SRC} = File::Spec->updir;
+$args->{MAN1PODS} = { bar => 1 };
+$$out = '';
+$res = $args->manifypods();
+is( $$out, '', '... should not warn if PERL_SRC provided' );
+like( $res, qr/bar \\\n\t1 \\\n\tfoo/, '... should join MAN1PODS and MAN3PODS');
+
+# test perl_archive
+my $libperl = $Config{libperl} || 'libperl.a';
+is( $args->perl_archive(), "\$(PERL_INC)/$libperl",
+   'perl_archive() should respect libperl setting' );
+
+# test import of $Verbose and &neatvalue
+can_ok( 'ExtUtils::MM_Cygwin', 'neatvalue' );
+is( $ExtUtils::MM_Cygwin::Verbose, $ExtUtils::MakeMaker::Verbose, 
+   'ExtUtils::MM_Cygwin should import $Verbose from ExtUtils::MakeMaker' );
+
+package FakeOut;
+
+sub TIEHANDLE {
+   bless(\(my $scalar), $_[0]);
+}
+
+sub PRINT {
+   my $self = shift;
+   $$self .= shift;
+}

lib.t (was Re: Untested modules update: There's more than we thought)

[Michael G Schwern]

On Fri, Dec 14, 2001 at 05:57:06PM -0500, Michael G Schwern wrote:
> > #how about use's?
> > is(
> > `perl "-Mlib=stuff -MYup" -e "print 'ok'"`,
> > 'ok',
> > 'affects use'
> > );
> 
> A simple way to trick 'use' is to clear the entry from %INC and then
> eval "use Yup".

I just thought of a better way.  Since all we're testing is that
lib.pm does the right things to @INC, we can presume that if one of
require(), do() or use() works, the rest will work.

So for most everything, set Yup.pm to return some unique value and
then just use do() for most everything.  That way it'll work multiple
times in the same program without eval or `` trickery.

print YUP  Kwalitee Is Job One
I was *meant* to mount your donuts.

Re: Untested modules update: There's more than we thought

[Jarkko Hietaniemi]

On Fri, Dec 14, 2001 at 05:50:16PM -0500, Michael G Schwern wrote:
> On Sat, Dec 15, 2001 at 12:21:08AM +0200, Jarkko Hietaniemi wrote:
> > On Fri, Dec 14, 2001 at 05:18:43PM -0500, Michael G Schwern wrote:
> > > The good news is Nick Clark just wrote a test for Benchmark!  The bad
> > > news is I made the mistake at looking closer at ext/B/ today, so the
> > > list just got bigger and quite a bit harder.
> > > 
> > > There's still some fruit that's below giraffe-level, lib and re.
> > 
> > Clutching to that money, eh? :-)
> 
> More like leading p5p along with a $500 bill on a stick. :)
> 
> I'll take a break from test driving bleadperl and handle the B::* and
> the *Loader nasties.  I might be able to do CPAN, too.  It'll give the
> rest of you time to catch up on all those make warnings *wink wink,
> nudge nudge*.

The cc warnings?  #13691.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Fri, Dec 14, 2001 at 02:48:07PM -0800, Brent Dax wrote:
> I don't know what directory this should go into, 

lib/lib.t I guess.


> so I'm not providing a
> patch, just pasting the body of lib.t in here.  Good enough?
> 
> #!./perl
> 
> use Test::More tests => 3;
> use lib 'stuff';
> 
> #create a directory and a module
> mkdir('stuff', 0666);
> open(MOD, '>stuff/Yup.pm');
> print MOD 'package Yup; 1;';
> close MOD;
> 
> 
> #did the directory show up?
> ok(scalar(grep {/stuff/} @INC), 'added to @INC');

I believe lib.pm is guaranteed to stick the new lib on the front.  So
it should appear on @INC[0].


> #does it change require's behavior?
> ok(
>   eval { require Yup; 1; },
>   'affects require'
> );

Check that $INC{'Yup'} contains the right file.


> #how about use's?
> is(
>   `perl "-Mlib=stuff -MYup" -e "print 'ok'"`,
>   'ok',
>   'affects use'
> );

A simple way to trick 'use' is to clear the entry from %INC and then
eval "use Yup".


> #cleanup
> unlink 'stuff/Yup.pm';
> unlink 'stuff';

rmdir 'stuff'?


Other things need checked.  

no lib 

the magic behavior when there's $dir/$archname/auto.

@lib::ORIG_INC

That trailing duplicate entries are removed (ie. so @INC doesn't
grow under mod_perl)



-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
humble abode cries
there is not enough paste here
restock erotica please
-- imploded

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Sat, Dec 15, 2001 at 12:21:08AM +0200, Jarkko Hietaniemi wrote:
> On Fri, Dec 14, 2001 at 05:18:43PM -0500, Michael G Schwern wrote:
> > The good news is Nick Clark just wrote a test for Benchmark!  The bad
> > news is I made the mistake at looking closer at ext/B/ today, so the
> > list just got bigger and quite a bit harder.
> > 
> > There's still some fruit that's below giraffe-level, lib and re.
> 
> Clutching to that money, eh? :-)

More like leading p5p along with a $500 bill on a stick. :)

I'll take a break from test driving bleadperl and handle the B::* and
the *Loader nasties.  I might be able to do CPAN, too.  It'll give the
rest of you time to catch up on all those make warnings *wink wink,
nudge nudge*.



-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
Anti-cow device, don't bother.
http://www.goats.com/archive/990323.html

RE: Untested modules update: There's more than we thought

[Brent Dax]

Michael G Schwern:
# lib *new*

I don't know what directory this should go into, so I'm not providing a
patch, just pasting the body of lib.t in here.  Good enough?

#!./perl

use Test::More tests => 3;
use lib 'stuff';

#create a directory and a module
mkdir('stuff', 0666);
open(MOD, '>stuff/Yup.pm');
print MOD 'package Yup; 1;';
close MOD;


#did the directory show up?
ok(scalar(grep {/stuff/} @INC), 'added to @INC');


#does it change require's behavior?
ok(
eval { require Yup; 1; },
'affects require'
);


#how about use's?
is(
`perl "-Mlib=stuff -MYup" -e "print 'ok'"`,
'ok',
'affects use'
);


#cleanup
unlink 'stuff/Yup.pm';
unlink 'stuff';

__END__

--Brent Dax
[EMAIL PROTECTED]
Configure pumpking for Perl 6

"Nothing important happened today."
--George III of England's diary entry for 4-Jul-1776

Re: Untested modules update: There's more than we thought

[Michael G Schwern]

On Fri, Dec 14, 2001 at 02:26:35PM -0800, Russ Allbery wrote:
> I forget where the wiki is.

WikiHere
http://www.pobox.com/~schwern/cgi-bin/perl-qa-wiki.cgi?UntestedModules

-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
The prefered method of receiving a paste enema is to go to St. Mark's
Place, stand in the middle of the street, and shout "I NEED A PASTE
ENEMA!"  You'll get something, anyway.

Untested modules update: There's more than we thought

[Michael G Schwern]

The good news is Nick Clark just wrote a test for Benchmark!  The bad
news is I made the mistake at looking closer at ext/B/ today, so the
list just got bigger and quite a bit harder.

There's still some fruit that's below giraffe-level, lib and re.

B::Asmdata  *new*
B::Assembler*new*
B::Bblock   *new*
B::Bytecode *new*
B::C*new*
B::CC   *new*
B::Concise  *new*
B::Disassembler *new*
B::Lint *new*
B::Stackobj *new*
B::Xref *new*
Byteloader  *new*
CPAN
Dynaloader  *new*
ExtUtils::MM_Cygwin
ExtUtils::MM_NW5
ExtUtils::MM_OS2
ExtUtils::MM_Unix
ExtUtils::MM_VMS
ExtUtils::MM_Win32
ExtUtils::Install
ExtUtils::Installed
ExtUtils::Liblist
ExtUtils::Mksymlists
lib *new*
Net::Cmd
Net::Domain
Net::Ping
Net::POP3
O
Pod::Functions (no docs!)
Pod::Html
Pod::InputObjects
#Pod::Man [[EMAIL PROTECTED]]
#Pod::ParseLink [[EMAIL PROTECTED]]
Pod::Select
#Pod::Text::Color [[EMAIL PROTECTED]]
#Pod::Text::Overstrike [[EMAIL PROTECTED]]
#Pod::Text::Termcap [[EMAIL PROTECTED]]
Pod::Usage
re  *new*
XSLoader*new*


-- 

Michael G. Schwern   <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance  <[EMAIL PROTECTED]> Kwalitee Is Job One
My breasts are arousing weapons.