Re: Cpan Dist Layout Issue
On Fri, May 15, 2009 at 1:29 PM, Smylers wrote: > Most of the information on that page is likely to be bewildering or > irrelevant to the user who's just stumbled upon the site, but it does > say mention that it's "a way to provide multi-platform testing for > modules" and provides "valuable feedback for users". Yes, well, I agree with you there. Care to draft some replacement text and pitch it to Barbie? I believe the source is here: http://github.com/barbie/cpan-www-testers/tree/master Maybe we need some simple, web 2.0 interface like Adam did for Strawberry Perl and explain CPAN Testers in 140 characters or less. :-) > Anyway, our sys-admin could easily end up deciding that Cpan Testers > results are clearly wrong, and think less of Cpan Testers or Cpan or > Perl or the Perl Community. Maybe in future when evaluating software > he'll be more inclined to choose something written in another language, > so he doesn't have to deal with the trials of Cpan again. So instead your sysadmin will go choose something in a language that doesn't have a code library repository of the breadth and depth of CPAN, one which doesn't bother to encourage a culture of testing, and doesn't bother to collect test results at all? Because it's simpler and less confusing than the 'trials' of CPAN? That doesn't say much for the quality of your sysadmin, does it? On the other hand, you're right -- people will often choose the lesser path because it doesn't require them to think as much. > And that's fair enough. But possibly there are those in the Perl > community who think this perception is unfortunate and are interested in > doing something about it, making the Perl World a friendlier or more > helpful place. Seen Alias's rants about Perl websites? Or the Iron Man blogging competition? I'm totally with you on the broad point, just not on the cheap shot at CPAN Testers. Do you have any idea how long it took to get the CPAN Testers websites handed over at all? (Years) Or how many flamewars there have been on how to make CPAN Testers less annoying? (At least a couple a year, though less since we stopped emailing authors directly.) Or how much work is being done to improve the quality and utility of the reports? Or to switch away from email as a transport mechanism? It's not that we don't know that parts of it suck, it's just that we're working on other things. Rant aside -- please do send patches if you think you can make it better. We could use all the help we can get. > [*1] Though actually is there anything stopping somebody from checking > whether modules actuall install and sending FAIL results to Cpan Testers > for those that didn't? Surely they'd just display like other FAILs. > Yes that'd be more involved than what current testers are doing, but I > don't think it means it couldn't be done by somebody. Nothing is there to stop it, though I'd personally oppose it and push to drop those reports on the floor so it doesn't confuse matters. One of the reasons that we decided to have Build/make failures become UNKNOWN was to help authors distinguish them from test FAIL reports. I've supported expanding the grades in the past and would do so again if someone were to argue for an INSTALLFAIL grade or equilvalent for the case you describe. -- David
Re: Cpan Dist Layout Issue
On May 15, 2009, at 12:00 PM, Smylers wrote: Graham Barr writes: It is documented in ExtUtils::MakeMaker PMLIBDIRS Ref to array of subdirectories containing library files. Thanks. So for this modules it will scan lib/ and DWIW/ neither of which exist. It also scans . Having never tried to use a directory layout other than ./lib/ I hadn't thought about where this came from, and having seen the correct filename mentioned in somewhere in Makefile.PL presumed that meant MakeMaker knew about it. It actually predates MM supporting lib/ Many XS extensions have the main pm file in the top directory So would it make sense for MakeMaker to warn if a file mentioned in VERSION_FROM (or ABSTRACT_FROM -- any others?) is in a directory that isn't in PMLIBDIRS? I actually have a dist which deliberately does just that Graham.
Re: Cpan Dist Layout Issue
David Golden writes:
> On Fri, May 15, 2009 at 12:11 PM, Smylers wrote:
>
> > That Perl has this QA thing which tells you which platforms a
> > module will work on but that doesn't actually mean it'll install
> > only adds to external perceptions of Perl being confusing and
> > awkward.
>
> Of course CPAN Testers doesn't claim to check whether a module
> installs, only whether it passes tests.
You say "of course", but I'm not sure. _I_ know what Cpan Testers
does[*1], but I don't think it's so obvious to those outside (or new to)
the Perl community.
Suppose somebody finds a distribution on Cpan Search:
http://search.cpan.org/~jzawodny/DBIx-DWIW-0.49/
There's a mention of "CPAN Testers", without explanation. "PASS (46)"
doesn't say what it has passed. 'View Reports' goes to:
http://www.cpantesters.org/show/DBIx-DWIW.html#DBIx-DWIW-0.49
Again there's nothing there which says what is being tested. There's a
promising looking 'Help' box, which starts "So you've stumbled upon this
site and are wondering what it all means ..."; that links to:
http://www.cpantesters.org/help.html
Most of the information on that page is likely to be bewildering or
irrelevant to the user who's just stumbled upon the site, but it does
say mention that it's "a way to provide multi-platform testing for
modules" and provides "valuable feedback for users".
So I'm struggling to see where our sys-admin would learn that Cpan
Testers don't actually test that modules install on those platforms.
Remember, he isn't a Perl programmer. In fact, he isn't a programmer at
all: he's only involved in this at all because he wishes to run a
program, and that program happens to be written in Perl, with some Cpan
dependencies.
> It doesn't even tell you whether there are good tests. It could be
> t/pass.t with a single pass("fooled you") test and it would still get
> a PASS grade on CPAN Testers.
Again, I know that but I'm not sure it's obvious to somebody who doesn't
(though it's probably easier to deduce than not checking installation).
Anyway, our sys-admin could easily end up deciding that Cpan Testers
results are clearly wrong, and think less of Cpan Testers or Cpan or
Perl or the Perl Community. Maybe in future when evaluating software
he'll be more inclined to choose something written in another language,
so he doesn't have to deal with the trials of Cpan again.
One possible response to this is that he's wrong: he's made an incorrect
presumption, and if he goes off elsewhere that's his loss.
And that's fair enough. But possibly there are those in the Perl
community who think this perception is unfortunate and are interested in
doing something about it, making the Perl World a friendlier or more
helpful place.
[*1] Though actually is there anything stopping somebody from checking
whether modules actuall install and sending FAIL results to Cpan Testers
for those that didn't? Surely they'd just display like other FAILs.
Yes that'd be more involved than what current testers are doing, but I
don't think it means it couldn't be done by somebody.
Smylers
Re: Cpan Dist Layout Issue
Graham Barr writes: > It is documented in ExtUtils::MakeMaker > > PMLIBDIRS > Ref to array of subdirectories containing library files. Thanks. > So for this modules it will scan lib/ and DWIW/ neither of which > exist. It also scans . Having never tried to use a directory layout other than ./lib/ I hadn't thought about where this came from, and having seen the correct filename mentioned in somewhere in Makefile.PL presumed that meant MakeMaker knew about it. So would it make sense for MakeMaker to warn if a file mentioned in VERSION_FROM (or ABSTRACT_FROM -- any others?) is in a directory that isn't in PMLIBDIRS? Smylers
Re: Cpan Dist Layout Issue
On Fri, May 15, 2009 at 12:11 PM, Smylers wrote:
> That Perl has this QA thing which tells you which platforms a module
> will work on but that doesn't actually mean it'll install only adds to
> external perceptions of Perl being confusing and awkward.
Of course CPAN Testers doesn't claim to check whether a module
installs, only whether it passes tests. It doesn't even tell you
whether there are good tests. It could be t/pass.t with a single
pass("fooled you") test and it would still get a PASS grade on CPAN
Testers.
Automated testing doesn't remove to need for real people to find bugs
and report them.
-- David
Re: Cpan Dist Layout Issue
On May 15, 2009, at 11:11 AM, Smylers wrote: I recently encountered a Cpan module which doesn't install from its distribution yet passes all its tests, and I was wondering whether the glitch is a generic one which should be addressed. This particular module is not my concern (its maintainer responded speedily and is now looking at fixing it), but it illustrates the issue: http://cpansearch.perl.org/src/JZAWODNY/DBIx-DWIW-0.49/ Note the module source is in the tarball at ./DBIx/DWIW.pm, and that is reflected in Makefile.PL. Running make creates the ./blib/lib/DBIx/ directory but doesn't copy the module to it (nor generate the manpage). It appears to work fine with the source at either ./DWIW.pm or ./lib/DBIx/DWIW.pm (and a corresponding Makefile.PL). * Is this a known limitation of MakeMaker or a bug? If a bug then I'll report it. If a limitation, would it be possible for MakeMaker to detect this condition and bail out (on both make dist and make), rather than giving the impression it's succeeded? It is documented in ExtUtils::MakeMaker PMLIBDIRS Ref to array of subdirectories containing library files. Defaults to [ 'lib', $(BASEEXT) ]. The directories will be scanned and any files they contain will be installed in the corresponding location in the library. A libscan() method can be used to alter the behaviour. Defining PM in the Makefile.PL will override PMLIBDIRS. (Where BASEEXT is the last component of NAME.) So for this modules it will scan lib/ and DWIW/ neither of which exist. It also scans . The install mapping for this module would be foo.pm => DBIx/foo.pm DWIW/bar.pm => DBIx/DWIW/bar.pm lib/baz.pm => baz.pm Graham.
Cpan Dist Layout Issue
I recently encountered a Cpan module which doesn't install from its
distribution yet passes all its tests, and I was wondering whether the
glitch is a generic one which should be addressed.
This particular module is not my concern (its maintainer responded
speedily and is now looking at fixing it), but it illustrates the issue:
http://cpansearch.perl.org/src/JZAWODNY/DBIx-DWIW-0.49/
Note the module source is in the tarball at ./DBIx/DWIW.pm, and that is
reflected in Makefile.PL. Running make creates the ./blib/lib/DBIx/
directory but doesn't copy the module to it (nor generate the manpage).
It appears to work fine with the source at either ./DWIW.pm or
./lib/DBIx/DWIW.pm (and a corresponding Makefile.PL).
* Is this a known limitation of MakeMaker or a bug? If a bug then I'll
report it. If a limitation, would it be possible for MakeMaker to
detect this condition and bail out (on both make dist and make),
rather than giving the impression it's succeeded?
* make test succeeded because perl puts the current directory at the end
of @INC. Because DBIx::DWIW isn't in ./blib/, the one at the top of
the tarball is found and used for the tests.
Obviously this doesn't happen with other tarball layouts because
./lib/ isn't in the path for make test (and ./DWIW.pm can't be in the
path, because the parent directory isn't called DBIx; that would be an
issue for single-level modules though).
* That the module doesn't install at all makes a slight mockery of the
many Cpan Testers reports giving it a PASS:
http://www.cpantesters.org/show/DBIx-DWIW.html#DBIx-DWIW-0.49
Yes, I know why that is, but that doesn't make it any better. It was
a sys-admin colleague with little Perl experience who encountered this
problem, while trying to install a program which uses this module.
That Perl has this QA thing which tells you which platforms a module
will work on but that doesn't actually mean it'll install only adds to
external perceptions of Perl being confusing and awkward.
* Even if MakeMaker were modified so that this distribution would
install, it's still concerning that modules with this distribution
layout get picked up by make test: there are other sorts of mistakes a
module author could make which would result in a bogus test pass
because of this.
* So it'd be nice if when running tests the current directory weren't in
@INC. Putting the following line in the Test and Test::More modules
would achieve this:
BEGIN { pop @INC if $INC[-1] eq '.' }
But that's so non-backwards-compatible that I'm guessing it would
cause problems elsewhere.
* An alternative would be to have make test do the above (for example by
putting the above line in a module and having make test -M that module
when running a test script). That limits the scope of the change, but
it's still backwards-incompatible -- and it means anybody running an
individual test directly still risks including a file they didn't
intend to.
* Or make test could cd into the blib/lib/ directory, such that the
current directory is one that should be in the path. But Michael
Schwern a few days ago argued that tests should be run from the parent
directory:
http://www.nntp.perl.org/group/perl.qa/2009/05/msg12215.html
Any thoughts?
Smylers
