Re: CPAN Test Data
[EMAIL PROTECTED] (Adam Turoff) wrote: >Allow me to clarify -- I'm interested in hacking the cpan-testers data. >This will most likely lead to a facelift for testers.cpan.org. :-) > >Graham asked me to discuss the changes I'd like to see with the >testers site. Here's what I have in mind: > > - Provide a downloadable version of the test database (generated >periodically, much like http://dmoz.org/rdf). > > - Provide summaries by module of the entire test history for >that module in a single file: > - module version tested > - Perl version tested (perl -V output, and parsed configuration) > - OS (+platform) tested > > - Provide brief summaries of what modules have been tested by >Perl release > > - Provide cross-reference pass/fail reports based on Perl configuration >option and release > > - Output should be available in multiple formats: XML, Text, and >if this does wind up being a facelift for testers.perl.org, HTML. :-) I would add to that: - Enhancement of cpantest.pl so that testers can easily check the testing history of a module on their platform The reason is that I generally just submit test results blindly, without checking whether someone else has already reported a particular package/version/platform combination. This works okay for me because I seem to be the only one reporting tests for the Darwin platform, but it would be nice to be assured that I'm not reporting redundant data. >Ideally, a rich API layer could be added on top of testers.cpan.org, so >that an install client (CPAN.pm) can provide relevant test information >to a user before a module is installed. Agreed. And it would be nice to build the cpantest.pl functionality into CPAN.pm too, for those that want it. >Maintaining a local copy of the (relevant) test data could help >smoke-test clients burn through a local CPAN mirror and torture test >CPAN. One might need to maintain a list of modules that can't just be downloaded and tested without some additional manual system configuration, because these may well fail in a rapid-fire smoke test. This includes modules like GD.pm (requires several external libraries) or HTML::Mason (has interactive 'make test'). ------ Ken Williams Last Bastion of Euclidity [EMAIL PROTECTED]The Math Forum
Re: Supressing "make test" jibber-jabber
Dave Rolsky <[EMAIL PROTECTED]> wrote: > On Tue, 2 Oct 2001, Michael G Schwern wrote: > >> Any particularly good reason we're showing all that "PERL_DL_NONLAZY" >> noise as part of "make test"? What about just: > > I do some weird stuff with that piece to pass data around to my tests. > Sometimes its useful to see it. How about only showing it if TEST_VERBOSE > is true? I agree, MakeMaker is an opaque enough process already, it won't necessarily help to silence it when it's trying to tell you what it's doing. TEST_VERBOSE should show the info if it goes away in the default case. -Ken
Re: Supressing "make test" jibber-jabber
Michael G Schwern <[EMAIL PROTECTED]> wrote: > On Tue, Oct 02, 2001 at 04:21:17PM -0500, Dave Rolsky wrote: >> On Tue, 2 Oct 2001, Michael G Schwern wrote: >> >> > Any particularly good reason we're showing all that "PERL_DL_NONLAZY" >> > noise as part of "make test"? What about just: >> >> I do some weird stuff with that piece to pass data around to my tests. > > You capture the output of 'make test' and manipuate that line > directly?? Eek! No, he(/we) override(s) the 'make test' command, and for informational purposes wants to see the modified command when running 'make test'. -Ken
Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]
On Thursday, August 29, 2002, at 05:29 PM, Michael G Schwern wrote:
> There's already the beginnings of something like this in
> t/TestInit.pm, but
> it wasn't updated for the PERL_CORE trick. Nor do I know if it can be
> entirely trusted.
>
> Also, Nick's example is a little odd. You usually don't
> want '.' (ie. t/)
> in your @INC. It's more like this:
>
> BEGIN {
> if($ENV{PERL_CORE}) {
> chdir 't';
> @INC = '../lib';
> }
> }
>
> but in some cases you need to include something more. For example,
> MakeMaker does this:
>
> BEGIN {
> if($ENV{PERL_CORE}) {
> chdir 't';
> @INC = ('../lib', 'lib');
> }
> else {
> unshift @INC, 't/lib';
> }
> }
>
> so it can see specialty helper modules in t/lib/.
I admit that I was a little surprised to see 'lib' stuff here
and not 'blib' stuff. Don't blibs get created when building
modules in the core?
-Ken
Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]
On Thursday, August 29, 2002, at 06:25 PM, Michael G Schwern wrote: > On Thu, Aug 29, 2002 at 06:23:26PM +1000, Ken Williams wrote: >> I admit that I was a little surprised to see 'lib' stuff here >> and not 'blib' stuff. Don't blibs get created when building >> modules in the core? > > Everything winds up in lib/ before the tests are run. All non-XS > modules > are never really "built", they're already in lib/. All XS modules are > built > and their results put into lib/ and then the tests are run. > > So, effectively, the core "installs" every modules into lib/ then runs > all > their tests. This is so module A's tests can find module B. Oh, one big lib/, not several different ones? So then why can't it be run with perl -Mblib=lib t/foo/bar.t ? I'm just trying to push the decision into a more appropriate place, though I guess in the end it's not such a big deal. -Ken
Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]
On Thursday, August 29, 2002, at 05:53 PM, Nicholas Clark wrote: > For information, my example was from the top of t/integer.t in Storable > It looks like the same bit of code was cargo-cult cut & pasted (er, by > me I think) from the other Storable tests (such as freeze.t) where they > have to require 'st-dump.pl'; to get a library of useful test routines. > > I'm not sure if Storable was actually the best (cleanest, simplest) > thing to > snag the example code from. I should point out my hidden agenda of making dual-life modules (on CPAN and in core) easier to manage. Ideally, I'd like for core modules to still have a CPAN life, and then some certain version just gets schlurped into the core. This is because I've been frustrated a number of times by modules that died on CPAN when they got integrated into the core, but needed bugs to be fixed, but wouldn't see new versions until perl got maintenance releases... -Ken
Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Tuesday, September 3, 2002, at 08:56 AM, Michael G Schwern wrote: > On Mon, Sep 02, 2002 at 08:13:06AM +0100, Nick Ing-Simmons wrote: >> I understand all that. My point was that while test itself may care >> where it is run, blib.pm does not mind as much. Also blib.pm's job >> is to make running an un-installed module "easy" which is what you >> want to do for a pre-install test. So if there is boiler-plate stuff >> to do blib.pm is an appropriate place to do it. >> >> Thus is you need to be in t this might suit >> >> cd t; perl -Mblib=lib foo/bar.t I'd be happier if that 'cd t;' happened inside blib.pm as 'chdir "t"'. I'd be happier still if tests didn't run from t/, since that's not how most CPAN modules work. > There's a chicken/egg problem here. blib.pm is going to find the lib/ > directory so we can load uninstalled modules. So... how do you find > blib.pm? This doesn't seem like a real problem. The test suite can find it because it knows where it's being run from. It can set up @INC however it wants, to make sure blib.pm is found. Then blib.pm sets things up for the rest of the tests. > Other problems include that we can't assume blib.pm works. > Don't want the > whole basic test suite falling apart because of a module failure. For core perl, we can indeed assume blib.pm works. If it doesn't, we want to know about it right away. > Finally, since you have to run from t/ anyway, blib.pm is > overkill. This > works just as well: > > cd t; ./perl -I../lib foo/bar.t I have to wonder why it's preferable to put code on command lines rather than in modules. > While the result is still ugly, it means we can expand and alter the > requirements for running a core test. For example, the PERL_CORE > environment variable should be set (t/TestInit.pm currently > doesn't). So > the full command is really something like this: > > cd t; > PERL_CORE=1 ./perl -I../lib path/to/test.t > > Which could be reduced somewhat to: > > ./perl -It -MTestInit t/path/to/test.t That's reasonable to me. I don't really care whether the code goes in TestInit.pm or blib.pm, but a disadvantage of blib.pm is that it's got a rather closed command-line interface, all you can do is specify a directory to treat as a blib/ . Really, it would be nice if you could tell the module about certain situations, like -MTestInit=core or whatever, and that would do whatever's necessary for a core test run (including chdir, @INC adjustments, etc.). Heck: with a smart enough TestInit, it could even adjust @ARGV so that you could specify all *.t paths in Unix format, if that would be helpful. > and perhaps reduced a little further by moving/linking > TestInit.pm into the > top level directory. > > ./perl -MTestInit t/path/to/test.t > > but that will cause trouble when running tests with -T (because . is no > longer in the path). '.' won't necessarily be in the path anyway. -Ken
Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Tuesday, September 3, 2002, at 03:10 PM, Michael G Schwern wrote: > On Tue, Sep 03, 2002 at 11:26:01AM +1000, Ken Williams wrote: >>>> Thus is you need to be in t this might suit >>>> >>>> cd t; perl -Mblib=lib foo/bar.t >> >> I'd be happier if that 'cd t;' happened inside blib.pm as 'chdir "t"'. > > It already happens inside TestInit. Cool. > [...] > So they can't run from the top level directory, and at that > point you're > already different than CPAN. A pity - it means that if they write files, etc. (which is presumably the only case where this consideration matters) they have to do it differently in the core vs. regular CPAN if they want the effect to be the same. Just another obstacle to having things lead dual lives. >>> There's a chicken/egg problem here. blib.pm is going to find >>> the lib/ >>> directory so we can load uninstalled modules. So... how do you find >>> blib.pm? >> >> This doesn't seem like a real problem. The test suite can find >> it because it knows where it's being run from. It can set up >> @INC however it wants, to make sure blib.pm is found. Then >> blib.pm sets things up for the rest of the tests. > > I thought the whole idea was to run blib.pm to set @INC. If > the test suite > is setting up @INC we don't need blib. blib.pm only adds to @INC. Seems like the core requires something to strip @INC down to a small set, not add to it. So blib.pm as it currently works isn't really the right tool I guess. Nick was suggesting blib.pm could be changed, but it sounds like you'd rather create a different TestInit tool. Anyway, what I meant was that @INC needs to be one value to find blib.pm, and then blib.pm changes @INC to be another value, so there's no chicken/egg problem. > And if we're not using blib to setup > @INC there's no point in shoving extra functionality unrelated > to finding > modules into it. Even more importantly, there may not be room in the blib.pm interface for other stuff. >> I have to wonder why it's preferable to put code on command >> lines rather than in modules. > > As mentioned below, it's not. That's why it's already in > TestInit.pm (are > you sensing a theme here?) I was just spelling out the > equivalent command to show blib wasn't buying us anything. Ah, I see. >>> and perhaps reduced a little further by moving/linking >>> TestInit.pm into the >>> top level directory. >>> >>>./perl -MTestInit t/path/to/test.t >>> >>> but that will cause trouble when running tests with -T >>> (because . is no >>> longer in the path). >> >> '.' won't necessarily be in the path anyway. > > Sorry, I ment perl's @INC. That's what I meant too. People can build Perl without '.' in @INC, right? -Ken
Removing CPAN's dependency on Test::More
Hi, Twice recently, I've just been through the task of upgrading CPAN.pm from 1.59_54 to 1.63 on a solaris system because something about MD5 was broken enough that no modules could be installed. It took a long time to do manually, because of the dependency on Test::More, which depends on Test::Harness, and because I didn't immediately know to look for Test::More in Test::Simple. So why does CPAN.pm depend on Test::More anyway? It only uses it during its 'make test' phase, not during runtime operations. And when you can't install modules very easily because you don't have CPAN.pm working properly, it's, well, not very easy to install other modules. ;-) Seems like one of the following would be better: 1) Include a copy of Test::More and Test::Harness in a t/lib/ directory and use that 2) Rework the two Test::More-using tests (mirroredby.t and Nox.t) to just use Test.pm Or is this a sneaky way to get more people to install Test::More on their machines? -Ken
