Re: Frequency of test.pl
demerphq wrote: > On 11/1/06, Michael G Schwern <[EMAIL PROTECTED]> wrote: >> This distribution uses 'test.pl' for its tests. The output of >> 'test.pl' is not parsed. When >> "make test" is run by automated installers (for example, the CPAN > shell) your tests will >> always appear to have passed no matter what the output of 'test.pl'. > > Is that true when test.pl dies? Maybe it should say that if you must > use test.pl it should die on test failure. Technically that is correct. However, 99% of the time folks are using test.pl because that's what h2xs gave them and not because they prefer it over t/*.t. Its not worth mentioning workarounds in a succinct explanation, there's little reason reason to put tests in test.pl.
Re: Frequency of test.pl
Michael G Schwern schrieb: This doesn't really explain why test.pl is bad. [...] Agreed! This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not parsed. When "make test" is run by automated installers (for example, the CPAN shell) your tests will always appear to have passed no matter what the output of 'test.pl'. Please rename 'test.pl' to a '*.t' file in the t/ directory (for example, t/basic.t). All of t/*.t will be run, and parsed, by "make test". Much better! Steffen
Re: Frequency of test.pl
On 11/1/06, Michael G Schwern <[EMAIL PROTECTED]> wrote: This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not parsed. When "make test" is run by automated installers (for example, the CPAN shell) your tests will always appear to have passed no matter what the output of 'test.pl'. Is that true when test.pl dies? Maybe it should say that if you must use test.pl it should die on test failure. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
Re: Integrating Test::Run into an ExtUtils::MakeMaker+Test::Manifest Setup
Shlomi Fish wrote:
> Hi all!
>
> See http://xrl.us/sw5o for a recipe for integrating "make runtest" and "make
> distruntest" targets into a Makefile.PL-generated Makefile that makes use of
> Test::Manifest.
>
> I needed that when working on XML::RSS, so I wrote it.
What has me scratching my head here is why you're hacking in a new target for
Test::Run rather than just overriding the test target using the much cleaner
test_via_harness(). Either the author wants to use Test::Run or they don't.
If the intention is for your own personal use while hacking other's modules, it
seems much easier to simply have a prove command-line equivalent for running
tests with Test::Run and Test::Manifest then to have to hack up each author's
Makefile.PL.
As for the implementation, here's some notes to avoid MakeMaker land mines:
Your "perl" is hard coded. You should be using $(PERLRUNINST) so that the
tests run with the same perl used to run Makefile.PL.
You probably want to stash the bulk of the command line code off in a function
somewhere like ExtUtils::Command::MM::test_harness() and
Test::Manifest::run_t_manifest(). Long command lines mean portability problems.
This is not how you override methods in MakeMaker. Its not backwards
compatible and it pokes at the guts which are likely to move.
sub ExtUtils::MM_Any::test_via_harness {
Instead, do this:
{
package MY;
sub test_via_harness {
...
}
}
Re: Frequency of test.pl
Hi,
Michael G Schwern schrieb:
So making "no_test_pl" a kwalitee test is a good idea.
So adding this to Module::CPANTS::Kwalitee::Files in the obvious place
should do exactly that. Perhaps we should call it "old_testing_scheme"
or so instead, but I'll go with your suggestion:
{
name=>'no_test_pl',
error=>q{This distribution contains a file called 'test.pl' in
its main distribution directory. This indicates that it uses the old
style of creating test suites. Nowadays, tests belong into one or more
*.t files in the t/ subdirectory.},
remedy=>q{Refactor the monolithic test.pl script into one or
more t/*.t test scripts.},
code=>sub {
my $d=shift;
return 0 if $d->{file_test_pl};
return 1;
},
},
HTH,
Steffen
Re: Frequency of test.pl
Hi again, Michael G Schwern schrieb: So making "no_test_pl" a kwalitee test is a good idea. of course, adding a mention in the docs helps, too. So I should have made it a patch. Sorry. I didn't have the svn sources. Hence: =item * has_test_pl And if people decide that this should be optional, add an "is_extra => 1," into the hash ref of the last mail.
Re: CPAN code searches
On Nov 1, 2006, at 1:38 PM, Michael G Schwern wrote: A few people have asked how I do my CPAN scans. Code search engine Krugle will be at the Hackathon on Nov 11 to talk to us about code searching. See http://rakudo.org/hackathon-chicago/index.cgi?krugle_presentation Maybe someone who is coming to the Hackathon would like to present ideas for the non-present from perl-qa? xoa -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:petdance
Re: Frequency of test.pl
Steffen Mueller wrote:
> {
> name=>'no_test_pl',
> error=>q{This distribution contains a file called 'test.pl' in
> its main distribution directory. This indicates that it uses the old
> style of creating test suites. Nowadays, tests belong into one or more
> *.t files in the t/ subdirectory.},
This doesn't really explain why test.pl is bad. It says "its old" but there's
nothing wrong with old, if it works. The real problem is that test.pl is
unparsed by "make test" and thus cannot be checked by automated testing. Try
something like...
This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not
parsed. When "make test" is run by automated installers (for example, the CPAN
shell) your tests will always appear to have passed no matter what the output
of 'test.pl'. Please rename 'test.pl' to a '*.t' file in the t/ directory (for
example, t/basic.t). All of t/*.t will be run, and parsed, by "make test".
Integrating Test::Run into an ExtUtils::MakeMaker+Test::Manifest Setup
Hi all! See http://xrl.us/sw5o for a recipe for integrating "make runtest" and "make distruntest" targets into a Makefile.PL-generated Makefile that makes use of Test::Manifest. I needed that when working on XML::RSS, so I wrote it. Enjoy! Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ Chuck Norris wrote a complete Perl 6 implementation in a day but then destroyed all evidence with his bare hands, so no one will know his secrets.
CPAN code searches
A few people have asked how I do my CPAN scans. I keep a minicpan handy and have a little script called "grep_cpan" which reads the index file, unpacks each distribution, does a "find | grep" and then cleans up. It allows you to search for a given pattern in a given set of files. # search for all uses of the word "flubber" in minicpan grep_cpan flubber # search for the LICENSE field in Makefile.PL grep_cpan LICENSE Makefile.PL I'll often replace the "find | grep" part with whatever custom logic I need, such as "does it have a test.pl". This exercise is left for the reader. http://schwern.org/src/grep_cpan Its slow. http://code.google.com is less flexible but much faster. I have a handy Firefox keyword to restrict it to Perl code. "http://www.google.com/codesearch?q=lang:perl %s"
Re: Test.pm does not return false on failed tests.
David Golden wrote: > On 11/1/06, Michael G Schwern <[EMAIL PROTECTED]> wrote: >> Contact all the authors making use of it and suggest they switch to .t >> files. Maybe I could throw a warning into MakeMaker. Also a kwalitee >> check sounds like a good idea "not_only_test.pl" or something. > > It shouldn't be "not_only". If there is t/*.t and test.pl, > ExtUtils::MakeMaker runs them separately, first passing t/*.t to > Test::Harness and then separately running test.pl. If the > Test::Harness run passes, then we're right back to the same problem as > if test.pl was just by itself. As pointed out earlier, some distributions use test.pl as a general "do some interesting stuff" program which are not necessarily tests. DBI, for example, uses it to do benchmarking. I don't believe this usage to be widespread at all. That can be taken as both an argument for making it a kwalitee test, "Its almost always a mistake to have both t/*.t and test.pl" or against "So few people actually do this, let's leave them be". I'm doing a CPAN scan now to see which distributions only have a test.pl and which have both.
Re: Test.pm does not return false on failed tests.
On 11/1/06, Michael G Schwern <[EMAIL PROTECTED]> wrote: Contact all the authors making use of it and suggest they switch to .t files. Maybe I could throw a warning into MakeMaker. Also a kwalitee check sounds like a good idea "not_only_test.pl" or something. It shouldn't be "not_only". If there is t/*.t and test.pl, ExtUtils::MakeMaker runs them separately, first passing t/*.t to Test::Harness and then separately running test.pl. If the Test::Harness run passes, then we're right back to the same problem as if test.pl was just by itself. David
Re: Test.pm does not return false on failed tests.
Jos Boumans wrote: > Hmm, I hope this doesn't mean you plan on removing the 'non-zero exit status > on failed make test'.. that's be a real PITA for automated installers... No, that is done by Test::Harness dying on test failure. >> Neither CPAN nor CPANPLUS are under any obligation to parse their output. >> The solution is to get authors to stop using test.pl. > > It does of course generate questions when the human sees 'make test > failed' on > his screen and CPAN/CPANPLUS say 'make test passed, installing'. Is there > anything > we could use to DTRT thing here, or is it time for Yet Another Disclaimer? Contact all the authors making use of it and suggest they switch to .t files. Maybe I could throw a warning into MakeMaker. Also a kwalitee check sounds like a good idea "not_only_test.pl" or something.
