Re: unit tests or functional tests

[Andrew Savige]

> There is this big hairball of under-tested code. (Nothing new here)
> So the question is, which to tackle first - unit tests, or functional tests.

Generally, I agree with the earlier responders endorsing your approach
and I've little to add to their thoughtful remarks.

Perhaps the major benefit of unit tests is that they improve design:

 * writing a test first forces you to focus on interface
 * hard to test code is often hard to use
 * simpler interfaces are easier to test
 * functions that are encapsulated and easy to test are easy to reuse
 * components that are easy to mock are more flexible/extensible

That is, unit tests provide more value for new development work.
If you already have a big ball-o-mud, unit tests provide less value.
The time to write unit tests for an existing system, is when you
choose to rewrite a component/subsystem.

This sort of question has been asked a few times at Perl monks.
See, for example:

 http://www.perlmonks.org/?node_id=355203
 http://www.perlmonks.org/?node_id=560614

> Why am I wrong?

You're not. :-)

Cheers,
/-\


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Time for a Revolution

[Andrew Savige]

--- Nicholas Clark wrote:
> On Sat, Jul 15, 2006 at 08:36:54AM +1000, Andrew Savige wrote:
> > Who's Chromatic?
> 
> And it wasn't even the start of a sentence. :-)
> 
> [When doing the perl 6 summaries, Piers reconciled the forces of accuracy and
> traditional grammar by ensuring by always rephrasing to some sentence order
> that didn't start with "chromatic"]

I thought Chromatic might be the name of chromatic's father or older brother.

/-\







 
On Yahoo!7 
Messenger - Make free PC-to-PC calls to your friends overseas. 
http://au.messenger.yahoo.com 

Re: Time for a Revolution

[Andrew Savige]

--- Clayton O'Neill wrote:
> I think a core difference between your list and Chromatic's is that
> yours would be part of the standard library in a lot of languages,
> whereas Chromatic seems to be aiming more for things that would be
> part of the language.

Who's Chromatic?

/-\







 
On Yahoo!7 
Messenger - Make free PC-to-PC calls to your friends overseas. 
http://au.messenger.yahoo.com 

Re: Continuous testing tools

[Andrew Savige]

--- Adam Kennedy wrote:
> I know it's somewhat vapour at the moment, and I'm keeping somewhat 
> quiet, but the new post-Audrey'fied PITA design is aiming at exactly 
> what you have described.

Thanks for the reminder about PITA. I'd (unforgivably) forgotten about
that project when I first enquired. Sorry 'bout that. ;-)

> Initial deployment targets include a pugs smoker, parrot smoker, and 
> CPAN Testers 2.
> 
> Of course, I have no idea how you projects are structured, but it may 
> well be that it would be appropriate.
> 
> Of course, it partly depends on how quickly you need it... if you are 
> in a hurry, then I'll get back to you in a few months.

I don't see an urgent need, though it's something we'd like to have.
I guess the main pressure is coming from the grass roots level where
a Java programmer might enquire "why don't we use CruiseControl?" or
a .NET guy might suggest something else or someone else just saw an
article on BuildBot and asks why don't we use that. I do find it a bit
disturbing that noone seems to ask about Perl-based solutions. ;-)

In the name of empowerment or anarchy, we could allow each group to
do their own thing and set up their own preferred tool themselves.
OTOH, we might try to build a single system that worked with Java,
.NET, C++, Perl, and so on. In terms of requirements, our Perl stuff
is standard TAP, while our C++ stuff has very home-grown non-standard
multi-platform builds and regression test suites.

Thanks,
/-\


Send instant messages to your online friends http://au.messenger.yahoo.com 

Continuous testing tools

[Andrew Savige]

We are looking at introducing continuous builds/smoke tests at
work across a number of platforms (mainly Windows and Unix),
building a number of different languages (mainly C++).

I quick google uncovered the list below.

Anyone got any advice?

Thanks,
/-\


Perl


* AutoBuild: http://www.autobuild.org/,
http://search.cpan.org/dist/Test-AutoBuild/

* http://search.cpan.org/dist/Test-Smoke/

* http://search.cpan.org/dist/Test-Nightly/


Other Open Source
-

* BuildBot (Python): http://buildbot.sourceforge.net/

* DamageControl (Ruby):
http://opensource.thoughtworks.com/projects/damagecontrol.jsp

* CruiseControl (Java): http://cruisecontrol.sourceforge.net/

* CruiseControl.NET:
http://confluence.public.thoughtworks.org/display/CCNET/Welcome+to+CruiseControl.NET

* Eclipse plug-in: http://www.eclipse.org/

* Tinderbox: http://www.mozilla.org/tinderbox.html

* BusyB: http://sourceforge.net/projects/busyb

Commercial
--

* ParaBuild: http://www.viewtier.com/products/parabuild/index.htm

* BuildForge: http://www.buildforge.com/


Send instant messages to your online friends http://au.messenger.yahoo.com 

RE: Is there an "integrated" test suite/module to test all standard modules of Perl itself?

[Andrew Savige]

--- "Fu, Elva" wrote:
> It seems there are really an ¡°Integrated¡± test suites existed to test Perl
> itself. Who could give me a hand to find it? Thanks in advance.

I thought chromatic already this question by pointing you at the t/
directories in the Perl source code distribution.

To clarify, when you build Perl from source code (as described in the
INSTALL file in the Perl source distribution) with:

 sh Configure -de
 make
 make test

"make test" automatically runs all the tests for the Perl core and all
Perl core modules.

I've never tried running the test suite against an already built perl.
I imagine you could do that by untar'ing the Perl source distribution
and running t/TEST and/or t/harness. See the file t/README in the Perl
source distribution for more details. See also the standard
Test::Harness "prove" command:

 http://search.cpan.org/dist/Test-Harness/bin/prove

which is a generally handy way of running test suites outside of
"make test".

HTH,
/-\


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: [OT] TDD only works for simple things...

[Andrew Savige]

--- Tels wrote:
> although I still can only guess what TDD stands for :)

Tolkien Driven Development?

Googling around for examples of real world large systems developed
using TDD, I found http://www.agiledata.org/essays/tdd.html which
states:

 The first reaction that many people have to agile techniques is
 that they're ok for small projects, perhaps involving a handful
 of people for several months, but that they wouldn't work for
 "real" projects that are much larger. That’s simply not true.
 Beck (2003) reports working on a Smalltalk system taking a
 completely test-driven approach which took 4 years and 40
 person years of effort, resulting in 250,000 lines of
 functional code and 250,000 lines of test code. There are
 4000 tests running in under 20 minutes, with the full suite
 being run several times a day. Although there are larger
 systems out there, I've personally worked on systems where
 several hundred person years of effort were involved, it is
 clear that TDD works for good-sized systems. 

Apart from that I didn't find much. :-(

Oh, apart from this article describing TDD:

 http://msdn2.microsoft.com/en-us/library/ms182521.aspx

which got pulled by Microsoft after polling 1.44 out of 9
from 576 votes. ;-)

/-\







 
On Yahoo!7 
Messenger - Make free PC-to-PC calls to your friends overseas. 
http://au.messenger.yahoo.com 

Re: New kwalitee metric - eg/ directory

[Andrew Savige]

--- Tyler MacDonald wrote:
> Well so far the only ones I've seen are "eg", "examples", and from that
> renegade GD::Graph, "samples".

And from that eccentric Acme::Bleach, "demo".

/-\
 



 
On yahoo!7 
Avatars:  Dress up like your "Dancing with the Stars" favourites! 
http://au.avatars.yahoo.com 

Re: How to best have statements execute only during dev/testing?

[Andrew Savige]

--- "A. Pagaltzis" wrote:
> Additive filters that the same code can run without are
> sane when used carefully, and they're easy to create if the
> trigger is a special comment or better yet POD section:

Damian's Smart::Comments module filters specially formatted
comments and can do assertions and quite a lot more.

>From its doco:

  "This module is probably even relatively safe with other
  Filter::Simple modules since it is very specific and limited
  in what it filters"

Also, as noted at:

  http://www.oreillynet.com/pub/wlg/8894

Perl 5.10 will have assertions.

/-\




 
Do you Yahoo!? 
Check out gigs in your area on the comprehensive Yahoo! Music Gig Guide 
http://au.music.yahoo.com/gig-guide

Re: CPANTS new

[Andrew Savige]

--- David Landgren wrote:
> Seriously though, I have a module whose test suite includes Test::Pod 
> and Test::Pod::Coverage, except that I use the following construct:
> 
> SKIP: {
>  skip( 'Test::Pod not installed on this system', 1 )
>  unless do {
>  eval qq{ use Test::Pod };
>  $@ ? 0 : 1;
>  };
> 
>  pod_file_ok( 'foobar.pm' );
> }
> 
> The cpants analysis fails to recognise this as valid. What is it looking 
> for and/or could it be taught to look for this? I thought that it was 
> only looking for a string eval of "use Test::Pod".

I based mine on the Test::Pod::Coverage docs:

 use Test::More;
 eval "use Test::Pod::Coverage 1.00";
 plan skip_all => "Test::Pod::Coverage 1.00 required for testing POD coverage"
if $@;
 all_pod_coverage_ok();

and scored the coverage kwalitee point... Now, if I can just get cog to
pre-req my module (he teased me by saying he would ;-).

BTW, Module::Starter auto-generates a similar pod-coverage.t.

/-\


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Why are we adding more kwalitee tests?

[Andrew Savige]

--- Thomas Klausner wrote:
> *) CPANTS is not describing the one and only way how to write Perl / pack
> distribution. It's more of an online mutliplayer game where people submit
> their dists which than fight against my evil metrics.

The multiplayer game you describe is subject to the testing
phenomenon known as the "pesticide paradox", described here:

http://www.ece.cmu.edu/~koopman/des_s99/sw_testing/

Which means to keep finding new bugs in distributions you must
continuously find new kwalitee tests to break distributions
that have built up immunity to your earlier tests. :-)

/-\







 
Do you Yahoo!? 
The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full 
Actor Database. 
http://au.movies.yahoo.com

Re: Why are we adding more kwalitee tests?

[Andrew Savige]

--- Andy Lester wrote:
> But will the author actually care?  Will the author even know this
> exists?  Are you going to send email to Bob and say "Hey, Bob, you only
> passed 7 of 23 things"?  What's Bob going to say in return?  I see a
> couple of options:

Ah, now I see where you are coming from. I would say we should definitely
not spam CPAN authors with kwalitee metric emails; that is guaranteed to
really annoy a lot of them.

I feel pushily promoting kwalitee metrics will do more harm than good.
And I agree with chromatic that officially endorsing any particular
kwalitee metric is a mistake that is likely to cause unproductive
flame wars. Best, at least for now, is to simply publish some
kwalitee metrics as an optional aid to enthusiastic CPAN authors.
If they prove good and useful, they will naturally become better known.

/-\







 
Do you Yahoo!? 
The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full 
Actor Database. 
http://au.movies.yahoo.com

Re: Why are we adding more kwalitee tests?

[Andrew Savige]

--- Andy Lester <[EMAIL PROTECTED]> wrote:
> Why are we worrying about these automated kwalitee tests?  What will 
> happen once we find that DBIx::Wango has only passed 7 of these 23 
> items on the checklist?

I am not the one to answer this, but I'm curious to know where you are
coming from. Is it:

1) Metrics, in general, are often misused and do more harm than good.

2) The Module-CPANTS kwalitee metrics are poorly chosen.

3) The Module-CPANTS kwalitee metrics are an unwelcome distraction
   to CPAN authors who should be focusing on improving their module
   quality, not on improving their Module-CPANTS kwalitee score.

4) Schwern's original kwalitee vision in
 http://www.nntp.perl.org/group/perl.qa/149
   is not worth worrying about.

Or all of the above?

I have sympathy for 1) above because I've seen first hand the dysfunctional
behaviour that can flow from people focusing on improving their metrics
at the expense of doing a good job. For example, a support group whose
performance was measured by the number of calls they closed per month,
so they started to close customer calls and reopen the same issue with
a new call (which annoyed the customer, but, hey, they improved their
"performance"). Many other examples could be given. See also Brian Marick's
"How to misuse Code Coverage" paper.

To clarify point 2) above do you equally feel we shouldn't worry about
the CPAN top 100 code coverage metrics published by Paul Johnson?

Personally, I see the publication of CPAN code coverage and kwalitee
metrics as useful data for CPAN authors to help them improve their
offerings and catch blunders. Of course, they shouldn't become a
distraction from the primary task of producing a "quality" module,
i.e. one with great user interface, doco, performance, robustness,
maintainability, portability, error handling, extensibility, ...

/-\







 
Do you Yahoo!? 
The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full 
Actor Database. 
http://au.movies.yahoo.com

Re: Adding more kwalitee tests

[Andrew Savige]

--- Thomas Klausner wrote:
> --- Adam Kennedy wrote:
>> has_perl_dependency:
>> 
>> In the META.yml (assuming it exists) there is a dependency on the 
>> version of perl required to install the dist.
>> 
>> The goal of this is to make life a little easier on installers and CPAN 
>> testers and a few other things.
>> 
>> Many many modules should pass this test already.
>> 
>> The main goal however, is to help enable the NEXT kwalitee test to have 
>> better coverage.
> 
> Hmm. I never added this of any to my dists.

I think you should. ;-)

Apart from a user wanting to know which version of perl a module
works with, it's a general "quality" (not kwalitee;-) principle to
first specify, then test that the specification is met. So I would
say that specifying which versions of perl your module is meant to
work with should be done before testing it against those versions.

In addition to META.yml, your module's version requirements should be
enforced programmatically. Well, Damian says so. :-) (p.405 of Perl
Best Practices: "Enforce your version requirements programmatically").
Another possible kwalitee check: the perl version in META.yml should
match the version enforced programmatically.

BTW, is there any plan to mine PBP for possible new kwalitee tests?

/-\


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Testing Net-SSLeay

[Andrew Savige]

--- Andy Lester wrote:
> I'd throw my hands up and let it go, then.  One of the key functions of
> Phalanx is to modernize the testing infrastructure of the modules we
> touch.  If he needs it to stay compatible back to the relative dark
> ages, then let's just leave it that way.

Though many modules must remain compatible way back, their Phalanx test
suites do not, their primary goal being to test against Ponie.

How about the perhaps silly or wild idea of decoupling a module's test
suite from the module itself? You could even have multiple test suites
for the same module. What triggered this idea is the recent impressive
work from Autrijus (Pugs) and Ingy (FreePAN) using a model with committer
rights handed out liberally, wiki-style. I could imagine a Phalanx svn
respository with all Phalanxers having commit rights, and a free-for-all,
where any Phalanxer could write any test for any Phalanx module, always
using modern test tools. Low barrier of entry, low communication costs,
lots of test duplication perhaps -- but in the testing world, variety
and duplication can be your friend.

I'm sorry but I don't have the skill or time to provide much practical
assistance for such a project but thought I'd throw it out there anyway.

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

How to fix "command line too long" with "make test" (when you have vast numbers of tests)

[Andrew Savige]

As described here:

http://www.nntp.perl.org/group/perl.perl6.compiler/413

'nmake test' on the latest release of Pugs blew up under Windows with
"command line too long" (using ActiveState perl-5.8.6).

Is there a "standard" way to fix this? What do other distributions with
vast numbers of tests do? BTW, "prove -r" does not have this command
line limit because it builds @tests and invokes T::H runtests @tests.

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: Testing .pl executables

[Andrew Savige]

--- Anuradha Dissanayake <[EMAIL PROTECTED]> wrote:
> I'm new to unit testing in Perl, so this may be a stupid question.
> 
> Using Test::More to test .pm perl modules is quite simple as you just
> "use" the module in your test executable and call the module's
> subroutines inside your tests.
> 
> However, I was wondering if there is a standard, safe way of testing
> simple .pl scripts?

Hmmm. The identical question was asked a bit over a week ago:

http://www.nntp.perl.org/group/perl.qa/3722

I suggest you start by browsing the answers given there.

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: Kwalitee mascot

[Andrew Savige]

--- Michael G Schwern wrote: 
> I hereby propose the Koaladile (Kah-wah-lah-dile) for perl-qa mascot!
> 
>
http://i.somethingawful.com/inserts/articlepics/photoshop/12-10-04-animals/AirbagSML.jpg
> 
> "Koaladile is for Kwalitee".  (Kah-wah-lah-dile is for Kah-wal-i-tee)
> 
> From Something Awful's "Animal Anomalies 2"
> http://www.somethingawful.com/articles.php?a=2539

"Kah-wah-lah-dile is for Kah-wal-i-tee" ... Yes, I agree Schwern, that's
an excellent perl-qa slogan. I just created an ascii art koaladile, for
use in Perl qa programs:

http://search.cpan.org/src/ASAVIGE/Acme-EyeDrops-1.45/lib/Acme/EyeDrops/koaladile.eye

Though I still prefer this one:

http://search.cpan.org/src/ASAVIGE/Acme-EyeDrops-1.45/lib/Acme/EyeDrops/schwern.eye

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: [ANNOUNCE] Test::Simple/More/Builder 0.54 (problem with signa ture?)

[Andrew Savige]

Steve Hay wrote:
> And this program (500,000 small extensions to a string):
>
> my $a = '';
> my $start = time;
> for my $i (1 .. 50) {
>   print "$i\n" if $i % 1000 == 0;
>   $a .= '.' x 20;
> }
> printf "OK (%d seconds)\n", time - $start;
>
> is even worse: 1 second again on 5.8.6/perl-malloc versus 56
> seconds on 5.8.4/system-malloc!

I'm pretty sure realloc() is the culprit here.
A common trick used by string classes is to double the memory
allocation whenever you need to grow a string, and so avoid
O(n-squared) performance when growing a string one char at a time.

For cheap thrills, I created the following patch against 5.8.6
to hack this memory doubling trick into Perl_sv_catpvn_flags():

--- sv-orig.c   2004-11-02 03:01:54.0 +1100
+++ sv.c2004-12-16 15:04:39.0 +1100
@@ -4394,9 +4394,14 @@
 {
 STRLEN dlen;
 char *dstr;
+STRLEN neededlen;
 
 dstr = SvPV_force_flags(dsv, dlen, flags);
-SvGROW(dsv, dlen + slen + 1);
+neededlen = dlen + slen + 1;
+if (SvLEN(dsv) < neededlen) {
+STRLEN s2 = SvLEN(dsv) * 2;
+SvGROW(dsv, s2 < neededlen ? neededlen : s2);
+}
 if (sstr == dstr)
sstr = SvPVX(dsv);
 Move(sstr, SvPVX(dsv) + dlen, slen, char);

Though all tests passed, I don't know Perl internals well enough to
judge whether this hacky change is sound or not. I did it only out
of curiosity to see how it affected Steve's little benchmark.

After applying this patch, the performance of Steve's test program
above dropped from 26 seconds to 1 second when run on Windows XP
(when built with default system malloc). BTW, on Linux, the
improvement was hardly noticeable: from 0.86 secs to 0.51.

I'm interested to hear opinions on whether these sort of memory
heuristics are best done in the perl core or left to realloc().

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

RE: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- "Clayton, Nik" wrote: 
> Any "Writing thread safe libraries for dummies" texts you could point
> me at?

I recommend "Programming with POSIX Threads" by David Butenhof.

Re the varargs ok() business, I assume you'll be using some sort of
config.h with your libtap library. Any plans on using autoconf or
similar tool?

One way around this __VA_ARGS__ portability issue is to let
configure work it out and write your code for some sort of
"VA_ARGS" capability. There doesn't appear to be a standard
autoconf symbol for this, at least I couldn't find one.
Googling for HAVE_VA_ARGS uncovered only two dubious hits.

asked a question about this issue, but no response.

I noticed this in a "issue with glib.h" gtk-devel-list thread:

I think we should just use the __STDC_VERSION__ define -- no need
for autoconf.

#if defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L
# define g_message(...) g_log (DOM, LOG_MSG, __VA_ARGS__)
#elif defined __GNUC__
# define g_message(format_args...) g_log (DOM, LOG_MSG, format_args)
#else
...
#endif

Finally, ACE C++ library uses:

#if defined (__GNUC__) && (__GNUC__ >= 3 || __GNUC_MINOR__ > 95)
  // use GNU __VA_ARGS__ capability ...

HTH,
/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- /-\ wrote: 
> In an attempt to do varargs ok() without the magical __VA_ARGS__,
> I've come up with two little example solutions shown below.
> Improvements welcome.

Sorry, but curiosity got the better of me and I took a look at how
the very widely used and portable C++ ACE library does it.
The version below is very loosely based on the ACE library
ACE_ERROR() macro.

Note the funky (( and )) macro hackery in the ok's below. This is
almost as much fun as Perl golf. :-)

AFAICT, in the ACE version, instead of gr_save() below it calls a
setter method on a singleton "logger" object and instead of gr()
it calls the log method of the said singleton logger object.

/* Version very loosely based on ACE C++ library ACE_ERROR() - */
#include 
#include 

#define ok(X)  \
  do { \
  gr_save(__FILE__, __LINE__); \
  gr X; \
  } while (0)

static const char* g_file;
static unsigned long g_line;

void gr_save(const char* file, unsigned long line) {
g_file = file;
g_line = line;
}

void gr(int cond, const char* fmt, ...) {
va_list t;
printf("cond:%d file:%s line:%lu, ", cond, g_file, g_line);
va_start(t, fmt);
vprintf(fmt, t);
va_end(t);
}

int main() {
ok(( 1==1, "test %s %d\n", "one eq one",  42 ));
ok(( 0==1, "test %s %d\n", "zero eq one", 43 ));
return 0;
}

/* --- */

/-\



Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- muppet wrote: 
> writing your own printf-style macros is actually a very common idiom 
> for getting around this sort of technical problem.  they, of course, 
> raise the further problem that GCC's vararg macros are not portable, 
> and C99's vararg macros are not yet universally supported.

Agreed. I missed that __VA_ARGS__ earlier; that won't work on many
current C compilers.

In an attempt to do varargs ok() without the magical __VA_ARGS__,
I've come up with two little example solutions shown below.
Improvements welcome.

/* Sound version but with ugly XX symbol - */

#include 
#include 

#define XX  __FILE__,__LINE__

void ok(int cond, const char* file, unsigned long line, const char* fmt, ...) {
va_list t;
printf("cond:%d file:%s line:%lu, ", cond, file, line);
va_start(t, fmt);
vprintf(fmt, t);
va_end(t);
}

int main() {
ok(1==1, XX, "test %s %d\n", "one eq one",  42);
ok(0==1, XX, "test %s %d\n", "zero eq one", 43);
return 0;
}

/* Horrible hacky thread-unsafe version but no XX  */

#include 
#include 

#define ok  ok_save(__FILE__, __LINE__)
typedef void (*ok_t)(int, const char*, ...);

static const char* g_file;
static unsigned long g_line;

void gr(int cond, const char* fmt, ...) {
va_list t;
printf("cond:%d file:%s line:%lu, ", cond, g_file, g_line);
va_start(t, fmt);
vprintf(fmt, t);
va_end(t);
}
ok_t ok_save(const char* file, unsigned long line) {
g_file = file;
g_line = line;
return gr;
}

int main() {
ok(1==1, "test %s %d\n", "one eq one",  42);
ok(0==1, "test %s %d\n", "zero eq one", 43);
return 0;
}

/* --- */

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

RE: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- "Clayton, Nik" wrote: 
> Andrew Savige wrote:
> > 2) A uniform mechanism for test programs to handle command line
> > arguments would be nice. For example:
> > 
> > int main(int argc, char* argv[])
> > {
> > tap_init(argc, argv);  /* mythical new tap function */ // ...
> > }
> > 
> > Some possible command line arguments are:
> >   -v verbose
> >   -d debug
> >   -t give timing information (via ANSI C clock() function, say)
> 
> Isn't that more the province of the harness that's running the tests?
> Like prove(1)?

Yeah, you're right. Sorry, I wasn't up-to-date with the latest
petdance goings on when I first responded. (Googling for TAP
protocol didn't help much, but I finally uncovered this:
<http://use.perl.org/~petdance/journal/22057>).

That said, when writing my xUnit-style C++ unit test suite, I found
it handy to provide hooks to tweak the behaviours of individual test
programs outside of the driver. In particular, I wanted to provide
timing information at a finer level of granularity than provided by
the driver (e.g. prove).

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- Michael G Schwern <[EMAIL PROTECTED]> wrote: 
> Yucko.
> 
> Test::More implements cmp_ok() using an eval.  Could a macro prove useful
> here to do something similar?
> 
> cmp_ok(foo, 'int', '==', bar);

Good idea Schwern.
These test suites inevitably degenerate into macro crack-pipe
smoking sessions. Lacking 'eval', it is always going to be very
hard work to even get close to Perl's Test::More functionality
and ease-of-use in C.

I don't have a filthy enough mind to make your:
  cmp_ok(foo, 'int', '==', bar);
macro work (mainly because '==' is not a valid C identifier).
But I can make this one work:
  cmp_ok(foo, int, eq, bar);
pretty easily:

#include 
void cmp_ok_inteq(int s, int t) {
printf("in cmp_ok_inteq: %d %d\n", s, t);
}
void cmp_ok_streq(const char* s, const char* t) {
printf("in cmp_ok_streq: '%s' '%s'\n", s, t);
}
#define cmp_ok(s, y, o, t)  cmp_ok_##y##o(s, t)
int main() {
int ifoo = 42;
int ibar = 43;
static const char* sfoo = "foo";
static const char* sbar = "bar";
cmp_ok(ifoo, int, eq, ibar);
cmp_ok(sfoo, str, eq, sbar);
return 0;
}

/-\



Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- "Clayton, Nik" <[EMAIL PROTECTED]> wrote: 
> Having done the initial work to get most of FreeBSD's regression testing
> infrastructure producing Test::Harness TAP compatible output, I've started
> putting together a C library that makes it easier to write tests in C.

Great!
This is something I'm interested in and wish I had more spare time
to pursue. Here are a few random ideas.

1) You might be able to steal some implementation ideas from:

http://cutest.sourceforge.net/

This is a very simple C Unit Testing framework.
In particular, it keeps state in a CuTest struct which is passed
around as the first (mostly hidden) parameter to most functions.

2) A uniform mechanism for test programs to handle command line
arguments would be nice. For example:

int main(int argc, char* argv[])
{
tap_init(argc, argv);  /* mythical new tap function */
// ...
}

Some possible command line arguments are:
  -v verbose
  -d debug
  -t give timing information (via ANSI C clock() function, say)

3) Implementing cmp_ok() in C is a challenge. ;-)

xUnit/cutest have things like:

AssertStrEquals
AssertIntEquals
...

The trouble with plain old ok() is that investigating test failures
is a pest. I suppose you could support equals for the basic C types.
For example:

cmp_ok_int_equals
cmp_ok_double_equals
cmp_ok_cstr_equals
cmp_ok_mem_equals
...

and so on. In the cstr/mem case, you can have the function check
for NULL pointers and give a nice diagnostic rather than crashing.

4) Is thread-safety a requirement of your library?

If so, you better think about it now because it's very hard to
retrofit.

HTH,
/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- Michael G Schwern <[EMAIL PROTECTED]> wrote: 
> On Mon, Dec 06, 2004 at 02:25:42PM -0800, Andrew Savige wrote:
> > --- Michael G Schwern <[EMAIL PROTECTED]> wrote:
> > > Why add that extra auto-sprintf complexity?  Can't the user do the exact
> > > same thing with:
> > > 
> > >   ok(some_func(i), sprintf("some_func(%d)", i));
> > 
> > No. sprintf in C needs a buffer and you don't know how big to make it.
> 
> If the user doesn't know how will ok() know how big to make it?

By the magic of the ANSI C macros: va_start(), va_arg(), va_end().
These macros are provided by the C runtime library and typically require
some assembler to implement. In Nik's _gen_result() function (tap.c) he
employs these macros in harness with vprintf(): in this way, he does not
need to allocate a buffer at all, nor does he need worry about possible
buffer overruns. FWIW, I agree with this implemenatation decision, and
feel it will be well received by C programmers ... unless anyone can
suggest a better alternative, of course.

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C implementation of Test::Harness' TAP protocol

[Andrew Savige]

--- Michael G Schwern <[EMAIL PROTECTED]> wrote:
> Why add that extra auto-sprintf complexity?  Can't the user do the exact
> same thing with:
> 
>   ok(some_func(i), sprintf("some_func(%d)", i));

No. sprintf in C needs a buffer and you don't know how big to make it.

> > ok2() is for situations where the code to test is sufficiently 
> > self-documenting that you don't need to provide a test name.
> > 
> > ok2(code to test); /* test name is automatically the same as the code
> */
> > 
> > E.g.,
> > 
> > ok(1 == 1, "1 equals one"); /* PRINT: ok 1 - 1 equals 1 */
> > ok2(1 == 2);/* PRINT: not ok 2 - 1 == 2 */
> 
> Why the split?  You can do variable length argument lists in C.

Not with the pre-processor. And you need the pre-processor for
__LINE__ and __FILE__.

An alternative is to drop the ugly ok2() and force the test writer
to use:

  ok(1 == 2, "");

I prefer that since I want to make it hard for people to avoid giving
a test a name, er, comment.

> > Finally, there's exit_status(), which returns an int suitable for use
> > when return'ing from main(), or calling exit().  You should always do one
> > of:
> > 
> > return exit_status();
> > exit(exit_status());
> 
> What is this for?  <--- possible C ignorance
> 
> I hope you're not emulating Test::More's exit code == # of tests failed
> "feature" that I'm planning on getting rid of.

I tried returning the number of failed tests in the exit status in
a C++ regression test suite a while back and dropped it because:
i) you are limited to 0-255 for this exit status;
ii) you can always tell the number of failed tests by parsing stdout;
iii) it's nice to cleanly detect a program crash; if it don't
return zero, it crashed.

/-\


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Test::Simple 0.51 prerelease

[Andrew Savige]

--- Michael G Schwern wrote: 
> Excuse me for a moment.
> 
> I AM SO SICK OF THREADING BUGS!!!  BLARHGAGHAGHAHGAH!  I don't even USE
> THREADS and I seem to find all the bugs!!  ARGH!!!
> 
> *ahem*

To try and cheer you up a bit, I'm delighted to report that your new
Test-Simple-0.51 passed all tests on Windows XP under Perl 5.8.5
using NMAKE.

C:\balls\Test-Simple-0.51>nmake test

Microsoft (R) Program Maintenance Utility Version 7.10.3077
Copyright (C) Microsoft Corporation.  All rights reserved.

C:\perl585\bin\perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib\lib', 'blib\ar
ch')" t\00signature.t t\00test_harness_check.t t\bad_plan.t t\buffer.t
t\Builder.t t\curr_test.t t\d
etails.t t\diag.t t\eq_set.t t\exit.t t\extra.t t\extra_one.t t\fail-like.t
t\fail-more.t t\fail.t t
\fail_one.t t\filehandles.t t\fork.t t\harness_active.t t\has_plan.t
t\has_plan2.t t\import.t t\is_d
eeply.t t\maybe_regex.t t\missing.t t\More.t t\no_diag.t t\no_ending.t
t\no_header.t t\no_plan.t t\o
k_obj.t t\output.t t\overload.t t\plan.t t\plan_is_noplan.t t\plan_no_plan.t
t\plan_skip_all.t t\res
et.t t\simple.t t\skip.t t\skipall.t t\thread_taint.t t\threads.t t\todo.t
t\undef.t t\use_ok.t t\us
eing.t
t\00signature.ok
1/1 skipped: Next time around, consider install Module::Signature, so
you can verify the int
egrity of this distribution.
t\00test_harness_checkok
t\bad_planok
t\buffer..ok
t\Builder.ok
t\curr_test...ok
t\details.ok
t\diagok
t\eq_set..ok
t\exitok
t\extra...ok
t\extra_one...ok
t\fail-like...ok
t\fail-more...ok
t\failok
t\fail_oneok
t\filehandles.ok
t\forkskipped
all skipped: This system cannot fork
t\harness_active..ok
t\has_planok
t\has_plan2...ok
t\import..ok
t\is_deeply...ok
t\maybe_regex.ok
t\missing.ok
t\Moreok
t\no_diag.ok
t\no_ending...ok
t\no_header...ok
t\no_plan.ok
t\ok_obj..ok
t\output..ok
t\overloadok
1/5 skipped: not really skipped, testing overloaded reason
t\planok
t\plan_is_noplan..ok
t\plan_no_planok
t\plan_skip_all...skipped
all skipped: Just testing plan & skip_all
t\reset...ok
t\simple..ok
t\skipok
7/15 skipped: Just testing the skip interface.
t\skipall.ok
t\thread_taintok
t\threads.ok
t\todook
t\undef...ok
t\use_ok..ok
t\useing..ok
All tests successful, 2 tests and 9 subtests skipped.
Files=47, Tests=298,  5 wallclock secs ( 0.00 cusr +  0.00 csys =  0.00 CPU)

Oh, and thankyou for working around sort threading bug #30333.

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Test::Harness/prove: printing the test name when a test fails

[Andrew Savige]

I am currently trying to sell Test::More/prove and TDD at work.
I received a complaint from a newbie workmate today.
He complained that when a test, for example:
  ok( 0 == 1,  "This is my test name" );
fails, Test::Harness (and the prove command) by default do not
print the test name ("This is my test name" above).

I told him to use verbose mode (prove -v) but he still complained.
Actually, I agree with him that when a test fails (even when not
in verbose mode) it makes sense to print out as much useful
infomation as possible (including the test name).
Is there a way to make Test::Harness do this?
Any chance of getting this behaviour changed?

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Aegis "NO RESULT" versus Test::More skip

[Andrew Savige]

Does Test::More have an equivalent concept to Aegis "NO RESULT"?
Is Aegis "NO RESULT" equivalent to Test::More skip?

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: C/C++ White-Box Unit Testing and Test::More

[Andrew Savige]

Ovid wrote:
> For white box testing C code, I just use assert().

assert() is ok, but ok() is better. :-)
I will prolly roll my own custom ok() macro, so instead of:
  assert(x==y);
I can write:
  ok(x==y, "test that x equals y");
Writing a lot of tests, I want to be able to easily label each test.
(I think I will need an ok() macro rather than a function to get
__FILE__ and __LINE__ also for failing tests).

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

C/C++ White-Box Unit Testing and Test::More

[Andrew Savige]

I am currently using Test::More for my Perl white-box unit tests.
I also need to write some C/C++ white-box unit tests and would like
to use something similar in spirit to Test::More.

Smalltalk's SUnit-style framework has been ported to many languages
(JUnit, cppunit, Test::Unit, Test::Class, ...).
Has anyone ported a Test::More style framework to C/C++ or Java?
How do Perl5/Parrot white-box unit test their C code?

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Mapping test cases to bug databases

[Andrew Savige]

Suppose I fix a bug with a unique bug ID in a bug tracking system.
I start by dutifully adding 15 new asserts, say, to an existing unit
test program, to duplicate the bug before I fix it. What if I later
want some way to map the bug ID back to the these 15 new asserts?
Should I somehow assign unique IDs to my unit tests -- if so, at what
level of granularity? If I keep some sort of external test case
database, I'm worried about the overhead of keeping my (typically
fairly volatile) unit test programs in sync with the database.

Does the Perl core deal with this somehow? That is, suppose you have
an RT ticket, is there a way to find out which of Perl's 80,164 tests
tests for it?

/-\


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: Testing Inline::C

[Andrew Savige]

Michael G Schwern <[EMAIL PROTECTED]> wrote:
> To catch memory mistakes in C, I'd normally use something like Electric
> Fence or other malloc replacement.  Basically something that replaces the
> memory allocation functions with those that put in magic so that if your
> program walks outside its allocated memory space it immediately pukes.

Recently, I searched the Web for the best C/C++ memory checking tools.
After finding an awful lot of crappy ones, I finally stumbled on mpatrol,
which I like a lot: it is powerful, mature, portable, works with both
C and C++, nice licensing, good quality code and documentation, ...
However, there is quite a long learning curve and you may need to
recompile for best results.

The other standout tool I noticed is valgrind (Linux only, I think): it's
powerful and very easy to use (not requiring recompilation or relinking).
It is a pest though if you want to check setuid executables and it won't
find all memory errors (overwriting a stack array, for instance) -- but
it does find a lot and is really convenient to use.

If anyone knows of other good memory checking tools, please let me know.

/-\


http://greetings.yahoo.com.au - Yahoo! Greetings
Send your love online with Yahoo! Greetings - FREE!

Re: Using environment variables to control long running tests (again)

[Andrew Savige]

Michael G Schwern wrote:
> Disabling tests for subjective reasons (they take "too long", they don't
> test critical functionality, etc...) is a slippery slope.  For that reason
> I'd agree with Curtis and say that everything is always run by default
> and users can then elect what to turn off.  PERL_SKIP_LONG_TESTS seems like
> a clear name.
> 
> If users find tests annoying then can turn them off, or ignore them or
> not run them at all.  Because the user impulse is more often to weaken
> the tests than the strengthen them, the default should always be the 
> strongest testing position.

I think PERL_SKIP_LONG_TESTS is a fine name.

Given the sometimes heated discussion in this thread:
  http://www.perlmonks.org/index.pl?node_id=254011
I doubt the two camps will ever agree on this issue.

My view is that it is up to the individual CPAN author how he
wants to treat the naive user (i.e. the user who installs his
module without explicitly setting any environment variable).
That is, the CPAN author can choose whether he wants to use
PERL_SMOKE or PERL_SKIP_LONG_TESTS (based on his good taste
and sound judgement, after all it is his module).

I think the two environment variables can happily coexist as
shown in the table below:

  PERL_SMOKE   PERL_SKIP_LONG_TESTS
Naive User0 0
Contrary User 1 1
Impatient User0 1
Robot 1 0

Naive: Gets default behaviour desired by CPAN author
Contrary:  Gets opposite of default behaviour desired by CPAN author
Impatient: Never runs long tests
Robot: Always runs long tests

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Using environment variables to control long running tests (again)

[Andrew Savige]

Ovid wrote:
> --- Kate L Pugh wrote:
> > This was discussed on this list back in June.  I'm wanting to
> > implement it now and am wondering if Andrew's suggestion (below) has
> > been taken up by anyone.  Is PERL_TEST_LONG what people here
> > generally
> > expect to be the right environment variable to set to enable
> > long-running tests?

I followed merlyn's suggestion and used PERL_SMOKE. That is, the long-
running tests are skipped by default; they are run only if the user sets
the PERL_SMOKE environment variable.

Since the name is arbitrary, it makes little sense for each developer
to invent her own different name for the same purpose, hence my initial
query.

> I don't know if something like that's been implemented or not, but I
> think it's something that should be implemented by the user and
> probably not be pushed into the core testing suite.  

Having an agreed name requires no changes to any test harnesses, the
user controls simply by setting the PERL_SMOKE environment variable.

> Also, I would recommend something like PERL_SKIP_LONG_TESTS.  By
> default, all tests should be run to prevent the user accidentally
> forget to run some tests.

If some tests take hours to run, running them by default will annoy
interactive users, in particular those running the CPAN shell.

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Refactoring a test program: advice sought

[Andrew Savige]

Michael G Schwern wrote:
> I use t/lib so the top level t/ directory doesn't get cluttered (and for
> compatibility with the Perl core which may be important later for A::T).

Yes, I like that. Should I call it:
  t/lib/Test/Archive/Tar...
or:
  t/lib/Archive/Tar/Test...
or something else?

I took a quick look a mod_perl and Template Toolkit (TT).
TT has a:
  lib/Template/Test.pm
which looks wrong to me (should that not be under t/lib instead?).

Not sure, but mod_perl seems to have unbundled the test suite
into a separate Apache-Test distribution. Again, why should
that be called Apache-Test rather than Test-Apache?

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Refactoring a test program: advice sought

[Andrew Savige]

As part of the phalanx project, I've added quite a few new tests to
02_methods.t in the Archive::Tar test suite. Though I'm jubilant the
new tests have uncovered a number of bugs, the test code itself has
been getting progressively uglier, ripe for refactoring, in fact.

To avoid code duplication between tests, I'd like to store common
code outside the tests. To see how others have solved this problem,
I took a look at YAML and ExtUtils-MakeMaker.

YAML has t/TestYAML.pm and its .t files do this:
  use lib 't';
  use TestYAML;
ExtUtils-MakeMaker has t/lib/MakeMaker/Test/Utils.pm and its .t
files do this:
  BEGIN {
  if( $ENV{PERL_CORE} ) {
  chdir 't' if -d 't';
  @INC = ('../lib', 'lib');
  }
  else {
  unshift @INC, 't/lib';
  }
  }
  chdir 't';
  use MakeMaker::Test::Utils;
Which model should I follow? Or are there better models out there?

Anyway, the current 02_methods.t is embedded below; any advice on
how it could be improved or refactored is most welcome! (I know it
has a bug in that it does not skip the correct number of tests).

/-\

use Test::More 'no_plan';
use strict;

use Cwd;
use IO::File;
use File::Path;
use File::Spec  ();
use File::Spec::Unix();
use File::Basename  ();

use Archive::Tar;
use Archive::Tar::Constant;

my $tar = Archive::Tar->new;
my $tarbin  = Archive::Tar->new;
my $tarx= Archive::Tar->new;

for my $obj ( $tar, $tarbin, $tarx ) {
isa_ok( $obj, 'Archive::Tar', 'Object created' );
}

my $file =
qq[directory/really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-really-long-directory-name/myfile];

my $expect = {
c   => qr/^\s*$/,
d   => qr/^\s*$/,
};

my $all_chars = join '', "\r\n", map( chr, 0..255 ), "zzz\n\r";

### @expectbin is used to ensure that $tarbin is written in the right   ###
### order and that the contents and order match exactly when extracted  ###
my @expectbin = (
###  filename  contents   ###
['bIn11',  $all_chars x 11 ],
['bIn3',   $all_chars x  3 ],
['bIn4',   $all_chars x  4 ],
['bIn1',   $all_chars  ],
['bIn2',   $all_chars x  2 ],
);

### @expectx is used to ensure that $tarx is written in the right   ###
### order and that the contents and order match exactly when extracted  ###
my $xdir = 'x';
my @expectx = (
###  filename  contentsdirs ###
['k'   ,   '', [ $xdir ]],
[$xdir ,   'j',[ $xdir ]],   # failed before A::T 1.08
);

### wintendo can't deal with too long paths, so we might have to skip tests ###
my $TOO_LONG=   ($^O eq 'MSWin32' or $^O eq 'cygwin')
&& length( cwd(). $file ) > 247;

if( $TOO_LONG ) {
SKIP: {
skip( "No long filename support - long filename extraction disabled", 0
);
}
} else {
$expect->{$file} = qr/^hello\s*$/ ;
}

my @root = grep { length }   File::Basename::dirname($0),
'src', $TOO_LONG ? 'short' : 'long';

my $archive= File::Spec->catfile( @root, 'bar.tar' );
my $compressed = File::Spec->catfile( @root, 'foo.tgz' );
my $archivebin = File::Spec->catfile( @root, 'outbin.tar' );
my $compressedbin  = File::Spec->catfile( @root, 'outbin.tgz' );
my $archivex   = '0';
my $compressedx= '1';
my $zlib   = eval { require IO::Zlib; 1 };
my $NO_UNLINK  = scalar @ARGV ? 1 : 0;

### error tests ###
{
local $Archive::Tar::WARN  = 0;
my $init_err   = $tar->error;
my @list   = $tar->read();
my $read_err   = $tar->error;
my $obj= $tar->add_data( '' );
my $add_data_err   = $tar->error;

is( $init_err, '',  "The error string is empty" );
is( scalar @list, 0,"Function read returns 0 files on
error" );
ok( $read_err,  "   and error string is non empty" );
like( $read_err, qr/create/,"   and error string contains create"
);
unlike( $read_err, qr/add/, "   and error string does not contain
add" );
ok( ! defined( $obj ),  "Function add_data returns undef on
error" );
ok( $add_data_err,  "   and error string is non empty" );
like( $add_data_err, qr/add/,   "   and error string contains add" );
unlike( $add_data_err, qr/create/,  "   and error string does not contain
create" );
}

### read tests ###
my $gzip = 0;
for my $type( $archive, $compressed ) {

my $state = $gzip ? 'compressed' : 'uncompressed';

SKIP: {
   
skip(   "No IO::Zlib - can not read compressed archives",
4 + 2 * (scalar keys %$expect)
) if( $gzip and !$zlib);

{
my @list= $tar->read( $type );

Re: Phalanx

[Andrew Savige]

"Jeays, Mark" wrote:
> Hi, 
> 
> I'm writing on behalf of Ottawa Perl Mongers. At our last meeting, one of
> our members gave us a quick introduction to the Phalanx project and its
> goals. A number of the members were interested in the idea and we have been
> looking for a group project. How would we go about getting involved? Can we
> pick a module we're interested in? Do you think doing tests for a module
> would be suitable for a group to take on? Is there a particular methodology
> to be followed? 

Ottawa eh? Well, if the infamous `/anick is involved, I'm sure we are all
in for a lot of fun. ;-) I think some of the bigger modules are too big
for one person, ideal for a team from Ottawa, in fact. Three bigger and
important ones that spring to mind are:

1) DBI (Tim Bunce specifically asked for help in re-designing the test
   suite).

2) CPANPLUS.

3) Module::Build.

Having said that, you better check with Andy. But I think you can
just pick any untaken module from the top 100 that appeals to you.

As for how to get started, I just followed the excellent checklist
at http://qa.perl.org/phalanx/modules.html and it has worked like
a charm. If a module you really really like is already taken I
think it is fine to email the hoplite and offer to help.

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Trying to spear a phalanx shield for pod

[Andrew Savige]

Michael G Schwern wrote:
> On Sun, Oct 26, 2003 at 04:45:48PM +1100, Andrew Savige wrote:
>> There is a misprint in this line:
>>my $have_testpod = !$@ and $Test::Pod::VERSION >= 0.95;
>> It should read:
>>my $have_testpod = !$@ &&  $Test::Pod::VERSION >= 0.95;
>
> I deliberately used and instead of &&.  I'd noticed the original was
> using explicit parens to disambiuate the precendence to make sure there
> was no chance of it accidentally evaluating as:
> 
> (!$@ && $Test::Pod::VERSION) >= 0.95
> 
> for those of us that don't have the symbol table memorized.  Instead of
> parens, I went with 'and'.
> 
> But looking at the precdence table, either version will work.

Your version does not work (unless I've had a brain malfunction):

# cat sch.pl
use strict;
eval 'use Test::Pod';
my $have_testpod = !$@ and $Test::Pod::VERSION >= 0.95;
print "have_testpod='$have_testpod'\n";
print "tpv=", $Test::Pod::VERSION, "\n";

# perl -w sch.pl
Useless use of numeric ge (>=) in void context at sch.pl line 3.
have_testpod='1'
tpv=0.94

# perl -MO=Deparse -e \
> 'my $have_testpod = !$@ and $Test::Pod::VERSION >= 0.95'
$Test::Pod::VERSION >= 0.95 if my $have_testpod = !$@;
-e syntax OK

>> Based on the excellent work from Lester & Schwern, I could not restrain
>> myself from shortening this a little more (since it will be included
>> in many distributions).
> 
> The logic escapes me.  Clarity is more important than length if you're
> going to be sticking this code all over the place.

I'm not going to pretend it's rational, but for some reason I feel more
comfortable with cut n' pasting if it's only a small amount of code.

This reminds me of the many recent discussions re the merit of:
   $s = do { local $/; <$fh> };
versus using File::Slurp. Psychologically, it may be easier to justify
cut n' pasting if you can say "oh, it's just a one-liner".

> But if we're going to encourage people to copy this code, Test::Pod
> should probably just have a subroutine to do this for you.  That's
> what libraries are for!

I hope Andy is listening. ;-)
Yes, I agree it'd be nice for Test::Pod to provide a little more help
for this very common chore for CPAN module authors. I do feel most
uncomfortable cut n' pasting this code all over the place.

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Trying to spear a phalanx shield for pod

[Andrew Savige]

Michael G Schwern wrote:
> Since skip_all will exit immediately you can fold that big "everything
> inside the else block" away.
> 
> eval 'use Test::Pod';
> my $have_testpod = !$@ and $Test::Pod::VERSION >= 0.95;
> plan skip_all => "Test::Pod v0.95 required for testing POD" 
> unless $have_testpod;
> 
> my @files;
> my $blib = File::Spec->catfile(qw(blib lib));
> ...

There is a misprint in this line:
   my $have_testpod = !$@ and $Test::Pod::VERSION >= 0.95;
It should read:
   my $have_testpod = !$@ &&  $Test::Pod::VERSION >= 0.95;

Based on the excellent work from Lester & Schwern, I could not restrain
myself from shortening this a little more (since it will be included
in many distributions). My current short version is:

use Test::More;
use File::Spec;
use File::Find;
use strict;
eval 'use Test::Pod';
plan skip_all => "Test::Pod v0.95 required for testing POD"
if $@ || $Test::Pod::VERSION < 0.95;
my @files;
find( sub {push @files, $File::Find::name if /\.p(?:l|m|od)$/},
File::Spec->catfile( qw(blib lib) ) );
plan tests => scalar @files;
foreach my $file (@files) { pod_file_ok($file) }

Two questions:

1) Is using $File::Find::name portable on VMS and Mac OS?
   (I'm confused about File::Find's use of 'Unix' names).

2) Which is preferred:
   eval 'use Test::Pod';
   or:
   eval { require Test::Pod };
   ...
   Test::Pod->import;
   or does it not matter?

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Trying to spear a phalanx shield for pod

[Andrew Savige]

I'm about to add a POD test program to my phalanx distro.
Before I do that, just want to check I'm using the best model.
I plan on using the one from WWW::Mechanize (shown below) --
unless someone can suggest a better model.

Is it worth trying to agree on a de facto standard name for
such a beast: 99-pod.t/99_pod.t/99.pod.t/99pod.t?

use Test::More;

use File::Spec;
use File::Find;
use strict;

eval {
require Test::Pod;
};

my $ok = !$@ && ($Test::Pod::VERSION >= '0.95');

if (!$ok) {
plan skip_all => "Test::Pod v0.95 required for testing POD";
} else {
Test::Pod->import;
my @files;
my $blib = File::Spec->catfile(qw(blib lib));
find( sub {push @files, $File::Find::name if /\.p(l|m|od)$/}, $blib);
plan tests => scalar @files;
foreach my $file (@files) {
pod_file_ok($file);
}
}

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Taint mode testing and project Phalanx

[Andrew Savige]

Michael G Schwern wrote:
> On Tue, Oct 21, 2003 at 12:34:44PM -0500, Dave Rolsky wrote:
>> Anyway, my taint mode experience has been that random things break in very
>> weird ways when using it.
> 
> All the more reason to test with it on. :)

Given the differences in behaviour with taint mode, it seems to me
that for a "taint mode test" (i.e. one with -wT in its first line)
Test::Harness should run the test twice -- once with taint mode and
once without. Though I suppose there might be a case where you want
to run the test in taint mode only, so maybe Test::Harness needs
some options to control this.

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: No more code coverage

[Andrew Savige]

Tim Bunce wrote:
> p.s. Could someone suggest a pure-perl module with lots of tests as
> a suitable testbed for Devel::Cover? 

http://search.cpan.org/dist/Acme-EyeDrops has 22 test programs,
769 tests and no dependencies.

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Taint mode testing and project Phalanx

[Andrew Savige]

I noticed in Test::Tutorial:
"Taint mode is a funny thing. It's the globalest of all global features.
Once you turn it on it effects all code in your program and all modules
used (and all the modules they use). If a single piece of code isn't
taint clean, the whole thing explodes. With that in mind, it's very
important to ensure your module works under taint mode."

I don't have much practical experience with taint mode testing,
however, and was wondering how important taint mode testing is
to project Phalanx.

What are general opinions re taint tests for CPAN modules?
(i.e. a test with #!perl -Tw as the first line of the test).

Further to Schwern's Test::Tutorial preach to the kwire above,
taint mode tests seem worthwhile to me because:

1) They may catch silly errors in module code. For example:
 $x = `date`;
   is not taint-safe, while:
 $x = scalar localtime;
   is. Clearly, the latter is preferable.

2) They further exercise Perl itself (part of the Phalanx charter).

On the down-side: while taint tests seem to work fine for me with
Perl 5.6.0 and above, I experienced problems with Perl 5.005 which
failed with something like "cannot locate Fred/Fred.pm in @INC"
when running taint tests. I have hardly investigated this: does
anyone out there regularly run taint mode tests with Perl 5.005?

/-\


http://personals.yahoo.com.au - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

Re: Devel::Cover can't find loaded modules

[Andrew Savige]

Ovid wrote:
> I was running some test code with Devel::Cover and I've had no problem using
> it or generating pretty reports that make coworkers "ooh" and "ahh".
> Unfortunately, I started running my test suite on a different set of
> tests and started getting some strange errors...

I have no experience with Devel::Cover yet, but remember this post:
  http://nntp.x.perl.org/group/perl.qa/2213
which states that Devel::Cover cannot be run from the 't' directory
(I know your 'grind' command cd's to t/, because I use it :-).

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Ovid wrote:
> --shuffle will shuffle the order in which the tests are run to ensure that
> you have no accidental dependency on test order.
>
> --fast sets and environment variable that can be checked in the test scripts.
> For example, if you have a couple of tests that double the time of your test
> run and you want a fast check of integrity, you can put this in your code:
>
>   SKIP: {
> skip 'Fast Tests', 1 if $ENV{FAST_TESTS};
> ok( really_long_running_function(), '... successfully kills time');
>   }

Erm, am I the only one to take Merlyn seriously?
  http://nntp.x.perl.org/group/perl.qa/1932
Well, that's what google tells me (unsurprisingly, zero people took my
PERL_TEST_LONG suggestion seriously).

Instead of FAST_TESTS, may I humbly suggest something like PERL_TEST_FAST,
the point being that environment variables share the global name space.

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Ovid wrote:
> I've just made it available at
> http://users.easystreet.com/ovid/cgi_course/downloads/grind.gz
>
> It needs more work, including allowing descending into directories (via
> File::Find or a similar mechanism) and having pre and post actions.
> I haven't figured out the best way to do the latter (that would be
> things like setting up a test db, tearing it down afterwards, etc.)

Thanks, I've started using it already. :-) Using your suggested google-fu
search technique, I noticed that, in addition to Test::Verbose (mentioned
by Adrian Howard), it may be worth keeping an eye on libwww-perl-5.69
(t/TEST) and mod_perl-1.28 (t/TEST.PL) since they perform a similar
function to your grind. Apache-Test-1.03 (used in modperl2) seems to have
evolved into something much more comprehensive; the old mod_perl1 TEST.PL
functionality seems to be superseded by Apache::TestHarness run() method.

All of the above seem to first chdir into the 't' directory, which I found
a bit odd because "make test" runs from the directory below that one.
And I imagine quite a few CPAN modules have tests that will fail if run
from the 't' directory. Or is writing a test that assumes you are in the
directory below the 't' directory considered bad style? (if so, I couldn't
find it mentioned in the docs).

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Ovid wrote:
> I do something like the following to get this effect:
> 
>   #!/usr/bin/perl -w
>   use strict;
>   use Test::Harness;
>   use Getopt::Long;
>   use Pod::Usage;
> 
>   GetOptions(
> 'help|?'=> sub { pod2usage(-verbose => 2); exit },
> 'verbose!'  => \$Test::Harness::verbose,
> 'quiet' => sub { $Test::Harness::verbose = 0 },
> 'include=s' => \my @include,
> 'exclude=s' => \my @exclude,
> 'shuffle'   => \my $shuffle_tests,
> 'fast'  => \$ENV{FAST_TESTS},
>   );
> 
> In this case, this is part of my driver script and I call it like:
> 
>   grind --fast
> 
> That runs through all of my tests, but the "FAST_TESTS" environment
> variable is available in the test programs that I run.

Oh, that 'grind' looks like a very handy command but I'm a bit
confused about how you use it. Is it just a handy general-purpose
command or do you use it specifically as part of "make test" in
your CPAN distributions? If you have used it in a CPAN module,
please let me know which one, so I can take a look.

I assume grind calls Test::Harness:runtests() shortly after accepting
these command-line options, running all "t/*.t" tests by default, and
using @include and @exclude to include/exclude tests. Is that right?
What is --shuffle and --fast used for? Can you please give some
examples of how you use it?

BTW, is there a web site that allows you to search CPAN source code?
For example, I might like to search all of CPAN for any distribution
that contains Test::Harness in any file in the t/ directory.

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Fergal Daly wrote on 14 July 2003:
>   is it possible with Test::Harness and MakeMaker to pass arguments to
> my test scripts? I think it's not but I just want to check for sure.
> The module I'm working on is getting a new "optimised" mode so I'd like
> to be able to run all the tests twice, once with and once without the
> optimiser.

Which module is this? What did you end up doing? I'm interested to study
what you did, so as to learn new testing techniques.

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: Phalanx has started, and I need perl-qa's help

[Andrew Savige]

> Net_SSLeay.pm

Just noticed that's a kinda odd name for a distribution that contains
the modules Net::SSLeay and Net::SSLeay::Handle. I wonder why is it
not called Net-SSLeay?

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: Phalanx has started, and I need perl-qa's help

[Andrew Savige]

Andy Lester wrote:
> The Phalanx project has started its rampup to an official 
> announcement.  Phalanx is going to beef up the tests, coverage and 
> docs on Perl and 100 heavily-used modules from CPAN.
> 
> The project page is at http://qa.perl.org/phalanx/.  Please take a 
> look, tell me your thoughts, and if there are any serious ommissions 
> from the Phalanx 100 module list...
> 
> I'm turning to the perl-qa group for feedback BEFORE I announce to 
> the world, so I appreciate any comments.

Looks like a most excellent start to me. Thanks Andy.

Here is the top 100 list sorted by most prolific author
(just in case anyone is interested :-):

8 GAAS  : HTML-Parser libwww-perl MIME-Base64 URI Digest-HMAC
  Digest-SHA1 Image-Info MD5
4 JWIED : DBD-mysql Msql-Mysql-modules Net-Daemon Text-CSV_XS
3 GBARR : Convert-ASN1 perl-ldap Scalar-List-Utils
3 INGY  : CGI-Kwiki Inline YAML
2 ABW   : Template-Toolkit AppConfig
2 DPARIS: Crypt-Blowfish Crypt-DES
2 DROLSKY   : HTML-Mason Params-Validate
2 ERYQ  : IO-stringy MIME-tools
2 KANE  : CPANPLUS Archive-Tar
2 LDS   : GD Crypt-CBC
2 MARKOV: MailTools Mail-Box
2 MSERGEANT : XML-SAX XML-XPath
2 MVERB : GDGraph GDTextUtil
2 RGIERSIG  : Expect IO-Tty
2 SBURKE: HTML-Tagset HTML-Tree
2 STBEY : Date-Calc Bit-Vector
2 TIMB  : DBI DBD-Oracle
1 ABH   : Apache-DBI
1 ABIGAIL   : Regexp-Common
1 AKSTE : Data-ShowTable
1 AREIBENS  : PDF-API2
1 AUTRIJUS  : PAR
1 BEHROOZI  : IO-Socket-SSL
1 BIRNEY: bioperl
1 BTROTT: Net-SSH-Perl
1 CHAMAS: Crypt-SSLeay
1 CNANDOR   : MP3-Info
1 COOPERCL  : XML-Parser
1 CREIN : Net-DNS
1 CWINTERS  : SPOPS
1 DCLINTON  : Cache-Cache
1 DCONWAY   : Parse-RecDescent
1 DMEGG : XML-Writer
1 DTOWN : Net-SNMP
1 GRANTM: XML-Simple
1 GSAR  : libwin32
1 ILYAZ : Term-ReadLine-Perl
1 JBAKER: Apache-Session
1 JCRISTY   : PerlMagick
1 JERLBAUM  : CGI-Application
1 JESSE : DBIx-SearchBuilder
1 JMASON: Mail-SpamAssassin
1 JMCNAMARA : Spreadsheet-WriteExcel
1 JOESUF: libapreq
1 JROGERS   : Net-Telnet
1 JURL  : DBD-ODBC
1 JZUCKER   : DBD-CSV
1 KGB   : PDL
1 KJALB : TermReadKey
1 KMACLEOD  : libxml-perl
1 KULCHENKO : SOAP-Lite
1 KWILLIAMS : Module-Build
1 KWITKNR   : Spreadsheet-ParseExcel
1 MERGL : DBD-Pg
1 MIVKOVIC  : Mail-Sendmail
1 MPIOTR: Text-Iconv
1 MUIR  : Time-modules
1 NEDKONZ   : Archive-Zip
1 PETDANCE  : WWW-Mechanize
1 PHISH : XML-LibXML
1 PMQS  : Compress-Zlib
1 RCAPUTO   : POE
1 RJRAY : Image-Size
1 RSE   : TimeDate
1 SAMPO : Net_SSLeay.pm
1 SAMTREGAR : HTML-Template
1 SBECK : DateManip
1 SHERZODR  : CGI-Session
1 TJMATHER  : XML-DOM
1 TMTM  : Class-DBI
1 UARUN : Error
1 WADG  : Config-IniFiles
1 YVES  : MIME-Lite

/-\



http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: Passing arguments to tests

[Andrew Savige]

A moron once wrote:
> I have a similar problem; I'd like some of my test programs
> to generate other test programs on the fly, then run them.
>
> This seems to work:
>
> use strict;
> use Test::Harness;
> my $outf = 'out.tmp';
> print "1..1\n";
> local *SAVOUT; open(SAVOUT, ">&STDOUT");  # save original STDOUT
> open(STDOUT, '>'.$outf) or die "error: open '$outf': $!";
> my $success = runtests('t/some_generated_test.t');
> open(STDOUT, ">&SAVOUT"); # restore STDOUT
> $success or print "not ";
> print "ok 1\n";
>
> but I feel odd.

Hey Schwern! How come you let this stand? Looks like you gave the right
advice tonight in huesserm's use.perl.org journal:

"A more general solution is in TieOut.pm of which copies are scattered
around in t/lib directories of various modules, a little invention of
chromatic's. Here's one.
http://search.cpan.org/src/MSCHWERN/ExtUtils-MakeMaker-6.16/t/lib/. Here's
an example of its use.
http://search.cpan.org/src/MSCHWERN/ExtUtils-MakeMaker-6.16/t/prompt.t
That also shows TieIn for input filehandles.

Yes, one of these days I'll get around to writing Test::Filehandles. Hey,
someone could pay me to write it! ;)"

Or am I missing something? (again).

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Michael G Schwern wrote:
> On Wed, Aug 20, 2003 at 07:18:54PM +1000, Andrew Savige wrote:
> > I admit to asking my original question as a joke since, as of Perl
> 5.6.1,
> > Test::Harness was pure functional (Test::Harness::Straps no there).
> > However, I am not joking now when I say I have no idea what "Straps"
> > means.
> 
> harness
>n 1: a support consisting of an arrangement of straps for holding
> something to the body (especially one supporting a
> person suspended from a parachute)
>2: stable gear consisting of an arrangement of leather straps
>   fitted to a draft animal so that it can be attached to and
>   pull a cart
> 
> Test::Harness::Parser would have been a better name, but hey, you're
> talking to the author of Ima::DBI.

Not to mention Semi::Semicolons.
You'll have to take my word for it when I claim that I finally realised
why you'd chosen "Straps" ... right after I pressed the send button.
Maybe after that "is my bitch" business, I'm looking for slang from
"the hood" everywhere. Or become too obsessed with Leon's prize.
I don't know. I suck.

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: passing arguments to tests

[Andrew Savige]

Michael G Schwern wrote:
> On Thu, Jul 24, 2003 at 10:55:57AM +1000, Andrew Savige wrote:
>> I'd be interested to see an example of sub-classing Test::Harness.
> 
> See examples/mini_harness.plx in Test::Harness.
> 
> The straps interface is not yet entirely usable.

Thanks. Looks interesting.
I admit to asking my original question as a joke since, as of Perl 5.6.1,
Test::Harness was pure functional (Test::Harness::Straps no there).
However, I am not joking now when I say I have no idea what "Straps"
means. Assuming it has nothing to do with Leon's recent successful bid
at YAPC::Europe auction, I'm guessing it's Mercan idiom/slang ??
(I needed HFB to explain "schwern is my bitch" to me also:
http://archive.develooper.com/[EMAIL PROTECTED]/msg00012.html).

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: Scrutinizing CPAN distributions (was Testing for valid path names...)

[Andrew Savige]

Leon Brocard wrote:
> I like the is_impolite / is_naughty ideas, and will roll them into the
> next version. If you have a simple metric for a good cross-platform
> filename, that'd be good.

I'll see what I can come up with.

> I'm not sure about how you mean a "good" Changes. For a start, people
> call them different things (Changes, CHANGES, ChangeLog etc.), and
> format them differently.

Yeah, this is probably too hard by the low-hanging fruit theory.
I can normally tell a good Changes file from a poor one (for example,
one that spells my name "Savage" ;-) but it is very hard to get a
machine to recognize the difference. There is a poor man's Inline::C
(whose name escapes me) that is lacking any Changes or README file or
any documentation at all -- now, that would be easy to detect.
 
> What is a good README? To be honest, now that we have web interfaces
> to docs on search.cpan.org I don't think that READMEs are terribly
> important.

Agreed.

> > 2) POD tester. Use Test::Pod/Pod::Coverage, say, on all POD in a
> >distribution.
> 
> I don't want to eval the whole of CPAN. Maybe we could get the CPAN
> Testers to do this?
>
> > 4) Test suite analyser. How good is the test suite? Use perhaps
> >Devel::Cover to determine how much of the code is covered by
> >the distribution's test suite.
>
> Sure, convince CPAN Testers to do this ;-)
>  
> > 3) Static Perl code analyser. I suppose PerlTidy, Module::Info,
> >B::Lint may be helpful. Any others?
> 
> To achieve what?

It doesn't have to eval the whole of CPAN to be useful.
I see the mythical Module::Scrutinize as perhaps a little orthogonal
to Module::CPANTS, as something that may help individual CPAN authors
produce a higher quality product, running their CPAN module through it
pointing out things that might be improved. It may also be handy to
help in-depth module reviewers produce reviews with more substance than
most of those posted to cpanratings -- which I still quite like, BTW,
despite the smutty DBI reviews (on the other hand, I learnt some new
slang terms I'd never heard of before).
  
> > 6) Prerequisite checker.
> 
> What would you check, exactly?

That the prerequisites in Makefile.PL/Build.PL/META.yml match the code.

> > 7) Version checker.
> 
> What would you check, exactly?

That multiple versions in multiple files match.
  
> It's great that somebody else thinks that metrics is a good idea.
> I thought it was just Schwern, Thomas and I! ;-)

Four completely normal peeps there. ;-) Judging by the popularity of
cricket and golf statistics, I think there are a lot of lurkers too.

/-\


http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Scrutinizing CPAN distributions (was Testing for valid path names...)

[Andrew Savige]

Though this started as an innocent question, I think it would be
nice to have a module/script to scrutinize a CPAN distribution.

1) Archive nit-picker. Archive::Any's is_impolite/is_naughty is a
   start. Also test for: "good cross-platform" file names (my
   original question); "good" Changes, README, t/ directory etc..

2) POD tester. Use Test::Pod/Pod::Coverage, say, on all POD in a
   distribution.

3) Static Perl code analyser. I suppose PerlTidy, Module::Info,
   B::Lint may be helpful. Any others?

4) Test suite analyser. How good is the test suite? Use perhaps
   Devel::Cover to determine how much of the code is covered by
   the distribution's test suite.

5) Signature checker.
6) Prerequisite checker.
7) Version checker.

Anything else? Does such a module/script already exist?

I'm an enthusiastic admirer of Schwern's CPANTS kwalitee vision and
aware of Module::CPANTS but unsure about the current state of play.
Tips on other stuff I should look at are welcome.

/-\



http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Testing for valid path names in CPAN distributions

[Andrew Savige]

Running variants of:

tar tzf perl-5.8.0.tar.gz | perl -lne'print if tr|-_./a-zA-Z0-9||c'

suggests only [-_./a-zA-Z0-9] are valid characters in a path name.

Then I noticed 'perldoc perlport' lists the portable filename
characters as defined by ANSI C and various other restrictions.
What is the length limit of each path name component?
What is the length limit of file extensions? I heard YAML changed
from .yaml to .yml, for instance, yet Perl itself has many files
with long extensions -- runtime.porting, for example.

It'd be nice to have a standard test for valid "portable" path names.
Does such a test exist? I noticed Archive::Any has is_impolite() and
is_naughty() but didn't see any checks for basic path name validity.
BTW, is Archive::Any a "dead camel"?

/-\



http://search.yahoo.com.au - Yahoo! Search
- Looking for more? Try the new Yahoo! Search

Re: Testers & PASS

[Andrew Savige]

Leon Brocard wrote:
> Leon Brocard sent the following bits through the ether:
>
> > Secondly, who do I need to convince to add the "make test" results for
> > PASSes too? ;-)
>
> So, does anyone actually have an opinion on this?

*Puts up hand*. I agree with you. Seems useful and trivial to implement.
But who owns it?

/-\


http://personals.yahoo.com.au - Yahoo! Personals
-  New people, new possibilities! Try Yahoo! Personals, FREE for a limited period!

Re: passing arguments to tests

[Andrew Savige]

> #!/usr/bin/perl
> use strict;
> use warnings;
> 
> sub my_fn { print "in sub my_fn, args='@_'\n" }
> 
> # This happily calls my_fn(): the parameters received by my_fn are
> # 'sample.t' and 'def'. But why?
> 
> 'sample.t'->main::my_fn('def');
> 
> # ... yet this fails with: Can't call method "main::my_fn" without
> # a package or object reference
> 
> './sample.t'->main::my_fn('def');

For cheap thrills, I grep'ed the Perl C sources for the error message
and found it in file pp_hot.c/S_method_common():

if (!packname ||
((UTF8_IS_START(*packname) && DO_UTF8(sv))
? !isIDFIRST_utf8((U8*)packname)
: !isIDFIRST(*packname)
))
{
Perl_croak(aTHX_ "Can't call method \"%s\" %s", name,
   SvOK(sv) ? "without a package or object reference"
: "on an undefined value");
}
/* assume it's a package name */
...

Hmm, it seems that if the first char is [A-Za-z_] it's assumed to be
a package -- which explains why 'sample.t' worked and './sample.t'
didn't. To test that theory further:

#!/usr/bin/perl
sub What::the::_::is::going::on { print+reverse$/,@_ }
'[EMAIL PROTECTED]&?!! is going on?'->What::the::_::is::going::on('What the ');

Not sure if this is a bug or a feature, but it sure is fun. :-)

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: passing arguments to tests

[Andrew Savige]

Andrew P's test program does indeed "work" with Perl 5.8.0
on both Unix and Windows with a test name of 'sample.t' in
THDriver.pl, yet changing it to './sample.t' results in a
failure of: Can't call method "SUPER::runtests" without a
package or object reference. But why?

The snippet below (tested with Perl 5.8.0) is hopefully a
simplified example. Can anyone explain what is going on?

--snip---snip---snip---
#!/usr/bin/perl
use strict;
use warnings;

sub my_fn { print "in sub my_fn, args='@_'\n" }

# This happily calls my_fn(): the parameters received by my_fn are
# 'sample.t' and 'def'. But why?

'sample.t'->main::my_fn('def');

# ... yet this fails with: Can't call method "main::my_fn" without
# a package or object reference

'./sample.t'->main::my_fn('def');

# And this fails with: Can't locate object method "my_fn" via
# package "sample.t" (perhaps you forgot to load "sample.t"?)

'sample.t'->my_fn('def');
--snip---snip---snip---

/-\



http://personals.yahoo.com.au - Yahoo! Personals
-  New people, new possibilities! Try Yahoo! Personals, FREE for a limited period!

Re: black-box test tool

[Andrew Savige]

Danny Faught wrote:
> I've often talked about the difference between my black-box test
> experience and the unit testing context that most of you are working in.
>   I've come across an interesting example of an open source black-box
> tool - QMTest, http://www.codesourcery.com/qm/qmtest.

Just curious, has anyone had STAF?
  http://staf.sourceforge.net/index.php
I've been asked to investigate using this at work.
Originally developed at IBM, it looks very comprehensive, running on
a wide variety of platforms. The core is written in C++; it has plugins
for many languages, including Perl, Python, REXX, Java...

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: passing arguments to tests

[Andrew Savige]

"Potozniak, Andrew" wrote:
> Create a sub class of Test::Harness and or MakeMaker that will over-ride
> all of their methods/subroutines and then add what you want it to do,
> don't forget to call super classes' method/subroutine that you are
> over-riding. I can give an example of this if need be.

I'd be interested to see an example of sub-classing Test::Harness.

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: passing arguments to tests

[Andrew Savige]

Fergal Daly wrote:
> Hi,
>   is it possible with Test::Harness and MakeMaker to pass arguments
> to my test scripts? I think it's not but I just want to check for sure.
> The module I'm  working on is getting a new "optimised" mode so I'd like
> to be able to run all the tests twice, once with and once without the
> optimiser.

I have a similar problem; I'd like some of my test programs
to generate other test programs on the fly, then run them.

This seems to work:

use strict;
use Test::Harness;
my $outf = 'out.tmp';
print "1..1\n";
local *SAVOUT; open(SAVOUT, ">&STDOUT");  # save original STDOUT
open(STDOUT, '>'.$outf) or die "error: open '$outf': $!";
my $success = runtests('t/some_generated_test.t');
open(STDOUT, ">&SAVOUT"); # restore STDOUT
$success or print "not ";
print "ok 1\n";

but I feel odd. Or should I generate them in Makefile.PL?

Does anyone know of a CPAN module that generates its test programs?
More generally, which CPAN modules have particularly good test
suites, worthy of study?

/-\



http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: Using environment variables to control long running tests

[Andrew Savige]

Mark Fowler wrote:
> On Thu, 5 Jun 2003, [iso-8859-1] Andrew Savige wrote:
> 
> > While a standard name is not required, having one would allow
> > automated CPAN test harnesses to run the longer tests.
> 
> While we're on it, it'd be nice to have a standard enviromental
> variable that turned off all interactivty.

Since no experienced perl-qa'er has leapt in, and against my better
judgement, I suggest an environment variable name of:

PERL_TEST_LONG

Paranoid/masochistic folks and automated test harnesses can set this
variable to ensure all tests are run, even the long running ones.
More well-adjusted people (such as merlyn) would not set it, obviously.

I suppose you could further assign meanings to its value (0, 1, 2,
etc.) but I don't see a need for that right now. I guess you could
also add some attribute/s to META.yml, such as whether a module
supports PERL_TEST_LONG or perhaps uses some other environment
variable/s or some other testing scheme.

As for a name for an environment variable to switch off interactivity,
I defer to twoshortplanks. :-)

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: Using environment variables to control long running tests

[Andrew Savige]

Randal L. Schwartz wrote:
> All I'm asking for in my quest is to keep the "install health check"
> tests down to a minute or two.  Remember that CPAN.pm insists that
> "make test" works before I can install.

I agree. Modules should commonly have long-running exhaustive tests
and/or stress tests. These tests are primarily for the benefit of
module maintainers; it's unreasonable to expect users to wait hours
for them to run. I think this is a common enough scenario to warrant
defining a standard way to do it -- which may be as simple as
agreeing on a standard name of a single environment variable
(I don't have enough experience to suggest a good name, however).

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: Using environment variables to control long running tests

[Andrew Savige]

Randal L. Schwartz wrote:
> No, it doesn't need a standard name.  The default should be the
> "install tests" without any special envar.  The developer can select
> their own set of envars to control various *longer* tests.

While a standard name is not required, having one would allow
automated CPAN test harnesses to run the longer tests.

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Using environment variables to control long running tests

[Andrew Savige]

Merlyn's use perl journal of April 29, 2003:
 http://use.perl.org/~merlyn/journal/
suggests using environment variables to spare the installer of
CPAN modules a long wait while running 'make test'. One reply
noted that DBD, HTML::Mason and other modules already do this.

Is there a standard name/s for such environment variables?
If not, is it desirable to standardise on at least one environment
variable name to control whether long tests are run?
Is anyone using alternative mechanisms to solve this problem?

/-\


http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

Re: [ Memory ] Re: Thought

[Andrew . Savige]

En op 5 oktober 2002 sprak Ann Barcomb:
> I hadn't looked in to how I could solve my question, and because
> it was given to me as a low priority task, I wasn't sure I was going
> to have a chance to either.  But you can count me as someone who will
> be very happy about the module :)

I noticed CPAN Proc::ProcessTable, which is worth a look, if only
to peer at the code and note the portability, er, challenges.
If the module that reports memory usage itself consumes significant
memory, that is a pest when trying to figure out how much memory
a Perl data structure is using. That is the nice thing about
Merijn's original sbrk() -- it is lightweight in the extreme.

/-\

Re: [ Memory ] Re: Thought

[Andrew . Savige]

En op 4 oktober 2002 sprak Michael G Schwern:
> The problem is when you put those two next to each other, one
> promising a friendly interface, one a bare-metal interface,
> it confuses the intent of the module. Is it for Joe End User
> or is it for Joe Core Hacker?

A module to report memory usage of any process (not just yourself),
is do-able: on some Unices you might trample thru /proc file system;
on Windows, the Performance Registry, and so on. But has it been done?
I searched CPAN and could not find such a module. And, if you were to
write such a module, what should you call it?

I would find such a module generally useful in that it gives a sysadmin
view of a Perl process (is it a memory pig compared to a Java process,
say). Admittedly, such a module may not be very useful in figuring out
how much memory a perl data structure is using, but I don't see how
sbrk is any better in that regard. For a more fine-grained view, you
need hooks into Perl internals (such as the Perl malloc).

/-\

Getting started with CPANPLUS/cpantest on Windows

[Andrew . Savige]

I was very impressed with Autrijus' article on www.cpan.org.
This has inspired me to try to do some Windows CPAN testing.
Please bear in mind that I am very new to all this.

The cpantest script from http://testers.cpan.org does not seem
to work out-of-the-box on Windows. It failed for me with a:
  Died at G:/Perl/site/lib/Mail/Mailer.pm line 291
It seems to be trying to run mail or sendmail or some other
exe which does not exist on Windows.

Anyway, I then tried installing CPANPLUS 0.033. Success!
When I tried:
  i Acme::Buffy
it worked very nicely. But when I try to send my test report
from cpanp shell, it hangs for about 5 minutes when running
cpantest. The mail does not get sent but no error is reported.

In desperation, I hack cpantest.bat (I think this was supplied
with CPANPLUS) and change the smtp Server from onion.perl.org
to my local SMTP server. This stops the 5-minute hang, but I
am now defeated by internal security and suffer a "Relaying
is prohibited" message.

Why did CPANPLUS need to hack cpantest script with this smtp
Server onion.perl.org business? Does the standard cpantest
script not work out-of-the-box on Windows? If so, why is
there not a note to that effect on http://testers.cpan.org?

Anyway, I now know how to defeat this "Relaying is prohibited"
via a custom Perl script that uses SMTP authentication.
So I should now be able to report test results via CPANPLUS
shell by using a custom version of cpantest script with
custom SMTP authentication. Before I do that, however,
I am interested to hear if anyone else has had similar
problems, and how they solved them.

/-\ndrew