named tests, do_all_tests(), use autotest/selftest/testpod?

[barries]

On Thu, Feb 15, 2001 at 11:37:12AM -0500, [EMAIL PROTECTED] wrote:
> Why have a static count?  Rocco (I think) made the point that
> sometimes tests will just mysteriously never run!  And if those tests
> came at the end of the run (or the test program aborted but exited
> normally for some reason) Test::Harness might think it was a
> successful run.  Tests should be paranoid and inflexible by design.

Just to be clear: the list-o-test-subs example I sent emitted a test
count before tests are run without having to manually maintain it.
The new todo() make that an even more useful idiom for me, many thanks!

I'd appreciate hearing of any shortcomings people see in it.  The major
ones I've had are the lack of todo() and the need to occasionally have
fake "tests" that do setup or cleanup functions.  Numbers are chap,
though :).

> Test::Harness aware of it (it currently simply ignores it) for better
> failed test reports (I'd rather it reported by name than by number) as
> well as add it to Test.pm's interface.

Very cool, named tests will be useful to me.  I already use the comment
field of ok() a lot for this.

<ideation quality="to be determined">

What do folks think of adding something like the following to Test.pm:

   my $tname ; ## ok(), skip(), todo() can get the test name from here
   my $is_todo ; ## ok() could look at this and adjust it's output

   sub do_all_tests {
      plan tests => scalar( grep ref eq "CODE", @_ ) ;

      while ( @_ ) {
         $tname = ref $_[0] eq "CODE" ? undef : shift ;
         $is_todo = defined $tname && $tname eq "todo" ;
         $name = undef if $is_todo ;
         if ( defined $tname && $tname eq "skip" ) {
            skip( 1, 0, 0 ) ;
            shift ;
            next ;
         }
         shift->( $ntest ) ;
      }
   }

This would make for very succinct easy to maintain test suites, if your
test suite is simple enough:

   use Test qw( do_tests ) ;

   do_all_tests(
      frobnitz => sub {
         ok( 1 ) ;
      },
      skip => sub {
         ok( 1 ) ;
      },
      todo => sub {
         ok( 1 ) ;
      },
   ) ;

#####################################

Or even a filtering mode, like so:

   use testfilter ;

   sub frobnitz_test {
      ok( 1 ) ;
   }

   sub other_non_test_sub { ## for example }
   }

   sub flitzer_todo_test {
      ok( 0 ) ;
   }

   sub foofer_test {
   }

   sub test {  ## Unnumbered test, the filter would name mangle for
               ## uniqueness
   }

#####################################

If we wanted to embed tests in module files, perhaps a

   use selftest ;

that would look for $ENF{SELFTEST} or for @ARGV to contain '--selftest'
and run the test suite if found.  A

   use selftest qw( -nobloat ) ;

could also tell it to try to filter out test subs if neither SELFTEST or
--selftest is set.

#####################################

Hmmm, if we wanted to worry pod-people a bit, perhaps:

   use testpod ;   ## A special variant of POD

   =begin test frobnitz

      ok( 1 ) ;

   =end test

   =begin todo 

      ok( 0 ) ;

   =end todo

<!-- ideation is open-ended -->

- Barrie