On 4 Apr 2007, at 18:59, Eric Wilhelm wrote:
# from Andy Armstrong
# on Wednesday 04 April 2007 05:58 am:
runtests t/*.t --select regression,docs
Would select only regression and documentation tests. That's just an
example plucked out of the ether of course - it wouldn't /have/ to
work like that.
Meaning that every test has to decide what sort it is and whether
or not
it should run?
In that case yes - but that was just an off-the-top-of-my-head
example of what you might want to use parameters for. Or ENV
variables as Schwern pointed out.
I added a concept of "test profiles" to Module::Build
for just this sort of thing. You make your gui tests be '.gt' files,
your author tests '.at' files, and your network tests '.nt' files.
Then run testgui, etc or testall for everything. Of course, this is
intended mostly to provide a mechanism for tests that don't run by
default (e.g. you have no access to the required data or your machine
environment needs to be configured just so.) I'm not sure if I'm
exactly happy with it, but it is a start. Perhaps the setup should be
turned into a yml file that could be utilized by runtests, etc.
I think I'd rather put my tests into subdirectories to group them -
but either way that only gets you a single taxonomy. In practice
you're quite likely to want tests to be classified in more than one way.
For running categories of tests which are also in the default set, I
suppose we would need something of a label. It would be nice to have
that natively supported in runtests. It would also eliminate code
from
the test files themselves.
Of course, if the desire is to run only part of a particular test
file,
I'm more inclined to say that the tests need to be divided out into
smaller pieces. Not sure how that plays with Test::Class (or rather,
whether Test::Class can learn to play nicely with the test-files
concept.)
I imagine something not far from the existing skip mechanism could be
used there. I'd like, for example, to be able to re-run just a single
failing test in a test script that might contain many tests.
--
Andy Armstrong, hexten.net
