On Fri, Jan 23, 2009 at 6:39 PM, Dave Whipp <d...@dave.whipp.name> wrote:
> A spec-test is (or should be) different from an ad-hoc test. I want to be
> able to say "test S09.237 passes on pugs but not on Rakudo" (perhaps with a
> nicer name). Unique identifiers allow comparisons of specific tests across
> multiple implementations, and over time. It is possible to derive IDs using
> line numbers (perhaps block-relative), but that's only a good idea if the
> test suite is reasonably stable (and it requires tool support).
>
That sounds useful on the surface but often turns out to be more difficult
to do than you might think. There are many cases where tests are performed
from within loops. Something like S09.237 may or may not be in a loop, may
be difficult to identify in files with many tests. This sort of test name
could be the test message output by Test.pm's verbose output, but it then
makes the verbose output virtually useless in that Test.pm could just keep
records of the test numbers instead. There can also be multiple tests per
single line of code, especially if provided as an adverb, such as :ok
Test labels seems like an aspect that is highly susceptible to bit-rot due
to the ever evolving nature. Given the multitude of things that can go wrong
trying to keep records, it might not be a good idea to focus on this.
Rather, it might be a good idea to have the language provide a base test and
a means to extend the test. This would allow for the tests previously
written to transparently change the back-end testing mechanism.
Here's a very crude example. Lets say that ok() is defined by the Core (and
thus the language):
multi sub ok(Bool $test, Str $msg) { if $test { say "ok $msg" } else { say
"not ok $msg" } }
Then let's say I don't want the default (psuedo)-tap test output, I could
redefine what ok() does:
multi sub ok(Bool $test) { say "A test has failed at some point somewhere"
if $test }
ok(?($x == 4), "no good has come of this"); #calls Core's ok()
ok(?($x == 2)); #calls my crappy ok()
That's just an example to show that the language could provide a basic
version that is extensible with various implementations and various
compilers such that I don't have to write constantly unique test names (or
poorly identified names) and still only have to write a test once.
-Jason "s1n" Switzer
