On Fri, Jan 23, 2009 at 6:39 PM, Dave Whipp <d...@dave.whipp.name> wrote:
> A spec-test is (or should be) different from an ad-hoc test. I want to be > able to say "test S09.237 passes on pugs but not on Rakudo" (perhaps with a > nicer name). Unique identifiers allow comparisons of specific tests across > multiple implementations, and over time. It is possible to derive IDs using > line numbers (perhaps block-relative), but that's only a good idea if the > test suite is reasonably stable (and it requires tool support). > That sounds useful on the surface but often turns out to be more difficult to do than you might think. There are many cases where tests are performed from within loops. Something like S09.237 may or may not be in a loop, may be difficult to identify in files with many tests. This sort of test name could be the test message output by Test.pm's verbose output, but it then makes the verbose output virtually useless in that Test.pm could just keep records of the test numbers instead. There can also be multiple tests per single line of code, especially if provided as an adverb, such as :ok Test labels seems like an aspect that is highly susceptible to bit-rot due to the ever evolving nature. Given the multitude of things that can go wrong trying to keep records, it might not be a good idea to focus on this. Rather, it might be a good idea to have the language provide a base test and a means to extend the test. This would allow for the tests previously written to transparently change the back-end testing mechanism. Here's a very crude example. Lets say that ok() is defined by the Core (and thus the language): multi sub ok(Bool $test, Str $msg) { if $test { say "ok $msg" } else { say "not ok $msg" } } Then let's say I don't want the default (psuedo)-tap test output, I could redefine what ok() does: multi sub ok(Bool $test) { say "A test has failed at some point somewhere" if $test } ok(?($x == 4), "no good has come of this"); #calls Core's ok() ok(?($x == 2)); #calls my crappy ok() That's just an example to show that the language could provide a basic version that is extensible with various implementations and various compilers such that I don't have to write constantly unique test names (or poorly identified names) and still only have to write a test once. -Jason "s1n" Switzer