[parrot-porters and p6l have been removed from this thread, as discussion of
testing tools is not directly related to those lists]
On 8/11/06, Agent Zhang <[EMAIL PROTECTED]> wrote:
On 8/12/06, jerry gay <[EMAIL PROTECTED]> wrote:
this last testing rule i mentioned becomes somewhat problematic when
> designing a test suite for use by multiple perl6 implementations. it
> is obvious that some implementations of perl6 will not pass all tests,
> and will need to mark certain tests as todo() and skip() in order to
> run the suite successfully.
>
IIRC, for the p6-on-p5 implementation, clkao has implemented some kind
of magic (in Makefile.PL?) to copying passing test files from Pugs
test suit to v6-alpha's own directory. Of course, it's far from ideal,
but i think it's a good enough solution to the current (early) stage
of development.
yes, it's a good step, but it's not ideal. i'm hoping discussion with those
familiar with test suite design, test tools, and test libraries can help us
find what we want, find the tools we can use to help, and develop new tools
we need in order to get results.
I've added the plan for the neutral todo mechanism to Pugs' TASKS
file, getting help from many others on #perl6. The new todo marks look
like this:
todo :pugs<6.28.0>, :p6p5<0.110>, :parrot<1.00>;
is $got, $expected; # affected by the preceding todo call
or equivalently,
is $got, $expected, todo :pugs<6.2.13>, :p6p5<0.110>;
this is a proposed api change to the test library for perl6. i expect that
since these are the first tests written in perl6, they'll be used my many as
a starting point for writing their own tests in the future. i'd like to hear
from the perl-qa folks on what they think about supporting this feature: is
this an acceptable proposal, or is there some other (better) way?
~jerry
