On Fri, Jan 23, 2009 at 08:01:14AM -0800, Dave Whipp wrote: > For example, I could conceive of a trait: > > ok foo, :broken<rakudo> > > which might downgrade the error to a warning on rakudo, but not on other > implementations.
On the surface that seems like a good idea, and pugs started out doing things this way, but we discovered that it's a Terrible Mistake to mix platform dependencies in with the notation of the actual test, which is why we now use the "fudge" preprocessor approach, where any platform-dependent cheating is listed on its own line and looks like a comment to other platforms. Plus it's very easy to measure whether you're passing the test or not--you just turn off all the fudging, which leaves all the annotations as mere comments. If you mix the notation in with the test, then the test harness has to explicitly ignore the notations both for other platforms and also for this platform when a complete validation is desired; whether that is being done correctly is more difficult to prove, and it opens up the test harness to potential accusations of perfidious cheating. With the current approach it's drop-dead easy to see whether or not the tests are cheating--either you're running "fudge" or you're not. All that being said, fudge is a preprocessor, and preprocessors are a form of evil, so I'd certainly be open to the actual parser doing the fudging during compilation if explicitly requested to do so. My main concern is that the fudging directives not be intermixed with the actual test, and that they not look like real code. Larry
