The tests in mcs/tests are already partitioned into On Wed, 26 Jun 2002, Nick Drochak wrote:
> My goal is to get us down to 0 bugs and then introduce a "unit test > tinderbox" system that will automagically notify someone if their > commit breaks any unit tests. This is also on my radar - hence my patch to make the tests in mcs/errors and mcs/tests much more uniform. A tinderbox-like system would be way cool though. It's much more useful to have a test suite which runs at 100% rather than one which is expected to run at a lower success rate. For example, several of the tests in mcs/errors are 'expected' to fail because the compiler isn't perfect yet. This means I can't just run 'make test' and get a boolean ok/fail result - I have to manually go through the results and see if they were 'as expected'. You could address that by trying to fix all the outstanding issues in the compiler, or you could partition the existing tests into 'working' and 'nonworking' (like mcs/tests does) so that running 'make test' only runs the working ones. Therefore, you will always expect it to run at 100%. The non-working tests in mcs/errors should be moved to a 'test-broken' rule for example. Then, if you want to improve the compiler you can pick one of the tests from the 'test-broken:' rule, fix the problem, and move the test onto the main 'test:' rule. Also, unit tests should not require user interaction. There's a test in mcs/tests which pops up a box and requires you to click OK. That should be changed so the tests can run unattended. Oops, I'm being a bit verbose ... I'll just send a patch next time. ;-) Andrew - www.tardis.ed.ac.uk/~adb - _______________________________________________ Mono-list maillist - [EMAIL PROTECTED] http://lists.ximian.com/mailman/listinfo/mono-list
