--- Peter Van Eynde <[EMAIL PROTECTED]> wrote:
> On Monday 19 December 2005 22:02, C Y wrote: > > c) Have part of the install process be to run the full > > unit test for every library being installed on the target > > lisp. > > The problem with tests like this, and the reason why the sbcl > build script no longer runs the tests is that sometimes these > tests get stuck in endless loops. And then what you do? Actually, that might be addressible by proper handling of the test framework itself. How about the following: Build the test framework so that any one test does not run longer than X seconds, where X is either some sane default (e.g. 10 sec. per test max) - if it runs longer, the test is terminated and an error recorded. For tests that are expected to run longer, have a parameter which can be optionally passed to the framework on a per test basis telling it to let this test run no more than n seconds. The trick, of course, is how to make sure that the loop doesn't hinder the test being shut down after the time limit expires. Not sure offhand how to do that - first thought is to customize the eval routine to checkpoint things every 10 seconds and allow the check, but I don't know how to do it offhand. > In the past common-lisp-controller would rebuild all libraries > for all implementations and send root spam if it failed. > > Certain libraries failed to build for months without _any_ > bugreports. I concluded that these libraries are just not > used :-(. The other option, and maybe a simpler one, is that rather than just sending failure reports, have a once a week report published reporting both success and failure. So if a report doesn't come, one knows something is wrong. Getting fancy, there could be a webpage which reports a matrix of libraries and common lisp distributions, with a green box reporting success and other colors for various failure states like, say, 10 of 1000 tests failed, vs. total failure. (Like the cffi porting status matrix, for example.) That way, anybody can do an instant "state of support" check for any lisp/library. Maybe this could be a framework on common-lisp.net, or some other logical central setup. It might not even be needed to use just one testing tool - I know cl-utilities has defined some tests using one, and doubtless others are used in other places. All that need be done is to establish a way for each test package to report success or failure that is understandable. Some work would be needed to make things robust, and someone would have to keep watch over the testing routines in case something went wrong, but I'll bet it's doable. Cheers, CY __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ Gardeners mailing list [email protected] http://www.lispniks.com/mailman/listinfo/gardeners
