James Carlson wrote: > Roland Mainz writes: > > Mike Kupfer wrote: > > > >>>>> "Craig" == Craig Mohrman <Craig.Mohrman at eng.sun.com> writes: > > > > > > Craig> So it really boils down to these tests failing and then causing a > > > Craig> subsequent exit in make. > > > > > > Are you planning to putback with the tests enabled as part of the > > > standard build? I know Danek has voiced opposition to this in the past. > > > > Known problem... I'd like to convince him somehow - but that depends on > > whether we can get the test suite to run "stable" ... > > We still intentionally put test cases into a separate gate. This is > to allow for independent regression tests, and for tests that take a > long time to run or (in some cases, obviously not this one) require > special hardware or configuration.
The test suite is quite fast (however there is one catch: The test run in usr/src/cmd/ksh/Makefile.com calls the test suite for various locales to catch multibyte handing bugs - on a system with all locales installed 18 runs are done, covering major locales to ensure ksh93 works thee (I guess the japanese customers or those at *.gov.cn who like GB18030 will NOT be happy about a shell which can't deal with such a locale... =:-) )) and it does not require a specialised test environment. And I am not happy about the idea to disable the test suite run by default (but would accept gatekeepers decision if they wish to have it disabled) since the test suite is the KEY for the success of this project (or better: the test suite is, not neccesarily running it with each build). It caught many many bugs in the whole project development cycle and should be used in the future to make sure that no incompatibility between the ksh93 shipped with OS/Net and the versions on other operating systems creps in. This was the issue ruined the current Solaris /usr/bin/ksh and I really don't like the idea that this may happen to ksh93 in Solaris, too. > The test code doesn't belong in ON, so, yes, you are going to get > opposition to a plan that includes integrating it. It'd be far better > if the folks helping you with the integration could segregate the test > code out and put it back to one of the test gates instead. How can the ksh93 test suite then be used outside Sun ? The test gate is AFAIK not public, rendering the ksh93 test suite inaccessible. Beyond that it is not "easily" sepeateable from the remaining code. The physical files could be moved, however the test suite must be exactly in sync with the ksh93/libshell sources or you get some new and interesting "heisenbugs", rendering the purpose of the test suite more or less useless (and I have no information whether or if there is a syncronisation process between the test gate and Nevada (based on earlier complaints about syncronisation problems I doubt that...)). BTW: The test suite is shipped in usr/demo/ksh/tests/ for now (the pro and cons were discussed earlier in the list) as outlined in the original ARC case. > If it's not neatly separable, then you might get some to grudgingly go > along, but only if it's very fast and perfectly reliable. I doubt > that Danek or anyone else on the gate team would tolerate something > that causes builds to fail intermittently. It will only kick the build if there is major problem with ksh93 (e.g. I've disabled some of the non-important bugs and added detailed comments for each known issue in usr/src/cmd/ksh/Makefile.com). BTW: The problem Craig found has been fixed (it was some kind of bug triggered under high stress) and the fix gets included in the ksh93-integration prototype tree with the next update from the upstream sources. ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)
