Roland Mainz writes:
> > Agreed on all those points (particularly about _not_ having end-users
> > "testing" the system,
>
> Erm, the majority of people who use Solaris Nevada are developers and it
> may be usefull to get their feedback. For example if the i18n-parts of
> libc are broken it may be helpfull to have the test suite around to
> narrow-down the problem.
That alone doesn't argue for ON integration of those test parts. (In
fact, if there are significant known bugs, that argues against
integration, but that's probably a different conversation.)
Unlike ordinary end users, developers can presumably do special
things, such as download additional software (packages or otherwise)
and compile parts that aren't delivered as objects. As end users have
no business "testing" the system, and we really do not want to
encourage them to think in that way, having tests as part of the
delivered system is suspect.
> > as that's likely not a good way to build
> > confidence that we're actually doing our jobs),
>
> Uhm... why do you give tools like "dtrace" or "truss" to the end-users ?
> :-) :-)
Because:
- They're useful to end-users to test their own applications. I
don't think we have any expectation that end-users are going to
plumb the depths of (say) VM and report back bugs to us.
- They're general-purpose observability tools.
What you're proposing here appears to be neither of those.
> > but a fair compromise
> > might be to let this in temporarily on the condition that it
> > eventually migrates over to the test consolidation -- once that
> > consolidation is actually open.
>
> Mhhh... the test suite is included in the ksh93 codebase and should
> remain there (unless there is a way to keep both codebases in sync and
> prevent users from running the wrong version of the test suite against
> ksh93) - moving the code to a seperate codebase is tricky as the test
> suite and the ksh93 version must be exactly in sync - it is IMO useless
> to run the shell against a different version of the test suite.
In my opinion, this is yet another reason why ksh is a poor match for
ON. That's not to say that it's a bad thing (at all!), but rather
that the development style doesn't match this consolidation.
There are a few *intentionally* unpackaged and undelivered test hooks
and features in the ON gate, but as a matter of cleanliness, we split
the bulk of testing out into a separate consolidation.
The fact that the ON test consolidation isn't available on
opensolaris.org yet, and thus (except for a handful of existing open
source test suites) Open Solaris participants can neither do all of
the testing that we do nor can they contribute to the tests is a big
problem. I see that it's on the road map, but it's a problem here.
So, while I'd certainly agree that integrating the source for the ksh
tests is an expedient solution for right now, and I might even agree
that packaging them for /usr/demo, though distasteful, might be a good
stop-gap effort for the team, I don't agree that it's a good long-term
solution.
When the test consolidation becomes available, the testing tools
should move where they belong.
--
James Carlson, KISS Network <james.d.carlson at sun.com>
Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
