Nicolas Williams wrote:
> On Tue, Sep 01, 2009 at 10:46:55AM -0700, Garrett D'Amore wrote:
>
>> Nicolas Williams wrote:
>>
>>> Actually, I wasn't. The points of my two posts in this sub-thread were:
>>>
>>> - The ARC should stay out of design issues (in this case the design
>>> issue of using Tcl to configure shell environment modules);
>>>
>>>
>> Why? If the design imposes upon the consumers, then ARC should be
>>
>
> Because design review is not normally in scope for the ARC.
>
Where the design turns into interfaces, or has reaching ramifications
that affect system architecture, it *is* in scope. The boundaries are
no where near as cut and dried as you seem to believe. Nor should they
be, IMO.
>
>> concerned. As a specific case here, Tcl imposes two things on consumers:
>>
>> 1) a scripting syntax/language heretofore not used for core system bits
>>
>
> If from the user's point of view it's just a few extra curly braces,
> then who cares?
>
Can the user access other bits of Tcl syntax? Are there possibly
unexpected behaviors here? And, ultimately, does it boil down to just a
few extra curly braces?
Inventing new configuration syntaxes is something we usually try to
discourage. If this isn't 'invention', but inheritance from Tcl, then
we need to look at any possible ramifications. If the intent is not to
impose Tcl on our audience, but instead to have a limited config syntax
which just happens to be parseable as Tcl, that's fine, but then the
syntax still needs review.
>
>> 2) a dependency on the Tcl packages itself.
>>
>
> The dependency on Tcl is something to be documented. It is not remotely
> a problem, particularly given the stability of Tcl.
>
The issue is not stability, the issue is introducing a new dependency on
something that might not be installed by default. This ultimately can
lead to other kinds of poor choices, which is why ask for this
information to be reviewed.
For example, would you like to see ifconfig modified to have a
dependency on GTK+? There are lots of handy functions in GTK+ (ignoring
licensing issues for the moment), so it might not seem to some
developers to be worth architectural review. But I think most people on
this list would agree that having such a dependency would ultimately be
a poor choice, since it would require GTK+ (and possibly other heavy
weight items) in environments where they are not needed. (There's also
that whole cross consolidation thing...)
The point is, dependencies *do* warrant at least looking at, at least
when they're not super obvious. This one is not super obvious to me as
either a good or bad idea, but instead deserves to be looked at.
Again, I'd give the project team a free ride for FOSS compatibility if
this wasn't being proposed as a building block in our larger systems
architecture.
>
>> Both of those considerations fall, IMO, well within ARC oversight.
>>
>
> I don't agree.
>
That's your prerogative I suppose.
>
>>> - /contrib is a not good place for Sun projects to integrate into --
>>> aim for /dev & /release if you can;
>>>
>> Why? I think /contrib is perfectly reasonable for projects that just
>> want to enable something, and don't want to make support guarantees. We
>> have no other way to deliver 'caveat emptor' bits that I can see. If
>> /contrib works for the rest of the community, why not also Sun?
>>
>
> Precisely because there's no commitment to keep anything in /contrib
> working. That's fine for: a) random third parties, b) as a stepping
> stone to /dev and /release (e.g., for alpha and beta testing purposes).
>
But its that lack of commitment that might be just the reason someone
inside Sun delivers something to /contrib. As an example, imagine a
service engineer decides he needs some tool to help out a customer. He
builds it and sets it up in /contrib. What's wrong with that? Should
he be forbidden from doing this just because he's a Sun employee?
>
>>> - What is the architectural status of /contrib? IMO: none. /contrib
>>> is for third party contributions that are not yet ready to integrate
>>> into /dev & /release;
>>>
>> True. Or things that don't need/want to integrate into /dev or
>> /release. For example, joe random software package that someone sets
>> up, but doesn't want to commit to sustaining forever.
>>
>
> Only in so far as it may be the only reasonable option if the ARC
> rejects a case. The case that led to this sub-thread should not be
> rejected.
>
When this looked like it was just delivering joe random software without
a sustaining commitment, and not as part of a larger project, /contrib
looked reasonable. Since then its been clear that the project team
needs something beyond /contrib, because they want it to be part of
their underlying architecture.
>
>>> - What to do about FOSS backwards compatibility breakage? (This was in
>>> my follow-up.) My suggestion: documented stability levels can and
>>> should be used to set expectations, but we need to be much more
>>> flexible with respect to backwards compatibility, and for the reasons
>>> that you pointed out (i.e., we're in violent agreement).
>>>
>> Yes. I'd like to point out that FOSS gets a special exception when it
>> is delivered strictly for FOSS compatibility. I think that exception
>> goes away when we start using it as a building block for our own
>> architecture pieces. I.e., its bad if project team A delivers a new
>> version of a package that breaks project team B.
>>
>
> I don't think that's true _now_, except, perhaps, as a matter of ARC
> discretion. I'm arguing that the interface and/or the release binding
> taxonomies need to explicitly state this.
>
ARC discretion does exist to a certain degree. We are intelligent (well
at least the other members are :-) folks at ARC, and can make decisions
without being hamstrung by a policy that forbids us from taking what is
clearly the right action. (Such as allowing sensible or non-risky
interface breakage.)
>
>>> Specifically, IMO: we should have very frequent Minor releases, or at
>>> least treat releases as "Minor for some things, Mico/Patch for most
>>> things", or otherwise allow for out of cycle breaks, plus suitable
>>> warnings on breaks.
>>>
>> This topic is not a matter for ARC to decide. Release timelines are
>> decided by the PAC and business teams. ARC can however relax some of
>> the breakage rules. We've already widely done that for a number of
>> items in Nevada/OpenSolaris.
>>
>
> The notion of "Minor for some things, Mico/Patch for most things" is not
> in the release binding taxonomy, IIRC. Suppose the PAC proposes that.
> Then what? IMO: the release binding taxonomy should have to be updated,
> for it'd be silly for the ARC to deny the PAC on such a matter. But the
> ARC could update that taxonomy now -- we already know that OpenSolaris
> will be "Major for some things", so what's the ARC waiting for?
>
I'm not following you here? You want some clarifications on what the
next Solaris release will be? *I* don't know. We've been following the
only guidance that Sr. management has given us so far, which is some
major binding style breakage is permissible, but minor for pretty much
everything else.
Its interesting that the OpenSolaris release are not covered by any
interface taxonomy at present -- for all intents and purposes at ARC
they don't even exist. This is a bug, but it is made worse by the fact
that the various management and product teams have heretofore been
unwilling to give us any more guidance on this; this is not something
ARC can declare and expect everyone to follow.
Otherwise the OpenSolaris releases would have been far better covered by
ARC a long time ago.
>
>>> Alt. description: keep existing ARC interface taxonomy, but make it
>>> easier to make out of cycle breaks by adding to the release binding
>>> taxonomy.
>>>
>> I'm not sure how that will help. Release taxonomies need to follow user
>> expectations. Otherwise they become useless. (By that I mean, a
>> "minor" release needs to be associated with a real versioned product, so
>> end users know where the boundaries are.)
>>
>
> Expectations are set according to past track record, but track record is
> not exactly a failsafe predictor when it comes to third parties. _We_
> can make strong interface commitments, but we can't really do it on
> behalf of others. Yet labeling all FOSS as Volatile is hardly helpful.
>
So project teams do the best they can... they assess what the upstream
is likely to observe, and assign an appropriate commitment level.
You're right in that its not failsafe, but neither do we automatically
have to immediately integrate a change that breaks compatibility.
It is sensible to review such changes, if only to learn whether the
impact of the change is severe enough to warrant either delaying its
integration or coming up with some other mitigation plan (such as
parallel releases ala perl.)
Looking for a silver bullet that automatically gives a free ride to
sources that come from elsewhere seems like a recipe for anarchy, and
isn't a plan I support.
>
>> We used to have better ways of expressing this. Like "Evolving". But
>> folks decided that the fine grained stability levels were not useful.
>>
>
> I don't agree.
>
That's fine. I doubt you're alone; heck I liked the more fine-grained
stability levels. But that decision has been made, and recently enough
that I don't think its productive to go back and question it.
>
>> I'd really like to have something like "External", which would mean that
>> Sun (or the Solaris org) makes no guarantees about interface stability,
>> we just follow the upstream. Caveat emptor sort of thing.
>>
>
> Yes, I agree with this. We need an adjective that indicates that
> something is of third party origin and that advertised interface
> stability is advisory only, not necessarily reliable.
>
It would be nice indeed.
>
>>> The vast majority of the time upstream will manage to stay backwards
>>> compatible, so the majority of the time Committed and Uncommitted will
>>> be much more useful than Volatile.
>>>
>> It only takes one nasty breakage to be really annoying. And it depends
>> on your upstream. Some are pretty good at understanding binary
>> compatibility. Others, much less so. (OpenSSL, I'm looking squarely at
>> you.)
>>
>
> And yet what are we doing to keep OpenSSL compatible yet full-featured?
> Nothing. In OpenSolaris we just update it and _try_ to fix every
> consumer in OpenSolaris consolidations.
>
Actually, we try not to export its interfaces to third parties. Which
is hard... I know. But at least people aren't reading man pages and
getting false information about stability that way.
OpenSSL is a classic example where some mitigation strategy was
necessary. We might not have liked how it has played out, but I daresay
its been far better than just letting new versions integrate willy nilly.
>
>>>> ... The question isn't "should we provide a new
>>>> version?", but rather "how can we provide a *set* of versions that can
>>>> be used to meet the various needs articulated above?".
>>>>
>>>>
>>> Sometimes you get into DLL hell if you provide multiple versions. At
>>> least with direct binding that's less likely than it used to be.
>>>
>>> We might even need to deliver more than two versions of some FOSS.
>>>
>> This way generally lies madness. There are some specific counter
>> examples were it has worked out (multiple perl versions for example),
>> but doing this generally for shared libraries will cause huge kinds of
>> problems.
>>
>
> We will have these choices when the upstream community breaks
> compatibility:
>
> a) don't update;
> b) warn, update and break compatibility and oh well;
> c) deliver multiple versions, risk DLL hell;
> d) fund an effort to restore backwards compatibility and contribute the
> fixes back upstream;
> e) fund an effort to restore backwards compatibility and fork;
>
> The ideal choice is (d), but it's not realistic in many cases (we
> can't fund everything, and not all upstream communities would welcome
> such changes).
>
> (e) is a nightmare. (c) is great, if you can get away with it. (a) is
> not tolerable for long.
>
> (b) is only practical if our users will deal. To make that likely we
> should set expectations that stable-looking FOSS interfaces might break
> on release boundaries that normally Sun-owned code wouldn't, and warn
> loudly.
>
There are multiple situations here, and there is not just one right answer.
a) usually leads to one of the others eventually, or to just removing it
because it becomes stale.
b) sucks, but for *some* cases it is the best choice. E.g. when a fix
is made to close a security bug, or when the breakage is not to a widely
used portion of the interface.
c) doesn't necessarily involve DLL hell. We do this with Apache (or we
used to at least), Perl, and even Java. It does require 'effort' though.
e) isn't that bad... can you say OpenSS vs. SunSSH? Sometimes the fork
is the right answer. I disagree with the notion that forks are
inherently Evil. They often are, but sometimes they are the least of
the Evils available.
>
>> I don't care what crapware we import for end-users to use at their own
>> risk; but if we ought to be *architecting* things so that the bits we
>> rely on and use are not subject to constant or unplanned breakage.
>>
>
> Those two statements are in conflict. Please resolve it :)
>
Huh? If we rely on stuff as a building block, then it deserves higher
level review/commitment. If its only put out there as courtesy for our
customers (for example the "unison" package), then I don't care as
much. (And in such cases, I contend, that /contrib might well be a
better choice.)
>
>> Put another way, IMO a project that is a "leaf project" (no dependents)
>> can do whatever it wants as long as it sets end-user expectations
>> appropriately.
>>
>
> A project that is a leaf today may not be tomorrow. I can see some app
> depending on Environment Modules; can't you?
>
Yes, I can. It would be nice if our commitment tables warned about this
risk, so that when a project imported such a bit that it triggered
closer review.
-- Garrett
