John Plocher wrote:
> James Carlson wrote:
>> We used to think that incompatible changes to
>> interfaces found in/on Solaris were always Bad,
>> and that evolutionary stability was always a
>> Good Thing.
>
> Yes, stability is important, but it is not the only thing that
> matters. Under your words, I hear "everything will be OK if we can
> achieve interface stability; if we can't, all we can do is punt" (and,
> yes, I am taking your statements to an unwarranted extreme :-)
Lack of stability implies that someone -- somewhere -- is constantly
rewriting his dependent software so that it continues to work over time,
despite the shifting sands under his feet. Or it means that nobody
actually uses the exposed volatile parts so there's no dependency to
worry about. Or it means that the people who use it just don't care
that it sometimes falls apart -- though I frankly don't know of any
cases like that.
As I said before, those sorts of schemes do indeed work. It likely does
in fact mean something like "Volatile," at least for the portions of the
software that behave in that way. And I have no problem whatsoever
using that attribute when it really does fit the situation.
It doesn't, though, match up with the way most software really behaves,
and applying it at random or based on external considerations (such as
authorship or "control") is quite counter-productive.
The subset that behaves this way is small, and I suspect much of it --
like some of the GNOME libraries -- is more properly regarded as
Consolidation Private than as Volatile. It's rapidly moving because
it's internal to a larger project that is itself actively evolving and
part of a coherent community. Those outside such a community shouldn't
consider using it, but those inside have no trouble keeping up.
> I think we need to admit up front that the traditional Sun Interface
> Taxonomy simply does not work for most "pass it on" FOSS projects -
> the only pragmatic answer to "what does this project team promise to
> commit to" is "nothing", because no matter what the past track record
> is from the upstream provider, they have no choice but to follow their
> lead going forward.
Don't get so hung up on "promises." What we're really after here is
(regardless of origin) a prediction about the future of the software
itself -- i.e., how often does it change, in what ways, and on what
sorts of boundaries? It turns out that people inside Sun are no better
or worse at those predictions than those outside, so "promises," though
nice, are not really the right thing to look for. (How many times has
Sun said "Stable" or "Committed" and then later nuked it? How does that
make those non-FOSS things any better or worse than FOSS?)
Instead, what we're after is _intent_, which can be expressed in many
ways, some explicit and some implicit. An example of an explicit
statement is (of course) documentation saying what changes and when. A
bit of mature FOSS has such material, though much does not. An example
of an implicit statement is historical info -- "they wanted to change X,
but they didn't, because they knew several other projects were using it"
or "this part of the library has been the same since 1995" or even "if
this were to change incompatibly, then this other project would break,
and there's nobody maintaining that other project anymore."
Another part of implicit stability is design: things that look
half-finished, or like they're intended for use by humans alone, are
probably less than stable. Those that look polished are probably more so.
All of that -- including the guesswork -- is relevant, and establishes
what we're after, which is whether project teams can establish
dependencies without _substantial_ fear of failure.
Obviously, all software breaks, and has bugs. Bad things will happen.
The intent is only to keep bad things from happening due to simple
ignorance, not to achieve some sort of magical perfection. That's not
going to happen, so don't think that "Committed" really means it'll
never ever change.
> The traditional Interface Taxonomy assumes that we have the option of
> not passing on (or re-engineering or deferring or ...) incompatible
> changes so we can live up to our commitments of stability.
Those aren't the only options available for ordinary FOSS projects
integrating using higher stability levels in the traditional Interface
Taxonomy. Among the others are:
- Mark some things as stable and others as not, depending on the
situation. It's often the case that there are things that are
more stable than others, and that someone who understands the
software can separate these out.
- Get involved with the upstream community. If the component is
so important to you that you're going to depend on it, then don't
be shy. Suggest alternatives to incompatible changes and help
plan out the ones that must happen so that they're less onerous.
- Provide multiple versions, if that happens to be feasible with the
component in question, and if it works for the consumers.
- Figure out some way to sequence the release of changes, such that
consumers are notified and are able to make the switch when the
incompatible lumps come down the pipeline. Timing is everything.
(That is to say, "contracts.")
- Create a compatibility layer. Sometimes this is a simple answer
and sometimes it's infeasible.
- If the situation warrants, then fork the project if it gets in a
bad state. Obviously, forking is a serious prospect, but sometimes
it makes sense. Often it does not, but if you're going to be
serious about any sort of engineering, then you can't arbitrarily
toss viable answers off the table a priori.
Obviously, a combination of approaches will be needed for some parts.
There are likely other solutions that are situation-specific or that
don't come to mind at the moment.
There are many ways to solve these sorts of problems, but pretending
that they don't exist or that FOSS is a special new world where software
engineering no longer matters is just not one of them. The issues
described by the Interface and Release Taxonomies aren't abstract
notions cooked up in a back room somewhere; they're actually a
reflection of Mother Nature, as she shows up in software. If I call my
function foo() today, and you call it, then you're going to be sad if I
call it bar() tomorrow. No amount of appealing to the magic of open
source will change the output of ld.so.1.
> In today's
> environment, we don't have that option, and furthermore, nobody but us
> expects us to use those options in the first place. If GDB does
> something we ourselves wouldn't have done, we still have to ship the
> new version on to our users - or those same users will go elsewhere to
> get it. They (and we) don't have the option of doing without it, and
> we don't have the resources to re-engineer it.
Nobody suggested either was a necessary outcome.
> 3) We must allow for other versions of the thing to be installed, in
> additional locations if need be, even if those versions are
> incompatible with our stacks or other components. However it is done,
> installation of another version of something must not negatively
> impact existing installed silos.
I think you're missing something important here. *EVERY* solution to
the problem has a downside -- either by adding bloat, or risk, or
expense, or by actually being infeasible to make work. Yes, allowing
multiple installed versions to work around an incompatibility is one of
the tools in the toolbox, but we cannot pretend that it's the only one,
or even that it's the best one.
It's not.
> To me, this sounds a lot like the JDK/JRE work that jek3 did - and the
> work that was going on in WSARC around architecting multiple coherent
> silos, with symlinks from /usr/bin and application ld.so linkage
> dependencies into one specific silo along with the ability to create
> and install alternative (potentially sparse) silos.
Again, there are architectural trade-offs to be made there. That's
reasonable in some particular situations, but not in others. If those
"silos" ever end up sharing things -- datasets, libraries,
configuration, or the like -- the problem becomes far more complex. If
there's a bug that affects all of them, then even more so.
What you're describing isn't at all new. It's not novel. And it's not
the panacea that I think you're making it out to be. It doesn't solve
the basic problem of managing dependencies between software components.
> Rather than having an architecture that only allows for a single
> global silo of "plays well with others" apps and libs, this moves us
> towards a world of multiple silos. Each silo would have an assumption
> of stability within the stack, and no promises outside itself. - sort
> of like multiple co-instralled WOS's...
Unless there's a superhuman being who writes all the software in one
silo, you still need to manage dependencies *within* each silo. I think
you've just moved the problem from one place to another ... and created
a few new scaling problems in the process.
Yes, it's hard to demand a single global version of each component.
It's a good thing over a long period of time, but it's certainly hard,
and perhaps *too* hard for some parts. That doesn't make it less of a
useful goal, or something that should be tossed aside lightly.
> This isn't easy - but it isn't impossible, either, given some
> leadership and a desire to make something that works. Rather than
> dumping a boil-the-ocean requirement on each and every FOSS project to
> acomplish by themselves, mostly in the dark, and without any role
> models, maybe we could nail down some architectural principles and
> offer some best practices... Please?
I think you're using the multiple-supported-versions thing as a cart to
drag around an architectural horse, but if you insist, I'll say that the
ARC isn't necessarily the right place to _solve_ that problem. Instead,
the community groups that would be most affected by the requirements
(say, Install and Packaging) should figure out what to do with the
solutions that someone who cares about this angle (say, *you*) creates.
Once there are solutions, bring 'em to the ARC for review. That's what
the "R" part of ARC is about.
The "Architecture Creating Council" is two doors down on the left. ;-}
--
James Carlson 42.703N 71.076W <carlsonj at workingcode.com>
