With apologies for cross-posting. Dear all,
I have copied below the text of a blog post* I wrote a few minutes ago, because it addresses an issue in Debian and Debian-derived distros that I've encountered several times, and which no doubt many people encounter frequently. It's an issue which seems, to judge by forum posts and mailing list requests, to generate a fair amount of traffic and expenditure of effort, but which has not really been tackled in a co-ordinated way as far as I am aware. Consequently, I've written this piece (which is intended for a general audience) in order to raise awareness of the issue, and to seek a co-ordinated effort to tackle it. Yours respectfully, Sam Kuper *URL: http://www.sampablokuper.com/blog/2008/11/05/when-stability-is-pointless/ When stability is pointless =========================== Many Linux distributions (and other software environments too) use package managers to facilitate the installation, upgrading and uninstallation of software packages as needed. At least, that's the idea. Why have package managers? -------------------------- Are package managers necessary? Well, no. One way of managing software is simply to install individual software programs/libraries as needed, and allow each item to handle its own updating or uninstallation (or even just leave that to the user to do manually). That's pretty much how Windows handles things. It works OK if it isn't crucial that your programs aren't able to communicate with each other beyond basic, operating system level mechanisms like cut-and-paste. If your programs depend on each other, however, you'd be in trouble if you removed a piece of software that another piece depended on, or installed a piece without installing its dependencies, etc. Linux distributions usually have many pieces of software that are inter-dependent. A package manager can keep track of those dependencies and can, for instance, inform a user about to uninstall a piece of software of which other packages will be affected by this. That's important, because otherwise the user might accidentally disable something crucial. Package managers can also make upgrading software, to take onboard the latest security patches, trivial, requiring only one or two commands in order to automatically upgrade all the packages on an entire system for which upgrades are available. I haven't investigated the historical origins of package management, so I can't claim to know the original reason why package managers came to exist. But I can state from experience that package managers, when they work well, make life easier than it would otherwise be for system administrators running Linux systems with interdependent packages. Stability --------- Package managers facilitate stability. Given the foregoing, this should come as no surprise. Yet in the context of many Linux distributions, stability means something more specific than the colloquial "reliable, consistent" sort of meaning that the term normally implies. Stability, in the context of these distributions, means "unchanging, except with regard to security". One such distribution is Debian. Through the magic of package managers, Debian maintainers maintain multiple packages of each original software item they want to make available to users. The reason for the multiple packages is normally to achieve stability. The stability is achieved by packaging a version of the original software that has been tested sufficiently to warrant confidence in its security: it is not known to compromise systems on which it runs (except in cases where it may do so by explicit design). This "stable" package is then offered to users for, typcially, many years, and is not altered in any way except if a security flaw is discovered in it. Only if a security is found will the package be modified: to patch the flaw. Meanwhile, the maintainer will keep an eye on new versions of the original software. When the maintainer thinks a new version is worth packaging, she may package it as an "unstable" package to begin with, and then subsequently a "testing" package. If, as the release of a new version of Debian is approaching, the "testing" package meets sufficient criteria indicating its suitability for being regarded as "stable", then it, too, is marked as "stable" for that new release of Debian, and is thereafter treated as described above. As such, Debian may have multiple packages for the same piece of software: "stable" packages for current and old versions of Debian, and "unstable" and/or "testing" packages. Usually, each of these packages will be based upon a different version of the original software. In a hypothetical case, the "stable" package for the current Debian stable release might contain version 3.1 of the original software (perhaps with security patches, as described above); the previous Debian stable release might have packaged version 1.0 of the original software. The current "testing" version of Debian might package version 3.9 of the original software, and the "unstable" version of Debian might package version 4.2 of the original software. But… ---- Sometimes, stability lets you down. My perception is that the greatest problems with the system of "stability" practised by Debian and other Linux communities arise when the upstream developer has not maintained the documentation for earlier versions of the software he has written. This leads to a disconnect between users reliant on package managemers and interested in dependability, and developers interested in making software that is faster, more fully featured, or otherwise different from the earlier versions of their software. An example ---------- Here is my scenario. I have a server running Ubuntu 8.04 LTS: a "stable", recent release of a Debian-based Linux distribution. I wish to install a security-related program called "psad" (short for "Port Scan Attack Detector) on that server. However, the stable package of psad for Ubuntu 8.04 turns out to house version 2.1 of psad. That wouldn't bother me, except that… I can't set it up! The reason I'm having difficulty setting it up is that the documentation on installing psad refer not to version 2.1 but to version 2.1.4, which requires setting up differently to 2.1. The developer's recommendation is that I upgrade to a more recent version, but two questions arise: * how? * why? The how question has several possible answers. I could forego package management altogether for psad and install version 2.1.4 straight from the source code. But that would entail the problems outlined above: the problems package management solves. Alternatively, I could perhaps configure my server to attempt to use an "ubstable" or "testing" Ubuntu psad package in place of the stable one for 8.04. But why? Why should I have to do this, if the "stable" package is good enough that it was chosen for "stable" release? This is where the disconnect I alluded to above becomes apparent. Developers may not always explicitly deprecate their old code, but they nearly always do deprecate it anyway. The response of Michael Rash, the psad developer, was characteristic of this. Yet as far as users of stable distributions are concerned, that same sofware - the oldish versions of the program - has not been deprecated: it's still in the latest "stable" repositories. There's a contradiction here. Unfortunately, it's an unresolved contradiction, as far as I can tell. I encountered it previously with rkhunter, also a piece of security software I wanted to install onto a software running a "stable" version of Linux. In that case, it was even worse: the upstream developer had explicitly deprecated the version that was in the current (read: un-deprecated) "stable" package, and was entirely unwilling to support it. Clarification ------------- Despite these problems, I'm still very grateful to - and have a lot of respect for - the developers and maintainers of the software I use (including the software I've mentioned above). Michael Rash's software is innovative and free (in both senses of the word), and he was certainly under no formal obligation to reply to my queries. Furthermore, I have learned a good deal from reading his book. Equally, rkhunter *can* be a useful tool even if you aren't using the very latest version of it, and the maintainers who answered my queries also did so voluntarily and courteously. Yet the problem remains: when developers deprecate software that maintainers have not deprecated, users are left in the lurch with software they can't use, can't get much support for, or can't find documetation for. Solution -------- In some ways, I'm still very much a newcomer to the free software world. It's only in the last 2-3 years that I've really begun feeding back to the communities whose tools I use, and I have not become a developer or maintainer myself. As such, I'm not yet in a position to implement change to, say, Debian's release policy. Yet, I do have some ideas for solving the problem I've outlined in this piece: * Raise awareness of the problem. * Propose that maintainers open a dialogue with the developers whose software they package to request that developers continue to support, or at least maintain documentation for, old versions of their software until the maintainers have deprecated it. * Ask the maintainers to shoulder this burden where the developers cannot or will not do so. * Seek further suggestions from the community. In the first instance, I'm going to attempt to make some progress towards realising these solutions by posting a link to this piece to the Debian and Ubuntu mailing lists. Then I'll leave it in the community's hands unless I find I have any more to add.