Since we're explicitly not talking about tools here, I've removed tools-discuss from the CC. The cross-posting in this thread seems a bit out of hand, and I'm also unsure whether desktop-discuss is still interested.
On Fri, May 05, 2006 at 09:48:05PM +0100, Peter Tribble wrote: > It starts off with the initial problem that configure is often > completely incapable of making sensible decisions - or even running > to completion without failing itself. (As opposed to exiting because > it found something that needed to be fixed.) I'm unsure whether this > is a fundamntal flaw in the design, or whether it's always being > misused (or both). It's some of both - autoconf and automake have a bunch of macro packages that can perform standard checks, and they also allow developers to create their own tests using some combination of gm4 macros and arbitrary text to be interpreted by the shell or make. It's not at all uncommon for developers to encode environment-specific assumptions into both their use of generic checks and their local additions. This is how you can end up with configure scripts that require ksh or even bash, and makefiles that only work with GNU make. The single greatest design flaw in these tools is the assumption that the build environment and the runtime environment will be the same (that is, that every machine that will execute the resulting binaries will be either the build machine or a mirror image thereof). For example, while the default X library path checking in autoconf's libs.m4 is smart enough to know about /usr/openwin/lib, it also adds identical -L and -R options to the linker command line if libraries are found there. That's great for software built on the system it's going to run on, but it's wrong if you want to link against a proto area of which the contents will be found elsewhere at runtime. Since autoconf as shipped knows about -lsocket and /usr/openwin but not /usr/sfw, it's unclear whether the intent is truly to encode all of these environmental quirks into autotools, or simply to provide a framework in which the person compiling the software can use command-line and environment options to inform the build system. If the vision for autotools is that building software Just Works, then it's obvious that there's a long way to go. > Generally, assuming configure actually works, the problems boil > down to missing libraries or APIs. So off you go and suffer the > pain of getting all the dependent libraries built. (Or updated.) > How to fix that? Simply bundle more libraries, and keep the ones > we do supply up to date. Often the libraries that are needed are already present, and neither autoconf nor the user knows it. This is perhaps the most unfortunate aspect of /usr/sfw. Certainly, OpenSolaris doesn't include every library that's ever been written. I would be interested in ideas for distinguishing between libraries that should be a part of OpenSolaris proper and those that are best delivered by something like the companion common base contemplated on companion-discuss. > Then there are missing APIs. Sometimes the author is just careless > and uses a non-portable version of a portable api, and that isn't > too hard to fix. Sometimes, it would be easier to bite the bullet > and simply add the API. As an example, I filed 6421095 requesting > that strcasestr() be provided. Yes, and there are precedents for adding these APIs if they are widely used and sufficiently stable. The most recent example is mkstemps/mkdtemp[0], added by Casper Dik in February. > Then you have to fight libtool; if that fails there's no > effective way that I've yet found of fighting back. Indeed, libtool is a candidate for the worst piece of software still in use. Short of instituting the death penalty for using it, there's not much that can be done. Whatever version of libtool happens to be on a developer's system when he or she libtoolizes the build system ends up encoded in that distribution forever, so even fixing some of the bugs that make libtool such a nightmare (such as failing to build self-contained objects) will have little immediate impact. The more serious design flaws, such as the very existence of '.la' files, put the nail in the coffin of any effort to make this software useful. That's not to say libtool never had a purpose - during the transition from static to dynamic linking, it was worthwhile to have a tool that could cope. But today, every interesting OS uses dynamic linking, and the only remaining purpose of libtool is to store dependency information in a sea of '.la' files. That's a travesty, because ELF and other modern binary formats already provide standard ways to store that information in the binaries themselves. > My experience is also that software isn't necessarily any > easier to compile under Linux. If it is, it's because the > libraries and code are similar, and dependencies are more > likely to be met out of the box. But other than that it > wasn't any easier - sometimes harder, because I usually only > had the single (OS provided) working environment and was > stuck with it, whereas under Solaris I could switch between > several different environments to try and get better results. > I suspect that binary distribution is sufficently advanced > that ab-intio building of software by end users is getting > relatively rare under Linux. That's certainly true; GNU/Linux users with support contracts usually limit themselves to vendor-supplied packages, users of Debian-based distributions use apt, and even the source-based distributions already encode all the required information needed to build the software successfully. Building one's own software in these extremely rich environments is almost never required unless one is actually contributing to a distribution. The downside to that approach is a brutal lack of testing. Component testing is extremely limited; most contributors only test their packages on a single system, and if you don't happen to use the same architecture as that contributor, it's likely that you will be running a particular binary for the first time. Unit testing within these components is hit-or-miss as well; little of this software contains built-in test suites, and many distributions never run them on the software they deliver. Finally, integration testing is practically nonexistent, in some cases by design. When you ship 20,000 components it's simply not possible to assess the impact of upgrading one of those components, especially when few of them offer any kind of API stability. Jamie Zawinski recently blogged a typical example[1] of this problem. Contemplating upgrading all 20,000 of those components simultaneously is recognising that large-scale Free Software aggregations are dancing bears. So what would you have us do? A few possibilities: 1. Bundle a huge assortment of untested libraries and provide no support for them. Even in the best case the libraries delivered with Solaris releases are sure to be 6+ months out of date on the first day they ship. People using current OpenSolaris sources or Solaris Express have at least the possibility of getting fresher bits, but there's still no guarantee that they'll work, and in any case someone has to do the work to keep this software up to date in whatever consolidation it resides. 2. Leave the existing consolidations as they are, and use the Companion Common Base (or whatever name it takes) as your main source of unstable third-party software. This would seem to do little but isolate the problem; it does not seem functionally different from the previous option. 3. Take the time to test a possibly smaller subset of this software thoroughly, integrate it properly with the rest of OpenSolaris, and attach a slightly higher stability to it - Unstable, perhaps. This seems attractive to me in many ways - it offers users a consistent, well-defined environment with each OS release. But it does nothing for anyone who wants to build a piece of software released yesterday that depends on a new and incompatible version of a library released last week. This takes us back to the Companion discussion[2]: what problems are we trying to solve? Why should an administrator want to install the Companion? What problems do we need to solve for people who can't or won't install the Companion, and what cost are we willing to bear to solve them? [0] PSARC/2006/107 mkstemps(3C)/mkdtemp(3C) [1] http://www.dnalounge.com/backstage/log/2006/04.html#29 [2] http://www.opensolaris.org/jive/forum.jspa?forumID=126 -- Keith M Wesolowski "Sir, we're surrounded!" Solaris Kernel Team "Excellent; we can attack in any direction!"
