Re: Debian concordance
Em Dom, 2005-06-19 às 16:22 +0100, Scott James Remnant escreveu: A definitive example would be the (eventually abandoned) attempt by Ximian to provide debs for Helix GNOME. I was working in a GNOME2 backport back then, IIRC. I remember the Helix GNOME debs were simply low quality, with non-sensical, almost bizarre in some cases, Depends. Nevertheless, they would install pretty well on a stable system and with some difficulty and manual intervention to fix conflicts in unstable, at the time. See ya, -- [EMAIL PROTECTED]: Gustavo Noronha http://people.debian.org/~kov Debian: http://www.debian.org * http://www.debian-br.org signature.asc Description: This is a digitally signed message part
Re: Debian concordance
On Sun, Jun 19, 2005 at 05:19:08PM +0100, Scott James Remnant wrote: A definitive example would be the (eventually abandoned) attempt by Ximian to provide debs for Helix GNOME. Didn't that have more to do with it being experimental, rather flakey, and conflicting badly with the gnome stuff in Debian? The main problem they had was that they created the debs for potato, and they were perfectly installable on that. But Debian changed things hugely in unstable, so they weren't installable there -- and then introduced testing, making three incompatible systems. It was impossible to create a single set of debs that would work on all three (stable, testing, unstable) distributions of Debian at the same time. I thought the big problem was that the Helix GNOME packages were packaged without any coordination with Debian, and no effort was made on the part of those packagers (and therefore, not by anyone else either) to ensure the official Debian packages for woody included a smooth upgrade path? There are some fundamental technical problems with the way Debian does things, and the way our software works, that makes deriving from Debian or providing third-party debs very hard or impossible. Unfortunately Debian has a habit of blaming the derivative or third party for working around these problems :-( Let's use a popular example... I make a package that requires /usr/bin/bzgrep. In Debian, I would have to read the debian/changelog for bzip2 and discover that this wasn't introduced until 1.0.1-3, and thus Depends: bzip2 (= 1.0.1-3) But in a Debian-derivative with a different release schedule (perhaps a system used in Schools and sync'd on the start of a school year), that might have been backported and added in 0.9.5d-3school1 Depends: bzip2 (= 0.9.5d-3school1) There's no way to express both of these relationships in the same binary (as 1.0.1-2 would satisfy the relationship for the Debian-derivative). Sure there is -- Provides: bzgrep But that requires coordination between the maintainers of the two packages. And while file-based dependencies could address this particular class of issue (and I think there's far from universal agreement that it's a good idea), there are certainly other issues where packaging system changes can't possibly be a substitute. It's also certainly not realistic to expect Debian maintainers to track down whatever arbitrary changes any Debian derivative is making to their packages: there are far too many Debian derivatives to go around. So yes, when Debian derivatives make changes without communicating with Debian about them, I do blame the derivatives for the resulting incompatibilities. Similar problems exist for shared libraries, I might need a symbol added in a particular revision in one derivative and a different one in another. And do you have any real-world examples of this happening? Patching upstream libraries in this fashion is strongly discouraged within Debian. Even glibc, which seems to have started this discussion, hasn't suffered from such a problem between Debian and Ubuntu. I don't disagree with, and in fact, I utterly support the idea that Debian should be an excellent base for derivative distributions and third-party packages. I just don't think sarge is there, in fact, I think sarge is about as far from that ideal as you can get. We need social and technical changes to make Debian suitable as a standard base, I think we should do it and I think we _can_ do it. But first Debian needs to stop blaming derivatives and third parties for breaking compatibility, and instead ask what we did wrong that required them to break compatibility in order to reach their goals. I think that's a very unrealistic position to take. If the derivatives care about breaking compatibility, why aren't they here *telling* us when we've done something wrong? And if upwards compatibility isn't a priority for them, why assume that this is any fault of Debian's? -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On 6/18/05, Michael K. Edwards [EMAIL PROTECTED] wrote: In any case, Ubuntu packages aren't Debian packages any more than Mandrake packages are Red Hat packages. If Ubuntu sees itself to Debian as Mandrake was to Red Hat, then that certainly explains a lot. If you want binary compatibility, you need to build a system whose engineering outcome is binary compatibility That's precisely what I'm proposing we should do here! There will never be a better time. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On 6/18/05, Matt Zimmerman [EMAIL PROTECTED] wrote: For open source software as a rule, the most important interface level is the source code. [...] [...] The cost of guaranteeing ABI compatibility is high, and the benefit to free software is marginal. It is a problem for proprietary software vendors to be concerned with, and we should leave it to them. We have more important work to do, and we're doing it in source code form. I don't know if you release this, but this is exactly what Red Hat says too. RHEL is free, because we provide the source code. Binaries aren't important to free software. Well, they're pretty damned important to Red Hat, to the tune of about $200 million per year (and growing at an impressive rate too). No wonder they don't want anyone else to think they're important. Surely you can see that there is quite a lot more to what we do than GNOME and X.org, both technologically and organizationally, and our release process is part of it. Sure, I've never disputed that. I'd argue, though, that your release process *was* part of it. Now that sarge is out, we have an opportunity to fix the problem at its source, rather than continuing to provide a workaround. Why not take advantage of that? If Ubuntu had somehow been constructed atop Woody, rather than unstable, consider how much more difficult it would have been for Ubuntu patches to be used in unstable. This would have been like forward-porting patches from Linux 2.2 to Linux 2.6. You're being dramatic again. I'm not suggesting Ubuntu should track a Debian stable that's released whenever it's ready. I'm saying Ubuntu should base on stable if and only if Debian can fix the release management problems. If, 12 or 18 months from today, Debian seems no closer to fixing these problems, Debian will deserve what it gets, and I'll be Ubuntu's biggest chearleader. In the meantime, let's give Debian a chance. That's all I'm saying. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On 6/18/05, Steve Langasek [EMAIL PROTECTED] wrote: Is Progeny interested in working with other Debian (+Ubuntu) folks to solve the fundamental limitations of the shlibs system that cause sarge and hoary to be incompatible due to a single-symbol difference, and that will cause similar breakage in the other direction with sarge and breezy? Am I willing to put my money where my mouth is? Absolutely. In fact, this could fit pretty well with some work Jeff Licquia is already doing to help Debian achieve LSB 2.0/3.0 compliance without breaking sarge compatibility: http://www.licquia.org/archives/2005/06/16/a-new-approach-to-the-lsb-part-2/ It doesn't really matter to me *how* we fix the compatibility problem, so long as we fix it. In fact, if we can find a way to fix it that makes it easier for the derivatives, so much the better--don't forget, I'm in that business too. ... going it alone, like when Matthias Klose ran his plans for the gcc 4 transition past the Debian release team before implementing it in Ubuntu, and is now proceeding to implement the same transition in Debian? Mea culpa. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
* Scott James Remnant | A definitive example would be the (eventually abandoned) attempt by | Ximian to provide debs for Helix GNOME. At the same time, I've never had a problem Opera debs provided by Opera Software. -- Tollef Fog Heen,''`. UNIX is user friendly, it's just picky about who its friends are : :' : `. `' `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/19/05, Scott James Remnant [EMAIL PROTECTED] wrote: On Sun, 2005-06-19 at 11:42 -0400, Joey Hess wrote: Scott James Remnant wrote: Walking up to a man on the street, if anything, you'll find Debian has a far worse reputation than RPM and RedHat-derived distributions. The general feeling is that third-party RPMs will almost always install on any system, while third-party .debs are practically useless. That's strange, this is not the impression I've gotten from ten years of reading the debian-user mailing list, participating in Linux and Debian user groups, or hearing people discuss services such as backports.org and apt-get.org, or from using them myself. Try hanging out outside of the immediate Debian community; I spend a fair amount of time loitering around the GNOME and Freedesktop.org communities, for example. You see a wildly different picture there. I spend a fair amount of time with executives in technology companies, big and small. What they tell me is that they very badly want to support Debian but aren't quite sure how to do it. The demand is clearly there. The main problem, as they see it, is that they're not quite sure what Debian is. They'd like it to be Debian stable, but that hasn't been realistic up to now because Debian stable has been too old, and it's been impossible to know when the next one will be out. In that environment, the Ubuntu approach is reasonable. I guess the difference between your point of view and mine is that you seem to take it as a given that it has to be this way while I don't. It was impossible to create a single set of debs that would work on all three (stable, testing, unstable) distributions of Debian at the same time. That may be partially true (and it's not quite that dramatic--I've been mixing and matching for years without major problems). However, as I've said several times in this thread, to the vast majority of the world, Debian is Debian stable. Historically, people have used testing because of the lack of a predictable release cycle around stable, and Debian developers are the primary users of unstable. Also, as Joey has pointed out, all three of these distros are under a single umbrella, Debian, so transitions can be and are carefully coordinated. Fix the release problem, and lots of day-to-day users will stop using testing; the remainder will continue to use testing because they, well, want to help test it. Furthermore, for the most part, as has already been pointed out, packages built against stable tend to work on unstable just fine, particularly if there isn't a three year gap between releases. The other situations are bugs. As the comment that started this thread stated, packages built against glibc 2.3.2 run fine against glibc 2.3.5 but not vice versa, and this is mostly true when the upstreams are careful about compatibility. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On Mon, Jun 20, 2005 at 05:44:33AM -0500, Ian Murdock wrote: I don't know if you release this, but this is exactly what Red Hat says too. RHEL is free, because we provide the source code. Binaries aren't important to free software. Well, they're pretty damned important to Red Hat, to the tune of about $200 million per year (and growing at an impressive rate too). No wonder they don't want anyone else to think they're important. As I explained when Joey made this same interpretation, I obviously don't believe that binaries are unimportant. Binaries are the most important transport format for getting software into the hands of users. However, they aren't a very useful tool for software development collaboration, and work to make binaries more universal does not promote the development of open source software. Sure, I've never disputed that. I'd argue, though, that your release process *was* part of it. Now that sarge is out, we have an opportunity to fix the problem at its source, rather than continuing to provide a workaround. Why not take advantage of that? Debian and Ubuntu already are taking advantage of the space created by the Sarge release, by coordinating major transitions, and feeding more Ubuntu features and packages into Debian. But if what you're suggesting is that the Sarge release means that Ubuntu should stop making its own releases based on unstable, that doesn't make sense for our developers or our users. You seem to believe that Ubuntu's approach to releasing was a one-time workaround for a delayed Sarge release, but that has never been the case. Ubuntu 5.10 will be released in October of this year, based on the breezy development tree, regardless of any pending discussions about Debian's release strategy. You're being dramatic again. I'm not suggesting Ubuntu should track a Debian stable that's released whenever it's ready. I'm saying Ubuntu should base on stable if and only if Debian can fix the release management problems. If, 12 or 18 months from today, Debian seems no closer to fixing these problems, Debian will deserve what it gets, and I'll be Ubuntu's biggest chearleader. In this case, your suggestion is entirely hypothetical, and we'll discuss this when there's something concrete to discuss. The world could be a very different place in 12 or 18 months, and we aren't going to plan the next 2-3 Ubuntu releases based on an oversimplified proposal to improve Debian's release cycle. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/20/05, Ian Murdock [EMAIL PROTECTED] wrote: On 6/18/05, Michael K. Edwards [EMAIL PROTECTED] wrote: In any case, Ubuntu packages aren't Debian packages any more than Mandrake packages are Red Hat packages. If Ubuntu sees itself to Debian as Mandrake was to Red Hat, then that certainly explains a lot. Well, I haven't used Mandrake in a while, and if there's bad blood there that I don't know about then it's probably a bad analogy. (And remember, I have no affiliation with Ubuntu either.) All I'm saying is that, unlike a CDD or a Debian-pony-with-one-extra-trick (Knoppix, etc.), Ubuntu is a full distro which tracks Debian at the source code level rather than using Debian binary packages. This has consequences for ISVs not unlike those of the early Mandrake releases, when Mandrake tracked Red Hat's code and releases but optimized for Pentium and wrote an alternate installer. If you wanted your third-party application to run on both, you couldn't just sort of pretend you were part of the Red Hat release cycle; you needed to concoct a build environment whose products were distro-agnostic. Choice is good; but choice doesn't always make things easier. If you want binary compatibility, you need to build a system whose engineering outcome is binary compatibility That's precisely what I'm proposing we should do here! There will never be a better time. Building that system doesn't mean cajoling Ubuntu into holding breezy back. It means (as I see it) constructing an apt repository and a debootstrap recipe for Debian Standard Base 05.03 and 05.09 -- build environments for sarge+hoary+breezy+etch-compatible binary debs and breezy+etch-compatible debs respectively. Presumably packages built in 05.03 won't be able to use ABI fixes, etc. introduced at the last minute in sarge and/or hoary, and anything that we can already tell won't run on breezy or etch should also be excluded. If that leaves a build environment that won't build much other than C programs with few library dependencies, maybe we should think about formalizing more backwards compatibility mechanisms in breezy/etch. If, that is, we care about ISVs and other poor sods who don't want their application upgrade cycle dictated by their O/S upgrade cycle (and vice versa). Cheers, - Michael
Re: Debian concordance
On 6/18/05, Joey Hess [EMAIL PROTECTED] wrote: Matt Zimmerman wrote: Practically speaking, the differences in compatibility between Ubuntu and Debian is of as much concern as those between Debian stable and Debian unstable. New interfaces are added in unstable constantly, and software is adapted to use them. Binary packages from unstable are rarely installable on stable without upgrading other supporting packages. Third party packagers must choose whether to provide builds for stable (if the package builds), unstable or both. So far, this has not resulted in a problem for Debian. Except unstable is capable of running packages built on stable, and stable is to some extent (partial upgrades) capable of running packages built on unstable. And if it doesn't work, a dependency will tell you it doesn't work. And Debian is able to decide we want to make it work better and fix things. So I don't think your analogy holds up very well. After six months, I suspect that sid will have evolved to where no binary package of any great complexity from sarge will install on it without a stack of oldlibs; and backports will be (as usual) a royal pain. Better just to run a carefully selected sid snapshot. Test your backups frequently, though. :-) But Joey's right that having two stable releases, neither of which has systematically greater version numbers than the other, complicates the graph a lot. This isn't necessarily a problem; in a way, it's a nice business opportunity for people with a good grasp on the whole process to build customized distros for businesses that need them for embedded / ISV purposes. But that opportunity already existed mid-Debian-release-cycle; it's just sort of changed shape. A former client of mine was very appreciative of the glibc 2.2.5-final-that-never-was build that I did for them (extending the useful life of their woody systems, for their particular purposes, by a year or so). In a hypothetical similar situation six months from now on sarge, I would probably suggest that they try breezy for a while rather than go custom; but they would need custom work from a different angle to port their internal code sideways and re-tune their automated install procedures. And it works both ways, of course; developers who jumped immediately to hoary may decide that they want python 2.3.x by default after all, and need help persuading sarge to play nice with multiarch. It's dull work in some ways, but it's bread and butter for the local distro wrangler. I'd sure rather have several Debian-style arrows in my quiver than have to choose between Good Luck Enterprise Linux flavors R, S, and W. :-) The cost of guaranteeing ABI compatibility is high, and the benefit to free software is marginal. It is a problem for proprietary software vendors to be concerned with, and we should leave it to them. We have more important work to do, and we're doing it in source code form. If I were only interested in source code, I would not be a contributor to this distribution. I am interested in whole, working systems which are accessible to end users. Amen, brother. But Ubuntu's end users shouldn't be pointing their sources.list at J. R. Hacker's apt repository, even when J. R. Hacker is pronounced Debian, and vice versa. On the other hand, I respectfully differ from Matt about whether the creation of an ISV-friendly build environment should be left to ISVs. Coaxing, say, a major proprietary RDBMS vendor onto Debian/Ubuntu, and reaping the benefit of their benchmark-driven advice in tuning our kernels and libraries, would benefit everyone, including those who prefer open-source databases. Debian packages just work, in the environment for which they were intended No, Debian packages just work, if dpkg allows you to install them on your system. Unless, now, they happen to be built by someone running the other distribution. I can think of several ways that this could happen, but I haven't actually seen any of them yet. Would you mind adducing some examples? This has nothing to do with binary compatibility, and everything to do with rigorous packaging practices (which is the true basis for this selling point). I agree that ABI compatability is only part of the picture, though it seems like one of the more important parts. However, the other parts of the picture suffer from similar problems. It's important if you like, say, sarge for compile farms and hoary as a base system for demo LiveCDs. Just as a random example, Ubuntu's fork of debhelper has a hack[1] in dh_buildeb to run pkgstriptranslations, an Ubuntu-specific command which removes mo files from usr/share/locale. That works ok until Debian adds a pkgstriptranslations that does something else. Or until the Debian version of debhelper is installed on someone's Ubuntu system and begins building packages without calling this tool. I agree with Joey that mucking with the actual packaging
Re: Debian concordance
On Sat, Jun 18, 2005 at 10:33:06PM -0400, Joey Hess wrote: Except unstable is capable of running packages built on stable Trivial packages which only link against libc, yes. In general, no. And many packages from unstable won't build correctly (or at all) on stable during most of the release cycle. , and stable is to some extent (partial upgrades) capable of running packages built on unstable. This is not the same thing. binary compatible package installations do not require upgrading system libraries. And if it doesn't work, a dependency will tell you it doesn't work. Usually, yes, but I don't see the relevance. dpkg can tell you that a binary built with glibc 2.3.2.ds1-21 can't be installed on a glibc 2.3.2.ds1-20 system, but this doesn't change the fact that the package is not compatible with your system. If I were only interested in source code, I would not be a contributor to this distribution. I am interested in whole, working systems which are accessible to end users. That's interesting, but not particularly relevant. Of course source code isn't the only medium which matters. I only said that it isn't important that binary packages be universal. No, Debian packages just work, if dpkg allows you to install them on your system. I disagree, but again, I don't see your point regarding dependency checks. Either the package is compatible, or it isn't. The fact that the tools can detect some types of incompatibilities and say no doesn't make the packages any more or less compatible. Unless, now, they happen to be built by someone running the other distribution. The only incompatibilities being discussed in this thread have involved incompatible package dependencies, but you seem to be alluding to some other type of incompatibility. Are you concerned about the pkgstriptranslations example you describe below, or something else? Just as a random example, Ubuntu's fork of debhelper has a hack[1] in dh_buildeb to run pkgstriptranslations, an Ubuntu-specific command which removes mo files from usr/share/locale. That works ok until Debian adds a pkgstriptranslations that does something else. That would be a bit pathological, don't you think? Ubuntu also has a binary called gcc-4.0, but I wasn't concerned about Debian creating a gcc-4.0 which did something else. Or until the Debian version of debhelper is installed on someone's Ubuntu system and begins building packages without calling this tool. This particular scenario causes no problem, and works as designed (though in general, mixing packages from derivatives is not a smart thing to do for other reasons). No other distribution has ever seen the need to fork debhelper The content of the fork helps put this in perspective: http://people.ubuntu.com/~scott/patches/debhelper/debhelper_4.2.33ubuntu1_unknown.patch (and modify/fork 1500 other packages), or recompile the entire Debian archive from scratch. But today there do now exist Debian derivatives which want to do such devious things as support different versions of Python than Debian. No other distribution except perhaps Knoppix has attracted enough users and developers to make compatability issues more than minor annoyances. Is there a reason why you don't have a problem with Knoppix, but you object to Ubuntu? What about the other Debian derivatives which are based on snapshots of testing or unstable (even packages from experimental), equally incompatible with stable Debian releases? [1] which I would not accept into debhelper if asked because it violates design principles of dh_builddeb I'm not sure that we need this patch going forward, but at any rate it's extremely simple and got the job done. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Sat, Jun 18, 2005 at 11:22:35PM -0700, Michael K. Edwards wrote: On 6/18/05, Joey Hess [EMAIL PROTECTED] wrote: Matt Zimmerman wrote: Practically speaking, the differences in compatibility between Ubuntu and Debian is of as much concern as those between Debian stable and Debian unstable. New interfaces are added in unstable constantly, and software is adapted to use them. Binary packages from unstable are rarely installable on stable without upgrading other supporting packages. Third party packagers must choose whether to provide builds for stable (if the package builds), unstable or both. So far, this has not resulted in a problem for Debian. Except unstable is capable of running packages built on stable, and stable is to some extent (partial upgrades) capable of running packages built on unstable. And if it doesn't work, a dependency will tell you it doesn't work. And Debian is able to decide we want to make it work better and fix things. So I don't think your analogy holds up very well. After six months, I suspect that sid will have evolved to where no binary package of any great complexity from sarge will install on it without a stack of oldlibs; and backports will be (as usual) a royal pain. Better just to run a carefully selected sid snapshot. Test your backups frequently, though. :-) Of 596 lib packages in woody (loosely identified), 325 are still present in sarge. That's after three years of more or less constant development. Where did you come up with this absurd idea that all binary packages of any great complexity will become uninstallable after only six *months*? -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On 6/19/05, Steve Langasek [EMAIL PROTECTED] wrote: Of 596 lib packages in woody (loosely identified), 325 are still present in sarge. That's after three years of more or less constant development. Where did you come up with this absurd idea that all binary packages of any great complexity will become uninstallable after only six *months*? The examples that come to mind immediately are those with substantial components in both native code and an interpreted or bytecode language, such as Perl XSUBs and Python extensions. Last time around, I seem to recall that Perl was updated shortly after the release, and you couldn't very well install a binary package containing a Perl module (especially an XSUB) without also installing the old Perl, by which time you might as well have a stable chroot. And what are the odds of my tweaked Python profiler, built to divert individual files within the Python library tree, working with sid's stock Python build come December? The next example to pop into my head is the Midgard content management framework, which involves both an Apache module and piles of PHP code. The chances of a binary package built on sarge installing (let alone working) against next year's apache and php packages in sid probably aren't that high. The fact is that, while the C dynamic library naming system and the split into libfoo2 and libfoo-dev allows multiple historical versions to co-exist, the same cannot be said of all languages and development frameworks. Biggish software packages tend to have a few too many paths and versions compiled into them to survive the first six months after a release without source code changes and recompilation. Think, say, the LyX WYSIWYM front end to LaTeX (which knows the paths to all sorts of document format converters) or a build of gvim with all of the scripting language bells and whistles. Doubtless others who have had more occasion to attempt this sort of thing than I have will provide more examples if needed. Cheers, - Michael
Re: Debian concordance
Michael K. Edwards wrote: No, Debian packages just work, if dpkg allows you to install them on your system. Unless, now, they happen to be built by someone running the other distribution. I can think of several ways that this could happen, but I haven't actually seen any of them yet. Would you mind adducing some examples? I haven't bothered to find them, but given what I'm hearing about the glibc incompatabilities in this thread, I'm sure they exist, right? I agree with Joey that mucking with the actual packaging system (or even a very popular helper kit) is one of the more fork-like things that a derivative distro can do. But this is certainly an area where sarge+hoary is no worse than woody+sid-after-twelve-months was; many backported source packages were broken in gross or subtle ways if you didn't start by building yourself an up-to-date debhelper. However, debhelper is excessively careful to preserve backwards compatability and when a package's build dependencies don't express its need for a newer debhelper, we file a bug report. Here I can disagree from experience. RPM hell is being unable to reproduce your vendor's binaries as a starting point for subsequent modifications (with or without third-party help), and it was already gaping wide as of Red Hat 5.2. That is not how I've heard most users define the term. It's certianly a valid problem. Ubuntu is the first Debian derivative not to be more or less a random sid snapshot plus the deriver's pet hacks. Well random sid/testing/stable snapshots. With the exception of most CDDs; cf skolelinux, debian-edu, etc. -- see shy jo signature.asc Description: Digital signature
Re: Debian concordance
Matt Zimmerman wrote: I disagree, but again, I don't see your point I think this sums up your entire response nicely, which is why I won't reply to it point-by-point. You're not interested in trying to understand these concerns, but you dismiss them out of hand. Fine. -- see shy jo signature.asc Description: Digital signature
Re: Debian concordance
On Sat, 2005-06-18 at 11:35 -0500, Ian Murdock wrote: Debian packages just work has been a truism for *years*, and it's been one of our key technical selling points. I don't want to see that fall by the wayside. This thread is a perfect example of what will happen if we don't worry about this stuff *now*. I've seen this movie before. Actually, this is a conceit that for some reason Debian Developers (who, of course, only run Debian on their systems) still maintain while the rest of the world believes the exact opposite. Debian has had a long history of being hostile towards any attempt to provide .debs that aren't in the archive, both socially and technically. Walking up to a man on the street, if anything, you'll find Debian has a far worse reputation than RPM and RedHat-derived distributions. The general feeling is that third-party RPMs will almost always install on any system, while third-party .debs are practically useless. A definitive example would be the (eventually abandoned) attempt by Ximian to provide debs for Helix GNOME. Scott -- Have you ever, ever felt like this? Had strange things happen? Are you going round the twist? signature.asc Description: This is a digitally signed message part
Re: Debian concordance
Scott James Remnant wrote: Walking up to a man on the street, if anything, you'll find Debian has a far worse reputation than RPM and RedHat-derived distributions. The general feeling is that third-party RPMs will almost always install on any system, while third-party .debs are practically useless. That's strange, this is not the impression I've gotten from ten years of reading the debian-user mailing list, participating in Linux and Debian user groups, or hearing people discuss services such as backports.org and apt-get.org, or from using them myself. A definitive example would be the (eventually abandoned) attempt by Ximian to provide debs for Helix GNOME. Didn't that have more to do with it being experimental, rather flakey, and conflicting badly with the gnome stuff in Debian? -- see shy jo signature.asc Description: Digital signature
Re: Debian concordance
On Sun, 2005-06-19 at 11:42 -0400, Joey Hess wrote: Scott James Remnant wrote: Walking up to a man on the street, if anything, you'll find Debian has a far worse reputation than RPM and RedHat-derived distributions. The general feeling is that third-party RPMs will almost always install on any system, while third-party .debs are practically useless. That's strange, this is not the impression I've gotten from ten years of reading the debian-user mailing list, participating in Linux and Debian user groups, or hearing people discuss services such as backports.org and apt-get.org, or from using them myself. Try hanging out outside of the immediate Debian community; I spend a fair amount of time loitering around the GNOME and Freedesktop.org communities, for example. You see a wildly different picture there. A definitive example would be the (eventually abandoned) attempt by Ximian to provide debs for Helix GNOME. Didn't that have more to do with it being experimental, rather flakey, and conflicting badly with the gnome stuff in Debian? The main problem they had was that they created the debs for potato, and they were perfectly installable on that. But Debian changed things hugely in unstable, so they weren't installable there -- and then introduced testing, making three incompatible systems. It was impossible to create a single set of debs that would work on all three (stable, testing, unstable) distributions of Debian at the same time. There are some fundamental technical problems with the way Debian does things, and the way our software works, that makes deriving from Debian or providing third-party debs very hard or impossible. Unfortunately Debian has a habit of blaming the derivative or third party for working around these problems :-( Let's use a popular example... I make a package that requires /usr/bin/bzgrep. In Debian, I would have to read the debian/changelog for bzip2 and discover that this wasn't introduced until 1.0.1-3, and thus Depends: bzip2 (= 1.0.1-3) But in a Debian-derivative with a different release schedule (perhaps a system used in Schools and sync'd on the start of a school year), that might have been backported and added in 0.9.5d-3school1 Depends: bzip2 (= 0.9.5d-3school1) There's no way to express both of these relationships in the same binary (as 1.0.1-2 would satisfy the relationship for the Debian-derivative). In the RPM world, my package can simply depend on the file /usr/bin/bzgrep existing. Similar problems exist for shared libraries, I might need a symbol added in a particular revision in one derivative and a different one in another. I don't disagree with, and in fact, I utterly support the idea that Debian should be an excellent base for derivative distributions and third-party packages. I just don't think sarge is there, in fact, I think sarge is about as far from that ideal as you can get. We need social and technical changes to make Debian suitable as a standard base, I think we should do it and I think we _can_ do it. But first Debian needs to stop blaming derivatives and third parties for breaking compatibility, and instead ask what we did wrong that required them to break compatibility in order to reach their goals. Scott -- Have you ever, ever felt like this? Had strange things happen? Are you going round the twist? signature.asc Description: This is a digitally signed message part
Re: Debian concordance
On 6/19/05, Scott James Remnant [EMAIL PROTECTED] wrote: Let's use a popular example... I make a package that requires /usr/bin/bzgrep. In Debian, I would have to read the debian/changelog for bzip2 and discover that this wasn't introduced until 1.0.1-3, and thus Depends: bzip2 (= 1.0.1-3) But in a Debian-derivative with a different release schedule (perhaps a system used in Schools and sync'd on the start of a school year), that might have been backported and added in 0.9.5d-3school1 Depends: bzip2 (= 0.9.5d-3school1) There's no way to express both of these relationships in the same binary (as 1.0.1-2 would satisfy the relationship for the Debian-derivative). In the RPM world, my package can simply depend on the file /usr/bin/bzgrep existing. What if you need a specific feature in or version of bzgrep?
Re: Debian concordance
Scott James Remnant [EMAIL PROTECTED] writes: On Sat, 2005-06-18 at 11:35 -0500, Ian Murdock wrote: Debian packages just work has been a truism for *years*, and it's been one of our key technical selling points. I don't want to see that fall by the wayside. This thread is a perfect example of what will happen if we don't worry about this stuff *now*. I've seen this movie before. Actually, this is a conceit that for some reason Debian Developers (who, of course, only run Debian on their systems) still maintain while the rest of the world believes the exact opposite. Debian has had a long history of being hostile towards any attempt to provide .debs that aren't in the archive, both socially and technically. Er, huh? I got started with Debian packaging by providing an extensive set of local .debs for Stanford internal use for quite some time before I started getting them into the base archive. Everything just worked. Not only did everything just work, but it was *significantly* easier to set this up and make it work properly with Debian than with Red Hat. Walking up to a man on the street, if anything, you'll find Debian has a far worse reputation than RPM and RedHat-derived distributions. The general feeling is that third-party RPMs will almost always install on any system, while third-party .debs are practically useless. This is, to me, an utterly bizarre statement. My personal experience could not possibly be more completely opposite to this. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
Scott James Remnant [EMAIL PROTECTED] writes: The main problem they had was that they created the debs for potato, and they were perfectly installable on that. But Debian changed things hugely in unstable, so they weren't installable there -- and then introduced testing, making three incompatible systems. It was impossible to create a single set of debs that would work on all three (stable, testing, unstable) distributions of Debian at the same time. So build three sets of .debs. I'm not sure why people would consider that so hard? Debian provides a ton of excellent tools to help you do exactly that. You have to learn pbuilder and set up a couple of chroots, which while sounding intimidating is actually a matter of a couple of hours work to produce an environment that will then serve you well for years and is quite easy to maintain. Certainly you're not arguing that one never has to build multiple RPMs, are you? Most software I use that provides RPMs has multiple variants for similar reasons (one for Fedora, one for RHEL3, one for RHEL4, etc.). Let's use a popular example... I make a package that requires /usr/bin/bzgrep. In Debian, I would have to read the debian/changelog for bzip2 and discover that this wasn't introduced until 1.0.1-3, and thus Depends: bzip2 (= 1.0.1-3) But in a Debian-derivative with a different release schedule (perhaps a system used in Schools and sync'd on the start of a school year), that might have been backported and added in 0.9.5d-3school1 Depends: bzip2 (= 0.9.5d-3school1) There's no way to express both of these relationships in the same binary (as 1.0.1-2 would satisfy the relationship for the Debian-derivative). This seems like an extremely artificial example. Why would the school distribution backport a new feature to a release that's supposed to be extremely stable? In the RPM world, my package can simply depend on the file /usr/bin/bzgrep existing. I can see the merits of this feature in some circumstances (among other things, it means not having to work out the alternative packages that supply a particular binary and don't provide a virtual package). I wouldn't object if someone found a way to make it work well with Debian. You still will need to list what package to install to get that file if you don't already have it, though; I really don't like the idea of apt-get using apt-file to try to guess at what package to install to satisfy the dependency. RPM used to use this mechanism for libraries as well; I sure hope you're not proposing that, as Debian's method is far, far more reliable. We need social and technical changes to make Debian suitable as a standard base, I think we should do it and I think we _can_ do it. But first Debian needs to stop blaming derivatives and third parties for breaking compatibility, and instead ask what we did wrong that required them to break compatibility in order to reach their goals. Well, this I can mostly agree with, since I don't think blame is usually the right way to approach these sorts of things. I'm not sure that blame is really what's going on so much as concern, and I do think the concern is warranted. Certainly, if there are infrastructure improvements that Debian can make without sacrificing its quality that make it easier for derivative distributions to not diverge, I'm all in favor of that and will help as much as I can. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Sun, Jun 19, 2005 at 11:13:31AM -0400, Joey Hess wrote: Michael K. Edwards wrote: I can think of several ways that this could happen, but I haven't actually seen any of them yet. Would you mind adducing some examples? I haven't bothered to find them, but given what I'm hearing about the glibc incompatabilities in this thread, I'm sure they exist, right? I've responded in depth regarding the glibc situation elsewhere in this thread, and there has never been any more to it than shlibdeps. There have been no reports whatsoever of mysterious silent breakage, contrary to your earlier implication. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Sun, Jun 19, 2005 at 01:41:47AM -0700, Michael K. Edwards wrote: On 6/19/05, Steve Langasek [EMAIL PROTECTED] wrote: Of 596 lib packages in woody (loosely identified), 325 are still present in sarge. That's after three years of more or less constant development. Where did you come up with this absurd idea that all binary packages of any great complexity will become uninstallable after only six *months*? The examples that come to mind immediately are those with substantial components in both native code and an interpreted or bytecode language, such as Perl XSUBs and Python extensions. Yes, these are specific examples of packages that will be rendered uninstallable in unstable by an ABI change on a particular package. Your claim was that *all* packages of any great complexity would be uninstallable after six months. And while perl XSUBs from woody are not installable in sarge, python extensions from woody *are* generally installable in sarge (for reasons we shouldn't be particularly proud of, but still). And what are the odds of my tweaked Python profiler, built to divert individual files within the Python library tree, working with sid's stock Python build come December? A pathological case if ever I heard one... The next example to pop into my head is the Midgard content management framework, which involves both an Apache module and piles of PHP code. The chances of a binary package built on sarge installing (let alone working) against next year's apache and php packages in sid probably aren't that high. Which still doesn't prove a claim that *no* packages will be installable after six months. -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On 6/19/05, Steve Langasek [EMAIL PROTECTED] wrote: On Sun, Jun 19, 2005 at 01:41:47AM -0700, Michael K. Edwards wrote: The examples that come to mind immediately are those with substantial components in both native code and an interpreted or bytecode language, such as Perl XSUBs and Python extensions. Yes, these are specific examples of packages that will be rendered uninstallable in unstable by an ABI change on a particular package. Your claim was that *all* packages of any great complexity would be uninstallable after six months. Perhaps that reflects my idea of what constitutes any great complexity -- a high-level implementation language plus some glue and accelerators in C/C++, maybe a GUI library or two, maybe a module for a separately packaged daemon. I also wrote without a stack of 'oldlibs' -- and even packages whose dependencies are completely captured by ldd on a few binaries are going to be in that boat real soon now. And while perl XSUBs from woody are not installable in sarge, python extensions from woody *are* generally installable in sarge (for reasons we shouldn't be particularly proud of, but still). OK; but a program that makes use of them probably doesn't work unless it was future-proofed in advance by systematically avoiding references to python without an explicit version. And anything that uses, say, woody's libwxbase2.2 is out of luck. And python-gdbm isn't the version built against python 2.1 anymore, so packages built against it will have the wrong dependencies for sarge. And so on and so forth. And what are the odds of my tweaked Python profiler, built to divert individual files within the Python library tree, working with sid's stock Python build come December? A pathological case if ever I heard one... Maybe so; but then all efforts to use the packaging system to update bits of a language's standard library, without rebuilding (and taking security update responsibility for) the whole shebang, are equally pathological. That would apply to large fractions of CPAN and CTAN and many emacs modes as well as local customizations like my experimental profiler. The next example to pop into my head is the Midgard content management framework, which involves both an Apache module and piles of PHP code. The chances of a binary package built on sarge installing (let alone working) against next year's apache and php packages in sid probably aren't that high. Which still doesn't prove a claim that *no* packages will be installable after six months. What I said was: After six months, I suspect that sid will have evolved to where no binary package of any great complexity from sarge will install on it without a stack of oldlibs; and backports will be (as usual) a royal pain. Better just to run a carefully selected sid snapshot. Test your backups frequently, though. :-) Would you settle for few binary packages of any great complexity, etc.? Cheers, - Michael
Re: Debian concordance
On Fri, Jun 17, 2005 at 09:32:49AM +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 12:15:25AM -0700, Steve Langasek wrote: On Fri, Jun 17, 2005 at 08:07:34AM +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. Once we begin to deploy icheck, we will have all this information. Haven't yet figured out how to do anything with it. It is not sufficient to track when symbols are added to a package. You must also check when their meaning changes. I have not yet been able to find a way to do this on a per-symbol basis, only a per-library one (I can find examples that break all the 'obvious' approaches). However, breaking the meaning of any symbol is supposed to mean that we punt by changing the soname, no? The notable exception would be glibc, which is the really interesting case here. Right; the issue there being that you have to figure out which version of the symbol the function in your headers currently maps to, not that ELF symbols are themselves repurposed without changing the soname. -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On 6/17/05, Matt Zimmerman [EMAIL PROTECTED] wrote: As I've said to you privately already, I do not feel that demanding binary compatibility between Debian and Ubuntu is the best way to address your concerns. You seem to disagree strongly, as is of course your right, but I think that some of the comments that you've made in support of this cause have been misleading. Please don't be dramatic. I'm not demanding anything. I'm expressing a concern, and a legitimate one. The fact is that Hoary *was* binary compatible (in both directions) with both sarge and sid when it was released. Later, the Debian glibc maintainers and release managers considered changing the ABI in order to fix a bug. In the course of a lengthy discussion[0], including expression of concerns about inter-distribution compatibility, they weighed the options and decided to go ahead with it. I fully support their decision, and I do not consider the resulting incompatibility to be a significant obstacle to the continuing growth and success of either Debian or Ubuntu. Presumably, neither did they. I wasn't even aware of the sarge/hoary incompatibility till it came up in this thread. And, based on what you and others have said, I'd agree it wasn't your fault, though it was certainly unfortunate. I'm more worried about the future; and I still haven't seen anyone address my initial question, which is why Ubuntu is tracking sid on core things like libc in the first place. The value you add is around the edges with stuff like X.org and GNOME 2.10. I'd like to see you do that in a manner that promotes compatibility with sarge, just as we're doing at Progeny as we move forward in these same areas. But I certainly understand why you want to move forward in these areas.. I do as well. The core is a completely different issue. Where at the core do you add value? Ok, perhaps you can get a bug fix in here, better support for an architecture here. But are those things worth breaking compatibility? If your changes are important enough, they should be in Debian too. If they aren't, they're not as important as compatibility. Debian packages just work has been a truism for *years*, and it's been one of our key technical selling points. I don't want to see that fall by the wayside. This thread is a perfect example of what will happen if we don't worry about this stuff *now*. I've seen this movie before. If there's ever been or ever will be a perfect time for Debian and Ubuntu to sync up, it's now. Sarge is out, and there is significant momentum within the project behind the idea of fixing the release cycle problem, so it seems likely that etch will be out in some predictable and reasonable amount of time. Why not take advantage of that? Better yet, why not help make it happen? Why not, for example, work with Debian on putting together a plan for migrating to GCC 4 rather than just plowing ahead on your own? Going it alone is sure to cause compatibility problems that make the current ones pale by comparison. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On Jun 18, Ian Murdock [EMAIL PROTECTED] wrote: Debian packages just work has been a truism for *years*, and it's been And now people are learning that Debian packages are not Debian/unstable packages nor Ubuntu packages. Big deal. I still do not see any harm caused by this, except some speculation. -- ciao, Marco signature.asc Description: Digital signature
Re: Debian concordance
On 6/18/05, Ian Murdock [EMAIL PROTECTED] wrote: I'm more worried about the future; and I still haven't seen anyone address my initial question, which is why Ubuntu is tracking sid on core things like libc in the first place. The value you add is around the edges with stuff like X.org and GNOME 2.10. I'd like to see you do that in a manner that promotes compatibility with sarge, just as we're doing at Progeny as we move forward in these same areas. But I certainly understand why you want to move forward in these areas.. I do as well. X.org isn't exactly around the edges in my book. And in any case, if you think of Ubuntu exclusively as a desktop distro, I don't think you have it straight; hoary works very nicely as a server platform. From my perspective, Ubuntu adds value mostly on process fronts: predictable release cycles, a clear distinction between supported and best effort (and a systematic team approach to that support), and a commercial support model for people who want that sort of thing. Each of these has its down side as well, and Debian works differently for mostly good reasons; but they have inevitable engineering consequences, and not just around the edges either. The core is a completely different issue. Where at the core do you add value? Ok, perhaps you can get a bug fix in here, better support for an architecture here. But are those things worth breaking compatibility? If your changes are important enough, they should be in Debian too. If they aren't, they're not as important as compatibility. Debian packages just work has been a truism for *years*, and it's been one of our key technical selling points. I don't want to see that fall by the wayside. This thread is a perfect example of what will happen if we don't worry about this stuff *now*. I've seen this movie before. In case you hadn't gleaned it from Matt's timeline, most of Ubuntu's work on glibc late in the hoary cycle did make it into sarge (or, if you like, Ubuntu-the-company sponsored some excellent work by some of Debian's glibc team, which made it under the wire for both releases), and I for one am very glad that it did. I really didn't want to be stuck with Ubuntu #7897 (Debian #300943) for the next three years. In any case, Ubuntu packages aren't Debian packages any more than Mandrake packages are Red Hat packages. If you want binary compatibility, you need to build a system whose engineering outcome is binary compatibility, which will look a lot more like the LSB than any one distro. The fact that most packages compiled on sid three months ago will run on both sarge and hoary is no more significant than the fact that most packages compiled on Red Hat 6.0 ran on both Red Hat 7.0 and Mandrake 7.0. If there's ever been or ever will be a perfect time for Debian and Ubuntu to sync up, it's now. Sarge is out, and there is significant momentum within the project behind the idea of fixing the release cycle problem, so it seems likely that etch will be out in some predictable and reasonable amount of time. Why not take advantage of that? Better yet, why not help make it happen? Why not, for example, work with Debian on putting together a plan for migrating to GCC 4 rather than just plowing ahead on your own? Going it alone is sure to cause compatibility problems that make the current ones pale by comparison. Debian and Ubuntu are syncing up, with the maintainers' eyes on the future instead of the past. I think the evidence is overwhelming that Ubuntu is not going it alone or plowing ahead on their own. To take your example: if you hadn't noticed, Matthias Klose is the only person who has ever uploaded GCC 4.0 to either Debian or Ubuntu. I suspect that he coordinates with appropriate teams on both sides. The glibc teams obviously coordinate very closely, and the shlibver problem smells more like the law of unintended consequences than anything else; I think it very improbable that it will recur in the next cycle. The X.org packagers on both sides are doing their best -- it's a very challenging situation -- and it looks like they will sync up where it matters, on the modular tree. Ubuntu's collective patience and tolerance has been quite remarkable -- as has Debian's, with few exceptions. It doesn't make much sense to me, looking from the outside, to ask Ubuntu to freeze the breezy ABIs at the place where the sarge roulette wheel stopped. How about if everybody puts the sarge/hoary baggage behind them and works on etch/breezy? Cheers, - Michael
Re: Debian concordance
On Sat, Jun 18, 2005 at 11:35:21AM -0500, Ian Murdock wrote: Please don't be dramatic. I'm not demanding anything. I'm expressing a concern, and a legitimate one. I'm not the only one who isn't convinced of the accuracy of the predictions which form the basis of your concerns. First, they're based on the RPM story, and there are many differences (technological, social and logical) which lead me to feel that it doesn't model Debian very well at all. We live in a different world, and we did even then. The fate of Debian does not rest on the possibility of creating one binary package which is compatible with every Debian derivative, (which is fortunate, considering that this has not been feasible for a very long time already) and Ubuntu doesn't change that. For open source software as a rule, the most important interface level is the source code. GNOME developers, for example, maintain a very successful collaboration, despite using a wide variety of quite incompatible development platforms. Linus Torvalds' stated position on interface compatibility in the kernel[0] is in the same vein: it's the kernel programming API which is important, not the binary module ABI. [0] http://kerneltrap.org/node/1758 Practically speaking, the differences in compatibility between Ubuntu and Debian is of as much concern as those between Debian stable and Debian unstable. New interfaces are added in unstable constantly, and software is adapted to use them. Binary packages from unstable are rarely installable on stable without upgrading other supporting packages. Third party packagers must choose whether to provide builds for stable (if the package builds), unstable or both. So far, this has not resulted in a problem for Debian. The cost of guaranteeing ABI compatibility is high, and the benefit to free software is marginal. It is a problem for proprietary software vendors to be concerned with, and we should leave it to them. We have more important work to do, and we're doing it in source code form. I'm more worried about the future; and I still haven't seen anyone address my initial question, which is why Ubuntu is tracking sid on core things like libc in the first place. Michael K. Edwards provided you with one explanation in [EMAIL PROTECTED], earlier in this thread, which is pretty close to the heart of the matter. It has to do with the fact that Ubuntu is on a radically different release schedule than Debian. These things are entering our development branch now because they're going to be shipping on CD-ROM in October. The value you add is around the edges with stuff like X.org and GNOME 2.10. I'd like to see you do that in a manner that promotes compatibility with sarge, just as we're doing at Progeny as we move forward in these same areas. But I certainly understand why you want to move forward in these areas.. I do as well. The core is a completely different issue. Where at the core do you add value? Ok, perhaps you can get a bug fix in here, better support for an architecture here. But are those things worth breaking compatibility? Surely you can see that there is quite a lot more to what we do than GNOME and X.org, both technologically and organizationally, and our release process is part of it. It is common sense that new feature development should be based on a development branch, not the previous stable release. If Ubuntu had somehow been constructed atop Woody, rather than unstable, consider how much more difficult it would have been for Ubuntu patches to be used in unstable. This would have been like forward-porting patches from Linux 2.2 to Linux 2.6. If your changes are important enough, they should be in Debian too. If they aren't, they're not as important as compatibility. This seems to imply a belief that the binary compatibility differences which concern you are the result of Ubuntu-specific patches. This is not the case, and never has been. If instead you meant changes in a broader sense (such as earlier adoption of certain versions of software), see above regarding release cycles. Debian packages just work has been a truism for *years*, and it's been one of our key technical selling points. I don't want to see that fall by the wayside. This thread is a perfect example of what will happen if we don't worry about this stuff *now*. I've seen this movie before. Debian packages just work, in the environment for which they were intended would be a more accurate assessment. This has nothing to do with binary compatibility, and everything to do with rigorous packaging practices (which is the true basis for this selling point). It has never just worked to mix and match binary packages from different releases of Debian, or packages from different Debian derivatives. If there's ever been or ever will be a perfect time for Debian and Ubuntu to sync up, it's now. Sarge is out, and there is significant momentum within the project behind the idea of fixing the
Re: Debian concordance
Matt Zimmerman wrote: Practically speaking, the differences in compatibility between Ubuntu and Debian is of as much concern as those between Debian stable and Debian unstable. New interfaces are added in unstable constantly, and software is adapted to use them. Binary packages from unstable are rarely installable on stable without upgrading other supporting packages. Third party packagers must choose whether to provide builds for stable (if the package builds), unstable or both. So far, this has not resulted in a problem for Debian. Except unstable is capable of running packages built on stable, and stable is to some extent (partial upgrades) capable of running packages built on unstable. And if it doesn't work, a dependency will tell you it doesn't work. And Debian is able to decide we want to make it work better and fix things. So I don't think your analogy holds up very well. The cost of guaranteeing ABI compatibility is high, and the benefit to free software is marginal. It is a problem for proprietary software vendors to be concerned with, and we should leave it to them. We have more important work to do, and we're doing it in source code form. If I were only interested in source code, I would not be a contributor to this distribution. I am interested in whole, working systems which are accessible to end users. Debian packages just work, in the environment for which they were intended No, Debian packages just work, if dpkg allows you to install them on your system. Unless, now, they happen to be built by someone running the other distribution. This has nothing to do with binary compatibility, and everything to do with rigorous packaging practices (which is the true basis for this selling point). I agree that ABI compatability is only part of the picture, though it seems like one of the more important parts. However, the other parts of the picture suffer from similar problems. Just as a random example, Ubuntu's fork of debhelper has a hack[1] in dh_buildeb to run pkgstriptranslations, an Ubuntu-specific command which removes mo files from usr/share/locale. That works ok until Debian adds a pkgstriptranslations that does something else. Or until the Debian version of debhelper is installed on someone's Ubuntu system and begins building packages without calling this tool. It has never just worked to mix and match binary packages from different releases of Debian, or packages from different Debian derivatives. No other distribution has ever seen the need to fork debhelper (and modify/fork 1500 other packages), or recompile the entire Debian archive from scratch. No other distribution except perhaps Knoppix has attracted enough users and developers to make compatability issues more than minor annoyances. People didn't start talking about rpm hell until Mandrake or the like showed up. -- see shy jo [1] which I would not accept into debhelper if asked because it violates design principles of dh_builddeb signature.asc Description: Digital signature
Re: Debian concordance
On Sat, Jun 18, 2005 at 11:35:21AM -0500, Ian Murdock wrote: The fact is that Hoary *was* binary compatible (in both directions) with both sarge and sid when it was released. Later, the Debian glibc maintainers and release managers considered changing the ABI in order to fix a bug. In the course of a lengthy discussion[0], including expression of concerns about inter-distribution compatibility, they weighed the options and decided to go ahead with it. I fully support their decision, and I do not consider the resulting incompatibility to be a significant obstacle to the continuing growth and success of either Debian or Ubuntu. Presumably, neither did they. I wasn't even aware of the sarge/hoary incompatibility till it came up in this thread. And, based on what you and others have said, I'd agree it wasn't your fault, though it was certainly unfortunate. I'm more worried about the future; and I still haven't seen anyone address my initial question, which is why Ubuntu is tracking sid on core things like libc in the first place. The value you add is around the edges with stuff like X.org and GNOME 2.10. I'd like to see you do that in a manner that promotes compatibility with sarge, just as we're doing at Progeny as we move forward in these same areas. But I certainly understand why you want to move forward in these areas.. I do as well. Is Progeny interested in working with other Debian (+Ubuntu) folks to solve the fundamental limitations of the shlibs system that cause sarge and hoary to be incompatible due to a single-symbol difference, and that will cause similar breakage in the other direction with sarge and breezy? If there's ever been or ever will be a perfect time for Debian and Ubuntu to sync up, it's now. Sarge is out, and there is significant momentum within the project behind the idea of fixing the release cycle problem, so it seems likely that etch will be out in some predictable and reasonable amount of time. Why not take advantage of that? Better yet, why not help make it happen? Why not, for example, work with Debian on putting together a plan for migrating to GCC 4 rather than just plowing ahead on your own? Going it alone is sure to cause compatibility problems that make the current ones pale by comparison. ... going it alone, like when Matthias Klose ran his plans for the gcc 4 transition past the Debian release team before implementing it in Ubuntu, and is now proceeding to implement the same transition in Debian? -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. Once we begin to deploy icheck, we will have all this information. Haven't yet figured out how to do anything with it. It is not sufficient to track when symbols are added to a package. You must also check when their meaning changes. I have not yet been able to find a way to do this on a per-symbol basis, only a per-library one (I can find examples that break all the 'obvious' approaches). -- .''`. ** Debian GNU/Linux ** | Andrew Suffield : :' : http://www.debian.org/ | `. `' | `- -- | signature.asc Description: Digital signature
Re: Debian concordance
On Fri, Jun 17, 2005 at 08:07:34AM +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. Once we begin to deploy icheck, we will have all this information. Haven't yet figured out how to do anything with it. It is not sufficient to track when symbols are added to a package. You must also check when their meaning changes. I have not yet been able to find a way to do this on a per-symbol basis, only a per-library one (I can find examples that break all the 'obvious' approaches). However, breaking the meaning of any symbol is supposed to mean that we punt by changing the soname, no? -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
On Fri, Jun 17, 2005 at 12:15:25AM -0700, Steve Langasek wrote: On Fri, Jun 17, 2005 at 08:07:34AM +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. Once we begin to deploy icheck, we will have all this information. Haven't yet figured out how to do anything with it. It is not sufficient to track when symbols are added to a package. You must also check when their meaning changes. I have not yet been able to find a way to do this on a per-symbol basis, only a per-library one (I can find examples that break all the 'obvious' approaches). However, breaking the meaning of any symbol is supposed to mean that we punt by changing the soname, no? The notable exception would be glibc, which is the really interesting case here. -- .''`. ** Debian GNU/Linux ** | Andrew Suffield : :' : http://www.debian.org/ | `. `' | `- -- | signature.asc Description: Digital signature
Re: Debian concordance
On Fri, 2005-06-17 at 09:32 +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 12:15:25AM -0700, Steve Langasek wrote: On Fri, Jun 17, 2005 at 08:07:34AM +0100, Andrew Suffield wrote: On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. Once we begin to deploy icheck, we will have all this information. Haven't yet figured out how to do anything with it. It is not sufficient to track when symbols are added to a package. You must also check when their meaning changes. I have not yet been able to find a way to do this on a per-symbol basis, only a per-library one (I can find examples that break all the 'obvious' approaches). However, breaking the meaning of any symbol is supposed to mean that we punt by changing the soname, no? The notable exception would be glibc, which is the really interesting case here. Who are historically pretty well behaved about changing symbol versions if they change the API. Scott -- Have you ever, ever felt like this? Had strange things happen? Are you going round the twist? signature.asc Description: This is a digitally signed message part
Re: Debian concordance
On 6/16/05, Michael K. Edwards [EMAIL PROTECTED] wrote: On 6/16/05, Ian Murdock [EMAIL PROTECTED] wrote: glibc. Shipping X.org and GNOME 2.10 adds value, since sarge doesn't ship them. Shipping glibc 2.6.5 vs. glibc 2.6.2 just adds incompatibilities. Speaking as someone with no Ubuntu affiliation (and IANADD either), I think that statement is based on a somewhat shallow analysis of how glibc is handled. [...] I don't doubt there were changes, even some worthwhile changes, between the version of libc in sarge and the versions in hoary/breezy. My question is: Are the changes worth breaking compatibility? It's a cost/benefit thing. And if they're important enough, why aren't they going into Debian directly? I understand why Ubuntu was moving ahead of Debian before, since Debian was so far behind. But now that sarge is out, don't you think it would be worthwhile to give Debian a chance to fix its release cycle problems and, better yet, to try to help fix them, rather than simply saying Debian is too slow/unpredictable for us? Again, as I've said before, it's *sarge* the rest of the world thinks of as Debian, not sid. So, we're getting out patches into sid or we're tracking sid or whatever doesn't really help anything. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On Fri, Jun 17, 2005 at 01:55:57PM -0500, Ian Murdock wrote: snip I don't doubt there were changes, even some worthwhile changes, between the version of libc in sarge and the versions in hoary/breezy. My question is: Are the changes worth breaking compatibility? It's a cost/benefit thing. And if they're important enough, why aren't they going into Debian directly? I understand why Ubuntu was moving ahead of Debian before, since Debian was so far behind. But now that sarge is out, don't you think it would be worthwhile to give Debian a chance to fix its release cycle problems and, better yet, to try to help fix them, rather than simply saying Debian is too slow/unpredictable for us? Pardon me, but wouldn't hoary and sarge be (roughly) compatible, and breezy with etch (hoping on a slightly faster release cycle). Wouter van Heyst -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/17/05, Ian Murdock [EMAIL PROTECTED] wrote: On 6/16/05, Michael K. Edwards [EMAIL PROTECTED] wrote: Speaking as someone with no Ubuntu affiliation (and IANADD either), I think that statement is based on a somewhat shallow analysis of how glibc is handled. [...] I don't doubt there were changes, even some worthwhile changes, between the version of libc in sarge and the versions in hoary/breezy. My question is: Are the changes worth breaking compatibility? It's a cost/benefit thing. And if they're important enough, why aren't they going into Debian directly? Well, if anyone broke compatibility in the sarge/hoary timeframe, it wasn't Ubuntu. The sched_[gs]et_affinity change in sarge is the only one that broke an ABI and bumped shlib_dep_ver. The Ubuntu folks quite consciously refrained from rolling out 2.3.[45] in hoary; demanding that they merge 2.3.2.ds1-22 and nothing else into breezy, and therefore further postpone upstream's improved ppc64 support and compatibility with other toolchain updates, seems like a lot to ask. I understand why Ubuntu was moving ahead of Debian before, since Debian was so far behind. But now that sarge is out, don't you think it would be worthwhile to give Debian a chance to fix its release cycle problems and, better yet, to try to help fix them, rather than simply saying Debian is too slow/unpredictable for us? I really don't think they're saying that. Goto-san and the rest of the Debian glibc team are appropriately cautious about moving 2.3.5 from experimental to unstable at the same time that many other things are in flux. But if Ubuntu is going to move the toolchain forward in breezy at all, they just can't wait. If anything, this will help smooth the transition in Debian and speed up the etch cycle accordingly. There's no particular reason why etch can't ship in six or nine months, with better application compatibility between etch and breezy than there is now between sarge and hoary. Again, as I've said before, it's *sarge* the rest of the world thinks of as Debian, not sid. So, we're getting out patches into sid or we're tracking sid or whatever doesn't really help anything. I basically agree with you there; what would help, in my view, is a sort of Debian/Ubuntu mini-LSB, ideally with a white box testing framework that helps validate that a .deb is installable and likely to function properly on some combination of sarge, hoary, breezy, and etch. If, that is, ISV support is of interest, and you don't want to go the LCC multi-distro golden binaries route. Cheers, - Michael
Re: Debian concordance
* Daniel Stone: Breezy (like current sid) is built against 2.3.5. Current sid on which platform? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Fri, Jun 17, 2005 at 03:58:35AM +1000, Daniel Stone wrote: Hoary (like sarge) is built against 2.3.2. Breezy (like current sid) is built against 2.3.5. No, 2.3.5 is still in experimental. -- Daniel Jacobowitz CodeSourcery, LLC -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Jun 17, Ian Murdock [EMAIL PROTECTED] wrote: I don't doubt there were changes, even some worthwhile changes, between the version of libc in sarge and the versions in hoary/breezy. My question is: Are the changes worth breaking compatibility? It's a cost/benefit thing. And if they're important enough, why aren't they going into Debian directly? You may want to ask the Debian glibc team. Or understand that different distributions may have different goals and priorities. I understand why Ubuntu was moving ahead of Debian before, since Debian was so far behind. But now that sarge is out, don't you think it would be worthwhile to give Debian a chance to fix its release cycle problems and, better yet, to try to help fix them, rather than simply saying Debian is too slow/unpredictable for us? Considering the track record of past debian releases, I'd say no. Again, as I've said before, it's *sarge* the rest of the world thinks of as Debian, not sid. So, we're getting out patches into Not since sarge has been about 1-2 years late. -- ciao, Marco signature.asc Description: Digital signature
Re: Debian concordance
On Fri, Jun 17, 2005 at 01:55:57PM -0500, Ian Murdock wrote: I don't doubt there were changes, even some worthwhile changes, between the version of libc in sarge and the versions in hoary/breezy. My question is: Are the changes worth breaking compatibility? It's a cost/benefit thing. And if they're important enough, why aren't they going into Debian directly? I understand why Ubuntu was moving ahead of Debian before, since Debian was so far behind. But now that sarge is out, don't you think it would be worthwhile to give Debian a chance to fix its release cycle problems and, better yet, to try to help fix them, rather than simply saying Debian is too slow/unpredictable for us? Let's slow down for a minute. No one has said Debian is too slow/unpredictable for us, no one is denying Debian a chance to address the issues with its release cycle, and Hoary did not break glibc ABI compatibility with Sarge. I think the following timeline might help to clarify the situation: 2004-12-27 glibc 2.3.2.ds1-20 uploaded to sid (http://lists.debian.org/debian-devel-changes/2004/12/msg01481.html) 2005-04-08 Ubuntu 5.04 (Hoary Hedgehog) released, with glibc based on (and compatible with) sid's (and sarge's) 2.3.2.ds1-20 (http://www.ubuntulinux.org/504Released) 2005-04-16 glibc 2.3.2.ds1-21 uploaded to sid (http://lists.debian.org/debian-devel-changes/2005/04/msg01457.html) 2005-04-18 glibc 2.3.5 uploaded to experimental (http://lists.debian.org/debian-devel-changes/2005/04/msg01579.html) 2005-04-28 glibc 2.3.2.ds1-21 accepted into sarge (http://release.debian.org/sarge-hints/vorlon) 2005-05-17 glibc 2.3.5 uploaded to breezy (http://lists.ubuntu.com/archives/breezy-changes/2005-May/004798.html) 2005-06-06 Debian 3.1 (Sarge) released, with glibc 2.3.2.ds1-22 (http://lists.debian.org/debian-announce/debian-announce-2005/msg3.html) 2005-06-?? glibc 2.3.5 expected to enter sid sometime this month As I've said to you privately already, I do not feel that demanding binary compatibility between Debian and Ubuntu is the best way to address your concerns. You seem to disagree strongly, as is of course your right, but I think that some of the comments that you've made in support of this cause have been misleading. The fact is that Hoary *was* binary compatible (in both directions) with both sarge and sid when it was released. Later, the Debian glibc maintainers and release managers considered changing the ABI in order to fix a bug. In the course of a lengthy discussion[0], including expression of concerns about inter-distribution compatibility, they weighed the options and decided to go ahead with it. I fully support their decision, and I do not consider the resulting incompatibility to be a significant obstacle to the continuing growth and success of either Debian or Ubuntu. Presumably, neither did they. [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=297769 Again, you may disagree with me on this point, but there is no justification for claiming that Ubuntu created this situation, regardless of your opinion about it. Again, as I've said before, it's *sarge* the rest of the world thinks of as Debian, not sid. So, we're getting out patches into sid or we're tracking sid or whatever doesn't really help anything. I don't know what you mean by this. Are you trying to say that: - Patches received from Ubuntu should have been pushed into sarge more aggressively? - Ubuntu should base its development branch on sarge rather than sid? Neither of these interpretations make sense to me. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Thu, Jun 16, 2005 at 12:54:08PM -0500, Ian Murdock wrote: Daniel Stone wrote: libc6 added interfaces between 2.3.2 and 2.3.5 and made several other major changes, so all packages built with .5 depend on .5 or above, in case you use one of the new interfaces. A binary built with 2.3.2 can run with .5, but a binary built with .5 can't necessarily run with .2. Then why not build your packages against 2.3.2? That would ensure maximum compatibility with Debian proper (which to most of the world is sarge, *not* sid, so don't answer that you're almost the same as sid). Hoary (like sarge) is built against 2.3.2. Breezy (like current sid) is built against 2.3.5. I don't begrudge your attempt to innovate, but I doubt your users consider a slightly newer libc innovation, particularly when it introduces problems like this. Ironically, the problem in this case stems from sid innovating too fast for Hoary, the latest stable Ubuntu release. Ho hum. I strongly suspect they're more interested in your X.org and GNOME 2.10. Given that, a lot of this divergence seems pretty gratutious to me. Yes, these are both very interesting to users. Which 'divergence' do you mean when you reference that -- X.Org/GNOME 2.10, or glibc? Cheers, Daniel signature.asc Description: Digital signature
Re: Debian concordance
On 6/16/05, Daniel Stone [EMAIL PROTECTED] wrote: On Thu, Jun 16, 2005 at 12:54:08PM -0500, Ian Murdock wrote: Daniel Stone wrote: libc6 added interfaces between 2.3.2 and 2.3.5 and made several other major changes, so all packages built with .5 depend on .5 or above, in case you use one of the new interfaces. A binary built with 2.3.2 can run with .5, but a binary built with .5 can't necessarily run with .2. Then why not build your packages against 2.3.2? That would ensure maximum compatibility with Debian proper (which to most of the world is sarge, *not* sid, so don't answer that you're almost the same as sid). Hoary (like sarge) is built against 2.3.2. Breezy (like current sid) is built against 2.3.5. Unfortunately, it's worse than this. A last-minute ABI change in sarge (backporting some glibc 2.3.4 symbols to 2.3.2) has the effect that any package whose autogenerated shlibdeps includes libc6, when built on sarge, isn't installable on hoary. Any package that doesn't use the affected APIs can be hacked to work around this (by lowering the versioned dependency on libc6), but it's quite an inconvenience. In general, it's not trivial to set up a build environment that reliably produces binary packages that are installable on both sarge and hoary. (I happen to have such an environment at work, based on a part-careful-part-lucky snapshot of sid, but it's not something I would care to support for more than a few packages.) It would be awfully nice if Debian and Ubuntu could coordinate on a 90% solution; I don't necessarily expect to be able easily to build python packages that will run on both (given Ubuntu's early move to 2.4) but how about basic C, C++, and perl ABI compatibility? (Yes, I know this is what the LSB is for, but Debian and Ubuntu are so closely related that the 90% solution probably isn't that hard. An apt repository containing a few packages with lowest-common-denominator ABIs, plus a debootstrap script for use with pbuilder, would probably do it.) Cheers, - Michael
Re: Debian concordance
Daniel Stone wrote: libc6 added interfaces between 2.3.2 and 2.3.5 and made several other major changes, so all packages built with .5 depend on .5 or above, in case you use one of the new interfaces. A binary built with 2.3.2 can run with .5, but a binary built with .5 can't necessarily run with .2. Then why not build your packages against 2.3.2? That would ensure maximum compatibility with Debian proper (which to most of the world is sarge, *not* sid, so don't answer that you're almost the same as sid). I don't begrudge your attempt to innovate, but I doubt your users consider a slightly newer libc innovation, particularly when it introduces problems like this. I strongly suspect they're more interested in your X.org and GNOME 2.10. Given that, a lot of this divergence seems pretty gratutious to me. P.S. - This whole thread is *exactly* the kind of thing I'm talking about when I talk about Ubuntu divergence from Debian and the kinds of headaches that are naturally going to arise from that. For debian-devel's benefit, the thread starts here: http://lists.ubuntu.com/archives/ubuntu-devel/2005-June/008125.html P.S. - Don't tell me build from source is the answer--with a package system as advanced as Debian's, this shouldn't be necessary. And, as above, to most of the world, this is a non-started for many reasons. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/16/05, Daniel Stone [EMAIL PROTECTED] wrote: Hoary (like sarge) is built against 2.3.2. Breezy (like current sid) is built against 2.3.5. Why? -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
On 6/16/05, Daniel Stone [EMAIL PROTECTED] wrote: I strongly suspect they're more interested in your X.org and GNOME 2.10. Given that, a lot of this divergence seems pretty gratutious to me. Yes, these are both very interesting to users. Which 'divergence' do you mean when you reference that -- X.Org/GNOME 2.10, or glibc? glibc. Shipping X.org and GNOME 2.10 adds value, since sarge doesn't ship them. Shipping glibc 2.6.5 vs. glibc 2.6.2 just adds incompatibilities. -- Ian Murdock 317-578-8882 (office) http://www.progeny.com/ http://ianmurdock.com/ A nerd is someone who uses a telephone to talk to other people about telephones. --Douglas Adams
Re: Debian concordance
Ian Murdock wrote: On 6/16/05, Daniel Stone [EMAIL PROTECTED] wrote: I strongly suspect they're more interested in your X.org and GNOME 2.10. Given that, a lot of this divergence seems pretty gratutious to me. Yes, these are both very interesting to users. Which 'divergence' do you mean when you reference that -- X.Org/GNOME 2.10, or glibc? glibc. Shipping X.org and GNOME 2.10 adds value, since sarge doesn't ship them. Shipping glibc 2.6.5 vs. glibc 2.6.2 just adds incompatibilities. As well as bugfixes and things like ppc64 NPTL. Thiemo -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/16/05, Ian Murdock [EMAIL PROTECTED] wrote: glibc. Shipping X.org and GNOME 2.10 adds value, since sarge doesn't ship them. Shipping glibc 2.6.5 vs. glibc 2.6.2 just adds incompatibilities. Speaking as someone with no Ubuntu affiliation (and IANADD either), I think that statement is based on a somewhat shallow analysis of how glibc is handled. Jeff Bailey and Goto Masanori, and a couple of other glibc packagers, are probably the people who can make genuinely informed comments; but having watched the final stages of glibc convergence for both hoary and sarge, I can say that they are significantly different under the hood despite both being labeled 2.3.2. On the Ubuntu side, divergences from the last Debian glibc drop that was merged into hoary (2.3.2.ds1-20) include subtle but important fixes to NPTL/TLS (with particular implications for heavily multithreaded apps on Sun's JRE), a fix for a sigsetjmp() issue that hits gcc 4.0, substantial rework of UTF-8 locale handling, and Tollef Fog Heen's multiarch support. The first two of these appear to have been addressed similarly in Debian's 2.3.2.ds1-21, but there's also a change to sched_[gs]et_affinity that resulted in GLIBC_2.3.4 versioned symbols and thus bumped shlib_dep_ver up to 2.3.2.ds1-21. That's why many (most?) packages compiled on sarge won't install on hoary. All of this is on top of the extensive backports to 2.3.2 of fragments of later glibc snapshots that were already in Debian's 2.3.2.ds1-20. Goto-san has expressed the intention of moving sid to glibc 2.3.5 ASAP (his first upload of 2.3.4 to experimental was in March), and Ubuntu has synced up to his latest upload and moved forward with build fixes. If it is Ubuntu's desire not to diverge too much from sid in the breezy timeframe, they really did have to merge from Debian experimental immediately after the hoary release. It doesn't seem reasonable to me to ask Ubuntu to continue shipping a fork labeled 2.3.2-whatever for the duration of stable=sarge. If it were possible for breezy to bump shlib_dep_ver to 2.3.2.ds1-21 and hold it there, that would be great; but it looks to me like Jeff has bumped it to 2.3.4-1 for good and sufficient reason. At least, with some luck, packages compiled on sarge will run on breezy (but not vice versa without manual shlibver mangling), just as packages compiled on hoary will run on sarge (ditto). Cheers, - Michael
Re: Debian concordance
On 6/16/05, Matthias Klose [EMAIL PROTECTED] wrote: Python is basic for Ubuntu. Given the long freeze of sarge, Debian had to support 2.1 (jython), 2.2 (for zope 2.6) and 2.3 for sarge. I'm happy we did have a possibility to ship 2.4.1 with sarge. Maybe not with the best packaging, but it's included in the release. We will have a better packaging for etch/breezy. No criticism intended of Ubuntu's work on python packaging, which is awesome. I just meant to point out that it is pretty much inevitable that python packages need to be compiled separately for sarge and hoary if they are to be used with the default runtime. Please stop spreading fud about C++ and perl ABI compatibility. There is none! Both sarge and hoary did ship with ABI compatible Perl and C++ ABIs. Basic ABI/API updates are done at the start of a new release cycle, there is no difference how that is done in Debian and Ubuntu. There is no point comparing the state of development and unstable versions. I'm sorry, I really don't mean to be spreading FUD. I meant those as criteria for a lowest-common-denominator build environment, not as statements about what is or isn't compatible between sarge and hoary. There is of course a great deal more involved in practical ABI compatibility than compiler / standard library divergences; addressing the libc6 shlibver issue may resolve the Perl XSUB problems (I haven't tried it yet), but I haven't even checked whether there are other last-minute divergences in things like libxml2, glib, libnspr4, libssl, etc. In general, I think it's normal for it to take some work to set up a build environment for binary packages that are supposed to install and function on multiple distros, no matter how closely related the distros are. It would be a really wonderful thing if Debian and Ubuntu (and any other Debian derivatives willing to pitch in) could do that work instead of sticking ISVs (commercial or otherwise) with it. Cheers, - Michael
Re: Debian concordance
Michael K. Edwards writes: In general, it's not trivial to set up a build environment that reliably produces binary packages that are installable on both sarge and hoary. (I happen to have such an environment at work, based on a part-careful-part-lucky snapshot of sid, but it's not something I would care to support for more than a few packages.) It would be awfully nice if Debian and Ubuntu could coordinate on a 90% solution; I don't necessarily expect to be able easily to build python packages that will run on both (given Ubuntu's early move to 2.4) but how about basic C, C++, and perl ABI compatibility? Python is basic for Ubuntu. Given the long freeze of sarge, Debian had to support 2.1 (jython), 2.2 (for zope 2.6) and 2.3 for sarge. I'm happy we did have a possibility to ship 2.4.1 with sarge. Maybe not with the best packaging, but it's included in the release. We will have a better packaging for etch/breezy. Please stop spreading fud about C++ and perl ABI compatibility. There is none! Both sarge and hoary did ship with ABI compatible Perl and C++ ABIs. Basic ABI/API updates are done at the start of a new release cycle, there is no difference how that is done in Debian and Ubuntu. There is no point comparing the state of development and unstable versions. Matthias PS: From those people spending time reasoning about releases, I'd like to see the same amount of work going into release work itself :-/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On Thu, Jun 16, 2005 at 04:03:32PM -0700, Michael K. Edwards wrote: On 6/16/05, Ian Murdock [EMAIL PROTECTED] wrote: glibc. Shipping X.org and GNOME 2.10 adds value, since sarge doesn't ship them. Shipping glibc 2.6.5 vs. glibc 2.6.2 just adds incompatibilities. Speaking as someone with no Ubuntu affiliation (and IANADD either), I think that statement is based on a somewhat shallow analysis of how glibc is handled. Jeff Bailey and Goto Masanori, and a couple of other glibc packagers, are probably the people who can make genuinely informed comments; but having watched the final stages of glibc convergence for both hoary and sarge, I can say that they are significantly different under the hood despite both being labeled 2.3.2. On the Ubuntu side, divergences from the last Debian glibc drop that was merged into hoary (2.3.2.ds1-20) include subtle but important fixes to NPTL/TLS (with particular implications for heavily multithreaded apps on Sun's JRE), a fix for a sigsetjmp() issue that hits gcc 4.0, substantial rework of UTF-8 locale handling and Tollef Fog Heen's multiarch support. The first two of these appear to have been addressed similarly in Debian's 2.3.2.ds1-21, So if they were merged, why is it relevant that they were merged from hoary to sarge instead of the other way around? but there's also a change to sched_[gs]et_affinity that resulted in GLIBC_2.3.4 versioned symbols and thus bumped shlib_dep_ver up to 2.3.2.ds1-21. Indeed; and the shlibs model of identifying dependencies unfortunately is not very fine-grained, with the result that it's a lose-lose choice between making people manually set shlibs when building on sarge if they want hoary compatibility, or shipping glibc in sarge with an incorrectly relaxed shlibs that could let you install packages on hoary that won't work there. So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
Re: Debian concordance
Ian Murdock writes: On 6/16/05, Daniel Stone [EMAIL PROTECTED] wrote: Hoary (like sarge) is built against 2.3.2. Breezy (like current sid) is built against 2.3.5. Why? - Ubuntu supports its powerpc users with a ppc64 toolchain and kernels. - Ubuntu does toolchain upgrades at the beginning of a release cycle as Debian does. Matthias -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian concordance
On 6/16/05, Steve Langasek [EMAIL PROTECTED] wrote: On Thu, Jun 16, 2005 at 04:03:32PM -0700, Michael K. Edwards wrote: On the Ubuntu side, divergences from the last Debian glibc drop that was merged into hoary (2.3.2.ds1-20) include subtle but important fixes to NPTL/TLS (with particular implications for heavily multithreaded apps on Sun's JRE), a fix for a sigsetjmp() issue that hits gcc 4.0, substantial rework of UTF-8 locale handling and Tollef Fog Heen's multiarch support. The first two of these appear to have been addressed similarly in Debian's 2.3.2.ds1-21, So if they were merged, why is it relevant that they were merged from hoary to sarge instead of the other way around? It's not particularly; they just happened to be things I was aware of on the Ubuntu side, and in verifying the corresponding changes in -21 I was relieved to see that similar (not, I think, identical) patches went into sarge. Looks like Ubuntu has since adopted Debian's fix for at least the TLS fix (not sure about the others, they may already be dealt with upstream in 2.3.[45]). but there's also a change to sched_[gs]et_affinity that resulted in GLIBC_2.3.4 versioned symbols and thus bumped shlib_dep_ver up to 2.3.2.ds1-21. Indeed; and the shlibs model of identifying dependencies unfortunately is not very fine-grained, with the result that it's a lose-lose choice between making people manually set shlibs when building on sarge if they want hoary compatibility, or shipping glibc in sarge with an incorrectly relaxed shlibs that could let you install packages on hoary that won't work there. Right. As I said, it's not surprising, and not cause for criticism, that there's a need for a specialized environment if you want to build multi-distro binary packages. It's a bit like building LSB RPMs, except it's less painful because of the shared heritage and the reduced need to lag drastically when there are fewer distros involved. So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... It would be pretty cool to actually extract the external symbols referenced in a package's ELF objects and deduce the minimum happy library versions accordingly. Alternately, one could construct a set of chroots with various historical library mixes and grind through the ELF objects with ldd. That's really more along the lines of automated white box testing, and might fit better into the sort of grandiose post-build QA-role-signature chain I've proposed a couple times than it does in the build process per se. This holds especially when it comes to ISV certifications, to the extent that that's of interest to Debian and/or derivatives. (Oracle on Ubuntu, anyone?) ISV build processes tend to be rather stinky, and post-build touch-ups to metadata are more practical than retrofits to the build. (There's likely to be an alien / java-package style stage in the process anyway.) Cheers, - Michael
Re: Debian concordance
On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. One easy way would be to simply make it an error for there to be any new symbols while building a source package -- so you'd have to update the table before building so it gets put into the source; this is a bit invasive though. (It also could have repercussions for buildds, if there are symbols that only exist on a particular architecture.) I haven't come up with another yet. Scott -- Have you ever, ever felt like this? Had strange things happen? Are you going round the twist? signature.asc Description: This is a digitally signed message part
Re: Debian concordance
On Fri, Jun 17, 2005 at 04:26:36AM +0100, Scott James Remnant wrote: On Thu, 2005-06-16 at 17:20 -0700, Steve Langasek wrote: So, maybe it's time to revisit the weaknesses of the shlibs system, particularly as they apply to glibc. Scott James Remnant had done some poking in this area about a year ago, which involved tracking when individual symbols were added to a package -- apparently, many packages would actually be happy with glibc 2.1 or so (at least on i386), but we have no way to track this... I was just thinking the same with this thread ... The principal problem with the shlibsyms stuff was that in order to track when symbols are added to a package, you need the list of the set of symbols that were in the last version -- and as the source packages are put together before the binary, the source package wouldn't contain the updated set of symbols. One easy way would be to simply make it an error for there to be any new symbols while building a source package -- so you'd have to update the table before building so it gets put into the source; this is a bit invasive though. http://lists.debian.org/debian-devel/2005/06/msg01185.html :) Invasive, yes, but I think it's time to take our library QA tools to the next level. (It also could have repercussions for buildds, if there are symbols that only exist on a particular architecture.) Yep. And glibc happens to be the chief candidate for that, unfortunately. Also an issue are any C++ libs being built using different compilers on different architectures. So there does seem to be a need for per-architecture override lists. I'm not sure how user-friendly the support for that would need to be initially, though. -- Steve Langasek postmodern programmer signature.asc Description: Digital signature
