Re: libraries being removed from the archive
On Tue, 05 Aug 2003 07:32:57 +0200, Matthias Urlichs wrote: > Hi, Peter Mathiasson wrote: > >> "[...] distcc sends the complete preprocessed source code across the >> network for each job." > > Hmm, OK, but that would just speedup the actual compilation. Granted, > that's the largest chunk, but cpp/asm/ld could do with a speed-up too. As a point of information, distcc runs the assembler remotely too (in this case, on the x86). cpp's CPU usage is usually negligible, and while ld can be slow it is often a small fraction of the total time. Building a cross suite with the excellent Debian toolchain-source package is easy, and I regularly use that to have several x86 machines help with ia64 builds. For many packages the biggest problem is actually ./configure, since it is slow and not parallelizable. -- Martin
Re: libraries being removed from the archive
Richard Braakman wrote:
> On Mon, Aug 04, 2003 at 10:08:04AM +0200, Jérôme Marant wrote:
> > > Hence the need for policy to dictate to the maintainer not to allow the
> > > package to be removed before all other packages have transitioned. It
> > > usually doesn't take much more work as long as the maintainer is even
> > > aware of what will happen.
> >
> > It is not policy problem, it is a common sense one!
>
> Common sense says otherwise :) You see, before we had katie and the
> testing scripts, such removal of orphan libraries was done manually.
> ("orphan" because they no longer had a source package that built them).
> Our experience was that packages that depended on them did not even start
> to get updated until after we removed the old library. As long as the
> old library was there, there was apparently no incentive for anyone
> to recompile.
>
> That's when we decided to just remove such libraries immediately,
> and just let unstable be broken for a while. With most libraries
> this works fine. There were a few libraries with so many dependencies
> that an "oldlibs" version was necessary -- ncurses was in that
> category, for example. But they were the exception, not the rule.
That's experience, not common sense.
Regards,
Joey
--
Long noun chains don't automatically imply security. -- Bruce Schneier
Please always Cc to me when replying to me on the lists.
Re: libraries being removed from the archive
> On Tue, Aug 05, 2003 at 01:07:42AM +0400, Nikita V. Youshchenko wrote: > > So buidd + distcc on a slow m68k/arm/whatever, and distccd on a fast > > P4 or Athlon, or even on several of those. This is expected to reduce > > the compile time to almost the same as it is on x86 :). > > I'm not sure that's true; but it would be interesting to know for > sure. Many packages spend a lot of time running tests and generating > documentation. Maybe this can be compensated by running distccd on several fast x86 machines, and run parallel make while package building? Distcc is known to scale almost lineary for small number of hosts. I use distcc at regular basis and can confirm this. Perhaps someone who has access to both m68k and a fast x86 connected by a fast network should try this.
Re: libraries being removed from the archive
On Tue, Aug 05, 2003 at 01:07:42AM +0400, Nikita V. Youshchenko wrote: > So buidd + distcc on a slow m68k/arm/whatever, and distccd on a fast P4 or > Athlon, or even on several of those. This is expected to reduce the compile > time to almost the same as it is on x86 :). I'm not sure that's true; but it would be interesting to know for sure. Many packages spend a lot of time running tests and generating documentation. These would still take forever. Also, when using distcc, the m68k is still responsible for preprocessing source, assembling compiler output, and linking objects. These operations take up a nontrivial amount of time on m68k. Plus, the buildd's currently install every non-essential build dependency before building a specific package, and remove them after. Running dpkg is slow on m68k, especially if you're installing a giant chain of build dependencies like GNOME or KDE. I'm sure distcc would help though ;).
Re: libraries being removed from the archive
Hi, Peter Mathiasson wrote:
> "[...] distcc sends the complete preprocessed source code across
> the network for each job."
Hmm, OK, but that would just speedup the actual compilation. Granted,
that's the largest chunk, but cpp/asm/ld could do with a speed-up too.
Anyway, thanks for the pointers to existing software everybody.
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | [EMAIL PROTECTED]
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
--
If Columbus had had an advisory committee he would probably still be at
the dock.
-- Justice Arthur Goldberg
Re: libraries being removed from the archive
At Mon, 04 Aug 2003 15:54:56 +0200, Matthias Urlichs wrote: > > Hi, Nikita V. Youshchenko wrote: > > >> Guess how many hours it takes for the m68k buildd to rebuild > >> kdegraphics. OVER 38 HOURS! > > > > By the way, isn't it a good time to rise up a discussion about package > > cross-compiling infrastructure? > > Surprise, I was thinking about the same thing, yesterday. Basic idea: > mount the slow system's build chroot from the fast server, replace > gcc/g++/ld with scripts that call the server's version remotely. The > biggest problem will probably be (a) getting all those pathnames right. http://www.dodes.org/dodes/ DODES: DODES is a system for the transparent remote execution of the binaries for the foreign architectures. For an example, you can build a MIPS binary with cross compiler on host x86 GNU/Linux system and can "execute it" on host. regards, junichi
Re: libraries being removed from the archive
At Mon, 04 Aug 2003 15:54:56 +0200, Matthias Urlichs wrote: > > Hi, Nikita V. Youshchenko wrote: > > >> Guess how many hours it takes for the m68k buildd to rebuild > >> kdegraphics. OVER 38 HOURS! > > > > By the way, isn't it a good time to rise up a discussion about package > > cross-compiling infrastructure? > > Surprise, I was thinking about the same thing, yesterday. Basic idea: > mount the slow system's build chroot from the fast server, replace > gcc/g++/ld with scripts that call the server's version remotely. The > biggest problem will probably be (a) getting all those pathnames right. http://www.dodes.org/dodes/ DODES: DODES is a system for the transparent remote execution of the binaries for the foreign architectures. For an example, you can build a MIPS binary with cross compiler on host x86 GNU/Linux system and can "execute it" on host. regards, junichi
Re: libraries being removed from the archive
On Tue, Aug 05, 2003 at 01:07:42AM +0400, Nikita V. Youshchenko wrote: > I'm not sure that current distcc in unstable can support such configuration, > but it should be really easy to add this support. In fact, as far as I can > remember, it is mentioned in distcc documentation that machines that run > distcc and distccd may have different architecture. "Unlike other distributed build systems, distcc distcc does not require all machines to share a filesystem, or to have the same libraries or header files installed. distcc sends the complete preprocessed source code across the network for each job." Sounds like it might work out of the box if you just exclude "localhost" from DISTCC_HOSTS. -- Peter Mathiasson, peter at mathiasson dot nu, http://www.mathiasson.nu GPG Fingerprint: A9A7 F8F6 9821 F415 B066 77F1 7FF5 C2E6 7BF2 F228
Re: libraries being removed from the archive
> > If you want to be productive, how about setting a buildd and trying to > > crosscompile the distribution and then post statsistics of > > failed/succeeded crosscompilings? > This is a good idea. Maybe I will try after my vacation. Is > documentation/hints abould how to do it available anywhere? THere have been some attempts to recompile the entire distribution on this list you may to search for. buildd is available at http://m68k.debian.org/buildd/getting.html . You will probably also be able to use pentium-builder package to trick the right compliler flags past the configure scripts. -- Riku Voipio|[EMAIL PROTECTED] | kirkkonummentie 33 |+358 40 8476974 --+-- 02140 Espoo| | Facts do not cease to exist because they are ignored. |
Re: libraries being removed from the archive
On Sun, 2003-08-03 at 03:32, Chris Cheney wrote: > Today I was reminded of something that causes apps not to migrate into > sarge. When maintainers remove old libraries from the archive! Today > for example libexif8 was removed by Christophe Barbe and replaced by > libexif9. Guess what that does... any package which depends on libexif8 > now becomes... you guessed it... UNINSTALLABLE! Why does this annoy me > in particular, because I just uploaded kdegraphics yesterday which was > built against libexif8. Guess how many hours it takes for the m68k > buildd to rebuild kdegraphics. OVER 38 HOURS! > OH NO! THIS IS TERRIBLE! It'll take a WHOLE 38 HOURS LONGER for your package to enter testing! Just THINK of all the TERRIBLE THINGS that could happen in that space of time! Users could find some bugs in the package which should really be fixed before it's a candidate package for the next release, wait, uh, that's not terrible. Sarge could release without it, wait, that's not going to happen just yet. Uh... Seriously, sheesh, stop over-reacting. A dependency just had a SONAME bump, eh? Well you're going to have to recompile against that new library sooner or later. Or do you only fix even the most critical bugs on a testing-delay cycle to make sure every package you release becomes a candidate for the next stable release, no matter how buggy they are? Scott signature.asc Description: This is a digitally signed message part
Re: libraries being removed from the archive
> On Mon, Aug 04, 2003 at 03:54:56PM +0200, Matthias Urlichs wrote: >> Surprise, I was thinking about the same thing, yesterday. Basic idea: >> mount the slow system's build chroot from the fast server, replace >> gcc/g++/ld with scripts that call the server's version remotely. The >> biggest problem will probably be (a) getting all those pathnames right. > > Wouldn't it be possible to use something like distcc for this? I wanted to suggest just the same when read this message. It is somewhat difficult to setup correct complete cross-compile environment which handles build-depends and all cases when programs other than compiler are executed during the build process. But it is not difficult to build a cross version of "cc1"/"cc1plus" and "as" programs, and just use distcc to run those on a fast x86-based computer. This is expected to decrease compile time dramatically. I'm not sure that current distcc in unstable can support such configuration, but it should be really easy to add this support. In fact, as far as I can remember, it is mentioned in distcc documentation that machines that run distcc and distccd may have different architecture. So buidd + distcc on a slow m68k/arm/whatever, and distccd on a fast P4 or Athlon, or even on several of those. This is expected to reduce the compile time to almost the same as it is on x86 :).
Re: libraries being removed from the archive
On 04-Aug-03, 12:42 (CDT), Adam Heath <[EMAIL PROTECTED]> wrote: > On Mon, 4 Aug 2003, Richard Braakman wrote: > > Uh, no. Changing the binary package name the way we've always > > handled soname changes, except with a small number of very popular > > libraries. It's a lot less work, and it doesn't require creating a > > new package that will be orphaned almost instantly. If it turns out > > to be a problem for a particular library, and oldlibs package can > > be created for it afterwards, when the need for it has been demonstrated. > > And of the users? Please read the social contract. How are the users harmed? The binary package name and library soname *change*, so the old library will not be removed, so currently installed software will continue to work. For a few days, in *unstable*, some packages will not be installable. I hardly see this as a violation of the Social Contract. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 12:42:27PM -0500, Adam Heath wrote: > And of the users? Please read the social contract. I read it every day, just before bedtime. Richard Braakman
Re: libraries being removed from the archive
On Mon, 4 Aug 2003, Richard Braakman wrote: > On Mon, Aug 04, 2003 at 10:53:00AM -0500, Adam Heath wrote: > > In this case, libexif8 -> libexif9, this is a major soname bump, so should > > have required a new source package. The maintainer was probably derelict in > > this case. > > Uh, no. Changing the binary package name the way we've always > handled soname changes, except with a small number of very popular > libraries. It's a lot less work, and it doesn't require creating a > new package that will be orphaned almost instantly. If it turns out > to be a problem for a particular library, and oldlibs package can > be created for it afterwards, when the need for it has been demonstrated. And of the users? Please read the social contract.
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 08:07:56PM +0300, Richard Braakman wrote: > On Mon, Aug 04, 2003 at 10:53:00AM -0500, Adam Heath wrote: > > In this case, libexif8 -> libexif9, this is a major soname bump, so should > > have required a new source package. The maintainer was probably derelict in > > this case. > > Uh, no. Changing the binary package name the way we've always > handled soname changes, except with a small number of very popular > libraries. It's a lot less work, and it doesn't require creating a > new package that will be orphaned almost instantly. If it turns out > to be a problem for a particular library, and oldlibs package can > be created for it afterwards, when the need for it has been demonstrated. > > Richard Braakman Amen. -- Christophe Barbé <[EMAIL PROTECTED]> GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8 F67A 8F45 2F1E D72C B41E Cats are intented to teach us that not everything in nature has a function. --Garrison Keillor
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 10:53:00AM -0500, Adam Heath wrote: > In this case, libexif8 -> libexif9, this is a major soname bump, so should > have required a new source package. The maintainer was probably derelict in > this case. Uh, no. Changing the binary package name the way we've always handled soname changes, except with a small number of very popular libraries. It's a lot less work, and it doesn't require creating a new package that will be orphaned almost instantly. If it turns out to be a problem for a particular library, and oldlibs package can be created for it afterwards, when the need for it has been demonstrated. Richard Braakman
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 10:53:00AM -0500, Adam Heath wrote: > In this case, libexif8 -> libexif9, this is a major soname bump, so should > have required a new source package. The maintainer was probably derelict in > this case. The source package is libexif independently of the soname. Are you suggesting that ALL library packages should come with a new source package for each major soname change? I was not derelict. If you have good arguments to answer yes to the above question, please give them to me. In the mean time I will keep doing it the way I believe makes the more sense and which happens to be a very common practice. Christophe -- Christophe Barbé <[EMAIL PROTECTED]> GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8 F67A 8F45 2F1E D72C B41E Whether you believe you can, or whether you believe you can't, you're absolutely right. --Henry Ford
Re: libraries being removed from the archive
Adam Heath wrote: > Perhaps someone should write a script to detect these uninstallable issues, > and notify the maintainers of the dependant packages when they occur. Like [0]? (Not my work, but such a script certainly seems to exist.) If done at all, probably a two (or something) day grace period before mail is sent should be implemented for people noticing themselves. Cheers T. 0. http://ftp-master.debian.org/unmet-deps/ pgpL4fR66vSlh.pgp Description: PGP signature
Re: libraries being removed from the archive
Chris Cheney <[EMAIL PROTECTED]> writes: > On Sun, Aug 03, 2003 at 03:55:41PM -0400, David Z Maze wrote: >> Chris Cheney <[EMAIL PROTECTED]> writes: >> >> > IMHO we need to make an addition to policy stating that an old lib can >> > not be removed from the archive until no other packages still depend on >> > it. >> >> So say I maintain foo. The source package produces two binary >> packages, foo and libfoo1. Now, there's a new foo release, that >> changes libfoo's soname. In the current scheme, I package the new >> upstream release and upload foo and libfoo2; since there's no source >> package for libfoo1, it eventually gets removed from unstable. >> >> Are you proposing that (a) the ftpmasters not remove libfoo1, or (b) >> that package maintainers of library packages are now compelled to >> package the last version of foo's source providing libfoo1 separately, >> potentially for multiple release cycles for a widely used library? >> Option (b) sounds problematic to me... > > libfoo1 gets automatically removed immediately upon installation of > libfoo2 in the archive currently. The proper way to fix this issue > is when the maintainer uploads new libfoo source with libfoo2 package > in it to also upload a source called libfoo1 that only provides the > libfoo1 binary package [0]. I have done this myself for libao0 in the > past. Once libfoo1 is no longer used by anything you can simply remove > it from the archive without having to modify anything. I seriously > doubt at the speed of Debian's "release cycle" you would need to have > the old library in the archive for more than one release, probably not > even that long. You do realize that Debian's "release cycle" is > currently 2 years per release...? > I'd just upload a package for the new source, as long as it only breaks binary compatibility (vs. source-level-compat), so a simple rebuild will fix depending packages. A little unstable breakage isn't that bad IMO, as long as there aren't a lot of packages depending on the lib. For libraries that bump soname often (i.e. most C++ libraries, at least), your method will put more strain on the ftpmasters (not that I'm one, but whatever), since they will have to accept the new packages, and then (additionally) remove the old source package (when it's no longer used). Regards, Andy -- Andreas Rottmann | [EMAIL PROTECTED] | [EMAIL PROTECTED] | [EMAIL PROTECTED] http://www.8ung.at/rotty | GnuPG Key: http://www.8ung.at/rotty/gpg.asc Fingerprint | DFB4 4EB4 78A4 5EEE 6219 F228 F92F CFC5 01FD 5B62 Packages should build-depend on what they should build-depend.
Re: libraries being removed from the archive
On Sun, 3 Aug 2003, Chris Cheney wrote: > Seriously, if we want to ever release sarge we are going to need to stop > making libraries disappear, every time we rebuild something it takes > another 10 days for it to migrate into testing and everything that > depends on it is also pushed back another 10 days. Even if the person > causing the breakage NMU's all the affected packages it still causes > them to wait another 10 days to migrate, and causes unneeded load on > the buildds, possibly with the packages no longer being able to be > built since gcc 3.3 is so anal now. (/me wonders how many RC bugs are > around just for gcc 3.3 related crap) So, let's say libexif9 is uploaded, and libexif8 stays around. Do you really want to release a package into stable, that requires an old version of a library? Wouldn't it be better to be compiled against the latest? If the old libraries stay around, then this could easily happen. Removing the old libraries(or packages), means that these problems can be spotted easily. Perhaps someone should write a script to detect these uninstallable issues, and notify the maintainers of the dependant packages when they occur. That would be more useful, imho, then keeping the archive bloated with stale code/source.
Re: libraries being removed from the archive
On Sun, 3 Aug 2003, Eduard Bloch wrote: > #include > * LapTop006 [Sun, Aug 03 2003, 03:13:57PM]: > > > > IMHO we need to make an addition to policy stating that an old lib can > > > not be removed from the archive until no other packages still depend on > > > it. > > How about old libraries can not be removed until either no packages > > depend on it OR the author files RC bugs with any dependent pakcages > > (and hopefully waits for at least a day in case of any major problems)? > > Old libraries are removed since only one version can exist in the same > distro branch to the same time. If the library maintainer decided not to > fork the source package but change the binary package name inside of > existing three then he does not care about the users of his package. > Such actions (in-place transition) should always be done in coordination > with fellow developers as happened in the Perl transition, for example. In this case, libexif8 -> libexif9, this is a major soname bump, so should have required a new source package. The maintainer was probably derelict in this case.
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 03:54:56PM +0200, Matthias Urlichs wrote: > Surprise, I was thinking about the same thing, yesterday. Basic idea: > mount the slow system's build chroot from the fast server, replace > gcc/g++/ld with scripts that call the server's version remotely. The > biggest problem will probably be (a) getting all those pathnames right. Wouldn't it be possible to use something like distcc for this? -- Peter Mathiasson, peter at mathiasson dot nu, http://www.mathiasson.nu GPG Fingerprint: A9A7 F8F6 9821 F415 B066 77F1 7FF5 C2E6 7BF2 F228
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 10:18:37AM +0200, J?r?me Marant wrote: > Usually, you can use apt-cache showpkg libexif8 and send a message to > every maintainer whose package depends on it, asking to rebuild against > the new libexif9. When everyone has rebuilt against the new lib, > then you can ask for the removal of the old library. Read the thread, which is very informative. The first thing you will learn is that packages are removed from the archive automatically. Christophe > > Cheers, > > -- > J?r?me Marant > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] -- Christophe Barbé <[EMAIL PROTECTED]> GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8 F67A 8F45 2F1E D72C B41E Dogs believe they are human. Cats believe they are God.
Re: libraries being removed from the archive
Hi, Nikita V. Youshchenko wrote:
>> Guess how many hours it takes for the m68k buildd to rebuild
>> kdegraphics. OVER 38 HOURS!
>
> By the way, isn't it a good time to rise up a discussion about package
> cross-compiling infrastructure?
Surprise, I was thinking about the same thing, yesterday. Basic idea:
mount the slow system's build chroot from the fast server, replace
gcc/g++/ld with scripts that call the server's version remotely. The
biggest problem will probably be (a) getting all those pathnames right.
I plan to play with the idea a bit once I get my Mac/68k running under
2.6. (I need strace to find a persistent chroot-environment problem. :-/ )
Hmm, if I bind-mount the real arch's /{lib,usr/{bin,lib}} on the server's
chroot, that'll be visible only on the server but not on the client, so a
chrooted cross-compiler should Just Work...
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | [EMAIL PROTECTED]
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
--
:bathtub curve: n. Common term for the curve (resembling an end-to-end
section of one of those claw-footed antique bathtubs) that describes the
expected failure rate of electronics with time: initially high, dropping
to near 0 for most of the system's lifetime, then rising again as it
`tires out'. See also {burn-in period}, {infant mortality}.
Re: libraries being removed from the archive
> On Mon, Aug 04, 2003 at 04:28:49PM +0400, Nikita V. Youshchenko wrote: > > > Guess how many hours it takes for the m68k > > > buildd to rebuild kdegraphics. OVER 38 HOURS! > > > > By the way, isn't it a good time to rise up a discussion about package > > cross-compiling infrastructure? > > Isn't it good idea to research first before rising discussion? Just > raising discussion without referring to any previous discussion about > a subject is very counterproductive, as it inevitably leads to > reitarating all the same arguments that where discussed last time. Well, I just thought about cross compiling while readed the cited post :) > > It's of course possible to cross-compile single packages where you > > know that it works. But compiling *all* packages is a different thing. > > configure too often makes problem, besides Makefiles that never > > thought of cross-compiling... I've written not just "cross-compile", but "cross-compile infrastructure". To face the mentioned problem, there may be a "Cross-Compile-Possible:" field in control file that maintainer should set to "yes" if he claims that package may be cross-compiled. > If you want to be productive, how about setting a buildd and trying to > crosscompile the distribution and then post statsistics of > failed/succeeded crosscompilings? This is a good idea. Maybe I will try after my vacation. Is documentation/hints abould how to do it available anywhere?
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 04:28:49PM +0400, Nikita V. Youshchenko wrote: > Chris Cheney wrote: > > Guess how many hours it takes for the m68k buildd to rebuild > > kdegraphics. OVER 38 HOURS! > > By the way, isn't it a good time to rise up a discussion about package > cross-compiling infrastructure? With reasonably important though big packages, I tend to take the attitude that helping the release process is more important than autobuilder time, and just upload anyway. If the m68k folks are really having problems, they'll shout. From http://buildd.debian.org/stats/ I take it that they're not falling behind to any particularly serious degree. -- Colin Watson [EMAIL PROTECTED]
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 04:28:49PM +0400, Nikita V. Youshchenko wrote: > > Guess how many hours it takes for the m68k > > buildd to rebuild kdegraphics. OVER 38 HOURS! > By the way, isn't it a good time to rise up a discussion about package > cross-compiling infrastructure? Isn't it good idea to research first before rising discussion? Just raising discussion without referring to any previous discussion about a subject is very counterproductive, as it inevitably leads to reitarating all the same arguments that where discussed last time. Try groups.google.com and keywords "autobuilder" "crosscompile" for a starter. Executive summary: Thus said Roman Hodek: > It's of course possible to cross-compile single packages where you > know that it works. But compiling *all* packages is a different thing. > configure too often makes problem, besides Makefiles that never > thought of cross-compiling... If you want to be productive, how about setting a buildd and trying to crosscompile the distribution and then post statsistics of failed/succeeded crosscompilings? -- Riku Voipio|[EMAIL PROTECTED] | kirkkonummentie 33 |+358 40 8476974 --+-- 02140 Espoo| | Facts do not cease to exist because they are ignored. |
Re: libraries being removed from the archive
> Today I was reminded of something that causes apps not to migrate into > sarge. When maintainers remove old libraries from the archive! Today > for example libexif8 was removed by Christophe Barbe and replaced by > libexif9. Guess what that does... any package which depends on libexif8 > now becomes... you guessed it... UNINSTALLABLE! Why does this annoy me > in particular, because I just uploaded kdegraphics yesterday which was > built against libexif8. Guess how many hours it takes for the m68k > buildd to rebuild kdegraphics. OVER 38 HOURS! > > IMHO we need to make an addition to policy stating that an old lib can > not be removed from the archive until no other packages still depend on > it. You are possibly right, but for now, while situation is as it is, why not fix Build-Depends and re-upload immidiatly? This is a one-line fix, so should not take you long. But an installable version of kdegraphics will become available faster.
Re: libraries being removed from the archive
> Guess how many hours it takes for the m68k > buildd to rebuild kdegraphics. OVER 38 HOURS! By the way, isn't it a good time to rise up a discussion about package cross-compiling infrastructure?
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 08:33:31AM +0200, Thomas Viehmann wrote:
> Steve Langasek wrote:
> > I think a better approach would simply be to regard application
> > uninstallable-in-sid bugs as non-RC for testing purposes. Since the
> > testing scripts will already refuse to process new libs that render
> > applications uninstallable, the only impact here will be that certain
> > packages will be uninstallable on a new, completely pristine unstable
> > system -- which, frankly, is not a significant concern of mine.
> > (Putting both testing and unstable in sources.list works just fine,
> > IME.) Tagging such bugs appropriately and ignoring them until after the
> > application makes its way into testing would probably serve our release
> > process better than worrying about uninstallability problems caused by
> > versions of packages which are themselves not yet release candidates.
> Maybe I'm getting this wrong, but wouldn't that just accelerate
> application's push into testing (somewhat) at the price of slowing
> down library updates even further? Application developers could just
> impede libraries progress by building against testing ("oh, this is
> more important than the update to your package, so I'll be building
> against you previous version instead..."). But then certainly
> noone would ever do this, would they?
It would be amusing to watch them try; they would have to build by hand
on all 11 platforms, because the autobuilders will simply build against
unstable. I'm only talking about the case where the application has
/already/ been built, and is rendered uninstallable in sid because of an
soversion increment.
Yes, there are still some tradeoffs, and I expect it to be something
that gets discussed -- at least among the developers affected, if not on
debian-devel or debian-release -- before the maintainer makes an
executive decision on this point. I think a reasonable decision can be
reached in most cases without having to resort to discussions of a
package's "importance", either: in most cases, just looking at the
relative testing wait times for the packages in question, together with
an idea of the typical wait times for a library vs. a non-trivial
application, is likely to point to a solution. E.g.: libexif8 has the
same version in testing and sarge; kdegraphics has *no* version in
testing, even though the package has been in the archive for 11 months.
And honestly, my other idea would be to impose a moratorium on
dependency-changing library uploads three months before a release push;
since the above solution still doesn't address the issue of applications
rebuilding against libraries whose shlibs are not satisfied in testing.
(Samba, which I thought had a relatively simple dependency chain, has
been blocked for about 6 months now because of library dependencies
alone.)
> The advantage of the current situation is that bugs are assigned there where
> changes are needed for a resolution, which seems a good thing to me.
The disadvantage is that it prioritizes installability in sid over
currency in sarge. The reality is that uninstallability due to
dependency unavailability in sid is not *per se* an RC bug, because
unstable isn't what we *release*.
--
Steve Langasek
postmodern programmer
pgpfM55ftNiXA.pgp
Description: PGP signature
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 12:49:51PM +0300, Richard Braakman wrote:
> Common sense says otherwise :) You see, before we had katie and the
> testing scripts, such removal of orphan libraries was done manually.
> ("orphan" because they no longer had a source package that built them).
> Our experience was that packages that depended on them did not even start
> to get updated until after we removed the old library. As long as the
> old library was there, there was apparently no incentive for anyone
> to recompile.
Even now, there are a number of maintainers whose response to "your
package is uninstallable in unstable because libfoo1 was removed" is
"I'll wait until libfoo2 gets into testing and rebuild then". It often
requires some gentle education to demonstrate the deadlock, namely that
libfoo2 will not get into testing until the package in question is
rebuilt.
> Of course, these days we have gnome and kde depending on every library
> they can possibly find, and every other package depending on gnome
> or kde (or both, if they can manage it), so the terrain may have
> shifted somewhat...
It's certainly great fun when a library on which bits of both GNOME and
KDE depend changes soname (pilot-link, at the moment). I'd be almost
inclined towards the multiple source package approach at that point; you
can still provide incentive for people to recompile by removing the old
-dev package and filing "won't build on unstable" bugs (see gdbm).
--
Colin Watson [EMAIL PROTECTED]
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 10:08:04AM +0200, Jérôme Marant wrote:
> > Hence the need for policy to dictate to the maintainer not to allow the
> > package to be removed before all other packages have transitioned. It
> > usually doesn't take much more work as long as the maintainer is even
> > aware of what will happen.
>
> It is not policy problem, it is a common sense one!
Common sense says otherwise :) You see, before we had katie and the
testing scripts, such removal of orphan libraries was done manually.
("orphan" because they no longer had a source package that built them).
Our experience was that packages that depended on them did not even start
to get updated until after we removed the old library. As long as the
old library was there, there was apparently no incentive for anyone
to recompile.
That's when we decided to just remove such libraries immediately,
and just let unstable be broken for a while. With most libraries
this works fine. There were a few libraries with so many dependencies
that an "oldlibs" version was necessary -- ncurses was in that
category, for example. But they were the exception, not the rule.
Of course, these days we have gnome and kde depending on every library
they can possibly find, and every other package depending on gnome
or kde (or both, if they can manage it), so the terrain may have
shifted somewhat...
Richard Braakman
Re: libraries being removed from the archive
Quoting christophe barbe <[EMAIL PROTECTED]>: > Ok, sorry for being rude in my previous mail. > > I understand the general problem that you are facing with KDE and > will try in the future to announce upcomming soname changes. > > Concerning the removal, I don't really see the point of not removing > older libraries from unstable. Most of the time, rebuilding the package > is enough to fix the UNINSTALLABLE problem. > My understanding of what you want is some kind of direct upload to > testing which is not the intented purpose of unstable. Usually, you can use apt-cache showpkg libexif8 and send a message to every maintainer whose package depends on it, asking to rebuild against the new libexif9. When everyone has rebuilt against the new lib, then you can ask for the removal of the old library. Cheers, -- Jérôme Marant
Re: libraries being removed from the archive
Quoting Chris Cheney <[EMAIL PROTECTED]>: > > Old libraries are removed since only one version can exist in the same > > distro branch to the same time. If the library maintainer decided not to > > fork the source package but change the binary package name inside of > > existing three then he does not care about the users of his package. > > Such actions (in-place transition) should always be done in coordination > > with fellow developers as happened in the Perl transition, for example. > > Hence the need for policy to dictate to the maintainer not to allow the > package to be removed before all other packages have transitioned. It > usually doesn't take much more work as long as the maintainer is even > aware of what will happen. It is not policy problem, it is a common sense one! -- Jérôme Marant
Re: libraries being removed from the archive
Steve Langasek wrote:
> I think a better approach would simply be to regard application
> uninstallable-in-sid bugs as non-RC for testing purposes. Since the
> testing scripts will already refuse to process new libs that render
> applications uninstallable, the only impact here will be that certain
> packages will be uninstallable on a new, completely pristine unstable
> system -- which, frankly, is not a significant concern of mine.
> (Putting both testing and unstable in sources.list works just fine,
> IME.) Tagging such bugs appropriately and ignoring them until after the
> application makes its way into testing would probably serve our release
> process better than worrying about uninstallability problems caused by
> versions of packages which are themselves not yet release candidates.
Maybe I'm getting this wrong, but wouldn't that just accelerate application's
push into testing (somewhat) at the price of slowing down library updates even
further? Application developers could just impede libraries progress by building
against testing ("oh, this is more important than the update to your package, so
I'll be building against you previous version instead..."). But then certainly
noone would ever do this, would they?
The advantage of the current situation is that bugs are assigned there where
changes are needed for a resolution, which seems a good thing to me.
Cheers
T.
pgp6c2k2fej5D.pgp
Description: PGP signature
Re: libraries being removed from the archive
On Mon, Aug 04, 2003 at 01:37:43AM +0200, Thomas Viehmann wrote: > Chris Cheney wrote: > ... > > for example libexif8 was removed by Christophe Barbe and replaced by > > libexif9. Guess what that does... any package which depends on libexif8 > ... > > not be removed from the archive until no other packages still depend on > > it. > Well, if it's uninstallable for a couple of days, does it matter that much? > My guess is that while it would be nice to have other ways dealing with this, > your suggestion would create a ton of problems for maintainers to have to > follow > too many old versions of their packages. The problem, which is a very real one, is this: - large application at the top of the dependency change is uploaded. - library it depends on has an soname bump before the application makes it into testing. - bug is filed against the application, because it's now uninstallable on sid. - recompiled application is uploaded; testing counter resets. That said, I don't think Chris's recommendation is the right one. In one sense, the status quo results in a strong incentive for application maintainers to "trim the fat" from their dependency trees. Moreover, requiring library maintainers to maintain multiple copies simultaneously seems rather extreme, for issues that really only apply to a subset of packages. I think a better approach would simply be to regard application uninstallable-in-sid bugs as non-RC for testing purposes. Since the testing scripts will already refuse to process new libs that render applications uninstallable, the only impact here will be that certain packages will be uninstallable on a new, completely pristine unstable system -- which, frankly, is not a significant concern of mine. (Putting both testing and unstable in sources.list works just fine, IME.) Tagging such bugs appropriately and ignoring them until after the application makes its way into testing would probably serve our release process better than worrying about uninstallability problems caused by versions of packages which are themselves not yet release candidates. -- Steve Langasek postmodern programmer pgphoxRZRksQg.pgp Description: PGP signature
Re: libraries being removed from the archive
Chris Cheney wrote: ... > for example libexif8 was removed by Christophe Barbe and replaced by > libexif9. Guess what that does... any package which depends on libexif8 ... > not be removed from the archive until no other packages still depend on > it. Well, if it's uninstallable for a couple of days, does it matter that much? My guess is that while it would be nice to have other ways dealing with this, your suggestion would create a ton of problems for maintainers to have to follow too many old versions of their packages. In your particular case, the problem seems to be 'grep Package kdegraphics-3.1.3/debian/control | wc -l' generating large build-dependencies. (And yes, I know that this is a design decision by upstream.) Usually recompiling for soname-changes should not be that big a problem. I'd rather a slower development process than many packages depending on obsolete versions of libraries during shipping. Cheers T. P.S.: I was today hit by #199049 when wanting to reinstall my system, so I've been thinking about library packages 'disappearing' and recompiling of stuff today (or yesterday, really). pgpgsh3IDi0GM.pgp Description: PGP signature
Re: libraries being removed from the archive
Ok, sorry for being rude in my previous mail. I understand the general problem that you are facing with KDE and will try in the future to announce upcomming soname changes. Concerning the removal, I don't really see the point of not removing older libraries from unstable. Most of the time, rebuilding the package is enough to fix the UNINSTALLABLE problem. My understanding of what you want is some kind of direct upload to testing which is not the intented purpose of unstable. Btw a new libgphoto2 is comming soon ;-) Christophe On Sun, Aug 03, 2003 at 05:16:59PM -0500, Chris Cheney wrote: > On Sun, Aug 03, 2003 at 05:31:37PM -0400, christophe barbe wrote: > > You are kidding right? > > > > I have not removed an old library, I have uploaded a newer upstream with > > a different soname. That's the way it works, a new library is uploaded, > > then packages using it are rebuilt and when they are all ready they > > migrate in testing. > > > > As the gphoto2 maintainer, I don't remember getting mails from you > > announcing an upcoming libusb package with a new soname. Perhaps that's > > because I was waiting for a few months to get a working one for our > > powerpc users. > > > > IMHO we need to make an addition to the policy stating that not being an > > asshole on the mailing-list is welcome. I don't remember sending mails > > to the mailing list when the libusb packaging was broken for a few > > months, but I do remember sending you polite mails. > > > > For you information, some packages using libexif need libexif9. > > I agree that I could (should) have sent a prior notice before uploading it > > (more than a week ago, BEFORE your kdegraphics upload), but that's not > > an excuse to be an asshole. > > > > Christophe > > You aren't the only one that has broken things, many other people > including myself have as well, I most notably with libvorbis ;). However, > this libusb soname change you mention last happened on Feb 27 2002, > which was changed due to a RC bug regarding its naming. (BTW libusb's > soname is odd, which is why I didn't catch it sooner). Also you mention > that libusb was broken for several months, which is true, but it was > only broken on one arch (powerpc). Also, from what I can tell from > looking back at it by the time you determined it wasn't a bug in > gphoto2 you NMU'd it within a week. I don't recall if I was actually > aware of the bug before you NMU'd it. > > Also, I was not stating that libexif9 should not be uploaded, only that > old libraries should not be removed from the archive until they are no > longer used, which apparently was not the case for libexif8. I don't > recall if I stated this earlier but each time I have uploaded KDE in the > past several months it has been broken by library removals within about > a week and recompiling KDE sources is not light work for the buildds. > > Seriously, if we want to ever release sarge we are going to need to stop > making libraries disappear, every time we rebuild something it takes > another 10 days for it to migrate into testing and everything that > depends on it is also pushed back another 10 days. Even if the person > causing the breakage NMU's all the affected packages it still causes > them to wait another 10 days to migrate, and causes unneeded load on > the buildds, possibly with the packages no longer being able to be > built since gcc 3.3 is so anal now. (/me wonders how many RC bugs are > around just for gcc 3.3 related crap) > > BTW - For those wondering Woody was released over a year ago... > > Thanks, > > Chris Cheney > > PS - I apologize for sounding like an asshole, however this general > problem really does need fixing. > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] -- Christophe Barbé <[EMAIL PROTECTED]> GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8 F67A 8F45 2F1E D72C B41E Cats seem go on the principle that it never does any harm to ask for what you want. --Joseph Wood Krutch
Re: libraries being removed from the archive
On Sun, Aug 03, 2003 at 05:31:37PM -0400, christophe barbe wrote: > You are kidding right? > > I have not removed an old library, I have uploaded a newer upstream with > a different soname. That's the way it works, a new library is uploaded, > then packages using it are rebuilt and when they are all ready they > migrate in testing. > > As the gphoto2 maintainer, I don't remember getting mails from you > announcing an upcoming libusb package with a new soname. Perhaps that's > because I was waiting for a few months to get a working one for our > powerpc users. > > IMHO we need to make an addition to the policy stating that not being an > asshole on the mailing-list is welcome. I don't remember sending mails > to the mailing list when the libusb packaging was broken for a few > months, but I do remember sending you polite mails. > > For you information, some packages using libexif need libexif9. > I agree that I could (should) have sent a prior notice before uploading it > (more than a week ago, BEFORE your kdegraphics upload), but that's not > an excuse to be an asshole. > > Christophe You aren't the only one that has broken things, many other people including myself have as well, I most notably with libvorbis ;). However, this libusb soname change you mention last happened on Feb 27 2002, which was changed due to a RC bug regarding its naming. (BTW libusb's soname is odd, which is why I didn't catch it sooner). Also you mention that libusb was broken for several months, which is true, but it was only broken on one arch (powerpc). Also, from what I can tell from looking back at it by the time you determined it wasn't a bug in gphoto2 you NMU'd it within a week. I don't recall if I was actually aware of the bug before you NMU'd it. Also, I was not stating that libexif9 should not be uploaded, only that old libraries should not be removed from the archive until they are no longer used, which apparently was not the case for libexif8. I don't recall if I stated this earlier but each time I have uploaded KDE in the past several months it has been broken by library removals within about a week and recompiling KDE sources is not light work for the buildds. Seriously, if we want to ever release sarge we are going to need to stop making libraries disappear, every time we rebuild something it takes another 10 days for it to migrate into testing and everything that depends on it is also pushed back another 10 days. Even if the person causing the breakage NMU's all the affected packages it still causes them to wait another 10 days to migrate, and causes unneeded load on the buildds, possibly with the packages no longer being able to be built since gcc 3.3 is so anal now. (/me wonders how many RC bugs are around just for gcc 3.3 related crap) BTW - For those wondering Woody was released over a year ago... Thanks, Chris Cheney PS - I apologize for sounding like an asshole, however this general problem really does need fixing.
Re: libraries being removed from the archive
You are kidding right? I have not removed an old library, I have uploaded a newer upstream with a different soname. That's the way it works, a new library is uploaded, then packages using it are rebuilt and when they are all ready they migrate in testing. As the gphoto2 maintainer, I don't remember getting mails from you announcing an upcoming libusb package with a new soname. Perhaps that's because I was waiting for a few months to get a working one for our powerpc users. IMHO we need to make an addition to the policy stating that not being an asshole on the mailing-list is welcome. I don't remember sending mails to the mailing list when the libusb packaging was broken for a few months, but I do remember sending you polite mails. For you information, some packages using libexif need libexif9. I agree that I could (should) have sent a prior notice before uploading it (more than a week ago, BEFORE your kdegraphics upload), but that's not an excuse to be an asshole. Christophe On Sat, Aug 02, 2003 at 09:32:37PM -0500, Chris Cheney wrote: > Today I was reminded of something that causes apps not to migrate into > sarge. When maintainers remove old libraries from the archive! Today > for example libexif8 was removed by Christophe Barbe and replaced by > libexif9. Guess what that does... any package which depends on libexif8 > now becomes... you guessed it... UNINSTALLABLE! Why does this annoy me > in particular, because I just uploaded kdegraphics yesterday which was > built against libexif8. Guess how many hours it takes for the m68k > buildd to rebuild kdegraphics. OVER 38 HOURS! > > IMHO we need to make an addition to policy stating that an old lib can > not be removed from the archive until no other packages still depend on > it. > > Chris Cheney > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] -- Christophe Barbé <[EMAIL PROTECTED]> GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8 F67A 8F45 2F1E D72C B41E A qui sait comprendre, peu de mots suffisent. (Intelligenti pauca.)
Re: libraries being removed from the archive
On Sun, Aug 03, 2003 at 03:55:41PM -0400, David Z Maze wrote: > Chris Cheney <[EMAIL PROTECTED]> writes: > > > IMHO we need to make an addition to policy stating that an old lib can > > not be removed from the archive until no other packages still depend on > > it. > > So say I maintain foo. The source package produces two binary > packages, foo and libfoo1. Now, there's a new foo release, that > changes libfoo's soname. In the current scheme, I package the new > upstream release and upload foo and libfoo2; since there's no source > package for libfoo1, it eventually gets removed from unstable. > > Are you proposing that (a) the ftpmasters not remove libfoo1, or (b) > that package maintainers of library packages are now compelled to > package the last version of foo's source providing libfoo1 separately, > potentially for multiple release cycles for a widely used library? > Option (b) sounds problematic to me... libfoo1 gets automatically removed immediately upon installation of libfoo2 in the archive currently. The proper way to fix this issue is when the maintainer uploads new libfoo source with libfoo2 package in it to also upload a source called libfoo1 that only provides the libfoo1 binary package [0]. I have done this myself for libao0 in the past. Once libfoo1 is no longer used by anything you can simply remove it from the archive without having to modify anything. I seriously doubt at the speed of Debian's "release cycle" you would need to have the old library in the archive for more than one release, probably not even that long. You do realize that Debian's "release cycle" is currently 2 years per release...? Chris Cheney [0] Both the libfoo and libfoo1 source will be marked NEW so assuming that both get processed at the same time packages depending on libfoo1 won't become uninstallable.
Re: libraries being removed from the archive
Chris Cheney <[EMAIL PROTECTED]> writes: > IMHO we need to make an addition to policy stating that an old lib can > not be removed from the archive until no other packages still depend on > it. So say I maintain foo. The source package produces two binary packages, foo and libfoo1. Now, there's a new foo release, that changes libfoo's soname. In the current scheme, I package the new upstream release and upload foo and libfoo2; since there's no source package for libfoo1, it eventually gets removed from unstable. Are you proposing that (a) the ftpmasters not remove libfoo1, or (b) that package maintainers of library packages are now compelled to package the last version of foo's source providing libfoo1 separately, potentially for multiple release cycles for a widely used library? Option (b) sounds problematic to me... -- David Maze [EMAIL PROTECTED] http://people.debian.org/~dmaze/ "Theoretical politics is interesting. Politicking should be illegal." -- Abra Mitchell
Re: libraries being removed from the archive
On Sun, Aug 03, 2003 at 08:55:48AM +0200, Eduard Bloch wrote: > #include > * LapTop006 [Sun, Aug 03 2003, 03:13:57PM]: > > > > IMHO we need to make an addition to policy stating that an old lib can > > > not be removed from the archive until no other packages still depend on > > > it. > > How about old libraries can not be removed until either no packages > > depend on it OR the author files RC bugs with any dependent pakcages > > (and hopefully waits for at least a day in case of any major problems)? > > Old libraries are removed since only one version can exist in the same > distro branch to the same time. If the library maintainer decided not to > fork the source package but change the binary package name inside of > existing three then he does not care about the users of his package. > Such actions (in-place transition) should always be done in coordination > with fellow developers as happened in the Perl transition, for example. Hence the need for policy to dictate to the maintainer not to allow the package to be removed before all other packages have transitioned. It usually doesn't take much more work as long as the maintainer is even aware of what will happen. Chris
Re: libraries being removed from the archive
#include * LapTop006 [Sun, Aug 03 2003, 03:13:57PM]: > > IMHO we need to make an addition to policy stating that an old lib can > > not be removed from the archive until no other packages still depend on > > it. > How about old libraries can not be removed until either no packages > depend on it OR the author files RC bugs with any dependent pakcages > (and hopefully waits for at least a day in case of any major problems)? Old libraries are removed since only one version can exist in the same distro branch to the same time. If the library maintainer decided not to fork the source package but change the binary package name inside of existing three then he does not care about the users of his package. Such actions (in-place transition) should always be done in coordination with fellow developers as happened in the Perl transition, for example. MfG, Eduard. -- Genossen können nicht die Faust ballen, weil sie überall die Finger drin haben. pgpkyvlyMg8FK.pgp Description: PGP signature
Re: libraries being removed from the archive
On Sat, Aug 02, 2003 at 09:32:37PM -0500, Chris Cheney arranged a set of bits into the following: > Today I was reminded of something that causes apps not to migrate into > sarge. When maintainers remove old libraries from the archive! Today > for example libexif8 was removed by Christophe Barbe and replaced by > libexif9. Guess what that does... any package which depends on libexif8 > now becomes... you guessed it... UNINSTALLABLE! Why does this annoy me > in particular, because I just uploaded kdegraphics yesterday which was > built against libexif8. Guess how many hours it takes for the m68k > buildd to rebuild kdegraphics. OVER 38 HOURS! > > IMHO we need to make an addition to policy stating that an old lib can > not be removed from the archive until no other packages still depend on > it. How about old libraries can not be removed until either no packages depend on it OR the author files RC bugs with any dependent pakcages (and hopefully waits for at least a day in case of any major problems)? This way ancient libraries can go, and the maintainers of packages depending on these libraries don't have to constantly keep an eye on bugs for ftp.d.o. The only downside would be filing all the bugs, and this might be a good way to dissuade people from removing libraries that are widely used. pgpSiCkA9ofoH.pgp Description: PGP signature
libraries being removed from the archive
Today I was reminded of something that causes apps not to migrate into sarge. When maintainers remove old libraries from the archive! Today for example libexif8 was removed by Christophe Barbe and replaced by libexif9. Guess what that does... any package which depends on libexif8 now becomes... you guessed it... UNINSTALLABLE! Why does this annoy me in particular, because I just uploaded kdegraphics yesterday which was built against libexif8. Guess how many hours it takes for the m68k buildd to rebuild kdegraphics. OVER 38 HOURS! IMHO we need to make an addition to policy stating that an old lib can not be removed from the archive until no other packages still depend on it. Chris Cheney
