Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-20 Thread Phillip Susi
Morten Kjeldgaard wrote: > Let me add my 2 cents' worth. I don't know what algorithm is used by > lzma, but I think there are other factors than CPU speed and size that > matters. Namely memory. lzma IS the name of the algorithm, and it is used by the 7zip program. > As an example, I can tell y

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-19 Thread Morten Kjeldgaard
Let me add my 2 cents' worth. I don't know what algorithm is used by lzma, but I think there are other factors than CPU speed and size that matters. Namely memory. As an example, I can tell you that in the past we have experienced problems with the quite serious memory requirements of bunzi

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-19 Thread Lars Wirzenius
On ke, 2007-12-19 at 13:34 -0500, Mackenzie Morgan wrote: > I upgrade a between 2 days and 2 weeks before release to avoid waiting > 4 hours for the upgrade to download on release day, like I did with > Edgy. For releases, one can almost certainly wait a few days for the worst rush to be over. F

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-19 Thread Mackenzie Morgan
On Dec 19, 2007 8:42 AM, Scott Ritchie <[EMAIL PROTECTED]> wrote: > We should consider that it's not always the user's download that gets > saturated, but the mirror's upload. This is especially true on release > day. > > A 30% reduction in filesize would translate to a 30% reduction in > downloa

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-19 Thread Scott Ritchie
Emmet Hikory wrote: > On Dec 18, 2007 9:09 AM, Thilo Six wrote: >> comparation of a whole install (download time + extract time): >> >> download time gz (39084/384)= 101.78s + 14.278s = 116.06s >> download time 7z (27358/384)= 71.24s + 143.783s = 215.02s > > Comparisons like this have far to

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-18 Thread Bryan Quigley
Apologies if I don't understand this correctly. I have a few questions. How will this affect users dist-upgrading? Does 7.10 + 6.06 both have LZMA already installed to allow the upgrade? Will this add a step for Ubuntu developers to make packages for Debian? I like this idea as it looks like i

apt-sync ( was: Changing dpkg-deb default compression from gzip to lzma for Hardy )

2007-12-18 Thread Phillip Susi
Aaron Whitehouse wrote: > On 18/12/2007, Mario Vukelic <[EMAIL PROTECTED]> wrote: >> A while ago I read about changing apt/dpkg to allow for the handling of >> security updates through binary patches. Does anyone know what came out >> of this? > > https://wiki.ubuntu.com/apt-sync The wiki says th

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-18 Thread Matthew Garrett
On Tue, Dec 18, 2007 at 11:09:50AM +0100, Thilo Six wrote: > Only with OOo core moved from gz to 7z we save: 39084 - 27358 = 11726kB on > the CDs!! The LiveCDs don't include the binary debs - the compression is handled by squashfs. I /suspect/ that the limiting factor in the liveCDs is the spee

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-18 Thread Thilo Six
Emmet Hikory wrote the following on 18.12.2007 01:51 > On Dec 18, 2007 9:09 AM, Thilo Six wrote: >> comparation of a whole install (download time + extract time): >> >> download time gz (39084/384)= 101.78s + 14.278s = 116.06s >> download time 7z (27358/384)= 71.24s + 143.783s = 215.02s > >

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-18 Thread Krzysztof Lichota
Thilo Six napisał(a): > Krzysztof Lichota wrote the following on 17.12.2007 13:05 > > recently i also tried 7z for my backups and in comparation to bz2 it seems to > take a bit longer during compressing but is faster on extraction. > Also the compressed file with 7z is ~30% compared to a bz2 one.

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Aaron Whitehouse
On 18/12/2007, Mario Vukelic <[EMAIL PROTECTED]> wrote: > A while ago I read about changing apt/dpkg to allow for the handling of > security updates through binary patches. Does anyone know what came out > of this? https://wiki.ubuntu.com/apt-sync On the main issue, https://wiki.ubuntu.com/Dpkg7Z

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Mario Vukelic
On Tue, 2007-12-18 at 01:09 +0100, Thilo Six wrote: > ~30% less download time A while ago I read about changing apt/dpkg to allow for the handling of security updates through binary patches. Does anyone know what came out of this? It seems to me that for slower connections, binary patches are t

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Mackenzie Morgan
On Dec 17, 2007 7:51 PM, Emmet Hikory <[EMAIL PROTECTED]> wrote: >Bandwidth will always be location dependent, so package size > related to bandwidth is not an easy item for discussion. Smaller is > nice, but for some it doesn't matter very much, as their connections > are either fast enough

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Lars Wirzenius
On ma, 2007-12-17 at 13:05 +0100, Krzysztof Lichota wrote: > It is hard to judge best compression using only one package. On a lark, I ran a little script that re-compresses the components of a .deb with the desired compression program, and reports results: bzip2: Packages, total count: 22485 Or

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Emmet Hikory
On Dec 18, 2007 9:09 AM, Thilo Six wrote: > comparation of a whole install (download time + extract time): > > download time gz (39084/384)= 101.78s + 14.278s = 116.06s > download time 7z (27358/384)= 71.24s + 143.783s = 215.02s Comparisons like this have far too many factors (as you mention

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Thilo Six
Kristian Erik Hermansen wrote the following on 17.12.2007 23:56 > We are talking about a 10x increase in the time it takes to create > DEBs if moving from gz -> lzma. Is this acceptable? deb creation aka compression shouldn´t bother us as much as installing aka extraction. buildds should have eno

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Christian Leber
On Mon, Dec 17, 2007 at 01:05:26PM +0100, Krzysztof Lichota wrote: > > I've heard that lzma will be included by default in main for Hardy. > > This is a very good idea. Changing package build scripts to manually > > pass lzma compression using dh_builddeb -- -Z lzma would be very > > tedious, how

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Kristian Erik Hermansen
We are talking about a 10x increase in the time it takes to create DEBs if moving from gz -> lzma. Is this acceptable? Also, it is more than doubling the work placed on the installer's CPU. This may be acceptable moving forward, if that's what the community and developers decide. However, just

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Thilo Six
Thilo Six wrote the following on 17.12.2007 15:54 > atached is a more detailed analysis trying to compare gz, bz2 and 7z (aka > LZMA). > I tried it once for (mostly) textfiles as source and once on a binary file in > both compressing and extraction. I did the comparion on binaries again. this ti

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Mackenzie Morgan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Scott Ritchie wrote: > I believe lzma has a fairly efficient decompression time. We should > note, however, that package installation time is one of the least > important places to optimize CPU usage - it's not user-interactive, > and is very frequent

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Thilo Six
Krzysztof Lichota wrote the following on 17.12.2007 13:05 recently i also tried 7z for my backups and in comparation to bz2 it seems to take a bit longer during compressing but is faster on extraction. Also the compressed file with 7z is ~30% compared to a bz2 one. > It is hard to judge best

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread David MacKinnon
On Dec 18, 2007 1:00 AM, Krzysztof Lichota <[EMAIL PROTECTED]> wrote: > Why do you think so? I always run system update when I am doing other > things. And I can notice when installing packages starts, although I > have 3 GHz CPU. My personal experience, is that on modern machines, install time i

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Krzysztof Lichota
Scott Ritchie napisał(a): > It's been shown that lzma is, in general, much better. If we happen to > find a specific case where it's not, then we can always set that package > to a non-default by tweaking the dh_builddeb line. I couldn't find any paper about lzma. But you are right, if it can be

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Scott Ritchie
Krzysztof Lichota wrote: > Scott Ritchie napisał(a): >> I did some experimentation with my Wine package. Here's the filesize of >> the latest .deb passing different options to dpkg-deb: >> >> 11081456 default >> 10090930 bzip2 >> 7682608 lzma >> >> That's over a 30% reduction in bandwidth for me a

Re: Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Krzysztof Lichota
Scott Ritchie napisał(a): > I did some experimentation with my Wine package. Here's the filesize of > the latest .deb passing different options to dpkg-deb: > > 11081456 default > 10090930 bzip2 > 7682608 lzma > > That's over a 30% reduction in bandwidth for me and my humble third > party reposi

Changing dpkg-deb default compression from gzip to lzma for Hardy

2007-12-17 Thread Scott Ritchie
I did some experimentation with my Wine package. Here's the filesize of the latest .deb passing different options to dpkg-deb: 11081456 default 10090930 bzip2 7682608 lzma That's over a 30% reduction in bandwidth for me and my humble third party repository. I've heard that lzma will be included