Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-10-14 Thread Christian PERRIER
Quoting Paul Wise (p...@debian.org): > On Thu, Sep 25, 2014 at 10:23 PM, Thomas Goirand wrote: > > > As wrote by others earlier, that's the amount of memory needed for > > compression. 65 MB of RAM is needed for decompression. That's nothing!!! > > That is half the RAM available on my Debian-base

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-10-14 Thread Sebastian Andrzej Siewior
On 2014-09-25 22:23:19 [+0800], Thomas Goirand wrote: > On 09/25/2014 06:02 PM, Wouter Verhelst wrote: > As wrote by others earlier, that's the amount of memory needed for > compression. 65 MB of RAM is needed for decompression. That's nothing!!! just imagine you have one of the recent smaller box

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-28 Thread Lasse Collin
On 2014-09-25 Henrique de Moraes Holschuh wrote: > On Thu, 25 Sep 2014, Riku Voipio wrote: > > On Wed, Sep 24, 2014 at 03:18:02PM -0300, Henrique de Moraes > > Holschuh wrote: > > > OTOH, using -z9 on datasets smaller than the -z8 dictionary size > > > *is* a waste of memory (I don't know about cpu

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-26 Thread Wouter Verhelst
On Thu, Sep 25, 2014 at 10:23:19PM +0800, Thomas Goirand wrote: > On 09/25/2014 06:02 PM, Wouter Verhelst wrote: > > What about the buildd machines that your packages are being built on? [...] > Also, only OpenStack specific packages are compressed with -z9, other > Python modules which may be used

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Paul Wise
On Thu, Sep 25, 2014 at 10:23 PM, Thomas Goirand wrote: > As wrote by others earlier, that's the amount of memory needed for > compression. 65 MB of RAM is needed for decompression. That's nothing!!! That is half the RAM available on my Debian-based phone. Having to shut down the UI just to insta

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Thomas Goirand
On 09/25/2014 06:02 PM, Wouter Verhelst wrote: > Because you can't know what your users *actually* use? Let's say someone > wants to use openstack on a bunch of ARM devices or some such, and they > *don't* have two gigs of RAM? I'd be curious what kind of workload you'd be running on this kind of

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Henrique de Moraes Holschuh
On Thu, 25 Sep 2014, Riku Voipio wrote: > On Wed, Sep 24, 2014 at 03:18:02PM -0300, Henrique de Moraes Holschuh wrote: > > OTOH, using -z9 on datasets smaller than the -z8 dictionary size *is* a > > waste of memory (I don't know about cpu time, and xz(1) doesn't say anything > > on that matter). T

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Riku Voipio
Hi Henrique, On Wed, Sep 24, 2014 at 03:18:02PM -0300, Henrique de Moraes Holschuh wrote: > OTOH, using -z9 on datasets smaller than the -z8 dictionary size *is* a > waste of memory (I don't know about cpu time, and xz(1) doesn't say anything > on that matter). The same goes for -z8 and datasets

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Wouter Verhelst
On Thu, Sep 25, 2014 at 03:09:16PM +0800, Thomas Goirand wrote: > On 09/25/2014 02:18 AM, Henrique de Moraes Holschuh wrote: > > On Thu, 25 Sep 2014, Thomas Goirand wrote: > >> On 09/02/2014 09:39 PM, Henrique de Moraes Holschuh wrote: > >>> For -z9, it is as bad as ~670MiB to > >>> compress, and ~

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-25 Thread Thomas Goirand
On 09/25/2014 02:18 AM, Henrique de Moraes Holschuh wrote: > On Thu, 25 Sep 2014, Thomas Goirand wrote: >> On 09/02/2014 09:39 PM, Henrique de Moraes Holschuh wrote: >>> For -z9, it is as bad as ~670MiB to >>> compress, and ~65MiB to decompress. >> >> I'd say this really depends on what you do. For

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Henrique de Moraes Holschuh
On Thu, 25 Sep 2014, Thomas Goirand wrote: > On 09/02/2014 09:39 PM, Henrique de Moraes Holschuh wrote: > > For -z9, it is as bad as ~670MiB to > > compress, and ~65MiB to decompress. > > I'd say this really depends on what you do. For what I do (eg: OpenStack > packages), I don't see how 65MB cou

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Thomas Goirand
On 09/04/2014 04:58 PM, Ansgar Burchardt wrote: > On 09/04/2014 10:49, Thorsten Glaser wrote: >> On Wed, 3 Sep 2014, Christian Kastner wrote: >>> That is the key question, and I believe considering the worst possible >>> cost -- a package that cannot be unpacked, as in #757740 -- the >>> trade-off

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Thomas Goirand
On 09/04/2014 03:10 AM, Christian Kastner wrote: > Benefits: from the numbers posted in this thread, the size savings > compared to the default compression level for some sample packages are > somewhere around 3% to 4%. I claim that *practical* benefits from this > saving are insignificant to minor

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Thomas Goirand
On 09/03/2014 01:49 PM, Andrey Rahmatullin wrote: > Decompression costs were mentioned too, and they always matter I don't agree. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Thomas Goirand
On 09/02/2014 09:39 PM, Henrique de Moraes Holschuh wrote: > For -z9, it is as bad as ~670MiB to > compress, and ~65MiB to decompress. I'd say this really depends on what you do. For what I do (eg: OpenStack packages), I don't see how 65MB could be a problem. I do compress with -z9, and have no in

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-24 Thread Hideki Yamane
Hi, It's not abuse, "possible" abuse (IMO, at least :) - xz -z9 is very efficient for some cases (e.g. huge font package) - good for mirror infrastructures and low-bandwidth client On Thu, 4 Sep 2014 10:49:59 +0200 Thorsten Glaser wrote: > Such as mips? Surely it would be not comfortable

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-05 Thread Charles Plessy
Le Fri, Sep 05, 2014 at 11:13:40AM +0200, Thorsten Glaser a écrit : > > So please, do not outright dismiss scenarios you personally > cannot imagine. Please consider that your comments on the limited imagination of others can be felt as deliberately offensive. Cheers, -- Charles Plessy -- T

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-05 Thread Thorsten Glaser
On Fri, 5 Sep 2014, Changwoo Ryu wrote: > As I said, such lowmem embeded devices don't even need to install big > packages. Just you saying so doesn’t make it (more) true. Debian is a universal operating system… at least it tries to. Maybe one of these packages contains _one_ file you need to bui

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-04 Thread Changwoo Ryu
2014-09-04 4:10 GMT+09:00 Christian Kastner : > On 2014-09-03 13:10, Changwoo Ryu wrote: >> 2014-09-03 17:04 GMT+09:00 Simon McVittie : >>> On 02/09/14 21:17, Changwoo Ryu wrote: For fonts-nanum, the default is ~300 KiB 3.5% larger than -9e. And -9e is not better than -8e. >>> >>> I don't

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-04 Thread Ondřej Surý
On Thu, Sep 4, 2014, at 10:58, Ansgar Burchardt wrote: > What about a lintian warning (error, whatever) instead? I support that... lintian warning for non-default compression lintian (auto-reject) error for > 7 O. -- Ondřej Surý Knot DNS (https://www.knot-dns.cz/) – a high-performance DNS serv

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-04 Thread Ansgar Burchardt
On 09/04/2014 10:49, Thorsten Glaser wrote: > On Wed, 3 Sep 2014, Christian Kastner wrote: >> That is the key question, and I believe considering the worst possible >> cost -- a package that cannot be unpacked, as in #757740 -- the >> trade-off is not worth it. > > IIRC, I asked last year already

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-04 Thread Thorsten Glaser
Jonathan Dowland wrote: > I suppose we should always be sympathetic towards such architectures, but at > the end of the day we should primarily concern ourselves with release > architectures. Such as mips? On Wed, 3 Sep 2014, Christian Kastner wrote: > That is the key question, and I believe c

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Jonathan Dowland
On Wed, Sep 03, 2014 at 10:59:44AM +0200, Thorsten Glaser wrote: > Not on avr32, and it hurts sh4, m68k and others as well. I suppose we should always be sympathetic towards such architectures, but at the end of the day we should primarily concern ourselves with release architectures. -- To UNS

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Christian Kastner
On 2014-09-03 13:10, Changwoo Ryu wrote: > 2014-09-03 17:04 GMT+09:00 Simon McVittie : >> On 02/09/14 21:17, Changwoo Ryu wrote: >>> For fonts-nanum, the default is ~300 KiB 3.5% larger than -9e. And -9e >>> is not better than -8e. >> >> I don't think anyone is arguing that higher compression setti

Re: Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Changwoo Ryu
2014-09-04 1:18 GMT+09:00 Fabian Greffrath : > Hi Changwoo Ryu, > >> I think yes. The cost is 24 MiB extra memory on installation, and >> benefits are bandwidth and mirror size saving of big packages. > > I beg to differ. Those few kiB of bandwidth (yes, I mean it like that) > saved when downloadin

Re: Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Fabian Greffrath
Hi Changwoo Ryu, > I think yes. The cost is 24 MiB extra memory on installation, and > benefits are bandwidth and mirror size saving of big packages. I beg to differ. Those few kiB of bandwidth (yes, I mean it like that) saved when downloading a package are worthless if the decompresion routine f

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Thorsten Glaser
On Wed, 3 Sep 2014, Changwoo Ryu wrote: > I can't imagine any 31 MiB machine which needs to render megabytes of It’s *additional* memory. On top of kernel, libc, apt, dpkg, and whatever the user is running in parallel. bye, //mirabilos -- Sometimes they [people] care too much: pretty printers [

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Changwoo Ryu
2014-09-03 17:04 GMT+09:00 Simon McVittie : > On 02/09/14 21:17, Changwoo Ryu wrote: >> For fonts-nanum, the default is ~300 KiB 3.5% larger than -9e. And -9e >> is not better than -8e. > > I don't think anyone is arguing that higher compression settings don't > produce better compression ratios. H

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Thorsten Glaser
On Wed, 3 Sep 2014, Changwoo Ryu wrote: > I think 65MIB for decompressing is OK with current hardwares as long > as it saves good amount of space and bandwidth. Not on avr32, and it hurts sh4, m68k and others as well. bye, //mirabilos -- >> Why don't you use JavaScript? I also don't like enabl

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-03 Thread Simon McVittie
On 02/09/14 21:17, Changwoo Ryu wrote: > For fonts-nanum, the default is ~300 KiB 3.5% larger than -9e. And -9e > is not better than -8e. I don't think anyone is arguing that higher compression settings don't produce better compression ratios. However: Preset DictSize Com

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Changwoo Ryu
2014-09-03 14:49 GMT+09:00 Andrey Rahmatullin : > On Wed, Sep 03, 2014 at 07:32:49AM +0200, Christian PERRIER wrote: >> At the time, we (font team) decided to go with z9, the fact that >> packages were arch:all (and therefore that the memory cost of >> compression had only an impact on the machine

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Andrey Rahmatullin
On Wed, Sep 03, 2014 at 07:32:49AM +0200, Christian PERRIER wrote: > At the time, we (font team) decided to go with z9, the fact that > packages were arch:all (and therefore that the memory cost of > compression had only an impact on the machine of the developer who > builds packages), was a strong

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Christian PERRIER
Quoting Holger Levsen (hol...@layer-acht.org): > On Dienstag, 2. September 2014, Jonathan Dowland wrote: > > They're all arch: all though, right, so in practice no buildds are actually > > building these packages? > > yet. > > :-) > > At the time, we (font team) decided to go with z9, the fact

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Changwoo Ryu
2014-09-03 3:42 GMT+09:00 Sebastian Andrzej Siewior : > On 2014-09-02 14:08:57 [+0900], Changwoo Ryu wrote: >> "dh_builddeb -- -Zxz -Sextreme -z9" has been introduced to the >> pkg-font team when dpkg-deb default is not xz. >> >> In my quick experiments with some font packages, "-Sextream -z9" >> o

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Sebastian Andrzej Siewior
On 2014-09-02 14:08:57 [+0900], Changwoo Ryu wrote: > "dh_builddeb -- -Zxz -Sextreme -z9" has been introduced to the > pkg-font team when dpkg-deb default is not xz. > > In my quick experiments with some font packages, "-Sextream -z9" > option still gives ~4% smaller size than the default. IMO th

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Holger Levsen
On Dienstag, 2. September 2014, Jonathan Dowland wrote: > They're all arch: all though, right, so in practice no buildds are actually > building these packages? yet. :-) signature.asc Description: This is a digitally signed message part.

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Henrique de Moraes Holschuh
On Tue, 02 Sep 2014, Jonathan Dowland wrote: > On Tue, Sep 02, 2014 at 01:45:30PM +0200, Adam Borowski wrote: > > For any standardish font, taking any extra memory is a no-no. You might be > > running in a chroot on a 256MB RAM phone, etc. > > > > On the other hand, for a 400MB game, not using -z

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Jonathan Dowland
On Tue, Sep 02, 2014 at 01:45:30PM +0200, Adam Borowski wrote: > For any standardish font, taking any extra memory is a no-no. You might be > running in a chroot on a 256MB RAM phone, etc. > > On the other hand, for a 400MB game, not using -z9 is a pure waste of space. They're all arch: all thou

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Adam Borowski
On Tue, Sep 02, 2014 at 01:24:40PM +0200, Thorsten Glaser wrote: > On Tue, 2 Sep 2014, Changwoo Ryu wrote: > > > In my quick experiments with some font packages, "-Sextream -z9" > > option still gives ~4% smaller size than the default. IMO this is > > still significant for big font packages. > >

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-02 Thread Thorsten Glaser
On Tue, 2 Sep 2014, Changwoo Ryu wrote: > In my quick experiments with some font packages, "-Sextream -z9" > option still gives ~4% smaller size than the default. IMO this is > still significant for big font packages. Maybe, but please stick to -Sextreme -z7 at most, nevertheless. Thanks, //mir

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-01 Thread Christian PERRIER
Quoting Changwoo Ryu (cw...@debian.org): > In my quick experiments with some font packages, "-Sextream -z9" > option still gives ~4% smaller size than the default. IMO this is > still significant for big font packages. Yes, that choice of the fonts team is based on a detailed experiment conducte

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-01 Thread Changwoo Ryu
2014-09-01 21:20 GMT+09:00 Guillem Jover : > There seems to be some packages overriding the default compression > level for xz to 9. This means dpkg-deb will require way more memory on > both compression and decompression usually for extremely little gain, > and might even fail on some systems with

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-01 Thread Bastien ROUCARIES
Le 1 sept. 2014 14:21, "Guillem Jover" a écrit : > > Hi! > > I had noticed this a while ago while reading changelogs, but didn't > realize at the time this poses actual problems, besides being possibly > just a dubious practice. > > There seems to be some packages overriding the default compressio

Re: Possible abuse of dpkg-deb -z9 for xz compressed binary packages

2014-09-01 Thread Daniel Baumann
On 09/01/2014 02:20 PM, Guillem Jover wrote: > it seems that most packages are or have been maintained by Daniel Baumann > or the Fonts Team (both CCed). i've switched to the default compression december 2012/january 2013 in all my packages (so they use an inherited -z6 as everyone else uses/shoul