/usr/share/doc/ files and gzip/xz/no compression

[Lars Wirzenius]

On Mon, Aug 15, 2011 at 05:16:55PM +0900, Charles Plessy wrote:
> Le Mon, Aug 15, 2011 at 01:48:50AM +0200, Adam Borowski a écrit :
> > 
> > * A year ago, I repacked CD1, .xz took 66% space needed by .gz.  This time,
> >   on the whole archive, gains are somewhat smaller: 72%.  I guess that CD1
> >   is code-heavy while packages of lower priorities tend to have more data.
> 
> Also, many files in /usr/share/doc are gzipped as per §12.3; that can prevent
> to get the full benefit of xz compression.  In some of my packages containing
> mostly such files, the benefit of switching to xz is almost null.  I wonder if
> it still makes sense to compress these files by default:
> 
>  - Most systems have enough space to keep them uncompressed,
>  - others systems just do not install these files,
>  - some filesystems are compressed on the fly,
>  - the binary packages themselves are compressed.

On the other hand, many computers now have an SSD drive, for speed,
which is relatively small. Further, most users will likely need files in
/usr/share/doc rarely, if ever, so not compressing things risks wasting
a bunch of disk space for no particular benefit.

To get some actual numbers, I wrote the attached script. On my laptop
running squeeze, it reports:

Total size of *.gz files in /usr/share/doc: 170542915
Total size of uncompressed *.gz files in /usr/share/doc: 611945610
Total size of *.gz files in /usr/share/doc converted into *.xz: 140588208

That indicates that compressing documentation with xz instead of gz
does not save a whole lot (but does save some), but not compressing at
all wastes a lot. Putting the numbers into a table for easier comparison:

 raw gz  xz
 584163 134 file sizes (MiB)
   0421 450 savings compared to raw (MiB)
-421  0  29 savings compared to current gz (MiB)

So I would definitely vote for continuing to compress files in
/usr/share/doc. (Note that these numbers cover only files that are
currently *.gz, not all files in /usr/share/doc. See script for
details.)

I'm OK with allowing use of xz for compressing the files.

-- 
Freedom-based blog/wiki/web hosting: http://www.branchable.com/
#!/bin/sh

set -e

gzsum=$(find /usr/share/doc -type f -name '*.gz' -printf '%s\n' | 
awk '{ s += $1 } END { print s }')
echo "Total size of *.gz files in /usr/share/doc: $gzsum"

rawsum=$(find /usr/share/doc -type f -name '*.gz' -exec zcat '{}' + | wc -c)
echo "Total size of uncompressed *.gz files in /usr/share/doc: $rawsum"

xzsum=$(find /usr/share/doc -type f -name '*.gz' -print0 |
xargs -0n1 -I'{}' -- sh -c 'zcat {} | xz | wc -c' |
awk '{ s += $1 } END { print s }')
echo "Total size of *.gz files in /usr/share/doc converted into *.xz: $xzsum"

Re: /usr/share/doc/ files and gzip/xz/no compression

[Thorsten Glaser]

Carsten Hey dixit:

>IIRC bzip2 had a better compression.  Compressing dpkg's changelog on
>stable seems confirm this:

xz’s default compression level -6 is not good for files
smaller than 8 MiB. Try -2 instead, maybe -2e (slower).
Besides, it decompresses a lot faster than bzip2, so even
in case of a (slight) size increase from bzip2 to xz, I’d
choose xz every day anyway.

bye,
//mirabilos, wondering how many *.gz don’t use gzip -n9 either
-- 
FWIW, I'm quite impressed with mksh interactively. I thought it was much
*much* more bare bones. But it turns out it beats the living hell out of
ksh93 in that respect. I'd even consider it for my daily use if I hadn't
wasted half my life on my zsh setup. :-) -- Frank Terbeck in #!/bin/mksh


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/pine.bsm.4.64l.1205221227450.23...@herc.mirbsd.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Andreas Barth]

* Lars Wirzenius (l...@liw.fi) [110815 19:36]:
>  584163 134 file sizes (MiB)

Thanks for comparing these numbers. That tells me that at least in the
average case we just can continue with gz, and not care much about the
relativly small difference to xz.


Andi


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815174146.gc15...@mails.so.argh.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Lucas Nussbaum]

On 15/08/11 at 19:41 +0200, Andreas Barth wrote:
> * Lars Wirzenius (l...@liw.fi) [110815 19:36]:
> >  584163 134 file sizes (MiB)
> 
> Thanks for comparing these numbers. That tells me that at least in the
> average case we just can continue with gz, and not care much about the
> relativly small difference to xz.

I wouldn't call -20% a relatively small difference.

The question is: are there downsides to switching to xz for
/usr/share/doc?

For example, in the past, some PDF readers could not directly open
.pdf.gz files, so one had to uncompress the file manually first, which
was pretty annoying.  I just checked, and evince, okular and xpdf can
open .pdf.xz files. But maybe other tools have similar issues.

- Lucas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815191525.ga1...@xanadu.blop.info

Re: /usr/share/doc/ files and gzip/xz/no compression

[Carsten Hey]

* Lars Wirzenius [2011-08-15 18:33 +0100]:
>  raw gz  xz
>  584163 134 file sizes (MiB)
>0421 450 savings compared to raw (MiB)
> -421  0  29 savings compared to current gz (MiB)

Years ago I compared sizes of compressed files in /usr/share/doc using
different compression methods too, possibly restricting to specific file
types (for example changelog and copyright).

> I'm OK with allowing use of xz for compressing the files.

IIRC bzip2 had a better compression.  Compressing dpkg's changelog on
stable seems confirm this:

$ zcat /usr/share/doc/dpkg/changelog.gz | bzip2 | wc -c
145586
$ zcat /usr/share/doc/dpkg/changelog.gz | xz | wc -c
167844


Regards
Carsten


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815210451.ga25...@furrball.stateful.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Lars Wirzenius]

On Mon, Aug 15, 2011 at 11:04:51PM +0200, Carsten Hey wrote:
> * Lars Wirzenius [2011-08-15 18:33 +0100]:
> >  raw gz  xz
> >  584163 134 file sizes (MiB)
> >0421 450 savings compared to raw (MiB)
> > -421  0  29 savings compared to current gz (MiB)
> 
> Years ago I compared sizes of compressed files in /usr/share/doc using
> different compression methods too, possibly restricting to specific file
> types (for example changelog and copyright).
> 
> > I'm OK with allowing use of xz for compressing the files.
> 
> IIRC bzip2 had a better compression.  Compressing dpkg's changelog on

Adding bzip2 support to my script and re-running gives me

Total size of *.gz files in /usr/share/doc converted into *.bz2: 135179633

In other words, it's 130 MiB against xz's 134 MiB. I'll leave it to
others to decide if it's a significatn difference.

-- 
Freedom-based blog/wiki/web hosting: http://www.branchable.com/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815212515.ga24...@havelock.liw.fi

Re: /usr/share/doc/ files and gzip/xz/no compression

[Andreas Barth]

* Lars Wirzenius (l...@liw.fi) [110815 23:27]:
> On Mon, Aug 15, 2011 at 11:04:51PM +0200, Carsten Hey wrote:
> > * Lars Wirzenius [2011-08-15 18:33 +0100]:
> > >  raw gz  xz
> > >  584163 134 file sizes (MiB)
> > >0421 450 savings compared to raw (MiB)
> > > -421  0  29 savings compared to current gz (MiB)

> In other words, it's 130 MiB against xz's 134 MiB. I'll leave it to
> others to decide if it's a significatn difference.

bzip2 is definitly a more conservative choice than xz. If it's
smaller, than it's superior to xz.


Andi


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815215907.gd15...@mails.so.argh.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Iustin Pop]

On Mon, Aug 15, 2011 at 11:59:07PM +0200, Andreas Barth wrote:
> * Lars Wirzenius (l...@liw.fi) [110815 23:27]:
> > On Mon, Aug 15, 2011 at 11:04:51PM +0200, Carsten Hey wrote:
> > > * Lars Wirzenius [2011-08-15 18:33 +0100]:
> > > >  raw gz  xz
> > > >  584163 134 file sizes (MiB)
> > > >0421 450 savings compared to raw (MiB)
> > > > -421  0  29 savings compared to current gz (MiB)
> 
> > In other words, it's 130 MiB against xz's 134 MiB. I'll leave it to
> > others to decide if it's a significatn difference.
> 
> bzip2 is definitly a more conservative choice than xz. If it's
> smaller, than it's superior to xz.

AFAIK, bzip2 has much worse decompression performance than xz: I have
taken dpkg's changelog, concatenated it to itself 10 times (11MB size),
and:

gzip: 0.377s, down to 2.7MB
gunzip: 0.077s

bzip2: 1.45s, down to 1.8M
bunzip2: 0.420s

xz: 4.4s(!), down to 204K(!)
xz -d: 0.035s

So here bzip is an order of magnitude slower at decompression.

I've repeated the test on uncompressible data (/dev/urandom), 10MB, and
the numbers are even worse for bzip2:

gzip:  0.410s / 0.060s
bzip2: 2.400s / 0.960s
xz:4.040s / 0.027s

So while xz is costly for compression, it's faster than even gzip for
decompression. bzip2's cost for decompresion (huge!) is what kept me
personally from using it seriously before xz appeared.

There is also information on Wikipedia about various compression
benchmarks, but IMHO if we want to switch from gzip then bzip2
doesn't make sense for /usr/share/doc.

regards,
iustin


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Carsten Hey]

* Andreas Barth [2011-08-15 23:59 +0200]:
> * Lars Wirzenius (l...@liw.fi) [110815 23:27]:
> > On Mon, Aug 15, 2011 at 11:04:51PM +0200, Carsten Hey wrote:
> > > * Lars Wirzenius [2011-08-15 18:33 +0100]:
> > > >  raw gz  xz
> > > >  584163 134 file sizes (MiB)
> > > >0421 450 savings compared to raw (MiB)
> > > > -421  0  29 savings compared to current gz (MiB)
>
> > In other words, it's 130 MiB against xz's 134 MiB. I'll leave it to
> > others to decide if it's a significatn difference.
>
> bzip2 is definitly a more conservative choice than xz. If it's
> smaller, than it's superior to xz.

bzip2 has a better compression on average for some filetypes, xz[1] has
a better compression on average for others:

   gzip  bzip2   xz bzip2+xz[3]
  text files[2]   94312922  73496587  77783076  73496587
  other files 16577181  14609893  14275484  14275484
  sum110890103  88106480  92058560  87772071

Among the "other files" are also a lot of text files, if we would
compress Debian packages instead, xz would win presumably.

Anyway, I don't think this difference of 4 MiB on a desktop system is
significant.


I would prefer to avoid bloating the set of pseudo essential packages
without a good reason and I think users should be able to decompress all
files in /u/s/d.  There are plans to let dpkg depend on liblzma2 instead
of xz and it already depends on libbz2-1.0.  If dpkg's dependency on
libbz2 is planned to be removed in future, I would prefer to let libbz2
vanish from the pseudo essential set and use xz also for /u/s/d,
otherwise I would prefer using bzip2 over xz for /u/s/d.


Carsten


 [1] I did not use -e nor -9, but the difference should not be that big
 on files in /usr/share/doc.
 [2] find ... -regex '.*\(changelog\|copyright\|README\|TODO\|NEWS\).*[.]gz'
 [3] bzip2 for text files and xz for other files.  This is of course
 nothing we should consider doing.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110815230257.ga27...@furrball.stateful.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Russell Coker]

On Tue, 16 Aug 2011, Iustin Pop  wrote:
> There is also information on Wikipedia about various compression
> benchmarks, but IMHO if we want to switch from gzip then bzip2
> doesn't make sense for /usr/share/doc.

I'd like to see zless work transparently with bzip and xz compressed files.  
There's really no need for three different wrapper programs when the zless 
program can just consult the magic db to determine which decompression program 
to use.

A switch inevitably involves a period of time where we have a mixture of 
compression methods in use.  Even after that there will be a variety of old 
data (I'm sure that I'm not the only person who has been using gzip for most 
things because it's usually good enough and it's a matter of habit).  As most 
Unix commands aren't consciously typed having multiple variants of zless would 
be a real drag.

-- 
My Main Blog http://etbe.coker.com.au/
My Documents Bloghttp://doc.coker.com.au/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201108161043.04786.russ...@coker.com.au

Re: /usr/share/doc/ files and gzip/xz/no compression

[Ben Hutchings]

On Tue, 2011-08-16 at 10:43 +1000, Russell Coker wrote:
> On Tue, 16 Aug 2011, Iustin Pop  wrote:
> > There is also information on Wikipedia about various compression
> > benchmarks, but IMHO if we want to switch from gzip then bzip2
> > doesn't make sense for /usr/share/doc.
> 
> I'd like to see zless work transparently with bzip and xz compressed files.  
> There's really no need for three different wrapper programs when the zless 
> program can just consult the magic db to determine which decompression 
> program 
> to use.
[...]

+1

After all, it isn't called gzless.

Ben.



signature.asc
Description: This is a digitally signed message part

Re: /usr/share/doc/ files and gzip/xz/no compression

[Chris Knadle]

On Monday, August 15, 2011, Ben Hutchings wrote:
> On Tue, 2011-08-16 at 10:43 +1000, Russell Coker wrote:
> > On Tue, 16 Aug 2011, Iustin Pop  wrote:
> > > There is also information on Wikipedia about various
> > > compression benchmarks, but IMHO if we want to switch from
> > > gzip then bzip2 doesn't make sense for /usr/share/doc.
> > 
> > I'd like to see zless work transparently with bzip and xz
> > compressed files. There's really no need for three different
> > wrapper programs when the zless program can just consult the
> > magic db to determine which decompression program to use.
> 
> [...]
> 
> +1
> 
> After all, it isn't called gzless.

Actually, that's not far off.  Right now these functions are spread 
across 4 different binaries, each of which is in a different package.

File Package
==   ===
/bin/bzless -> bzmorebzip2
/bin/bzmore  bzip2
/bin/zless   gzip
/usr/bin/less -> /bin/less   less
/bin/lessless
/usr/bin/xzless  xz-utils


  -- Chris

-- 

Chris Knadle
chris.kna...@coredump.us


signature.asc
Description: This is a digitally signed message part.

Re: /usr/share/doc/ files and gzip/xz/no compression

[Russell Coker]

On Tue, 16 Aug 2011, Chris Knadle  wrote:
> > After all, it isn't called gzless.
> 
> Actually, that's not far off.  Right now these functions are spread 
> across 4 different binaries, each of which is in a different package.

Probably the best thing to do would be to just incorporate the functionality 
in the less package.

zless works with uncompressed files and is well known to work that way.  I'm 
sure that lots of people run "zless *" without regard to whether there are 
compressed or uncompressed files in the directory.  Having zless and less do 
the same thing and support a wide variety of compression methods would be 
ideal.  It's not as if anyone would use less to view the binary contents of a 
gzip data stream.

-- 
My Main Blog http://etbe.coker.com.au/
My Documents Bloghttp://doc.coker.com.au/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201108161306.34484.russ...@coker.com.au

Re: /usr/share/doc/ files and gzip/xz/no compression

[James Vega]

On Tue, Aug 16, 2011 at 01:06:34PM +1000, Russell Coker wrote:
> On Tue, 16 Aug 2011, Chris Knadle  wrote:
> > > After all, it isn't called gzless.
> > 
> > Actually, that's not far off.  Right now these functions are spread 
> > across 4 different binaries, each of which is in a different package.
> 
> Probably the best thing to do would be to just incorporate the functionality 
> in the less package.

You mean like lesspipe(1)?  Seems like it might need to be updated to
handle *.xz, but other than that looks like it fits the bill.

-- 
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega 


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Chris Knadle]

On Monday, August 15, 2011, Russell Coker wrote:
> On Tue, 16 Aug 2011, Chris Knadle  wrote:
> > > After all, it isn't called gzless.
> > 
> > Actually, that's not far off.  Right now these functions are
> > spread across 4 different binaries, each of which is in a
> > different package.
> 
> Probably the best thing to do would be to just incorporate the
> functionality in the less package.
> 
> zless works with uncompressed files and is well known to work that
> way.  I'm sure that lots of people run "zless *" without regard to
> whether there are compressed or uncompressed files in the
> directory.

I commonly do that.  Not just with zless but even more commonly with 
zfgrep.  [And this hints that this issue is larger than just 'less'.]

Apparently bzmore and xzless are likewise able to read plaintext 
files, so these binaries are also doing file type recognition.

> Having zless and less do the same thing and support a
> wide variety of compression methods would be ideal.  It's not as
> if anyone would use less to view the binary contents of a gzip
> data stream.

I think you're onto a good idea.

It seems like it should be possible to build one 'less' such that it 
will read all of the compressed file types, and have zless xzless and 
bzless all point to 'less' via softlinks.  (Or hardlinks, if they're 
in the same directory, similar to how used to be done with separate 
'git-' binaries after they were consolidated into one 'git' 
binary.)

I'm trying to think of what undesired effects this might have.

One unexpected consequence would be that calling 'less' by default 
wouldn't skip over compressed file types that it would know how to 
handle.  If it were desired to be able to skip over compressed files, 
'less' could be crafted to change behavior based on the filename that 
was actually called (as was likewise done with git), thus zless bzless 
and xzless would open compressed files, but 'less' itself could avoid 
doing so.  Or add yet another command-line option as to whether to 
open compressed file types or not.

  -- Chris

-- 

Chris Knadle
chris.kna...@coredump.us


signature.asc
Description: This is a digitally signed message part.

Re: /usr/share/doc/ files and gzip/xz/no compression

[Alexander Reichle-Schmehl]

Hi!

Am 16.08.2011 02:43, schrieb Russell Coker:

> I'd like to see zless work transparently with bzip and xz compressed files.  
> There's really no need for three different wrapper programs when the zless 
> program can just consult the magic db to determine which decompression 
> program 
> to use.

FWIW, if you the following in your bashrc:

if [ -f /usr/bin/lesspipe ]; then
eval "$(lesspipe)"
fi

you can just "less somegzfile" or "less somebzipfile".  In squeeze you
can't use less on a xz compressed file, but I guess that could be added.


The bashrc snippet

Best regards,
  Alexander


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4a258d.6090...@debian.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Chow Loong Jin]

On 16/08/2011 16:08, Alexander Reichle-Schmehl wrote:
> Hi!
> 
> Am 16.08.2011 02:43, schrieb Russell Coker:
> 
>> I'd like to see zless work transparently with bzip and xz compressed files.  
>> There's really no need for three different wrapper programs when the zless 
>> program can just consult the magic db to determine which decompression 
>> program 
>> to use.
> 
> FWIW, if you the following in your bashrc:
> 
> if [ -f /usr/bin/lesspipe ]; then
> eval "$(lesspipe)"
> fi
> 
> you can just "less somegzfile" or "less somebzipfile".  In squeeze you
> can't use less on a xz compressed file, but I guess that could be added.

That works when you do "less somefile.gz" or something, but doesn't work out so
well on gz/bz2 streams, unfortunately. Isn't it possible to detect this based on
some magic strings at the beginning of the stream?

-- 
Kind regards,
Loong Jin



signature.asc
Description: OpenPGP digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Adam Borowski]

On Tue, Aug 16, 2011 at 10:08:45AM +0200, Alexander Reichle-Schmehl wrote:
> Am 16.08.2011 02:43, schrieb Russell Coker:
> 
> > I'd like to see zless work transparently with bzip and xz compressed files. 
> >  
> > There's really no need for three different wrapper programs when the zless 
> > program can just consult the magic db to determine which decompression 
> > program 
> > to use.
> 
> FWIW, if you the following in your bashrc:
> 
> if [ -f /usr/bin/lesspipe ]; then
> eval "$(lesspipe)"
> fi
> 
> you can just "less somegzfile" or "less somebzipfile".  In squeeze you
> can't use less on a xz compressed file, but I guess that could be added.

lesspipe doesn't support xz in unstable, too.

There's a patch in #572228 (and at least one fool just wrote a duplicate
before checking for existing ones), but the patch is rotting in the BTS.

Anibal: would you care to apply it?

-- 
1KB // Yo momma uses IPv4!


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Luca Capello]

Hi there!

On Tue, 16 Aug 2011 10:08:45 +0200, Alexander Reichle-Schmehl wrote:
> Am 16.08.2011 02:43, schrieb Russell Coker:
>
>> I'd like to see zless work transparently with bzip and xz compressed files.  
>> There's really no need for three different wrapper programs when the zless 
>> program can just consult the magic db to determine which decompression 
>> program 
>> to use.
>
> FWIW, if you the following in your bashrc:
>
> if [ -f /usr/bin/lesspipe ]; then

I would use the -x expression instead ;-)

And I would also say that this test should be included in the
lesspipe(1) manpage...

Thx, bye,
Gismo / Luca


pgpvk0q8rDLom.pgp
Description: PGP signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Adam Borowski]

On Tue, Aug 16, 2011 at 04:44:07PM +0800, Chow Loong Jin wrote:
> > Am 16.08.2011 02:43, schrieb Russell Coker:
> >> I'd like to see zless work transparently with bzip and xz compressed 
> >> files.  
> >> There's really no need for three different wrapper programs when the zless 
> >> program can just consult the magic db to determine which decompression 
> >> program 
> >> to use.
> > 
> > eval "$(lesspipe)"
> > 
> > you can just "less somegzfile" or "less somebzipfile".  In squeeze you
> > can't use less on a xz compressed file, but I guess that could be added.
> 
> That works when you do "less somefile.gz" or something, but doesn't work out 
> so
> well on gz/bz2 streams, unfortunately. Isn't it possible to detect this based 
> on
> some magic strings at the beginning of the stream?

lesspipe tries to support a long list of compression formats.  On a .pdf, it
will go through pdftotext, on a .iso, it will list the files contained, and
so on.  It'd be tricky to implement all of that as magic rather than
extension checks, especially for twice encapsulated files (.tar.gz, etc).
Doable of course, but on non-seekable streams it'd require some work.

On the other hand, zcat and zless are nothing but thin shell wrappers over
"gzip -cd", without loads of extra functionality lesspipe has.  I'd turn
them into a C program linked against zlib (priority: required), liblzma2
(priority: required) and libbz2 (priority: important, might be dlopened
instead).

"zcat" already supports multiple formats: gzip and compress, it's just that
support for uncompressing the latter is included in gzip's binary.  Since
it's not called "gzcat" but generic "zcat", I think that's a better idea
than proliferation of bzcat, xzcat, my-newest-formatcat.

What would you guys say?

-- 
1KB // Yo momma uses IPv4!


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Adam Borowski]

On Tue, Aug 16, 2011 at 11:50:49AM +0200, Luca Capello wrote:
> On Tue, 16 Aug 2011 10:08:45 +0200, Alexander Reichle-Schmehl wrote:
> > FWIW, if you the following in your bashrc:
> >
> > if [ -f /usr/bin/lesspipe ]; then
> 
> I would use the -x expression instead ;-)
> 
> And I would also say that this test should be included in the
> lesspipe(1) manpage...

/etc/skel/.profile has it, commented out:

# make less more friendly for non-text input files, see lesspipe(1)
#[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

-- 
1KB // Yo momma uses IPv4!


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Vincent Lefevre]

On 2011-08-15 23:29:17 -0400, James Vega wrote:
> You mean like lesspipe(1)?  Seems like it might need to be updated to
> handle *.xz, but other than that looks like it fits the bill.

lesspipe(1) from the less package is a bit primitive. How about using
Wolfgang Friebel's version, which already supports xz?

http://www-zeuthen.desy.de/~friebel/unix/lesspipe.html

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110816104339.gl20...@prunille.vinc17.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Luca Capello]

Hi there!

On Tue, 16 Aug 2011 12:43:39 +0200, Vincent Lefevre wrote:
> On 2011-08-15 23:29:17 -0400, James Vega wrote:
>> You mean like lesspipe(1)?  Seems like it might need to be updated to
>> handle *.xz, but other than that looks like it fits the bill.
>
> lesspipe(1) from the less package is a bit primitive. How about using
> Wolfgang Friebel's version, which already supports xz?
>
> http://www-zeuthen.desy.de/~friebel/unix/lesspipe.html

Some reasons from the old maintainer:

  

It happened I saw that it contains code from you, Vincent:

  

  
Thx, bye,
Gismo / Luca


pgpuTHTFFuuhb.pgp
Description: PGP signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Russ Allbery]

Adam Borowski  writes:

> On the other hand, zcat and zless are nothing but thin shell wrappers
> over "gzip -cd", without loads of extra functionality lesspipe has.  I'd
> turn them into a C program linked against zlib (priority: required),
> liblzma2 (priority: required) and libbz2 (priority: important, might be
> dlopened instead).

One angle of this discussion is confusing me: zcat and zless are upstream
binaries that are part of a package that we build from upstream source.
That upstream is gzip; presumably the gzip maintainers aren't particularly
interested in adding build dependencies on other compression formats.  (I
wouldn't be if I were them; it seems a bit outside their bailiwick.)

Are people who are proposing that we change these programs proposing we
build a new package with binaries of those same names and use
alternatives?  That Debian fork gzip?  Something else?

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87pqk5xn19@windlord.stanford.edu

Re: /usr/share/doc/ files and gzip/xz/no compression

[Ben Hutchings]

On Tue, Aug 16, 2011 at 10:51:30AM -0700, Russ Allbery wrote:
> Adam Borowski  writes:
> 
> > On the other hand, zcat and zless are nothing but thin shell wrappers
> > over "gzip -cd", without loads of extra functionality lesspipe has.  I'd
> > turn them into a C program linked against zlib (priority: required),
> > liblzma2 (priority: required) and libbz2 (priority: important, might be
> > dlopened instead).
> 
> One angle of this discussion is confusing me: zcat and zless are upstream
> binaries that are part of a package that we build from upstream source.
> That upstream is gzip; presumably the gzip maintainers aren't particularly
> interested in adding build dependencies on other compression formats.  (I
> wouldn't be if I were them; it seems a bit outside their bailiwick.)
> 
> Are people who are proposing that we change these programs proposing we
> build a new package with binaries of those same names and use
> alternatives?  That Debian fork gzip?  Something else?
 
I propose that we provide a separate package containing zcat, zless,
etc., remove them from gzip and make gzip depend on the new package.
As for who maintains the new package, no I am not volunteering.

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
  - Albert Camus


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110816175953.gq29...@decadent.org.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Jakub Wilk]

* Adam Borowski , 2011-08-16, 12:18:
zcat and zless are nothing but thin shell wrappers over "gzip -cd", 
without loads of extra functionality lesspipe has. I'd turn them into a 
C program linked against zlib (priority: required), liblzma2 (priority: 
required) and libbz2 (priority: important, might be dlopened instead).


See http://packages.debian.org/sid/zutils - though the way it's 
currently packaged is not particularly impressive.


--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110816180105.ga8...@jwilk.net

Re: /usr/share/doc/ files and gzip/xz/no compression

[Eduard Bloch]

#include 
* Adam Borowski [Tue, Aug 16 2011, 12:18:40PM]:

> "zcat" already supports multiple formats: gzip and compress, it's just that
> support for uncompressing the latter is included in gzip's binary.  Since
> it's not called "gzcat" but generic "zcat", I think that's a better idea
> than proliferation of bzcat, xzcat, my-newest-formatcat.

ucat from the unp package does just that - print with any streamable
format unp knows about.

Regards,
Eduard.

-- 
* h01ger winkt Tolimar und seiner zukünftigen Ex-Freundin :)


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110816192608.ga16...@rotes76.wohnheim.uni-kl.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Roger Lynn]

On 16/08/11 00:10, Carsten Hey wrote:
> bzip2 has a better compression on average for some filetypes, xz[1] has
> a better compression on average for others:
> 
>gzip  bzip2   xz bzip2+xz[3]
>   text files[2]   94312922  73496587  77783076  73496587
>   other files 16577181  14609893  14275484  14275484
>   sum110890103  88106480  92058560  87772071
> 
> Among the "other files" are also a lot of text files, if we would
> compress Debian packages instead, xz would win presumably.
> 
> Anyway, I don't think this difference of 4 MiB on a desktop system is
> significant.
> 
> 
> I would prefer to avoid bloating the set of pseudo essential packages
> without a good reason and I think users should be able to decompress all
> files in /u/s/d.  There are plans to let dpkg depend on liblzma2 instead
> of xz and it already depends on libbz2-1.0.  If dpkg's dependency on
> libbz2 is planned to be removed in future, I would prefer to let libbz2
> vanish from the pseudo essential set and use xz also for /u/s/d,
> otherwise I would prefer using bzip2 over xz for /u/s/d.

One advantage of gzip /usr/share/doc is that when served by an
appropriately configured web server .gz files will be transparently
decompressed and displayed by most web browsers. I believe Policy
requires Debian web servers to make /u/s/d available at
http://localhost/doc/. While this obviously isn't an overriding
consideration it is a nice easy way to browse the documentation. Can
same be done for any other compression formats?

Roger


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/7e7rh8-j14@silverstone.rilynn.me.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Bob Proulx]

Ben Hutchings wrote:
> Russ Allbery wrote:
> > Are people who are proposing that we change these programs proposing we
> > build a new package with binaries of those same names and use
> > alternatives?  That Debian fork gzip?  Something else?
>  
> I propose that we provide a separate package containing zcat, zless,
> etc., remove them from gzip and make gzip depend on the new package.
> As for who maintains the new package, no I am not volunteering.

This has been proposed and discussed upstream:

  https://lists.gnu.org/archive/html/bug-gzip/2009-11/threads.html

The web mail archive doesn't thread across months.  Here are
continuing months of discussion:

  https://lists.gnu.org/archive/html/bug-gzip/2010-12/threads.html

  https://lists.gnu.org/archive/html/bug-gzip/2011-01/threads.html

Bob


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Wouter Verhelst]

On Tue, Aug 16, 2011 at 10:43:04AM +1000, Russell Coker wrote:
> On Tue, 16 Aug 2011, Iustin Pop  wrote:
> > There is also information on Wikipedia about various compression
> > benchmarks, but IMHO if we want to switch from gzip then bzip2
> > doesn't make sense for /usr/share/doc.
> 
> I'd like to see zless work transparently with bzip and xz compressed files.
> There's really no need for three different wrapper programs when the zless
> program can just consult the magic db to determine which decompression
> program to use.

I'd like to see _less_ work transparently with bzip and xz compressed
files, too. It's not impossible, I've seen it done on default RedHat
installations.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110817220300.go9...@grep.be

Re: /usr/share/doc/ files and gzip/xz/no compression

[Marc Haber]

On Mon, 15 Aug 2011 18:33:48 +0100, Lars Wirzenius  wrote:
>On the other hand, many computers now have an SSD drive, for speed,
>which is relatively small. Further, most users will likely need files in
>/usr/share/doc rarely, if ever, so not compressing things risks wasting
>a bunch of disk space for no particular benefit.

How about making this configurable within dpkg, so that the people who
want their /usr/share/doc to be uncompressed can tell dpkg to do it
for them without breaking dpkg's databases?

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1quthk-0002ff...@swivel.zugschlus.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Guillem Jover]

On Fri, 2011-08-19 at 19:55:12 +0200, Marc Haber wrote:
> On Mon, 15 Aug 2011 18:33:48 +0100, Lars Wirzenius  wrote:
> >On the other hand, many computers now have an SSD drive, for speed,
> >which is relatively small. Further, most users will likely need files in
> >/usr/share/doc rarely, if ever, so not compressing things risks wasting
> >a bunch of disk space for no particular benefit.
> 
> How about making this configurable within dpkg, so that the people who
> want their /usr/share/doc to be uncompressed can tell dpkg to do it
> for them without breaking dpkg's databases?

If you mean doing that at unpack time, I only see disadvantages with
this: it makes the files take more space on disk, it makes the unpack
slower as it has to uncompress those files, and it makes dpkg more
complicated.

regards,
guillem


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110819205047.ga2...@gaara.hadrons.org

Re: /usr/share/doc/ files and gzip/xz/no compression

[Darren Salt]

I demand that James Vega may or may not have written...

> On Tue, Aug 16, 2011 at 01:06:34PM +1000, Russell Coker wrote:
>> On Tue, 16 Aug 2011, Chris Knadle  wrote:
[snip; *zless]
>>> Right now these functions are spread across 4 different binaries, each of
>>> which is in a different package.
>> Probably the best thing to do would be to just incorporate the
>> functionality in the less package.

> You mean like lesspipe(1)?  Seems like it might need to be updated to
> handle *.xz, but other than that looks like it fits the bill.

The following in ~/.lessfilter adds that support (as a workaround until
lesspipe has it):

#! /bin/sh
case "$1" in
  *.xz)
xzcat "$1"
;;
  *)
exit 1
;;
esac
exit 0

-- 
| Darren Salt  | linux or ds at  | nr. Ashington, |  _  ASCII ribbon
| using Debian | youmustbejoking | Northumberland | ( ) campaign against
| GNU/Linux| ,demon,co,uk||  X  HTML e-mail
/ \ www.asciiribbon.org
I'd like to, but I'm being deported.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/520b4b9cd8%li...@youmustbejoking.demon.co.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Darren Salt]

I demand that Adam Borowski may or may not have written...

[snip]
> There's a patch in #572228 (and at least one fool just wrote a duplicate
> before checking for existing ones), but the patch is rotting in the BTS.

I'd forgotten that I'd written that patch. It's still sufficiently cleanly
applicable, at least, and still seems to do the job.

It's worth mentioning that man-db has had xz support since March last year
(upstream). This is available in testing.

-- 
| Darren Salt  | linux or ds at  | nr. Ashington, |  _  ASCII ribbon
| using Debian | youmustbejoking | Northumberland | ( ) campaign against
| GNU/Linux| ,demon,co,uk||  X  HTML e-mail
/ \ www.asciiribbon.org
Never replicate a successful experiment.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/520b4c0fb0%li...@youmustbejoking.demon.co.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Colin Watson]

On Sat, Aug 20, 2011 at 08:30:24PM +0100, Darren Salt wrote:
> It's worth mentioning that man-db has had xz support since March last year
> (upstream). This is available in testing.

Although I'd also like to mention that I expect that it would take
rather longer for mandb to process /usr/share/man if all of the manual
pages there were xz-compressed rather than gzipped, as it would have to
exec an external command for every page rather than using a library.  I
support xz-compressed pages because you sent me a reasonable patch and
it might occasionally help somebody, but I don't recommend it for global
use in Debian.

-- 
Colin Watson   [cjwat...@debian.org]


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110821150357.ga16...@riva.dynamic.greenend.org.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Iustin Pop]

On Sun, Aug 21, 2011 at 04:03:57PM +0100, Colin Watson wrote:
> On Sat, Aug 20, 2011 at 08:30:24PM +0100, Darren Salt wrote:
> > It's worth mentioning that man-db has had xz support since March last year
> > (upstream). This is available in testing.
> 
> Although I'd also like to mention that I expect that it would take
> rather longer for mandb to process /usr/share/man if all of the manual
> pages there were xz-compressed rather than gzipped, as it would have to
> exec an external command for every page rather than using a library.  I
> support xz-compressed pages because you sent me a reasonable patch and
> it might occasionally help somebody, but I don't recommend it for global
> use in Debian.

Since liblzmaX exists, would it be a simple matter of using it in order
to make mandb handle such compressed man pages without having to fork?

regards,
iustin


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Marc Haber]

On Fri, 19 Aug 2011 22:50:47 +0200, Guillem Jover 
wrote:
>If you mean doing that at unpack time, 

Yes, that's what I mean.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qvd54-0001fy...@swivel.zugschlus.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Adam Borowski]

On Sun, Aug 21, 2011 at 04:03:57PM +0100, Colin Watson wrote:
> On Sat, Aug 20, 2011 at 08:30:24PM +0100, Darren Salt wrote:
> > It's worth mentioning that man-db has had xz support since March last year
> > (upstream). This is available in testing.
> 
> Although I'd also like to mention that I expect that it would take
> rather longer for mandb to process /usr/share/man if all of the manual
> pages there were xz-compressed rather than gzipped, as it would have to
> exec an external command for every page rather than using a library.

This is not a real argument as liblzma2 works just as well as zlib.

> I support xz-compressed pages because you sent me a reasonable patch and
> it might occasionally help somebody, but I don't recommend it for global
> use in Debian.

However, I do agree that it's generally not a good idea, but for a different
reason: xz has a slow start, and below a few thousand bytes of compressed
size gzip usually packs better.

Man pages tend to be smaller than that: out of 94752 man pages in main,
71966(!) are smaller with gzip, and 94088 have gzip < 110% of xz.

This goes in stark contrast to whole packages, where even empty ones usually
have enough junk (copyright, changelog.Debian.gz) that they tend to compress
better with xz -- and sizes of dummies are lost in underflow compared to
real ones.

Thus, just as I strongly recommend using xz everywhere for .deb files, let's
use gzip everywhere for man pages with no more than a few exceptions.

_All_ files where changing compression gives a gain of >10KB are:

 gz | xz | gz-xz | filename
  44492 |  32888 | 11604 | man3/QGauss.3.gz
  46174 |  32716 | 13458 | man3/Color::Library::Dictionary::NBS_ISCC::M.3pm.gz
  66121 |  51340 | 14781 | man3/InfFE.3.gz
  67316 |  52272 | 15044 | man3/ContinuationSystem.3.gz
  76441 |  58656 | 17785 | man1/gm.1.gz
  99346 |  73964 | 25382 | man5/postconf.5.gz
 136446 | 106680 | 29766 | ru/man1/mplayer.1.gz
 179569 | 134548 | 45021 | man3/Image::ExifTool::TagNames.3pm.gz
 153320 | 100544 | 52776 | man3/FE.3.gz


-- 
1KB // Yo momma uses IPv4!


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Adam Borowski]

On Sun, Aug 21, 2011 at 09:39:55PM +0200, Adam Borowski wrote:
> Man pages tend to be smaller than that: out of 94752 man pages in main,
> 71966(!) are smaller with gzip, and 94088 have gzip < 110% of xz.
> 
> Thus, just as I strongly recommend using xz everywhere for .deb files, let's
> use gzip everywhere for man pages with no more than a few exceptions.
> 
> _All_ files where changing compression gives a gain of >10KB are:
> 
>  gz | xz | gz-xz | filename
>   44492 |  32888 | 11604 | man3/QGauss.3.gz
>   46174 |  32716 | 13458 | man3/Color::Library::Dictionary::NBS_ISCC::M.3pm.gz
>   66121 |  51340 | 14781 | man3/InfFE.3.gz
>   67316 |  52272 | 15044 | man3/ContinuationSystem.3.gz
>   76441 |  58656 | 17785 | man1/gm.1.gz
>   99346 |  73964 | 25382 | man5/postconf.5.gz
>  136446 | 106680 | 29766 | ru/man1/mplayer.1.gz
>  179569 | 134548 | 45021 | man3/Image::ExifTool::TagNames.3pm.gz
>  153320 | 100544 | 52776 | man3/FE.3.gz

Oops, my bad, these results were mistakenly filtered to:
  xz*510KB, 9
with diff >20KB.  And obviously the additional ones give less than 25%
benefit...

-- 
1KB // Yo momma uses IPv4!


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Colin Watson]

On Sun, Aug 21, 2011 at 05:39:08PM +0200, Iustin Pop wrote:
> Since liblzmaX exists, would it be a simple matter of using it in order
> to make mandb handle such compressed man pages without having to fork?

I don't want to add more linkage, especially in light of Adam's point
about xz not being worth it for most manual pages anyway.

-- 
Colin Watson   [cjwat...@debian.org]


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110822082725.ga5...@riva.dynamic.greenend.org.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Iustin Pop]

On Mon, Aug 22, 2011 at 09:27:25AM +0100, Colin Watson wrote:
> On Sun, Aug 21, 2011 at 05:39:08PM +0200, Iustin Pop wrote:
> > Since liblzmaX exists, would it be a simple matter of using it in order
> > to make mandb handle such compressed man pages without having to fork?
> 
> I don't want to add more linkage, especially in light of Adam's point
> about xz not being worth it for most manual pages anyway.

Indeed, with that context, agreed.

iustin


signature.asc
Description: Digital signature

Re: /usr/share/doc/ files and gzip/xz/no compression

[Eduard Bloch]

#include 
* Colin Watson [Mon, Aug 22 2011, 09:27:25AM]:
> On Sun, Aug 21, 2011 at 05:39:08PM +0200, Iustin Pop wrote:
> > Since liblzmaX exists, would it be a simple matter of using it in order
> > to make mandb handle such compressed man pages without having to fork?
> 
> I don't want to add more linkage, especially in light of Adam's point
> about xz not being worth it for most manual pages anyway.

Would you consider additional linkage to libdl? That is less than 15kB
size and brings possibility to load liblzma as needed.

In fact, I have an almost complete patch for man-db which dlopens
on-the-fly. Let me know if I should proceed with that.

Regards,
Eduard.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110822134904.ga19...@rotes76.wohnheim.uni-kl.de

Re: /usr/share/doc/ files and gzip/xz/no compression

[Colin Watson]

On Mon, Aug 22, 2011 at 03:49:04PM +0200, Eduard Bloch wrote:
> #include 
> * Colin Watson [Mon, Aug 22 2011, 09:27:25AM]:
> > I don't want to add more linkage, especially in light of Adam's point
> > about xz not being worth it for most manual pages anyway.
> 
> Would you consider additional linkage to libdl? That is less than 15kB
> size and brings possibility to load liblzma as needed.
> 
> In fact, I have an almost complete patch for man-db which dlopens
> on-the-fly. Let me know if I should proceed with that.

Given that it apparently doesn't gain much in the way of size
improvements, I don't really see the point.  But I'd look at a patch if
it doesn't make the code significantly less portable or significantly
harder to maintain.

-- 
Colin Watson   [cjwat...@debian.org]


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110822203514.gb5...@riva.dynamic.greenend.org.uk

Re: /usr/share/doc/ files and gzip/xz/no compression

[Eduard Bloch]

#include 
* Colin Watson [Mon, Aug 22 2011, 09:35:14PM]:
> On Mon, Aug 22, 2011 at 03:49:04PM +0200, Eduard Bloch wrote:
> > #include 
> > * Colin Watson [Mon, Aug 22 2011, 09:27:25AM]:
> > > I don't want to add more linkage, especially in light of Adam's point
> > > about xz not being worth it for most manual pages anyway.
> > 
> > Would you consider additional linkage to libdl? That is less than 15kB
> > size and brings possibility to load liblzma as needed.
> > 
> > In fact, I have an almost complete patch for man-db which dlopens
> > on-the-fly. Let me know if I should proceed with that.
> 
> Given that it apparently doesn't gain much in the way of size
> improvements, I don't really see the point.  But I'd look at a patch if
> it doesn't make the code significantly less portable or significantly
> harder to maintain.

Agreed. I stopped coding after realizing that you don't gain anything if
the typical file size is within one or two filesystem blocks.

A copy is stored on 
http://alioth.debian.org/~blade/mandb_dlopens_liblzma.diff , feel free
to add it or ignore it.

Regards,
Eduard.
-- 
Naja, Garbage Collector eben. Holt den Müll sogar vom Himmel.
   (Heise Trollforum über Java in der Flugzeugsteuerung)


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20110822210339.ga13...@rotes76.wohnheim.uni-kl.de