On Oct 24, 2010, at 2:13 PM, Sriram Narayanan wrote:
> What does the "optional" support for bzip2, LZMA, gzip, etc mean ?
>
As originally planned, @rpm5.org was set up so that
Have it your own way!
could be done.
In practice (after 3 years of experience), "Have it your own way!"
is largely unmaintainable (it creates a combinatorial failure
for build options to test) and its quite possible to build
@rpm5.org with _EVERYTHING_ diabled. I.e. rpm is utterly
useless.
So "optional" dates back to an earlier time when minimizing build
pre-requsites was deemed more important than having a functional RPM.
Traditionally, RPM has uses zlib for payload compression.
Mandriva (and a few other distros) tried bzip2 and found out
that the reduced package size leads to 5-7x slower installs,
and huge memory footprints (-9 has entirely different meanings
for gzip and bzip2).
Most distros are attempting XZ/LZMA these days, but what is tricky
there is that there's no "official" release of XZ yet, so its
all caveat emptor and differently broken.
> When does it make sense to add support for these compression schemes ?
>
Personally, the issue isn't the compression format that is important,
but rather how compression can be used effectively.
E.g. the zlib (but _NOT_ the XZ) support in rpm can be quite
efficiently rsync'd (its essentially gzip --rsyncable though
its achieved by flushing, not padding, the compression).
Another e.g. Bzip2 per se isn't necessarily 5-7x slower, its
the size/number of blocks used in the BWT
http://en.wikipedia.org/wiki/Burrows–Wheeler_transform
that force 5-7x slower: if you have only one block, then you
are forced to decompress one block serially.
Note that @rpm5.org _ALREADY_ does what pbzip2 does, concatenate
multiple BWT blocks, so that each block can be decompressed
in parallel. I haven't bothered to do the multi-threaded
bzip2 decompression largely because all the recent interest
is on XZ (which is _ALSO_ neither --rsyncable nor parallelizable
afaik, noone has bothered to do the work, too busy changing
XZ! XZ! XZ! XZ!
Short answer:
Add all the compressions. And ask for LZO which has certain
advantages like KISS/FAST decompression in RPM.
> Does rpm5 take a decision to compress a file using a specific library
> (e.g. bzip2 for binaries, gzip for text, etc) ?
>
rpmbuild choses compression through macro configuration.
rpm uses magic and/or metadata tags to find which decompressor to use.
The killer is this:
If the needed compression isn't available, rpm2cpio.sh MUST be used for
*.rpm.
73 de Jeff______________________________________________________________________
RPM Package Manager http://rpm5.org
Developer Communication List rpm-devel@rpm5.org
