[opensc-devel] move to dist-xz format / use .tar.xz instead of.tar.gz

2010-02-16 Thread Andreas Jellinghaus
Hi,

distributions like slackware and fedora moved to the xz compression
a while ago, even the kernel developers think about abandoning the
.tar.gz file format in favor of alternatives with better compression
like .tar.bz2 or .tar.xz (or short: .txz).

if we want to try doing something like that too, opensc 0.12 would be
a good time. for reference, I checked current trunk with all options:

sizefile name / format
2416782 opensc-0.12.0-svn.shar.gz
1135736 opensc-0.12.0-svn.tar.bz2
1519708 opensc-0.12.0-svn.tar.gz
1009746 opensc-0.12.0-svn.tar.lzma
1009948 opensc-0.12.0-svn.tar.xz
2508011 opensc-0.12.0-svn.tar.Z
1922452 opensc-0.12.0-svn.zip

uncompression time:
opensc-0.12.0-svn.shar.gz  real0m10.836s user0m3.760s sys 0m7.120s
opensc-0.12.0-svn.tar.bz2  real0m0.328s user0m0.280s sys 0m0.040s
opensc-0.12.0-svn.tar.gz   real0m0.116s user0m0.030s sys 0m0.030s
opensc-0.12.0-svn.tar.lzma real0m0.156s user0m0.120s sys 0m0.090s
opensc-0.12.0-svn.tar.xz   real0m0.173s user0m0.070s sys 0m0.120s
opensc-0.12.0-svn.tar.Zreal0m0.119s user0m0.030s sys 0m0.080s
opensc-0.12.0-svn.zip  real0m0.416s user0m0.150s sys 0m0.070s

so tar.xz would be 50% smaller than tar.gz, but still uncompress quite fast.

compression of xz files is of course much slower than gzip (about factor 20),
and uses a lot of memory, but that only affects people creating new tar.xz 
files.

most linux distributions have xz-utils installed or available in their 
repository, so users would need to do nothing / simply install those.
command line: tar xf file works on all formats - tar, tar.gz, tar.xz etc,
so simply drop the z option when extracting, and tar will figure out which
compression format the file has and how to decompress it.

for reference, the packaging option is j for bzip2 and J for xz I think.
AM_AUTOMAKE_INIT takes options like dist-bzip2 or dist-xz and will then
generate those matching files.

windows users can continue to use the free 7-zip to extract tar.gz files,
but for tar.xz they need to use the 9.00 version (currently in beta test),
the old 4.* version won't work.


in summary:
I'm not sure at all, if it is worth the trouble to switch from tar.gz to
some better compression. we don't have many people downloading files, no
bandwidth issues or anything like that. on the other hand distributions
are happy to save disk space etc. if source packages are using a better
compression mechanism.

I reviewed latest trends, what other people do and so on, and I believe
if we want to swich to a different compression, then tar.xz is the current
recommendation. it is new, and not everyone has those tools for it installed,
but already enough people use it so it won't be a big pain.

what do you think? keep tar.gz? switch to tar.xz? something else?

Regards, Andreas
___
opensc-devel mailing list
opensc-devel@lists.opensc-project.org
http://www.opensc-project.org/mailman/listinfo/opensc-devel


Re: [opensc-devel] move to dist-xz format / use .tar.xz instead of.tar.gz

2010-02-16 Thread Ralf Schlatterbeck
On Tue, Feb 16, 2010 at 09:02:29AM +0100, Andreas Jellinghaus wrote:
 
 distributions like slackware and fedora moved to the xz compression
 a while ago, even the kernel developers think about abandoning the
 .tar.gz file format in favor of alternatives with better compression
 like .tar.bz2 or .tar.xz (or short: .txz).

I've never heard of xz before your post (not that I claim to be up to
date with current developments in compression technology).
And debian stable (aka lenny) doesn't have it:

% apt-cache search xz  
makexvpics - updates .xvpics thumbnails from the command line
xzgv - Picture viewer for X with a thumbnail-based selector
xzip - Interpreter of Infocom-format story-files
xzoom - magnify part of X display, with real-time updates
zblast-x11 - X11 version of zblast, shoot 'em up space game
horae-examples - ATHENA and ARTEMIS examples and tutorials

or is this in a package with an obscure name?
And debian stable tar doesn't have a -J option.
(And you don't think the unsecure .shar format is a canditate?)

Does it really matter if the distribution is 1M or 1.5M?
I can see this makes a difference for large distributions like the
kernel, though.

So I vote for keeping .tar.gz -- or if size really matters move to bz2.

Ralf

-- 
Dr. Ralf Schlatterbeck  Tel:   +43/2243/26465-16
Open Source Consulting  Fax:   +43/2243/26465-23
Reichergasse 131www:   http://www.runtux.com
A-3411 Weidling email: off...@runtux.com
osAlliance member   email: r...@osalliance.com
___
opensc-devel mailing list
opensc-devel@lists.opensc-project.org
http://www.opensc-project.org/mailman/listinfo/opensc-devel


Re: [opensc-devel] move to dist-xz format / use .tar.xz instead of.tar.gz

2010-02-16 Thread Kalev Lember
On 02/16/2010 11:05 AM, Ralf Schlatterbeck wrote:
 On Tue, Feb 16, 2010 at 09:02:29AM +0100, Andreas Jellinghaus wrote:
 distributions like slackware and fedora moved to the xz compression
 a while ago, even the kernel developers think about abandoning the
 .tar.gz file format in favor of alternatives with better
 compression
 like .tar.bz2 or .tar.xz (or short: .txz).
snip
 Does it really matter if the distribution is 1M or 1.5M?
 I can see this makes a difference for large distributions like the
 kernel, though.

 So I vote for keeping .tar.gz -- or if size really matters move to
 bz2.

An option would be to have the same tarball compressed with both gzip
and xz, and distribute both of them side-by-side. That way both parties
should be happy: new distributions who want to use best compression
technology can get tar.xz, and people who want to install new opensc on
older distributions can still keep using tar.gz.

-- 
Kalev
___
opensc-devel mailing list
opensc-devel@lists.opensc-project.org
http://www.opensc-project.org/mailman/listinfo/opensc-devel