Re: 7-Zip / Bzip2
Samuel J. Greear [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Matthew Dillon [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] :Hi, : :Posted this to kernel@ by accident, please reply here instead :) : :I just wanted to know if there's any interest for the devs to add :something like p7zip to the base install; even if it's a simple fork :that only supports 7z. While 7zip is about as obnoxiously slow as :bzip2, it usually gets much better compression. : :That's not why I'm suggesting it though - what really gets me is that :bzip2 has no list option. Does that 10 gb bzip2 backup archive :contain 100gb of data, or 200gb? Other than dumping the entire :archive to /dev/null through wc, there's really no way to do it. Gzip :will list files, but its compression ratio is awful. : :I imagine that other OSes are going to be watching Dragonfly very :carefully in the next while as new the features (especially HAMMER) :mature. Maybe adding 7z will get yet another bandwagon going and :there will be support across the board :) : :Best Regards, :Ben Cadieux Well, I think not in base, at least not unless a lot of people are using it. p7zip is readily available via the pkgsrc tree and that's the most reasonable method of accessibility for now. -Matt Matthew Dillon [EMAIL PROTECTED] I do not know if DragonFly is actively tracking libarchive, but it seems to be the ticket for implementing new archiving/compression methods through a common mechanism. If one wanted to see 7z functionality in base implementing a libarchive provider is probably the way to go. I did a quick test at one point on the dfly distribution ISO and as I recall 7z was ~60% the size of bzip2 using standard settings. food+thought, etc. Sam I just noticed I still have these sitting here so I figured I would post sizes. 1.12.1 ISO 294M, bz2 108M, 7z 74M, zip 118M, gz 120M. Sam
Re: 7-Zip / Bzip2
Samuel J. Greear [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Matthew Dillon [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] :Hi, : :Posted this to kernel@ by accident, please reply here instead :) : :I just wanted to know if there's any interest for the devs to add :something like p7zip to the base install; even if it's a simple fork :that only supports 7z. While 7zip is about as obnoxiously slow as :bzip2, it usually gets much better compression. : :That's not why I'm suggesting it though - what really gets me is that :bzip2 has no list option. Does that 10 gb bzip2 backup archive :contain 100gb of data, or 200gb? Other than dumping the entire :archive to /dev/null through wc, there's really no way to do it. Gzip :will list files, but its compression ratio is awful. : :I imagine that other OSes are going to be watching Dragonfly very :carefully in the next while as new the features (especially HAMMER) :mature. Maybe adding 7z will get yet another bandwagon going and :there will be support across the board :) : :Best Regards, :Ben Cadieux Well, I think not in base, at least not unless a lot of people are using it. p7zip is readily available via the pkgsrc tree and that's the most reasonable method of accessibility for now. -Matt Matthew Dillon [EMAIL PROTECTED] I do not know if DragonFly is actively tracking libarchive, but it seems to be the ticket for implementing new archiving/compression methods through a common mechanism. If one wanted to see 7z functionality in base implementing a libarchive provider is probably the way to go. I did a quick test at one point on the dfly distribution ISO and as I recall 7z was ~60% the size of bzip2 using standard settings. food+thought, etc. Sam I got yelled at on IRC for not providing times, so here: Key: Selected Archiver - Compression Time - Threads - FileSize Source file: dfly-1.12.2_REL.iso - 295,512KB Bzip2 - 0:27 - 4 - 108,742KB Bzip2 - 1:24 - 1 - 108,742KB Decompress: 0:10 GZip - 0:55 - 1 - 118,024KB Decompress: 0:10 Zip - 0:55 - 4 - 118,024KB Zip - 0:55 - 1 - 118,024KB Decompress: 0:11 7Zip - 1:50 - 2 - 73,952KB 7Zip - 3:09 - 1 - 73,952KB Decompress: 0:12 All done via the most unscientific methods available (reporting user time as displayed by the 7Zip user interface), so treat it as such. Tested on an Intel Q6600 (4x2.4GHz) w/ single SATA disk. Default compression settings were used across the board. Everything was tested using the windows 7-Zip program, which integrates all of these algorithms. Sam
Re: 7-Zip / Bzip2
Hello, On Tue, 2008-05-06 at 09:51 -0700, Samuel J. Greear wrote: Samuel J. Greear [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Matthew Dillon [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] :Hi, : :Posted this to kernel@ by accident, please reply here instead :) : :I just wanted to know if there's any interest for the devs to add :something like p7zip to the base install; even if it's a simple fork :that only supports 7z. While 7zip is about as obnoxiously slow as :bzip2, it usually gets much better compression. : :That's not why I'm suggesting it though - what really gets me is that :bzip2 has no list option. Does that 10 gb bzip2 backup archive :contain 100gb of data, or 200gb? Other than dumping the entire :archive to /dev/null through wc, there's really no way to do it. Gzip :will list files, but its compression ratio is awful. : :I imagine that other OSes are going to be watching Dragonfly very :carefully in the next while as new the features (especially HAMMER) :mature. Maybe adding 7z will get yet another bandwagon going and :there will be support across the board :) : :Best Regards, :Ben Cadieux Well, I think not in base, at least not unless a lot of people are using it. p7zip is readily available via the pkgsrc tree and that's the most reasonable method of accessibility for now. -Matt Matthew Dillon [EMAIL PROTECTED] I do not know if DragonFly is actively tracking libarchive, but it seems to be the ticket for implementing new archiving/compression methods through a common mechanism. If one wanted to see 7z functionality in base implementing a libarchive provider is probably the way to go. I did a quick test at one point on the dfly distribution ISO and as I recall 7z was ~60% the size of bzip2 using standard settings. food+thought, etc. Sam I got yelled at on IRC for not providing times, so here: Key: Selected Archiver - Compression Time - Threads - FileSize Source file: dfly-1.12.2_REL.iso - 295,512KB Bzip2 - 0:27 - 4 - 108,742KB Bzip2 - 1:24 - 1 - 108,742KB Decompress: 0:10 GZip - 0:55 - 1 - 118,024KB Decompress: 0:10 Zip - 0:55 - 4 - 118,024KB Zip - 0:55 - 1 - 118,024KB Decompress: 0:11 7Zip - 1:50 - 2 - 73,952KB 7Zip - 3:09 - 1 - 73,952KB Decompress: 0:12 All done via the most unscientific methods available (reporting user time as displayed by the 7Zip user interface), so treat it as such. Tested on an Intel Q6600 (4x2.4GHz) w/ single SATA disk. Default compression settings were used across the board. Everything was tested using the windows 7-Zip program, which integrates all of these algorithms. Sam The compression time is not an issue for install disks, and decompression extra time of 7z is insignificant, but the best usage for this would be when packaging sources, not the installer CD, whose size will grow slower than the size of CD/DVD/BlueRay... Anyway our CD image could easily fit a mini-CD, so we don't need this, for now. Regards, Cristi. -- Cristi Magherusan, Universitatea Tehnica din Cluj - Napoca Centrul de Comunicatii Pusztai Kalman Tel. 0264/401247 http://cc.utcluj.ro signature.asc Description: This is a digitally signed message part
Re: 7-Zip / Bzip2
:I just noticed I still have these sitting here so I figured I would post :sizes. 1.12.1 ISO 294M, bz2 108M, 7z 74M, zip 118M, gz 120M. : :Sam Those are impressive numbers. I will keep an eye on it. If 7z is consistently better then bz2 I expect it will become widely adopted. I just think it is a bit early to put it into base when it is so readily accessible via pkgsrc. -Matt Matthew Dillon [EMAIL PROTECTED]
Re: 7-Zip / Bzip2
On Tue, 06 May 2008 20:29:53 +0300 Cristi Magherusan [EMAIL PROTECTED] wrote: The compression time is not an issue for install disks, and decompression extra time of 7z is insignificant, but the best usage for this would be when packaging sources, not the installer CD, whose size will grow slower than the size of CD/DVD/BlueRay... Anyway our CD image could easily fit a mini-CD, so we don't need this, for now. There is still the little matter of download time and bandwidth utilisation. Certainly if I saw a 73MB file and a 108MB file with the same contents I know which one I would choose to download (once I'd assured myself of being able to unpack it). -- C:WIN | Directable Mirror Arrays The computer obeys and wins.| A better way to focus the sun You lose and Bill collects. |licences available see |http://www.sohara.org/
Re: 7-Zip / Bzip2
Well, the issue is not so much one of download bandwidth as it is one of accessibility. Everyone has gzip, not everyone has 7z (yet). I would rather not have multiple compression formats on the download site, nor require that people have 7z in order to be able to unpack our release CDs! -Matt Matthew Dillon [EMAIL PROTECTED]
Re: 7-Zip / Bzip2
Well, the issue is not so much one of download bandwidth as it is one of accessibility. Everyone has gzip, not everyone has 7z (yet). I would rather not have multiple compression formats on the download site, nor require that people have 7z in order to be able to unpack our release CDs! This is the 'bandwagon' I was referring to --- it's a chicken and egg problem. I can safely say I would rarely be using gzip or bzip2 if 7z were in the base installs of all Linux/BSD releases. No one uses compress any more because gzip/bzip2 are in the base. If they weren't, you'd still see .Z files everywhere. Best Regards, Ben Cadieux
Re: 7-Zip / Bzip2
For everyone concerned with timing, I tested compressing a 4gb image of a quad boot USB key with a variety of junk on it at compression level 4. It still beats bzip2 by 90 mb and was almost 3 minutes quicker: # time -h bzip2 -k key.img 11m59.71s real 10m39.48s user 5.03s sys # time -h 7z a -mx=4 key.img.7z key.img 8m39.02s real 7m38.57s user 4.95s sys # ls -la | grep key -rw-r- 1 sol wheel 4102029312 Mar 8 11:27 key.img -rw--- 1 root wheel 904489376 May 6 11:38 key.img.7z -rw-r- 1 sol wheel 997632056 Mar 8 11:27 key.img.bz2 Best Regards, Ben Cadieux
Re: 7-Zip / Bzip2
Samuel J. Greear wrote: I just noticed I still have these sitting here so I figured I would post sizes. 1.12.1 ISO 294M, bz2 108M, 7z 74M, zip 118M, gz 120M. lrz 69M It doesn't mean that I care though. In fact, I use gzip most of time - bandwith is quite cheap nowadays. -- Hasso Tepper
