On Thursday, August 25, 2011 07:20:20 PM Charles Curley did opine: > On Thu, 25 Aug 2011 09:45:38 -0400 > > gene heskett <ghesk...@wdtv.com> wrote: > > That could be handy if it is easily configured to be used, which I > > have not investigated. > > http://wiki.zmanda.com/index.php/How_To:Use_pigz_to_speed_up_compression > > > Making amanda use its parallelism might be a > > challenging project however. > > Other emails on this list indicate that using pigz does speed up amanda > runs. > > Just for the halibut, I did a bit of testing on an otherwise idle > desktop with two AMD cores. I created a tarball using my /var > directory. Then: > > root@dzur:/# time pigz var.test.tar > > real 6m1.411s > user 9m5.290s > sys 0m44.550s > root@dzur:/# ll var.test.tar.gz > -rw-r--r-- 1 root root 6786436232 2011-08-25 10:07 var.test.tar.gz > root@dzur:/# time gzip -d var.test.tar.gz > > real 4m16.611s > user 1m43.080s > sys 0m38.410s > root@dzur:/# ll var.test.tar > -rw-r--r-- 1 root root 9458923520 2011-08-25 10:07 var.test.tar > root@dzur:/# tar -tf var.test.tar > # output omitted; no error messages > root@dzur:/# time gzip var.test.tar > > real 9m45.992s > user 8m23.070s > sys 0m32.970s > root@dzur:/# ll var.test.tar.gz > -rw-r--r-- 1 root root 6785153296 2011-08-25 10:07 var.test.tar.gz > root@dzur:/# gunzip var.test.tar.gz > root@dzur:/# ll var.test.tar > -rw-r--r-- 1 root root 9458923520 2011-08-25 10:07 var.test.tar > root@dzur:/# pigz --version > pigz 2.1.6 > root@dzur:/# > > > Note that gzip took noticeably longer but produced a marginally smaller > output file (possibly due to different default compression values; I > did not investigate). Testing the output file as above produced no error > message. The uncompressed tarball retained its size. > > I used gzip -d to test rather than pigz -d in order to verify > compatibility. > > While we're at it, I am glad I never did implement bzip2 compression > for Amanada: > > root@dzur:/# time bzip2 var.test.tar > > real 157m54.710s > user 156m41.360s > sys 0m35.550s > root@dzur:/# ll var.test.tar.bz2 > -rw-r--r-- 1 root root 6780647698 2011-08-25 10:07 var.test.tar.bz2 > root@dzur:/#
This latter is not a bit surprising. And its been known to make mistakes in the decompress without reporting an error. I once unpacked a kernel tree but could not get it to build, not once but 3 times and each time the tree was missing about a megabytes worth of a driver subdir. By then I was getting schmardter from the error messages & knew which subdir was missing. Prepared to download it again, I blew that tree away and unpacked the .bz2 again. On the 4th try, that subdir finally showed up, and it then built as usual. From that time on, I have never pulled the .bz2 unless the .gz was not available. Cheers, gene -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) My web page: <http://204.111.67.138:85/gene> When you die, you lose a very important part of your life. -- Brooke Shields
