Hi!
I just installed pbzip2 and runned it on the same file and ended up with
the same packed file. If you run pbzip2 will the created bz2-file differ
from bzip2 on the same file? If so, you have something wrong with pbzip2.
Do you really need to compress that much files all the time?
If we look at the options you use. For me that indicates that you pack the
same files over and over again (--force). If so why not just pack the
files that have been changed? Then you would end up with packing a lot
less files.
Something like this would probably work in that case (if runned daily):
# find . -mtime -1d30m -exec bzip2 --compress --keep --force --best {} \;
/Bjorn
On Thu, 21 Jan 2010, Philipp Reddigau wrote:
> Hi,
> Thank you for the reply.
> But this will compress up to 10 files at once.
> It will not compress one file on more than one core.
> It would be better and much faster if one file compressed by 8 or 16 cores.
>
> An Example: (we are do it like your skript)
> 5000 files:
> normal method(one file at time one core):
> 7h 34m
>
> like your skript(15 files on 15 cores): !extreme load, hdd is unusable!
> 1h 37m
>
> pbzip2(8 cores): !all is right, hdd usuable!
> 1h 43m
>
> Take care at the number of cores.
>
> Mit freundlichen Grüßen,
> Philipp Reddigau (GKServer)
>
> GKServer
> Daniel Zschau
> Niederfährer Straße 38
> 01662 Meißen
>
>
> -----Ursprüngliche Nachricht-----
> Von: hlds_linux-boun...@list.valvesoftware.com
> [mailto:hlds_linux-boun...@list.valvesoftware.com] Im Auftrag von kama
> Gesendet: Donnerstag, 21. Januar 2010 14:17
> An: Half-Life dedicated Linux server mailing list
> Betreff: Re: [hlds_linux] BZip2 compression methods
>
>
>
> On Thu, 21 Jan 2010, Philipp Reddigau wrote:
>
> > Hi,
> > we are using a automatic *.bz2 for our content servers (fastdownload) on
> > nix/lenny.
> > It works fine with bzip2 --compress --keep --force --best "$file"
> > But we have one Problem..we have many files to compress and one bzip2 is
> not
> > enough.
> > So we have tried pbzip2 after compress join on server "Could not CRC
> > blabla".
> > So the game cannot decompress the file..
> > I there a way to get it clean with any other compress program?
> > (multithreaded)
> > Anyone knows witch exact format is required, that the games can decompress
> > it?
> > I cannot find any information about the decompression.
>
> maybe do a script that does it for you instead...
>
> This script will keep you at max 10 instances of bzip:
>
> -- snip--
> #!/usr/local/bin/bash
>
> max=10
>
> for file in $*
> do
> bzip2 --compress --keep --force --best "$file" &
> while [ `ps aux | grep bzip | awk 'END { print NR }'` -ge $max ]
> do
> sleep 1
> done
> done
> -- snap --
>
> This is just an example. There are probably ways to tune it and make it
> better.
>
> /Bjorn
>
> _______________________________________________
> To unsubscribe, edit your list preferences, or view the list archives,
> please visit:
> http://list.valvesoftware.com/mailman/listinfo/hlds_linux
>
>
> _______________________________________________
> To unsubscribe, edit your list preferences, or view the list archives, please
> visit:
> http://list.valvesoftware.com/mailman/listinfo/hlds_linux
>
_______________________________________________
To unsubscribe, edit your list preferences, or view the list archives, please
visit:
http://list.valvesoftware.com/mailman/listinfo/hlds_linux
