ID: 33070 Updated by: [EMAIL PROTECTED] Reported By: lindsay at bitleap dot com -Status: Open +Status: Verified Bug Type: Performance problem Operating System: Linux 2.6.10 kernel -PHP Version: 5.0.3 +PHP Version: 5.*, 4.* New Comment:
I stand corrected, it's dead slow. :) Previous Comments: ------------------------------------------------------------------------ [2005-05-27 17:20:01] lindsay at bitleap dot com sniper at php dot net: I'll suggest a faster cpu is not the solution. On the same machine, the bunzip2 binary decodes the same 4MB bzip'd file in under 2 seconds. time bunzip2 4M.bz2 real 0m1.198s user 0m1.162s sys 0m0.034s And bzdecompress did it in: 476 seconds or 7 minutes and 56 seconds. Almost 8 minutes versus 2 seconds on the same machine with the same data sample would suggest a problem with bzdecompress. ------------------------------------------------------------------------ [2005-05-27 09:11:43] [EMAIL PROTECTED] Get faster machine or use some other compression method than bz2.. (try your test using the bunzip2 binary and you'll see it's equally slow..) ------------------------------------------------------------------------ [2005-05-23 21:19:02] lindsay at bitleap dot com I noticed the new code was crashing. You can reliably reproduce the results by creating a file which compresses well. I created my data sample with the following commands: dd if=/dev/zero bs=1024 count=65536 of=large bzip2 large When I run the script on this, I get the following crash: file length 256K ran in: 0 seconds file length 512K ran in: 1 seconds file length 1M ran in: 1 seconds file length 2M ran in: 2 seconds file length 3M ran in: 2 seconds file length 4M ran in: 3 seconds file length large ran in: *** glibc detected *** double free or corruption (!prev): 0x0853f418 *** ------------------------------------------------------------------------ [2005-05-23 20:09:43] lindsay at bitleap dot com I applied the patch suggested by jeff at vertexdev dot com. I also added another test for 64M data chunks. Just to be sure, I verified php's bzdecompress w/ the patch produces data creating the same md5sum as bzunzip. file length 256K ran in: 0 seconds file length 512K ran in: 0 seconds file length 1M ran in: 2 seconds file length 2M ran in: 1 seconds file length 3M ran in: 3 seconds file length 4M ran in: 3 seconds file length 64M ran in: 38 seconds ------------------------------------------------------------------------ [2005-05-23 16:50:02] lindsay at bitleap dot com It looks like BZ2_bzRead() requires a file to read from. Would: BZ2_bzDecompressInit loop BZ2_bzDecompress //decompress in chunks BZ2_bzDecompressEnd work? The source code for BZ2_bzBuffToBuffDecompress seems like it could almost be used if the BZ2_bzDecompress was looped through 'source' in chunks. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/33070 -- Edit this bug report at http://bugs.php.net/?id=33070&edit=1