Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-07 Thread Steven Chamberlain
On 07/06/12 11:20, Vincent Lefevre wrote: > Actually these are only MD5 errors (from my own checks), no > "bunzip2: Data integrity error when decompressing" errors. Couldn't you pipe the uncompressed data directly into your integrity checking program? Without using bzip2/bunzip2 or any other comp

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-07 Thread Vincent Lefevre
On 2012-06-07 10:49:32 +0200, Vincent Lefevre wrote: > > You might try the same test with data from your server's local disks > > too. Or try saving the bz2 output to a local disk to see if it makes > > the integrity errors go away. (You would still see the MD5 mismatches > > though). > > I get

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-07 Thread Vincent Lefevre
On 2012-06-07 03:01:17 +0100, Steven Chamberlain wrote: > On 07/06/12 02:35, Vincent Lefevre wrote: > > bunzip2: Data integrity error when decompressing. > > Input file = mpfr-0-8207.bz2, output file = (stdout) > > > > It is possible that the compressed file(s) have become corrupted. > >

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Steven Chamberlain
On 07/06/12 02:35, Vincent Lefevre wrote: > bunzip2: Data integrity error when decompressing. > Input file = mpfr-0-8207.bz2, output file = (stdout) > > It is possible that the compressed file(s) have become corrupted. Were you writing the .bz2 files to that fileserver? You could try get

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Vincent Lefevre
On 2012-06-07 03:20:37 +0200, Vincent Lefevre wrote: > Now, the way bunzip2 does integrity checking when decompressing is > poorly documented: does bunzip2 guarantee that every byte from the > stream is correct (e.g. integrity checking is done chunk by chunk, > and a chunk is output only once CRC c

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Vincent Lefevre
I've found an old discussion saying that there are major problems with the file server. Indeed on 2012-02-28, I got an error: bunzip2: Data integrity error when decompressing. Input file = mpfr-0-8054.bz2, output file = (stdout) (when I tested bunzip2 manually, I think, but I'm not sure). However

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Vincent Lefevre
Hi, On 2012-06-06 20:50:17 +0100, Steven Chamberlain wrote: > This could be a symptom of bad RAM in the machine. (Although from the > system info, it sounds like you might have ECC RAM). When there are RAM problems, there are error messages in the /var/log/kern.log file, like: ECC/ChipKill ECC e

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Steven Chamberlain
Hi, This could be a symptom of bad RAM in the machine. (Although from the system info, it sounds like you might have ECC RAM). Have you considered checking it, e.g. with memtest86+? If you can't take the machine offline, memtester might give you some idea (as root, test an amount approx. equal

Bug#676377: bzip2: bunzip2 sometimes generate corrupt data via a pipe

2012-06-06 Thread Vincent Lefevre
Package: bzip2 Version: 1.0.5-6+squeeze1 Severity: grave Justification: causes non-serious data loss I have a Perl script that calls bunzip2 on a .bz2 file and does some md5 checking on the output. But while the .bz2 file hasn't changed, the md5 is sometimes wrong (this is unreproducible in genera