I am struggling to understand why this problem occurs: I scp a number of binary files (e.g. videos) from box A to box B. I have bz2'd them beforehand. At box B I run bzip2 -tv file.bz2, which reports that all is OK.
Then I run bzip2 -dv file.bz2 but it tells me that some of the contents have been corrupted. Fair enough, I run bzip2recover which runs its CRC checks and does not report anything being wrong. So, I know run bzip2 -tv rec00*file.bz2 and it finds a number of files with "data integrity (CRC) error in data". I run bzip2recover on box A and scp afresh copies of the affected rec00* files. Some of the rec00* files seem to be copied without further corruption, but one or two report again corruption (on box B only). Unable to understand why this is so, I decide to use vanilla ftp to another box (in a data center this time) to copy over the same files from box A. Then run ftp get from box B. bzip2 -tv shows no more corruption (although this is not universal - some files still show corruption when tested on box B). My ISP applies traffic shaping (especially on the obscure ssh port that I have chosen for the transmission) so some packets may be dropped, but this does not explain why ftp occasionally works. Can you explain why this seemingly random corruption occurs? Can I trust bzip2 and its CRC tests? -- Regards, Mick
signature.asc
Description: This is a digitally signed message part.
