On Thu, 2005-05-05 at 11:52 -0600, Clint Silvester wrote: > I tried rdiff last night instead of rdiff-backup on the same 25 gig > files that rdiff-backup gave me an error 107 and rdiff worked fine on > the files. I got a full copy of the newer file (what I was trying to do > with rdiff-backup). This was just with standard options, no block size > changes or anything. Here are the timings I got: [...] > So that's over 4 and a half hours. I'm thinking maybe I'll need to play > with block sizes but that only mentions changing the block sizes for the > signature file, so I don't know if it will make any difference for > creating the delta. Anyway, I'll try some more things, but it looks
The block size will have a significant impact on speed, as it "walks" through the file faster when there are hits. I would recommend using a blocksize that is the square-root of the file size... ie 1M file, 1K blocksize 1G file, 32K blocksize 4G file, 64K blocksize 16G file, 128K blocksize 64G file, 256K blocksize (this also reduces the probablity of blocksum collisions, which though very unlikely, will cause corruption). However, if you know some characteristics of the file contents, you will benefit by taking that into account.. if know it is full of fixed sized records, you should pick a block size that is a multiple of the record size (taking into account the recommended size above). > like rdiff doesn't suffer from the same problem rdiff-backup does. I'm > still testing and it takes a while to run these. I'll report back again. I _think_ rdiff-backup uses an extension module that hooks into librsync itself. It is possible the rdiff-backup extension module is not correctly compiled with 64 bit support. I suspect the whole python interpreter would need to be compiled with 64 bit support. -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ _______________________________________________ rdiff-backup-users mailing list at [email protected] http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
