>>>>> Donovan Baarda <[EMAIL PROTECTED]> >>>>> wrote the following on Fri, 06 May 2005 12:22:16 +1000
> The block size will have a significant impact on speed, as it > "walks" through the file faster when there are hits. I would > recommend using a blocksize that is the square-root of the file > size... ie > > 1M file, 1K blocksize > 1G file, 32K blocksize > 4G file, 64K blocksize > 16G file, 128K blocksize > 64G file, 256K blocksize > > (this also reduces the probablity of blocksum collisions, which > though very unlikely, will cause corruption). Ahh rdiff-backup chooses the blocksize to be approximately 1/2000th of the length of the file, witha minimum of 512 bytes (see find_blocksize in Rdiff.py). So if the file is 25 gigs large, perhaps Clint could try running rdiff with a blocksize of 13421568. But I'm not sure how I ever came up with that formula, and probably there was no sound reasoning behind it. Should I switch to the square-root thing (minimum blocksize 512, blocksize always multiple of 512?)? I remember there being some discussion about this, but I probably never updated rdiff-backup with the correct function. > I _think_ rdiff-backup uses an extension module that hooks into > librsync itself. It is possible the rdiff-backup extension module is > not correctly compiled with 64 bit support. I suspect the whole > python interpreter would need to be compiled with 64 bit support. Yes, that is correct, see _librsyncmodule.c. If you want to test rdiff-backup's librsync stuff separately, you may want to check out python-rdiff, which is a simple port of rdiff to rdiff-backup's librsync extension module. You can download it from the rdiff-backup CVS at: http://savannah.nongnu.org/cgi-bin/viewcvs/*checkout*/rdiff-backup/rdiff-backup/python-rdiff?rev=HEAD&content-type=text/plain Note the blocksize there is fixed, so you'll have to edit the top to test different sizes. Also I haven't tested it recently, but it looks like it should still work. -- Ben Escoto
pgpinEMjA9KrH.pgp
Description: PGP signature
_______________________________________________ rdiff-backup-users mailing list at [email protected] http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
