Just a quick update: I tried making these changes on both sides and it really didn't make a difference. Full backup of 202852072 Kbytes required 2267m25.913s (previously it took 2346m57.800s, so it only sped up a factor of 3%.
Only thing I have not yet tried is running a raw rsync to see how fast that runs. I'll do that next. So, back to my orignal question: anyone have any idea how to get initial transfers to run faster (or indeed any significant data transfers)? Thanks, -derek "Derek Atkins" <de...@ihtfp.com> writes: > Hi, > > I'm trying to use rdiff-backup to backup a bunch of servers. One > particular server contains about 160GB of data, but when I try to perform > the rdiff-backup it's saving the data at a measly 1MB/s. > > Here's my configuration: > > [server] <--ssh--> [backup-server]{encfs} <--nfs--> [freenas] > > I ran a bunch of tests to try to figure out my bottlenecks. > > I ran a bunch of dd tests (using dd if=/dev/zero bs=1M count=1000) on the > backup server. Going directly to FreeNAS via NFS (bybassing encfs) I get > 50.2MB/s. If I run dd directly on the backup server (through encfs) I get > 20.1MB/s. If I go over SSH from the backup server to the target server > and run the dd on the target server, then write to FreeNas through encfs > declines to 7.6MB/s. > > Note that in those SSH tests, however, I forgot to turn off compression. > When I do that, the throughput for the dd test reduced to 6.6BM/s. (Note > that this is running simultaneously with a running rdiff-backup, so it's > possible that they are reducing performance). > > Then I ran an scp test to the same target server; copying about 1.4GB of > photos. Files ranged in size from 10KB to 5MB. When run in standard mode > (displaying each file status) I got 4.4MB/s. Running in quiet mode I get > 5.1MB/s. > > So clearly the bottleneck is in rdiff-backup -- performance (IMHO) should > not be significantly slower than the last dd-over-ssh test. It appears > rdiff-backup is slowing me down by a factor of 5x throughput versus scp. > > I found a message from Ben from 2005 where he suggests increasing the > blocksize and conn_bufsiz settings in Globals.py: > https://lists.gnu.org/archive/html/rdiff-backup-users/2005-10/msg00062.html > > What he didn't say was whether this needed to be changed on the target > server, the backup server, or both. Nor do I know if that would actually > help this situation. > > Do you have any ideas? > > Thanks, > > -derek > > PS: According to rpm, both systems are running version 1.2.8. -- Derek Atkins 617-623-3745 de...@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki