Hi, Bowie Bailey wrote on 2018-06-14 15:38:44 -0400 [Re: [BackupPC-users] Slow local backup]: > On 6/14/2018 3:27 PM, Michael Stowe wrote: > > On 2018-06-14 10:05, Bowie Bailey wrote: > > > > I just installed BackupPC v4 on a CentOS 7 server with 4G ram. I > > am trying to back up a local 318G filesystem. I am using rsyncd > > over the loopback connection. It has been running for 17 hours so > > far and has backed up less than half of the directory (based on > > the size of the backup filesystem). Running top does not show any > > excessive cpu or iowait. ???free??? shows no swap usage and 1.5G > > available memory.
if I'm not miscalculating, that is roughly 10GB/hour or 3MB/s. From what I've read about BackupPCv3 performance, that wouldn't seem extremely unreasonable, especially for a first backup. For V4, I have no idea what the common figures are. The C implementation (rsync_bpc) might have a performance benefit. But I would expect one core of your CPU to be almost 100% busy, while the others may be idle. Depending on which figure you are looking at, that might not seem excessive, but it's the most you can get for single threaded compression performance (you *are* using compression, right?). > [...] > I use rsyncd rather than rsync to avoid the ssh overhead. For a local backup, you can achieve the same by using 'sudo' instead of 'ssh'. At least you could with V3. I've forgotten if and how you can use 'sudo' with rsync in V4. > I expected a backup done via the loopback interface to be fast since it > doesn't have the normal networking bandwidth limitations. Well, yes, but you still have a network stack and probably at least two copies between kernel and user space. loopback networking does not come for free. A quick 'netperf' test gives me a bit less than 9 Gbit/s throughput. That's certainly faster than Gbit networking, but only by a factor of 10. More importantly, the other limitations don't change - disk and compression speed, for instance. If your bottleneck is not the network, a faster network won't change anything. Keep in mind that a local backup means that client and server are the same computer, so a loopback backup might actually be *slower* than a remote backup. Your source file system and BackupPC pool are on different physical disks, hopefully? > > Is it normal for the backup to take this long? > > > > While that's hard to guess without knowing the particulars of your > > system, I'm going to go out on a limb and say, no. No it is not. I believe it is still important whether it is the first backup or not. The recommendation used to be "ignore the timing of the first backup - fix your problem only if the second (or third) backup is still too slow". That would still seem to apply to BackupPCv4. > > Is there a better way to back up a local filesystem? > > > > Personally, I use rsync (not rsyncd) and in the cases where I have > > experienced slowness, it was due to poorly chosen rsync parameters (I > > note that this would not differ between rsync and rsyncd), a broken > > filesystem, or a specific bug in a specific version of rsync. > > How do you go about setting up rsync so that it does a local copy rather > than going through ssh over the network? I would try setting $Conf {RsyncSshArgs} = [ '-e', '/usr/bin/sudo' ], but you might run into quoting problems. You could then try using a script containing something like 'exec /usr/bin/sudo $@'. Or see if using 'ssh' makes much of a difference at all ... Hope that helps. Regards, Holger ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/