Hi Ted, On 02.08.19 20:09, Ted Toal wrote: > Hi Alex, > > Ok, thanks for that suggestion, I’d thought of it, but wasn’t sure if rsync > would complain if the arg appeared twice, but apparently it doesn’t. > > I am NOT sure whether bandwidth limitation is what I want. I am actually > trying to throttle down not only the network bandwidth used but also the I/O > load. This is a shared file system with hundreds of users accessing it. I’m > only backing up our lab’s small portion of the data, and I’m only backing up > files less than 1 MB in size. The full backups are done separately by > someone else in a different manner. For my <1 MB files, I am doing a full > backup once a year and an incremental backup once an hour.
> I want to have essentially 0 impact on the network bandwidth and on the I/O > load between the server that talks to BackupPC and the network storage device. I'm not 100% sure, but this sounds way more complicated than throttling the bandwidth between the BackupPC server and the host. IIUC, your situation is: BPC (1) ---(a)--- host (2) ---(b)--- NAS (3) BPC (1) is the BackupPC server; host (2) is the system you want to back up, i.e., the client from BackupPC's perspective; and NAS (3) is the server providing the shared file system. You want to limit I/O on 3 as well as bandwidth on link b, with privileged access to only 1, no access to 3, and probably no chance of changing the way 2 communicates with 3, correct? (E.g., to set up a dedicated NFS connection where the server side (3) is I/O-limited.) Here's my gut feeling: (disclaimer: unconfirmed, highly dependent on your exact setup, and I'm not an expert on NFS setups) In that situation, ionice on 2 won't help; the rsync instance running on host 2 is purely cpu- and network-bound, but has negligible local I/O (controlled by ionice). And limiting cpu (via nice) and network bandwidth (via trickle, e.g.) on 2 won't help, either: just listing files on an NFS is usually a bottleneck, because individual requests have to pass the link b. If you somehow manage to limit the bandwidth across b, actual *content* transfer will be horribly slow. (And I expect this to be difficult, as the NFS is probably pre-mounted via a mechanism that you can't control.) The only reasonable idea, AFAICS, would be to rate-limit the *number* of files accessed. But I do not see how this could be done, short of modifying the rsync-sender on host 2. IMHO, the one and only *proper* way to install such a backup solution would be to ask your friendly staff managing the NAS 3 (hopefully experts on how their setup works, if it serves 100+ users) to grant you access to their backups (which they surely have), or give you read-only direct access to NAS 3 with proper limits. What you're trying to do sounds like their job, and even if you have reasons to think that you might do better or have specific requirements they won't be able to fulfill, you're not in best position to implement it. Just my 2 pennies from someone who enjoys not having do deal with NFS a lot... Alex > Since I’m just starting, I’m doing the first full backups, and they are > taking forever. I have a bandwidth limit of 1 MB/s, very low. I need to > explore how high I can go without impacting other’s access, and how high I > need to go to finish the full backups and incremental backups in a timely > fashion. I’m thinking a higher bandwidth limit for the full backups would > get them done quicker with still little impact. For the incrementals, I > haven’t done one yet so I don’t know how long it will take, but I may > discover I have to increase that bandwidth also, and/or decrease the > frequency of the incrementals. > > Based on that, do you think I should be using ionice too? And by the way, I > do not have root access to the server. > > Ted
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
