On Apr 18, 2008, at 2:00 PM, Tony Schreiner wrote: > dedicated backup server, 64-bit CentOS on 4 GB RAM. dstat doesn't > show any paging. the clients tend to have much more RAM
Very good, that sounds adequate, more cores and more RAM on the server could still help a bit, or an additional server or two (but again budget) but sounds like not the main cause of the problem. > The BackupPC host summary web page shows speeds between 9 MB/s and > 35 MB/s depending on the client, I think that's in line with what > others are seeing on Gbit networks. The storage is a 3Ware 9550 with > 10 disks ( I admit that the controller BBU has failed and needs > replacing, and that is slowing down my write speed). Sounds about right. > When I watch the progress of backups on the server, what really > seems to be the slowest part is when large files ( > several Gb ) > are being compared against the pool. I don't know enough about the > internals of the software to know quite what is happening there. This is good info, could get you answers more suited to your particular set up. First, a word of caution, it's possible that changing compression or checksum caching will cause BackupPC to immediately eat up 2x as much disk space until old files expire from the pool, someone here should be better qualified to explain if that could be a problem. I do know for sure that happens if you change certain parameters that affect the hashing function. You should use "Rsync checksum caching" if you do not already do so, that may help quite a bit, pretty good description in the faq in the section with the same title, this will take advantage of rsync's built- in checksum functions. There is some risk that if a file gets corrupted in the pool for some reason, you won't learn about it until you try to restore, so you should keep that in mind especially since the RAID controller has dead BBU. http://backuppc.sourceforge.net/faq/BackupPC.html If these huge files are not more or less "static" and you can manage enough disk space, you may want to turn off compression, or perhaps exclude those files from BackupPC and use a different method to back them up, at least try a smaller value for $Conf{CompressLevel} See also previous faq topic "Compressed file format". If the files themselves are already compressed on the source, then BackupPC should "detect" that compressing them more doesn't help and switch to flush() in which case turning off compression probably won't help. I haven't dug around in the guts of BackupPC for some time and all of the configuration options that are possible, there may well be some option to not compress files over some size limit, or other tweaks that you could make, hopefully someone here could give you some better answers on that. Jonathan ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/