Re: [BackupPC-users] 2.1 versus 3.0
The real question is why your backups and your nightly runs are taking so long to complete. One reason might be network bandwidth. In which case, you're probably stuck. I'm gonna guess this isn't the problem though. If you're on a LAN almost certainly this isn't it. It's easy to measure this. Another reason might be CPU on the backup server. This is easy enough to monitor. I kind of doubt this is your problem either. This leaves us with resource constraints on the backup clients - this is possible on busy file servers, but doesn't explain why nightly jobs take so long. My bet is that your problem is disk I/O on the backup server. BackupPC's (very clever, a neat hack and i love it) pooling scheme has a problem. All those hard links all over the place mean that your disk heads need to seek a lot. This is a real bummer and really slows down your disk I/O, especially with ATA (including most SATA) disks. all parallel ATA & most serial ATA disks can only handle a single request at a time. this means they service a request, then get another and service it. This means that each and every request, even for a tiny block of data, needs a rotation of the disk to occur (probably on average half a rotation?). This shouldn't make much of a difference on the actual data transfer stage of your backups, but it will make your backuppc_link and backuppc_nightly processes take forever. SCSI disks with tagged command queuing (most of them, i think) and SATA disks that support Native Command Queuing (definitely not most of them, and your OS & maybe your SATA card need to support it too) can potentially service many more requests, because the disk can service the requests out of order. Seeks happen much faster than a single rotation of a disk, especially on 7200rpm (and 5400rpm, egads) disks. Another possibility for slow i/o is RAID5. Writes to RAID5 are significantly slower than writes to RAID1 (or simple unraid'd disks). RAID10 is better than RAID5 but is probably not optimal either. For this kind of workload, you probably do not want RAID 10, but rather you want to have multiple RAID 1 devices concatenated together (what linux md calls LINEAR and solaris disksuite calls CONCAT). This allows the disk pairs to seek independently of one another. RAID10 is designed for high raw throughput, not to optimize seeks. With all this in mind, if you think this is causing your problems, there are a few ways to improve your situation. 1) the most obvious is to get disks that support command queueing. 2) the next most obvious is to get rid of RAID5 or RAID10 and move to concatenated RAID1's. * Getting rid of RAID5 might be more bang for the buck than getting command queuing. 3) Even if you don't need the extra space you may find that things improve just by adding extra RAID1 devices to your concatenated disk. This will spread the seeks out a bit. Ideally, ensure that all new stuff gets written to the new disk by doing something like this: - Fill the existing disk with empty files - add the new disk at the end of your device - grow your filesystem to match the size of your device. Of course, during this process, pray that it all works and that you don't whack your backup store. Back it up first if you can. Hint, standard backup tools like tar or rsync or dump won't work worth a damn on the backuppc store, for the same kind of reasons that i discuss above, but you can just dd the raw device to a backup disk or tape. On Wed, Jan 31, 2007 at 04:55:12PM -0600, Jason Hughes wrote: > James Ward wrote: > > it looks like they're going to all get started at the same time again > > due to waiting on the nightly process to complete after the longest > > of these backups. > > > > Does version 3 get me away from this scenario? > Yes. Version 3 doesn't need nightly processing to be mutually exclusive > to backups. They should fire off whenever they're due to start. > However, if you having trouble with one taking more than 24 hours, and > the period between backups is less than this, it will pretty much always > be backing up those machines and falling further behind. Since you > mention having multiple backup servers, perhaps putting the largest file > server hosts onto different backuppc servers would help? > > And on my other > > server that's backing up 200 machines (some remote), will it be able > > to just backup 24x7 with version 3? Right now it spends most of > > every day from the wee hours until the afternoon doing the nightly > > cleanup. > > > Again, this should be alleviated in version 3. Even if the processing > is still lengthy, it should not bunch up your backups anymore, so > theoretically, the same server has greater capacity in version 3 than in > version 2. > > JH > > - > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly
Re: [BackupPC-users] 2.1 versus 3.0
Le jeudi, 1 février 2007 11:01, vous avez écrit : > Linux sees four processors in my servers: Intel(R) Xeon(TM) CPU 3.20GHz > So 4 is a good number for me? There are only 2 physical CPUs however. Hyperthreading (at least I guess 3.2GHz Xeons are hyperthreaded, and not dual-core) doesn't give you 2 full CPUs. The number that was circulated in the time when hypethreading was still new was that a CPU with 2 hyperthreads acts as 1.3 physical CPUs. So, I believe it's still better to use MaxBackups=2 in your case. The hyperthreaded CPUs will probably help a lot (and better than MaxBackups=4) with the I/O being done simultaneously with the rsync/compression/hashing crunching. Of course, you can do experimenting. Just look at the time backups take for a particular big host the nights before before and the ones after the modifying of this parameter (you get this info in the page of each host). (and with BackupPC3.0, you can play with these configs directly in the cgi interface, many thanks Craig!) -- Cristian Tibirna(1-418-) 656-2131 / 4340 Laval University - Quebec, CAN ... http://www.giref.ulaval.ca/~ctibirna Research professional at GIREF ... [EMAIL PROTECTED] - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] 2.1 versus 3.0
Le mercredi, 31 janvier 2007 17:20, Carl Wilhelm Soderstrom a écrit :
> You probably also want to change:
> $Conf{MaxBackups} = 4;
>
> to something lower. that's the number of backups which will run
> simultaneously. I've found that the default of '4' is too high for most
> backup servers, and that '2' is a better value in most cases.
Just to underline the goodness of this advice.
Since the beginning, MaxBackups was set on my servers to be equal to number of
processors in the backup server. This way, I ensure that the load isn't so
high that it makes the kernel spend too much time in scheduling (or this is
the idilic interpretation that I give to my choice), especially given that
quite a bit of processor cycles have to be spent in managing the disk io (and
I also use raid and lvm on the backup partition). I never had too much
overload trouble. Backup_nightly starts to get a bit annoyingly longish, but
it's manageable for now.
--
Cristian Tibirna(1-418-) 656-2131 / 4340
Laval University - Quebec, CAN ... http://www.giref.ulaval.ca/~ctibirna
Research professional at GIREF ... [EMAIL PROTECTED]
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
Re: [BackupPC-users] 2.1 versus 3.0
James Ward wrote: > it looks like they're going to all get started at the same time again > due to waiting on the nightly process to complete after the longest > of these backups. > > Does version 3 get me away from this scenario? Yes. Version 3 doesn't need nightly processing to be mutually exclusive to backups. They should fire off whenever they're due to start. However, if you having trouble with one taking more than 24 hours, and the period between backups is less than this, it will pretty much always be backing up those machines and falling further behind. Since you mention having multiple backup servers, perhaps putting the largest file server hosts onto different backuppc servers would help? > And on my other > server that's backing up 200 machines (some remote), will it be able > to just backup 24x7 with version 3? Right now it spends most of > every day from the wee hours until the afternoon doing the nightly > cleanup. > Again, this should be alleviated in version 3. Even if the processing is still lengthy, it should not bunch up your backups anymore, so theoretically, the same server has greater capacity in version 3 than in version 2. JH - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] 2.1 versus 3.0
On 01/31 02:52 , James Ward wrote:
> I have three servers (2.x) in production. One of them is backing up
> three big fileservers. Any one of these backups can take more than
> 24 hours. I recently started full backups of each of the servers on
> different days to see if I could stagger the network load, but it
> looks like they're going to all get started at the same time again
> due to waiting on the nightly process to complete after the longest
> of these backups.
would changing the IncrPeriod to greater than 24 hours help?
the default is:
$Conf{IncrPeriod} = 0.97;
this is basically what causes backups to go off once a day.
perhaps for these machines you can set this to 1.47, or 1.97; so they would
get backed up once every day and a half or two days?
You probably also want to change:
$Conf{MaxBackups} = 4;
to something lower. that's the number of backups which will run
simultaneously. I've found that the default of '4' is too high for most
backup servers, and that '2' is a better value in most cases.
--
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
[BackupPC-users] 2.1 versus 3.0
I have three servers (2.x) in production. One of them is backing up three big fileservers. Any one of these backups can take more than 24 hours. I recently started full backups of each of the servers on different days to see if I could stagger the network load, but it looks like they're going to all get started at the same time again due to waiting on the nightly process to complete after the longest of these backups. Does version 3 get me away from this scenario? And on my other server that's backing up 200 machines (some remote), will it be able to just backup 24x7 with version 3? Right now it spends most of every day from the wee hours until the afternoon doing the nightly cleanup. Thanks in advance, James - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
