Re: [Veritas-bu] Slow Informix Logs backups
On Thu, 2 Apr 2009, Larry Mascarenhas wrote: Thanks. Tried this and it eventually does catch up. But this is not the correct solution to the problem. It masks the real issue ie the Informix Module generates a job for each log. The processing of this job is the real reason the log backups are slow (even for disk). In reviewing how ONBAR works, it opens 1 connection to NBU and ships the logs sequentially to it. It is the same onbar process that delivers every log, not separate ones. But NBU treats each log as a separate request and generates a job for it. Can you imagine a STREAM for a FS and each file is a separate request and therefore a separate job? I have to escalate this. Yikes-- agree, this is broken, let us know if you have any updates, thanks. But thanks for this. It is a workaround. No problem, better to have something working then no backups/problematic backups. Justin. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Slow Informix Logs backups
Hi Patrick, Thanks for your reply. 1. I'm running NBU v6.5.1. 2. I believe I see a lot of subjobs for the logs and yes I have multistreaming enabled on the policy. In fact the 300G DB backs up in a 1hour, but 30Gig of logs takes 3 hours. 3. Onbar streams for DB backups. For logs, it opens 1 onbar process (the same one that backed up the DB) and ships the logs sequentially. That's non-configurable (I'm pretty sure). Patrick wrote: I didn't notice which version of NBU so I'll assume the latest. In the activity monitor do you see a parent job with a lot of sub jobs for the log files? Does the policy allow multistreaming? On the ONBAR side can you specify single or multiple streams? Just some thoughts. Regards, Patrick Whelan VERITAS Certified NetBackup Support Engineer for UNIX. VERITAS Certified NetBackup Support Engineer for Windows. netbac...@whelan-consulting.co.uk -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Larry Mascarenhas Sent: 03 April 2009 01:55 To: Justin Piszcz Cc: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Slow Informix Logs backups Thanks. Tried this and it eventually does catch up. But this is not the correct solution to the problem. It masks the real issue ie the Informix Module generates a job for each log. The processing of this job is the real reason the log backups are slow (even for disk). In reviewing how ONBAR works, it opens 1 connection to NBU and ships the logs sequentially to it. It is the same onbar process that delivers every log, not separate ones. But NBU treats each log as a separate request and generates a job for it. Can you imagine a STREAM for a FS and each file is a separate request and therefore a separate job? I have to escalate this. But thanks for this. It is a workaround. Justin Piszcz wrote: On Fri, 27 Mar 2009, Larry Mascarenhas wrote: Hello, I'm running backups on Informix7 DBs. We're able to stream the DB backups using parallelism 4 and get good throughput (100MB/s, yes megabytes/sec, it is a SAN Media Server). A 300Gig DB takes less than 1 hour. However (I know ;)), when it backs up the LOGS (all backups go to tape, I don't have Disk backups), each log comes in as a separate request. Each log is therefore a separate job. So the time taken to process this request is greater than the time to backup each log. ie each log is 100MB and the time taken is 2secs (when the resource is allocted and ready), but it takes 1+min to create the job/assign the resource/and begin writing. This is resulting in my never being able to catch up with the rate of logs generation although the amount to be backed up is minimal eg 30logs is just 3Gig, but takes approx 30mins and by that time 10 more logs are created. I am using media_unmount_delay so the tapes are not re-mounting for every request. I cannot increase the size of the LOGS as this would affect replication and recoverability (double whammy). This is Symantec's recommendation. I'm looking for experiences that people have had to speed up the Informix backups. Thanks. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Why not use a Disk Staging Unit if the logs are small enough? Send to disk first - then shoot to tape? Justin. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Slow Informix Logs backups
Thanks. Tried this and it eventually does catch up. But this is not the correct solution to the problem. It masks the real issue ie the Informix Module generates a job for each log. The processing of this job is the real reason the log backups are slow (even for disk). In reviewing how ONBAR works, it opens 1 connection to NBU and ships the logs sequentially to it. It is the same onbar process that delivers every log, not separate ones. But NBU treats each log as a separate request and generates a job for it. Can you imagine a STREAM for a FS and each file is a separate request and therefore a separate job? I have to escalate this. But thanks for this. It is a workaround. Justin Piszcz wrote: On Fri, 27 Mar 2009, Larry Mascarenhas wrote: Hello, I'm running backups on Informix7 DBs. We're able to stream the DB backups using parallelism 4 and get good throughput (100MB/s, yes megabytes/sec, it is a SAN Media Server). A 300Gig DB takes less than 1 hour. However (I know ;)), when it backs up the LOGS (all backups go to tape, I don't have Disk backups), each log comes in as a separate request. Each log is therefore a separate job. So the time taken to process this request is greater than the time to backup each log. ie each log is 100MB and the time taken is 2secs (when the resource is allocted and ready), but it takes 1+min to create the job/assign the resource/and begin writing. This is resulting in my never being able to catch up with the rate of logs generation although the amount to be backed up is minimal eg 30logs is just 3Gig, but takes approx 30mins and by that time 10 more logs are created. I am using media_unmount_delay so the tapes are not re-mounting for every request. I cannot increase the size of the LOGS as this would affect replication and recoverability (double whammy). This is Symantec's recommendation. I'm looking for experiences that people have had to speed up the Informix backups. Thanks. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Why not use a Disk Staging Unit if the logs are small enough? Send to disk first - then shoot to tape? Justin. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Slow Informix Logs backups
I agree with the DSU suggestion -- this is exactly the reason we use that method with an Informix DB here. Waiting for tape to do these backups just isn't fast enough. The DSU solves this problem completely. -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Justin Piszcz Sent: Friday, March 27, 2009 4:48 PM To: Larry Mascarenhas Cc: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Slow Informix Logs backups On Fri, 27 Mar 2009, Larry Mascarenhas wrote: Hello, I'm running backups on Informix7 DBs. We're able to stream the DB backups using parallelism 4 and get good throughput (100MB/s, yes megabytes/sec, it is a SAN Media Server). A 300Gig DB takes less than 1 hour. However (I know ;)), when it backs up the LOGS (all backups go to tape, I don't have Disk backups), each log comes in as a separate request. Each log is therefore a separate job. So the time taken to process this request is greater than the time to backup each log. ie each log is 100MB and the time taken is 2secs (when the resource is allocted and ready), but it takes 1+min to create the job/assign the resource/and begin writing. This is resulting in my never being able to catch up with the rate of logs generation although the amount to be backed up is minimal eg 30logs is just 3Gig, but takes approx 30mins and by that time 10 more logs are created. I am using media_unmount_delay so the tapes are not re-mounting for every request. I cannot increase the size of the LOGS as this would affect replication and recoverability (double whammy). This is Symantec's recommendation. I'm looking for experiences that people have had to speed up the Informix backups. Thanks. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Why not use a Disk Staging Unit if the logs are small enough? Send to disk first - then shoot to tape? Justin. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] Slow Informix Logs backups
Hello, I'm running backups on Informix7 DBs. We're able to stream the DB backups using parallelism 4 and get good throughput (100MB/s, yes megabytes/sec, it is a SAN Media Server). A 300Gig DB takes less than 1 hour. However (I know ;)), when it backs up the LOGS (all backups go to tape, I don't have Disk backups), each log comes in as a separate request. Each log is therefore a separate job. So the time taken to process this request is greater than the time to backup each log. ie each log is 100MB and the time taken is 2secs (when the resource is allocted and ready), but it takes 1+min to create the job/assign the resource/and begin writing. This is resulting in my never being able to catch up with the rate of logs generation although the amount to be backed up is minimal eg 30logs is just 3Gig, but takes approx 30mins and by that time 10 more logs are created. I am using media_unmount_delay so the tapes are not re-mounting for every request. I cannot increase the size of the LOGS as this would affect replication and recoverability (double whammy). This is Symantec's recommendation. I'm looking for experiences that people have had to speed up the Informix backups. Thanks. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Slow Informix Logs backups
On Fri, 27 Mar 2009, Larry Mascarenhas wrote: Hello, I'm running backups on Informix7 DBs. We're able to stream the DB backups using parallelism 4 and get good throughput (100MB/s, yes megabytes/sec, it is a SAN Media Server). A 300Gig DB takes less than 1 hour. However (I know ;)), when it backs up the LOGS (all backups go to tape, I don't have Disk backups), each log comes in as a separate request. Each log is therefore a separate job. So the time taken to process this request is greater than the time to backup each log. ie each log is 100MB and the time taken is 2secs (when the resource is allocted and ready), but it takes 1+min to create the job/assign the resource/and begin writing. This is resulting in my never being able to catch up with the rate of logs generation although the amount to be backed up is minimal eg 30logs is just 3Gig, but takes approx 30mins and by that time 10 more logs are created. I am using media_unmount_delay so the tapes are not re-mounting for every request. I cannot increase the size of the LOGS as this would affect replication and recoverability (double whammy). This is Symantec's recommendation. I'm looking for experiences that people have had to speed up the Informix backups. Thanks. -- Larry Mascarenhas lmasc...@comcast.net ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Why not use a Disk Staging Unit if the logs are small enough? Send to disk first - then shoot to tape? Justin. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
