Re: Smoothing active backup sessions
Jim, Thanks for the new idea. I'll have to ponder that one. Best wishes, Keith
Re: Smoothing active backup sessions
Wanda, Thank you. It's helpful to know you have been doing that routinely. Best wishes, Keith
Re: Smoothing active backup sessions
Is it possible to control this with MAXSESSIONS? Associate a large number of servers with a long backup schedule and set MAXSESSIONS to 40. With a MAXSCHEDSESSIONS value of 50 (percent) you would only run 20 backups at a time. The optimistic UNTESTED result would be to start new backups when to session total falls below 20. I have not tried this, but it might work. Regards, Jim Schneider -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@vm.marist.edu] On Behalf Of Keith Arbogast Sent: Friday, May 27, 2011 9:51 AM To: ADSM-L@vm.marist.edu Subject: [ADSM-L] Smoothing active backup sessions A bar graph of our active backup sessions during the night is far from a Golden Rectangle. It's more like a big city skyline. We are backing up 510 clients now, but that number is increasing steadily. I am mulling over ways to smooth our session load. The hoped for benefit would be to make more effective use of available resources, and to delay the need for more TSM servers. We run TSM server version 6.2.2.2 on RHEL5 on a Dell R810 server. It's fibre connected to a Hitachi SAN. Till now we have just aimed to average the number of associations within each schedule.That isn't effective since clients have widely different run times. Both of the following techniques use changes in Scheduling options to attempt to effect session smoothing. I would be glad to hear opinions on how well, or not, they might work. 1) The first idea is to overlap Schedule Start Times. The randomization server option is there to avoid starting all backups in a Schedule together. That's a major knob for smoothing, but the highest setting for it is '50' which distributes actual start times over only the first half of a backup window. This allows clients to retry before the window expires. Since actual start times are spread over the first half of the window only, session count is necessarily higher in that half, which effectively de-smoothes the two halves of the window. Sawtooth City. We have few client communication errors, so we could set each Schedule's start time to be midway through the window of the previous Schedule. Then, the lightly used second half of the previous schedule would be filled with sessions from the first half of the overlapping schedule. If Schedule Durations were an hour, Schedules would start every half-hour. There would be more defined Schedules, but fewer Associations per schedule. To be most effective, this technique may depend on the elapsed backup times of the clients in the overlapping schedules being more or less equal. So, it may need to be combined with the next technique which defines associations based on the experienced or expected elapsed run time of the client. 2) Another way to smooth sessions could be to make schedule Durations inversely proportional to the usual elapsed time of the clients associated with them rather than setting all Durations to an hour. About twenty-five percent of our backups finish in one minute or less. About fifty percent finish in five minutes or less. The easiest way to spread the good effect of all those light loads is to put them in a long window and let Randomization distribute their actual start times. I am thinking of starting with a four hour duration for the clients with run times of less than one minute. Clients with long elapsed times would be associated with schedules of shorter duration so their start times could be more tightly controlled. Clients with intermediate elapsed times would be associated with schedules of intermediate duration. Since this technique depends on Randomization working as well with long Schedule Durations as with shorter, does anyone have experience with Durations of four hours or more? Did it go alright? Any reactions to these ideas would be gratefully accepted. With my thanks and best wishes, Keith Arbogast Indiana University
Re: Smoothing active backup sessions
>>Since this technique depends on Randomization working as well with long >>Schedule Durations as with shorter, does anyone have experience with >>Durations of four hours or more? Did it go alright? Do it all the time. Whenever I set up a new customer, I tell them to make the Duration window as big as they can tolerate (usually 8-12 hours), throw all the clients in. Then see what clients cause a problem, and make special schedules for the clients that need to be forced to more particular start/run times.
Smoothing active backup sessions
A bar graph of our active backup sessions during the night is far from a Golden Rectangle. It's more like a big city skyline. We are backing up 510 clients now, but that number is increasing steadily. I am mulling over ways to smooth our session load. The hoped for benefit would be to make more effective use of available resources, and to delay the need for more TSM servers. We run TSM server version 6.2.2.2 on RHEL5 on a Dell R810 server. It's fibre connected to a Hitachi SAN. Till now we have just aimed to average the number of associations within each schedule.That isn't effective since clients have widely different run times. Both of the following techniques use changes in Scheduling options to attempt to effect session smoothing. I would be glad to hear opinions on how well, or not, they might work. 1) The first idea is to overlap Schedule Start Times. The randomization server option is there to avoid starting all backups in a Schedule together. That's a major knob for smoothing, but the highest setting for it is '50' which distributes actual start times over only the first half of a backup window. This allows clients to retry before the window expires. Since actual start times are spread over the first half of the window only, session count is necessarily higher in that half, which effectively de-smoothes the two halves of the window. Sawtooth City. We have few client communication errors, so we could set each Schedule's start time to be midway through the window of the previous Schedule. Then, the lightly used second half of the previous schedule would be filled with sessions from the first half of the overlapping schedule. If Schedule Durations were an hour, Schedules would start every half-hour. There would be more defined Schedules, but fewer Associations per schedule. To be most effective, this technique may depend on the elapsed backup times of the clients in the overlapping schedules being more or less equal. So, it may need to be combined with the next technique which defines associations based on the experienced or expected elapsed run time of the client. 2) Another way to smooth sessions could be to make schedule Durations inversely proportional to the usual elapsed time of the clients associated with them rather than setting all Durations to an hour. About twenty-five percent of our backups finish in one minute or less. About fifty percent finish in five minutes or less. The easiest way to spread the good effect of all those light loads is to put them in a long window and let Randomization distribute their actual start times. I am thinking of starting with a four hour duration for the clients with run times of less than one minute. Clients with long elapsed times would be associated with schedules of shorter duration so their start times could be more tightly controlled. Clients with intermediate elapsed times would be associated with schedules of intermediate duration. Since this technique depends on Randomization working as well with long Schedule Durations as with shorter, does anyone have experience with Durations of four hours or more? Did it go alright? Any reactions to these ideas would be gratefully accepted. With my thanks and best wishes, Keith Arbogast Indiana University
Re: Tsm server possibly limiting backup performance
The networking itself is just one element in the overall backup operation... You don't say what measurement point you are looking at for your throughput numbers. If it is the "Network data transfer rate" TSM summary statistic, that reflects TSM interacting with the network layer of the operating system, with affecters such as TSM TCPWindowsize, Path MTU Discovery where that is in effect, etc. Unexpected, elongated network routing can delay things; but your FTP test should have run into that if were a factor...unless conditions are changing depending upon time of day. When a TDP is involved, it can only pass data as fast as the back end application provides it. A traditional B/A backup can be impaired by the paging incited by the amount of memory taken by the Active files list. If standard measurement points don't reveal a cause, a client trace with a representative data set can be illuminating. Richard Sims
Re: Tsm server possibly limiting backup performance
Sorry, thought I had my cases correct. 1 gb (gigabit) ethernet ocnnection. Ftp and other tests yielded 800 megabit performance, Tsm sessions average 200 to 250 megabit. Gary Lee Senior System Programmer Ball State University phone: 765-285-1310 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Paul Fielding Sent: Friday, May 27, 2011 10:30 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Tsm server possibly limiting backup performance I think clarification is needed on mb/s vs. MB/s. a 1gb (gigabit) (note lowercase) connection will certainly do 800 mb/s (megabit/s). it won't, however, do 800 MB/s (megabyte/s) Paul On Fri, May 27, 2011 at 8:09 AM, Richard Rhodes wrote: > A little more explanation is going to be needed. A 1GB ethernet is not > going to give 800mb/s, let alone 200mb/s. 1GB ethernet is only going to > provide one direction throughput of 110mb/s. Do you mean a 10gb ethernet > connection? > > > > > > From: "Lee, Gary D." > To: ADSM-L@VM.MARIST.EDU > Date: 05/27/2011 09:35 AM > Subject:Tsm server possibly limiting backup performance > Sent by:"ADSM: Dist Stor Manager" > > > > Tsm server 6.2.2 > OS redhat enterprise linux 6.0 > Client tdp for exchange, sqlserver, and backup archive client > > Client connected via 1 gb ethernet connection. > > In my experience, I yhave never been able to se throughput of much more > than 200 mb/s during a tsm backup, whether tdp or regular client. > I wonder, is TSM limiting a single session's bandwidth so as to insure > availability for other client sessions which might begin; i.e. prevent one > backup stream from monopolizing the server's network connection? > > I have run ftp and other tests to the server, and achieved throughput of > around 800 mb/s over the same connection. > > > > Gary Lee > Senior System Programmer > Ball State University > phone: 765-285-1310 > > > > > > - > The information contained in this message is intended only for the > personal and confidential use of the recipient(s) named above. If > the reader of this message is not the intended recipient or an > agent responsible for delivering it to the intended recipient, you > are hereby notified that you have received this document in error > and that any review, dissemination, distribution, or copying of > this message is strictly prohibited. If you have received this > communication in error, please notify us immediately, and delete > the original message. >
Re: Tsm server possibly limiting backup performance
I think clarification is needed on mb/s vs. MB/s. a 1gb (gigabit) (note lowercase) connection will certainly do 800 mb/s (megabit/s). it won't, however, do 800 MB/s (megabyte/s) Paul On Fri, May 27, 2011 at 8:09 AM, Richard Rhodes wrote: > A little more explanation is going to be needed. A 1GB ethernet is not > going to give 800mb/s, let alone 200mb/s. 1GB ethernet is only going to > provide one direction throughput of 110mb/s. Do you mean a 10gb ethernet > connection? > > > > > > From: "Lee, Gary D." > To: ADSM-L@VM.MARIST.EDU > Date: 05/27/2011 09:35 AM > Subject:Tsm server possibly limiting backup performance > Sent by:"ADSM: Dist Stor Manager" > > > > Tsm server 6.2.2 > OS redhat enterprise linux 6.0 > Client tdp for exchange, sqlserver, and backup archive client > > Client connected via 1 gb ethernet connection. > > In my experience, I yhave never been able to se throughput of much more > than 200 mb/s during a tsm backup, whether tdp or regular client. > I wonder, is TSM limiting a single session's bandwidth so as to insure > availability for other client sessions which might begin; i.e. prevent one > backup stream from monopolizing the server's network connection? > > I have run ftp and other tests to the server, and achieved throughput of > around 800 mb/s over the same connection. > > > > Gary Lee > Senior System Programmer > Ball State University > phone: 765-285-1310 > > > > > > - > The information contained in this message is intended only for the > personal and confidential use of the recipient(s) named above. If > the reader of this message is not the intended recipient or an > agent responsible for delivering it to the intended recipient, you > are hereby notified that you have received this document in error > and that any review, dissemination, distribution, or copying of > this message is strictly prohibited. If you have received this > communication in error, please notify us immediately, and delete > the original message. >
Re: Tsm server possibly limiting backup performance
To make sure we're all talking the same language here, can we agree that 'b' means 'bit' and 'B' means 'byte'? This conversation could get very confusing otherwise... /DMc On 27 May 2011, at 15:09, Richard Rhodes wrote: > A little more explanation is going to be needed. A 1GB ethernet is not > going to give 800mb/s, let alone 200mb/s. 1GB ethernet is only going to > provide one direction throughput of 110mb/s. Do you mean a 10gb ethernet > connection? > > > > > > From: "Lee, Gary D." > To: ADSM-L@VM.MARIST.EDU > Date: 05/27/2011 09:35 AM > Subject:Tsm server possibly limiting backup performance > Sent by:"ADSM: Dist Stor Manager" > > > > Tsm server 6.2.2 > OS redhat enterprise linux 6.0 > Client tdp for exchange, sqlserver, and backup archive client > > Client connected via 1 gb ethernet connection. > > In my experience, I yhave never been able to se throughput of much more > than 200 mb/s during a tsm backup, whether tdp or regular client. > I wonder, is TSM limiting a single session's bandwidth so as to insure > availability for other client sessions which might begin; i.e. prevent one > backup stream from monopolizing the server's network connection? > > I have run ftp and other tests to the server, and achieved throughput of > around 800 mb/s over the same connection. > > > > Gary Lee > Senior System Programmer > Ball State University > phone: 765-285-1310 > > > > > > - > The information contained in this message is intended only for the > personal and confidential use of the recipient(s) named above. If > the reader of this message is not the intended recipient or an > agent responsible for delivering it to the intended recipient, you > are hereby notified that you have received this document in error > and that any review, dissemination, distribution, or copying of > this message is strictly prohibited. If you have received this > communication in error, please notify us immediately, and delete > the original message.
Re: Tsm server possibly limiting backup performance
A little more explanation is going to be needed. A 1GB ethernet is not going to give 800mb/s, let alone 200mb/s. 1GB ethernet is only going to provide one direction throughput of 110mb/s. Do you mean a 10gb ethernet connection? From: "Lee, Gary D." To: ADSM-L@VM.MARIST.EDU Date: 05/27/2011 09:35 AM Subject:Tsm server possibly limiting backup performance Sent by:"ADSM: Dist Stor Manager" Tsm server 6.2.2 OS redhat enterprise linux 6.0 Client tdp for exchange, sqlserver, and backup archive client Client connected via 1 gb ethernet connection. In my experience, I yhave never been able to se throughput of much more than 200 mb/s during a tsm backup, whether tdp or regular client. I wonder, is TSM limiting a single session's bandwidth so as to insure availability for other client sessions which might begin; i.e. prevent one backup stream from monopolizing the server's network connection? I have run ftp and other tests to the server, and achieved throughput of around 800 mb/s over the same connection. Gary Lee Senior System Programmer Ball State University phone: 765-285-1310 - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Tsm server possibly limiting backup performance
Tsm server 6.2.2 OS redhat enterprise linux 6.0 Client tdp for exchange, sqlserver, and backup archive client Client connected via 1 gb ethernet connection. In my experience, I yhave never been able to se throughput of much more than 200 mb/s during a tsm backup, whether tdp or regular client. I wonder, is TSM limiting a single session's bandwidth so as to insure availability for other client sessions which might begin; i.e. prevent one backup stream from monopolizing the server's network connection? I have run ftp and other tests to the server, and achieved throughput of around 800 mb/s over the same connection. Gary Lee Senior System Programmer Ball State University phone: 765-285-1310
how to take backup in CD/DVD using TSM server?
Hi Team, kindly assit me how to take backup in CD/DVD using TSM server? provide me the steps for the backup to be done thanks in advance Arul +-- |This was sent by arulkumar9...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +--
