Re: TSM database write performance
Yes- we have a mx limit of 256 AIO serversruns great -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of GWDVMS::MOELLER Sent: Tuesday, December 09, 2003 5:44 AM To: [EMAIL PROTECTED] Subject: Re: TSM database write performance (TSM server 5.1.6.5 on AIX 5.1) So, does anyone use AIXASYNCIO ? Apparently there is no success story yet to be found in the archives or on the net. So please, raise your hands! Wolfgang J. Moeller, Tel. +49 551 201-1516/-1510, [EMAIL PROTECTED] GWDG, D-37077 Goettingen, F.R.Germany |Disclaimer: No claim intended! http://www.gwdg.de/~moeller/ <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
Re: TSM database write performance
Hi Wolfgang, I'd recommend looking into the following parameters for AIX/TSM in your server options file: AIXASYNCIO YES AIXDIRECTIOYES MIRRORWRITE DB PARALLEL Good luck- John -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of GWDVMS::MOELLER Sent: Wednesday, December 03, 2003 3:02 PM To: [EMAIL PROTECTED] Subject: TSM database write performance (TSM server 5.1.6.5 on AIX 5.1 with essentially standard options) >From observing DSMSERV's I/O behaviour via "topas", I get the impression that it *writes* to the database volumes (currently 14 files each covering a whole JFS-mirrored SSA disk) in essentially single-threaded fashion, i.e. never exceeding the I/O rate that you would expect from a single disk. This is quite different from the *read* behaviour, where many disks (= DB volumes) are being used in parallel. Is there some tuning option that would allow for a higher write I/O rate, or is the sequential processing of writes fundamental to TSM's database operation? I didn't find anything pertinent in the "Quickfacts" ... What write I/O rates do you see? Wolfgang J. Moeller, Tel. +49 551 201-1516/-1510, [EMAIL PROTECTED] GWDG, D-37077 Goettingen, F.R.Germany |Disclaimer: No claim intended! http://www.gwdg.de/~moeller/ <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
Re: Reclamation Pr. Hanging
Recycling all 3 instances of TSM (incl. library manager inst.) fixed the problem. Tivoli Level 2 didn't ref. any APARs for this one, but recommended updating our Atape and atldd code on AIX-- -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Richard Sims Sent: Wednesday, November 26, 2003 9:48 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging >This one is a real buggerturns out the tape is physically in the library >slot and the 3494 library manager knows about it, but the TSM library manager >instance still thinks it's dismounting...and the process is hanging. The only >thing I can think of beyond is to recycle the library manager instance of TSM to >clear the mount state for the volume. No locks, no deadlocks, just an MIA >process? Interesting. This is why we systems people all want video cameras in the computer room. I'll bet an operator responded to the problem, manually extricated the tape from the drive, and either put it into cell 1 so that the robot would put it back into its home cell (most likely), or manually put the tape back into its cell. But because of the manual action, TSM has not received information about the tape dismounting. A situation like this often can't be fixed by an action less than recycling the TSM server. I would first try going to the 3494 panel and rendering the drive Unavailable for a while, to see if that state change gets back to TSM so that it clears things, then put it back to Available via the panel. Post watch on that questionable tape drive, which may need service. It's always something. Well, it's nice to be employed. Richard Sims, BU
Re: Reclamation Pr. Hanging
It certainly is...Happy Thanksgiving and thanks for your help!~ -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Richard Sims Sent: Wednesday, November 26, 2003 9:48 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging >This one is a real buggerturns out the tape is physically in the library >slot and the 3494 library manager knows about it, but the TSM library manager >instance still thinks it's dismounting...and the process is hanging. The only >thing I can think of beyond is to recycle the library manager instance of TSM to >clear the mount state for the volume. No locks, no deadlocks, just an MIA >process? Interesting. This is why we systems people all want video cameras in the computer room. I'll bet an operator responded to the problem, manually extricated the tape from the drive, and either put it into cell 1 so that the robot would put it back into its home cell (most likely), or manually put the tape back into its cell. But because of the manual action, TSM has not received information about the tape dismounting. A situation like this often can't be fixed by an action less than recycling the TSM server. I would first try going to the 3494 panel and rendering the drive Unavailable for a while, to see if that state change gets back to TSM so that it clears things, then put it back to Available via the panel. Post watch on that questionable tape drive, which may need service. It's always something. Well, it's nice to be employed. Richard Sims, BU
Re: Reclamation Pr. Hanging
This one is a real buggerturns out the tape is physically in the library slot and the 3494 library manager knows about it, but the TSM library manager instance still thinks it's dismounting...and the process is hanging. The only thing I can think of beyond is to recycle the library manager instance of TSM to clear the mount state for the volume. No locks, no deadlocks, just an MIA process? -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Levi, Ralph Sent: Tuesday, November 25, 2003 3:35 PM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging John, Here are some more ideas. Cancel the reclamation. Run: audit vol 600379 fix=yes Then do a move data: move data 600379 stgpool=(stg pool name) . Try start the reclamation after that. Hope it helps. Ralph -Original Message- From: Merryman, John A. [mailto:[EMAIL PROTECTED] Sent: November 25, 2003 11:26 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging Ralph, Thanks for the idea- that's a good one. checkout libvol cny3494 600379 remove=no 11/25/03 10:56:03 ANR0985I Process 1138 for CHECKOUT LIBVOLUME running in the BACKGROUND completed with completion state SUCCESS at 10:56:03. Ran audit library for library clients- checkin libv cny3494 600379 checkl=no status=private owner=aixtsm3 11/25/03 11:06:19 ANR0985I Process 1143 for CHECKIN LIBVOLUME running in the BACKGROUND completed with completion state SUCCESS at 11:06:19. On my library manager ANR8377I 3590 volume 600379 is mounted R/O, status: DISMOUNTING. Reclamation pr. is still hanging-- any other ideas? Many Thanks, John -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Levi, Ralph Sent: Tuesday, November 25, 2003 10:51 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging Try to do a checkout. I suspect it will not be found either, meaning your tsm DB is not in sync with your library. Manually take the tape out of the library and then check it in. You may also have to do an audit on it once you check it in. Ralph -Original Message----- From: Merryman, John A. [mailto:[EMAIL PROTECTED] Sent: November 25, 2003 10:42 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging Hi Folks- I've got a reclamation process hanging...waiting for the mount of a volume which is clearly accessible to the library/drives. Tried updating both vols in the process to acc=unavail and also cancel pr (pending now), along with updating the stgpoolreclaim=100 but no dice. Any ideas? Cheers, John
Re: Reclamation Pr. Hanging
Ralph, Thanks for the idea- that's a good one. checkout libvol cny3494 600379 remove=no 11/25/03 10:56:03 ANR0985I Process 1138 for CHECKOUT LIBVOLUME running in the BACKGROUND completed with completion state SUCCESS at 10:56:03. Ran audit library for library clients- checkin libv cny3494 600379 checkl=no status=private owner=aixtsm3 11/25/03 11:06:19 ANR0985I Process 1143 for CHECKIN LIBVOLUME running in the BACKGROUND completed with completion state SUCCESS at 11:06:19. On my library manager ANR8377I 3590 volume 600379 is mounted R/O, status: DISMOUNTING. Reclamation pr. is still hanging-- any other ideas? Many Thanks, John -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Levi, Ralph Sent: Tuesday, November 25, 2003 10:51 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging Try to do a checkout. I suspect it will not be found either, meaning your tsm DB is not in sync with your library. Manually take the tape out of the library and then check it in. You may also have to do an audit on it once you check it in. Ralph -Original Message- From: Merryman, John A. [mailto:[EMAIL PROTECTED] Sent: November 25, 2003 10:42 AM To: [EMAIL PROTECTED] Subject: Re: Reclamation Pr. Hanging Hi Folks- I've got a reclamation process hanging...waiting for the mount of a volume which is clearly accessible to the library/drives. Tried updating both vols in the process to acc=unavail and also cancel pr (pending now), along with updating the stgpoolreclaim=100 but no dice. Any ideas? Cheers, John
Re: Reclamation Pr. Hanging
Hi Folks- I've got a reclamation process hanging...waiting for the mount of a volume which is clearly accessible to the library/drives. Tried updating both vols in the process to acc=unavail and also cancel pr (pending now), along with updating the stgpoolreclaim=100 but no dice. Any ideas? Cheers, John
Re: Antwort: Re: TSM performance and network options
[EMAIL PROTECTED]/> lslpp -w /usr/samples/kernel/vmtune FileFileset Type /usr/samples/kernel/vmtune bos.adt.samples File [EMAIL PROTECTED]/> lslpp -l bos.adt.samples Fileset Level State Description Path: /usr/lib/objrepos bos.adt.samples 5.1.0.50 COMMITTED Base Operating System Samples The vmtune64 is the command parameter we use in /etc/inittab on startup; this command set may only be installed if you are running in 64bit mode AIX when applying the filesets- This is an excellent resource- http://www.redbooks.ibm.com/redpieces/pdfs/sg246039.pdf ref. ch 14 Good luck! -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Ben Bullock Sent: Wednesday, November 19, 2003 6:24 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: TSM performance and network options Just to stray a ~little~ off-topic I'm interested in the vmtune64 command you are using. I recently migrated a host to aix5.2 and am running it in 64-bit mode, but still only have the vmtune command, not vmtune64... root:># lslpp -w /usr/samples/kernel/vmtune FileFileset Type /usr/samples/kernel/vmtune bos.adt.samples File root:># lslpp -l bos.adt.samples Fileset Level State Description Path: /usr/lib/objrepos bos.adt.samples 5.2.0.10 COMMITTED Base Operating System Samples Which fileset is this "vmtune64" command included in, I can't seem to find a 64-bit version of this fileset. Thanks, Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Merryman, John A. Sent: Wednesday, November 19, 2003 12:43 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: TSM performance and network options Make sure file system caching is off for JFS filesystems- and consider running vmtune on aix. We have the following parameters that made a tremendous difference. I would suggest testing in a test environment first if you have the option: vmtune64 -p 10 -P 20 -s1 -W16 -c8 -R256 -F512 -u25 -b2200 -B2200 note- we are running 64bit mode AIX 5.1 ML4, hence vmtune64. Also, using Direct I/O on TSM and AIO servers on AIX can help with performance, but overall the underlying disk infrastrucutre is going to be just as important as the logical layout and tuning. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of French, Michael Sent: Wednesday, November 19, 2003 2:33 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: TSM performance and network options I can't really speak to Aix performance with RAW volumes, maybe someone else can. I use Solaris on all of my TSM servers and I can tell a huge difference between RAW and logical. In basic benchmark testing, it took 4-6 minutes to backup 1 500MB file from the local disk on the TSM server using logical, vxfs formatted volumes. Doing the same test with RAW volumes took about 40 secs, a huge difference. I was able to replicate these results many times over. You will also see a big performance boost in operations such as expiration and file space deletions. Michael French Savvis Communications IDS01 Santa Clara, CA (408)450-7812 -- desk (408)239-9913 -- mobile -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Frank Mueller Sent: Wednesday, November 19, 2003 11:29 AM To: [EMAIL PROTECTED] Subject: Antwort: Re: TSM performance and network options Hi *, I use on both server logical volumes and jfs-filesystems. All (DB, log and disk pool) are on jfs-filesystems. Is it better when I change to RAW devices? Best regards, Frank Mueller "French, Michael" <[EMAIL PROTECTED]An: [EMAIL PROTECTED] AVVIS.NET> Kopie: Gesendet von:Thema:Re: TSM performance and network options "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] .EDU> 19.11.2003 20:23 Bitte antworten an "ADSM: Dist Stor Manager" Are you using RAW volumes or logical, file system mounted volumes for DB, log, and disk pool? RAW volumes make a HUGE difference. You should also have your client's NIC'
Re: Antwort: Re: TSM performance and network options
Make sure file system caching is off for JFS filesystems- and consider running vmtune on aix. We have the following parameters that made a tremendous difference. I would suggest testing in a test environment first if you have the option: vmtune64 -p 10 -P 20 -s1 -W16 -c8 -R256 -F512 -u25 -b2200 -B2200 note- we are running 64bit mode AIX 5.1 ML4, hence vmtune64. Also, using Direct I/O on TSM and AIO servers on AIX can help with performance, but overall the underlying disk infrastrucutre is going to be just as important as the logical layout and tuning. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of French, Michael Sent: Wednesday, November 19, 2003 2:33 PM To: [EMAIL PROTECTED] Subject: Re: Antwort: Re: TSM performance and network options I can't really speak to Aix performance with RAW volumes, maybe someone else can. I use Solaris on all of my TSM servers and I can tell a huge difference between RAW and logical. In basic benchmark testing, it took 4-6 minutes to backup 1 500MB file from the local disk on the TSM server using logical, vxfs formatted volumes. Doing the same test with RAW volumes took about 40 secs, a huge difference. I was able to replicate these results many times over. You will also see a big performance boost in operations such as expiration and file space deletions. Michael French Savvis Communications IDS01 Santa Clara, CA (408)450-7812 -- desk (408)239-9913 -- mobile -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Frank Mueller Sent: Wednesday, November 19, 2003 11:29 AM To: [EMAIL PROTECTED] Subject: Antwort: Re: TSM performance and network options Hi *, I use on both server logical volumes and jfs-filesystems. All (DB, log and disk pool) are on jfs-filesystems. Is it better when I change to RAW devices? Best regards, Frank Mueller "French, Michael" <[EMAIL PROTECTED]An: [EMAIL PROTECTED] AVVIS.NET> Kopie: Gesendet von:Thema:Re: TSM performance and network options "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] .EDU> 19.11.2003 20:23 Bitte antworten an "ADSM: Dist Stor Manager" Are you using RAW volumes or logical, file system mounted volumes for DB, log, and disk pool? RAW volumes make a HUGE difference. You should also have your client's NIC's forced to 100/Full (the ones using fastethernet), don't let them auto negotiate, kill's backup performance. Michael French Savvis Communications IDS01 Santa Clara, CA (408)450-7812 -- desk (408)239-9913 -- mobile -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Frank Mueller Sent: Wednesday, November 19, 2003 11:19 AM To: [EMAIL PROTECTED] Subject: TSM performance and network options Hi *, our TSM environment is very slow (backup from the clients and the backup storage pool from the first to the second server). So I would like to describe our environment once roughly: We have 2 TSM server (Version 5.1.8) which runs both on AIX boxes with AIX 5.1. On the first server is an 3494 library (with 4 3590 drives) connected and on the second server an 3583 (with 3 LTO Ultrium1 drives). The clients (AIX-clients and Windows clients) are connected over fastethernet or gigaethernet. First TSM server has both adapter (fast and giga). The connection between the first and the second TSM server is gigaethernet. Can you give me some tips to increase the performance? I think that are some TSM and AIX options (TCPWindowsize etc on TSM side and son no-Paramter on AIX site). What is a good start for this options? Best regards, Frank Mueller
Re: Catalog Size and large number of files
We have a similar scenario For short term retention (5 years or less), we use dual node names Node_ARC and send their data to a backup management class that functions for an incremental archive. The downside is our database and dirmc size will continue to grow for 2 years. For long term retention (>5 years) traditional archiving is the way to go. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Pasquale, Joe Sent: Tuesday, November 18, 2003 11:08 AM To: [EMAIL PROTECTED] Subject: Re: Catalog Size and large number of files Full backups are required because of retention requirements set by government regulations, so the "incremental forever" philosophy is NOT a solution. -Original Message- From: David E Ehresman [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 18, 2003 11:04 AM To: [EMAIL PROTECTED] Subject: Re: Catalog Size and large number of files > If so, could you please forward some >possible solutions? > The obvious solution is to use TSM's "incremental forever" philosophy and not do unnecessary full backups. David * This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.
Backup Versioning / Retention Policy Survey
Hi TSMers, It's our understanding that industry standard TSM backup versioning policies (if there is such a thing) are the following: TSM Policy Setting Industry Standard Versions Data Exist 7 / 14** Versions Data Deleted 7/ 14** Retain Extra Version30 Retain Only Version 30 *7 versions of file system type files, and 14 versions of database type files. Please feedback if this is accurate- all comments and alternate examples are welcome. Many Thanks, John
