Re: inactivate TDP SQL transaction logs?
I have Del- and to me they were silent on the inactivation (it's quite different than Oracle/RMAN). The question on the management class parameters was because the TSM Admins determined that they didn't need to follow the Redbook's recommendations on unlimited/unlimited and I am unable to provide hard cold reasons why they should follow the Redbook. What they gave me: Created Policy Domain , Set up management class Standard. For now (until business side gets back to us with retention) set up the MC with 0 days between backups, 7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file f! or 60 days. Set up separate collocation group for X. My comments regarding: How does this MC affect backup of the SQL transaction logs (since it is backup and not archive)? I am concerned that the 7 versions existing 1 deleted may cause issues doing a point-in-time restore (i.e.- the log that needs to be restored rolled off TSM now the point in time recovery fails). Regarding metadata being retained on disk vs. on tape. We may not be able to query database objects for restore if the metadata is on tape. This is another concern I have regarding only one STANDARD management class. What I asked for: I have relied heavily on the IBM Redbook Backing up Microsoft SQL Server with IBM Tivoli Storage Manager (SG-24-6148-01) and its recommended setup 1. I suggest you collocate by node or filesystem, depending on size of filesystems maxsize of the tapes--but if you can meet the business-side's RTO ! without collocation, then whatever works. 2. Separate storage pools (stgp) be set up to backup the SQL server environment. (Redbook pg 81) 2.a Dedicated disk storage pools for the primary stgps for the SQL DB servers the TDPs DRSQLDBDISK- primary pool for DB Data DRSQLLOGDISK- primary pool for DB logs DRSQLDISKCOPY-copypool for DRSQLLOGDISK DRSQLMETADISK) DRSQLTAPECOPY- copy pool for DRSQLDBDISK in tape library REUSEDELAY=2 DR_DRIMVDL- DR offsite tapes REUSEDELAY=2 DRSQLMETADISK-this is for metadata when using VSS. However, a system utilizing transaction log backup without using VSS staging was not illustrated in the Redbook, so it's unclear we'd need this for the legacy backup solution. The documentation discusses ! when working with VSS backups another separate disk stgp be set up for the metadata. The problem with this is that in the Redbook team's test environment, they assume all clustered servers would be utilizing VSS-either staging, offloaded or hardware. I think that this is something we should look into-given the projected data volume and the RTOs-perhaps even the backup window. We could do the staging or off-loaded now-we just need to install the IBM TSM for Copy Services. And some more configuration. 3. Separate Policy Domain (PD) (Redbook ppg 83 84) 3.a Management Class (MC) *DRSQLDAILY- for the daily weekly backups with retention policy based on date- VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 **Default MC *DRSQL_LONG- monthly backups archives (shouldn't need to include meta logs if the archive/(full) backup is done correctly-right? I would also suggest that the scheduled job! s use dates in the name of the backup for the archives, so we don't run into issues with version retention for clarity) VEREXISTS=2 VERDELETED=1 RETEXTRA=400 RETONLY=400 *DRSQL_LOGS- for the transaction logs VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 *DRSQL_VSSLOCAL- for local (on the SQL server or the proxy) VSS backups- retention policy enforced by version not time- VEREXISTS=3 VERDELETED=3 RETEXTA=nolimit RETONLY=nolimit *DRSQL_VSSFULLTSM -daily VSS backups sent to TSM-retention policy based on time= VEREXISTS=nolimit RETEXTRA=35 RETONLY=35 I am concerned about PIT recovery since I don't know what their tweaking will do to our ability to recover. The business side is willing to pay for what we need, but the service provider is unwilling to provide what we ask for.. If I could bother with one more stupid question... I am vague on how/if the transaction log space empties-- For example, I have a difffull running every night, then backup the logs and every 2 hours a backup of the logs occurs-- If I get a notification (SNMP) that the drive designated for the SQL logs is 95% full -- Do I have to worry about it? thanks! lisa -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Del Hoobler Sent: Thursday, September 04, 2008 6:50 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] inactivate TDP SQL transaction logs? Lisa, For Data Protection for SQL, you do not need to schedule an inactivate logs. log backups are stored as backups, and are inactivated automatically on the next full backup of the associated database. I encourage you to read the Backup overview, Backup strategies, and Recommended Tivoli Storage Manager
Re: inactivate TDP SQL transaction logs?
First, Unlimited is excessive for everything. However, we have Unlimited set for # of versions, but we only keep the backups for 30 days for our regular node, 5 years for our monthly node, and 10 years for our quarterly nodes when it comes to SQL server. The key settings for SQL Server are unlimited versions then limit the # of Days, this way you can run regular log backups and actually recover. See Ya' Howard -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Laughlin, Lisa Sent: Friday, September 05, 2008 9:38 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] inactivate TDP SQL transaction logs? I have Del- and to me they were silent on the inactivation (it's quite different than Oracle/RMAN). The question on the management class parameters was because the TSM Admins determined that they didn't need to follow the Redbook's recommendations on unlimited/unlimited and I am unable to provide hard cold reasons why they should follow the Redbook. What they gave me: Created Policy Domain , Set up management class Standard. For now (until business side gets back to us with retention) set up the MC with 0 days between backups, 7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file f! or 60 days. Set up separate collocation group for X. My comments regarding: How does this MC affect backup of the SQL transaction logs (since it is backup and not archive)? I am concerned that the 7 versions existing 1 deleted may cause issues doing a point-in-time restore (i.e.- the log that needs to be restored rolled off TSM now the point in time recovery fails). Regarding metadata being retained on disk vs. on tape. We may not be able to query database objects for restore if the metadata is on tape. This is another concern I have regarding only one STANDARD management class. What I asked for: I have relied heavily on the IBM Redbook Backing up Microsoft SQL Server with IBM Tivoli Storage Manager (SG-24-6148-01) and its recommended setup 1. I suggest you collocate by node or filesystem, depending on size of filesystems maxsize of the tapes--but if you can meet the business-side's RTO ! without collocation, then whatever works. 2. Separate storage pools (stgp) be set up to backup the SQL server environment. (Redbook pg 81) 2.a Dedicated disk storage pools for the primary stgps for the SQL DB servers the TDPs DRSQLDBDISK- primary pool for DB Data DRSQLLOGDISK- primary pool for DB logs DRSQLDISKCOPY-copypool for DRSQLLOGDISK DRSQLMETADISK) DRSQLTAPECOPY- copy pool for DRSQLDBDISK in tape library REUSEDELAY=2 DR_DRIMVDL- DR offsite tapes REUSEDELAY=2 DRSQLMETADISK-this is for metadata when using VSS. However, a system utilizing transaction log backup without using VSS staging was not illustrated in the Redbook, so it's unclear we'd need this for the legacy backup solution. The documentation discusses ! when working with VSS backups another separate disk stgp be set up for the metadata. The problem with this is that in the Redbook team's test environment, they assume all clustered servers would be utilizing VSS-either staging, offloaded or hardware. I think that this is something we should look into-given the projected data volume and the RTOs-perhaps even the backup window. We could do the staging or off-loaded now-we just need to install the IBM TSM for Copy Services. And some more configuration. 3. Separate Policy Domain (PD) (Redbook ppg 83 84) 3.a Management Class (MC) *DRSQLDAILY- for the daily weekly backups with retention policy based on date- VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 **Default MC *DRSQL_LONG- monthly backups archives (shouldn't need to include meta logs if the archive/(full) backup is done correctly-right? I would also suggest that the scheduled job! s use dates in the name of the backup for the archives, so we don't run into issues with version retention for clarity) VEREXISTS=2 VERDELETED=1 RETEXTRA=400 RETONLY=400 *DRSQL_LOGS- for the transaction logs VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 *DRSQL_VSSLOCAL- for local (on the SQL server or the proxy) VSS backups- retention policy enforced by version not time- VEREXISTS=3 VERDELETED=3 RETEXTA=nolimit RETONLY=nolimit *DRSQL_VSSFULLTSM -daily VSS backups sent to TSM-retention policy based on time= VEREXISTS=nolimit RETEXTRA=35 RETONLY=35 I am concerned about PIT recovery since I don't know what their tweaking will do to our ability to recover. The business side is willing to pay for what we need, but the service provider is unwilling to provide what we ask for.. If I could bother with one more stupid question... I am vague on how/if the transaction log space empties-- For example, I have a difffull running every
TSM library master for two tape libraries causes problems
Greetings, We have a TSM instance that serves as the library master for two tape libraries. One is a IBM3584 tape library with 24 LTO4 drives. The second is a virtual tape library, an EMC EDL configured as a IBM3584 with 128 LTO1 tape drives. The library master has 9 other TSM instances and 4 Lan-free clients that are library clients, and appeal to it for tape mounts. Most of the time this works just fine. All the TSM instances are running TSM 5.4.3.0. A few weeks ago we upgraded them from 5.4.2.0 in order to pick up a patch to provide the LIBSHRTIMEOUT parameter. This may not have anything to do with our problem, but it IS a recent change. The problem comes when we have any problem with the real IBM3584 tape library. Two weeks ago a tape got stuck in the gripper. Last week the gripper itself actually broke, so no tapes could get mounted. Last night it was a single tape drive that got an error and went Polling. For some reason, in each one of these cases, after an hour or two all tape mounts start hanging, even those belonging to the virtual tape library. When we would do a 'q mount' they all showed up in Reserved status. So before long all backups going to the virtual tape library ground to a halt. Can any of you see a reason why the TSM library master should get into such a problem? Shouldn't all tape mounts be asynchronous? It seems like to me a single tape drive getting into problems should not keep all other mounts from proceeding. And it doesn't happen instantly. It seems to happen gradually. I probably should also mention that this is a fairly busy environment during the night. It isn't unusual for us to have over 100 virtual tape mounts simultaneously. That is the reason we needed the LIBSHRTIMEOUT parameter (mentioned above). Before we had that, we sometimes would get timeouts that caused tape mount failures, because the TSM library master's queue of tape mounts polls would get overrun. Since we put on that patch, and added 'LIBSHRTIMEOUT 60' to the options file, that problem has gone away. But now this problem seems to have taken it's place. Best Regards, John D. Schneider Lead Systems Administrator - Storage Sisters of Mercy Health Systems 3637 South Geyer Road St. Louis, MO 63127 Phone: 314-364-3150 Cell: 314-750-8721 Email: [EMAIL PROTECTED] This e-mail contains information which (a) may be PROPRIETARY IN NATURE OR OTHERWISE PROTECTED BY LAW FROM DISCLOSURE, and (b) is intended only for the use of the addressee(s) named above. If you are not the addressee, or the person responsible for delivering this to the addressee(s), you are notified that reading, copying or distributing this e-mail is prohibited. If you have received this e-mail in error, please contact the sender immediately.
Re: inactivate TDP SQL transaction logs?
7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file for 60 Days Is substantially different from VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 And I am worried about recovery. If I could bother with one more stupid question... I am vague on how/if the transaction log space empties-- For example, I have a difffull running every night, then backup the logs and every 2 hours a backup of the logs occurs-- If I get a notification (SNMP) that the drive designated for the SQL logs is 95% full -- Do I have to worry about it? thanks! lisa -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Howard Coles Sent: Friday, September 05, 2008 10:20 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] inactivate TDP SQL transaction logs? First, Unlimited is excessive for everything. However, we have Unlimited set for # of versions, but we only keep the backups for 30 days for our regular node, 5 years for our monthly node, and 10 years for our quarterly nodes when it comes to SQL server. The key settings for SQL Server are unlimited versions then limit the # of Days, this way you can run regular log backups and actually recover. See Ya' Howard -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Laughlin, Lisa Sent: Friday, September 05, 2008 9:38 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] inactivate TDP SQL transaction logs? I have Del- and to me they were silent on the inactivation (it's quite different than Oracle/RMAN). The question on the management class parameters was because the TSM Admins determined that they didn't need to follow the Redbook's recommendations on unlimited/unlimited and I am unable to provide hard cold reasons why they should follow the Redbook. What they gave me: Created Policy Domain , Set up management class Standard. For now (until business side gets back to us with retention) set up the MC with 0 days between backups, 7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file f! or 60 days. Set up separate collocation group for X. My comments regarding: How does this MC affect backup of the SQL transaction logs (since it is backup and not archive)? I am concerned that the 7 versions existing 1 deleted may cause issues doing a point-in-time restore (i.e.- the log that needs to be restored rolled off TSM now the point in time recovery fails). Regarding metadata being retained on disk vs. on tape. We may not be able to query database objects for restore if the metadata is on tape. This is another concern I have regarding only one STANDARD management class. What I asked for: I have relied heavily on the IBM Redbook Backing up Microsoft SQL Server with IBM Tivoli Storage Manager (SG-24-6148-01) and its recommended setup 1. I suggest you collocate by node or filesystem, depending on size of filesystems maxsize of the tapes--but if you can meet the business-side's RTO ! without collocation, then whatever works. 2. Separate storage pools (stgp) be set up to backup the SQL server environment. (Redbook pg 81) 2.a Dedicated disk storage pools for the primary stgps for the SQL DB servers the TDPs DRSQLDBDISK- primary pool for DB Data DRSQLLOGDISK- primary pool for DB logs DRSQLDISKCOPY-copypool for DRSQLLOGDISK DRSQLMETADISK) DRSQLTAPECOPY- copy pool for DRSQLDBDISK in tape library REUSEDELAY=2 DR_DRIMVDL- DR offsite tapes REUSEDELAY=2 DRSQLMETADISK-this is for metadata when using VSS. However, a system utilizing transaction log backup without using VSS staging was not illustrated in the Redbook, so it's unclear we'd need this for the legacy backup solution. The documentation discusses ! when working with VSS backups another separate disk stgp be set up for the metadata. The problem with this is that in the Redbook team's test environment, they assume all clustered servers would be utilizing VSS-either staging, offloaded or hardware. I think that this is something we should look into-given the projected data volume and the RTOs-perhaps even the backup window. We could do the staging or off-loaded now-we just need to install the IBM TSM for Copy Services. And some more configuration. 3. Separate Policy Domain (PD) (Redbook ppg 83 84) 3.a Management Class (MC) *DRSQLDAILY- for the daily weekly backups with retention policy based on date- VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 **Default MC *DRSQL_LONG- monthly backups archives (shouldn't need to include meta logs if the archive/(full) backup is done correctly-right? I would also suggest that the scheduled job! s use dates in the name of the backup for the archives, so we don't run into issues with version retention for clarity) VEREXISTS=2 VERDELETED=1
Re: inactivate TDP SQL transaction logs?
Hi Lisa, SQL Server log truncation is different than Oracle and even Exchange Server. Just because you truncate the logs, doesn't necessarily mean that you recapture the physical disk space. Please read this article, it explains it fully. It is a good read: http://msdn.microsoft.com/en-us/library/aa174524(SQL.80).aspx You will see this statement: Shrinking a log is dependent on first truncating the log. Data Protection for SQL will truncate the logs (unless you tell it not to). If you are concerned about shrinking, the article above should help you. Thanks, Del ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU wrote on 09/05/2008 12:15:01 PM: [image removed] Re: inactivate TDP SQL transaction logs? Laughlin, Lisa to: ADSM-L 09/05/2008 12:16 PM Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Please respond to ADSM: Dist Stor Manager 7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file for 60 Days Is substantially different from VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 And I am worried about recovery. If I could bother with one more stupid question... I am vague on how/if the transaction log space empties-- For example, I have a difffull running every night, then backup the logs and every 2 hours a backup of the logs occurs-- If I get a notification (SNMP) that the drive designated for the SQL logs is 95% full -- Do I have to worry about it? thanks! lisa
TSM library problem caused by IBM3584 virtual I/O
Greetings, We are running TSM 5.4.3.0 on AIX 5.3ML5. We have a library master instance, and 9 other TSM instances that are library clients. They all share an IBM3584 with 24 LTO4 tape drives, and an EMC EDL virtual library emulating an IBM3584 with 128 LTO1 drives. Recently our IBM CE told us we should be running with virtual I/O, a feature of the IBM3584 library. The reason he recommended it is because we frequently have more than 32 outgoing tapes every day, and sometimes the Operators don't get around to taking the tapes out of the I/O doors, and checkouts have to wait. With virtual I/O turned on, the checkouts go ahead and run to completion, even though the tapes don't actually go into the I/O doors. Then later when the I/O doors get empty, the tape library moves the rest of the tapes into the I/O doors. That part seems to be working as expected. After we turned virtual I/O on, we started getting weird symptoms in TSM, like tapes that we would check back in to the library, but later TSM could not find them. So we decided that maybe virtual I/O changed the element number map, and we should have redefined the library to TSM. So we: 1) Deleted the drive paths, drives, library path, and library on the library master instance, and all client instances. 2) From the Tape library Web interface, performed a complete library inventory (just in case) 3) Defined the library, library path, drives, and drive paths on the library master instance, and all client instances. 4) Checked back in the scratch tapes 5) Checked back in the private tapes 6) Did an Audit library on the library master and all library clients. It was only a few days later that we started getting errors from TSM of the form: 09/04/08 22:00:56 ANR8300E I/O error on library SUN2079 (OP=6C03, CC=314, KEY=05, ASC=3B, ASCQ=0E, SENSE=70.00.05.00.00.00- .00.0A.00.00.00.00.3B.0E.00.C0.00.04., Description=The source slot or drive was empty in an attempt to move a volume). Refer to Appendix C in the 'Messages' manual for recommended action. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8312E Volume 101781L4 could not be located in library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8358E Audit operation is required for library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8381E NAS volume 101781L4 could not be mounted in drive LTO4_F2_D09 (c576t0l0). (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1402W Mount request denied for volume 101781L4 - volume unavailable. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1410W Access mode for volume 101781L4 now set to unavailable. (SESSION: 395703, PROCESS: 487) It is different tapes every time, so we now have over a dozen tapes that are missing on account of this. Did we do something wrong with we turned on virtual I/O for this library? I found this technote, that sounds like it is supported. It also says we need to restart the TSM server instance, which we have done now a couple of times since this problem started. With SAN Device Mapping implemented (since TSM520 for Windows and TSM530 for most other platforms), when hardware changes are done to the library, the Tivoli Storage Manager server needs to be restarted. During server initialization, Tivoli Storage Manager server will access the library. If the library inventory has changed, it will be refreshed at that time. If drives are added or deleted from the library or drive element addresses are changed, this information will be refreshed at server initialization also. If the library path has changed, then the path needs to be updated with the new device name for the new library path. The same holds true when dealing with a 3584 library with the ALMS feature. This is discussed in the following document : http://www-1.ibm.com/support/docview.wss?uid=swg21053638 That document will mention that TSM supports the Advanced Library Management System (ALMS) and Virtual I/O Slots (VIOS) features of IBM 3584. When changing the number of drive, storage, or import/export elements for a logical library, the TSM server must be restarted. Is there something else I needed to do in TSM? Some option in the options file, or setting in the library? Best Regards, John D. Schneider Lead Systems Administrator - Storage Sisters of Mercy Health Systems 3637 South Geyer Road St. Louis, MO 63127 Phone: 314-364-3150 Cell: 314-750-8721 Email: [EMAIL PROTECTED] This e-mail contains information which (a) may be PROPRIETARY IN NATURE OR OTHERWISE PROTECTED BY LAW FROM DISCLOSURE, and (b) is intended only for the use of the addressee(s) named above. If you are not the addressee, or the person
Re: inactivate TDP SQL transaction logs?
Thank you Del-- I'll check it out! thanks! lisa -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Del Hoobler Sent: Friday, September 05, 2008 11:48 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] inactivate TDP SQL transaction logs? Hi Lisa, SQL Server log truncation is different than Oracle and even Exchange Server. Just because you truncate the logs, doesn't necessarily mean that you recapture the physical disk space. Please read this article, it explains it fully. It is a good read: http://msdn.microsoft.com/en-us/library/aa174524(SQL.80).aspx You will see this statement: Shrinking a log is dependent on first truncating the log. Data Protection for SQL will truncate the logs (unless you tell it not to). If you are concerned about shrinking, the article above should help you. Thanks, Del ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU wrote on 09/05/2008 12:15:01 PM: [image removed] Re: inactivate TDP SQL transaction logs? Laughlin, Lisa to: ADSM-L 09/05/2008 12:16 PM Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Please respond to ADSM: Dist Stor Manager 7 versions of each file kept, 30 days to keep inactive versions, 1 deleted version, keep last file for 60 Days Is substantially different from VEREXISTS=nolimit VERDELETED=nolimit RETEXTRA=35 RETONLY=90 And I am worried about recovery. If I could bother with one more stupid question... I am vague on how/if the transaction log space empties-- For example, I have a difffull running every night, then backup the logs and every 2 hours a backup of the logs occurs-- If I get a notification (SNMP) that the drive designated for the SQL logs is 95% full -- Do I have to worry about it? thanks! lisa
Re: TSM library problem caused by IBM3584 virtual I/O
John, It almost sounds like something is moving the tapes without telling TSM. Is your VTL attached to the library? Cheers, Neil Strand Storage Engineer - Legg Mason Baltimore, MD. (410) 580-7491 Whatever you can do or believe you can, begin it. Boldness has genius, power and magic. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Schneider, John Sent: Friday, September 05, 2008 12:51 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] TSM library problem caused by IBM3584 virtual I/O Greetings, We are running TSM 5.4.3.0 on AIX 5.3ML5. We have a library master instance, and 9 other TSM instances that are library clients. They all share an IBM3584 with 24 LTO4 tape drives, and an EMC EDL virtual library emulating an IBM3584 with 128 LTO1 drives. Recently our IBM CE told us we should be running with virtual I/O, a feature of the IBM3584 library. The reason he recommended it is because we frequently have more than 32 outgoing tapes every day, and sometimes the Operators don't get around to taking the tapes out of the I/O doors, and checkouts have to wait. With virtual I/O turned on, the checkouts go ahead and run to completion, even though the tapes don't actually go into the I/O doors. Then later when the I/O doors get empty, the tape library moves the rest of the tapes into the I/O doors. That part seems to be working as expected. After we turned virtual I/O on, we started getting weird symptoms in TSM, like tapes that we would check back in to the library, but later TSM could not find them. So we decided that maybe virtual I/O changed the element number map, and we should have redefined the library to TSM. So we: 1) Deleted the drive paths, drives, library path, and library on the library master instance, and all client instances. 2) From the Tape library Web interface, performed a complete library inventory (just in case) 3) Defined the library, library path, drives, and drive paths on the library master instance, and all client instances. 4) Checked back in the scratch tapes 5) Checked back in the private tapes 6) Did an Audit library on the library master and all library clients. It was only a few days later that we started getting errors from TSM of the form: 09/04/08 22:00:56 ANR8300E I/O error on library SUN2079 (OP=6C03, CC=314, KEY=05, ASC=3B, ASCQ=0E, SENSE=70.00.05.00.00.00- .00.0A.00.00.00.00.3B.0E.00.C0.00.04., Description=The source slot or drive was empty in an attempt to move a volume). Refer to Appendix C in the 'Messages' manual for recommended action. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8312E Volume 101781L4 could not be located in library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8358E Audit operation is required for library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8381E NAS volume 101781L4 could not be mounted in drive LTO4_F2_D09 (c576t0l0). (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1402W Mount request denied for volume 101781L4 - volume unavailable. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1410W Access mode for volume 101781L4 now set to unavailable. (SESSION: 395703, PROCESS: 487) It is different tapes every time, so we now have over a dozen tapes that are missing on account of this. Did we do something wrong with we turned on virtual I/O for this library? I found this technote, that sounds like it is supported. It also says we need to restart the TSM server instance, which we have done now a couple of times since this problem started. With SAN Device Mapping implemented (since TSM520 for Windows and TSM530 for most other platforms), when hardware changes are done to the library, the Tivoli Storage Manager server needs to be restarted. During server initialization, Tivoli Storage Manager server will access the library. If the library inventory has changed, it will be refreshed at that time. If drives are added or deleted from the library or drive element addresses are changed, this information will be refreshed at server initialization also. If the library path has changed, then the path needs to be updated with the new device name for the new library path. The same holds true when dealing with a 3584 library with the ALMS feature. This is discussed in the following document : http://www-1.ibm.com/support/docview.wss?uid=swg21053638 That document will mention that TSM supports the Advanced Library Management System (ALMS) and Virtual I/O Slots (VIOS) features of IBM 3584. When changing the number of drive, storage, or import/export elements for a logical library, the TSM server must be restarted. Is there something else I needed to do in TSM? Some option in the
Re: TSM library problem caused by IBM3584 virtual I/O
Neil, The VTL does not connect to the real library. It does not mount tapes itself. All tape mounting comes from the TSM library master. We do not use the feature that allows a VTL to mount tapes and copy virtual tape data to a real tape. It is a good thought, Neil, but I don't see anything in our configuration besides the TSM library master that can control the tape library. Best Regards, John D. Schneider Phone: 314-364-3150 Cell: 314-750-8721 Email: [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Strand, Neil B. Sent: Friday, September 05, 2008 1:20 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM library problem caused by IBM3584 virtual I/O John, It almost sounds like something is moving the tapes without telling TSM. Is your VTL attached to the library? Cheers, Neil Strand Storage Engineer - Legg Mason Baltimore, MD. (410) 580-7491 Whatever you can do or believe you can, begin it. Boldness has genius, power and magic. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Schneider, John Sent: Friday, September 05, 2008 12:51 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] TSM library problem caused by IBM3584 virtual I/O Greetings, We are running TSM 5.4.3.0 on AIX 5.3ML5. We have a library master instance, and 9 other TSM instances that are library clients. They all share an IBM3584 with 24 LTO4 tape drives, and an EMC EDL virtual library emulating an IBM3584 with 128 LTO1 drives. Recently our IBM CE told us we should be running with virtual I/O, a feature of the IBM3584 library. The reason he recommended it is because we frequently have more than 32 outgoing tapes every day, and sometimes the Operators don't get around to taking the tapes out of the I/O doors, and checkouts have to wait. With virtual I/O turned on, the checkouts go ahead and run to completion, even though the tapes don't actually go into the I/O doors. Then later when the I/O doors get empty, the tape library moves the rest of the tapes into the I/O doors. That part seems to be working as expected. After we turned virtual I/O on, we started getting weird symptoms in TSM, like tapes that we would check back in to the library, but later TSM could not find them. So we decided that maybe virtual I/O changed the element number map, and we should have redefined the library to TSM. So we: 1) Deleted the drive paths, drives, library path, and library on the library master instance, and all client instances. 2) From the Tape library Web interface, performed a complete library inventory (just in case) 3) Defined the library, library path, drives, and drive paths on the library master instance, and all client instances. 4) Checked back in the scratch tapes 5) Checked back in the private tapes 6) Did an Audit library on the library master and all library clients. It was only a few days later that we started getting errors from TSM of the form: 09/04/08 22:00:56 ANR8300E I/O error on library SUN2079 (OP=6C03, CC=314, KEY=05, ASC=3B, ASCQ=0E, SENSE=70.00.05.00.00.00- .00.0A.00.00.00.00.3B.0E.00.C0.00.04., Description=The source slot or drive was empty in an attempt to move a volume). Refer to Appendix C in the 'Messages' manual for recommended action. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8312E Volume 101781L4 could not be located in library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8358E Audit operation is required for library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8381E NAS volume 101781L4 could not be mounted in drive LTO4_F2_D09 (c576t0l0). (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1402W Mount request denied for volume 101781L4 - volume unavailable. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1410W Access mode for volume 101781L4 now set to unavailable. (SESSION: 395703, PROCESS: 487) It is different tapes every time, so we now have over a dozen tapes that are missing on account of this. Did we do something wrong with we turned on virtual I/O for this library? I found this technote, that sounds like it is supported. It also says we need to restart the TSM server instance, which we have done now a couple of times since this problem started. With SAN Device Mapping implemented (since TSM520 for Windows and TSM530 for most other platforms), when hardware changes are done to the library, the Tivoli Storage Manager server needs to be restarted. During server initialization, Tivoli Storage Manager server will access the library. If the library inventory has changed, it will be refreshed at that time. If drives are added or deleted from
Re: TSM library problem caused by IBM3584 virtual I/O
Hi John, We are seeing similar issues, albeit on a much smaller library. Two of our TSM Servers run with 3584 libraries on version 5.4.2.0. The nine LTO3 library does not have virtual I/O enabled and we see no problems, but the ten LTO3 library with virtual I/O enabled does get the ANR8300E errors that you documented. The effect we see are tapes going into unavailable status. To correct the problem, we need to idle tape mounts, audit the library and update the tape status to read/write. But it would be better to find the root cause of the problem and correct it. Robert R. Price ADSM/TSM Administrator Computer Sciences Corporation Phone: 412-374-3247 Fax: 412-374-6371 [EMAIL PROTECTED] This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind CSC to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. Schneider, John [EMAIL PROTECTED] ERCY.NET To Sent by: ADSM: ADSM-L@VM.MARIST.EDU Dist Stor cc Manager [EMAIL PROTECTED] Subject .EDU TSM library problem caused by IBM3584 virtual I/O 09/05/2008 12:51 PM Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] .EDU Greetings, We are running TSM 5.4.3.0 on AIX 5.3ML5. We have a library master instance, and 9 other TSM instances that are library clients. They all share an IBM3584 with 24 LTO4 tape drives, and an EMC EDL virtual library emulating an IBM3584 with 128 LTO1 drives. Recently our IBM CE told us we should be running with virtual I/O, a feature of the IBM3584 library. The reason he recommended it is because we frequently have more than 32 outgoing tapes every day, and sometimes the Operators don't get around to taking the tapes out of the I/O doors, and checkouts have to wait. With virtual I/O turned on, the checkouts go ahead and run to completion, even though the tapes don't actually go into the I/O doors. Then later when the I/O doors get empty, the tape library moves the rest of the tapes into the I/O doors. That part seems to be working as expected. After we turned virtual I/O on, we started getting weird symptoms in TSM, like tapes that we would check back in to the library, but later TSM could not find them. So we decided that maybe virtual I/O changed the element number map, and we should have redefined the library to TSM. So we: 1) Deleted the drive paths, drives, library path, and library on the library master instance, and all client instances. 2) From the Tape library Web interface, performed a complete library inventory (just in case) 3) Defined the library, library path, drives, and drive paths on the library master instance, and all client instances. 4) Checked back in the scratch tapes 5) Checked back in the private tapes 6) Did an Audit library on the library master and all library clients. It was only a few days later that we started getting errors from TSM of the form: 09/04/08 22:00:56 ANR8300E I/O error on library SUN2079 (OP=6C03, CC=314, KEY=05, ASC=3B, ASCQ=0E, SENSE=70.00.05.00.00.00- .00.0A.00.00.00.00.3B.0E.00.C0.00.04., Description=The source slot or drive was empty in an attempt to move a volume). Refer to Appendix C in the 'Messages' manual for recommended action. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8312E Volume 101781L4 could not be located in library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8358E Audit operation is required for library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8381E NAS volume 101781L4 could not be mounted in drive LTO4_F2_D09 (c576t0l0). (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1402W Mount request denied for volume 101781L4 - volume unavailable. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1410W Access mode for volume 101781L4 now set to unavailable. (SESSION: 395703, PROCESS: 487) It is different tapes every time, so we now have over a dozen tapes that are missing on account of this. Did we do something wrong with we turned on virtual I/O for this library? I found this technote, that sounds like it is supported. It also says we need to restart the TSM
Re: TSM library problem caused by IBM3584 virtual I/O
First (and it may be redundant), make sure the library, device drivers, and TSM server version are up to date enough to handle the feature being turned on. Nothing brings out bugs like turning on a new feature. IIRC, V-I/O implies ALMS. When V-I/O is turned on, the operator may be asked which library to put the tapes in. I think there is a default setting for which library to put the tapes in. If this dialog is not responded to correctly, the tapes may be going to an incorrect library partition. Its been a long week. [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Schneider, John Sent: Friday, September 05, 2008 9:51 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] TSM library problem caused by IBM3584 virtual I/O Greetings, We are running TSM 5.4.3.0 on AIX 5.3ML5. We have a library master instance, and 9 other TSM instances that are library clients. They all share an IBM3584 with 24 LTO4 tape drives, and an EMC EDL virtual library emulating an IBM3584 with 128 LTO1 drives. Recently our IBM CE told us we should be running with virtual I/O, a feature of the IBM3584 library. The reason he recommended it is because we frequently have more than 32 outgoing tapes every day, and sometimes the Operators don't get around to taking the tapes out of the I/O doors, and checkouts have to wait. With virtual I/O turned on, the checkouts go ahead and run to completion, even though the tapes don't actually go into the I/O doors. Then later when the I/O doors get empty, the tape library moves the rest of the tapes into the I/O doors. That part seems to be working as expected. After we turned virtual I/O on, we started getting weird symptoms in TSM, like tapes that we would check back in to the library, but later TSM could not find them. So we decided that maybe virtual I/O changed the element number map, and we should have redefined the library to TSM. So we: 1) Deleted the drive paths, drives, library path, and library on the library master instance, and all client instances. 2) From the Tape library Web interface, performed a complete library inventory (just in case) 3) Defined the library, library path, drives, and drive paths on the library master instance, and all client instances. 4) Checked back in the scratch tapes 5) Checked back in the private tapes 6) Did an Audit library on the library master and all library clients. It was only a few days later that we started getting errors from TSM of the form: 09/04/08 22:00:56 ANR8300E I/O error on library SUN2079 (OP=6C03, CC=314, KEY=05, ASC=3B, ASCQ=0E, SENSE=70.00.05.00.00.00- .00.0A.00.00.00.00.3B.0E.00.C0.00.04., Description=The source slot or drive was empty in an attempt to move a volume). Refer to Appendix C in the 'Messages' manual for recommended action. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8312E Volume 101781L4 could not be located in library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8358E Audit operation is required for library SUN2079. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR8381E NAS volume 101781L4 could not be mounted in drive LTO4_F2_D09 (c576t0l0). (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1402W Mount request denied for volume 101781L4 - volume unavailable. (SESSION: 395703, PROCESS: 487) 09/04/08 22:00:56 ANR1410W Access mode for volume 101781L4 now set to unavailable. (SESSION: 395703, PROCESS: 487) It is different tapes every time, so we now have over a dozen tapes that are missing on account of this. Did we do something wrong with we turned on virtual I/O for this library? I found this technote, that sounds like it is supported. It also says we need to restart the TSM server instance, which we have done now a couple of times since this problem started. With SAN Device Mapping implemented (since TSM520 for Windows and TSM530 for most other platforms), when hardware changes are done to the library, the Tivoli Storage Manager server needs to be restarted. During server initialization, Tivoli Storage Manager server will access the library. If the library inventory has changed, it will be refreshed at that time. If drives are added or deleted from the library or drive element addresses are changed, this information will be refreshed at server initialization also. If the library path has changed, then the path needs to be updated with the new device name for the new library path. The same holds true when dealing with a 3584 library with the ALMS feature. This is discussed in the following document : http://www-1.ibm.com/support/docview.wss?uid=swg21053638 That document will mention that TSM supports the Advanced
Re: TSM library master for two tape libraries causes problems
Was the single tape drive that had an error your library control path? Is Atape up to date? What OS is the TSM server running? Are there any errors in the errpt / error log / syslog, etc? [RC] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Schneider, John Sent: Friday, September 05, 2008 9:14 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] TSM library master for two tape libraries causes problems Greetings, We have a TSM instance that serves as the library master for two tape libraries. One is a IBM3584 tape library with 24 LTO4 drives. The second is a virtual tape library, an EMC EDL configured as a IBM3584 with 128 LTO1 tape drives. The library master has 9 other TSM instances and 4 Lan-free clients that are library clients, and appeal to it for tape mounts. Most of the time this works just fine. All the TSM instances are running TSM 5.4.3.0. A few weeks ago we upgraded them from 5.4.2.0 in order to pick up a patch to provide the LIBSHRTIMEOUT parameter. This may not have anything to do with our problem, but it IS a recent change. The problem comes when we have any problem with the real IBM3584 tape library. Two weeks ago a tape got stuck in the gripper. Last week the gripper itself actually broke, so no tapes could get mounted. Last night it was a single tape drive that got an error and went Polling. For some reason, in each one of these cases, after an hour or two all tape mounts start hanging, even those belonging to the virtual tape library. When we would do a 'q mount' they all showed up in Reserved status. So before long all backups going to the virtual tape library ground to a halt. Can any of you see a reason why the TSM library master should get into such a problem? Shouldn't all tape mounts be asynchronous? It seems like to me a single tape drive getting into problems should not keep all other mounts from proceeding. And it doesn't happen instantly. It seems to happen gradually. I probably should also mention that this is a fairly busy environment during the night. It isn't unusual for us to have over 100 virtual tape mounts simultaneously. That is the reason we needed the LIBSHRTIMEOUT parameter (mentioned above). Before we had that, we sometimes would get timeouts that caused tape mount failures, because the TSM library master's queue of tape mounts polls would get overrun. Since we put on that patch, and added 'LIBSHRTIMEOUT 60' to the options file, that problem has gone away. But now this problem seems to have taken it's place. Best Regards, John D. Schneider Lead Systems Administrator - Storage Sisters of Mercy Health Systems 3637 South Geyer Road St. Louis, MO 63127 Phone: 314-364-3150 Cell: 314-750-8721 Email: [EMAIL PROTECTED] This e-mail contains information which (a) may be PROPRIETARY IN NATURE OR OTHERWISE PROTECTED BY LAW FROM DISCLOSURE, and (b) is intended only for the use of the addressee(s) named above. If you are not the addressee, or the person responsible for delivering this to the addressee(s), you are notified that reading, copying or distributing this e-mail is prohibited. If you have received this e-mail in error, please contact the sender immediately. DISCLAIMER: This message is intended for the sole use of the addressee, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not the addressee you are hereby notified that you may not use, copy, disclose, or distribute to anyone the message or any information contained in the message. If you have received this message in error, please immediately advise the sender by reply email and delete this message.
Cross Site Vaulting
Good Morning I am looking for direction on which way to go. We are in the process of setting up cross site vaulting and not 100% sure on how to incorporate both nodes and libraries to share the libraries. I assume the most efficient way would be to have a library manager instance that controls both libraries from our remote site? If we were to loose the library manager we would then configure the libraries to each LPAR to manage their own libraries? Also consider library partitioning. Our implementation has a restricted budget where as we are attempting to set this up with existing infrastructure besides a couple of new 595 LPARS. An overview of what we have is x2 3584's one with an expansion and the other as standalone. A CWDM link with a 8gb trunked link over 15klm's and two 595 LPARS 1 cpu and 8gb mem. Two HBA's for disk and four HBA's for tape drives. Prod data will go across the wire to our non prod site which will have our prod backup server and vice versa. We are limited to 4 drive slots in the smaller frame library and have 4 LTO3 drives and 9 LTO2 drives in total to manage the data. We are planning on using two LTO3 and two LTO2 drives in the smaller library which will hold prod primary storage pool data and the rest in the non prod library. As the restriction of the size of the prod library we are only planning on copying prod data across sites and copying non prod data within the non prod library. Currently hold approximately 200TB's backed up data across two servers. The current two instances are 5.5.1 and all associated clients. Can someone point me to a doco or have a similar experience? I have been trawling through redbooks and google is my friend. Look forward to any responses Regards Bunnings Legal Disclaimer: 1) This email is confidential and may contain legally privileged information. If you are not the intended recipient, you must not disclose or use the information contained in it. If you have received this email in error, please notify us immediately by return email and delete the document. 2) All emails sent to and sent from Bunnings Group Limited. are scanned for content. Any material deemed to contain inappropriate subject matter will be reported to the email administrator of all parties concerned.
