Re: Slow Reclamation from disk
We initially got bit on this with the DIRMC storage pool, but I installed TSM at a site where their concern was restore speed and we elected to turn on CACHE for the disk storage pool. Turns out that their estimates of how much data would be backed up a day was too high and they ended up being able to keep almost a week's worth of data CACHE'd on DISK. So on the weekend when the COPYPOOL reclamation started, it would still be running 48-hours later with no end in site. Working with the client, we turned off CACHE, but to help on the restore speed I moved the MIGRATION tasks to the end of the day, around 5:00pm. That way restores from last nights' backups would come from disk, and that was the majority of their restoresusers trashing their file and want it back from yesterday. We ended up using this as a model for other sites we manage and moved the MIGRATION tasks to later in the day, but well before the backup windows start. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of David Longo Sent: Wednesday, June 26, 2002 8:14 PM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk It has more to do with LTO tapes becuase they don't handle the little files spoon fed one at a time as well as say 3570 tapes. I have both here and can compare. The root cause is TSM design though. (I still like LTO, it is really great in my view! Now that I understand what's happening, I just adjusted things so this one problem is not much of a problem.) They are correct, keeping on disk will speed up restores especially if you do a lot. But balance this out for each sites requirements. If you don't do many restores and the reclamation is difficult to complete, then keep less on disk. Similar to other tuning that we do with TSM. David Longo [EMAIL PROTECTED] 06/26/02 05:03PM I did try it and it does move a lot faster.. So this has nothing to do with the fact that my diskpool is on a SAN drives right? So keeping data on diskpool is not a good idea in small installations? I was told by Tivoli techs to try and keep a couple of days of backups on disk if possible to speed up restores... Is this not a good idea? Thanks for the help, Etienne Brodeur Bill Boyer [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 06/26/2002 11:55 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk When the primary location of a file is in a primary DISK random access storage pool, then this is working as designed. (That's the answer we got from Tivoli) In this case the reclamation task processes 1 file at a time per transaction. When the primary storage pool is a SEQUENTIAL media (either tape or FILE) then the files are batched up by the MOVEBATCHSIZE/MOVESIZETHRESH parameters. Move the files to a SEQENTIAL primary pool before running reclamation. Bill Boyer DSS, INc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Etienne Brodeur Sent: Wednesday, June 26, 2002 9:54 AM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkdllavdgen 31D 192 79 SOLLENTUNA Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about
Re: Slow Reclamation from disk
I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkällavägen 31D 192 79 SOLLENTUNA Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ==
Re: Slow Reclamation from disk
When the primary location of a file is in a primary DISK random access storage pool, then this is working as designed. (That's the answer we got from Tivoli) In this case the reclamation task processes 1 file at a time per transaction. When the primary storage pool is a SEQUENTIAL media (either tape or FILE) then the files are batched up by the MOVEBATCHSIZE/MOVESIZETHRESH parameters. Move the files to a SEQENTIAL primary pool before running reclamation. Bill Boyer DSS, INc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Etienne Brodeur Sent: Wednesday, June 26, 2002 9:54 AM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkdllavdgen 31D 192 79 SOLLENTUNA Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves
Re: Slow Reclamation from disk
This was from an old email of mine. We got info that my last paragraph was a good guess, it is a TSM design problem. Notice carefully if when the Reclamation is running slow, that there is just an output tape and no input tape. This means it is getting input files from disk and not tape, if you are going tape to tape it should be fast like mine. Also LTO tapes tend to exaggerate this problem due to backhitching when writing small files or something like that. That's why I noticed it more on LTO than 3575 library. There apparently isn't a fix planned by TSM, takes a lot of recoding and redesign as I understand it. David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] [EMAIL PROTECTED] 06/26/02 09:54AM I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkällavägen 31D 192 79 SOLLENTUNA Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies
Re: Slow Reclamation from disk
I did try it and it does move a lot faster.. So this has nothing to do with the fact that my diskpool is on a SAN drives right? So keeping data on diskpool is not a good idea in small installations? I was told by Tivoli techs to try and keep a couple of days of backups on disk if possible to speed up restores... Is this not a good idea? Thanks for the help, Etienne Brodeur Bill Boyer [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 06/26/2002 11:55 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk When the primary location of a file is in a primary DISK random access storage pool, then this is working as designed. (That's the answer we got from Tivoli) In this case the reclamation task processes 1 file at a time per transaction. When the primary storage pool is a SEQUENTIAL media (either tape or FILE) then the files are batched up by the MOVEBATCHSIZE/MOVESIZETHRESH parameters. Move the files to a SEQENTIAL primary pool before running reclamation. Bill Boyer DSS, INc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Etienne Brodeur Sent: Wednesday, June 26, 2002 9:54 AM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkdllavdgen 31D 192 79 SOLLENTUNA Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03
Re: Slow Reclamation from disk
It has more to do with LTO tapes becuase they don't handle the little files spoon fed one at a time as well as say 3570 tapes. I have both here and can compare. The root cause is TSM design though. (I still like LTO, it is really great in my view! Now that I understand what's happening, I just adjusted things so this one problem is not much of a problem.) They are correct, keeping on disk will speed up restores especially if you do a lot. But balance this out for each sites requirements. If you don't do many restores and the reclamation is difficult to complete, then keep less on disk. Similar to other tuning that we do with TSM. David Longo [EMAIL PROTECTED] 06/26/02 05:03PM I did try it and it does move a lot faster.. So this has nothing to do with the fact that my diskpool is on a SAN drives right? So keeping data on diskpool is not a good idea in small installations? I was told by Tivoli techs to try and keep a couple of days of backups on disk if possible to speed up restores... Is this not a good idea? Thanks for the help, Etienne Brodeur Bill Boyer [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 06/26/2002 11:55 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk When the primary location of a file is in a primary DISK random access storage pool, then this is working as designed. (That's the answer we got from Tivoli) In this case the reclamation task processes 1 file at a time per transaction. When the primary storage pool is a SEQUENTIAL media (either tape or FILE) then the files are batched up by the MOVEBATCHSIZE/MOVESIZETHRESH parameters. Move the files to a SEQENTIAL primary pool before running reclamation. Bill Boyer DSS, INc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Etienne Brodeur Sent: Wednesday, June 26, 2002 9:54 AM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk I am suddenly having the same problem... Reclamation is very slow. I barely manage to reclaim 1 GB/hour, which wasn't the case in the beginning (I think after all in the beginning I didn't have so much data outside). I update the stg to reclaim=60. I have a diskpool on a SAN (FastT500) and an LTO 3583 (also SAN attached through a San data GW). My server is on AIX 5 (TSM 4.2.1.9). Is there any reason why this should take forever like this? Thanks for the help. Etienne Brodeur Daniel Sparrman [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 03/11/2002 11:25 AM Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkdllavdgen 31D 192 79 SOLLENTUNA Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again
Slow Reclamation from disk
I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ==
Re: Slow Reclamation from disk
Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkällavägen 31D 192 79 SOLLENTUNA Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete
Re: Slow Reclamation from disk
Yep, it is a design flaw! Apar IC15925 describes this issue. ERROR DESCRIPTION During an offsite reclamation process; where files are being transfered from disk to tape. ADSM fails to group the files together into transaction groups. Instead the files are processed sequentially. Hence a file will be reclaimed and committed before the next file will be process. ADSM needs to honor the movebatchsize and movesizethresh options in the server options file, so that the files will be grouped together properly for reclimation. As a note, this problem will only be seen where off-site reclamation is occuring from disk to tape. COMMENTS: After taking a very close look at the solution for this APAR, it was determined that the code needed for this performance enhancement is significant because it requires a restructure in the offsite reclamation transaction processing. A requirement can be taken to include this performance improvement in the next release or version. I have submitted a design request - and actually had a TSM developer call me asking for details on this last September. I haven't heard anything since so I'm thinking they are not persuing it! Please submit your reqests for a design change - perhaps if enough people ask ... Tim Rushforth City of Winnipeg -Original Message- From: David Longo [mailto:[EMAIL PROTECTED]] Sent: Monday, March 11, 2002 10:24 AM To: [EMAIL PROTECTED] Subject: Slow Reclamation from disk I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ==
Re: Slow Reclamation from disk
That's it, alright! That APAR was from ADSM 2.1 closed in July 1997!! I guess the fix is on the back burner. David Longo [EMAIL PROTECTED] 03/11/02 11:33AM Yep, it is a design flaw! Apar IC15925 describes this issue. ERROR DESCRIPTION During an offsite reclamation process; where files are being transfered from disk to tape. ADSM fails to group the files together into transaction groups. Instead the files are processed sequentially. Hence a file will be reclaimed and committed before the next file will be process. ADSM needs to honor the movebatchsize and movesizethresh options in the server options file, so that the files will be grouped together properly for reclimation. As a note, this problem will only be seen where off-site reclamation is occuring from disk to tape. COMMENTS: After taking a very close look at the solution for this APAR, it was determined that the code needed for this performance enhancement is significant because it requires a restructure in the offsite reclamation transaction processing. A requirement can be taken to include this performance improvement in the next release or version. I have submitted a design request - and actually had a TSM developer call me asking for details on this last September. I haven't heard anything since so I'm thinking they are not persuing it! Please submit your reqests for a design change - perhaps if enough people ask ... Tim Rushforth City of Winnipeg -Original Message- From: David Longo [mailto:[EMAIL PROTECTED]] Sent: Monday, March 11, 2002 10:24 AM To: [EMAIL PROTECTED] Subject: Slow Reclamation from disk I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. == MMS health-first.org made the following annotations on 03/11
Re: Slow Reclamation from disk
We went round-and-round with Tivoli on this issue. They say it's working as designed. Here's what's happening... When the location of a file is on DISK which is RANDOM access, then TSM processes the files ONE AT A TIME instead of batching them together like on SEQuential media. We first discovered this with the DIRMC...our policy put the directories into a DISK storage pool and then copied them to offsite each day. Then the reclamation ran forever. Tivoli says that RANDOM DISK is processed 1 file at a time and they have no plans to change this. SO you can imaging how many 1 at a times there are reclaiming your DIRMC pool. Now we configure up with a first-level DISK pool that we migrate for a FILE devclass. The FILE is a SEQuential media and reclamation runs OK. So, if you can migrate the data before you start the reclamation you should get better response times. HTH, Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Daniel Sparrman Sent: Monday, March 11, 2002 11:26 AM To: [EMAIL PROTECTED] Subject: Re: Slow Reclamation from disk Hi How is your diskpool configured? Is it placed locally on the machine, or is it SAN attached? Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Bergkällavägen 31D 192 79 SOLLENTUNA Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 David Longo David.Longo@HEALTHTo: [EMAIL PROTECTED] -FIRST.ORGcc: Sent by: ADSM:Subject: Slow Reclamation from disk Dist Stor Manager [EMAIL PROTECTED] DU 2002-03-11 17:23 Please respond to ADSM: Dist Stor Manager I have old TSM server 3.7.4.0 on AIX 4.3.3 with 3575-L32 library with C-XL drives. New server is TSM 4.2.1.9 on AIX 4.3.3 ML09 and IBM 3584-L32 library with 8 FC-AL drives. Having the 3584 running for about 6 weeks now, I see what some of the talk about slownes in the last year actually is. On BOTH TSM systems there is slowness in offsite tape reclamation WHEN some of the files being reclaimed are still on the Disk stgpool. When tape reclamation starts, it figures out which tapes to reclaim and where the data is - understand. Then when it actually starts it will copy files that are STILL in disk stgpool first before files that are only on tape - makes sense. Then move to files that are only on tape. However, when it is copying files from disk it is MUCH slower than when copying from tape - I would say 10 times slower. But when doing a MIGRATION disk to tape or BACKUP STGPOOL fromdisk to tape on either system, it's like going from idle to 4th gear! This slownes can happen on thses system when there is NO OTHER ACTIVITY but this single reclamation. Having two different systems to compare at the same time seems to indicate that this is NOT a disk problem or a tape problem (except that the 3584 may not handle it quite as well). Also our old system tended to have disk pool emptier when reclaimation started and this wasn't really noticed on that system until I started paying close attention. I again point out that my 3584 really smokes when doing everything else BUT reclamation from disk to tape. So does the 3575 - relatively speaking. (My 3584 library is library F/W 2360 and drives are 18N2 as delivered, I don't think this is problem but included for completeness). It seems that this is some inherent Tivoli design flaw/feature in the RECLAMATION process. Anyone with detail knowledge of Reclamation can provide some insite? Andy Raibeck? Thanks, David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5525 [EMAIL PROTECTED] MMS health-first.org made the following annotations on 03/11/02 11:37:16 -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views
Re: Slow Reclamation from disk
That's it, alright! That APAR was from ADSM 2.1 closed in July 1997!! I guess the fix is on the back burner. I know it's not Friday, but this is even funnier then all the I'm not here in about 8 languages;-) Cheers, Henk.
