Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Yes it still works that way when you don't specify a DIRMC. Kyle fred johanson <[EMAIL PROTECTED]> wrote: Does anyone know if TSM still puts directories in the mgmtc with the longest retention period? On one of my machines, that belongs to a special group of machines with all sorts of special handling. I've used DIRMC to ensure the directory of some desktop doesn't get treated in the same way. At 01:25 PM 3/19/2005 -0800, you wrote: >Paul, >Using a separate pool for directories is something that many have been >doing for a long time and just kept doing even after IBM implemented the >new directory restore method (restore order processing). If you look at a >directory as a small file then you can see why keeping it in a separate >pool so that you can keep that data on disk might help. This is one >reason why we have not yet stopped using the DIRMC. With that being said >more and more of my customers are implementing file device classes or >VTL's which keep most of the primary data on disk. As a result I no >longer see the need for separating out the directories to another disk >location. > >About a year ago many of my larger customers would complain about how long >the DIRMC disk pools would take to backup. In working with support we >found that this was a WAD feature. I think the issue was (can't remember >for sure) that each file in a disk pool is evaluated on every backup where >sequential access pools are evaluated differently. As a result we started >taking our DIRMC pools and giving them a small pool built with a file >device class definition. We made sure the data migrated from disk to file >device class each day and as a result the storage pool copy problem went away. > >Now the fact that we are using file device classes as described above is >why I am concerned about the issue that was mentioned in this thread about >the larger default block size. > >All of these issues together lead me to believe that DIRMC pools are no >longer as necessary as they used to be. > >Kyle > > >Paul Fielding wrote: >Hi Richard, > >I took a look through the Quickfacts (something I should have done long >ago). It does indeed suggest that surrogate directories are created and the >real directories are restored as they are hit. > >Has anyone really observed this to be genuinely true? I have in the past >observed the double-tape-mount theory, and though I understand it is >supposedly fixed, I haven't heard anyone say "I have seen it, I know it >works, you no longer need to keep a dirmc diskpool". > >Of course, if it is indeed working as designed now, it doesn't resolve the >other dirmc issues currently being discussed in this thread. > >Is there anyone on the list who has in recent history decided to ditch using >a dirmc diskpool altogether and done so with success on the restore side? > >regards, > >Paul > >- Original Message - >From: "Richard Sims" >To: >Sent: Saturday, March 19, 2005 4:44 AM >Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation performance issues >resolved or not. > > > > Paul - > > > > This generally falls under the TSM term Restore Order processing. We've > > discussed it on the List before. I have an entry on it in ADSM > > QuickFacts which you can refer to as a preliminary to further pursuit > > in IBM doc. > > > > Richard Sims http://people.bu.edu/rbs > > > > On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: > > > >> I'd be interested in more discussion on this point. My original > >> understanding was actually a bit different that that. The impression > >> I had > >> was that originally directory tree structures were restored before any > >> files > >> happened, period. Following that, files would be restored. Net result > >> - > >> tapes might get mounted twice. > >> > >> Is my understanding incorrect? (could well be). If this behavior has > >> indeed > >> been fixed so that directories are restored as they are hit on the tape > >> (with a pre-created non-ACLed directory being created first) then it > >> would > >> indeed make sense that a DIRMC pool is no longer needed. > >> > >> Is there any documentation on this somewhere I can reference? > > > > > >- >Do you Yahoo!? > Yahoo! Small Business - Try our new resources site! Fred Johanson ITSM Administrator University of Chicago 773-702-8464 - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Does anyone know if TSM still puts directories in the mgmtc with the longest retention period? On one of my machines, that belongs to a special group of machines with all sorts of special handling. I've used DIRMC to ensure the directory of some desktop doesn't get treated in the same way. At 01:25 PM 3/19/2005 -0800, you wrote: Paul, Using a separate pool for directories is something that many have been doing for a long time and just kept doing even after IBM implemented the new directory restore method (restore order processing). If you look at a directory as a small file then you can see why keeping it in a separate pool so that you can keep that data on disk might help. This is one reason why we have not yet stopped using the DIRMC. With that being said more and more of my customers are implementing file device classes or VTL's which keep most of the primary data on disk. As a result I no longer see the need for separating out the directories to another disk location. About a year ago many of my larger customers would complain about how long the DIRMC disk pools would take to backup. In working with support we found that this was a WAD feature. I think the issue was (can't remember for sure) that each file in a disk pool is evaluated on every backup where sequential access pools are evaluated differently. As a result we started taking our DIRMC pools and giving them a small pool built with a file device class definition. We made sure the data migrated from disk to file device class each day and as a result the storage pool copy problem went away. Now the fact that we are using file device classes as described above is why I am concerned about the issue that was mentioned in this thread about the larger default block size. All of these issues together lead me to believe that DIRMC pools are no longer as necessary as they used to be. Kyle Paul Fielding <[EMAIL PROTECTED]> wrote: Hi Richard, I took a look through the Quickfacts (something I should have done long ago). It does indeed suggest that surrogate directories are created and the real directories are restored as they are hit. Has anyone really observed this to be genuinely true? I have in the past observed the double-tape-mount theory, and though I understand it is supposedly fixed, I haven't heard anyone say "I have seen it, I know it works, you no longer need to keep a dirmc diskpool". Of course, if it is indeed working as designed now, it doesn't resolve the other dirmc issues currently being discussed in this thread. Is there anyone on the list who has in recent history decided to ditch using a dirmc diskpool altogether and done so with success on the restore side? regards, Paul - Original Message - From: "Richard Sims" To: Sent: Saturday, March 19, 2005 4:44 AM Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation performance issues resolved or not. > Paul - > > This generally falls under the TSM term Restore Order processing. We've > discussed it on the List before. I have an entry on it in ADSM > QuickFacts which you can refer to as a preliminary to further pursuit > in IBM doc. > > Richard Sims http://people.bu.edu/rbs > > On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: > >> I'd be interested in more discussion on this point. My original >> understanding was actually a bit different that that. The impression >> I had >> was that originally directory tree structures were restored before any >> files >> happened, period. Following that, files would be restored. Net result >> - >> tapes might get mounted twice. >> >> Is my understanding incorrect? (could well be). If this behavior has >> indeed >> been fixed so that directories are restored as they are hit on the tape >> (with a pre-created non-ACLed directory being created first) then it >> would >> indeed make sense that a DIRMC pool is no longer needed. >> >> Is there any documentation on this somewhere I can reference? > - Do you Yahoo!? Yahoo! Small Business - Try our new resources site! Fred Johanson ITSM Administrator University of Chicago 773-702-8464
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Restore Order processing did definitely fix one problem. DIRMC still solves other problems if you are storing data on tape. We used to tell our users that backups were stored on disk from the previous night's backups so if they did restores during the day then there would be no tape mounts. We have 1 group that uses TSM to clone DB's from prod onto a test box. All DB files would be backed up the previous night, but when the user went to do the restore tape mounts were occurring (and to make matters worse we were experiencing the corrupt CM issue on LTO tapes at the time!). I thought the reason tapes were being mounted was perhaps because they were selecting a directory plus all of the files to be restored (and the directory of course was backed up days, months, or years ago) so was on tape. But this wasn't the case, I don't remember the exact restore syntax now but this made us implement DIRMC and we've been happy since! (Perhaps this was some other bug here?) So DIRMC is being used successfully for us so that no unnecessary tape mounts occur to restore a directory entry here and there. (We have limited tape drives, usually doing reclaims during the day - sometimes with huge exchange db files which take a while to preempt). So ensure you understand all of the issues before you decide to dump DIRMC. Tim Rushforth City of Winnipeg -Original Message- From: Kenneth & Susan Bury [mailto:[EMAIL PROTECTED] Sent: Saturday, March 19, 2005 3:17 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. Paul, It is definitely, absolutely, positively, seen it myself - fixed Been fixed for years. Forget DIRMC. Ken > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of Paul Fielding > Sent: Saturday, March 19, 2005 16:06 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: DIRMC - Are copypool reclamation performance > issues resolved or not. > > > Hi Richard, > > I took a look through the Quickfacts (something I should have > done long ago). It does indeed suggest that surrogate > directories are created and the real directories are restored > as they are hit. > > Has anyone really observed this to be genuinely true? I have > in the past observed the double-tape-mount theory, and though > I understand it is supposedly fixed, I haven't heard anyone > say "I have seen it, I know it works, you no longer need to > keep a dirmc diskpool". > > Of course, if it is indeed working as designed now, it > doesn't resolve the other dirmc issues currently being > discussed in this thread. > > Is there anyone on the list who has in recent history decided > to ditch using a dirmc diskpool altogether and done so with > success on the restore side? > > regards, > > Paul > > - Original Message - > From: "Richard Sims" <[EMAIL PROTECTED]> > To: > Sent: Saturday, March 19, 2005 4:44 AM > Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation > performance issues resolved or not. > > > > Paul - > > > > This generally falls under the TSM term Restore Order processing. > > We've discussed it on the List before. I have an entry on > it in ADSM > > QuickFacts which you can refer to as a preliminary to > further pursuit > > in IBM doc. > > > > Richard Simshttp://people.bu.edu/rbs > > > > On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: > > > >> I'd be interested in more discussion on this point. My original > >> understanding was actually a bit different that that. The > impression > >> I had was that originally directory tree structures were restored > >> before any files > >> happened, period. Following that, files would be restored. > Net result > >> - > >> tapes might get mounted twice. > >> > >> Is my understanding incorrect? (could well be). If this > behavior has > >> indeed been fixed so that directories are restored as they > are hit on > >> the tape (with a pre-created non-ACLed directory being > created first) > >> then it would > >> indeed make sense that a DIRMC pool is no longer needed. > >> > >> Is there any documentation on this somewhere I can reference? > > >
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Paul, Using a separate pool for directories is something that many have been doing for a long time and just kept doing even after IBM implemented the new directory restore method (restore order processing). If you look at a directory as a small file then you can see why keeping it in a separate pool so that you can keep that data on disk might help. This is one reason why we have not yet stopped using the DIRMC. With that being said more and more of my customers are implementing file device classes or VTL's which keep most of the primary data on disk. As a result I no longer see the need for separating out the directories to another disk location. About a year ago many of my larger customers would complain about how long the DIRMC disk pools would take to backup. In working with support we found that this was a WAD feature. I think the issue was (can't remember for sure) that each file in a disk pool is evaluated on every backup where sequential access pools are evaluated differently. As a result we started taking our DIRMC pools and giving them a small pool built with a file device class definition. We made sure the data migrated from disk to file device class each day and as a result the storage pool copy problem went away. Now the fact that we are using file device classes as described above is why I am concerned about the issue that was mentioned in this thread about the larger default block size. All of these issues together lead me to believe that DIRMC pools are no longer as necessary as they used to be. Kyle Paul Fielding <[EMAIL PROTECTED]> wrote: Hi Richard, I took a look through the Quickfacts (something I should have done long ago). It does indeed suggest that surrogate directories are created and the real directories are restored as they are hit. Has anyone really observed this to be genuinely true? I have in the past observed the double-tape-mount theory, and though I understand it is supposedly fixed, I haven't heard anyone say "I have seen it, I know it works, you no longer need to keep a dirmc diskpool". Of course, if it is indeed working as designed now, it doesn't resolve the other dirmc issues currently being discussed in this thread. Is there anyone on the list who has in recent history decided to ditch using a dirmc diskpool altogether and done so with success on the restore side? regards, Paul - Original Message - From: "Richard Sims" To: Sent: Saturday, March 19, 2005 4:44 AM Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation performance issues resolved or not. > Paul - > > This generally falls under the TSM term Restore Order processing. We've > discussed it on the List before. I have an entry on it in ADSM > QuickFacts which you can refer to as a preliminary to further pursuit > in IBM doc. > > Richard Sims http://people.bu.edu/rbs > > On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: > >> I'd be interested in more discussion on this point. My original >> understanding was actually a bit different that that. The impression >> I had >> was that originally directory tree structures were restored before any >> files >> happened, period. Following that, files would be restored. Net result >> - >> tapes might get mounted twice. >> >> Is my understanding incorrect? (could well be). If this behavior has >> indeed >> been fixed so that directories are restored as they are hit on the tape >> (with a pre-created non-ACLed directory being created first) then it >> would >> indeed make sense that a DIRMC pool is no longer needed. >> >> Is there any documentation on this somewhere I can reference? > - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Paul, It is definitely, absolutely, positively, seen it myself - fixed Been fixed for years. Forget DIRMC. Ken > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of Paul Fielding > Sent: Saturday, March 19, 2005 16:06 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: DIRMC - Are copypool reclamation performance > issues resolved or not. > > > Hi Richard, > > I took a look through the Quickfacts (something I should have > done long ago). It does indeed suggest that surrogate > directories are created and the real directories are restored > as they are hit. > > Has anyone really observed this to be genuinely true? I have > in the past observed the double-tape-mount theory, and though > I understand it is supposedly fixed, I haven't heard anyone > say "I have seen it, I know it works, you no longer need to > keep a dirmc diskpool". > > Of course, if it is indeed working as designed now, it > doesn't resolve the other dirmc issues currently being > discussed in this thread. > > Is there anyone on the list who has in recent history decided > to ditch using a dirmc diskpool altogether and done so with > success on the restore side? > > regards, > > Paul > > - Original Message - > From: "Richard Sims" <[EMAIL PROTECTED]> > To: > Sent: Saturday, March 19, 2005 4:44 AM > Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation > performance issues resolved or not. > > > > Paul - > > > > This generally falls under the TSM term Restore Order processing. > > We've discussed it on the List before. I have an entry on > it in ADSM > > QuickFacts which you can refer to as a preliminary to > further pursuit > > in IBM doc. > > > > Richard Simshttp://people.bu.edu/rbs > > > > On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: > > > >> I'd be interested in more discussion on this point. My original > >> understanding was actually a bit different that that. The > impression > >> I had was that originally directory tree structures were restored > >> before any files > >> happened, period. Following that, files would be restored. > Net result > >> - > >> tapes might get mounted twice. > >> > >> Is my understanding incorrect? (could well be). If this > behavior has > >> indeed been fixed so that directories are restored as they > are hit on > >> the tape (with a pre-created non-ACLed directory being > created first) > >> then it would > >> indeed make sense that a DIRMC pool is no longer needed. > >> > >> Is there any documentation on this somewhere I can reference? > > >
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Hi Richard, I took a look through the Quickfacts (something I should have done long ago). It does indeed suggest that surrogate directories are created and the real directories are restored as they are hit. Has anyone really observed this to be genuinely true? I have in the past observed the double-tape-mount theory, and though I understand it is supposedly fixed, I haven't heard anyone say "I have seen it, I know it works, you no longer need to keep a dirmc diskpool". Of course, if it is indeed working as designed now, it doesn't resolve the other dirmc issues currently being discussed in this thread. Is there anyone on the list who has in recent history decided to ditch using a dirmc diskpool altogether and done so with success on the restore side? regards, Paul - Original Message - From: "Richard Sims" <[EMAIL PROTECTED]> To: Sent: Saturday, March 19, 2005 4:44 AM Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation performance issues resolved or not. Paul - This generally falls under the TSM term Restore Order processing. We've discussed it on the List before. I have an entry on it in ADSM QuickFacts which you can refer to as a preliminary to further pursuit in IBM doc. Richard Simshttp://people.bu.edu/rbs On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: I'd be interested in more discussion on this point. My original understanding was actually a bit different that that. The impression I had was that originally directory tree structures were restored before any files happened, period. Following that, files would be restored. Net result - tapes might get mounted twice. Is my understanding incorrect? (could well be). If this behavior has indeed been fixed so that directories are restored as they are hit on the tape (with a pre-created non-ACLed directory being created first) then it would indeed make sense that a DIRMC pool is no longer needed. Is there any documentation on this somewhere I can reference?
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Paul - This generally falls under the TSM term Restore Order processing. We've discussed it on the List before. I have an entry on it in ADSM QuickFacts which you can refer to as a preliminary to further pursuit in IBM doc. Richard Simshttp://people.bu.edu/rbs On Mar 19, 2005, at 3:06 AM, Paul Fielding wrote: I'd be interested in more discussion on this point. My original understanding was actually a bit different that that. The impression I had was that originally directory tree structures were restored before any files happened, period. Following that, files would be restored. Net result - tapes might get mounted twice. Is my understanding incorrect? (could well be). If this behavior has indeed been fixed so that directories are restored as they are hit on the tape (with a pre-created non-ACLed directory being created first) then it would indeed make sense that a DIRMC pool is no longer needed. Is there any documentation on this somewhere I can reference?
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
I'd be interested in more discussion on this point. My original understanding was actually a bit different that that. The impression I had was that originally directory tree structures were restored before any files happened, period. Following that, files would be restored. Net result - tapes might get mounted twice. Is my understanding incorrect? (could well be). If this behavior has indeed been fixed so that directories are restored as they are hit on the tape (with a pre-created non-ACLed directory being created first) then it would indeed make sense that a DIRMC pool is no longer needed. Is there any documentation on this somewhere I can reference? regards, Paul - Original Message - From: "TSM_User" <[EMAIL PROTECTED]> To: Sent: Thursday, March 17, 2005 3:54 PM Subject: Re: [ADSM-L] DIRMC - Are copypool reclamation performance issues resolved or not. If V5.3 in fact only writes in larger blocks in the smaller directories may take up more space that required. Still, that issue aside you should no longer need to have a DIRMC pool. At one time there was a feature (or call it a bug) where every directory had to be restored as it came up which would cause many more mounts of tape drives. For some time now a restore create a directory (without ACL's) so that the restore can continue. Then when the directory itself is hit it will simply restore over top of the directory that was created. This will ensure each tape is still only ready once. True, directories are like small files and just like small files restoring from disk would be faster but the bug that used to exist has long since been fixed. Further as people implement file device class storage pools and other disk only solutions like VTL's I don't see the need for seperating the directories into a seperate pool. Kyle "Rushforth, Tim" <[EMAIL PROTECTED]> wrote: What in 5.3 warrants new consideration? The reason we implemented DIRMC is so that when a user restores a file(s) there are not extra tape mounts to restore the directories We ran into this on multiple occasions, even when all files were on disk, tape mounts would occur because the directories were on tape. Thanks, Tim Rusforth City of Winnipeg -Original Message- From: TSM_User [mailto:[EMAIL PROTECTED] Sent: Wed 3/16/2005 6:48 PM To: ADSM-L@VM.MARIST.EDU Cc: Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed but the reason there have been suggestions to use a file type device class is because disk pools unline sequential pools are scanned from begining to end for every storage pool backup. I have had some customers that have millions of directories in their DIRMC pool. Even when none change they backup runs from hours on that pool. With a file type device class only the new volumes would be backed up resulting in a much faster backup. Now all that being said this new feature in V5.3 warrents new consideration. My new consideration is to stop using DIRMC pools as the reason they were created in the first place has also long been fixed. Kyle "Thorneycroft, Doug" wrote: OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe. - Do you Yahoo!? Yahoo! Small Business - Try our new resources site! - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
If V5.3 in fact only writes in larger blocks in the smaller directories may take up more space that required. Still, that issue aside you should no longer need to have a DIRMC pool. At one time there was a feature (or call it a bug) where every directory had to be restored as it came up which would cause many more mounts of tape drives. For some time now a restore create a directory (without ACL's) so that the restore can continue. Then when the directory itself is hit it will simply restore over top of the directory that was created. This will ensure each tape is still only ready once. True, directories are like small files and just like small files restoring from disk would be faster but the bug that used to exist has long since been fixed. Further as people implement file device class storage pools and other disk only solutions like VTL's I don't see the need for seperating the directories into a seperate pool. Kyle "Rushforth, Tim" <[EMAIL PROTECTED]> wrote: What in 5.3 warrants new consideration? The reason we implemented DIRMC is so that when a user restores a file(s) there are not extra tape mounts to restore the directories We ran into this on multiple occasions, even when all files were on disk, tape mounts would occur because the directories were on tape. Thanks, Tim Rusforth City of Winnipeg -Original Message- From: TSM_User [mailto:[EMAIL PROTECTED] Sent: Wed 3/16/2005 6:48 PM To: ADSM-L@VM.MARIST.EDU Cc: Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed but the reason there have been suggestions to use a file type device class is because disk pools unline sequential pools are scanned from begining to end for every storage pool backup. I have had some customers that have millions of directories in their DIRMC pool. Even when none change they backup runs from hours on that pool. With a file type device class only the new volumes would be backed up resulting in a much faster backup. Now all that being said this new feature in V5.3 warrents new consideration. My new consideration is to stop using DIRMC pools as the reason they were created in the first place has also long been fixed. Kyle "Thorneycroft, Doug" wrote: OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe. - Do you Yahoo!? Yahoo! Small Business - Try our new resources site! - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Storage pools consist of one or more volumes, generally disk or tape. The storage pool gets its volumes via the device class which has a maxscr setting to limit the volume count and max capacity to estimate or assign the max size of the volume. The device class also points to a directory which in the case of windows is drive:\path. Within that path TSM will create volumes used by the storage pool up to the count and size limits or you can predefine the volumes to the storage pool using the TSM DEF VOL command. Don't know too much about AIX and TSM. Mike wrote: On Thu, 17 Mar 2005, Steve Bennett might have said: Wanda, I just added a sata disk array in TSM v5.2 so I'll jump in here. If you are using one disk partition in Windows for the device class then you can let TSM define the number of vols it needs up to maxscr or out of disk condition. Each volume name will be unique and assigned by TSM. If you use more than one one disk partition for the device class you need to dsmfmt as many volumes as you need and you must then define those volumes to the storage pool. Only the volume names you specified will be used and reused. Are you saying that if I have one huge disk that TSM will carve it up into some size of logical slices and then use those slices? I'm on AIX, not Windows, does this make a difference? Currently we use dsmfmt to create the files on disk that TSM uses as storage pools. I'm not the TSM admin. I think the individual storage pools are manually given to a 'storage class'? We have enough disk for a night's backup (of incrementals). I'd like to give all the disk to TSM to manage as little volumes (or whatever the term is) so that backups are quick (node->network->tsm). Mike -- Steve Bennett, (907) 465-5783 State of Alaska, Enterprise Technology Services, Technical Services Section
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
On Thu, 17 Mar 2005, Steve Bennett might have said: > Wanda, > > I just added a sata disk array in TSM v5.2 so I'll jump in here. > > If you are using one disk partition in Windows for the device class then > you can let TSM define the number of vols it needs up to maxscr or out > of disk condition. Each volume name will be unique and assigned by TSM. > > If you use more than one one disk partition for the device class you > need to dsmfmt as many volumes as you need and you must then define > those volumes to the storage pool. Only the volume names you specified > will be used and reused. Are you saying that if I have one huge disk that TSM will carve it up into some size of logical slices and then use those slices? I'm on AIX, not Windows, does this make a difference? Currently we use dsmfmt to create the files on disk that TSM uses as storage pools. I'm not the TSM admin. I think the individual storage pools are manually given to a 'storage class'? We have enough disk for a night's backup (of incrementals). I'd like to give all the disk to TSM to manage as little volumes (or whatever the term is) so that backups are quick (node->network->tsm). Mike
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
On Thu, Mar 17, 2005 at 07:38:16AM -0500, Richard Sims wrote: > blocks of 256 KiB minimum...". Could you provide a documentation or web > site reference for that 5.3 change? No, sorry. Just the info I received through the PMR. I made the suggestion to include this in e.g. a README, and that suggestion was passed on. > According to 2004 Technote 1167281, the block size for FILE and DISK > devtypes has been a fixed size of 256 KB for some time. > > I'd like to see definitive information on this before committing my 5.3 > beliefs. Hmmm. The only thing I know for certain is that the exact same setup that works with 5.2 and below, doesn't work with 5.3 due to FILE volumes filling up way too quickly. In the PMR I provided trace information, etc. etc. Eventually the result was that this was working as designed, and that a design change in TSM 5.3 caused the volumes to fill up to only 1,5% before encountering end-of-volume. The description given in the PMR seems to fit the symptoms quite well. -- Jurjen Oskam
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
You could skip the dsmfmt step - I don't think it actually buys you anything. We've just done define volume (you just need to calculate how much space you have first to determine the # of volumes to define). The disk only backup document states that dsmfmt may buy you less fragmentation but this is not the case (at least on windows, on 5.2.2.4 - discussed on the list). Tim -Original Message- From: Steve Bennett [mailto:[EMAIL PROTECTED] Sent: Thursday, March 17, 2005 10:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. Wanda, I just added a sata disk array in TSM v5.2 so I'll jump in here. If you are using one disk partition in Windows for the device class then you can let TSM define the number of vols it needs up to maxscr or out of disk condition. Each volume name will be unique and assigned by TSM. If you use more than one one disk partition for the device class you need to dsmfmt as many volumes as you need and you must then define those volumes to the storage pool. Only the volume names you specified will be used and reused. Prather, Wanda wrote: > Tim: > > We are looking at using all disk now for our onsite disk pool with our > next capital$ buy. > Something I've never been sure of - > Whenf you use a type=file devclass for backups, > > 1) Do you predefine volumes somehow, or just let TSM create a new one > after the first hits max capacity? > 2) Do reclaims happen by themselves, or do you have to force it somehow? > > Thanks > Wanda > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of > Rushforth, Tim > Sent: Wednesday, March 16, 2005 5:31 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: DIRMC - Are copypool reclamation performance issues > resolved or not. > > > It is fixed (somewhere around 5.1.5.2). > > -Original Message- > From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] > Sent: Wednesday, March 16, 2005 4:25 PM > To: ADSM-L@VM.MARIST.EDU > Subject: DIRMC - Are copypool reclamation performance issues resolved or > not. > > OK, after spending a large portion of my day reviewing adsm-l post going > back to > 2000, I'm still not sure. Does anyone know if there is still a > performance problem > running reclamation on a DIRMC random access disk pool? > I came across one post that said it was supposedly fixed, but > recommended using > a file type diskpool to be safe. > -- Steve Bennett, (907) 465-5783 State of Alaska, Enterprise Technology Services, Technical Services Section
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Tim/Steve Thanks - got it! -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Thursday, March 17, 2005 11:17 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. 1. You can predefine or use scratch. (Below 5.3 if you are using scratch you are limited to one file system) We have multiple 1 TB file systems on 5.2.2.4 so we've predefined all of our volumes (via define volume). Have you seen the Disk only Backups presentation? http://www-1.ibm.com/support/docview.wss?uid=swg21193983&aid=1 2. Reclaims are exactly the same as Tape reclaims - set your reclamation threshold and reclamation will be done for volumes that meet the criteria. Tim Rushforth City of Winnipeg -Original Message- From: Prather, Wanda [mailto:[EMAIL PROTECTED] Sent: Thursday, March 17, 2005 9:49 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. Tim: We are looking at using all disk now for our onsite disk pool with our next capital$ buy. Something I've never been sure of - Whenf you use a type=file devclass for backups, 1) Do you predefine volumes somehow, or just let TSM create a new one after the first hits max capacity? 2) Do reclaims happen by themselves, or do you have to force it somehow? Thanks Wanda -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Wednesday, March 16, 2005 5:31 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe.
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
1. You can predefine or use scratch. (Below 5.3 if you are using scratch you are limited to one file system) We have multiple 1 TB file systems on 5.2.2.4 so we've predefined all of our volumes (via define volume). Have you seen the Disk only Backups presentation? http://www-1.ibm.com/support/docview.wss?uid=swg21193983&aid=1 2. Reclaims are exactly the same as Tape reclaims - set your reclamation threshold and reclamation will be done for volumes that meet the criteria. Tim Rushforth City of Winnipeg -Original Message- From: Prather, Wanda [mailto:[EMAIL PROTECTED] Sent: Thursday, March 17, 2005 9:49 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. Tim: We are looking at using all disk now for our onsite disk pool with our next capital$ buy. Something I've never been sure of - Whenf you use a type=file devclass for backups, 1) Do you predefine volumes somehow, or just let TSM create a new one after the first hits max capacity? 2) Do reclaims happen by themselves, or do you have to force it somehow? Thanks Wanda -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Wednesday, March 16, 2005 5:31 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe.
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Wanda, I just added a sata disk array in TSM v5.2 so I'll jump in here. If you are using one disk partition in Windows for the device class then you can let TSM define the number of vols it needs up to maxscr or out of disk condition. Each volume name will be unique and assigned by TSM. If you use more than one one disk partition for the device class you need to dsmfmt as many volumes as you need and you must then define those volumes to the storage pool. Only the volume names you specified will be used and reused. Prather, Wanda wrote: Tim: We are looking at using all disk now for our onsite disk pool with our next capital$ buy. Something I've never been sure of - Whenf you use a type=file devclass for backups, 1) Do you predefine volumes somehow, or just let TSM create a new one after the first hits max capacity? 2) Do reclaims happen by themselves, or do you have to force it somehow? Thanks Wanda -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Wednesday, March 16, 2005 5:31 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe. -- Steve Bennett, (907) 465-5783 State of Alaska, Enterprise Technology Services, Technical Services Section
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Tim: We are looking at using all disk now for our onsite disk pool with our next capital$ buy. Something I've never been sure of - Whenf you use a type=file devclass for backups, 1) Do you predefine volumes somehow, or just let TSM create a new one after the first hits max capacity? 2) Do reclaims happen by themselves, or do you have to force it somehow? Thanks Wanda -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Wednesday, March 16, 2005 5:31 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe.
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Jurjen - In this thread, and the "Minor gotcha on upgrade to 5.3" thread, you indicate that TSM 5.3 has changed things such that "...the handling of FILE volumes was changed. All writes to such a volume is now done in blocks of 256 KiB minimum...". Could you provide a documentation or web site reference for that 5.3 change? The only "256 KB" related change that I can find in 5.3 is a Windows-specific, tape-only maximum block size change (max: 256 KB), controlled by a new DSMMAXSG utility for tweaking Host Based Adapters. (ADSM/TSM always used a maximal tape drive block size on Unix systems, but Windows was problematic.) This is documented in the TSM 5.3 Technical Guide redbook, as well as the TSM 5.3 for Windows Admin Ref and Admin Guide manuals. According to 2004 Technote 1167281, the block size for FILE and DISK devtypes has been a fixed size of 256 KB for some time. I'd like to see definitive information on this before committing my 5.3 beliefs. thanks, Richard Sims On Mar 17, 2005, at 2:29 AM, Jurjen Oskam wrote: On Wed, Mar 16, 2005 at 07:29:50PM -0600, Rushforth, Tim wrote: [DIRMC] What in 5.3 warrants new consideration? Probably the fact that sequential volumes are written to in blocks of at least 256 KB, even when the data is only 1500 bytes. This can cause a lot of overhead, and the effective capacity of sequential volumes could be reduced by a factor of 60 or more. Note that a great number of factors influence the above statement. On the one end, it could cause a perfectly OK setup under 5.2 to be unusable under 5.3. On the opposite end, you might notice no adverse effects and even experience a performance improvement. In the case of FILE storagepools used as a DIRMC destination, you might find yourself in the first scenario (works under <= 5.2, doesn't work under 5.3). You might be able to alleviate this by adjusting the TXNGROUPMAX server setting and the TXNBYTELIMIT client setting. Unfortunately, this doesn't affect data that's already in a storage pool. And unfortunately again, storagepools used as DIRMC destination often have quite generous retention settings, as per the documentation and general recommendation. Part of the reason that these retention settings could be generous was that directories were so small that it wouldn't matter a lot. With the new handling of sequential volumes, it does start to matter (and sometimes a lot). -- Jurjen Oskam
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
On Wed, Mar 16, 2005 at 07:29:50PM -0600, Rushforth, Tim wrote: [DIRMC] > What in 5.3 warrants new consideration? Probably the fact that sequential volumes are written to in blocks of at least 256 KB, even when the data is only 1500 bytes. This can cause a lot of overhead, and the effective capacity of sequential volumes could be reduced by a factor of 60 or more. Note that a great number of factors influence the above statement. On the one end, it could cause a perfectly OK setup under 5.2 to be unusable under 5.3. On the opposite end, you might notice no adverse effects and even experience a performance improvement. In the case of FILE storagepools used as a DIRMC destination, you might find yourself in the first scenario (works under <= 5.2, doesn't work under 5.3). You might be able to alleviate this by adjusting the TXNGROUPMAX server setting and the TXNBYTELIMIT client setting. Unfortunately, this doesn't affect data that's already in a storage pool. And unfortunately again, storagepools used as DIRMC destination often have quite generous retention settings, as per the documentation and general recommendation. Part of the reason that these retention settings could be generous was that directories were so small that it wouldn't matter a lot. With the new handling of sequential volumes, it does start to matter (and sometimes a lot). -- Jurjen Oskam
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
- Original Message - in a much faster backup. Now all that being said this new feature in V5.3 warrents new consideration. My new consideration is to stop using DIRMC pools as the reason they were created in the first place has also long been fixed. Which reason is this that has been fixed? How about quick restoration of ACLed directories (still an issue, no?)... Paul
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
What in 5.3 warrants new consideration? The reason we implemented DIRMC is so that when a user restores a file(s) there are not extra tape mounts to restore the directories We ran into this on multiple occasions, even when all files were on disk, tape mounts would occur because the directories were on tape. Thanks, Tim Rusforth City of Winnipeg -Original Message- From: TSM_User [mailto:[EMAIL PROTECTED] Sent: Wed 3/16/2005 6:48 PM To: ADSM-L@VM.MARIST.EDU Cc: Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed but the reason there have been suggestions to use a file type device class is because disk pools unline sequential pools are scanned from begining to end for every storage pool backup. I have had some customers that have millions of directories in their DIRMC pool. Even when none change they backup runs from hours on that pool. With a file type device class only the new volumes would be backed up resulting in a much faster backup. Now all that being said this new feature in V5.3 warrents new consideration. My new consideration is to stop using DIRMC pools as the reason they were created in the first place has also long been fixed. Kyle "Thorneycroft, Doug" <[EMAIL PROTECTED]> wrote: OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe. - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
It is fixed but the reason there have been suggestions to use a file type device class is because disk pools unline sequential pools are scanned from begining to end for every storage pool backup. I have had some customers that have millions of directories in their DIRMC pool. Even when none change they backup runs from hours on that pool. With a file type device class only the new volumes would be backed up resulting in a much faster backup. Now all that being said this new feature in V5.3 warrents new consideration. My new consideration is to stop using DIRMC pools as the reason they were created in the first place has also long been fixed. Kyle "Thorneycroft, Doug" <[EMAIL PROTECTED]> wrote: OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe. - Do you Yahoo!? Yahoo! Small Business - Try our new resources site!
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
Thanks Tim Doug Thorneycroft Systems Analyst Computer Technology Section County Sanitation Districts of Los Angeles County (562) 699-7411 Ext. 1058 FAX (562) 699-6756 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Wednesday, March 16, 2005 2:31 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: DIRMC - Are copypool reclamation performance issues resolved or not. It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe.
Re: DIRMC - Are copypool reclamation performance issues resolved or not.
It is fixed (somewhere around 5.1.5.2). -Original Message- From: Thorneycroft, Doug [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 16, 2005 4:25 PM To: ADSM-L@VM.MARIST.EDU Subject: DIRMC - Are copypool reclamation performance issues resolved or not. OK, after spending a large portion of my day reviewing adsm-l post going back to 2000, I'm still not sure. Does anyone know if there is still a performance problem running reclamation on a DIRMC random access disk pool? I came across one post that said it was supposedly fixed, but recommended using a file type diskpool to be safe.
