Re: D2D on AIX
Our server was collocated by node and we did have resource utilization set to 10 with the mount limit set to 10 so it used as many drives as it could. Lets get back on track. What I am saying is the seak on a tape costs time. Lots of time. If you have a file server that has a high change rate then it will take longer to restore it a year after you start to back it up then the day after. With tape it will be much longer with disk not so much. We have run a number of tests and we have seen time and time again that the fragmentation of a backup running for a year has a very small effect on disk but a large effect on tape. Daniel Sparrman <[EMAIL PROTECTED]> wrote: Hi No, the restore test was not done with "new" data. The server had been backing up data for several months, and utilized a total of 63 9840B cartridges.However, mount time isnt an issue when having the possibility of using multi-mount, multi-session restore. Its an even smaller issue when having a tape library and tape drives that have a small time-to-data. However, a large fileserver (+300GB) should never be non-collocated. This will, as you say, spread the data over a non-acceptable number of tapes. This however doesnt have to do with tape vs disk performance, rather TSM design issues. A good design will always give you a good performance from tape. The problem is, as servers are growing, no actions are taken to withold good restore performance from the tape drives. Designing the TSM system to have bad performance on either disk or tape isnt that hard. The challenge is to design the TSM system with good performance on both disk aswell as tape. Mvh // Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Propellervdgen 6B 183 62 TDBY Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 TSM_User Sent by: "ADSM: Dist Stor Manager" 2004-09-22 19:16 Please respond to "ADSM: Dist Stor Manager" To [EMAIL PROTECTED] cc Subject Re: D2D on AIX These numbers are from STK 9840B drives. I am not talking about a backup and then a restore. I am talking about a daily backup for 8 months and then a restore. File fragemenation dramatically effects your througput over time. Sure we can spin 9840 drives to 100 GB/hr for large files. I have even backed up a file server to 38 GB/hr (throgh 1 GB NIC's) with millions of small files. But over time the speed is effected. It's being unfair it is what it is. For that restore test was it right after a the backup was done? What is the change rate of the data. If it was after 8 months and you still got 40 GB/hr throughput then good deal. I haven't seen that. Daniel Sparrman wrote: Hi Comparing these types of numbers are abit unfair. We have customers running 9840 and LTO-2. They have alot higher throughput than 8-12GB/hour over a GB nic. For example, we have a customer running Netware. The TSM server is an AIX server(pSeries 615) connected to a 3584-L32 library with 3 LTO-2 drives. The Netware server has about 200GB of data. The AIX server has three 100Mbs nic, bundled togheter in an Etherchannel interface(theoretic speed is 300Mbs or 30MB/s). The netware server is connected through 100Mbs ethernet(single adapter). The server have a restore time of about 5= hours which means we have an hourly throughput of almost 40GB/hour. Average networkspeed is 11MB/s. The Netware server utilizes multi-session restore, which means it can mount multiple volumes at once for restores. We have another customer running a pSeries 650 clustrer. The cluster is attached to a 3584-L32 library with 9 LTO-2 drives. The pSeries server is equipped with an Etherchannel interface which consists of 2 GB nics. During testing of a restore scenario on one of their Lotus Domino servers(300GB of data), they reached about 50MB/s restoring directly from tape. In this case, we didnt utilize multi-session restore, which meant that the single LTO-2 drive could deliver 180GB/hour. Today, the new tape technologies can easily outrun disks. To match LTO-2 drives against disks, you'll ned large, fiber-attached disk subsystems, with no other load than the TSM server load. Internal SCSI-disks can never outrun fiber-attached LTO-2 drives. The LTO-2 drive has a native speed of 35MB/s, compressed around 50-70MB/s depending on the type of data. They also have dynamic speed, which means you dont get the back-hitch as long as you keep writing data with at least 15MB/s. We've seen theese drives push up to 90MB/s on database backups and restores. During the testing phase of the implementation, we had up to 380MB/s from the disks(two mirrored FAStT900 connected through 4 FC HBA:s with 34 15K 36.4GB fiber disks per FAStT system) and almost 650MB/s from the drives(9 LTO-2 drives connected through 4 FC HBA:s). The speed of the drives is all about design. If you attach a large number of drives to a single FC HBA, you'll easily get back-hitch. With the LTO
Re: D2D on AIX
Hi No, the restore test was not done with "new" data. The server had been backing up data for several months, and utilized a total of 63 9840B cartridges.However, mount time isnt an issue when having the possibility of using multi-mount, multi-session restore. Its an even smaller issue when having a tape library and tape drives that have a small time-to-data. However, a large fileserver (+300GB) should never be non-collocated. This will, as you say, spread the data over a non-acceptable number of tapes. This however doesnt have to do with tape vs disk performance, rather TSM design issues. A good design will always give you a good performance from tape. The problem is, as servers are growing, no actions are taken to withold good restore performance from the tape drives. Designing the TSM system to have bad performance on either disk or tape isnt that hard. The challenge is to design the TSM system with good performance on both disk aswell as tape. Mvh // Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Propellervägen 6B 183 62 TÄBY Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 TSM_User <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> 2004-09-22 19:16 Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To [EMAIL PROTECTED] cc Subject Re: D2D on AIX These numbers are from STK 9840B drives. I am not talking about a backup and then a restore. I am talking about a daily backup for 8 months and then a restore. File fragemenation dramatically effects your througput over time. Sure we can spin 9840 drives to 100 GB/hr for large files. I have even backed up a file server to 38 GB/hr (throgh 1 GB NIC's) with millions of small files. But over time the speed is effected. It's being unfair it is what it is. For that restore test was it right after a the backup was done? What is the change rate of the data. If it was after 8 months and you still got 40 GB/hr throughput then good deal. I haven't seen that. Daniel Sparrman <[EMAIL PROTECTED]> wrote: Hi Comparing these types of numbers are abit unfair. We have customers running 9840 and LTO-2. They have alot higher throughput than 8-12GB/hour over a GB nic. For example, we have a customer running Netware. The TSM server is an AIX server(pSeries 615) connected to a 3584-L32 library with 3 LTO-2 drives. The Netware server has about 200GB of data. The AIX server has three 100Mbs nic, bundled togheter in an Etherchannel interface(theoretic speed is 300Mbs or 30MB/s). The netware server is connected through 100Mbs ethernet(single adapter). The server have a restore time of about 5= hours which means we have an hourly throughput of almost 40GB/hour. Average networkspeed is 11MB/s. The Netware server utilizes multi-session restore, which means it can mount multiple volumes at once for restores. We have another customer running a pSeries 650 clustrer. The cluster is attached to a 3584-L32 library with 9 LTO-2 drives. The pSeries server is equipped with an Etherchannel interface which consists of 2 GB nics. During testing of a restore scenario on one of their Lotus Domino servers(300GB of data), they reached about 50MB/s restoring directly from tape. In this case, we didnt utilize multi-session restore, which meant that the single LTO-2 drive could deliver 180GB/hour. Today, the new tape technologies can easily outrun disks. To match LTO-2 drives against disks, you'll ned large, fiber-attached disk subsystems, with no other load than the TSM server load. Internal SCSI-disks can never outrun fiber-attached LTO-2 drives. The LTO-2 drive has a native speed of 35MB/s, compressed around 50-70MB/s depending on the type of data. They also have dynamic speed, which means you dont get the back-hitch as long as you keep writing data with at least 15MB/s. We've seen theese drives push up to 90MB/s on database backups and restores. During the testing phase of the implementation, we had up to 380MB/s from the disks(two mirrored FAStT900 connected through 4 FC HBA:s with 34 15K 36.4GB fiber disks per FAStT system) and almost 650MB/s from the drives(9 LTO-2 drives connected through 4 FC HBA:s). The speed of the drives is all about design. If you attach a large number of drives to a single FC HBA, you'll easily get back-hitch. With the LTO-2 drives, a fair number of drives/adapter is around 3-4 / adapter. Designing disk to match the tape drives is all about cost. S-ATA drives can never outrun LTO-2 drives, at least not when it comes to large files or database backups and restores. Designing FC disks to match the drives will mean the cost is 10 times the cost of the tape drives. This is all my opinion, and I'm sure that there are others out there that dont agree. Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Propellervdg
Re: Client Compression (was D2D on AIX)
I've done some tests in the past (but I have to search for my results ...). Note that these are with 100 Mbs Ethernet. Recent ones: Exchange TDP Exchange DB compressed 50.3 GB - 1:19:30 elapsed time, 10.81 MB/sec (Backup) Exchange DB uncompressed 63.1 GB - 1:36:30 elapsed time, 11.15 MB/sec (Backup) We backup Oracle directly with the b/a client, (No TDP) and always get huge compressions rates (88% compression). This was a multi-session backup test a while ago: (Again on 100 Mbs Ethernet), 9.9GB of source data (compressed to 1.19GB) elapsed time of 152 seconds - this translates to 65 MB/sec which we could not achieve with 100 Mbs Ethernet without the compression. (We tend to max out on CPU on these types of backups). With client compression on backups you have to make sure no files are growing during compression and being resent - this will add time to your backups. We exclude files like .zip etc from compression and also scan the client logs for any files that grow during compression (you can also use compressalways to avoid the resends). I'm going to search for other tests I've done (or do some again) and I'll post those results. Tim Rushforth City of Winnipeg -Original Message- From: TSM_User [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 22, 2004 12:19 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX This is true, I was just wondering if others were seeing the same thing. We expected it to take longer but not two to three times as long. In the end after compression there would be less data to transfer so we thought there would be some gain there. Richard Rhodes <[EMAIL PROTECTED]> wrote: >Tim, we recently ran a bunch of tests on client side compression. >In every test the backup ran for 2 to 3 times longer. In some cases >this wouldn't be a big deal when you look at the backup alone being >incremental and all. However, we also believed that it would also >cause the restore to run 2 to 3 times as long to uncompress the data. >As a result of these tests and thoughts we decided not to >implement client side compression. Uncompress should be much faster and less cpu intensive than compression. In compression you are searching for redundant tokens. In uncompression you are basically performing token substitution. Rick - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: D2D on AIX
This is true, I was just wondering if others were seeing the same thing. We expected it to take longer but not two to three times as long. In the end after compression there would be less data to transfer so we thought there would be some gain there. Richard Rhodes <[EMAIL PROTECTED]> wrote: >Tim, we recently ran a bunch of tests on client side compression. >In every test the backup ran for 2 to 3 times longer. In some cases >this wouldn't be a big deal when you look at the backup alone being >incremental and all. However, we also believed that it would also >cause the restore to run 2 to 3 times as long to uncompress the data. >As a result of these tests and thoughts we decided not to >implement client side compression. Uncompress should be much faster and less cpu intensive than compression. In compression you are searching for redundant tokens. In uncompression you are basically performing token substitution. Rick - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: D2D on AIX
These numbers are from STK 9840B drives. I am not talking about a backup and then a restore. I am talking about a daily backup for 8 months and then a restore. File fragemenation dramatically effects your througput over time. Sure we can spin 9840 drives to 100 GB/hr for large files. I have even backed up a file server to 38 GB/hr (throgh 1 GB NIC's) with millions of small files. But over time the speed is effected. It's being unfair it is what it is. For that restore test was it right after a the backup was done? What is the change rate of the data. If it was after 8 months and you still got 40 GB/hr throughput then good deal. I haven't seen that. Daniel Sparrman <[EMAIL PROTECTED]> wrote: Hi Comparing these types of numbers are abit unfair. We have customers running 9840 and LTO-2. They have alot higher throughput than 8-12GB/hour over a GB nic. For example, we have a customer running Netware. The TSM server is an AIX server(pSeries 615) connected to a 3584-L32 library with 3 LTO-2 drives. The Netware server has about 200GB of data. The AIX server has three 100Mbs nic, bundled togheter in an Etherchannel interface(theoretic speed is 300Mbs or 30MB/s). The netware server is connected through 100Mbs ethernet(single adapter). The server have a restore time of about 5= hours which means we have an hourly throughput of almost 40GB/hour. Average networkspeed is 11MB/s. The Netware server utilizes multi-session restore, which means it can mount multiple volumes at once for restores. We have another customer running a pSeries 650 clustrer. The cluster is attached to a 3584-L32 library with 9 LTO-2 drives. The pSeries server is equipped with an Etherchannel interface which consists of 2 GB nics. During testing of a restore scenario on one of their Lotus Domino servers(300GB of data), they reached about 50MB/s restoring directly from tape. In this case, we didnt utilize multi-session restore, which meant that the single LTO-2 drive could deliver 180GB/hour. Today, the new tape technologies can easily outrun disks. To match LTO-2 drives against disks, you'll ned large, fiber-attached disk subsystems, with no other load than the TSM server load. Internal SCSI-disks can never outrun fiber-attached LTO-2 drives. The LTO-2 drive has a native speed of 35MB/s, compressed around 50-70MB/s depending on the type of data. They also have dynamic speed, which means you dont get the back-hitch as long as you keep writing data with at least 15MB/s. We've seen theese drives push up to 90MB/s on database backups and restores. During the testing phase of the implementation, we had up to 380MB/s from the disks(two mirrored FAStT900 connected through 4 FC HBA:s with 34 15K 36.4GB fiber disks per FAStT system) and almost 650MB/s from the drives(9 LTO-2 drives connected through 4 FC HBA:s). The speed of the drives is all about design. If you attach a large number of drives to a single FC HBA, you'll easily get back-hitch. With the LTO-2 drives, a fair number of drives/adapter is around 3-4 / adapter. Designing disk to match the tape drives is all about cost. S-ATA drives can never outrun LTO-2 drives, at least not when it comes to large files or database backups and restores. Designing FC disks to match the drives will mean the cost is 10 times the cost of the tape drives. This is all my opinion, and I'm sure that there are others out there that dont agree. Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Propellervdgen 6B 183 62 TDBY Vdxel: 08 - 754 98 00 Mobil: 070 - 399 27 51 TSM_User Sent by: "ADSM: Dist Stor Manager" 2004-09-22 04:27 Please respond to "ADSM: Dist Stor Manager" To [EMAIL PROTECTED] cc Subject Re: D2D on AIX Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The data was fragemented on tape for a little over a year for the first test. The data was fragmented over disk for nearly 8 months. Steve Harris wrote: How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape
Re: D2D on AIX
>Tim, we recently ran a bunch of tests on client side compression. >In every test the backup ran for 2 to 3 times longer. In some cases >this wouldn't be a big deal when you look at the backup alone being >incremental and all. However, we also believed that it would also >cause the restore to run 2 to 3 times as long to uncompress the data. >As a result of these tests and thoughts we decided not to >implement client side compression. Uncompress should be much faster and less cpu intensive than compression. In compression you are searching for redundant tokens. In uncompression you are basically performing token substitution. Rick - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: D2D on AIX
Could someone please email me the presentation as well? Or send me the link? I just spent 20 minutes on IBM's website and couldn't find it. Thanks! -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Robert HECKO Sent: Wednesday, September 22, 2004 4:14 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Hello Can you send this presentation also to me ? thank you. best regards Robert Hecko - Original Message - From: "Johnson, Milton" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, September 20, 2004 7:57 PM Subject: Re: D2D on AIX > It depends upon how you configure things. For dynamic allocation of > volumes, then yes you are limited to the size of the file system that > you mount on that mount point. However if you define the stgpool > volumes explicitly using the DEFINE VOLUME command, you can place the > volumes across as many file systems as you want. I will email you a > PDF presentation IBM has on Disk Only backups. > > > H. Milton Johnson > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf > Of Eliza Lau > Sent: Monday, September 20, 2004 12:11 PM > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are > looking into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has > a max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] >
Re: D2D on AIX
==> In article <[EMAIL PROTECTED]>, Eliza Lau <[EMAIL PROTECTED]> writes: > True. Seek time is tiny compared to tape mounts. I am just concerned that > the TSM db has to keep track of thousands of volume. How much will it increase > the size of the db. Ours is already 90G at 70% utilized. It's always a good idea to keep the DB size in the back of your mind; but my take is that you probably don't need to think too hard about adding something with size 'thousands' to it. I started to feel that I had too many volumes at one point when I had remote server volumes at 2G, and I had several hundred thousands of them. To help yourself feel more comfortable with this, I suggest that you take a 'Q occ' of a few nodes you consider "small", "moderate", and "large". Count the total number of objects, and compare. I find that even my small nodes have hundreds of thousands of objects. I don't think that the db size per object is particularly close to the size per volume (i.e. the per-volume overhead is probably much much less) , but you can get a taste of the general order of how big is big. - Allen S. Rout
Re: D2D on AIX
Hi Comparing these types of numbers are abit unfair. We have customers running 9840 and LTO-2. They have alot higher throughput than 8-12GB/hour over a GB nic. For example, we have a customer running Netware. The TSM server is an AIX server(pSeries 615) connected to a 3584-L32 library with 3 LTO-2 drives. The Netware server has about 200GB of data. The AIX server has three 100Mbs nic, bundled togheter in an Etherchannel interface(theoretic speed is 300Mbs or 30MB/s). The netware server is connected through 100Mbs ethernet(single adapter). The server have a restore time of about 5½ hours which means we have an hourly throughput of almost 40GB/hour. Average networkspeed is 11MB/s. The Netware server utilizes multi-session restore, which means it can mount multiple volumes at once for restores. We have another customer running a pSeries 650 clustrer. The cluster is attached to a 3584-L32 library with 9 LTO-2 drives. The pSeries server is equipped with an Etherchannel interface which consists of 2 GB nics. During testing of a restore scenario on one of their Lotus Domino servers(300GB of data), they reached about 50MB/s restoring directly from tape. In this case, we didnt utilize multi-session restore, which meant that the single LTO-2 drive could deliver 180GB/hour. Today, the new tape technologies can easily outrun disks. To match LTO-2 drives against disks, you'll ned large, fiber-attached disk subsystems, with no other load than the TSM server load. Internal SCSI-disks can never outrun fiber-attached LTO-2 drives. The LTO-2 drive has a native speed of 35MB/s, compressed around 50-70MB/s depending on the type of data. They also have dynamic speed, which means you dont get the back-hitch as long as you keep writing data with at least 15MB/s. We've seen theese drives push up to 90MB/s on database backups and restores. During the testing phase of the implementation, we had up to 380MB/s from the disks(two mirrored FAStT900 connected through 4 FC HBA:s with 34 15K 36.4GB fiber disks per FAStT system) and almost 650MB/s from the drives(9 LTO-2 drives connected through 4 FC HBA:s). The speed of the drives is all about design. If you attach a large number of drives to a single FC HBA, you'll easily get back-hitch. With the LTO-2 drives, a fair number of drives/adapter is around 3-4 / adapter. Designing disk to match the tape drives is all about cost. S-ATA drives can never outrun LTO-2 drives, at least not when it comes to large files or database backups and restores. Designing FC disks to match the drives will mean the cost is 10 times the cost of the tape drives. This is all my opinion, and I'm sure that there are others out there that dont agree. Best Regards Daniel Sparrman --- Daniel Sparrman Exist i Stockholm AB Propellervägen 6B 183 62 TÄBY Växel: 08 - 754 98 00 Mobil: 070 - 399 27 51 TSM_User <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> 2004-09-22 04:27 Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To [EMAIL PROTECTED] cc Subject Re: D2D on AIX Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The data was fragemented on tape for a little over a year for the first test. The data was fragmented over disk for nearly 8 months. Steve Harris <[EMAIL PROTECTED]> wrote: How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files
Re: D2D on AIX
Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The data was fragemented on tape for a little over a year for the first test. The data was fragmented over disk for nearly 8 months. Steve Harris <[EMAIL PROTECTED]> wrote: How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout > *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. *** __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: D2D on AIX
Hello Can you send this presentation also to me ? thank you. best regards Robert Hecko - Original Message - From: "Johnson, Milton" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, September 20, 2004 7:57 PM Subject: Re: D2D on AIX > It depends upon how you configure things. For dynamic allocation of > volumes, then yes you are limited to the size of the file system that > you mount on that mount point. However if you define the stgpool > volumes explicitly using the DEFINE VOLUME command, you can place the > volumes across as many file systems as you want. I will email you a PDF > presentation IBM has on Disk Only backups. > > > H. Milton Johnson > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of > Eliza Lau > Sent: Monday, September 20, 2004 12:11 PM > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are looking > into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has a > max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] >
Re: D2D on AIX
Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The data was Steve Harris <[EMAIL PROTECTED]> wrote:How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout > *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. *** - Do you Yahoo!? vote.yahoo.com - Register online to vote today!
Re: D2D on AIX
Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The d Steve Harris <[EMAIL PROTECTED]> wrote:How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout > *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. *** - Do you Yahoo!? vote.yahoo.com - Register online to vote today!
Re: D2D on AIX
Good questions. Our real world example:We went from around 8 - 12 GB/hr restore off of tape to over 40 GB/hr from the file device classes. Our test was a file server with a little over 300 GB of data. The File server and the TSM server both had 1 GB NIC's. Resource utilization was set to 10 in both cases. The data Steve Harris <[EMAIL PROTECTED]> wrote:How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout > *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. *** __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: D2D on AIX
Tim, we recently ran a bunch of tests on client side compression. In every test the backup ran for 2 to 3 times longer. In some cases this wouldn't be a big deal when you look at the backup alone being incremental and all. However, we also believed that it would also cause the restore to run 2 to 3 times as long to uncompress the data. As a result of these tests and thoughts we decided not to implement client side compression. Have you run any tests to see how compression effects your backups and restores? "Rushforth, Tim" <[EMAIL PROTECTED]> wrote: We use 5 days for reuse delay. I did a quick comparison using 25GB and 4GB volumes on our pilot with the following results: Disk Volumes - 25 GB Volumes Stored Data - 236 GB # of Disk Vols - 14 (including 2 pending volumes) Total allocation - 14 * 25GB = 350 GB 67% Utilization Disk Volumes - 4 GB Volumes (TSM2) Stored Data - 333 GB # of Disk Vols - 100 (including 12 pending volumes) Total allocation - 100 * 4GB = 400 GB 83% Utilization Reclamation was set at 25% for both of these. It would be interesting to see others peoples results. Thanks, Tim Rushforth City of Winnipeg -Original Message- From: Johnson, Milton [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 1:04 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX What do use for a reuse delay? How many pending volumes do you average? H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eliza: At the Disk only Backups Technical Exchange, IBM recommended 2-4 GB volume size. (This was stated by the presenter, it was not written on the PDF presentation.) We started with 25 GB volumes and have now switched to 4 GB volumes. Using smaller volume sizes allows a better utilization of space and increases restore performance with multi-session restore. (Also helps eliminate contention if multiple clients are restoring from the same volume) Tim Rushforth City of Winnipeg -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes > cannot be used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are > looking into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has > a max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** > - Do you Yahoo!? vote.yahoo.com - Register online to vote today!
Re: D2D on AIX
Eliza, We are using 25 GB volumes right now without any issues but we are still collocateing by node. We are evaluting the savings of using smaller volumes when we move to noncollocated storage pools. I agree that it doesn't make sense to collocate file device class pools but managment wouldn't let us change that at first. If you are talking about a Windows server then you need to think about file handles and their effect on the 256 MB of nonpagged memory. Also, the size of your MFT should be considered. Using more smaller files will have an effect on these things. I think as more and more of us implement these solutions we will have more collective knowledge from which to guide these decisions. Again, 25 GB volumes have been working well for us for nearly 8 months. We set our storage pools to relaim=30. We run expiration at 6:00 AM which starts a reclamation process soon after. We never have reclamation processes running past 8:30 AM. We have run many restores and have been extremely pleased with our speeds, much faster than tape for servers with many small files. "Rushforth, Tim" <[EMAIL PROTECTED]> wrote: Eliza: At the Disk only Backups Technical Exchange, IBM recommended 2-4 GB volume size. (This was stated by the presenter, it was not written on the PDF presentation.) We started with 25 GB volumes and have now switched to 4 GB volumes. Using smaller volume sizes allows a better utilization of space and increases restore performance with multi-session restore. (Also helps eliminate contention if multiple clients are restoring from the same volume) Tim Rushforth City of Winnipeg -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes cannot be > used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding > another frame or upgrading to 3590H or 3592 tapes we are looking into > setting > up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 > has a max file system size of 1TB. Does it mean the largest stgpool > I can define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** > - Do you Yahoo!? vote.yahoo.com - Register online to vote today!
Re: D2D on AIX
How does TSM access the data on file volumes? Does it keep an offset of the start of every file or aggregate? If it does, then yes we could skip to the start of each file or aggregate. If it does not, then we need to read through the volume to find the file we are going to restore. Where we have a large number of concurrent restores happening, this could cause performance issues on the array. Now TSM has some smarts on later technology tape drives that have block addressability and on-cartridge memory and can find a spot on the tape quickly, but does this translate to file volumes? Regards Steve. >>> [EMAIL PROTECTED] 22/09/2004 4:49:55 >>> True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau <[EMAIL PROTECTED]> writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout > *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. ***
Re: D2D on AIX
True. Seek time is tiny compared to tape mounts. I am just concerned that the TSM db has to keep track of thousands of volume. How much will it increase the size of the db. Ours is already 90G at 70% utilized. Eliza > > ==> In article <[EMAIL PROTECTED]>, Eliza Lau <[EMAIL PROTECTED]> writes: > > > What is the recommended volume size. I have seen someone mentioned 5G, but > > then the number of volumes will explode from about 800 (current # of 3590 > > primary tapes) to thousands. > > Consider, this doesn't really cost you much. Seek time in a directory of > thousands of files is still tiny compared to tape behavior. > > I probably wouldn't go as low as 5G, but 10G (much less than the average size > of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my > perspective. > > > > > How about keeping the staging space so clients backup to staging then > > migrate to FILE volumes. Then every volume will be filled up. > > > I like this, too. > > > - Allen S. Rout >
Re: D2D on AIX
==> In article <[EMAIL PROTECTED]>, Eliza Lau <[EMAIL PROTECTED]> writes: > What is the recommended volume size. I have seen someone mentioned 5G, but > then the number of volumes will explode from about 800 (current # of 3590 > primary tapes) to thousands. Consider, this doesn't really cost you much. Seek time in a directory of thousands of files is still tiny compared to tape behavior. I probably wouldn't go as low as 5G, but 10G (much less than the average size of my 3590 vols) seems pretty reasonable to me. 20G is getting big, from my perspective. > How about keeping the staging space so clients backup to staging then > migrate to FILE volumes. Then every volume will be filled up. I like this, too. - Allen S. Rout
Re: D2D on AIX
We use 5 days for reuse delay. I did a quick comparison using 25GB and 4GB volumes on our pilot with the following results: Disk Volumes - 25 GB Volumes Stored Data - 236 GB # of Disk Vols - 14 (including 2 pending volumes) Total allocation - 14 * 25GB = 350 GB 67% Utilization Disk Volumes - 4 GB Volumes (TSM2) Stored Data - 333 GB # of Disk Vols - 100 (including 12 pending volumes) Total allocation - 100 * 4GB = 400 GB 83% Utilization Reclamation was set at 25% for both of these. It would be interesting to see others peoples results. Thanks, Tim Rushforth City of Winnipeg -Original Message- From: Johnson, Milton [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 1:04 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX What do use for a reuse delay? How many pending volumes do you average? H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eliza: At the Disk only Backups Technical Exchange, IBM recommended 2-4 GB volume size. (This was stated by the presenter, it was not written on the PDF presentation.) We started with 25 GB volumes and have now switched to 4 GB volumes. Using smaller volume sizes allows a better utilization of space and increases restore performance with multi-session restore. (Also helps eliminate contention if multiple clients are restoring from the same volume) Tim Rushforth City of Winnipeg -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes > cannot be used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are > looking into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has > a max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** >
Re: D2D on AIX
What do use for a reuse delay? How many pending volumes do you average? H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eliza: At the Disk only Backups Technical Exchange, IBM recommended 2-4 GB volume size. (This was stated by the presenter, it was not written on the PDF presentation.) We started with 25 GB volumes and have now switched to 4 GB volumes. Using smaller volume sizes allows a better utilization of space and increases restore performance with multi-session restore. (Also helps eliminate contention if multiple clients are restoring from the same volume) Tim Rushforth City of Winnipeg -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes > cannot be used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are > looking into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has > a max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** >
Re: D2D on AIX
Now we get into religion. IBM did offer a figure of ~5GB during the webinar, but there are a lot of factors that would affect this such as: REUSE DELAY: you want to be able to use those TSM DB backups RECLAMATION THRESHOLD: A lower threshold should lead to more efficient usage of volumes except that it causes more frequent tape reclamation leading to more pending volumes causing wasted space. Of course the exact opposite is true regarding higher reclamation thresholds. What yin yang is right for you? Experiment and find out. AVG SIZE OF STORED OBJECTS? EXPIRATION RATE OF STORED OBJECTS? I'm sure others will bring up other factors. How many volumes are too many? If TSM is keeping track of the volumes and you are not handling the physical volumes (i.e. loading/unloading tapes), is 4,000 too many? If so, why? H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Eliza Lau Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes > cannot be used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are > looking into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has > a max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** >
Re: D2D on AIX
Eliza: At the Disk only Backups Technical Exchange, IBM recommended 2-4 GB volume size. (This was stated by the presenter, it was not written on the PDF presentation.) We started with 25 GB volumes and have now switched to 4 GB volumes. Using smaller volume sizes allows a better utilization of space and increases restore performance with multi-session restore. (Also helps eliminate contention if multiple clients are restoring from the same volume) Tim Rushforth City of Winnipeg -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:20 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes cannot be > used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding > another frame or upgrading to 3590H or 3592 tapes we are looking into > setting > up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 > has a max file system size of 1TB. Does it mean the largest stgpool > I can define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. > ** >
Re: D2D on AIX
I haven't tried this setup but that makes sense. So you would ensure MAXNUMP would be equal to the number of backup sessions you wanted. (You want to update MAXNUMP for restores anyway when restoring from file device class.) I asked the presenter of the Disk Only Backup Technical Exchange about collocating file stgpools and he said it made no sense. I tend to agree. Tim Rushforth City of Winnipeg -Original Message- From: Bill Boyer [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 12:34 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes, you would still get the multi-session backup, but only to the limit of your MAXNUMMP,right? What if you're running the FILE stgpool as collocated? -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 12:36 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes that is basically what we are doing. We are doing it more for fault tolerance. Our file storage pool is on a different disk subsystem. If there were major problems with that we could still do the nightly backups. Note that if you went directly to a File storage pool you would get multi-session backup also - you don't need a DISK pool for that. -Original Message- From: Bill Boyer [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:09 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX What about combining both worlds...have the DISK storage pool for your daily backups to get the multi-session backups and faster backups, then migrate to a FILE storage pool for retention. Now you'll get the multi-session restore, less overhead than the large DISK pool, but still have to do reclamation. Just substitute FILE for the onsite TAPE. Bill Boyer DSS, Inc.
Re: D2D on AIX
Yes, you would still get the multi-session backup, but only to the limit of your MAXNUMMP,right? What if you're running the FILE stgpool as collocated? -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 12:36 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes that is basically what we are doing. We are doing it more for fault tolerance. Our file storage pool is on a different disk subsystem. If there were major problems with that we could still do the nightly backups. Note that if you went directly to a File storage pool you would get multi-session backup also - you don't need a DISK pool for that. -Original Message- From: Bill Boyer [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:09 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX What about combining both worlds...have the DISK storage pool for your daily backups to get the multi-session backups and faster backups, then migrate to a FILE storage pool for retention. Now you'll get the multi-session restore, less overhead than the large DISK pool, but still have to do reclamation. Just substitute FILE for the onsite TAPE. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 11:16 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes, try again. If it works it is a bug (don't tell IBM)! If data is on a DISK device class and Tape (or file device class) you can have 1 session from disk and other sessions from tape. -Original Message- From: Stapleton, Mark [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 9:51 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim >And with DISK device class there is no multi-session restore. Are you sure? I seem to recall using RESOURCEUTILIZATION to run a multi-threaded restore or two from DISKPOOL. -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks Office 262.521.5627
Re: D2D on AIX
Yes that is basically what we are doing. We are doing it more for fault tolerance. Our file storage pool is on a different disk subsystem. If there were major problems with that we could still do the nightly backups. Note that if you went directly to a File storage pool you would get multi-session backup also - you don't need a DISK pool for that. -Original Message- From: Bill Boyer [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 11:09 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX What about combining both worlds...have the DISK storage pool for your daily backups to get the multi-session backups and faster backups, then migrate to a FILE storage pool for retention. Now you'll get the multi-session restore, less overhead than the large DISK pool, but still have to do reclamation. Just substitute FILE for the onsite TAPE. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 11:16 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes, try again. If it works it is a bug (don't tell IBM)! If data is on a DISK device class and Tape (or file device class) you can have 1 session from disk and other sessions from tape. -Original Message- From: Stapleton, Mark [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 9:51 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim >And with DISK device class there is no multi-session restore. Are you sure? I seem to recall using RESOURCEUTILIZATION to run a multi-threaded restore or two from DISKPOOL. -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks Office 262.521.5627
Re: D2D on AIX
Eric, What is the recommended volume size. I have seen someone mentioned 5G, but then the number of volumes will explode from about 800 (current # of 3590 primary tapes) to thousands. How about keeping the staging space so clients backup to staging then migrate to FILE volumes. Then every volume will be filled up. Eliza > > Hi Eliza! > You do want several smaller files, rather than a few very large files > because each client session will allocate a volume. File volumes cannot be > used concurrently by more than one session. > Kindest regards, > Eric van Loon > KLM Royal Dutch Airlines > > > -Original Message- > From: Eliza Lau [mailto:[EMAIL PROTECTED] > Sent: Monday, September 20, 2004 19:11 > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding > another frame or upgrading to 3590H or 3592 tapes we are looking into > setting > up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 > has a max file system size of 1TB. Does it mean the largest stgpool > I can define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > > > ** > For information, services and offers, please visit our web site: http://www.klm.com. > This e-mail and any attachment may contain confidential and privileged material > intended for the addressee only. If you are not the addressee, you are notified that > no part of the e-mail or any attachment may be disclosed, copied or distributed, and > that any other action related to this e-mail or attachment is strictly prohibited, > and may be unlawful. If you have received this e-mail by error, please notify the > sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart > Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for > the incorrect or incomplete transmission of this e-mail or any attachments, nor > responsible for any delay in receipt. > ** >
Re: D2D on AIX
What about combining both worlds...have the DISK storage pool for your daily backups to get the multi-session backups and faster backups, then migrate to a FILE storage pool for retention. Now you'll get the multi-session restore, less overhead than the large DISK pool, but still have to do reclamation. Just substitute FILE for the onsite TAPE. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Rushforth, Tim Sent: Tuesday, September 21, 2004 11:16 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Yes, try again. If it works it is a bug (don't tell IBM)! If data is on a DISK device class and Tape (or file device class) you can have 1 session from disk and other sessions from tape. -Original Message- From: Stapleton, Mark [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 9:51 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim >And with DISK device class there is no multi-session restore. Are you sure? I seem to recall using RESOURCEUTILIZATION to run a multi-threaded restore or two from DISKPOOL. -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks Office 262.521.5627
Re: D2D on AIX
Yes, try again. If it works it is a bug (don't tell IBM)! If data is on a DISK device class and Tape (or file device class) you can have 1 session from disk and other sessions from tape. -Original Message- From: Stapleton, Mark [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 9:51 AM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim >And with DISK device class there is no multi-session restore. Are you sure? I seem to recall using RESOURCEUTILIZATION to run a multi-threaded restore or two from DISKPOOL. -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks Office 262.521.5627
Re: D2D on AIX
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Rushforth, Tim >And with DISK device class there is no multi-session restore. Are you sure? I seem to recall using RESOURCEUTILIZATION to run a multi-threaded restore or two from DISKPOOL. -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks Office 262.521.5627
Re: D2D on AIX
And with DISK device class there is no multi-session restore. -Original Message- From: Ian Hobbs [mailto:[EMAIL PROTECTED] Sent: Monday, September 20, 2004 5:26 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Question, Why not use the DISK device class with RAW volumes? Personally, I find FILE classes a pain for user storage because you DO have to perform reclamation on them. Ian Hobbs On Mon, 20 Sep 2004 14:48:07 -0400, Eliza Lau wrote: >Okay. I got it. It is a pain to manually define the volumes, but it can be >done. I also received your pdf file. > >Thanks to everyone who answered, >Eliza > >> >> It depends upon how you configure things. For dynamic allocation of >> volumes, then yes you are limited to the size of the file system that >> you mount on that mount point. However if you define the stgpool >> volumes explicitly using the DEFINE VOLUME command, you can place the >> volumes across as many file systems as you want. I will email you a PDF >> presentation IBM has on Disk Only backups. >> >> >> H. Milton Johnson >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of >> Eliza Lau >> Sent: Monday, September 20, 2004 12:11 PM >> To: [EMAIL PROTECTED] >> Subject: D2D on AIX >> >> Our 3494 with 3590K tapes in 3 frames is getting full. Instead of >> adding another frame or upgrading to 3590H or 3592 tapes we are looking >> into setting up a bunch of cheap ATA disks as primary storage. >> >> The FILE devclass defines a directory as its destination and JFS2 has a >> max file system size of 1TB. Does it mean the largest stgpool I can >> define is 1TB? >> >> My Exchange stgpool alone has 8TB of data. Do I have to split it up >> into 8 pieces? >> >> server: TSM 5.2.2.5 on AIX 5.2 >> database 90GB at 70% >> Total backup data - 22TB >> >> Eliza Lau >> Virginia Tech Computing Center >> [EMAIL PROTECTED] >> >> Ian Hobbs [EMAIL PROTECTED] === "Never argue with an idiot. They drag you down to their level then beat you with experience." -Dilbert
Re: D2D on AIX
IBM gave a webinar on DISK ONLY backups including the advantages of DISK vs. FILE device classes. While your mileage may vary, in general it seems that a FILE devclass will give better performance for large pools (read TB not GB). Two quick examples: 1) With DISK TSM keeps track of each 4K block in the DISK volumes. This means that TSM must maintain a map of all those blocks and search/update that map every time a file is saved/expired. Also your files will be fragmented within those DISK volumes leading to further performance problems. 2) When it's time to backup what's on disk to offsite tapes, TSM has a speedy shortcut with SEQUENTIAL device classes. TSM keeps a flag for each sequential volume, when the sequential volume is backed the flag is set, and when the sequential volume is written to the flag is cleared. This means that when it's time to back-up those primary stgpool sequential volumes to a copypool TSM only needs to examine those files in the sequential volumes with the flag cleared. With the way TSM works, this greatly reduces the amount of time required by TSM to determine what data needs to be backed up. With a DISK device class, TSM has no choice but to examine each file in the STGPOOL being backed up to determine if it has been previously backed-up to the copypool. Incidentally, IBM hinted that a future enhancement would be to allow a list of mount points (directories) to be assigned as the destination to FILE device classes. This would allow utilization of dynamic allocation across multiple file systems. Of course one drawback with dynamic allocation is that fragmentation can occur overtime. Your particular OS will greatly influence the severity of this problem, however defining the stgpool volume explicitly will prevent that problem. H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Ian Hobbs Sent: Monday, September 20, 2004 5:26 PM To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Question, Why not use the DISK device class with RAW volumes? Personally, I find FILE classes a pain for user storage because you DO have to perform reclamation on them. Ian Hobbs On Mon, 20 Sep 2004 14:48:07 -0400, Eliza Lau wrote: >Okay. I got it. It is a pain to manually define the volumes, but it >can be done. I also received your pdf file. > >Thanks to everyone who answered, >Eliza > >> >> It depends upon how you configure things. For dynamic allocation of >> volumes, then yes you are limited to the size of the file system that >> you mount on that mount point. However if you define the stgpool >> volumes explicitly using the DEFINE VOLUME command, you can place the >> volumes across as many file systems as you want. I will email you a >> PDF presentation IBM has on Disk Only backups. >> >> >> H. Milton Johnson >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf >> Of Eliza Lau >> Sent: Monday, September 20, 2004 12:11 PM >> To: [EMAIL PROTECTED] >> Subject: D2D on AIX >> >> Our 3494 with 3590K tapes in 3 frames is getting full. Instead of >> adding another frame or upgrading to 3590H or 3592 tapes we are >> looking into setting up a bunch of cheap ATA disks as primary storage. >> >> The FILE devclass defines a directory as its destination and JFS2 has >> a max file system size of 1TB. Does it mean the largest stgpool I >> can define is 1TB? >> >> My Exchange stgpool alone has 8TB of data. Do I have to split it up >> into 8 pieces? >> >> server: TSM 5.2.2.5 on AIX 5.2 >> database 90GB at 70% >> Total backup data - 22TB >> >> Eliza Lau >> Virginia Tech Computing Center >> [EMAIL PROTECTED] >> >> Ian Hobbs [EMAIL PROTECTED] === "Never argue with an idiot. They drag you down to their level then beat you with experience." -Dilbert
Re: D2D on AIX
Hi Eliza! You do want several smaller files, rather than a few very large files because each client session will allocate a volume. File volumes cannot be used concurrently by more than one session. Kindest regards, Eric van Loon KLM Royal Dutch Airlines -Original Message- From: Eliza Lau [mailto:[EMAIL PROTECTED] Sent: Monday, September 20, 2004 19:11 To: [EMAIL PROTECTED] Subject: D2D on AIX Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding another frame or upgrading to 3590H or 3592 tapes we are looking into setting up a bunch of cheap ATA disks as primary storage. The FILE devclass defines a directory as its destination and JFS2 has a max file system size of 1TB. Does it mean the largest stgpool I can define is 1TB? My Exchange stgpool alone has 8TB of data. Do I have to split it up into 8 pieces? server: TSM 5.2.2.5 on AIX 5.2 database 90GB at 70% Total backup data - 22TB Eliza Lau Virginia Tech Computing Center [EMAIL PROTECTED] ** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. **
Re: D2D on AIX
Hi Ian! Because you can't reclaim DISK volumes. TSM stores files in aggregates which will only be freed when all files within the aggregate become expired. Kindest regards, Eric van Loon KLM Royal Dutch Airlines -Original Message- From: Ian Hobbs [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 21, 2004 00:26 To: [EMAIL PROTECTED] Subject: Re: D2D on AIX Question, Why not use the DISK device class with RAW volumes? Personally, I find FILE classes a pain for user storage because you DO have to perform reclamation on them. Ian Hobbs On Mon, 20 Sep 2004 14:48:07 -0400, Eliza Lau wrote: >Okay. I got it. It is a pain to manually define the volumes, but it can be >done. I also received your pdf file. > >Thanks to everyone who answered, >Eliza > >> >> It depends upon how you configure things. For dynamic allocation of >> volumes, then yes you are limited to the size of the file system that >> you mount on that mount point. However if you define the stgpool >> volumes explicitly using the DEFINE VOLUME command, you can place the >> volumes across as many file systems as you want. I will email you a PDF >> presentation IBM has on Disk Only backups. >> >> >> H. Milton Johnson >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of >> Eliza Lau >> Sent: Monday, September 20, 2004 12:11 PM >> To: [EMAIL PROTECTED] >> Subject: D2D on AIX >> >> Our 3494 with 3590K tapes in 3 frames is getting full. Instead of >> adding another frame or upgrading to 3590H or 3592 tapes we are looking >> into setting up a bunch of cheap ATA disks as primary storage. >> >> The FILE devclass defines a directory as its destination and JFS2 has a >> max file system size of 1TB. Does it mean the largest stgpool I can >> define is 1TB? >> >> My Exchange stgpool alone has 8TB of data. Do I have to split it up >> into 8 pieces? >> >> server: TSM 5.2.2.5 on AIX 5.2 >> database 90GB at 70% >> Total backup data - 22TB >> >> Eliza Lau >> Virginia Tech Computing Center >> [EMAIL PROTECTED] >> >> Ian Hobbs [EMAIL PROTECTED] === "Never argue with an idiot. They drag you down to their level then beat you with experience." -Dilbert ** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. **
Re: D2D on AIX
Question, Why not use the DISK device class with RAW volumes? Personally, I find FILE classes a pain for user storage because you DO have to perform reclamation on them. Ian Hobbs On Mon, 20 Sep 2004 14:48:07 -0400, Eliza Lau wrote: >Okay. I got it. It is a pain to manually define the volumes, but it can be >done. I also received your pdf file. > >Thanks to everyone who answered, >Eliza > >> >> It depends upon how you configure things. For dynamic allocation of >> volumes, then yes you are limited to the size of the file system that >> you mount on that mount point. However if you define the stgpool >> volumes explicitly using the DEFINE VOLUME command, you can place the >> volumes across as many file systems as you want. I will email you a PDF >> presentation IBM has on Disk Only backups. >> >> >> H. Milton Johnson >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of >> Eliza Lau >> Sent: Monday, September 20, 2004 12:11 PM >> To: [EMAIL PROTECTED] >> Subject: D2D on AIX >> >> Our 3494 with 3590K tapes in 3 frames is getting full. Instead of >> adding another frame or upgrading to 3590H or 3592 tapes we are looking >> into setting up a bunch of cheap ATA disks as primary storage. >> >> The FILE devclass defines a directory as its destination and JFS2 has a >> max file system size of 1TB. Does it mean the largest stgpool I can >> define is 1TB? >> >> My Exchange stgpool alone has 8TB of data. Do I have to split it up >> into 8 pieces? >> >> server: TSM 5.2.2.5 on AIX 5.2 >> database 90GB at 70% >> Total backup data - 22TB >> >> Eliza Lau >> Virginia Tech Computing Center >> [EMAIL PROTECTED] >> >> Ian Hobbs [EMAIL PROTECTED] === "Never argue with an idiot. They drag you down to their level then beat you with experience." -Dilbert
Re: D2D on AIX
Okay. I got it. It is a pain to manually define the volumes, but it can be done. I also received your pdf file. Thanks to everyone who answered, Eliza > > It depends upon how you configure things. For dynamic allocation of > volumes, then yes you are limited to the size of the file system that > you mount on that mount point. However if you define the stgpool > volumes explicitly using the DEFINE VOLUME command, you can place the > volumes across as many file systems as you want. I will email you a PDF > presentation IBM has on Disk Only backups. > > > H. Milton Johnson > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of > Eliza Lau > Sent: Monday, September 20, 2004 12:11 PM > To: [EMAIL PROTECTED] > Subject: D2D on AIX > > Our 3494 with 3590K tapes in 3 frames is getting full. Instead of > adding another frame or upgrading to 3590H or 3592 tapes we are looking > into setting up a bunch of cheap ATA disks as primary storage. > > The FILE devclass defines a directory as its destination and JFS2 has a > max file system size of 1TB. Does it mean the largest stgpool I can > define is 1TB? > > My Exchange stgpool alone has 8TB of data. Do I have to split it up > into 8 pieces? > > server: TSM 5.2.2.5 on AIX 5.2 > database 90GB at 70% > Total backup data - 22TB > > Eliza Lau > Virginia Tech Computing Center > [EMAIL PROTECTED] > >
Re: D2D on AIX
On Sep 20, 2004, at 2:19 PM, [EMAIL PROTECTED] wrote: Actually, JFS2 has a max filesystem size quite a bit bigger than that. I've got several 2.5TB filesystems right now, and I've had them up to 10TB. For completeness: See also IBM site article "Maximum capacity of an ITSM disk volume." (search on reference number "1170255"). Richard Sims
Re: D2D on AIX
/testfs may grow to 1 TB
there is a way to cheat:
def v filepool /test2fs/file
def v filepool /test2fs/file0001
...
so using a script you can manually define more volumes outside the
devclass-defined directory. their sizes are as defined in the devclass.
another approach might be hierarchy of filepools:
def devclass fileclass1 devtype=file maxcap=64g dir=/test1fs
def stgpool filepool1 fileclass1 pooltype=primary maxscratch=100
def devclass fileclass2 devtype=file maxcap=64g dir=/test2fs
def stgpool filepool2 fileclass2 pooltype=primary maxscratch=100
upd stg filepool1 next=filepool2
def devclass fileclass3 devtype=file maxcap=64g dir=/test3fs
def stgpool filepool3 fileclass3 pooltype=primary maxscratch=100
upd stg filepool2 next=filepool3
...
You have to take into account that the latter is unusual and during the
years there were some nasty APARs for hierarchies of more than two storage
pools (DISKPOOL -> TAPEPOOL).
Zlatko Krastev
IT Consultant
Eliza Lau <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
20.09.2004 20:52
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc: (bcc: ADSM-L/ACIT)
Subject:Re: D2D on AIX
Mark,
Where do you define this
I use this command to define a diskpool:
def devclass fileclass devtype=file maxcap=64g dir=/testfs
def stgpool filepool fileclass pooltype=primary maxscratch=100
/testfs is a JFS2 filesystem. How big can /testfs grow to? The documents
say 1TB.
When volumes are created in the stgpool filepool, it creates a volume of
64G,
which is the max value you can specify. Where do you define the 500GB you
said.
Eliza
>
> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On
> Behalf Of Eliza Lau
> >Our 3494 with 3590K tapes in 3 frames is getting full.
> >Instead of adding another frame or upgrading to 3590H or 3592
> >tapes we are looking into setting up a bunch of cheap ATA
> >disks as primary storage.
> >
> >The FILE devclass defines a directory as its destination and
> >JFS2 has a max file system size of 1TB. Does it mean the
> >largest stgpool I can define is 1TB?
>
> No. What it means is that the largest single volume in your diskpool can
> be 1TB. You could have, say, 30 volumes @ 500GB per volume, making a
> total storage pool size of 15TB. Every two volumes would be in their own
> filesystem.
>
> If you're using a disk farm as your primary storage pool, fault
> tolerance is strongly recommended. RAID0 and RAID1+0 would be more
> expensive; RAID5 might make more sense, as long as you were using a
> proper monitoring system (properly set up) to watch the health of your
> disks. Are you using CACHE=YES in your proposed disk solution?
>
> --
> Mark Stapleton ([EMAIL PROTECTED])
> Berbee Information Networks
>
>
Re: D2D on AIX
==> In article <[EMAIL PROTECTED]>, Eliza Lau <[EMAIL PROTECTED]> writes: > The FILE devclass defines a directory as its destination and JFS2 > has a max file system size of 1TB. Does it mean the largest stgpool > I can define is 1TB? Actually, JFS2 has a max filesystem size quite a bit bigger than that. I've got several 2.5TB filesystems right now, and I've had them up to 10TB. I'd suggest FILE devclass sizes that are small enough to feel manageable; tens of GB, not hundreds. - Allen S. Rout
Re: D2D on AIX
It depends upon how you configure things. For dynamic allocation of volumes, then yes you are limited to the size of the file system that you mount on that mount point. However if you define the stgpool volumes explicitly using the DEFINE VOLUME command, you can place the volumes across as many file systems as you want. I will email you a PDF presentation IBM has on Disk Only backups. H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Eliza Lau Sent: Monday, September 20, 2004 12:11 PM To: [EMAIL PROTECTED] Subject: D2D on AIX Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding another frame or upgrading to 3590H or 3592 tapes we are looking into setting up a bunch of cheap ATA disks as primary storage. The FILE devclass defines a directory as its destination and JFS2 has a max file system size of 1TB. Does it mean the largest stgpool I can define is 1TB? My Exchange stgpool alone has 8TB of data. Do I have to split it up into 8 pieces? server: TSM 5.2.2.5 on AIX 5.2 database 90GB at 70% Total backup data - 22TB Eliza Lau Virginia Tech Computing Center [EMAIL PROTECTED]
Re: D2D on AIX
Mark, Where do you define this I use this command to define a diskpool: def devclass fileclass devtype=file maxcap=64g dir=/testfs def stgpool filepool fileclass pooltype=primary maxscratch=100 /testfs is a JFS2 filesystem. How big can /testfs grow to? The documents say 1TB. When volumes are created in the stgpool filepool, it creates a volume of 64G, which is the max value you can specify. Where do you define the 500GB you said. Eliza > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > Behalf Of Eliza Lau > >Our 3494 with 3590K tapes in 3 frames is getting full. > >Instead of adding another frame or upgrading to 3590H or 3592 > >tapes we are looking into setting up a bunch of cheap ATA > >disks as primary storage. > > > >The FILE devclass defines a directory as its destination and > >JFS2 has a max file system size of 1TB. Does it mean the > >largest stgpool I can define is 1TB? > > No. What it means is that the largest single volume in your diskpool can > be 1TB. You could have, say, 30 volumes @ 500GB per volume, making a > total storage pool size of 15TB. Every two volumes would be in their own > filesystem. > > If you're using a disk farm as your primary storage pool, fault > tolerance is strongly recommended. RAID0 and RAID1+0 would be more > expensive; RAID5 might make more sense, as long as you were using a > proper monitoring system (properly set up) to watch the health of your > disks. Are you using CACHE=YES in your proposed disk solution? > > -- > Mark Stapleton ([EMAIL PROTECTED]) > Berbee Information Networks > >
Re: D2D on AIX
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Eliza Lau >Our 3494 with 3590K tapes in 3 frames is getting full. >Instead of adding another frame or upgrading to 3590H or 3592 >tapes we are looking into setting up a bunch of cheap ATA >disks as primary storage. > >The FILE devclass defines a directory as its destination and >JFS2 has a max file system size of 1TB. Does it mean the >largest stgpool I can define is 1TB? No. What it means is that the largest single volume in your diskpool can be 1TB. You could have, say, 30 volumes @ 500GB per volume, making a total storage pool size of 15TB. Every two volumes would be in their own filesystem. If you're using a disk farm as your primary storage pool, fault tolerance is strongly recommended. RAID0 and RAID1+0 would be more expensive; RAID5 might make more sense, as long as you were using a proper monitoring system (properly set up) to watch the health of your disks. Are you using CACHE=YES in your proposed disk solution? -- Mark Stapleton ([EMAIL PROTECTED]) Berbee Information Networks
D2D on AIX
Our 3494 with 3590K tapes in 3 frames is getting full. Instead of adding another frame or upgrading to 3590H or 3592 tapes we are looking into setting up a bunch of cheap ATA disks as primary storage. The FILE devclass defines a directory as its destination and JFS2 has a max file system size of 1TB. Does it mean the largest stgpool I can define is 1TB? My Exchange stgpool alone has 8TB of data. Do I have to split it up into 8 pieces? server: TSM 5.2.2.5 on AIX 5.2 database 90GB at 70% Total backup data - 22TB Eliza Lau Virginia Tech Computing Center [EMAIL PROTECTED]
