Re: Tier'ed library
Tivoli may be looking at this for futures? As a matter of fact they will. Version 5.3 (expected 4th. quarter) will address several disk-only implementation enhancements. Kindest regards, Eric van Loon KLM Royal Dutch Airlines -Original Message- From: Rushforth, Tim [mailto:[EMAIL PROTECTED] Sent: Thursday, June 17, 2004 17:05 To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Using sequential-access File volumes with TSM seems to result in a lot of file level fragmentation. We are doing a mini-pilot with 25GB File volumes for the storage pool volumes for some nodes. These volumes end up very fragmented (some of the files are in 9000 fragments). This could have performance implications. Tivoli may be looking at this for futures? A VTL may address this. I did a quick restore test of a node as follows: 36 GB restored 219,170 objects On fragmented File Volumes: Time 34.6 minutes 17.7 MB/sec After defragmenting the file Volumes: Time 21.9 minutes 27.9 MB/sec It was not a controlled test as it was run on a production server so there could have been other things affecting the two tests. TSM Server 5.2.2.4 on Windows 2003 TSM Client 5.2.2.9 on Windows 2003 Tim Rushforth City of Winnipeg -Original Message- From: Tab Trepagnier [mailto:[EMAIL PROTECTED] Sent: June 16, 2004 8:48 AM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C. ** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. **
Re: Tier'ed library
Yes, file level fragmentation (not internal TSM). If you scroll down you will see it is Windows 2003. The file system was NTFS. Tim -Original Message- From: Steve Harris [mailto:[EMAIL PROTECTED] Sent: June 17, 2004 6:05 PM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Thanks for the post Tim. When you say fragmented, do you mean that the blocks making up this file are scattered all over the filesystem or some sort of internal TSM fragmentation. From your earlier posts, I think that you are in a Windows shop. What OS/Filesystem are we talking here? Regards Steve Steve Harris AIX and TSM Admin Queensland Health, Brisbane Australia [EMAIL PROTECTED] 18/06/2004 1:05:21 Using sequential-access File volumes with TSM seems to result in a lot of file level fragmentation. We are doing a mini-pilot with 25GB File volumes for the storage pool volumes for some nodes. These volumes end up very fragmented (some of the files are in 9000 fragments). This could have performance implications. Tivoli may be looking at this for futures? A VTL may address this. I did a quick restore test of a node as follows: 36 GB restored 219,170 objects On fragmented File Volumes: Time 34.6 minutes 17.7 MB/sec After defragmenting the file Volumes: Time 21.9 minutes 27.9 MB/sec It was not a controlled test as it was run on a production server so there could have been other things affecting the two tests. TSM Server 5.2.2.4 on Windows 2003 TSM Client 5.2.2.9 on Windows 2003 Tim Rushforth City of Winnipeg -Original Message- From: Tab Trepagnier [mailto:[EMAIL PROTECTED] Sent: June 16, 2004 8:48 AM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C. *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. ***
Re: Tier'ed library
Using sequential-access File volumes with TSM seems to result in a lot of file level fragmentation. We are doing a mini-pilot with 25GB File volumes for the storage pool volumes for some nodes. These volumes end up very fragmented (some of the files are in 9000 fragments). This could have performance implications. Tivoli may be looking at this for futures? A VTL may address this. I did a quick restore test of a node as follows: 36 GB restored 219,170 objects On fragmented File Volumes: Time 34.6 minutes 17.7 MB/sec After defragmenting the file Volumes: Time 21.9 minutes 27.9 MB/sec It was not a controlled test as it was run on a production server so there could have been other things affecting the two tests. TSM Server 5.2.2.4 on Windows 2003 TSM Client 5.2.2.9 on Windows 2003 Tim Rushforth City of Winnipeg -Original Message- From: Tab Trepagnier [mailto:[EMAIL PROTECTED] Sent: June 16, 2004 8:48 AM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C.
Re: Tier'ed library
Thanks for the post Tim. When you say fragmented, do you mean that the blocks making up this file are scattered all over the filesystem or some sort of internal TSM fragmentation. From your earlier posts, I think that you are in a Windows shop. What OS/Filesystem are we talking here? Regards Steve Steve Harris AIX and TSM Admin Queensland Health, Brisbane Australia [EMAIL PROTECTED] 18/06/2004 1:05:21 Using sequential-access File volumes with TSM seems to result in a lot of file level fragmentation. We are doing a mini-pilot with 25GB File volumes for the storage pool volumes for some nodes. These volumes end up very fragmented (some of the files are in 9000 fragments). This could have performance implications. Tivoli may be looking at this for futures? A VTL may address this. I did a quick restore test of a node as follows: 36 GB restored 219,170 objects On fragmented File Volumes: Time 34.6 minutes 17.7 MB/sec After defragmenting the file Volumes: Time 21.9 minutes 27.9 MB/sec It was not a controlled test as it was run on a production server so there could have been other things affecting the two tests. TSM Server 5.2.2.4 on Windows 2003 TSM Client 5.2.2.9 on Windows 2003 Tim Rushforth City of Winnipeg -Original Message- From: Tab Trepagnier [mailto:[EMAIL PROTECTED] Sent: June 16, 2004 8:48 AM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C. *** This email, including any attachments sent with it, is confidential and for the sole use of the intended recipient(s). This confidentiality is not waived or lost, if you receive it and you are not the intended recipient(s), or if it is transmitted/received in error. Any unauthorised use, alteration, disclosure, distribution or review of this email is prohibited. It may be subject to a statutory duty of confidentiality if it relates to health service matters. If you are not the intended recipient(s), or if you have received this email in error, you are asked to immediately notify the sender by telephone or by return email. You should also delete this email and destroy any hard copies produced. ***
Re: Tier'ed library
We have outgrown our 3583 and 3575s, so I'm looking for a new tape library. At first I though my only choices were another standalone library like the 3583 or go with a frame system like the 3584. I have problems with both approaches. While I have no personal experience with and so cannot give a recommendation pro or con, you might look at a library like the ADIC Scalar 1000. I believe it is reasonably priced and can grow from 1 frame to 4 frames. - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: Tier'ed library
Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C. Johnson, Milton [EMAIL PROTECTED] 06/15/2004 03:13 PM To: Tab Trepagnier [EMAIL PROTECTED] cc: Subject:Tier'ed library Tab, We are faced with the same options and are seriously looking at Virtual Tape Libraries (VTL), an appliance that is physically a large SATA Raid Array but presents itself to TSM as a tape library. The product we are looking at is the Sepaton S2100-ES (www.sepaton.com). The things we like include: 1) It's TSM certified, meaning that it has passed the same certification the real tape libraries passed. 2) Cheap. We have been quoted $30K (USD) for the first 3TB and $18K (USD) for each additional 3TB. 3) Modular: Purchase the first 3Tb then expand in 3TB increments up to 200TB. After that you purchase another VTL. 4) Performance: Since it's disk based it's very fast, up to 1.6 TB/hour. Since the tape drives are virtual tape drives you can configure a tape library to have 200 tape drives, eliminating the tape drive bottleneck. Tape mounts happen instantly. 5) Because of the high performance and large number of available virtual tape drives, you should be able to reclaim the virtual tapes when they are only 25% reclaimable, instead of waiting until they are 50% reclaimable. This should allow much more efficient usage of the tape space. We will just use our present 3494 ATL to cut off-site tapes. We haven't implemented it yet, but we are actively pursuing it. Our contact is: David Littman The More Group 47 East Grove Street Middleboro, MA 02346 [EMAIL PROTECTED] 508-946-2255 x19 H. Milton Johnson
Re: Tier'ed library
Tab, We've asked the same thing and came to the following points: 1) The $/TB is about as cheap as you can get. 2) With a VTL you can do LAN free backups. 3) To create an equiv Primary StgPool using sequential-access FILE volumes you would have to create a 200TB file system. Even using JFS2, I'm not sure you would get the same throughput as with a VTL. 4) Sepaton claims that by second quarter 2005, they will also have compression which would increase the VTL's usable capacity. AIX does not offer a compressed JFS2 filesystem, and if it did it would have to have a serious impact on performance. 5) A tape library can be shared between systems which may/may not be relevant to you. 6) We could not think of a technical advantage that favored using sequential-access FILE volumes. The biggest hurtle is changing your mind set to allow the use of disks versus tape. We have come to the following conclusions: 1) If RAID-5 is that unreliable then why are we using it for our on-line databases? 2) We will not be a tape free environment, we would just be replacing our on-site tapes with disks. The amount of time when a backup is on disks only is very short. 3) The VTL costs are low enough so that we can have a mirrored on-site tape pool. The mirror would be in another building and TSM would simultaneously write the backup to both stgpools. This would give a level of protection that would be very costly reproduce using physical tapes. The fact that IBM provides the service for the Sepaton VTL is an added plus in our shop. Please note that I originally posted this to Tab instead of the list because I did not want it to seem that I was using the list to promote Sepaton. H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Tab Trepagnier Sent: Wednesday, June 16, 2004 8:48 AM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Milton, Thanks for the info. I briefly looked at Sepaton, but I had no idea they were that inexpensive. I will probably give them a second look. But one thing that I'm struggling with is why a VTL? Between random-access DISK volumes and sequential-access FILE volumes what does a VTL buy me that I couldn't implement using those two volume types in TSM? Thanks. Tab Trepagnier TSM Administrator Laitram, L.L.C. Johnson, Milton [EMAIL PROTECTED] 06/15/2004 03:13 PM To: Tab Trepagnier [EMAIL PROTECTED] cc: Subject:Tier'ed library Tab, We are faced with the same options and are seriously looking at Virtual Tape Libraries (VTL), an appliance that is physically a large SATA Raid Array but presents itself to TSM as a tape library. The product we are looking at is the Sepaton S2100-ES (www.sepaton.com). The things we like include: 1) It's TSM certified, meaning that it has passed the same certification the real tape libraries passed. 2) Cheap. We have been quoted $30K (USD) for the first 3TB and $18K (USD) for each additional 3TB. 3) Modular: Purchase the first 3Tb then expand in 3TB increments up to 200TB. After that you purchase another VTL. 4) Performance: Since it's disk based it's very fast, up to 1.6 TB/hour. Since the tape drives are virtual tape drives you can configure a tape library to have 200 tape drives, eliminating the tape drive bottleneck. Tape mounts happen instantly. 5) Because of the high performance and large number of available virtual tape drives, you should be able to reclaim the virtual tapes when they are only 25% reclaimable, instead of waiting until they are 50% reclaimable. This should allow much more efficient usage of the tape space. We will just use our present 3494 ATL to cut off-site tapes. We haven't implemented it yet, but we are actively pursuing it. Our contact is: David Littman The More Group 47 East Grove Street Middleboro, MA 02346 [EMAIL PROTECTED] 508-946-2255 x19 H. Milton Johnson
Re: Tier'ed library
Your posting caught my eye with 6 years and 10 different libraries. We have been on TSM(adsm) for almost 8 years now. When the system was first speced out, they did some cost analysis and decided that the 3494 tape library with the 3590 tape drives were the best solution for the long-term. As you know that is a frame system and it is NOT CHEAP. There was a lot of weeping, wailing and gnashing of teeth, with folk's wanting cheaper solutions up front. The evaluation group kept to their guns and the library was bought. Well, we are now 8 years down the road and the libraries are still in use. They are the oldest things in the computer rooms, long depreciated off the books, but still working like a champ. We've extended on and on and on as the storage in the environment has exploded with very little re-work or downtime. Yes we had to upgrade the 3590 drives to the various iterations of capacity, but the library is still the same. I feel it was a wise investment that is still paying dividends today. We have some remote sites which went cheap on some DLTs at the same time and they are once again looking at their 3rd iteration of tape libraries. They once again want to go cheap, but we are pressing them to get a 3584 with LTO2 tapes so they can install it once and then expand it. Just my 2 cents. Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Tab Trepagnier Sent: Tuesday, June 15, 2004 12:23 PM To: [EMAIL PROTECTED] Subject: Tier'ed library TSM Server 5.1.9.0 on AIX 5.2 ML-2 TSM Server 5.2 media on hand We have outgrown our 3583 and 3575s, so I'm looking for a new tape library. At first I though my only choices were another standalone library like the 3583 or go with a frame system like the 3584. I have problems with both approaches. The standalone library, while cheaper, means that in 2-3 years I will have filled up yet another library now destined to collect dust in a corner. Since ADSM 2 six years ago we have use 10 different libraries or autoloaders on our system. Five of them are still in service. That is ridiculous. The frame system is just too expensive for our little company. We've obtained quotes for four and six drive configurations using LVDS and FC, and the price range is $125K - $180K. That's a LOT of money to us. So I'm thinking that a standalone library that can be expanded in tiers would be an effective way to split the difference. Start smaller with one tier, and simply add a tier once we've outgrown it. I'm thinking four LTO-2 drives using LVDS connectivity to start. Initial *real* capacity should be about 10 TB. It should accept at least two additional tiers before maxing out. It must be supported by some variant of TSM 5.x. What has been the experience of forum users with libraries of that type? Do you have any brand/model recommendations? Models to avoid? Thanks in advance. Tab Trepagnier TSM Administrator Laitram, L.L.C.
Re: Tier'ed library
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Tab Trepagnier TSM Server 5.1.9.0 on AIX 5.2 ML-2 TSM Server 5.2 media on hand We have outgrown our 3583 and 3575s, so I'm looking for a new tape library. At first I though my only choices were another standalone library like the 3583 or go with a frame system like the 3584. I have problems with both approaches. Whatever you go to, try to stay with LTO. Not only will you be able to use the tape inventory you have at hand, but most other media just won't take the punishment TSM metes out to tape. I think you're going to have a hard time finding something between the scales of a 3583 and a 3584. Have you thought about just adding another 3583 library? A full-up 3583 with LTO gen 2 tape drives has 24TB+ of capacity and will give you six additional drives. The newer 3583s support logical partitioning and fiber connectivity. Mr. Bullock's post in this thread is also of merit. You might consider his alternative as well. -- Mark Stapleton
Re: Tier'ed library
Same experience here - Buying a cheap library is like knowing you've accepted the low bidder for brain surgery ;) We've been through exactly the same analysis, same strategy, same arguments, and fortunately have always had the same results - more cost effective in the long run to buy the big library up front. (BTW, in addition to the capital $$, remember that YOUR TIME for managing multiple libraries and changing libraries in and out is worth a LOT.) BUT, if you add another 3583, with LTO2, and upgrade your existing 3583 to LTO2 - KNOWING that in a couple of years you should be able to go to LTO3 - would that do it? A BIG advantage of the LTOx media is that the standard calls for write-compatibility for 1 generation back and read compatibility for 2 generations back. If you're growing so fast that 2 3583's with LTO2 won't hold your data for 3 years - I think you're gonna end up with a frame-based system anyway! Condolences, Wanda -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Ben Bullock Sent: Tuesday, June 15, 2004 2:50 PM To: [EMAIL PROTECTED] Subject: Re: Tier'ed library Your posting caught my eye with 6 years and 10 different libraries. We have been on TSM(adsm) for almost 8 years now. When the system was first speced out, they did some cost analysis and decided that the 3494 tape library with the 3590 tape drives were the best solution for the long-term. As you know that is a frame system and it is NOT CHEAP. There was a lot of weeping, wailing and gnashing of teeth, with folk's wanting cheaper solutions up front. The evaluation group kept to their guns and the library was bought. Well, we are now 8 years down the road and the libraries are still in use. They are the oldest things in the computer rooms, long depreciated off the books, but still working like a champ. We've extended on and on and on as the storage in the environment has exploded with very little re-work or downtime. Yes we had to upgrade the 3590 drives to the various iterations of capacity, but the library is still the same. I feel it was a wise investment that is still paying dividends today. We have some remote sites which went cheap on some DLTs at the same time and they are once again looking at their 3rd iteration of tape libraries. They once again want to go cheap, but we are pressing them to get a 3584 with LTO2 tapes so they can install it once and then expand it. Just my 2 cents. Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Tab Trepagnier Sent: Tuesday, June 15, 2004 12:23 PM To: [EMAIL PROTECTED] Subject: Tier'ed library TSM Server 5.1.9.0 on AIX 5.2 ML-2 TSM Server 5.2 media on hand We have outgrown our 3583 and 3575s, so I'm looking for a new tape library. At first I though my only choices were another standalone library like the 3583 or go with a frame system like the 3584. I have problems with both approaches. The standalone library, while cheaper, means that in 2-3 years I will have filled up yet another library now destined to collect dust in a corner. Since ADSM 2 six years ago we have use 10 different libraries or autoloaders on our system. Five of them are still in service. That is ridiculous. The frame system is just too expensive for our little company. We've obtained quotes for four and six drive configurations using LVDS and FC, and the price range is $125K - $180K. That's a LOT of money to us. So I'm thinking that a standalone library that can be expanded in tiers would be an effective way to split the difference. Start smaller with one tier, and simply add a tier once we've outgrown it. I'm thinking four LTO-2 drives using LVDS connectivity to start. Initial *real* capacity should be about 10 TB. It should accept at least two additional tiers before maxing out. It must be supported by some variant of TSM 5.x. What has been the experience of forum users with libraries of that type? Do you have any brand/model recommendations? Models to avoid? Thanks in advance. Tab Trepagnier TSM Administrator Laitram, L.L.C.
Re: Tier'ed library
... The standalone library, while cheaper, means that in 2-3 years I will have filled up yet another library now destined to collect dust in a corner. ... Not necessarily. The Ultrium roadmap: http://www.lto-technology.com/newsite/html/format_roadmap.html http://www.qualstar.com/146252.htm talks of doubling capacity every generation, which could mean just swapping out drives (which could be dispositioned by inserting into stand-along cabinets for local system use, or sell on the used market). It may depend upon your timeframe relative to LTO evolution, and the vendor's inclination to allow the next generation to fit into the old drive positions. (A big hint to your IBM rep could help keep this true.) IBM has been wonderful about evolving existing libraries and retaining cartridge dimensions, making a long-term investment very economical. Richard Sims
Re: Tier'ed library
== In article [EMAIL PROTECTED], Ben Bullock [EMAIL PROTECTED] writes: Your posting caught my eye with 6 years and 10 different libraries. I'm going to try to make this not just a 'Me Too'. I've only had two libraries, but the 3494 was here when I started messing with ADSM in '97. I've just added some 3592 drives to it, which will take me well into the next generation of tape tech. Upgrade path, reliability, flexibility, featureset. Try really hard to convince the folks that you're going to -save- money in the 3-year timeframe, and will absolutely rake it in on the 7-year. - Allen S. Rout
