Re: How to Incorporate a CDL into TSM environment?
We are considering purchasing a CDL 4100 and EMC is telling us we will get 1200MBps performance. Can anyone provide feedback on how their VTL -- specifically EMC DL 4100 is performing? I have spoken to some friends who have implemented one and are getting much lower performance #s... they say 100MB-200MB. Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
Re: How to Incorporate a CDL into TSM environment?
In my case 10TB of type=file will store 10TB of data. 10TB of VTL will store nearly 20TB of data. That is my only reason. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of caldwem01 Sent: Tuesday, June 12, 2007 5:21 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? How come no one has commented on the TSM device type=file as an alternative to VTL? I know it doesn't do de-dup. But doesn't it have a place? please comment this has baffled me since the first VLT hit the market. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +-- This e-mail (including any attachments) is confidential and may be legally privileged. If you are not an intended recipient or an authorized representative of an intended recipient, you are prohibited from using, copying or distributing the information in this e-mail or its attachments. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message and any attachments. Thank you.
Re: How to Incorporate a CDL into TSM environment?
In my world I am limited to 800MB to the VTL because I only have four 2GB fiber connections to the VTL. Migrations are limited to 400MB because I have only two 2GB connections to the disk pools. When the Ins are going to the right Outs I get close to the above speeds. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of lowneil Sent: Tuesday, June 12, 2007 5:38 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? We are considering purchasing a CDL 4100 and EMC is telling us we will get 1200MBps performance. Can anyone provide feedback on how their VTL -- specifically EMC DL 4100 is performing? I have spoken to some friends who have implemented one and are getting much lower performance #s... they say 100MB-200MB. Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +-- This e-mail (including any attachments) is confidential and may be legally privileged. If you are not an intended recipient or an authorized representative of an intended recipient, you are prohibited from using, copying or distributing the information in this e-mail or its attachments. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message and any attachments. Thank you.
Re: How to Incorporate a CDL into TSM environment?
Validate with EMC that the the 1200MBps is Native or compressed, we had the same discussion with our CDL 740 and the actual max was 469MBS. I'm not sure if the newer 4100 is clustering the VTL Heads to get an actual 1200MBS. Charles Hart lowneil [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/12/2007 05:37 PM Please respond to ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] How to Incorporate a CDL into TSM environment? We are considering purchasing a CDL 4100 and EMC is telling us we will get 1200MBps performance. Can anyone provide feedback on how their VTL -- specifically EMC DL 4100 is performing? I have spoken to some friends who have implemented one and are getting much lower performance #s... they say 100MB-200MB. Thanks. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +-- This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.
Re: How to Incorporate a CDL into TSM environment?
How come no one has commented on the TSM device type=file as an alternative to VTL? I know it doesn't do de-dup. But doesn't it have a place? please comment this has baffled me since the first VLT hit the market. +-- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] +--
Re: How to Incorporate a CDL into TSM environment?
Richard Rhodes said: I'd love to have a couple vtl's. When we've priced them out they come out to be much more costly (several times) that of tape for our environment. We keep lots of old/stale data around which drives seems to drive the cost of the VTL way up. I was hoping possibly use a vtl for only primary data with the new feature of TSM v5.4, but that's not going to work out. VTL cost (even after dedupe) will never compare with the cost of tape media alone, so tape will always be a cheaper medium if you take it out of the library. When I say that VTLs are close to the price of tape, I mean close to the price of a similarly-sized, fully-populated-with-media tape library. I personal opinion is that VTL's are a stop-gap solution. I think compression and de-dupe have much wider application within a normal disk subsystem where it could apply to a much wider range of situations. Pretty much everybody who is following the industry believes it will morph into the intelligent disk target industry. VTL will continue to be a personality they offer, but as other backup software products are better able to back up to filesystems (TSM does it just fine), more VTL vendors will offer a filesystem interface. Right now, the only one that does a filesystem interface and de-dupe is Data Domain. (Copan has a filesystem interface, but I don't think they're doing de-dupe through it yet. Any day now.) This is the bit problem I see with Tape. It seems to me that the latest generations of tape drives have rated speeds that almost defy the any ability to supply them with data. I almost which I could purchase a modern tape drive that actually was slower. Mmm... LTO-4: 120 MB/s native speed, 180 MB/s typical with compression. (I'm using 1.5:1 compression, which is what I see most often as an average actual compression ratio.) So... It wants 180 MB/s, and I've supposed to feed that with a 60-80 MB/s GbE connection. (The only saving grace of modern tape drives is that they have variable speeds. The LTO-4, for example, can go as slow as about 40 MB/s plus compression.) The problem is capacity. As vendors push capacity, they do so by pushing the bits closer together. As they do that, the drive gets faster.
Re: How to Incorporate a CDL into TSM environment?
On Tue, 12 Jun 2007 03:20:18 -0400, Curtis Preston [EMAIL PROTECTED] said: Pretty much everybody who is following the industry believes it will morph into the intelligent disk target industry. I see this trend too. It makes me think of ATM. Remember ATM? :) Oy, I sound like an old fart. Sorry. Topical, topical: The problem I see with the intelligent disk target notion is that, the smarter you try to get, the more management you have to do, and the more statistical duplication you have to push under the covers. How would you feel if you had to map and tune RAID-5 regions between platters? And, (and here's the psychological kicker) the more you are out of control of the actual processes. This does not offend most administrators of e.g. windows file-service servers: the performance required is sufficiently modest, and the demand sufficiently diffuse, that things like Which platters is this coming from never arise. But if you really want to understand your performance and bottlenecks, then you're in vendor motel land, with extra-price tools to help you. :) It's a storage cloud, and just trust us, it'll all work out. ATM. For administrators accustomed to operating the presented interface, this is not worrying. For administrators accustomed to understanding and controlling the underlying behavior, it is disconcerting, and in the way. This doesn't mean it won't be a great way to make money. The advantage to selling restricted and concealing interfaces is that you can access a market which is incompetent to challenge you, and each flaw in product version N can be a sales opportunity for N.1. Witness Windows. - Allen S. Rout
Re: How to Incorporate a CDL into TSM environment?
Hi Everyone! I have successfully completed the following: Defined a scsi library: cdlb_dev Defined a path from the tsm server to the cdl library: define path tsmdev cdlb_dev srctype=server desttype=library device=/dev/smc0 Define LTO drives for the cdl library: define drive cdlb_dev lto2-27 Define tape paths for the lto2 virtual drives: define path tsmdev lto2-27 srctype=server desttype=drive library=cdlb_dev device=/dev/rmt27 Define a device class for the cdl library: define devclass lto2_cdlb library=cdlb_dev devtype=lto format=ultrium2 mountlimit=drives Define a cdl storage pool: define stgpool cdlb_aix lto2_cdlb pool=primary maxsize=5G maxscratch=100 next=tape_aix hi=90 lo=70 reclaim=50 reclaimpr=1 collocate=no migdelay=30 migpr=1 (I have set the mgdelay to 30 so that we keep 30 days of backups within the cdl until it moves to onsite tape) Label virtual volumes: label libvol cdlb_dev search=yes checkin=scratch volrange=db,db00209 labelsource=barcode In order to start using the cdl and the cdl storage pool cdlb_aix in the order of: aix (disk pool) -- cdlb_aix (cdl aix primary sequential storage pool) -- tape_aix (primary sequential storage pool which is physical lto2 tape) -- copy_aix (offsite lto2 tape pool for offsite disaster recovery purposes) Would I just have to update the aix disk pool's next storage pool to be: cdlb_aix? And then the data will stay on the cdl for 30 days since I have the migdelay set to 30? And what order does it make sense to migrate and backup the data? Scenerio 1 Client backup to aix disk pool Migrate data to cdlb_aix backup cdlb_aix to copy_aix backup tape_aix to copy_aix Migrate cdlb_aix to tape_aix after 30 days Will I ever need to run a job to backup the aix disk pool to the copy_aix pool? Scenerio 2 Client backup to aix disk pool Backup aix disk pool to copy_aix Migrate data from aix disk pool to cdlb_aix storage pool backup cdlb_aix to copy_aix backup tape_aix to copy_aix Migrate cdlb_aix to tape_aix after 30 days Any suggestions are greatly appreciated! Thanks! Joni Moyer Highmark Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966 Fax: (717) 302-9826 [EMAIL PROTECTED] Allen S. Rout [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/12/2007 10:00 AM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: How to Incorporate a CDL into TSM environment? On Tue, 12 Jun 2007 03:20:18 -0400, Curtis Preston [EMAIL PROTECTED] said: Pretty much everybody who is following the industry believes it will morph into the intelligent disk target industry. I see this trend too. It makes me think of ATM. Remember ATM? :) Oy, I sound like an old fart. Sorry. Topical, topical: The problem I see with the intelligent disk target notion is that, the smarter you try to get, the more management you have to do, and the more statistical duplication you have to push under the covers. How would you feel if you had to map and tune RAID-5 regions between platters? And, (and here's the psychological kicker) the more you are out of control of the actual processes. This does not offend most administrators of e.g. windows file-service servers: the performance required is sufficiently modest, and the demand sufficiently diffuse, that things like Which platters is this coming from never arise. But if you really want to understand your performance and bottlenecks, then you're in vendor motel land, with extra-price tools to help you. :) It's a storage cloud, and just trust us, it'll all work out. ATM. For administrators accustomed to operating the presented interface, this is not worrying. For administrators accustomed to understanding and controlling the underlying behavior, it is disconcerting, and in the way. This doesn't mean it won't be a great way to make money. The advantage to selling restricted and concealing interfaces is that you can access a market which is incompetent to challenge you, and each flaw in product version N can be a sales opportunity for N.1. Witness Windows. - Allen S. Rout
Re: How to Incorporate a CDL into TSM environment?
I'd love to have a couple vtl's . . . . ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU wrote on 06/08/2007 05:34:03 PM: Yes, tape is still cheaper, but if you compare the price of a large VTL with de-dupe to an equivalently sized tape library, they'll be a lot closer than you think I'd love to have a couple vtl's. When we've priced them out they come out to be much more costly (several times) that of tape for our environment. We keep lots of old/stale data around which drives seems to drive the cost of the VTL way up. I was hoping possibly use a vtl for only primary data with the new feature of TSM v5.4, but that's not going to work out. The second thing VTLs bring to the table is hardware compression. Then, of course, there's de-dupe, which most surveys are showing to be the got-to-have technology of this year. It's here. It's real. And it really does shrink the amount of disk you need to use by a factor of 10-20:1, and even more depending on how you do your backups. Agreed. I'm convinced that compression and de-dupe is what you're purchasing in a vtl as opposed to just straight disk. It would be a lot cleaper to just purchase disk, but then no compression and no de-dupe. I personal opinion is that VTL's are a stop-gap solution. I think compression and de-dupe have much wider application within a normal disk subsystem where it could apply to a much wider range of situations. A long time ago a company we purchased had a old STK Iceberg disk subsystem . . .yea, the one with with log based writes like NetApp except with hardware compression. The guys who used it have nothing but praise for it (although it had it's problems!!!). NetApp is adding de-dupe to their disk systems . . . .now if they would only add hdwr compression . . . The reason that VTL/disk can outperform tape is that disk can go whatever speed your backup is going and tape cannot. This is the bit problem I see with Tape. It seems to me that the latest generations of tape drives have rated speeds that almost defy the any ability to supply them with data. I almost which I could purchase a modern tape drive that actually was slower. Most environments never get anywhere near their tape's capabilities and about half or so are getting a small fraction of their tape drive's capabilities. Exactly . . . . Just my $.02. --- W. Curtis Preston Add my $.02 Rick - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: How to Incorporate a CDL into TSM environment?
John, After all the talks I've given promoting the use of VTLs with TSM and other products, it's good to finally hear from someone who has been able to actually DO it in multiple environments. I concur with almost all of your comments. I do have questions about some of them, and then I have some comments about the overall VTL industry. It sounds like you've got much more real-world experience with the TSM/VTL combination than I have, so please take my comments as curiosity and/or request for confirmation, not confrontation. The first question that I have is what size environments have you been able to implement these recommendations for? Many of them strike me as perfect for small-to-medium shops, but not easy to implement in large shops. (Our customers are some of the largest TSM shops in the world.) The second is a Redbook from IBM about their VTL solution. There is a chapter in it specific to how a VTL can help in a TSM environment. You do realize that both products are Falconstor underneath, right? So at this point, the only material difference between the two is the hardware that IBM/EMC puts around it. Sun uses it as well. 1) Design the solution and size the CDL so that most or all Primary storage pools can fit on the CDL. I couldn't agree more. The chaleng that I've found is that most of our large customers have been unable to justify the cost of VTLs that are the same size as their tape libraries. The advent of de-dupe is changing all that, as a 200 TB tape library can be replaced by 10 TB of disk. Let me speak to this for a second. De-dupe ratios definitely are an area where your mileage will vary. TSM filesystem progressive incrementals will not get the same level of de-dupe as other shops that do frequent full backups, as that is where a lot of the duplicated data comes from. However, duplicated data also comes from the same files being placed in multiple places (emails, filesystems, multiple users using the same doc but putting it in multiple places, etc.). It also comes from repeated incrementals of the same file that changes just a little bit each day, such as a spreadsheet that someone updates every day. So TSM environments will still see plenty de-dupe on their filesystem backups, just not 20:1. They will also see the same de-dupe ratios as everybody else when they backup Oracle, DB2, Exchange, SQL Server, etc, as TSM does the typical full/incremental backups there. The bummer thing about de-dupe is that it's not available in most of the major OEM VTLs. I believe Sun is selling Falconstor's de-dupe, and HDS is definitely selling Diligent's Protectier. IBM hasn't let me know what they're doing yet, and EMC is still saying they're going to write their own. HP's VTL (provided by SEPATON) doesn't yet offer their de-dupe feature. NetApp's VTL doesn't yet offer de-dupe. That means that those same large shops that I'm saying should use de-dupe won't use it because it's not available from their OEM. (As I said, you're fine if you use HDS or Sun, but not if you want to buy it from EMC, HP, or IBM -- yet.) Bummer. I'm pretty bullish on de-dupe and I think it's ready for prime time as long as everything is also on tape. (Your copies you're creating for offsite DR will do.) It solves a lot of problems. It reduces acquisition cost (by a factor as much as 20:1) and reduces power and cooling cost by the same factor. And as long as everything is also on tape for DR, you've got a risk mitigation copy in case you picked the wrong de-dupe product and it completely goes toes-up on you. So, I'd say that your idea is totally implementable in large shops if they use de-dupe. Otherwise, we're talking way too much disk when you consider that most people have 10 GB on tape for every 1 GB they have on disk. Direct the client backups directly to the virtual tapes, instead of going to disk storage pool. Again, I think this will work fine in many environments. Our large TSM customers have hundreds (or thousands) of clients backing up simultaneously to their disk pools. You can't define 500 or 1000 virtual tape drives, and you wouldn't want to if you could. So these customers would have an issue implementing your suggestion. This will save you hours of time in the schedule not having to migrate from disk to tape. Again, if you can do it, I agree. Many people can't, unfortunately. There is no particular advantage to collocating storage pools in a virtual tape environment. I'm not sure I'm sold on this. I'm not saying I disagree; I'm just not sold. My fellow consultants and I have discussed this ad nauseum. My experience is that mounting a virtual tape still takes a finite amount of time and when you multiple that finite amount times the number of tape mounts you may experience in a completely uncollocated world, it may add up to a significant amount of time. (Again, this may be a much bigger deal in larger environments.) I would have to do a test on a couple of
Re: How to Incorporate a CDL into TSM environment?
Josef Weingand said: why do you want to use a VTL with TSM? I get this question a lot in my talks. The first thing I'd say is for the same reasons we want to do it in other environments: performance and manageability. Backups and restores rock with a VTL and that's all there is to it. Second, the manageability of a GOOD large VTL vs many smaller disk pools in a large environment is significantly different. If you have a good VTL that can scale to meet your needs, you get to manage one device. Do you want to replace with a VTL your physical tape, or do you want to replace the TSM disk pool? If you read my other response to John's post, you'll see that my opinion on this is the former. I still think you need a disk pool of some size (e.g. one day's backups) to reduce tape mount contention. But I think it can replace that part of your tape library that's meant to store all on-site tapes. Or do you want to have more LAN-free backup? Not an or, but an and. If I don't want to, I don't have to worry about tape drive sharing, or purchase 3rd party products (i.e. Gresham) to share a tape library (you have to do that to share an ACSLS tape library between multiple LAN-free clients). A VTL allows me to give every large server that needs a tape library its OWN tape library if that's what I want to do. Replacing the disk pool, you might just get the advantage of compression, however, compression with VTL you always need to consider the performance impacts with compression enabled. Most VTLs are using hardware compression now just like your tape drive. If you're using NAS as your disk storage pool, another thing you may consider is Storewiz. It does compression of data going to NAS mounts, and it does it inline just like tape compression, actually improving performance not hurting it. Replacing physical tape, leads to more pwr (and cooling) cost. A 200 TB VTL needs about 280 000 Euro/U$ more (for pwr and cooling) over a 6 years time frame compared to tape. Ok, some countries on the world does not care about pwr and green house gas (yet). Hey, I think that was aimed at us. ;) First I can tell you our contry's position on that matter is likely to change drastically in about 18 months. Second, I can tell you that it's a bigger problem over here than you think for datacenters. The problem is that in many places, you actually can't get more power. If you need more power, you have to buy somebody else's building and take their power -- no kidding. Finally, I'll tell you that the concept of green storage is one of the hottest topics at the conferences I've been attending lately. Now, on to your comment. I'd say it's true w/o de-dupe. However, if I'm storing multiple versions on a de-dupe VTL and it reduces the data by 20:1 (or anything near that), the amount of power it consumes changes drastically. Another thing is that disk drives, while they usually spin all day long (although not in all VTLS: see copansys.com), they are much more power efficient than tape drives. Then consider how many hours in a day tape drives spin in a typical TSM environment. I'd wager that a de-dupe VTL spinning disk all day would consume less energy than a similarly sized tape library running anywhere near 24x7. I've actually seen a case study where this was done and the de-dupe vendor won. thinking about100 - 200 session in parallel, you are not able to handle 100 - 200 tape drives on your TSM server. Therefore, in my eyes, a VTL will not replace a prim disk pool in a medium to large TSM environment. Copycat. ;) Using the IBM Tape Device Driver is only allowed for IBM tape products. It is illegal to use EMC CDL with the IBM Tape Device Driver!!! Only the IBM VTL TS7510 and TS7520 are using the IBM Tape Device Driver. Illegal is a strong word. Do you mean unsupported? That falls into the gimme a break category for me. Especially since the software inside the TS7510 is the same exact software as what's inside the EMC CDL. If it works for the 7510, it's going to work for the CDL. I have, however, been in some shops where they not do something that was obviously a Good Idea just because their vendor said we won't support that configuration. I'm too much of a customer is always right type. If I were put in that situation by IBM, I would mention that that this appears to make their position as a combo software/tape hardware/disk hardware/OS platform provider a liability instead of an asset. Perhaps I should go to a vendor that won't hold things over my head like that. I've used this same tactic with Symantec when the force customers to use their volume manager or explain how they don't like to work with EMC products since EMC bought NetWorker. I've used it with EMC when they tell me they come out with a cool new feature on a DMX/Symmetrix that's only usable if I use their backup product. this is a general sizing question. In a well designed/sized TSM environment, there should no prbl with reclamation
Re: How to Incorporate a CDL into TSM environment?
Josef Weingand said: why do you want to use a VTL with TSM? I get this question a lot in my talks. The first thing I'd say is for the same reasons we want to do it in other environments: performance and manageability. Backups and restores rock with a VTL and that's all there is to it. Second, the manageability of a GOOD large VTL vs many smaller disk pools in a large environment is significantly different. If you have a good VTL that can scale to meet your needs, you get to manage one device. ...I understood restore performance suffered with a VTL - the way it has been described to me is that, should a restore need to come from a volume that has been destaged from disk to tape in the VTL, then a restore of a single file from the volume would first have to wait for the vtl to rebuild the tape on disk? Or have I got the wrong end of the stick? This message and any attachments (the message) is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. - Ce message et toutes les pieces jointes (ci-apres le message) sont etablis a l'intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le detruire et d'en avertir immediatement l'expediteur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, BNP PARIBAS (et ses filiales) decline(nt) toute responsabilite au titre de ce message, dans l'hypothese ou il aurait ete modifie.
Re: How to Incorporate a CDL into TSM environment?
...I understood restore performance suffered with a VTL - the way it has been described to me is that, should a restore need to come from a volume that has been destaged from disk to tape in the VTL, then a restore of a single file from the volume would first have to wait for the vtl to rebuild the tape on disk? Or have I got the wrong end of the stick? Um. Both. Most VTL's are disk-only devices that emulate tape, and do not have the staging issue you describe. Many VTL's will make restores FASTER because the tape mount time goes from potentially minutes to a second or less. (You also don't have to worry about collocating data in a VTL, so your migration times are generally faster as well.) Now that goes with a caveat - you have to PIN YOUR VENDOR TO THE WALL and get documentation about throughput rates. ALL VTL's work about the same way, but they all have different hardware inside the box, so you can get drastically different results. You can easily create a case where restoring 1 VERY LARGE file will take longer on a slow VTL than with fast tape (Say a TS1120, which run get more than 100MB/sec.) It depends on WHICH VTL you are talking about, the speed of the disk in it, the size of the cache in it the speed of your SAN connection and/or HBAs compared to which tape drive, and whether you are talking about restoring lots of little files or a few huge ones. A VTS (don't they make this confusing?) is an IBM-only mixture of disk/tape that emulates tape. It has to pull data off tape and stage it back to disk before you can restore. Normally the VTS is used in a mainframe environment. IBM also makes VTLs, the TS7510 and TS7520, for use in open environments. They are all disk.
Re: How to Incorporate a CDL into TSM environment?
Greetings, A lot of the chatter about VTL's being good or bad seems to stem from which vendors you listen to, and what they are trying to sell you. There are a lot of dogmatic statements made by people on both sides of this issue, usually by people with no personal experience about what they are talking about. Somebody has fed them a sales line and they dutifully parrot it back. EMC sold their CDL product for about two years before IBM entered the market. During that time you would not believe how many times I heard IBM pooh-pooh the CDL saying it wasn't a good fit for TSM, didn't perform well, whatever they had to say to compete against it. I even heard someone recently say it was against the law to use a CDL if you used the IBM drivers to talk to it. Against what law exactly? Then after two years IBM came out with their VTL the TS7510, and almost immediately came out with a Redbook about it with a TSM chapter explaining why the TS7510 was such a good fit for TSM! Huh? And not because it was a better product than the EMC one, it was actually slightly slower and only scaled to about a fourth the size of the largest EMC VTL. The only difference is that now IBM had something in the marketplace, and that changed everything. As Wanda has said, a lot of the distinctions fall down to how you use the VTL, and if your expectations are set correctly. It is easy for a vendor presentation to promise the moon without qualifying it's claims. A single-engine DL4100 from EMC can sustain a 1100MB/sec (3.7 TB per hour) write speed like they claim IF: 1) You are writing multiple simultaneous virtual tape streams (like 16 or more), 2) You balance the I/O across at least 4 FC streams coming in the VTL engine, 3) You have at least 5 or more disk drawers to spread out the I/O load. 4) You are not compressing at the VTL engine. If you compress at the VTL engine, your performance will drop off, perhaps as low as a third as fast. This is because the compression is done in software. If you want hardware compression, go with one of the DL6000 series that has an optional hardware compression engine. But the presentations only say 1100MB/sec performance, and so customers install one, set up a single backup to a single virtual tape drive, and when it pegs at ~100MB/sec they think they have been lied to. The other complaint I hear a lot is the claim of 3:1 compression. Almost every vendor puts that in their literature as if it is a solid fact, and not a typical value. I had a customer once get so mad they almost yanked the whole box out and made the vendor take it back because they bought a 10TB VTL, which they sized on the assumption of 3:1 compression. Never mind that the compression they were getting on their existing LTO tape library was on 1.2:1, they were told the VTL would do 3:1, so it should. I had another customer almost throw out IBM because they bought 12 new 3592 tape drives, and they wouldn't perform anywhere near their rated performance. Never mind the fact that data was coming in through a single GigE connection, and the 12 tape drives had an aggregate throughput rating at about times that. Customers looking to purchase any tape or disk technology would be wise to ask questions about how performance numbers were achieved, and look at their own situation to see what results they should expect. Best Regards, John D. Schneider Sr. System Administrator - Storage Sisters of Mercy Health System 3637 South Geyer Road St. Louis, MO. 63127 Email: [EMAIL PROTECTED] Office: 314-364-3150, Cell: 314-486-2359 -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Prather, Wanda Sent: Friday, June 08, 2007 10:56 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? ...I understood restore performance suffered with a VTL - the way it has been described to me is that, should a restore need to come from a volume that has been destaged from disk to tape in the VTL, then a restore of a single file from the volume would first have to wait for the vtl to rebuild the tape on disk? Or have I got the wrong end of the stick? Um. Both. Most VTL's are disk-only devices that emulate tape, and do not have the staging issue you describe. Many VTL's will make restores FASTER because the tape mount time goes from potentially minutes to a second or less. (You also don't have to worry about collocating data in a VTL, so your migration times are generally faster as well.) Now that goes with a caveat - you have to PIN YOUR VENDOR TO THE WALL and get documentation about throughput rates. ALL VTL's work about the same way, but they all have different hardware inside the box, so you can get drastically different results. You can easily create a case where restoring 1 VERY LARGE file will take longer on a slow VTL than with fast tape (Say a TS1120, which run get more than 100MB/sec.) It depends
Re: How to Incorporate a CDL into TSM environment?
You go John! (And a BIG ditto on the compression rate issue - I've NEVER had a customer that got 3:1 over the whole TSM environment.) And let's step back a minute for a sanity check and ask, what IS a VTL anyway? It's disk with some cache and software in front. So if you need to back up 20 TB of disk, why not do as Kelly says and just buy another 20 TB of disk? Answer: In most cases, people buy a 20TB VTL because it's cheaper than adding another 20 TB in their disk array of choice. Why do you think that is? Is it because the vendors are really nice guys? Well, they may be really nice guys, but it's not because they want to give disk away. It's because the VTLs are built of A LESS EXPENSIVE KIND OF DISK. The cheaper disk is slower. Using cheaper disk, the VTL vendors have made it practical and cost effective to eliminate tape backups, FOR SOME CUSTOMERS. When people say they can back up or restore with a VTL faster than tape, it may mean 1) they are replacing slow tape drives 2) they are eliminating tape mount times 3) they no longer have to wait for a tape drive It doesn't mean there aren't cases where tape is faster. There are cases where a VTL really rocks. My favorite is using a VTL for OFFSITE storage and backing up to it directly over fibre. In case of a major problem, you aren't limited in the number of tape drives you have available for restore (you ARE still limited by the size of your fibre pipe). You don't have to physically move tapes around, and the media never leaves your control (If I never spend another minute doing a manual audit looking for misplaced tapes...etc.). And you don't have to collocate in a VTL, since there is zero effective tape mount time. And it is a good solution for people who want to do more Lan-free backups, and are short of tape drives. But you should be buying a VTL for one of THOSE reasons, not for raw speed. You can always create a scenario where you get down to the actual device speed of the underlying technology and hit that bottleneck. Many people never run into that scenario. But some do. Also, FWIW, tape is still cheaper per MB of storage than a VTL. There are price points where they are comparable, or where the benefits of a VTL outweigh the cost differential. But in general, the larger your site in terms of TB to store, the more difference you will see in cost if you go with a VTL vs. tape, with tape still being lower. You gotta first know what you are trying to do, THEN figure out where your bottlenecks are, THEN figure out what technology matches your need and fits your budget. Wanda (I think I'm done for the day now and I'm sure glad it's Friday) Prather From: ADSM: Dist Stor Manager on behalf of Schneider, John Sent: Fri 6/8/2007 1:00 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: How to Incorporate a CDL into TSM environment? Greetings, A lot of the chatter about VTL's being good or bad seems to stem from which vendors you listen to, and what they are trying to sell you. There are a lot of dogmatic statements made by people on both sides of this issue, usually by people with no personal experience about what they are talking about. Somebody has fed them a sales line and they dutifully parrot it back. EMC sold their CDL product for about two years before IBM entered the market. During that time you would not believe how many times I heard IBM pooh-pooh the CDL saying it wasn't a good fit for TSM, didn't perform well, whatever they had to say to compete against it. I even heard someone recently say it was against the law to use a CDL if you used the IBM drivers to talk to it. Against what law exactly? Then after two years IBM came out with their VTL the TS7510, and almost immediately came out with a Redbook about it with a TSM chapter explaining why the TS7510 was such a good fit for TSM! Huh? And not because it was a better product than the EMC one, it was actually slightly slower and only scaled to about a fourth the size of the largest EMC VTL. The only difference is that now IBM had something in the marketplace, and that changed everything. As Wanda has said, a lot of the distinctions fall down to how you use the VTL, and if your expectations are set correctly. It is easy for a vendor presentation to promise the moon without qualifying it's claims. A single-engine DL4100 from EMC can sustain a 1100MB/sec (3.7 TB per hour) write speed like they claim IF: 1) You are writing multiple simultaneous virtual tape streams (like 16 or more), 2) You balance the I/O across at least 4 FC streams coming in the VTL engine, 3) You have at least 5 or more disk drawers to spread out the I/O load. 4) You are not compressing at the VTL engine. If you compress at the VTL engine, your performance will drop off, perhaps as low as a third as fast. This is because the compression is done in software. If you want hardware
Re: How to Incorporate a CDL into TSM environment?
Hi Everyone, I have 1 question about the order of the steps that I must take to fold the CDL into an environment where I will still have a disk storage pool, but will migrate the data to the cdl and hold it there for 30 days before moving it to physical tape media. (Which in my case is LTO2.) Here is what I have planned so far when we implement this: Define CDL library (Emulating IBM 3584) Define path from the server to the CDL library Define drives (Emulating LTO2) Define drive paths Define device class for LTO2 Define primary storage pools with hi=90, lo=70, migdelay=30, next=tape stgpool Label tapes: label libvol library_name search=yes checkin=scratch volrange=A0,A00203 Update the disk storage pools so that they have a next storage pool directed to the cdl storage pool After this point I'm a little confused as to when migrations the backups of the pools should be done and in what order. So, for example: AIX -- disk stgpool AIX_CDL -- cdl sequential access stgpool AIX_TAPE -- sequential access stgpool LTO2 AIX_COPY -- offsite stgpool LTO2 Would I do the following? I guess I'm getting a little confused on exactly what order of operations would be best? Client bkups to AIX pool After 24 hrs. migrate AIX pool to AIX_CDL pool Backup stgpool AIX_CDL to AIX_COPY Backup stgpool AIX to AIX_COPY Migrate data to AIX_TAPE after 30 days Backup stgpool AIX_TAPE to AIX_COPY Also, I have heard that you can turn compression on at the CDL drive level. I have purchased an EMC CDL 4400. Would it be best to turn compression on at this level? Or is it possible to define the device class with ultrium2c instead of just ultrium2? Or should I just turn TSM client compression on the larger servers such as Oracle databases and Lotus Notes? I don't want to have too negative of a performance impact, but yet I can't afford to not compress anything... I'm trying to wrap my brain around this whole concept and the best methods/practices to use the CDL. Any thoughts/opinions are greatly appreciated! Joni Moyer Highmark Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966 Fax: (717) 302-9826 [EMAIL PROTECTED] Prather, Wanda [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/08/2007 02:25 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: How to Incorporate a CDL into TSM environment? You go John! (And a BIG ditto on the compression rate issue - I've NEVER had a customer that got 3:1 over the whole TSM environment.) And let's step back a minute for a sanity check and ask, what IS a VTL anyway? It's disk with some cache and software in front. So if you need to back up 20 TB of disk, why not do as Kelly says and just buy another 20 TB of disk? Answer: In most cases, people buy a 20TB VTL because it's cheaper than adding another 20 TB in their disk array of choice. Why do you think that is? Is it because the vendors are really nice guys? Well, they may be really nice guys, but it's not because they want to give disk away. It's because the VTLs are built of A LESS EXPENSIVE KIND OF DISK. The cheaper disk is slower. Using cheaper disk, the VTL vendors have made it practical and cost effective to eliminate tape backups, FOR SOME CUSTOMERS. When people say they can back up or restore with a VTL faster than tape, it may mean 1) they are replacing slow tape drives 2) they are eliminating tape mount times 3) they no longer have to wait for a tape drive It doesn't mean there aren't cases where tape is faster. There are cases where a VTL really rocks. My favorite is using a VTL for OFFSITE storage and backing up to it directly over fibre. In case of a major problem, you aren't limited in the number of tape drives you have available for restore (you ARE still limited by the size of your fibre pipe). You don't have to physically move tapes around, and the media never leaves your control (If I never spend another minute doing a manual audit looking for misplaced tapes...etc.). And you don't have to collocate in a VTL, since there is zero effective tape mount time. And it is a good solution for people who want to do more Lan-free backups, and are short of tape drives. But you should be buying a VTL for one of THOSE reasons, not for raw speed. You can always create a scenario where you get down to the actual device speed of the underlying technology and hit that bottleneck. Many people never run into that scenario. But some do. Also, FWIW, tape is still cheaper per MB of storage than a VTL. There are price points where they are comparable, or where the benefits of a VTL outweigh the cost differential. But in general, the larger your site in terms of TB to store, the more difference you will see in cost if you go with a VTL vs. tape, with tape still being lower. You gotta first know what you are trying to do, THEN figure out where
Re: How to Incorporate a CDL into TSM environment?
Maybe this will help... What we did with our CDLs and we are happy with it: - We defined a Scalar i2000 tape library - 8 SDLT tape drives in each VTL (2 CDLs per TSM server(long story)) We did eight drives in library because we calculated that would be about all our hardware could handle when doing Backup Storage pool to our physical 3590E drives. It is never a good idea to starve an actual tape drive. We wanted a different virtual tape drives so we would not have any driver version conflicts with the physical tape drives. - 100GB cartridges. The consultant said that was a good size, it works and we did not try any other sizes. We backup to disk pools unless the file is greater than 2GB. This was little slower to tape because of the stops and starts. We do compression on the CDL and get nearly 2:1 on average. This makes VTL better than straight disks. We let TSM do all data movement. We do not have the CDL do any tape to tape copies. A backup takes the path of: Host -- AIX_Disks -- CDL All copy tapes are physical tapes. Each TSM server moves about 2TB per night. When we added the CDL's, we simply changed where our primary tape pools where and waited our retention period for attrition to reduce the primary tapes, then moved the rest to the CDL. Reclaiming to a different pool is a really nice feature. This config may change when we do our server refresh this year. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Joni Moyer Sent: Friday, June 08, 2007 2:02 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? Hi Everyone, I have 1 question about the order of the steps that I must take to fold the CDL into an environment where I will still have a disk storage pool, but will migrate the data to the cdl and hold it there for 30 days before moving it to physical tape media. (Which in my case is LTO2.) Here is what I have planned so far when we implement this: Define CDL library (Emulating IBM 3584) Define path from the server to the CDL library Define drives (Emulating LTO2) Define drive paths Define device class for LTO2 Define primary storage pools with hi=90, lo=70, migdelay=30, next=tape stgpool Label tapes: label libvol library_name search=yes checkin=scratch volrange=A0,A00203 Update the disk storage pools so that they have a next storage pool directed to the cdl storage pool After this point I'm a little confused as to when migrations the backups of the pools should be done and in what order. So, for example: AIX -- disk stgpool AIX_CDL -- cdl sequential access stgpool AIX_TAPE -- sequential access stgpool LTO2 AIX_COPY -- offsite stgpool LTO2 Would I do the following? I guess I'm getting a little confused on exactly what order of operations would be best? Client bkups to AIX pool After 24 hrs. migrate AIX pool to AIX_CDL pool Backup stgpool AIX_CDL to AIX_COPY Backup stgpool AIX to AIX_COPY Migrate data to AIX_TAPE after 30 days Backup stgpool AIX_TAPE to AIX_COPY Also, I have heard that you can turn compression on at the CDL drive level. I have purchased an EMC CDL 4400. Would it be best to turn compression on at this level? Or is it possible to define the device class with ultrium2c instead of just ultrium2? Or should I just turn TSM client compression on the larger servers such as Oracle databases and Lotus Notes? I don't want to have too negative of a performance impact, but yet I can't afford to not compress anything... I'm trying to wrap my brain around this whole concept and the best methods/practices to use the CDL. Any thoughts/opinions are greatly appreciated! Joni Moyer Highmark Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966 Fax: (717) 302-9826 [EMAIL PROTECTED] Prather, Wanda [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/08/2007 02:25 PM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: How to Incorporate a CDL into TSM environment? You go John! (And a BIG ditto on the compression rate issue - I've NEVER had a customer that got 3:1 over the whole TSM environment.) And let's step back a minute for a sanity check and ask, what IS a VTL anyway? It's disk with some cache and software in front. So if you need to back up 20 TB of disk, why not do as Kelly says and just buy another 20 TB of disk? Answer: In most cases, people buy a 20TB VTL because it's cheaper than adding another 20 TB in their disk array of choice. Why do you think that is? Is it because the vendors are really nice guys? Well, they may be really nice guys, but it's not because they want to give disk away. It's because the VTLs are built of A LESS EXPENSIVE KIND OF DISK. The cheaper disk is slower. Using cheaper disk, the VTL vendors have made it practical and cost effective to eliminate tape backups, FOR
Re: How to Incorporate a CDL into TSM environment?
Wanda, We appear to have very different sets of data, because what I've seen in the VTL and backup world is very different than what you're describing. With respect to what you've seen, let me describe what I've seen. VTLs are using the same type of disks that any other ATA-based storage arrays are using. If you're saying they're not using Fibre-Channel - of course they're not. But do you need Fibre Channel drives to receive 10-20 streams of data? I don't think so. Fibre Channel drives are perfect for random I/O, which means they're probably perfect for a TSM disk storage pool, as it may be receiving hundreds of simultaneous backup jobs, which is very different than receiving 10-20-50 streams that were designed to go to tape. ATA drives do very well in streaming operations -- exactly what backups going to tape generate. In fact, I've seen tests where ATA drives actually outperform Fibre Channel drives in streaming ops. Yes, tape is still cheaper, but if you compare the price of a large VTL with de-dupe to an equivalently sized tape library, they'll be a lot closer than you think -- and the power and cooling will surprise you too. I've seen many scenarios where the VTL's power and cooling needs were less than the tape library. The people that I'm working with are not buying VTLs because they're cheaper; in many cases they're more expensive than an ATA-based array of equivalent capacity. They're buying a VTL for the things VTLs bring to the table, and those things come in play much more when we are talking about large environment. The first is ease of management. It's one thing to buy a 20 TB array and put that behind a single TSM server. It's another to buy that 20 TB array and split it up into properly sized partitions for several TSM servers. Provisioning is just as big of a pain in the TSM world as it is in the online world, and VTLs remove that problem. They use thin and/or over-provisioning where each server only consumes the amount of storage it sends to the VTL, not the amount you said it could have. The second thing VTLs bring to the table is hardware compression. Notice I said hardware compression. I'm not a fan of VTL software compression. So I get to buy a 20 TB VTL that's a little more expensive than a 20 TB ATA disk array, but what I get is 30-40 TB VTL, depending on my compression ratio -- and I don't lose performance. Then, of course, there's de-dupe, which most surveys are showing to be the got-to-have technology of this year. It's here. It's real. And it really does shrink the amount of disk you need to use by a factor of 10-20:1, and even more depending on how you do your backups. VTLs do indeed increase the speed of almost anyone's backups. It's not that disk/VTL is technically faster than tape. IMO, tapes are now much faster than disk. The reason that VTL/disk can outperform tape is that disk can go whatever speed your backup is going and tape cannot. If I send an 80 MB/s tape a 10 MB/s backup, it will shoe-shine and actually write 5 MB/s. A VTL would write 10 MB/s. Most environments never get anywhere near their tape's capabilities and about half or so are getting a small fraction of their tape drive's capabilities. VTL and disk bring THIS to the table. And yes, what ends up happening in almost every environment I've put a VTL in is that backups go faster. (There are some bad VTLs that actually slow down backups.) Finally, to one of the original questions of why would anyone use a VTL in a TSM environment, I say the following. Most VTL companies are telling me that a significant proportion of their customers are TSM customers. Why is that? First, TSM customers have the same reasons that other products' customers have for going to VTL. Second, TSM customers can benefit from a significantly higher number of tape drives being available -- without having to pay for those tape drives. Reclamation goes much faster with two virtual drives than two physical drives, and you can throw as many drives at reclamation as you wish. Just my $.02. --- W. Curtis Preston -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Prather, Wanda Sent: Friday, June 08, 2007 11:25 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? You go John! (And a BIG ditto on the compression rate issue - I've NEVER had a customer that got 3:1 over the whole TSM environment.) And let's step back a minute for a sanity check and ask, what IS a VTL anyway? It's disk with some cache and software in front. So if you need to back up 20 TB of disk, why not do as Kelly says and just buy another 20 TB of disk? Answer: In most cases, people buy a 20TB VTL because it's cheaper than adding another 20 TB in their disk array of choice. Why do you think that is? Is it because the vendors are really nice guys? Well, they may be really nice guys, but it's not because they want to give disk away. It's
Re: How to Incorporate a CDL into TSM environment?
Joni, Disclaimer: Before the job I have now, I spent two years at EMC as a TSM consultant, helping customers use EMC products (including the CDL) in their TSM environment. I have implemented a lot of CDLs with TSM. I am still doing this at my current job. So if I seem biased in favor of this solution, well, I guess I am. I have two resources for you. The first is a whitepaper I was permitted to write while I was at EMC. You will need a Powerlink account to get access to it, but you can sign up for one for free if you don't have one already: https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1 /en_US/Offering_Technical/White_Paper/H2095_TSM_WP_ldv.pdf?mtcs=ZXZlbnRU eXBlPUttQ2xpY2tTZWFyY2hSZXN1bHRzRXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwMTkw YWRhLGRhdGFTb3VyY2U9RENUTV9lbl9VU18w You can also find it in Powerlink using the search string TSM CDL. It was the first link listed when I did it. That search will also give you lots of other valuable hits. The second is a Redbook from IBM about their VTL solution. There is a chapter in it specific to how a VTL can help in a TSM environment. Here is the link: http://www.redbooks.ibm.com/abstracts/sg247189.html?Open My favorite way to use the CDL with TSM is: 1) Design the solution and size the CDL so that most or all Primary storage pools can fit on the CDL. This will give you the most flexible solution, and the fastest restores. There is not a lot of advantage to a CDL if you only make is as big as your original disk storage pool would be. Don't think of it as a temporary holding place until the data can be migrated to tape the next day. Think of it as the tape library for your primary storage pools. 2) Emulate a IBM3584 library with LTO1 tape drives. Use the IBM driver for the library and tape drives. I have had excellent success on both AIX and Windows with that emulation working trouble free. The IBM3584 emulation can support up to 4096 slots and 72 tape drives, more than enough for most people's environment. 3) Don't be afraid to create enough virtual tape drives to solve the problem. Even in a small TSM server (30-50) clients, I create 16 virtual tape drives. In a larger environment (50-200 clients) I create 32-64. It takes a little longer to set them up, but afterwards you will reap the benefits. 4) After you have created the virtual tape library, drives, and tapes, define it to TSM the way you ordinarily would with a real tape library and drives. Simplify the setup by using the setting to automatically discover the serial and element numbers. 5) Direct the client backups directly to the virtual tapes, instead of going to disk storage pool. If you have enough virtual tape drives and spread your client schedules out into groups, you can drive the data straight through to virtual tape. This will save you hours of time in the schedule not having to migrate from disk to tape. There may be some older slower clients you could leave behind on disk storage pool if you have a lot of clients like that, it is up to you. 6) There is no particular advantage to collocating storage pools in a virtual tape environment. You can restore a client spread across two dozen virtual tapes (almost) just as fast as if it was two virtual tapes, because the mounts, dismounts, and seeks are so fast. By turning off collocation, you can get better overal utilization of the disk space in the CDL. 7) Doing backup storage pools from primary to copy pools will only require half as many real tape drives as it did before, so you can increase MAXPR if you need to to speed things up. 8) In a real tape environment, most people have to set aside time in the daily schedule to reclaim both primary storage pool and copy storage pool tapes. They need to have a script or admin schedule to start and stop reclamation, to keep it out of the way of other processes that would require tapes. If your primary storage pool is on a CDL, set the reclamation threshold at 50% (or whatever you prefer) and leave it there. As long as you have enough virtual tape drives so that you aren't running out of virtual drives, reclamation will simply grab a couple virtual tape drives, mount the tapes, and go about reclaiming tapes when it needs to. The whole process will happen in the background and you won't even notice it. This can free up hours so you can reclaim more copy storage pool tapes, or whatever. Those are my thoughts. Feel free to ask me other questions if you need to. Also, don't hesitate to ask for help from the EMC BURA (BackUp, Recovery, and Archive) Practice at EMC. That is what they are there for. Best Regards, John D. Schneider Sr. System Administrator - Storage Sisters of Mercy Health System 3637 South Geyer Road St. Louis, MO. 63127 Email: [EMAIL PROTECTED] Office: 314-364-3150, Cell: 314-486-2359 -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Joni Moyer Sent: Tuesday, June
Re: How to Incorporate a CDL into TSM environment?
Hi John, Thank you so much for all of the information! I have a lot of data to go through, but it is a tremendous help! I was just wondering what the name of the document you wrote is called? I tried the link and it said that it had moved on Powerlink. Thanks again! Joni Moyer Highmark Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966 Fax: (717) 302-9826 [EMAIL PROTECTED] Schneider, John [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/05/2007 10:56 AM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: How to Incorporate a CDL into TSM environment? Joni, Disclaimer: Before the job I have now, I spent two years at EMC as a TSM consultant, helping customers use EMC products (including the CDL) in their TSM environment. I have implemented a lot of CDLs with TSM. I am still doing this at my current job. So if I seem biased in favor of this solution, well, I guess I am. I have two resources for you. The first is a whitepaper I was permitted to write while I was at EMC. You will need a Powerlink account to get access to it, but you can sign up for one for free if you don't have one already: https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1 /en_US/Offering_Technical/White_Paper/H2095_TSM_WP_ldv.pdf?mtcs=ZXZlbnRU eXBlPUttQ2xpY2tTZWFyY2hSZXN1bHRzRXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwMTkw YWRhLGRhdGFTb3VyY2U9RENUTV9lbl9VU18w You can also find it in Powerlink using the search string TSM CDL. It was the first link listed when I did it. That search will also give you lots of other valuable hits. The second is a Redbook from IBM about their VTL solution. There is a chapter in it specific to how a VTL can help in a TSM environment. Here is the link: http://www.redbooks.ibm.com/abstracts/sg247189.html?Open My favorite way to use the CDL with TSM is: 1) Design the solution and size the CDL so that most or all Primary storage pools can fit on the CDL. This will give you the most flexible solution, and the fastest restores. There is not a lot of advantage to a CDL if you only make is as big as your original disk storage pool would be. Don't think of it as a temporary holding place until the data can be migrated to tape the next day. Think of it as the tape library for your primary storage pools. 2) Emulate a IBM3584 library with LTO1 tape drives. Use the IBM driver for the library and tape drives. I have had excellent success on both AIX and Windows with that emulation working trouble free. The IBM3584 emulation can support up to 4096 slots and 72 tape drives, more than enough for most people's environment. 3) Don't be afraid to create enough virtual tape drives to solve the problem. Even in a small TSM server (30-50) clients, I create 16 virtual tape drives. In a larger environment (50-200 clients) I create 32-64. It takes a little longer to set them up, but afterwards you will reap the benefits. 4) After you have created the virtual tape library, drives, and tapes, define it to TSM the way you ordinarily would with a real tape library and drives. Simplify the setup by using the setting to automatically discover the serial and element numbers. 5) Direct the client backups directly to the virtual tapes, instead of going to disk storage pool. If you have enough virtual tape drives and spread your client schedules out into groups, you can drive the data straight through to virtual tape. This will save you hours of time in the schedule not having to migrate from disk to tape. There may be some older slower clients you could leave behind on disk storage pool if you have a lot of clients like that, it is up to you. 6) There is no particular advantage to collocating storage pools in a virtual tape environment. You can restore a client spread across two dozen virtual tapes (almost) just as fast as if it was two virtual tapes, because the mounts, dismounts, and seeks are so fast. By turning off collocation, you can get better overal utilization of the disk space in the CDL. 7) Doing backup storage pools from primary to copy pools will only require half as many real tape drives as it did before, so you can increase MAXPR if you need to to speed things up. 8) In a real tape environment, most people have to set aside time in the daily schedule to reclaim both primary storage pool and copy storage pool tapes. They need to have a script or admin schedule to start and stop reclamation, to keep it out of the way of other processes that would require tapes. If your primary storage pool is on a CDL, set the reclamation threshold at 50% (or whatever you prefer) and leave it there. As long as you have enough virtual tape drives so that you aren't running out of virtual drives, reclamation will simply grab a couple virtual tape drives, mount the tapes, and go about
Re: How to Incorporate a CDL into TSM environment?
Joni, Sorry, yes the URL I posted must have been specific to my search, and the results thrown away afteward. The whitepaper is called EMC CLARiiON Backup Storage Solutions: The Value of CLARiiON Disk Library with TSM. If you log on to Powerlink, then search on TSM CDL, one of the results coming back will be that whitepaper; I tried it earlier today to make sure they still had it. Best Regards, John D. Schneider Sr. System Administrator - Storage Sisters of Mercy Health System 3637 South Geyer Road St. Louis, MO. 63127 Email: [EMAIL PROTECTED] Office: 314-364-3150, Cell: 314-486-2359 -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Joni Moyer Sent: Tuesday, June 05, 2007 1:18 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to Incorporate a CDL into TSM environment? Hi John, Thank you so much for all of the information! I have a lot of data to go through, but it is a tremendous help! I was just wondering what the name of the document you wrote is called? I tried the link and it said that it had moved on Powerlink. Thanks again! Joni Moyer Highmark Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966 Fax: (717) 302-9826 [EMAIL PROTECTED] Schneider, John [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU 06/05/2007 10:56 AM Please respond to ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU To ADSM-L@VM.MARIST.EDU cc Subject Re: How to Incorporate a CDL into TSM environment? Joni, Disclaimer: Before the job I have now, I spent two years at EMC as a TSM consultant, helping customers use EMC products (including the CDL) in their TSM environment. I have implemented a lot of CDLs with TSM. I am still doing this at my current job. So if I seem biased in favor of this solution, well, I guess I am. I have two resources for you. The first is a whitepaper I was permitted to write while I was at EMC. You will need a Powerlink account to get access to it, but you can sign up for one for free if you don't have one already: https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1 /en_US/Offering_Technical/White_Paper/H2095_TSM_WP_ldv.pdf?mtcs=ZXZlbnRU eXBlPUttQ2xpY2tTZWFyY2hSZXN1bHRzRXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwMTkw YWRhLGRhdGFTb3VyY2U9RENUTV9lbl9VU18w You can also find it in Powerlink using the search string TSM CDL. It was the first link listed when I did it. That search will also give you lots of other valuable hits. The second is a Redbook from IBM about their VTL solution. There is a chapter in it specific to how a VTL can help in a TSM environment. Here is the link: http://www.redbooks.ibm.com/abstracts/sg247189.html?Open My favorite way to use the CDL with TSM is: 1) Design the solution and size the CDL so that most or all Primary storage pools can fit on the CDL. This will give you the most flexible solution, and the fastest restores. There is not a lot of advantage to a CDL if you only make is as big as your original disk storage pool would be. Don't think of it as a temporary holding place until the data can be migrated to tape the next day. Think of it as the tape library for your primary storage pools. 2) Emulate a IBM3584 library with LTO1 tape drives. Use the IBM driver for the library and tape drives. I have had excellent success on both AIX and Windows with that emulation working trouble free. The IBM3584 emulation can support up to 4096 slots and 72 tape drives, more than enough for most people's environment. 3) Don't be afraid to create enough virtual tape drives to solve the problem. Even in a small TSM server (30-50) clients, I create 16 virtual tape drives. In a larger environment (50-200 clients) I create 32-64. It takes a little longer to set them up, but afterwards you will reap the benefits. 4) After you have created the virtual tape library, drives, and tapes, define it to TSM the way you ordinarily would with a real tape library and drives. Simplify the setup by using the setting to automatically discover the serial and element numbers. 5) Direct the client backups directly to the virtual tapes, instead of going to disk storage pool. If you have enough virtual tape drives and spread your client schedules out into groups, you can drive the data straight through to virtual tape. This will save you hours of time in the schedule not having to migrate from disk to tape. There may be some older slower clients you could leave behind on disk storage pool if you have a lot of clients like that, it is up to you. 6) There is no particular advantage to collocating storage pools in a virtual tape environment. You can restore a client spread across two dozen virtual tapes (almost) just as fast as if it was two virtual tapes, because the mounts, dismounts, and seeks are so fast. By turning off collocation, you can get better overal
