Re: Fw: How to Incorporate a CDL into TSM environment?
2) Sepaton does hardware compression so as far as I can tell it performs like compression on physical drives, I imagine other vendors may also use hardware compression. Besides what's important to me is that my TSM clients and TSM server are not taking a performance hit from doing compression because compression has been off loaded to the VTL. Even if the VTL performs slower when compressing vs not compressing, it's not an issue unless your TSM server can over whelm your VTL. Any time your TSM server or TSM client is doing compression, your backup session performance is impacted. 3) This is an area that merits actual testing which is one reason I expressed it as a personal opinion. This seems to me that the results would be very dependent upon the exact hardware configuration being tested. At the least, using a VTL would off-load the file system management activities from the TSM server to the VTL freeing up TSM CPU cycles to pump data from the input stream to the output stream. 4) Read the article by Curtis Preston (referenced in the "Just how does a VTL work?" thread) for a discussion on in-band versus out-of band de-dup and the effect on performance. I agree this technology does not have a long track record and due diligence in progressing on this, and other new backup technologies, is strongly advised. As I stated I have not used this as of "yet". 5) If your VTL must run FSCK against 100TB, yes you do have a problem but: 5A) At least your TSM server is up. 5B) Your TSM server is a much more complex system compared to a VTL. Hence is most likely more susceptible to crashing. Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Josef Weingand Sent: Sunday, June 17, 2007 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? 1.) yes 2.) pls give me vendor / machine type of VTLs which does compression without performance impacts. physical tape drives gets performance improvements of compression, but VTL gets normally much slower performance if compression is enabled 3.) you need compare the same technology. You need compare a FC Disk Subsystems, like DS4000, with a VTL, then you get the same performance (maybe even more) from a File Device Type as from a VTL. 4.)current de-dups does not bring enough performance for most of the environments, and in addition, most vendors have just announced it, but does not have it ready for GA yet. 5.)what happens if the VTL crashed Mit freundlichen Grüßen / Kind regards Josef Weingand Senior IT Specialist Technical Sales Systems Storage Mobil +49 171 55 26 783 - Homeoffice Tel. +49 8845 757421 Fax +49 171 13 5526783 email: [EMAIL PROTECTED] SMS/eMail: [EMAIL PROTECTED] IBM Deutschland GmbH Vorsitzender des Aufsichtsrats: Hans Ulrich Maerki Geschäftsführung: Martin Jetter (Vorsitzender), Rudolf Bauer, Christian Diedrich, Christoph Grandpierre, Matthias Hartmann, Thomas Fell, Michael Diemer Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 "Johnson, Milton" <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" 15.06.2007 21:08 Please respond to "ADSM: Dist Stor Manager" To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? Why a VTL vs FILE devclass volumes on local drives? 1) With a VTL you can do LAN free backups. 2) Data Compression: 2A) TSM Client does compression: Big performance hit on the client, slower backups/restores 2B) TSM server does compression (FILE devclass volumes on compressed file systems): Big performance hit on server, slower backups/reclaims/restores 2C) VTL does compression: No performance hits (just as with using physical drives with compression) 3) I doubt the TSM server could write large amounts of data simultaneously streaming in from 30 different clients to 30 different FILE devclasses volumes as fast as it can write that data to a fibre channel adapter. 3A) 30 input streams means 30 mounted FILE devclass volumes 3B) The drive heads would always be out of position for the next write. 4) Data Reduction/Data Deduplication/Content Aware Compression: What ever the VTL vendor calls it, you don't have that available with FILE devclass volumes. (I realize that from previous posts you are not comfortable with this technology, I myself have not used it.) 5) Do you really want to manage an AIX system with a 25TB, 50TB or a 100TB file system? After a system crash how long will a FSCK of 100Tb take? Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Allen S. Rout Sent: Thursday, June 14, 2007 3:22 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? >> On Wed, 13 Jun 2007 08:43:43 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> said: > The VTL is
Re: Fw: How to Incorporate a CDL into TSM environment?
>2.) pls give me vendor / machine type of VTLs which does compression >without performance impacts. Several of them are now using hardware compression. They use the same chip as the tape drives do, therefore, they get what you're asking for. (I don't want to give vendor names cause the names will change tomorrow.) >3.) you need compare the same technology. You need compare a FC Disk >Subsystems, like DS4000, with a VTL, then you get the same performance >(maybe even more) from a File Device Type as from a VTL. With a WHOLE LOT more management and none of the other cool stuff we're talking about. >4.)current de-dups does not bring enough performance for most of the >environments, and in addition, most vendors have just announced it, but >does not have it ready for GA yet. You hit the nail on the head there. Not sure if I'd say "most" environments, but definitely "many." If you look at the stats, most environments are actually quite small and will work with just about anything on the market. The currently GA de-dupe products (Data Domain, HDS, Diligent, NetApp) support 200-300 MB/s. That's 6-9 TB in an eight-hour window. That'll meet a LOT of people's needs. Falconstor & SEPATON are currently in limited availability (think something just beyond Beta). That doesn't necessarily mean it's buggy. It's a new technology and they want to know what customers should expect before they sell it to the world. I'm personally fine with using it in production as long as you also copy all your backups to tape. That copy will verify your original and will act as the backup of your backup if the de-dupe software does have a problem. 5.)what happens if the VTL crashed Same thing that happens when a disk array crashes or your tape library is caught in a fire. That's why it should never be your only copy of the data.
Re: Fw: How to Incorporate a CDL into TSM environment?
1.) yes 2.) pls give me vendor / machine type of VTLs which does compression without performance impacts. physical tape drives gets performance improvements of compression, but VTL gets normally much slower performance if compression is enabled 3.) you need compare the same technology. You need compare a FC Disk Subsystems, like DS4000, with a VTL, then you get the same performance (maybe even more) from a File Device Type as from a VTL. 4.)current de-dups does not bring enough performance for most of the environments, and in addition, most vendors have just announced it, but does not have it ready for GA yet. 5.)what happens if the VTL crashed Mit freundlichen Grüßen / Kind regards Josef Weingand Senior IT Specialist Technical Sales Systems Storage Mobil +49 171 55 26 783 - Homeoffice Tel. +49 8845 757421 Fax +49 171 13 5526783 email: [EMAIL PROTECTED] SMS/eMail: [EMAIL PROTECTED] IBM Deutschland GmbH Vorsitzender des Aufsichtsrats: Hans Ulrich Maerki Geschäftsführung: Martin Jetter (Vorsitzender), Rudolf Bauer, Christian Diedrich, Christoph Grandpierre, Matthias Hartmann, Thomas Fell, Michael Diemer Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 "Johnson, Milton" <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" 15.06.2007 21:08 Please respond to "ADSM: Dist Stor Manager" To ADSM-L@VM.MARIST.EDU cc Subject Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? Why a VTL vs FILE devclass volumes on local drives? 1) With a VTL you can do LAN free backups. 2) Data Compression: 2A) TSM Client does compression: Big performance hit on the client, slower backups/restores 2B) TSM server does compression (FILE devclass volumes on compressed file systems): Big performance hit on server, slower backups/reclaims/restores 2C) VTL does compression: No performance hits (just as with using physical drives with compression) 3) I doubt the TSM server could write large amounts of data simultaneously streaming in from 30 different clients to 30 different FILE devclasses volumes as fast as it can write that data to a fibre channel adapter. 3A) 30 input streams means 30 mounted FILE devclass volumes 3B) The drive heads would always be out of position for the next write. 4) Data Reduction/Data Deduplication/Content Aware Compression: What ever the VTL vendor calls it, you don't have that available with FILE devclass volumes. (I realize that from previous posts you are not comfortable with this technology, I myself have not used it.) 5) Do you really want to manage an AIX system with a 25TB, 50TB or a 100TB file system? After a system crash how long will a FSCK of 100Tb take? Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Allen S. Rout Sent: Thursday, June 14, 2007 3:22 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? >> On Wed, 13 Jun 2007 08:43:43 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> said: > The VTL is a Sepaton S2100-ES and yes it is disk only. > I don't see the benefit that a "tape backed" system would bring, how > does that really differ from a physical tape ATL with TSM providing a > DISKPOOL front end? Well, exactly. :) But the distinction I wanted to make clear was: if you've decided to store all your data on disk, then TSM has all the primitives necessary to make that disk manageable, and you can discard the intermediate appliance that makes the disk pretend to be a bunch of tape drives. That's what everyone's getting at when they talk about FILE devclasses. So if you bought 23 TB of slow disk plus a pretend-im-tape-box, then the tape box was a waste, if you're using TSM. If you're using something without TSM's volume primitives, it could be extremely important. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
I think you hit every nail right on the head, Milton. I would emphasize #5, as people tend to minimize how big of a deal it is to provision and administer large amounts of disk. A GOOD VTL (not all are good) will make that provisioning issue go away. As to de-dupe, it's considered by many to be the hottest technology right now, and it is real, and is saving lots of people lots of money. It actually (in most cases) makes a VTL roughly equivalent (if not cheaper) than a similarly-sized tape solution. --- W. Curtis Preston -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Johnson, Milton Sent: Friday, June 15, 2007 12:09 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? Why a VTL vs FILE devclass volumes on local drives? 1) With a VTL you can do LAN free backups. 2) Data Compression: 2A) TSM Client does compression: Big performance hit on the client, slower backups/restores 2B) TSM server does compression (FILE devclass volumes on compressed file systems): Big performance hit on server, slower backups/reclaims/restores 2C) VTL does compression: No performance hits (just as with using physical drives with compression) 3) I doubt the TSM server could write large amounts of data simultaneously streaming in from 30 different clients to 30 different FILE devclasses volumes as fast as it can write that data to a fibre channel adapter. 3A) 30 input streams means 30 mounted FILE devclass volumes 3B) The drive heads would always be out of position for the next write. 4) Data Reduction/Data Deduplication/Content Aware Compression: What ever the VTL vendor calls it, you don't have that available with FILE devclass volumes. (I realize that from previous posts you are not comfortable with this technology, I myself have not used it.) 5) Do you really want to manage an AIX system with a 25TB, 50TB or a 100TB file system? After a system crash how long will a FSCK of 100Tb take? Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Allen S. Rout Sent: Thursday, June 14, 2007 3:22 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? >> On Wed, 13 Jun 2007 08:43:43 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> said: > The VTL is a Sepaton S2100-ES and yes it is disk only. > I don't see the benefit that a "tape backed" system would bring, how > does that really differ from a physical tape ATL with TSM providing a > DISKPOOL front end? Well, exactly. :) But the distinction I wanted to make clear was: if you've decided to store all your data on disk, then TSM has all the primitives necessary to make that disk manageable, and you can discard the intermediate appliance that makes the disk pretend to be a bunch of tape drives. That's what everyone's getting at when they talk about FILE devclasses. So if you bought 23 TB of slow disk plus a pretend-im-tape-box, then the tape box was a waste, if you're using TSM. If you're using something without TSM's volume primitives, it could be extremely important. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
Why a VTL vs FILE devclass volumes on local drives? 1) With a VTL you can do LAN free backups. 2) Data Compression: 2A) TSM Client does compression: Big performance hit on the client, slower backups/restores 2B) TSM server does compression (FILE devclass volumes on compressed file systems): Big performance hit on server, slower backups/reclaims/restores 2C) VTL does compression: No performance hits (just as with using physical drives with compression) 3) I doubt the TSM server could write large amounts of data simultaneously streaming in from 30 different clients to 30 different FILE devclasses volumes as fast as it can write that data to a fibre channel adapter. 3A) 30 input streams means 30 mounted FILE devclass volumes 3B) The drive heads would always be out of position for the next write. 4) Data Reduction/Data Deduplication/Content Aware Compression: What ever the VTL vendor calls it, you don't have that available with FILE devclass volumes. (I realize that from previous posts you are not comfortable with this technology, I myself have not used it.) 5) Do you really want to manage an AIX system with a 25TB, 50TB or a 100TB file system? After a system crash how long will a FSCK of 100Tb take? Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Allen S. Rout Sent: Thursday, June 14, 2007 3:22 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? >> On Wed, 13 Jun 2007 08:43:43 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> said: > The VTL is a Sepaton S2100-ES and yes it is disk only. > I don't see the benefit that a "tape backed" system would bring, how > does that really differ from a physical tape ATL with TSM providing a > DISKPOOL front end? Well, exactly. :) But the distinction I wanted to make clear was: if you've decided to store all your data on disk, then TSM has all the primitives necessary to make that disk manageable, and you can discard the intermediate appliance that makes the disk pretend to be a bunch of tape drives. That's what everyone's getting at when they talk about FILE devclasses. So if you bought 23 TB of slow disk plus a pretend-im-tape-box, then the tape box was a waste, if you're using TSM. If you're using something without TSM's volume primitives, it could be extremely important. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
>> On Wed, 13 Jun 2007 08:43:43 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> >> said: > The VTL is a Sepaton S2100-ES and yes it is disk only. > I don't see the benefit that a "tape backed" system would bring, how > does that really differ from a physical tape ATL with TSM providing > a DISKPOOL front end? Well, exactly. :) But the distinction I wanted to make clear was: if you've decided to store all your data on disk, then TSM has all the primitives necessary to make that disk manageable, and you can discard the intermediate appliance that makes the disk pretend to be a bunch of tape drives. That's what everyone's getting at when they talk about FILE devclasses. So if you bought 23 TB of slow disk plus a pretend-im-tape-box, then the tape box was a waste, if you're using TSM. If you're using something without TSM's volume primitives, it could be extremely important. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
The VTL is a Sepaton S2100-ES and yes it is disk only. I don't see the benefit that a "tape backed" system would bring, how does that really differ from a physical tape ATL with TSM providing a DISKPOOL front end? "Fewer tape-head-hours": I understand your confusion, there was no way to reduce the required tape-head-hours, but with a VTL if I need 30 physical tape drives then I configure the VTL as having 45 virtual drives, more than enough. There is no price difference for configuring my VTL as having 1 tape drive or 64 tape drives. Off topic: Configuration Based Pricing: I pray Sepaton will not start, and other vendors will not continue, a pricing scheme based up how you configure the software you PURCHASED. What's next, you have to may extra to your OS vendor for each file system you create? Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Allen S. Rout Sent: Tuesday, June 12, 2007 3:41 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? >> On Tue, 12 Jun 2007 12:16:27 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> said: > Why a VTL? With us we found that when we out grew our physical > library we would have to have to buy over 30 physical drives in order > to be able to do backups, restores, cut off-site tapes and reclaim > on/off site tapes in the time allowed. That amounted to some serious > money, more than our VTL costs. I'm interested in the details on this. The VTL is disk-only, or is it tape backed? I'm confused about how you need fewer tape-head-hours when you virtualize processes. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
>> On Tue, 12 Jun 2007 12:16:27 -0400, "Johnson, Milton" <[EMAIL PROTECTED]> >> said: > Why a VTL? With us we found that when we out grew our physical > library we would have to have to buy over 30 physical drives in > order to be able to do backups, restores, cut off-site tapes and > reclaim on/off site tapes in the time allowed. That amounted to > some serious money, more than our VTL costs. I'm interested in the details on this. The VTL is disk-only, or is it tape backed? I'm confused about how you need fewer tape-head-hours when you virtualize processes. - Allen S. Rout
Re: Fw: How to Incorporate a CDL into TSM environment?
Yes a virtual tape volume can be accessed only by one client at a time and if two processes/clients try to access the same volume at the same time one process/client must wait. Again smaller volume sizes decreases the chance that a contention would happen and also decrease the contention duration. Why a VTL? With us we found that when we out grew our physical library we would have to have to buy over 30 physical drives in order to be able to do backups, restores, cut off-site tapes and reclaim on/off site tapes in the time allowed. That amounted to some serious money, more than our VTL costs. When you also take into account the costs of the much larger DISKPOOL a physical tape library requires, growing a physical tape library in a TSM environment is not cheap. The VTL footprint is also smaller which also should considered in the total cost of ownership. Sorry, but we had to justify our VTL purchase. Thanks, H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Nicholas Cassimatis Sent: Tuesday, June 12, 2007 10:55 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? I'm not looking at the spinning through the volume to find the file, I'm focused on the fact that a volume can only be accessed by one client at a time. You have to read the data to be restored, which takes time. If you have one client reading the volume, any other access to that volume has to queue up. With a slow client (or a fast one pulling a large file), you can develop some access contention, which is a bottleneck that collocation resolves. That's why I still see collocation playing with VTL's. It all comes back to "Why do you want a VTL?" which is another way of asking, "What problem are you trying to solve/avoid?" I'm sure there are people who are getting VTL's because they have to spend their budget or they lose it - and the rest of us are jealous of them for that! But, as with most other technologies, implementing a VTL just moves the bottleneck/weakest link to another spot, which may not be the best solution for a given environment. Nick Cassimatis - Forwarded by Nicholas Cassimatis/Raleigh/IBM on 06/12/2007 11:41 AM - "ADSM: Dist Stor Manager" wrote on 06/12/2007 11:13:30 AM: > As I see it, there are two areas where you get performance hits when > restoring from non-collocated volumes: > 1) Tapes Mounts: In my experience my VTL makes this problem > insignificant. > > 2) Spinning Sequential Media: Yes, VTL volumes are sequential and if > you define your tapes as 50GB native and then with compression get > 100GB written to the tape, you may have to spin through 99.9GB of data > to retrieve a 0.1Gb file. However if you define 10GB volumes you only > have to spin through 1/5 of the data to reach your 0.1GB file. Also > with smaller volumes you are more likely to get "natural collocation" > because a client that writes directly to tape is more likely to fill up a tape. > Obviously if you define smaller and smaller volumes at some point you > will have a "tape mount bottle neck". > > Just one way to manage the trade offs. > > H. Milton Johnson
Re: Fw: How to Incorporate a CDL into TSM environment?
As I see it, there are two areas where you get performance hits when restoring from non-collocated volumes: 1) Tapes Mounts: In my experience my VTL makes this problem insignificant. 2) Spinning Sequential Media: Yes, VTL volumes are sequential and if you define your tapes as 50GB native and then with compression get 100GB written to the tape, you may have to spin through 99.9GB of data to retrieve a 0.1Gb file. However if you define 10GB volumes you only have to spin through 1/5 of the data to reach your 0.1GB file. Also with smaller volumes you are more likely to get "natural collocation" because a client that writes directly to tape is more likely to fill up a tape. Obviously if you define smaller and smaller volumes at some point you will have a "tape mount bottle neck". Just one way to manage the trade offs. H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Nicholas Cassimatis Sent: Monday, June 11, 2007 12:45 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Fw: How to Incorporate a CDL into TSM environment? A couple of comments about what Wanda said about collocation and VTL's: At some point, you do have a finite number of mount points defined for your VTL. Even if virtual tape "mounts" are near instant, there is still some overhead. A large number of clients "mounting" virtual tape after virtual tape after virtual tape will have some sort of negative effect on the overall throughput of their sessions. I'm not saying it will be significant, but it could get there, depending on the VTL technology. A few milliseconds here, a few there, and a controller that gets bogged down under the mount request queue, you could cause yourself some issues. And don't forget that virtual tapes are the same as physical tapes in one major factor - they're sequential! So non-collocated storage pools could have multiple clients asking for the same virtual tape, so there would be a wait queue for the virtual tape. A VTL doesn't resolve this type of contention, as it's at the TSM level. I would argue that the cost of creating collocated volumes in a VTL is negligible, and still has benefits on the restore side. To echo a number of others comments in the thread - if you don't plan it out right, it's not going to work. That goes for just about anything, from vacations to VTL's! Nick Cassimatis - Forwarded by Nicholas Cassimatis/Raleigh/IBM on 06/11/2007 01:33 PM - > And you don't have to collocate in a VTL, since there is zero > effective tape mount time.