Re: tsm and data domain
Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments.
Re: tsm and data domain
On Jun 16, 2011, at 8:34 PM, Paul Zarnowski wrote: At 05:59 PM 6/16/2011, Nick Laflamme wrote: We need to do a bake-off -- or study someone else's -- between using deduplication in a DataDomain box and using both client-side deduplication and server-side deduplication in TSM V6 and then writing to relatively inexpensive, relatively simple (but replicating) storage arrays. However, we keep pushing the limits of stability with our TSM V6 servers, so we haven't dared tried such a back-off yet. Nick, We are heading down this path. My analysis is that in a TSM environment, the fairly low dedup ratio does not justify the higher price of duduping VTLs. Commodity disk arrays have gotten very inexpensive. We're using DS3500s which are nice building blocks. We put some behind IBM SVCs for servers, some attached to TSM or Exchange servers (without SVC). Common technology - different uses. We use them for both TSM DB, LOG and FILE (different RPM disks, obviously). Using cheap disk vs VTLs has different pros and cons. using disk allows for source-mode (client-side) dedup, which a VTL will not do. VTLs, on the other hand, allow for global dedup pools and LAN-free virtual tape targets. deduping VTLs will be more effective in TSM environments where you have known duplicate data, such as lots of Oracle or MSSQL full backups, or other cases where you have multiple full backups. For normal progressive incremental file backups, however, TSM already does a good job of reducing data so VTL dedup doesn't get you as much, and in this case IMHO cheap disk is, well, cheaper and gets you source-mode dedup as well. We are in process of implementing this, but I know a few others are a bit further along. It's too bad there isn't a USA-based users group for TSM that meets annually or even semi-annually for things like user presentations and panels on topics like this. :-) I'd love to go to a TSM Workshop at some university campus to geek out on topics like this. Dedupe ratios are all over the place for us. We've got some in the high teens, and I already mentioned the low end, low single-digits. Part of me wishes we'd broken our library volumes out into smaller replication pools (and corresponding TSM library pools) so we could get a little more granularity on dedupe ratios, but what I really want is volume-by-volume (or file-by-file for NFS mounts? Directory-by-directory?) dedupe ratios. Adding a copy storage pool on the same DDR is cheap from a DDR point of view; it seems to dedupe great when I do that. The TSM DB size becomes the limiting factor in that case. We will continue to use TSM Backup Stgpool to replicate offsite. We're replicating at the HW level; TSM doesn't know it. That makes me a little nervous. I forgot one thing I don't like: having specific cleaning cycles on the DDR is annoying. We run it two or three times a week on a couple of our DDRs to keep them from getting too high (above 80%) in utilization, but I wish it just constantly did its own garbage collection. ..Paul Nick
Re: tsm and data domain
We just implemented a pair of DD880's. Here are some thoughts/comments about this implementation in no particular order . . . - We like them a lot. - Fairly straight forward system. Did not go to training - just spent time with the manual. - The tsm servers are not real high performers, so they don't put a big I/O load on the DD. - The TSM servers are old Sun T2000. - Each DD has both primary data for onsite TSM servers and DR copy for the other. - Used as NFS filepools using 30gb file volumes over 10g ethernet. - We really like the NFS/Filepool setup. It greatly simplified the TSM environment and has worked very well. - We kept the diskpool to front most backups, then run migrations to the filepool. - Big database backups that went straight to tape now go straight to the filepool. - We've seen max throughput of about 180mb/s, and we believe this limit is the Sun servers. - We use DD replication - no copy pools. - Because no copy pools, we use DD snapshots to protect from major logical corruption. - TSM db backups go to a separate small filepool on the DD. - We keep the snapshots for 3 days, with 3 day tsm database retension, with 3 day reuse delay. - Data is a mix of Windows servers, Unix, and databases. - During the RFP faze DD went with a 4.5 dedup ratio - we wanted to be very conservative. - Overall dedup ratio has turned out to be 8.5. - Freed up space on the DD is made available for use by running a CLEAN process. This included the space used by old snapshots. - DD recommends running CLEAN only once per week. - In performing DR testing learned that TSM can't access a filepool via a symlink. - If interested and want more info than sales folks give, ask for a temp account on the support site. - Started conversion of the TAPEPOOL to DD FILEPOOL using reclamation. Halfway throught we switched to MOVEDATA, which seemed to perform better. - Took 8 weeks to move all the TAPEPOOL's of 3 TSM servers to DD Filepools. - Total footprint in the DD is 550TB (local primary + DR from other site) stored on about 60TB of disk. (8x dedup) Rick From: Tim Brown tbr...@cenhud.com To: ADSM-L@VM.MARIST.EDU Date: 06/16/2011 03:54 PM Subject:tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: tsm and data domain
We too implemented 1 DD690( for mainframe data), and DD880(for open systems), and replicate both to DD880 @ a DR location about 9 months ago. Our setup is 90% VTL and 10% NFS for archive db logs. We also have disk pools in front of our file system data.Our implementation for the Open Systems was challenging in the beginning but everything now seems to be running better once we made some adjustments and upgraded the DD880 a bunch of times (Problems with dedupe rates promised,undersized storage overall,and were hit with some bugs from Data Domain for our DD880 and waiting for new fixes coming out soon for a couple of small bugs. Couple things to keep in mind: (Lessons Learned). Background: TSM, Version 5.5.5.0, AIX OS 5.3 TL12 DD880 Version: 4.9.2.6-226914 DD880 setup like a VTL, emulating LTO2, tape size 200GB Reclamation:DD recommends setting TSM reclamation and 90%, we always ran reclamation pretty aggressively in our old environments(EMC EDLs, and/or LTO physical tape worlds). We cannot run reclamation during Data Domain cleaning without getting disk or tape errors(yes, false tape errors), so we just don't run reclamation for 2 days to avoid problems. Choosing what type of data goes on the Data Domain and replication: We chose to replicate all our data to our DR site, TIER 1,2,3,4 and we are now rethinking this from a business side that would we really do if catastrophic disaster occurred. The footprint at our DR location is costly not to mention disk costs. Would we really restore all our data or are there other options, staging a copy of a prod database for our Dev,Model,Test, environments. SQL-Backtrack for SYBASE Databases - You will NOT get any additional dedupe on this type of data. We had to implement LAN FREE clients(another cost) for our Sybase databases seeing that they took up 40% of our storage. Nancy Leugemors Enterprise Systems HealthNow, NY 716-887-7979 From: Richard Rhodes rrho...@firstenergycorp.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 08:33 AM Subject: Re: [ADSM-L] tsm and data domain Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU We just implemented a pair of DD880's. Here are some thoughts/comments about this implementation in no particular order . . . - We like them a lot. - Fairly straight forward system. Did not go to training - just spent time with the manual. - The tsm servers are not real high performers, so they don't put a big I/O load on the DD. - The TSM servers are old Sun T2000. - Each DD has both primary data for onsite TSM servers and DR copy for the other. - Used as NFS filepools using 30gb file volumes over 10g ethernet. - We really like the NFS/Filepool setup. It greatly simplified the TSM environment and has worked very well. - We kept the diskpool to front most backups, then run migrations to the filepool. - Big database backups that went straight to tape now go straight to the filepool. - We've seen max throughput of about 180mb/s, and we believe this limit is the Sun servers. - We use DD replication - no copy pools. - Because no copy pools, we use DD snapshots to protect from major logical corruption. - TSM db backups go to a separate small filepool on the DD. - We keep the snapshots for 3 days, with 3 day tsm database retension, with 3 day reuse delay. - Data is a mix of Windows servers, Unix, and databases. - During the RFP faze DD went with a 4.5 dedup ratio - we wanted to be very conservative. - Overall dedup ratio has turned out to be 8.5. - Freed up space on the DD is made available for use by running a CLEAN process. This included the space used by old snapshots. - DD recommends running CLEAN only once per week. - In performing DR testing learned that TSM can't access a filepool via a symlink. - If interested and want more info than sales folks give, ask for a temp account on the support site. - Started conversion of the TAPEPOOL to DD FILEPOOL using reclamation. Halfway throught we switched to MOVEDATA, which seemed to perform better. - Took 8 weeks to move all the TAPEPOOL's of 3 TSM servers to DD Filepools. - Total footprint in the DD is 550TB (local primary + DR from other site) stored on about 60TB of disk. (8x dedup) Rick From: Tim Brown tbr...@cenhud.com To: ADSM-L@VM.MARIST.EDU Date: 06/16/2011 03:54 PM Subject:tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended
Re: tsm and data domain
Brian, The DD has dedup by volume. If you lookup the contents of a volume on TSM, you can get an feeling for how dedup varies. The resulting information is approximate, due to the fact some of the data on a DD volume has expired from TSM's point of view, and will not show up in the volumeusage table (I don't think.) Also, the first copy of any chunk will have a dedup ratio of 1. filesys show compression /backup/vtc/*/* /backup/vtc/Default/N00022L3: mtime: 1302728324652916847, bytes: 47,868,905,726, g_comp: 47,521,219,818, l_comp: 8,403,079,871, meta-data: 154,714,124, bytes/storage_used: 5.6 Richard -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. To reply to our email administrator directly, please send an email to postmas...@sbsplanet.com.
Re: tsm and data domain
Hi Richard, Thanks for the info, I'll have a closer look at this ! Cheers. Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Cowen, Richard Sent: Friday, 17 June, 2011 15:15 To: ADSM-L@VM.MARIST.EDU Subject: Re: tsm and data domain Brian, The DD has dedup by volume. If you lookup the contents of a volume on TSM, you can get an feeling for how dedup varies. The resulting information is approximate, due to the fact some of the data on a DD volume has expired from TSM's point of view, and will not show up in the volumeusage table (I don't think.) Also, the first copy of any chunk will have a dedup ratio of 1. filesys show compression /backup/vtc/*/* /backup/vtc/Default/N00022L3: mtime: 1302728324652916847, bytes: 47,868,905,726, g_comp: 47,521,219,818, l_comp: 8,403,079,871, meta-data: 154,714,124, bytes/storage_used: 5.6 Richard -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. To reply to our email administrator directly, please send an email to postmas...@sbsplanet.com.
Re: tsm and data domain
I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. I have cold stand-by TSM servers at my DR site that can be recovered and online within 1-2 hours and ready to recover systems. Plus I save the cost of VTL licenses and the storage requirements for the copy poolsand using the file device class means I really do not have to be concerned with mount point limits. Interested in feedback. ~Rick Adamson Jacksonville, FL. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments.
Re: tsm and data domain
Has anyone used the DD NFS mount point for Devclass type of File? If so to what degree? How hard could you push it? 1, 2 or 3 TBH? (10Gige = approx 2.25TBH) We have a fairly robust backup server platform I ask as we like the idea of no tape / robot devices from OS to app layer(TSM). This thread is a great read! -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Cowen, Richard Sent: Friday, June 17, 2011 8:15 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Brian, The DD has dedup by volume. If you lookup the contents of a volume on TSM, you can get an feeling for how dedup varies. The resulting information is approximate, due to the fact some of the data on a DD volume has expired from TSM's point of view, and will not show up in the volumeusage table (I don't think.) Also, the first copy of any chunk will have a dedup ratio of 1. filesys show compression /backup/vtc/*/* /backup/vtc/Default/N00022L3: mtime: 1302728324652916847, bytes: 47,868,905,726, g_comp: 47,521,219,818, l_comp: 8,403,079,871, meta-data: 154,714,124, bytes/storage_used: 5.6 Richard -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. To reply to our email administrator directly, please send an email to postmas...@sbsplanet.com. This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby
Re: tsm and data domain
We use ours as VTL because it was a very simple replacement of the existing VTL. We only use 48 drives and use disk pools to catch the incoming backups. The other advantage of VTL for us is speed. We have very limited 10Gb Ethernet and loads of 4Gb FC. 1Gb would be far too slow. We did look and using NFS, but the network ruled that out. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rick Adamson Sent: Friday, June 17, 2011 8:35 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. I have cold stand-by TSM servers at my DR site that can be recovered and online within 1-2 hours and ready to recover systems. Plus I save the cost of VTL licenses and the storage requirements for the copy poolsand using the file device class means I really do not have to be concerned with mount point limits. Interested in feedback. ~Rick Adamson Jacksonville, FL. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. This e-mail (including any attachments) is confidential and may be legally privileged. If you are not an intended recipient or an authorized representative of an intended recipient, you are prohibited from using, copying or distributing the information in this e-mail or its attachments. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message and any attachments. Thank you.
Re: tsm and data domain
Richard, We too experienced the pretty long initialization times for the TSM server attached to the VTL with 128 drives(15 -30 minutes).We ended up changing this setting and it fixed the issue. Check to see if you have this set? update library resetd=no *we had resetd=yes tsm: TSMq library f=d Library Name: Library Type: SCSI ACS Id: Private Category: Scratch Category: WORM Scratch Category: External Manager: Shared: Yes LanFree: ObeyMountRetention: Primary Library Manager: WWN: Serial Number: x AutoLabel: No Reset Drives: No --we originally had this set to YES,default I believe Relabel Scratch: Yes Last Update by (administrator): x Last Update Date/Time: 03/17/11 12:49:58 Another issue was that if we performed a delete volhist t=dbb, with the Data Domains, we would hose up the TSM server and have to restart it.I relabel process would kick out and then the TSM server would get hosed up. That APAR was fixed it when we went from TSM Server code level 5.5.4.0 to 5.5.5.0, there was an APAR out there for it.On a side note. Since going to 5.5.5.0 we are experiencing the recovery log pinning issue almost daily which IBM TSM support is working on currently with us and other TSMers feeling that pain. We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... Nancy Leugemors Enterprise Systems HealthNow, NY 716-887-7979 From: Cowen, Richard rco...@sbsplanet.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 09:20 AM Subject: Re: [ADSM-L] tsm and data domain Sent by: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Brian, The DD has dedup by volume. If you lookup the contents of a volume on TSM, you can get an feeling for how dedup varies. The resulting information is approximate, due to the fact some of the data on a DD volume has expired from TSM's point of view, and will not show up in the volumeusage table (I don't think.) Also, the first copy of any chunk will have a dedup ratio of 1. filesys show compression /backup/vtc/*/* /backup/vtc/Default/N00022L3: mtime: 1302728324652916847, bytes: 47,868,905,726, g_comp: 47,521,219,818, l_comp: 8,403,079,871, meta-data: 154,714,124, bytes/storage_used: 5.6 Richard -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager
Re: tsm and data domain
I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. Rick A not uncommon configuration, I have also used DD's of NFS for disk pools as well. Then only time I've seen them as VTL's is when LAN Free was required. I would however think again about not having a copy pool as you are leaving yourself open to TSM logically corrupting data and having no backup.
Re: tsm and data domain
Steven, I agree, you comment is regard to the copy pools is something I am a little concerned with. I am preparing the move from 5.5 to ver 6.2 now and am in conversations with management on that very subject so I can procure the needed resources. The talks are somewhat challenging to say the least as the DD reps had already told them that the copy pools were not necessary. My concern led to the age-old question what are the chances that something like that would happen to us? Just in case I do have the proverbial warning email archived away. ~Rick -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Steven Langdale Sent: Friday, June 17, 2011 10:30 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. Rick A not uncommon configuration, I have also used DD's of NFS for disk pools as well. Then only time I've seen them as VTL's is when LAN Free was required. I would however think again about not having a copy pool as you are leaving yourself open to TSM logically corrupting data and having no backup.
Re: tsm and data domain
We've moved TSM almost entirely off physical tape, and onto VTL. (Other VTL, not DD.) We do have some DD, but are using it for an RMAN target via NFS. And with RMAN managing its own retention, TSM becomes a bit superfluous. Just about anything can be replaced with a bunch of cheap intel boxes running Linux. [RC] From: Rick Adamson rickadam...@winn-dixie.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 06:47 AM Subject:Re: [ADSM-L] tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. I have cold stand-by TSM servers at my DR site that can be recovered and online within 1-2 hours and ready to recover systems. Plus I save the cost of VTL licenses and the storage requirements for the copy poolsand using the file device class means I really do not have to be concerned with mount point limits. Interested in feedback. ~Rick Adamson Jacksonville, FL. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC Brion Arnaud Sent: Friday, June 17, 2011 3:11 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain Tim, We are right conducting a POC with a pair of DD860, and are relatively satisfied with them. The machines are mostly used as VTL except for a small NFS partition which is used or TSM DB backups. Deduplication rate is OK : around 10 so far, with a good mix of Exchange, Oracle, DB2 as well as Win and AIX data. Ingestion rate is corresponding to our needs, slightly more than 1 GB/s , and the possibility to define plenty drives. Negative or no so impressive points, so far : deduplication rate is a global factor, impossible to know what type of data dedupes better than another one, thus lowering the granularity of deduplication monitoring . We also experienced pretty long initialization times for the TSM server attached to a virtual library having 100 drives : at restart TSM server needs at least 15 minutes to recognize all the drives, thus generating numerous client errors, as they try to get a drive which is defined but not available. We still have to experiment if defining more smaller libraries would solve that issue ... At least replication : it is still unclear for me what would be happening if we do rely on replication to replace copy storage pools, and if our TSM server crashes on primary site while replication is not completed. To my mind the virtual volumes contents would not be matching TSM DB content : not sure so far on how to handle such a situation ... Hope this helped ! Cheers Arnaud ** Corporate IT Systems Datacenter Responsible Panalpina Management Ltd., Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 Direct: +41 (61) 226 19 78 e-mail: arnaud.br...@panalpina.com ** -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, 16 June, 2011 21:50 To: ADSM-L@VM.MARIST.EDU Subject: tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. U.S. BANCORP made the following annotations - Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by electronic communications privacy laws, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply
Re: tsm and data domain
We debated having a copy pool when doing our DD setup. We finally decided that using a daily DD snapshot was an acceptable solution. We take a snapshot a little while after the TSM db backup completes (tsm db backup is on the DD in it's own filepool). Snapshots are kept for 3 days which is our tsm db backup retension and also our reuse delay. The DD snapshot feature was a disappointment. It's all or nothing, covering the entire DD filesystem. To be useful it needs to be made per some defined subdirectory or replication context. Rick From: Rick Adamson rickadam...@winn-dixie.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 11:10 AM Subject:Re: tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Steven, I agree, you comment is regard to the copy pools is something I am a little concerned with. I am preparing the move from 5.5 to ver 6.2 now and am in conversations with management on that very subject so I can procure the needed resources. The talks are somewhat challenging to say the least as the DD reps had already told them that the copy pools were not necessary. My concern led to the age-old question what are the chances that something like that would happen to us? Just in case I do have the proverbial warning email archived away. ~Rick -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Steven Langdale Sent: Friday, June 17, 2011 10:30 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. Rick A not uncommon configuration, I have also used DD's of NFS for disk pools as well. Then only time I've seen them as VTL's is when LAN Free was required. I would however think again about not having a copy pool as you are leaving yourself open to TSM logically corrupting data and having no backup. - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: tsm and data domain
Good discussion. We also use a DD690 onsite and replicate to 2 DD580's at remove sites. We are not using the VTL function, just mounting them up NFS to our AIX TSM server (v5.5). NFS performance is good enough for us to not need the extra expense of the VTL functions. We are using device type of FILE with a size of 50GB/vol. We were running the 1Gb NICs at about 100MB/second, so we recently added a 10Gb NIC for better throughput. Positives: Replication works great, as the dedupe lets us use a much smaller WAN connection to move the data compared to the original size. Current Dedupe rate 14.9x, typically around the 12-14x range. Fitting 340TB of data on 23TB of disk is very cool. Mixture of all types of data: filesystems, server OS, SQL, Exchange. In some cases we have the DBAs dump directly to the DD and bypass TSM for faster DR recovery (they can get the files before TSM is back up). Negatives: It is difficult to hunt down data that is not deduping well. It seems to take a combination of: 1. Looking for new volumes in that stg pool (q volhist typ=stgnew) 2. Looking at the dedupe rate on that volume (filesys show compression /backup/xxx/xxx.BFS) 3. Once you have the vols that have poor compression, Looking to see what TSM has on the volume. Easier if you are using collocation. (q vol /BACKUP/xxx.BFS count=100). 4. Then figure out what you are going to do with that non-compressible data. We still have a Tape library attached for some non-critical data we can put on LT04 tape and conserve DD space. GUI is poor, but it has gotten somewhat better in V 4.9.2. Had problems with Windows 2008 R2 AD authentication, but that has now been fixed. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, June 16, 2011 1:50 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. The BCI Email Firewall made the following annotations - *Confidentiality Notice: This E-Mail is intended only for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you have received this communication in error, please do not distribute, and delete the original message. Thank you for your compliance. You may contact us at: Blue Cross of Idaho 3000 E. Pine Ave. Meridian, Idaho 83642 1.208.345.4550 -
Re: tsm and data domain
Check out DD OS 5.0 and mtrees for some enhancements around snapshot granularity. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Richard Rhodes Sent: Friday, June 17, 2011 12:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain We debated having a copy pool when doing our DD setup. We finally decided that using a daily DD snapshot was an acceptable solution. We take a snapshot a little while after the TSM db backup completes (tsm db backup is on the DD in it's own filepool). Snapshots are kept for 3 days which is our tsm db backup retension and also our reuse delay. The DD snapshot feature was a disappointment. It's all or nothing, covering the entire DD filesystem. To be useful it needs to be made per some defined subdirectory or replication context. Rick From: Rick Adamson rickadam...@winn-dixie.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 11:10 AM Subject:Re: tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU Steven, I agree, you comment is regard to the copy pools is something I am a little concerned with. I am preparing the move from 5.5 to ver 6.2 now and am in conversations with management on that very subject so I can procure the needed resources. The talks are somewhat challenging to say the least as the DD reps had already told them that the copy pools were not necessary. My concern led to the age-old question what are the chances that something like that would happen to us? Just in case I do have the proverbial warning email archived away. ~Rick -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Steven Langdale Sent: Friday, June 17, 2011 10:30 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] tsm and data domain I have had DD implemented for about a year now, but I fail to understand why anyone would utilize the DD VTL license when using TSM? Mine are setup as a simple SAN device with defined directories that correspond to my TSM primary storage pools. I have the device calss in TSM set as the type file and let TSM manage the virtual volumes as it would any other disk storage. There is another DD system that is located at our DR facility, and all data including TSM DB backups are replicated to that location. This allows me to no longer have copy pools. Rick A not uncommon configuration, I have also used DD's of NFS for disk pools as well. Then only time I've seen them as VTL's is when LAN Free was required. I would however think again about not having a copy pool as you are leaving yourself open to TSM logically corrupting data and having no backup. - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message. The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. To reply to our email administrator directly, please send an email to postmas...@sbsplanet.com.
Re: tsm and data domain
We debated having a copy pool when doing our DD setup. We finally decided that using a daily DD snapshot was an acceptable solution. We take a Rick The problem with that is that you'd just be replicating a logical corruption. So unless you noticed it before your last snapshot expired, you be stuck. Obvoliusly a simple solution would be to have a copy stpool on exactly the same DD - inefficient copying wize, but it would dedupe down to nothing.
Re: tsm and data domain
I guess I don't see the difference. If logical corruption (rm -r * in the filepool or tsm screws up and deletes vols), the copy pool could be just as corrupt as the main pool. if the copy pool is in the same dd, then a dd failure would trash both anyway and you're in a DR situation. The snapshot is only over the local dd, so we cut snapshots on both dd's at the same time. Rick From: Steven Langdale steven.langd...@gmail.com To: ADSM-L@VM.MARIST.EDU Date: 06/17/2011 01:22 PM Subject:Re: tsm and data domain Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU We debated having a copy pool when doing our DD setup. We finally decided that using a daily DD snapshot was an acceptable solution. We take a Rick The problem with that is that you'd just be replicating a logical corruption. So unless you noticed it before your last snapshot expired, you be stuck. Obvoliusly a simple solution would be to have a copy stpool on exactly the same DD - inefficient copying wize, but it would dedupe down to nothing. - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: tsm and data domain
I guess I don't see the difference. If logical corruption (rm -r * in the filepool or tsm screws up and deletes vols), the copy pool could be just as corrupt as the main pool. if the copy pool is in the same dd, then a dd failure would trash both anyway and you're in a DR situation. The snapshot is only over the local dd, so we cut snapshots on both dd's at the same time. What I meant by logical corruption was more along the lines of maybe an aggregate bitmap somehow getting screwed up (which I've had) etc, and having to restore from a copy volume. A local copy should mitigate that, and you could still have the dd replication for a real DR. Low risk I know, and perhaps I'm being over cautious because I've had it happen to me. Steven
tsm and data domain
Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments.
Re: tsm and data domain
I use DDs as a VTL. Works good, in the aggregate it is fast. It can restore faster the 1Gb Ethernet (no 10Gb yet), I have heard some people complain that the restore was slow, I have not experienced it being slow. Meets my needs. I do not use NFS and I do not replicate yet. The GUI needs lots of work, but I try to avoid it anyway. Andy Huebner -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tim Brown Sent: Thursday, June 16, 2011 2:50 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] tsm and data domain Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. Thanks, Tim Brown Systems Specialist - Project Leader Central Hudson Gas Electric 284 South Ave Poughkeepsie, NY 12601 Email: tbr...@cenhud.com mailto:tbr...@cenhud.com Phone: 845-486-5643 Fax: 845-486-5921 Cell: 845-235-4255 This message contains confidential information and is only for the intended recipient. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please notify the sender immediately by replying to this note and deleting all copies and attachments. This e-mail (including any attachments) is confidential and may be legally privileged. If you are not an intended recipient or an authorized representative of an intended recipient, you are prohibited from using, copying or distributing the information in this e-mail or its attachments. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message and any attachments. Thank you.
Re: tsm and data domain
On Jun 16, 2011, at 2:49 PM, Tim Brown wrote: Any one use emc's data domain devices for storage pools and replication Would like to here positive and negative issues. My current employer has dozens of DataDomain DD690s and DD880s, all arranged in pairs in which one is primary and one is a DR replication target. (Each of our data centers is primary for some stuff and the DR site for other stuff, so there's no clear DR site vs. main site.) We mostly use them as VTLs; it was an easier conversion that way than to overcome the resident anti-NFS bias and convert to FILE class devices while doing a migration to new hardware. We like them. We keep them relatively small for each head unit (20-30 effective TB, well below maximum capacities), so our cost per GB might be a bit higher than it has to be, but it saves us from having nightmares about DR scenarios when everyone wants to do restores at once and it all grinds down. (Only two FBA adapters? Compared to EMC's older CDL's, that's distinctly smaller.) We're just now converting from sometimes very large replication pools of tapes to replicate into several smaller pools of tapes to restore. We found out the hard way that when our WAN providers have bad days or when our long-haul switches start dropping packets, the restoration (and catch up) of the replication can be painfully slow when service is restored if you only have a couple of pools of volumes. Replication gets bound by housekeeping, not network bandwidth, if you have too few pools. Probably like any VTL technology, tape errors are almost completely a thing of the past. Capacity planning is a bit tricky; some workloads dedupe nicely, but some of our DDRs have dedupe ratios of 5 or less, We need to do a bake-off -- or study someone else's -- between using deduplication in a DataDomain box and using both client-side deduplication and server-side deduplication in TSM V6 and then writing to relatively inexpensive, relatively simple (but replicating) storage arrays. However, we keep pushing the limits of stability with our TSM V6 servers, so we haven't dared tried such a back-off yet. Nick
Re: tsm and data domain
At 05:59 PM 6/16/2011, Nick Laflamme wrote: We need to do a bake-off -- or study someone else's -- between using deduplication in a DataDomain box and using both client-side deduplication and server-side deduplication in TSM V6 and then writing to relatively inexpensive, relatively simple (but replicating) storage arrays. However, we keep pushing the limits of stability with our TSM V6 servers, so we haven't dared tried such a back-off yet. Nick, We are heading down this path. My analysis is that in a TSM environment, the fairly low dedup ratio does not justify the higher price of duduping VTLs. Commodity disk arrays have gotten very inexpensive. We're using DS3500s which are nice building blocks. We put some behind IBM SVCs for servers, some attached to TSM or Exchange servers (without SVC). Common technology - different uses. We use them for both TSM DB, LOG and FILE (different RPM disks, obviously). Using cheap disk vs VTLs has different pros and cons. using disk allows for source-mode (client-side) dedup, which a VTL will not do. VTLs, on the other hand, allow for global dedup pools and LAN-free virtual tape targets. deduping VTLs will be more effective in TSM environments where you have known duplicate data, such as lots of Oracle or MSSQL full backups, or other cases where you have multiple full backups. For normal progressive incremental file backups, however, TSM already does a good job of reducing data so VTL dedup doesn't get you as much, and in this case IMHO cheap disk is, well, cheaper and gets you source-mode dedup as well. We are in process of implementing this, but I know a few others are a bit further along. We will continue to use TSM Backup Stgpool to replicate offsite. ..Paul -- Paul ZarnowskiPh: 607-255-4757 Manager, Storage Services Fx: 607-255-8521 719 Rhodes Hall, Ithaca, NY 14853-3801Em: p...@cornell.edu
Re: TSM and Data Domain
Kenneth, Bradberry, Kenneth schrieb: I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? We just tested a DataDomain machine with TSM. The results were not as good as we expected. It shows a very good compression with database backups (Exchange backups should also work well) but not with normal user data, like homedirectories or web data. -- Regards, Dirk Kastens Universitaet Osnabrueck, Rechenzentrum (Computer Center) Albrechtstr. 28, 49069 Osnabrueck, Germany Tel.: +49-541-969-2347, FAX: -2470
Re: TSM and Data Domain
We are averaging about 11/1 with data domain on a mix of data. One thing we found was the need to update a setting for tsm tape marker parameter, which almost doubled our performance. Cory Heikel Tivoli Systems Administrator Hershey Medical Center (717) 531-7972 Dirk Kastens [EMAIL PROTECTED] 10/10/2007 2:45 AM Kenneth, Bradberry, Kenneth schrieb: I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? We just tested a DataDomain machine with TSM. The results were not as good as we expected. It shows a very good compression with database backups (Exchange backups should also work well) but not with normal user data, like homedirectories or web data. -- Regards, Dirk Kastens Universitaet Osnabrueck, Rechenzentrum (Computer Center) Albrechtstr. 28, 49069 Osnabrueck, Germany Tel.: +49-541-969-2347, FAX: -2470
Re: TSM and Data Domain
Can you share the details of the tape marker setting? -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Cory Heikel Sent: Wednesday, October 10, 2007 9:38 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM and Data Domain We are averaging about 11/1 with data domain on a mix of data. One thing we found was the need to update a setting for tsm tape marker parameter, which almost doubled our performance. Cory Heikel Tivoli Systems Administrator Hershey Medical Center (717) 531-7972 Dirk Kastens [EMAIL PROTECTED] 10/10/2007 2:45 AM Kenneth, Bradberry, Kenneth schrieb: I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? We just tested a DataDomain machine with TSM. The results were not as good as we expected. It shows a very good compression with database backups (Exchange backups should also work well) but not with normal user data, like homedirectories or web data. -- Regards, Dirk Kastens Universitaet Osnabrueck, Rechenzentrum (Computer Center) Albrechtstr. 28, 49069 Osnabrueck, Germany Tel.: +49-541-969-2347, FAX: -2470
Re: TSM and Data Domain
The command can be different depending on which version of the software you are
running. In the latest version it is:
option set marker-type {nw1 | cv1 | tsm1 | tsm2 | eti1 | hpdp1 | none} Set
the tape marker type for data from NetWorker; CommVault; Tivoli Storage Manager
on Linux, Windows and AIX; Tivoli Storage Manager on Solaris; HP NonStop with
ETI-NET EZX-BackBox; or HP Data Protector.
The problem is there are two options for tsm, so you have to test to see which
one gets triggered for you (we are running on aix and ended up with setting
tsm2 which doesn't seem to match with the doc), and it only applies to newly
written data. Best bet is to work with the data domain tech people.
cory
Cory Heikel
Tivoli Systems Administrator
Hershey Medical Center
(717) 531-7972
William Jean [EMAIL PROTECTED] 10/10/2007 10:03 AM
Can you share the details of the tape marker setting?
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Cory Heikel
Sent: Wednesday, October 10, 2007 9:38 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM and Data Domain
We are averaging about 11/1 with data domain on a mix of data. One thing
we found was the need to update a setting for tsm tape marker parameter,
which almost doubled our performance.
Cory Heikel
Tivoli Systems Administrator
Hershey Medical Center
(717) 531-7972
Dirk Kastens [EMAIL PROTECTED] 10/10/2007 2:45 AM
Kenneth,
Bradberry, Kenneth schrieb:
I am currently looking at Data Domain as a possible VTL and
de-duplication vender. Can anyone share there experiences with Data
Domain with TSM?
We just tested a DataDomain machine with TSM. The results were not as
good as we expected. It shows a very good compression with database
backups (Exchange backups should also work well) but not with normal
user data, like homedirectories or web data.
--
Regards,
Dirk Kastens
Universitaet Osnabrueck, Rechenzentrum (Computer Center)
Albrechtstr. 28, 49069 Osnabrueck, Germany
Tel.: +49-541-969-2347, FAX: -2470
Re: TSM and Data Domain
That would match what I've heard most often. While users of most other backup products are seeing an average of 20:1 on most data, that ratio relies on having full backups to compare against. TSM's progressive incremental backup of filesystems significantly reduces the commonality that you'll achieve with filesystem backups. However, you should achieve the same commonality that other backup product users see on database and application backups because you have full backups to compare against. VERY important: You will not see de-duplication of any kind if you're only using your Data Domain box (or any other de-dupe target) as a disk pool for caching. If you're not leaving previous versions (or previous full backups of databases) on the device, it won't have ANYTHING to compare against, and you'll end up with 2:1, as some users have mentioned. (You get 2:1 because after any de-dupe stages, they also run compression against data sent to their device.) And since most de-dupe products' pricing is based on getting a 20:1 or so de-dupe ratio, you're kind of getting ripped off. (This means that after you test what de-dupe ratio you're going to get with your data, make sure you compare the price of the de-dupe target against what you would pay for just a regular disk array. If you're getting a very low de-dupe ratio, they may be very close in price.) Finally, if you're OK with just getting 2:1, you might consider Storewiz. They are a gateway in front of your storage that does in-line Limpel-Ziv compression on your data allegedly without any performance degradation. (The idea is that while compressing obviously takes some cycles, they make up those cycles by having your disk array store 50% fewer blocks.) --- W. Curtis Preston Backup Blog @ www.backupcentral.com VP Data Protection, GlassHouse Technologies -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Cory Heikel Sent: Wednesday, October 10, 2007 6:38 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM and Data Domain We are averaging about 11/1 with data domain on a mix of data. One thing we found was the need to update a setting for tsm tape marker parameter, which almost doubled our performance. Cory Heikel Tivoli Systems Administrator Hershey Medical Center (717) 531-7972 Dirk Kastens [EMAIL PROTECTED] 10/10/2007 2:45 AM Kenneth, Bradberry, Kenneth schrieb: I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? We just tested a DataDomain machine with TSM. The results were not as good as we expected. It shows a very good compression with database backups (Exchange backups should also work well) but not with normal user data, like homedirectories or web data. -- Regards, Dirk Kastens Universitaet Osnabrueck, Rechenzentrum (Computer Center) Albrechtstr. 28, 49069 Osnabrueck, Germany Tel.: +49-541-969-2347, FAX: -2470
TSM and Data Domain
I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? I am also looking at SEPATON as an out of band solution as well. Thanks, Ken Bradberry.
Re: TSM and Data Domain
I curently have the quantum DXI 5500 setup with Tivoli and it is performing fantastic. However, since this is a new product you will face some growing pains. What I like is that it does hardware compression rather than software like DD -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Bradberry, Kenneth Sent: Tuesday, October 09, 2007 11:49 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] TSM and Data Domain I am currently looking at Data Domain as a possible VTL and de-duplication vender. Can anyone share there experiences with Data Domain with TSM? I am also looking at SEPATON as an out of band solution as well. Thanks, Ken Bradberry. --- Confidentiality Notice: The information in this e-mail and any attachments thereto is intended for the named recipient(s) only. This e-mail, including any attachments, may contain information that is privileged and confidential and subject to legal restrictions and penalties regarding its unauthorized disclosure or other use. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or the taking of any action or inaction in reliance on the contents of this e-mail and any of its attachments is STRICTLY PROHIBITED. If you have received this e-mail in error, please immediately notify the sender via return e-mail; delete this e-mail and all attachments from your e-mail system and your computer system and network; and destroy any paper copies you may have in your possession. Thank you for your cooperation.
