Not really sure why you would need +24 hours to handle copying data to your secondary site.
With features like: - Copy storage pool feature for client backups - Copy storage pool feature on migration - Normal backup storage pool With the first 2, there shouldnt really be much data left to copy for the 3rd one. I've seen sites handling up to 35TB per day using these features and they never had to spend 24 hours doing backup storagepool. Ofc, it also depends on the throughput your getting. There are VTL's out there that can handle up to 45TB/hour that costs alot less than 1 million. However, it does require the infrastructure (fiber) connections to reach those numbers. As with the hash conflict, the DD uses SHA-1 with a variable block length for deduplication. Theoretically, there is a 2^160 chance it will happen. Doesnt seem to be that bad, but your first hash collision is randomly more likely to happen than that number suggests. Best Regards Daniel Daniel Sparrman Exist i Stockholm AB Växel: 08-754 98 00 Fax: 08-754 97 30 daniel.sparr...@exist.se http://www.existgruppen.se Posthusgatan 1 761 30 NORRTÄLJE -----"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> skrev: ----- Till: ADSM-L@VM.MARIST.EDU Från: Shawn Drew Sänt av: "ADSM: Dist Stor Manager" Datum: 10/05/2011 00:14 Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool We were using the copystg feature of the storage pools before, and even then our TSM cycle was growing, growing and eventually passed 24 hours. For us, it wasn't 4 hours vs 1 hour. It was 24+hours+finger crossing vs one hour. We reached the point where we had to kill the backup-stg's and try and catch up on the weekends. As far as the hash conflict, I think all of these systems use cryptographic level hashes (MD5/SHA1). From my understanding, the chances of a collision are way, way, exponentially lower than .1%. Where did that number come from, Is that a commonly accepted number for this issue? Maybe I'm not understanding the proposed error. Are you referring to a cryptographic hash collision? or something else? In order to update our environment to handle the traditional TSM copypool architecture, we would have had to spend an additional million dollars (at a minimum) and many more expensive WAN links. Considering this is for short-term data (30 days or less) , we still have weekly tape backups, and I've never seen anything to convince me that the chances of these specific errors are anywhere near the quoted .1%, I'm still confident this was the right move for us. I do keep looking for where I could be wrong, so I want to make sure I completely understand what you are saying. Regards, Shawn ________________________________________________ Shawn Drew Internet daniel.sparr...@exist.se Sent by: ADSM-L@VM.MARIST.EDU 10/04/2011 04:44 PM Please respond to ADSM-L@VM.MARIST.EDU To ADSM-L cc Subject [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool Not entirely true since you dont have todo backup stgpool as a only resort with TSM 6.2. Since the simultaneous copy feature is now not only available for client, but for server processes, there is nothing saying you have to copy all the data by using backup stgpool. And it's not a matter if it's a risk or not getting a logical error. The risk is there, and if it hits you, you dont loose a few hours of data. A hash conflict would strike you across both primary and secondary storage, and it could strike you for a huge amount of data if you're unlucky. You get a hash conflict on a common hash key, and it could strike you all across the board. So, having your offsite replicated within the hour instead of 4 hours with the risk of loosing most of it, or letting it take 4 hours, but be sure you'll be able to recover? The descriptions around here sometimes scares me (not in particular this description). Some people seem to think that 0.1% chance of loosing it all is worth it for other benefits. If you told your boss (not your IT manager, but your business developer for example) that there's a 0.1% chance you'll loose all your backups and all your versions, think he'd be ok with it? My guess? I'd say he wouldnt even answer it... If you feel replication is such a good way of going, you can always go sync or async mirroring instead. You're doing just the same (since you can actually have verification on your mirroring). You're syncing/replicating bits and bytes, but there's no way for the mirroring / replication to be sure it's actually readable for the app. Unlike having your app creating a 2ndary copy. Best Regards & nice evening Daniel Daniel Sparrman Exist i Stockholm AB Växel: 08-754 98 00 Fax: 08-754 97 30 daniel.sparr...@exist.se http://www.existgruppen.se Posthusgatan 1 761 30 NORRTÄLJE -----"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> skrev: ----- Till: ADSM-L@VM.MARIST.EDU Från: Shawn Drew Sänt av: "ADSM: Dist Stor Manager" Datum: 10/04/2011 20:17 Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool The other side of this is that with 3rd party deduplicated replication, my offsite copy is rarely more than one hour behind the source copy. Before I moved to this, we would schedule our backup-stg pools to run once a day, and they would have to run several hours before it was in sync. If there was a real DR situation, we would lose much more data by being up to 24-hours out of sync with the old "backup-stg" solution. And it's always recent "just backed-up" data that is the most desirable after a DR situation. The chances of a real DR situation seem higher to me than a logical error. Perhaps just feeling like that after seeing a minor earthquake and a hurricane within a couple weeks of each other. I have seen a few logical errors and a few DR situations in my career and even in my conservative bank environment, a single-pool-3rd-party-replicated is lower risk than the slow-backup-stg solution. We still keep periodic longer term backups on Tape, so we would have a "last resort" restore source if there was a logical error, but it is still not a daily backup. In the end, its a risk trade-off decision that you have to make for yourself. And decide if you can afford the time that a backup-stg takes each day. Regards, Shawn ________________________________________________ Shawn Drew Internet steven.langd...@gmail.com Sent by: ADSM-L@VM.MARIST.EDU 10/04/2011 02:57 AM Please respond to ADSM-L@VM.MARIST.EDU To ADSM-L cc Subject Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool The "logical error" question has come up before. With no TSM managed copy pool you are perhaps at a slightly higher risk. An option is to still have a copy pool, but on the same DD. So little real disk usage, but some protection from a TSM logical error. That obviously does not protect you from a DD induced one. FWIW, when we implement VTL's, and if the bandwidth allows, we use TSM to create the copy. Small sites with limited bandwidth, we rely on the appliance. Steven On 4 October 2011 06:41, Daniel Sparrman <daniel.sparr...@exist.se> wrote: > > If someone puts a high-caliber bullet through my Gainesville DD, then > > I recover it from the replicated offsite DD, perhaps selecting a > snapshot. > > > > If someone puts a high-caliber bullet through both of them, then I > > have lost my backups of a bunch of important databases. > > And if you have a logical error on your primary box, which is then > replicated to your 2nd box? Or even worse, a hash conflict? > > I dont consider someone putting a bullet through both the boxes a high > risk, I do however consider other errors to be more of a high risk. > > Best Regards > > Daniel > > > > > > Daniel Sparrman > Exist i Stockholm AB > Växel: 08-754 98 00 > Fax: 08-754 97 30 > daniel.sparr...@exist.se > http://www.existgruppen.se > Posthusgatan 1 761 30 NORRTÄLJE > > > -----"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> skrev: ----- > Till: ADSM-L@VM.MARIST.EDU > Från: "Allen S. Rout" > Sänt av: "ADSM: Dist Stor Manager" > Datum: 10/03/2011 23:38 > Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] > vtl versus file systems for pirmary pool > > On 09/28/2011 02:16 AM, Daniel Sparrman wrote: > > > In this mail, it really sounds like you're using your DD as both > > primary storage and for TSM storage. > > I am, right now, using the DD as a target for direct-written database > backups, only. So that's not really "primary storage", as I think > about it. > > > > If the DD box fails, what are your losses? > > If someone puts a high-caliber bullet through my Gainesville DD, then > I recover it from the replicated offsite DD, perhaps selecting a snapshot. > > If someone puts a high-caliber bullet through both of them, then I > have lost my backups of a bunch of important databases. > > > > > Sorry for all the questions, I'm just trying to get an idea how > > you're using this box. > > No problem. Our conversation is fuzzed by the fact that I am also > talking about how one _might_ use it for TSM storage. I'm > contemplating it, but not doing it at the moment. > > > [ ... if you lose a DD, then ... ] you have to restore the data from > > somewhere else (tape?). > > > In my planning, the DD gets copied / offsited to a remote DD, so > that's the somewhere else. > > - Allen S. Rout > This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc. This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.
