I'm not convinced either. Although our numbers are a little different, you and I end up roughly at the same place. There are a number of vendors whose de-dupe targets top out at about 200-300 MB/s, which is roughly the speed of 2-3 LTO-3 drives, depending on how well you use them. If you need more than that, you need to buy another box. (BTW, Data Domain's numbers have increased to about 200 MB/s.)
These numbers work just fine when we're talking backups via the LAN to LAN-based backup servers. You're going to need at least two, possibly three network-based backup servers to generate 200 MB/s. Assuming 70 MB/s or so per master/media server, you buy one de-dupe unit per three master/media servers or so. You can scale pretty far that way. You will need to make sure that backup A is always sent to de-dupe unit A, and backup B is always sent to de-dupe unit B, and so on. (If you send backup B to de-dupe unit A after initially sending it to de-dupe unit A, its first backup will not get de-duped against anything, resulting in a significant decrease in overall de-duplication ratio.) While you won't get as big of a de-dupe ratio as you would if you could have a single device that could do 1000s of MB/s, there is an argument to be made that you won't get much de-dupe when de-duping the backups of server A against those of server B -- unless they have similar data. So a very large setup like this will require a bit of planning, but I think the benefits outweigh the extra planning required. Now, if you happen to have a SINGLE SAN media server that needs MORE than 200 MB/s, then you're going to want a device that can handle that level of throughput. This is going to be a pretty big server, BTW, as a 200 MB/s device can back up about 6 TB in 8 hours. And notice I said SAN media server, not a regular media server, as a regular media server isn't going to be able to generate more than 200 MB/s, as it's getting its backups via IP. But a SAN media server is backing up its own data locally, so it can go much faster. This also means you're really looking at a SAN/block device, which means you're really looking at a VTL. (Yes, I'm aware of the Puredisk storage unit around the corner. I think you'll find it's not going after this part of the market.) If you need this kind of throughput, there are a few products that are advertising several hundred or thousands of MB/s within a single de-dupe setup. These are the newer kids on the de-dupe block, of course, so they're not going to have as many customer references as the vendors that have been selling de-dupe as long. But from what I've seen, they're worth a look. --- W. Curtis Preston Backup Blog @ www.backupcentral.com VP Data Protection, GlassHouse Technologies -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ed Wilts Sent: Monday, September 24, 2007 9:44 PM To: 'Clem Kruger'; [EMAIL PROTECTED]; 'Jeff Lightner' Cc: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Tapeless backup environments? I'm not convinced that writing to a DataDomain is going to be faster than writing to multiple LTO-3 drives over a SAN. The DD is limited to about 90MB/sec which is on par with 1-2 LTO-3 drives and not much more than that. Unless, of course, you consider adding extra DD units for every 2 LTO-3 drives you currently have and that's going to bump your costs up even higher (which might be offset by the requirement for a Decru FC520 encrypting appliance for every 2-3 LTO-3 drives today). I don't think that NetBackup 6.5 includes de-duplication. It's provided by PureDisk which is a separately licensed product. With 6.5.1, you'll be able to use PureDisk as a storage unit, something that's not there yet today. .../Ed -- Ed Wilts, RHCE, BCFP, BCSD Mounds View, MN, USA mailto:[EMAIL PROTECTED] > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:veritas-bu- > [EMAIL PROTECTED] On Behalf Of Clem Kruger > Sent: Monday, September 24, 2007 11:32 AM > To: [EMAIL PROTECTED]; Jeff Lightner > Cc: veritas-bu@mailman.eng.auburn.edu > Subject: Re: [Veritas-bu] Tapeless backup environments? > > Hi Dave, > > Yes it is a difficult decision I have looked at DataDomain with > NetBackup. I have found that the backups are faster and there is a vast > amount of disk being saved. > > NetBackup 6.5 includes de-duplication and I have become a great friend > of it. To use the words of a supplier, "Saving me Time, Saving me Space > and Saving me Money" :) > > > Kind Regards, > Clem Kruger > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dave > Markham > Sent: 24 September 2007 17:35 PM > To: Jeff Lightner > Cc: veritas-bu@mailman.eng.auburn.edu > Subject: Re: [Veritas-bu] Tapeless backup environments? > > Guys i've just read this thread and can say im very interested in it. > The first thing is i learned a new term called deduplication which i > didn't know existed. > > Question : I gather Deduplication is using other software. DataDomain i > think i saw mentioned. Where does this fit in with Netbackup and does > the software reside on every client or just a server somewhere? > > Ok, so im trying to kit refresh a backup environment for a customer > which has 2 sites. Production and DR about 200 miles apart. There is a > link between the sites but the customer will probably frown on > increased > bandwidth charges to transfer backup data across for DisasterRecovery > purposes. > > Data is probably only 1 TB for the site with perhaps 70% being required > to be transfered daily to offsite media. > > Currently i use tape and i was just speccing a new tape system as i > thought by using disk based backups, and retentions of weekly/monthly > backups lasting say 6 weeks, im going to need a LOT of disk, plus the > bandwidth transfer costs to DR site > > LTO3 tapes are storing 200gb a tape which is pretty good compared to > disk i thought. > > I guess in my set up its a trade off between :- > > Initial cost of disk array vs initial cost of tape library, drives and > media > > Time take to backup ( network will be bottle neck here. Still on 100Meg > lan with just 2 DB servers using GigaBit lan to backup server. > > Offsite transfer of tapes daily to offsite location vs Cost of > increased > bandwith between sites to transfer backup data. > > > Im now confused what to propose :) _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu