Re: Datadomain / Exagrid - Backup Times over Cat5
True. I'm trying to backup ~4TB in under 12 hours. 8 hours would be nice... I think a single 1Gig Cat5 cable is going to get me around 23 hours at around 850Mbps On Tue, Apr 9, 2013 at 11:47 PM, Ben Scott mailvor...@gmail.com wrote: On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. Gigabit Ethernet can stream 125,000,000 8-bit quantities per second. Framing and protocol overhead rob significantly from that. Let's assume 75% efficiency, just to have a number. That's 93 megabytes per second, or 337 gigabytes in one hour. If you're only backing up a terabyte, that might be just fine. If you're backing up a petabyte, not so much. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
RE: Datadomain / Exagrid - Backup Times over Cat5
Not necessarily, don't mix capacity with speed. A 2 link Port-Channel is still 2 x 1 Gbs link. If the backup is between only 2 IP (not multiplexing backup from multiple sources), the Port-Channel algorithm will still give you only 1 Gbs per IP pairs or port pair, etc. You have to check the load-balancing option of the Port-Channel in your system. There is multiple algorithm available to load-balance in the Port-Channel. You have to chose the correct one for your system. De : Andrew S. Baker [mailto:asbz...@gmail.com] Envoyé : 10 avril 2013 10:27 À : NT System Admin Issues Objet : Re: Datadomain / Exagrid - Backup Times over Cat5 A two-port trunk will reduce that significantly, of course... ASB http://XeeMe.com/AndrewBaker http://xeeme.com/AndrewBaker Providing Virtual CIO Services (IT Operations Information Security) for the SMB market... On Wed, Apr 10, 2013 at 9:52 AM, Jon D rekcahp...@gmail.com wrote: True. I'm trying to backup ~4TB in under 12 hours. 8 hours would be nice... I think a single 1Gig Cat5 cable is going to get me around 23 hours at around 850Mbps On Tue, Apr 9, 2013 at 11:47 PM, Ben Scott mailvor...@gmail.com wrote: On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. Gigabit Ethernet can stream 125,000,000 8-bit quantities per second. Framing and protocol overhead rob significantly from that. Let's assume 75% efficiency, just to have a number. That's 93 megabytes per second, or 337 gigabytes in one hour. If you're only backing up a terabyte, that might be just fine. If you're backing up a petabyte, not so much. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Datadomain / Exagrid - Backup Times over Cat5
On Wed, Apr 10, 2013 at 9:52 AM, Jon D rekcahp...@gmail.com wrote: That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. True. I'm trying to backup ~4TB in under 12 hours. 8 hours would be nice... I think a single 1Gig Cat5 cable is going to get me around 23 hours at around 850Mbps This is where the other things come into play. If the change-delta, de-dupe, compression, and other magic happens on the agent side of the backup, you may see significant gains. For example, if you're backing up a bunch of small files, and churn is low, and the backup solution uses the maintain a full mirror of original model, and it only copies the changed files, your nightly backup might only need to ship a tiny amount of data over the wire. Or if they're large files but it has good change-delta handling. Even without such magic, you may also be able to do a longer full backup on weekends, then an incremental/differential on weeknights. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Datadomain / Exagrid - Backup Times over Cat5
We don't use DD or Exagrid but i can give you some numbers on that size. We do a NAS backup not through NDMP. 4.3TB ~5.6 mil files. NAS and backup media server are LACP'ed at 4Gig. We hit at least 2.4Gbps on the media server nightly We dedup at the mediaserver for the NAS A new full takes ~2-3 days using one stream. Multi-streams i could get it under 1 day. Incr takes 9 hours single stream, pretty regular at 140GBs and 1.4 mil files We no longer do real fulls and instead do synthetic with take 16 hours to compile. Our Friday INCR which includes PST files takes 19 hours, ~1.4 mil files but the sized is increase to ~640GBs We have another file server with 12TB and some where close to 20 mil files We multi stream this server with Dedup at the client Incr backups of 1.6 mil files at ~36Gbs takes about 4 hours. A raw full takes almost a week. While backup technology is changing for the most part when an incremental is completed you are still scanning every file to check the modify date or archive bit. I am not sure about DD or Exagrid but for us Netbackup they started tracking file changes to dramatically decrease the full backups. Not that we have the latest version to take advantage of this. If network bandwidth is really a challenge and not file count then look to spreading out your fulls less often. Do a Full backup every 1-2 months, a differential every week and an incremental every day. We do this for our DNA sequencers which spit our 2TBs a patient. We do a twice a year, differential every month and incremental every day. On Wed, Apr 10, 2013 at 11:20 AM, Ben Scott mailvor...@gmail.com wrote: On Wed, Apr 10, 2013 at 9:52 AM, Jon D rekcahp...@gmail.com wrote: That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. True. I'm trying to backup ~4TB in under 12 hours. 8 hours would be nice... I think a single 1Gig Cat5 cable is going to get me around 23 hours at around 850Mbps This is where the other things come into play. If the change-delta, de-dupe, compression, and other magic happens on the agent side of the backup, you may see significant gains. For example, if you're backing up a bunch of small files, and churn is low, and the backup solution uses the maintain a full mirror of original model, and it only copies the changed files, your nightly backup might only need to ship a tiny amount of data over the wire. Or if they're large files but it has good change-delta handling. Even without such magic, you may also be able to do a longer full backup on weekends, then an incremental/differential on weeknights. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Datadomain / Exagrid - Backup Times over Cat5
If all of your backups are from Box A to Box B, then both will need to be trunked for there to be any benefits in throughput. However, if there are multiple concurrent backups at play from multiple sources, then trunking on the backup device will still provide for more throughput than a single network connection. Of course, as Kevin points out, there are several other factors to consider in a backup scenario. *ASB **http://XeeMe.com/AndrewBaker* http://xeeme.com/AndrewBaker* **Providing Virtual CIO Services (IT Operations Information Security) for the SMB market…*** On Wed, Apr 10, 2013 at 10:58 AM, Bourque Daniel daniel.bour...@loto-quebec.com wrote: ** Not necessarily, don't mix capacity with speed. A 2 link Port-Channel is still 2 x 1 Gbs link. If the backup is between only 2 IP (not multiplexing backup from multiple sources), the Port-Channel algorithm will still give you only 1 Gbs per IP pairs or port pair, etc. You have to check the load-balancing option of the Port-Channel in your system. There is multiple algorithm available to load-balance in the Port-Channel. You have to chose the correct one for your system. -- *De :* Andrew S. Baker [mailto:asbz...@gmail.com] *Envoyé :* 10 avril 2013 10:27 *À :* NT System Admin Issues *Objet :* Re: Datadomain / Exagrid - Backup Times over Cat5 A two-port trunk will reduce that significantly, of course... *ASB **http://XeeMe.com/AndrewBaker* http://xeeme.com/AndrewBaker* **Providing Virtual CIO Services (IT Operations Information Security) for the SMB market.*** On Wed, Apr 10, 2013 at 9:52 AM, Jon D rekcahp...@gmail.com wrote: True. I'm trying to backup ~4TB in under 12 hours. 8 hours would be nice... I think a single 1Gig Cat5 cable is going to get me around 23 hours at around 850Mbps On Tue, Apr 9, 2013 at 11:47 PM, Ben Scott mailvor...@gmail.com wrote: On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. Gigabit Ethernet can stream 125,000,000 8-bit quantities per second. Framing and protocol overhead rob significantly from that. Let's assume 75% efficiency, just to have a number. That's 93 megabytes per second, or 337 gigabytes in one hour. If you're only backing up a terabyte, that might be just fine. If you're backing up a petabyte, not so much. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin Mise en garde concernant la confidentialité : Le présent message, comprenant tout fichier qui y est joint, est envoyé à l'intention exclusive de son destinataire; il est de nature confidentielle et peut constituer une information protégée par le secret professionnel. Si vous n'êtes pas le destinataire, nous vous avisons que toute impression, copie, distribution ou autre utilisation de ce message est strictement interdite. Si vous avez reçu ce courriel par erreur, veuillez en aviser immédiatement l'expéditeur par retour de courriel et supprimer le courriel. Merci! Confidentiality Warning: This message, including any attachment, is sent only for the use of the intended recipient; it is confidential and may constitute privileged information. If you are not the intended recipient, you are hereby notified that any printing, copying, distribution or other use of this message is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email, and delete it. Thank you! ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums
Re: Datadomain / Exagrid - Backup Times over Cat5
Can't speak to Exagrid, but think of the DD boxes as if they are NAS devices. My (older model) 530 can ingest data as fast as I can throw information at it. Regarding speed, I suppose too slow is as too slow does. GigE is fast enough for my backups given their size. Here are some statistics about amount of data written over the last week and compression ratios. Pre-Comp (GB) Post-Comp(GB) Global-Comp Factor Local-Comp Factor Total-Comp Factor (Reduction %) --- - --- -- - Last 7 days 8718.2 331.9 18.6x 1.4x26.3x (96.2) Last 24 hrs 1412.140.1 24.2x 1.5x35.3x (97.2) --- - --- -- - On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. I know you can combine ports, but how much does that really help? Can anyone tell me how much data a full backup is for them, and how long it takes their Data Domain or Exagrid to back it up? Thanks in advace, Jon ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Datadomain / Exagrid - Backup Times over Cat5
Thanks everyone. It seems like it's as simple as the Cat5 cable is the bottle neck. I think DD does have a pre-backup dedupe, but only if it's talking to a server with a client loaded. Wouldn't help with backing up file shares on an EMC SAN. Somehow I need to figure out how to get data on 1 EMC SAN(acting like a NAS) to get to a backup appliance with speed. Without dropping 10G money. lol On Tue, Apr 9, 2013 at 2:46 PM, Richard Stovall rich...@gmail.com wrote: Good point. In that respect, DD probably shouldn't be considered a 'backup' product. Deduplication file storage might be a better moniker. On Tue, Apr 9, 2013 at 2:33 PM, Andrew S. Baker asbz...@gmail.com wrote: I can't speak about the DD products specifically, but dedupe can be independent of data deltas... *ASB **http://XeeMe.com/AndrewBaker* http://xeeme.com/AndrewBaker* **Providing Virtual CIO Services (IT Operations Information Security) for the SMB market…*** On Tue, Apr 9, 2013 at 2:10 PM, Richard Stovall rich...@gmail.comwrote: The DD products dedupe after receiving the complete data, so there is no bandwidth savings. On Apr 9, 2013 2:04 PM, Andrew S. Baker asbz...@gmail.com wrote: Most backup products of this sort are not copying 100% of your full data set across the wire. They are sending only the changed bits (deltas) so as to improve both performance and storage consumption. *ASB **http://XeeMe.com/AndrewBaker* http://xeeme.com/AndrewBaker* **Providing Virtual CIO Services (IT Operations Information Security) for the SMB market…*** On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. I know you can combine ports, but how much does that really help? Can anyone tell me how much data a full backup is for them, and how long it takes their Data Domain or Exagrid to back it up? Thanks in advace, Jon ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Datadomain / Exagrid - Backup Times over Cat5
On Tue, Apr 9, 2013 at 10:55 AM, Jon D rekcahp...@gmail.com wrote: I'm trying to wrap my head around the speed of backup appliances like Data Domain and Exagrid. The thing that doesn't make sense to me is the backups are going across Cat5. It seems like they would be really slow for a full backup. That depends how fast the network you're running is, and how much data you've got to worry about, and maybe other things. Gigabit Ethernet can stream 125,000,000 8-bit quantities per second. Framing and protocol overhead rob significantly from that. Let's assume 75% efficiency, just to have a number. That's 93 megabytes per second, or 337 gigabytes in one hour. If you're only backing up a terabyte, that might be just fine. If you're backing up a petabyte, not so much. -- Ben ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
