Re: switch 10G standalone TOR, core to DC
Anyone have worked with the switching vendor Quanta for their 10ge switching as TOR? [1] Their spec looked interesting and they are quiet cheap. [1] http://www.quantaqct.com/en/01_product/02_detail.php?mid=30sid=114id=116qs=63 -bn 0216331C On Tue, Feb 12, 2013 at 7:45 AM, Nick Hilliard n...@foobar.org wrote: On 12/02/2013 14:23, Piotr wrote: shared 9 MB packet buffer pool that is allocated dynamically to ports that are congested 9MB is a standard size of port buffers.. That's pretty standard for a cut-thru ToR switch of this style. Cut-thru switches generally need a lot less packet buffer space than store-n-forward switches. Also, ToR boxes tend not to have complex qos requirements. Having said that, you need to be careful deploying small-buffer boxes. If you're not careful, you will end up with bad packet loss. Nick
Re: switch 10G standalone TOR, core to DC
On Tue, Feb 19, 2013 at 8:21 PM, Bao Nguyen ngq...@gmail.com wrote: Anyone have worked with the switching vendor Quanta for their 10ge switching as TOR? [1] Their spec looked interesting and they are quiet cheap. [1] http://www.quantaqct.com/en/01_product/02_detail.php?mid=30sid=114id=116qs=63 -bn 0216331C Based on the specs, the Quanta switches look like they use Broadcom merchant silicon and should have similar performance to other switches based on the same chipset: http://blog.sflow.com/2011/12/merchant-silicon.html While many vendors use merchant silicon, there is variability in firmware, exposed features, CLI etc.
Re: switch 10G standalone TOR, core to DC
I have fairly extensive experience with the Quanta LY2 10GE switches, and they work very well for some environments. Here are some basic impressions: - Broadcom Trident chipset - Similar performance to other Trident switches (ideally line rate, but small buffers) - Cisco-like configuration interface (similar, not the same) - Custom Linux kernel and OS - Basic look-and-feel, but so far the quality has not been a disappointment - Decent support for topologies with no Spanning-Tree - Good compatibility with SFP+ transceivers, direct connections, and optics from various sources. - Basic feature set (OSPF/RIP, but no BGP) - Somewhat limited troubleshooting and debug tools One very pleasant aspect of working with Quanta is that they are very responsive to feature requests, often working closely with customers. On the other hand, their release schedules are somewhat non-specific. I've been waiting for full MLAG support for a while (it's supposedly right around the corner). They are particularly convenient if you are putting them at the top of racks full of Quanta servers, since they have logistics and full-rack staging/shipping. I wish they had better MIB support, BGP, scriptability, and policy-based routing, but they don't. They are cheap enough, however, that you may be able to get two LY2 switches for the price of one of some of their competitors. -- Dan Sneddon On Tuesday, February 19, 2013 at 8:21 PM, Bao Nguyen wrote: Anyone have worked with the switching vendor Quanta for their 10ge switching as TOR? [1] Their spec looked interesting and they are quiet cheap. [1] http://www.quantaqct.com/en/01_product/02_detail.php?mid=30sid=114id=116qs=63 -bn 0216331C
Re: switch 10G standalone TOR, core to DC
On Thu, Feb 7, 2013 at 12:55 PM, Nick Hilliard n...@foobar.org wrote: On 29/01/2013 11:58, Nick Hilliard wrote: None of them will do trill. The Extreme X670 and Juniper EX4550 will both do VPLS, though. The X670 won't do BGP. this is incorrect: the ex4550 will do l2vpn/l3vpn but not vpls. The X480 does vpls, but not the X670. Nick I normally just lurk but I thought I would post to clear up the confusion. Full disclosure, I am an Extreme Networks TAC engineer. The x450 does not support any VPLS/H-VPLS/MPLS and is discontinued. It was replaced with the x460 which does support VPLS/H-VPLS/VPWS. The x480 and x670 both support VPLS/H-VPLS/VPWS. The x460, x480 and x670 all support BGP. However, only the x480 can hold the BGP full-view in hardware. So while you can run BGP on the x460 or x670 they are really only suitable for iBGP. All switches require a Core license to run BGP and an MPLS license to run VPLS/H-VPLS/VPWS. Andrew
Re: switch 10G standalone TOR, core to DC
On 12/02/2013 12:09, Andrew McConachie wrote: I normally just lurk but I thought I would post to clear up the confusion. Full disclosure, I am an Extreme Networks TAC engineer. The x450 does not support any VPLS/H-VPLS/MPLS and is discontinued. It was replaced with the x460 which does support VPLS/H-VPLS/VPWS. The x480 and x670 both support VPLS/H-VPLS/VPWS. Thanks for the clarification on this. The data sheet on the x670 doesn't actually mention vpls: www.extremenetworks.com/libraries/products/DSSumX670_1777.pdf ... just that there is an mpls feature set license, but with no details of what it contains. Normally vendors can't tell enough about useful features like this, so in the absence of mentioning it I had assumed incorrectly that it wasn't supported on this platform. Maybe you could get the documentation updated to include this information, because this is an important feature? Nick
Re: switch 10G standalone TOR, core to DC
On Tue, Feb 12, 2013 at 1:15 PM, Nick Hilliard n...@foobar.org wrote: On 12/02/2013 12:09, Andrew McConachie wrote: I normally just lurk but I thought I would post to clear up the confusion. Full disclosure, I am an Extreme Networks TAC engineer. The x450 does not support any VPLS/H-VPLS/MPLS and is discontinued. It was replaced with the x460 which does support VPLS/H-VPLS/VPWS. The x480 and x670 both support VPLS/H-VPLS/VPWS. Thanks for the clarification on this. The data sheet on the x670 doesn't actually mention vpls: www.extremenetworks.com/libraries/products/DSSumX670_1777.pdf ... just that there is an mpls feature set license, but with no details of what it contains. Normally vendors can't tell enough about useful features like this, so in the absence of mentioning it I had assumed incorrectly that it wasn't supported on this platform. Maybe you could get the documentation updated to include this information, because this is an important feature? Nick Thanks for pointing that out. Documentation folks have been alerted. For a quick comparison of all switches I like the Comparison Guide the best. http://www.extremenetworks.com/libraries/products/MSComparisonChart_1636.pdf --Andrew
Re: switch 10G standalone TOR, core to DC
W dniu 2013-02-07 22:54, Sergey Marunich pisze: Hi Peter, http://www.aristanetworks.com/media/system/pdf/Datasheets/7050S_Datasheet.pdf Arista 7050S-64 48 x 10GE + 4 x 40 GE, price around 25k$ in gpl. Large buffers, supports MLAG, DCB, wire-speed L2/L3 (OSPF,BGP), but doesn't have any kind of TRILL implementation. from documentation: shared 9 MB packet buffer pool that is allocated dynamically to ports that are congested 9MB is a standard size of port buffers.. regards, Piotr
Re: switch 10G standalone TOR, core to DC
On 12/02/2013 14:23, Piotr wrote: shared 9 MB packet buffer pool that is allocated dynamically to ports that are congested 9MB is a standard size of port buffers.. That's pretty standard for a cut-thru ToR switch of this style. Cut-thru switches generally need a lot less packet buffer space than store-n-forward switches. Also, ToR boxes tend not to have complex qos requirements. Having said that, you need to be careful deploying small-buffer boxes. If you're not careful, you will end up with bad packet loss. Nick
Re: switch 10G standalone TOR, core to DC
On 29/01/2013 11:58, Nick Hilliard wrote: None of them will do trill. The Extreme X670 and Juniper EX4550 will both do VPLS, though. The X670 won't do BGP. this is incorrect: the ex4550 will do l2vpn/l3vpn but not vpls. The X480 does vpls, but not the X670. Nick
Re: switch 10G standalone TOR, core to DC
Well, talking about HP´s A5920/A5900 series. Last time I was looking, their virtual routing instances haven´t supported IPv4 multicast, nor IPv6 multicast/unicast, nor any policy based routing. Michael
RE: switch 10G standalone TOR, core to DC
Hi Peter, http://www.aristanetworks.com/media/system/pdf/Datasheets/7050S_Datasheet.pdf Arista 7050S-64 48 x 10GE + 4 x 40 GE, price around 25k$ in gpl. Large buffers, supports MLAG, DCB, wire-speed L2/L3 (OSPF,BGP), but doesn't have any kind of TRILL implementation. Have it in production, but for now using for L2 only with MLAG. As option also can be considered: Brocade VDX6720, has own TRILL-like protocol to make STP-free topology, also can do L3, DCB but pay attention licensing is painful with Brocade. Best Regards,Sergey Date: Thu, 7 Feb 2013 18:13:58 +0100 From: excel...@gmx.com To: nanog@nanog.org Subject: Re: switch 10G standalone TOR, core to DC Well, talking about HP´s A5920/A5900 series. Last time I was looking, their virtual routing instances haven´t supported IPv4 multicast, nor IPv6 multicast/unicast, nor any policy based routing. Michael
Re: switch 10G standalone TOR, core to DC
Someone use this switches ? 1. Alacatel lucent omniswitch OS6900-X40 Deep packet buffers for simultaneous high-burst absorption in all ports gpl 28k$ 2. Hp 5900 af 48xg large buffer options - configurable buffers gpl 30k$ What is, exactly, buffer size ? I can't find in documentation best, Piotr
Re: switch 10G standalone TOR, core to DC
W dniu 2013-01-30 12:59, Ingo Flaschberger pisze: Am 30.01.2013 11:30, schrieb Piotr: 2. Hp 5900 af 48xg large buffer options - configurable buffers gpl 30k$ small: Memory and processor 512 MB flash, 2 GB SDRAM; packet buffer size: 9 MB http://h17007.www1.hp.com/us/en/products/switches/HP_5900_Switch_Series/index.aspx#Layer%202%20switching Tab models small in 5900 series: High-performance 10 GbE switching — cut-through and nonblocking architecture delivers industry-leading low latency (~1 microsecond) and very demanding enterprise applications; the switch delivers a 1.28 Tbps switching capacity and 952.32 Mpps packet forwarding rate in addition to incorporating 9 MB of packet buffers or big in 5920: High-performance 10GbE switching — enables you to scale your server-edge 10GbE ToR deployments with 24 high-density 10GbE ports delivered in a 1RU design; delivers a 480 Gbps (357.12 Mpps) switching capacity in addition to incorporating 3.6 GB of packet buffers Ultra-deep packet buffering — provides up to a 3.6 GB packet buffer to reduce network congestion at the I/O that is associated with the heavy use of server virtualization, as well as bursty multimedia, storage applications, and other critical services http://h17007.www1.hp.com/us/en/products/switches/HP_5920_Switch_Series/index.aspx
RE: switch 10G standalone TOR, core to DC
Cisco also now has the Nexus 6001 but I don't know of its ability to do BGP or support things like Netflow. 48x10GE+4x40GE in 1RU. Also likely doesn't have huge packet buffers. From: Piotr Sent: 1/30/2013 5:32 To: nanog@nanog.org Subject: Re: switch 10G standalone TOR, core to DC Someone use this switches ? 1. Alacatel lucent omniswitch OS6900-X40 Deep packet buffers for simultaneous high-burst absorption in all ports gpl 28k$ 2. Hp 5900 af 48xg large buffer options - configurable buffers gpl 30k$ What is, exactly, buffer size ? I can't find in documentation best, Piotr
switch 10G standalone TOR, core to DC
Hello, I looking some 10G switches, it should work as TOR or core in DC. It should have more than 40 port 10G in one unit, wirespeed L2 L3, with virtual routers and some other ip functions like some BGP, OSPF, policy routing, 1-2U, MLAG, g.8032 (ERPS) trill-like ? Other important features are big port buffers ( something similar to Juniper EX8200 - 512 MB per slot), defined counters accessible via snmp (like in junos), L3 statistics accessible via snmp Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. What vendor, model You prefer or suggest as a solution ? thanks for help best, Peter
Re: switch 10G standalone TOR, core to DC
On 29/01/2013 11:27, Piotr wrote: Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. the extreme x670, juniper ex4550, brocade ICX6550 and arista 7150 will most of this, and probably many others too. None of them will do trill. The Extreme X670 and Juniper EX4550 will both do VPLS, though. The X670 won't do BGP. You won't find a box of this form with large port buffers. There don't appear to be any of these boxes on the market at the moment, probably because none of the switch vendors want to bite the bottom out of their more lucrative chassis-based switches. This is a good market opening for a new vendor - there is no technical reason why this couldn't be done. Nick
Fwd: Re: switch 10G standalone TOR, core to DC
a...@shady.org replied: Subject: Re: switch 10G standalone TOR, core to DC Date: Tue, 29 Jan 2013 12:25:57 + From: andy a...@shady.org To: Nick Hilliard n...@foobar.org CC: Piotr piotr.1...@interia.pl, nanog@nanog.org Force10's S4810 isnt bad, we use these for a 10G 48 port box that doesnt require Ultra Low latency. http://www.scribd.com/doc/90301756/Dell-Force10-S4810-Spec-Sheet Supports TRILL in some way too, Ive not had any major issues with this box, 1 or 2 bugs, but force10 (now dell) seem quick to fix these. They also have the Z9000 series, Ive not looked at this, but it might be worth having a quick look and see if it fits. on the rest, what nick said. :) On Tue, Jan 29, 2013 at 11:58:14AM +, Nick Hilliard wrote: On 29/01/2013 11:27, Piotr wrote: Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. the extreme x670, juniper ex4550, brocade ICX6550 and arista 7150 will most of this, and probably many others too. None of them will do trill. The Extreme X670 and Juniper EX4550 will both do VPLS, though. The X670 won't do BGP. You won't find a box of this form with large port buffers. There don't appear to be any of these boxes on the market at the moment, probably because none of the switch vendors want to bite the bottom out of their more lucrative chassis-based switches. This is a good market opening for a new vendor - there is no technical reason why this couldn't be done. Nick -- andya...@shady.org --- Never argue with an idiot. They drag you down to their level, then beat you with experience. JNCIE #742 ---
RE: switch 10G standalone TOR, core to DC
We use IBM networking (used to be BLADE networks) Rackswitch 8264. They will do TRILL, and have multi-chassis link aggregation, they call vLAG. We use this for cross datacenter aggregation. They do have the L3 features you are looking for and BGP as a possibility, but no full tables. It is a cut-through switch (although this can be toggled in software to store and forward in later switch os). I believe, although I can't find the doc where I read this at the moment, the packet buffer is 2G, but shared among ports. Enterasys S-Series is also an option, but the 10G port densities are much lower. S-Series has large packet buffers, chassis bonding, and L3 features (some modules support full bgp tables).
Re: switch 10G standalone TOR, core to DC
although everyone here seems to hold Cisco in contempt, the Nexux 5548 is a rock-solid switch - at least that has been my experience with it. On Tue, Jan 29, 2013 at 6:27 AM, Piotr piotr.1...@interia.pl wrote: Hello, I looking some 10G switches, it should work as TOR or core in DC. It should have more than 40 port 10G in one unit, wirespeed L2 L3, with virtual routers and some other ip functions like some BGP, OSPF, policy routing, 1-2U, MLAG, g.8032 (ERPS) trill-like ? Other important features are big port buffers ( something similar to Juniper EX8200 - 512 MB per slot), defined counters accessible via snmp (like in junos), L3 statistics accessible via snmp Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. What vendor, model You prefer or suggest as a solution ? thanks for help best, Peter -- To him who is able to keep you from falling and to present you before his glorious presence without fault and with great joy
Re: switch 10G standalone TOR, core to DC
Hi, I do suggest you go over EN offering with a fine tooth comb. We experienced a whole lot of issues with 6 x650: . from hardware licensing (start at shipping from the fab and not when the customers get them); . software licensing (have to license every box even the ones in the labs); . known eeprom defect limiting upgrade from XOS 12 to 15; . 1 vlan-translation causing all sort of head-aches with port-grouping (ether-channel); . EAPS packets being silently filtered out of VMAN's when you do not use the Core license; ( Undocumented and that is not acceptable when trying to transport customers owns EAPS traffic on their VLAN's ) . no VLAN flapping logging; Don't get me wrong, they are good campus switches... just not designed for our L2 Core purposes. And the Licensing is just an exercise in frustration. I can understand the business purpose, just not the way they go about doing it. As for L3 support, it is fine: . include IP tracking in VRRP with is a plus for us . Virtual Routers We don't need them for BGP and we do not have a MPLS network yet. As for the x670, maybe most of the hardware issue has been addressed, but I doubt the licensing and undocumented limitations is better. PS: We're using them (x650), and are planning to keep using/recommending EN products, but it did cost us a lot of man hours and un-planned crashes that could have been prevented with better documentation and support. Good luck with your project =D - Alain Hebertaheb...@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.netFax: 514-990-9443 On 01/29/13 06:27, Piotr wrote: Hello, I looking some 10G switches, it should work as TOR or core in DC. It should have more than 40 port 10G in one unit, wirespeed L2 L3, with virtual routers and some other ip functions like some BGP, OSPF, policy routing, 1-2U, MLAG, g.8032 (ERPS) trill-like ? Other important features are big port buffers ( something similar to Juniper EX8200 - 512 MB per slot), defined counters accessible via snmp (like in junos), L3 statistics accessible via snmp Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. What vendor, model You prefer or suggest as a solution ? thanks for help best, Peter
Re: switch 10G standalone TOR, core to DC
Peter, Network visibility wasn't mentioned as a requirement, but it is worth considering since the ToR switches are the best place monitor server network I/O, tunneled traffic (VxLAN, GRE etc), storage (iSCSI, FCoE, HDFS etc). The Nexus 5548 switch does not include monitoring (i.e. no NetFlow/sFlow). The Nexus 3048, along with all the other 10G ToR switches so far mentioned on this thread, supports sFlow and provides wire speed 10G/40G monitoring. The following article provides additional background: http://blog.sflow.com/2012/02/10-gigabit-ethernet.html Cheers, Peter On Tue, Jan 29, 2013 at 7:15 AM, Steven Fischer sfischer1...@gmail.com wrote: although everyone here seems to hold Cisco in contempt, the Nexux 5548 is a rock-solid switch - at least that has been my experience with it. On Tue, Jan 29, 2013 at 6:27 AM, Piotr piotr.1...@interia.pl wrote: Hello, I looking some 10G switches, it should work as TOR or core in DC. It should have more than 40 port 10G in one unit, wirespeed L2 L3, with virtual routers and some other ip functions like some BGP, OSPF, policy routing, 1-2U, MLAG, g.8032 (ERPS) trill-like ? Other important features are big port buffers ( something similar to Juniper EX8200 - 512 MB per slot), defined counters accessible via snmp (like in junos), L3 statistics accessible via snmp Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big.. What vendor, model You prefer or suggest as a solution ? thanks for help best, Peter -- To him who is able to keep you from falling and to present you before his glorious presence without fault and with great joy
