Re: [j-nsp] MX304 Port Layout
On 6/27/23 19:44, Gert Doering wrote: The issues we see / have with "satellites that are not real satellites" are - l2 link down -> l3 not going down - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic down to a 100M customer port, saturating the "vlan on trunk" link for no good (we run config-management-system managed EX3400s off an Arista MLAG pair, so the benefits outweigh the drawbacks for us) Same here. I think this is the architecture most operators run, especially because despite centralized routers being less disjointed, they are still pricier than attaching a switch to a router port via an 802.1Q trunk. If you are aggregating plenty of 1Gbps or 10Gbps ports that aren't each carrying lots of traffic, centralized ports will be more expensive than a traditional switch<=>router architecture compared to even the cheapest and most dense centralized router. I think centralized routers make a lot of sense when aggregating 100Gbps services, because doing these on a switch<=>router architecture is going to be tough when you try to aggregate N x 100Gbps links, or even a handful of 400Gbps links for the 802.1Q trunk, as most of those customers will be riding their 100Gbps port really hard, i.e., there is enough distance between 10Gbps and 100Gbps, while there really isn't any between 100Gbps and 400Gbps. One of the biggest risks I find with a standard switch<=>router architecture is the outage that could happen if that trunk goes down. Yes, you could mitigate that by making the trunk a LAG of multiple links, but the challenge that creates is how to do policing on the router sub-interface because the policer is split across the member links in the LAG. On the other hand, one could get around this commercially by letting customers know the SLA associated with being connected to only "one device", and provide the option of connecting to "a second device". The other issue I find with the switch<=>router architecture is where to do policing. With LAG's, shaping on the switch ports makes sense because you don't run into having to police across member links in a LAG on the router port. Most aggregation switches do not support egress policing with Broadcom chips, so shaping is your only tool. It has worked well enough for us on the Arista switches that we have deployed for this, but was not a reasonable option when we used to run the EX4550 that only had 4MB of packet buffers. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
Hi, On Tue, Jun 27, 2023 at 06:32:49PM +0200, Mark Tinka via juniper-nsp wrote: > How? The issues we see / have with "satellites that are not real satellites" are - l2 link down -> l3 not going down - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic down to a 100M customer port, saturating the "vlan on trunk" link for no good (we run config-management-system managed EX3400s off an Arista MLAG pair, so the benefits outweigh the drawbacks for us) gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany g...@greenie.muc.de signature.asc Description: PGP signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On 6/27/23 18:45, Tarko Tikan via juniper-nsp wrote: Previously mentioned centralized boxes are actually becoming more and more common now (in addition to non-redundant pizzabox formfactor that has been available for ages) that single NPU can do 2+ Tbps. For BCM J2 see ACX7509, Nokia IXR-R6d or certain Cisco NCS models. All pretty good fits for SP aggregation networks. Single NPU doesn't mean non-redundant - those devices run two (or 4 in ACX case) BCM NPUs and switch "linecards" over to backup NPU when required. All without true fabric and distributed NPUs to keep the cost down. IMHO these are very compelling options for SP market, you get selection of linecards for different applications/architectures yet the overall size of the device (and power consumption) is small. I've been trying to explain to one of the major vendors that they should build similar device with their own silicon so you get similar benefits while keeping the rich featureset that comes with vendor silicon compared to BCM. Ah, okay - I'm with you now. I had a totally different definition in my head for what "centralized" meant. Yes, agreed - these make sense because of just how cheap, yet fast, Broadcom chips are. Like you, I've been pushing our friendly vendors to build similar architectures with their in-house silicon, and like you, it has fallen on deaf ears. Because IP traffic is becoming more and more public, the need for features that drove end-to-end on-net VPN's in silicon will continue to decline. If that helps operators to simplify their product technical configuration to where Broadcom can handle 90% of all scenarios, giving up the other 10% may be worth the capex/opex savings. It is that 10% that many operators want to keep, and the traditional vendors are locking up that 10% in their own silicon to print money. With every passing year, Broadcom adds to the things I want that it could not do. I am close to the 90% for some of these platforms, to where I can consider them now. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
hey, While I'm not sure operators want that, they will take a look if the lower price does not impact performance. Previously mentioned centralized boxes are actually becoming more and more common now (in addition to non-redundant pizzabox formfactor that has been available for ages) that single NPU can do 2+ Tbps. For BCM J2 see ACX7509, Nokia IXR-R6d or certain Cisco NCS models. All pretty good fits for SP aggregation networks. Single NPU doesn't mean non-redundant - those devices run two (or 4 in ACX case) BCM NPUs and switch "linecards" over to backup NPU when required. All without true fabric and distributed NPUs to keep the cost down. IMHO these are very compelling options for SP market, you get selection of linecards for different applications/architectures yet the overall size of the device (and power consumption) is small. I've been trying to explain to one of the major vendors that they should build similar device with their own silicon so you get similar benefits while keeping the rich featureset that comes with vendor silicon compared to BCM. -- tarko ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On 6/27/23 17:07, Saku Ytti wrote: Yes. How? Like cat6500/7600 linecards without DFC, so SP gear with centralised logic, and dumb 'low performance' linecards. Given low performance these days is multi Tbps chips. While I'm not sure operators want that, they will take a look if the lower price does not impact performance. There is more to just raw speed. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On Tue, 27 Jun 2023 at 17:40, Mark Tinka wrote: > Would that be high-density face-plate solutions for access aggregation > in the data centre, that they are> Are you suggesting standard 802.1Q/Q-in-Q > trunking from a switch into a > "pricey" router line card that support locally-significant VLAN's per > port is problematic? Yes. > I'm still a bit unclear on what you mean by "centralized"... in the > context of satellite, or standalone? Like cat6500/7600 linecards without DFC, so SP gear with centralised logic, and dumb 'low performance' linecards. Given low performance these days is multi Tbps chips. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On 6/27/23 09:02, Saku Ytti wrote: Juniper messaging seems to be geo-specific, in EU their sales seems to sell them more willingly than in US. My understanding is that basically fusion is dead, but they don't actually have solution for access/SP market front-plate, so some sales channels are still pitching it as the solution. Would that be high-density face-plate solutions for access aggregation in the data centre, that they are struggling with? I haven't used their EX platform since the EX4600 broke things, and I have never tried their QFX platform. So not sure if Juniper have any decent switch offerings for large scale data centre aggregation in 1U form factors, that customers are actually happy with. Nokia seems very committed to it. I think the solution space is a) centralised lookup engines - so you have cheap(er) line cards for high density low pps/bps b) satellite c) vlan aggregation Satellite is basically a specific scenario of c), but it does bring significant derisking compared to vlan aggregation, as a single instance is designing it and can solve some problems better than can be solved by vendor agnostic vlan aggregation. Vlan aggregation looks very simple on the surface but is fraught with problems, many of which are slightly better solved in satellites, and these problems will not be identified ahead of time but during the next two decades of operation. Are you suggesting standard 802.1Q/Q-in-Q trunking from a switch into a "pricey" router line card that support locally-significant VLAN's per port is problematic? Centralised boxes haven't been available for quite a few years, but hopefully Cisco is changing that, I think it's the right compromise for SPs. I'm still a bit unclear on what you mean by "centralized"... in the context of satellite, or standalone? Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
Hi, > On 26 Jun 2023, at 20:56, Jackson, William via juniper-nsp > wrote: > >> The MX204 is an MPC7E, so whatever H-QoS is on the MPC7E is what the >> MX204 will also do. > >> We have used them as an edge router on a temporary basis at new sites, >> with an Arista switch hanging off of them via an 802.1Q trunk, until we >> can get our standard MX480 to site. They are capable for such a >> use-case. But usually, we use them for peering and value-added traffic. > > Similar use case here but we use a QFX as a fusion satellite if port > expansion is required. > Works well as an small site start up option. I have done the same. One thing to remember is that even though you can run dual-RE on the MX304, the QFX won’t be able to do an upgrade without rebooting. Don’t plan to do ISSU in such a setup. Cheers, Sander ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On Tue, 27 Jun 2023 at 06:02, Mark Tinka via juniper-nsp wrote: > > Similar use case here but we use a QFX as a fusion satellite if port > > expansion is required. > > Works well as an small site start up option. > > Are vendors still pushing their satellite switches :-)? > > That technology looked dodgy to me when Cisco first proposed it with > 9000v, and then Juniper and Nokia followed with their own implementations. Juniper messaging seems to be geo-specific, in EU their sales seems to sell them more willingly than in US. My understanding is that basically fusion is dead, but they don't actually have solution for access/SP market front-plate, so some sales channels are still pitching it as the solution. Nokia seems very committed to it. I think the solution space is a) centralised lookup engines - so you have cheap(er) line cards for high density low pps/bps b) satellite c) vlan aggregation Satellite is basically a specific scenario of c), but it does bring significant derisking compared to vlan aggregation, as a single instance is designing it and can solve some problems better than can be solved by vendor agnostic vlan aggregation. Vlan aggregation looks very simple on the surface but is fraught with problems, many of which are slightly better solved in satellites, and these problems will not be identified ahead of time but during the next two decades of operation. Centralised boxes haven't been available for quite a few years, but hopefully Cisco is changing that, I think it's the right compromise for SPs. But in reality I'm not sure if centralised actually makes sense, since I don't think we can axiomatically assume it costs less to the vendor, even though there is less BOM, the centralised design does add more engineering cost. It might be basically a way to sell boxes to some market at lower margins, while ensuring that hyperscalers don't buy them, instead of directly benefiting from the cost reduction. -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
