Re: [j-nsp] MX304 Port Layout

2023-07-05 Thread Saku Ytti via juniper-nsp
On Wed, 5 Jul 2023 at 04:45, Mark Tinka wrote: > This is one of the reasons I prefer to use Ethernet switches to > interconnect devices in large data centre deployments. > > Connecting stuff directly into the core routers or directly together > eats up a bunch of ports, without necessarily using

Re: [j-nsp] MX304 Port Layout

2023-07-04 Thread Mark Tinka via juniper-nsp
On 7/4/23 09:11, Saku Ytti wrote: You must have misunderstood. When they fully scale the current design, the design offers 100T capacity, but they've bought 400T of ports. 3/4 ports are overhead to build the design, to connect the pizzaboxes together. All ports are used, but only 1/4 are

Re: [j-nsp] MX304 Port Layout

2023-07-04 Thread Saku Ytti via juniper-nsp
On Tue, 4 Jul 2023 at 08:34, Mark Tinka wrote: > Yes, I watched this NANOG session and was also quite surprised when they > mentioned that they only plan for 25% usage of the deployed capacity. > Are they giving themselves room to peak before they move to another chip > (considering that they

Re: [j-nsp] MX304 Port Layout

2023-07-03 Thread Mark Tinka via juniper-nsp
On 7/2/23 18:04, Saku Ytti wrote: Not disagreeing here, but how do we define oversubscribed here? Are all boxes oversubscribed which can't do a) 100% at max size packet and b) 100% at min size packet and c) 100% of packets to delay buffer, I think this would be quite reasonable definition,

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Saku Ytti via juniper-nsp
On Sun, 2 Jul 2023 at 17:15, Mark Tinka wrote: > Technically, do we not think that an oversubscribed Juniper box with a > single Trio 6 chip with no fabric is feasible? And is it not being built > because Juniper don't want to cannibalize their other distributed > compact boxes? > > The MX204,

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Mark Tinka via juniper-nsp
On 7/2/23 15:19, Saku Ytti wrote: Right as is MX304. I don't think this is 'my definition', everything was centralised originally, until Cisco7500 came out, which then had distributed forwarding capabilities. Now does centralisation truly mean BOM benefit to vendors? Probably not, but it

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Saku Ytti via juniper-nsp
On Sun, 2 Jul 2023 at 15:53, Mark Tinka via juniper-nsp wrote: > Well, by your definition, the ASR9903, for example, is a distributed > platform, which has a fabric ASIC via the RP, with 4x NPU's on the fixed > line card, 2x NPU's on the 800Gbps PEC and 4x NPU's on the 2Tbps PEC. Right as is

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Mark Tinka via juniper-nsp
On 6/28/23 09:29, Saku Ytti via juniper-nsp wrote: This of course makes it more redundant than distributed box, because distributed boxes don't have NPU redundancy. Well, by your definition, the ASR9903, for example, is a distributed platform, which has a fabric ASIC via the RP, with 4x

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Mark Tinka via juniper-nsp
On 7/2/23 11:18, Saku Ytti wrote: In this context, these are all distributed platforms, they have multiple NPUs and fabric. Centralised has a single forwarding chip, and significantly more ports than bandwidth. So to clarify your definition of "centralized", even if there is no

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Saku Ytti via juniper-nsp
On Sun, 2 Jul 2023 at 12:11, Mark Tinka wrote: > Well, for data centre aggregation, especially for 100Gbps transit ports > to customers, centralized routers make sense (MX304, MX10003, ASR9903, > e.t.c.). But those boxes don't make sense as Metro-E routers... they can > aggregate Metro-E

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Mark Tinka via juniper-nsp
On 7/2/23 10:42, Saku Ytti wrote: Yes. Satellite is basically VLAN aggregation, but a little bit less broken. Both are much inferior to MPLS. I agree that using vendor satellites solves this problem. The issue, IIRC, is was what happens when you need to have the satellites in rings?

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Saku Ytti via juniper-nsp
On Sun, 2 Jul 2023 at 11:38, Mark Tinka wrote: > So all the above sounds to me like scenarios where Metro-E rings are > built on 802.1Q/Q-in-Q/REP/STP/e.t.c., rather than IP/MPLS. Yes. Satellite is basically VLAN aggregation, but a little bit less broken. Both are much inferior to MPLS. But

Re: [j-nsp] MX304 Port Layout

2023-07-02 Thread Mark Tinka via juniper-nsp
On 6/28/23 08:44, Saku Ytti wrote: Apart from obvious stuff like QoS getting difficult, not full feature parity with VLAN and main interface, or counters becoming less useful as many are port level so identifying true source port may not be easy. There are things that you'll just discover

Re: [j-nsp] MX304 Port Layout

2023-06-28 Thread Saku Ytti via juniper-nsp
On Tue, 27 Jun 2023 at 19:47, Tarko Tikan via juniper-nsp wrote: > Single NPU doesn't mean non-redundant - those devices run two (or 4 in > ACX case) BCM NPUs and switch "linecards" over to backup NPU when > required. All without true fabric and distributed NPUs to keep the cost > down. This

Re: [j-nsp] MX304 Port Layout

2023-06-28 Thread Saku Ytti via juniper-nsp
On Tue, 27 Jun 2023 at 19:32, Mark Tinka wrote: > > Yes. > > How? Apart from obvious stuff like QoS getting difficult, not full feature parity with VLAN and main interface, or counters becoming less useful as many are port level so identifying true source port may not be easy. There are things

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Mark Tinka via juniper-nsp
On 6/27/23 19:44, Gert Doering wrote: The issues we see / have with "satellites that are not real satellites" are - l2 link down -> l3 not going down - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic down to a 100M customer port, saturating the "vlan on trunk"

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Gert Doering via juniper-nsp
Hi, On Tue, Jun 27, 2023 at 06:32:49PM +0200, Mark Tinka via juniper-nsp wrote: > How? The issues we see / have with "satellites that are not real satellites" are - l2 link down -> l3 not going down - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic down to a 100M

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Mark Tinka via juniper-nsp
On 6/27/23 18:45, Tarko Tikan via juniper-nsp wrote: Previously mentioned centralized boxes are actually becoming more and more common now (in addition to non-redundant pizzabox formfactor that has been available for ages) that single NPU can do 2+ Tbps. For BCM J2 see ACX7509, Nokia

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Tarko Tikan via juniper-nsp
hey, While I'm not sure operators want that, they will take a look if the lower price does not impact performance. Previously mentioned centralized boxes are actually becoming more and more common now (in addition to non-redundant pizzabox formfactor that has been available for ages) that

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Mark Tinka via juniper-nsp
On 6/27/23 17:07, Saku Ytti wrote: Yes. How? Like cat6500/7600 linecards without DFC, so SP gear with centralised logic, and dumb 'low performance' linecards. Given low performance these days is multi Tbps chips. While I'm not sure operators want that, they will take a look if the

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Saku Ytti via juniper-nsp
On Tue, 27 Jun 2023 at 17:40, Mark Tinka wrote: > Would that be high-density face-plate solutions for access aggregation > in the data centre, that they are> Are you suggesting standard 802.1Q/Q-in-Q > trunking from a switch into a > "pricey" router line card that support locally-significant

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Mark Tinka via juniper-nsp
On 6/27/23 09:02, Saku Ytti wrote: Juniper messaging seems to be geo-specific, in EU their sales seems to sell them more willingly than in US. My understanding is that basically fusion is dead, but they don't actually have solution for access/SP market front-plate, so some sales channels are

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Sander Steffann via juniper-nsp
Hi, > On 26 Jun 2023, at 20:56, Jackson, William via juniper-nsp > wrote: > >> The MX204 is an MPC7E, so whatever H-QoS is on the MPC7E is what the >> MX204 will also do. > >> We have used them as an edge router on a temporary basis at new sites, >> with an Arista switch hanging off of them

Re: [j-nsp] MX304 Port Layout

2023-06-27 Thread Saku Ytti via juniper-nsp
On Tue, 27 Jun 2023 at 06:02, Mark Tinka via juniper-nsp wrote: > > Similar use case here but we use a QFX as a fusion satellite if port > > expansion is required. > > Works well as an small site start up option. > > Are vendors still pushing their satellite switches :-)? > > That technology

Re: [j-nsp] MX304 Port Layout

2023-06-26 Thread Mark Tinka via juniper-nsp
On 6/26/23 20:56, Jackson, William via juniper-nsp wrote: Similar use case here but we use a QFX as a fusion satellite if port expansion is required. Works well as an small site start up option. Are vendors still pushing their satellite switches :-)? That technology looked dodgy to me

Re: [j-nsp] MX304 Port Layout

2023-06-26 Thread Jackson, William via juniper-nsp
Of Mark Tinka via juniper-nsp Sent: Saturday, June 10, 2023 11:03 AM To: juniper-nsp@puck.nether.net Subject: Re: [j-nsp] MX304 Port Layout On 6/9/23 17:46, Andrey Kostin via juniper-nsp wrote: >  We have two MX204s running in pair with 2x100G taken for links > between them and remaining

Re: [j-nsp] MX304 Port Layout

2023-06-17 Thread Mark Tinka via juniper-nsp
Junos 22.4R2 that fixes this issue has been released... Mark. On 6/10/23 13:55, Mark Tinka wrote: On 6/10/23 13:50, Jason Lixfeld wrote: Do either of you two have PRs for your respective issues? If you could share, I, for one anyway, would be grateful :) For the PTX1000 issue:

Re: [j-nsp] MX304 Port Layout

2023-06-10 Thread Mark Tinka via juniper-nsp
On 6/10/23 13:50, Jason Lixfeld wrote: Do either of you two have PRs for your respective issues? If you could share, I, for one anyway, would be grateful :) For the PTX1000 issue: https://supportportal.juniper.net/s/article/PTX1000-resources-exhaustion-causing-host-loopback-wedge

Re: [j-nsp] MX304 Port Layout

2023-06-10 Thread Mark Tinka via juniper-nsp
On 6/9/23 17:46, Andrey Kostin via juniper-nsp wrote:  We have two MX204s running in pair with 2x100G taken for links between them and remaining BW is 6x100G for actual forwarding in/out. In this case it's kind of at the same level for price/100G value. Yeah, using the MX204 like this

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Saku Ytti via juniper-nsp
On Fri, 9 Jun 2023 at 20:37, Andrey Kostin wrote: > Sounds more like a datacenter setup, and for DC operator it could be > attractive to do at scale. For a traditional ISP with relatively small > PoPs spread across the country it may be not the case. Sure, not suggesting everyone is in the

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Hi Saku, Saku Ytti писал(а) 2023-06-09 12:09: On Fri, 9 Jun 2023 at 18:46, Andrey Kostin wrote: I'm not in this market, have no qualification and resources for development. The demand in such devices should be really massive to justify a process like this. Are you not? You use a lot of

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Thank you very much, Jeff, for sharing your experience. Will watch closely Release Notes for upcoming Junos releases. And kudos to Juniper for finding and fixing it, 1,5 week is very fast reaction!. Kind regards, Andrey Litterick, Jeff (BIT) писал(а) 2023-06-09 12:41: This is why we got the

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Litterick, Jeff (BIT) via juniper-nsp
9, 2023 11:09 AM To: Mark Tinka Cc: Saku Ytti ; juniper-nsp Subject: Re: [EXT] [j-nsp] MX304 Port Layout Mark Tinka писал(а) 2023-06-09 10:26: > On 6/9/23 16:12, Saku Ytti wrote: > >> I expect many people in this list have no need for more performance >> than single Trio YT

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Saku Ytti via juniper-nsp
On Fri, 9 Jun 2023 at 19:15, Andrey Kostin wrote: > Can anything else be inserted in this socket? If not, then what's the > point? For server CPUs there are many models with different clocking and > number of cores, so socket provides a flexibility. If there is only one > chip that fits the

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Litterick, Jeff (BIT) via juniper-nsp
testing all releases with multiple REs now. I guess that was not true before we found the bug. -Original Message- From: juniper-nsp On Behalf Of Mark Tinka via juniper-nsp Sent: Thursday, June 8, 2023 10:53 PM To: juniper-nsp@puck.nether.net Subject: Re: [EXT] [j-nsp] MX304 Port Layout

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Saku Ytti писал(а) 2023-06-09 10:35: LGA8371 socketed BRCM TH4. Ostensibly this allows a lot more switches to appear in the market, as the switch maker doesn't need to be friendly with BRCM. They make the switch, the customer buys the chip and sockets it. Wouldn't surprise me if FB, AMZN and

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Saku Ytti via juniper-nsp
On Fri, 9 Jun 2023 at 18:46, Andrey Kostin wrote: > I'm not in this market, have no qualification and resources for > development. The demand in such devices should be really massive to > justify a process like this. Are you not? You use a lot of open source software, because someone else did

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Mark Tinka писал(а) 2023-06-09 10:26: On 6/9/23 16:12, Saku Ytti wrote: I expect many people in this list have no need for more performance than single Trio YT in any pop at all, yet they need ports. And they are not adequately addressed by vendors. But they do need the deep features of NPU.

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Saku Ytti писал(а) 2023-06-09 10:12: On Fri, 9 Jun 2023 at 16:58, Andrey Kostin via juniper-nsp wrote: Not sure why it's eye-watering. The price of fully populated MX304 is basically the same as it's predecessor MX10003 but it provides 3.2T BW capacity vs 2.4T. If you compare with MX204, then

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Mark Tinka via juniper-nsp
On 6/9/23 16:35, Saku Ytti wrote: I'm not convinced at all that leaba is being sold. I think it's sold conditionally when customers would otherwise be lost. Probably - it's a "grain of salt" situation when you hear the news. I don't think Meta and Microsoft have not bought zero of the

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Saku Ytti via juniper-nsp
On Fri, 9 Jun 2023 at 17:26, Mark Tinka wrote: > Well, the story is that Cisco are doing this with Meta and Microsoft on > their C8000 platform, and apparently, doing billions of US$ in business > on the back of that. I'm not convinced at all that leaba is being sold. I think it's sold

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Mark Tinka via juniper-nsp
On 6/9/23 16:12, Saku Ytti wrote: I expect many people in this list have no need for more performance than single Trio YT in any pop at all, yet they need ports. And they are not adequately addressed by vendors. But they do need the deep features of NPU. This. There is sufficient

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Mark Tinka via juniper-nsp
On 6/9/23 15:57, Andrey Kostin wrote: Hi Mark, Not sure why it's eye-watering. The price of fully populated MX304 is basically the same as it's predecessor MX10003 but it provides 3.2T BW capacity vs 2.4T. That's true, but the premium being paid for 400Gbps capability that some houses

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Saku Ytti via juniper-nsp
On Fri, 9 Jun 2023 at 16:58, Andrey Kostin via juniper-nsp wrote: > Not sure why it's eye-watering. The price of fully populated MX304 is > basically the same as it's predecessor MX10003 but it provides 3.2T BW > capacity vs 2.4T. If you compare with MX204, then MX304 is about 20% > expensive

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Hi Mark, Not sure why it's eye-watering. The price of fully populated MX304 is basically the same as it's predecessor MX10003 but it provides 3.2T BW capacity vs 2.4T. If you compare with MX204, then MX304 is about 20% expensive for the same total BW, but MX204 doesn't have redundant RE and

Re: [j-nsp] MX304 Port Layout

2023-06-09 Thread Andrey Kostin via juniper-nsp
Hi Jeff, Thank you very mush for sharing this information. Do you know in what publicly available release it's going to be fixed? Knowing PR number would be the best but I guess it may be internal-only. Kind regards, Andrey Litterick, Jeff (BIT) via juniper-nsp писал(а) 2023-06-08 18:03:

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Mark Tinka via juniper-nsp
On 6/9/23 00:03, Litterick, Jeff (BIT) via juniper-nsp wrote: The big issue we ran into is if you have redundant REs then there is a super bad bug that after 6 hours (1 of our 3 would lock up after reboot quickly and the other 2 would take a very long time) to 8 days will lock the entire

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Litterick, Jeff (BIT) via juniper-nsp
this back at the end of April) -Original Message- From: juniper-nsp On Behalf Of Thomas Bellman via juniper-nsp Sent: Thursday, June 8, 2023 2:09 PM To: juniper-nsp Subject: Re: [EXT] [j-nsp] MX304 Port Layout On 2023-06-08 17:18, Kevin Shymkiw via juniper-nsp wrote: > Along with this

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Thomas Bellman via juniper-nsp
On 2023-06-08 17:18, Kevin Shymkiw via juniper-nsp wrote: > Along with this - I would suggest looking at Port Checker ( > https://apps.juniper.net/home/port-checker/index.html ) to make sure > your port combinations are valid. The port checker claims an interresting "feature": if you have

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Mark Tinka via juniper-nsp
On 6/8/23 18:39, Giuliano C. Medalha wrote: but you have the flex model. With license for capacity and features.  Advanced and Premium. Which isn't a new thing with vendors. The prices are just terrible, even with discounts. fib is better now - 12M sampling rate for ipfix is better

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Giuliano C. Medalha via juniper-nsp
> Hello good afternoon. > > Please have a look at the following documentation: > > >

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Mark Tinka via juniper-nsp
On 6/8/23 17:35, Giuliano C. Medalha wrote: Hello good afternoon. Please have a look at the following documentation: https://community.juniper.net/blogs/reema-ray/2023/03/28/mx304-deepdive Thanks, this is most useful! It will have everything you need to do with it, including the

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Giuliano C. Medalha via juniper-nsp
, 2023 12:25 PM To: Kevin Shymkiw Cc: juniper-nsp Subject: Re: [j-nsp] MX304 Port Layout On 6/8/23 17:18, Kevin Shymkiw wrote: > Along with this - I would suggest looking at Port Checker ( > https://apps/ > .juniper.net%2Fhome%2Fport-checker%2Findex.html=05%7C01%7Cgiuliano%40wzte

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Mark Tinka via juniper-nsp
On 6/8/23 17:18, Kevin Shymkiw wrote: Along with this - I would suggest looking at Port Checker ( https://apps.juniper.net/home/port-checker/index.html ) to make sure your port combinations are valid. We've had ample experience with Juniper's MPC7E, MX204, PTX1000 and PTX10001 to know how

Re: [j-nsp] MX304 Port Layout

2023-06-08 Thread Kevin Shymkiw via juniper-nsp
Along with this - I would suggest looking at Port Checker ( https://apps.juniper.net/home/port-checker/index.html ) to make sure your port combinations are valid. Kevin On Thu, Jun 8, 2023 at 9:16 AM Mark Tinka via juniper-nsp wrote: > > So, we decided to give the MX304 another sniff, and

[j-nsp] MX304 Port Layout

2023-06-08 Thread Mark Tinka via juniper-nsp
So, we decided to give the MX304 another sniff, and needed to find out why Juniper charge a license for 16x 100Gbps ports per line card, and yet the data sheet suggests the box can handle 48x 100Gbps ports chassis-wide. Well, turns out that if you deploy it with redundant RE's, you get 32x