Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 5/3/24 19:54, Lee Starnes wrote:


Hello Mark,

Thanks for asking. This is eBGP and the issue is that there have been 
failures whereby the link does not fail, and thus can't track that 
routes should be removed. BGP session has stayed up in some cases as 
well, yet no traffic.


Yeah, if the physical media is unable to indicate link-layer failure to 
the system, BFD is probably you best bet at detecting link failure.


Have you worked with your eBGP neighbor (I'm guessing your transit 
provider) to see what they can do to improve link failure detection at 
the link layer? Perhaps avoid an intermediate Ethernet switch between 
both your routers, e.t.c.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 4/29/24 17:42, Lee Starnes via juniper-nsp wrote:

As for BFD and stability with aggressive settings, we don't run too
aggressive on this, but certainly do require it because the physical links
have not gone down in our cases when we have had issues, causing a larger
delay in killing the routes for that path. Not being able to rely on link
state failure leaves us with requiring the use of BFD.


Is this link carrying eBGP or iBGP?

If the latter, have you considered using BFD to track the IGP instead of 
BGP?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 4/29/24 09:13, Saku Ytti wrote:


100%, what Mark implied was not what I was trying to communicate.
Sure, go ahead and damp flapping interfaces, but to penalise on first
down event, when most of them are just that, one event, to me, is just
bad policy made by people who don't feel the cost.


Yes, agree with this. Didn't mean to cause a mix-up.

As before, my perspective is from circuits where it can be continuous 
events in, let's say, a 12-hour period every few moments. Yes, this is 
not the norm in most mature markets, but we have had to deal with this 
sort of thing several times a year, down here, and it can get complex 
especially if the route you are dealing with has no suitable alternative 
options other than going round the continent and back.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 4/29/24 09:15, Saku Ytti wrote:


You are making this unnecessarily complicated.

You could simply configure that first down event doesn't add enough
points to damp, 2nd does. And you are wildly better off.

Perfect is the enemy of done and kills all movement towards better.


Fair enough.

My perspective is from this side of the world where backbone is not the 
greatest experience in most of the inland markets. But I grant that such 
scenarios are not the norm in more mature regions.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 4/29/24 09:06, Gert Doering wrote:


Yes, but that's a slightly different tangent.  If the underlay is unstable,
I think we're all in agremeent that higher layers should not send packets
there.


It comes down to how you classify stable (well-behaved) vs. unstable 
(misbehaving) interfaces.


This will vary for networks, backbones, providers, e.t.c.

In many cases, manual intervention will be required because even the 
most aggressive or the most conservative dampening settings will not be 
able to account for what stable and unstable interfaces means. I suppose 
one could "AI" it, but that's outside the realm of my abilities.


In other words, a one-size-fits-all is unlikely to work here. Plenty of 
tools exist, and I think it is up to the operator to educate themselves 
on all of them and make the best decision for a given scenario.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP timer

[Mark Tinka via juniper-nsp]

On 4/29/24 08:31, Saku Ytti via juniper-nsp wrote:


But why is this desirable? Why do I want to prioritise stability
always, instead of prioritising convergence on well-behaved interfaces
and stability on poorly behaved interfaces?

If I can pick just one, I'll prioritise convergence every time for both.

That is, if I cannot have exponential back-off, I won't kill
convergence 'just in case', because it's not me who will feel the pain
of my decisions, it's my customers. Netengs and particularly infosec
people quite often are unnecessarily conservative in their policies,
because they don't have skin in the game, they feel the upside, but
not the downside.


Over the decades, I've had a handful of customers that preferred uptime 
to convergence, because they were measured on that by their boss, 
organization or auditors.


You know - the kind of people that would refuse to reboot a router to 
implement new code, because "Last Reboot: 5y, 6w ago" looks far better 
than "Last Reboot: 15min ago" - those people.


Protocols staying up despite the underlay being unstable means traffic 
dies and users are not happy. It's really that simple.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] L3VPNs and on-prem DDoS scrubbing architecture

[Mark Tinka via juniper-nsp]

On 4/3/24 18:06, Tom Beecher wrote:



My first thought was also to use BGP-LU.


Would a virtual router with an lt- interface connecting the VRF to the 
global table be too expensive?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] L3VPNs and on-prem DDoS scrubbing architecture

[Mark Tinka via juniper-nsp]

On 4/3/24 08:45, Saku Ytti wrote:


Actually I think I'm confused. I think it will just work. Because even
as the EgressPE does IP lookup due to table-label, the IP lookup still
points to egressMAC, instead looping back, because it's doing it in
the CleanVRF.
So I think it just works.

So OP just needs to copy the direct route as-is, not as host/32 into
cleanVRF, with something like this:

routing-options {
   interface-routes {
 rib-groups {
   cleanVRF {
 import-rib [ inet.0 cleanVRF.inet.0 ];
 import-policy cleanVRF:EXPORT;
  

Now cleanVRF.inet.0 has the connected TableLabel, and as lookup is
done in the cleanVRF, without the Scrubber/32 route, it'll be sent to
the correct egress CE, despite doing egress IP lookup.


Sounds like it should if I logic through your example, but in our case, 
we took a different path.


We did not use RIB Groups. Everything happened in the virtual-router 
instance (including IS-IS + LDP + a dedicated Loopback interface), and 
then we connected it to the global table using an lt- interface (classic 
virtual-router vibes).


Basically, we cut the router in half (or doubled it, whichever way you 
look at it) so that one side of the router was dealing with traffic 
on-ramp to send to the scrubber for cleaning, and the other side of the 
router was dealing with traffic off-ramp to send the cleaned traffic 
toward the egress PE. Both sides of the router were virtually 
independent, even though in the same physical hardware.


You could achieve the same using two physical routers, but with the 
available tech., it would have been a waste.


We did this with an MX204, which means there could be a PFE penalty down 
the line if traffic grows, but I did not spend too much time digging 
into that, as at the time, we were only dealing with about 40Gbps of 
traffic, and needed to get the setup going ASAP.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] L3VPNs and on-prem DDoS scrubbing architecture

[Mark Tinka via juniper-nsp]

On 4/3/24 08:07, Saku Ytti via juniper-nsp wrote:


If I understand you correctly, the problem is not that you can't copy
direct into CleanVRF, the problem is that ScrubberPE that does clean
lookup in in CleanVRF, has label stack of [EgressPE TableLabel],
instead of [EgressPE EgressCE], this causes the EgressPE to do IP
lookup, which will then see the Direct/32 advertised by the scrubber,
causing loop. While what you want is end-to-end MPLS lookup, so that
egressPE MPLS lookup has egressMAC.

I believe in BGP-LU you could fix this, without actually paying for
duplicate RIB/FIB and without opportunistically copying routes to
CleanVRF, every prefix would be scrubbable by default. You'd have
per-ce for rest, but per-prefix for connected routes, I believe then
you would have [EgressPE EgressMAC_CE] label for connected routes, so
each host route would have their own label, allowing mac rewrite
without additional local IP lookup.

I'm not sure if this is the only way, I'm not sure if there would be a
way in CleanVRF to force each direct/32 to have a label as well,
avoiding the egress IP lookup loops. One doesn't immediately spring to
mind, but technically implementation could certainly allow such mode.


At old job, we managed to do this with a virtual-router VRF that carried 
traffic between the scrubbing PE and the egress PE via MPLS, to avoid 
the IP loop.


It was quite an involved configuration, but it actually worked. It sort 
of mimicked the ability of the scrubbing device being able to 
participate in your IGP (which is something I wanted but the scrubbing 
vendor was not keen to support).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX10008 power supply configuration

[Mark Tinka via juniper-nsp]

On 2/17/24 17:12, Vincent Bernat via juniper-nsp wrote:


Hey!

I am a bit lost on how the MX10008 power supplies work. My main 
question is how much power will be drawn if a power feed is lost?


If we take the JNP10K-PWR-AC2 dual feed with high power (30-A) 
setting, configured for dual feed, it can draw 5500 W. In Europe 
(230V), I suppose it means it will draw 12A on one feed and 12A on the 
other. This is not clearly stated.


What happens if we lost one feed? I suppose it would fall back at 5000 
W, so it would draw 22A on the remaining feed, but maybe it could just 
fallback to half the power at 12A?


If it would fallback at 5000 W, I don't understand why there are DIP 
switches to configured mono or dual feed. We could just not plug the 
second feed and be done with it.


Also, on the power supply, there are these markings:

INPUT1 OR INPUT2: PER INPUT 28.5A, 5000W MAX
INPUT1 AND INPUT2: PER INPUT 16A, 5500W MAX

But again, it is unclear if the choice is done by configuring the 
switches or if this depends on the number of actuel inputs used. If I 
configure the DIP switches to use both input, do I still get 16A per 
input if one feed fails?


Ideally, when configured in high power mode, it could fallback to 
~3700W when on one feed, but I suppose this would marked more clearly 
in the documentation. This would mean 33kW when full, 16.5kW when 
loosing half the power supplies (unlikely if they are dual feed) and 
22kW when loosing one feed.


Otherwise, we'll run in 20A mode, get 30kW with 6 power supplies, or 
15kW with half of them or 16.2kW with half of the feeds.


Sounds like the kind of question best answered by your SE.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Fwd: Stange issue on 100 Gbs interconnection Juniper - Cisco

[Mark Tinka via juniper-nsp]

On 2/11/24 12:10, james list via juniper-nsp wrote:


Dear experts
we have a couple of BGP peers over a 100 Gbs interconnection between
Juniper (MX10003) and Cisco (Nexus N9K-C9364C) in two different datacenters
like this:

DC1
MX1 -- bgp -- NEXUS1
MX2 -- bgp -- NEXUS2

DC2
MX3 -- bgp -- NEXUS3
MX4 -- bgp -- NEXUS4

The issue we see is that sporadically (ie every 1 to 3 days) we notice BGP
flaps only in DC1 on both interconnections (not at the same time), there is
still no traffic since once noticed the flaps we have blocked deploy on
production.

We've already changed SPF (we moved the ones from DC2 to DC1 and viceversa)
and cables on both the interconnetion at DC1 without any solution.

SFP we use in both DCs:

Juniper - QSFP-100G-SR4-T2
Cisco - QSFP-100G-SR4

over MPO cable OM4.

Distance is DC1 70 mt and DC2 80 mt, hence is less where we see the issue.

Any idea or suggestion what to check or to do ?


You don't say if these are eBGP or iBGP sessions. But assuming the 
latter, do the logs show a corresponding IGP outage?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 2/8/24 17:10, Tom Beecher wrote:



For any use cases that you want protocol interaction, but not 
substantive traffic forwarding capabilities , cRPD is by far the 
better option.


It can handle around 1M total RIB/FIB using around 2G RAM, right in 
Docker or k8. The last version of vMX I played with required at least 
5G RAM / 4 cores to even start the vRE and vPFEs up, plus you have to 
do a bunch of KVM tweaking and customization, along with NIC driver 
fun. All of that has to work right just to START the thing, even if 
you have no intent to use it for forwarding. You could have cRPD up in 
20 minutes on even a crappy Linux host. vMX has a lot more overhead.


Is the same true for VMware?

I had a similar experience trying to get CSR1000v on KVM going back in 
2014 (and Junos vRR, as it were). Gave up and moved to CSR1000v on 
VMware where it was all sweeter. Back then, vRR did not support 
VMware... only KVM.


On the other hand, if you are deploying one of these as an RR, hardware 
resources are going to be the least of your worries. In other words, 
some splurging is in order. I'd rather do that and be able to run a 
solid software-only OS than be a test-bed for cRPD in such a use-case.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX204 and IPv6 BGP announcements

[Mark Tinka via juniper-nsp]

On 2/8/24 16:29, Saku Ytti wrote:


In absence of more specifics, junos by default doesn't discard but
reject.


Right, which I wanted to clarify if it does the same thing with this 
specific feature, or if it does "discard"


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX204 and IPv6 BGP announcements

[Mark Tinka via juniper-nsp]

On 2/8/24 15:48, Jeff Haas wrote:

It’s rib-only.  If you wanted the usual other properties, you’d use 
the usual other features.




So internally, if it attracts any traffic for non-specific destinations, 
does Junos send it /dev/null in hardware? I'd guess so...


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 2/8/24 09:56, Saku Ytti via juniper-nsp wrote:


Same concerns, I would just push it back and be a late adopter. Rock
existing vRR while supported, not pre-empt into cRPD because vendor
says that's the future. Let someone else work with the vendor to
ensure feature parity and indeed perhaps get some appliance from the
vendor.


Agreed.


With HPE, I feel like there is a lot more incentive to sell integrated
appliances to you than before.


Is the MX150 still a current product? My understanding is it's an x86 
platform running vMX.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 2/8/24 09:56, Saku Ytti via juniper-nsp wrote:


Same concerns, I would just push it back and be a late adopter. Rock
existing vRR while supported, not pre-empt into cRPD because vendor
says that's the future. Let someone else work with the vendor to
ensure feature parity and indeed perhaps get some appliance from the
vendor.


Agreed.


With HPE, I feel like there is a lot more incentive to sell integrated
appliances to you than before.


Is the MX150 still a current product? My understand is it's an x86 
platform running vMX.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 2/8/24 09:50, Roger Wiklund via juniper-nsp wrote:


Hi

I'm curious, when moving from vRR to cRPD, how do you plan to manage/setup
the infrastructure that cRPD runs on?


I run cRPD on my laptop for nothing really useful apart from testing 
configuration commands, e.t.c.


I wouldn't consider cRPD for production. vRR (or vMX, if it's still a 
thing) seems to make more sense.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX204 and IPv6 BGP announcements

[Mark Tinka via juniper-nsp]

On 2/6/24 19:42, Jeff Haas wrote:


And for situations where you need it nailed up:

https://www.juniper.net/documentation/us/en/software/junos/cli-reference/topics/ref/statement/bgp-static-edit-routing-options.html


Interesting, never knew about this BGP-specific feature.

What does the router do with this in FIB? Same as a a regular static 
route pointing to 'discard'? Or does it just stay in RIB?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 2/6/24 18:53, Saku Ytti wrote:


Not just opinion, fact. If you see everything, ORR does nothing but adds cost.

You only need AddPath and ORR, when everything is too expensive, but
you still need good choices.

But even if you have resources to see all, you may not actually want
to have a lot of useless signalling and overhead, as it'll add
convergence time and risk of encouraging rare bugs to surface. In the
case where I deployed it, having all was not realistic possibly, in
that, having all would mean network upgrade cycle is determined when
enough peers are added, causing RIB scale to demand triggering full
upgrade cycle, despite not selling the ports already paid.
You shouldn't need to upgrade your boxes, because your RIB/FIB doesn't
scale, you should only need to upgrade your boxes, if you don't have
holes to stick paying fiber into.


I agree.

We started with 6 paths to see how far the network could go, and how 
well ECMP would work across customers who connected to us in multiple 
cities/countries with the same AS. That was exceedingly successful and 
customers were very happy that they could increase their capacity 
through multiple, multi-site links, without paying anything extra and 
improving performance all around.


Same for peers.

But yes, it does cost a lot of control plane for anything less than 32GB 
on the MX. The MX204 played well if you unleased it's "hidden memory" 
hack :-).


This was not a massive issue for the RR's which were running on CSR1000v 
(now replaced with Cat8000v). But certainly, it did test the 16GB 
Juniper RE's we had.


The next step, before I left, was to work on how many paths we can 
reduce to from 6 without losing the gains we had made for our customers 
and peers. That would have lowered pressure on the control plane, but 
not sure how it would have impacted the improvement in multi-site load 
balancing.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX204 and IPv6 BGP announcements

[Mark Tinka via juniper-nsp]

On 2/6/24 18:48, Lee Starnes via juniper-nsp wrote:


Hello everyone,

I was having difficulty in getting an announcement of a IPv6 /32 block
using prefix-lists rather than redistribution of the IP addresses in from
other protocols. We only have a couple /64 blocks in use at the moment but
want to be able to announce the entire /32. In cisco, that would just be a
holddown route and then announce. Not sure how it works to Juniper.

I configured a prefix-list that contained the /32 block in it. Then created
a policy statement with term 1 from prefix-list  and then term 2 then
accept. Set the export in BGP protocol peer of this policy statement and it
just ignores it.

Now this same setup in IPv4 works fine.

After a week of going round and round with Juniper TAC, they had me setup a
rib inet6 aggregate entry for the /32 and then use that in the policy
statement.


This is the equivalent of the "hold-down" route you refer to in 
Cisco-land. Useful if the route does not exist in the RIB from any other 
source.


I'm guessing your IPv4 route just works without a hold-down route 
because it is being learned from somewhere else (perhaps your IGP, iBGP 
or a static route), and as such, already exists in the router's RIB for 
your export policy to pick it up with no additional fiddling.


Typically, BGP will not originate a route to its neighbors unless it 
already exists in the routing table through some source. If it is an 
aggregate route, a hold-down pointing to "discard" (Null0 in Cisco) is 
enough. If it is a longer route assigned to a customer, that route 
pointing to the customer will do.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 12/8/23 19:36, Jared Mauch via juniper-nsp wrote:


I’ll also comment that many software suites don’t scale to 10’s or 100’s of 
million of paths

Keep in mind paths != routes and many folks don’t always catch the difference 
between them.  If you have a global network like 2914 (for example) you may be 
peering with someone in 10-20 places globally so if they send you 10k routes, * 
20 locations that’s 200k paths(exits), then move to someone with 100k or 400k 
prefixes like 3356 had at one point, those numbers go up quite a bit.


Our outfit was not as large as 2914 or 3356 when I worked there, but our 
RR's saw about 12.5 million IPv4 paths and 2.9 million IPv6 paths.


The clients saw about 6 million paths and 1.2 million paths, respectively.

The biggest issues to think about his how the RE handles path churn, 
which can be very high in a setup such as this, because while it 
provides excellent path stability for downstream eBGP customers, it 
creates a lot of noise inside your core.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 12/8/23 19:16, Saku Ytti via juniper-nsp wrote:


I tried to advocate for both, sorry if I was unclear.

ORR for good options, add-path for redundancy and/or ECMPability.


IME, when we got all available paths, ORR was irrelevant.

But yes, at the cost of some control plane resources.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 12/8/23 18:57, Saku Ytti via juniper-nsp wrote:


Given a sufficient count of path options, they're not really
alternatives, but you need both. Like you can't do add-path , as
the clients won't scale. And you probably don't want only ORR, because
of the convergence cost of clients not having a backup option or the
lack of ECMP opportunity.


I found that if you run 6 or more paths on an RR client, the need for 
ORR was negated.


But yes, it does put a lot of pressure on the RE, with a 64GB RAM system 
being the recommended minimum, I would say.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Hardware configuration for cRPD as RR

[Mark Tinka via juniper-nsp]

On 12/7/23 17:05, Saku Ytti via juniper-nsp wrote:


If you have a
low amount of duplicate RIB entries it might not be very useful, as
final collation of unique entries will be more or less single threaded
anyhow. But I believe anyone having a truly large RIB, like 20M, will
have massive duplication and will see significant benefit.


So essentially, outfits running BGP Add-Paths setups that have 6 or more 
paths per route, then...


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/11/24 02:56, Chris Kawchuk via juniper-nsp wrote:


Shall we start taking bets on what stays, and what goes?


I'm glad that Rami gets to stay as CEO for the networking side of 
things. Good lad, that...

Again, ACX was never a competitor to the ASR920 which I know Mr Tinka was very fond 
of. And the NCS540 "is the new ASR920”. There’s some long roads ahead for JNPR 
to wrestle back some of that marketshare.


The whole Metro-E industry is currently a balls-up. All vendors seem to 
have met at a secret location that served 20-year old wine and agreed 
not to pursue any Metro-E platforms built around custom silicon.


So really, what you are buying from either Cisco, Juniper, Nokia, Arista 
or Arrcus will be code maturity. No other differentiator.



ACX also did a ‘reboot’ of the product line in the 7000-series when they went 
Jericho, versus ACX5000 which (correct me if I’m wrong) that was 
QFX/Trident/Trident+ based and earlier ACX series which were 
$no-idea-i-didnt-look-very-hard-at-them…. so its almost “a new product” which 
may not have a lot of customer nor market traction; thus easier to kill off. 
Yes — even though previous generations of ACX did exist and likely had some 
customers..somewhere…., I know of absolutely nobody that bought them nor used 
them in anger for a large Metro-E/MPLS/eVPN/SR network role.

I'm happy to be proven wrong on ACX; as I don’t like the idea of handing an 
entire market segment to a single vendor.


I am aware of a few major orders of the ACX7024 that Juniper are working 
on. Of course, none of it will become materially evidential until the 
end of 2024. That said, I think HP will give the box a chance, as there 
is a market for it. They might just put a time line on it.


And for once in Juniper's history, they are beginning to take the 
Metro-E network a little seriously, although probably a tad later than 
they should have.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/10/24 21:30, Aaron Gould via juniper-nsp wrote:
https://newsroom.juniper.net/news/news-details/2024/HPE-to-Acquire-Juniper-Networks-to-Accelerate-AI-Driven-Innovation/ 



Glad to see Rami will be staying on.

Considering Juniper's current market cap of US$9.5 billion, that US$14 
billion price tag is rather hefty.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Difference in "MX204" and "MX204-HW-BASE"?

[Mark Tinka via juniper-nsp]

On 1/10/24 19:50, Richard McGovern via juniper-nsp wrote:


The “difference” is that either SKU above does not contain a [Flex] Feature 
License. Some Feature License, Adv or Prem, at some term (years or perpetual) 
must now be included if you want any MX to do any L3 or above features. So 
basically without some Feature License tied the HW SN via some Flex Feature 
License, it is a good boat anchor!

For information on Flex Licenses go here - 
https://www.juniper.net/documentation/us/en/software/license/juniper-licensing-user-guide/topics/concept/licenses-for-juniper-software.html

This change in how MX and other Juniper products has been driven by Stock 
Analysist and other vendors. “I don’t make the news, I just report it”


I can believe this.

If other vendors are dumping honor-based or perpetual licenses, it would 
be commercially silly to not follow suit.


Kind of like what Broadcom have done since picking up VMware, and 
telling customers to move to subscription-based billing in lieu of 
perpetual licenses.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/10/24 18:22, Tom Beecher wrote:



I wouldn't necessarily agree that was the wrong *technical* decision. 
Unfortunately, it was a perfect scenario to be exploited for the 
MBA-ification of everything that has greatly expanded in the past decade.


I agree.

Kind of like "not making a mistake of it, but being wrong at the same 
time", if you know what I mean :-).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/10/24 09:04, Forrest Christian (List Account) wrote:

I find it frustrating that things one would expect to be included in 
any layer 3 switch has become additional revenue opportunities.


"The switch hardware is $x.  Oh you want the software too?  Oh,  
that's an additional cost.   L3 switching?  Oh,  that's an extra 
feature.  OSPF? Oh that's not included with the L3 license so that 
will be extra too. Oh and by the way,  you aren't buying a perpetual 
license anymore so be sure to pay us the fees for all the software 
functionality every year".


Yes I know the above isn't completely 100% accurate but it definitely 
is how it seems anymore.


I get charging extra for advanced features,  but when basic features 
that pretty much everyone wants and uses becomes an add-on and not 
perpetual,  it tends to make me start looking for a different vendor.


In our hubris to "decouple the control plane from the data plane (tm)", 
we, instead, decoupled the software/hardware integration from a single 
vendor.


So hardware folk make their own cut, and software folk make their own 
cut. And they are not the same people.


Welcome to the "white box" and "software-only" era.

Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/9/24 11:47, Roger Wiklund wrote:



Yeah the ISP business is no fun, I feel like everyone secretly wishes 
they can start buying Huawei again, It seems it's all about the 
lowest price per 100G/400G port.


There is no shortage of cheap ports. The issue is how useful those ports 
are beyond just "speed".


This is where Trio does well, but this may not be enough to save the day.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Thanks for all the fish

[Mark Tinka via juniper-nsp]

On 1/9/24 10:55, Saku Ytti via juniper-nsp wrote:

What do we think of HPE acquiring JNPR?


I guess it was given that something's gotta give, JNPR has lost to
dollar as an investment for more than 2 decades, which is not
sustainable in the way we model our economy.

Out of all possible outcomes:
- JNPR suddenly starts to grow (how?)
- JNPR defaults
- JNPR gets acquired

It's not the worst outcome, and from who acquires them, HPE isn't the
worst option, nor the best. I guess the best option would have been,
several large telcos buying it through a co-owned sister company, who
then are less interested in profits, and more interested in having a
device that works for them. Worst would probably have been Cisco,
Nokia, Huawei.

I think the main concern is that SP business is kinda shitty business,
long sales times, low sales volumes, high requirements. But that's
also the side of JNPR that has USP.

What is the future of NPU (Trio) and Pipeline (Paradise/Triton), why
would I, as HP exec, keep them alive? I need JNPR to put QFX in my DC
RFPs, I don't really care about SP markets, and I can realise some
savings by axing chip design and support. I think Trio is the best NPU
on the market, and I think we may have a real risk losing it, and no
mechanism that would guarantee new players surfacing to replace it.

I do wish that JNPR had been more serious about how unsustainable it
is to lose to the dollar, and had tried more to capture markets. I
always suggested why not try Trio-PCI in newegg. Long tail is long,
maybe if you could buy it for 2-3k, there would be a new market of
Linux PCI users who want wire rate programmable features for multiple
ports? Maybe ESXi server integration for various pre-VPC protection
features at wire-rate? I think there might be a lot of potential in
NPU-PCI, perhaps even FAB-PCI, to have more ports than single NPU-PCI.


HP could do what Geely did for Volvo - give them cash, leave them alone, 
but force them to wake up and get into the real world.


I don't think HP can match Juniper intellectually in the networking 
space, so perhaps they add another sort of credibility to Juniper, as 
long as Juniper realize that they need to get cleverer at staying in 
business than just being technically smart.


I am concerned that if we lose Trio, it would be the end of half-decent 
line-rate networking, which would level the playing field around 
Broadcom... good for them, but perhaps not so great for operators. On 
the other hand, as you say, the ISP business is in a terrible place 
right now, and not looking to get any better as the core of the Internet 
continues to be owned by a small % of the content crew.


And then there was this, hehe:

    https://hpjuniper.com/en/signature-gin/

Hehe.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/26/23 16:10, Aaron Gould wrote:
After tshooting with JTAC yesterday, they've determined the built-in 
FPC to be a problem.  They are doing RMA.


Strange that when the 60-day trail license expired, I decided to 
reboot to see what would happen.  I rebooted "request system reboot 
both-routing-engines" and that's when the router never worked after 
that.  Strange that this would "fry" the FPC.  Maybe there was already 
something wrong with it... I don't know. Perhaps I'll try to reproduce 
it after the new chassis comes back.


-Aaron

I wonder if the "request vmhost reboot routing-engine both" would've 
done anything differently


According to the documentation, re1 is also seen as fpc0 if you put an 
LMIC in it. Would have been good to troubleshoot by putting an LMIC in 
the re1 slot to see if that works. But since re1 is fpc0/pic2, then it 
probably wouldn't work if JTAC are saying it's an FPC failure.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/26/23 15:47, Saku Ytti wrote:

I urge everyone to give them the same message as I've given.

Any type of license, even timed license, after it expires will not
cause an outage. And enforcement would be 'call home' via 'http(s)'
proxy, which reports the license-use data to Juniper sales, making it
a commercial problem between Juniper and you.

Proxy, so that you don't need Internet access on the device.
Potentially you could ask for encryption-less mode, if you want to log
on the proxy what is actually being sent to the vendor. I don't give
flying or any other method of locomotion fuck about leaking
information.

I believe this is a very reasonable give/take compromise which is
marketable, but if we try to start punching holes through esoteric
concerns, we'll get boxes which die periodically because someone
forgot to re-up. This is a real future that may happen, unless we
demand it must not.


I agree.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

So my SE came back to me a short while ago to say that at present, 
routing protocols will not be disabled if an MX304 (or some future 
box/code designed for the same authorization framework) does not have 
the appropriate license installed.


He did add, however, that Juniper are considering enforcing routing 
protocol licenses in the future, and that he cannot say, with any 
certainty, that this will not become a thing in the future.


I'd suggest staying very close to our SE's for the desired outcome we 
want for this development. As we have seen before, Juniper appear 
reasonably open to operator feedback, but we would need to give it to 
them to begin with.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/26/23 08:02, Saku Ytti wrote:

Even if you believe/think this, it is not in your best interest to
communicate anything like this, there is nothing you can win, and
significant downside potential.


As you can probably tell, I am not terribly politically correct :-). The 
coddle culture we live in today only ends up creating a generation that 
will not be competitive in the open market, because whether we like it 
or not, only competence gets compensated.


You really can't force people to give you their money for a poor, 
expensive job done, much as we may believe it should be the case. 8th 
place trophies should never be a model.

I believe the question is not about what data says, the question is,
why does the data say that. And the thesis/belief is, data should not
say that, that there is no fundamental reason why the data would say
so. The question is, is the culture reinforcing this from day0,
causing people to believe it is somehow inherent/natural.


The reason the data should not say that is because there has been a 
serious amount of investment in creating scientists, engineers, 
mathematicians, technologists, CEO's and inventors from the female 
community over the past couple of decades. And yet, all the metrics 
still show that women are "under-represented" in these areas.


So the explanation ends up going back to "upbringing, cultural 
socialization, oppression by men, e.t.c.". After all, if a woman has the 
opportunity to do male-dominated jobs or study male-dominated subjects, 
why wouldn't she, for whatever reason that may or may not be useful to 
her own person? After all, nothing screams equality like doing exactly 
what men can do, or what women can do.


In other words, the idea that women (and little girls) may not have any 
personal interest in things that men are inherently interested in is 
completely inconceivable.

>From scientific POV, we currently don't have any real reason to
believe there are unplastic differences in the brain from birth which
cause this. There might, but science doesn't know that. Scientifically
we should today expect very even distribution, unless culturally
biased.


When we refuse to believe that, in general, men prefer building things 
and women prefer dealing with people, we are essentially trying to fix 
innately biological differences with culture. And while culture, on 
paper, sounds and feels good because it either elicits emotion (instead 
of logic) or results in censorship (instead of discourse), more often 
than not, biology always wins out. It's a bit like weight loss... you 
may starve yourself to lose excess body fat, but eventually, hunger 
always wins. So you need another strategy, one which maximizes weight 
loss, but without leaving your ravenous.


The sad part is that by the time biology takes over, it is too late for 
the individual to benefit from a different decision they could have 
taken, earlier on in life. And what is worse for the next generation, is 
that those poor outcomes that afflict the individuals in their mid-40's 
or later, are never communicated down to the kids... because if that 
happens, the simulation that culture trumps biology would inevitably 
crumble. And that's just bad for business...



But of course inequality, inequitability is everywhere, not an
hyperbole, but you can't compare anything on how we choose who does
what and come up with anything that resembles fair distribution. Zip
code has a lot of predictive power where you'll end up in your life,
and that is hardly your fault or merit. Top level managers are not
just disproportionately men, but they are disproportionately men with
+1.5SD height, and there is no scientific reason to believe zip code
or height suggests stronger ability.

It is just a really unfair world to live in, but luckily I am on the
beneficiary side of the unfairness, which I am strong enough to
accept.


Well, the problem comes with how we define "fairness". If we say 
fairness is both men and women should have access to the same 
opportunities, that is fine. But if we say that fairness is both men and 
women should have the same outcomes, that becomes problematic. If equal 
outcome worked, half of all CEO's would be women, as much as half of all 
coal miners would be women. It's not like there is a shortage of 
corporations or mining companies looking to fill half of their staff 
with women... and yet they do not. Instead of looking deeply into why 
that is, popular culture will simply chalk it down to anything other 
than "opportunity" and "personal interest", e.g., being passed over, 
sexism, racism, pay gap, e.t.c.


Moreover, since the "pay gap" suggests that women will earn less than 
men in any field where both are employable, you'd think that those 
companies would be 90% female-dominated, as they would be a lower 
cost-to-company. But again, that is not happening... why? It certainly 
can't be the combination of personal interest + meritocracy, can it :-)?

I have 

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 21:02, Richard McGovern via juniper-nsp wrote:


I tried to get my daughter (now Sr at Uni) to look at this field. Her response 
was, “I don’t want to do anything like what you do” 


At the risk of derailing this thread, one item that is generally 
programmed into an agenda of your favourite NOG conference is "Women In 
Tech". For over 10 years, the agenda has always been the same... why 
aren't there more women in our field, what can we do about it, and what 
is the experience of those few women that are in the field?


After 10 years, I finally asked the question at a recent meeting earlier 
this year - could it be that we may be ignoring and/or underestimating 
the value and power of personal choice and interest?


While there are some women who enjoy engineering, and some men who enjoy 
nursing, most women don't enjoy engineering, and most men don't enjoy 
nursing. I think we would move much farther ahead if we accepted this, 
and stopped trying to force-feed people stuff that does not interest 
them. That way, we give a real chance to those that are interested and 
make sure we don't risk people's future by placing them in subjects and 
fields where they will not be able to compete fairly with those who 
really want to be there.


If you look at the data, on average, 70% of new enrollments at 
university are women, and 60% of all graduands are women. And yet, 90% 
of all STEM students are men, while 80% of all psychology students are 
women. Perhaps there is a clue in there :-)...


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 16:00, Gert Doering wrote:


What is "high-touch edge" for you?

Most things we could come up with do work, with the notable exception
of MAC accounting (or inclusion of MAC addresses in sflow/ipfix) - but
here the ASR9000 is one of the few platforms on the market that can
actually do it.  So more of a chip limitation.

We don't do extremely demanding QoS things, but basic shaping and policing
works fine on the new J2c+ boxes.  So not sure where the limits are.

Of course it's merchant silicon boxes, but the J2c chips have become
really impressive.


Good to hear.

The main things that have typically been an issue for us on 
Broadcom-based boxes have been:


    - Egress policing (trTCM).
    - uRPF (IPv4/IPv6).
    - Ingress/Egress marking (Policy Map a la Junos).
    - EVC + VLAN Tag Rewrite (push, pop, swap).
    - IPv4/IPv6 interface ACL's (should be pretty doable now).
    - LDPv6 (not chip related, just code).

Has your experience on Arista been that all those work?

We have a ton of Arista hardware, but we just use it purely for Layer 2 
switching.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 15:36, Gert Doering via juniper-nsp wrote:


There goes another vendor...

Now, if the base price would have been *lowered* by the amount the
L3 features of a *MX router* cost extra now, this might have been an
option... but for my understanding, the base MX304 is already insanely
pricey, and then add licenses on top...  nah, taking our money elsewhere.


Based on quotes we've seen so far, 75% of the list price of an MX304 is 
just the license for the entire chassis (separate from the line cards 
and without support).



Did I mention Arista is not spending valuable engineer time on all this
license shit, but on actually making great products?


What is the current experience of the code for IP/MPLS functions that go 
beyond simply peering and transit, i.e., high-touch edge?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 14:42, Saku Ytti via juniper-nsp wrote:


But we can reject licenses that expire in operation and cause an
outage. That I think is a very reasonable ask.  I know that IOS XE for
example will do this, you run out of license and your box breaks. I
swapped out from CRS1k to ASR1k because I knew the organisation would
eventually fail to fix the license ahead of expiry.


We had this happen to us in 2014 when we recovered a failed server 
running CSR1000v. The "installation evaluation" license expired after 60 
days, and since everyone forgot about it, the box went down.


So as part of our deployment/recovery procedure, we always procure 
CSR1000v licenses for each installation.


Of course, with things changing to Cat8000v, this could get juicy.



I'm happy if the device calls homes via https proxy, and reports my
license use, and the sales droid tells me I'm not compliant with
terms. Making it a commercial problem is fine, making it an acute
technical problem is not.


In your specific case, the ports never worked, you had to procure a
license, and the license never dies. So from my POV, this is fine. And
being absolutist here will not help, as then you can't even achieve
reasonable compromise.


I tend to agree.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 10:57, Sebastian Wiesinger via juniper-nsp wrote:

Yeah it depends. Our MX204 also needed licenses for subscriber
managment. Some options would produce a license warning and some other
stuff just failed silently which was worse. Also noone at Juniper
seemed to know WHICH licenses we needed for our usecase.

In the end our license list looked like this:

subscriber-accounting
subscriber-authentication
subscriber-address-assignment
subscriber-vlan
subscriber-ip
scale-subscriber
scale-l2tp
l2tp-inline-lns

So yeah.. that wasn't a nice experience at all.


Subscriber Management has always required real licenses on the MX since 
it started shipping BNG code.


You got 1,000 subscribers as standard, and then needed an enforceable 
license after that.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/25/23 08:01, Saku Ytti via juniper-nsp wrote:


Juniper had assured me multiple times that they strategically have
decided to NEVER do this. That it's an actual decision they've
considered at the highest level, that they will not downgrade devices
in operation. I guess 'reboot' is not in-operation?

Notion that operators are able to keep licenses up-to-date and valid
is naive, we can't keep SSL certificates valid and we've had decades
of time to learn, it won't happen. You will learn about the problem,
when shit breaks.

The right solution would be a phone-home, and a vendor sales rep
calling you 'hey you have expired licenses, let's solve this'. Not
breaking the boxes. Or 'your phone home hasn't worked, you need to fix
it before we can re-up your support contract'.


I spoke to my SE about this today, and he checked with the PLM. While a 
license can be purchased to quiet the logs, it is not necessary for 
routing protocols to work.


If you buy your MX304 from Juniper, you will pay for a license anyway.

He thinks the issue the OP experienced where routing protocols did not 
work after the reboot is an unrelated issue to licenses.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory - Resolved

[Mark Tinka via juniper-nsp]

On 10/19/23 06:48, Chris Kawchuk wrote:

Indeed. Apologies to all --- I too got an update from JNPR on the "show route" vs 
"show routing" CLI conflict a few weeks ago in early September -- but forgot to share it 
here.


CASE;
2023-0719-733950

Synopsis:
"show route" and "show routing" operational mode CLI conflict - JunOS 21+

Root Cause:
"show routing" CLI stanza was unhidden to implement "show routing 
transport-class" command for BGP CT (in JunOS 21+)

Resolution:
For finger memory convenience, hiding 'show routing' stanza again, and moving 
BGP CT command to under 'show route' stanza.
Understanding is: "show route" will now contain both:
 - route table related information, AND
 - routing daemon related information.


-

Looks like it's a win for the little guy!.. or at least... my little finger 
that wants to smash that [TAB] key early


Indeed!

We can be pleased.

Well done to all :-).

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/18/23 19:05, Chris Wopat via juniper-nsp wrote:


Only complaint is Junos related, with auto tab complete problems as
extensively discussed in a different thread.


I have an update on that...

Our request was granted, and Juniper are initially targeting to fix this 
in Junos 24.1. However, there are ongoing discussions to introduce this 
into 23.3R2.


So we may soon see the back of this.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/18/23 18:55, Tom Beecher wrote:

Juniper licensing is honor based. Won't impact functionality, will 
just grump at you on commits.


Okay, so usual licensing enforcement.

Just curious why warnings about OSPF and LDP... this is what a router is 
exactly for.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 - Edge Router

[Mark Tinka via juniper-nsp]

On 10/18/23 15:47, Aaron1 via juniper-nsp wrote:


Also, I get a license warning when committing OSPF and LDP.  We got a license 
from our account team and now we don’t see that message anymore


Any details on what this license does, and if there is any operational 
risk if you don't apply it?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] BGP Extended Communities

[Mark Tinka via juniper-nsp]

On 9/10/23 20:50, Mohammad Khalil via juniper-nsp wrote:


Greetings
Hope all is well.

I need to check if Juniper's BGP extended community settings are compatible
with Cisco's BGP extended community settings.

Is it possible to intercommunicate Juniper's BGP extended community with
Cisco BGP extended community ?


We use them for l3vpn VRF's between both vendors. Works standard.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] P2MP traffic loss over QFX and EX series

[Mark Tinka via juniper-nsp]

On 9/5/23 17:05, Gonzalo Caceres via juniper-nsp wrote:


Hi Mark, thanks for the quick reply!

It is a problem that we have been having for some time, and we have not
received concrete answers from our SE. That's why I checked here, as I
haven't seen colleagues expressing this flaw in other posts either. Thanks
again.


I suspect it could be an issue related to Broadcom, which I understand 
Juniper may have been able to workaround through code (with some 
limitations on performance). But your SE will be your best bet.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] P2MP traffic loss over QFX and EX series

[Mark Tinka via juniper-nsp]

On 9/3/23 01:07, Gonzalo Caceres via juniper-nsp wrote:

Hi everyone,

I have multicast traffic through a P2MP LSP and when this traffic goes
through an EX or QFX series switch (spot EX4600 and QFX5110/5120) it is
lost. We use these switches as multilayer, they have MPLS suite protocols
(ISIS, LDP and RSVP).



*>show versionModel: ex4600-40fJunos: 21.4R3.15 (recommended version at
day)*

The switch see the transit LSP from IP 2.2.2.2 to 1.1.1.1:




*> show mpls lspTransit LSP: 2 sessionsTo  FromState Rt Style
Labelin Labelout LSPname2.2.2.2 1.1.1.1 Up0  1 SE 62
  lsp1Total 2 displayed, Up 2, Down 0*

Traffic from 1.1.1.1 come's via ae0:


*> show route 1.1.1.11.1.1.1/32  *[IS-IS/18] via
ae0.01.1.1.1/32  *[LDP/9] via ae0.0, Push 315471*

   Traffic to 2.2.2.2 goes via ae3:


*> show route 2.2.2.22.2.2.2/32  *[IS-IS/18] 20:36:26,
via ae3.02.2.2.2/32  *[LDP/9] 20:36:25, via ae3.0*

The P2MP traffic see on input interface but they not replicate at the
output:








*Interface: ae0, Enabled, Link is UpInput bytes: 7282719382422 (785608168
bps) [207824323]Input packets: 5263502719 (70972 pps) [223103]Interface:
ae3, Enabled, Link is Up  Output bytes: 186620844 (26760 bps) [7545]
Output packets: 1225967 (13 pps) [47]*
Has anyone ever had a similar problem? or are you certain that this type of
LSP will NOT work on these switches?


I remember several late-night calls with Juniper, nearly a decade ago, 
when we were evaluating their Broadcom-based platforms for deployments 
where NG-MVPN had been anticipated, both via p2mp RSVP-TE as well as mLDP.


They were not comfortable committing to supporting NG-MVPN on their 
Broadcom boxes then. I'm not sure how far that has come in 2023, but 
probably worth reaching out to your SE and asking about whether what you 
are trying to do is supported on the Broadcom chips your boxes are running.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/19/23 06:22, Mark Tinka wrote:


I'm trying with my SE too. Let's stay in touch.


Got a ticket opened with JTAC which my SE upstreamed, and it has been 
linked to an internal PR that was raised on the back of Chris' ticket.


So all we can do now is wait.

Thanks, all.

Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/19/23 02:37, Chris Kawchuk wrote:


Hi Jeff

I'll open it with my SE here in Australia (Mark Barrett). Will advise once 
complete.


I'm trying with my SE too. Let's stay in touch.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/15/23 20:54, Crist Clark wrote:

I find it kind of telling that customers are just getting around to 
complaining about it two years after it was released. Not at all 
surprising, but illustrates how slow network operators update cycles 
can be compared to the pace of development.


To the Junos developers, this is ancient news, long forgotten in the 
dozens of sprints and multiple releases since.


While I do take your point, to be fair, this is one of those things 
operators are not likely to complain about because it would be coming 
from a place of priviledge. There are far more pressing issues than 
this, to be honest.


So like us who have been running Junos 21 for the past 8 or so months, 
we've been struggling with this and have been meaning to bring it up. 
But it's not a major catastrophe, so we haven't prioritized - and this 
post is really tongue-in-cheek. If it gets looked at, great. If it 
doesn't, it's not the end of the world.


If you run a network of significant size, you are going to be in a 
constant cycle of code upgrades. So it's not unreasonable to find 
operators being behind.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/12/23 15:45, Jeff Haas wrote:


You don't need to tell my fingers that. __

With the infrastructure as it is, the only "solution" is we stop adding things. 
 Good luck with that.

The general here is the explosion of keywords.  I have about 15 features 
sitting in my backlog that are small things to do to bgp policy.  The policy 
stanza is already a mess.

... and that's not including the work to let users match on flowspec filter 
components.

The CLI could be taught to not include certain auto-completions as a 
user-profile, locally, with hints from TACACS, etc... but it means we get into 
an inconsistent user experience.

Feel free to spend some collective thinking time on what a "this would suck 
less" solution would look like.  I suspect that the competing opinions on what to do 
will eventually involve a cage fight match.


Will any of these issues register significantly on Juniper's roadmap of 
how to make customers happier? Likely unlikely.


Will it stop customers from buying Juniper gear. Even more likely unlikely.

Does it hurt to ask? Nope.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/12/23 15:38, Chris Wopat wrote:

Another offender in 21. `protocols bgp` doesn't autocomplete as it did 
since `bgpmcast` was added.


me@r-mx304-lab-re1# set protocols bgp?
Possible completions:
> bgp                  BGP options
> bgpmcast             BGP multicast options


Yes, that has been painful too since Junos 21. But in the hierarchy of 
annoying changes, the one to "route" is higher.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/12/23 15:08, Vladimir Blazhkun wrote:


You can probably deny that command using the login class ACL.


As I'd mentioned to someone else already, not looking to create custom 
hacks that would not survive staff movement.


While it is an option, it would be one of extreme last resort.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

On 7/12/23 11:12, Chris Kawchuk wrote:


+1 Mark!


For transparency, it was Chris who gave me the nudge, so thanks for 
that, mate :-).




As any good problem begs for a solution, my suggestions to JNPR are as follows, 
as alternative places for this command:

"show route transport-class ..." (or really, is it even a routing thing? might 
be better w/the segment-routing or spring commands)i.e.:
"show segment-routing ..."
"show spring transport-class ."

...but whatever is done, please, for the love of 20+ years of finger-memory please do 
NOT make it conflict with "sh[TAB] rou[TAB]"


The other option that came to mind is - they could make it part of the 
BGP or MPLS command set, as it is specific to those features.




- CK.

(P.S. NOTE that since this new 'show' command is part of base JunOS 21 and 
above; it's also present on all EX, QFX, SRX, etc.. devices you would never 
ever think of ever running segment-routing on w/transport classes or colours, 
but that *do* have a basic routing table, so it's not just on PTX/ACX/EVO or 
MX/P/PE devices we run into this...)


The beauty of "One Junos to rule them all" :-).

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Junos 21+ Killing Finger Muscle Memory...

[Mark Tinka via juniper-nsp]

So, this is going to be a very priviledged post, and I have been 
spending the last several months mulling over even complaining about it 
either on here, or with my SE.


But a community friend sent me the exact same annoyance he is having 
with Junos 21 or later, which has given me a final reason to just go 
ahead and moan about it:


tinka@router> show rout
 ^
'rout' is ambiguous.
Possible completions:
  route    Show routing table information
  routing  Show routing information
{master}
tinka@router>

I'm going to send this to my Juniper SE and AM. Not sure what they'll 
make of it, as it is a rather priviledged complaint - but in truth, it 
does make working with Junos on a fairly historically commonly used 
command rather cumbersome, and annoying.


The smile that comes to my face when I touch a box running Junos 20 or 
earlier and run this specific command, is unconscionably satisfying :-).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 7/4/23 09:11, Saku Ytti wrote:


You must have misunderstood. When they fully scale the current design,
the design offers 100T capacity, but they've bought 400T of ports. 3/4
ports are overhead to build the design, to connect the pizzaboxes
together. All ports are used, but only 1/4 are revenue.


Thanks, makes sense.

This is one of the reasons I prefer to use Ethernet switches to 
interconnect devices in large data centre deployments.


Connecting stuff directly into the core routers or directly together 
eats up a bunch of ports, without necessarily using all the available 
capacity.


But to be fair, at the scale AWS run, I'm not exactly sure how I'd do 
things.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 7/2/23 18:04, Saku Ytti wrote:


Not disagreeing here, but how do we define oversubscribed here? Are
all boxes oversubscribed which can't do a) 100% at max size packet and
b) 100% at min size packet and c) 100% of packets to delay buffer, I
think this would be quite reasonable definition, but as far as I know,
no current device of non-modest scale would satisfy each 3, almost all
of them would only satisfy a).


Well, the typical operator will use "oversubscribed" in the context of 
number of ports vs. chip capacity. However, it is not unwise to consider 
packet handling as a function of oversubscription too.




Let's consider first gen trio serdes
1) 2/4 goes to fabric (btree replication)
2) 1/4 goes to delay buffer
3) 1/4 goes to WAN port
(and actually like 0.2 additionally goes to lookup engine)

So you're selling less than 1/4th of the serdes you ship, more than
3/4 are 'overhead'. Compared to say Silicon1, which is partially
buffered, they're selling almost 1/2 of the serdes they ship. You
could in theory put ports on all of these serdes in BPS terms, but not
in PPS terms at least not with off-chip memory.


To be fair, although Silicon One is Cisco's first iteration of the chip, 
it's not fair to compare it to Trio 1 :-).


But I take your point.



And in each case, in a pizza box case, you could sell those fabric
ports, as there is no fabric. So given NPU has always ~2x the bps in
pizza box format (but usually no more pps). And in MX80/MX104 Juniper
did just this, they sell 80G WAN ports, when in linecard mode it only
is 40G WAN port device. I don't consider it oversubscribed, even
though the minimum packet size went up, because the lookup capacity
didn't increase.


Makes sense, but what that means is that you are more concerned with pps 
while someone else could be more concerned with bps. I guess it depends 
on if your operation is more pps-heavy, while someone else's is more 
bps-heavy with average packet size.




Curiously AMZN told Nanog their ratio, when design is fully scaled to
100T is 1/4, 400T bought ports, 100T useful ports. Unclear how long
100T was going to scale, but obviously they wouldn't launch
architecture which needs to be redone next year, so when they decided
100T cap for the scale, they didn't have 100T need yet. This design
was with 112Gx128 chips, and boxes were single chip, so all serdes
connect ports, no fabrics, i.e. true pizzabox.
I found this very interesting, because the 100T design was, I think 3
racks? And last year 50T asics shipped, next year we'd likely get 100T
asics (224Gx512? or 112Gx1024?). So even hyperscalers are growing
slower than silicon, and can basically put their dc-in-a-chip, greatly
reducing cost (both CAPEX and OPEX) as no need for wasting 3/4th of
the investment on overhead.


Yes, I watched this NANOG session and was also quite surprised when they 
mentioned that they only plan for 25% usage of the deployed capacity. 
Are they giving themselves room to peak before they move to another chip 
(considering that they are likely in a never-ending installation/upgrade 
cycle), or trying to maintain line-rate across a vast number of packet 
sizes? Or both?




The scale also surprised me, even though perhaps it should not have,
they quoted +1M network devices, considering they quote +20M nitro
system shipped, that's like <20 revenue generating compute per network
device. Depending on the refresh cycle, this means amazon is buying
15-30k network devices per month, which I expect is significantly more
than cisco+juniper+nokia ship combined to SP infra, so no wonder SPs
get little love.


Well, the no-love to service providers has been going on for some time 
now. It largely started with the optical vendors, around 2015 when 
coherent gave us 400Gbps waves over medium-haul distances, and the 
content folk began deploying DCI's. Around the same time, submarine 
systems began deploying uncompensated cables, and with most of them 
being funded by the content folk, optical vendors focused 90% of their 
attention that way, ignoring the service providers.


The content folk are largely IETF people, so they had options around 
what they could do to optimize routing and switching (including building 
their own gear). But I see that there is some interest in what they can 
do with chips from Cisco, Juniper and Nokia if they have arrangements 
where those are opened up to them for self development; not to mention 
Broadcom, which means we - as network operators - are likely to see even 
far less love from routing/switching vendors, going forward.


But with AWS deploying that many nodes, even with tooling, it must be a 
mission staying on top of software (and hardware) upgrades.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 7/2/23 15:19, Saku Ytti wrote:


Right as is MX304.

I don't think this is 'my definition', everything was centralised
originally, until Cisco7500 came out, which then had distributed
forwarding capabilities.

Now does centralisation truly mean BOM benefit to vendors? Probably
not, but it may allow to address one lower margin market which as
lower per-port performance needs, without cannibilising larger margin
market.


Technically, do we not think that an oversubscribed Juniper box with a 
single Trio 6 chip with no fabric is feasible? And is it not being built 
because Juniper don't want to cannibalize their other distributed 
compact boxes?


The MX204, for example, is a single Trio 3 chip that is oversubscribed 
by an extra 240Gbps. So we know they can do it. The issue with the MX204 
is that most customers will run out of ports before they run out of 
bandwidth.


I don't think it's that vendors using Broadcom to oversubscribe a 
high-capacity chip is the issue. It's that other vendors with in-house 
silicon won't do the same with their own silicon.



Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/28/23 09:29, Saku Ytti via juniper-nsp wrote:


This of course makes it more redundant than distributed box, because
distributed boxes don't have NPU redundancy.


Well, by your definition, the ASR9903, for example, is a distributed 
platform, which has a fabric ASIC via the RP, with 4x NPU's on the fixed 
line card, 2x NPU's on the 800Gbps PEC and 4x NPU's on the 2Tbps PEC.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 7/2/23 11:18, Saku Ytti wrote:


In this context, these are all distributed platforms, they have
multiple NPUs and fabric. Centralised has a single forwarding chip,
and significantly more ports than bandwidth.


So to clarify your definition of "centralized", even if there is no 
replaceable fabric, and the line cards communicate via a fixed fabric 
ASIC, you'd still define that as a distributed platform?


By your definition, you are speaking about fixed form factor platforms 
with neither a replaceable fabric nor fabric ASIC, like the MX204, 
ASR920, ACX7024, 7520-IXR, e.t.c.?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 7/2/23 10:42, Saku Ytti wrote:


Yes. Satellite is basically VLAN aggregation, but a little bit less
broken. Both are much inferior to MPLS.


I agree that using vendor satellites solves this problem. The issue, 
IIRC, is was what happens when you need to have the satellites in rings?


Satellites work well when fibre is not an issue, and each satellite can 
hang off the PE router like a spur. But if you need to build rings in 
order to cover as many areas as possible at a reasonable cost, 
satellites seemed to struggled to have scalable ring topologies. This 
could have changed over time, not sure. I stopped tracking satellite 
technologies around 2010.




  But usually that's not the
comparison due to real or perceived cost reasons. So in absence of a
vendor selling you the front-plate you need, option space often
considered is satellite or vlan aggregation, instead of connecting
some smaller MPLS edge boxes to bigger aggregation MPLS boxes, which
would be, in my opinion, obviously better.


The cost you pay for a small Metro-E router optimized for ring 
deployments is more than paid back in the operational simplicity that 
comes with MPLS-based rings. Having ran such architectures for close to 
15 years now (since the Cisco ME3600X/3800X), I can tell you how much 
easier it has been for us to scale and keep customers because we did not 
have to run Layer 2 rings like our competitors did.




But as discussed, centralised chassis boxes are appearing as a new
option to the option space.


Well, for data centre aggregation, especially for 100Gbps transit ports 
to customers, centralized routers make sense (MX304, MX10003, ASR9903, 
e.t.c.). But those boxes don't make sense as Metro-E routers... they can 
aggregate Metro-E routers, but can't be Metro-E routers due to their cost.


I think there is still a use-case for distributed boxes like the MX480 
and MX960, for cases where you have to aggregate plenty of 1Gbps and 
10Gbps customers. Those line cards, especially the ones that are now 
EoS/EoL, are extremely cheap and more than capable of supporting 1Gbps 
and 10Gbps services in the data centre. At the moment, with modern 
centralized routers optimized for 100Gbps and 400Gbps, using them to 
aggregate 10Gbps services or lower maybe be costlier than, say, an MX480 
or MX960 with MPC2E or MPC7E line cards attached to a dense Ethernet 
switch via 802.1Q.


For the moment, the Metro-E router that makes the most sense to us is 
the ACX7024. Despite its Broadcom base, we seem to have found a way to 
make it work for us, and replace the ASR920.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/28/23 08:44, Saku Ytti wrote:


Apart from obvious stuff like QoS getting difficult, not full feature
parity with VLAN and main interface, or counters becoming less useful
as many are port level so identifying true source port may not be
easy. There are things that you'll just discover over time that don't
even come to your mind, and I don't know what those will be in your
deployment. I can give anecdotes

2*VXR termination of metro L2 ring
 - everything is 'ok'
 - ethernet pseudowire service is introduced to customers
 - occasionally there are loops now
 - well VXR goes to promisc mode when you add ethernet pseudowire,
because while it has VLAN local significancy, it doesn't have per-vlan
MAC filter.
 - now unrelated L3 VLAN, which is redundantly terminated to both
VXR has customer CE down in the L2 metro
 - because ARP timeout is 4h, and MAC timeout is 300s, the the
metro will forget the MAC fast, L3 slowly
 - so primary PE gets packet off of internet, sends to metro, metro
floods to all ports, including secondary PE
 - secondary PE sends packet to primary PE, over WAN
 - now you learned 'oh yeah, i should have ensured there is
per-vlan mac filter' and 'oh yeah, my MAC/ARP timeouts are
misconfigured'
 - but these are probably not the examples you'll learn, they'll be
something different
 - when you do satellite, you can solve lot of the problem scope by
software as you control L2 and L3, and can do proprietary code

L2 transparency
 - You do QinQ in L2 aggregation, to pass customer frame to
aggregation termination
 - You do MAC rewrite in/out of the L2 aggregation (customer MAC
addresses get rewritten coming in from customer, and mangled back to
legitimate MAC going out to termination). You need this to pass STP
and such in pseudowires from customer to termination
 - In termination hardware physically doesn't consider VLAN+ISIS
legitimate packet and will kill it, so you have no way of supporting
ISIS inside pseudowire when you have L2 aggregation to customer.
Technically it's not valid, technically ISIS isn't EthernetII, and
802.3 doesn't have VLANs. But technically correct rarely reduces the
red hue in customers faces when they inform about issues they are
experiencing.
 - even if this works, there are plenty of other ways pseudowire
transparency suffers with L2 aggregation, as you are experiencing set
of limitations from two box, instead of one box when it comes to
transparency, and these sets wont be identical
 - you will introduce MAC limit to your point-to-point martini
product, which didn't previously exist. Because your L2 ring is
redundant and you need mac learning. If it's just single switch, you
can turn off MAC learning per VLAN, and be closer to satellite
solution

Convergence
 - your termination no longer observes hardware liveness detection,
so you need some solution to transfer L2 port state to VLAN. Which
will occasionally break, as it's new complexity.


So all the above sounds to me like scenarios where Metro-E rings are 
built on 802.1Q/Q-in-Q/REP/STP/e.t.c., rather than IP/MPLS.


We run fairly large Metro-E rings, but we run them as IP/MPLS rings, and 
all the issues you describe above are the reasons we pushed the vendors 
(Cisco in particular) to provide boxes that were optimized for the 
Metro-E applications, but had proper IP/MPLS support. In other words, 
these are largely solved problems.


I think many - if not all - of the issues you raise above can be fixed 
by, say, a Cisco ASR920 deployed at scale in the Metro, running IP/MPLS 
for the backbone, end-to-end.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/27/23 19:44, Gert Doering wrote:


The issues we see / have with "satellites that are not real satellites"
are

  - l2 link down -> l3 not going down
  - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic
down to a 100M customer port, saturating the "vlan on trunk" link
for no good

(we run config-management-system managed EX3400s off an Arista MLAG pair,
so the benefits outweigh the drawbacks for us)


Same here.

I think this is the architecture most operators run, especially because 
despite centralized routers being less disjointed, they are still 
pricier than attaching a switch to a router port via an 802.1Q trunk. If 
you are aggregating plenty of 1Gbps or 10Gbps ports that aren't each 
carrying lots of traffic, centralized ports will be more expensive than 
a traditional switch<=>router architecture compared to even the cheapest 
and most dense centralized router.


I think centralized routers make a lot of sense when aggregating 100Gbps 
services, because doing these on a switch<=>router architecture is going 
to be tough when you try to aggregate N x 100Gbps links, or even a 
handful of 400Gbps links for the 802.1Q trunk, as most of those 
customers will be riding their 100Gbps port really hard, i.e., there is 
enough distance between 10Gbps and 100Gbps, while there really isn't any 
between 100Gbps and 400Gbps.


One of the biggest risks I find with a standard switch<=>router 
architecture is the outage that could happen if that trunk goes down. 
Yes, you could mitigate that by making the trunk a LAG of multiple 
links, but the challenge that creates is how to do policing on the 
router sub-interface because the policer is split across the member 
links in the LAG. On the other hand, one could get around this 
commercially by letting customers know the SLA associated with being 
connected to only "one device", and provide the option of connecting to 
"a second device".


The other issue I find with the switch<=>router architecture is where to 
do policing. With LAG's, shaping on the switch ports makes sense because 
you don't run into having to police across member links in a LAG on the 
router port. Most aggregation switches do not support egress policing 
with Broadcom chips, so shaping is your only tool. It has worked well 
enough for us on the Arista switches that we have deployed for this, but 
was not a reasonable option when we used to run the EX4550 that only had 
4MB of packet buffers.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/27/23 18:45, Tarko Tikan via juniper-nsp wrote:

Previously mentioned centralized boxes are actually becoming more and 
more common now (in addition to non-redundant pizzabox formfactor that 
has been available for ages) that single NPU can do 2+ Tbps. For BCM 
J2 see ACX7509, Nokia IXR-R6d or certain Cisco NCS models. All pretty 
good fits for SP aggregation networks.


Single NPU doesn't mean non-redundant - those devices run two (or 4 in 
ACX case) BCM NPUs and switch "linecards" over to backup NPU when 
required. All without true fabric and distributed NPUs to keep the 
cost down.


IMHO these are very compelling options for SP market, you get 
selection of linecards for different applications/architectures yet 
the overall size of the device (and power consumption) is small. I've 
been trying to explain to one of the major vendors that they should 
build similar device with their own silicon so you get similar 
benefits while keeping the rich featureset that comes with vendor 
silicon compared to BCM.


Ah, okay - I'm with you now.

I had a totally different definition in my head for what "centralized" 
meant.


Yes, agreed - these make sense because of just how cheap, yet fast, 
Broadcom chips are.


Like you, I've been pushing our friendly vendors to build similar 
architectures with their in-house silicon, and like you, it has fallen 
on deaf ears.


Because IP traffic is becoming more and more public, the need for 
features that drove end-to-end on-net VPN's in silicon will continue to 
decline. If that helps operators to simplify their product technical 
configuration to where Broadcom can handle 90% of all scenarios, giving 
up the other 10% may be worth the capex/opex savings. It is that 10% 
that many operators want to keep, and the traditional vendors are 
locking up that 10% in their own silicon to print money.


With every passing year, Broadcom adds to the things I want that it 
could not do. I am close to the 90% for some of these platforms, to 
where I can consider them now.


Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/27/23 17:07, Saku Ytti wrote:



Yes.


How?



Like cat6500/7600 linecards without DFC, so SP gear with centralised
logic, and dumb 'low performance' linecards. Given low performance
these days is multi Tbps chips.


While I'm not sure operators want that, they will take a look if the 
lower price does not impact performance.


There is more to just raw speed.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/27/23 09:02, Saku Ytti wrote:


Juniper messaging seems to be geo-specific, in EU their sales seems to
sell them more willingly than in US. My understanding is that
basically fusion is dead, but they don't actually have solution for
access/SP market front-plate, so some sales channels are still
pitching it as the solution.


Would that be high-density face-plate solutions for access aggregation 
in the data centre, that they are struggling with?


I haven't used their EX platform since the EX4600 broke things, and I 
have never tried their QFX platform. So not sure if Juniper have any 
decent switch offerings for large scale data centre aggregation in 1U 
form factors, that customers are actually happy with.




Nokia seems very committed to it.

I think the solution space is
a) centralised lookup engines - so you have cheap(er) line cards
for high density low pps/bps
b) satellite
c) vlan aggregation

Satellite is basically a specific scenario of c), but it does bring
significant derisking compared to vlan aggregation, as a single
instance is designing it and can solve some problems better than can
be solved by vendor agnostic vlan aggregation. Vlan aggregation looks
very simple on the surface but is fraught with problems, many of which
are slightly better solved in satellites, and these problems will not
be identified ahead of time but during the next two decades of
operation.


Are you suggesting standard 802.1Q/Q-in-Q trunking from a switch into a 
"pricey" router line card that support locally-significant VLAN's per 
port is problematic?




Centralised boxes haven't been available for quite a few years, but
hopefully Cisco is changing that, I think it's the right compromise
for SPs.


I'm still a bit unclear on what you mean by "centralized"... in the 
context of satellite, or standalone?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/26/23 20:56, Jackson, William via juniper-nsp wrote:


Similar use case here but we use a QFX as a fusion satellite if port expansion 
is required.
Works well as an small site start up option.


Are vendors still pushing their satellite switches :-)?

That technology looked dodgy to me when Cisco first proposed it with 
9000v, and then Juniper and Nokia followed with their own implementations.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

Junos 22.4R2 that fixes this issue has been released...

Mark.

On 6/10/23 13:55, Mark Tinka wrote:



On 6/10/23 13:50, Jason Lixfeld wrote:


Do either of you two have PRs for your respective issues?  If you could share, 
I, for one anyway, would be grateful :)


For the PTX1000 issue:

https://supportportal.juniper.net/s/article/PTX1000-resources-exhaustion-causing-host-loopback-wedge
https://prsearch.juniper.net/problemreport/PR1695183

Mark.


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/10/23 13:50, Jason Lixfeld wrote:


Do either of you two have PRs for your respective issues?  If you could share, 
I, for one anyway, would be grateful :)


For the PTX1000 issue:

https://supportportal.juniper.net/s/article/PTX1000-resources-exhaustion-causing-host-loopback-wedge
https://prsearch.juniper.net/problemreport/PR1695183

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/9/23 17:46, Andrey Kostin via juniper-nsp wrote:

 We have two MX204s running in pair with 2x100G taken for links 
between them and remaining BW is 6x100G for actual forwarding in/out. 
In this case it's kind of at the same level for price/100G value.


Yeah, using the MX204 like this (edge router functions) is costly on the 
ports it's already lacking.





I agree, and that's why I asked about HQoS experience, just to add 
more inexpensive low-speed switch ports via trunk but still be able to 
treat them more like separate ports from a router perspective.


The MX204 is an MPC7E, so whatever H-QoS is on the MPC7E is what the 
MX204 will also do.


We have used them as an edge router on a temporary basis at new sites, 
with an Arista switch hanging off of them via an 802.1Q trunk, until we 
can get our standard MX480 to site. They are capable for such a 
use-case. But usually, we use them for peering and value-added traffic.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/9/23 16:35, Saku Ytti wrote:


I'm not convinced at all that leaba is being sold. I think it's sold
conditionally when customers would otherwise be lost.


Probably - it's a "grain of salt" situation when you hear the news.

I don't think Meta and Microsoft have not bought zero of the C8000... 
but not to the degree that they would ignore more primary options, I think.




But NPU from newegg and community writes code that doesn't exist, and
I think it should and there would be volume in it, but no large volume
to any single customer.


Not enough foresight from traditional OEM's to see the potential here.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/9/23 16:12, Saku Ytti wrote:


I expect many people in this list have no need for more performance
than single Trio YT in any pop at all, yet they need ports. And they
are not adequately addressed by vendors. But they do need the deep
features of NPU.


This.

There is sufficient performance in Trio today (even a single Trio chip 
on the board) that people are willing to take an oversubscribed box or 
line card because in real life, they will run out of ports long before 
they run out of aggregate forwarding capacity.


The MX204, even though it's a pizza box, is a good example of how it 
could do with 8x 100Gbps ports, even though Trio on it will only forward 
400Gbps. Most use-cases will require another MX204 chassis, just for 
ports, before the existing one has hit anywhere close to capacity.


Really, folk are just chasing the Trio capability, otherwise they'd have 
long solved their port-count problems by choosing any Broadcom-based box 
on the market. Juniper know this, and they are using it against their 
customers, knowingly or otherwise. Cisco was good at this back in the 
day, over-subscribing line cards on their switches and routers. Juniper 
have always been a little more purist, but the market can't handle it 
because the rate of traffic growth is being out-paced by what a single 
Trio chip can do for a couple of ports, in the edge.




I keep hoping that someone is so disruptive that they take the
nvidia/gpu approach to npu. That is, you can buy Trio PCI from newegg
for 2 grand, and can program it as you wish. I think this market
remains unidentified and even adjusting to cannibalization would
increase market size.
I can't understand why JNPR is not trying this, they've lost for 20
years to inflation in valuation, what do they have to lose?


Well, the story is that Cisco are doing this with Meta and Microsoft on 
their C8000 platform, and apparently, doing billions of US$ in business 
on the back of that.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/9/23 15:57, Andrey Kostin wrote:


Hi Mark,

Not sure why it's eye-watering. The price of fully populated MX304 is 
basically the same as it's predecessor MX10003 but it provides 3.2T BW 
capacity vs 2.4T.


That's true, but the premium being paid for 400Gbps capability that some 
houses may not yet need is probably what is pushing that price up in 
comparison to the MX10003, which does not support 400Gbps.


But to be fair, it will come down to the discounts you can negotiate 
with Juniper. I'm perhaps more concerned because we got good pricing on 
the MX10003, even when we did a like-for-somewhat-like comparison with 
the MX304.


As much as we have struggled with Cisco in the past, this is forcing me 
to see what is available in their ASR99xx boxes. But off-the-bat, form 
factor and port density is poor on the Cisco side, compared to the MX304 
and MX10003.



If you compare with MX204, then MX304 is about 20% expensive for the 
same total BW, but MX204 doesn't have redundant RE and if you use it 
in redundant chassis configuration you will have to spend some BW on 
"fabric" links, effectively leveling the price if calculated for the 
same BW. I'm just comparing numbers, not considering any real 
topology, which is another can of worms. Most probably it's not worth 
to try to scale MX204s to more than a pair of devices, at least I 
wouldn't do it and consider it ;)


The use-case for MX204 and MX304 is very very different. As you say, 
MX304 is a better alternative for the MX10003 (which I am getting 
conflicting information about re: sale availability from Juniper).


We use the MX204 extensively, but only for peering and routing for 
value-added services too small to plug into a larger MX.



I'd rather call eye-watering prices for MPC7 and MPC10 to upgrade 
existing MX480 routers if you still to use their low-speed ports. Two 
MPC10s with SCB3s upgrade cost more than MX304, but gives 30% less BW 
capacity. For MPC7 this ratio is even worse.


Agreed - the MPC7 and MPC10's only make sense for large capacity 
aggregation or backbone links, not as an access port for 100Gbps 
customers. The MX10003 and MX304 are better boxes for 100Gbps access for 
customers.


Conversely, trying to use the MX304 or MX10003 as a core box is too 
costly, since you are paying the premium for edge features in Trio, when 
all you need is basic Ethernet, IS-IS and MPLS.


So the MPC7/MPC10 vs. MX304/10003 use-cases are clearly defined, if 
money is an object.



This brings a question, does anybody have an experience with HQoS on 
MX304? I mean just per-subinterface queueing on an interface to a 
switch, not BNG subscribers CoS which is probably another big topic. 
At least I'm not dare yet to try MX304 in BNG role, maybe later ;)


In this world where the kind of traffic you will be pushing through an 
MX304 most likely being majority off-net content, do you really need 
H-QoS :-)?


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/9/23 00:03, Litterick, Jeff (BIT) via juniper-nsp wrote:


The big issue we ran into is if you have redundant REs then there is a super 
bad bug that after 6 hours (1 of our 3 would lock up after reboot quickly and 
the other 2 would take a very long time) to 8 days will lock the entire chassis 
up solid where we had to pull the REs physical out to reboot them. It is 
fixed now, but they had to manually poke new firmware into the ASICs on each RE 
when they were in a half-powered state,  Was a very complex procedure with tech 
support and the MX304 engineering team.  It took about 3 hours to do all 3 
MX304s  one RE at a time.   We have not seen an update with this built-in yet.  
(We just did this back at the end of April)


Oh dear, that's pretty nasty. So did they say new units shipping today 
would come with the RE's already fixed?


We've been suffering a somewhat similar issue on the PTX1000, where a 
bug was introduced via regression in Junos 21.4, 22.1 and 22.2 that 
causes CPU queues to get filled up by unknown MAC address frames, and 
are not cleared. It takes 64 days for this packet accumulation to grow 
to a point where the queues get exhausted, causing a host loopback wedge.


You would see an error like this in the logs:

   alarmd[27630]: Alarm set: FPC id=150995048, 
color=RED, class=CHASSIS, reason=FPC 0 Major Errors
   fpc0 Performing action cmalarm for error 
/fpc/0/pfe/0/cm/0/Host_Loopback/0/HOST_LOOPBACK_MAKE_CMERROR_ID[1] 
(0x20002) in module: Host Loopback with scope: pfe category: functional 
level: major
   fpc0 Cmerror Op Set: Host Loopback: HOST 
LOOPBACK WEDGE DETECTED IN PATH ID 1  (URI: 
/fpc/0/pfe/0/cm/0/Host_Loopback/0/HOST_LOOPBACK_MAKE_CMERROR_ID[1])
Apr 1 03:52:28  PTX1000 fpc0 CMError: 
/fpc/0/pfe/0/cm/0/Host_Loopback/0/HOST_LOOPBACK_MAKE_CMERROR_ID[3] 
(0x20004), in module: Host Loopback with scope: pfe category: functional 
level: major


This causes the router to drop all control plane traffic, which, 
basically, makes it unusable. One has to reboot the box to get it back 
up and running, until it happens again 64 days later.


The issue is resolved in Junos 21.4R3-S4, 22.4R2, 23.2R1 and 23.3R1.

However, these releases are not shipping yet, so Juniper gave us a 
workaround SLAX script that automatically runs and clears the CPU queues 
before the 64 days are up.


We are currently running Junos 22.1R3.9 on this platform, and will move 
to 22.4R2 in a few weeks to permanently fix this.


Junos 20.2, 20.3 and 20.4 are not affected, nor is anything after 23.2R1.

I understand it may also affect the QFX and MX, but I don't have details 
on that.


Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/8/23 18:39, Giuliano C. Medalha wrote:

but you have the flex model. With license for capacity and features. 
 Advanced and Premium.


Which isn't a new thing with vendors. The prices are just terrible, even 
with discounts.




fib is better now - 12M

sampling rate for ipfix is better too.

but you have other parameters for mpls and bng too


Yes, like I said, increased Trio capacity aside, it's straight forward. 
Just not sure the price is justified. I expect a premium for custom 
silicon over Broadcom, but that seems excessive.




But you need the correct drivers on Junos


Yes, that is assumed, of course, especially if you want to talk to a 
ROADM. There is varying support amongst vendors, but as with everything, 
they will converge in time.



Juniper now has good prices ( common optics ) for 400G ( JCO part 
numbers )


Mixing $vendor_name and "good optics prices" has always ended in tears.



Low 40 km or maximum 80 km direct with ZR high power ( end of the year )


Okay.

Our use-case is 100Gbps customer edge, so within the data centre.

We operate an optical network for anything longer than 80km.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/8/23 17:35, Giuliano C. Medalha wrote:


Hello good afternoon.

Please have a look at the following documentation:


https://community.juniper.net/blogs/reema-ray/2023/03/28/mx304-deepdive


Thanks, this is most useful!



It will have everything you need to do with it, including the pictures.

Our first boxes are arriving this next month in Brazil.

By the specs of the new chipset (TRIO6) it's a very good box. A lot of 
enhancements.


Trio capacity aside, based on our experience with the MPC7E, MX204 and 
MX10003, we expect it to be fairly straight forward.


What is holding us back is the cost. The license for each 16-port line 
card is eye-watering. While I don't see anything comparable in ASR99xx 
Cisco-land (in terms of form factor and 100Gbps port density), those 
prices are certainly going to force Juniper customers to look at other 
options. They would do well to get that under control.




And it supports 400G already ( ZR and ZR+ need to check ) ( 16 x 100 or 4 x 400 
) per LMIC.


The LMIC won't care whether it's ZR, ZR+, FR4 or DR4. It will be 
compatible with whatever pluggable is used, as long as it can do 400Gbps.


Unless, of course, you mean whether Juniper provide an interface into 
the optic for use-cases where you are plugging into a ROADM... that, I 
don't know.


Are you intending to use this router for long-distance applications?

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

On 6/8/23 17:18, Kevin Shymkiw wrote:

Along with this - I would suggest looking at Port Checker (
https://apps.juniper.net/home/port-checker/index.html ) to make sure
your port combinations are valid.


We've had ample experience with Juniper's MPC7E, MX204, PTX1000 and 
PTX10001 to know how they structure this from a philosophical 
standpoint. So not a major drama there.


It's just interesting to me that the data sheet does not mention needing 
to sacrifice an RE to get to the chassis' advertised full port 
compliment. Unless the data sheet was updated and I missed it.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] MX304 Port Layout

[Mark Tinka via juniper-nsp]

So, we decided to give the MX304 another sniff, and needed to find out 
why Juniper charge a license for 16x 100Gbps ports per line card, and 
yet the data sheet suggests the box can handle 48x 100Gbps ports 
chassis-wide.


Well, turns out that if you deploy it with redundant RE's, you get 32x 
100Gbps ports (2x line cards of 16x ports each).


However, to take another 16 ports that gets you to 48x 100Gbps ports, 
you need to sacrifice one RE to use its slot :-).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] JunOS RPKI/ROA database in non-default routing instance, but require an eBGP import policy in inet.0 (default:default LI:RI) to reference it.

[Mark Tinka via juniper-nsp]

On 6/6/23 09:27, Saku Ytti wrote:


I am not implying it is pragmatic or possible, just correct from a
design point of view.

Commercial software deals with competing requirements, and these
requirements are not constructive towards producing maintainable clean
code. Over time commercial software becomes illiquid with its
technical debt.

There is no real personal reward for paying technical debt, because
almost invariably it takes a lot of time, brings no new revenue and
non-coder observing your work only sees the outages the debt repayment
caused. While another person who creates this debt creating new
invoiceable features and bug fixes in ra[pb]id manner is a star to the
non-coder observers.

Not to say our open source networking is always great either, Linux
developers are notorious about not asking SMEs 'how has this problem
been solved in other software'. There are plenty of anecdotes to
choose from, but I'll give one.

- In 3.6 kernel, FIB was introduced to replace flow-cache, of course
anyone dealing with networking could have told kernel developers day1
why flow-cache was a poor idea, and what FIB is, how it is done, and
why it is a better idea.
- In 3.6 FIB implementation, ECMP was solved by essentially randomly
choosing 1 option of many, per-packet. Again they could have asked
even junior network engineers 'how does ECMP work, how should it be
done, I'm thinking of doing like this, why do you think they've not
done this in other software?' But they didn't.
- in 4.4 Random ECMP was changed to do hashed ECMP

I still continue to catch discussions about poor TCP performance on
Linux ECMP environment, then I first ask what kernel do you have, then
I explain to them why per-packet + cubic will never ever perform. So
for 4 years ECMP was completely broke, and reading ECMP release notes
in 4.4 not even developers had completely understood just how bad the
problem one, so we can safely assume people were not running ECMP.

Another example was when I tried to explain to the OpenSSH mailing
list, that ''TOS' isn't a thing, and got a confident reply that TOS
absolutely is a thing, prec/DSCP are not. Luckily a few years later
Job fixed OpenSSH packet classification.

But these examples are everywhere, so it seems you either choose
software written by people who understand the problem but are forced
to write unmaintainable code, or you choose software by people who are
just now learning about the problem and then solve it without
discovering prior art, usually wrong.


I think being able to write code is one thing. Being able to write code 
to build and run an IP/MPLS network is - not a-whole-other - but another 
thing. I say this because people that know how to write code do not 
always understand how IP/MPLS networks work. And for better or worse, we 
need code to run the routers and switches that deliver IP/MPLS 
capability to network operators.


The reason traditional networking OEM's build usable code that allows us 
to run IP/MPLS networks is that their raison d'être is, well, shifting 
packets around the world as quickly as possible. General-purpose OS 
developers optimize for service/app performance, leaving the problem of 
network performance to the networking folk, for the most part. So it 
does not surprise me that developers who code for a general-purpose OS 
would think RIP is better than IS-IS, for example, just because it has 
the word "Routing" in it and they can write code for it. It's not 
because they don't know how to write code for IS-IS... they just don't 
have the organizational structure setup to care about why IS-IS is a 
better idea than RIP. Their organization setup is app, app, app.


Unfortunately, not everybody can be a Cisco, Juniper, Google or AWS, who 
have the benefit of plenty of people that can more easily integrate 
writing code for its down sake with writing code for networking.


It is the reason most large scale network operators will still continue 
to find value in IOS XR, Junos, EOS, ArcOS, e.t.c., than, say, a NOS 
that was put together by someone that knows how to interpret an RFC and 
spit out an implementation on Linux, with zero understanding of the 
overall TCP/UDP/IP/MPLS/Ethernet stack and how it all ties in together 
at scale.


I like what folk like pfSense (Netgate) are doing with FRR, and also 
what folk like Mikrotik can pack in 13MB of software... but at a certain 
scale, you simply can't ignore traditional networking OEM, try as we might.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] JunOS RPKI/ROA database in non-default routing instance, but require an eBGP import policy in inet.0 (default:default LI:RI) to reference it.

[Mark Tinka via juniper-nsp]

On 6/5/23 23:26, Jeff Haas via juniper-nsp wrote:


[Note that I've already inquired internally about the original problem.  I 
don't recall the answer from top of head and don't have time for code 
spelunking...]

As to the point below, we get to these headaches one commit at a time.  Junos 
is long-lived enough that VRFs started as a hack on a system that didn't have 
them.  A completely new system written from scratch likely would just assume 
arbitrary tables that can be spaghetti configured into whatever you want to do 
with them.

When I was first studying the Juniper implementation of L3VPN VRFs, some of the 
rib-group constructs didn’t make a lot of sense to me.  It was rather clearer 
after discovering that when the features were released in the 9.x (?) releases 
that all of the niceties we have today used to be completely done at a manual 
level with those same rib-groups. __

While I have a lot of sympathy for Saku's pragmatism, I prefer to file off the 
ugly edges of old justifications when I can... but it's done one commit at a 
time.


I think Junos' VRF implementation is so far down the line, it makes more 
sense to just carry on with the architecture and add knobs, support and 
fixes as issues are either discovered or as customers ask for them. As 
you say, "One commit at a time".


Going back to re-do the implementation from scratch would be a 
non-starter. There is simply too much water under this bridge.


It is not unlike all the stuff you get in IOS (and IOS XE) after so many 
years of assuming so many things in the 90's and early 2000's, not least 
of which that IP is not the only protocol routers will ever run. It is 
so hard for Cisco to remove or re-do those assumptions. At some point, 
you just have to accept that picking up an OS-specific book that 
eloquently describes its eccentricities is part of the process of 
raising a network engineer.


Newer OS's like EOS, ArcOS have a cleaner, simpler and more sensible 
approach to this sort of thing because they are benefiting from all the 
good things IOS/IOS XE/IOS XR and Junos have done in the past 25+ years, 
while dropping all the bad things those OS's have also done. Where the 
older OS's still display their strength is in their maturity, especially 
of routing protocols.


In the end, pick your poison, buy the book, and be happy.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] JunOS RPKI/ROA database in non-default routing instance, but require an eBGP import policy in inet.0 (default:default LI:RI) to reference it.

[Mark Tinka via juniper-nsp]

On 6/5/23 09:46, Saku Ytti via juniper-nsp wrote:


It is safer to put the service (internet) in VRF, and leave the main
table for signalling and NMS, if you want to create this distinction.
It will also make it a lot more convenient to leak between instances
and create subInternets, like peeringInternet, to avoid peers from
default routing to you.


I'm still in awe of operators who run their Internet and/or core network 
inside a VRF :-).


In all my years in the game, that is the one thing I have never been 
brave about. I find it cheaper and simpler to just separate functions 
with physical hardware.




I consider it a design error that NOS conceptually has two types of
instances, main instance and VRF instance. This distinction makes it
more expensive to write and maintain the NOS and it makes it more
fragile.


I can see reasons for and against it, but no doubt, it does add 
complexity into the code base, even though it may "simplify" network 
operations to have the distinction.


Cisco's approach is less onerous, with simply applying the VRF wherever 
it is needed inside the global configuration. I'd imagine that is 
simpler to code for, and operationally, it is reasonably simple as well.




General principle applies, do boring things, get boring results. Same
reason why IPv6 continues to be 2nd class citizen and is a bad
candidate for your NMS/signalling AFI, people don't use it, so if you
do, you will be responsible for driving vendors to fix it. Time which
you likely should be spending doing something else.


I, most likely, do not speak for the majority when I say we are a 
dual-stack house and signal both on IPv4 and IPv6 for just about 
everything... SSH, TACACS+, RADIUS, SNMP, NTP, ROV, DNS, Syslog, LDP, 
Netflow, e.t.c. I do remember a time when vendors did not have parity 
for this, but that was, at worst, 9 years ago.


It does take some effort to have signaling parity between IPv4 and IPv6, 
and I think that is the biggest hurdle for network operators... that 
boring is so good, it's a mission thinking about bringing some 
excitement into your life :-). It has been easier for us because we got 
the chance to greenfield our network revamp 11 years ago, so we got 
dual-stack done at the same time. I can see a complication where 
longstanding deployments are very IPv4-centric, that going back to work 
IPv6 natively in is insurmountable.


DNSSEC, RPKI, DMARC, DANE and DKIM suffer from the same inattention as IPv6.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX960 PEMs and 3 phase power

[Mark Tinka via juniper-nsp]

On 3/23/23 14:15, Tom Storey via juniper-nsp wrote:


We have A+B power delivered 3 phase to the racks, broken out on PDUs, but I
think I must have been thinking about another platform in regards to not
splitting PSUs across phases.


Right, data centre providers will deliver power to your rack either via 
a single- or 3-phase bus, and leave you to decide how to use it.


Unless you are doing something special with the data centre provider for 
you to yield the desired outcome, I'm not sure splitting phases across 
the chassis adds any more value than A+B, especially with the MX960 
power supplies which come with 2 AC receptacles per power supply that 
you, ideally, will already split to redundant power sources.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX960 PEMs and 3 phase power

[Mark Tinka via juniper-nsp]

On 3/23/23 10:29, Gert Doering wrote:


"typical" data centres over here have 2 primary feeds, either one
with UPS and one with generator, or both with (different) UPSes - but
depending on the load drawn by a rack, might be presented as multiple
circuits with 16A each, and individual circuit breakers.


Same here, and also what we have seen at most of the larger data centres 
we work with.




Now, if another device next to your MX960 has a PSU failure and kicks
one of the circuit breakers, what do you want your MX960 to do...  as
well, what do you want to happen if the UPS fails, and one of the primary
feeds goes down.


We typically deploy a full power complement in all our routers to 
protect against issues such as these, even when we may not necessarily 
need all that power Day 1.




We do not have MX960s, but we did have other devices with 3 PSUs that
needed 2 to fully power all line cards, and this was quite a bit annoying
- "it says redundant PSUs, but no full feed A / feed B resiliency"...


Even with the 2nd generation high-capacity AC power supplies, the MX960 
requires at least 2 power supplies at a minimum, to power the 2 zones in 
the system.


These power supplies have 2 receptacles per unit, to allow you to drive 
a full chassis running at speed with just 2 of them in the box. 
Otherwise, with just a single receptacle per power supply, you only get 
about half of the energy out of the power supply.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX960 PEMs and 3 phase power

[Mark Tinka via juniper-nsp]

On 3/22/23 17:27, Tom Storey via juniper-nsp wrote:


Hi all.

I had this idea in my head that MX960 power supplies should not be split
across phases, but I cant find anything in any documentation that says that.

Can anyone comment on whether multiple phases per PEM are supported, or
whether its even a reasonable idea to put into practice?

My thought is one PEM on one phase, such that if that phase drops, you drop
that PEM entirely. Or is it worthwhile spreading across phases to keep even
half of a PEM alive and supplying the chassis?


I can't think of any plausible reason why this would not work, but it 
sounds awfully complicated.


If you are in a typical data centre, all your power is going to come 
from at least one UPS anyway, maybe even two. So not sure you need to 
worry about this.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] ACX7100 route scale

[Mark Tinka via juniper-nsp]

On 12/31/22 14:04, Giuliano C. Medalha via juniper-nsp wrote:


Good Morning

Think it is 10M RIB and 1.5M FIB ( need to check )


CPU is x86 running with 32GB of RAM.

FIB is 1 - 2 million entries.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] polishing an antique m7i

[Mark Tinka via juniper-nsp]

On 7/2/22 20:00, Randy Bush via juniper-nsp wrote:

- old m7i with RE-B-1800X1-4G-S
- currently running 14.2R7.5
- hard disk dying
- have nice new 1tb sata ssd for it
- juniper support download is pushing 15.1R7.9 at me
- should i worry about increased memory use or license changes in 15?
- if so, where the heck is 14?


If 14 is missing from the repository, then it's probably because it is 
EoL. I can't find 14 for even the MX, so chances are Juniper stopped 
maintaining it a while ago. I recall debuting 14 into our network back 
in 2014, and it had tons of problems. I'd be surprised if Juniper are 
still actively supporting it.


For the M7i, chances are your memory footprint will bulge with 15, but 
likely not as much as if you went to higher code (which the M7i doesn't 
support - it tops out at 15.1).


I'd be keen to hear if you can get the SSD drive to boot, though.



- and with the support portal rearrangement, i can not find destructions
   for making a bootable usb stick from install-media-15.1R7.dms (on a
   mac or an rPi, of course:) 


This is dated, but might offer some help:

http://networkarch.blogspot.com/2013/02/building-bootable-juniper-usb-stick-on.html

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE tunnels on a QFX10002-60C

[Mark Tinka via juniper-nsp]

On 6/27/22 21:22, Mark Tinka wrote:



I'm having a meeting with them about all this on Thursday. If anything 
is exciting, and I can share, I will.


As promised, below is what has developed:

As it pertains to the both the MX204 and MX10003, Juniper have made the 
following amendments:


    EoS = 2023.
    End of new features = 2024.
    End of bug fixes = 2028.
    End of critical features = 2028.
    EoL = 2029.

FWIW.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE tunnels on a QFX10002-60C

[Mark Tinka via juniper-nsp]

On 6/27/22 18:22, Olivier Benghozi via juniper-nsp wrote:


I guess that the right thing to do would be to provide a licence based model 
for MX304 with an entry level capacity licence priced as the MX204 currently 
is...


That would, indeed, be the right thing. But it's the wild west in 
vendor-land, now. Can't believe anything they say to stay true for any 
reasonable period of time.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE tunnels on a QFX10002-60C

[Mark Tinka via juniper-nsp]

On 6/27/22 18:15, Giuliano C. Medalha wrote:


MX204 was announced at EoS

We have used MX204 for last 5 years. It was a huge success for any 
function it was designed.


We intend to keep running ours until the 2nd coming :-).

I don't have time for Juniper's nonsense.




Juniper only recommend to us ( about gre projects ) in mx product line

Are you going to use mx304 after this year ?

Even this box is more expensive ( 5x at least )

Or Juniper is thinking to launch a new router with TRIO 6 ?


I'm having a meeting with them about all this on Thursday. If anything 
is exciting, and I can share, I will.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE tunnels on a QFX10002-60C

[Mark Tinka via juniper-nsp]

On 6/24/22 11:01, Saku Ytti wrote:


Many ways to skin the cat. If you can dedicate small router to the
scrubber (or routing-instance if you can't) and you run BGP-LU, so you
avoid useless egress IP lookup, you just ensure that the scrubber PE
or scrubber instance doesn't have the more specific routes, then it'll
follow the BGP-LU path to egress CE.
You can scrub any and all prefixes, without any scale implications as
you never need to touch the network to handle clean traffic.


Agreed.

We use the MX204 as the dedicated router attached to the scrubber, so 
there is no performance issue (yet).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE tunnels on a QFX10002-60C

[Mark Tinka via juniper-nsp]

On 6/24/22 09:28, Saku Ytti via juniper-nsp wrote:


In many ways filter based decapsulation is actually preferable to
interface, so I have no large qualms here. What I'd actually want is
egress filter decap, instead of ingress. So I could point my GRE
tunnels to random addresses at customer network, and have in my edge
filters static decap statement which is never updated. Like 'from
scurbber/32 to anywhere, protocol gre, decap'. This way my scrubber
would launch GRE tunnels to any address at customer site, routing
would follow best BGP path to egress and just at the last moment,
packet would get decapped.


After failing to get Netscout to natively support IS-IS, we came up with 
a rather convoluted - but elegant - way to transport on-ramp/off-ramp 
traffic into and out of our scrubbers.


Basically, we use lt-* (logical tunnel) interfaces that sit both in the 
global table and a VRF. We loop them to each other, and use IS-IS + BGP 
+ LDP to tunnel traffic natively using MPLS-based LSP's signaled by LDP 
(as opposed to GRE), so that traffic an always follow the best IS-IS + 
iBGP path, without the hassle of needing to run GRE between routers and 
scrubbers.


The scrubbers inject the /32 or /128 route that needs protection, and 
the routers responsible for this "Frankenstein" setup do the rest. I'm 
quite proud of the setup, actually :-).


Essentially, it's equivalent to getting the Netscout TMS to be part of 
your IS-IS and MPLS/LDP domain, even though it supports neither.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Request for data from PTX PE/Paradise (e.g. PTX1000, LC1101) operators

[Mark Tinka via juniper-nsp]

On 6/17/22 11:05, Saku Ytti via juniper-nsp wrote:

Hi,

I'd like to return to this topic. I was confused earlier,
misattributing the issue I'm seeing to MX. And now it is clear it must
be on PTX.

I'd again solicit for input for anyone seeing in their syslogs:
   a) junos: 'received pdu - length mismatch for lacp'
   b) iosxr: 'ROUTING-ISIS-4-ERR_BAD_PDU_LENGTH' or
'ROUTING-ISIS-4-ERR_BAD_PDU_FORMAT'
   c) or any other NOS which might log this, when another end is PTX

The issue is very rare, so ideally you'd look at syslog for all
periods you've had Paradise in the network.


I've checked all our links where a PTX1000 is on end and a CRS-X is on 
another, and can't match any such logs.


I'll keep digging.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] bgp graceful-shutdown receiver

[Mark Tinka via juniper-nsp]

On 5/7/22 16:32, Michael Hare wrote:

Mark,

I was told by JTAC that the 19.1 default "'set protocols bgp graceful-shutdown 
receiver" does exactly that [blindly trusts 65535:0 as zero] for iBGP learned 
routes, but not eBGP.  Unless I'm daft that's not what their public documentation implies.

A set command that means "no really, I want 65535:0 to mean localpref=X for whatever 
group hierarchy I configure said feature" might be neat.  It would have saved me a 
few hours as I am a tool assisted CLI jockey, not fully pushing from a database.  But the 
ship has long since sailed as I ended up changing my import/export policies instead.  It 
was a worthwhile endeavor anyway as I also incorporated prepending and MEDs as part of 
our outbound shutdown approach.


Thanks for the feedback, MIchael. Most appreciated.

For me, this somewhat amounts to what Cisco do with RPKI in IOS XE, 
where they automatically apply policy based on RPKI state. I don't like 
that at all. Things can get tricky between code upgrades when this 
"automation" is broken, or changes with little to no documentation.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] bgp graceful-shutdown receiver

[Mark Tinka via juniper-nsp]

On 4/18/22 17:24, Michael Hare via juniper-nsp wrote:

Hello,

Is anyone using "bgp graceful-shutdown receiver" successfully out-of-the-box 
for eBGP peers without modifying their import policies to account for 65535:0?

For example our production AS peers with lab AS over eBGP.  Import policy on the 
production side sets local preference.  "I was assuming" that the reception of 
65535:0 would set localpref to 0 and that's not what I see.  JTAC is claiming this is 
expected and that any import policy that sets local preference will override having 
graceful-shutdown receiver enabled.

Yes, I have confirmed gshut is enabled and I am indeed receiving 65535:0.   If 
I change my import policy to match 65535:0 and set local pref to 0, it 
unsurprisingly works.  Thankfully I have disaggregated the terms that accept a 
route from those that set local preference so I'm just looking at a major 
annoyance instead of major pain, but I find this a bit unbelievable as a 
default behavior.  But perhaps I'm missing the concept of why the hook to set 
localpref appears to be at start of policy evaluation and not after route 
acceptance inside RPD.


My understanding is that 65535:0 is "universally" accepted across eBGP 
neighbors to signal LOCAL_PREF=0. However, receiving operators need to 
setup the policy infrastructure to make that happen.


I'd find it rather odd if a vendor automatically changed the LOCAL_PREF 
to 0 for a route that shipped with 65535:0, without the operator 
specifically pre-defining the policy infrastructure to support that. I 
certainly wouldn't want that from my vendor.


Unless I am missing something...

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] juniper security advisories rss feeds broken this year?

[Mark Tinka via juniper-nsp]

On 5/5/22 22:46, Joel Jaeggli via juniper-nsp wrote:



What are people doing at this point to subscribe to vendor specific 
juniper vulnerability publication.


We rely on the TSB (Technical Support Bulletin) e-mails that come 
through with vulnerabilities, and the code in which they are fixed.


I believe you need to subscribe to the TSB here:

    https://supportportal.juniper.net/s/knowledge?language=en_US

Mark.

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MPC4E-3D-2CGE-8XGE

[Mark Tinka via juniper-nsp]

On 4/13/22 18:25, Chris Wopat via juniper-nsp wrote:


Dario -

Friendly heads up that the annual support costs on MPC7E is incredibly
expensive.

So high that pre-pandemic prices it would've been cheaper to just buy
one off of eBay than pay for a year of support (If this matters to
you)


I intentionally neglected to mention that the MPC7E on the open market 
is a steal. I figured the OP would, well, figure that out :-).


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MPC4E-3D-2CGE-8XGE

[Mark Tinka via juniper-nsp]

--- Begin Message ---



On 4/13/22 15:50, Dario Amaya via juniper-nsp wrote:

Hello,

Does anybody know if this card (MPC4E-3D-2CGE-8XGE) is compatible with
the SCBE-MX ?


Nope, never used it before.



This site says yes:
https://apps.juniper.net/hct/model/?component=MPC4E-3D-2CGE-8XGE=sComponents


Yep, looks like it supports the SCBE-MX, SCBE2-MX and SCBE3-MX.



Is anybody using these cards? We need 100G port on our MX480's but
it's so expensive!


The MPC7E might be a better option for 100Gbps. But if you don't have 
significant port and chassis requirements, the MX204 is worth considering.


Mark.

--- End Message ---
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 20 - slow RPD

[Mark Tinka via juniper-nsp]

On 3/25/22 11:21, Mihai via juniper-nsp wrote:

In my case I just upgraded one MX204 in the lab to 21.2R2, enabled 
rib-sharding and increased the JunosVM memory to 24G and things look 
better now.


Glad to hear!

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Junos 20 - slow RPD

[Mark Tinka via juniper-nsp]

On 3/25/22 02:58, Gustavo Santos via juniper-nsp wrote:


Hi,
I think that I was the only one with this issue.


From their feedback, it seems the issue of scaling outbound updates of 
full tables to eBGP neighbors is known within Juniper, because they told 
us they have had to come up with all manner of hacks for many of their 
large scale customers as well.


So it's a fundamental problem, one I'm not sure they are addressing very 
well.


We can't keep throwing hardware at the problem.



it is better to lost NSR than blackhole
traffic for over an hour..


Agreed - we had gotten to the point where we were willing to give up NSR 
until we figure this out.


Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp