Re: [j-nsp] BGP timer
On 2024-04-27 09:44, Lee Starnes via juniper-nsp wrote: > Having difficulty finding a way to prevent BGP from re-establishing after a > BFD down detect. I am looking for a way to keep the session from > re-establishing for a configured amount of time (say 5 minutes) to ensure > we don't have a flapping session for a. link having issues. Isn't that what the holddown-interval setting does? It is limited to 255 seconds (4 minutes 15 seconds), though, and for BGP it is only allowed for EBGP sessions, not iBGP sessions. The documentation also says that you need to set holddown-interval on *both* ends. I'm gueesing that the holddown only prevents your end from initiating the BGP session, but that it will still accept a connection initiated from the other end. https://www.juniper.net/documentation/us/en/software/junos/cli-reference/topics/ref/statement/bfd-liveness-detection-edit-protocols-bgp.html I haven't used BFD for BGP myself, though, only for static routes on a couple of links. But there I do use holddown-interval, and at least when I set it up several years ago, it seemed to do what I expected: after the link and the BFD session came up again, it waited (in my case) 15 seconds before enabling my static route again. -- Thomas Bellman, National Supercomputer Centre, Linköping Univ., Sweden "We don't understand the software, and sometimes we don't understand the hardware, but we can *see* the blinking lights!" signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] QSA28 adapter in QFX5120-32C
Has anyone managed to get a 25G transceiver to work in a QFX5120-32C
switch with a QSA28 adapter, i.e, an adapter from QSFP28 to SFP28? If
so, what brand and model, what Junos version, and what configuration
magic did you need?
We have tried both an adapter from SmartOptics, and one from Dell,
but the switch doesn't seem to recognize it as a 25G transceiver at
all. The channelized interfaces (et-0/0/0:[0-3]) don't show up, only
a non-channelized interface (et-0/0/0), and the switch says it has a
speed of 40 Gbit/s.
The transceivers themselves (also from SmartOptics) seem to work, or
are at least *recognized*, when we put them in QFX5120-48Y switches,
but those have native SFP28 ports, so we are not using an adapter
there.
QSA adapters for putting a 10G transceiver into it (i.e. not QSA28),
works without any problems in the switch, and likewise does break-out
DAC cables to give us 4×25G from each QSFP28 port.
We have configured the ports with
chassis {
fpc 0 {
pic 0 {
port 0 { channel-speed 25g; }
port 1 { channel-speed 25g; }
port 2 { channel-speed 25g; }
}
}
}
We have also tried "speed 100g" instead of "channel-speed 25g", as
well as no speed configuration at all, to no avail.
We have tested the QSA28 adapter in a Dell Z9100 switch as well, and
it seems to be OK with the adapter, and recognizes the transceiver as
25Gbase-LR (but then thinks it is not Dell qualified, and refuses to
use it, but the adapter itself seems OK).
Junos version 23.2R1-S1.6, but we have also tried 22.2R2-S1.5 (which
the switch arrived pre-installed with) and 22.4R2-S2.6 (which seems
to be the one with latest release date).
SmartOptics says they get their adapter to work in a PTX10001-36MR,
if you configure it with
interfaces {
et-0/1/4 {
number-of-sub-ports 4;
speed 25g;
}
}
but that syntax is not accepted by Junos on QFX5120 (I suppose it is
specific to Junos Evolved).
--
Thomas Bellman, National Supercomputer Centre, Linköping Univ., Sweden
"We don't understand the software, and sometimes we don't understand
the hardware, but we can *see* the blinking lights!"
signature.asc
Description: OpenPGP digital signature
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] MX304 Port Layout
On 2023-06-08 17:18, Kevin Shymkiw via juniper-nsp wrote: > Along with this - I would suggest looking at Port Checker ( > https://apps.juniper.net/home/port-checker/index.html ) to make sure > your port combinations are valid. The port checker claims an interresting "feature": if you have anything in port 3, then *all* the other ports in that port group must also be occupied. So if you use all those four ports for e.g. 100GE, everything is fine, but if you then want to stop using either of ports 0, 1 or 2, the configuration becomes invalid... (And similarly for ports 5, 8 and 14 in their respective groups.) I hope that's a bug in the port checker, not actual behaviour by the MX304... -- Thomas Bellman, National Supercomputer Centre, Linköping Univ., Sweden "We don't understand the software, and sometimes we don't understand the hardware, but we can *see* the blinking lights!" signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] VRRP for IPv6
On 2022-01-25 22:53, Chris Adams via juniper-nsp wrote: > I wasn't planning to use a virtual link-local address, so I didn't put > one. The JUNOS VRRP for v6 example doesn't include one, although then > the JUNOS documentation for virtual-link-local-address is oddly > confusing: For IPv6, the VRRP protocol requires that the link-local address is virtual; it *must* be present in the list of virtual addresses a VRRP node announces. But Junos does indeed generate one automatically for you; you don't need to add a virtual-link-local-address stanza. And the link-local address should show up when you run 'show vrrp': bellman@Bluegrass2> show vrrp Interface State Group VR state VR Mode TimerType Address irb.214up 1 master Active A 0.461 lcl 2001:6b0:17:180::3 vip fe80::200:5eff:fe00:201 vip 2001:6b0:17:180::1 (Unfortunately I don't have any immediate ideas of why VRRP for IPv6 doesn't work for you, or why you don't see the outgoing packets using 'monitor traffic'. When I test on a couple of QFX:es and EX4600:s, I can see both outgoing and incoming VRRP packets.) /Bellman signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Cut through and buffer questions
On 2021-11-19 10:07, Saku Ytti via juniper-nsp wrote: > Cut-through does nothing, because your egress is congested, you can > only use cut-through if egress is not congested. Cut-through actually *can* help a little bit. The buffer space in the Trident and Tomahawk chips is mostly shared between all ports; only a small portion of it is dedicated per port[1]. If you have lots of traffic on some ports, with little or no congestion, enabling cut-through will leave more buffer space available for the congested ports, as the packets will leave the switch/router quicker. One should note though that these chips will fall back to store- and-forward if the ingress port and egress port run at different speeds. (In theory, it should be possible to do cut-through as long as the egress port is not faster than the ingress port, but as far as I know, any speed mismatch causes store-and-forward to be used). Also, if you have rate limiting or shaping enabled on the ingress or egress port, the chips will fall back to store-and-forward. Whether this helps *enough*, is another question. :-) I believe in general, it will only make a pretty small difference in buffer usage. I enabled cut-through forwarding on our QFX5xxx:es and EX4600:s a few years ago, and any change in packet drop rates or TCP performance (both local and long-distance) was lost way down in the noise. But I have seen reports from others that saw a meaningful, if not exactly huge, difference; that was several years ago, though, and I didn't save any reference to the report, so you might want to classify that as hearsay... (I have kept cut-through enabled on our devices, since I don't know of any practical disadvantages, and it *might* help a tiny little bit in some cases.) [1] Of the 12 Mbyte buffer space in Trident 2, which is used in QFX5100 and EX4600, 3 Mbyte is used for per-port dedicated buffers, and 9 Mbyte is shared between all ports. I believe on later chips an even larger percentage is shared. -- Thomas Bellman, National Supercomputer Centre, Linköping Univ., Sweden "We don't understand the software, and sometimes we don't understand the hardware, but we can *see* the blinking lights!" signature.asc Description: OpenPGP digital signature ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] Cut through and buffer questions
On 2021-11-19 09:49, james list via juniper-nsp wrote:
> I try to rephrase the question you do not understand: if I enable cut
> through or change buffer is it traffic affecting ?
On the QFX 5xxx series and (at least) EX 46xx series, the forwarding
ASIC needs to reset in order to change between store-and-forward and
cut-through, and traffic will be lost until the reprogramming has been
completed. Likewise, changing buffer config will need to reset the
ASIC. When I have tested it, this has taken at most one second, though,
so for many people it will be a non-event.
One thing to remember when using cut-through forwarding, is that packets
that have suffered bit errors or truncation, so the CRC checksum is
incorrect, will still be forwarded, and not be discarded by the switch.
This is usually not a problem in itself, but if you are not aware of it,
it is easy to get confused when troubleshooting bit errors (you see
ingress errors on one switch, and think it is the link to the switch
that has problems, but in reality it might just be that the switch on
the other end that is forwarding broken packets *it* received).
> Regarding the drops here the outputs (15h after clear statistics):
[...abbreviated...]
> Queue: 0, Forwarding classes: best-effort
> Transmitted:
> Packets :6929684309190446 pps
> Bytes: 4259968408584 761960360 bps
> Total-dropped packets: 1592 0 pps
> Total-dropped bytes : 2244862 0 bps
[...]> Queue: 7, Forwarding classes: network-control
> Transmitted:
> Packets : 59234 0 pps
> Bytes: 4532824 504 bps
> Total-dropped packets: 0 0 pps
> Total-dropped bytes : 0 0 bps
> Queue: 8, Forwarding classes: mcast
> Transmitted:
> Packets : 655370488 pps
> Bytes:5102847425663112 bps
> Total-dropped packets: 279 0 pps
> Total-dropped bytes :423522 0 bps
These drop figures don't immediately strike me as excessive. We
certainly have much higher drop percentages, and don't see much
practical performance problems. But it will very much depend on
your application. The one thing I note is that you have much
more multicast than we do, and you see drops in that forwarding
class.
I didn't quite understand if you see actual application or
performance problems.
> show class-of-service shared-buffer
> Ingress:
> Total Buffer : 12480.00 KB
> Dedicated Buffer : 2912.81 KB
> Shared Buffer: 9567.19 KB
> Lossless : 861.05 KB
> Lossless Headroom : 4305.23 KB
> Lossy : 4400.91 KB
This looks like a QFX5100 or EX4600, with the 12 Mbyte buffer in the
Broadcom Trident 2 chip. You probably want to read this page, to
understand how to configure buffer allocation for your needs:
https://www.juniper.net/documentation/us/en/software/junos/traffic-mgmt-qfx/topics/concept/cos-qfx-series-buffer-configuration-understanding.html
In my network, we only have best-effort traffic, and very little
multi- or broadcast traffic (basically just ARP/Neighbour discovery,
DHCP, and OSPF), so we use these settings on our QFX5100 and EX4600
switches:
forwarding-options {
cut-through;
}
class-of-service {
/* Max buffers to best-effort traffic, minimum for lossless ethernet */
shared-buffer {
ingress {
percent 100;
buffer-partition lossless { percent 5; }
buffer-partition lossless-headroom { percent 0; }
buffer-partition lossy { percent 95; }
}
egress {
percent 100;
buffer-partition lossless { percent 5; }
buffer-partition lossy { percent 75; }
buffer-partition multicast { percent 20; }
}
}
}
(On our QFX5120 switches, I have moved even more buffer space to
the "lossy" classes.) But you need to tune to *your* needs; the
above is for our needs.
/Bellman
signature.asc
Description: OpenPGP digital signature
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] QFX5100-48S-AFI/AFO vs QFX5100-48S-3AFI/AFO
On 2021-10-26 23:27, Han Hwei Woo via juniper-nsp wrote:
> Does anyone know if there are any differences between the
> QFX5100-48S versions with or without the '3'?
The normal version of QFX5100 have two management ethernet ports, one
SFP port and one twisted pair ("RJ45") port, while the 3AFI/3AFO
versions have three management ports, two SFP ports and one twisted
pair port.
I have never seen or used the 3AFI/3AFO version in the real world
myself, but from reading the hardware guide
(https://www.juniper.net/documentation/us/en/hardware/qfx5100/qfx5100.pdf)
it looks like the TP port and one of the SFP ports are actually
shared, so you can use *either*, but not both at the same time, and
that becomes the em0 interface. (That's what some vendors call a dual
personality port.) The other SFP port is then the em1 interface.
Otherwise they appear to be identical.
More modern Juniper models appear to only have twisted pair ports,
and it seems pretty random which ones have one port and which ones
have two ports. For example, in the QFX5120 line, the -48Y and
-48YM modules have two management ports, while the -48T and -32C
models have one. Weird.
(And hey, Juniper, how about making those management ports actually
useful, and connect them to an IPMI controller with support for Serial
Over LAN? That would be super helpful, especially when you are e.g.
upgrading Junos on them.)
/Bellman
signature.asc
Description: OpenPGP digital signature
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] QFX3500 and... multicast forward (VRRP related)
On 2021-09-13 13:56, Xavier Beaudouin wrote:
> I have a strange clue with an QFX3500-48S4Q, and with "simple" VRRP
> setup.
>
> On port xe-0/0/6.0 I have a infrastructure (cisco switches) with a
> VLAN 3016 who want to be VRRP with an MX204 on et-0/1/0.0.
>
> Current config of the switch :
>
> [...]
>
> Pretty "simple" configuration.
>
> When I monitor traffic interface et-0/1/0.0 match vrrp I see the VRRP
> comming MX204 and on xe-0/0/6.0 I see also VRRP comming TO the QFX...
> Is there any reason why VRRP on the VLAN is not forwarded between
> et-0/1/0.0 and xe-0/0/6.0 ?
>
> Did I missed something ?
The 'monitor traffic' command will only show packets that actually go
to or from the routing engine ("supervisor" in Cisco speak). Traffic
that is just forwarded by the packet forwarding engine (the line cards
on an MX, or the ASIC on the QFX3500) don't show up. Traffic that is
generated by or handled by the PFE itself, e.g. BFD on some platforms,
*also* are not visible to 'monitor traffic'. (Also worth noting, is
that incoming traffic that *is* destined for the routing engine, but
is discarded by a firewall rule executing on the PFE, also won't be
seen by 'monitor traffic'.)
The fact that you see VRRP packets from both the Cisco and the MX,
is probably because it is multicast, so it is forwarded both to the
other port, *and* to the routing-engine. But since the QFX3500 is
not itself doing any VRRP, you won't see any outgoing VRRP packets.
The interresting question is if the MX and the Cisco see each other's
VRRP packets. Do they both think they are master? (It seems like the
command to use is 'show vrrp' both in Junos and IOS.) If both sides
believe they are master, then you have a real forwarding problem, but
if one side think they are backup, and says the other is master, then
the VRRP packets are forwarded as they should.
This is what it looks like on a Juniper:
bellman@...> show vrrp
Interface State Group VR state VR Mode TimerType Address
irb.13 up 1 backupActive D 2.949 lcl 192.168.13.18
vip 192.168.13.19
mas 192.168.13.20
This says that my router's own IP address ("lcl") is 192.168.13.18,
the virtual address the routers are battling for ("vip") is
192.168.13.19, and the current master ("mas") is 192.168.13.20.
(I don't have a Cisco at hand to check what the output looks like
there.)
Note though, that if you run 'show vrrp' on the VRRP master, you
won't see any information about which nodes are backups. That's
because in the VRRP protocol, only the master, or those that want
to take over the master role, sends any packets; those that are
backup and happy with being so, are silent. (This is unlike
Cisco's older HSRP protocol, where both master and backup where
sending, allowing you to see all participating nodes by running
'show hsrp' on any of the nodes.)
If you run the 'monitor traffic' command on the MX, you should be
able to see the VRRP traffic it receives and/or sends. (I don't
know if there is any equivalent on command on Cisco's operating
systems.)
/Bellman
signature.asc
Description: OpenPGP digital signature
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
