Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
For info, the issue was related to a carrier mediaconverter sending frames with a private unknown ethertype not decoded by EX4300. Thanks for your help all Cheers James Il Lun 20 Apr 2020, 01:31 Chuck Anderson ha scritto: > Well, that was an easy fix on my MX480s: > > set protocols lldp interface xe-0/0/1 disable > > Now I'm not seeing CRC errors incrementing 2-3 times per minute on the > EX3400s connected directly to the MX480s. > > I'm not excluding any protocols from MACsec--LLDP runs end-to-end between > the EX3400s just fine. > > Check if the carrier is running LLDP or CDP or similar. > > > On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote: > > Yes, I see CRC errors on EX3400s with MACsec termination, but only on > one side. > > > > Here is my topology: > > > > From A to B: > > > > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 > vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B] > > MACsec L2 connection L2 xconnect > MACsec > > > > From B to A: > > > > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 > vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B] > > MACsec L2 connection L2 xconnect > MACsec > > > > I also have a redundant path with EX3400-C (different local switch) and > EX3400-B (same remote switch). > > > > I see the CRC errors increasing at a rate of about 2-3 per minute, but > only on EX3400-A and EXX3400-C. > > > > All EX3400s were initially running 15.1X53-D57. Now A and C are running > 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been > consistent throughout all releases, no improvement with upgrades. > > > > I wonder if something the carrier's ASR9k is sending down the VLAN > towards EX3400-A and -C is causing this? If not, maybe it is the MX480s > sending something locally to EX3400-A and -C? > > > > The following PRs don't seem relevant--I'm not doing anywhere close to > 60% utilization: > > > > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567 > > > > And I'm not seeing "runts": > > > > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663 > > > > I'm only seeing Framing errors (CRC/Align errors): > > > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 > > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed > discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: > 0, FIFO errors: 0, Resource errors: 0 > > CRC/Align errors2279110 > > > > A few seconds later, it increased to 227913: > > > > MAC statistics: Receive Transmit > > Total octets179536471171563221741316352 > > Total packets 13200126465 7010832956 > > Unicast packets13194022205 7004785539 > > Broadcast packets 52720 > > Multicast packets 6098988 6047417 > > CRC/Align errors2279130 > > FIFO errors 00 > > MAC control frames 00 > > MAC pause frames 00 > > Oversized frames 0 > > Jabber frames0 > > Fragment frames 0 > > VLAN tagged frames 13196813130 > > Code violations 0 > > > > Rate is only 24 Mbps, 2200 pps: > > > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps" > > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, > BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: > None, MAC-REWRITE Error: None, > >Input bytes : 17954037433802 24242136 bps > >Output bytes :3221749625049 498504 bps > >Input packets: 13200416854 2190 pps > >Output packets: 7010941872 830 pps > > > > > > > > On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote: > > > Dear experts, > > > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error > > > counter increase also if the traffic is very low. > > > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but > > > traffic max is actually 100 Mbs and on average 10 Mbs. > > > On this interface I've enabled macsec, if I disable macsec the issue > is not > > > in place but unfortunately macsec is mandatory to be kept enabled. > > > > > > I cannot sniff since the packet is encrypted but to me it seems that > > > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs > > > outside to DataCenter. > > > > > > Due to the fact that monitoring system contantly raise an alert, I'd > like > > > to know how to fix it or at least let the EX4300 do not
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
--- Begin Message ---
Thanks Chuck
Richard McGovern
Sr Sales Engineer, Juniper Networks
978-618-3342
I’d rather be lucky than good, as I know I am not good
I don’t make the news, I just report it
On 4/21/20, 11:53 AM, "Chuck Anderson" wrote:
[External Email. Be cautious of content]
As I said, I'm not excluding any protocols from MACsec. With that
configuration, LLDP apparently doesn't work "outside the tunnel"--I never see
any directly attached neighbors. LLDP does work between the MACsec
endpoints--they show as if they are directly connected neighbors. I'm fine
with that result in my case.
The solution to eliminate the spurious framing errors appears to be: Ask
your carrier to shut off LLDP/CDP/any other L2 protocols running on their
interfaces directly attached to your MACsec endpoint devices.
On Tue, Apr 21, 2020 at 02:59:53PM +, Richard McGovern wrote:
> Based upon Chuck’s reply:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on
the EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end
between the EX3400s just fine.
>
> Don’t exclude LLDP from MACSEC and either stop or block LLDP from the
Carrier/ISP. In Chuck’s case the Carrier (to the EX3400s) was his MX. Then
EX3400s should see each other via LLDP, but not see the carrier. I am not sure
if today, you have an LLDP neighbor with your Carrier/ISP or not?
>
> This is the way I now read his response.
>
> Yes?
>
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
> [signature_1140633420]
>
> From: james list
> Date: Tuesday, April 21, 2020 at 10:53 AM
> To: Richard McGovern
> Cc: Chuck Anderson , Juniper List
> Subject: Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
>
> [External Email. Be cautious of content]
>
> Hi Richard
> lldp and lacp are excluded:
>
> > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode
static-cak
> > > set security macsec connectivity-association MAC pre-shared-key
ckn
> > > set security macsec connectivity-association MAC pre-shared-key
cak
> > > "t"
> > > set security macsec connectivity-association MAC exclude-protocol
lldp
> > > set security macsec connectivity-association MAC exclude-protocol
lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association
MAC
>
> I did not catch the connection with Framing errors counter...
>
> Please detail if you can.
>
> Cheers
>
>
> Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern
mailto:rmcgov...@juniper.net>> ha scritto:
> Chuck, I thought you were running both LLDP and LACP outside the MACSEC
tunnel, no?
>
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association
connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to
be secured using MACsec:
>
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the
specified protocol—in this case, LLDP—that are sent or received on the link.
>
> BEST PRACTICEWe recommend that any protocol other than MACsec being used
on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing
protocols, should be excluded and moved outside of the MACsec tunnel.
>
> Is this not working properly for LLDP?
>
> Rich
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
>
> On 4/19/20, 7:31 PM, "Chuck Anderson" mailto:c...@wpi.edu>>
wrote:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on
the EX3400s connected directly to the MX480s.
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
As I said, I'm not excluding any protocols from MACsec. With that configuration, LLDP apparently doesn't work "outside the tunnel"--I never see any directly attached neighbors. LLDP does work between the MACsec endpoints--they show as if they are directly connected neighbors. I'm fine with that result in my case. The solution to eliminate the spurious framing errors appears to be: Ask your carrier to shut off LLDP/CDP/any other L2 protocols running on their interfaces directly attached to your MACsec endpoint devices. On Tue, Apr 21, 2020 at 02:59:53PM +, Richard McGovern wrote: > Based upon Chuck’s reply: > > Well, that was an easy fix on my MX480s: > > set protocols lldp interface xe-0/0/1 disable > > Now I'm not seeing CRC errors incrementing 2-3 times per minute on the > EX3400s connected directly to the MX480s. > > I'm not excluding any protocols from MACsec--LLDP runs end-to-end between > the EX3400s just fine. > > Don’t exclude LLDP from MACSEC and either stop or block LLDP from the > Carrier/ISP. In Chuck’s case the Carrier (to the EX3400s) was his MX. Then > EX3400s should see each other via LLDP, but not see the carrier. I am not > sure if today, you have an LLDP neighbor with your Carrier/ISP or not? > > This is the way I now read his response. > > Yes? > > > Richard McGovern > Sr Sales Engineer, Juniper Networks > 978-618-3342 > > I’d rather be lucky than good, as I know I am not good > I don’t make the news, I just report it > > [signature_1140633420] > > From: james list > Date: Tuesday, April 21, 2020 at 10:53 AM > To: Richard McGovern > Cc: Chuck Anderson , Juniper List > Subject: Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled > > [External Email. Be cautious of content] > > Hi Richard > lldp and lacp are excluded: > > > > @EX4300-A> show configuration security macsec | display set > > > set security macsec connectivity-association MAC security-mode > static-cak > > > set security macsec connectivity-association MAC pre-shared-key ckn > > > > set security macsec connectivity-association MAC pre-shared-key cak > > > "t" > > > set security macsec connectivity-association MAC exclude-protocol lldp > > > set security macsec connectivity-association MAC exclude-protocol lacp > > > set security macsec interfaces ge-0/0/0 connectivity-association MAC > > I did not catch the connection with Framing errors counter... > > Please detail if you can. > > Cheers > > > Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern > mailto:rmcgov...@juniper.net>> ha scritto: > Chuck, I thought you were running both LLDP and LACP outside the MACSEC > tunnel, no? > > (Optional) Exclude a protocol from MACsec: > [edit security macsec connectivity-association connectivity-association-name] > user@switch# set exclude-protocol protocol-name > For instance, if you did not want Link Level Discovery Protocol (LLDP) to be > secured using MACsec: > > [edit security macsec connectivity-association ca-dynamic1] > user@switch# set exclude-protocol lldp > When this option is enabled, MACsec is disabled for all packets of the > specified protocol—in this case, LLDP—that are sent or received on the link. > > BEST PRACTICEWe recommend that any protocol other than MACsec being used on > the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing protocols, > should be excluded and moved outside of the MACsec tunnel. > > Is this not working properly for LLDP? > > Rich > > Richard McGovern > Sr Sales Engineer, Juniper Networks > 978-618-3342 > > I’d rather be lucky than good, as I know I am not good > I don’t make the news, I just report it > > > On 4/19/20, 7:31 PM, "Chuck Anderson" mailto:c...@wpi.edu>> > wrote: > > Well, that was an easy fix on my MX480s: > > set protocols lldp interface xe-0/0/1 disable > > Now I'm not seeing CRC errors incrementing 2-3 times per minute on the > EX3400s connected directly to the MX480s. > > I'm not excluding any protocols from MACsec--LLDP runs end-to-end between > the EX3400s just fine. > > Check if the carrier is running LLDP or CDP or similar. > > > On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote: > > Yes, I see CRC errors on EX3400s with MACsec termination, but only on > one side. > > > > Here is my topology: > > > > From A to B: > > > > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 > vlan-->-[Carrier-ASR9
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
Hi Richard lldp and lacp are excluded: > > @EX4300-A> show configuration security macsec | display set > > set security macsec connectivity-association MAC security-mode static-cak > > set security macsec connectivity-association MAC pre-shared-key ckn > > set security macsec connectivity-association MAC pre-shared-key cak > > "t" > > set security macsec connectivity-association MAC exclude-protocol lldp > > set security macsec connectivity-association MAC exclude-protocol lacp > > set security macsec interfaces ge-0/0/0 connectivity-association MAC I did not catch the connection with Framing errors counter... Please detail if you can. Cheers Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern < rmcgov...@juniper.net> ha scritto: > Chuck, I thought you were running both LLDP and LACP outside the MACSEC > tunnel, no? > > (Optional) Exclude a protocol from MACsec: > [edit security macsec connectivity-association > connectivity-association-name] > user@switch# set exclude-protocol protocol-name > For instance, if you did not want Link Level Discovery Protocol (LLDP) to > be secured using MACsec: > > [edit security macsec connectivity-association ca-dynamic1] > user@switch# set exclude-protocol lldp > When this option is enabled, MACsec is disabled for all packets of the > specified protocol—in this case, LLDP—that are sent or received on the link. > > BEST PRACTICEWe recommend that any protocol other than MACsec being used > on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing > protocols, should be excluded and moved outside of the MACsec tunnel. > > Is this not working properly for LLDP? > > Rich > > Richard McGovern > Sr Sales Engineer, Juniper Networks > 978-618-3342 > > I’d rather be lucky than good, as I know I am not good > I don’t make the news, I just report it > > > On 4/19/20, 7:31 PM, "Chuck Anderson" wrote: > > Well, that was an easy fix on my MX480s: > > set protocols lldp interface xe-0/0/1 disable > > Now I'm not seeing CRC errors incrementing 2-3 times per minute on the > EX3400s connected directly to the MX480s. > > I'm not excluding any protocols from MACsec--LLDP runs end-to-end > between the EX3400s just fine. > > Check if the carrier is running LLDP or CDP or similar. > > > On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote: > > Yes, I see CRC errors on EX3400s with MACsec termination, but only > on one side. > > > > Here is my topology: > > > > From A to B: > > > > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 > vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B] > > MACsec L2 connection L2 > xconnectMACsec > > > > From B to A: > > > > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 > vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B] > > MACsec L2 connection L2 > xconnectMACsec > > > > I also have a redundant path with EX3400-C (different local switch) > and EX3400-B (same remote switch). > > > > I see the CRC errors increasing at a rate of about 2-3 per minute, > but only on EX3400-A and EXX3400-C. > > > > All EX3400s were initially running 15.1X53-D57. Now A and C are > running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been > consistent throughout all releases, no improvement with upgrades. > > > > I wonder if something the carrier's ASR9k is sending down the VLAN > towards EX3400-A and -C is causing this? If not, maybe it is the MX480s > sending something locally to EX3400-A and -C? > > > > The following PRs don't seem relevant--I'm not doing anywhere close > to 60% utilization: > > > > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567 > > > > And I'm not seeing "runts": > > > > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663 > > > > I'm only seeing Framing errors (CRC/Align errors): > > > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 > > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, > Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch > timeouts: 0, FIFO errors: 0, Resource errors: 0 > > CRC/Align errors2279110 > > > > A few seconds later, it increased to 227913: > > > > MAC statistics: Receive Transmit > > Total octets179536471171563221741316352 > > Total packets 13200126465 7010832956 > > Unicast packets13194022205 7004785539 > > Broadcast packets 52720 > > Multicast packets 6098988 6047417 > > CRC/Align errors
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
--- Begin Message --- Chuck, I thought you were running both LLDP and LACP outside the MACSEC tunnel, no? (Optional) Exclude a protocol from MACsec: [edit security macsec connectivity-association connectivity-association-name] user@switch# set exclude-protocol protocol-name For instance, if you did not want Link Level Discovery Protocol (LLDP) to be secured using MACsec: [edit security macsec connectivity-association ca-dynamic1] user@switch# set exclude-protocol lldp When this option is enabled, MACsec is disabled for all packets of the specified protocol—in this case, LLDP—that are sent or received on the link. BEST PRACTICEWe recommend that any protocol other than MACsec being used on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing protocols, should be excluded and moved outside of the MACsec tunnel. Is this not working properly for LLDP? Rich Richard McGovern Sr Sales Engineer, Juniper Networks 978-618-3342 I’d rather be lucky than good, as I know I am not good I don’t make the news, I just report it On 4/19/20, 7:31 PM, "Chuck Anderson" wrote: Well, that was an easy fix on my MX480s: set protocols lldp interface xe-0/0/1 disable Now I'm not seeing CRC errors incrementing 2-3 times per minute on the EX3400s connected directly to the MX480s. I'm not excluding any protocols from MACsec--LLDP runs end-to-end between the EX3400s just fine. Check if the carrier is running LLDP or CDP or similar. On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote: > Yes, I see CRC errors on EX3400s with MACsec termination, but only on one side. > > Here is my topology: > > From A to B: > > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B] > MACsec L2 connection L2 xconnectMACsec > > From B to A: > > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B] > MACsec L2 connection L2 xconnectMACsec > > I also have a redundant path with EX3400-C (different local switch) and EX3400-B (same remote switch). > > I see the CRC errors increasing at a rate of about 2-3 per minute, but only on EX3400-A and EXX3400-C. > > All EX3400s were initially running 15.1X53-D57. Now A and C are running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent throughout all releases, no improvement with upgrades. > > I wonder if something the carrier's ASR9k is sending down the VLAN towards EX3400-A and -C is causing this? If not, maybe it is the MX480s sending something locally to EX3400-A and -C? > > The following PRs don't seem relevant--I'm not doing anywhere close to 60% utilization: > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567 > > And I'm not seeing "runts": > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663 > > I'm only seeing Framing errors (CRC/Align errors): > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 > CRC/Align errors2279110 > > A few seconds later, it increased to 227913: > > MAC statistics: Receive Transmit > Total octets179536471171563221741316352 > Total packets 13200126465 7010832956 > Unicast packets13194022205 7004785539 > Broadcast packets 52720 > Multicast packets 6098988 6047417 > CRC/Align errors2279130 > FIFO errors 00 > MAC control frames 00 > MAC pause frames 00 > Oversized frames 0 > Jabber frames0 > Fragment frames 0 > VLAN tagged frames 13196813130 > Code violations 0 > > Rate is only 24 Mbps, 2200 pps: > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps" > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, >Input bytes : 17954037433802 24242136 bps >Output bytes :3221749625049
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
Well, that was an easy fix on my MX480s: set protocols lldp interface xe-0/0/1 disable Now I'm not seeing CRC errors incrementing 2-3 times per minute on the EX3400s connected directly to the MX480s. I'm not excluding any protocols from MACsec--LLDP runs end-to-end between the EX3400s just fine. Check if the carrier is running LLDP or CDP or similar. On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote: > Yes, I see CRC errors on EX3400s with MACsec termination, but only on one > side. > > Here is my topology: > > From A to B: > > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 > vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B] > MACsec L2 connection L2 xconnect > MACsec > > From B to A: > > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 > vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B] > MACsec L2 connection L2 xconnect > MACsec > > I also have a redundant path with EX3400-C (different local switch) and > EX3400-B (same remote switch). > > I see the CRC errors increasing at a rate of about 2-3 per minute, but only > on EX3400-A and EXX3400-C. > > All EX3400s were initially running 15.1X53-D57. Now A and C are running > 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent > throughout all releases, no improvement with upgrades. > > I wonder if something the carrier's ASR9k is sending down the VLAN towards > EX3400-A and -C is causing this? If not, maybe it is the MX480s sending > something locally to EX3400-A and -C? > > The following PRs don't seem relevant--I'm not doing anywhere close to 60% > utilization: > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567 > > And I'm not seeing "runts": > > https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663 > > I'm only seeing Framing errors (CRC/Align errors): > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed > discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: > 0, FIFO errors: 0, Resource errors: 0 > CRC/Align errors2279110 > > A few seconds later, it increased to 227913: > > MAC statistics: Receive Transmit > Total octets179536471171563221741316352 > Total packets 13200126465 7010832956 > Unicast packets13194022205 7004785539 > Broadcast packets 52720 > Multicast packets 6098988 6047417 > CRC/Align errors2279130 > FIFO errors 00 > MAC control frames 00 > MAC pause frames 00 > Oversized frames 0 > Jabber frames0 > Fragment frames 0 > VLAN tagged frames 13196813130 > Code violations 0 > > Rate is only 24 Mbps, 2200 pps: > > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps" > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU > Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, > MAC-REWRITE Error: None, >Input bytes : 17954037433802 24242136 bps >Output bytes :3221749625049 498504 bps >Input packets: 13200416854 2190 pps >Output packets: 7010941872 830 pps > > > > On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote: > > Dear experts, > > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error > > counter increase also if the traffic is very low. > > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but > > traffic max is actually 100 Mbs and on average 10 Mbs. > > On this interface I've enabled macsec, if I disable macsec the issue is not > > in place but unfortunately macsec is mandatory to be kept enabled. > > > > I cannot sniff since the packet is encrypted but to me it seems that > > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs > > outside to DataCenter. > > > > Due to the fact that monitoring system contantly raise an alert, I'd like > > to know how to fix it or at least let the EX4300 do not raise the counter > > increase. > > > > I've opened a JTAC case but they found a PR which is currently related to a > > Broadcom chipset raising framing errors during spikes (ie 70% of the > > interface bandwidth). > > > > https://kb.juniper.net/InfoCenter/index?page=content=KB32264=METADATA > > > > Also enabling flow-control as described in the KB do not
Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
Yes, I see CRC errors on EX3400s with MACsec termination, but only on one side. Here is my topology: >From A to B: [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B] MACsec L2 connectionL2 xconnect MACsec >From B to A: [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B] MACsec L2 connectionL2 xconnect MACsec I also have a redundant path with EX3400-C (different local switch) and EX3400-B (same remote switch). I see the CRC errors increasing at a rate of about 2-3 per minute, but only on EX3400-A and EXX3400-C. All EX3400s were initially running 15.1X53-D57. Now A and C are running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent throughout all releases, no improvement with upgrades. I wonder if something the carrier's ASR9k is sending down the VLAN towards EX3400-A and -C is causing this? If not, maybe it is the MX480s sending something locally to EX3400-A and -C? The following PRs don't seem relevant--I'm not doing anywhere close to 60% utilization: https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1261567 And I'm not seeing "runts": https://prsearch.juniper.net/InfoCenter/index?page=prcontent=PR1469663 I'm only seeing Framing errors (CRC/Align errors): admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791 Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 CRC/Align errors2279110 A few seconds later, it increased to 227913: MAC statistics: Receive Transmit Total octets179536471171563221741316352 Total packets 13200126465 7010832956 Unicast packets13194022205 7004785539 Broadcast packets 52720 Multicast packets 6098988 6047417 CRC/Align errors2279130 FIFO errors 00 MAC control frames 00 MAC pause frames 00 Oversized frames 0 Jabber frames0 Fragment frames 0 VLAN tagged frames 13196813130 Code violations 0 Rate is only 24 Mbps, 2200 pps: Manager@sw-gp1-macsec-1> show interfaces extensive xe-0/2/0 |match "bps|pps" Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Input bytes : 17954037433802 24242136 bps Output bytes :3221749625049 498504 bps Input packets: 13200416854 2190 pps Output packets: 7010941872 830 pps On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote: > Dear experts, > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error > counter increase also if the traffic is very low. > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but > traffic max is actually 100 Mbs and on average 10 Mbs. > On this interface I've enabled macsec, if I disable macsec the issue is not > in place but unfortunately macsec is mandatory to be kept enabled. > > I cannot sniff since the packet is encrypted but to me it seems that > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs > outside to DataCenter. > > Due to the fact that monitoring system contantly raise an alert, I'd like > to know how to fix it or at least let the EX4300 do not raise the counter > increase. > > I've opened a JTAC case but they found a PR which is currently related to a > Broadcom chipset raising framing errors during spikes (ie 70% of the > interface bandwidth). > > https://kb.juniper.net/InfoCenter/index?page=content=KB32264=METADATA > > Also enabling flow-control as described in the KB do not change the > behaviour. > > I'm wondering if there could the option we're receiving some sort on > "unknown protocol" from the carrier (I remeber Cisco has something like > that) or could be an harware issue.. > > On the other side of the link, the other EX4300 (side B) do not experience > the same issue but the traffic is mostly from side B to side A. > > Here an example of the output, statistics cleared and after 1 minute I get > 12 framing errors with 2 Mbs running on the WAN link: > > @EX4300-A> show interfaces ae0 extensive > Physical interface: ae0, Enabled, Physical link is Up > Interface index: 220, SNMP
