Re: [j-nsp] Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
Hey, Can you please provide - show filter dram - show filter hw X - show filter hw X show_term_info I lost a fight with JTAC about whether the TCAM exhausting filter should be a commit failure or not. Argument was along the line 'well you can keep adding routes even if you exhaust TCAM, so this should be the same'. I'm absolutely certain there are many QFX and EX networks out there with wildly different filters programmed than what they believe they have. On Wed, 12 Oct 2022 at 05:33, Chuck Anderson via juniper-nsp wrote: > > Has anyone seen these errors and know what the cause is? > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-624-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-626-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-631-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-632-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-632-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-633-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-633-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-634-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-634-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-638-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-638-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-647-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-647-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-656-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-656-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-657-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-657-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-655-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-652-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-652-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-653-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-653-5-1" is NOT programmed in HW > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter > pfe-cos-cl-654-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : > "pfe-cos-cl-654-5-1" is NOT programmed in HW > > There is plenty of TCAM space for IRACL/IPACL entries, so this seems to be > some issue with a different TCAM partition? > > ex4300-48mp> show pfe filter hw summary > > Slot 0 > > Unit:0: > GroupGroup-ID Allocated Used Free > --- > > Ingress filter groups: > iRACL group33 2048 1148 900 > iPACL group25 51212 500 > > Egress filter groups: > > Slot 1 > > Unit:0: > GroupGroup-ID Allocated Used Free > --- > > Ingress filter groups: > iRACL group33 2048 1148 900 > iPACL group25 51212 500 > > Egress filter groups: > > ___ > juniper-nsp mailing list juniper-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp -- ++ytti ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
Has anyone seen these errors and know what the cause is? Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-624-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-626-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-631-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-632-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-632-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-633-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-633-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-634-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-634-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-638-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-638-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-647-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-647-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-656-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-656-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-657-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-657-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-655-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-652-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-652-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-653-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-653-5-1" is NOT programmed in HW Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-654-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-654-5-1" is NOT programmed in HW There is plenty of TCAM space for IRACL/IPACL entries, so this seems to be some issue with a different TCAM partition? ex4300-48mp> show pfe filter hw summary Slot 0 Unit:0: GroupGroup-ID Allocated Used Free --- > Ingress filter groups: iRACL group33 2048 1148 900 iPACL group25 51212 500 > Egress filter groups: Slot 1 Unit:0: GroupGroup-ID Allocated Used Free --- > Ingress filter groups: iRACL group33 2048 1148 900 iPACL group25 51212 500 > Egress filter groups: ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] port-mirror with source inside routing-instance type vrf
Chuck,
Thanks for the suggestion. I have tried it at least four ways; both with and
without the static-arp entry and with egress interface in global and egress
interface in VRF. When I tried without static-arp, I forced mirror up with a
ping from our mirroring device. My fw counters imply > 100pps hitting the
relevant firewall "then" clause.
@re0# run show forwarding-options port-mirroring
Oct 11 11:00:33
Instance Name: uwwhitewater
Instance Id: 3
Input parameters:
Rate : 1
Run-length: 0
Maximum-packet-length : 0
Output parameters:
Family State Destination Next-hop
inetupxe-0/0/4:2.3124 10.235.43.1
-Michael
> -Original Message-
> From: juniper-nsp On Behalf Of
> Chuck Anderson via juniper-nsp
> Sent: Tuesday, October 11, 2022 10:59 AM
> To: juniper-nsp@puck.nether.net
> Subject: Re: [j-nsp] port-mirror with source inside routing-instance type vrf
>
> Did you try creating a static ARP entry for the port mirroring destination?
>
> interfaces {
> xe-0/0/4:2 {
> vlan-tagging;
> mtu 9192;
> encapsulation flexible-ethernet-services;
> unit 3124 {
> description "mirror test";
> vlan-id 3124;
> family inet {
> no-redirects;
> no-neighbor-learn;
> address 10.235.43.0/31 {
> arp 10.235.43.1 mac 02:02:02:02:02:02;
> }
> }
> }
> }
> }
>
> On Tue, Oct 11, 2022 at 02:37:47PM +, Michael Hare via juniper-nsp
> wrote:
> > show interfaces xe-0/0/4:2 | no-more
> > enable;
> > vlan-tagging;
> > mtu 9192;
> > encapsulation flexible-ethernet-services;
> > ...
> > ...
> > unit 3124 {
> > description "mirror test";
> > vlan-id 3124;
> > family inet {
> > address 10.235.43.0/31;
> > }
> > }
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] port-mirror with source inside routing-instance type vrf
Did you try creating a static ARP entry for the port mirroring destination?
interfaces {
xe-0/0/4:2 {
vlan-tagging;
mtu 9192;
encapsulation flexible-ethernet-services;
unit 3124 {
description "mirror test";
vlan-id 3124;
family inet {
no-redirects;
no-neighbor-learn;
address 10.235.43.0/31 {
arp 10.235.43.1 mac 02:02:02:02:02:02;
}
}
}
}
}
On Tue, Oct 11, 2022 at 02:37:47PM +, Michael Hare via juniper-nsp wrote:
> show interfaces xe-0/0/4:2 | no-more
> enable;
> vlan-tagging;
> mtu 9192;
> encapsulation flexible-ethernet-services;
> ...
> ...
> unit 3124 {
> description "mirror test";
> vlan-id 3124;
> family inet {
> address 10.235.43.0/31;
> }
> }
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
[j-nsp] port-mirror with source inside routing-instance type vrf
Hello,
Cluebats appreciated, I can contact JTAC on this but am trying to avoid the
timesink of opening a case.
Topic is filter based port mirroring for family inet with the wrinkle being
that I'm trying to mirror traffic from inside "instance-type vrf". I've done
this countless times before successfully [including today as a sanity check]
with source being in global table.
So far I've tried putting the output interface both inside the same VRF and in
global; no traffic seems to mirror. What is the correct stance? Yes, I've
tried to prime the macaddr pump with ICMP from the mx10003 doing the mirroring.
I am aware of mirroring "family any" but am unsure if that applies here, as the
source interface I am trying to mirror is edge of VRF and doesn't have family
mpls on the logical interface of interest.
I'm confident the traffic I want to mirror is hitting my filter term based on
incrementing counters.
Lightly sanitized config below.
# I confirmed this is attached to the interface of question and counters are
incrementing.
term mirror-2 {
then {
count :mirror:all;
port-mirror-instance uw;
next term;
}
}
show forwarding-options
port-mirroring {
instance {
uw {
input {
rate 1;
}
family inet {
output {
interface xe-0/0/4:2.3124 {
next-hop 10.235.43.1;
}
}
}
}
}
}
show chassis
fpc 0 {
...
port-mirror-instance uw;
sampling-instance ins1;
}
show interfaces xe-0/0/4:2 | no-more
enable;
vlan-tagging;
mtu 9192;
encapsulation flexible-ethernet-services;
...
...
unit 3124 {
description "mirror test";
vlan-id 3124;
family inet {
address 10.235.43.0/31;
}
}
and then I've put xe-0/0/4:2.3124 inside and outside the relevant
routing-instance as tests.
-Michael
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
