Re: [c-nsp] TCAM utilization on Nexus 9396
James, Thanks for wonderful explanation.. now it’s much clear in my mind how to hand it better way, I had multiple subnets which I summarized and make it single line rule ;) thanks for that tip. Sent from my iPhone > On Mar 22, 2019, at 5:22 AM, James Bensley wrote: > >> On Wed, 20 Mar 2019 at 19:14, Satish Patel wrote: >> >> Thanks for clarification, i have noticed when i add 1 rules number >> bump +1 but i believe you can't go above 510 right? that is hard limit >> if i am not wrong. >> >> also changing in resource required reload. > > Hi Satish, > > I don't know this platform at all but, general rules for platforms with TCAMs: > > Whether the TCAM is at 1% utilisation or 99% there should be no impact > to traffic forwarding rate for the features that use the TCAM (e.g. > ACLs, QoS, SPAN). > > Yes you can sometimes fit more entries into TCAM than the stated number. > For example if in your config you have two entries in an ACL which are > contiguous e.g.: > 1. 192.168.0.0/24 > 2. 192.168.0.1/24 > These will often be aggregated into one single entry: 192.168.0.0/23 > However, I wouldn't rely on this. If you devices supports 512 ACLs and > you need 512, you should probably chose a difference device to allow > for future growth or adjust your ACL plan/design. > > It is generally OK to run a TCAM at 100%, for example if you have an > ACL that is 512 entries long and your TCAM can store 512 entries, this > is technically fine. However, as soon as you add the 513th entry chaos > may ensure. WIth ACLs as Tim said, the config often won't be applied > and an error presented on the CLI. With routing entries which are > dynamically learnt though and not explicitly configured, it is a > different story. If your routing space inside TCAM is full, when a > packet ingresses the device and there is no match for the destination > inside the TCAM the packet must be CPU punted to check the FIB in slow > memory. These slows will be very slow and experinace high packet drops > rates and your support desk phones will light up etc. > > Often you can TCAM paritioning to give more or less space to a certain > feature however, this always required a reboot to implement. > > Cheers, > James. > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
On Wed, 20 Mar 2019 at 19:14, Satish Patel wrote: > > Thanks for clarification, i have noticed when i add 1 rules number > bump +1 but i believe you can't go above 510 right? that is hard limit > if i am not wrong. > > also changing in resource required reload. Hi Satish, I don't know this platform at all but, general rules for platforms with TCAMs: Whether the TCAM is at 1% utilisation or 99% there should be no impact to traffic forwarding rate for the features that use the TCAM (e.g. ACLs, QoS, SPAN). Yes you can sometimes fit more entries into TCAM than the stated number. For example if in your config you have two entries in an ACL which are contiguous e.g.: 1. 192.168.0.0/24 2. 192.168.0.1/24 These will often be aggregated into one single entry: 192.168.0.0/23 However, I wouldn't rely on this. If you devices supports 512 ACLs and you need 512, you should probably chose a difference device to allow for future growth or adjust your ACL plan/design. It is generally OK to run a TCAM at 100%, for example if you have an ACL that is 512 entries long and your TCAM can store 512 entries, this is technically fine. However, as soon as you add the 513th entry chaos may ensure. WIth ACLs as Tim said, the config often won't be applied and an error presented on the CLI. With routing entries which are dynamically learnt though and not explicitly configured, it is a different story. If your routing space inside TCAM is full, when a packet ingresses the device and there is no match for the destination inside the TCAM the packet must be CPU punted to check the FIB in slow memory. These slows will be very slow and experinace high packet drops rates and your support desk phones will light up etc. Often you can TCAM paritioning to give more or less space to a certain feature however, this always required a reboot to implement. Cheers, James. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
--- Begin Message --- Please check the config guide. I am not as familiar w/the 1st gen switches as 2nd gen, but there should be at least some level of reconfigurability of the regions in gen 1. So you may be able to size up the region you want by removing entries from some other region. Yes, region resizing requires a switch reboot. Tim -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 12:12 PM To: Tim Stevenson (tstevens) Cc: Cisco Network Service Providers ; Nick Cutting Subject: Re: TCAM utilization on Nexus 9396 Thanks for clarification, i have noticed when i add 1 rules number bump +1 but i believe you can't go above 510 right? that is hard limit if i am not wrong. also changing in resource required reload. On Wed, Mar 20, 2019 at 2:07 PM Tim Stevenson (tstevens) wrote: > > Yes, ACL lines consume space in the TCAM. TCAM can be recarved according to > the features in use/required. > > As long as the policy fits in the available TCAM space for that feature > (software will complain and fail your config if it won't), enforcement is at > full rate, no performance penalty for that. > > Tim > > -Original Message- > From: Satish Patel > Sent: Wednesday, March 20, 2019 10:46 AM > To: Cisco Network Service Providers ; Nick Cutting > ; Tim Stevenson (tstevens) > Subject: TCAM utilization on Nexus 9396 > > Folks and ( Tim/Nick ) > > I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 > Access-list to block certain traffic ) today i was reading about TCAM > and when i look at switch i found following utilization, so trying to > understand how ACL relationship with TCAM. > > - Does number of ACL impact TCAM utilization or traffic ? > > > # show hardware access-list resource utilization > > slot 1 > === > > > > INSTANCE 0x0 > - > > > ACL Hardware Resource Utilization (Mod 1) > -- > UsedFreePercent > Utilization > --- > Ingress IPv4 PACL 3 509 0.59 > Ingress IPv4 Port QoS 4 252 1.56 > Ingress IPv4 VACL 2 510 0.39 > Ingress IPv4 RACL 226 286 44.14 > Egress IPv4 VACL3 509 0.59 > Egress IPv4 RACL3 253 1.17 > SUP COPP205 51 80.08 > SUP COPP Reason Code TCAM 6 122 4.69 > Redirect2 510 0.39 > SPAN21 235 8.20 > VPC Convergence 1 255 0.39 > > LOU 2 22 8.33 > Both LOU Operands 2 > Single LOU Operands 0 > LOU L4 src port:1 > LOU L4 dst port:1 > LOU L3 packet len: 0 > LOU IP tos: 0 > LOU IP dscp:0 > LOU ip precedence: 0 > LOU ip TTL: 0 > TCP Flags 0 16 0.00 > > Protocol CAM2 244 0.81 > Mac Etype/Proto CAM 0 14 0.00 > > L4 op labels, Tcam 00 10230.00 > L4 op labels, Tcam 21 62 1.58 > L4 op labels, Tcam 60 20470.00 > > Ingress Dest info table 0 512 0.00 > > Egress Dest info table 0 512 0.00 --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
Thanks for clarification, i have noticed when i add 1 rules number bump +1 but i believe you can't go above 510 right? that is hard limit if i am not wrong. also changing in resource required reload. On Wed, Mar 20, 2019 at 2:07 PM Tim Stevenson (tstevens) wrote: > > Yes, ACL lines consume space in the TCAM. TCAM can be recarved according to > the features in use/required. > > As long as the policy fits in the available TCAM space for that feature > (software will complain and fail your config if it won't), enforcement is at > full rate, no performance penalty for that. > > Tim > > -Original Message- > From: Satish Patel > Sent: Wednesday, March 20, 2019 10:46 AM > To: Cisco Network Service Providers ; Nick Cutting > ; Tim Stevenson (tstevens) > Subject: TCAM utilization on Nexus 9396 > > Folks and ( Tim/Nick ) > > I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 > Access-list to block certain traffic ) today i was reading about TCAM > and when i look at switch i found following utilization, so trying to > understand how ACL relationship with TCAM. > > - Does number of ACL impact TCAM utilization or traffic ? > > > # show hardware access-list resource utilization > > slot 1 > === > > > > INSTANCE 0x0 > - > > > ACL Hardware Resource Utilization (Mod 1) > -- > UsedFreePercent > Utilization > --- > Ingress IPv4 PACL 3 509 0.59 > Ingress IPv4 Port QoS 4 252 1.56 > Ingress IPv4 VACL 2 510 0.39 > Ingress IPv4 RACL 226 286 44.14 > Egress IPv4 VACL3 509 0.59 > Egress IPv4 RACL3 253 1.17 > SUP COPP205 51 80.08 > SUP COPP Reason Code TCAM 6 122 4.69 > Redirect2 510 0.39 > SPAN21 235 8.20 > VPC Convergence 1 255 0.39 > > LOU 2 22 8.33 > Both LOU Operands 2 > Single LOU Operands 0 > LOU L4 src port:1 > LOU L4 dst port:1 > LOU L3 packet len: 0 > LOU IP tos: 0 > LOU IP dscp:0 > LOU ip precedence: 0 > LOU ip TTL: 0 > TCP Flags 0 16 0.00 > > Protocol CAM2 244 0.81 > Mac Etype/Proto CAM 0 14 0.00 > > L4 op labels, Tcam 00 10230.00 > L4 op labels, Tcam 21 62 1.58 > L4 op labels, Tcam 60 20470.00 > > Ingress Dest info table 0 512 0.00 > > Egress Dest info table 0 512 0.00 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] TCAM utilization on Nexus 9396
--- Begin Message --- Yes, ACL lines consume space in the TCAM. TCAM can be recarved according to the features in use/required. As long as the policy fits in the available TCAM space for that feature (software will complain and fail your config if it won't), enforcement is at full rate, no performance penalty for that. Tim -Original Message- From: Satish Patel Sent: Wednesday, March 20, 2019 10:46 AM To: Cisco Network Service Providers ; Nick Cutting ; Tim Stevenson (tstevens) Subject: TCAM utilization on Nexus 9396 Folks and ( Tim/Nick ) I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 Access-list to block certain traffic ) today i was reading about TCAM and when i look at switch i found following utilization, so trying to understand how ACL relationship with TCAM. - Does number of ACL impact TCAM utilization or traffic ? # show hardware access-list resource utilization slot 1 === INSTANCE 0x0 - ACL Hardware Resource Utilization (Mod 1) -- UsedFreePercent Utilization --- Ingress IPv4 PACL 3 509 0.59 Ingress IPv4 Port QoS 4 252 1.56 Ingress IPv4 VACL 2 510 0.39 Ingress IPv4 RACL 226 286 44.14 Egress IPv4 VACL3 509 0.59 Egress IPv4 RACL3 253 1.17 SUP COPP205 51 80.08 SUP COPP Reason Code TCAM 6 122 4.69 Redirect2 510 0.39 SPAN21 235 8.20 VPC Convergence 1 255 0.39 LOU 2 22 8.33 Both LOU Operands 2 Single LOU Operands 0 LOU L4 src port:1 LOU L4 dst port:1 LOU L3 packet len: 0 LOU IP tos: 0 LOU IP dscp:0 LOU ip precedence: 0 LOU ip TTL: 0 TCP Flags 0 16 0.00 Protocol CAM2 244 0.81 Mac Etype/Proto CAM 0 14 0.00 L4 op labels, Tcam 00 10230.00 L4 op labels, Tcam 21 62 1.58 L4 op labels, Tcam 60 20470.00 Ingress Dest info table 0 512 0.00 Egress Dest info table 0 512 0.00 --- End Message --- ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] TCAM utilization on Nexus 9396
Folks and ( Tim/Nick ) I have Cisco Nexus 9396 L3 switch and running bunch of ACL ( IPv4 Access-list to block certain traffic ) today i was reading about TCAM and when i look at switch i found following utilization, so trying to understand how ACL relationship with TCAM. - Does number of ACL impact TCAM utilization or traffic ? # show hardware access-list resource utilization slot 1 === INSTANCE 0x0 - ACL Hardware Resource Utilization (Mod 1) -- UsedFreePercent Utilization --- Ingress IPv4 PACL 3 509 0.59 Ingress IPv4 Port QoS 4 252 1.56 Ingress IPv4 VACL 2 510 0.39 Ingress IPv4 RACL 226 286 44.14 Egress IPv4 VACL3 509 0.59 Egress IPv4 RACL3 253 1.17 SUP COPP205 51 80.08 SUP COPP Reason Code TCAM 6 122 4.69 Redirect2 510 0.39 SPAN21 235 8.20 VPC Convergence 1 255 0.39 LOU 2 22 8.33 Both LOU Operands 2 Single LOU Operands 0 LOU L4 src port:1 LOU L4 dst port:1 LOU L3 packet len: 0 LOU IP tos: 0 LOU IP dscp:0 LOU ip precedence: 0 LOU ip TTL: 0 TCP Flags 0 16 0.00 Protocol CAM2 244 0.81 Mac Etype/Proto CAM 0 14 0.00 L4 op labels, Tcam 00 10230.00 L4 op labels, Tcam 21 62 1.58 L4 op labels, Tcam 60 20470.00 Ingress Dest info table 0 512 0.00 Egress Dest info table 0 512 0.00 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
