Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
> On 22 Oct 2015, at 17:42, Gert Doering wrote: > > Hi, > > On Tue, Oct 20, 2015 at 09:20:43PM +0200, ??ukasz Bromirski wrote: > [..] >>> you're in trouble; >> Well, not exactly. > [..] >> In other words - you???re safe, the box won???t melt, but the situation >> will require fixing & reload. > > Well, "require reload" definitely smells like "in trouble", no? ;-) Sure, but just look at the bright side - after couple of years we finally managed to get mls rate-limiter protection. Before it was node suddenly vanishing from the network. An edge node vanishing may be trouble squared ;P -- Łukasz Bromirski, luk...@bromirski.net CCIE R&S/SP #15929, CCDE #2012::17 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
Hi, On Tue, Oct 20, 2015 at 09:20:43PM +0200, ??ukasz Bromirski wrote: [..] > > you're in trouble; > Well, not exactly. [..] > In other words - you???re safe, the box won???t melt, but the situation > will require fixing & reload. Well, "require reload" definitely smells like "in trouble", no? ;-) gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de signature.asc Description: PGP signature ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
> On 20 Oct 2015, at 10:55, James Bensley wrote: > > I will probably aim for 60k IPv6 routes, so it's enough to phase out > the boxes and that's it. Be careful that these boxes will start to CPU > switch packets before you run out of TACM. When you see these logs > you're in trouble; Well, not exactly. Last I remember, it was changed in 12.2(33)SXH - when the PFC hits exception on TCAM, it’ll switch “exception” packets (packets to destination that’s outside of known TCAM programmed entries) with a mls hardware-limiter set to 10kpps. In other words - you’re safe, the box won’t melt, but the situation will require fixing & reload. -- Łukasz Bromirski, luk...@bromirski.net CCIE R&S/SP #15929, CCDE #2012::17 ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
We currently have 560163 routes in Internet VRF for IPv4, with 120K available for IPv6, My BGP process is taking ~456MB memory. Router is running fairly new IOS15.2(4)S4a. We only import default routes per VRF per PE for IPv4 and same plan for IPv6. any future scale limit? Krunal On Tue, Oct 20, 2015 at 11:42 AM, Pete Templin wrote: > Just a tiny tidbit related to TCAM reallocation, make sure the SP > bootvar's config register matches the RP bootvar's config register. In > tech-speak, 'sh bootv | i eg' should match 'rem com sw sh bootv | i eg'. If > it doesn't, "conf t; config-register 0x2142; end; conf t; config-register > 0x2102; end; copy run start" and recheck. A mismatch in how the SP > pre-configures itself is immaterial for the basics of IOS configuration > stuff, but fatal with respect to TCAM; the box will forcibly reload after 5 > minutes endlessly until fixed. > > > On 10/20/2015 1:55 AM, James Bensley wrote: > >> On 14 October 2015 at 13:32, krunal shah wrote: >> >>> hi NSPs, >>> >>> Is any one doing 6VPE and importing full IPv4 and IPv6 routes in same >>> VRF? >>> >>> I am planning to implement full IPv6 and IPv4 routes in same VRF that is >>> used for internet service and other PE routers would only get subset or >>> default routes. Suip has 4 GB of memory, My 1M TCAM space is carved with >>> >>> FIB TCAM maximum routes : >>> === >>> Current :- >>> --- >>> IPv4- 768k >>> MPLS- 16k (default) >>> IPv6 + IP Multicast - 120k (default) >>> >> >> We are not doing this on any 7600 's but will be shortly so we'll >> start testing soon. Our 7600's are being bumped up to 15.3(3)S6, all >> running with RSP720-3XCL-10GE's and the TCAMs will be repartitioned, >> this will then hopefully see them out until they are decomissioned. >> >> The TCAMs need reallocating as they are currently carrying a lot of >> VPNv4 routes as well as the full IPv4 table and 6VPE was not >> envisioned, however your partitioning of the TCAM seems a litte too >> favourable for IPv6 for our needs; >> >> FIB TCAM maximum routes : >> === >> Current :- >> --- >> IPv4 + MPLS - 960k (default) >> IPv6- 16k >> IP Multicast- 16k >> >> FIB TCAM usage: TotalUsed %Used >> 72 bits (IPv4, MPLS, EoM) 983040 658740 67% >> >> I will probably aim for 60k IPv6 routes, so it's enough to phase out >> the boxes and that's it. Be careful that these boxes will start to CPU >> switch packets before you run out of TACM. When you see these logs >> you're in trouble; >> >> %MLSCEF-SP-4-FIB_EXCEPTION_THRESHOLD: Hardware CEF entry >> usage is at 95% capacity for IPv4 unicast protocol >> >> %MLSCEF-DFC4-7-FIB_EXCEPTION: FIB TCAM exception, Some >> entries will be software switched >> >> %MLSCEF-SP-7-FIB_EXCEPTION: FIB TCAM exception, Some >> entries will be software switched >> >> >> Cheers, >> James. >> ___ >> cisco-nsp mailing list cisco-nsp@puck.nether.net >> https://puck.nether.net/mailman/listinfo/cisco-nsp >> archive at http://puck.nether.net/pipermail/cisco-nsp/ >> >> > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ > ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
Just a tiny tidbit related to TCAM reallocation, make sure the SP bootvar's config register matches the RP bootvar's config register. In tech-speak, 'sh bootv | i eg' should match 'rem com sw sh bootv | i eg'. If it doesn't, "conf t; config-register 0x2142; end; conf t; config-register 0x2102; end; copy run start" and recheck. A mismatch in how the SP pre-configures itself is immaterial for the basics of IOS configuration stuff, but fatal with respect to TCAM; the box will forcibly reload after 5 minutes endlessly until fixed. On 10/20/2015 1:55 AM, James Bensley wrote: On 14 October 2015 at 13:32, krunal shah wrote: hi NSPs, Is any one doing 6VPE and importing full IPv4 and IPv6 routes in same VRF? I am planning to implement full IPv6 and IPv4 routes in same VRF that is used for internet service and other PE routers would only get subset or default routes. Suip has 4 GB of memory, My 1M TCAM space is carved with FIB TCAM maximum routes : === Current :- --- IPv4- 768k MPLS- 16k (default) IPv6 + IP Multicast - 120k (default) We are not doing this on any 7600 's but will be shortly so we'll start testing soon. Our 7600's are being bumped up to 15.3(3)S6, all running with RSP720-3XCL-10GE's and the TCAMs will be repartitioned, this will then hopefully see them out until they are decomissioned. The TCAMs need reallocating as they are currently carrying a lot of VPNv4 routes as well as the full IPv4 table and 6VPE was not envisioned, however your partitioning of the TCAM seems a litte too favourable for IPv6 for our needs; FIB TCAM maximum routes : === Current :- --- IPv4 + MPLS - 960k (default) IPv6- 16k IP Multicast- 16k FIB TCAM usage: TotalUsed %Used 72 bits (IPv4, MPLS, EoM) 983040 658740 67% I will probably aim for 60k IPv6 routes, so it's enough to phase out the boxes and that's it. Be careful that these boxes will start to CPU switch packets before you run out of TACM. When you see these logs you're in trouble; %MLSCEF-SP-4-FIB_EXCEPTION_THRESHOLD: Hardware CEF entry usage is at 95% capacity for IPv4 unicast protocol %MLSCEF-DFC4-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched %MLSCEF-SP-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched Cheers, James. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
On 20 October 2015 at 09:55, James Bensley wrote: > I will probably aim for 60k IPv6 routes 2 byte boundary, should have said 64k routes, doh! James. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
On 14 October 2015 at 13:32, krunal shah wrote: > hi NSPs, > > Is any one doing 6VPE and importing full IPv4 and IPv6 routes in same VRF? > > I am planning to implement full IPv6 and IPv4 routes in same VRF that is > used for internet service and other PE routers would only get subset or > default routes. Suip has 4 GB of memory, My 1M TCAM space is carved with > > FIB TCAM maximum routes : > === > Current :- > --- > IPv4- 768k > MPLS- 16k (default) > IPv6 + IP Multicast - 120k (default) We are not doing this on any 7600 's but will be shortly so we'll start testing soon. Our 7600's are being bumped up to 15.3(3)S6, all running with RSP720-3XCL-10GE's and the TCAMs will be repartitioned, this will then hopefully see them out until they are decomissioned. The TCAMs need reallocating as they are currently carrying a lot of VPNv4 routes as well as the full IPv4 table and 6VPE was not envisioned, however your partitioning of the TCAM seems a litte too favourable for IPv6 for our needs; FIB TCAM maximum routes : === Current :- --- IPv4 + MPLS - 960k (default) IPv6- 16k IP Multicast- 16k FIB TCAM usage: TotalUsed %Used 72 bits (IPv4, MPLS, EoM) 983040 658740 67% I will probably aim for 60k IPv6 routes, so it's enough to phase out the boxes and that's it. Be careful that these boxes will start to CPU switch packets before you run out of TACM. When you see these logs you're in trouble; %MLSCEF-SP-4-FIB_EXCEPTION_THRESHOLD: Hardware CEF entry usage is at 95% capacity for IPv4 unicast protocol %MLSCEF-DFC4-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched %MLSCEF-SP-7-FIB_EXCEPTION: FIB TCAM exception, Some entries will be software switched Cheers, James. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] 6VPE on 7600 RSP720 3CXL
On 14/Oct/15 14:32, krunal shah wrote: > hi NSPs, > > Is any one doing 6VPE and importing full IPv4 and IPv6 routes in same VRF? > > I am planning to implement full IPv6 and IPv4 routes in same VRF that is > used for internet service and other PE routers would only get subset or > default routes. Suip has 4 GB of memory, My 1M TCAM space is carved with > > FIB TCAM maximum routes : > === > Current :- > --- > IPv4- 768k > MPLS- 16k (default) > IPv6 + IP Multicast - 120k (default) > > > Just want to make sure that this can be achieved. We are doing 6VPE on all our Cisco and Juniper gear in the same VRF, but not the full Internet table. Mark. ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] 6VPE on 7600 RSP720 3CXL
hi NSPs, Is any one doing 6VPE and importing full IPv4 and IPv6 routes in same VRF? I am planning to implement full IPv6 and IPv4 routes in same VRF that is used for internet service and other PE routers would only get subset or default routes. Suip has 4 GB of memory, My 1M TCAM space is carved with FIB TCAM maximum routes : === Current :- --- IPv4- 768k MPLS- 16k (default) IPv6 + IP Multicast - 120k (default) Just want to make sure that this can be achieved. Krunal ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
