Re: [j-nsp] jtree0 Memory full on MX480?
So the SCB itself is only responsible for the available bandwidth per slot but is not and will never be a memory limitation? Correct on all points. - CK. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Thank's for clearification, that helps. So the SCB itself is only responsible for the available bandwidth per slot but is not and will never be a memory limitation? Best, Jeff Am 22.07.2015 um 23:51 schrieb Chris Kawchuk: On 23/07/2015, at 1:30 AM, Jeff Meyers jeff.mey...@gmx.net wrote: yes, we did (at least since yesterday) although we are not really requiring more ports or bandwidth right now. If I understand that correctly, I need to upgrade to SCB2 as well? nope -- no need to go to MPC+SCB2 combo. original SCBs work fine with MPC1 MPC2.2E, etc.. albeit limited to 120G/slot only, which is fine as MPC2 can only do 80G anyways in/out ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
On 23/Jul/15 15:54, Jeff Meyers wrote: Thank's for clearification, that helps. So the SCB itself is only responsible for the available bandwidth per slot but is not and will never be a memory limitation? That's right. The SCB provides inter-slot bandwidth. It is not impacted by FIB memory. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Hi, I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. yes, we did (at least since yesterday) although we are not really requiring more ports or bandwidth right now. If I understand that correctly, I need to upgrade to SCB2 as well? It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) I agree :) ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Hi, The size of the firewall configuration could be concern if you use the box for subscriber management and have tons of dynamic interface with filters attached. Otherwise you should be safe to use that knob. At the moment you have 11.5MB in segment 1, when you enable that know it will go down to 6MB or 7MB. And you will be able to accommodate two full feeds in the FIB. 11650408 bytes available (11609600 bytes from free pages) I would recommend to check for any know PR using that feature with 11.4 Ivan, On Wed, Jul 22, 2015 at 4:31 PM, Jeff Meyers jeff.mey...@gmx.net wrote: Hi, thanks for the hint, didn't know about that option. This will certainly safe us if we are running in to limits. We don't have too many filters, mostly the basic stuff to protect the RE and a few filters on some vlans with basic white- and/or blacklisting. So really nothing fance although I have no idea mow many many filters are on a box like the MX. Is it 1,000 terms or more like 1,000,000 terms? Best, Jeff Am 22.07.2015 um 13:06 schrieb Ivan Ivanov: Hi, The 'route' option on 'memory-enhanced' will give you some time before upgrade to MPC. Actually you should be okay for quite a long time considering the size of the table you have at the moment. https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/junos-software-jtree-memory-repartitioning.html https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/junos-software-jtree-memory-repartitioning.html Note, that this will use the part of the memory reserved for filters (Jtree segment 1) for storing route information. You that feature only if don't have many filters configured. Ivan, On Wed, Jul 22, 2015 at 7:52 AM, Mark Tinka mark.ti...@seacom.mu mailto:mark.ti...@seacom.mu wrote: On 22/Jul/15 02:59, Chris Kawchuk wrote: I know that a ton of fixes on BGP convergence time son MX80 is definitely a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be applicable. I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. For things like Fusion or BNG or any other new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any new device we turn up, and have already started testing 15.1 for the new NG cards we will likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), etc... preventing starvation if you happen to have re-used Queue 3 as not-so-high priority, etc)... the list goes on. It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) We've been on 14.2 for a while now, and settling into 14.2R3.8. Happy, to be honest. Only real problem is policing on LAG's, but it's manageable. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net mailto:juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp -- Best Regards! Ivan Ivanov -- Best Regards! Ivan Ivanov ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
On 22/Jul/15 17:30, Jeff Meyers wrote: yes, we did (at least since yesterday) although we are not really requiring more ports or bandwidth right now. If I understand that correctly, I need to upgrade to SCB2 as well? Hehehehe, where have I heard that before :-)? IPv4 BGP table havin' us runnin' around like headless chicken :-)... Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Hi, thanks for the hint, didn't know about that option. This will certainly safe us if we are running in to limits. We don't have too many filters, mostly the basic stuff to protect the RE and a few filters on some vlans with basic white- and/or blacklisting. So really nothing fance although I have no idea mow many many filters are on a box like the MX. Is it 1,000 terms or more like 1,000,000 terms? Best, Jeff Am 22.07.2015 um 13:06 schrieb Ivan Ivanov: Hi, The 'route' option on 'memory-enhanced' will give you some time before upgrade to MPC. Actually you should be okay for quite a long time considering the size of the table you have at the moment. https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/junos-software-jtree-memory-repartitioning.html https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/junos-software-jtree-memory-repartitioning.html Note, that this will use the part of the memory reserved for filters (Jtree segment 1) for storing route information. You that feature only if don't have many filters configured. Ivan, On Wed, Jul 22, 2015 at 7:52 AM, Mark Tinka mark.ti...@seacom.mu mailto:mark.ti...@seacom.mu wrote: On 22/Jul/15 02:59, Chris Kawchuk wrote: I know that a ton of fixes on BGP convergence time son MX80 is definitely a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be applicable. I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. For things like Fusion or BNG or any other new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any new device we turn up, and have already started testing 15.1 for the new NG cards we will likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), etc... preventing starvation if you happen to have re-used Queue 3 as not-so-high priority, etc)... the list goes on. It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) We've been on 14.2 for a while now, and settling into 14.2R3.8. Happy, to be honest. Only real problem is policing on LAG's, but it's manageable. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net mailto:juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp -- Best Regards! Ivan Ivanov ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
On 22/Jul/15 02:59, Chris Kawchuk wrote: I know that a ton of fixes on BGP convergence time son MX80 is definitely a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be applicable. I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. For things like Fusion or BNG or any other new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any new device we turn up, and have already started testing 15.1 for the new NG cards we will likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), etc... preventing starvation if you happen to have re-used Queue 3 as not-so-high priority, etc)... the list goes on. It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) We've been on 14.2 for a while now, and settling into 14.2R3.8. Happy, to be honest. Only real problem is policing on LAG's, but it's manageable. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Hi, The 'route' option on 'memory-enhanced' will give you some time before upgrade to MPC. Actually you should be okay for quite a long time considering the size of the table you have at the moment. https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/ junos-software-jtree-memory-repartitioning.html Note, that this will use the part of the memory reserved for filters (Jtree segment 1) for storing route information. You that feature only if don't have many filters configured. Ivan, On Wed, Jul 22, 2015 at 7:52 AM, Mark Tinka mark.ti...@seacom.mu wrote: On 22/Jul/15 02:59, Chris Kawchuk wrote: I know that a ton of fixes on BGP convergence time son MX80 is definitely a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be applicable. I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. For things like Fusion or BNG or any other new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any new device we turn up, and have already started testing 15.1 for the new NG cards we will likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), etc... preventing starvation if you happen to have re-used Queue 3 as not-so-high priority, etc)... the list goes on. It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) We've been on 14.2 for a while now, and settling into 14.2R3.8. Happy, to be honest. Only real problem is policing on LAG's, but it's manageable. Mark. ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp -- Best Regards! Ivan Ivanov ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
On 23/07/2015, at 1:30 AM, Jeff Meyers jeff.mey...@gmx.net wrote: yes, we did (at least since yesterday) although we are not really requiring more ports or bandwidth right now. If I understand that correctly, I need to upgrade to SCB2 as well? nope -- no need to go to MPC+SCB2 combo. original SCBs work fine with MPC1 MPC2.2E, etc.. albeit limited to 120G/slot only, which is fine as MPC2 can only do 80G anyways in/out ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Over the years, we have run into a couple of issues that translated to either
exhausting FPC memory or corrupting the JTree. Currently, life is good on
13.3R6, which we run on all MX's globally. I haven't run into this specific
issue, and I am just assuming that behavior is improved.
Best Regards,
-Phil
On Jul 21, 2015, at 8:49 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hi,
yes, an upgrade is absolutely possible but since there are no major issues
with that release, we didn't do that yet. Are you just assuming a newer
software improves that or did Juniper really do something on that side?
Best,
Jeff
Am 22.07.2015 um 02:45 schrieb Phil Rosenthal:
Disabling Basic-Table certainly bought you some time.
Agree that it still does not look good. I suspect that you are running into
a software issue. 11.4 is no longer a supported version, 12.3 is the
minimum supported today, with 13.3R6 as the recommended version. Is it
possible for you to upgrade?
Best Regards,
-Phil
On Jul 21, 2015, at 7:23 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hi Phil,
sure:
{master}
jeff@cr0 show configuration | display set | match rpf-check
{master}
nico@FRA4.cr0 show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package
[11.4R9.4]
JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]
{master}
nico@FRA4.cr0 show route summary
Autonomous system number: X
Router ID: A.B.C.D
inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown,
2994 hidden)
Direct: 1143 routes, 1140 active
Local: 1144 routes, 1144 active
OSPF: 81 routes, 18 active
BGP: 1745429 routes, 542631 active
Static:100 routes, 95 active
IGMP: 1 routes, 1 active
Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5
holddown, 0 hidden)
Direct: 2283 routes, 1140 active
Local: 2288 routes, 1144 active
OSPF: 17 routes, 17 active
BGP: 210387 routes, 210382 active
Static: 95 routes, 95 active
inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113
hidden)
Direct:451 routes,368 active
Local:373 routes,373 active
OSPF3: 9 routes, 9 active
BGP: 38399 routes, 22571 active
Static: 10 routes, 9 active
Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3
holddown, 0 hidden)
Direct:366 routes,366 active
Local:373 routes,373 active
OSPF3: 8 routes, 8 active
BGP: 11539 routes, 11536 active
Static: 9 routes, 9 active
{master}
I actually thought this Basic-Table was inactive. It is not so I'm going
to deactive it now. Since it was holding 200k routes, this is for sure a
lot. Doing that made the syslog message disappear but it didn't actually
free up as much as I was hoping for:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT: 15192 bytes unusable
GOT: 32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT: 24739 pages partially used
GOT:1691 pages free (max contiguous = 380)
Still doesn't look to glorious, right?
Best,
Jeff
Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:
Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum
DPC should have enough memory for ~1M FIB. This can get divided in half
if you
Re: [j-nsp] jtree0 Memory full on MX480?
Disabling Basic-Table certainly bought you some time.
Agree that it still does not look good. I suspect that you are running into a
software issue. 11.4 is no longer a supported version, 12.3 is the minimum
supported today, with 13.3R6 as the recommended version. Is it possible for
you to upgrade?
Best Regards,
-Phil
On Jul 21, 2015, at 7:23 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hi Phil,
sure:
{master}
jeff@cr0 show configuration | display set | match rpf-check
{master}
nico@FRA4.cr0 show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package
[11.4R9.4]
JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]
{master}
nico@FRA4.cr0 show route summary
Autonomous system number: X
Router ID: A.B.C.D
inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown, 2994
hidden)
Direct: 1143 routes, 1140 active
Local: 1144 routes, 1144 active
OSPF: 81 routes, 18 active
BGP: 1745429 routes, 542631 active
Static:100 routes, 95 active
IGMP: 1 routes, 1 active
Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5
holddown, 0 hidden)
Direct: 2283 routes, 1140 active
Local: 2288 routes, 1144 active
OSPF: 17 routes, 17 active
BGP: 210387 routes, 210382 active
Static: 95 routes, 95 active
inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113
hidden)
Direct:451 routes,368 active
Local:373 routes,373 active
OSPF3: 9 routes, 9 active
BGP: 38399 routes, 22571 active
Static: 10 routes, 9 active
Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3
holddown, 0 hidden)
Direct:366 routes,366 active
Local:373 routes,373 active
OSPF3: 8 routes, 8 active
BGP: 11539 routes, 11536 active
Static: 9 routes, 9 active
{master}
I actually thought this Basic-Table was inactive. It is not so I'm going to
deactive it now. Since it was holding 200k routes, this is for sure a lot.
Doing that made the syslog message disappear but it didn't actually free up
as much as I was hoping for:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT: 15192 bytes unusable
GOT: 32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT: 24739 pages partially used
GOT:1691 pages free (max contiguous = 380)
Still doesn't look to glorious, right?
Best,
Jeff
Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:
Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum
DPC should have enough memory for ~1M FIB. This can get divided in half if
you are using RPF. If you have multiple routing instances, this also can
contribute to the problem.
Best Regards,
-Phil Rosenthal
On Jul 21, 2015, at 6:56 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hello list,
we seem to be running into limits with a MX480 with RE-2000 and 2x
DPCE-4XGE-R since we are seeing these new messages in the syslog:
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree
Instance:jtree0-seg0 Type:free-dwords Available:83072 is less than LWM
limit:104857, rsmon_syslog_limit()
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree
Instance:jtree1-seg0 Type:free-pages Available:1326 is less than LWM
limit:1638,
Re: [j-nsp] jtree0 Memory full on MX480?
Hi,
yes, an upgrade is absolutely possible but since there are no major
issues with that release, we didn't do that yet. Are you just assuming a
newer software improves that or did Juniper really do something on that
side?
Best,
Jeff
Am 22.07.2015 um 02:45 schrieb Phil Rosenthal:
Disabling Basic-Table certainly bought you some time.
Agree that it still does not look good. I suspect that you are running into a
software issue. 11.4 is no longer a supported version, 12.3 is the minimum
supported today, with 13.3R6 as the recommended version. Is it possible for
you to upgrade?
Best Regards,
-Phil
On Jul 21, 2015, at 7:23 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hi Phil,
sure:
{master}
jeff@cr0 show configuration | display set | match rpf-check
{master}
nico@FRA4.cr0 show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package [11.4R9.4]
JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]
{master}
nico@FRA4.cr0 show route summary
Autonomous system number: X
Router ID: A.B.C.D
inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown, 2994
hidden)
Direct: 1143 routes, 1140 active
Local: 1144 routes, 1144 active
OSPF: 81 routes, 18 active
BGP: 1745429 routes, 542631 active
Static:100 routes, 95 active
IGMP: 1 routes, 1 active
Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5
holddown, 0 hidden)
Direct: 2283 routes, 1140 active
Local: 2288 routes, 1144 active
OSPF: 17 routes, 17 active
BGP: 210387 routes, 210382 active
Static: 95 routes, 95 active
inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113 hidden)
Direct:451 routes,368 active
Local:373 routes,373 active
OSPF3: 9 routes, 9 active
BGP: 38399 routes, 22571 active
Static: 10 routes, 9 active
Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3
holddown, 0 hidden)
Direct:366 routes,366 active
Local:373 routes,373 active
OSPF3: 8 routes, 8 active
BGP: 11539 routes, 11536 active
Static: 9 routes, 9 active
{master}
I actually thought this Basic-Table was inactive. It is not so I'm going to
deactive it now. Since it was holding 200k routes, this is for sure a lot. Doing that
made the syslog message disappear but it didn't actually free up as much as I was hoping for:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT: 15192 bytes unusable
GOT: 32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT: 24739 pages partially used
GOT:1691 pages free (max contiguous = 380)
Still doesn't look to glorious, right?
Best,
Jeff
Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:
Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum
DPC should have enough memory for ~1M FIB. This can get divided in half if you
are using RPF. If you have multiple routing instances, this also can contribute
to the problem.
Best Regards,
-Phil Rosenthal
On Jul 21, 2015, at 6:56 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hello list,
we seem to be running into limits with a MX480 with RE-2000 and 2x DPCE-4XGE-R
since we are seeing these new messages in the syslog:
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-dwords Available:83072 is less than LWM limit:104857,
Re: [j-nsp] jtree0 Memory full on MX480?
I know that a ton of fixes on BGP convergence time son MX80 is definitely a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be applicable. I see you're running DPC cards, have you considered shifting those links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda =)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on the new -NG cards coming out now). As the FIB is pushed to hardware, it may be some silly DPC thing you're running into. For things like Fusion or BNG or any other new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any new device we turn up, and have already started testing 15.1 for the new NG cards we will likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), etc... preventing starvation if you happen to have re-used Queue 3 as not-so-high priority, etc)... the list goes on. It's not a case of if it aint broke, don't fix it once you get 4-5 years behind. You'll benefit from the years of Oh, we finally fixed LLDP ascii decoding stuff that ends up getting traction; plus JTAC would really really like it if you weren't on 11.4 =) - Ck. On 22 Jul 2015, at 10:49 am, Jeff Meyers jeff.mey...@gmx.net wrote: Hi, yes, an upgrade is absolutely possible but since there are no major issues with that release, we didn't do that yet. Are you just assuming a newer software improves that or did Juniper really do something on that side? Best, Jeff ___ juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [j-nsp] jtree0 Memory full on MX480?
Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum
DPC should have enough memory for ~1M FIB. This can get divided in half if you
are using RPF. If you have multiple routing instances, this also can contribute
to the problem.
Best Regards,
-Phil Rosenthal
On Jul 21, 2015, at 6:56 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hello list,
we seem to be running into limits with a MX480 with RE-2000 and 2x
DPCE-4XGE-R since we are seeing these new messages in the syslog:
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-dwords Available:83072 is less than LWM limit:104857,
rsmon_syslog_limit()
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0
Type:free-pages Available:1326 is less than LWM limit:1638,
rsmon_syslog_limit()
Jul 22 00:50:36 cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-pages Available:1316 is less than LWM limit:1638,
rsmon_syslog_limit()
Jul 22 00:50:37 cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-dwords Available:84224 is less than LWM limit:104857,
rsmon_syslog_limit()
Jul 22 00:50:37 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0
Type:free-dwords Available:84864 is less than LWM limit:104857,
rsmon_syslog_limit()
Here is some more output from the FPC:
jeff@cr0 request pfe execute target fpc0 command show rsmon
SENT: Ukern command: show rsmon
GOT:
GOT: categoryinstancetypetotal lwm_limit hwm_limit free
GOT: --- - -
GOT:jtree jtree0-seg0 free-pages32768 1638 4915 1245
GOT:jtree jtree0-seg0 free-dwords 209715210485731457279680
GOT:jtree jtree0-seg1 free-pages32768 1638 491522675
GOT:jtree jtree0-seg1 free-dwords 2097152104857314572 1451200
GOT:jtree jtree1-seg0 free-pages32768 1638 4915 1267
GOT:jtree jtree1-seg0 free-dwords 209715210485731457281088
GOT:jtree jtree1-seg1 free-pages32768 1638 491523743
GOT:jtree jtree1-seg1 free-dwords 2097152104857314572 1519552
GOT:jtree jtree2-seg0 free-pages32768 1638 4915 1266
GOT:jtree jtree2-seg0 free-dwords 209715210485731457281024
GOT:jtree jtree2-seg1 free-pages32768 1638 491523732
GOT:jtree jtree2-seg1 free-dwords 2097152104857314572 1518848
GOT:jtree jtree3-seg0 free-pages32768 1638 4915 1232
GOT:jtree jtree3-seg0 free-dwords 209715210485731457278848
GOT:jtree jtree3-seg1 free-pages32768 1638 491523731
GOT:jtree jtree3-seg1 free-dwords 2097152104857314572 1518784
LOCAL: End of file
{master}
jeff@cr0 request pfe execute target fpc0 command show jtree 0 memory
extensive
SENT: Ukern command: show jtree 0 memory extensive
GOT:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:15299920 bytes used
GOT: 1459080 bytes available (660480 bytes from free pages)
GOT:3024 bytes wasted
GOT: 15192 bytes unusable
GOT: 32768 pages total
GOT: 26528 pages used (2568 pages used in page alloc)
GOT:4950 pages partially used
GOT:1290 pages free (max contiguous = 373)
GOT:
GOT: Partially Filled Pages (In bytes):-
GOT: UnitAvail Overhead
GOT: 8 6743440
GOT: 16 1078400
GOT: 2413296 4792
GOT: 32 2880
GOT: 48 283210400
GOT:
GOT: Free Page Lists(Pg Size = 512 bytes):-
GOT:Page Bucket Avail(Bytes)
GOT:1-1 140288
GOT:2-2 112640
GOT:3-376800
GOT:4-449152
GOT:5-5 7680
GOT:6-615360
GOT:7-725088
GOT:8-8 8192
GOT: 9-11 5632
GOT: 12-17 6656
GOT: 18-2622016
GOT: 27-32768 190976
GOT:
GOT: Fragmentation Index = 0.869, (largest free = 190976)
GOT: Counters:
GOT: 465261655 allocs (0 failed)
GOT: 0 releases(partial 0)
GOT: 463785484 frees
GOT: 0 holds
GOT: 9 pending frees(pending bytes 88)
GOT: 0 pending forced
GOT: 0 times free blocked
GOT: 0 sync writes
GOT: Error Counters:-
GOT: 0 bad params
GOT: 0 failed frees
GOT: 0 bad cookie
GOT:
GOT: Jtree memory segment 1 (Context: 0x449f87e8)
GOT: ---
GOT: Memory
Re: [j-nsp] jtree0 Memory full on MX480?
Hi Phil,
sure:
{master}
jeff@cr0 show configuration | display set | match rpf-check
{master}
nico@FRA4.cr0 show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package
[11.4R9.4]
JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]
{master}
nico@FRA4.cr0 show route summary
Autonomous system number: X
Router ID: A.B.C.D
inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown,
2994 hidden)
Direct: 1143 routes, 1140 active
Local: 1144 routes, 1144 active
OSPF: 81 routes, 18 active
BGP: 1745429 routes, 542631 active
Static:100 routes, 95 active
IGMP: 1 routes, 1 active
Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5
holddown, 0 hidden)
Direct: 2283 routes, 1140 active
Local: 2288 routes, 1144 active
OSPF: 17 routes, 17 active
BGP: 210387 routes, 210382 active
Static: 95 routes, 95 active
inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113
hidden)
Direct:451 routes,368 active
Local:373 routes,373 active
OSPF3: 9 routes, 9 active
BGP: 38399 routes, 22571 active
Static: 10 routes, 9 active
Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3
holddown, 0 hidden)
Direct:366 routes,366 active
Local:373 routes,373 active
OSPF3: 8 routes, 8 active
BGP: 11539 routes, 11536 active
Static: 9 routes, 9 active
{master}
I actually thought this Basic-Table was inactive. It is not so I'm
going to deactive it now. Since it was holding 200k routes, this is
for sure a lot. Doing that made the syslog message disappear but it
didn't actually free up as much as I was hoping for:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT: 15192 bytes unusable
GOT: 32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT: 24739 pages partially used
GOT:1691 pages free (max contiguous = 380)
Still doesn't look to glorious, right?
Best,
Jeff
Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:
Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum
DPC should have enough memory for ~1M FIB. This can get divided in half if you
are using RPF. If you have multiple routing instances, this also can contribute
to the problem.
Best Regards,
-Phil Rosenthal
On Jul 21, 2015, at 6:56 PM, Jeff Meyers jeff.mey...@gmx.net wrote:
Hello list,
we seem to be running into limits with a MX480 with RE-2000 and 2x DPCE-4XGE-R
since we are seeing these new messages in the syslog:
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-dwords Available:83072 is less than LWM limit:104857,
rsmon_syslog_limit()
Jul 22 00:50:36 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0
Type:free-pages Available:1326 is less than LWM limit:1638, rsmon_syslog_limit()
Jul 22 00:50:36 cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-pages Available:1316 is less than LWM limit:1638, rsmon_syslog_limit()
Jul 22 00:50:37 cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0
Type:free-dwords Available:84224 is less than LWM limit:104857,
rsmon_syslog_limit()
Jul 22 00:50:37 cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0
Type:free-dwords Available:84864 is less than LWM limit:104857,
rsmon_syslog_limit()
Here
