Re: [j-nsp] GRE interfaces doesn't show up

[Jeff Meyers]

Hi Wojciech,

we are using DPCE 4x 10GE R. I had a typo in my original email, xe-5/2/0
would be the correct interface of course but the config matches the
correct numbers.
The network-services mode is "IP"
The DPC is fine since we are using all other ports for ethernet services.

I just tested again on the 2nd router (I originally didn't want to use
for that purpose) and with the bandwidth 10g statement the port is
finally being created on Junos 13. On the Junos 15 router still not
working, the bandwidth is correctly set to 10g. Does it make a different
if an XFP is inserted in the slot..?


Thanks,
Jeff

Am 30.08.2019 um 00:17 schrieb Wojciech Janiszewski:

Hi Jeff,

What type of DPC card you're using?  As per your initial email I
wouldn't expect PIC 4 on DPC card related to gr-5/4/0 interface to be
present as PICs can take numbers between 0 and 3.
What is your network-services mode (show chassis network-services)?
Is DPC card up (show chassis fpc)?

As per documentation, if you specify a bandwidth that is not
compatible, tunnel services are not activated. For example, you cannot
specify a bandwidth of 1 Gbps for a Packet Forwarding Engine on a
10-Gigabit Ethernet 4-port DPC.
Have you verified with "show interfaces gr-* terse" if any GRE
interface is created?
If you're using DPC 40x1GE then tunnel interfaces shall be
gr-x/[0-3]/10 (set chassis fpc X pic Y bandwidth 1g) and with DPC
4x10GE it shall be gr-x/[0-3]/0 (set chassis fpc X pic Y bandwidth 10g).

Regards,
Wojciech


czw., 29 sie 2019 o 23:40 Jeff Meyers mailto:jeff.mey...@gmx.net>> napisał(a):

Hi Jim,

thanks for the quick reply! However, unfortunately that did not do the
job and the interface still doesn't show up, neither with 1g nor
10g for
the bandwidth:

> # show chassis fpc 0
> pic 0 {
>     tunnel-services {
>     bandwidth 10g;
>     }
> }

> # show interfaces gr-0/0/0
> unit 0 {
>     description Tunnel;
>     tunnel {
>     source 1.2.3.4;
>     destination 4.3.2.1;
>     }
> }

> # run show interfaces gr-0/0/0
> error: device gr-0/0/0 not found

Any ideas? :-(


Thanks,
Jeff



Am 29.08.2019 um 12:38 schrieb Jim Alias:
> On DPCs you need to set the bandwidth setting.
>
> set chassis fpc  pic  tunnel-services bandwidth <10g - 100g>
>
> Be advised that this will burn a 10 gig port for each 10 gigs
you assign to tunnel-services.
>
> Regard,
> Clay Haynes
>
> Sent from my iPhone
>
>> On 29 Aug 2019, at 10:50, Jeff Meyers mailto:jeff.mey...@gmx.net>> wrote:
>>
>> Hi list,
>>
>> I'm trying to setup a GRE tunnel interface on a MX480 with DPCE
>> linecards. I want to use xe-5/4/0 and have therefore configured
this
>> interface under 'edit chassis' for tunnel-services. The interface
>> vanishes as expected. I have then configured the interface
gr-5/4/0 with
>> unit 0 but is not being created. It's the same behaviour on a
secondary
>> MX router so I might be missing something. I am running Junos 13.3
>> respectively 15.1 on the routers I tested this with.
>>
>> I am unable to identify the reason right now, maybe someone
from this
>> list can help me with that? Thanks in advance!
>>
>> Best
>> Jeff
>> ___
>> juniper-nsp mailing list juniper-nsp@puck.nether.net
<mailto:juniper-nsp@puck.nether.net>
>> https://puck.nether.net/mailman/listinfo/juniper-nsp

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
<mailto:juniper-nsp@puck.nether.net>
https://puck.nether.net/mailman/listinfo/juniper-nsp



___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] GRE interfaces doesn't show up

[Jeff Meyers]

Hi Jim,

thanks for the quick reply! However, unfortunately that did not do the
job and the interface still doesn't show up, neither with 1g nor 10g for
the bandwidth:


# show chassis fpc 0
pic 0 {
    tunnel-services {
    bandwidth 10g;
    }
}



# show interfaces gr-0/0/0
unit 0 {
    description Tunnel;
    tunnel {
    source 1.2.3.4;
    destination 4.3.2.1;
    }
}



# run show interfaces gr-0/0/0
error: device gr-0/0/0 not found


Any ideas? :-(


Thanks,
Jeff



Am 29.08.2019 um 12:38 schrieb Jim Alias:

On DPCs you need to set the bandwidth setting.

set chassis fpc  pic  tunnel-services bandwidth <10g - 100g>

Be advised that this will burn a 10 gig port for each 10 gigs you assign to 
tunnel-services.

Regard,
Clay Haynes

Sent from my iPhone


On 29 Aug 2019, at 10:50, Jeff Meyers  wrote:

Hi list,

I'm trying to setup a GRE tunnel interface on a MX480 with DPCE
linecards. I want to use xe-5/4/0 and have therefore configured this
interface under 'edit chassis' for tunnel-services. The interface
vanishes as expected. I have then configured the interface gr-5/4/0 with
unit 0 but is not being created. It's the same behaviour on a secondary
MX router so I might be missing something. I am running Junos 13.3
respectively 15.1 on the routers I tested this with.

I am unable to identify the reason right now, maybe someone from this
list can help me with that? Thanks in advance!

Best
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] GRE interfaces doesn't show up

[Jeff Meyers]

Hi list,

I'm trying to setup a GRE tunnel interface on a MX480 with DPCE
linecards. I want to use xe-5/4/0 and have therefore configured this
interface under 'edit chassis' for tunnel-services. The interface
vanishes as expected. I have then configured the interface gr-5/4/0 with
unit 0 but is not being created. It's the same behaviour on a secondary
MX router so I might be missing something. I am running Junos 13.3
respectively 15.1 on the routers I tested this with.

I am unable to identify the reason right now, maybe someone from this
list can help me with that? Thanks in advance!

Best
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] SNMP_EVLIB_FAILURE - snmp not working anymore

[Jeff Meyers]

Thanks, worked!

Am 21.12.2018 um 02:05 schrieb Olivier Benghozi:

PR1270686

restart statistics-service


Le 21 déc. 2018 à 01:23, Jeff Meyers  a écrit :

Dec 21 01:20:40  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out of 
transfer credits with PFE.Failed to get stats. ifl index: 373

I already did a snmp process restart without any success. Google doesn't reveal 
anything helpful to me. Next thing I want to try is a manual RE failover but 
with maintenance notifications, I cannot do that instantly. Did anyone here see 
this before and does by any chance know a solution for this?


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] SNMP_EVLIB_FAILURE - snmp not working anymore

[Jeff Meyers]

Hi,

we had a faulty DPCE-R in an older MX240 a few days ago and replaced it 
during operation. Since inserting the spare module, we cannot snmp query 
the device anymore and seeing those entries a lot:


Dec 21 01:20:25  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 0
Dec 21 01:20:27  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 371
Dec 21 01:20:31  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 0
Dec 21 01:20:33  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 372
Dec 21 01:20:37  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 0
Dec 21 01:20:40  fra4-cr2 mib2d[67435]: SNMP_EVLIB_FAILURE: PFED ran out 
of transfer credits with PFE.Failed to get stats. ifl index: 373



I already did a snmp process restart without any success. Google doesn't 
reveal anything helpful to me. Next thing I want to try is a manual RE 
failover but with maintenance notifications, I cannot do that instantly. 
Did anyone here see this before and does by any chance know a solution 
for this?



Thanks!
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] [c-nsp] LACP between router VMs

[Jeff Meyers]

Hi Adam,

LACP packets (so called slow packets in the linux kernel) are never 
forwarded by the bridging code. If it didn't change in the last ~2 
years, you will have to hack your kernel in order to let them pass. It's 
actually only a minor change in a couple of lines. The same applies btw 
to STP packets.



Jeff

Am 08.11.2017 um 17:09 schrieb adamv0...@netconsultings.com:

Hi folks,

  


Slightly off topic but I'm gonna give it a shot anyways.

Would anyone know how can I make linux bridges or better OVS to forward LACP
PDUs instead of swallowing 'em?

Basically "l2protocol forward lacp" equivalent.

Couldn't find a single article on this and just can't believe I'm the first
person that ever tried this.

Aren't linux bridges or OVS MEF compliant :-) ?

  


Thanks

  


adam

  


___
cisco-nsp mailing list  cisco-...@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] FIB size at CFEB-E M7i

[Jeff Meyers]

show chassis cfeb0 (or alike) is the most interesting info since it 
shows the SRAM usage on this very, very old machine. This is what 
actually limits the number of routes in the FIB. But 96% on the RE is 
not funny either and a replacement should be scheduled on short notice. 
Best is to skip the MX80 with it's limited 2GB of RAM on the RE as well 
and the slow CPU.


Am 10.04.2017 um 21:00 schrieb Eduardo Schoedler:

Hi Robert,

This output is from a M7i:

usr@rtr> show version
Hostname: rtr
Model: m7i
Junos: 13.3R8.7



usr@rtr> show route summary
Autonomous system number: x
Router ID: x.x.x.x

inet.0: 641678 destinations, 1301206 routes (641372 active, 19
holddown, 3258 hidden)
Restart Complete
   Direct:  9 routes,  9 active
Local:  9 routes,  9 active
 OSPF: 12 routes, 10 active
  BGP: 1301162 routes, 641331 active
   Static: 14 routes, 13 active

inet6.0: 37365 destinations, 80265 routes (37335 active, 30 holddown, 1 hidden)
Restart Complete
   Direct: 13 routes,  9 active
Local: 12 routes, 12 active
OSPF3:  2 routes,  2 active
  BGP:  80235 routes,  37309 active
   Static:  3 routes,  3 active



usr@rtr> show route forwarding-table summary
Routing table: default.inet
Internet:
  user: 641332 routes
  perm:  5 routes
  intf: 18 routes
  dest: 142 routes

Routing table: default.inet6
Internet6:
  user: 37314 routes
  perm:  4 routes
  intf: 48 routes
  dest: 128 routes



usr@rtr> show chassis routing-engine
Routing Engine status:
 Temperature 34 degrees C / 93 degrees F
 CPU temperature 35 degrees C / 95 degrees F
 DRAM  1536 MB (2048 MB installed)
 Memory utilization  96 percent
 CPU utilization:
   User   0 percent
   Background 0 percent
   Kernel 3 percent
   Interrupt  1 percent
   Idle  95 percent
 Model  RE-850
 Serial ID  x
 Start time 2016-01-21 18:56:42 BRST
 Uptime 444 days, 22 hours, 1 minute, 44 seconds
 Last reboot reason Router rebooted after a normal shutdown.
 Load averages: 1 minute   5 minute  15 minute
0.09   0.18   0.13

Regards,


2017-04-10 15:24 GMT-03:00 Robert Hass :

Hi
What is supported FIB size for M7i router with CFEB-E ?
Is it will handle 1M of routes in FIB ?

Rob
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp





___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] R: EX4550: storm-control on commit

[Jeff Meyers]

Hi,



I also have 2 x EX4550 in VC with storm-control enabled, but it never
happens to me to get that message on commit;

I have this configuration for storm-control:

set ethernet-switching-options storm-control interface all

I am wondering what is your configuration for the ae2.0 interface:
can you check that ? which physical interfaces are on ae2 ? what are
their configurations? Maybe there is a loop somewhere in between
those.


we have this one:

interface all {
level 1;
}


ae2 is actually a LACP channel containing 2x 10GE to a customer. 
Generally yes there might be a potential loop behind. The curious part 
is however that this message mostly appears on commits and not 
necessarily also without a commit. This is not limited to our EX4550 VC 
but can also be seen on some EX3300 ToR switches. Typically all with 
storm-control level 1-5 configured.



Also, can you check your log messages ? if you do a "help syslog
ESWD_ST_CTL_ERROR_IN_EFFECT" it tells you:


Yes, that happens every once in a while. Although I cannot guarantee 
this is not loop-caused, the impact to our network caused by loops is 
was typically clearly visible by disappearing ARP entries on the routers 
and/or jumping MACs (at least until we set storm-control low enough 
which is even < 1% so we use the bandwidth option here).



Best,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] EX4550: storm-control on commit

[Jeff Meyers]

Hello everybody,

we are having a kind of strange haviour here. It's not really an issue 
but at least a curiousity. On a couple of EX4550 in VC we have 
storm-control enabled. This works fine so far. But on every commit, we 
see a "storm-control in effect" message in our logs:


Jan 26 11:50:45  cs0 eswd[1298]: ESWD_ST_CTL_ERROR_IN_EFFECT: ae2.0: 
storm control in effect on the port
Jan 26 11:51:47  cs0 eswd[1298]: ESWD_ST_CTL_ERROR_IN_EFFECT: ae2.0: 
storm control in effect on the port



I wonder where this comes from. I doubt that this is real since most 
commits only contain a vlan configuration change for some ports but no 
major adjustments. Maybe someone can bring some light in this.



Thanks!
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] EX4550: LACP channels short lost after commit

[Jeff Meyers]

Hi Olivier,

thanks for your reply. At least it is something, that can be solved then ;-)

I have two follow-up questions:

- can I change it to periodic slow without any impact (because the other 
side is still on slow) or will that have to happen simultaneously 
because expecting fast does also mean it sends those packets fast?


- since slow means longer conversion times: will a link down event still 
wait for the channel member to timeout or will this be handled as an 
instant member failure?



Thanks!

Jeff

Am 30.10.2015 um 15:05 schrieb Olivier Benghozi:

Infamous LAG LACP flap...
I think there are several factors: crappy Junos code, LACP is CPU
managed (at least on EX), so a pike in CPU (or a latency in the
dedicated process) can break LAGs (make them flap in fact).
I had some similar issues with SRX code.

Go for slow LACP rate (30 seconds instead of 1 second, like Cisco by
default), it will probably work around it efficiently...
Suggestion for your ae config:

aggregated-ether-options {
 link-speed 10g;
 lacp {
 active;
 periodic slow;
 }
}



Le 30 oct. 2015 à 14:38, Jeff Meyers mailto:jeff.mey...@gmx.net>> a écrit :

Hi everybody,

yesterday we had a very small hickup on a bunch of EX4550 doing Layer2
only in a Virtual-Chassis. According to the logfiles, there were some
LACP changes on multiple independent channels on various fpcs so I
couldn't spot just one VC member as a source.

Now, just 1 hour ago, I saw more or less the same messages after a
commit: http://nopaste.linux-dev.org/?797124


Is this something I should worry about? At least there didn't seem to
be any problem visible in the network but I'm quite unsure if this is
just related to the commit (but doesn't happen every commit) because
the EX needs to activate the new configuration (just added one vlan to
a 10GE port not in a channel) or if there is something suspicious
going on.

We are currently running 12.3R4.6 on the VC with 3 members.


Thanks for any suggestions!


Best,
Jeff



___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] EX4550: LACP channels short lost after commit

[Jeff Meyers]

Hi everybody,

yesterday we had a very small hickup on a bunch of EX4550 doing Layer2 
only in a Virtual-Chassis. According to the logfiles, there were some 
LACP changes on multiple independent channels on various fpcs so I 
couldn't spot just one VC member as a source.


Now, just 1 hour ago, I saw more or less the same messages after a 
commit: http://nopaste.linux-dev.org/?797124



Is this something I should worry about? At least there didn't seem to be 
any problem visible in the network but I'm quite unsure if this is just 
related to the commit (but doesn't happen every commit) because the EX 
needs to activate the new configuration (just added one vlan to a 10GE 
port not in a channel) or if there is something suspicious going on.


We are currently running 12.3R4.6 on the VC with 3 members.


Thanks for any suggestions!


Best,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Thank's for clearification, that helps. So the SCB itself is only 
responsible for the available bandwidth per slot but is not and will 
never be a memory limitation?



Best,
Jeff

Am 22.07.2015 um 23:51 schrieb Chris Kawchuk:


On 23/07/2015, at 1:30 AM, Jeff Meyers  wrote:


yes, we did (at least since yesterday) although we are not really requiring 
more ports or bandwidth right now. If I understand that correctly, I need to 
upgrade to SCB2 as well?



nope -- no need to go to MPC+SCB2 combo.

original SCBs work fine with MPC1 MPC2.2E, etc.. albeit limited to 120G/slot 
only, which is fine as MPC2 can only do 80G anyways in/out





___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Hi,

thanks for the hint, didn't know about that option. This will certainly 
safe us if we are running in to limits. We don't have too many filters, 
mostly the basic stuff to protect the RE and a few filters on some vlans 
with basic white- and/or blacklisting. So really nothing fance although 
I have no idea mow many "many" filters are on a box like the MX. Is it 
1,000 terms or more like 1,000,000 terms?



Best,
Jeff

Am 22.07.2015 um 13:06 schrieb Ivan Ivanov:

Hi,

The 'route' option on 'memory-enhanced' will give you some time before
upgrade to MPC. Actually you should be okay for quite a long time
considering the size of the table you have at the moment.

https://www.juniper.net/documentation/en_US/junos11.4/topics/task/configuration/junos-software-jtree-memory-repartitioning.html


Note, that this will use the part of the memory reserved for filters
(Jtree segment 1) for storing route information. You that feature only
if don't have many filters configured.

Ivan,


On Wed, Jul 22, 2015 at 7:52 AM, Mark Tinka mailto:mark.ti...@seacom.mu>> wrote:



On 22/Jul/15 02:59, Chris Kawchuk wrote:
> I know that a ton of fixes on BGP convergence time son MX80 is definitely 
a reason to be 'moving up'... however as you're on RE-2000s on MX480 may not be 
applicable.
>
> I see you're running DPC cards, have you considered shifting those links 
onto an MPC/Trio Card? (newer chip, more RAM, more horsepower, yadda yadda yadda 
=)..) DPC was EOL a while ago, and everything has been Trio (and now Trio-NG on 
the new -NG cards coming out now). As the FIB is pushed to hardware, it may be 
some silly DPC thing you're running into.
>
> For things like Fusion or BNG or any other 
new/advanced/this-is-what-PLM-is-thinking functions, we're already putting in 14.2 on any 
new device we turn up, and have already started testing 15.1 for the new NG cards we will 
likely be buying. Rest of our network is now on 12.3R8 or 13.3 in many cases. (lots of BFD 
bugs have been squashed, some HQoS issues fixed, host-outbound-traffic for BFD keepalives 
now honour the c-o-s knobs, and are finally out of Queue 3 and into the Queue we want (7), 
etc... preventing starvation if you happen to have re-used Queue 3 as 
"not-so-high" priority, etc)... the list goes on.
>
> It's not a case of "if it aint broke, don't fix it" once you get 4-5 years behind. 
You'll benefit from the years of "Oh, we finally fixed LLDP ascii decoding" stuff that ends 
up getting traction; plus JTAC would really really like it if you weren't on 11.4 =)

We've been on 14.2 for a while now, and settling into 14.2R3.8.

Happy, to be honest. Only real problem is policing on LAG's, but it's
manageable.

Mark.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net

https://puck.nether.net/mailman/listinfo/juniper-nsp




--
Best Regards!

Ivan Ivanov

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Hi,


I see you're running DPC cards, have you considered shifting those
links onto an MPC/Trio Card? (newer chip, more RAM, more horsepower,
yadda yadda yadda =)..) DPC was EOL a while ago, and everything has
been Trio (and now Trio-NG on the new -NG cards coming out now). As
the FIB is pushed to hardware, it may be some silly DPC thing you're
running into.


yes, we did (at least since yesterday) although we are not really 
requiring more ports or bandwidth right now. If I understand that 
correctly, I need to upgrade to SCB2 as well?



It's not a case of "if it aint broke, don't fix it" once you get 4-5
years behind. You'll benefit from the years of "Oh, we finally fixed
LLDP ascii decoding" stuff that ends up getting traction; plus JTAC
would really really like it if you weren't on 11.4 =)


I agree :)


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Hi,

yes, an upgrade is absolutely possible but since there are no major 
issues with that release, we didn't do that yet. Are you just assuming a 
newer software improves that or did Juniper really do something on that 
side?



Best,
Jeff

Am 22.07.2015 um 02:45 schrieb Phil Rosenthal:

Disabling Basic-Table certainly bought you some time.

Agree that it still does not look good. I suspect that you are running into a 
software issue.  11.4 is no longer a supported version, 12.3 is the minimum 
supported today, with 13.3R6 as the recommended version.  Is it possible for 
you to upgrade?

Best Regards,
-Phil

On Jul 21, 2015, at 7:23 PM, Jeff Meyers  wrote:

Hi Phil,

sure:


{master}
jeff@cr0> show configuration | display set | match rpf-check

{master}
nico@FRA4.cr0> show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package [11.4R9.4]
JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]

{master}
nico@FRA4.cr0> show route summary
Autonomous system number: X
Router ID: A.B.C.D

inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown, 2994 
hidden)
  Direct:   1143 routes,   1140 active
   Local:   1144 routes,   1144 active
OSPF: 81 routes, 18 active
 BGP: 1745429 routes, 542631 active
  Static:100 routes, 95 active
IGMP:  1 routes,  1 active

Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5 
holddown, 0 hidden)
  Direct:   2283 routes,   1140 active
   Local:   2288 routes,   1144 active
OSPF: 17 routes, 17 active
 BGP: 210387 routes, 210382 active
  Static: 95 routes, 95 active

inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113 hidden)
  Direct:451 routes,368 active
   Local:373 routes,373 active
   OSPF3:  9 routes,  9 active
 BGP:  38399 routes,  22571 active
  Static: 10 routes,  9 active

Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3 
holddown, 0 hidden)
  Direct:366 routes,366 active
   Local:373 routes,373 active
   OSPF3:  8 routes,  8 active
 BGP:  11539 routes,  11536 active
  Static:  9 routes,  9 active

{master}


I actually thought this "Basic-Table" was inactive. It is not so I'm going to 
deactive it now. Since it was holding > 200k routes, this is for sure a lot. Doing that 
made the syslog message disappear but it didn't actually free up as much as I was hoping for:

GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT:   15192 bytes unusable
GOT:   32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT:   24739 pages partially used
GOT:1691 pages free (max contiguous = 380)


Still doesn't look to glorious, right?


Best,
Jeff


Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:

Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum

DPC should have enough memory for ~1M FIB.  This can get divided in half if you 
are using RPF. If you have multiple routing instances, this also can contribute 
to the problem.

Best Regards,
-Phil Rosenthal

On Jul 21, 2015, at 6:56 PM, Jeff Meyers  wrote:

Hello list,

we seem to be running into limits with a MX480 with RE-2000 and 2x DPCE-4XGE-R 
since we are seeing these new messages in the syslog:


Jul 22 00:50:36  cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree0-seg0 
Type:free-dwords Available:83072 is less than LWM limit:

Re: [j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Hi Phil,

sure:


{master}
jeff@cr0> show configuration | display set | match rpf-check

{master}
nico@FRA4.cr0> show version
Hostname: cr0
Model: mx480
JUNOS Base OS boot [11.4R9.4]
JUNOS Base OS Software Suite [11.4R9.4]
JUNOS Kernel Software Suite [11.4R9.4]
JUNOS Crypto Software Suite [11.4R9.4]
JUNOS Packet Forwarding Engine Support (M/T Common) [11.4R9.4]
JUNOS Packet Forwarding Engine Support (MX Common) [11.4R9.4]
JUNOS Online Documentation [11.4R9.4]
JUNOS Voice Services Container package [11.4R9.4]
JUNOS Border Gateway Function package [11.4R9.4]
JUNOS Services AACL Container package [11.4R9.4]
JUNOS Services LL-PDF Container package [11.4R9.4]
JUNOS Services PTSP Container package [11.4R9.4]
JUNOS Services Stateful Firewall [11.4R9.4]
JUNOS Services NAT [11.4R9.4]
JUNOS Services Application Level Gateways [11.4R9.4]
JUNOS Services Captive Portal and Content Delivery Container package 
[11.4R9.4]

JUNOS Services RPM [11.4R9.4]
JUNOS Services HTTP Content Management package [11.4R9.4]
JUNOS AppId Services [11.4R9.4]
JUNOS IDP Services [11.4R9.4]
JUNOS Services Crypto [11.4R9.4]
JUNOS Services SSL [11.4R9.4]
JUNOS Services IPSec [11.4R9.4]
JUNOS Runtime Software Suite [11.4R9.4]
JUNOS Routing Software Suite [11.4R9.4]

{master}
nico@FRA4.cr0> show route summary
Autonomous system number: X
Router ID: A.B.C.D

inet.0: 546231 destinations, 1747898 routes (545029 active, 11 holddown, 
2994 hidden)

  Direct:   1143 routes,   1140 active
   Local:   1144 routes,   1144 active
OSPF: 81 routes, 18 active
 BGP: 1745429 routes, 542631 active
  Static:100 routes, 95 active
IGMP:  1 routes,  1 active

Basic-Table.inet.0: 212783 destinations, 215070 routes (212778 active, 5 
holddown, 0 hidden)

  Direct:   2283 routes,   1140 active
   Local:   2288 routes,   1144 active
OSPF: 17 routes, 17 active
 BGP: 210387 routes, 210382 active
  Static: 95 routes, 95 active

inet6.0: 23331 destinations, 39242 routes (23330 active, 1 holddown, 113 
hidden)

  Direct:451 routes,368 active
   Local:373 routes,373 active
   OSPF3:  9 routes,  9 active
 BGP:  38399 routes,  22571 active
  Static: 10 routes,  9 active

Basic-Table.inet6.0: 12295 destinations, 12295 routes (12292 active, 3 
holddown, 0 hidden)

  Direct:366 routes,366 active
   Local:373 routes,373 active
   OSPF3:  8 routes,  8 active
 BGP:  11539 routes,  11536 active
  Static:  9 routes,  9 active

{master}


I actually thought this "Basic-Table" was inactive. It is not so I'm 
going to deactive it now. Since it was holding > 200k routes, this is 
for sure a lot. Doing that made the syslog message disappear but it 
didn't actually free up as much as I was hoping for:


GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:14613176 bytes used
GOT: 2145824 bytes available (865792 bytes from free pages)
GOT:3024 bytes wasted
GOT:   15192 bytes unusable
GOT:   32768 pages total
GOT:6338 pages used (2568 pages used in page alloc)
GOT:   24739 pages partially used
GOT:1691 pages free (max contiguous = 380)


Still doesn't look to glorious, right?


Best,
Jeff


Am 22.07.2015 um 01:06 schrieb Phil Rosenthal:

Can you paste the output of these commands:
show conf | display set | match rpf-check
show ver
show route sum

DPC should have enough memory for ~1M FIB.  This can get divided in half if you 
are using RPF. If you have multiple routing instances, this also can contribute 
to the problem.

Best Regards,
-Phil Rosenthal

On Jul 21, 2015, at 6:56 PM, Jeff Meyers  wrote:

Hello list,

we seem to be running into limits with a MX480 with RE-2000 and 2x DPCE-4XGE-R 
since we are seeing these new messages in the syslog:


Jul 22 00:50:36  cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree0-seg0 
Type:free-dwords Available:83072 is less than LWM limit:104857, 
rsmon_syslog_limit()
Jul 22 00:50:36  cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0 
Type:free-pages Available:1326 is less than LWM limit:1638, rsmon_syslog_limit()
Jul 22 00:50:36  cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 
Type:free-pages Available:1316 is less than LWM limit:1638, rsmon_syslog_limit()
Jul 22 00:50:37  cr0 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 
Type:free-dwords Available:84224 is less than LWM limit:104857, 
rsmon_syslog_limit()
Jul 22 00:50:37  cr0 fpc0 RSMON: Resource Category:jtree Instance:jtree1-seg0 
Type:free-dwords Available:84864 is less than LWM limit:104857, 
rsm

[j-nsp] jtree0 Memory full on MX480?

[Jeff Meyers]

Hello list,

we seem to be running into limits with a MX480 with RE-2000 and 2x 
DPCE-4XGE-R since we are seeing these new messages in the syslog:



Jul 22 00:50:36  cr0 fpc0 RSMON: Resource Category:jtree 
Instance:jtree0-seg0 Type:free-dwords Available:83072 is less than LWM 
limit:104857, rsmon_syslog_limit()
Jul 22 00:50:36  cr0 fpc0 RSMON: Resource Category:jtree 
Instance:jtree1-seg0 Type:free-pages Available:1326 is less than LWM 
limit:1638, rsmon_syslog_limit()
Jul 22 00:50:36  cr0 fpc1 RSMON: Resource Category:jtree 
Instance:jtree0-seg0 Type:free-pages Available:1316 is less than LWM 
limit:1638, rsmon_syslog_limit()
Jul 22 00:50:37  cr0 fpc1 RSMON: Resource Category:jtree 
Instance:jtree0-seg0 Type:free-dwords Available:84224 is less than LWM 
limit:104857, rsmon_syslog_limit()
Jul 22 00:50:37  cr0 fpc0 RSMON: Resource Category:jtree 
Instance:jtree1-seg0 Type:free-dwords Available:84864 is less than LWM 
limit:104857, rsmon_syslog_limit()



Here is some more output from the FPC:


jeff@cr0> request pfe execute target fpc0 command "show rsmon"
SENT: Ukern command: show rsmon
GOT:
GOT: categoryinstancetypetotal  lwm_limit hwm_limit free
GOT:  ---   - - 
GOT:jtree jtree0-seg0   free-pages32768  1638  4915 1245
GOT:jtree jtree0-seg0  free-dwords  209715210485731457279680
GOT:jtree jtree0-seg1   free-pages32768  1638  491522675
GOT:jtree jtree0-seg1  free-dwords  2097152104857314572  1451200
GOT:jtree jtree1-seg0   free-pages32768  1638  4915 1267
GOT:jtree jtree1-seg0  free-dwords  209715210485731457281088
GOT:jtree jtree1-seg1   free-pages32768  1638  491523743
GOT:jtree jtree1-seg1  free-dwords  2097152104857314572  1519552
GOT:jtree jtree2-seg0   free-pages32768  1638  4915 1266
GOT:jtree jtree2-seg0  free-dwords  209715210485731457281024
GOT:jtree jtree2-seg1   free-pages32768  1638  491523732
GOT:jtree jtree2-seg1  free-dwords  2097152104857314572  1518848
GOT:jtree jtree3-seg0   free-pages32768  1638  4915 1232
GOT:jtree jtree3-seg0  free-dwords  209715210485731457278848
GOT:jtree jtree3-seg1   free-pages32768  1638  491523731
GOT:jtree jtree3-seg1  free-dwords  2097152104857314572  1518784
LOCAL: End of file

{master}
jeff@cr0> request pfe execute target fpc0 command "show jtree 0 memory 
extensive"

SENT: Ukern command: show jtree 0 memory extensive
GOT:
GOT: Jtree memory segment 0 (Context: 0x44976cc8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT:15299920 bytes used
GOT: 1459080 bytes available (660480 bytes from free pages)
GOT:3024 bytes wasted
GOT:   15192 bytes unusable
GOT:   32768 pages total
GOT:   26528 pages used (2568 pages used in page alloc)
GOT:4950 pages partially used
GOT:1290 pages free (max contiguous = 373)
GOT:
GOT:  Partially Filled Pages (In bytes):-
GOT:   UnitAvail Overhead
GOT:  8   6743440
GOT: 16   1078400
GOT: 2413296 4792
GOT: 32  2880
GOT: 48 283210400
GOT:
GOT:  Free Page Lists(Pg Size = 512 bytes):-
GOT:Page Bucket Avail(Bytes)
GOT:1-1   140288
GOT:2-2   112640
GOT:3-376800
GOT:4-449152
GOT:5-5 7680
GOT:6-615360
GOT:7-725088
GOT:8-8 8192
GOT:   9-11 5632
GOT:  12-17 6656
GOT:  18-2622016
GOT:   27-32768   190976
GOT:
GOT:  Fragmentation Index = 0.869, (largest free = 190976)
GOT:  Counters:
GOT:   465261655 allocs (0 failed)
GOT:   0 releases(partial 0)
GOT:   463785484 frees
GOT:   0 holds
GOT:   9 pending frees(pending bytes 88)
GOT:   0 pending forced
GOT:   0 times free blocked
GOT:   0 sync writes
GOT:  Error Counters:-
GOT:   0 bad params
GOT:   0 failed frees
GOT:   0 bad cookie
GOT:
GOT: Jtree memory segment 1 (Context: 0x449f87e8)
GOT: ---
GOT: Memory Statistics:
GOT:16777216 bytes total
GOT: 5123760 bytes used
GOT:11650408 bytes available (11609600 bytes from free pages)
GOT:2704 bytes wasted
GOT: 344 bytes unusable
GOT:   32768 pages total
GOT:9912 pages used (8976 pages used in page alloc)
GOT: 181 pages partially used
GOT:   22675 pages free (max contiguous = 22672)
GOT:
GOT:  Partially Filled Pages (In bytes):-
GOT:   UnitAvail Overhead
GOT:  8253520
G

Re: [j-nsp] Proxmox with Multicast & Juniper EX

[Jeff Meyers]

Hi Tore,


However these intial IGMP membership reports are not forwarded to the
EX4550, as it is not a multicast-router interface (from the EX4200s'
points of view, at least). So the EX4550 won't be able to learn of the
existence of the multicast group at all. Also, since the uplink to the
EX4550 isn't a multicast-router interface, nor a regular host interface
that's a member of the multicast group, packets destined for the
multicast group won't be forwarded to the EX4550 either.

You really need a multicast querier in the network for this to work...


I will give that a shot, thanks for the help!


Regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Proxmox with Multicast & Juniper EX

[Jeff Meyers]

Hi list,

I hope to get some experience and tips from you regarding the usage of a 
Proxmox cluster using Multicast in a (juniper-based) network. Since our 
multicast experience is quite low and was never required before Proxmox 
became quickly popular and is meanwhile widely used by our customer, we 
are a little stuck here. The situation is as follows:


Proxmox cluster communicates using multicast between the nodes being a 
member of the cluster. This works so far with the default configuration 
(igmp-snooping enabled on Juniper EX for all vlans, nothing further 
configured) if the nodes are on the same device. It is not working in 
the following setup:



 MX480MX80
   |   |
 +---+
 | EX4550-VC |
 +---+
  ||   ||
+---+ +---+
| EX4200-VC | | EX4200-VC |
+---+ +---+
   | |   |
Node 1   | Node 3
   Node 2


Node 1 & Node 2 see each other but they don't see Node 3. It doesn't 
matter on which EX4200 Node 1 & 2 are placed, it always works locally 
but doesn't as soon as data has to travel through the EX4550 
core-switch. Most likely a solution would be to simply disable 
igmp-snooping on the EX4550 for the vlans, where I have to have a 
working multicast communication. I'd really like to avoid that since we 
are using vlan-range configurations instead of explicitly configuration 
each vlan but the latter is required, in order to disable igmp-snooping 
for just this specific vlan.


I am mostly confused why the packets passing the core makes a difference 
at all. For my understanding, igmp-snooping inspects the communication 
and passes multicast traffic to exactly those who shall receive it. Why 
isn't this working? I read that this requires an icmp querier. Would it 
help to configure that querier on one of the routers (it's two routers 
because of VRRP)? Can anyone explain why it is working on a local switch 
but not anymore as soon as a 2nd switch is involved in the path?



Hopefully some of you guys are working with setups like this as well and 
can help to solve our issue.



Thanks in advance!

Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hi Matt,


conf t
fault-finder broadcast-storm

sorry should be "fault-finder broadcast-storm sensitivity high"

I think the ProCurve-proprietary "loop-protect" on the downstream port
of the 2824 might be a better fit here.

See eg
http://larsmichelsen.com/monitoring/hp-procurve-network-loop-detection/


maybe, as long as the ESXi VMs do not drop those kind of packets as well.


Best,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hi Patrick,


The problem is that any broadcast packets across the loop get amplified
pretty quickly and this propagates across the entire broadcast domain
(all related switches that have trunks containing affected vlans for
transit).


of course, I always forget the 3rd party broadcasts when talking about 
loops.



For the procurve I think they call the feature "flow control"

so ...

conf t
fault-finder broadcast-storm
interface x flow-control


I'm not sure if flow-control helps here. For my understanding, it only 
sends out pause frames and helps to make sure that one server does not 
fully utilize the uplink - but probably only if the other side has 
flow-control enabled as well.



Should do it. Ideally I'd turn that on all ports. Note that these are
the defaults (or at least should be). The only other way to reduce the
effects of this is to reduce your broadcast domain (read: ports affected
by amplified looping broadcasts) by routing (rather than switching)
closer to the customer.


Yes, that's unfortunately almost impossible in this case because 
customer needs to use most of his vlans in multiple rooms in the 
datacenter. Maybe we give the customer an own routerport and split him 
from the general L2 infrastructure..



Best,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hi Alex,


I think we are missing some important details here.
AFAIK, in order to detect MAC moves, the port must be in a
bridge-domain/VPLS instance.
So Your MX480 ae0 must be a L2/"bridged" port, not a L3/routed one.


yes, that is the case - historically actually. In the past we migrated 
from L3 only to L2 interfaces with IRB and RSTP active with the MX480 as 
the root bridge and one MX80 as the backup root. But we quickly saw that 
RSTP was poorly implemented on the MX-routers because a topology change 
leads to a complete flush of the arp table on the router, causing 
packet-loss in the network because our ~35.000 entries couldn't be 
learned that quick again.



So the question would be -  are there any other ports on this MX480 in
same bridge-domain(BD)/VPLS instance?


Not anymore, there was in the past when we had all vlans bridged over to 
the MX80 as well. We don't do that anymore. But that's actually an 
important information that this can't/shouldn't happen on L3 interfaces. 
Since we don't need the L2 functionality anymore anyways, I guess we 
will migrate back to L3-only then.



If not, but You have an IRB interface in this BD, does it have "IS-IS
passive" enabled by any chance? "IS-IS passive" does not actually stop
ES-IS PDUs being sent out, so these pesky ES-IS mcast frames could be
the ones which looped.


No, we do OSPF internally but not on the L2 interface - only on "real" 
router ports.



Additionally, MAC move limiting is not supported on EX4550 VC and in
mixed EX4200-4500/4550 VC so if Your EX4200 VC is actually a mixed
EX4200-4500/4550 VC there is no chance getting it stopped on EX.
https://www.juniper.net/techpubs/en_US/release-independent/junos/topics/concept/ex-series-software-features-overview-vc.html#port-security-features-by-platform-table


It's two separate VCs because of the drawbacks a mixed VC causes but the 
L2 core is a EX4550 VC.



Thanks,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hi David,


Once you draw your diagram correctly you'll see what you're up against
(and it ain't pretty).

 Juniper MX480  no RSTP
   ||
   ae0
   ||
Juniper EX4550 VC   RSTP bridge id 0
   ||
   ae0
   ||
Juniper EX4200 VC   RSTP bridge id 16k
|
  ProCurve 2824 RSTP bridge id 32k
|
Windows Hostno RSTP
 (virtual switch)
 /   \
/ \
VM-host1  VM-host2  virtual hosts with bridging potential (no RSTP)
  \  /
   \/   loop via clients bridging causing ARP 'move'
broadcast storm


So your problem is that the final two virtual-switch layers don't
participate in your RSTP but can be looped causing your ARP storm.


alright, but why would the ProCurve and everything above care? Isn't the 
loop only inside the Windows Host since from the perspective of the 
ProCurve, packets come in the same interface where they were sent out?
Furthermore, what is required for the router to see this "mac move" 
thing? Since there is only one physical port (ae0) towards the L2 
infrastructure, this can only be macs moving between Vlans, right?


But the windows host connects to an access-port and the trouble spread 
over the whole network, even to uninvolved vlans not present on the 
ProCurve or the EX4200 above.



You can prevent it by fiat (limit that pro-curve port to only 1 or 2
MAC addresses). Force the user to run in VM-nat mode or only run
one VM at a time.


I'm afraid that's hard to achieve, especially with customers of 
customers.. :(



You may be able to control the damage by limiting the broadcast/storm
thresholds on the leaf ports.


It's currently set to 85% on the EX4200, the ProCurve doesn't even 
support that feature. I guess this will require a value much smaller in 
the 10% region?



I don't think that the "mac-move-limit" feature will help you as the
mac changes are all coming in on the same physical port. The switches
don't care about ARP MAC<->IP flapping, only the router cares about it.


No, doesn't sound suitable for this case - I agree.


Good luck


Thanks!


Regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hi Michele,


So STP didn't protect you and you faced the loop.


okay, but how is this a loop from the perspective of the switches in the 
higher levels?  The Procurve sees packets coming in from the same port 
where they were sent out. Isn't that by definition not a loop?



When the loop occurs, frames that your switches send down keep coming
back up, and you see MAC flaps. The 2824 is not smart enough to warn you
about that.


But why would that cause trouble the way it did? We had:

- trouble even in Vlans not even present on the switch
- mac moving messages on the router - although there is just one 
physical downlink - probably indicating, that macs move between vlans(?) 
but the port in question is an access-port


Do you have an idea how that is even possible?


You might try to use PVST in your EXs, if they support it, because it
uses a different MAC, but I bet the vswitch would drop that as well.


I guess any solution that can be bypassed simply by filtering packets by 
the customer, will be abused eventually. How do others solve that problem?



Best regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Network, trouble after customer created a loop *inside* a VM host

[Jeff Meyers]

Hello everybody,

I'm writing to this list because I can't seem to find the reason for 
what we saw twice meanwhile. Here is the setup:




   Juniper MX480no RSTP
 ||
 ae0
 ||
  Juniper EX4550 VC RSTP bridge id 0
 ||
 ae0
 ||
  Juniper EX4200 VC RSTP bridge id 16k
  |
ProCurve 2824   RSTP bridge id 32k
  |
  Windows Host


So the router itself is not part of the Spanning-Tree, everything below 
is. On the Windows host, the customer is running ESXi with just one 
uplink towards the HP ProCurve switch so there is not even a real danger 
for a physical loop. Now: on the host are two VMs running. Each of them 
has a virtual NIC which is bridged to the physical one of the host. 
Because of a mistake, the customer accidentally bridged his two VMs 
together as well which caused a loop inside the Host. So far, so good.


The trouble begins at this point because immediately we saw partial 
network outages resulting in router messages like this:


Nov  7 14:30:47  cr0 l2ald[2545]: L2ALD_MAC_MOVE_NOTIFICATION: MAC Moves 
detected in the system



This message repeated over and over and the ARP counter decreased 
continueously. Host flapped and vanished for seconds or minutes and 
internal smokeping measured a lot of loss.


The HP ProCurve logged only excessive broadcast for the customer port 
and that's it. Spanning-Tree didn't recognize anything. The same applies 
to the EX4200 VC and the EX4550 VC: nothing was detected by the loop 
preventing procotol and it was only a lucky shot, that we knew where to 
look because the customer called by phone and told us, what he did.


The question is: how can that be and what can I do?

On the EX-series switches, each downlink port is configured with

set protocols rstp interface ge-0/0/0 no-root-port

storm-control is enabled on all ports with 85% (but none was detected). 
There is no special configuration on the ProCurve besides the general 
RSTP activation (which is set to RSTP and not STP).



So can anybody help with that? I am really stuck here.. :(


Thanks in advance,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Weird behaviour in network after customer created a bridge inside a Windows VM

[Jeff Meyers]

I'm not aware of any inherent loop protection when running a VC setup that
substitutes for STP.


This is not the case. VC is being used as well as RSTP. VC for the 3 
virtual-chassis configuration (seperate from each other, one VC with 2x 
EX4550 and 2x VC with 2x EX4200) and RSTP for the typical loop-detection 
and prevention.



Thanks,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Weird behaviour in network after customer created a bridge inside a Windows VM

[Jeff Meyers]

Hi everybody,

recently we saw a strange bahviour in our network. A customer with a 
Proxmox server had a Windows 2k8 VM with 2 virtual NICs (both bridged to 
eth0 of the server which faces the internet) and bridged them together 
INSIDE the VM. This caused immediately high latency and partial 
packet-loss within the whole network with the following messages in the 
router log:


May 29 04:06:33  cr0 l2ald[2545]: L2ALD_MAC_MOVE_NOTIFICATION: MAC Moves 
detected in the system



This is all I saw, no other device detected anything suspicious. This is 
the setup of the network:



MX480 router with DPCE and irb-interfaces but no (R)STP or any other STP 
flavour. This router connects 2x 10G as ae0 to a virtual-chassis 
consisting of 2x EX4550. This is a pure Layer2 device and the RSTP root 
bridge with priority 0. Furthermore, each server room is equipped with 
2x EX4200 in a VC as well with a RSTP priority of 16k. The ToR switches 
might have RSTP enabled or not and are usually connected with 1x GE to 
the EX4200 stack. Here is a scheme which describes the setup hopefully 
good enough:




 +---+
 | MX480 |   --- L3 edge-router, no STP
 +---+
 ||  --- ae0 with 2x XGE
 ++
 | EX4550 |  --- L2 only
 ++  --- RSTP priority 0
 | EX4550 |
 ++
   ||  ||
  +--+ +--+
  |  EX4200  | |  EX4200  |
  +--+ +--+  --- RSTP priority 16k
  |  EX4200  | |  EX4200  |
  +--+ +--+
   |||   --- 1GE links to ToR
   +---+ +---+ ...
   | HP SW | | HP SW |   --- RSTP priority 32k
   +---+ +---+
|||
|...
   +--+
   | Proxmox Host |
   +--+



In this particular scenario, the HP ProCurve switch had STP disabled and 
did not participate in the spanning-tree protocol. However, for my 
understanding that shouldn't be required anyways because the VC of the 
EX4200 switches should identify a potential loop on their own as long as 
there is no BPDU filter present.
The visible behaviour was high latency to several devices and sometimes 
up to 100% packet-loss. Not only within the customer's vlan but globally 
in every L2 segment. The arp count did not change on the router, I 
checked that. No TC event was recognized by any device. The questions 
are now:


- How is it possible that a bridge on a host with just ONE physical 
uplink can cause such problems?
- Am I correct that RSTP on the ToR switches is not required as long as 
they do not filter BPDUs?


- Why would the MX router create such a message and indicate, that the 
MAC changes although there is only one interface (ae0) facing the L2 
network? Even if a MAC moves on the EX4550 stack from one port to 
another, the MX would never see that.



Please let me know if you need any further details. I am very curious to 
find out what went wrong here. Do I have a misunderstanding on how STP 
behaves in that scenario? Are there any ways to dig deeper into analysis 
of the causes for that strange MX log message?



Thanks a lot!


Best regards,
Jeff

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Network design problem in a bridged setup with 2x Juniper MX and some Brocade SuperX

[Jeff Meyers]

Hello list,

I'm currently a little stuck and might need some help in order to decide 
how to improve the current setup. We are running a network where all 
customer vlans are bridged because the same Vlan is usually required in 
different areas in the network. This is the setup:




Room A:  ++
 | SX1600 |> [ 2nd SuperX not installed yet ]
 ++
  | |
  | |
 ++++
 | MX480  ||  MX80  |
 ++++
  | |
  | |
Room B:  ++++
 | SX400  || SX400  |
 ++++



Both MX routers have a 10G link between each other with RSTP active, so 
the the two SuperXes in Room B. These are the priorities:


MX480: 0 (root bridge)
MX80: 4k (backup root)
SX400: both 16k


Because topology changes caused some minor packet loss in Room B, I 
installed the SX1600 with MSTP instead of RSTP to see if that performs 
better. During some tests before connecting customers to the SX1600, 
results looked fine. We proceeded with the setup and replaced the old 
Cisco 6509/sup32 with the SX1600 and turned all routed Vlans active on 
the Cisco into bridged Vlans.


I'm running just one instance of MSTP (CIST) on the SX1600 with the 
following configuration:



mstp scope all
mstp instance 0 vlan 1
mstp instance 0 vlan 19
...
mstp instance 0 priority 16384
mstp edge-port-auto-detect
mstp start


On this SX1600, most uplinks go to switches on their own, usually HP 
ProCurve 2600 or 2800 series. Although we manage those switches, 
customers can install cables on their own. And here is where the problem 
actually starts: a rack with two ProCurve switches installed receives 
two uplinks from the same SX1600 and those switches are connected with 
each other, causing a loop. No matter what I did, the loop continued to 
cause trouble to the whole network because the MX routers saw topology 
changes all the time (between a few and 200 seconds or so) and flushed 
the whole arp cache. With about 90.000 active arp entries, this caused a 
more or less heavy impact on the servers behind of course. Although STP 
was active on both HP switches, the problem didn't vanish but the 
topolgy change itself was not visible on the SX1600 as it seems. In 
order to solve the issue, we had to remove the cable causing the loop 
but of course this can't be the solution since customers may install a 
new loop anytime and what's the point in running STP if you need to care 
about that?


The question is now how to proceed and how to improve the setup 
generally? Does it make sense to change RSTP to MSTP on the MX routers 
in the first place? Is there any configuration I should perform on any 
of those devices involved?
Since many of you are most likely from the Cisco world, here is a list 
of the available commands on the SuperX running in MST mode:


SSH@A.cs0 (config)#mst
  admin-edge-port Define this port to be an edge port
  admin-pt2pt-mac Define this port to be a point-to-point link
  disable Disable MSTP on this interface
  edge-port-auto-detect   Enable/Disable auto-detect edge port
  force-migration-check   Trigger port's migration state machine check
  force-version   Configure MSTP force version
  forward-delay   Configure bridge parameter forward-delay
  hello-time  Configure bridge parameter hello-time
  instanceConfigure MSTP instance VLAN membership
  max-age Configure bridge parameter max-age
  max-hopsConfigure MSTP max-hops
  nameConfigure MSTP configuration name
  revisionConfigure MSTP revision level
  scope   Configure MSTP scope
  start   Start/stop MSTP operation


Inside the interface configuration, there is no way to configure e.g. a 
bpdu-protect on the port but root-protect is configured on every port 
towards customer switches.



I will be gladly thankful for any hints and I am aware that some of you 
might declare the setup to be broken but on the other hand, for 
colocation services where the same vlan might be required campus-wide, 
it's hard to improve that without installing tons of cables.
Furthermore, we want to eliminate the dependency of just one big 
core-switch. Both rooms are equally important and in the past, we had a 
big core in Room A with downlinks going to smaller core-switches in Room 
B but with the big core having a problem, everything was going down.



Thanks so far for reading this and hopefully some great ideas will 
follow. Any help will rewarded with a cold beer in Frankfurt, Germany 
anytime! ;-)




Best regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puc

Re: [j-nsp] MPLS for management VPN question

[Jeff Meyers]

Truman Boyes schrieb:

Hi,

thanks for your answer so far.

You then need to define a route-distinguisher, and route targets (or 
simply vrf-target under the VRF) to import/export the routes for this 
VPN from other PEs.


Can you provide an example for that? That would be a L3VPN, right? Why 
would I need any routes to be known on the router? Basically I only need 
192.168.0.0/16 to be the management subnet globally without any default 
gateways.


Later on you might want to connect some of your NMS/OSS systems into the 
VRF so they can reach the the devices on the management VPN.


So I simply add the devices to the vlan 100 on the existing ae Link with 
.1q tagged vlans? No special encapsulation needed on juniper side?



Thanks,
Jeff

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] MPLS for management VPN question

[Jeff Meyers]

Hi,

we currently have a small number of PoPs, each equippped with a Juniper 
M-series router. On each PoP we use a local Vlan 100 for the management 
with RFC1918 ip adresses - 192.168.0.0/16.


Unfortunately, this results in scalability problems as the network grows 
since it's not possible to manage and monitor all network devices (e.g. 
network switches) from one central point.


Therefore, we want to setup MPLS in our network and create a network 
wide VPN for the management. Since my experience with MPLS is very, very 
low (as in "there is none"), I could need some help here. So here we go:


The PoPs are connected over dedicated transport links and iBGP as well 
as OSPF is running fine so far. The transport link ends in a Foundry 
core-switch, the core-switch itself is connected via aggregated-ethernet 
to the juniper m-series router. On that ae-link, we're running dot1q 
vlan tagging.



First problem: what exactly will I need for my purpose? L2VPN? L3VPN? 
Something else? The management vpn shall be reachable from every 
management device on 2 or more PoPs.



I managed to got basic MPLS running as follows:


- enabled mpls under "protocols mpls" and created a label-switched-path
- enabled rsvp for the interface ospf and iBGP is running on


Here is where I'm stuck: what would be the next steps in order to create 
the desired management VPN? The routers itself doesn't need a RFC1918 
adress within that VPN.
What encapsulation would I need on the specific interface for the 
described setup?



Thanks for any help & best regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Mixing AC & DC in M10/M20

[Jeff Meyers]

Hi list,

is it possible to mix AC and DC power supplies in M10 and/or M20 routers?

Thanks,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] VRRP with Juniper, what is needed around?

[Jeff Meyers]

Pekka Savola schrieb:

Hi,

> Strictly speaking, you don't need full tables from upstream.  For 
> example, a default route or default + some more specifics is also OK.

of course, yes. But for a little traffic engineering we prefere 
full-tables here :-)

> In your simple setup, you don't necessarily need IGP, because the only 
> thing the other box needs to know is where is the other router's 
> loopback address.  The rest can be propagated in iBGP.  You can 

Even if I don't do VRRP for every single customer? What happens if 
traffic transits the 2nd router and it does not know that the subnet is 
added on the 1st router in vlan 200? I guess in this case, traffic will 
be discarded.

> I guess there are two main ways to build a redundant router/switch 
> solution like this:
> 
>  R1-R2
>  |  |
>  SW1---SW2
> 
> or:
> 
>  R1\ /R2
>  |  X |
>  | / \|
>  SW1 SW2
> 
> in the latter diagram you can also add a direct link between routers 
> and/or switches if you want but you can also live without it.

I guess the first solution is what we want. I guess the direct link 
between R1 and R2 is just a logical link for iBGP and maybe IGP?

> The former is simpler and is usually sufficient when the switches and 
> routers are located in the same premises (i.e. you don't need to be too 
> worried about fiber breaks etc. -- this assumes that if a link between 
> switch and router fails, the router sees the link down event). In this 
> scenario, you may want to use two links between SW1 and SW2 (and run 
> LACP or some such to bundle them up unless you just use STP) just in 
> case a switch port fails.  Spanning tree is not required in this setup.

Great, thanks for your detailled reply!

Best regards,
Jeff

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] VRRP with Juniper, what is needed around?

[Jeff Meyers]

Prasanna Kumar A.S schrieb:

Hi,

>   I guess your topology with two m10s will look this
> 
> Uplink1 - +---+ - +-+
> Uplink2 - | M10 1 |ae0| Core-Switch | - Customers
>   +---+ - + |
>   | |
> Uplink1 - +---+ - + |
> Uplink2 - | M10 2 |ae0| | 
>   +---+ - +-+

exactly, yes.

> And
> - You will have to replicate the BGP configurations on the second M10
> box, ( hope you are advertising the ae-ifl's subnet into BGP)
  ^

What do you mean by that? What is ae-ifl? (aggregated ethernet interface 
l...?)

> - configure VRRP on the ae ifls of both the m10 boxes connecting to the
> core-switch and you can configure VRRP in two ways 
> 1) Configure separate ip address as the VR-IP/VIP, Here only one router
> will be doing the FWding at any point of time

At least outgoing, yes. As long as the customer does not use the IP of 
M10-2 as his default gateway.

> 2) configure two vrrp-groups on each ifl, one with interface ip of M10-1
> as VIP and other group with interface ip of m10-2 as VIP, and here you
> can do load sharing by configuring 50% of the customers with gateway as
> m10-1 and rest with m10-2, this way you achieve load sharing (when both
> the boxes are up) as well as redundancy(when any one of the box is down)

I guess we will go with option #1.

> I am not sure how we can provide redundancy at switch level as the
> costumer can connect to only one switch with one physical link.

The next step will be a 2nd switch where a customer can get another 
uplink if he wants.

> Please get me the current configuration of your m10 box to understand
> your topology better

Pretty basic stuff:

- a couple of prefixes being announced to two transit providers
- 2x GigE member links on ae0
- 1x GigE for Upstream #1
- Upstream #2 is connected to the core switch
- All subnets are assigned to various vlans on ae0, e.g. ae0.100 for 
customer1, ae0.101 for customer2 and so on..

Just BGP, no IGP is running there. Only one router for everything and 
the Switch is doing Layer2 only.


Best regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] VRRP with Juniper, what is needed around?

[Jeff Meyers]

Hello list,

we operate a relatively small network with one Juniper M10 router for 
everything. Since availability becomes more and more important, we want 
to raise this by installing a second M10 with VRRP.

Our current setup is pretty simple:


Uplink1 - +-+ - +-+
Uplink2 - | M10 |ae0| Core-Switch | - Customers
   +-+ - +-+


Where Uplink 2 is physically connected to the Core-Switch and the 
logical connection is done with dot1q Vlans.

We just do very basic BGP and configure all default gateways for the 
customers directly on logical units on ae0. Unfortunately, my experience 
with VRRP and IGPs is very limited and I did not find helpful 
documentation on how a VRRP setup affects everything else.

Here is the way I think it works:

- a second router needs to have at least one full-table upstream on it's own
- the routers have to do iBGP with each other
- I have to configure VRRP on both sides for specific subnets(just a 
few, not all)
- the routers have to do some IGP with each other(which would you suggest?)

Please correct me here if I am wrong.


The first step is only 2 routers for Layer3 redundancy. We consider that 
necessary because we had too many problems in the past with the juniper 
box. The Core-Switch is redundant in several ways(and doing it's job 
rock stable), so for now we won't install a 2nd Core-Switch. However, if 
we did: how would that affect the setup? Which extra links would be 
necessary in which configuration?

As far as I know, the following links usually exist with this setup:

- Router 1 <-> Router 2

- Router 1 <-> Switch 1
- Router 1 <-> Switch 2

- Router 2 <-> Switch 1
- Router 2 <-> Switch 2

- Switch 1 <-> Switch 2


Obviously, Router 1 and Router 2 share a more or less identical 
configuration for VRRP with the same VLAN-IDs and so on..
But what about the link each router has to each switch? Since that is 
"real" router interfaces, VLAN 200 from Link #1(to Switch1) is not equal 
to VLAN 200 from Link #2(to Switch2).

Which extra configuration(e.g. Spanning-Tree) should be done here?


I suppose it's obvious that I am having some trouble here finding the 
correct solution. I hope some of you can help destroy some 
misunderstanding and enlighten me and maybe some other guys too ;)


Looking forward to your answers!

Regards,
Jeff
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp