[j-nsp] EX4550 QSFP bounce

[Ross Halliday via juniper-nsp]

Hello fellow Juniperers,

I'm playing with some EX4550s as boxes for 1 and 10 Gbps breakouts. For uplinks 
I've installed the 2x QSFP expansion module in the front slot, and am using 
QSFP DACs.

In testing I noticed that whenever something is inserted into et-0/1/0, 
et-0/1/1 stops working for a brief moment. et-0/0/1 recovers as fast as it can, 
subject to whatever additional constraints. Tried with and without LAG, the 
interface set to disable, manipulating the other end, different firmwares on 
the transceivers, and tweaking config knobs. It does not seem to matter what 
sort of transceiver I connect into et-0/1/0, or even if it has link. I've tried 
"deleting" this as a VC port, and no commands show it considering the 
interfaces VC ports, so I don't think that's a problem. I tried a few different 
JUNOS releases in the 15.1R7 SR train. 12.3 is not an option for this hardware.

As far as redundancy goes this can be worked around but I'd prefer the switch 
to not break when optics are inserted...

Has anybody else on this list run into this before? If you haven't but you ARE 
running this module with both ports active, if you would mind sharing what 
software release you're on that would be extremely helpful!

Thanks!
Ross



--- Logs from when this happens. I think the logging on the switch is 
incomplete as afterwards et-0/1/1 is actually an active interface participating 
in ae0. The MX204 is reporting that properly ---



Sep  2 16:49:59  lab-mx204-2 kernel: lag_bundlestate_ifd_change: bundle ae0: 
bundle IFD minimum bandwidth or minimum links not met, Bandwidth (Current : 
Required) 0 : 400 Number of links (Current : Required) 0 : 1
Sep  2 16:49:59  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_DOWN: ifIndex 575, 
ifAdminStatus up(1), ifOperStatus down(2), ifName ae0
Sep  2 16:49:59  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_DOWN: ifIndex 570, 
ifAdminStatus up(1), ifOperStatus down(2), ifName et-0/0/3
Sep  2 16:50:00  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 570, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/3
Sep  2 16:50:00  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 602, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/3.300
Sep  2 16:50:00  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 604, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/3.32767
Sep  2 16:50:02  lab-mx204-2 kernel: lag_bundlestate_ifd_change: bundle ae0 is 
now Up. uplinks 1 >= min_links 1
Sep  2 16:50:02  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 575, 
ifAdminStatus up(1), ifOperStatus up(1), ifName ae0
Sep  2 16:50:02  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 580, 
ifAdminStatus up(1), ifOperStatus up(1), ifName ae0.300
Sep  2 16:50:02  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 581, 
ifAdminStatus up(1), ifOperStatus up(1), ifName ae0.32767
Sep  2 16:50:06  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 599, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/1
Sep  2 16:50:06  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 601, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/1.300
Sep  2 16:50:06  lab-mx204-2 mib2d[22305]: SNMP_TRAP_LINK_UP: ifIndex 603, 
ifAdminStatus up(1), ifOperStatus up(1), ifName et-0/0/1.32767

Sep  2 16:49:57  lab-ex4550-2 chassism[1279]: xcvr_is_qsfpplus  DAC PASSIVE 
CABLE!!!
Sep  2 16:49:57  lab-ex4550-2 /kernel: drv_ge_misc_handler: ifd:155  new 
address:dc:38:e1:9c:69:43
Sep  2 16:49:57  lab-ex4550-2 /kernel: if_pfe_msg_send_vlan_data: peer ioctl 
message is NULL
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: PHY:MLD workaround passed for 
port:0, count:1, phy_id:0
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: PHY:MLD workaround passed for 
port:0, count:2, phy_id:1
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: PHY:MLD workaround passed for 
port:0, count:3, phy_id:2
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: PHY:MLD workaround passed for 
port:0, count:3, phy_id:3
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc419,for port:0, 
phy_id:0, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc41a,for port:0, 
phy_id:0, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc419,for port:0, 
phy_id:1, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc41a,for port:0, 
phy_id:1, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc419,for port:0, 
phy_id:2, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc41a,for port:0, 
phy_id:2, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc419,for port:0, 
phy_id:3, value:0xc044
Sep  2 16:49:59  lab-ex4550-2 chassism[1279]: NLP:Reg:0xc41a,for port:0, 
phy_id:3, value:0xc044
Sep  2 16:50:00  lab-ex4550-2 /kernel: ae_bundlestate_ifd_change: bundle ae0: 
bundle IFD minimum bandwidth or minimum links not met, Bandwidth (Current : 
Required) 0 : 1 Number of links (Current : Required) 0 : 1
Sep  2 16:50:00  lab-ex4550-2 

Re: [j-nsp] vRR License Key

[Ross Halliday]

Thank you Michael, it does indeed! Have an excellent night!

Cheers
Ross

From: Michael Hobl 
Sent: May 28, 2021 10:12 AM
To: Ross Halliday 
Cc: juniper-nsp@puck.nether.net
Subject: [EXTERNAL] Re: [j-nsp] vRR License Key

Hi Ross,

As far as I'm aware, vRR doesn't require any licensing within the instance 
(which probably also makes sense given it does not permit any forwarding-plane 
traffic).

Likewise, a quick review of a production vRR box shows nothing interesting:

admin@rr.bne01<mailto:admin@rr.bne01>> show version
Hostname: rr.bne01
Model: vrr
Junos: 18.4R1-S1.1
[...]

admin@rr.bne01<mailto:admin@rr.bne01>> show chassis hardware
Hardware inventory:
Item Version  Part number  Serial number Description
ChassisVRxx  VRR
Midplane
Routing Engine   RE-VRR

admin@rr.bne01<mailto:admin@rr.bne01>> show system license
License usage:
 Licenses LicensesLicensesExpiry
  Feature name   usedinstalled  needed
  scale-subscriber  0   10   0permanent
  scale-l2tp0 1000   0permanent
  scale-mobile-ip   0 1000   0permanent

Licenses installed: none

admin@rr.bne01<mailto:admin@rr.bne01>>

HTH

- Michael


On Sat, May 29, 2021 at 12:02 AM Ross Halliday 
mailto:ross.halli...@wtccommunications.ca>> 
wrote:
Hello everyone,

I'm hoping that someone here with vRR in production is able to answer me this 
simple question:

After obtaining the appropriate license, is there a key that needs to be 
activated or installed?

Is there a command like setting R or IR mode on a hardware MPC? Is this just a 
paper license???

None of the information we received from Juniper upon purchase gives any 
indication. I have been trying to get an answer officially all week. I have 
been through our VAR, technical and sales reps, regional sales rep, and a 
Customer Service Leader, and not one of them has been able to tell me whether 
or not I should have a key, or generate one in the Agile portal, or what. I've 
been waiting on an admin ticket for two days on what should be a "yes" or "no" 
question.

pls halp


Thanks
Ross
___
juniper-nsp mailing list 
juniper-nsp@puck.nether.net<mailto:juniper-nsp@puck.nether.net>
https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] vRR License Key

[Ross Halliday]

Hello everyone,

I'm hoping that someone here with vRR in production is able to answer me this 
simple question:

After obtaining the appropriate license, is there a key that needs to be 
activated or installed?

Is there a command like setting R or IR mode on a hardware MPC? Is this just a 
paper license???

None of the information we received from Juniper upon purchase gives any 
indication. I have been trying to get an answer officially all week. I have 
been through our VAR, technical and sales reps, regional sales rep, and a 
Customer Service Leader, and not one of them has been able to tell me whether 
or not I should have a key, or generate one in the Agile portal, or what. I've 
been waiting on an admin ticket for two days on what should be a "yes" or "no" 
question.

pls halp


Thanks
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX204 and QSFP+ breakouts

[Ross Halliday]

Thanks to everyone who responded. I actually do have an FSbox lying around, and 
forgot it had a QSFP+/QSFP28 port! Chris, our modules obviously have different 
firmware, I'll see if I can get something from FS about this.

Thanks!

Ross


-Original Message-
From: juniper-nsp  On Behalf Of Chris Adams
Sent: April 30, 2021 6:50 PM
To: juniper-nsp@puck.nether.net
Subject: [EXTERNAL] Re: [j-nsp] MX204 and QSFP+ breakouts

Once upon a time, Ross Halliday  said:
> Do FS QSFP+ breakout DACs and AOCs work on this platform? Is there some magic 
> sauce firmware I'm too daft to find?

I've got a couple of MX204s in service since they were released with
Fiberstore 4x10G optics (reported as QSFP-PLR4-40G in JUNOS).  I haven't
had any trouble with any of them on 18.1 and 18.4 JUNOS.  No magic sauce
in use as far as I know.

-- 
Chris Adams 
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] MX204 and QSFP+ breakouts

[Ross Halliday]

Dear list,

On a lark we picked up 4x10GbE breakouts for our MX204. Currently we've tried a 
couple different versions, a plain ol' passive DAC, and one that breaks out 
into 8 multimode fibers.

I see that QSFPP-4X10GE-SR is supported on MX204 since 17.4R1. We're running 
19.4R3. *SHOULD* work, right?

With the appropriate configuration they come up and pass data. Great. But they 
flap every few minutes, and the PFE log is flooded with 

MQSS(0): CHMAC1: Detected Ethernet MAC Local Fault Delta Event for Port 0 
(xe-0/0/0:0)
MQSS(0): CHMAC1: Cleared Ethernet MAC Local Fault Delta Event for Port 0 
(xe-0/0/0:0)

...for every single channel interface that's got link on it. The same happens 
with the DACs and AOCs.

Of course, these are all from FS, so firmware is the big question in my mind. 
These are Fiberstore QFX-QSFP-DACBO-2M and EX-QSFP-8LC-AOC5M. The MX CLI shows 
them as "QSFP+-40G-CU?M". These are switch-oriented part numbers, but I must 
admit this is my first foray into these things and in SFP+ land I haven't 
encountered any issues where an optic labelled "EX" doesn't just work in 
everything. We don't have an MX204 available to try random JUNOS loads on (yet) 
so I turn to the collective:

Do FS QSFP+ breakout DACs and AOCs work on this platform? Is there some magic 
sauce firmware I'm too daft to find?

(I've talked to JTAC, of course they blame the third-party transceiver)

Thanks

Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Juniper support offline?

[Ross Halliday]

Seems to be back albeit a bit rocky - I could not get in with my previous 
password and had to reset. SRM and Downloads appear to be functioning

Ross


-Original Message-
From: juniper-nsp  On Behalf Of Thomas 
Scott
Sent: January 26, 2020 6:14 AM
To: Nathan Ward 
Cc: Juniper NSP 
Subject: Re: [j-nsp] Juniper support offline?

Just got off the phone with JTAC and was informed the website was down for
"maintenance". We managed to open a case last night, but were unable to
this morning... I'm hoping that the "issue" doesn't last too long..

Phone number I called was 1-888-314-5822.

- Thomas Scott | mr.thomas.sc...@gmail.com  


On Sun, Jan 26, 2020 at 5:28 AM Nathan Ward  wrote:

> Hi,
>
> The published number on the Juniper website - 0080025864737 doesn’t work,
> and the +1888 US number went to Juniper, but to voicemail.
>
> I’ve called the number Liam has posted here (same as above but with +
> rather than 00), which worked - the agent had told me that there was
> maintenance yesterday and now there is no way to get copies of images
> apparently, even JTAC.
> I was told that there is no ETA for login being restored. All they can do
> is open a case so I get notified if and when it’s fixed. (Yeah, the agent
> really said “if and when”).
>
> Prsearch disappeared what, Jan 2019.. this year will it be case management
> and download access we lose for almost a year?
>
>
> On the off chance someone has them and is able to share, I need packages
> for 18.2R3-S1 for MX204 (so, VMHost), and 18.4R2-S2 for QFX5120.
> Those are the JTAC recommended versions, so I imagine they’ll be knocking
> about on plenty of hard drives..
> Luckily, checksums are still visible on the public site :-)
>
> > On 26/01/2020, at 8:22 PM, Liam Farr  wrote:
> >
> > I just messaged some local at Juniper NZ and they advised that
> +80025864737 is working for support.
> >
> > Seems to work from my 2D mobile here too.
> >
> >
> > Cheers
> >
> > Liam
> >
> > On Sun, 26 Jan 2020 at 8:16 PM, Nathan Ward  > wrote:
> > Hi,
> >
> > Looks to me and colleagues of mine like Juniper support is offline.
> >
> > Last night, I was able to log in but trying to download an image got to
> some stage of the redirect process and hung, then a please try again later
> message. It persisted for the next few hours of me trying every now and
> then.
> >
> > Today, I can’t log in at all - Invalid user/password.
> > Password reset process works, but, still doesn’t let me in. Different
> browsers, cleared cache, all the usual “is it on at the wall sir” debugging.
> >
> > Hearing the same story for others.
> >
> > I’ve called both the NZ 00800 (international 800) and the US +1888
> number. The former says “call cannot be completed”. The US number says
> “high volume of calls please leave a message”.
> >
> > We’re in New Zealand - unsure if that’s relevant.
> >
> >
> > Are others having these same issues?
> >
> > Any insight in to what’s going on?
> > It’s a long weekend here, so the local sales/SE/etc. folks I usually
> deal with are likely not anywhere near their phones.
> >
> > --
> > Nathan Ward
> >
> > ___
> > juniper-nsp mailing list juniper-nsp@puck.nether.net  juniper-nsp@puck.nether.net>
> > https://puck.nether.net/mailman/listinfo/juniper-nsp <
> https://puck.nether.net/mailman/listinfo/juniper-nsp>
> > --
> > Kind Regards
> >
> >
> > Liam Farr
> >
> > Maxum Data
> > +64-9-950-5302
>
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Rock-solid JUNOS for QFX5100

[Ross Halliday]

Dear List,

I'm curious if anybody can recommend a JUNOS release for QFX5100 that is 
seriously stable. Right now we're on the previously-recommended version 
17.3R3-S1.5. Everything's been fine in testing, and suddenly out of the blue 
there will be weird issues when I make a change. I suspect maybe they are 
related to VSTP or LAG, or both.

1. Add a VLAN to a trunk port, all the access ports on that VLAN completely 
stopped moving packets. Disable/delete disable all of the broken interfaces 
restored function. This happened during the day. I opened a JTAC ticket and 
they'd never heard of an issue like this, of course we couldn't reproduce it. I 
no longer recall with confidence, but I think the trunk port may have been a 
one-member LAG (replacement of a downstream switch).

2. New trunk port (a two-port LACP LAG) not sending VSTP BPDUs for some VLANs. 
I'm not sure if it was coincidence or always broken as I had recently began 
feeding new VSTP BPDUs (thus the root bridge changed) before I even looked at 
this. Other trunk ports did not exhibit the same issue. Completely deleted the 
LAG and rolled back to fix. This was on a fresh turnup and luckily wasn't in a 
topology that could form a loop.

Features I'm using include:

- BGP
- OSPF
- PIM
- VSTP
- LACP
- VRRP
- IGMPv2 and v3
- Routing-instance
- CoS for multicast
- CoS for unicast
- CoS classification by ingress filter
- IPv4-only
- ~7k routes in FIB (total of all tables)
- ~1k multicast groups


There are no automation features, no MPLS, no MC-LAG, no EVPN, VXLAN, etc. 
These switches are L3 boxes that hand off IP to an MX core. Management is in 
the default instance/table, everything else is in a routing instance.

These boxes have us scared to touch them outside of a window as seemingly basic 
changes risk blowing the whole thing up. Is this a case where an ancient 
version might be a better choice or is this release a lemon? I recall that JTAC 
used to recommend two releases, one being for if you didn't require "new 
features". I find myself stuck between the adages of "If it ain't broke, don't 
fix it" and "Software doesn't age like wine". Given how poorly multicast seems 
to be understood by JTAC I'm very hesitant to upgrade to significantly newer 
releases.

If anybody can give advice or suggestions I would appreciate it immensely!

Thanks
Ross

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Going Juniper

[Ross Halliday]

Indeed, I remember our discussions on the topic before! I still haven't made 
much headway. It's worth pointing out, though, that the "not configured" state 
can pop up when you least expect it, such as an aggregate filtering action 
applied after a broadcast storm (which you THOUGHT you fixed, but for some 
reason a bunch of stuff doesn't work still and you can't get into the box and 
aren't sure why). Good to watch out for since a network where unexpected things 
don't happen isn't connected to anything and certainly doesn't have any 
administrators ;)

 - Ross

-Original Message-
From: Jared Mauch [mailto:ja...@puck.nether.net] 
Sent: Tuesday, April 17, 2018 7:39 PM
To: Saku Ytti
Cc: Ross Halliday; juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] Going Juniper



> On Apr 17, 2018, at 7:02 PM, Saku Ytti <s...@ytti.fi> wrote:
> 
> 
> DDoS protection out-of-the-box is for all practical purposes not
> configured at all, which is unfortunate as that is what most people
> run. When configured correctly Trio has best CoPP I know of in the
> market, certainly better than Cisco or Arista have.

I do wish that Juniper would look at what IOS-XR has done to make
configuring it much easier out of the box.  An ASR 9K w/ default LPTS
is much nicer than a Juniper with default firewall filters.

- Jared
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Going Juniper

[Ross Halliday]

A little late to the party, but I've been accused of worse. 

We transitioned our network from Cisco 6500 platform to MX104s, and at the same 
time converged our Internet Edge onto those MXes too. It's the only Juniper 
router I'm aware of that actually fits *nicely* into a two-post rack, and they 
draw little power for what they are (approx. 150W on DC at idle at lower 
temperatures).

The low-end MXes can do a lot of things, but that doesn't mean you SHOULD 
necessarily do them. Anything CPU-heavy is a good example. Convergence time on 
three full feeds takes about 10-15 minutes in my experience, say in the case a 
major upstream drops. This isn't a big deal for everyone and for my employer 
certainly not enough to justify a bigger box on its own. One of the platform's 
strengths is also its weakness, in that the MX104 is essentially fabricless. 
Each of the "PIC" slots is a carved out chunk of a single forwarding engine. 
This is why they're limited to the 2-XFP MICs for 10 GbE options, unlike bigger 
MXes that take MICs with a much greater port density and support more than 4 
SFP+es. This is also why you have things that will affect the entire chassis at 
once, such as one of my favourite optics bug (plug in an SFP too slowly and all 
SFPs reset) and the DDoS Protection feature (blackholes an entire class of 
traffic on ALL ports). They are not suitable for core use for th
 is reason. 

They are very handy but less than great. MX104 is really made for telecom 
remotes and small COs with limited floor plans. MX80 and variants would be a 
convenient solution if you got one from the used market and have a 4-post rack. 
Both are really intended as 1 GbE aggregation routers. The MX104 is better 
since it has redundant REs, and if you just need some stuff off of an MS-MIC 
then it'd be a reliable low-maintenance box. But be aware that you're paying a 
premium for the form factor and not for something that will be in your network 
for a long time as they have very limited upgradability.  We were originally 
pretty excited but we're replacing a lot now since they just can't deliver the 
10G port density we need. Heck I think you can get smart lightbulbs these days 
that come with 10GbE ports...


Based on things I see in the mailing list, I'd take a serious look at splitting 
things up into more scalable components, such as the vMX and maybe an MX204 - 
systems that have real upgrade potential - although I have no idea what the 
pricing is like since we're going the "Big MX" route.


On Wednesday, April 11, 2018 5:32 AM, Saku Ytti wrote:
> 
> On 11 April 2018 at 04:31, Chris via juniper-nsp
>  wrote:
>
> > Since the MX104 has user replacable RE's I really wish Juniper would at
> > least offer a different option with a more beefy CPU/RAM but I don't think
> > that would ever happen...
>
> I think JNPR believes MX204 is the 'next gen MX104'. I bet if there
> was MX204 with 2xQSFP28 + 24-48xSFP+, sold attractively priced with BW
> license, so that SFP+ as 1GE and sub would make sense, those would
> sell really well.

Indeed it would. Even better if they hardened up the MX204 so it could be 
installed in the same environments.

> New RE for MX104 was on the table early on in the MX104 history, but
> then JNPR changed tracks, citing XEON not being thermally possible on
> it.

I would bet money that this has as much to do with it as planned limited 
lifetime of the platform. Maybe two thirds of our MX104s have baked PEM0 (the 
most down-wind power supply) out of existence. Any better CPU in there would 
have to be something like an ARM to keep the thing from roasting both power 
supplies simultaneously.

> I suspect more correct reason is that they don't see sufficient market
> potential in device like MX104. I think Cisco and Juniper are very
> confused about market, they appear to think entire market consists
> solely of large scale DC providers. That only addressable market is
> market which wants extremely dense 100GE boxes. 1GE is dead, 10GE is
> almost dead.

I have noticed this as well. A lot of noise about ultra-dense machines that 
take up way too much space (depth) and need half a rack's worth of break-out 
cables to do anything interesting. That is, as long as "interesting" means 
"inside the building". I think vendors are taking "cloud" too seriously, as it 
seems that everybody's forgotten that the access and aggregation layers 
actually exist. Err I mean to say, just use hyperconverged Next Gen 5G. It's 
magic! Right?



On Thursday, April 12, 2018 3:58 AM, James Harrison wrote:
>
> On 11/04/2018 10:51, James Bensley wrote:
> > I would agree with
> > you that low port coun't, good, and reasonably priced mixed 1G/10G
> > devices aren't plentiful in choice from vendors. We open a lot of
> > small PoPs so stuff like ME3600X/ASR920s, ASR9001, MX104 are great for
> > us but each with their own caveats.
>
> Particularly if you include the requirement for temperature hardening -

Re: [j-nsp] Help needed regarding the Eompls tunnel in Juniper & Cisco

[Ross Halliday]

Hi Ahsan,

A Catalyst 6500 and the ME 6500 are slightly different inside. The problem is 
with the Catalyst 6500: The feature you're trying to use requires you to set 
Gi2/2 as a switchport, you can then terminate your L3 using an SVI.

Read more here: 
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/15-0SY/configuration/guide/15_0_sy_swcg/mpls.html#pgfId-1430355

Cheers
Ross


-Original Message-
From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of 
Ahsan Rasheed
Sent: Thursday, December 01, 2016 5:25 PM
To: juniper-nsp@puck.nether.net
Subject: [j-nsp] Help needed regarding the Eompls tunnel in Juniper & Cisco

Hi All,

We are having some serious issue with one customer circuit.We are using eompls 
vlan based & we are unable to pass traffic over eompls (l2)tunnel between Cisco 
3550 switches if we use specifically Cisco 6503 ,Cisco 6504 & 6506 etc. If we 
use Cisco switch 6524 instead of Cisco 6503 it is working.

{(Cisco 3550 switch)--->(Cisco 6524)>(Juniper ACX 4000)>(Cisco 3550) 
}-->This setup is working.I am able to pass traffic end to end between Cisco 
3550's.

{(Cisco 3550 switch1)--->(Cisco 6503 or Cisco 6506))>(Juniper ACX 
4000)>(Cisco 3550 switch2) }-->This setup is not working.

Cisco 3550 switch1 vlan 1089(1.1.1.1/30)---trunk->sub interface eompls vlan 
1089(Cisco 6503)->(ACX 4000)terminating tunnel on sub interface vlan 
1089->Cisco 3550 switch2-trunk-vlan 1089(1.1.1.2/30)

We are using bgp & ospf between Cisco 6503 & Juniper ACX 4000. Vlan 1089  as 
svi we are using on Cisco 3550 switch1 and allowing vlan 1089 as trunk 
connecting back to Cisco 6503,eompls vlan 1089 tunnel is configured on sub int 
on 6503 facing Cisco 3550 switch 1.Cisco 6503 is connected with juniper ACX 
4000 & running bgp & ospf between each other.On ACX 4000 juniper eompls vlan 
based tunnel is terminating on sub interface facing Cisco 3550 switch 2. With 
Sup720 I was unable to pass traffic over tunnels although l2 eompls tunnel 1089 
is up on both (Cisco 6503 & Juniper). See below.


Below are the outputs & commands which i was running.


ACX 4000 Juniper:

chi> show l2circuit connections
Layer-2 Circuit Connections:
Neighbor: 63.250.238.225
Interface Type  St Time last up  # Up trans
ge-1/1/0.1089(vc 1089)rmt   Up Jan  2 12:45:23 2010   1
  Remote PE: 63.250.238.225, Negotiated control-word: No
  Incoming label: 299776, Outgoing label: 19
  Negotiated PW status TLV: No
  Local interface: ge-1/1/0.1089, Status: Up, Encapsulation: VLAN
chi> show ospf neighbor
Address  Interface  State ID   Pri  Dead
10.252.0.85  xe-0/2/0.0 Full  63.250.238.225 139

chi> show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table  Tot Paths  Act Paths SuppressedHistory Damp StatePending
inet.0
  15 13  0  0  0  0
Peer AS  InPkt OutPktOutQ   Flaps Last Up/Dwn 
State|#Active/Received/Accepted/Damped...
63.250.238.22530373179200   0   0 1:21:40 
13/15/15/0   0/0/0/0

show ldp neighbor
AddressInterface  Label space ID Hold time
63.250.238.225 lo0.0  63.250.238.225:0 40
63.250.250.219 lo0.0  0.0.0.0:00
10.252.0.85xe-0/2/0.0 63.250.238.225:0 11

set interfaces xe-0/2/0 mtu 9192
set interfaces xe-0/2/0 unit 0 bandwidth 10g
set interfaces xe-0/2/0 unit 0 family inet mtu 1546
set interfaces xe-0/2/0 unit 0 family inet address 10.252.0.86/30
set interfaces xe-0/2/0 unit 0 family mpls

set interfaces ge-1/1/0 vlan-tagging
set interfaces ge-1/1/0 mtu 1564
set interfaces ge-1/1/0 media-type copper
set interfaces ge-1/1/0 encapsulation flexible-ethernet-services
set interfaces ge-1/1/0 unit 0 vlan-id 2062
set interfaces ge-1/1/0 unit 0 family inet address 10.254.62.9/29 primary
set interfaces ge-1/1/0 unit 0 family inet address 63.250.226.153/30
set interfaces ge-1/1/0 unit 1089 encapsulation vlan-ccc
set interfaces ge-1/1/0 unit 1089 vlan-id 1089

set protocols mpls interface xe-0/2/0.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface xe-0/2/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface xe-0/2/0.0 authentication md5 1 key 
"$9$a9JUHf5F6CuZU9puOSyX7-wgJDikqP5ZGtu1IcS"
set protocols ldp interface xe-0/2/0.0 allow-subnet-mismatch

set protocols ldp interface lo0.0
set protocols l2circuit neighbor 63.250.238.225 interface ge-1/1/0.1089 
virtual-circuit-id 1089


ACX 4000 i am using Junos:jinstall-ppc-12.3X54-D27.1-domestic-signed.tgz

Cisco 6503:
Test#show mpls l2transport vc detail
Local interface: Gi2/2.1089 up, line protocol up, Eth VLAN 1089 up
  Destination address: 63.250.250.225, VC ID: 1089, VC status: up
Output interface: 

Re: [j-nsp] QoS when there is no congestion

[Ross Halliday]

On Nov 17, 2016, at 8:07 AM, Jason Healy  wrote:
> 
> We're actually starting to experience this now.  We have a QFX with 10g and 
> 1g links, and seeing significant drops on the 1g interfaces when traffic 
> arrives on a 10g.  We have QoS enabled on our WAN connection, but haven't put 
> it on the internal interfaces (where the drops are).
> 
> Do you have links to guidelines for basic QoS guidelines where you have this 
> mismatched interface speed?  I haven't had a lot of luck finding out things 
> like underlying buffer size, so that makes troubleshooting a little tricky.


Well, my experience has been with specific Cisco models, and I'm basing things 
I do based on that (lesson I learned: don't count on things to "just work"). If 
you do a search for QoS for iSCSI on the 3750 you will find a plethora of posts 
on the cisco-nsp list. What actually got me on to that was the documentation 
from our SAN vendor. Obviously we're kind of comparing apples to oranges here 
but it's all fruit, right? 

Funny, now that you mention it, I went and checked our EX4550s with 10G VMware 
hosts to our old 1G SAN... output drops! Nothing worse than 1259 out of 
1868189159 for the iSCSI queue, but apparently I need to update my strategy. 
Since it's just the SAN on it I probably don't need so much buffer space 
allocated to the queue I created for vMotion etc (202229 packets overall).


On Nov 17, 2016, at 3:54 PM, Aaron  wrote:
>
> H, interestingly, I'm getting word from someone that is running test
traffic through my network that they are seeing out of sequence with small
test frames (64 or 68 bytes)but any frames larger than that are fine.
Your mention of "reordering" gives me concern.  I wonder how/where I would
solve that.

Sorry Aaron, I probably should have used another word. I was thinking of jitter 
when I wrote that. Outright reordering/resequencing would be interesting to see 
- I don't have enough experience in that department to comment.


- Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] QoS when there is no congestion

[Ross Halliday]

> My opinion on QoS for networks with low bandwidth is to always implement
> it. It's really not that difficult and you never know when microbursts
> could be affecting things. Believe me, even if your upstream link is a
> 1Gb/s circuit, and your outbound traffic is less than 10Mb/s, you can
> still have drops due to microbursts.
> 
> Voice and video, depending on your use of them, are normally very
> important and very sensitive to drops/delays. It will never cost you
> anything (besides time) to learn to implement some simple QoS policies,
> however, if you have customers who complain about bad voice/video quality,
> it will cost you your reputation and possibly revenue when said customers
> cancel their contracts because of a service you cannot reliably provide.
> 
> -evt

As a convert from the "bandwidth is always the answer" camp, I'd like to echo 
these sentiments. And apologize for the incoming wall of text.

Our network is not 'congested' - at least not in the sense that you'd pick up 
with a Cacti bandwidth graph. A better way to think of things is that we have 
many points of contention.

For us, it comes down to a matter of allocation of buffer space and I guess 
what you could call the "funnelling" of packets. If you have a device that only 
has two interfaces at the same speed, there's nothing to think about. The 
challenge is when there are more interfaces: Consider a simple 3-interface 
situation, where traffic from two (interfaces A and B) is destined out one (Z). 
All are 1 Gbps. Say A and B hit the router at 100 Mbps. It's easy to think that 
200 Mbps should fit into 1 Gbps, but this isn't completely accurate. The 
packets are not nicely interleaved so that they "fit together" - in reality 
some of them are occupying the same point in time as each other. The overall 
bitrate measured over an entire second is comparatively low but they're 
arriving at 1 Gbps speeds. If you want a car analogy, think of a multi-lane 
freeway. Two cars travelling in different lanes going in the same direction, 
going the same speed. They are the only two cars for a quarter mile. Just 
because 
 the highway is rated for 80 cars in that space at that speed doesn't mean 
there won't be a crash if one suddenly changes lanes.

Through buffering, the router is able to take this data and cram it into 
interface Z, but it needs a large enough packet queue to deal with the packets 
that arrive at the same time.

This is called a "microburst", and WILL cause packet delay and reordering if 
the buffer isn't large enough. Anyone operating an IP SAN should be familiar 
with this concept. This is a big issue issue with switches used for iSCSI, such 
as the Cisco 3750s we started out with (despite common notions, QoS actually 
has to be enabled as the 'default'/'disabled' buffers are insufficient to deal 
with microbursts).

If you want a really blatant example of this in action, you need to look no 
further than Cisco's 2960, which has default buffer allocations so small that 
it experiences problems sending data out of a 100 Mbps port if it arrives on a 
1 Gbps port. (CSCte64832 
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2960/software/release/12-2_53_se/release/notes/OL22233.html)


Back in our early days of MPLS, our network was almost entirely Cisco 6500s. We 
had a number of these boxes as P/PE that just used the SFP ports built in to 
the supervisors for optics. Most of our small sites were fine, but as Internet 
traffic levels grew, we noticed TV problems (~400 Mbps at the time). The issues 
appeared similar to packet loss and were loosely coupled to peak Internet usage 
- it was very easy to see this on the graphs in the evening, but issues were 
observed during the day, as well. We did not understand why a gigabit link that 
only had 600-800 Mbps running over it would be having these issues. We 
eventually figured out the problem and moved everything onto interfaces with 
real buffers and the problems disappeared. (we had not yet figured out DSCP to 
MPLS EXP mapping.)

This highlighted for us that we cannot know or control what will happen with 
data to or from our customers on the Internet side. Using the SUP720s' onboard 
ports was a bad decision for a variety of reasons, but what would happen if a 
real DoS hit our equipment? Since our network handles voice and video as well, 
it is extremely important for us to protect our sensitive services. The only 
way you can guarantee that is by allocating sufficient buffer space for the 
things you deem important. I think a couple of our MXes are still running 
default queues but all marking is enforced at ingress.

Of course since we implement QoS throughout the network, everything scales out 
very well down to the access equipment. It is a lot of work but it would be 
nearly impossible to reliably operate a converged network without the ability 
to tell traffic apart and prioritize important stuff.

My two cents.

Cheers
Ross

Re: [j-nsp] ACX for subscriber aggregation

[Ross Halliday]

> ACX as a BNG, hmmmh...
> 
> Why don't you go virtual instead, e.g., vMX, CSR1000v or XRv?
> 
> Mark.

Unfortunately our towers aren't quite tall enough to reach the cloud and 
require some kind of intermediate device ;)



> Ross,
> 
> You might want to try searching the archives for ACX. A few of us have posted 
> the good, the bad, the ugly.
> 
> TL;DR: look at the ASR920.
> 
> Dan

Thanks Dan. I've been skimming the archives but I will look closer. I've seen 
the ASR920 before but they use too much power. I forgot to mention in my 
original posting to the list that we're designing solar-only sites going 
forward. Aren't they essentially the same hardware, anyway?



> I use ACX5048
> 
> I have a few vrf's and a few irb's with dhcp relay for FTTH broadband
> mainly
> 
> I use dhcp relay... works good, but watch out for those access-internal
> /32
> routes that are created for all dhcp client associations...the /32 host
> routes are also auto-redistributed into the MPLS L3VPN's VRF's !! ..unless
> you suppress them with, "forwarding-options dhcp-relay group ftth
> route-suppression access-internal"
> 
> - Aaron

Fantastic, and good to hear from you again Aaron! Currently we're used to that 
sort of behaviour as our PPPoE LNS dumps /32 routes for each subscriber. But 
thanks for the heads up and mention of your usage!



Thanks gents
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] ACX for subscriber aggregation

[Ross Halliday]

Hi list,

We run a bunch of fixed wireless broadband towers where we bring MPLS right to 
the site. Subscribers are terminated right there. Today we use Cisco 7301 in a 
PPPoE LAC capacity for dynamic subscribers and deal with BGP sessions to higher 
end customers that have managed CE.

We're looking at the ACX1100 to replace said 7301 and additional switches. 
Because of the lack of PPPoE-related features we're considering using DHCP for 
subscriber access. 

Does anyone have any experience with the ACX routers doing this kind of thing? 
Particularly the DHCP relay side, features supported in IRBs, and practical 
experience with the MPLS features is what I'm wondering about. We'd load it up 
with maybe half a dozen VRFs.


Thanks!
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Dealing with multihomed customer BGP primary/backup links

[Ross Halliday]

If I had no control over the far end I would enforce received routes with a 
prefix-list/routemap/policy (which you should be doing anyway), use 
metrics/localpref internally, and lock it down with strict uRPF.

However, my preferred approach is to place a CPE on site. We've never sold 
links on the basis of active/backup. We sell "redundant solutions". Gear is 
cheap, headaches aren't!

Cheers
Ross



> -Original Message-
> From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf
> Of Cydon Satyr
> Sent: Wednesday, July 13, 2016 4:37 AM
> To: juniper-nsp@puck.nether.net
> Subject: [j-nsp] Dealing with multihomed customer BGP primary/backup links
> 
> What would be the optimal way to deal with following scenario.
> 
> The customer of ours has a primary bgp connection over primary link on one
> router, and a backup bgp connection (up) on backup link on our other
> router. The customer may or may not (usually not) terminate both
> primary/backup links on the same router.
> 
> We want to stop customer using backup link at all as long as the primary
> link is up. Since we police both primary and backup link, customer can
> just
> load balance and use both links.
> 
> Without asking changes on his side (so something link MC-LAG won't fit
> here, I guess?), what are way to deal with this.
> 
> I can think of making a script which will not import their routes as links
> a primary link route is in our table.
> 
> The bgp conditional policy doesn't work for importing routes, only
> exporting... so that won't work either.
> 
> Any other suggestions maybe?
> 
> Regards
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX104 capabilities question

[Ross Halliday]

Hi Saku,

> I don't see how this makes it any less of a box, in my mind this makes
> it superior box. You lost single PFE/linecard, which happens to be
> only linecard you have.
> In my mind fabricless single-linecard design is desirable, as it
> reduces delay and costs significantly. Not only can you omit fabric
> chip, but you can get >2x the ports on faceplate, as no capacity is
> wasted on fabric side.

This is a good point but kind of tangential to what I was getting at. Before we 
were really familiar with the MX104, we went on sales and marketing material 
that talked about "the little" MXes and "MXes with multiple slots". It's very 
misleading. Even JUNOS MX documentation talks about FPCs being separate in 
control and forwarding plane operations, when in reality there's only AFEB0 and 
that's the whole box. No isolation, and "slot diversity" is basically only a 
little bit better than adjacent ports... Again, contrary to what the popular 
advice about "multi-slot MX routers" is. The MX104 is not really a multi-slot 
router in the traditional sense, it just takes more MICs.
 
> Regarding PR1031696, years ago I had bunch of 3rd party SFPs which
> would crash MX PFE. I practically begged JTAC to fix it. The issue was
> caused by SFP being sluggish to answer to I2C polling, and the code
> which was expecting an answer crashed when it couldn't receive I2C
> answer fast enough. I tried to explain to them, it's only matter of
> time before original SFP develops I2C error, at which point you'll see
> this from customer buying 1st party optics. JTAC was unconvinced, told
> me to re-open if I see it on 1st party.
> I used many channels to complain, but no avail. To me this was
> absolutely appalling and short-sighted behaviour.

Yes, and then it crashes every single SFP... brilliant design backed with 
brilliant support... give me a break!

> But all platforms can have all kind of problems, and if you would have
> multiple linecards, sure, in this case you'd only crash one of them.
> But just having multiple linecards won't help that much, you can still
> crash all linecards due to RE problem, so you're still going to need
> second router for proper redundancy, at which point it becomes
> immaterial if you have this 'linecard redundancy' or not.

All kinds of problems happen, yes the only "real" safeguard is to put every 
customer on their own PE. You might remember a previous conversation we had 
regarding the DDoS Protection mechanism. This thing is a major thorn in my 
side. Thanks to this "faster" design, when one of these filters kicks in, any 
traffic matching that class on the ENTIRE box is blackholed. I don't think this 
is appropriate behaviour: In my experience, it actually *creates* a DoS 
situation on these boxes.

These routers have their place, they're definitely a Swiss Army Knife type of 
machine, it's just that the handle is really small...

Oh - and something I forgot to mention in my original email: The MX104 doesn't 
support ISSU like the "real" MX routers do. ISSU isn't an option until 
somewhere in the 14.x train, I think, and JTAC's recommended stable release is 
still in the 13.x train. Kind of ticked about that one.

Cheers
Ross


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX104 capabilities question

[Ross Halliday]

We made the same move, from a network of SUP720-3CXLs up to MX104, over the 
last two years. Our busiest MX104 ingests 3 full feeds and around 100k prefixes 
via an IXP.

I'll echo Mark's sentiments: The MX104 has similar problems in that the CPU 
must update the FIB when the RIB changes. Small things like individual peers or 
normal churn aren't a big deal, but if we drop a major peer (say 300k prefixes 
destined to them), it's not pretty. It's not a slug like the 600 MHz MIPS 
SUP720 RP, but you can anticipate a couple minutes of reachability problems 
while the hardware shortcuts are updated to point out somewhere that accepts 
packets.

The MX104 is a great value box, but lacks some features which make it a "big 
boys" router. Aside from the RE, the control plane isn't as segregated as I 
thought, leading to interesting bugs like PR1031696 - which caused me to 
briefly lose all 10G optics in the entire chassis while installing an XFP; as 
well as some irritating behaviour from the ddos-protection mechanism. The 
larger chassis are *sort of* like running DFC cards in a 6500, were forwarding 
operations are pushed down. MX104 entirely relies on an integral system in the 
chassis and all you install are MICs, *sort of* like running a module with CFC 
and relying on the SUP.

I agree that you should spring for the MX240, which allows you the better RE, 
if you can afford/justify it. Unfortunately the larger MX platforms require the 
use of MPCs (not just MICs) so you wind up with a pretty awful number "just" to 
get a better RE, but there's a significant architectural difference, too. 
Depends on whether these are worth it to you or not. I imagine we'll eventually 
upgrade some of our key MX104s to the grown-up versions and reuse them 
elsewhere, but for now they do the job! 


Cheers
Ross


> -Original Message-
> From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf
> Of Mark Tinka
> Sent: Monday, June 06, 2016 4:16 AM
> To: Ralph E. Whitmore, III
> Cc: juniper-nsp@puck.nether.net
> Subject: Re: [j-nsp] MX104 capabilities question
> 
> 
> 
> On 6/Jun/16 10:01, Ralph E. Whitmore, III wrote:
> 
> > I am in the process of replacing my old cisco650x hardware and was
> steered to this list to pose the following questions:
> >
> > I have 4 primary BGP transits  each delivering 600k+ routes to me and we
> will be adding another probably 600k+peer in the near future.  The sales
> rep recommended the MX 104 to us first, but then came back to us and said
> "Sorry this router isn't adequate for your needs you need to be in the
> MX240 Chassis" I read the spec's I can find and it says from a routing
> engine perspective (RE-S-MX104)  that it will handle the routes with room
> to grow on."
> >
> > >From Juniper:
> > IPv4 unicast FIB 1 million
> > IPv6 unicast FIB  512K
> >
> > Ipv4 RIB   4 million
> > IPv6 RIB  3 million
> >
> >
> > So the question is:  is there some other limiting factor(s)  that should
> steer me away from the MX104 to the MX240 Chassis? Is the sales rep
> blowing smoke?  I am hoping to find someone here who has tried this config
> and will either say yes this is great solution or  OMG, I'd never try that
> again.
> 
> The MX104 will support 4x BGP sessions in the control plane (4GB RAM).
> The problem is that the CPU on this router is massively slow, so you
> will be in pain if you've got lots of churn.
> 
> FIB-wise, you will be fine with the MX104.
> 
> If you have the cash, go for the Intel-based RE's (MX240/480/960). If
> you don't mind Cisco, consider the ASR1000-X platforms.
> 
> Mark.
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] MX104 ISSU

[Ross Halliday]

Dear List,

I'm trying to practice upgrades on my MX104s with the least amount of network 
explosion. 

This KB article (showing the error I get) claims that ISSU is only supported 
14.1R1 and higher:

https://kb.juniper.net/InfoCenter/index?page=content=KB29381

...but the ISSU feature page, 
http://www.juniper.net/documentation/en_US/junos15.1/topics/reference/requirements/issu-system-requirements.html#section-issu-supported-releases
 , says "use the Feature Explorer", which lists 13.3R1 through 13.3R9 as 
supporting ISSU.

Running 13.3R4.6, trying to move to 13.3R8.7.

I'm assuming that the Feature Explorer is wrong.

Is there any way other than the extremely ungraceful switchover method to 
upgrade an MX104 on 13.3Rx?

Thanks
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX punting packets to RE - why?

[Ross Halliday]

Hey again,

> No, something like this:
> edit system ddos-protection protocols resolve mcast-v4
> set bandwidth 100
> set burst 100
> set flow-level-bandwidth logical-interface 20
> set flow-level-detection subscriber off
> set flow-level-detection logical-interface on
> 
> So we allow on aggregate 100pps of mcast-v4 resolve, but only 20pps
> per IFL. So even if one IFL is misbehaving, another IFL's mcast-v4
> resolve works fine.
> 
> There are only 4k policers available in HW for ddos-protection, which
> makes me turn off subscriber detection, as it's easy for attacker to
> generate 'new subscriber' (like in BGP change SADDR => new subscriber)
> and congest those 4k policer slots.
> I make logical-interface on (instead of 'automatic', which means they
> are added dynamically if aggregate policer if out-of-contract, but
> with 'on' they are always there, this guarantees well-behaving IFL
> does not suffer or the duration software detects this and adds the IFL
> policers)
> 
> This is just random recommendation. And funny thing is, you'd need to
> make similar thing to _all_ protools available for 'ddos-protection',
> there are quite many of them, like maybe 100, so you'll get several
> thousand new lines of config just for this.
> I wish there was way to set default explicit values, but there isn't.

Oh dear, that sounds like quite the chore. I don't understand your reasoning 
behind lowering the parameters so far from the defaults, though. 3000 pps/5000 
packet burst is how the box ships. Or am I to read between the lines re: 
"random recommendation"? lol

Maybe this is something I should talk with JTAC about at this point. I don't 
want to slam the RE but I don't want to have such a massive cutout, either.

> Hmm, then the redundancy really should have worked, unless you're
> doing some IGMP Snooping or something in the switches (on by default)
> which might require convergence on multicast states on the L2 too.
> If you're rocking RSTP  or MST, and it's correctly configured (all
> non-l2-core ports _MUST_ be portfast, because MST will not unblock
> downstream ports, until there is explicit permission from all upstream
> ports, and if you don't have portfast on in 1 port which is not
> speaking MST, then this port will block whole MST convergence, as
> you're waiting for explicit permission from that port, which will
> never come).

Oh, the redundancy definitely works, don't get me wrong. For some reason the MX 
is deciding it has to resolve packets instead of just sending whatever comes in 
with that VLAN tag into an l2circuit.

> Yeah multicast is tricky subject

That's the understatement of the year!

> I dislike
> multicast, I believe there are probably lot of yet unknown DoS vectors
> in it, and I would never run internet multicast. But for well
> controlled internal applications, it may sometimes be least bad
> solution.

Internet multicast, as we have things now, would be an absolute nightmare. But 
as far as unknown DoS vectors and other quirkiness, I compare it to IPv6 a few 
years ago. Everybody basically does it half-assed because nobody uses it. The 
only applications we have for multicast are TV service delivery and some timing 
protocols here and there.

> But getting help on the setup probably is huge chore, just getting
> external person to understand the network takes time.

Yes, that's for sure. Thank you very much for all of your feedback and effort 
into trying to understand what's going on here. I'm going to see if our CSE is 
subscribed to this list and ask him to take a peek.

Thanks again.

Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX punting packets to RE - why?

[Ross Halliday]

Thanks Michael. Looks like I'm at 66 pps like Dragan mentioned.

Some night I'll set up a maintenance window and play with this knob...

Cheers
Ross



-Original Message-
From: Michael Hare [mailto:michael.h...@wisc.edu] 
Sent: Monday, February 01, 2016 10:19 PM
To: Ross Halliday
Cc: juniper-nsp@puck.nether.net
Subject: RE: [j-nsp] MX punting packets to RE - why?

Ross-

Change 'fpc0' to 'afeb0' in your failed command.  I got goose eggs, but this 
lab chassis isn't doing multicast which may play a part.

$user@mx104-lab-re0> request pfe execute target afeb0 command "show nhdb mcast 
resolve" 
SENT: Ukern command: show nhdb mcast resolve
GOT:
GOT: Nexthop Info:
GOT:ID  TypeProtocolResolve-Rate
GOT: -    --  ---
LOCAL: End of file

-Michael

-Original Message-
From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of 
Ross Halliday
Sent: Monday, February 1, 2016 2:38 PM
To: Dragan Jovicic <dragan...@gmail.com>; Saku Ytti <s...@ytti.fi>
Cc: juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] MX punting packets to RE - why?

Hi Saku and Dragan,

Thank you for the responses, and apologies for the ambiguity.

The EXes are our video source switches. PIM RP is shared with MSDP to an 
anycast address. The MXes connect to the EXes at L3 via BGP - MX1/EX1 link is 
de-prioritized with a metric. Most of our receivers ride off of MX2, with a few 
further downstream.

Due to some interop issues and our use of VBR we've settled on a single MDT for 
this VRF. Being the default MDT it is of course joined on all PEs with this 
VRF. During normal operation, MX1, which doesn't have any active traffic for 
this VRF, has a full list of mcast routes with the source interface of the MDT.

So, in the first failure scenario - let's say EX2 or MX2 totally dies - MX1 
will lose a preferred BGP route to the RP and sources and see everything over 
the MX1/EX1 link, so all of the S,G entries will need to be updated from 
mt-0/0/0.1081344 to xe-0/3/0.3812.

If I am understanding what you guys are saying correctly, this would cause 
everything to get punted to the CPU until a new hardware shortcut is created, 
and in the meantime - since our entire channel lineup is in there - this would 
hammer the DoS protection mechanism?

Can the rate at which the joins are sent out be slowed? I can live with a bit 
of a delay on the channels coming back to life, but not with the entire slot 
getting blackholed... I am also open to tweaking the DoS protection settings 
but it seems to me that a 10x increase would be opening myself up to really 
slamming the RE and causing even bigger problems. I come from SUP720 world, and 
I rather like having a box that can process BFD and BGP updates at the same 
time LOL


The other failure scenario is when the EX1/EX2 link goes down. When this 
happens, all devices are still up, so as far as BGP or really anything on the 
MX "knows", nothing has changed. Metric and next-hops are identical to the PEs. 
Instead of pulling video from the direct link, EX1 & EX2 can only see each 
other through VLANs that the MXes carry as EoMPLS l2circuits. This is what 
truly baffles me, as none of what you guys mentioned with regards to should 
apply to an l2circuit.


Also,
> request pfe execute target fpc0 command "show nhdb mcast resolve"
error: command is not valid on the mx104

:(

Thanks for your help guys!

Ross



From: Dragan Jovicic [mailto:dragan...@gmail.com] 
Sent: Sunday, January 31, 2016 7:44 AM
To: Saku Ytti
Cc: Ross Halliday; juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] MX punting packets to RE - why?

Correct me if I'm wrong, this looks like MX doesn't have multicast cache for 
all those S,G routes (in inet.1).
So first packet of each S,G entry must first be resolved by kernel and 
downloaded to PFE.
DDOS feature is activated because large influx of unresolved packets are 
passing trough the router. You could change default DDOS setting for this type 
of traffic on your FPC.
Another thing that comes to mind is that kernel itself has limited number of 
resolves per second, which is 66. That is, 66 different NH S,G entries will be 
resolved per second.

dj@mx-re0> request pfe execute target fpc0 command "show nhdb mcast resolve" 
SENT: Ukern command: show nhdb mcast resolve
GOT:
GOT: Nexthop Info:
GOT:    ID  Type    Protocol    Resolve-Rate
GOT: -    --  ---
GOT:  1927   Resolve    IPv6   66
GOT:  1962   Resolve    IPv4   66
LOCAL: End of file
This is modified by (hidden) knob:

dj@mx-re0# set forwarding-options multicast resolve-rate ?  
Possible completions:
     Multicast resolve rate (100..1000 per second)
{master}[edit]
Mind you, I haven't tested this.
HTH,
Regards

On Sat, Jan 30, 2016 at 12:04 PM, Saku Ytti <s...@ytti.fi> wrote:
Hey Ross,

It's not clear to me i

Re: [j-nsp] MX punting packets to RE - why?

[Ross Halliday]

Hello,

> > If I am understanding what you guys are saying correctly, this would cause 
> > everything to get punted to the CPU until a new hardware shortcut is 
> > created, and in the meantime - since our entire channel lineup is in there 
> > - this would hammer the DoS protection mechanism?
>
> Yes, if ingress interface does not match, they will be punted.

Okay, thanks

> > Can the rate at which the joins are sent out be slowed? I can live with a 
> > bit of a delay on the channels coming back to life, but not with the entire 
> > slot getting blackholed...
>
> What do you mean 'entire slot being blackholed', do you mean losing unrelated 
> control-plane stuff, like BGP/ARP etc?

Yes, on the entire MPC I will see unrelated control plane protocols bounce, eg. 
spanning-tree. If I recall correctly some protocols are handled by the TRIO 
chips, right? I don't see any of my BFD-managed ISIS adjacencies drop.

> If yes, just limit the mcast resolve to something reasonable 100pps should be 
> plenty, provided we're not competing with  actual attack traffic.
>
> I would start with ddos-protection fixes and see if it behaves better with 
> more restricted punting.

I assume you're referring to "set forwarding-options multicast resolve-rate", 
right?

> Further research might involve figuring out if both MX boxes have multicast 
> state with source towards the local EX port, and clients subscribed. So that 
> no convergence is needed.

Interesting concept! Doesn't bother me at all, we like the idea of having our 
multicast available everywhere anyway.

> It wasn't obvious to me what kind of negative impact you observe when the 
> EX-EX link goes down. How are you now stopping loop in the EX network? You 
> have direct physical link between them, then you have l2circuit as well? But 
> it looks like you're not carrying BPDU over the l2circuit? So if you rely on 
> STP, I'm not entirely sure how the L2 redundancy works, which port is 
> normally being blocked? The actual physical link between switches or the link 
> via l2circuit, since my first guess would be that there would be L2 loop in 
> the topology and nothing to stop it, so I'm not sure I understand why it 
> works at all.

I'm in the habit of running VSTP for everything (the Cisco half of my brain 
keeps trying to type rapid-pvst+) that isn't a two-port affair. BPDUs are 
definitely making it through, everything checks out. The paths over the 
l2circuits are normally blocked via increased interface cost.

One of the VLANs carried as an l2circuit by the MXes between the EXes is 
actually not spanning-tree controlled, but a "backup" PIM interface. 
Essentially a clone of the EX-EX direct link, but with higher metric. Unlike 
the other VLANs this one always has the PIM and BGP adjacency sending traffic 
over it. The ddos-protection resolve-mcast4 action trips when multicast is 
slammed over that or one of the VSTP-managed VLANs transitions to a forwarding 
state.


I can do up a diagram if that would help. I'm really not sure how I'd explain 
this to JTAC and wanted to get some real-world experience from guys who are 
working with this stuff.

Thanks for all your help!

Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX punting packets to RE - why?

[Ross Halliday]

Hi Saku and Dragan,

Thank you for the responses, and apologies for the ambiguity.

The EXes are our video source switches. PIM RP is shared with MSDP to an 
anycast address. The MXes connect to the EXes at L3 via BGP - MX1/EX1 link is 
de-prioritized with a metric. Most of our receivers ride off of MX2, with a few 
further downstream.

Due to some interop issues and our use of VBR we've settled on a single MDT for 
this VRF. Being the default MDT it is of course joined on all PEs with this 
VRF. During normal operation, MX1, which doesn't have any active traffic for 
this VRF, has a full list of mcast routes with the source interface of the MDT.

So, in the first failure scenario - let's say EX2 or MX2 totally dies - MX1 
will lose a preferred BGP route to the RP and sources and see everything over 
the MX1/EX1 link, so all of the S,G entries will need to be updated from 
mt-0/0/0.1081344 to xe-0/3/0.3812.

If I am understanding what you guys are saying correctly, this would cause 
everything to get punted to the CPU until a new hardware shortcut is created, 
and in the meantime - since our entire channel lineup is in there - this would 
hammer the DoS protection mechanism?

Can the rate at which the joins are sent out be slowed? I can live with a bit 
of a delay on the channels coming back to life, but not with the entire slot 
getting blackholed... I am also open to tweaking the DoS protection settings 
but it seems to me that a 10x increase would be opening myself up to really 
slamming the RE and causing even bigger problems. I come from SUP720 world, and 
I rather like having a box that can process BFD and BGP updates at the same 
time LOL


The other failure scenario is when the EX1/EX2 link goes down. When this 
happens, all devices are still up, so as far as BGP or really anything on the 
MX "knows", nothing has changed. Metric and next-hops are identical to the PEs. 
Instead of pulling video from the direct link, EX1 & EX2 can only see each 
other through VLANs that the MXes carry as EoMPLS l2circuits. This is what 
truly baffles me, as none of what you guys mentioned with regards to should 
apply to an l2circuit.


Also,
> request pfe execute target fpc0 command "show nhdb mcast resolve"
error: command is not valid on the mx104

:(

Thanks for your help guys!

Ross



From: Dragan Jovicic [mailto:dragan...@gmail.com] 
Sent: Sunday, January 31, 2016 7:44 AM
To: Saku Ytti
Cc: Ross Halliday; juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] MX punting packets to RE - why?

Correct me if I'm wrong, this looks like MX doesn't have multicast cache for 
all those S,G routes (in inet.1).
So first packet of each S,G entry must first be resolved by kernel and 
downloaded to PFE.
DDOS feature is activated because large influx of unresolved packets are 
passing trough the router. You could change default DDOS setting for this type 
of traffic on your FPC.
Another thing that comes to mind is that kernel itself has limited number of 
resolves per second, which is 66. That is, 66 different NH S,G entries will be 
resolved per second.

dj@mx-re0> request pfe execute target fpc0 command "show nhdb mcast resolve" 
SENT: Ukern command: show nhdb mcast resolve
GOT:
GOT: Nexthop Info:
GOT:    ID  Type    Protocol    Resolve-Rate
GOT: -    --  ---
GOT:  1927   Resolve    IPv6   66
GOT:  1962   Resolve    IPv4   66
LOCAL: End of file
This is modified by (hidden) knob:

dj@mx-re0# set forwarding-options multicast resolve-rate ?  
Possible completions:
     Multicast resolve rate (100..1000 per second)
{master}[edit]
Mind you, I haven't tested this.
HTH,
Regards

On Sat, Jan 30, 2016 at 12:04 PM, Saku Ytti <s...@ytti.fi> wrote:
Hey Ross,

It's not clear to me if the mcast is only inside the EX or if it's
also on the MX's. And it's not clear to me how the faults impact the
multicast distribution tree. On stable state, do both MX80's have
mcast states for groups? Or only one of them?

Trio maps each multicast group into an input interface, if mismatch
occurs, that is group ingresses from other input interface than the
specified, I believe this causes host punt.

Alas DDoS-protection limits are quite insane, like 20kpps for many
protocols, that's more than NPU=>LC_PCU punting allows for, so it'll
kill pretty much everything. I'd set protocols I don't need to
10-100pps, non-critical protocols I need to 4kpps and critical
protocols I need to 8kpps.
And yes, configure each and every ddos-protocol, it'll inflate the
config quite a bit, but there is always 'set apply-flags omit', which
can be useful way to reduce config cruft about standard-configs you
don't really want to review in normally.


On 29 January 2016 at 23:36, Ross Halliday
<ross.halli...@wtccommunications.ca> wrote:
> Hi list,
>
> I've run into an oddity that's been causing us some issues. First, a diagram!
>

[j-nsp] MX punting packets to RE - why?

[Ross Halliday]

Hi list,

I've run into an oddity that's been causing us some issues. First, a diagram!

EX1EX2
 |  |
 |  |
MX1MX2

EX1 and EX2 are independent switches (not VC) that run a ton of video traffic. 
EX4200 on 12.3R8.7
MX1 and MX2 are MPLS PEs that ingest video and send it out to our network. 
MX104 on 13.3R4.6
Several VLANs span EX1 and EX2 as each switch has a server that requires Layer 
2 to the other unit. (active/active middleware)
EX1-EX2 link is direct fiber carrying VLANs
MX1-MX2 link is MPLS

The MX ports facing the EXes terminate L3 as well as hauling L2:

MX1:

xe-0/3/0 {
description "EX1 xe-3/1/0";
flexible-vlan-tagging;
hold-time up 5000 down 0;
encapsulation flexible-ethernet-services;
unit 3810 {
description "Backup link between TV switches";
encapsulation vlan-ccc;
vlan-id-list [ 304 810-811 3810 3813 3821-3822 ];
}
unit 3812 {
description "Video feed 2/2 from head end switch";
vlan-id 3812;
family inet {
address MX1/31;
}
}
}
l2circuit {
neighbor MX2 {
interface xe-0/3/0.3810 {
virtual-circuit-id 3810;
description "IPTV switch redundant link";
no-control-word;
}
}
}

MX2:

xe-0/3/0 {
description "EX1 xe-0/1/0";
flexible-vlan-tagging;
hold-time up 5000 down 0;
encapsulation flexible-ethernet-services;
unit 3810 {
description "Backup link between TV switches";
encapsulation vlan-ccc;
vlan-id-list [ 304 810-811 3813 3821-3822 ];
}
unit 3811 {
description "Video feed 1/2 from head end switch";
vlan-id 3811;
family inet {
address MX2/31;
}
}
}
l2circuit {
neighbor MX1 {
interface xe-0/3/0.3810 {
virtual-circuit-id 3810;
description "IPTV switch redundant link";
no-control-word;
}
}
}

We have dual L3 feeds from "the switches" to "the routers", and VLANs are 
carried over an l2circuit should the direct link between EX1 & EX2 bite the 
dust. It should be noted that MX1 is basically a "backup" - traffic normally 
flows EX1-EX2-MX2. The goal of this setup is so that we can take out any link 
and still have our video working.

It works... eventually.

The problem I am running into is that when a fail occurs, or I simply pull a 
VLAN from the EX1-EX2 link, multicast is suddenly slammed either across or into 
the MXes. When that happens, I get this lovely message:

jddosd[1527]: DDOS_PROTOCOL_VIOLATION_SET: Protocol resolve:mcast-v4 is 
violated at fpc 0 for 38 times, started at 2016-01-27 04:59:55 EST
jddosd[1527]: DDOS_PROTOCOL_VIOLATION_CLEAR: Protocol resolve:mcast-v4 has 
returned to normal. Violated at fpc 0 for 38 times, from 2016-01-27 04:59:55 
EST to 2016-01-27 04:59:55 EST

...and traffic (maybe of just offending class) on that slot is dumped for a 
little while.

> show ddos-protection protocols resolve statistics

  Packet type: mcast-v4
System-wide information:
  Bandwidth is no longer being violated
No. of FPCs that have received excess traffic: 1
Last violation started at: 2016-01-27 04:59:55 EST
Last violation ended at:   2016-01-27 04:59:55 EST
Duration of last violation: 00:00:00 Number of violations: 38
  Received:  4496939 Arrival rate: 0 pps
  Dropped:   2161644 Max arrival rate: 45877 pps
Routing Engine information:
  Policer is never violated
  Received:  130584  Arrival rate: 0 pps
  Dropped:   0   Max arrival rate: 1 pps
Dropped by aggregate policer: 0
FPC slot 0 information:
  Policer is no longer being violated
Last violation started at: 2016-01-27 04:59:57 EST
Last violation ended at:   2016-01-27 04:59:57 EST
Duration of last violation: 00:00:00 Number of violations: 38
  Received:  4496939 Arrival rate: 0 pps
  Dropped:   2161644 Max arrival rate: 45877 pps
Dropped by this policer:  2161644
Dropped by aggregate policer: 0
Dropped by flow suppression:  0
  Flow counts:
Aggregation level Current   Total detected   State
Subscriber0 0Active

Once the thing recovers, everything works again. But I cannot change a VLAN, a 
spanning tree topology, or work on anything without risking serious impact to 
my network!

I understand that the 'resolve' protocol means these packets are being sent to 
the RE.

...why the hell are they being sent to the RE? Even when there's a change on 
traffic that gets sent into that l2circuit - shouldn't this just be 

Re: [j-nsp] Breaking an EX cluster?

[Ross Halliday]

Since you want to nuke the config anyway, break the switch out of the VC and 
use 

request system zeroize

You may want to assign the soon-to-be-former member an RE role, if it's not an 
automatically elected cluster, just to make things a little easier.

Cheers
Ross


-Original Message-
From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of 
Scott Granados
Sent: Thursday, August 13, 2015 9:23 PM
To: juniper-nsp
Subject: [j-nsp] Breaking an EX cluster?

Hi,
Have some EX 4300s that I want to break apart and start like they were factory 
new and reboot.  I know about the factory default button on the front and the 
configuration option but no matter how I apply that I still have the node boot 
thinking it’s a member of the previous chassis.  How do I delete it’s 
membership when it’s active / a stand alone node?

Any pointers are most appreciated.

Thank you
Scott

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Firewall filter with apply-path

[Ross Halliday]

Hi list,

Would someone be so kind as to apply a working example configuration to protect 
the RE using apply-path to generate prefix lists? I *THINK* I have the actual 
apply-path part working, as show configuration ... inheritance shows what 
should be in there, but when I set the firewall filter on lo0.0 it doesn't 
match any packets and everything is just dropped. :(

I am new to JUNOS so a sanitized working example would be absolutely fantastic.

I'm running MX104s on 13.3R4.6, if that matters.

Thanks!
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MX104 Limitations

[Ross Halliday]

Saku Ytti wrote:

  1) It’s 3.5U high, making rack planning a little weird, and requiring me to 
  buy a hard to find half-U blank panel
 
 It is targeting metro applications, where racks often are telco racks. job-1
 and job-2 were thrilled to get MX104 form-factor, MX80 was very problematic
 and lead to 'creative' installations.

Telco here. I love the MX104's format... in general. Most of our installations 
are DC so the .5U part is really negligible as approx. 1U is required off the 
bottom of the unit for clearance anyway. What drives me nuts about the height 
is actually the spacing of holes on the ears. Racking them is clumsy for this 
reason.

I'd love to see a switch in a similar form factor. We've had to put some 
EX4200s in one of our COs, and man, what a pain in the cavity those are. Flimsy 
little ears and way too deep. 

  2) It uses unusual power connectors on its power supplies, so you have to 
  plan to buy special power cords just for this.
 
 It's standard C15/C16 which is temperature enchanced (120c) version of
 standard C13/C14. Lot of vendors are doing that these days, I'd like to
 understand why. Is there some new recommendation for fire safety or what has
 triggered the change.

Thank you for this explanation!!! We have one AC unit at a tower site and were 
wondering what the story was. Our company now has *TWO* NEMA 5-15P to IEC 320 
C15 cables.

About the only other thing that annoys me about the MX104 is the location of 
the chassis ground. Right in the corner with the fan tray. Seriously?

In general I love these little routers.

Cheers
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] RE switch master to backup

[Ross Halliday]

 Gents,
 
 has anybody seen a dual RE MX gear switch the routing engine master to
 backup due to a “possibly bad console cable” ?

 Jul  1 11:21:29.941 2015  MX480LON_0 init: getty repeating too quickly on
 port /dev/ttyd0, sleeping 30 secs

Did you happen to find a resolution for this? That message seems the most 
telling as to why it would fail over.

On Sun servers there used to be a way cool method to get them to reboot: Send a 
trillion BREAKs over the console. Same thing could happen if you rebooted your 
console device a few times, or, indeed, if the cable was bad. I'm not sure if 
JUNOS operates like this internally, but as they have common BSD roots it could 
be related.


Cheers
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] Q-in-Q with VSTP

[Ross Halliday]

 I'm bashing away at a conundrum here. I'm trying to lab a setup for a 
 multi-VLAN subscriber over some GPON gear. The setup is:

 MX104 --2x-- OLT -- ONT -- subscriber

 The ONT is able to strip the outer VLAN tag facing the subscriber, so the CE 
 can hit all of the inner VLANs directly. The OLT presents frames with both 
 outer  and inner
 tags to the MX, so I must strip the outer in order to work with the 
 subscriber's services. Link bundling/aggregation between the MX and OLT are 
 not an option, so I
 need to use spanning tree.

 So, I am trying to create a bridge domain with VSTP enabled for VLANs that 
 aren't accessible with a simple vlan-id-list or vlan-id.

 I cannot get this to work. I've tried vlan-tags outer 9 inner 9 with 
 separate units per inner VLAN, as well as sending everything into a vswitch 
 like detailed here
 http://www.juniper.net/techpubs/en_US/junos14.2/topics/topic-map/layer-2-services-stp-vstp-on-trunk-port-with-tagged-traffic-example.html
  ...but the STP
 instance just doesn't seem to recognize the interface, and my CE doesn't see 
 any BPDUs on that VLAN.

 I can't even find if this feature is supported - my searches on Juniper's 
 site don't show anything with spanning tree and Q-in-Q or double-tags or 
 inner tags etc on the
 same page.

 Is anybody else trying to do this? Or suggested solution? If the MX supported 
 Redundant Trunk Groups (like Cisco FlexLink) this would be so much easier.


Figured I should follow up on this in case someone tries similar in the future.

What ended up working was to run spanning tree on the outer VLAN. What I find 
odd, being a Cisco graduate, is that the outer VLAN doesn't technically exist 
as a bridge domain... just as something VSTP uses.

protocols {
vstp {
vlan 2090 {
bridge-priority 4k;
interface ge-1/0/2;
interface ge-1/1/2;
}
}
}
interfaces {
ge-1/0/2 {
description GPON Card 1;
flexible-vlan-tagging;
encapsulation flexible-ethernet-services;
unit 2090 {
description Q-in-Q to subscriber;
vlan-id 2090;
family bridge {
interface-mode trunk;
inner-vlan-id-list [ 100 200 300 ];
}
}
}
ge-1/1/2 {
description GPON Card 12;
flexible-vlan-tagging;
encapsulation flexible-ethernet-services;
unit 2090 {
description Q-in-Q to subscriber;
vlan-id 2090;
family bridge {
interface-mode trunk;
inner-vlan-id-list [ 100 200 300 ];
}
}
}
}
bridge-domains {
Jimbob-Phones {
description Jimbob's IP phones;
vlan-id 100;
routing-interface irb.100;
}
Jimbob-Internet {
description Jimbob's Internet access;
vlan-id 200;
routing-interface irb.200;
}
Jimbob-TLS {
description Jimbob's L3VPN to remote offices;
vlan-id 300;
routing-interface irb.300;
}
}

Works great. VSTP on 2090 prevents loops between the MX and GPON access gear, 
and since the access gear strips the outer tag going to the subscriber the CE 
doesn't even know it's there. STP not required on the inner VLANs like I had 
originally tried.

Hope this helps someone.

Cheers
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] Q-in-Q with VSTP

[Ross Halliday]

Hi list,

First time caller, long time admirer (okay so we're a mostly Cisco shop and 
just got some Juniper stuff in the last few months - whatever...)

I'm bashing away at a conundrum here. I'm trying to lab a setup for a 
multi-VLAN subscriber over some GPON gear. The setup is:

MX104 --2x-- OLT -- ONT -- subscriber

The ONT is able to strip the outer VLAN tag facing the subscriber, so the CE 
can hit all of the inner VLANs directly. The OLT presents frames with both 
outer  and inner tags to the MX, so I must strip the outer in order to work 
with the subscriber's services. Link bundling/aggregation between the MX and 
OLT are not an option, so I need to use spanning tree.

So, I am trying to create a bridge domain with VSTP enabled for VLANs that 
aren't accessible with a simple vlan-id-list or vlan-id.

I cannot get this to work. I've tried vlan-tags outer 9 inner 9 with separate 
units per inner VLAN, as well as sending everything into a vswitch like 
detailed here 
http://www.juniper.net/techpubs/en_US/junos14.2/topics/topic-map/layer-2-services-stp-vstp-on-trunk-port-with-tagged-traffic-example.html
 ...but the STP instance just doesn't seem to recognize the interface, and my 
CE doesn't see any BPDUs on that VLAN.

I can't even find if this feature is supported - my searches on Juniper's site 
don't show anything with spanning tree and Q-in-Q or double-tags or inner tags 
etc on the same page.

Is anybody else trying to do this? Or suggested solution? If the MX supported 
Redundant Trunk Groups (like Cisco FlexLink) this would be so much easier.

Thanks
Ross
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp