Re: [j-nsp] MC-LAG reliability

[Vincent Bernat]

 ❦ 22 décembre 2016 15:15 +0100, Vincent Bernat  :

> How reliable should MC-LAG be considered on EX and QFX series (in a pure
> L2 setup)?
>
> I had a few bad experiences with virtual chassis where a hiccup usually
> translates to both switches becoming unavailable. This is pretty rare of
> course. MC-LAG would avoid those coordinated faults but is it otherwise
> as reliable as virtual chassis?

A quick feedback on my tests with MC-LAG: it doesn't work well for
me. While pure L2 operations worked fine, I had huge difficulties
running BGP sessions terminated at the IRB, either by using MAC
synchronization or VRRP (or neither of them). Local IP delivery on the
IRB seems patchy: the MAC address for local delivery to the remove
switch is learned on the ICCP VLAN instead of the IRB VLAN.

I didn't tried too hard, so maybe I missed something. I didn't want an
HA setup for local delivery, so I first tried with neither MAC sync nor
VRRP. Then I have tried MAC synchronization (which was an improvement
from "reproducibly don't work for some BGP sessions" to "work for all
BGP sessions but break after one hour"), then VRRP (without MAC
synchronization). After reverting the whole stuff, even L2 operations
didn't work correctly until a reboot. Maybe only MAC synchronization is
broken and VRRP would have worked.

The issues also affected ICMP handling.

Tested with 14.1X53-D35 on a pair of QFX5100.
-- 
Don't comment bad code - rewrite it.
- The Elements of Programming Style (Kernighan & Plauger)
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Aaron]

What about the ACX5048 ?  Do y'all know if it can do VC or MC-LAG yet in the 
newer version of Junos ?

- Aaron

-Original Message-
From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of 
Vincent Bernat
Sent: Thursday, December 22, 2016 2:29 PM
To: juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] MC-LAG reliability

Hey!

I also think that the VC is quite reliable. However, by design, it is a bit 
fragile. rpd can die and take the whole VC down. I also remember quite a few 
problems with upgrades but this is quite ancient, so maybe this doesn't apply 
any more.

I didn't test much, but even on the EX3300 with 15.5, you seem to have MC-LAG 
support (and no warnings from the CLI when using it). Dunno if this is recent 
or not.
--
Don't just echo the code with comments - make every comment count.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Raphael Mazelier <r...@futomaki.net>
 Sent: 22 décembre 2016 18:32 +0100
 Subject: Re: [j-nsp] MC-LAG reliability
 To: juniper-nsp@puck.nether.net

> Hey,
>
> My experience with VirtualChassis with a lot of them (you know where) 
> is globally positive. In fact I dot not remember when a VC completly 
> fail. This is not a perfect techno but it do the job for very low cost 
> of setup.
>
> On EX series you have no choice, afaik MC-LAG is not supported (unless 
> on highend series).
>
> On QFX I would hesitate. My tests are OK.
> Running independent switches is more reliable indeed, but even with 
> automation tool the cost of setup/maintenance is bit higher. (and in 
> my actual work I have just no time to spend with network config 
> unfortunately).
>
> --
> Raphael Mazelier
>
> On 22/12/2016 15:15, Vincent Bernat wrote:
>> Hey!
>>
>> How reliable should MC-LAG be considered on EX and QFX series (in a 
>> pure
>> L2 setup)?
>>
>> I had a few bad experiences with virtual chassis where a hiccup 
>> usually translates to both switches becoming unavailable. This is 
>> pretty rare of course. MC-LAG would avoid those coordinated faults 
>> but is it otherwise as reliable as virtual chassis?
>>
>> Thanks!
>>
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net 
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net 
https://puck.nether.net/mailman/listinfo/juniper-nsp

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Aaron]

S  if there are problems with VC , I don't want my EX4550's to hear 
about it LOL  They've been behaving just fine for so long I forgot they 
were there.

{master:1}
root@sabn-dcvc-4550> show system uptime | grep "days|fpc"
fpc0:
--
 7:13PM  up 1285 days,  6:57, 0 users, load averages: 0.16, 0.13, 0.09
fpc1:
--
 7:13PM  up 1285 days,  6:57, 2 users, load averages: 0.18, 0.15, 0.13

{master:1}
root@sabn-dcvc-4550>



{master:1}
root@stlr-dcvc-4550> show system uptime | grep "days|fpc"
fpc0:
--
 7:16PM  up 1289 days,  5:27, 0 users, load averages: 0.35, 0.21, 0.17
fpc1:
--
 7:16PM  up 1289 days,  5:48, 1 user, load averages: 0.10, 0.10, 0.09

{master:1}


- Aaron


-Original Message-
From: juniper-nsp [mailto:juniper-nsp-boun...@puck.nether.net] On Behalf Of 
Vincent Bernat
Sent: Thursday, December 22, 2016 2:29 PM
To: juniper-nsp@puck.nether.net
Subject: Re: [j-nsp] MC-LAG reliability

Hey!

I also think that the VC is quite reliable. However, by design, it is a bit 
fragile. rpd can die and take the whole VC down. I also remember quite a few 
problems with upgrades but this is quite ancient, so maybe this doesn't apply 
any more.

I didn't test much, but even on the EX3300 with 15.5, you seem to have MC-LAG 
support (and no warnings from the CLI when using it). Dunno if this is recent 
or not.
--
Don't just echo the code with comments - make every comment count.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Raphael Mazelier <r...@futomaki.net>
 Sent: 22 décembre 2016 18:32 +0100
 Subject: Re: [j-nsp] MC-LAG reliability
 To: juniper-nsp@puck.nether.net

> Hey,
>
> My experience with VirtualChassis with a lot of them (you know where) 
> is globally positive. In fact I dot not remember when a VC completly 
> fail. This is not a perfect techno but it do the job for very low cost 
> of setup.
>
> On EX series you have no choice, afaik MC-LAG is not supported (unless 
> on highend series).
>
> On QFX I would hesitate. My tests are OK.
> Running independent switches is more reliable indeed, but even with 
> automation tool the cost of setup/maintenance is bit higher. (and in 
> my actual work I have just no time to spend with network config 
> unfortunately).
>
> --
> Raphael Mazelier
>
> On 22/12/2016 15:15, Vincent Bernat wrote:
>> Hey!
>>
>> How reliable should MC-LAG be considered on EX and QFX series (in a 
>> pure
>> L2 setup)?
>>
>> I had a few bad experiences with virtual chassis where a hiccup 
>> usually translates to both switches becoming unavailable. This is 
>> pretty rare of course. MC-LAG would avoid those coordinated faults 
>> but is it otherwise as reliable as virtual chassis?
>>
>> Thanks!
>>
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net 
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net 
https://puck.nether.net/mailman/listinfo/juniper-nsp

___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Vincent Bernat]

Hey!

I also think that the VC is quite reliable. However, by design, it is a
bit fragile. rpd can die and take the whole VC down. I also remember
quite a few problems with upgrades but this is quite ancient, so maybe
this doesn't apply any more.

I didn't test much, but even on the EX3300 with 15.5, you seem to have
MC-LAG support (and no warnings from the CLI when using it). Dunno if
this is recent or not.
-- 
Don't just echo the code with comments - make every comment count.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Raphael Mazelier <r...@futomaki.net>
 Sent: 22 décembre 2016 18:32 +0100
 Subject: Re: [j-nsp] MC-LAG reliability
 To: juniper-nsp@puck.nether.net

> Hey,
>
> My experience with VirtualChassis with a lot of them (you know where)
> is globally positive. In fact I dot not remember when a VC completly
> fail. This is not a perfect techno but it do the job for very low cost
> of setup.
>
> On EX series you have no choice, afaik MC-LAG is not supported (unless
> on highend series).
>
> On QFX I would hesitate. My tests are OK.
> Running independent switches is more reliable indeed, but even with
> automation tool the cost of setup/maintenance is bit higher. (and in
> my actual work I have just no time to spend with network config
> unfortunately).
>
> --
> Raphael Mazelier
>
> On 22/12/2016 15:15, Vincent Bernat wrote:
>> Hey!
>>
>> How reliable should MC-LAG be considered on EX and QFX series (in a pure
>> L2 setup)?
>>
>> I had a few bad experiences with virtual chassis where a hiccup usually
>> translates to both switches becoming unavailable. This is pretty rare of
>> course. MC-LAG would avoid those coordinated faults but is it otherwise
>> as reliable as virtual chassis?
>>
>> Thanks!
>>
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Vincent Bernat]

Hey!

It's for a quite basic ToR setup: each server in the rack dual
linked. L2 only. No STP.

All servers have a couple of BGP sessions running through the
switches but they don't do any routing. Some of the QFX will act as
route reflectors, so they can have an IP on their IRB, but that should
be quite independant of this MC-LAG stuff, right?
-- 
Make sure comments and code agree.
- The Elements of Programming Style (Kernighan & Plauger)

 ――― Original Message ―――
 From: Alberto Santos <albertofsan...@gmail.com>
 Sent: 22 décembre 2016 17:29 +0100
 Subject: Re: [j-nsp] MC-LAG reliability
 To: Vincent Bernat
 Cc: juniper-nsp@puck.nether.net

> Hi,
>
> last time a colleague checked the PR list, VC had more issues than MC LAG.
> How your L2 setup looks like? Do you configure STP on your L2 network ? which 
> version? make sure the STP version you have today is officially
> supported by juniper when you implement it.
>
> if you have any routing protocol such as eBGP or OSPF runnung through these 
> switches, you should make sure you test before you convert it to a
> MC LAG setup.
>
> With juniper pyez, there is no excuse to keep this in a VC setup, you should 
> really consider MC LAG :), another good advantage that you get is
> ISSU instead of NSSU :D 
>
> cheers and merry xmas
>
> Alberto Santos CCIE #26648
> JNCIP-SP
> "...Fix your DNS, make it dual-stack, take your mail server and make it 
> dual-stack, take your web server and make it dual-stack..." by Randy
> Bush/RIPE IPv6
>
> On 22 December 2016 at 15:15, Vincent Bernat <ber...@luffy.cx> wrote:
>
>  Hey!
>
>  How reliable should MC-LAG be considered on EX and QFX series (in a pure
>  L2 setup)?
>
>  I had a few bad experiences with virtual chassis where a hiccup usually
>  translates to both switches becoming unavailable. This is pretty rare of
>  course. MC-LAG would avoid those coordinated faults but is it otherwise
>  as reliable as virtual chassis?
>
>  Thanks!
>  --
>  Many pages make a thick book, except for pocket Bibles which are on very
>  very thin paper.
>  ___
>  juniper-nsp mailing list juniper-nsp@puck.nether.net
>  https://puck.nether.net/mailman/listinfo/juniper-nsp
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Raphael Mazelier]

Hey,

My experience with VirtualChassis with a lot of them (you know where) is 
globally positive. In fact I dot not remember when a VC completly fail. 
This is not a perfect techno but it do the job for very low cost of setup.


On EX series you have no choice, afaik MC-LAG is not supported (unless 
on highend series).


On QFX I would hesitate. My tests are OK.
Running independent switches is more reliable indeed, but even with 
automation tool the cost of setup/maintenance is bit higher. (and in my 
actual work I have just no time to spend with network config 
unfortunately).


--
Raphael Mazelier

On 22/12/2016 15:15, Vincent Bernat wrote:

Hey!

How reliable should MC-LAG be considered on EX and QFX series (in a pure
L2 setup)?

I had a few bad experiences with virtual chassis where a hiccup usually
translates to both switches becoming unavailable. This is pretty rare of
course. MC-LAG would avoid those coordinated faults but is it otherwise
as reliable as virtual chassis?

Thanks!


___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Re: [j-nsp] MC-LAG reliability

[Alberto Santos]

Hi,

last time a colleague checked the PR list, VC had more issues than MC LAG.
How your L2 setup looks like? Do you configure STP on your L2 network ?
which version? make sure the STP version you have today is officially
supported by juniper when you implement it.

if you have any routing protocol such as eBGP or OSPF runnung through these
switches, you should make sure you test before you convert it to a MC LAG
setup.

With juniper pyez, there is no excuse to keep this in a VC setup, you
should really consider MC LAG :), another good advantage that you get is
ISSU instead of NSSU :D

cheers and merry xmas


Alberto Santos CCIE #26648
JNCIP-SP
*"...Fix your DNS, make it dual-stack, take your mail server and make it
dual-stack, take your web server and make it dual-stack..." by Randy
Bush/RIPE IPv6*

On 22 December 2016 at 15:15, Vincent Bernat  wrote:

> Hey!
>
> How reliable should MC-LAG be considered on EX and QFX series (in a pure
> L2 setup)?
>
> I had a few bad experiences with virtual chassis where a hiccup usually
> translates to both switches becoming unavailable. This is pretty rare of
> course. MC-LAG would avoid those coordinated faults but is it otherwise
> as reliable as virtual chassis?
>
> Thanks!
> --
> Many pages make a thick book, except for pocket Bibles which are on very
> very thin paper.
> ___
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[j-nsp] MC-LAG reliability

[Vincent Bernat]

Hey!

How reliable should MC-LAG be considered on EX and QFX series (in a pure
L2 setup)?

I had a few bad experiences with virtual chassis where a hiccup usually
translates to both switches becoming unavailable. This is pretty rare of
course. MC-LAG would avoid those coordinated faults but is it otherwise
as reliable as virtual chassis?

Thanks!
-- 
Many pages make a thick book, except for pocket Bibles which are on very
very thin paper.
___
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp