Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[John E Drake]

Weiguo,

Comment inline.

Yours Irrespectively,

John

> -Original Message-
> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Haoweiguo
> Sent: Saturday, November 21, 2015 2:18 AM
> To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy
> yong; Henderickx, Wim (Wim); bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> Hi Jim & John,
> Firstly i think option-C interconnection between cloud data center and MPLS
> VPN network exists. Secondly, i think for different encapsulations, different
> interconnecting methods are needed. For VXLAN/NVGRE, VTEP IP/UDP Port
> allocation solutions are needed.

[JD]  I don't think this is either the best or the only solution and I don't 
think we should be in any rush to adopt it. 

> Only for MPLSoGRE/oUDP, both standard
> MPLS+MPLS+GRE/UDP and VTEP IP/UDP Port allocation solution can work, it
> shoud be measured which one is more suitable or both solutions are
> proposed to the industry for reference.
> Thanks,
> weiguo
> 
> 
> From: BESS [bess-boun...@ietf.org] on behalf of UTTARO, JAMES
> [ju1...@att.com]
> Sent: Saturday, November 21, 2015 1:51
> To: John E Drake; EXT - thomas.mo...@orange.com; Lucy yong; Henderickx,
> Wim (Wim); bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> +1
> 
> -Original Message-
> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
> Sent: Friday, November 20, 2015 12:19 PM
> To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim);
> bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> Lucy,
> 
> My apologies, I misunderstood.
> 
> I think we have to accept the fact that we will have to deal with a 
> multiplicity
> of different encapsulations in the data plane along a packet's e2e path and
> that we should take a more measured approach to deciding how to deal with
> this in a general and extensible way before accepting any solutions.
> 
> Yours Irrespectively,
> 
> John
> 
> > -Original Message-
> > From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
> > Sent: Friday, November 20, 2015 12:04 PM
> > To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
> > Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >
> > 2015-11-20, John E Drake:
> > > That presupposes that the group likes either of the two proposed
> > > solutions
> > in your draft.
> >
> > John, I think Lucy's "two solutions" was referring to
> > draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label
> > Optionc MPLS/MPLS/UDP solution described by Wim.
> >
> > -Thomas
> >
> >
> >
> > >
> > >> -Original Message-
> > >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
> > >> Sent: Friday, November 20, 2015 11:49 AM
> > >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
> > >> bess@ietf.org
> > >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> > >>
> > >> Share my 2 cent.
> > >>
> > >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> > >> Option C is a way to realize it. Both solutions summarized by
> > >> Thomas have no change on WAN VPN side and seamlessly work with
> WAN
> > >> VPN
> > option C.
> > >> However, to support either solution, DC has to do some enhancement
> > >> on DC BR or ToR switch, etc, which dictates to different
> > >> implementations within a DC. Option C pro and con remains
> > >> regardless what implementation is used in a DC.
> > >>
> > >> Two solutions should not coexist in one DC (does not make a sense),
> > >> but it does not matter if one DC operator prefers to use one
> > >> solution and another DC prefers to use another solution. Since
> > >> there are many cloud providers today, it is worth for the WG to
> > >> document both solutions and point out the implementation
> > >> requirements on impacted components. Then, up to vendors and
> > >> operators to select a solution for
> > their DC.
> > >>
> > >> It does not make a sense for us to vote which solution is better
> > >> here because a better solution for a DC depends on many factors.
> > >>
> > >>
> > >> Lucy
> > >>
> > >>
> > >>
> > >>
> > >>
> > >&g

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

It also works with VXLAN/GPE btw. On top the solution I outlined is future 
proof as well.

The key question is do people want a native VXLAN/NVGRE solution in the DC 
using existing TOR HW and/or vwitches. This will imply dealing with:
- Global VNID/labels -> driven by current TOR HW (most people will never deploy 
this)
- We have not spoken about the MAC manipulation in all details but will most 
likely require a routing lookup in the TOR and will limit performance in some 
HW TOR(s).

- An implementation on ASBR that only is required for this use case. It will be 
very expensive to carry forward -> the cost versus the benefit is big and if 
people want a uniform data plane VXLAN-GPE is an option which is very close to 
native VXLAN.



On 20/11/15 22:21, "Haoweiguo" <haowei...@huawei.com> wrote:

>Jorge,
>We all know that Wim's MPLS/MPLS/UDP solution works, but it's not the only 
>choice. Wim's solution requires MPLSoGRE/UDP encap in data center, but many 
>data centers only use VXLAN/NVGRE encapsulation for both north-south and 
>east-west bound traffic, how to interconnect with outside MPLS VPN network for 
>these data centers? So VXLAN/NVGRE and MPLS VPN network interworking is needed.
>And for the interconnection solution, we suggest both no TOR/NVE hardware 
>enhancement solution and future proof solution should be provided.
>Thanks,
>weiguo
>
>From: BESS [bess-boun...@ietf.org] on behalf of Rabadan, Jorge (Jorge) 
>[jorge.raba...@alcatel-lucent.com]
>Sent: Saturday, November 21, 2015 3:31
>To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy yong; 
>Henderickx, Wim (Wim); bess@ietf.org
>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
>IMHO if TOR chip vendors can confirm they are seriously looking at 
>MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
>scales.
>My 2 cents.
>
>Jorge
>
>
>
>On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org 
>on behalf of ju1...@att.com> wrote:
>
>>+1
>>
>>-Original Message-
>>From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
>>Sent: Friday, November 20, 2015 12:19 PM
>>To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
>>bess@ietf.org
>>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>
>>Lucy,
>>
>>My apologies, I misunderstood.
>>
>>I think we have to accept the fact that we will have to deal with a 
>>multiplicity of different encapsulations in the data plane along a packet's 
>>e2e path and that we should take a more measured approach to deciding how to 
>>deal with this in a general and extensible way before accepting any solutions.
>>
>>Yours Irrespectively,
>>
>>John
>>
>>> -Original Message-
>>> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
>>> Sent: Friday, November 20, 2015 12:04 PM
>>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>>
>>> 2015-11-20, John E Drake:
>>> > That presupposes that the group likes either of the two proposed solutions
>>> in your draft.
>>>
>>> John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter-
>>> nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution
>>> described by Wim.
>>>
>>> -Thomas
>>>
>>>
>>>
>>> >
>>> >> -Original Message-
>>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
>>> >> Sent: Friday, November 20, 2015 11:49 AM
>>> >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
>>> >> bess@ietf.org
>>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>> >>
>>> >> Share my 2 cent.
>>> >>
>>> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>>> >> Option C is a way to realize it. Both solutions summarized by Thomas
>>> >> have no change on WAN VPN side and seamlessly work with WAN VPN
>>> option C.
>>> >> However, to support either solution, DC has to do some enhancement on
>>> >> DC BR or ToR switch, etc, which dictates to different implementations
>>> >> within a DC. Option C pro and con remains regardless what
>>> >> implementation is used in a DC.
>>> >>
>>> >> Two solutions should not coexist in 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

In-line with WH>




On 20/11/15 01:01, "BESS on behalf of thomas.mo...@orange.com" 
<bess-boun...@ietf.org on behalf of thomas.mo...@orange.com> wrote:

>2015-11-19, Henderickx, Wim (Wim):
>> WH> I vote for a an evolution of switches/TORs that have proper
>> support for this. I hope some HW vendors of TOR chips shime in, but I
>> am told the MPLS solution is possible in the next generation chips
>> they are working on.
>
>Well, it looks like the key questions are:
>- when would ToR chips support MPLS/MPLS/UDP ?  (the generation that has 
>been released recently but not present in most shipping ToRs yet, the 
>next one ?)
WH> the next generation TOR chips seem to support it, at least this is the info 
I got from 3 vendors
>- do we want an interim VXLAN-based solution ? (that will involve at 
>best a performance penalty with existing chips, and at worse impossible 
>to implement -- we haven't seen clear information in this thread)
WH> in my view no, since we also have the global VNID issue and a continuous 
tax and ASBR boxes for this.
>
>-Thomas
>
>
>>> Zhuangshunwan  :
>>>>
>>>> Hi Diego,
>>>>
>>>> Thanks for your comments. Pls see inline with [Vincent].
>>>>
>>>> Vincent
>>>>
>>>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del
>>>> Rio *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题:*Re: [bess]
>>>> draft-hao-bess-inter-nvo3-vpn-optionc
>>>>
>>>> Some comments from my side, I think the draft makes quite a few
>>>> assumptions on specific implementation details that are way too
>>>> general to be considered widely available. Most of the TOR
>>>> devices today already pay a double-pass penalty when doing
>>>> routing of traffic in/out of vxlan-type tunnels. Only the newest
>>>> generation can route into tunnels without additional passes. And
>>>> are definitively limited in being able to arbitrary select UDP
>>>> ports on a per tunnel basis. In fact, most are even limited at
>>>> using more than one VNID per "service" of sorts. [Vincent]: Yes,
>>>> the new generation BCM chipset has already supported VXLAN
>>>> routing without additional passes. For OVS/TOR, VXLAN
>>>> encapsulation is more popular than MPLSoGRE/UDP, and has better
>>>> performance. The IP-addressed based implementation which would, I
>>>> assume, imply assigning one or more CIDRs to a loopback interface
>>>> on the ASBR-d is also quite arbitrary and does not seem like a
>>>> technically "clean" solution. (besides burning tons of IPs). As a
>>>> side-note, most PE-grade routers i've worked with were quite
>>>> limited in terms of IP addresses used for tunnel termination and
>>>> it wasn't that just a simple pool can be used. [Vincent]: I think
>>>> the larger VTEP IP address range on ASBR-d has no limitations.
>>>> For the hardware on ASBR-d, it has capability to terminate
>>>> multiple VXLAN tunnels with arbitrary destination VTEP IP
>>>> addresses. Wim's mentions on cases where the Application itself,
>>>> hosted in a datacenter, would be part of the option-C
>>>> interconnect, was dismissed without much discussion so far,
>>>> while, if we look in detail at the type of users which will even
>>>> consider a complex topology like model-C its most likely users
>>>> and operators very familiar with MPLS VPNs in the WAN. Those type
>>>> of operators will most likely be very interested in deploying
>>>> MPLS or WAN-grade applications (i.e., virtual-routers or other
>>>> VNFs) in the DC and thus its highly likely that the interconnect
>>>> would not terminate at the NVE itself but rather the TS (the
>>>> virtual machine). Also, the use of UDP ports at random would
>>>> imply quite complex logic on the ASBR-d IMHO. Im not saying its
>>>> impossible, but since the received packet now not only has to be
>>>> received on a random (though locally chosen) UDP port and this
>>>> information preserved in the pipeline to be able to do the
>>>> double-tunnel-stitching, it sounds quite complex. I am sure
>>>> someone in the list will mention this has already been
>>>> implemented somewhere, and I won't argue with that. And let's
>>>> not even bring into account what this would do to any DC
>>>> middlebox that now has to look at vxlan over almost any random
>>>> port. We have to go b

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

On 20/11/15 01:05, "thomas.mo...@orange.com" <thomas.mo...@orange.com> wrote:


>2015-11-20, Haoweiguo:
>> WH> ok now we have not discussed the constraints some HW vendors have
>> with respect to global VNIDs. To make this work all VNID/Labels need
>> to be globally unique. Hm
> > [weiguo2]:  In SDN scenario, a virtual
>> network normally is represented by a global VN ID or MPLS VPN Label
>> to simplify network management. I think it's not strange to allocate
>> global unique MPLS VPN Label or VN ID for a virtual network now.
>
>In an option C scenario you have to take into account what is done in 
>the WAN. Globally-assigned labels are not an option there at all.

WH> Agreed. As I said before the proposal in the draft has too many 
implications/restrictions in existing network deployments + requires a 
dedicated data plane in ASBR devices, which we cannot reuse and will add a 
continuous tax in SW/HW (add cost for nothing).
This is why I am against this draft and we should drive the industry to a 
proper solution that does not have these implications.
 
>
>-Thomas
>
>
>>>
>>> The alternative approach, put forward by Wim, consist in using existing
>>> Option-C with a variant of the 3-label variant relying on an
>>> MPLS/MPLS/UDP-or-GRE encap (instead of the 3-label MPLS stack).  This
>>> approach would not necessitate new standardized procedures, would
>>> require less changes on ASBRs. However it is not supported today by
>>> vswitches or ToRs (unless the bottom MPLS label is passed to the VM,
>>> which I think would restrict the application to a subset of the
>>> scenarios for which Option C is interesting). Having it supported in
>>> vswitches may be a simple matter of writing the software, but ToR
>>> support seems to require evolution in chipsets ; whether such a support
>>> is likely to appear hasn't been discussed in the thread.
>>>
>>> All in all, it seems there is no solution to cover an Option C scenario
>>> with today's generation ToRs, and the question for tomorrow's generation
>>> ToRs hasn't been really discussed.
>>>
>>> Based on the above, I would think the question boils down to whether it
>>> is desirable to specify an Option C variant usable with vswitches as
>>> they are today and possibly usable with a performance hit with today's
>>> ToRs chipsets, at the expense of a required evolution on ASBRs.  The
>>> alternative requiring some evolution on vswitches and waiting for future
>>> ToRs chipsets.
>> WH> I vote for a an evolution of switches/TORs that have proper support for 
>> this.
>> I hope some HW vendors of TOR chips shime in, but I am told the MPLS 
>> solution is possible in the next generation chips they are working on.
>> [weiguo2]:  I think it's better not to change current TOR hardware to 
>> realize option-C interconnection. No TOR hardware modification solution must 
>> be provided. But for future case, we can propose extra hardware requirements 
>> for TORs.
>> The other options requires baggage forever on the ASBR elements that does 
>> not bring value and it is better to avoid this and build an architecture 
>> which is future proof and more cost effective. My 2 cents
>>>
>>> -Thomas
>>
>>>
>>> Zhuangshunwan  :
>>>>
>>>> Hi Diego,
>>>>
>>>> Thanks for your comments. Pls see inline with [Vincent].
>>>>
>>>> Vincent
>>>>
>>>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del Rio
>>>> *发送时间:*2015年11月18日14:25
>>>> *收件人:*bess@ietf.org
>>>> *主题:*Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>>>
>>>> Some comments from my side,
>>>> I think the draft makes quite a few assumptions on specific
>>>> implementation details that are way too general to be considered
>>>> widely available.
>>>> Most of the TOR devices today already pay a double-pass penalty when
>>>> doing routing of traffic in/out of vxlan-type tunnels. Only the newest
>>>> generation can route into tunnels without additional passes. And are
>>>> definitively limited in being able to arbitrary select UDP ports on a
>>>> per tunnel basis. In fact, most are even limited at using more than
>>>> one VNID per "service" of sorts.
>>>> [Vincent]: Yes, the new generation BCM chipset has already supported
>>>> VXLAN routing without additional passes. For OVS/TOR, VXLAN
>>>> encapsulation i

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[thomas.morin]

2015-11-20, John E Drake:

That presupposes that the group likes either of the two proposed solutions in 
your draft.


John, I think Lucy's "two solutions" was referring to 
draft-hao-bess-inter-nvo3-vpn-optionc solution and the 3-label Optionc 
MPLS/MPLS/UDP solution described by Wim.


-Thomas






-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
Sent: Friday, November 20, 2015 11:49 AM
To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Share my 2 cent.

Cloud providers want to tunnel its customer traffic through DC (AS)BR.
Option C is a way to realize it. Both solutions summarized by Thomas have no
change on WAN VPN side and seamlessly work with WAN VPN option C.
However, to support either solution, DC has to do some enhancement on DC
BR or ToR switch, etc, which dictates to different implementations within a
DC. Option C pro and con remains regardless what implementation is used in
a DC.

Two solutions should not coexist in one DC (does not make a sense), but it
does not matter if one DC operator prefers to use one solution and another
DC prefers to use another solution. Since there are many cloud providers
today, it is worth for the WG to document both solutions and point out the
implementation requirements on impacted components. Then, up to
vendors and operators to select a solution for their DC.

It does not make a sense for us to vote which solution is better here because
a better solution for a DC depends on many factors.


Lucy





-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of
thomas.mo...@orange.com
Sent: Friday, November 20, 2015 3:02 AM
To: Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-19, Henderickx, Wim (Wim):

WH> I vote for a an evolution of switches/TORs that have proper
support for this. I hope some HW vendors of TOR chips shime in, but I
am told the MPLS solution is possible in the next generation chips
they are working on.


Well, it looks like the key questions are:
- when would ToR chips support MPLS/MPLS/UDP ?  (the generation that has
been released recently but not present in most shipping ToRs yet, the next
one ?)
- do we want an interim VXLAN-based solution ? (that will involve at best a
performance penalty with existing chips, and at worse impossible to
implement -- we haven't seen clear information in this thread)

-Thomas



Zhuangshunwan  :


Hi Diego,

Thanks for your comments. Pls see inline with [Vincent].

Vincent

*发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia

del Rio

*发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题

:*Re: [bess]

draft-hao-bess-inter-nvo3-vpn-optionc

Some comments from my side, I think the draft makes quite a few
assumptions on specific implementation details that are way too
general to be considered widely available. Most of the TOR devices
today already pay a double-pass penalty when doing routing of
traffic in/out of vxlan-type tunnels. Only the newest generation can
route into tunnels without additional passes. And are definitively
limited in being able to arbitrary select UDP ports on a per tunnel
basis. In fact, most are even limited at using more than one VNID
per "service" of sorts. [Vincent]: Yes, the new generation BCM
chipset has already supported VXLAN routing without additional
passes. For OVS/TOR, VXLAN encapsulation is more popular than
MPLSoGRE/UDP, and has better performance. The IP-addressed based
implementation which would, I assume, imply assigning one or more
CIDRs to a loopback interface on the ASBR-d is also quite arbitrary
and does not seem like a technically "clean" solution. (besides
burning tons of IPs). As a side-note, most PE-grade routers i've
worked with were quite limited in terms of IP addresses used for
tunnel termination and it wasn't that just a simple pool can be
used. [Vincent]: I think the larger VTEP IP address range on ASBR-d
has no limitations.
For the hardware on ASBR-d, it has capability to terminate multiple
VXLAN tunnels with arbitrary destination VTEP IP addresses. Wim's
mentions on cases where the Application itself, hosted in a
datacenter, would be part of the option-C interconnect, was
dismissed without much discussion so far, while, if we look in
detail at the type of users which will even consider a complex
topology like model-C its most likely users and operators very
familiar with MPLS VPNs in the WAN. Those type of operators will
most likely be very interested in deploying MPLS or WAN-grade
applications (i.e., virtual-routers or other
VNFs) in the DC and thus its highly likely that the interconnect
would not terminate at the NVE itself but rather the TS (the virtual
machine). Also, the use of UDP ports at random would imply quite
complex logic on the ASBR-d IMHO. Im not saying its impossible, but
since the received packet now not on

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Share my 2 cent.

Cloud providers want to tunnel its customer traffic through DC (AS)BR. Option C 
is a way to realize it. Both solutions summarized by Thomas have no change on 
WAN VPN side and seamlessly work with WAN VPN option C. However, to support 
either solution, DC has to do some enhancement on DC BR or ToR switch, etc, 
which dictates to different implementations within a DC. Option C pro and con 
remains regardless what implementation is used in a DC. 

Two solutions should not coexist in one DC (does not make a sense), but it does 
not matter if one DC operator prefers to use one solution and another DC 
prefers to use another solution. Since there are many cloud providers today, it 
is worth for the WG to document both solutions and point out the implementation 
requirements on impacted components. Then, up to vendors and operators to 
select a solution for their DC.

It does not make a sense for us to vote which solution is better here because a 
better solution for a DC depends on many factors.


Lucy



  

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of thomas.mo...@orange.com
Sent: Friday, November 20, 2015 3:02 AM
To: Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-19, Henderickx, Wim (Wim):
> WH> I vote for a an evolution of switches/TORs that have proper
> support for this. I hope some HW vendors of TOR chips shime in, but I 
> am told the MPLS solution is possible in the next generation chips 
> they are working on.

Well, it looks like the key questions are:
- when would ToR chips support MPLS/MPLS/UDP ?  (the generation that has been 
released recently but not present in most shipping ToRs yet, the next one ?)
- do we want an interim VXLAN-based solution ? (that will involve at best a 
performance penalty with existing chips, and at worse impossible to implement 
-- we haven't seen clear information in this thread)

-Thomas


>> Zhuangshunwan  :
>>>
>>> Hi Diego,
>>>
>>> Thanks for your comments. Pls see inline with [Vincent].
>>>
>>> Vincent
>>>
>>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del Rio 
>>> *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题:*Re: [bess] 
>>> draft-hao-bess-inter-nvo3-vpn-optionc
>>>
>>> Some comments from my side, I think the draft makes quite a few 
>>> assumptions on specific implementation details that are way too 
>>> general to be considered widely available. Most of the TOR devices 
>>> today already pay a double-pass penalty when doing routing of 
>>> traffic in/out of vxlan-type tunnels. Only the newest generation can 
>>> route into tunnels without additional passes. And are definitively 
>>> limited in being able to arbitrary select UDP ports on a per tunnel 
>>> basis. In fact, most are even limited at using more than one VNID 
>>> per "service" of sorts. [Vincent]: Yes, the new generation BCM 
>>> chipset has already supported VXLAN routing without additional 
>>> passes. For OVS/TOR, VXLAN encapsulation is more popular than 
>>> MPLSoGRE/UDP, and has better performance. The IP-addressed based 
>>> implementation which would, I assume, imply assigning one or more 
>>> CIDRs to a loopback interface on the ASBR-d is also quite arbitrary 
>>> and does not seem like a technically "clean" solution. (besides 
>>> burning tons of IPs). As a side-note, most PE-grade routers i've 
>>> worked with were quite limited in terms of IP addresses used for 
>>> tunnel termination and it wasn't that just a simple pool can be 
>>> used. [Vincent]: I think the larger VTEP IP address range on ASBR-d 
>>> has no limitations.
>>> For the hardware on ASBR-d, it has capability to terminate multiple 
>>> VXLAN tunnels with arbitrary destination VTEP IP addresses. Wim's 
>>> mentions on cases where the Application itself, hosted in a 
>>> datacenter, would be part of the option-C interconnect, was 
>>> dismissed without much discussion so far, while, if we look in 
>>> detail at the type of users which will even consider a complex 
>>> topology like model-C its most likely users and operators very 
>>> familiar with MPLS VPNs in the WAN. Those type of operators will 
>>> most likely be very interested in deploying MPLS or WAN-grade 
>>> applications (i.e., virtual-routers or other
>>> VNFs) in the DC and thus its highly likely that the interconnect 
>>> would not terminate at the NVE itself but rather the TS (the virtual 
>>> machine). Also, the use of UDP ports at random would imply quite 
>>> complex logic on the ASBR-d IMHO

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

That is my 2 cents. Anyone can share his/her opinion. :)
Lucy

-Original Message-
From: John E Drake [mailto:jdr...@juniper.net] 
Sent: Friday, November 20, 2015 10:54 AM
To: Lucy yong; EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); 
bess@ietf.org
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

That presupposes that the group likes either of the two proposed solutions in 
your draft.  

Yours Irrespectively,

John

> -Original Message-
> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
> Sent: Friday, November 20, 2015 11:49 AM
> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); 
> bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> Share my 2 cent.
> 
> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> Option C is a way to realize it. Both solutions summarized by Thomas 
> have no change on WAN VPN side and seamlessly work with WAN VPN option C.
> However, to support either solution, DC has to do some enhancement on 
> DC BR or ToR switch, etc, which dictates to different implementations 
> within a DC. Option C pro and con remains regardless what 
> implementation is used in a DC.
> 
> Two solutions should not coexist in one DC (does not make a sense), 
> but it does not matter if one DC operator prefers to use one solution 
> and another DC prefers to use another solution. Since there are many 
> cloud providers today, it is worth for the WG to document both 
> solutions and point out the implementation requirements on impacted 
> components. Then, up to vendors and operators to select a solution for their 
> DC.
> 
> It does not make a sense for us to vote which solution is better here 
> because a better solution for a DC depends on many factors.
> 
> 
> Lucy
> 
> 
> 
> 
> 
> -Original Message-
> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of 
> thomas.mo...@orange.com
> Sent: Friday, November 20, 2015 3:02 AM
> To: Henderickx, Wim (Wim); bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> 2015-11-19, Henderickx, Wim (Wim):
> > WH> I vote for a an evolution of switches/TORs that have proper
> > support for this. I hope some HW vendors of TOR chips shime in, but 
> > I am told the MPLS solution is possible in the next generation chips 
> > they are working on.
> 
> Well, it looks like the key questions are:
> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that 
> has been released recently but not present in most shipping ToRs yet, 
> the next one ?)
> - do we want an interim VXLAN-based solution ? (that will involve at 
> best a performance penalty with existing chips, and at worse 
> impossible to implement -- we haven't seen clear information in this 
> thread)
> 
> -Thomas
> 
> 
> >> Zhuangshunwan  :
> >>>
> >>> Hi Diego,
> >>>
> >>> Thanks for your comments. Pls see inline with [Vincent].
> >>>
> >>> Vincent
> >>>
> >>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia
> del Rio
> >>> *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题
> :*Re: [bess]
> >>> draft-hao-bess-inter-nvo3-vpn-optionc
> >>>
> >>> Some comments from my side, I think the draft makes quite a few 
> >>> assumptions on specific implementation details that are way too 
> >>> general to be considered widely available. Most of the TOR devices 
> >>> today already pay a double-pass penalty when doing routing of 
> >>> traffic in/out of vxlan-type tunnels. Only the newest generation 
> >>> can route into tunnels without additional passes. And are 
> >>> definitively limited in being able to arbitrary select UDP ports 
> >>> on a per tunnel basis. In fact, most are even limited at using 
> >>> more than one VNID per "service" of sorts. [Vincent]: Yes, the new 
> >>> generation BCM chipset has already supported VXLAN routing without 
> >>> additional passes. For OVS/TOR, VXLAN encapsulation is more 
> >>> popular than MPLSoGRE/UDP, and has better performance. The 
> >>> IP-addressed based implementation which would, I assume, imply 
> >>> assigning one or more CIDRs to a loopback interface on the ASBR-d 
> >>> is also quite arbitrary and does not seem like a technically 
> >>> "clean" solution. (besides burning tons of IPs). As a side-note, 
> >>> most PE-grade routers i've worked with were quite limited in terms 
> >>> of IP addresses used for tunnel termination and it wasn't that 
&g

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[UTTARO, JAMES]

+1

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
Sent: Friday, November 20, 2015 12:19 PM
To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

My apologies, I misunderstood.

I think we have to accept the fact that we will have to deal with a 
multiplicity of different encapsulations in the data plane along a packet's e2e 
path and that we should take a more measured approach to deciding how to deal 
with this in a general and extensible way before accepting any solutions. 
  
Yours Irrespectively,

John

> -Original Message-
> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
> Sent: Friday, November 20, 2015 12:04 PM
> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> 
> 2015-11-20, John E Drake:
> > That presupposes that the group likes either of the two proposed solutions
> in your draft.
> 
> John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter-
> nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution
> described by Wim.
> 
> -Thomas
> 
> 
> 
> >
> >> -Original Message-
> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
> >> Sent: Friday, November 20, 2015 11:49 AM
> >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
> >> bess@ietf.org
> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >>
> >> Share my 2 cent.
> >>
> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
> >> Option C is a way to realize it. Both solutions summarized by Thomas
> >> have no change on WAN VPN side and seamlessly work with WAN VPN
> option C.
> >> However, to support either solution, DC has to do some enhancement on
> >> DC BR or ToR switch, etc, which dictates to different implementations
> >> within a DC. Option C pro and con remains regardless what
> >> implementation is used in a DC.
> >>
> >> Two solutions should not coexist in one DC (does not make a sense),
> >> but it does not matter if one DC operator prefers to use one solution
> >> and another DC prefers to use another solution. Since there are many
> >> cloud providers today, it is worth for the WG to document both
> >> solutions and point out the implementation requirements on impacted
> >> components. Then, up to vendors and operators to select a solution for
> their DC.
> >>
> >> It does not make a sense for us to vote which solution is better here
> >> because a better solution for a DC depends on many factors.
> >>
> >>
> >> Lucy
> >>
> >>
> >>
> >>
> >>
> >> -Original Message-
> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of
> >> thomas.mo...@orange.com
> >> Sent: Friday, November 20, 2015 3:02 AM
> >> To: Henderickx, Wim (Wim); bess@ietf.org
> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >>
> >> 2015-11-19, Henderickx, Wim (Wim):
> >>> WH> I vote for a an evolution of switches/TORs that have proper
> >>> support for this. I hope some HW vendors of TOR chips shime in, but
> >>> I am told the MPLS solution is possible in the next generation chips
> >>> they are working on.
> >>
> >> Well, it looks like the key questions are:
> >> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that
> >> has been released recently but not present in most shipping ToRs yet,
> >> the next one ?)
> >> - do we want an interim VXLAN-based solution ? (that will involve at
> >> best a performance penalty with existing chips, and at worse
> >> impossible to implement -- we haven't seen clear information in this
> >> thread)
> >>
> >> -Thomas
> >>
> >>
> >>>> Zhuangshunwan  :
> >>>>>
> >>>>> Hi Diego,
> >>>>>
> >>>>> Thanks for your comments. Pls see inline with [Vincent].
> >>>>>
> >>>>> Vincent
> >>>>>
> >>>>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia
> >> del Rio
> >>>>> *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题
> >> :*Re: [bess]
> >>>>> draft-hao-bess-inter-nvo3-vpn-optionc
> >>>>>
> >>>>> Som

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Rabadan, Jorge (Jorge)]

IMHO if TOR chip vendors can confirm they are seriously looking at 
MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org 
on behalf of ju1...@att.com> wrote:

>+1
>
>-Original Message-
>From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
>Sent: Friday, November 20, 2015 12:19 PM
>To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
>bess@ietf.org
>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
>Lucy,
>
>My apologies, I misunderstood.
>
>I think we have to accept the fact that we will have to deal with a 
>multiplicity of different encapsulations in the data plane along a packet's 
>e2e path and that we should take a more measured approach to deciding how to 
>deal with this in a general and extensible way before accepting any solutions. 
>  
>Yours Irrespectively,
>
>John
>
>> -Original Message-
>> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
>> Sent: Friday, November 20, 2015 12:04 PM
>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> 
>> 2015-11-20, John E Drake:
>> > That presupposes that the group likes either of the two proposed solutions
>> in your draft.
>> 
>> John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter-
>> nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution
>> described by Wim.
>> 
>> -Thomas
>> 
>> 
>> 
>> >
>> >> -Original Message-----
>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
>> >> Sent: Friday, November 20, 2015 11:49 AM
>> >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
>> >> bess@ietf.org
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> Share my 2 cent.
>> >>
>> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>> >> Option C is a way to realize it. Both solutions summarized by Thomas
>> >> have no change on WAN VPN side and seamlessly work with WAN VPN
>> option C.
>> >> However, to support either solution, DC has to do some enhancement on
>> >> DC BR or ToR switch, etc, which dictates to different implementations
>> >> within a DC. Option C pro and con remains regardless what
>> >> implementation is used in a DC.
>> >>
>> >> Two solutions should not coexist in one DC (does not make a sense),
>> >> but it does not matter if one DC operator prefers to use one solution
>> >> and another DC prefers to use another solution. Since there are many
>> >> cloud providers today, it is worth for the WG to document both
>> >> solutions and point out the implementation requirements on impacted
>> >> components. Then, up to vendors and operators to select a solution for
>> their DC.
>> >>
>> >> It does not make a sense for us to vote which solution is better here
>> >> because a better solution for a DC depends on many factors.
>> >>
>> >>
>> >> Lucy
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -Original Message-
>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of
>> >> thomas.mo...@orange.com
>> >> Sent: Friday, November 20, 2015 3:02 AM
>> >> To: Henderickx, Wim (Wim); bess@ietf.org
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> 2015-11-19, Henderickx, Wim (Wim):
>> >>> WH> I vote for a an evolution of switches/TORs that have proper
>> >>> support for this. I hope some HW vendors of TOR chips shime in, but
>> >>> I am told the MPLS solution is possible in the next generation chips
>> >>> they are working on.
>> >>
>> >> Well, it looks like the key questions are:
>> >> - when would ToR chips support MPLS/MPLS/UDP ?  (the generation that
>> >> has been released recently but not present in most shipping ToRs yet,
>> >> the next one ?)
>> >> - do we want an interim VXLAN-based solution ? (that will involve at
>> >> best a performance penalty with existing chips, and at worse
>> >> impossible to implement -- we haven't seen clear information in this
>> 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

IMHO: voting on this thread does not make a sense. Both solutions will work and 
scales.

Lucy 

-Original Message-
From: Rabadan, Jorge (Jorge) [mailto:jorge.raba...@alcatel-lucent.com] 
Sent: Friday, November 20, 2015 1:32 PM
To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy yong; 
Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

IMHO if TOR chip vendors can confirm they are seriously looking at 
MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org 
on behalf of ju1...@att.com> wrote:

>+1
>
>-Original Message-
>From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
>Sent: Friday, November 20, 2015 12:19 PM
>To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
>bess@ietf.org
>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
>Lucy,
>
>My apologies, I misunderstood.
>
>I think we have to accept the fact that we will have to deal with a 
>multiplicity of different encapsulations in the data plane along a packet's 
>e2e path and that we should take a more measured approach to deciding how to 
>deal with this in a general and extensible way before accepting any solutions. 
>  
>Yours Irrespectively,
>
>John
>
>> -Original Message-
>> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
>> Sent: Friday, November 20, 2015 12:04 PM
>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> 
>> 2015-11-20, John E Drake:
>> > That presupposes that the group likes either of the two proposed 
>> > solutions
>> in your draft.
>> 
>> John, I think Lucy's "two solutions" was referring to 
>> draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label 
>> Optionc MPLS/MPLS/UDP solution described by Wim.
>> 
>> -Thomas
>> 
>> 
>> 
>> >
>> >> -Original Message-----
>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
>> >> Sent: Friday, November 20, 2015 11:49 AM
>> >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); 
>> >> bess@ietf.org
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> Share my 2 cent.
>> >>
>> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>> >> Option C is a way to realize it. Both solutions summarized by 
>> >> Thomas have no change on WAN VPN side and seamlessly work with WAN 
>> >> VPN
>> option C.
>> >> However, to support either solution, DC has to do some enhancement 
>> >> on DC BR or ToR switch, etc, which dictates to different 
>> >> implementations within a DC. Option C pro and con remains 
>> >> regardless what implementation is used in a DC.
>> >>
>> >> Two solutions should not coexist in one DC (does not make a 
>> >> sense), but it does not matter if one DC operator prefers to use 
>> >> one solution and another DC prefers to use another solution. Since 
>> >> there are many cloud providers today, it is worth for the WG to 
>> >> document both solutions and point out the implementation 
>> >> requirements on impacted components. Then, up to vendors and 
>> >> operators to select a solution for
>> their DC.
>> >>
>> >> It does not make a sense for us to vote which solution is better 
>> >> here because a better solution for a DC depends on many factors.
>> >>
>> >>
>> >> Lucy
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -Original Message-
>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of 
>> >> thomas.mo...@orange.com
>> >> Sent: Friday, November 20, 2015 3:02 AM
>> >> To: Henderickx, Wim (Wim); bess@ietf.org
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> 2015-11-19, Henderickx, Wim (Wim):
>> >>> WH> I vote for a an evolution of switches/TORs that have proper
>> >>> support for this. I hope some HW vendors of TOR chips shime in, 
>> >>> but I am told the MPLS solution is possible in the next 
>> >>> generation chips they are working on.
>> >>
>> >> Well, it looks like the key quest

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Martin Vigoureux]

Lucy,

there is no such thing as voting in IETF WGs
And I haven't seen anything like voting as part of this discussion.

Thank you
Martin

Le 20/11/2015 20:57, Lucy yong a écrit :

IMHO: voting on this thread does not make a sense. Both solutions will work and 
scales.

Lucy

-Original Message-
From: Rabadan, Jorge (Jorge) [mailto:jorge.raba...@alcatel-lucent.com]
Sent: Friday, November 20, 2015 1:32 PM
To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy yong; 
Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

IMHO if TOR chip vendors can confirm they are seriously looking at 
MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org on 
behalf of ju1...@att.com> wrote:


+1

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
Sent: Friday, November 20, 2015 12:19 PM
To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim);
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

My apologies, I misunderstood.

I think we have to accept the fact that we will have to deal with a 
multiplicity of different encapsulations in the data plane along a packet's e2e 
path and that we should take a more measured approach to deciding how to deal 
with this in a general and extensible way before accepting any solutions.

Yours Irrespectively,

John


-Original Message-
From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
Sent: Friday, November 20, 2015 12:04 PM
To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-20, John E Drake:

That presupposes that the group likes either of the two proposed
solutions

in your draft.

John, I think Lucy's "two solutions" was referring to
draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label
Optionc MPLS/MPLS/UDP solution described by Wim.

-Thomas






-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
Sent: Friday, November 20, 2015 11:49 AM
To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Share my 2 cent.

Cloud providers want to tunnel its customer traffic through DC (AS)BR.
Option C is a way to realize it. Both solutions summarized by
Thomas have no change on WAN VPN side and seamlessly work with WAN
VPN

option C.

However, to support either solution, DC has to do some enhancement
on DC BR or ToR switch, etc, which dictates to different
implementations within a DC. Option C pro and con remains
regardless what implementation is used in a DC.

Two solutions should not coexist in one DC (does not make a
sense), but it does not matter if one DC operator prefers to use
one solution and another DC prefers to use another solution. Since
there are many cloud providers today, it is worth for the WG to
document both solutions and point out the implementation
requirements on impacted components. Then, up to vendors and
operators to select a solution for

their DC.


It does not make a sense for us to vote which solution is better
here because a better solution for a DC depends on many factors.


Lucy





-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of
thomas.mo...@orange.com
Sent: Friday, November 20, 2015 3:02 AM
To: Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-19, Henderickx, Wim (Wim):

WH> I vote for a an evolution of switches/TORs that have proper
support for this. I hope some HW vendors of TOR chips shime in,
but I am told the MPLS solution is possible in the next
generation chips they are working on.


Well, it looks like the key questions are:
- when would ToR chips support MPLS/MPLS/UDP ?  (the generation
that has been released recently but not present in most shipping
ToRs yet, the next one ?)
- do we want an interim VXLAN-based solution ? (that will involve
at best a performance penalty with existing chips, and at worse
impossible to implement -- we haven't seen clear information in
this
thread)

-Thomas



Zhuangshunwan  :


Hi Diego,

Thanks for your comments. Pls see inline with [Vincent].

Vincent

*发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia

del Rio

*发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题

:*Re: [bess]

draft-hao-bess-inter-nvo3-vpn-optionc

Some comments from my side, I think the draft makes quite a few
assumptions on specific implementation details that are way too
general to be considered widely available. Most of the TOR
devices today already pay a double-pass penalty when doing
routing of traffic in/out of vxlan-type tunnels. Only the
newest generation can route into tunnels without additional
passes. And are defin

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Martin,

Sorry not to express my mind precisely.

I mean to say, debating which solution is a better solution here does not make 
a sense. There is no need for the two solutions interwork.

Regards,
Lucy

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Martin Vigoureux
Sent: Friday, November 20, 2015 2:13 PM
To: bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

there is no such thing as voting in IETF WGs And I haven't seen anything like 
voting as part of this discussion.

Thank you
Martin

Le 20/11/2015 20:57, Lucy yong a écrit :
> IMHO: voting on this thread does not make a sense. Both solutions will work 
> and scales.
>
> Lucy
>
> -Original Message-
> From: Rabadan, Jorge (Jorge) [mailto:jorge.raba...@alcatel-lucent.com]
> Sent: Friday, November 20, 2015 1:32 PM
> To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy 
> yong; Henderickx, Wim (Wim); bess@ietf.org
> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
> IMHO if TOR chip vendors can confirm they are seriously looking at 
> MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works 
> and scales.
> My 2 cents.
>
> Jorge
>
>
>
> On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" 
> <bess-boun...@ietf.org on behalf of ju1...@att.com> wrote:
>
>> +1
>>
>> -Original Message-
>> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
>> Sent: Friday, November 20, 2015 12:19 PM
>> To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
>> bess@ietf.org
>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>
>> Lucy,
>>
>> My apologies, I misunderstood.
>>
>> I think we have to accept the fact that we will have to deal with a 
>> multiplicity of different encapsulations in the data plane along a packet's 
>> e2e path and that we should take a more measured approach to deciding how to 
>> deal with this in a general and extensible way before accepting any 
>> solutions.
>>
>> Yours Irrespectively,
>>
>> John
>>
>>> -----Original Message-----
>>> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
>>> Sent: Friday, November 20, 2015 12:04 PM
>>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>>
>>> 2015-11-20, John E Drake:
>>>> That presupposes that the group likes either of the two proposed 
>>>> solutions
>>> in your draft.
>>>
>>> John, I think Lucy's "two solutions" was referring to
>>> draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label 
>>> Optionc MPLS/MPLS/UDP solution described by Wim.
>>>
>>> -Thomas
>>>
>>>
>>>
>>>>
>>>>> -Original Message-
>>>>> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
>>>>> Sent: Friday, November 20, 2015 11:49 AM
>>>>> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); 
>>>>> bess@ietf.org
>>>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>>>>
>>>>> Share my 2 cent.
>>>>>
>>>>> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>>>>> Option C is a way to realize it. Both solutions summarized by 
>>>>> Thomas have no change on WAN VPN side and seamlessly work with WAN 
>>>>> VPN
>>> option C.
>>>>> However, to support either solution, DC has to do some enhancement 
>>>>> on DC BR or ToR switch, etc, which dictates to different 
>>>>> implementations within a DC. Option C pro and con remains 
>>>>> regardless what implementation is used in a DC.
>>>>>
>>>>> Two solutions should not coexist in one DC (does not make a 
>>>>> sense), but it does not matter if one DC operator prefers to use 
>>>>> one solution and another DC prefers to use another solution. Since 
>>>>> there are many cloud providers today, it is worth for the WG to 
>>>>> document both solutions and point out the implementation 
>>>>> requirements on impacted components. Then, up to vendors and 
>>>>> operators to select a solution for
>>> their DC.
>>>>>
>>>>> It does not make a sense for us to vote which solution is better 
>>>>> here because a better solution for a DC depe

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Martin Vigoureux]

Lucy,

Thanks for your clarification.
Yet, the fact that two solutions need not interwork does not render 
useless a debate on the merits of each.


-m

Le 20/11/2015 21:36, Lucy yong a écrit :

Martin,

Sorry not to express my mind precisely.

I mean to say, debating which solution is a better solution here does not make 
a sense. There is no need for the two solutions interwork.

Regards,
Lucy

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Martin Vigoureux
Sent: Friday, November 20, 2015 2:13 PM
To: bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

there is no such thing as voting in IETF WGs And I haven't seen anything like 
voting as part of this discussion.

Thank you
Martin

Le 20/11/2015 20:57, Lucy yong a écrit :

IMHO: voting on this thread does not make a sense. Both solutions will work and 
scales.

Lucy

-Original Message-
From: Rabadan, Jorge (Jorge) [mailto:jorge.raba...@alcatel-lucent.com]
Sent: Friday, November 20, 2015 1:32 PM
To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy
yong; Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

IMHO if TOR chip vendors can confirm they are seriously looking at 
MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org on 
behalf of ju1...@att.com> wrote:


+1

-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
Sent: Friday, November 20, 2015 12:19 PM
To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim);
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Lucy,

My apologies, I misunderstood.

I think we have to accept the fact that we will have to deal with a 
multiplicity of different encapsulations in the data plane along a packet's e2e 
path and that we should take a more measured approach to deciding how to deal 
with this in a general and extensible way before accepting any solutions.

Yours Irrespectively,

John


-Original Message-
From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
Sent: Friday, November 20, 2015 12:04 PM
To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-20, John E Drake:

That presupposes that the group likes either of the two proposed
solutions

in your draft.

John, I think Lucy's "two solutions" was referring to
draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label
Optionc MPLS/MPLS/UDP solution described by Wim.

-Thomas






-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
Sent: Friday, November 20, 2015 11:49 AM
To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim);
bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Share my 2 cent.

Cloud providers want to tunnel its customer traffic through DC (AS)BR.
Option C is a way to realize it. Both solutions summarized by
Thomas have no change on WAN VPN side and seamlessly work with WAN
VPN

option C.

However, to support either solution, DC has to do some enhancement
on DC BR or ToR switch, etc, which dictates to different
implementations within a DC. Option C pro and con remains
regardless what implementation is used in a DC.

Two solutions should not coexist in one DC (does not make a
sense), but it does not matter if one DC operator prefers to use
one solution and another DC prefers to use another solution. Since
there are many cloud providers today, it is worth for the WG to
document both solutions and point out the implementation
requirements on impacted components. Then, up to vendors and
operators to select a solution for

their DC.


It does not make a sense for us to vote which solution is better
here because a better solution for a DC depends on many factors.


Lucy





-Original Message-
From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of
thomas.mo...@orange.com
Sent: Friday, November 20, 2015 3:02 AM
To: Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

2015-11-19, Henderickx, Wim (Wim):

WH> I vote for a an evolution of switches/TORs that have proper
support for this. I hope some HW vendors of TOR chips shime in,
but I am told the MPLS solution is possible in the next
generation chips they are working on.


Well, it looks like the key questions are:
- when would ToR chips support MPLS/MPLS/UDP ?  (the generation
that has been released recently but not present in most shipping
ToRs yet, the next one ?)
- do we want an interim VXLAN-based solution ? (that will involve
at best a performance penalty with existing chips, and at worse
impossible to implement -- we haven't seen clear information in
this
thread)

-Thomas



Zhuangshunwan  :


Hi Diego,

Thanks for 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[thomas.morin]

2015-11-20, Haoweiguo:

WH> ok now we have not discussed the constraints some HW vendors have
with respect to global VNIDs. To make this work all VNID/Labels need
to be globally unique. Hm

> [weiguo2]:  In SDN scenario, a virtual

network normally is represented by a global VN ID or MPLS VPN Label
to simplify network management. I think it's not strange to allocate
global unique MPLS VPN Label or VN ID for a virtual network now.


In an option C scenario you have to take into account what is done in 
the WAN. Globally-assigned labels are not an option there at all.


-Thomas




The alternative approach, put forward by Wim, consist in using existing
Option-C with a variant of the 3-label variant relying on an
MPLS/MPLS/UDP-or-GRE encap (instead of the 3-label MPLS stack).  This
approach would not necessitate new standardized procedures, would
require less changes on ASBRs. However it is not supported today by
vswitches or ToRs (unless the bottom MPLS label is passed to the VM,
which I think would restrict the application to a subset of the
scenarios for which Option C is interesting). Having it supported in
vswitches may be a simple matter of writing the software, but ToR
support seems to require evolution in chipsets ; whether such a support
is likely to appear hasn't been discussed in the thread.

All in all, it seems there is no solution to cover an Option C scenario
with today's generation ToRs, and the question for tomorrow's generation
ToRs hasn't been really discussed.

Based on the above, I would think the question boils down to whether it
is desirable to specify an Option C variant usable with vswitches as
they are today and possibly usable with a performance hit with today's
ToRs chipsets, at the expense of a required evolution on ASBRs.  The
alternative requiring some evolution on vswitches and waiting for future
ToRs chipsets.

WH> I vote for a an evolution of switches/TORs that have proper support for 
this.
I hope some HW vendors of TOR chips shime in, but I am told the MPLS solution 
is possible in the next generation chips they are working on.
[weiguo2]:  I think it's better not to change current TOR hardware to realize 
option-C interconnection. No TOR hardware modification solution must be 
provided. But for future case, we can propose extra hardware requirements for 
TORs.
The other options requires baggage forever on the ASBR elements that does not 
bring value and it is better to avoid this and build an architecture which is 
future proof and more cost effective. My 2 cents


-Thomas




Zhuangshunwan  :


Hi Diego,

Thanks for your comments. Pls see inline with [Vincent].

Vincent

*发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del Rio
*发送时间:*2015年11月18日14:25
*收件人:*bess@ietf.org
*主题:*Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Some comments from my side,
I think the draft makes quite a few assumptions on specific
implementation details that are way too general to be considered
widely available.
Most of the TOR devices today already pay a double-pass penalty when
doing routing of traffic in/out of vxlan-type tunnels. Only the newest
generation can route into tunnels without additional passes. And are
definitively limited in being able to arbitrary select UDP ports on a
per tunnel basis. In fact, most are even limited at using more than
one VNID per "service" of sorts.
[Vincent]: Yes, the new generation BCM chipset has already supported
VXLAN routing without additional passes. For OVS/TOR, VXLAN
encapsulation is more popular than MPLSoGRE/UDP, and has better
performance.
The IP-addressed based implementation which would, I assume, imply
assigning one or more CIDRs to a loopback interface on the ASBR-d is
also quite arbitrary and does not seem like a technically "clean"
solution. (besides burning tons of IPs). As a side-note, most PE-grade
routers i've worked with were quite limited in terms of IP addresses
used for tunnel termination and it wasn't that just a simple pool can
be used.
[Vincent]: I think the larger VTEP IP address range on ASBR-d has no
limitations. For the hardware on ASBR-d, it has capability to
terminate multiple VXLAN tunnels with arbitrary destination VTEP IP
addresses.
Wim's mentions on cases where the Application itself, hosted in a
datacenter, would be part of the option-C interconnect, was dismissed
without much discussion so far, while, if we look in detail at the
type of users which will even consider a complex topology like model-C
its most likely users and operators very familiar with MPLS VPNs in
the WAN. Those type of operators will most likely be very interested
in deploying MPLS or WAN-grade applications (i.e., virtual-routers or
other VNFs) in the DC and thus its highly likely that the interconnect
would not terminate at the NVE itself but rather the TS (the virtual
machine).
Also, the use of UDP ports at random would imply quite complex logic
on the ASBR-d IMHO. Im not saying its impossible, but 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Zhuangshunwan]

+1

More popular deployments in data center is VXLAN/NVGRE, so the VXLAN/NVGRE and 
MPLS VPN network interconnecting is necessary. If all TOR/NVEs support 
MPLSoGRE/oUDP, Wim's solution also can be used, but currently it's not 
practical to require all TOR/NVEs to support MPLS/MPLS/UDPorGRE.

Vincent

From: Haoweiguo
Sent: Saturday, November 21, 2015 14:21
To: Rabadan, Jorge (Jorge); UTTARO, JAMES; John E Drake; EXT - 
thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Jorge,
We all know that Wim's MPLS/MPLS/UDP solution works, but it's not the only 
choice. Wim's solution requires MPLSoGRE/UDP encap in data center, but many 
data centers only use VXLAN/NVGRE encapsulation for both north-south and 
east-west bound traffic, how to interconnect with outside MPLS VPN network for 
these data centers? So VXLAN/NVGRE and MPLS VPN network interworking is needed.
And for the interconnection solution, we suggest both no TOR/NVE hardware 
enhancement solution and future proof solution should be provided.
Thanks,
weiguo

From: BESS [bess-boun...@ietf.org] on behalf of Rabadan, Jorge (Jorge) 
[jorge.raba...@alcatel-lucent.com]
Sent: Saturday, November 21, 2015 3:31
To: UTTARO, JAMES; John E Drake; EXT - thomas.mo...@orange.com; Lucy yong; 
Henderickx, Wim (Wim); bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

IMHO if TOR chip vendors can confirm they are seriously looking at 
MPLS/MPLS/UDP, Wim’s suggestion makes all the sense since we know it works and 
scales.
My 2 cents.

Jorge



On 11/20/15, 9:51 AM, "BESS on behalf of UTTARO, JAMES" <bess-boun...@ietf.org 
on behalf of ju1...@att.com> wrote:

>+1
>
>-Original Message-
>From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of John E Drake
>Sent: Friday, November 20, 2015 12:19 PM
>To: EXT - thomas.mo...@orange.com; Lucy yong; Henderickx, Wim (Wim); 
>bess@ietf.org
>Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>
>Lucy,
>
>My apologies, I misunderstood.
>
>I think we have to accept the fact that we will have to deal with a 
>multiplicity of different encapsulations in the data plane along a packet's 
>e2e path and that we should take a more measured approach to deciding how to 
>deal with this in a general and extensible way before accepting any solutions.
>
>Yours Irrespectively,
>
>John
>
>> -Original Message-
>> From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com]
>> Sent: Friday, November 20, 2015 12:04 PM
>> To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org
>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>
>> 2015-11-20, John E Drake:
>> > That presupposes that the group likes either of the two proposed 
>> > solutions
>> in your draft.
>>
>> John, I think Lucy's "two solutions" was referring to 
>> draft-hao-bess-inter- nvo3-vpn-optionc solution and the 3-label 
>> Optionc MPLS/MPLS/UDP solution described by Wim.
>>
>> -Thomas
>>
>>
>>
>> >
>> >> -Original Message-----
>> >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong
>> >> Sent: Friday, November 20, 2015 11:49 AM
>> >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); 
>> >> bess@ietf.org
>> >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>> >>
>> >> Share my 2 cent.
>> >>
>> >> Cloud providers want to tunnel its customer traffic through DC (AS)BR.
>> >> Option C is a way to realize it. Both solutions summarized by 
>> >> Thomas have no change on WAN VPN side and seamlessly work with WAN 
>> >> VPN
>> option C.
>> >> However, to support either solution, DC has to do some enhancement 
>> >> on DC BR or ToR switch, etc, which dictates to different 
>> >> implementations within a DC. Option C pro and con remains 
>> >> regardless what implementation is used in a DC.
>> >>
>> >> Two solutions should not coexist in one DC (does not make a 
>> >> sense), but it does not matter if one DC operator prefers to use 
>> >> one solution and another DC prefers to use another solution. Since 
>> >> there are many cloud providers today, it is worth for the WG to 
>> >> document both solutions and point out the implementation 
>> >> requirements on impacted components. Then, up to vendors and 
>> >> operators to select a solution for
>> their DC.
>> >>
>> >> It does not make a sense 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[UTTARO, JAMES]

Comments In-Line..

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 2:06 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:ju1...@att.com]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <osa...@microsoft.com<mailto:osa...@microsoft.com>>; UTTARO, 
JAMES <ju1...@att.com<mailto:ju1...@att.com>>; Haoweiguo 
<haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and 
a Cloud Provider connect an inter-AS model of some form is needed.. It is 
doubtful that it would ever be Option C as the exposes a tremendous amount of 
internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a 
customer router in the cloud that should only expose customer routes. I had 
argued previously in IETF that current practice of option A is not scalable.
[Jim U>] So, do you mean a CSC model where the NHs are the customer CEs, if so, 
then that is not Classical Option C where PE LBs are exchanged as the NHs for 
the customers path/routes.

At the end of the day network architecture/design whether internal/external to 
a provider is not mandated by the IETF or any standards body. This draft seems 
to mix encap, interconnect etc If the encap "requires" a specific 
connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Today many large enterprises connect to cloud providers using their existing 
MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

A concern I have with this draft is that it pre-supposes the network 
architecture in terms of how DCs and WAN are connected. It assumes that DCs and 
WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's 
forwarding table size, the motivation is same as traditional MPLS VPN option-C. 
Currently, the most common practise on ASBR-d is to terminate VXLAN 
encapsulation, look up local routing table, and then perform MPLS encapsulation 
to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C 
method, only transport layer information needed to be maintained at GW/ASBR-d, 
the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS 
VPN network interworking, because VXLAN is becoming pervasive in data center,  
the solution in this draft was proposed for the heterogeneous network 
interworking.

The advantage of this solution is that only VXLAN encapsulation is required for 
OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while 
north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware 
changes on ASBR-d. ASBR-d normally is router, it has capability to realize the 
hardware changes. It will consume many IP addresses and the IP pool for 
allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stit

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Osama Zia]

CIL

From: UTTARO, JAMES [mailto:ju1...@att.com]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <osa...@microsoft.com>; UTTARO, JAMES <ju1...@att.com>; Haoweiguo 
<haowei...@huawei.com>; John E Drake <jdr...@juniper.net>; Henderickx, Wim 
(Wim) <wim.henderi...@alcatel-lucent.com>; EXT - thomas.mo...@orange.com 
<thomas.mo...@orange.com>; BESS <bess@ietf.org>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and 
a Cloud Provider connect an inter-AS model of some form is needed.. It is 
doubtful that it would ever be Option C as the exposes a tremendous amount of 
internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a 
customer router in the cloud that should only expose customer routes. I had 
argued previously in IETF that current practice of option A is not scalable.

At the end of the day network architecture/design whether internal/external to 
a provider is not mandated by the IETF or any standards body. This draft seems 
to mix encap, interconnect etc If the encap "requires" a specific 
connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Today many large enterprises connect to cloud providers using their existing 
MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

A concern I have with this draft is that it pre-supposes the network 
architecture in terms of how DCs and WAN are connected. It assumes that DCs and 
WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's 
forwarding table size, the motivation is same as traditional MPLS VPN option-C. 
Currently, the most common practise on ASBR-d is to terminate VXLAN 
encapsulation, look up local routing table, and then perform MPLS encapsulation 
to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C 
method, only transport layer information needed to be maintained at GW/ASBR-d, 
the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS 
VPN network interworking, because VXLAN is becoming pervasive in data center,  
the solution in this draft was proposed for the heterogeneous network 
interworking.

The advantage of this solution is that only VXLAN encapsulation is required for 
OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while 
north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware 
changes on ASBR-d. ASBR-d normally is router, it has capability to realize the 
hardware changes. It will consume many IP addresses and the IP pool for 
allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If 
UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR 
and ASBR-d. ASBR-d also has capability to realize the hardware changes. For 
OVS, it also can realize the data plane changes. For TOR switch, it normally 
can't realize this function.  This solution mainly focuses on pure software 
based overlay network, it has more scalability. In public cloud data center, 
software based overlay network is the majority case.



Whether using solution 1 or 2 depends on the operators real envionment.



So I think our solution has no flaws, it works fine.

Thanks,

weiguo



_

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Diego Garcia del Rio]

gt; for this.
> I hope some HW vendors of TOR chips shime in, but I am told the MPLS
> solution is possible in the next generation chips they are working on.
>
> The other options requires baggage forever on the ASBR elements that does
> not bring value and it is better to avoid this and build an architecture
> which is future proof and more cost effective. My 2 cents
> >
> >-Thomas
> >
> >
> >Zhuangshunwan  :
> >>
> >> Hi Diego,
> >>
> >> Thanks for your comments. Pls see inline with [Vincent].
> >>
> >> Vincent
> >>
> >> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del Rio
> >> *发送时间:*2015年11月18日14:25
> >> *收件人:*bess@ietf.org
> >> *主题:*Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
> >>
> >> Some comments from my side,
> >> I think the draft makes quite a few assumptions on specific
> >> implementation details that are way too general to be considered
> >> widely available.
> >> Most of the TOR devices today already pay a double-pass penalty when
> >> doing routing of traffic in/out of vxlan-type tunnels. Only the newest
> >> generation can route into tunnels without additional passes. And are
> >> definitively limited in being able to arbitrary select UDP ports on a
> >> per tunnel basis. In fact, most are even limited at using more than
> >> one VNID per "service" of sorts.
> >> [Vincent]: Yes, the new generation BCM chipset has already supported
> >> VXLAN routing without additional passes. For OVS/TOR, VXLAN
> >> encapsulation is more popular than MPLSoGRE/UDP, and has better
> >> performance.
> >> The IP-addressed based implementation which would, I assume, imply
> >> assigning one or more CIDRs to a loopback interface on the ASBR-d is
> >> also quite arbitrary and does not seem like a technically "clean"
> >> solution. (besides burning tons of IPs). As a side-note, most PE-grade
> >> routers i've worked with were quite limited in terms of IP addresses
> >> used for tunnel termination and it wasn't that just a simple pool can
> >> be used.
> >> [Vincent]: I think the larger VTEP IP address range on ASBR-d has no
> >> limitations. For the hardware on ASBR-d, it has capability to
> >> terminate multiple VXLAN tunnels with arbitrary destination VTEP IP
> >> addresses.
> >> Wim's mentions on cases where the Application itself, hosted in a
> >> datacenter, would be part of the option-C interconnect, was dismissed
> >> without much discussion so far, while, if we look in detail at the
> >> type of users which will even consider a complex topology like model-C
> >> its most likely users and operators very familiar with MPLS VPNs in
> >> the WAN. Those type of operators will most likely be very interested
> >> in deploying MPLS or WAN-grade applications (i.e., virtual-routers or
> >> other VNFs) in the DC and thus its highly likely that the interconnect
> >> would not terminate at the NVE itself but rather the TS (the virtual
> >> machine).
> >> Also, the use of UDP ports at random would imply quite complex logic
> >> on the ASBR-d IMHO. Im not saying its impossible, but since the
> >> received packet now not only has to be received on a random (though
> >> locally chosen) UDP port and this information preserved in the
> >> pipeline to be able to do the double-tunnel-stitching, it sounds quite
> >> complex. I am sure someone in the list will mention this has already
> >> been implemented somewhere, and I won't argue with that. And let's not
> >> even bring into account what this would do to any DC middlebox that
> >> now has to look at vxlan over almost any random port. We have to go
> >> back to the "is it a 4 or is it a 6 in byte x" heuristics to try to
> >> guess whether the packet is vxlan or just something entirely different
> >> running over IP.
> >> [Vincent]: Using NP or multi-core CPU hardware technology, it can be
> >> implemented although deeper packet inspection is needed to perform UDP
> >> port and MPLS stitching.
> >> In general I feel the proposed solution seems to be fitting of a
> >> specific use-case which is not really detailed in the draft and does
> >> not describe   a solution that would "elegantly" solve the issues at
> >> hand. It just feels like we're using any available bit-space to stuff
> >> data into places that do not necesarily belong.
> >> Yes, MPLS encapsulations on vi

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Osama Zia]

Today many large enterprises connect to cloud providers using their existing 
MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <haowei...@huawei.com>; John E Drake <jdr...@juniper.net>; 
Henderickx, Wim (Wim) <wim.henderi...@alcatel-lucent.com>; EXT - 
thomas.mo...@orange.com <thomas.mo...@orange.com>; BESS <bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

A concern I have with this draft is that it pre-supposes the network 
architecture in terms of how DCs and WAN are connected. It assumes that DCs and 
WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's 
forwarding table size, the motivation is same as traditional MPLS VPN option-C. 
Currently, the most common practise on ASBR-d is to terminate VXLAN 
encapsulation, look up local routing table, and then perform MPLS encapsulation 
to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C 
method, only transport layer information needed to be maintained at GW/ASBR-d, 
the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS 
VPN network interworking, because VXLAN is becoming pervasive in data center,  
the solution in this draft was proposed for the heterogeneous network 
interworking.

The advantage of this solution is that only VXLAN encapsulation is required for 
OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while 
north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware 
changes on ASBR-d. ASBR-d normally is router, it has capability to realize the 
hardware changes. It will consume many IP addresses and the IP pool for 
allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If 
UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR 
and ASBR-d. ASBR-d also has capability to realize the hardware changes. For 
OVS, it also can realize the data plane changes. For TOR switch, it normally 
can't realize this function.  This solution mainly focuses on pure software 
based overlay network, it has more scalability. In public cloud data center, 
software based overlay network is the majority case.



Whether using solution 1 or 2 depends on the operators real envionment.



So I think our solution has no flaws, it works fine.

Thanks,

weiguo




From: BESS [bess-boun...@ietf.org] on behalf of John E Drake 
[jdr...@juniper.net]
Sent: Wednesday, November 18, 2015 2:49
To: Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
Hi,

I think Wim has conclusively demonstrated that this draft has fatal flaws and I 
don't support it.  I also agree with his suggestion that we first figure out 
what problem we are trying to solve before solving it.

Yours Irrespectively,

John

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

- Snip -

No, the spec as it is can be implemented in its VXLAN variant with existing 
vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the 
linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough 
VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work 
every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines 
solutions which should be implemented across the board HW/SW/etc. Even if some 
SW switches can do this, the proposal will impose so many issues in 
HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, 
we better do it in the right way and guide the industry to a solution which 
does not imply those complexities. Otherwise we will stick with 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[UTTARO, JAMES]

IMO an Option C model not CSC between to different administrative authorities 
where internal reachability state is exchanged is a non-starter.  But coming 
back to this draft it seems that tying the encap and the network architecture 
is a requirement or am I getting that wrong. If not, how should I interpret 
this draft? Should it be if I do Option C as I must have different AS domains 
between my internal DCs and WAN and my encap models are different in the WAN 
and DC, and I am using specific encaps in different places etcc... it would be 
in my solution space??

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 2:38 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:ju1...@att.com]
Sent: Thursday, November 19, 2015 11:16 AM
To: Osama Zia <osa...@microsoft.com<mailto:osa...@microsoft.com>>; UTTARO, 
JAMES <ju1...@att.com<mailto:ju1...@att.com>>; Haoweiguo 
<haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Comments In-Line..

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 2:06 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

CIL

From: UTTARO, JAMES [mailto:ju1...@att.com]
Sent: Thursday, November 19, 2015 10:56 AM
To: Osama Zia <osa...@microsoft.com<mailto:osa...@microsoft.com>>; UTTARO, 
JAMES <ju1...@att.com<mailto:ju1...@att.com>>; Haoweiguo 
<haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The assumption being based on one's view of what a DC is ;) Yes when an SP and 
a Cloud Provider connect an inter-AS model of some form is needed.. It is 
doubtful that it would ever be Option C as the exposes a tremendous amount of 
internal information.
[OZ] I would beg to disagree here. Customer traffic can be tunneled to a 
customer router in the cloud that should only expose customer routes. I had 
argued previously in IETF that current practice of option A is not scalable.
[Jim U>] So, do you mean a CSC model where the NHs are the customer CEs, if so, 
then that is not Classical Option C where PE LBs are exchanged as the NHs for 
the customers path/routes.
[OZ] Sorry for the confusion. My comment was not specific to the draft. I was 
responding to your comment  "Option C as the exposes a tremendous amount of 
internal information"

At the end of the day network architecture/design whether internal/external to 
a provider is not mandated by the IETF or any standards body. This draft seems 
to mix encap, interconnect etc If the encap "requires" a specific 
connectivity model than I am not sure how that exactly works.

Jim Uttaro

From: Osama Zia [mailto:osa...@microsoft.com]
Sent: Thursday, November 19, 2015 1:51 PM
To: UTTARO, JAMES; Haoweiguo; John E Drake; Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Today many large enterprises connect to cloud providers using their existing 
MPLS VPN networks. This would mean different AS Numbers at WAN and DC sites.

Regards,
Osama

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of UTTARO, JAMES
Sent: Thursday, November 19, 2015 9:46 AM
To: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>; John E Drake 
<jdr...@juniper.net<mailto:jdr...@juniper.net>>; Henderickx, Wim (Wim) 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>; 
EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com> 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>; BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

A concern I have with this draft is that it p

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

Thomas, thx for the time summarising this and great summary.




On 19/11/15 02:52, "BESS on behalf of Thomas Morin" <bess-boun...@ietf.org on 
behalf of thomas.mo...@orange.com> wrote:

>I don't believe the discussion can be usefully summarized in terms of 
>"fatal flaws".
>Let me try to summarize my understanding of the thing discussed in this 
>thread.
>
>The solution in the draft has an impact on ASBR hardware due to the new 
>kind of stitching required (as well summarized by Diego). Whether this 
>is a big issue or not depends on the vendor. The solution in the draft, 
>for the NVE part, would be implementable in software quite easily. But, 
>because of the limitations in the widespread ToR chipsets for VXLAN (and 
>their lack of support for MPLS/(GRE or UDP)), its multiple-UDP-port 
>variant would be harder to implement in ToRs or at least not without a 
>performance hit (people who know that better, feel free to correct).  

WH> some HW cannot do this at all, so lets dismiss this idea.

>The multiple-loopback approach has operational drawbacks, but whether or 
>not these are killer issues may depend on the targeted scale (in terms 
>of number of NVEs) and may depend on other factors (operator practices, 
>ability to automate ASBR configs).

WH> ok now we have not discussed the constraints some HW vendors have with 
respect to global VNIDs. To make this work all VNID/Labels need to be globally 
unique. Hm 

>
>The alternative approach, put forward by Wim, consist in using existing 
>Option-C with a variant of the 3-label variant relying on an 
>MPLS/MPLS/UDP-or-GRE encap (instead of the 3-label MPLS stack).  This 
>approach would not necessitate new standardized procedures, would 
>require less changes on ASBRs. However it is not supported today by 
>vswitches or ToRs (unless the bottom MPLS label is passed to the VM, 
>which I think would restrict the application to a subset of the 
>scenarios for which Option C is interesting). Having it supported in 
>vswitches may be a simple matter of writing the software, but ToR 
>support seems to require evolution in chipsets ; whether such a support 
>is likely to appear hasn't been discussed in the thread.
>
>All in all, it seems there is no solution to cover an Option C scenario 
>with today's generation ToRs, and the question for tomorrow's generation 
>ToRs hasn't been really discussed.
>
>Based on the above, I would think the question boils down to whether it 
>is desirable to specify an Option C variant usable with vswitches as 
>they are today and possibly usable with a performance hit with today's 
>ToRs chipsets, at the expense of a required evolution on ASBRs.  The 
>alternative requiring some evolution on vswitches and waiting for future 
>ToRs chipsets.

WH> I vote for a an evolution of switches/TORs that have proper support for 
this.
I hope some HW vendors of TOR chips shime in, but I am told the MPLS solution 
is possible in the next generation chips they are working on.

The other options requires baggage forever on the ASBR elements that does not 
bring value and it is better to avoid this and build an architecture which is 
future proof and more cost effective. My 2 cents
>
>-Thomas
>
>
>Zhuangshunwan  :
>>
>> Hi Diego,
>>
>> Thanks for your comments. Pls see inline with [Vincent].
>>
>> Vincent
>>
>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del Rio
>> *发送时间:*2015年11月18日14:25
>> *收件人:*bess@ietf.org
>> *主题:*Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
>>
>> Some comments from my side,
>> I think the draft makes quite a few assumptions on specific 
>> implementation details that are way too general to be considered 
>> widely available.
>> Most of the TOR devices today already pay a double-pass penalty when 
>> doing routing of traffic in/out of vxlan-type tunnels. Only the newest 
>> generation can route into tunnels without additional passes. And are 
>> definitively limited in being able to arbitrary select UDP ports on a 
>> per tunnel basis. In fact, most are even limited at using more than 
>> one VNID per "service" of sorts.
>> [Vincent]: Yes, the new generation BCM chipset has already supported 
>> VXLAN routing without additional passes. For OVS/TOR, VXLAN 
>> encapsulation is more popular than MPLSoGRE/UDP, and has better 
>> performance.
>> The IP-addressed based implementation which would, I assume, imply 
>> assigning one or more CIDRs to a loopback interface on the ASBR-d is 
>> also quite arbitrary and does not seem like a technically "clean" 
>> solution. (besides burning tons of IPs). As a side-note, most PE-grade 
>> rou

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[UTTARO, JAMES]

A concern I have with this draft is that it pre-supposes the network 
architecture in terms of how DCs and WAN are connected. It assumes that DCs and 
WAN networks are in different AS domains. Why is that??

Jim Uttaro

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Haoweiguo
Sent: Tuesday, November 17, 2015 11:24 PM
To: John E Drake; Henderickx, Wim (Wim); EXT - thomas.mo...@orange.com; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's 
forwarding table size, the motivation is same as traditional MPLS VPN option-C. 
Currently, the most common practise on ASBR-d is to terminate VXLAN 
encapsulation, look up local routing table, and then perform MPLS encapsulation 
to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C 
method, only transport layer information needed to be maintained at GW/ASBR-d, 
the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS 
VPN network interworking, because VXLAN is becoming pervasive in data center,  
the solution in this draft was proposed for the heterogeneous network 
interworking.

The advantage of this solution is that only VXLAN encapsulation is required for 
OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while 
north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware 
changes on ASBR-d. ASBR-d normally is router, it has capability to realize the 
hardware changes. It will consume many IP addresses and the IP pool for 
allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If 
UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR 
and ASBR-d. ASBR-d also has capability to realize the hardware changes. For 
OVS, it also can realize the data plane changes. For TOR switch, it normally 
can't realize this function.  This solution mainly focuses on pure software 
based overlay network, it has more scalability. In public cloud data center, 
software based overlay network is the majority case.



Whether using solution 1 or 2 depends on the operators real envionment.



So I think our solution has no flaws, it works fine.

Thanks,

weiguo




From: BESS [bess-boun...@ietf.org] on behalf of John E Drake 
[jdr...@juniper.net]
Sent: Wednesday, November 18, 2015 2:49
To: Henderickx, Wim (Wim); EXT - 
thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
Hi,

I think Wim has conclusively demonstrated that this draft has fatal flaws and I 
don't support it.  I also agree with his suggestion that we first figure out 
what problem we are trying to solve before solving it.

Yours Irrespectively,

John

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

- Snip -

No, the spec as it is can be implemented in its VXLAN variant with existing 
vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the 
linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough 
VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work 
every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines 
solutions which should be implemented across the board HW/SW/etc. Even if some 
SW switches can do this, the proposal will impose so many issues in 
HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, 
we better do it in the right way and guide the industry to a solution which 
does not imply those complexities. Otherwise we will stick with these specials 
forever with all consequences (bugs, etc).

- snip -

From: "thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
BESS <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss 

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

For this to work every tenant needs a different VXLAN UDP destination 
port/receive port.

[Lucy] this is not right. For the traffic from DC ASBR -> NVE, only one UDP dst 
port is used; for the traffic from NVE-> DC ASBR, DC ASBR signaled UDP port 
number will be used. Tenant traffic is segregated by VNID, not UDP port. At an 
NVE, the traffic going toward the same remote PE will be encapsulated w/ VXLAN 
and the same UDP dst port number even the traffic may belong to different VNs. 
In short, tenant and UDP port are orthogonal; the solution just uses UDP port 
to indicate the outgoing mpls label for the packets at DC ASBR when it receives 
packets from NVEs.

Lucy

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 11:49 AM
To: thomas.mo...@orange.com; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

No, the spec as it is can be implemented in its VXLAN variant with existing 
vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the 
linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough 
VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work 
every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines 
solutions which should be implemented across the board HW/SW/etc. Even if some 
SW switches can do this, the proposal will impose so many issues in 
HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, 
we better do it in the right way and guide the industry to a solution which 
does not imply those complexities. Otherwise we will stick with these specials 
forever with all consequences (bugs, etc).

- snip -

From: "thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
BESS <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe requirements, but 
the current proposal I cannot agree to for the reasons explained.

TM> Well, although discussing forever is certainly not the goal, the reasons 
for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane 
here, the use cases I have seen in this txt is always where an application 
behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.


My understanding is that the applications  are contexts where overlays are 
present is when workloads (VMs or baremetal) need to be interconnected with 
VPNs. In these contexts, there can be reasons to want Option C to reduce the 
state on ASBRs.

In these context, its not the workload (VM or baremetal) that would typically 
handle VRFs, but really the vswitch or ToR.

WH2> can it not be all cases: TOR/vswitch/Application. I would make the 
solution flexible to support all of these not?




2015-11-13, Henderickx, Wim (Wim):

TM> The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in 
this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

WH> b depends on the use case

I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application behind NVE/TOR 
requiring model C, than all the discussion on impact on NVE/TOR is void. As 
such I want to have a discussion on the real driver/requirement for option c 
interworking with an IP based Fabric.

Although I can agree than detailing requirements can always help, I don't think 
one can assume a certain application to dismiss the proposal.

WH> for me the proposal is not acceptable for the reasons explained: too much 
impact on the data-planes


I wrote the above based on the idea that the encap used in MPLS/MPLS/(UDP or 
GRE), which hence has to be supported on the ToRs and vswitches.
Another possibility would be service-label/middle-label/Ethernet assuming an L2 
adjacency between vswitches/ToRs and ASBRs, but this certainly does not match 
your typical DC architecture. Or perhaps had you something else in mind ?

WH> see above. The draft right now also requires changes in existing TOR/NVE so 
for me all this discussion/debate is void.

No, the spec as it is can be implemented in its VXLAN variant with existin

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Lucy, does not change anything but I did a typo
I meant to say every tenant needs a different VXLAN UDP destination port for 
upstream
[Lucy] In this solution, tenant and VXLAN UDP dst. port are orthogonal.  The 
VXLAN UDP dst. port on an upstream packet contains the info. for the outgoing 
label at DC ASBR.

Lucy
+ receive port for downstream on NVE

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Tuesday 17 November 2015 at 11:54
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
"thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>, BESS 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

For this to work every tenant needs a different VXLAN UDP destination 
port/receive port.

[Lucy] this is not right. For the traffic from DC ASBR -> NVE, only one UDP dst 
port is used; for the traffic from NVE-> DC ASBR, DC ASBR signaled UDP port 
number will be used. Tenant traffic is segregated by VNID, not UDP port. At an 
NVE, the traffic going toward the same remote PE will be encapsulated w/ VXLAN 
and the same UDP dst port number even the traffic may belong to different VNs. 
In short, tenant and UDP port are orthogonal; the solution just uses UDP port 
to indicate the outgoing mpls label for the packets at DC ASBR when it receives 
packets from NVEs.

Lucy

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 11:49 AM
To: thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

No, the spec as it is can be implemented in its VXLAN variant with existing 
vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the 
linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough 
VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work 
every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines 
solutions which should be implemented across the board HW/SW/etc. Even if some 
SW switches can do this, the proposal will impose so many issues in 
HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, 
we better do it in the right way and guide the industry to a solution which 
does not imply those complexities. Otherwise we will stick with these specials 
forever with all consequences (bugs, etc).

- snip -

From: "thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
BESS <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe requirements, but 
the current proposal I cannot agree to for the reasons explained.

TM> Well, although discussing forever is certainly not the goal, the reasons 
for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane 
here, the use cases I have seen in this txt is always where an application 
behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.


My understanding is that the applications  are contexts where overlays are 
present is when workloads (VMs or baremetal) need to be interconnected with 
VPNs. In these contexts, there can be reasons to want Option C to reduce the 
state on ASBRs.

In these context, its not the workload (VM or baremetal) that would typically 
handle VRFs, but really the vswitch or ToR.

WH2> can it not be all cases: TOR/vswitch/Application. I would make the 
solution flexible to support all of these not?





2015-11-13, Henderickx, Wim (Wim):

TM> The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in 
this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

WH> b depends on the use case

I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application behind NVE/TOR 
requiring model C, than all the discussion on impact on NVE/TOR is void. As 
such I want to have a discussion on the real driver/requirement

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Haoweiguo]

Hi,

The problem we are trying to solve is to reduce data center GW/ASBR-d's 
forwarding table size, the motivation is same as traditional MPLS VPN option-C. 
Currently, the most common practise on ASBR-d is to terminate VXLAN 
encapsulation, look up local routing table, and then perform MPLS encapsulation 
to the WAN network. ASBR-d needs to maintain all VM's MAC/IP. In Option-C 
method, only transport layer information needed to be maintained at GW/ASBR-d, 
the scalability will be greatly enhanced. Traditonal Option-C is only for MPLS 
VPN network interworking, because VXLAN is becoming pervasive in data center,  
the solution in this draft was proposed for the heterogeneous network 
interworking.

The advantage of this solution is that only VXLAN encapsulation is required for 
OVS/TOR. Unlike Wim's solution, east-west bound traffic uses VXLAN encap, while 
north-south bound traffic uses MPLSoGRE/UDP encap.

There are two solutions in this draft:

1. Using VXLAN tunnel destination IP for stitching at ASBR-d.

No data plane modification requirements on OVS or TOR switches, only hardware 
changes on ASBR-d. ASBR-d normally is router, it has capability to realize the 
hardware changes. It will consume many IP addresses and the IP pool for 
allocation needs to be configured on ASBR-d beforehand.

2. Using VXLAN destination UDP port for stitching at ASBR-d.

Compared with solution 1, less IP address will be consumed for allocation. If 
UDP port range is too large, we can combine with solution 1 and 2.

In this solution, both data plane modification changes are needed at OVS/TOR 
and ASBR-d. ASBR-d also has capability to realize the hardware changes. For 
OVS, it also can realize the data plane changes. For TOR switch, it normally 
can't realize this function.  This solution mainly focuses on pure software 
based overlay network, it has more scalability. In public cloud data center, 
software based overlay network is the majority case.



Whether using solution 1 or 2 depends on the operators real envionment.



So I think our solution has no flaws, it works fine.

Thanks,

weiguo




From: BESS [bess-boun...@ietf.org] on behalf of John E Drake 
[jdr...@juniper.net]
Sent: Wednesday, November 18, 2015 2:49
To: Henderickx, Wim (Wim); EXT - thomas.mo...@orange.com; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi,

I think Wim has conclusively demonstrated that this draft has fatal flaws and I 
don’t support it.  I also agree with his suggestion that we first figure out 
what problem we are trying to solve before solving it.

Yours Irrespectively,

John

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Tuesday, November 17, 2015 12:49 PM
To: EXT - thomas.mo...@orange.com; BESS
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

— Snip —

No, the spec as it is can be implemented in its VXLAN variant with existing 
vswitches (e.g. OVS allows to choose the VXLAN destination port, ditto for the 
linux kernel stack).

(ToR is certainly another story, most of them not having a flexible enough 
VXLAN dataplane nor support for any MPLS-over-IP.)

WH> and how many ports simultaneously would they support? For this to work 
every tenant needs a different VXLAN UDP destination port/receive port.
There might be SW elements that could do some of this, but IETF defines 
solutions which should be implemented across the board HW/SW/etc. Even if some 
SW switches can do this, the proposal will impose so many issues in 
HW/data-plane engines that I cannot be behind this solution.

To make this work generically we will have to make changes anyhow. Given this, 
we better do it in the right way and guide the industry to a solution which 
does not imply those complexities. Otherwise we will stick with these specials 
forever with all consequences (bugs, etc).

- snip -

From: "thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Tuesday 17 November 2015 at 01:37
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
BESS <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe requirements, but 
the current proposal I cannot agree to for the reasons explained.

TM> Well, although discussing forever is certainly not the goal, the reasons 
for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane 
here, the use cases I have seen in this txt is always where an application 
behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.


My understanding is that the applications  ar

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Haoweiguo]

Hi Wim,

WH> I first want to understand the requirements for this use case and I don’t 
agree to adoption since there are major DP implications in this.
[weiguo]: You proposed MPLS over GRE/UDP encapsulation for the traffic between 
data center inside and outside MPLS VPN network. However, for some TOR 
switches, they don't support MPLS Over GRE/UDP. Even if they support, they rely 
on internal loop and the forwarding performance will be reduced half.  For 
option-c scenario, TOR switches need to realize MPLS+MPLS+GRE/UDP, it's more 
complicated for TOR switches. Current most of the TOR switches support VXLAN 
encapsulation and have no half forwarding performance issue.  For ASBR-d, it is 
router device in most case, it has capability to realize more comlicated 
forwarding behavior.

Thanks,
weiguo

From: BESS [bess-boun...@ietf.org] on behalf of Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com]
Sent: Tuesday, November 17, 2015 0:52
To: Thomas Morin; bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

In-line

From: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Date: Monday 16 November 2015 at 08:29
To: Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>, 
"bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc



From: BESS <bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>> on behalf of 
Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Monday 16 November 2015 at 07:47
To: "bess@ietf.org<mailto:bess@ietf.org>" 
<bess@ietf.org<mailto:bess@ietf.org>>, Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

Henderickx, Wim (Wim) :
VXLAN has a dedicated UDP port and is very clear in the RFC7348

Well, having a port reserved for this use that won't be the default for another 
protocol is one thing, but that does not prevent in itself the same protocol to 
be applied on another range of ports. (Because HTTP specs says port 80 does not 
prevent the URI scheme to allow specifying the port in the URL)

Even reading the VXLAN (Informational) specs, we see room for flexibility wrt 
ports:

  -  Destination Port: IANA has assigned the value 4789 for the
 VXLAN UDP port, and this value SHOULD be used by default as the
 destination UDP port.  Some early implementations of VXLAN have
 used other values for the destination port.  To enable
 interoperability with these implementations, the destination
 port SHOULD be configurable.

I read "4789 SHOULD be used by default" and "SHOULD be configurable".

WH> true but it does not say to use multiple at the same time. My read on this 
is that it allows a non standard port to be used, but not multiple at the same 
time.




Will write something on what I proposed when I get some time, not soon

The co-chair in me needs to ask the following question: should this call for 
adoption be kept on hold until an outline of an alternative solution is 
provided ?

WH> I first want to understand the requirements for this use case and I don’t 
agree to adoption since there are major DP implications in this.


-Thomas






From: Lucy yong 
<<mailto:lucy.y...@huawei.com>lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<<mailto:wim.henderi...@alcatel-lucent.com>wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>,
 Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "<mailto:bess@ietf.org>bess@ietf.org<mailto:bess@ietf.org>" 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. E

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[thomas.morin]

Hi Wim, WG,

2015-11-16, Henderickx, Wim (Wim):


2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe 
requirements, but the current proposal I cannot agree to for the 
reasons explained.


TM> Well, although discussing forever is certainly not the goal, the 
reasons for rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN 
data-plane here, the use cases I have seen in this txt is always where 
an application behind a NVE/TOR is demanding option c, but none of the 
NVE/TOR elements.




My understanding is that the applications  are contexts where overlays 
are present is when workloads (VMs or baremetal) need to be 
interconnected with VPNs. In these contexts, there can be reasons to 
want Option C to reduce the state on ASBRs.


In these context, its not the workload (VM or baremetal) that would 
typically handle VRFs, but really the vswitch or ToR.




2015-11-13, Henderickx, Wim (Wim):

TM> The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior 
specified in this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to 
support MPLS/MPLS/(UDP or GRE)


WH> b depends on the use case


I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application 
behind NVE/TOR requiring model C, than all the discussion on impact on 
NVE/TOR is void. As such I want to have a discussion on the real 
driver/requirement for option c interworking with an IP based Fabric.


Although I can agree than detailing requirements can always help, I 
don't think one can assume a certain application to dismiss the proposal.





I wrote the above based on the idea that the encap used in 
MPLS/MPLS/(UDP or GRE), which hence has to be supported on the ToRs 
and vswitches.
Another possibility would be service-label/middle-label/Ethernet 
assuming an L2 adjacency between vswitches/ToRs and ASBRs, but this 
certainly does not match your typical DC architecture. Or perhaps had 
you something else in mind ?


WH> see above. The draft right now also requires changes in existing 
TOR/NVE so for me all this discussion/debate is void.


No, the spec as it is can be implemented in its VXLAN variant with 
existing vswitches (e.g. OVS allows to choose the VXLAN destination 
port, ditto for the linux kernel stack).


(ToR is certainly another story, most of them not having a flexible 
enough VXLAN dataplane nor support for any MPLS-over-IP.)




WH> and depending on implementation you don’t need to change any of 
the TOR/vswitches.


Does this mean that for some implementations you may not need to 
change any of the TOR/vswitches, but that for some others you may ?


WH> any proposal on the table requires changes, so for me this is not 
a valid discussion


See above, the proposal in the draft does not necessarily need changes 
in vswitches.




Let me take a practical example : while I can quite easily see how to 
implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc 
based on current vswitch implementations of VXLAN, the lack of 
MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to me as 
making that alternate solution you suggest harder to implement.


WH> I would disagree to this. Tell me which switch/TOR handles 
multiple UDP ports for VXLAN ?


I mentioned _v_switches, and many do support a variable destination port 
for VXLAN, which is sufficient to implement what the draft proposes.


-Thomas













From: Thomas Morin <thomas.mo...@orange.com>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx <wim.henderi...@alcatel-lucent.com>
Cc: "bess@ietf.org" <bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to 
implementations that supports the chosen encap with non-IANA ports 
(which may be hard to achieve for instance on hardware 
implementations, as you suggest), or to context where managing 
multiple IPs would be operationally viable.


However, it does not seem obvious to me how the alternative you 
propose [relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) 
encap] addresses the issue of whether the encap behavior is 
supported or not (e.g. your typical ToR chipset possibly may not 
support this kind of encap, and even software-based switches may not 
be ready to support that today).


My take is that having different options to adapt to various 
implementations constraints we may have would have value.


(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used 
with a dummy MAC address that would be replaced by the right MAC 
by a sender based on an ARP request when needed ?


I

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

In-line

From: BESS <bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>> on behalf of 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Sunday 15 November 2015 at 13:51
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Linda Dunbar <linda.dun...@huawei.com<mailto:linda.dun...@huawei.com>>, 
"bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

Thanks for your detail explanation.

In your solution, MPLSoGRE/oUDP needs to be supported in data center network. 
To support option-c interconnection with WAN network, TORs/NVEs need to add one 
BGP LSP Label in middle layer, so TORs/NVEs need to perform MPLS(VPN Layer) + 
MPLS (BGP LSP Label) + GRE/UDP encapsulation for the traffic from local 
connecting end systems to WAN network. In your solution, GRE/UDP layer is not 
used for transport layer stitching at ASBR-d, traditonal end to end BGP LSP is 
still used for transport layer.

WH> The proposal I made is supportable on NVE/TOR/Applications in the DC and 
this is something that is feasible today without any extra standardisation.


In our solution, VXLAN/NVGRE are two of the focuses, and for VXLAN/NVGRE and 
MPLS VPN network option-c mode interconnection, our solution seems to be the 
only choice. For MPLS Over GRE/UDP, it's the debating point.We propose the same 
interconnecting solution for MPLSoGRE/oUDP, your proposal provides another 
optional solution, i think it works and is good. However, i think an 
independent informational draft only for this point maybe not necessary, we can 
describe it in same single draft which includes all possible solutions for IP 
overlay network(VXLAN/NVGRE/VXLAN-GPE/MPLSoGRE/MPLSoUDP) and MPLS VPN network 
interworking using option-c mode.

WH> I understand what you try to achieve but your proposal requires changes in 
the NVE/TOR as well since the TOR/NVE(s) out there don’t support multiple VXLAN 
ports and on top the whole MAC handling is not described and will require 
additional changes in the NVE/TOR data-planes. So the proposal you have I don’t 
support for being adopted since it requires to drastic changes in the data 
plane elements.


In summary, Option-c mode interconnecting solution between IP overlay network 
and MPLS VPN network is needed for BESS working group, and we should cover all 
possible solutions for all possible mainstream IP overlay encapsulations in 
single draft.

WH> The point is that your proposal requires too drastic impact in the DP and 
this is why I oppose to the draft being adopted. On top I am also questioning 
the real need for an NVE/TOR requiring to support this. Most use case I have 
seen require the application inside the DC consume the 3rd label and as such is 
very well supported with the proposal I suggested.


Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Saturday, November 14, 2015 15:12
To: Linda Dunbar; bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Linda, first of all the draft does not say anything about this and 2nd I am not 
confused and 3rd I explained the reasons why this does not fly. So read them

From: Linda Dunbar <linda.dun...@huawei.com<mailto:linda.dun...@huawei.com>>
Date: Friday 13 November 2015 at 23:18
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
"bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

If the draft adds some description to say that the VxLAN header is 
decapsulated, and the MAC header is terminated. It is the inner IP data frames 
to be passed to the MPLS IP-VPNs.

Will it clear out the confusion?

Linda

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 8:34 AM
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Thomas Morin]

Lucy Yong :
[Lucy1] Is this one?  “If we want to describe a model C VPN 
interconnect with a IP fabric in a DC I recommend to do an 
informational RFC that describes this using VXLAN-GPE, MPLSoGRE or 
MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.”


If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support 
VXLAN-GPE for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for 
this case, requires three labels + GRE/UDP tunnel, huge overhead is 
introduced here. This is very complex in my opinion. Our solution can 
apply VXLAN-GRE encap. and also support MPLSoX encap (5.1.1 and 
5.1.2). BTW, this may be not what you suggested.


I don't think Wim mentioned 3-labels over UDP or GRE.
Wim proposal works with 2-labels over an IP encap (the topmost label of 
the typical 3-label option C being replaced by an IP transport).


WH> it is less bits then you proposal and no changes in any CP/DP that 
is standardised today. Where is the complexity? Works today and is 
deployed in many networks


[Lucy2] This is not compliant with many DCs’ architecture as Thomas 
pointed out.


( Note that I merely mentioned the problem of vswitch/ToR support for 
MPLS/MPLS/(UDP or GRE), but had mentioned nothing about overall DC 
architecture.  )


-Thomas




*From: *Lucy yong <lucy.y...@huawei.com <mailto:lucy.y...@huawei.com>>
*Date: *Friday 13 November 2015 at 17:10
*To: *Wim Henderickx <wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alcatel-lucent.com>>, Haoweiguo 
<haowei...@huawei.com <mailto:haowei...@huawei.com>>
*Cc: *"bess@ietf.org <mailto:bess@ietf.org>" <bess@ietf.org 
<mailto:bess@ietf.org>>

*Subject: *RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination 
Ass, ASBRs do not maintain and distribute the VN routes; a tunnel is 
built between the end NVEs in source and destination Ass for traffic 
transport. Due to the different data planes in DC and WAN, the tunnel 
is stitched by several segments, IP tunnels are used in DCs, and MPLS 
tunnels are used in WAN.


Traditional OptionC requires end-to-end MPLS, which may fit to some 
DCI cases. However, there are many DCs that do not support MPLS data 
plane. This draft is to provide the solution for this use case.


Although VXLAN has UDP port number, if tunnel ingress and egress can 
negotiate to use another UDP port for the VXLAN encapsulation, I don’t 
see it breaks anything. Not sure if hw has this restriction either. 
Even yes, we can consider using UDP source port for this purpose. UDP 
source port is used for transit ECMP and filled by flow entropy, 
tunnel egress can determine the flow entropy value and inform tunnel 
ingress. Thus, tunnel egress (i.e. DC ASBR) can maintain UDP source 
port and MPLS label table; when DC ASBR receives a packet from NVE, by 
lookup the table, it gets the label for the packet, push the label on 
the packet before sending toward WAN.


Hope we together work out the solution for this valid use case and 
like to hear any better alternative.


Regards,

Lucy

*From:*BESS [mailto:bess-boun...@ietf.org] *On Behalf Of *Henderickx, 
Wim (Wim)

*Sent:* Thursday, November 12, 2015 11:00 PM
*To:* Haoweiguo
*Cc:* bess@ietf.org <mailto:bess@ietf.org>
*Subject:* Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since 
there is simpler solutions I don’t support this proposal.


Arguments have been given as to why.

*From: *Haoweiguo <haowei...@huawei.com <mailto:haowei...@huawei.com>>
*Date: *Friday 13 November 2015 at 02:44
*To: *Wim Henderickx <wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alcatel-lucent.com>>
*Cc: *"bess@ietf.org <mailto:bess@ietf.org>" <bess@ietf.org 
<mailto:bess@ietf.org>>

*Subject: *RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data 
center network. For layer 3 traffic forwarding, the MAC address in 
VXLAN/NVGRE is restricted only in data center inside domain. For the 
traffic from data center inside to outside,  the inner MAC 
address(destination MAC is ASBR-d's MAC, src MAC is NVE's MAC) in 
VXLAN/NVGRE will be dropped at ASBR-d, only IP payload will continue 
to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.


This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP 
network to interconnect with MPLS VPN network. In VXLAN-GPE, it 
supports IP in IP encapsulation, so no inner MAC concerns.


Thanks,

weiguo



*From:*BESS [bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] on 
behalf of Henderickx

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Thomas Morin]

Wim,

Henderickx, Wim (Wim) :

VXLAN has a dedicated UDP port and is very clear in the RFC7348


Well, having a port reserved for this use that won't be the default for 
another protocol is one thing, but that does not prevent in itself the 
same protocol to be applied on another range of ports. (Because HTTP 
specs says port 80 does not prevent the URI scheme to allow specifying 
the port in the URL)


Even reading the VXLAN (Informational) specs, we see room for 
flexibility wrt ports:


  -  Destination Port: IANA has assigned the value 4789 for the
 VXLAN UDP port, and this value SHOULD be used by default as the
 destination UDP port.  Some early implementations of VXLAN have
 used other values for the destination port.  To enable
 interoperability with these implementations, the destination
 port SHOULD be configurable.

I read "4789 SHOULD be used by default" and "SHOULD be configurable".




Will write something on what I proposed when I get some time, not soon


The co-chair in me needs to ask the following question: should this call 
for adoption be kept on hold until an outline of an alternative solution 
is provided ?


-Thomas







From: Lucy yong <lucy.y...@huawei.com <mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx <wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alcatel-lucent.com>>, Haoweiguo 
<haowei...@huawei.com <mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org <mailto:bess@ietf.org>" <bess@ietf.org 
<mailto:bess@ietf.org>>

Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination 
Ass, ASBRs do not maintain and distribute the VN routes; a tunnel is 
built between the end NVEs in source and destination Ass for traffic 
transport. Due to the different data planes in DC and WAN, the tunnel 
is stitched by several segments, IP tunnels are used in DCs, and MPLS 
tunnels are used in WAN.


Traditional OptionC requires end-to-end MPLS, which may fit to some 
DCI cases. However, there are many DCs that do not support MPLS data 
plane. This draft is to provide the solution for this use case.


Although VXLAN has UDP port number, if tunnel ingress and egress can 
negotiate to use another UDP port for the VXLAN encapsulation, I don’t 
see it breaks anything. Not sure if hw has this restriction either. 
Even yes, we can consider using UDP source port for this purpose. UDP 
source port is used for transit ECMP and filled by flow entropy, 
tunnel egress can determine the flow entropy value and inform tunnel 
ingress. Thus, tunnel egress (i.e. DC ASBR) can maintain UDP source 
port and MPLS label table; when DC ASBR receives a packet from NVE, by 
lookup the table, it gets the label for the packet, push the label on 
the packet before sending toward WAN.


Hope we together work out the solution for this valid use case and 
like to hear any better alternative.


Regards,

Lucy

*From:*BESS [mailto:bess-boun...@ietf.org] *On Behalf Of *Henderickx, 
Wim (Wim)

*Sent:* Thursday, November 12, 2015 11:00 PM
*To:* Haoweiguo
*Cc:* bess@ietf.org <mailto:bess@ietf.org>
*Subject:* Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since 
there is simpler solutions I don’t support this proposal.


Arguments have been given as to why.

*From: *Haoweiguo <haowei...@huawei.com <mailto:haowei...@huawei.com>>
*Date: *Friday 13 November 2015 at 02:44
*To: *Wim Henderickx <wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alcatel-lucent.com>>
*Cc: *"bess@ietf.org <mailto:bess@ietf.org>" <bess@ietf.org 
<mailto:bess@ietf.org>>

*Subject: *RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data 
center network. For layer 3 traffic forwarding, the MAC address in 
VXLAN/NVGRE is restricted only in data center inside domain. For the 
traffic from data center inside to outside,  the inner MAC 
address(destination MAC is ASBR-d's MAC, src MAC is NVE's MAC) in 
VXLAN/NVGRE will be dropped at ASBR-d, only IP payload will continue 
to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.


This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP 
network to interconnect with MPLS VPN network. In VXLAN-GPE, it 
supports IP in IP encapsulation, so no inner MAC concerns.


Thanks,

weiguo



*From:*BESS [bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] on 
behalf of Henderickx, Wim (Wim) [wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alc

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

In-line

From: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Date: Monday 16 November 2015 at 08:29
To: Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>, 
"bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc



From: BESS <bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>> on behalf of 
Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Monday 16 November 2015 at 07:47
To: "bess@ietf.org<mailto:bess@ietf.org>" 
<bess@ietf.org<mailto:bess@ietf.org>>, Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

Henderickx, Wim (Wim) :
VXLAN has a dedicated UDP port and is very clear in the RFC7348

Well, having a port reserved for this use that won't be the default for another 
protocol is one thing, but that does not prevent in itself the same protocol to 
be applied on another range of ports. (Because HTTP specs says port 80 does not 
prevent the URI scheme to allow specifying the port in the URL)

Even reading the VXLAN (Informational) specs, we see room for flexibility wrt 
ports:

  -  Destination Port: IANA has assigned the value 4789 for the
 VXLAN UDP port, and this value SHOULD be used by default as the
 destination UDP port.  Some early implementations of VXLAN have
 used other values for the destination port.  To enable
 interoperability with these implementations, the destination
 port SHOULD be configurable.

I read "4789 SHOULD be used by default" and "SHOULD be configurable".

WH> true but it does not say to use multiple at the same time. My read on this 
is that it allows a non standard port to be used, but not multiple at the same 
time.




Will write something on what I proposed when I get some time, not soon

The co-chair in me needs to ask the following question: should this call for 
adoption be kept on hold until an outline of an alternative solution is 
provided ?

WH> I first want to understand the requirements for this use case and I don’t 
agree to adoption since there are major DP implications in this.


-Thomas






From: Lucy yong 
<<mailto:lucy.y...@huawei.com>lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<<mailto:wim.henderi...@alcatel-lucent.com>wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>,
 Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "<mailto:bess@ietf.org>bess@ietf.org<mailto:bess@ietf.org>" 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo 
<<mailto:haowei...@huawei.com>haowei...@huawei.com<mailto:haowei.

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

From: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Date: Monday 16 November 2015 at 08:29
To: Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

In-line

From: BESS <bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>> on behalf of 
Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Monday 16 November 2015 at 07:37
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
Thomas, we can discuss forever and someone need to describe requirements, but 
the current proposal I cannot agree to for the reasons explained.

Well, although discussing forever is certainly not the goal, the reasons for 
rejecting a proposal need to be thoroughly understood.
WH> my point is what is the real driver for supporting a plain VXLAN data-plane 
here, the use cases I have seen in this txt is always where an application 
behind a NVE/TOR is demanding option c, but none of the NVE/TOR elements.


(more below)



2015-11-13, Henderickx, Wim (Wim):
Thomas, we need a solution that is implementable and will scale in 
control/data-plane. As such the solution I outline is standardised today and 
does not need new extensions in control/data-plane. Hence informational draft 
and can be implemented if there is market demand.

The above argument would fully make sense to me if support MPLS/MPLS/(UDP or 
GRE) was common place in ToRs or vswitches.

With respect to MAC you need to address both directions and does not make sense 
to manage this if it is not required/doe snot add value.

On VXLAN and the Ethernet header: it looks to me that addressing both direction 
is just a matter of documenting in the draft (if not already there) what is the 
expected behavior for VXLAN/NVGRE. But VXLAN and NVGRE are not the only encaps 
proposed in the specs anyway.

Compared to 3-label option C, my understanding is that the added-value is to 
not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs and 
vswitches.

The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in 
this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

WH> b depends on the use case

I don't get what you mean by "b depends on the use case".
WH> see my above comment. If the real use case is an application behind NVE/TOR 
requiring model C, than all the discussion on impact on NVE/TOR is void. As 
such I want to have a discussion on the real driver/requirement for option c 
interworking with an IP based Fabric.

I wrote the above based on the idea that the encap used in MPLS/MPLS/(UDP or 
GRE), which hence has to be supported on the ToRs and vswitches.
Another possibility would be service-label/middle-label/Ethernet assuming an L2 
adjacency between vswitches/ToRs and ASBRs, but this certainly does not match 
your typical DC architecture. Or perhaps had you something else in mind ?

WH> see above. The draft right now also requires changes in existing TOR/NVE so 
for me all this discussion/debate is void.


WH> and depending on implementation you don’t need to change any of the 
TOR/vswitches.

Does this mean that for some implementations you may not need to change any of 
the TOR/vswitches, but that for some others you may ?

WH> any proposal on the table requires changes, so for me this is not a valid 
discussion

Let me take a practical example : while I can quite easily see how to implement 
the procedures in draft-hao-bess-inter-nvo3-vpn-optionc based on current 
vswitch implementations of VXLAN, the lack of MPLS/MPLS/(UDP, GRE) support in 
commonplace vswitches seems to me as making that alternate solution you suggest 
harder to implement.

WH> I would disagree to this. Tell me which switch/TOR handles multiple UDP 
ports for VXLAN ?


-Thomas







From: Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "<mailto:bess@ietf.org>bess@ietf.org<mailto:bess@ietf.org>" 
<<mailto:bess@ietf.org>bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to implementations 
th

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Haoweiguo]

Hi Wim,

Thanks for your detail explanation.

In your solution, MPLSoGRE/oUDP needs to be supported in data center network. 
To support option-c interconnection with WAN network, TORs/NVEs need to add one 
BGP LSP Label in middle layer, so TORs/NVEs need to perform MPLS(VPN Layer) + 
MPLS (BGP LSP Label) + GRE/UDP encapsulation for the traffic from local 
connecting end systems to WAN network. In your solution, GRE/UDP layer is not 
used for transport layer stitching at ASBR-d, traditonal end to end BGP LSP is 
still used for transport layer.

In our solution, VXLAN/NVGRE are two of the focuses, and for VXLAN/NVGRE and 
MPLS VPN network option-c mode interconnection, our solution seems to be the 
only choice. For MPLS Over GRE/UDP, it's the debating point.We propose the same 
interconnecting solution for MPLSoGRE/oUDP, your proposal provides another 
optional solution, i think it works and is good. However, i think an 
independent informational draft only for this point maybe not necessary, we can 
describe it in same single draft which includes all possible solutions for IP 
overlay network(VXLAN/NVGRE/VXLAN-GPE/MPLSoGRE/MPLSoUDP) and MPLS VPN network 
interworking using option-c mode.

In summary, Option-c mode interconnecting solution between IP overlay network 
and MPLS VPN network is needed for BESS working group, and we should cover all 
possible solutions for all possible mainstream IP overlay encapsulations in 
single draft.

Thanks,

weiguo


From: BESS [bess-boun...@ietf.org] on behalf of Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com]
Sent: Saturday, November 14, 2015 15:12
To: Linda Dunbar; bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Linda, first of all the draft does not say anything about this and 2nd I am not 
confused and 3rd I explained the reasons why this does not fly. So read them

From: Linda Dunbar <linda.dun...@huawei.com<mailto:linda.dun...@huawei.com>>
Date: Friday 13 November 2015 at 23:18
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
"bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

If the draft adds some description to say that the VxLAN header is 
decapsulated, and the MAC header is terminated. It is the inner IP data frames 
to be passed to the MPLS IP-VPNs.

Will it clear out the confusion?

Linda

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 8:34 AM
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Thomas Morin]

Hi Wim,

I agree on the analysis that this proposal is restricted to 
implementations that supports the chosen encap with non-IANA ports 
(which may be hard to achieve for instance on hardware implementations, 
as you suggest), or to context where managing multiple IPs would be 
operationally viable.


However, it does not seem obvious to me how the alternative you propose 
[relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] 
addresses the issue of whether the encap behavior is supported or not 
(e.g. your typical ToR chipset possibly may not support this kind of 
encap,  and even software-based switches may not be ready to support 
that today).


My take is that having different options to adapt to various 
implementations constraints we may have would have value.


(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used 
with a dummy MAC address that would be replaced by the right MAC by a 
sender based on an ARP request when needed ?


Is the above the issue you had in mind about VXLAN and NVGRE ?

WH> yes


I you don't mind me asking : why do you challenge that ?


___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Fully agree Tomas’s points (below).

Lucy

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of thomas.mo...@orange.com
Sent: Friday, November 13, 2015 10:37 AM
To: Henderickx, Wim (Wim)
Cc: bess@ietf.org
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
Thomas, we need a solution that is implementable and will scale in 
control/data-plane. As such the solution I outline is standardised today and 
does not need new extensions in control/data-plane. Hence informational draft 
and can be implemented if there is market demand.

The above argument would fully make sense to me if support MPLS/MPLS/(UDP or 
GRE) was common place in ToRs or vswitches.


With respect to MAC you need to address both directions and does not make sense 
to manage this if it is not required/doe snot add value.

On VXLAN and the Ethernet header: it looks to me that addressing both direction 
is just a matter of documenting in the draft (if not already there) what is the 
expected behavior for VXLAN/NVGRE. But VXLAN and NVGRE are not the only encaps 
proposed in the specs anyway.

Compared to 3-label option C, my understanding is that the added-value is to 
not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs and 
vswitches.

The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in 
this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

Note that the 3-label option C approach with an MPLS/MPLS/(UDP or GRE) encap 
also requires the introduction of a control plane in the DC (possibly in an 
'SDN' controller) to learn and advertise the middle MPLS label.  Whether one 
consider that the best path to a working solution may also depend on the 
context.

Best,

-Thomas




From: Thomas Morin <thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to implementations 
that supports the chosen encap with non-IANA ports (which may be hard to 
achieve for instance on hardware implementations, as you suggest), or to 
context where managing multiple IPs would be operationally viable.

However, it does not seem obvious to me how the alternative you propose 
[relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] addresses the 
issue of whether the encap behavior is supported or not (e.g. your typical ToR 
chipset possibly may not support this kind of encap,  and even software-based 
switches may not be ready to support that today).

My take is that having different options to adapt to various implementations 
constraints we may have would have value.

(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used with a dummy 
MAC address that would be replaced by the right MAC by a sender based on an ARP 
request when needed ?


Is the above the issue you had in mind about VXLAN and NVGRE ?

WH> yes

I you don't mind me asking : why do you challenge that ?



_



Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,

Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.



This message and its attachments may contain confidential or privileged 
information that may be protected by law;

they should not be distributed, used or copied without authorisation.

If you have received this email in error, please notify the sender and delete 
this message and its attachments.

As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.

Thank you.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[thomas.morin]

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
Thomas, we need a solution that is implementable and will scale in 
control/data-plane. As such the solution I outline is standardised 
today and does not need new extensions in control/data-plane. Hence 
informational draft and can be implemented if there is market demand.


The above argument would fully make sense to me if support 
MPLS/MPLS/(UDP or GRE) was common place in ToRs or vswitches.


With respect to MAC you need to address both directions and does not 
make sense to manage this if it is not required/doe snot add value.


On VXLAN and the Ethernet header: it looks to me that addressing both 
direction is just a matter of documenting in the draft (if not already 
there) what is the expected behavior for VXLAN/NVGRE. But VXLAN and 
NVGRE are not the only encaps proposed in the specs anyway.


Compared to 3-label option C, my understanding is that the added-value 
is to not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs 
and vswitches.


The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior 
specified in this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)


Note that the 3-label option C approach with an MPLS/MPLS/(UDP or GRE) 
encap also requires the introduction of a control plane in the DC 
(possibly in an 'SDN' controller) to learn and advertise the middle MPLS 
label.  Whether one consider that the best path to a working solution 
may also depend on the context.


Best,

-Thomas




From: Thomas Morin <thomas.mo...@orange.com 
<mailto:thomas.mo...@orange.com>>

Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx <wim.henderi...@alcatel-lucent.com 
<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org <mailto:bess@ietf.org>" <bess@ietf.org 
<mailto:bess@ietf.org>>

Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to 
implementations that supports the chosen encap with non-IANA ports 
(which may be hard to achieve for instance on hardware 
implementations, as you suggest), or to context where managing 
multiple IPs would be operationally viable.


However, it does not seem obvious to me how the alternative you 
propose [relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) 
encap] addresses the issue of whether the encap behavior is supported 
or not (e.g. your typical ToR chipset possibly may not support this 
kind of encap,  and even software-based switches may not be ready to 
support that today).


My take is that having different options to adapt to various 
implementations constraints we may have would have value.


(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used 
with a dummy MAC address that would be replaced by the right MAC by 
a sender based on an ARP request when needed ?


Is the above the issue you had in mind about VXLAN and NVGRE ?

WH> yes


I you don't mind me asking : why do you challenge that ?





_

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.
Thank you.

___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 20:42
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP 
port. If someone wants to stick on the dedicate UDP port, we can use the src 
UDP port to achieve that.
WH> they are used for entropy, you seem to be  creating a kludge.
[Lucy1] It works.
WH2> good luck, but does not mean IETF has to adopt it. Mine also works :-)
The proposal I gave works with an IP data plane
[Lucy] I am not clear what is your proposal in this piled thread. Could you 
state or copy/paste again?
WH> look at the archives it is in there
[Lucy1] Is this one?  “If we want to describe a model C VPN interconnect with a 
IP fabric in a DC I recommend to do an informational RFC that describes this 
using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS 
label we defined in RFC4364.”

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE 
for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, 
requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This 
is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also 
support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.
WH> it is less bits then you proposal and no changes in any CP/DP that is 
standardised today. Where is the complexity? Works today and is deployed in 
many networks

Lucy
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(des

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

VXLAN has a dedicated UDP port and is very clear in the RFC7348
The proposal I gave works with an IP data plane
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 20:15
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP 
port. If someone wants to stick on the dedicate UDP port, we can use the src 
UDP port to achieve that.
WH> they are used for entropy, you seem to be  creating a kludge.
The proposal I gave works with an IP data plane
[Lucy] I am not clear what is your proposal in this piled thread. Could you 
state or copy/paste again?
WH> look at the archives it is in there

Lucy
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

Thomas, we can discuss forever and someone need to describe requirements, but 
the current proposal I cannot agree to for the reasons explained.

More in-line.

From: "thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>" 
<thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Friday 13 November 2015 at 17:36
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

2015-11-13, Henderickx, Wim (Wim):
Thomas, we need a solution that is implementable and will scale in 
control/data-plane. As such the solution I outline is standardised today and 
does not need new extensions in control/data-plane. Hence informational draft 
and can be implemented if there is market demand.

The above argument would fully make sense to me if support MPLS/MPLS/(UDP or 
GRE) was common place in ToRs or vswitches.

With respect to MAC you need to address both directions and does not make sense 
to manage this if it is not required/doe snot add value.

On VXLAN and the Ethernet header: it looks to me that addressing both direction 
is just a matter of documenting in the draft (if not already there) what is the 
expected behavior for VXLAN/NVGRE. But VXLAN and NVGRE are not the only encaps 
proposed in the specs anyway.

Compared to 3-label option C, my understanding is that the added-value is to 
not require the support of an MPLS/MPLS/(UDP or GRE) encap in ToRs and 
vswitches.

The right trade-off to make may in fact depend on whether you prefer:
(a) a new dataplane stitching behavior on DC ASBRs (the behavior specified in 
this draft)
or (b) an evolution of the encaps on the vswitches and ToRs to support 
MPLS/MPLS/(UDP or GRE)

WH> b depends on the use case and depending on implementation you don’t need to 
change any of the TOR/vswitches.


Note that the 3-label option C approach with an MPLS/MPLS/(UDP or GRE) encap 
also requires the introduction of a control plane in the DC (possibly in an 
'SDN' controller) to learn and advertise the middle MPLS label.  Whether one 
consider that the best path to a working solution may also depend on the 
context.

WH> we already have BGP for this.


Best,

-Thomas



From: Thomas Morin 
<<mailto:thomas.mo...@orange.com>thomas.mo...@orange.com<mailto:thomas.mo...@orange.com>>
Organization: Orange
Date: Friday 13 November 2015 at 09:57
To: Wim Henderickx 
<<mailto:wim.henderi...@alcatel-lucent.com>wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "<mailto:bess@ietf.org>bess@ietf.org<mailto:bess@ietf.org>" 
<bess@ietf.org<mailto:bess@ietf.org>>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

I agree on the analysis that this proposal is restricted to implementations 
that supports the chosen encap with non-IANA ports (which may be hard to 
achieve for instance on hardware implementations, as you suggest), or to 
context where managing multiple IPs would be operationally viable.

However, it does not seem obvious to me how the alternative you propose 
[relying on 3-label option C with an MPLS/MPLS/(UDP|GRE) encap] addresses the 
issue of whether the encap behavior is supported or not (e.g. your typical ToR 
chipset possibly may not support this kind of encap,  and even software-based 
switches may not be ready to support that today).

My take is that having different options to adapt to various implementations 
constraints we may have would have value.

(+ one question below on VXLAN...)

-Thomas


2015-11-12, Henderickx, Wim (Wim):
On VXLAN/NVGRE, do you challenge the fact that they would be used with a dummy 
MAC address that would be replaced by the right MAC by a sender based on an ARP 
request when needed ?

Is the above the issue you had in mind about VXLAN and NVGRE ?

WH> yes

I you don't mind me asking : why do you challenge that ?




_

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP 
port. If someone wants to stick on the dedicate UDP port, we can use the src 
UDP port to achieve that.
WH> they are used for entropy, you seem to be  creating a kludge.
[Lucy1] It works.
The proposal I gave works with an IP data plane
[Lucy] I am not clear what is your proposal in this piled thread. Could you 
state or copy/paste again?
WH> look at the archives it is in there
[Lucy1] Is this one?  “If we want to describe a model C VPN interconnect with a 
IP fabric in a DC I recommend to do an informational RFC that describes this 
using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS 
label we defined in RFC4364.”

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE 
for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, 
requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This 
is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also 
support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.

Lucy
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, Novemb

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

termine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

Wim,

VXLAN has a dedicated UDP port and is very clear in the RFC7348
[Lucy] This does not prevent other implementation to use VXLAN with other UDP 
port. If someone wants to stick on the dedicate UDP port, we can use the src 
UDP port to achieve that.
WH> they are used for entropy, you seem to be  creating a kludge.
[Lucy1] It works.
WH2> good luck, but does not mean IETF has to adopt it. Mine also works :-)
The proposal I gave works with an IP data plane
[Lucy2] IETF does not specify how entropy value is generated yet. It can be 
provided by tunnel egress and tell the ingress.
WH3> the RFC is pretty clear:
— copy —


The UDP source port number

 be calculated using a hash of fields from the inner packet --

 one example being a hash of the inner Ethernet frame's headers.

 This is to enable a level of entropy for the ECMP/load-

 balancing of the VM-to-VM traffic across the VXLAN overlay.

 When calculating the UDP source port number in this manner, it

 is RECOMMENDED that the value be in the dynamic/private port

 range 49152-65535 [RFC6335<https://tools.ietf.org/html/rfc6335>].


— copy –

[Lucy3]  This is one way to calculate, perhaps current practice too. But it is 
just recommended by RFC, not mandated.
[Lucy] I am not clear what is your proposal in this piled thread. Could you 
state or copy/paste again?
WH> look at the archives it is in there
[Lucy1] Is this one?  “If we want to describe a model C VPN interconnect with a 
IP fabric in a DC I recommend to do an informational RFC that describes this 
using VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS 
label we defined in RFC4364.”

If yes, you suggest NVE to implement MPLSoGRE or MPLSoUDP, to support VXLAN-GPE 
for L3 payload. First of all, using MPLSoGRE or MPLSoUDP for this case, 
requires three labels + GRE/UDP tunnel, huge overhead is introduced here. This 
is very complex in my opinion. Our solution can apply VXLAN-GRE encap. and also 
support MPLSoX encap (5.1.1 and 5.1.2). BTW, this may be not what you suggested.
WH> it is less bits then you proposal and no changes in any CP/DP that is 
standardised today. Where is the complexity? Works today and is deployed in 
many networks
[Lucy2] This is not compliant with many DCs’ architecture as Thomas pointed 
out. To make them doing it, it becomes very complex.
WH3> it is even deployed in existing DC(s)

Lucy
Will write something on what I proposed when I get some time, not soon

From: Lucy yong <lucy.y...@huawei.com<mailto:lucy.y...@huawei.com>>
Date: Friday 13 November 2015 at 17:10
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>, 
Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Hi Wim,

OptionC is very useful for DCI use case. In this case, multi-hop EBGP 
redistributes VN routes between the end NVEs in source and destination Ass, 
ASBRs do not maintain and distribute the VN routes; a tunnel is built between 
the end NVEs in source and destination Ass for traffic transport. Due to the 
different data planes in DC and WAN, the tunnel is stitched by several 
segments, IP tunnels are used in DCs, and MPLS tunnels are used in WAN.

Traditional OptionC requires end-to-end MPLS, which may fit to some DCI cases. 
However, there are many DCs that do not support MPLS data plane. This draft is 
to provide the solution for this use case.

Although VXLAN has UDP port number, if tunnel ingress and egress can negotiate 
to use another UDP port for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim He

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Linda Dunbar]

Wim,

If the draft adds some description to say that the VxLAN header is 
decapsulated, and the MAC header is terminated. It is the inner IP data frames 
to be passed to the MPLS IP-VPNs.

Will it clear out the confusion?

Linda

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 8:34 AM
To: bess@ietf.org
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

in UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Lucy yong]

ort for the VXLAN encapsulation, I don’t see it breaks 
anything. Not sure if hw has this restriction either. Even yes, we can consider 
using UDP source port for this purpose. UDP source port is used for transit 
ECMP and filled by flow entropy, tunnel egress can determine the flow entropy 
value and inform tunnel ingress. Thus, tunnel egress (i.e. DC ASBR) can 
maintain UDP source port and MPLS label table; when DC ASBR receives a packet 
from NVE, by lookup the table, it gets the label for the packet, push the label 
on the packet before sending toward WAN.

Hope we together work out the solution for this valid use case and like to hear 
any better alternative.

Regards,
Lucy





From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 11:00 PM
To: Haoweiguo
Cc: bess@ietf.org<mailto:bess@ietf.org>
Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo <haowei...@huawei.com<mailto:haowei...@huawei.com>>
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
<wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>>
Cc: "bess@ietf.org<mailto:bess@ietf.org>" <bess@ietf.org<mailto:bess@ietf.org>>
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo


From: BESS [bess-boun...@ietf.org<mailto:bess-boun...@ietf.org>] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com<mailto:wim.henderi...@alcatel-lucent.com>]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org<mailto:bess@ietf.org>
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc
I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

Linda, first of all the draft does not say anything about this and 2nd I am not 
confused and 3rd I explained the reasons why this does not fly. So read them

From: Linda Dunbar >
Date: Friday 13 November 2015 at 23:18
To: Wim Henderickx 
>, 
"bess@ietf.org" >
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc

Wim,

If the draft adds some description to say that the VxLAN header is 
decapsulated, and the MAC header is terminated. It is the inner IP data frames 
to be passed to the MPLS IP-VPNs.

Will it clear out the confusion?

Linda

From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Henderickx, Wim (Wim)
Sent: Thursday, November 12, 2015 8:34 AM
To: bess@ietf.org
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Haoweiguo]

Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo



From: BESS [bess-boun...@ietf.org] on behalf of Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[Henderickx, Wim (Wim)]

The whole draft complicates any data plane we defined so far and since there is 
simpler solutions I don’t support this proposal.
Arguments have been given as to why.

From: Haoweiguo >
Date: Friday 13 November 2015 at 02:44
To: Wim Henderickx 
>
Cc: "bess@ietf.org" >
Subject: RE: draft-hao-bess-inter-nvo3-vpn-optionc


Hi Wim,

It is used for layer 3 VPN CE device to visit IP based overlay data center 
network. For layer 3 traffic forwarding, the MAC address in VXLAN/NVGRE is 
restricted only in data center inside domain. For the traffic from data center 
inside to outside,  the inner MAC address(destination MAC is ASBR-d's MAC, src 
MAC is NVE's MAC) in VXLAN/NVGRE will be dropped at ASBR-d, only IP payload 
will continue to be carried into MPLS network. I can emphasize this point in my 
later version, it doesn't have much impact on the whole solution.

This draft also is suitable for VXLAN-GPE and MPLSoGRE/MPLSoUDP network to 
interconnect with MPLS VPN network. In VXLAN-GPE, it supports IP in IP 
encapsulation, so no inner MAC concerns.



Thanks,

weiguo



From: BESS [bess-boun...@ietf.org] on behalf of 
Henderickx, Wim (Wim) 
[wim.henderi...@alcatel-lucent.com]
Sent: Thursday, November 12, 2015 22:33
To: bess@ietf.org
Subject: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

I don’t support the adoption of this draft as a WG. There is a major flaw in 
this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS IP-VPNs. 
VXLAN/NVGRE contains a MAC address and IP-VPNs don’t. The draft does not talk 
about any of this and introduces a lot of complexity for nothing.

If we want to describe a model C VPN interconnect with a IP fabric in a DC I 
recommend to do an informational RFC that describes this using VXLAN-GPE, 
MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS label we defined in 
RFC4364.
___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess

Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc

[thomas.morin]

Hi Wim,

Thanks for your feedback.

With my co-chair hat, let met ask a few questions for the sake of well 
understanding the issue you raise.


Henderickx, Wim (Wim):
I don’t support the adoption of this draft as a WG. There is a major 
flaw in this proposal:
Basically the encapsulation of VXLAN/NVGRE is incompatible with MPLS 
IP-VPNs. VXLAN/NVGRE contains a MAC address and IP-VPNs don’t.


This comment only relates to a subset of the encaps proposed in the 
solution, right ?

(e.g. does not apply to MPLS/GRE and MPLS/UDP)

On VXLAN/NVGRE, do you challenge the fact that they would be used with a 
dummy MAC address that would be replaced by the right MAC by a sender 
based on an ARP request when needed ?



The draft [...] introduces a lot of complexity for nothing.


Can you detail what you consider is a lot of complexity ?



If we want to describe a model C VPN interconnect with a IP fabric in 
a DC I recommend to do an informational RFC that describes this using 
VXLAN-GPE, MPLSoGRE or MPLSoUDP encapsulation and retain the E2E MPLS 
label we defined in RFC4364.


My understanding of this draft's value is the fact that it avoids having 
service-label related state (forwarding and BGP state) on the routers at 
the DC-WAN border.  This is one of the typical property of 4364 option 
C.It is not obvious at all to me, based on existing spec, how to get 
the same benefit on the DC-side router in an architecture where the 
DC-side router also terminates the overlay tunnels.


Do you challenge the goal itself (not having service-label related state 
on the overlay tunnels router) or suggest that the same benefit can be 
achieved with less complexity ?


-Thomas

_

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.
Thank you.

___
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess