Lucy, My apologies, I misunderstood.
I think we have to accept the fact that we will have to deal with a multiplicity of different encapsulations in the data plane along a packet's e2e path and that we should take a more measured approach to deciding how to deal with this in a general and extensible way before accepting any solutions. Yours Irrespectively, John > -----Original Message----- > From: thomas.mo...@orange.com [mailto:thomas.mo...@orange.com] > Sent: Friday, November 20, 2015 12:04 PM > To: John E Drake; Lucy yong; Henderickx, Wim (Wim); bess@ietf.org > Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc > > 2015-11-20, John E Drake: > > That presupposes that the group likes either of the two proposed solutions > in your draft. > > John, I think Lucy's "two solutions" was referring to draft-hao-bess-inter- > nvo3-vpn-optionc solution and the 3-label Optionc MPLS/MPLS/UDP solution > described by Wim. > > -Thomas > > > > > > >> -----Original Message----- > >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of Lucy yong > >> Sent: Friday, November 20, 2015 11:49 AM > >> To: EXT - thomas.mo...@orange.com; Henderickx, Wim (Wim); > >> bess@ietf.org > >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc > >> > >> Share my 2 cent. > >> > >> Cloud providers want to tunnel its customer traffic through DC (AS)BR. > >> Option C is a way to realize it. Both solutions summarized by Thomas > >> have no change on WAN VPN side and seamlessly work with WAN VPN > option C. > >> However, to support either solution, DC has to do some enhancement on > >> DC BR or ToR switch, etc, which dictates to different implementations > >> within a DC. Option C pro and con remains regardless what > >> implementation is used in a DC. > >> > >> Two solutions should not coexist in one DC (does not make a sense), > >> but it does not matter if one DC operator prefers to use one solution > >> and another DC prefers to use another solution. Since there are many > >> cloud providers today, it is worth for the WG to document both > >> solutions and point out the implementation requirements on impacted > >> components. Then, up to vendors and operators to select a solution for > their DC. > >> > >> It does not make a sense for us to vote which solution is better here > >> because a better solution for a DC depends on many factors. > >> > >> > >> Lucy > >> > >> > >> > >> > >> > >> -----Original Message----- > >> From: BESS [mailto:bess-boun...@ietf.org] On Behalf Of > >> thomas.mo...@orange.com > >> Sent: Friday, November 20, 2015 3:02 AM > >> To: Henderickx, Wim (Wim); bess@ietf.org > >> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc > >> > >> 2015-11-19, Henderickx, Wim (Wim): > >>> WH> I vote for a an evolution of switches/TORs that have proper > >>> support for this. I hope some HW vendors of TOR chips shime in, but > >>> I am told the MPLS solution is possible in the next generation chips > >>> they are working on. > >> > >> Well, it looks like the key questions are: > >> - when would ToR chips support MPLS/MPLS/UDP ? (the generation that > >> has been released recently but not present in most shipping ToRs yet, > >> the next one ?) > >> - do we want an interim VXLAN-based solution ? (that will involve at > >> best a performance penalty with existing chips, and at worse > >> impossible to implement -- we haven't seen clear information in this > >> thread) > >> > >> -Thomas > >> > >> > >>>> Zhuangshunwan : > >>>>> > >>>>> Hi Diego, > >>>>> > >>>>> Thanks for your comments. Pls see inline with [Vincent]. > >>>>> > >>>>> Vincent > >>>>> > >>>>> *发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia > >> del Rio > >>>>> *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题 > >> :*Re: [bess] > >>>>> draft-hao-bess-inter-nvo3-vpn-optionc > >>>>> > >>>>> Some comments from my side, I think the draft makes quite a few > >>>>> assumptions on specific implementation details that are way too > >>>>> general to be considered widely available. Most of the TOR devices > >>>>> today already pay a double-pass penalty when doing routing of > >>>>> traffic in/out of vxlan-type tunnels. Only the newest generation > >>>>> can route into tunnels without additional passes. And are > >>>>> definitively limited in being able to arbitrary select UDP ports > >>>>> on a per tunnel basis. In fact, most are even limited at using > >>>>> more than one VNID per "service" of sorts. [Vincent]: Yes, the new > >>>>> generation BCM chipset has already supported VXLAN routing > without > >>>>> additional passes. For OVS/TOR, VXLAN encapsulation is more > >>>>> popular than MPLSoGRE/UDP, and has better performance. The > >>>>> IP-addressed based implementation which would, I assume, imply > >>>>> assigning one or more CIDRs to a loopback interface on the ASBR-d > >>>>> is also quite arbitrary and does not seem like a technically > >>>>> "clean" solution. (besides burning tons of IPs). As a side-note, > >>>>> most PE-grade routers i've worked with were quite limited in terms > >>>>> of IP addresses used for tunnel termination and it wasn't that > >>>>> just a simple pool can be used. [Vincent]: I think the larger VTEP > >>>>> IP address range on ASBR-d has no limitations. > >>>>> For the hardware on ASBR-d, it has capability to terminate > >>>>> multiple VXLAN tunnels with arbitrary destination VTEP IP > >>>>> addresses. Wim's mentions on cases where the Application itself, > >>>>> hosted in a datacenter, would be part of the option-C > >>>>> interconnect, was dismissed without much discussion so far, while, > >>>>> if we look in detail at the type of users which will even consider > >>>>> a complex topology like model-C its most likely users and > >>>>> operators very familiar with MPLS VPNs in the WAN. Those type of > >>>>> operators will most likely be very interested in deploying MPLS or > >>>>> WAN-grade applications (i.e., virtual-routers or other > >>>>> VNFs) in the DC and thus its highly likely that the interconnect > >>>>> would not terminate at the NVE itself but rather the TS (the > >>>>> virtual machine). Also, the use of UDP ports at random would imply > >>>>> quite complex logic on the ASBR-d IMHO. Im not saying its > >>>>> impossible, but since the received packet now not only has to be > >>>>> received on a random (though locally chosen) UDP port and this > >>>>> information preserved in the pipeline to be able to do the > >>>>> double-tunnel-stitching, it sounds quite complex. I am sure > >>>>> someone in the list will mention this has already been implemented > >>>>> somewhere, and I won't argue with that. And let's not even bring > >>>>> into account what this would do to any DC middlebox that now has > >>>>> to look at vxlan over almost any random port. We have to go back > >>>>> to the "is it a 4 or is it a 6 in byte x" heuristics to try to > >>>>> guess whether the packet is vxlan or just something entirely > >>>>> different running over IP. [Vincent]: Using NP or multi-core CPU > >>>>> hardware technology, it can be implemented although deeper packet > >>>>> inspection is needed to perform UDP port and MPLS stitching. In > >>>>> general I feel the proposed solution seems to be fitting of a > >>>>> specific use-case which is not really detailed > >>>>> in the draft and does not describe a solution that would > >>>>> "elegantly" solve the issues at hand. It just feels like we're > >>>>> using any available bit-space to stuff data into places that do > >>>>> not necesarily belong. Yes, MPLS encapsulations on virtual > >>>>> switches are not yet fully available, and there can be some > >>>>> performance penalty on the TORs, but the solutions are much > >>>>> cleaner from a control and data plane point of view. Maybe I'm too > >>>>> utopic. [Vincent]: I think pure VXLAN solution has its scenario, > >>>>> it's general rather than specific. We can't require all OVS/NVEs > >>>>> support VXLAN + MPLSoGRE at the same time. Best regards, Diego > >>>>> ------------------------------------------------------------------ > >>>>> -- > >>>>> ------------- > >>>>> > >>>>> > >> Hi, > >>>>> The problem we are trying to solve is to reduce data center > >>>>> GW/ASBR-d's forwarding table size, the motivation is same as > >>>>> traditional MPLS VPN option-C. Currently, the most common practise > >>>>> on ASBR-d is to terminate VXLAN encapsulation, look up local > >>>>> routing table, and then perform MPLS encapsulation to the WAN > network. > >>>>> ASBR-d needs to maintain all VM's MAC/IP. In Option-C method, only > >>>>> transport layer information needed to be maintained at GW/ASBR-d, > >>>>> the scalability will be greatly enhanced. Traditonal Option-C is > >>>>> only for MPLS VPN network interworking, because VXLAN is > becoming > >>>>> pervasive in data center, the solution in this draft was proposed > >>>>> for the heterogeneous network interworking. The advantage of this > >>>>> solution is that only VXLAN encapsulation is required for OVS/TOR. > >>>>> Unlike Wim's solution, east-west bound traffic uses VXLAN encap, > >>>>> while north-south bound traffic uses MPLSoGRE/UDP encap. There > are > >>>>> two solutions in this draft: 1. Using VXLAN tunnel destination IP > >>>>> for stitching at ASBR-d. No data plane modification requirements > >>>>> on OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d > >>>>> normally is router, it has capability to realize the hardware > >>>>> changes. It will consume many IP addresses and the IP pool for > >>>>> allocation needs to be configured on ASBR-d beforehand. 2. Using > >>>>> VXLAN destination UDP port for stitching at ASBR-d. Compared with > >>>>> solution 1, less IP address will be consumed for allocation. If > >>>>> UDP port range is too large, we can combine with solution 1 and 2. > >>>>> In this solution, both data plane modification changes are needed > >>>>> at OVS/TOR and ASBR-d. ASBR-d also has capability to realize the > >>>>> hardware changes. For OVS, it also can realize the data plane > >>>>> changes. For TOR switch, it normally can't realize this function. > >>>>> This solution mainly focuses on pure software based overlay > >>>>> network, it has more scalability. In public cloud data center, > >>>>> software based overlay network is the majority case. Whether using > >>>>> solution 1 or 2 depends on the operators real envionment. So I > >>>>> think our solution has no flaws, it works fine. > >>>>> Thanks, weiguo ________________________________ From: BESS > >>>>> [bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] on behalf > >>>>> of John E Drake [jdr...@juniper.net <mailto:jdr...@juniper.net>] > >>>>> Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim > (Wim); > >>>>> EXT - thomas.mo...@orange.com > <mailto:thomas.mo...@orange.com>; > >> BESS > >>>>> Subject: Re: [bess] draft-hao-bess-inter-nvo3-vpn-optionc Hi, I > >>>>> think Wim has conclusively demonstrated that this draft has fatal > >>>>> flaws and I don’t support it. I also agree with his suggestion > >>>>> that we first figure out what problem we are trying to solve > >>>>> before solving it. Yours Irrespectively, John From: BESS > >>>>> [mailto:bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] On > >>>>> Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015 > >>>>> 12:49 PM To: EXT - thomas.mo...@orange.com > >>>>> <mailto:thomas.mo...@orange.com>; BESS Subject: Re: [bess] > >>>>> draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it > >>>>> is can be implemented in its VXLAN variant with existing vswitches > >>>>> (e.g. OVS allows to choose the VXLAN destination port, ditto for > >>>>> the linux kernel stack). (ToR is certainly another story, most of > >>>>> them not having a flexible enough VXLAN dataplane nor support for > >>>>> any > >>>>> MPLS-over-IP.) WH> and how many ports simultaneously would they > >>>>> support? For this to work every tenant needs a different VXLAN UDP > >>>>> destination port/receive port. There might be SW elements that > >>>>> could do some of this, but IETF defines solutions which should be > >>>>> implemented across the board HW/SW/etc. > >>>>> Even if some SW switches can do this, the proposal will impose so > >>>>> many issues in HW/data-plane engines that I cannot be behind this > >>>>> solution. To make this work generically we will have to make > >>>>> changes anyhow. Given this, we better do it in the right way and > >>>>> guide the industry to a solution which does not imply those > complexities. > >>>>> Otherwise we will stick with these specials forever with all > >>>>> consequences (bugs, etc). - snip - From: > >>>>> "thomas.mo...@orange.com > >>>>> > >> > <mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com > >>>>> <mailto:thomas.mo...@orange.com>>" <thomas.mo...@orange.com > >>>>> > >> > <mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com > >>>>> <mailto:thomas.mo...@orange.com>>> Organization: Orange Date: > >>>>> Tuesday 17 November 2015 at 01:37 To: Wim Henderickx > >>>>> <wim.henderi...@alcatel-lucent.com > >>>>> <mailto:wim.henderickx@alcatel- > >> lucent.com><mailto:wim.henderickx@alc > >>>>> atel-lucent.com > >>>>> > >>>>> > >> <mailto:wim.henderi...@alcatel-lucent.com>>>, BESS <bess@ietf.org > >>>>> <mailto:bess@ietf.org><mailto:bess@ietf.org > >>>>> <mailto:bess@ietf.org>>> Subject: Re: [bess] > >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16, > >>>>> Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim): > Thomas, > >> we > >>>>> can discuss forever and someone need to describe requirements, but > >>>>> the current proposal I cannot agree to for the reasons explained. > >>>>> TM> Well, although discussing forever is certainly not the goal, > >>>>> TM> the > >>>>> reasons for rejecting a proposal need to be thoroughly understood. > >>>>> WH> my point is what is the real driver for supporting a plain > >>>>> WH> VXLAN > >>>>> data-plane here, the use cases I have seen in this txt is always > >>>>> where an application behind a NVE/TOR is demanding option c, but > >>>>> none of the NVE/TOR elements. > >>>>> My understanding is that the applications are contexts where > >>>>> overlays are present is when workloads (VMs or baremetal) need to > >>>>> be interconnected with VPNs. In these contexts, there can be > >>>>> reasons to want Option C to reduce the state on ASBRs. In these > >>>>> context, its not the workload (VM or baremetal) that would > >>>>> typically handle VRFs, but really the vswitch or ToR. WH2> can it not > be all cases: > >>>>> TOR/vswitch/Application. I would make the solution flexible to > >>>>> support all of these not? 2015-11-13, Henderickx, Wim (Wim): TM> > >>>>> The right trade-off to make may in fact depend on whether you > prefer: > >>>>> (a) a new dataplane stitching behavior on DC ASBRs (the behavior > >>>>> specified in this draft) or > >>>>> (b) an evolution of the encaps on the vswitches and ToRs to > >>>>> support MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I > >>>>> don't get what you mean by "b depends on the use case". WH> see > my > >>>>> above comment. If the real use case is an application behind > >>>>> NVE/TOR requiring model C, than all the discussion on impact on > >>>>> NVE/TOR is void. As such I want to have a discussion on the real > >>>>> driver/requirement for option c interworking with an IP based > >>>>> Fabric. Although I can agree than detailing requirements can > >>>>> always help, I don't think one can assume a certain application to > >>>>> dismiss the proposal. WH> for me the proposal is not acceptable > >>>>> for the reasons explained: too much impact on the data-planes I > >>>>> wrote the above based on the idea that the encap used in > >>>>> MPLS/MPLS/(UDP or GRE), which hence has to be supported on the > ToRs and vswitches. > >>>>> Another possibility would be service-label/middle-label/Ethernet > >>>>> assuming an L2 adjacency between vswitches/ToRs and ASBRs, but > >>>>> this certainly does not match your typical DC architecture. Or > >>>>> perhaps had you something else in mind ? WH> see above. The draft > >>>>> right now also requires changes in existing TOR/NVE so for me all > >>>>> this discussion/debate is void. No, the spec as it is can be > >>>>> implemented in its VXLAN variant with existing vswitches (e.g. OVS > >>>>> allows to choose the VXLAN destination port, ditto for the linux > >>>>> kernel stack). (ToR is certainly another story, most of them not > >>>>> having a flexible enough VXLAN dataplane nor support for any > >>>>> MPLS-over-IP.) > >>>>> WH> and how many ports simultaneously would they support? WH> > and > >>>>> depending on implementation you don’t need to change any of the > >>>>> TOR/vswitches. Does this mean that for some implementations you > >>>>> may not need to change any of the TOR/vswitches, but that for some > >>>>> others you may ? WH> any proposal on the table requires changes, > >>>>> so for me this is not a valid discussion See above, the proposal > >>>>> in the draft does not necessarily need changes in vswitches. Let > >>>>> me take a practical example : while I can quite easily see how to > >>>>> implement the procedures in draft-hao-bess-inter-nvo3-vpn-optionc > >>>>> based on current vswitch implementations of VXLAN, the lack of > >>>>> MPLS/MPLS/(UDP, GRE) support in commonplace vswitches seems to > >> me as > >>>>> making that alternate solution you suggest harder to implement. > >>>>> WH> I would disagree to this. Tell me which switch/TOR handles > >>>>> multiple UDP ports for VXLAN ? I mentioned _v_switches, and many > >>>>> do support a variable destination port for VXLAN, which is > >>>>> sufficient to implement what the draft proposes. -Thomas From: > >>>>> Thomas Morin <thomas.mo...@orange.com > >>>>> > >> > <mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com > >>>>> <mailto:thomas.mo...@orange.com>>> Organization: Orange Date: > >>>>> Friday 13 November 2015 at 09:57 To: Wim Henderickx > >>>>> <wim.henderi...@alcatel-lucent.com > >>>>> <mailto:wim.henderickx@alcatel- > >> lucent.com><mailto:wim.henderickx@alc > >>>>> atel-lucent.com > >>>>> > >>>>> > >> <mailto:wim.henderi...@alcatel-lucent.com>>> > >>>>> Cc: "bess@ietf.org <mailto:bess@ietf.org><mailto:bess@ietf.org > >>>>> <mailto:bess@ietf.org>>" <bess@ietf.org > >>>>> <mailto:bess@ietf.org><mailto:bess@ietf.org > >>>>> <mailto:bess@ietf.org>>> Subject: Re: [bess] > >>>>> draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the > >>>>> analysis that this proposal is restricted to implementations that > >>>>> supports the chosen encap with non-IANA ports (which may be hard > >>>>> to achieve for instance on hardware implementations, as you > >>>>> suggest), or to context where managing multiple IPs would be > >>>>> operationally viable. However, it does not seem obvious to me how > >>>>> the alternative you propose [relying on 3-label option C with an > >>>>> MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the > >> encap > >>>>> behavior is supported or not (e.g. your typical ToR chipset > >>>>> possibly may not support this kind of encap, and even > >>>>> software-based switches may not be ready to support that today). > >>>>> My take is that having different options to adapt to various > >>>>> implementations constraints we may have would have value. (+ one > >>>>> question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim > >>>>> (Wim): On VXLAN/NVGRE, do you challenge the fact that they would > >>>>> be used with a dummy MAC address that would be replaced by the > >>>>> right MAC by a sender based on an ARP request when needed ? Is the > >>>>> above the issue you had in mind about VXLAN and NVGRE ? WH> yes I > >>>>> you don't mind me asking : why do you challenge that ? > >>>>> > >> > >> > __________________________________________________________ > >> > __________________________________________________________ > >> _____ > >> > >> Ce message et ses pieces jointes peuvent contenir des informations > >> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, > >> exploites ou copies sans autorisation. Si vous avez recu ce message > >> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi > >> que les pieces jointes. Les messages electroniques etant susceptibles > >> d'alteration, France Telecom - Orange decline toute responsabilite si > >> ce message a ete altere, deforme ou falsifie. Merci > >> > >> This message and its attachments may contain confidential or > >> privileged information that may be protected by law; they should not > >> be distributed, used or copied without authorization. > >> If you have received this email in error, please notify the sender > >> and delete this message and its attachments. > >> As emails may be altered, France Telecom - Orange shall not be liable > >> if this message was modified, changed or falsified. > >> Thank you. > >> > >> _______________________________________________ > >> BESS mailing list > >> BESS@ietf.org > >> https://www.ietf.org/mailman/listinfo/bess > >> _______________________________________________ > >> BESS mailing list > >> BESS@ietf.org > >> https://www.ietf.org/mailman/listinfo/bess > > > __________________________________________________________ > __________________________________________________________ > _____ > > Ce message et ses pieces jointes peuvent contenir des informations > confidentielles ou privilegiees et ne doivent donc pas etre diffuses, > exploites > ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez > le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les > messages electroniques etant susceptibles d'alteration, France Telecom - > Orange decline toute responsabilite si ce message a ete altere, deforme ou > falsifie. Merci > > This message and its attachments may contain confidential or privileged > information that may be protected by law; they should not be distributed, > used or copied without authorization. > If you have received this email in error, please notify the sender and delete > this message and its attachments. > As emails may be altered, France Telecom - Orange shall not be liable if this > message was modified, changed or falsified. > Thank you. _______________________________________________ BESS mailing list BESS@ietf.org https://www.ietf.org/mailman/listinfo/bess