2015-11-19, Henderickx, Wim (Wim):
WH> I vote for a an evolution of switches/TORs that have proper
support for this. I hope some HW vendors of TOR chips shime in, but I
am told the MPLS solution is possible in the next generation chips
they are working on.
Well, it looks like the key questions are:
- when would ToR chips support MPLS/MPLS/UDP ? (the generation that has
been released recently but not present in most shipping ToRs yet, the
next one ?)
- do we want an interim VXLAN-based solution ? (that will involve at
best a performance penalty with existing chips, and at worse impossible
to implement -- we haven't seen clear information in this thread)
-Thomas
Zhuangshunwan :
Hi Diego,
Thanks for your comments. Pls see inline with [Vincent].
Vincent
*发件人:*BESS [mailto:bess-boun...@ietf.org] *代表 *Diego Garcia del
Rio *发送时间:*2015年11月18日14:25 *收件人:*bess@ietf.org *主题:*Re: [bess]
draft-hao-bess-inter-nvo3-vpn-optionc
Some comments from my side, I think the draft makes quite a few
assumptions on specific implementation details that are way too
general to be considered widely available. Most of the TOR
devices today already pay a double-pass penalty when doing
routing of traffic in/out of vxlan-type tunnels. Only the newest
generation can route into tunnels without additional passes. And
are definitively limited in being able to arbitrary select UDP
ports on a per tunnel basis. In fact, most are even limited at
using more than one VNID per "service" of sorts. [Vincent]: Yes,
the new generation BCM chipset has already supported VXLAN
routing without additional passes. For OVS/TOR, VXLAN
encapsulation is more popular than MPLSoGRE/UDP, and has better
performance. The IP-addressed based implementation which would, I
assume, imply assigning one or more CIDRs to a loopback interface
on the ASBR-d is also quite arbitrary and does not seem like a
technically "clean" solution. (besides burning tons of IPs). As a
side-note, most PE-grade routers i've worked with were quite
limited in terms of IP addresses used for tunnel termination and
it wasn't that just a simple pool can be used. [Vincent]: I think
the larger VTEP IP address range on ASBR-d has no limitations.
For the hardware on ASBR-d, it has capability to terminate
multiple VXLAN tunnels with arbitrary destination VTEP IP
addresses. Wim's mentions on cases where the Application itself,
hosted in a datacenter, would be part of the option-C
interconnect, was dismissed without much discussion so far,
while, if we look in detail at the type of users which will even
consider a complex topology like model-C its most likely users
and operators very familiar with MPLS VPNs in the WAN. Those type
of operators will most likely be very interested in deploying
MPLS or WAN-grade applications (i.e., virtual-routers or other
VNFs) in the DC and thus its highly likely that the interconnect
would not terminate at the NVE itself but rather the TS (the
virtual machine). Also, the use of UDP ports at random would
imply quite complex logic on the ASBR-d IMHO. Im not saying its
impossible, but since the received packet now not only has to be
received on a random (though locally chosen) UDP port and this
information preserved in the pipeline to be able to do the
double-tunnel-stitching, it sounds quite complex. I am sure
someone in the list will mention this has already been
implemented somewhere, and I won't argue with that. And let's
not even bring into account what this would do to any DC
middlebox that now has to look at vxlan over almost any random
port. We have to go back to the "is it a 4 or is it a 6 in byte
x" heuristics to try to guess whether the packet is vxlan or just
something entirely different running over IP. [Vincent]: Using NP
or multi-core CPU hardware technology, it can be implemented
although deeper packet inspection is needed to perform UDP port
and MPLS stitching. In general I feel the proposed solution seems
to be fitting of a specific use-case which is not really detailed
in the draft and does not describe a solution that would
"elegantly" solve the issues at hand. It just feels like we're
using any available bit-space to stuff data into places that do
not necesarily belong. Yes, MPLS encapsulations on virtual
switches are not yet fully available, and there can be some
performance penalty on the TORs, but the solutions are much
cleaner from a control and data plane point of view. Maybe I'm
too utopic. [Vincent]: I think pure VXLAN solution has its
scenario, it's general rather than specific. We can't require all
OVS/NVEs support VXLAN + MPLSoGRE at the same time. Best
regards, Diego
---------------------------------------------------------------------------------
Hi,
The problem we are trying to solve is to reduce data center
GW/ASBR-d's forwarding table size, the motivation is same as
traditional MPLS VPN option-C. Currently, the most common
practise on ASBR-d is to terminate VXLAN encapsulation, look up
local routing table, and then perform MPLS encapsulation to the
WAN network. ASBR-d needs to maintain all VM's MAC/IP. In
Option-C method, only transport layer information needed to be
maintained at GW/ASBR-d, the scalability will be greatly
enhanced. Traditonal Option-C is only for MPLS VPN network
interworking, because VXLAN is becoming pervasive in data center,
the solution in this draft was proposed for the heterogeneous
network interworking. The advantage of this solution is that only
VXLAN encapsulation is required for OVS/TOR. Unlike Wim's
solution, east-west bound traffic uses VXLAN encap, while
north-south bound traffic uses MPLSoGRE/UDP encap. There are two
solutions in this draft: 1. Using VXLAN tunnel destination IP for
stitching at ASBR-d. No data plane modification requirements on
OVS or TOR switches, only hardware changes on ASBR-d. ASBR-d
normally is router, it has capability to realize the hardware
changes. It will consume many IP addresses and the IP pool for
allocation needs to be configured on ASBR-d beforehand. 2. Using
VXLAN destination UDP port for stitching at ASBR-d. Compared with
solution 1, less IP address will be consumed for allocation. If
UDP port range is too large, we can combine with solution 1 and
2. In this solution, both data plane modification changes are
needed at OVS/TOR and ASBR-d. ASBR-d also has capability to
realize the hardware changes. For OVS, it also can realize the
data plane changes. For TOR switch, it normally can't realize
this function. This solution mainly focuses on pure software
based overlay network, it has more scalability. In public cloud
data center, software based overlay network is the majority
case. Whether using solution 1 or 2 depends on the operators real
envionment. So I think our solution has no flaws, it works fine.
Thanks, weiguo ________________________________ From: BESS
[bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] on behalf
of John E Drake [jdr...@juniper.net <mailto:jdr...@juniper.net>]
Sent: Wednesday, November 18, 2015 2:49 To: Henderickx, Wim
(Wim); EXT - thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com>; BESS Subject: Re: [bess]
draft-hao-bess-inter-nvo3-vpn-optionc Hi, I think Wim has
conclusively demonstrated that this draft has fatal flaws and I
don’t support it. I also agree with his suggestion that we first
figure out what problem we are trying to solve before solving
it. Yours Irrespectively, John From: BESS
[mailto:bess-boun...@ietf.org <mailto:bess-boun...@ietf.org>] On
Behalf Of Henderickx, Wim (Wim) Sent: Tuesday, November 17, 2015
12:49 PM To: EXT - thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com>; BESS Subject: Re: [bess]
draft-hao-bess-inter-nvo3-vpn-optionc — Snip — No, the spec as it
is can be implemented in its VXLAN variant with existing
vswitches (e.g. OVS allows to choose the VXLAN destination port,
ditto for the linux kernel stack). (ToR is certainly another
story, most of them not having a flexible enough VXLAN dataplane
nor support for any MPLS-over-IP.) WH> and how many ports
simultaneously would they support? For this to work every tenant
needs a different VXLAN UDP destination port/receive port. There
might be SW elements that could do some of this, but IETF defines
solutions which should be implemented across the board HW/SW/etc.
Even if some SW switches can do this, the proposal will impose so
many issues in HW/data-plane engines that I cannot be behind this
solution. To make this work generically we will have to make
changes anyhow. Given this, we better do it in the right way and
guide the industry to a solution which does not imply those
complexities. Otherwise we will stick with these specials forever
with all consequences (bugs, etc). - snip - From:
"thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com>>" <thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com>>> Organization: Orange Date:
Tuesday 17 November 2015 at 01:37 To: Wim Henderickx
<wim.henderi...@alcatel-lucent.com
<mailto:wim.henderi...@alcatel-lucent.com><mailto:wim.henderi...@alcatel-lucent.com
<mailto:wim.henderi...@alcatel-lucent.com>>>, BESS <bess@ietf.org
<mailto:bess@ietf.org><mailto:bess@ietf.org
<mailto:bess@ietf.org>>> Subject: Re: [bess]
draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, WG, 2015-11-16,
Henderickx, Wim (Wim): 2015-11-13, Henderickx, Wim (Wim): Thomas,
we can discuss forever and someone need to describe requirements,
but the current proposal I cannot agree to for the reasons
explained. TM> Well, although discussing forever is certainly not
the goal, the reasons for rejecting a proposal need to be
thoroughly understood. WH> my point is what is the real driver
for supporting a plain VXLAN data-plane here, the use cases I
have seen in this txt is always where an application behind a
NVE/TOR is demanding option c, but none of the NVE/TOR elements.
My understanding is that the applications are contexts where
overlays are present is when workloads (VMs or baremetal) need to
be interconnected with VPNs. In these contexts, there can be
reasons to want Option C to reduce the state on ASBRs. In these
context, its not the workload (VM or baremetal) that would
typically handle VRFs, but really the vswitch or ToR. WH2> can it
not be all cases: TOR/vswitch/Application. I would make the
solution flexible to support all of these not? 2015-11-13,
Henderickx, Wim (Wim): TM> The right trade-off to make may in
fact depend on whether you prefer: (a) a new dataplane stitching
behavior on DC ASBRs (the behavior specified in this draft) or
(b) an evolution of the encaps on the vswitches and ToRs to
support MPLS/MPLS/(UDP or GRE) WH> b depends on the use case I
don't get what you mean by "b depends on the use case". WH> see
my above comment. If the real use case is an application behind
NVE/TOR requiring model C, than all the discussion on impact on
NVE/TOR is void. As such I want to have a discussion on the real
driver/requirement for option c interworking with an IP based
Fabric. Although I can agree than detailing requirements can
always help, I don't think one can assume a certain application
to dismiss the proposal. WH> for me the proposal is not
acceptable for the reasons explained: too much impact on the
data-planes I wrote the above based on the idea that the encap
used in MPLS/MPLS/(UDP or GRE), which hence has to be supported
on the ToRs and vswitches. Another possibility would be
service-label/middle-label/Ethernet assuming an L2 adjacency
between vswitches/ToRs and ASBRs, but this certainly does not
match your typical DC architecture. Or perhaps had you something
else in mind ? WH> see above. The draft right now also requires
changes in existing TOR/NVE so for me all this discussion/debate
is void. No, the spec as it is can be implemented in its VXLAN
variant with existing vswitches (e.g. OVS allows to choose the
VXLAN destination port, ditto for the linux kernel stack). (ToR
is certainly another story, most of them not having a flexible
enough VXLAN dataplane nor support for any MPLS-over-IP.) WH> and
how many ports simultaneously would they support? WH> and
depending on implementation you don’t need to change any of the
TOR/vswitches. Does this mean that for some implementations you
may not need to change any of the TOR/vswitches, but that for
some others you may ? WH> any proposal on the table requires
changes, so for me this is not a valid discussion See above, the
proposal in the draft does not necessarily need changes in
vswitches. Let me take a practical example : while I can quite
easily see how to implement the procedures in
draft-hao-bess-inter-nvo3-vpn-optionc based on current vswitch
implementations of VXLAN, the lack of MPLS/MPLS/(UDP, GRE)
support in commonplace vswitches seems to me as making that
alternate solution you suggest harder to implement. WH> I would
disagree to this. Tell me which switch/TOR handles multiple UDP
ports for VXLAN ? I mentioned _v_switches, and many do support a
variable destination port for VXLAN, which is sufficient to
implement what the draft proposes. -Thomas From: Thomas Morin
<thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com><mailto:thomas.mo...@orange.com
<mailto:thomas.mo...@orange.com>>> Organization: Orange Date:
Friday 13 November 2015 at 09:57 To: Wim Henderickx
<wim.henderi...@alcatel-lucent.com
<mailto:wim.henderi...@alcatel-lucent.com><mailto:wim.henderi...@alcatel-lucent.com
<mailto:wim.henderi...@alcatel-lucent.com>>>
Cc: "bess@ietf.org <mailto:bess@ietf.org><mailto:bess@ietf.org
<mailto:bess@ietf.org>>" <bess@ietf.org
<mailto:bess@ietf.org><mailto:bess@ietf.org
<mailto:bess@ietf.org>>> Subject: Re: [bess]
draft-hao-bess-inter-nvo3-vpn-optionc Hi Wim, I agree on the
analysis that this proposal is restricted to implementations that
supports the chosen encap with non-IANA ports (which may be hard
to achieve for instance on hardware implementations, as you
suggest), or to context where managing multiple IPs would be
operationally viable. However, it does not seem obvious to me how
the alternative you propose [relying on 3-label option C with an
MPLS/MPLS/(UDP|GRE) encap] addresses the issue of whether the
encap behavior is supported or not (e.g. your typical ToR chipset
possibly may not support this kind of encap, and even
software-based switches may not be ready to support that today).
My take is that having different options to adapt to various
implementations constraints we may have would have value. (+ one
question below on VXLAN...) -Thomas 2015-11-12, Henderickx, Wim
(Wim): On VXLAN/NVGRE, do you challenge the fact that they would
be used with a dummy MAC address that would be replaced by the
right MAC by a sender based on an ARP request when needed ? Is
the above the issue you had in mind about VXLAN and NVGRE ? WH>
yes I you don't mind me asking : why do you challenge that ?
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete
altere, deforme ou falsifie. Merci
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorization.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, France Telecom - Orange shall not be liable if this
message was modified, changed or falsified.
Thank you.
_______________________________________________
BESS mailing list
BESS@ietf.org
https://www.ietf.org/mailman/listinfo/bess
