Hi Larry, 1) A name for the service (lets's pick one, is it hybrid, combined or unified?)
[Lizhong] IRB, which have been used before. We could change it to virtual IRB. 2) To address it from a solution point of view (e.g. are packets forwarded based on MAC or based on IP when source and dest are in the same subnet?). [Lizhong] Yes, we should do that, and there are already some drafts there ( http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00). For IP forwarding for intra subnet traffic: 1. I don’t think it could be wildly accepted in next two or three years to have a pure IP datacenter. Pure Ethernet traffic and MAC forwarding will be there anyway. 2. From the dataplane point, you have to design larger IP routing table, and possibly larger TCAM (for prefix entry), and higher cost consequently. What’s more, should we even design aging for IP entry like MAC entry? 3. From the control plane point, you need to change the mapping entry rule, and may have new interoperable issue with legacy L2/L3 VPN. I don’t see much benefit. Could anyone give me some hints? Thanks Lizhong From: Larry Kreeger (kreeger) [mailto:[email protected]] Sent: 2013年10月24日 8:32 To: Xuxiaohu; Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas Narten Cc: [email protected]; [email protected] Subject: Re: 答复: [nvo3] Distributed Gateways [was Re: NVO3 Architecture document] I've now read through this whole thread twice. I see now that there may be some confusion between an L2/L3 service (what tenant's see) and an L2/L3 solution (how NVO3 provides the service). Prior to this thread, I may have agreed that we don't need to define a hybrid/combined/unified L2/L3 "service", but now I'm less sure. Just be be clear, I was not calling so much for a definition of the service, but that we need: 1) A name for the service (lets's pick one, is it hybrid, combined or unified?) 2) To address it from a solution point of view (e.g. are packets forwarded based on MAC or based on IP when source and dest are in the same subnet?). As I stated before, I believe the control plane requirements will be different for a combined service than for a pure L2 or pure L3 service, and that we if don't start specifically calling out the control plane requirements for a combined service (from a solution point of view), we will have a very low likelihood of interoperable solutions as each implementer fills in the gaps in their own way. Does the WG agree that we should do 1) and 2) above? I would also not object if we had a short description in the framework of what the combined service looks like compared to a pure L2 or pure L3 service, just so we can level set for everyone. Thanks, Larry From: Xuxiaohu <[email protected]> Date: Tuesday, October 22, 2013 11:38 PM To: Larry Kreeger <[email protected]>, Lizhong Jin <[email protected]>, "Yves Hertoghs (yhertogh)" <[email protected]>, Lucy yong < [email protected]>, Thomas Narten <[email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: 答复: [nvo3] Distributed Gateways [was Re: NVO3 Architecture document] 发件人: [email protected] [mailto:[email protected]] 代表 Larry Kreeger (kreeger) 发送时间: 2013年10月23日 11:27 收件人: Lizhong Jin; Yves Hertoghs (yhertogh); Lucy yong; Thomas Narten 抄送: [email protected]; [email protected] 主题: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture document] Hi Lizong, Let me give more details on my thinking. A pure L2 service only requires inner MAC address to outer IP address mappings. When a frame is delivered between two TS, the source and dest MAC addresses have not been changed. A pure L3 service only requires inner IP to outer IP address mappings. There is no expectation that two TS are on the same L2 segment, and the TS would always ARP for a router to reach another TS. When a frame is delivered between the two TS, both the source and dest MAC will have been rewritten. TTL will have been decremented at least once. I'm expecting a hybrid L2/L3 service to preserve L2 semantics within a subnet. TS ARP for the other TS on its subnet instead of the router. The source and dest MAC are not rewritten. TTL is not decremented. From the perspective of the two TS on the same subnet, they are connected to the same L2 segment. To achieve this, the NVEs must behave differently when forwarding IP packets within the same L2 VN vs between different L2 VNs. For example, the NVE would need to respond to ARP, not with its own MAC address, but with the remote TSs MAC address if the TS are in the same L2 VN. Furthermore, as I wrote below, there needs to be a mapping of the set of L2 VNs that belong to the L3 VN. If so, what’s the difference between hybrid L2/L3 service and IRB (Integrated Routing and Bridging)? In addition, I believe the hybrid L2/L3 service you mentioned above is different from what Yves meant by “combined L2/L3”. The former is about bridging for intra-subnet traffic and routing for inter-subnet traffic while the latter is about routing for (both intra-subnet and inter-subnet) IP traffic while bridging for non-IP traffic. Best regards, Xiaohu So, I still think NVEs will need some additional information for a hybrid service, and I think we will need to describe the behavior of a hybrid service beyond a sentence in the framework saying it is possible. Otherwise, I think it is quite likely that two different hybrid implementations will not interoperate. - Larry From: Lizhong Jin <[email protected]> Date: Tuesday, October 22, 2013 7:31 PM To: Larry Kreeger <[email protected]>, "Yves Hertoghs (yhertogh)" < [email protected]>, Lucy yong <[email protected]>, Thomas Narten < [email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: RE: [nvo3] Distributed Gateways [was Re: NVO3 Architecture document] > >With regards to the 'new service type' of a combined L2/L3 NVE, >http://tools.ietf.org/html/draft-hertoghs-nvo3-lisp-controlplane-unifie >d-0 >0 > describes what you are referring to , but i am not sure if it needs to >be called a new service-type. I see it more as a deployment choice >where you choose to deploy distributed gateways with uniform mac/IP >addressing across the DC, collocate them with the L2 overlay, and do >some traffic steering to make sure IP traffic gets sent to the L3 >overlay (at all times), and non-IP traffic gets sent across the L2 overlay. LK> I'll give you an example of why I think the protocol requirements will be different for L2 vs L3 vs a combined L2/L3 service. For an L2 VN, the VN needs to be identified (e.g. with a Name or ID). For an L3 VN, it similarly needs to be identified. For a combined L2/L3 service, I think the NVEs need to know both the L3 VN identity (one), and all the L2 VN identities that are part of the L3 VN. When doing distributed L3 forwarding between a TS on one L2 VN to one on another L2 VN, it will need to know not just the mapping of inner to outer address, but the mapping of inner L3 address to destination L2 VN and MAC address (so it can rewrite the MAC and the L2 VN). [Lizhong] I tend to agree with Yves, it is more of a deployment choice. The behavior you described for L3 forwarding between two L2 VPN is more of a standard L3 forwarding function, looking up with destination IP address, then getting the destination MAC, source MAC, VLAN and NVO3 tunnel from the table. The MAC and VLAN are resolved from the next hop within same L3 VN. We write a combined L2/L3 forwarding for NVO3 last year, just for reference http://tools.ietf.org/html/draft-jin-nvo3-virb-centralized-00. BTW, we may update it soon. Thanks Lizhong > >Yves > >On 21/10/13 10:28, "Larry Kreeger (kreeger)" <[email protected]> wrote: > >>Lucy, >> >>See inline with LK2>. >> >> - Larry >> >>On 10/20/13 8:20 PM, "Lucy yong" <[email protected]> wrote: >> >>>Hi Larry, >>> >>>Please see inline with [Lucy] >>> >>>-----Original Message----- >>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >>>Sent: Friday, October 18, 2013 6:39 PM >>>To: Lucy yong; Thomas Narten >>>Cc: [email protected] >>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>document] >>> >>>Lucy, >>> >>>See inline with LK>. >>> >>>Thanks, Larry >>> >>>On 10/18/13 3:21 PM, "Lucy yong" <[email protected]> wrote: >>> >>>>Larry, >>>> >>>>Distributed L3 gateway is very useful and some vendors already >>>>implement that. >>> >>>I agree. >>> >>>>Current framework also includes L2/L3 service as Marc agrees to add >>>>the text I proposed although we did not explicitly define as a >>>>service type. >>> >>>I think we will need to because the control plane requirements will >>>likely be different for a hybrid service. >>>[Lucy] Do you mean that we should define an l2/L3 service type? I >>>full agree and hope see more people support that too. We can either >>>define it in the existing framework, or in the framework addition >>>draft I submitted while ago. >> >>LK2> Yes, if the group is to seriously address a hybrid L2/L3 service, >>LK2> I >>think we need to define it as we will need to identify its >>requirements independently from a pure L2 or pure L3 service. >> >>> >>>> >>>>Current architecture clear states NVE and NVA roles, i.e. NVE >>>>performs forwarding, and NVA performs routing. >>> >>>I'm not sure what you mean by routing. Are you referring to L3 service? >>>If so, I don't agree. >>>[Lucy] Sorry to make you confuse. The routing here does not mean L3 >>>services. I should state that NVE performs data plan forwarding, NVA >>>performs control plane routing, which, IMO, it is not just simple DB >>>to have inner to outer mappings. NVA may get the routing policy from >>>operators or customer, then interpret the policy, then generates the >>>mapping of tenant/next-hop location and send to NVEs. An NVE receives >>>the mapping in which, if the location is the same as itself, it >>>translates to a tenant/VAP mapping; If not, it installs as an >>>inner/outer mapping. >>>Thus, it works regardless whether sender tenant and destination >>>tenant are one the same NVE or on different NVEs. The mapping between >>>tenant/location is fully controlled under NVA. In this way, operator >>>only needs to input the routing policies to NVA. NVE simply performs >>>the forwarding accordingly. This is also my view about the SDN based >>>architecture. >>> >>>>If NVA is not able to distribute routing policy to NVE at all, I do >>>>not know how NVA can perform route distribution control? >>> >>>You lost me. Are you referring to a hybrid L2/L3 service? >>>[Lucy] Let me know above explanation help or not. No, not particulate >>>to >>>L2/L3 service but that certainly applies to L2/L3 service. >> >>LK2> OK, if it isn't specific to L2/L3 service, than I'd rather leave >>LK2> it >>out of the discussion for now. And yes, the above helped me >>understand what you meant. The term "routing" is very overloaded. >> >>> >>>> If NVA only supports simple inner to outer mappings, how can NVE >>>>get information to perform local forwarding? >>> >>>Inner to outer mapping resolution works fine for pure L2 or pure L3 >>>service. Local forwarding doesnšt need mappings, the local NVE knows >>>what VAPs the TSI are connected to. >>>[Lucy] does pure L3 service means an L3VPN w/o any policy? For an L3 >>>service, you can implement it w/o policy or w policy. >> >>LK2> Yes, I was thinking of pure L3 service without adding policy. >>LK2> I'm >>pretty sure you need more than just inner to outer mappings to >>implement policy. >> >>>IMO: NVE is not the entity to enforce the policy. NVA is the entity >>>to enforce the policy regardless the tenant locations. Again, Network >>>virtualization overlay has to address how to support the policies and >>>tenant mobility. IMO: current architecture draft is vague in or lacks >>>of describing this but it is important to architect this when having >>>data plan and control plane separated on different entities. >> >>LK2> I'm not sure if I agree with you about the NVE not being the >>LK2> entity >>to enforce policy, but I think it is something better discussed in a >>conversation (not email). >> >>> >>>Thanks, >>>Lucy >>> >>>> >>>>Thanks, >>>>Lucy >>>> >>>>-----Original Message----- >>>>From: Larry Kreeger (kreeger) [mailto:[email protected]] >>>>Sent: Friday, October 18, 2013 5:00 PM >>>>To: Thomas Narten; Lucy yong >>>>Cc: [email protected] >>>>Subject: Re: [nvo3] Distributed Gateways [was Re: NVO3 Architecture >>>>document] >>>> >>>>Hi Thomas and Lucy, >>>> >>>>The WG needs to think hard about this one. >>>> >>>>Support of a distributed L3 gateway function between L2 VNs is a >>>>significant increase in scope of the NVA, and the NVE to NVA protocol. >>>>Where we had previously stated L2 service or L3 service and pretty >>>>much left a combined L2/L3 service as an exercise for the reader, we >>>>would now be adding whatever mechanisms are needed to the protocols. >>>>We will need to add cases for >>>>L2 service, L3 service and L2/L3 service. We no longer have simple >>>>inner to outer mappings, but now need NVEs to do MAC rewrites, local >>>>NVE ARP termination, and multiple lookups depending on the >>>>destination MAC address (first L2, then potentially L3). We will >>>>also need to distribute two different VN identifiers (one for L2 and >>>>one for L3), and somehow convey the containment relationship between >>>>the two (multiple L2 VNs within one >>>>L3 VN). While I think this is all very useful, I just want to make >>>>sure the WG agrees to this since I feel it is a significant >>>>change/increase in scope from my perspective. >>>> >>>>Thanks, Larry >>>> >>>> >>>> >>>>On 10/18/13 2:52 PM, "Thomas Narten" <[email protected]> wrote: >>>> >>>>>Hi Lucy. >>>>> >>>>>Lucy yong <[email protected]> writes: >>>>> >>>>>> Section 5.3 describes gateways. IMO: it misses an important use >>>>>> case. A Gateway, say overlay gateway, may be used to interconnect >>>>>> two or more overlay VNs. In this case, the traffic traversing >>>>>> between two overlay VNs must go through the gateway where the >>>>>> policy can be enforced. Furthermore, it is possible to implement >>>>>> centralized or distributed overlay gateway. The latter has >>>>>> overlay gateway function implemented on NVEs. Thus, it requests >>>>>> the cross-VN policies to be distributed to NVEs. >>>>> >>>>>> Current section seems very focus on overlay VN interconnect a >>>>>> non-overlay network, which centralized gateway architecture is >>>>>> practical. But in overlay networks, both centralized or >>>>>> distributed are possible and depend on the applications. >>>>> >>>>>Agreed. I propose adding a new section after 5.3 that says: >>>>> >>>>> <section title="Distributed Gateways"> >>>>> <t> >>>>> The relaying of traffic from one VN to another deserves >>>>> special consideration. The previous section described >>>>> gateways performing this function. If such gateways are >>>>> centralized, traffic between TSes on different VNs can take >>>>> suboptimal paths, i.e., triangular routing results in paths >>>>> that always traverse the gateway. As an optimization, >>>>> individual NVEs can be part of a distributed gateway that >>>>> performs such relaying, reducing or completely eliminating >>>>> triangular routing. In a distributed gateway, each ingress >>>>> NVE can perform such relaying activity directly, so long as >>>>> it has access to the policy information needed to determine >>>>> whether cross-VN communication is allowed. Having individual >>>>> NVEs be part of a distributed gateway allows them to tunnel >>>>> traffic directly to the destination NVE without the need to >>>>> take suboptimal paths. >>>>> </t> >>>>> <t> >>>>> The NVO3 architecture should [must? or just say it does?] >>>>> support distributed gateways. Such support requires that >>>>> NVO3 control protocols include mechanisms for the >>>>> maintenance and distribution of policy information about >>>>> what type of cross-VN communication is allowed so that NVEs >>>>> acting as distributed gateways can tunnel traffic from one >>>>> VN to another as appropriate. >>>>> </t> >>>>> </section> >>>>> >>>>>Thoughts? >>>>> >>>>>Thomas >>>>> >>>>>_______________________________________________ >>>>>nvo3 mailing list >>>>>[email protected] >>>>>https://www.ietf.org/mailman/listinfo/nvo3 >>>> >>> >> >>_______________________________________________ >>nvo3 mailing list >>[email protected] >>https://www.ietf.org/mailman/listinfo/nvo3 >
_______________________________________________ nvo3 mailing list [email protected] https://www.ietf.org/mailman/listinfo/nvo3
