Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR
Hi Numan, Thanks for the feedback. Submitted v7 with updated ovn-architecture.xml. Thanks Regards, Ankur From: Numan Siddique Sent: Wednesday, May 8, 2019 4:30 AM To: Ankur Sharma Cc: ovs-dev@openvswitch.org Subject: Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR On Wed, May 8, 2019 at 2:13 AM Ankur Sharma mailto:ankur.sha...@nutanix.com>> wrote: Hi Numan, Thank you so much for trying out the patch. Please find my replies inline regarding your test. For other comments in the code, i will be happy to make the change and v6 will have it. Thanks Regards, Ankur From: Numan Siddique mailto:nusid...@redhat.com>> Sent: Tuesday, May 7, 2019 10:07 AM To: Ankur Sharma mailto:ankur.sha...@nutanix.com>> Cc: ovs-dev@openvswitch.org<mailto:ovs-dev@openvswitch.org> Subject: Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR On Fri, May 3, 2019 at 3:04 AM Ankur Sharma mailto:ankur.sha...@nutanix.com>> wrote: Background: [1] https://mail.openvswitch.org/pipermail/ovs-dev/2018-October/353066.html [mail.openvswitch.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_pipermail_ovs-2Ddev_2018-2DOctober_353066.html&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=OerTfhGsADp01PypwIUM88kogzs3tjwSi5R4eg84yDo&e=> [2] https://docs.google.com/document/d/1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU/edit?usp=sharing [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU_edit-3Fusp-3Dsharing&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=-9fgFCK15HOs9DwR7cxbMskrngMo6LTzr-yatSFvQmM&e=> This Series: Layer 3 E-W and Layer 3 N-S (NO NAT) changes for vlan backed distributed logical router. This patch: A. Key difference between an overlay logical switch and vlan backed logical switch is that for vlan logical switches packets are not encapsulated. Hence, if a distributed router port is connected to vlan type logical switch, then router port mac as source mac could be seen from multiple hypervisors. Same pairs coming from multiple ports from a top of the rack switch (TOR) perspective could be seen as a security threat and it could send alarms, drop the packets or block the ports etc. This patch addresses the same by introducing the concept of chassis mac. A chassis mac is CMS provisioned unique mac per chassis. For any routed packet (i.e source mac is router port mac) going on the wire on a vlan type logical switch, we will replace its source mac with chassis mac. This replacing of source mac with chassis mac will happen in table=65 of the logical switch datapath. A flow is added at priority 150, which matches the source mac and replaces it with chassis mac if the value is a router port mac. Example flow: cookie=0x0, duration=67765.830s, table=65, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=150,reg15=0x1,metadata=0x4, dl_src=00:00:01:01:02:03 actions=mod_dl_src:aa:bb:cc:dd:ee:ff, mod_vlan_vid:1000,output:16 Here, 00:00:01:01:02:03 is router port mac and aa:bb:cc:dd:ee:ff is chassis mac. B. This patch adds one more change of associating "types" with logical switches. i.e a logical switch could be of type "overlay" or "bridged". This is done to explicitly call out that on a bridged logical switch there will no encapsulation. Just a localnet port's presence is not sufficient, as we do encap while redirecting the packet to gateway chassis. By marking the logical switch as bridged, we can either avoid redirection totally (if there is no NAT) or do redirection based on router port mac, rather than encap over a tunnel. Hi Ankur, I applied both the patches in the series and I tested them. For now I have only tested E/W scenario. I have one questions. In my setup, I created 2 bridged logical switches - sw0 and sw1 and attached a router to it. I have 2 chassis - c1 and c2. On c1 mac-bindings are configured as - sw0: c1-sw0-MAC, sw1:c1-sw1-MAC On c2 mac-bindings are configured as - sw0: c2-sw0-MAC, sw1:c2-sw1-MAC [ANKUR]: Sorry, a little confused here the chassis mac need not be on per logical switch, chassis mac should be configured on per physical bridge basis. I understand. I used this just as an example. In this particular case, I created localnet ports for both sw0 and sw1 withthe options "network_name=sw0/sw1". I have a logical port - sw0-port1 (which belongs to sw0 logical switch) with IP - 10.0.0.4 and another logical port - sw1-port1 (which belongs to sw1) with IP - 20.0.0.4. sw0-port1 is bound on chassis c1 and sw1-port1 on chassis c2. When sw0-port1 (10.0.0.4) pings sw1-port1 (20.0.0.4), the router po
Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR
On Wed, May 8, 2019 at 2:13 AM Ankur Sharma wrote: > Hi Numan, > > Thank you so much for trying out the patch. > Please find my replies inline regarding your test. > > For other comments in the code, i will be happy to make the change and v6 > will have it. > > Thanks > > Regards, > Ankur > > > > > *From:* Numan Siddique > *Sent:* Tuesday, May 7, 2019 10:07 AM > *To:* Ankur Sharma > *Cc:* ovs-dev@openvswitch.org > *Subject:* Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan > backed DVR > > > > > > > > On Fri, May 3, 2019 at 3:04 AM Ankur Sharma > wrote: > > Background: > [1] https://mail.openvswitch.org/pipermail/ovs-dev/2018-October/353066.html > [mail.openvswitch.org] > <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_pipermail_ovs-2Ddev_2018-2DOctober_353066.html&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=OerTfhGsADp01PypwIUM88kogzs3tjwSi5R4eg84yDo&e=> > [2] > https://docs.google.com/document/d/1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU/edit?usp=sharing > [docs.google.com] > <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU_edit-3Fusp-3Dsharing&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=-9fgFCK15HOs9DwR7cxbMskrngMo6LTzr-yatSFvQmM&e=> > > This Series: > Layer 3 E-W and Layer 3 N-S (NO NAT) changes for vlan > backed distributed logical router. > > This patch: > > A. > Key difference between an overlay logical switch and > vlan backed logical switch is that for vlan logical switches > packets are not encapsulated. > > Hence, if a distributed router port is connected to vlan type > logical switch, then router port mac as source mac could be > seen from multiple hypervisors. Same pairs coming > from multiple ports from a top of the rack switch (TOR) perspective > could be seen as a security threat and it could send alarms, drop > the packets or block the ports etc. > > This patch addresses the same by introducing the concept of chassis mac. > A chassis mac is CMS provisioned unique mac per chassis. For any routed > packet > (i.e source mac is router port mac) going on the wire on a vlan type > logical switch, we will replace its source mac with chassis mac. > > This replacing of source mac with chassis mac will happen in table=65 > of the logical switch datapath. A flow is added at priority 150, which > matches the source mac and replaces it with chassis mac if the value > is a router port mac. > > Example flow: > cookie=0x0, duration=67765.830s, table=65, n_packets=0, n_bytes=0, > idle_age=65534, hard_age=65534, priority=150,reg15=0x1,metadata=0x4, > dl_src=00:00:01:01:02:03 actions=mod_dl_src:aa:bb:cc:dd:ee:ff, > mod_vlan_vid:1000,output:16 > > Here, 00:00:01:01:02:03 is router port mac and aa:bb:cc:dd:ee:ff > is chassis mac. > > B. > This patch adds one more change of associating "types" with logical > switches. i.e a logical switch could be of type "overlay" or "bridged". > This is done to explicitly call out that on a bridged logical > switch there will no encapsulation. > Just a localnet port's presence is not sufficient, as we do > encap while redirecting the packet to gateway chassis. > By marking the logical switch as bridged, we can either > avoid redirection totally (if there is no NAT) or do redirection > based on router port mac, rather than encap over a tunnel. > > > > Hi Ankur, > > > > I applied both the patches in the series and I tested them. For now I have > only > > tested E/W scenario. > > > > I have one questions. In my setup, I created 2 bridged logical switches - > sw0 and sw1 and attached > > a router to it. I have 2 chassis - c1 and c2. > > On c1 mac-bindings are configured as - sw0: c1-sw0-MAC, sw1:c1-sw1-MAC > > On c2 mac-bindings are configured as - sw0: c2-sw0-MAC, sw1:c2-sw1-MAC > [ANKUR]: Sorry, a little confused here the chassis mac need not be on per > logical switch, chassis mac should be configured on per physical bridge > basis. > I understand. I used this just as an example. In this particular case, I created localnet ports for both sw0 and sw1 withthe options "network_name=sw0/sw1". > > > > I have a logical port - sw0-port1 (which belongs to sw0 logical switch) > with IP - 10.0.0.4 > > and another logical port - sw1-port1 (which belongs to sw1) with IP - > 20.0.0.4. > > > > sw0-port1 is bound on cha
Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR
Hi Numan, Thank you so much for trying out the patch. Please find my replies inline regarding your test. For other comments in the code, i will be happy to make the change and v6 will have it. Thanks Regards, Ankur From: Numan Siddique Sent: Tuesday, May 7, 2019 10:07 AM To: Ankur Sharma Cc: ovs-dev@openvswitch.org Subject: Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR On Fri, May 3, 2019 at 3:04 AM Ankur Sharma mailto:ankur.sha...@nutanix.com>> wrote: Background: [1] https://mail.openvswitch.org/pipermail/ovs-dev/2018-October/353066.html [mail.openvswitch.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_pipermail_ovs-2Ddev_2018-2DOctober_353066.html&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=OerTfhGsADp01PypwIUM88kogzs3tjwSi5R4eg84yDo&e=> [2] https://docs.google.com/document/d/1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU/edit?usp=sharing [docs.google.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU_edit-3Fusp-3Dsharing&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=H9msg0rIyejuyXKAo8pHjin4JOijHLSAE_kOrph7i-c&s=-9fgFCK15HOs9DwR7cxbMskrngMo6LTzr-yatSFvQmM&e=> This Series: Layer 3 E-W and Layer 3 N-S (NO NAT) changes for vlan backed distributed logical router. This patch: A. Key difference between an overlay logical switch and vlan backed logical switch is that for vlan logical switches packets are not encapsulated. Hence, if a distributed router port is connected to vlan type logical switch, then router port mac as source mac could be seen from multiple hypervisors. Same pairs coming from multiple ports from a top of the rack switch (TOR) perspective could be seen as a security threat and it could send alarms, drop the packets or block the ports etc. This patch addresses the same by introducing the concept of chassis mac. A chassis mac is CMS provisioned unique mac per chassis. For any routed packet (i.e source mac is router port mac) going on the wire on a vlan type logical switch, we will replace its source mac with chassis mac. This replacing of source mac with chassis mac will happen in table=65 of the logical switch datapath. A flow is added at priority 150, which matches the source mac and replaces it with chassis mac if the value is a router port mac. Example flow: cookie=0x0, duration=67765.830s, table=65, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=150,reg15=0x1,metadata=0x4, dl_src=00:00:01:01:02:03 actions=mod_dl_src:aa:bb:cc:dd:ee:ff, mod_vlan_vid:1000,output:16 Here, 00:00:01:01:02:03 is router port mac and aa:bb:cc:dd:ee:ff is chassis mac. B. This patch adds one more change of associating "types" with logical switches. i.e a logical switch could be of type "overlay" or "bridged". This is done to explicitly call out that on a bridged logical switch there will no encapsulation. Just a localnet port's presence is not sufficient, as we do encap while redirecting the packet to gateway chassis. By marking the logical switch as bridged, we can either avoid redirection totally (if there is no NAT) or do redirection based on router port mac, rather than encap over a tunnel. Hi Ankur, I applied both the patches in the series and I tested them. For now I have only tested E/W scenario. I have one questions. In my setup, I created 2 bridged logical switches - sw0 and sw1 and attached a router to it. I have 2 chassis - c1 and c2. On c1 mac-bindings are configured as - sw0: c1-sw0-MAC, sw1:c1-sw1-MAC On c2 mac-bindings are configured as - sw0: c2-sw0-MAC, sw1:c2-sw1-MAC [ANKUR]: Sorry, a little confused here the chassis mac need not be on per logical switch, chassis mac should be configured on per physical bridge basis. I have a logical port - sw0-port1 (which belongs to sw0 logical switch) with IP - 10.0.0.4 and another logical port - sw1-port1 (which belongs to sw1) with IP - 20.0.0.4. sw0-port1 is bound on chassis c1 and sw1-port1 on chassis c2. When sw0-port1 (10.0.0.4) pings sw1-port1 (20.0.0.4), the router port (20.0.0.1) MAC is replaced by c1-sw1-MAC in table 65 as expected. When sw1-port1 receives the packet it will receive the packet as - --- | SRC MAC (c1-sw1-MAC) | SRC IP (10.0.0.4) | DST MAC (sw1-port1 mac) | DST IP (20.0.0.4) | --- when sw1-port1 replies for the ping, it sends the packet as - ---
Re: [ovs-dev] [PATCH v5 1/2] OVN: Enable E-W Traffic, Vlan backed DVR
On Fri, May 3, 2019 at 3:04 AM Ankur Sharma wrote: > Background: > [1] > https://mail.openvswitch.org/pipermail/ovs-dev/2018-October/353066.html > [2] > https://docs.google.com/document/d/1uoQH478wM1OZ16HrxzbOUvk5LvFnfNEWbkPT6Zmm9OU/edit?usp=sharing > > This Series: > Layer 3 E-W and Layer 3 N-S (NO NAT) changes for vlan > backed distributed logical router. > > This patch: > > A. > Key difference between an overlay logical switch and > vlan backed logical switch is that for vlan logical switches > packets are not encapsulated. > > Hence, if a distributed router port is connected to vlan type > logical switch, then router port mac as source mac could be > seen from multiple hypervisors. Same pairs coming > from multiple ports from a top of the rack switch (TOR) perspective > could be seen as a security threat and it could send alarms, drop > the packets or block the ports etc. > > This patch addresses the same by introducing the concept of chassis mac. > A chassis mac is CMS provisioned unique mac per chassis. For any routed > packet > (i.e source mac is router port mac) going on the wire on a vlan type > logical switch, we will replace its source mac with chassis mac. > > This replacing of source mac with chassis mac will happen in table=65 > of the logical switch datapath. A flow is added at priority 150, which > matches the source mac and replaces it with chassis mac if the value > is a router port mac. > > Example flow: > cookie=0x0, duration=67765.830s, table=65, n_packets=0, n_bytes=0, > idle_age=65534, hard_age=65534, priority=150,reg15=0x1,metadata=0x4, > dl_src=00:00:01:01:02:03 actions=mod_dl_src:aa:bb:cc:dd:ee:ff, > mod_vlan_vid:1000,output:16 > > Here, 00:00:01:01:02:03 is router port mac and aa:bb:cc:dd:ee:ff > is chassis mac. > > B. > This patch adds one more change of associating "types" with logical > switches. i.e a logical switch could be of type "overlay" or "bridged". > This is done to explicitly call out that on a bridged logical > switch there will no encapsulation. > Just a localnet port's presence is not sufficient, as we do > encap while redirecting the packet to gateway chassis. > By marking the logical switch as bridged, we can either > avoid redirection totally (if there is no NAT) or do redirection > based on router port mac, rather than encap over a tunnel. > > Hi Ankur, I applied both the patches in the series and I tested them. For now I have only tested E/W scenario. I have one questions. In my setup, I created 2 bridged logical switches - sw0 and sw1 and attached a router to it. I have 2 chassis - c1 and c2. On c1 mac-bindings are configured as - sw0: c1-sw0-MAC, sw1:c1-sw1-MAC On c2 mac-bindings are configured as - sw0: c2-sw0-MAC, sw1:c2-sw1-MAC I have a logical port - sw0-port1 (which belongs to sw0 logical switch) with IP - 10.0.0.4 and another logical port - sw1-port1 (which belongs to sw1) with IP - 20.0.0.4. sw0-port1 is bound on chassis c1 and sw1-port1 on chassis c2. When sw0-port1 (10.0.0.4) pings sw1-port1 (20.0.0.4), the router port (20.0.0.1) MAC is replaced by c1-sw1-MAC in table 65 as expected. When sw1-port1 receives the packet it will receive the packet as - --- | SRC MAC (c1-sw1-MAC) | SRC IP (10.0.0.4) | DST MAC (sw1-port1 mac) | DST IP (20.0.0.4) | --- when sw1-port1 replies for the ping, it sends the packet as - | SRC MAC (sw1-port1 MAC) | SRC IP (20.0.0.4) | DST MAC ( he router port (20.0.0.1) MAC ) | DST IP (10.0.0.4) | I observed for the packets on the sw1-port1 interface on c2. When ovn-controller delivers the packet to sw1-port1, shouldn't it replace the c1-sw1-MAC with the router port (20.0.0.1) MAC ? Or it's fine ? Few comments inline Thanks Numan > Signed-off-by: Ankur Sharma > --- > ovn/controller/binding.c| 12 +-- > ovn/controller/chassis.c| 66 +++- > ovn/controller/chassis.h| 4 + > ovn/controller/ovn-controller.8.xml | 10 ++ > ovn/controller/ovn-controller.c | 2 +- > ovn/controller/ovn-controller.h | 5 +- > ovn/controller/physical.c | 96 ++ > ovn/northd/ovn-northd.c | 38 +++ > ovn/ovn-architecture.7.xml | 12 +++ > ovn/ovn-nb.ovsschema| 10 +- > ovn/ovn-nb.xml | 18 > ovn/ovn-sb.xml | 15 +++ > ovn/utilities/ovn-nbctl.c | 49 +++-- > tests/ovn-nbctl.at