Re: [ovs-dev] [PATCH v3] Avoid tunneling for VLAN packets redirected to a gateway chassis
On Fri, Jun 1, 2018 at 12:13 AM, Anil Venkata wrote: > Thanks Russell. > > On Thu, May 31, 2018 at 1:41 AM, Russell Bryant wrote: > >> One more general question: >> >> a major difference when doing the redirect to the gateway via a VLAN >> vs a geneve tunnel is the lack of metadata. You've demonstrated how >> it's easy enough to identify the network (the VLAN ID + the port it >> arrived on). How about the logical input / output IDs? What values >> are included when the packet is sent over the tunnel? Are we >> confident those values are not needed, or can be inferred another way >> in this scenario? >> > > When the packet is sent over the geneve tunnel, geneve tunnel key consists > of > below values > 1) router patch port connected to tenant vlan network as logical input port > 2) redirect chassis port as logical output port > 2) router datapath as logical datapath > When gateway chassis receives packet from tunnel, it ignores logical input > port > and directly sends it to table 33 (which sets output to distributed > gateway port) > for egress processing. > > When gateway chassis receives vlan tagged packet, as we don't have logical > input > & output details, this patch sets - > "router patch port connected to tenant vlan network" as logical input > port and > sends the packet through router ingress pipeline to determine logical > output port. > > If we check destination MAC along with VLAN ID in table 0 on gateway > chassis, > as suggested by Miguel and Jakub, and if the destination MAC is for > distributed > gateway port, then we can set output port to distributed gateway port in > table 0 > and directly pass packet to router egress pipeline(skipping router ingress > pipeline). > > >> On Wed, May 30, 2018 at 3:59 PM, Russell Bryant wrote: >> > On Fri, May 25, 2018 at 7:33 AM, wrote: >> >> From: venkata anil >> >> >> >> When a vm on a vlan tenant network sends traffic to an external >> network, >> >> it is tunneled from host chassis to gateway chassis. In the earlier >> >> discussion [1], Russel (also in his doc [2]) suggested if we can figure >> >> out a way for OVN to do this redirect to the gateway host over a VLAN >> >> network. This patch implements his suggestion i.e will redirect to >> >> gateway chassis using incoming tenant vlan network. Gateway chassis are >> >> expected to be configured with tenant vlan networks. In this approach, >> >> new logical and physical flows introduced for packet processing in both >> >> host and gateway chassis. >> > >> > I don't think we can impose the expectation that the gateway is on the >> > same vlan network as the original compute node. The previous behavior >> > of using the tunnel does not require that. >> > >> > > > I thought we can impose this expectation as in Openstack DVR environment, > to use centralized SNAT for DVR routers, centralized router(on network > node) > will have connectivity to same tenant vlan network which compute node has > connected. Indeed a new port is created on each tenant network and these > ports > are added to centralized SNAT router namespace. VM will send packet to > these > ports on centralized SNAT router though this tenant vlan network. > > Without this patch, the existing code tunnels the packet to gateway chassis. But the return path i.e gateway chassis to compute chassis is always through localnet port. So bridge mappings on gateway chassis is must for existing implementation as well. We can still implement falling back to tunnelling(for compute to gateway chassis communication) when gateway chassis is not having bridge mappings. > > Have you thought of whether we could use the new behavior >> > automatically if we know both chassis are on the same network, or fall >> > back to a tunnel if necessary? >> > >> > > > No, I didn't think about that. Thanks for the suggestion. > If we couldn't impose the above expectation, then we can implement your > suggestion > about falling back to tunnelling. > > >> >> >> >> Packet processing in the host chassis: >> >> 1) A new ovs flow added in physical table 65, which sets >> MLF_RCV_FROM_VLAN >> >>flag for packets from vlan network entering into router pipeline >> >> 2) A new flow added in lr_in_ip_routing, for packets output through >> >>distributed gateway port and matching MLF_RCV_FROM_VLAN flag, >> >>set REGBIT_NAT_REDIRECT i.e >> >>table=7 (lr_in_ip_routing ), priority=2, match=( >> >>ip4.dst == 0.0.0.0/0 && flags.rcv_from_vlan == 1 && >> >>!is_chassis_resident("cr-alice")), action=(reg9[0] = 1; next;) >> >>This flow will be set only on chassis not hosting chassisredirect >> >>port i.e compute node. >> >>When REGBIT_NAT_REDIRECT set, >> >>a) lr_in_arp_resolve, will set packet eth.dst to distibuted gateway >> >> port MAC >> >>b) lr_in_gw_redirect, will set chassisredirect port as outport >> >> 3) A new ovs flow added in physical table 32 will use source vlan >> tenant >> >>network tag as vlan ID
Re: [ovs-dev] [PATCH v3] Avoid tunneling for VLAN packets redirected to a gateway chassis
Thanks Russell. On Thu, May 31, 2018 at 1:41 AM, Russell Bryant wrote: > One more general question: > > a major difference when doing the redirect to the gateway via a VLAN > vs a geneve tunnel is the lack of metadata. You've demonstrated how > it's easy enough to identify the network (the VLAN ID + the port it > arrived on). How about the logical input / output IDs? What values > are included when the packet is sent over the tunnel? Are we > confident those values are not needed, or can be inferred another way > in this scenario? > When the packet is sent over the geneve tunnel, geneve tunnel key consists of below values 1) router patch port connected to tenant vlan network as logical input port 2) redirect chassis port as logical output port 2) router datapath as logical datapath When gateway chassis receives packet from tunnel, it ignores logical input port and directly sends it to table 33 (which sets output to distributed gateway port) for egress processing. When gateway chassis receives vlan tagged packet, as we don't have logical input & output details, this patch sets - "router patch port connected to tenant vlan network" as logical input port and sends the packet through router ingress pipeline to determine logical output port. If we check destination MAC along with VLAN ID in table 0 on gateway chassis, as suggested by Miguel and Jakub, and if the destination MAC is for distributed gateway port, then we can set output port to distributed gateway port in table 0 and directly pass packet to router egress pipeline(skipping router ingress pipeline). > On Wed, May 30, 2018 at 3:59 PM, Russell Bryant wrote: > > On Fri, May 25, 2018 at 7:33 AM, wrote: > >> From: venkata anil > >> > >> When a vm on a vlan tenant network sends traffic to an external network, > >> it is tunneled from host chassis to gateway chassis. In the earlier > >> discussion [1], Russel (also in his doc [2]) suggested if we can figure > >> out a way for OVN to do this redirect to the gateway host over a VLAN > >> network. This patch implements his suggestion i.e will redirect to > >> gateway chassis using incoming tenant vlan network. Gateway chassis are > >> expected to be configured with tenant vlan networks. In this approach, > >> new logical and physical flows introduced for packet processing in both > >> host and gateway chassis. > > > > I don't think we can impose the expectation that the gateway is on the > > same vlan network as the original compute node. The previous behavior > > of using the tunnel does not require that. > > > I thought we can impose this expectation as in Openstack DVR environment, to use centralized SNAT for DVR routers, centralized router(on network node) will have connectivity to same tenant vlan network which compute node has connected. Indeed a new port is created on each tenant network and these ports are added to centralized SNAT router namespace. VM will send packet to these ports on centralized SNAT router though this tenant vlan network. > > Have you thought of whether we could use the new behavior > > automatically if we know both chassis are on the same network, or fall > > back to a tunnel if necessary? > > > No, I didn't think about that. Thanks for the suggestion. If we couldn't impose the above expectation, then we can implement your suggestion about falling back to tunnelling. > >> > >> Packet processing in the host chassis: > >> 1) A new ovs flow added in physical table 65, which sets > MLF_RCV_FROM_VLAN > >>flag for packets from vlan network entering into router pipeline > >> 2) A new flow added in lr_in_ip_routing, for packets output through > >>distributed gateway port and matching MLF_RCV_FROM_VLAN flag, > >>set REGBIT_NAT_REDIRECT i.e > >>table=7 (lr_in_ip_routing ), priority=2, match=( > >>ip4.dst == 0.0.0.0/0 && flags.rcv_from_vlan == 1 && > >>!is_chassis_resident("cr-alice")), action=(reg9[0] = 1; next;) > >>This flow will be set only on chassis not hosting chassisredirect > >>port i.e compute node. > >>When REGBIT_NAT_REDIRECT set, > >>a) lr_in_arp_resolve, will set packet eth.dst to distibuted gateway > >> port MAC > >>b) lr_in_gw_redirect, will set chassisredirect port as outport > >> 3) A new ovs flow added in physical table 32 will use source vlan tenant > >>network tag as vlan ID for sending the packet to gateway chassis. > >>As this vlan packet destination MAC is distibuted gateway port MAC, > >>packet will only reach the gateway chassis. > >>table=32,priority=150,reg14=0x3,reg15=0x6,metadata=0x4 > >>actions=mod_vlan_vid:2010,output:25,strip_vlan > >>This flow will be set only on chassis not hosting chassisredirect > >>port i.e compute node. > >> > >> Packet processing in the gateway chassis: > >> 1) A new ovs flow added in physical table 0 to pass vlan traffic coming > >>from localnet port to the connected router pipeline(i.e router > >>attached to
Re: [ovs-dev] [PATCH v3] Avoid tunneling for VLAN packets redirected to a gateway chassis
One more general question: a major difference when doing the redirect to the gateway via a VLAN vs a geneve tunnel is the lack of metadata. You've demonstrated how it's easy enough to identify the network (the VLAN ID + the port it arrived on). How about the logical input / output IDs? What values are included when the packet is sent over the tunnel? Are we confident those values are not needed, or can be inferred another way in this scenario? On Wed, May 30, 2018 at 3:59 PM, Russell Bryant wrote: > On Fri, May 25, 2018 at 7:33 AM, wrote: >> From: venkata anil >> >> When a vm on a vlan tenant network sends traffic to an external network, >> it is tunneled from host chassis to gateway chassis. In the earlier >> discussion [1], Russel (also in his doc [2]) suggested if we can figure >> out a way for OVN to do this redirect to the gateway host over a VLAN >> network. This patch implements his suggestion i.e will redirect to >> gateway chassis using incoming tenant vlan network. Gateway chassis are >> expected to be configured with tenant vlan networks. In this approach, >> new logical and physical flows introduced for packet processing in both >> host and gateway chassis. > > I don't think we can impose the expectation that the gateway is on the > same vlan network as the original compute node. The previous behavior > of using the tunnel does not require that. > > Have you thought of whether we could use the new behavior > automatically if we know both chassis are on the same network, or fall > back to a tunnel if necessary? > >> >> Packet processing in the host chassis: >> 1) A new ovs flow added in physical table 65, which sets MLF_RCV_FROM_VLAN >>flag for packets from vlan network entering into router pipeline >> 2) A new flow added in lr_in_ip_routing, for packets output through >>distributed gateway port and matching MLF_RCV_FROM_VLAN flag, >>set REGBIT_NAT_REDIRECT i.e >>table=7 (lr_in_ip_routing ), priority=2, match=( >>ip4.dst == 0.0.0.0/0 && flags.rcv_from_vlan == 1 && >>!is_chassis_resident("cr-alice")), action=(reg9[0] = 1; next;) >>This flow will be set only on chassis not hosting chassisredirect >>port i.e compute node. >>When REGBIT_NAT_REDIRECT set, >>a) lr_in_arp_resolve, will set packet eth.dst to distibuted gateway >> port MAC >>b) lr_in_gw_redirect, will set chassisredirect port as outport >> 3) A new ovs flow added in physical table 32 will use source vlan tenant >>network tag as vlan ID for sending the packet to gateway chassis. >>As this vlan packet destination MAC is distibuted gateway port MAC, >>packet will only reach the gateway chassis. >>table=32,priority=150,reg14=0x3,reg15=0x6,metadata=0x4 >>actions=mod_vlan_vid:2010,output:25,strip_vlan >>This flow will be set only on chassis not hosting chassisredirect >>port i.e compute node. >> >> Packet processing in the gateway chassis: >> 1) A new ovs flow added in physical table 0 to pass vlan traffic coming >>from localnet port to the connected router pipeline(i.e router >>attached to vlan tenant network). >>This flow will set router metadata, reg14 to router's patch port(lrp) >>(i.e patch port connecting router and vlan tenant network) and a new >>MLF_RCV_FROM_VLAN flag. >>table=0,priority=150,in_port=67,dl_vlan=2010 actions=strip_vlan, >>load:0x4->OXM_OF_METADATA[],load:0x3->NXM_NX_REG14[], >>load:0x1->NXM_NX_REG10[5],resubmit(,8) >>This flow will be set only on chassis hosting chassisredirect >>port i.e gateway node. >> 2) A new flow added in lr_in_admission which checks MLF_RCV_FROM_VLAN >>and allows the packet. This flow will be set only on chassis hosting >>chassisredirect port i.e gateway node. >>table=0 (lr_in_admission), priority=100 , match=( >>flags.rcv_from_vlan == 1 && inport == "lrp-44383893-613a-4bfe-b483- >>e7d0dc3055cd" && is_chassis_resident("cr-lrp-a6e3d2ab-313a-4ea3- >>8ec4-c3c774a11f49")), action=(next;) >>Then packet will pass through router ingress and egress pipelines and >>then to external switch pipeline. >> >> In a scenario where the traffic between two vms in the same tenant vlan >> network across different chassis i.e if "vm1" on tenant vlan network >> "net1" is on host chassis "ch1" and "vm2" on same tenant vlan network >> "net1" is on gateway chassis "gw1". When the packet arrived on "gw1" >> chassis from localnet port, we still send it to router pipeline and router >> pipeline will send it to destination switch ("net1") pipeline. > > Why is this? Wouldn't the packet just have a destination MAC for "vm2"? > >> But in this case when packet arrives at "vm2", it will have router MAC as >> source MAC as the packet is routed in gateway chassis. This bevaviour can >> be seen only for destination vms hosted on gateway node. >> >> [1] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046557.html >> [2] Point 3 in section 3.3.1 -
Re: [ovs-dev] [PATCH v3] Avoid tunneling for VLAN packets redirected to a gateway chassis
On Fri, May 25, 2018 at 7:33 AM, wrote: > From: venkata anil > > When a vm on a vlan tenant network sends traffic to an external network, > it is tunneled from host chassis to gateway chassis. In the earlier > discussion [1], Russel (also in his doc [2]) suggested if we can figure > out a way for OVN to do this redirect to the gateway host over a VLAN > network. This patch implements his suggestion i.e will redirect to > gateway chassis using incoming tenant vlan network. Gateway chassis are > expected to be configured with tenant vlan networks. In this approach, > new logical and physical flows introduced for packet processing in both > host and gateway chassis. I don't think we can impose the expectation that the gateway is on the same vlan network as the original compute node. The previous behavior of using the tunnel does not require that. Have you thought of whether we could use the new behavior automatically if we know both chassis are on the same network, or fall back to a tunnel if necessary? > > Packet processing in the host chassis: > 1) A new ovs flow added in physical table 65, which sets MLF_RCV_FROM_VLAN >flag for packets from vlan network entering into router pipeline > 2) A new flow added in lr_in_ip_routing, for packets output through >distributed gateway port and matching MLF_RCV_FROM_VLAN flag, >set REGBIT_NAT_REDIRECT i.e >table=7 (lr_in_ip_routing ), priority=2, match=( >ip4.dst == 0.0.0.0/0 && flags.rcv_from_vlan == 1 && >!is_chassis_resident("cr-alice")), action=(reg9[0] = 1; next;) >This flow will be set only on chassis not hosting chassisredirect >port i.e compute node. >When REGBIT_NAT_REDIRECT set, >a) lr_in_arp_resolve, will set packet eth.dst to distibuted gateway > port MAC >b) lr_in_gw_redirect, will set chassisredirect port as outport > 3) A new ovs flow added in physical table 32 will use source vlan tenant >network tag as vlan ID for sending the packet to gateway chassis. >As this vlan packet destination MAC is distibuted gateway port MAC, >packet will only reach the gateway chassis. >table=32,priority=150,reg14=0x3,reg15=0x6,metadata=0x4 >actions=mod_vlan_vid:2010,output:25,strip_vlan >This flow will be set only on chassis not hosting chassisredirect >port i.e compute node. > > Packet processing in the gateway chassis: > 1) A new ovs flow added in physical table 0 to pass vlan traffic coming >from localnet port to the connected router pipeline(i.e router >attached to vlan tenant network). >This flow will set router metadata, reg14 to router's patch port(lrp) >(i.e patch port connecting router and vlan tenant network) and a new >MLF_RCV_FROM_VLAN flag. >table=0,priority=150,in_port=67,dl_vlan=2010 actions=strip_vlan, >load:0x4->OXM_OF_METADATA[],load:0x3->NXM_NX_REG14[], >load:0x1->NXM_NX_REG10[5],resubmit(,8) >This flow will be set only on chassis hosting chassisredirect >port i.e gateway node. > 2) A new flow added in lr_in_admission which checks MLF_RCV_FROM_VLAN >and allows the packet. This flow will be set only on chassis hosting >chassisredirect port i.e gateway node. >table=0 (lr_in_admission), priority=100 , match=( >flags.rcv_from_vlan == 1 && inport == "lrp-44383893-613a-4bfe-b483- >e7d0dc3055cd" && is_chassis_resident("cr-lrp-a6e3d2ab-313a-4ea3- >8ec4-c3c774a11f49")), action=(next;) >Then packet will pass through router ingress and egress pipelines and >then to external switch pipeline. > > In a scenario where the traffic between two vms in the same tenant vlan > network across different chassis i.e if "vm1" on tenant vlan network > "net1" is on host chassis "ch1" and "vm2" on same tenant vlan network > "net1" is on gateway chassis "gw1". When the packet arrived on "gw1" > chassis from localnet port, we still send it to router pipeline and router > pipeline will send it to destination switch ("net1") pipeline. Why is this? Wouldn't the packet just have a destination MAC for "vm2"? > But in this case when packet arrives at "vm2", it will have router MAC as > source MAC as the packet is routed in gateway chassis. This bevaviour can > be seen only for destination vms hosted on gateway node. > > [1] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046557.html > [2] Point 3 in section 3.3.1 - Future Enhancements > https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflHACp8WSRnKCFSgg/edit# > > Reported-at: > https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046543.html > > Signed-off-by: Venkata Anil > --- > ovn/controller/bfd.c| 3 +- > ovn/controller/binding.c| 10 +- > ovn/controller/ovn-controller.c | 3 + > ovn/controller/ovn-controller.h | 16 ++- > ovn/controller/physical.c | 94 - > ovn/lib/logical-fields.c| 4 + > ovn/lib/logical-fields.h| 2 + > ovn/northd/ovn-northd.c | 35
[ovs-dev] [PATCH v3] Avoid tunneling for VLAN packets redirected to a gateway chassis
From: venkata anilWhen a vm on a vlan tenant network sends traffic to an external network, it is tunneled from host chassis to gateway chassis. In the earlier discussion [1], Russel (also in his doc [2]) suggested if we can figure out a way for OVN to do this redirect to the gateway host over a VLAN network. This patch implements his suggestion i.e will redirect to gateway chassis using incoming tenant vlan network. Gateway chassis are expected to be configured with tenant vlan networks. In this approach, new logical and physical flows introduced for packet processing in both host and gateway chassis. Packet processing in the host chassis: 1) A new ovs flow added in physical table 65, which sets MLF_RCV_FROM_VLAN flag for packets from vlan network entering into router pipeline 2) A new flow added in lr_in_ip_routing, for packets output through distributed gateway port and matching MLF_RCV_FROM_VLAN flag, set REGBIT_NAT_REDIRECT i.e table=7 (lr_in_ip_routing ), priority=2, match=( ip4.dst == 0.0.0.0/0 && flags.rcv_from_vlan == 1 && !is_chassis_resident("cr-alice")), action=(reg9[0] = 1; next;) This flow will be set only on chassis not hosting chassisredirect port i.e compute node. When REGBIT_NAT_REDIRECT set, a) lr_in_arp_resolve, will set packet eth.dst to distibuted gateway port MAC b) lr_in_gw_redirect, will set chassisredirect port as outport 3) A new ovs flow added in physical table 32 will use source vlan tenant network tag as vlan ID for sending the packet to gateway chassis. As this vlan packet destination MAC is distibuted gateway port MAC, packet will only reach the gateway chassis. table=32,priority=150,reg14=0x3,reg15=0x6,metadata=0x4 actions=mod_vlan_vid:2010,output:25,strip_vlan This flow will be set only on chassis not hosting chassisredirect port i.e compute node. Packet processing in the gateway chassis: 1) A new ovs flow added in physical table 0 to pass vlan traffic coming from localnet port to the connected router pipeline(i.e router attached to vlan tenant network). This flow will set router metadata, reg14 to router's patch port(lrp) (i.e patch port connecting router and vlan tenant network) and a new MLF_RCV_FROM_VLAN flag. table=0,priority=150,in_port=67,dl_vlan=2010 actions=strip_vlan, load:0x4->OXM_OF_METADATA[],load:0x3->NXM_NX_REG14[], load:0x1->NXM_NX_REG10[5],resubmit(,8) This flow will be set only on chassis hosting chassisredirect port i.e gateway node. 2) A new flow added in lr_in_admission which checks MLF_RCV_FROM_VLAN and allows the packet. This flow will be set only on chassis hosting chassisredirect port i.e gateway node. table=0 (lr_in_admission), priority=100 , match=( flags.rcv_from_vlan == 1 && inport == "lrp-44383893-613a-4bfe-b483- e7d0dc3055cd" && is_chassis_resident("cr-lrp-a6e3d2ab-313a-4ea3- 8ec4-c3c774a11f49")), action=(next;) Then packet will pass through router ingress and egress pipelines and then to external switch pipeline. In a scenario where the traffic between two vms in the same tenant vlan network across different chassis i.e if "vm1" on tenant vlan network "net1" is on host chassis "ch1" and "vm2" on same tenant vlan network "net1" is on gateway chassis "gw1". When the packet arrived on "gw1" chassis from localnet port, we still send it to router pipeline and router pipeline will send it to destination switch ("net1") pipeline. But in this case when packet arrives at "vm2", it will have router MAC as source MAC as the packet is routed in gateway chassis. This bevaviour can be seen only for destination vms hosted on gateway node. [1] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046557.html [2] Point 3 in section 3.3.1 - Future Enhancements https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflHACp8WSRnKCFSgg/edit# Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046543.html Signed-off-by: Venkata Anil --- ovn/controller/bfd.c| 3 +- ovn/controller/binding.c| 10 +- ovn/controller/ovn-controller.c | 3 + ovn/controller/ovn-controller.h | 16 ++- ovn/controller/physical.c | 94 - ovn/lib/logical-fields.c| 4 + ovn/lib/logical-fields.h| 2 + ovn/northd/ovn-northd.c | 35 +++ tests/ovn.at| 227 9 files changed, 390 insertions(+), 4 deletions(-) diff --git a/ovn/controller/bfd.c b/ovn/controller/bfd.c index 8f020d5..cbbd3ba 100644 --- a/ovn/controller/bfd.c +++ b/ovn/controller/bfd.c @@ -139,8 +139,9 @@ bfd_travel_gw_related_chassis(struct local_datapath *dp, struct local_datapath_node, node); dp = dp_binding->dp; free(dp_binding); +const struct sbrec_datapath_binding *pdp; for (size_t i = 0; i < dp->n_peer_dps;