Re: [ovs-discuss] Debugging ct dnat openflow action
Thanks Guru. Yes, I have replaced the upstream kernel with OVS kernel module repo 2.8.1, I mean from [1] which add openvswitch: nat support in linux datapath seems also including below changes and those doesn't included in the OVS kernel module, so I am concerning is it enough for just replace kernel module from OVS repo. include/uapi/linux/netfilter/nf_conntrack_common.h | 12 +- include/uapi/linux/openvswitch.h | 49 ++ net/ipv4/netfilter/nf_nat_l3proto_ipv4.c | 30 +- net/ipv6/netfilter/nf_nat_l3proto_ipv6.c | 30 +- [1] https://www.mail-archive.com/netdev@vger.kernel.org/msg101556.html Cause the NAT doesn't work in my environment, I am trying to debug why, please see previous email, thanks much for your help. Hui. On Tue, Nov 14, 2017 at 4:53 AM, Guru Shettywrote: > > > On 12 November 2017 at 22:43, Hui Xiang wrote: > >> Does ovs linux dapath NAT work with linux kernel 4.4.70 version? >> > > If you use the kernel module that comes with OVS repo, it will work. If > you use the kernel module that comes by default with linux kernel, it > won't. You can look at ovs-vswitchd.log when ovs-vswitchd starts to see a > message of the form: > > 2017-11-13T20:53:01.635Z|00018|ofproto_dpif|INFO|system@ovs-system: > Datapath does not support ct_state_nat > > > >> >> I have seen below comments in the NEWS saying [1] >> " >> - Linux: >> * OVS Linux datapath now implements Conntrack NAT action with all >> supported Linux kernels. >> " >> However, the NAT support for ovs linux datath showed in [2] and >> [3](below) means they are merged since kernel 4.6 >> " >> FeatureLinux upstreamLinux OVS treeUserspaceHyper-V >> NAT 4.6 YES Yes NO >> " >> >> My understanding is that the NAT is only working with a minimal version >> of kernel 4.6? Thanks much for any help. >> >> [1] https://github.com/openvswitch/ovs/blob/master/NEWS >> [2] https://www.mail-archive.com/netdev@vger.kernel.org/msg101556.html >> [3] http://docs.openvswitch.org/en/latest/faq/releases/ >> >> >> Hui. >> >> >> On Fri, Nov 10, 2017 at 6:41 PM, Hui Xiang wrote: >> >>> Hi Folks, >>> >>> >>> I am now debugging OVN NAT with openstack, networking-ovn. now I am >>> blocked at the dnat action step, if anyone can give a help or hint would be >>> really appreciated. >>> >>> VM instance has fixedip 20.0.0.2 and floatingip 172.16.0.131 >>> >>> Below are the lflow-trace, openflow-trace and related openflow table. >>> >>> From lflow-trace, the ip4.dst=172.16.0.131 is expected turn to 20.0.0.2 >>> by ct_dnat, and then when go to next table, the nw_dst will be >>> 20.0.0.0/24, but actually from the openflow-trace after >>> ct_dnat(20.0.0.2), the nw_dst is still 172.16.0.0/24 in the next >>> routing table, does there's something wrong or I miss anything in the ct >>> dnat? it is using the ovs 2.8.1 kernel conntrack, where should I looked? >>> Thanks much. >>> >>> >>> # lflow trace >>> ct_snat /* assuming no un-snat entry, so no change */ >>> - >>> 4. lr_in_dnat (ovn-northd.c:5007): ip && ip4.dst == 172.16.0.131 && >>> inport == "lrp-640d04" && is_chassis_resident("cr-lrp-640d04"), >>> priority 100, uuid 5d67b33f >>> ct_dnat(20.0.0.2); >>> >>> ct_dnat(ip4.dst=20.0.0.2) >>> - >>> 5. lr_in_ip_routing (ovn-northd.c:4140): ip4.dst == 20.0.0.0/24, >>> priority 49, uuid e869d362 >>> ip.ttl--; >>> reg0 = ip4.dst; >>> reg1 = 20.0.0.1; >>> eth.src = fa:16:3e:b5:99:71; >>> outport = "lrp-82f211"; >>> flags.loopback = 1; >>> next; >>> >>> # corresponding openflow trace >>> 12. ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131, priority 100, cookie >>> 0x5d67b33f >>> ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) >>> nat(dst=20.0.0.2) >>> -> A clone of the packet is forked to recirculate. The forked >>> pipeline will be resumed at table 13. >>> >>> Final flow: unchanged >>> Megaflow: recirc_id=0x19,eth,ip,in_port=0,nw_dst=172.16.0.131,nw_frag=no >>> Datapath actions: ct(commit,zone=7,nat(dst=20.0.0.2)),recirc(0x1a) >>> >>> >>> === >>> recirc(0x1a) - resume conntrack with default ct_state=trk|new (use >>> --ct-next to customize) >>> >>> === >>> >>> Flow: recirc_id=0x1a,ct_state=new|trk,eth,icmp,reg11=0x7,reg12=0x3 >>> ,reg14=0x1,metadata=0x3,vlan_tci=0x,dl_src=00:00:00:00:0 >>> 0:00,dl_dst=fa:16:3e:2e:ea:e9,nw_src=172.16.0.2,nw_dst=172. >>> 16.0.131,nw_tos=0,nw_ecn=0,nw_ttl=32,icmp_type=0,icmp_code=0 >>> >>> bridge("br-ex") >>> --- >>> thaw >>> Resuming from table 13 >>> 13. ip,metadata=0x3,nw_dst=172.16.0.0/16, priority 33, cookie 0x9e4db527 >>> dec_ttl() >>> move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127] >>> -> NXM_NX_XXREG0[96..127] is now 0xac100083
Re: [ovs-discuss] Debugging ct dnat openflow action
On 12 November 2017 at 22:43, Hui Xiangwrote: > Does ovs linux dapath NAT work with linux kernel 4.4.70 version? > If you use the kernel module that comes with OVS repo, it will work. If you use the kernel module that comes by default with linux kernel, it won't. You can look at ovs-vswitchd.log when ovs-vswitchd starts to see a message of the form: 2017-11-13T20:53:01.635Z|00018|ofproto_dpif|INFO|system@ovs-system: Datapath does not support ct_state_nat > > I have seen below comments in the NEWS saying [1] > " > - Linux: > * OVS Linux datapath now implements Conntrack NAT action with all > supported Linux kernels. > " > However, the NAT support for ovs linux datath showed in [2] and [3](below) > means they are merged since kernel 4.6 > " > FeatureLinux upstreamLinux OVS treeUserspaceHyper-V > NAT 4.6 YES Yes NO > " > > My understanding is that the NAT is only working with a minimal version of > kernel 4.6? Thanks much for any help. > > [1] https://github.com/openvswitch/ovs/blob/master/NEWS > [2] https://www.mail-archive.com/netdev@vger.kernel.org/msg101556.html > [3] http://docs.openvswitch.org/en/latest/faq/releases/ > > > Hui. > > > On Fri, Nov 10, 2017 at 6:41 PM, Hui Xiang wrote: > >> Hi Folks, >> >> >> I am now debugging OVN NAT with openstack, networking-ovn. now I am >> blocked at the dnat action step, if anyone can give a help or hint would be >> really appreciated. >> >> VM instance has fixedip 20.0.0.2 and floatingip 172.16.0.131 >> >> Below are the lflow-trace, openflow-trace and related openflow table. >> >> From lflow-trace, the ip4.dst=172.16.0.131 is expected turn to 20.0.0.2 >> by ct_dnat, and then when go to next table, the nw_dst will be >> 20.0.0.0/24, but actually from the openflow-trace after >> ct_dnat(20.0.0.2), the nw_dst is still 172.16.0.0/24 in the next routing >> table, does there's something wrong or I miss anything in the ct dnat? it >> is using the ovs 2.8.1 kernel conntrack, where should I looked? Thanks >> much. >> >> >> # lflow trace >> ct_snat /* assuming no un-snat entry, so no change */ >> - >> 4. lr_in_dnat (ovn-northd.c:5007): ip && ip4.dst == 172.16.0.131 && >> inport == "lrp-640d04" && is_chassis_resident("cr-lrp-640d04"), priority >> 100, uuid 5d67b33f >> ct_dnat(20.0.0.2); >> >> ct_dnat(ip4.dst=20.0.0.2) >> - >> 5. lr_in_ip_routing (ovn-northd.c:4140): ip4.dst == 20.0.0.0/24, >> priority 49, uuid e869d362 >> ip.ttl--; >> reg0 = ip4.dst; >> reg1 = 20.0.0.1; >> eth.src = fa:16:3e:b5:99:71; >> outport = "lrp-82f211"; >> flags.loopback = 1; >> next; >> >> # corresponding openflow trace >> 12. ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131, priority 100, cookie >> 0x5d67b33f >> ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) >> nat(dst=20.0.0.2) >> -> A clone of the packet is forked to recirculate. The forked >> pipeline will be resumed at table 13. >> >> Final flow: unchanged >> Megaflow: recirc_id=0x19,eth,ip,in_port=0,nw_dst=172.16.0.131,nw_frag=no >> Datapath actions: ct(commit,zone=7,nat(dst=20.0.0.2)),recirc(0x1a) >> >> >> === >> recirc(0x1a) - resume conntrack with default ct_state=trk|new (use >> --ct-next to customize) >> >> === >> >> Flow: recirc_id=0x1a,ct_state=new|trk,eth,icmp,reg11=0x7,reg12=0x3 >> ,reg14=0x1,metadata=0x3,vlan_tci=0x,dl_src=00:00:00:00: >> 00:00,dl_dst=fa:16:3e:2e:ea:e9,nw_src=172.16.0.2,nw_dst= >> 172.16.0.131,nw_tos=0,nw_ecn=0,nw_ttl=32,icmp_type=0,icmp_code=0 >> >> bridge("br-ex") >> --- >> thaw >> Resuming from table 13 >> 13. ip,metadata=0x3,nw_dst=172.16.0.0/16, priority 33, cookie 0x9e4db527 >> dec_ttl() >> move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127] >> -> NXM_NX_XXREG0[96..127] is now 0xac100083 >> load:0xac100082->NXM_NX_XXREG0[64..95] >> set_field:fa:16:3e:2e:ea:e9->eth_src >> set_field:0x1->reg15 >> load:0x1->NXM_NX_REG10[0] >> resubmit(,14) >> >> >> # openflow table >> cookie=0x5d67b33f, duration=4600.548s, table=12, n_packets=3, >> n_bytes=294, priority=100,ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131 >> actions=ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) >> cookie=0xe869d362, duration=4600.551s, table=13, n_packets=3, >> n_bytes=294, priority=49,ip,metadata=0x3,nw_dst=20.0.0.0/24 >> actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: >> 0x1401->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:b5:99:7 >> 1->eth_src,set_field:0x3->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,14) >> cookie=0x9e4db527, duration=4600.547s, table=13, n_packets=0, n_bytes=0, >> priority=33,ip,metadata=0x3,nw_dst=172.16.0.0/16 >> actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: >>
Re: [ovs-discuss] Debugging ct dnat openflow action
Does ovs linux dapath NAT work with linux kernel 4.4.70 version? I have seen below comments in the NEWS saying [1] " - Linux: * OVS Linux datapath now implements Conntrack NAT action with all supported Linux kernels. " However, the NAT support for ovs linux datath showed in [2] and [3](below) means they are merged since kernel 4.6 " FeatureLinux upstreamLinux OVS treeUserspaceHyper-V NAT 4.6 YES Yes NO " My understanding is that the NAT is only working with a minimal version of kernel 4.6? Thanks much for any help. [1] https://github.com/openvswitch/ovs/blob/master/NEWS [2] https://www.mail-archive.com/netdev@vger.kernel.org/msg101556.html [3] http://docs.openvswitch.org/en/latest/faq/releases/ Hui. On Fri, Nov 10, 2017 at 6:41 PM, Hui Xiangwrote: > Hi Folks, > > > I am now debugging OVN NAT with openstack, networking-ovn. now I am > blocked at the dnat action step, if anyone can give a help or hint would be > really appreciated. > > VM instance has fixedip 20.0.0.2 and floatingip 172.16.0.131 > > Below are the lflow-trace, openflow-trace and related openflow table. > > From lflow-trace, the ip4.dst=172.16.0.131 is expected turn to 20.0.0.2 by > ct_dnat, and then when go to next table, the nw_dst will be 20.0.0.0/24, > but actually from the openflow-trace after ct_dnat(20.0.0.2), the nw_dst is > still 172.16.0.0/24 in the next routing table, does there's something > wrong or I miss anything in the ct dnat? it is using the ovs 2.8.1 kernel > conntrack, where should I looked? Thanks much. > > > # lflow trace > ct_snat /* assuming no un-snat entry, so no change */ > - > 4. lr_in_dnat (ovn-northd.c:5007): ip && ip4.dst == 172.16.0.131 && > inport == "lrp-640d04" && is_chassis_resident("cr-lrp-640d04"), priority > 100, uuid 5d67b33f > ct_dnat(20.0.0.2); > > ct_dnat(ip4.dst=20.0.0.2) > - > 5. lr_in_ip_routing (ovn-northd.c:4140): ip4.dst == 20.0.0.0/24, > priority 49, uuid e869d362 > ip.ttl--; > reg0 = ip4.dst; > reg1 = 20.0.0.1; > eth.src = fa:16:3e:b5:99:71; > outport = "lrp-82f211"; > flags.loopback = 1; > next; > > # corresponding openflow trace > 12. ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131, priority 100, cookie > 0x5d67b33f > ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) > nat(dst=20.0.0.2) > -> A clone of the packet is forked to recirculate. The forked > pipeline will be resumed at table 13. > > Final flow: unchanged > Megaflow: recirc_id=0x19,eth,ip,in_port=0,nw_dst=172.16.0.131,nw_frag=no > Datapath actions: ct(commit,zone=7,nat(dst=20.0.0.2)),recirc(0x1a) > > > === > recirc(0x1a) - resume conntrack with default ct_state=trk|new (use > --ct-next to customize) > > === > > Flow: recirc_id=0x1a,ct_state=new|trk,eth,icmp,reg11=0x7,reg12= > 0x3,reg14=0x1,metadata=0x3,vlan_tci=0x,dl_src=00:00: > 00:00:00:00,dl_dst=fa:16:3e:2e:ea:e9,nw_src=172.16.0.2,nw_ > dst=172.16.0.131,nw_tos=0,nw_ecn=0,nw_ttl=32,icmp_type=0,icmp_code=0 > > bridge("br-ex") > --- > thaw > Resuming from table 13 > 13. ip,metadata=0x3,nw_dst=172.16.0.0/16, priority 33, cookie 0x9e4db527 > dec_ttl() > move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127] > -> NXM_NX_XXREG0[96..127] is now 0xac100083 > load:0xac100082->NXM_NX_XXREG0[64..95] > set_field:fa:16:3e:2e:ea:e9->eth_src > set_field:0x1->reg15 > load:0x1->NXM_NX_REG10[0] > resubmit(,14) > > > # openflow table > cookie=0x5d67b33f, duration=4600.548s, table=12, n_packets=3, > n_bytes=294, priority=100,ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131 > actions=ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) > cookie=0xe869d362, duration=4600.551s, table=13, n_packets=3, > n_bytes=294, priority=49,ip,metadata=0x3,nw_dst=20.0.0.0/24 > actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: > 0x1401->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:b5:99: > 71->eth_src,set_field:0x3->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,14) > cookie=0x9e4db527, duration=4600.547s, table=13, n_packets=0, n_bytes=0, > priority=33,ip,metadata=0x3,nw_dst=172.16.0.0/16 > actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: > 0xac100082->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:2e:ea: > e9->eth_src,set_field:0x1->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,14) > > > Hui. > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Debugging ct dnat openflow action
Hi Folks, I am now debugging OVN NAT with openstack, networking-ovn. now I am blocked at the dnat action step, if anyone can give a help or hint would be really appreciated. VM instance has fixedip 20.0.0.2 and floatingip 172.16.0.131 Below are the lflow-trace, openflow-trace and related openflow table. >From lflow-trace, the ip4.dst=172.16.0.131 is expected turn to 20.0.0.2 by ct_dnat, and then when go to next table, the nw_dst will be 20.0.0.0/24, but actually from the openflow-trace after ct_dnat(20.0.0.2), the nw_dst is still 172.16.0.0/24 in the next routing table, does there's something wrong or I miss anything in the ct dnat? it is using the ovs 2.8.1 kernel conntrack, where should I looked? Thanks much. # lflow trace ct_snat /* assuming no un-snat entry, so no change */ - 4. lr_in_dnat (ovn-northd.c:5007): ip && ip4.dst == 172.16.0.131 && inport == "lrp-640d04" && is_chassis_resident("cr-lrp-640d04"), priority 100, uuid 5d67b33f ct_dnat(20.0.0.2); ct_dnat(ip4.dst=20.0.0.2) - 5. lr_in_ip_routing (ovn-northd.c:4140): ip4.dst == 20.0.0.0/24, priority 49, uuid e869d362 ip.ttl--; reg0 = ip4.dst; reg1 = 20.0.0.1; eth.src = fa:16:3e:b5:99:71; outport = "lrp-82f211"; flags.loopback = 1; next; # corresponding openflow trace 12. ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131, priority 100, cookie 0x5d67b33f ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) nat(dst=20.0.0.2) -> A clone of the packet is forked to recirculate. The forked pipeline will be resumed at table 13. Final flow: unchanged Megaflow: recirc_id=0x19,eth,ip,in_port=0,nw_dst=172.16.0.131,nw_frag=no Datapath actions: ct(commit,zone=7,nat(dst=20.0.0.2)),recirc(0x1a) === recirc(0x1a) - resume conntrack with default ct_state=trk|new (use --ct-next to customize) === Flow: recirc_id=0x1a,ct_state=new|trk,eth,icmp,reg11=0x7,reg12=0x3,reg14=0x1,metadata=0x3,vlan_tci=0x,dl_src=00:00:00:00:00:00,dl_dst=fa:16:3e:2e:ea:e9,nw_src=172.16.0.2,nw_dst=172.16.0.131,nw_tos=0,nw_ecn=0,nw_ttl=32,icmp_type=0,icmp_code=0 bridge("br-ex") --- thaw Resuming from table 13 13. ip,metadata=0x3,nw_dst=172.16.0.0/16, priority 33, cookie 0x9e4db527 dec_ttl() move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127] -> NXM_NX_XXREG0[96..127] is now 0xac100083 load:0xac100082->NXM_NX_XXREG0[64..95] set_field:fa:16:3e:2e:ea:e9->eth_src set_field:0x1->reg15 load:0x1->NXM_NX_REG10[0] resubmit(,14) # openflow table cookie=0x5d67b33f, duration=4600.548s, table=12, n_packets=3, n_bytes=294, priority=100,ip,reg14=0x1,metadata=0x3,nw_dst=172.16.0.131 actions=ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=20.0.0.2)) cookie=0xe869d362, duration=4600.551s, table=13, n_packets=3, n_bytes=294, priority=49,ip,metadata=0x3,nw_dst=20.0.0.0/24 actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: 0x1401->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:b5:99:71->eth_src,set_field:0x3->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,14) cookie=0x9e4db527, duration=4600.547s, table=13, n_packets=0, n_bytes=0, priority=33,ip,metadata=0x3,nw_dst=172.16.0.0/16 actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127],load: 0xac100082->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:2e:ea:e9->eth_src,set_field:0x1->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,14) Hui. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss