"dev" <dev-boun...@openvswitch.org> wrote on 07/19/2016 10:44:27 PM:
> From: Chen Li <lichen.hangz...@gmail.com> > To: dev@openvswitch.org > Date: 07/19/2016 10:44 PM > Subject: [ovs-dev] Issue when using ovn with Openstack > Sent by: "dev" <dev-boun...@openvswitch.org> > > Hi list, > > I have an all-in-one devstack environment with ovn enabled. > I create a neutron network. > Create a port A from the network with secgroup A > Create a vm from the network with secgroup B. > Secgroup B has both ICMP and tcp 22 enabled. > > Then I try to ping the VM from the dhcp namespace, since the Secgroup B has > enabled ICMP, I suppose this should work. But, unfortunately, this do not > work. And, the ssh failed too. > > Anyone can help me to solve this issue ? > > I did some basic checks and looks like flows are missing in table 52. > > Here are flows in table 52: > > sudo ovs-ofctl dump-flows br-int |grep table=52 > > cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0, > idle_age=7766, priority=65535,icmp6,metadata=0x4,icmp_type=135,icmp_code=0 > actions=resubmit(,53) > cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0, > idle_age=7766, priority=65535,icmp6,metadata=0x4,icmp_type=136,icmp_code=0 > actions=resubmit(,53) > cookie=0x0, duration=7766.195s, table=52, n_packets=4, n_bytes=1474, > idle_age=7744, priority=2002,udp,reg15=0x2,metadata=0x4,nw_src= > 192.168.0.0/24,tp_src=67,tp_dst=68 > actions=load:0x1->NXM_NX_REG0[1],resubmit(,53) > cookie=0x0, duration=7557.209s, table=52, n_packets=2, n_bytes=759, > idle_age=7548, priority=2002,udp,reg15=0x3,metadata=0x4,nw_src= > 192.168.0.0/24,tp_src=67,tp_dst=68 > actions=load:0x1->NXM_NX_REG0[1],resubmit(,53) > cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0, > idle_age=7766, priority=2001,ipv6,reg15=0x2,metadata=0x4 actions=drop > cookie=0x0, duration=7766.195s, table=52, n_packets=2, n_bytes=676, > idle_age=7548, priority=2001,ip,reg15=0x2,metadata=0x4 actions=drop > cookie=0x0, duration=7557.209s, table=52, n_packets=0, n_bytes=0, > idle_age=7557, priority=2001,ipv6,reg15=0x3,metadata=0x4 actions=drop > cookie=0x0, duration=7557.209s, table=52, n_packets=3979, n_bytes=389774, > idle_age=413, priority=2001,ip,reg15=0x3,metadata=0x4 actions=drop > cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0, > idle_age=7766, priority=1,ipv6,metadata=0x4 > actions=load:0x1->NXM_NX_REG0[1],resubmit(,53) > cookie=0x0, duration=7766.195s, table=52, n_packets=8, n_bytes=2733, > idle_age=7548, priority=1,ip,metadata=0x4 > actions=load:0x1->NXM_NX_REG0[1],resubmit(,53) > cookie=0x0, duration=7926.354s, table=52, n_packets=0, n_bytes=0, > idle_age=7926, priority=0,metadata=0x1 actions=resubmit(,53) > cookie=0x0, duration=7790.771s, table=52, n_packets=129, n_bytes=5418, > idle_age=408, priority=0,metadata=0x4 actions=resubmit(,53) > > Here are steps how I find flows are missing in table52: > > ovs-dpctl show > > port 0: ovs-system (internal) > port 1: br-int (internal) > port 2: tap446ef382-f0 (internal) > port 3: tapc7c9f581-2d (internal) => the dhcp port for the testing > network > port 4: o-hm0 (internal) => the port create from the > testing network with security group A > port 5: tap275a5a25-79 => the port for the vm in the > testing network with security group B > > > sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 arp -n > > Address HWtype HWaddress Flags Mask > Iface > 192.168.0.6 ether fa:16:3e:40:85:41 C > tapc7c9f581-2d > 192.168.0.12 ether fa:16:3e:5c:fe:86 C > tapc7c9f581-2d > > > sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 ping > 192.168.0.12 => This is the IP for the VM. > PING 192.168.0.12 (192.168.0.12) 56(84) bytes of data. > > > ovs-dpctl dump-flows > > recirc_id(0),in_port(3),eth(src=fa:16:3e:b6:f6:25,dst=fa:16:3e: > 5c:fe:86),eth_type(0x0806),arp(sip=192.168.0.2,tip=192.168.0. > 12,op=1/0xff,sha=fa:16:3e:b6:f6:25,tha=00:00:00:00:00:00), > packets:0, bytes:0, used:never, > actions:userspace(pid=4294958325,slow_path(action)) > recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00: > 00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src= > 192.168.0.0/255.255.255.0,proto=1,frag=no), packets:14, bytes:1372, > used:0.974s, actions:drop > > > sudo ovs-appctl ofproto/trace > "recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00: > 00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src= > 192.168.0.0/255.255.255.0,proto=1,frag=no)" > => This produce a long output, and here are the end of the output: > > OpenFlow actions=resubmit(,52) > > Resubmitted flow: unchanged > > Resubmitted regs: reg0=0x1 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 > reg6=0x0 reg7=0x0 reg8=0x0 reg9=0x0 reg10=0x0 reg11=0x0 reg12=0x0 > reg13=0x0 reg14=0x1 reg15=0x3 > > Resubmitted odp: drop > > Resubmitted megaflow: > recirc_id=0,icmp,reg0=0,reg1=0,reg2=0,reg3=0,reg4=0,reg5=0,reg6=0,reg7=0,reg8=0,reg9=0,reg14=0,reg15=0,metadata=0,in_port=4,vlan_tci=0x0000/ > 0x1000 > ,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src= > 192.168.0.0/24,nw_frag=no > > Rule: table=52 cookie=0 priority=2001,ip,reg15=0x3,metadata=0x4 > > OpenFlow actions=drop > Final flow: > icmp,reg0=0x1,reg14=0x1,reg15=0x3,metadata=0x4,in_port=4,vlan_tci=0x0000,dl_src=00: > 00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=192.168.0.0,nw_d > st=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,icmp_type=0,icmp_code=0 > Megaflow: > recirc_id=0,icmp,in_port=4,vlan_tci=0x0000/0x1000,dl_src=00:00:00: > 00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src= > 192.168.0.0/24,nw_fra > g=no > Datapath actions: drop > > > Here are some output from OVN commands: > > sudo ovn-nbctl show > > switch 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b > (neutron-e8586b01-6441-4c3d-a90d-91bb0a54ec80) > port 2c713237-ffc7-4ff1-9e4c-95c1337545e6 > addresses: ["fa:16:3e:40:85:41 192.168.0.6"] > port c7c9f581-2db9-4b06-86c6-bde2d1aa8ffb > addresses: ["fa:16:3e:b6:f6:25 192.168.0.2"] > port 275a5a25-794f-47b9-9b04-8a8da053c143 > addresses: ["fa:16:3e:5c:fe:86 192.168.0.12"] > > > ovn-nbctl acl-list 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b > > from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4) > allow-related > from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 > && (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp && > udp.src == 68 && udp.dst == 67) allow > from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip6) > allow-related > from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4) > allow-related > from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 > && (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp && > udp.src == 68 && udp.dst == 67) allow > from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip6) > allow-related > from-lport 1001 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip) > drop > from-lport 1001 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip) > drop > to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 > && icmp4) allow-related > to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 > && ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68) allow > to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 > && tcp && tcp.dst == 22) allow-related > to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 > && tcp && tcp.dst == 9443) allow-related > to-lport 1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 > && ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68) allow > to-lport 1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 > && udp && udp.dst == 5555) allow-related > to-lport 1001 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip) > drop > to-lport 1001 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip) > drop > > > ovn-sbctl lflow-list | grep ls_out_acl > > table=4 (ls_out_acl ), priority=0 , match=(1), action= (next;) > table=4 (ls_out_acl ), priority=0 , match=(1), action= (next;) > table=4 (ls_out_acl ), priority=65535, match=(!ct.est && ct.rel > && !ct.new && !ct.inv), action=(next;) > table=4 (ls_out_acl ), priority=65535, match=(ct.est && !ct.rel > && !ct.new && !ct.inv), action=(next;) > table=4 (ls_out_acl ), priority=65535, match=(ct.inv), > action=(drop;) > table=4 (ls_out_acl ), priority=65535, match=(nd), action= (next;) > table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport > == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && icmp4)), > action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport > == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst == 22)), > action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport > == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst == > 9443)), action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport > == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && udp && udp.dst == > 5555)), action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2002 , match=(outport == > "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && ip4.src == 192.168.0.0/24 > && udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2002 , match=(outport == > "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && ip4.src == 192.168.0.0/24 > && udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=2001 , match=(outport == > "275a5a25-794f-47b9-9b04-8a8da053c143" && ip), action=(drop;) > table=4 (ls_out_acl ), priority=2001 , match=(outport == > "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip), action=(drop;) > table=4 (ls_out_acl ), priority=1 , match=(ip), > action=(reg0[1] = 1; next;) > table=4 (ls_out_acl ), priority=0 , match=(1), action= (next;) > > > The last commit in my ovs code: > > commit 7efb1e09bb06270248d29c787978593b57101d4f > Author: Pravin B Shelar <pshe...@ovn.org> > Date: Sun Jul 17 19:24:07 2016 -0700 > > datapath: Add support for kernel 4.5 > > Signed-off-by: Pravin B Shelar <pshe...@ovn.org> > Acked-by: Jesse Gross <je...@kernel.org> > > > Here are some detailed information from openstack: > > 1. The vm: > > nova show test01 > +-------------------------------------- > +----------------------------------------------------------------+ > | Property | > Value | > +-------------------------------------- > +----------------------------------------------------------------+ > | OS-DCF:diskConfig | > MANUAL | > | OS-EXT-AZ:availability_zone | > nova | > | OS-EXT-SRV-ATTR:host | LB-dev-chenli | > | OS-EXT-SRV-ATTR:hostname | > test01 | > | OS-EXT-SRV-ATTR:hypervisor_hostname | LB-dev-chenli | > | OS-EXT-SRV-ATTR:instance_name | > instance-00000001 | > | OS-EXT-SRV-ATTR:kernel_id | > 261ca209-430e-4b8f-ac39-0e397df30a46 | > | OS-EXT-SRV-ATTR:launch_index | > 0 | > | OS-EXT-SRV-ATTR:ramdisk_id | > 04b6a65d-3cff-4eaf-b30b-582caa2379d7 | > | OS-EXT-SRV-ATTR:reservation_id | > r-wehkr5gi | > | OS-EXT-SRV-ATTR:root_device_name | > /dev/vda | > | OS-EXT-SRV-ATTR:user_data | > - | > | OS-EXT-STS:power_state | > 1 | > | OS-EXT-STS:task_state | > - | > | OS-EXT-STS:vm_state | > active | > | OS-SRV-USG:launched_at | > 2016-07-20T01:18:48.000000 | > | OS-SRV-USG:terminated_at | > - | > | accessIPv4 > | | > | accessIPv6 > | | > | config_drive | > True | > | created | > 2016-07-20T01:18:42Z | > | description | > - | > | flavor | m1.tiny > (1) | > | hostId | > 36ef28d2b661e38d2d07645d814903a15d62da769828b57029306ec0 | > | host_status | > UP | > | id | > 27264d62-6a7c-4fe9-be81-c06fca56ec00 | > | image | cirros-0.3.4-x86_64-uec > (aa86e8b5-0699-46a0-a624-7af794b21404) | > | key_name | > - | > | lb-mgmt-net network | > 192.168.0.12 | > | locked | > False | > | metadata | > {} | > | name | > test01 | > | os-extended-volumes:volumes_attached | > [] | > | progress | > 0 | > | security_groups | > lb-mgmt-sec-grp | > | status | > ACTIVE | > | tags | > [] | > | tenant_id | > 73aebe8aa8ab41f58d5e375a03e279bf | > | updated | > 2016-07-20T01:18:48Z | > | user_id | > 53f8c8e491e94d2fa9210f3a8e6a85e4 | > +-------------------------------------- > +----------------------------------------------------------------+ > > 2. the security group: > > neutron security-group-show lb-mgmt-sec-grp > +---------------------- > +--------------------------------------------------------------------+ > | Field | > Value | > +---------------------- > +--------------------------------------------------------------------+ > | description > | | > | id | > fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee | > | name | > lb-mgmt-sec-grp | > | security_group_rules | > { | > | | "remote_group_id": > null, | > | | "direction": > "ingress", | > | | "protocol": > "icmp", | > | | "description": > "", | > | | "ethertype": > "IPv4", | > | | "remote_ip_prefix": > null, | > | | "port_range_max": > null, | > | | "security_group_id": > "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", | > | | "port_range_min": > null, | > | | "tenant_id": > "73aebe8aa8ab41f58d5e375a03e279bf", | > | | "id": > "140677a5-5308-48b2-a5a2-bb5e17994ed5" | > | | > } | > | | > { | > | | "remote_group_id": > null, | > | | "direction": > "ingress", | > | | "protocol": > "tcp", | > | | "description": > "", | > | | "ethertype": > "IPv4", | > | | "remote_ip_prefix": > null, | > | | "port_range_max": > 22, | > | | "security_group_id": > "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", | > | | "port_range_min": > 22, | > | | "tenant_id": > "73aebe8aa8ab41f58d5e375a03e279bf", | > | | "id": > "39fccc0c-f832-497a-b03d-fa0e40e3f407" | > | | > } | > | | > { | > | | "remote_group_id": > null, | > | | "direction": > "egress", | > | | "protocol": > null, | > | | "description": > "", | > | | "ethertype": > "IPv6", | > | | "remote_ip_prefix": > null, | > | | "port_range_max": > null, | > | | "security_group_id": > "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", | > | | "port_range_min": > null, | > | | "tenant_id": > "73aebe8aa8ab41f58d5e375a03e279bf", | > | | "id": > "54d134c0-f4bc-4f3d-bf49-0e1d0ac9ef1c" | > | | > } | > | | > { | > | | "remote_group_id": > null, | > | | "direction": > "ingress", | > | | "protocol": > "tcp", | > | | "description": > "", | > | | "ethertype": > "IPv4", | > | | "remote_ip_prefix": > null, | > | | "port_range_max": > 9443, | > | | "security_group_id": > "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", | > | | "port_range_min": > 9443, | > | | "tenant_id": > "73aebe8aa8ab41f58d5e375a03e279bf", | > | | "id": > "b3e00b04-d398-450b-b1cf-b92fd3dc37a1" | > | | > } | > | | > { | > | | "remote_group_id": > null, | > | | "direction": > "egress", | > | | "protocol": > null, | > | | "description": > "", | > | | "ethertype": > "IPv4", | > | | "remote_ip_prefix": > null, | > | | "port_range_max": > null, | > | | "security_group_id": > "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", | > | | "port_range_min": > null, | > | | "tenant_id": > "73aebe8aa8ab41f58d5e375a03e279bf", | > | | "id": > "c528b1cf-b065-4498-986c-13adac4c2a0a" | > | | > } | > | tenant_id | > 73aebe8aa8ab41f58d5e375a03e279bf | > +---------------------- > +--------------------------------------------------------------------+ > _______________________________________________ > dev mailing list > dev@openvswitch.org > http://openvswitch.org/mailman/listinfo/dev For issues between ovn and openstack, the openstack-dev mailing list with a tag of [networking-ovn] is another mailing list that you should try. I've added the same in the CC of this message to help. You've shared a lot of information about the ovv/ovs side of the situation, but more information on the openstack side would also help. Since you say "all-in-one" is this running with devstack? Is this the master branch or a stable branch of neutron? What is the latest commit in the networking-ovn repository? What settings are there in the neutron.ini and networking-ovn.ini files? I've personally seen tip of the tree neutron, networking-ovn, and ovn work together in multi-node setups, so my initial thought is to check the code versions or configurations you are using... Ryan _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev