Hi, On 28/08/2018 02:38, menkeyi wrote: > [root@compute01 ~]# uname -r > 3.10.0-862.el7.x86_64 > > 1、Ovs vxlan mode is available > 2、Ovs + DPDK vxlan mode is not available >
Can you provide a bit more info here? What in specific do you see not working, or not available? And when you say "Ovs vxlan" above (without DPDK), is that the userspace or the kernel datapath you're referring to? > DPDK ovs info: > > [root@compute01 ~]# ovs-vswitchd --version > ovs-vswitchd (Open vSwitch) 2.9.0 > DPDK 17.11.0 > Just a note that the advised version for using with 2.9.0 is (at least) DPDK v17.11.2, for security concerns. More info here: http://mails.dpdk.org/archives/announce/2018-April/000192.html > [root@compute01 ~]# dpdk-devbind --status|head -n10 > Network devices using DPDK-compatible driver > ============================================ > 0000:81:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' > drv=vfio-pci unused= > 0000:84:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' > drv=vfio-pci unused= > > [root@compute01 ~]# ovs-vsctl show > b79a7e81-2d68-4ecd-8cd8-bc2b7d1c52ef > Manager "ptcp:6640:127.0.0.1" > is_connected: true > Bridge br-tun > Controller "tcp:127.0.0.1:6633" > is_connected: true > fail_mode: secure > Port patch-int > Interface patch-int > type: patch > options: {peer=patch-tun} > Port "vxlan-33000169" > Interface "vxlan-33000169" > type: vxlan > options: {df_default="true", in_key=flow, > local_ip="51.0.1.101", out_key=flow, remote_ip="51.0.1.105"} > Port br-tun > Interface br-tun > type: internal > Port "vxlan-330001ca" > Interface "vxlan-330001ca" > type: vxlan > options: {df_default="true", in_key=flow, > local_ip="51.0.1.101", out_key=flow, remote_ip="51.0.1.202"} > Bridge br-provider > Controller "tcp:127.0.0.1:6633" > is_connected: true > fail_mode: secure > Port br-provider > Interface br-provider > type: internal > Port phy-br-provider > Interface phy-br-provider > type: patch > options: {peer=int-br-provider} > Port "Team1" > Interface "team1-enp132s0f0" > type: dpdk > options: {dpdk-devargs="0000:84:00.0"} > Interface "team1-enp129s0f0" > type: dpdk > options: {dpdk-devargs="0000:81:00.0"} > Bridge br-int > Controller "tcp:127.0.0.1:6633" > is_connected: true > fail_mode: secure > Port "vhu15e890ef-36" > tag: 1 > Interface "vhu15e890ef-36" > type: dpdkvhostuserclient > options: > {vhost-server-path="/var/run/openvswitch/vhu15e890ef-36"} > Port patch-tun > Interface patch-tun > type: patch > options: {peer=patch-int} > Port int-br-provider > Interface int-br-provider > type: patch > options: {peer=phy-br-provider} > Port br-int > Interface br-int > type: internal > Port "vhu82caa59c-d3" > tag: 1 > Interface "vhu82caa59c-d3" > type: dpdkvhostuserclient > options: > {vhost-server-path="/var/run/openvswitch/vhu82caa59c-d3"} > Port "tapadb8cafa-a0" > tag: 4095 > Interface "tapadb8cafa-a0" > type: internal > ovs_version: "2.9.0" > > > > > Create a virtual machine and find that the error is found in the > ovs-vswitchd.log log. I don't know if it is related to this： > on vxlan_sys_4789 device failed: No such device > > This shouldn't be related. I'm seeing this on my system as well even though vxlan is working. It is a bit misleading though, so I'll check what can lead to this and try to clean it up. > > 2018-08-25T02:34:54.453Z|01178|dpdk|INFO|VHOST_CONFIG: vhost-user > client: socket created, fd: 83 > 2018-08-25T02:34:54.453Z|01179|netdev_dpdk|INFO|vHost User device > 'vhu82caa59c-d3' created in 'client' mode, using client socket > '/var/run/openvswitch/vhu82caa59c-d3' > 2018-08-25T02:34:54.453Z|01180|dpdk|WARN|VHOST_CONFIG: failed to connect > to /var/run/openvswitch/vhu82caa59c-d3: No such file or directory > 2018-08-25T02:34:54.453Z|01181|dpdk|INFO|VHOST_CONFIG: > /var/run/openvswitch/vhu82caa59c-d3: reconnecting... > 2018-08-25T02:34:54.634Z|01182|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp129s0f0' rx queue 0 (measured processing cycles > 587888). > 2018-08-25T02:34:54.634Z|01183|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp132s0f0' rx queue 0 (measured processing cycles > 402684). > 2018-08-25T02:34:54.634Z|01184|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu82caa59c-d3' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.635Z|01185|bridge|INFO|bridge br-int: added > interface vhu82caa59c-d3 on port 1 > 2018-08-25T02:34:54.723Z|01186|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp129s0f0' rx queue 0 (measured processing cycles > 587888). > 2018-08-25T02:34:54.723Z|01187|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp132s0f0' rx queue 0 (measured processing cycles > 402684). > 2018-08-25T02:34:54.723Z|01188|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu82caa59c-d3' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.770Z|01189|dpdk|INFO|VHOST_CONFIG: vhost-user > client: socket created, fd: 84 > 2018-08-25T02:34:54.770Z|01190|netdev_dpdk|INFO|vHost User device > 'vhu15e890ef-36' created in 'client' mode, using client socket > '/var/run/openvswitch/vhu15e890ef-36' > 2018-08-25T02:34:54.770Z|01191|dpdk|WARN|VHOST_CONFIG: failed to connect > to /var/run/openvswitch/vhu15e890ef-36: No such file or directory > 2018-08-25T02:34:54.770Z|01192|dpdk|INFO|VHOST_CONFIG: > /var/run/openvswitch/vhu15e890ef-36: reconnecting... > 2018-08-25T02:34:54.770Z|01193|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp129s0f0' rx queue 0 (measured processing cycles > 587888). > 2018-08-25T02:34:54.770Z|01194|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp132s0f0' rx queue 0 (measured processing cycles > 402684). > 2018-08-25T02:34:54.770Z|01195|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu82caa59c-d3' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.770Z|01196|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu15e890ef-36' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.771Z|01197|bridge|INFO|bridge br-int: added > interface vhu15e890ef-36 on port 4 > 2018-08-25T02:34:54.861Z|01198|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp129s0f0' rx queue 0 (measured processing cycles > 587888). > 2018-08-25T02:34:54.861Z|01199|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp132s0f0' rx queue 0 (measured processing cycles > 402684). > 2018-08-25T02:34:54.861Z|01200|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu82caa59c-d3' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.861Z|01201|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu15e890ef-36' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:54.987Z|00001|dpdk|INFO|VHOST_CONFIG: > /var/run/openvswitch/vhu82caa59c-d3: connected > 2018-08-25T02:34:54.987Z|00002|dpdk|INFO|VHOST_CONFIG: new device, > handle is 0 > 2018-08-25T02:34:55.194Z|00001|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2018-08-25T02:34:55.195Z|00002|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_PROTOCOL_FEATURES > 2018-08-25T02:34:55.195Z|00003|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_PROTOCOL_FEATURES > 2018-08-25T02:34:55.195Z|00004|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_QUEUE_NUM > 2018-08-25T02:34:55.195Z|00005|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_SLAVE_REQ_FD > 2018-08-25T02:34:55.195Z|00006|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_OWNER > 2018-08-25T02:34:55.195Z|00007|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2018-08-25T02:34:55.195Z|00008|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:34:55.195Z|00009|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > file:86 > 2018-08-25T02:34:55.195Z|00010|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:34:55.195Z|00011|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > file:87 > 2018-08-25T02:34:55.691Z|01202|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp129s0f0' rx queue 0 (measured processing cycles > 587888). > 2018-08-25T02:34:55.691Z|01203|dpif_netdev|INFO|Core 8 on numa node 1 > assigned port 'team1-enp132s0f0' rx queue 0 (measured processing cycles > 402684). > 2018-08-25T02:34:55.691Z|01204|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu82caa59c-d3' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:55.691Z|01205|dpif_netdev|INFO|Core 0 on numa node 0 > assigned port 'vhu15e890ef-36' rx queue 0 (measured processing cycles 0). > 2018-08-25T02:34:55.692Z|01206|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.692Z|01207|bridge|INFO|bridge br-tun: added > interface vxlan-330001ca on port 2 > 2018-08-25T02:34:55.693Z|01208|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.695Z|01209|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.709Z|01210|bridge|INFO|bridge br-tun: added > interface vxlan-33000169 on port 3 > 2018-08-25T02:34:55.710Z|01211|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.712Z|01212|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.713Z|01213|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on > vxlan_sys_4789 device failed: No such device > 2018-08-25T02:34:55.987Z|00003|dpdk|INFO|VHOST_CONFIG: > /var/run/openvswitch/vhu15e890ef-36: connected > 2018-08-25T02:34:55.987Z|00004|dpdk|INFO|VHOST_CONFIG: new device, > handle is 1 > 2018-08-25T02:34:56.196Z|00012|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2018-08-25T02:34:56.196Z|00013|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_PROTOCOL_FEATURES > 2018-08-25T02:34:56.196Z|00014|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_PROTOCOL_FEATURES > 2018-08-25T02:34:56.196Z|00015|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_QUEUE_NUM > 2018-08-25T02:34:56.196Z|00016|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_SLAVE_REQ_FD > 2018-08-25T02:34:56.196Z|00017|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_OWNER > 2018-08-25T02:34:56.196Z|00018|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2018-08-25T02:34:56.196Z|00019|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:34:56.196Z|00020|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > file:89 > 2018-08-25T02:34:56.196Z|00021|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:34:56.196Z|00022|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > file:90 > 2018-08-25T02:35:04.169Z|00023|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.169Z|00024|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.169Z|00025|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu15e890ef-36'changed > to 'enabled' > 2018-08-25T02:35:04.169Z|00026|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.169Z|00027|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:04.169Z|00028|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.169Z|00029|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.169Z|00030|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu15e890ef-36'changed > to 'enabled' > 2018-08-25T02:35:04.169Z|00031|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.169Z|00032|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:04.171Z|00033|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_FEATURES > 2018-08-25T02:35:04.171Z|00034|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_MEM_TABLE > 2018-08-25T02:35:04.171Z|00035|dpdk|INFO|VHOST_CONFIG: guest memory > region 0, size: 0xa0000 > guest physical addr: 0x0 > guest virtual addr: 0x7f3700000000 > host virtual addr: 0x2aaac0000000 > mmap addr : 0x2aaac0000000 > mmap size : 0x40000000 > mmap align: 0x40000000 > mmap off : 0x0 > 2018-08-25T02:35:04.172Z|00036|dpdk|INFO|VHOST_CONFIG: guest memory > region 1, size: 0x7ff40000 > guest physical addr: 0xc0000 > guest virtual addr: 0x7f37000c0000 > host virtual addr: 0x2aab000c0000 > mmap addr : 0x2aab00000000 > mmap size : 0x80000000 > mmap align: 0x40000000 > mmap off : 0xc0000 > 2018-08-25T02:35:04.172Z|00037|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_NUM > 2018-08-25T02:35:04.172Z|00038|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_BASE > 2018-08-25T02:35:04.172Z|00039|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ADDR > 2018-08-25T02:35:04.172Z|00040|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_KICK > 2018-08-25T02:35:04.172Z|00041|dpdk|INFO|VHOST_CONFIG: vring kick idx:0 > file:93 > 2018-08-25T02:35:04.172Z|00042|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:35:04.172Z|00043|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > file:94 > 2018-08-25T02:35:04.172Z|00044|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_NUM > 2018-08-25T02:35:04.172Z|00045|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_BASE > 2018-08-25T02:35:04.172Z|00046|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ADDR > 2018-08-25T02:35:04.172Z|00047|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_KICK > 2018-08-25T02:35:04.172Z|00048|dpdk|INFO|VHOST_CONFIG: vring kick idx:1 > file:89 > 2018-08-25T02:35:04.172Z|00049|dpdk|INFO|VHOST_CONFIG: virtio is now > ready for processing. > 2018-08-25T02:35:04.172Z|00050|netdev_dpdk|INFO|vHost Device > '/var/run/openvswitch/vhu15e890ef-36' has been added on numa node 0 > 2018-08-25T02:35:04.172Z|00051|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:35:04.172Z|00052|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > file:95 > 2018-08-25T02:35:04.172Z|00053|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.172Z|00054|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.172Z|00055|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu15e890ef-36'changed > to 'enabled' > 2018-08-25T02:35:04.172Z|00056|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.172Z|00057|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:04.955Z|00058|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.955Z|00059|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.955Z|00060|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu82caa59c-d3'changed > to 'enabled' > 2018-08-25T02:35:04.955Z|00061|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.955Z|00062|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:04.955Z|00063|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.955Z|00064|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.955Z|00065|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu82caa59c-d3'changed > to 'enabled' > 2018-08-25T02:35:04.955Z|00066|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.955Z|00067|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:04.956Z|00068|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_FEATURES > 2018-08-25T02:35:04.956Z|00069|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_MEM_TABLE > 2018-08-25T02:35:04.957Z|00070|dpdk|INFO|VHOST_CONFIG: guest memory > region 0, size: 0xa0000 > guest physical addr: 0x0 > guest virtual addr: 0x7fb240000000 > host virtual addr: 0x2aab80000000 > mmap addr : 0x2aab80000000 > mmap size : 0x40000000 > mmap align: 0x40000000 > mmap off : 0x0 > 2018-08-25T02:35:04.958Z|00071|dpdk|INFO|VHOST_CONFIG: guest memory > region 1, size: 0x7ff40000 > guest physical addr: 0xc0000 > guest virtual addr: 0x7fb2400c0000 > host virtual addr: 0x2aabc00c0000 > mmap addr : 0x2aabc0000000 > mmap size : 0x80000000 > mmap align: 0x40000000 > mmap off : 0xc0000 > 2018-08-25T02:35:04.958Z|00072|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_NUM > 2018-08-25T02:35:04.958Z|00073|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_BASE > 2018-08-25T02:35:04.958Z|00074|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ADDR > 2018-08-25T02:35:04.958Z|00075|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_KICK > 2018-08-25T02:35:04.958Z|00076|dpdk|INFO|VHOST_CONFIG: vring kick idx:0 > file:97 > 2018-08-25T02:35:04.958Z|00077|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:35:04.958Z|00078|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > file:98 > 2018-08-25T02:35:04.958Z|00079|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_NUM > 2018-08-25T02:35:04.958Z|00080|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_BASE > 2018-08-25T02:35:04.958Z|00081|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ADDR > 2018-08-25T02:35:04.958Z|00082|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_KICK > 2018-08-25T02:35:04.958Z|00083|dpdk|INFO|VHOST_CONFIG: vring kick idx:1 > file:86 > 2018-08-25T02:35:04.958Z|00084|dpdk|INFO|VHOST_CONFIG: virtio is now > ready for processing. > 2018-08-25T02:35:04.958Z|00085|netdev_dpdk|INFO|vHost Device > '/var/run/openvswitch/vhu82caa59c-d3' has been added on numa node 0 > 2018-08-25T02:35:04.958Z|00086|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2018-08-25T02:35:04.958Z|00087|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > file:99 > 2018-08-25T02:35:04.958Z|00088|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.958Z|00089|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 0 > 2018-08-25T02:35:04.958Z|00090|netdev_dpdk|INFO|State of queue 0 ( > tx_qid 0 ) of vhost device '/var/run/openvswitch/vhu82caa59c-d3'changed > to 'enabled' > 2018-08-25T02:35:04.958Z|00091|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_ENABLE > 2018-08-25T02:35:04.958Z|00092|dpdk|INFO|VHOST_CONFIG: set queue enable: > 1 to qp idx: 1 > 2018-08-25T02:35:05.018Z|01214|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633: > 11 flow_mods in the 1 s starting 10 s ago (11 adds) > 2018-08-25T02:35:05.245Z|01215|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: > 30 flow_mods in the 1 s starting 10 s ago (24 adds, 6 deletes) > > No vxlan_sys_4789 device found，but you can see it with ovs-appctl > dpctl/show. > I don't know what went wrong.vm of 2 compute nodes cannot communicate > through vxlan network > > [root@compute01 ~]# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN > group default qlen 1000 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: enp2s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state > DOWN group default qlen 1000 > link/ether 58:f9:87:5c:a4:9f brd ff:ff:ff:ff:ff:ff > 3: enp2s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state > DOWN group default qlen 1000 > link/ether 58:f9:87:5c:a4:a0 brd ff:ff:ff:ff:ff:ff > 5: enp2s0f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master > team0 state UP group default qlen 1000 > link/ether 58:f9:87:5c:a4:a1 brd ff:ff:ff:ff:ff:ff > 6: enp2s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master > team0 state UP group default qlen 1000 > link/ether 58:f9:87:5c:a4:a1 brd ff:ff:ff:ff:ff:ff > 7: enp132s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq > master team2 state UP group default qlen 1000 > link/ether 58:f9:87:5d:6b:25 brd ff:ff:ff:ff:ff:ff > 9: enp129s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq > master team2 state UP group default qlen 1000 > link/ether 58:f9:87:5d:6b:25 brd ff:ff:ff:ff:ff:ff > 10: team2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue > state UP group default qlen 1000 > link/ether 58:f9:87:5d:6b:25 brd ff:ff:ff:ff:ff:ff > inet 51.1.1.101/24 brd 51.1.1.255 scope global noprefixroute team2 > valid_lft forever preferred_lft forever > 11: team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue > state UP group default qlen 1000 > link/ether 58:f9:87:5c:a4:a1 brd ff:ff:ff:ff:ff:ff > inet 51.0.1.101/24 brd 51.0.1.255 scope global noprefixroute team0 > valid_lft forever preferred_lft forever > 12: ovs-netdev: <BROADCAST,PROMISC> mtu 1500 qdisc noop state DOWN group > default qlen 1000 > link/ether c2:ef:e0:1f:78:b5 brd ff:ff:ff:ff:ff:ff > 17: br-provider: <BROADCAST,PROMISC> mtu 1500 qdisc noop state DOWN > group default qlen 1000 > link/ether 58:f9:87:5d:6b:16 brd ff:ff:ff:ff:ff:ff > 37: br-int: <BROADCAST,PROMISC> mtu 1450 qdisc noop state DOWN group > default qlen 1000 > link/ether 82:f4:71:ba:69:4f brd ff:ff:ff:ff:ff:ff > 38: tapadb8cafa-a0: <BROADCAST,PROMISC> mtu 1450 qdisc noop state DOWN > group default qlen 1000 > link/ether 26:fd:d0:80:65:67 brd ff:ff:ff:ff:ff:ff > 42: br-tun: <BROADCAST,PROMISC> mtu 1500 qdisc noop state DOWN group > default qlen 1000 > link/ether 66:eb:c1:15:1b:45 brd ff:ff:ff:ff:ff:ff > > [root@compute01 ~]# ovs-appctl dpctl/show > netdev@ovs-netdev: > lookups: hit:95016 missed:28793 lost:3 > flows: 4 > port 0: ovs-netdev (tap) > port 1: br-int (tap) > port 2: vhu82caa59c-d3 (dpdkvhostuserclient: > configured_rx_queues=1, configured_tx_queues=1, mtu=1450, > requested_rx_queues=1, requested_tx_queues=1) > port 3: tapadb8cafa-a0 (tap) > port 4: vhu15e890ef-36 (dpdkvhostuserclient: > configured_rx_queues=1, configured_tx_queues=1, mtu=1450, > requested_rx_queues=1, requested_tx_queues=1) > port 5: br-tun (tap) > port 6: vxlan_sys_4789 (vxlan: packet_type=ptap) > port 7: team1-enp132s0f0 (dpdk: configured_rx_queues=1, > configured_rxq_descriptors=2048, configured_tx_queues=3, > configured_txq_descriptors=2048, mtu=1500, requested_rx_queues=1, > requested_rxq_descriptors=2048, requested_tx_queues=3, > requested_txq_descriptors=2048, rx_csum_offload=true) > port 8: team1-enp129s0f0 (dpdk: configured_rx_queues=1, > configured_rxq_descriptors=2048, configured_tx_queues=3, > configured_txq_descriptors=2048, mtu=1500, requested_rx_queues=1, > requested_rxq_descriptors=2048, requested_tx_queues=3, > requested_txq_descriptors=2048, rx_csum_offload=true) > port 9: br-provider (tap) > > I wasn't able to reproduce your fault. I've set up the example at [1], with OvS 2.9.0 and DPDK v17.11.0 (as you have mentioned), the only different being the kernel (I'm running 4.10). Vxlan encap and decap seem to work as expected. However, I suspect what might be happening here is that you're not using the tunneling-related commands for userspace [1]. As it happens, if you're using the Kernel, you can't rely on the ip tools in order to configure the interfaces, routing or ARP cache. Thus, when using the userspace datapath you'll need to configure routing and ARP using the tunneling-related commands, such as showed in [1]. I suspect this will be the cause, other than the Kernel you're running (3.10 vs the 4.10 I'm running). Can you confirm if you're already using the tunneling-related commands? If so, how are you setting up the routes and ARP caches? Regards, Tiago. [1] http://docs.openvswitch.org/en/latest/howto/userspace-tunneling/ > > _______________________________________________ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
