[dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
[root at localhost ~]# modinfo i40e filename: /lib/modules/3.10.0-327.18.2.el7.x86_64/updates/drivers/net/ethernet/intel/i40e/i40e.ko version:1.4.25 [root at localhost ~]# modinfo i40evf filename: /lib/modules/3.10.0-327.18.2.el7.x86_64/updates/drivers/net/ethernet/intel/i40evf/i40evf.ko version:1.4.15 dpdk ver : 16.04 nic type : 83:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) [root at localhost ~]# ethtool -i ens11 driver: i40e version: 1.4.25 firmware-version: 5.04 0x8000253b 0.0.0 bus-info: :83:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes [root at localhost ~]# Testcase: - Enable SRIOV # echo 4 > /sys/class/net/ens11/device/sriov_numvfs 83:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) 83:02.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) 83:02.1 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) 83:02.2 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) 83:02.3 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) 85:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) - modprobe vfio-pci [root at localhost ~]# /root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py --bind=vfio-pci :83:02.0 [root at localhost ~]# /root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py --bind=vfio-pci :83:02.1 [root at localhost ~]# /root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py --bind=vfio-pci :83:02.2 [root at localhost ~]# /root/ckannan/vpp/build-root/build-vpp-native/dpdk/dpdk-16.04/tools/dpdk_nic_bind.py --bind=vfio-pci :83:02.3 - launch testpmd ./testpmd -l 13,14,15,16 -n 4 -- -i -a --nb-cores=3 --nb-ports=1 I start seeing errors like these... Jul 01 17:23:29 localhost.localdomain kernel: dmar: DRHD: handling fault status reg 2 Jul 01 17:23:29 localhost.localdomain kernel: dmar: DMAR:[DMA Write] Request device [83:02.0] fault addr 0 DMAR:[fault reason 05] PTE Write access is not set Jul 01 17:23:29 localhost.localdomain kernel: i40e :83:00.0: TX driver issue detected, PF reset issued Jul 01 17:23:29 localhost.localdomain kernel: i40e :83:00.0: TX driver issue detected on VF 0 On Sun, Jun 5, 2016 at 8:47 AM, Zhang, Helin wrote: > Hi Kannan > > Could you help to provide more details of your test cases? How to > reproduce that issue? It would be better to try with DPDK example > applications. > I think the kernel driver version, DPDK version, XL710 firmware version > are the basic information we want to know. > If possible, I'd suggest you to try with latest version of above and try > if the issue is still there. > Thank you very much! > > Regards, > Helin > > > -Original Message- > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Chandrasekar > > Kannan > > Sent: Thursday, June 2, 2016 2:22 AM > > To: dev at dpdk.org > > Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci > > > > I have been attempting to VPP (dpdk) application with SR-IOV enabled. > > Using the vfio-pci driver/i40e/XL710nics/. > > I'm encountering a SEGV on the i40e code path. > > > > Has anyone else seen this behaviour ?. > > > > full mail thread discussion with vpp-dev is here - > > https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html > > > > > > > > here's a backtrace during vpp crash... > > > > Program received signal SIGSEGV, Segmentation fault. > > [Switching to Thread 0x7f2b811fd700 (LWP 8725)] > > 0x00455539 in rx_recv_pkts () > > (gdb) backtrace > > #0 0x00455539 in rx_recv_pkts () > > #1 0x00455e76 in i40e_recv_pkts_bulk_alloc () > > #2 0x7f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, > > rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=) > > at > > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp- > > native/dpdk/include/rte_ethdev.h:2641 > > #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=1, > > xd=0x7f2bb1898540) > > at > > /w/workspace/vpp-merge-master-centos7/build- > > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 > > #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=1, > > cpu_index=, node=0x7f2bb1ef59fc, xd=) > > at > > /w/workspace/vpp-merge-master-centos7/build- > > data/../vnet/vnet/devices/dpdk/node.c:508 > > #5 dpdk_input_rss (vm=, node=0x7f2bb1ef59fc, > > f= > out>) > > at > >
[dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
Hi Kannan Could you help to provide more details of your test cases? How to reproduce that issue? It would be better to try with DPDK example applications. I think the kernel driver version, DPDK version, XL710 firmware version are the basic information we want to know. If possible, I'd suggest you to try with latest version of above and try if the issue is still there. Thank you very much! Regards, Helin > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Chandrasekar > Kannan > Sent: Thursday, June 2, 2016 2:22 AM > To: dev at dpdk.org > Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci > > I have been attempting to VPP (dpdk) application with SR-IOV enabled. > Using the vfio-pci driver/i40e/XL710nics/. > I'm encountering a SEGV on the i40e code path. > > Has anyone else seen this behaviour ?. > > full mail thread discussion with vpp-dev is here - > https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html > > > > here's a backtrace during vpp crash... > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7f2b811fd700 (LWP 8725)] > 0x00455539 in rx_recv_pkts () > (gdb) backtrace > #0 0x00455539 in rx_recv_pkts () > #1 0x00455e76 in i40e_recv_pkts_bulk_alloc () > #2 0x7f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, > rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=) > at > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp- > native/dpdk/include/rte_ethdev.h:2641 > #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=1, > xd=0x7f2bb1898540) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 > #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=1, > cpu_index=, node=0x7f2bb1ef59fc, xd=) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/node.c:508 > #5 dpdk_input_rss (vm=, node=0x7f2bb1ef59fc, > f= out>) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/node.c:821 > #6 0x7f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f02cd4, > node=node at entry=0x7f2bb1ef59fc, > type=type at entry=VLIB_NODE_TYPE_INPUT, > dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING, > frame=frame at entry=0x0, last_time_stamp=1482097661494035) > at > /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 > #7 0x7f2be43db6d8 in dpdk_worker_thread_internal > (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/threads.c:209 > #8 dpdk_worker_thread (w=, io_name=, > callback=0x0) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/threads.c:265 > #9 0x7f2be3791530 in clib_calljmp () at > /w/workspace/vpp-merge-master-centos7/build- > data/../vppinfra/vppinfra/longjmp.S:110 > #10 0x7f2b811fcc40 in ?? () > #11 0x004eb9c7 in eal_thread_loop () ---Type to continue, > or q to quit--- > #12 0x in ?? () > (gdb) > (gdb) thread apply all bt > > Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)): > #0 0x7f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81 > #1 0x004ef178 in pci_vfio_mp_sync_thread () > #2 0x7f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at > pthread_create.c:308 > #3 0x7f2be2992ced in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 > > Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)): > #0 0x7f2be29932c3 in epoll_wait () at > ../sysdeps/unix/syscall-template.S:81 > #1 0x004efe04 in eal_intr_thread_main () > #2 0x7f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at > pthread_create.c:308 > #3 0x7f2be2992ced in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 > > Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)): > #0 0x0045526b in rx_recv_pkts () > #1 0x00455e76 in i40e_recv_pkts_bulk_alloc () > #2 0x7f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, > rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=) > at > /w/workspace/vpp-merge-master-centos7/build-root/install-vpp- > native/dpdk/include/rte_ethdev.h:2641 > #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=0, > xd=0x7f2bb1a1fd40) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 > ---Type to continue, or q to quit--- > #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=0, > cpu_index=, node=0x7f2bb1ea18cc, xd=) > at > /w/workspace/vpp-merge-master-centos7/build- > data/../vnet/vnet/devices/dpdk/node.c:508 >
[dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
I have been attempting to VPP (dpdk) application with SR-IOV enabled. Using the vfio-pci driver/i40e/XL710nics/. I'm encountering a SEGV on the i40e code path. Has anyone else seen this behaviour ?. full mail thread discussion with vpp-dev is here - https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html here's a backtrace during vpp crash... Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f2b811fd700 (LWP 8725)] 0x00455539 in rx_recv_pkts () (gdb) backtrace #0 0x00455539 in rx_recv_pkts () #1 0x00455e76 in i40e_recv_pkts_bulk_alloc () #2 0x7f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=) at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=1, xd=0x7f2bb1898540) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=1, cpu_index=, node=0x7f2bb1ef59fc, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ef59fc, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x7f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f02cd4, node=node at entry=0x7f2bb1ef59fc, type=type at entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING, frame=frame at entry=0x0, last_time_stamp=1482097661494035) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x7f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x7f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x7f2b811fcc40 in ?? () #11 0x004eb9c7 in eal_thread_loop () ---Type to continue, or q to quit--- #12 0x in ?? () (gdb) (gdb) thread apply all bt Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)): #0 0x7f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81 #1 0x004ef178 in pci_vfio_mp_sync_thread () #2 0x7f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at pthread_create.c:308 #3 0x7f2be2992ced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)): #0 0x7f2be29932c3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 #1 0x004efe04 in eal_intr_thread_main () #2 0x7f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at pthread_create.c:308 #3 0x7f2be2992ced in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)): #0 0x0045526b in rx_recv_pkts () #1 0x00455e76 in i40e_recv_pkts_bulk_alloc () #2 0x7f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256, rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=) at /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-native/dpdk/include/rte_ethdev.h:2641 #3 dpdk_rx_burst (dm=0x7f2be4673980 , queue_id=0, xd=0x7f2bb1a1fd40) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64 ---Type to continue, or q to quit--- #4 dpdk_device_input (dm=0x7f2be4673980 , queue_id=0, cpu_index=, node=0x7f2bb1ea18cc, xd=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:508 #5 dpdk_input_rss (vm=, node=0x7f2bb1ea18cc, f=) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/node.c:821 #6 0x7f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f237d4, node=node at entry=0x7f2bb1ea18cc, type=type at entry=VLIB_NODE_TYPE_INPUT, dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING, frame=frame at entry=0x0, last_time_stamp=148209754800) at /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996 #7 0x7f2be43db6d8 in dpdk_worker_thread_internal (have_io_threads=0, callback=0x0, vm=0x7f2bb1f237d4) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:209 #8 dpdk_worker_thread (w=, io_name=, callback=0x0) at /w/workspace/vpp-merge-master-centos7/build-data/../vnet/vnet/devices/dpdk/threads.c:265 #9 0x7f2be3791530 in clib_calljmp () at /w/workspace/vpp-merge-master-centos7/build-data/../vppinfra/vppinfra/longjmp.S:110 #10 0x7f2b819fdc40 in ?? () #11 0x004eb9c7 in eal_thread_loop () #12 0x in ?? () Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)): #0 0x00455539 in rx_recv_pkts () #1 0x00455e76 in
