Hi, I have a DPDK application running on QEMU-KVM environment using DPDK 1.6.0. I am trying to port the same to DPDK version 1.7.1.
I am using Virt-manager GUI to assign e1000 emulated port to the VM. This works fine in DPDK 1.6.0. The device is identified by IGB_UIO and initialized by my application as expected. However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to be recognized. Following is my analysis: 1. The API pci_get_uio_dev() is returning ERROR. This is called from pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()]. 2. Due to this, the PCI device is not getting mapped to the correct driver (EM ? Driver? ). 3. The reason for the error in [1] appears to be that "uio" sub-directory doesn't seem to be correctly created for interfaces assigned to this VM. 4. Upon further analysis i found that IGB_UIO probe function ["igbuio_pci_probe()"] is not getting triggered indicating the port has *not been assigned* to the IGB_UIO. Kindly refer to the attachments:- - "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not created for PCI devices in case of DPDK 1.7.1 - "Output of lspci -v" - indicating device not bound to driver in case of DPDK 1.7.1 - IGB_UIO init log snippet - indicating PCI devices not detected and initialized by IGB_UIO in case of DPDK 1.7.1 - CONFIG file used for DPDK compilation Not sure what has changed between 1.6.0 and 1.7.1 which is impacting this. Could someone throw light in this regard as to what i may be missing ? Thanks in advance. Regards, Manoj -------------- next part -------------- ---------------------------------------------- [A] IGB UIO INIT LOGS - VM running DPDK 1.6.0 ---------------------------------------------- Oct 20 05:14:31 localhost kernel: Use MSIX interrupt by default Oct 20 05:14:31 localhost kernel: Use MSIX interrupt by default Oct 20 05:14:31 localhost kernel: igb_uio 0000:00:04.0: setting latency timer to 64 Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en tries Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en tries Oct 20 05:14:31 localhost kernel: uio device registered with irq b -------------------------------------> PCI device bound to driver Oct 20 05:14:31 localhost kernel: uio device registered with irq b Oct 20 05:14:31 localhost kernel: igb_uio 0000:00:08.0: setting latency timer to 64 Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en tries Oct 20 05:14:31 localhost kernel: fail to enable pci msix, or not enough msix en tries Oct 20 05:14:31 localhost kernel: uio device registered with irq b -------------------------------------> PCI device bound to driver Oct 20 05:14:31 localhost kernel: uio device registered with irq b ---------------------------------------------- [B] IGB UIO INIT LOGS - VM running DPDK 1.7.1 ---------------------------------------------- Oct 20 05:10:40 localhost kernel: igb_uio: Use MSIX interrupt by default Oct 20 05:10:40 localhost kernel: igb_uio: Use MSIX interrupt by default =====> No output for PCI initialization by IGB_UIO -------------- next part -------------- ------------------------- [VM-2 running DPDK 1.6.0] ------------------------- 00:08.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controll er (rev 03) Subsystem: Red Hat, Inc Device 1100 Flags: bus master, fast devsel, latency 0, IRQ 11 Memory at f20a0000 (32-bit, non-prefetchable) [size=128K] I/O ports at c140 [size=64] Expansion ROM at f20c0000 [disabled] [size=128K] Kernel driver in use: igb_uio=================> Driver binding succesfull Kernel modules: igb_uio ==================> Driver binding succesfull ------------------------- [VM-1 running DPDK 1.7.1] ------------------------- 00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controll er (rev 03) Subsystem: Red Hat, Inc Device 1100 Flags: fast devsel, IRQ 11 Memory at f2040000 (32-bit, non-prefetchable) [size=128K] I/O ports at c080 [size=64] Expansion ROM at f2060000 [disabled] [size=128K] ===> Missing driver binding info -------------- next part -------------- @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ On VM-1 running DPDK 1.7.1 @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ bash-4.2# lspci | grep 82540EM 00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03) 00:05.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03) ##SYS-BUS bash-4.2# ls /sys/bus/pci/devices/ 0000:00:00.0 0000:00:01.2 0000:00:03.0 0000:00:06.0 0000:00:01.0 0000:00:01.3 0000:00:04.0 0000:00:07.0 0000:00:01.1 0000:00:02.0 0000:00:05.0 0000:00:08.0 bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/ broken_parity_status local_cpulist resource0 class local_cpus resource1 config modalias rom consistent_dma_mask_bits msi_bus subsystem/ device numa_node subsystem_device dma_mask_bits power/ subsystem_vendor enable remove uevent firmware_node/ rescan vendor irq resource bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/uio ls: cannot access /sys/bus/pci/devices/0000:00:04.0/uio: No such file or directory ============================> No directory bash-4.2# ls /sys/bus/pci/devices/0000\:00\:05.0/ broken_parity_status local_cpulist resource0 class local_cpus resource1 config modalias rom consistent_dma_mask_bits msi_bus subsystem/ device numa_node subsystem_device dma_mask_bits power/ subsystem_vendor enable remove uevent firmware_node/ rescan vendor irq resource bash-4.2# ls /sys/bus/pci/devices/0000\:00\:05.0/uio ls: cannot access /sys/bus/pci/devices/0000:00:05.0/uio: No such file or directory ============================> No directory bash-4.2# ##SYS-DEVICES bash-4.2# ls /sys/devices/pci0000\:00/ 0000:00:00.0/ 0000:00:01.3/ 0000:00:05.0/ firmware_node/ 0000:00:01.0/ 0000:00:02.0/ 0000:00:06.0/ pci_bus/ 0000:00:01.1/ 0000:00:03.0/ 0000:00:07.0/ power/ 0000:00:01.2/ 0000:00:04.0/ 0000:00:08.0/ uevent bash-4.2# ls /sys/devices/pci0000\:00/0000:00:04.0/uio ls: cannot access /sys/devices/pci0000:00/0000:00:04.0/uio: No such file or directory ============================> No directory bash-4.2# ls /sys/devices/pci0000\:00/0000:00:04.0/ broken_parity_status enable msi_bus resource0 uevent class firmware_node numa_node resource1 vendor config irq power rom consistent_dma_mask_bits local_cpulist remove subsystem device local_cpus rescan subsystem_device dma_mask_bits modalias resource subsystem_vendor bash-4.2# bash-4.2# ls /sys/devices/pci0000\:00/0000:00:05.0/uio ls: cannot access /sys/devices/pci0000:00/0000:00:05.0/uio: No such file or directory ============================> No directory bash-4.2# ls /sys/devices/pci0000\:00/0000:00:05.0/ broken_parity_status enable msi_bus resource0 uevent class firmware_node numa_node resource1 vendor config irq power rom consistent_dma_mask_bits local_cpulist remove subsystem device local_cpus rescan subsystem_device dma_mask_bits modalias resource subsystem_vendor -------------------------------------------------------------------------------------------------------------------------------------------------------- @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ On VM-2 running DPDK 1.6.0 @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ bash-4.2# lspci | grep 82540EM 00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03) 00:08.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03) ##SYS-BUS bash-4.2# /sys/bus/pci/devices/0000\:00\:04.0/uio/uio0/ ============================> Directory created correctly device/ maps/ portio/ power/ subsystem/ bash-4.2# ls /sys/bus/pci/devices/0000\:00\:04.0/uio/uio0/ dev device event maps name portio power subsystem uevent version bash-4.2# ls /sys/bus/pci/devices/0000\:00\:08.0/uio/uio1/ ============================> Directory created correctly dev device event maps name portio power subsystem uevent version ##SYS-DEVICES bash-4.2# ls /sys/devices/pci0000:00/0000:00:04.0/uio/uio0 dev device event maps name portio power subsystem uevent version bash-4.2#