Public bug reported: Guys,
After deep research on the following problem: Xenial - OpenvSwitch with DPDK binding to 10G NIC, not working: https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2016-March/016287.html I realized that, going against the DPDK documentation: http://dpdk.org/doc/guides/nics/bnx2x.html The option "CONFIG_RTE_LIBRTE_BNX2X_PMD" is DISABLED by default! Not enabled, as the doc points (CONFIG_RTE_LIBRTE_BNX2X_PMD (default y))... Take a look: --- cd ~/sources/dpdk apt source dpdk cd dpdk-2.2.0 $ grep PMD config/common_linuxapp | grep ^CONFIG | grep n CONFIG_RTE_LIBRTE_MLX4_PMD=n CONFIG_RTE_LIBRTE_MLX5_PMD=n CONFIG_RTE_LIBRTE_BNX2X_PMD=n CONFIG_RTE_LIBRTE_NFP_PMD=n CONFIG_RTE_LIBRTE_PMD_SZEDATA2=n CONFIG_RTE_LIBRTE_PMD_PCAP=n CONFIG_RTE_LIBRTE_PMD_XENVIRT=n CONFIG_RTE_LIBRTE_PMD_QAT=n CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=n CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n --- I strong believe that this is the main problem with OpenvSwitch + DPDK on Ubuntu, that causes OVS to not find any "dpdkX" interfaces! Even if DPDK-Compatible driver is there, being used!!! So, I added the following line: --- -e 's,(CONFIG_RTE_LIBRTE_BNX2X_PMD=).*,\1y,' \ --- To DPDK's debian/rules file, it now looks like this: --- build-config: dh_testdir $(MAKE) O=$(DPDK_STATIC_DIR) T=$(DPDK_CONFIG) config sed -ri -e 's,(RTE_MACHINE=).*,\1"default",' \ -e 's,(RTE_NEXT_ABI=).*,\1n,' \ -e 's,(RTE_APP_TEST=).*,\1n,' \ -e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' \ -e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' \ -e 's,(CONFIG_RTE_BUILD_COMBINE_LIBS=).*,\1y,' \ -e 's,(CONFIG_RTE_LIBRTE_BNX2X_PMD=).*,\1y,' \ -e 's,(LIBRTE_PMD_PCAP=).*,\1y,' \ -e 's,(LIBRTE_PMD_XENVIRT=).*,\1y,' \ $(DPDK_STATIC_DIR)/.config --- Note: There is a need to install: sudo apt install zlib1g-dev After that, now, I can see a very different behavior! Still not working... lol But some progress, look, the error on OVS log is very different now: --- ovs-ctl[2170]: EAL: PCI device 0000:01:00.0 on NUMA socket 0 ovs-ctl[2170]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd ovs-vswitchd[2551]: EAL: PCI device 0000:01:00.0 on NUMA socket 0 ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6180000000 ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6180800000 ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6181000000 ovs-vswitchd[2551]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd ovs-ctl[2170]: ovs-vswitchd: /home/ubuntu/sources/dpdk/dpdk-2.2.0/drivers/net/bnx2x/bnx2x_ethdev.c:453: bnx2x_common_dev_init: Assertion `sc->firmware' failed. ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6180000000 ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6180800000 ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6181000000 ovs-ctl[2170]: Aborted (core dumped) --- So, I tried something different, changed /etc/dpdk/interfaces from this: --- pci 0000:01:00.0 uio_pci_generic pci 0000:01:00.1 uio_pci_generic --- To this: --- pci 0000:01:00.0 vfio-pci pci 0000:01:00.1 vfio-pci --- And now, a completely different, error message (still not working): --- ovs-vswitchd[2950]: EAL: PCI device 0000:01:00.0 on NUMA socket 0 ovs-vswitchd[2950]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd ovs-vswitchd[2950]: EAL: 0000:01:00.0 VFIO group is not viable! ovs-vswitchd[2950]: EAL: Error - exiting with code: 1 ovs-vswitchd[2950]: Requested device 0000:01:00.0 cannot be used --- It looks much better with VFIO! There is no more "core dumped"! But still not working. So, I'm opening this bug report, so you guys can enable the PMD drivers for all the remaining drivers, like BNX2X, MLX4 & 5, XENVIRT, QAT, PCAP and etc (maybe I am missing other DPDK options that we must enable! Lets take a deep look into those disabled options)... However, I'm curious about the usage of BNX2X when with "uio_pci_generic", what is that firmware error followed by a core dump? Very intriguing... BTW, when with VFIO, I added the following options "iommu=pt intel_iommu=on" to /etc/default/grub, otherwise, VFIO doesn't work. And, after reading the VFIO OVS error message, "VFIO group is not viable", I google for it and I found the following blog post about this: [IOMMU] The error of "VFIO group is not viable": http://danny270degree.blogspot.com.br/2015/12/iommu-error-of-vfio-group-is-not-viable.html So, I think that I am almost there! It is just a matter of configuring this VFIO group thing but, I have no idea about how to do that... :-P Christian is using ixgbe driver, which have PMD enabled by default! That's why his setup works (ixgbe), and mine doesn't (bnx2x), am I right? Look: --- .../dpdk-2.2.0$ grep -i ixgbe_pmd config/common_linuxapp CONFIG_RTE_LIBRTE_IXGBE_PMD=y --- I can't think about any other reason for this problem... PMD is disabled for many drivers! It is Friday night, long week... I'll play with this during weekend, for sure! ^_^ Cheers! Thiago ** Affects: dpdk (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to dpdk in Ubuntu. https://bugs.launchpad.net/bugs/1559408 Title: Many PMD driver options are disabled, including "BNX2X"! To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1559408/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
