Hi Junzhi, Can you please change log_num_mgm_entry_size=-1 Can you please run testpmd and specify only pf 04:00.0
Thanks, Olga -----Original Message----- From: users [mailto:[email protected]] On Behalf Of Junzhi Gong Sent: Thursday, November 1, 2018 6:45 PM To: [email protected] Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC Hi everyone, I am sorry that I did not give many examples and details last time. The Mellanox NIC is Device #1: ---------- Device Type: ConnectX3Pro Part Number: MCX313A-BCC_Ax Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6 PSID: MT_1060111023 PCI Device Name: /dev/mst/mt4103_pci_cr0 Port1 MAC: ec0d9a07dc90 Port2 MAC: ec0d9a07dc91 Versions: Current Available FW 2.42.5000 N/A PXE 3.4.0752 N/A The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this: $ lspci | grep Mellanox 04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] 04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] All port is set to Ethernet Adapter: $ connectx_port_config -s -------------------------------- Port configuration for PCI device: 0000:04:00.0 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.1 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.2 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.3 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.4 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.5 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.6 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:00.7 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:01.0 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:01.1 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:01.2 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:01.3 is: eth -------------------------------- -------------------------------- Port configuration for PCI device: 0000:04:01.4 is: eth -------------------------------- In file /etc/modprobe.d/mlx4_core.conf: options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12 My DPDK version is 18.08, and in file /config/common_base, I set: CONFIG_RTE_LIBRTE_MLX4_PMD=y After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib I run several DPDK applications, and I got the same error message as follows. Thank you! Junzhi > On Oct 26, 2018, at 4:27 PM, Junzhi Gong <[email protected]> wrote: > > Hi everyone, > > Currently I have some trouble running DPDK applications using Mellanox NIC. > All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will > crash with the following error message: > > PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, > "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, > message: flow rule rejected by device > EAL: Error - exiting with code: 1 > Cause: rte_eth_dev_start:err=-12, port=0 > > An example running click has the following error messages: > > $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click > > EAL: Detected 12 lcore(s) > EAL: Detected 1 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: No free hugepages reported in hugepages-1048576kB > EAL: Probing VFIO support... > EAL: PCI device 0000:04:00.1 on NUMA socket 0 > EAL: probe driver: 15b3:1004 net_mlx4 > PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: > true) > PMD: net_mlx4: 1 port(s) detected > PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing > DPDK > PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, > "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, > message: flow rule rejected by device > mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’: > Cannot start DPDK port 0: error -12 > > 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried > several different DPDK applications (all applications have been re-compiled > using the new configured DPDK), and I have also tried reboot the OS, but the > problem still exists. > > I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu > 18.04. I have installed the Mellanox OFED (version 4.4, only the newest > version is compatible to Ubuntu 18.04) and configure it by instructions in > DPDK doc page > (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>). > The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver > because the driver mlx4_core should be used for Mellanox NICs. Currently I am > using 2MB hugepages (number: 1024). > > I am very grateful if you can solve my problem. Thank you! > > Best, > Junzhi
