Hi Olga, Thank you very much! It works after I changing log_num_mgm_entry_size=-1!
Junzhi > On Nov 1, 2018, at 12:52 PM, Olga Shern <[email protected]> wrote: > > Hi Junzhi, > > Can you please change log_num_mgm_entry_size=-1 > Can you please run testpmd and specify only pf 04:00.0 > > Thanks, > Olga > > > -----Original Message----- > From: users [mailto:[email protected] <mailto:[email protected]>] > On Behalf Of Junzhi Gong > Sent: Thursday, November 1, 2018 6:45 PM > To: [email protected] <mailto:[email protected]> > Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC > > Hi everyone, > > I am sorry that I did not give many examples and details last time. > > The Mellanox NIC is > > Device #1: > ---------- > > Device Type: ConnectX3Pro > Part Number: MCX313A-BCC_Ax > Description: ConnectX-3 Pro EN network interface card; 40GigE; > single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6 > PSID: MT_1060111023 > PCI Device Name: /dev/mst/mt4103_pci_cr0 > Port1 MAC: ec0d9a07dc90 > Port2 MAC: ec0d9a07dc91 > Versions: Current Available > FW 2.42.5000 N/A > PXE 3.4.0752 N/A > > The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at > them like this: > > $ lspci | grep Mellanox > 04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 > Pro] > 04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > 04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family > [ConnectX-3/ConnectX-3 Pro Virtual Function] > > All port is set to Ethernet Adapter: > > $ connectx_port_config -s > -------------------------------- > Port configuration for PCI device: 0000:04:00.0 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.1 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.2 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.3 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.4 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.5 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.6 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:00.7 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:01.0 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:01.1 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:01.2 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:01.3 is: > eth > -------------------------------- > -------------------------------- > Port configuration for PCI device: 0000:04:01.4 is: > eth > -------------------------------- > > In file /etc/modprobe.d/mlx4_core.conf: > > options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 > probe_vf=12 > > My DPDK version is 18.08, and in file /config/common_base, I set: > > CONFIG_RTE_LIBRTE_MLX4_PMD=y > > After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib > > I run several DPDK applications, and I got the same error message as follows. > > Thank you! > > Junzhi > > >> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <[email protected]> wrote: >> >> Hi everyone, >> >> Currently I have some trouble running DPDK applications using Mellanox NIC. >> All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will >> crash with the following error message: >> >> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, >> "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, >> message: flow rule rejected by device >> EAL: Error - exiting with code: 1 >> Cause: rte_eth_dev_start:err=-12, port=0 >> >> An example running click has the following error messages: >> >> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click >> >> EAL: Detected 12 lcore(s) >> EAL: Detected 1 NUMA nodes >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >> EAL: No free hugepages reported in hugepages-1048576kB >> EAL: Probing VFIO support... >> EAL: PCI device 0000:04:00.1 on NUMA socket 0 >> EAL: probe driver: 15b3:1004 net_mlx4 >> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: >> true) >> PMD: net_mlx4: 1 port(s) detected >> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing >> DPDK >> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, >> "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, >> message: flow rule rejected by device >> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’: >> Cannot start DPDK port 0: error -12 >> >> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried >> several different DPDK applications (all applications have been re-compiled >> using the new configured DPDK), and I have also tried reboot the OS, but the >> problem still exists. >> >> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family >> [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu >> 18.04. I have installed the Mellanox OFED (version 4.4, only the newest >> version is compatible to Ubuntu 18.04) and configure it by instructions in >> DPDK doc page >> (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 >> >> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0> >> >> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 >> >> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>>). >> The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver >> because the driver mlx4_core should be used for Mellanox NICs. Currently I >> am using 2MB hugepages (number: 1024). >> >> I am very grateful if you can solve my problem. Thank you! >> >> Best, >> Junzhi
