Hi, Olga,
I think I have solved this problem.
libibverbs & libmlx4 can't use dpdp.org and can't compile with static linking.
I try use libibverbs-1.1.8mlnx1-OFED.2.4.45.ga305acd.src.rpm &
libmlx4-1.0.6mlnx1-OFED.2.4.0.1.2.src and compile with dynamic. This two
package in MLNX_OFED version 2.4-1.0.0 source package.
and I use ofed_script connectx_port_config config Card with Ethernet mode .
The testpmd was started with no error! :)
Thanks a million!! :)
Best Regards!
Kang
------------------ Original ------------------
From: "Arthas";<kangzy1982 at qq.com>;
Date: Mon, Apr 27, 2015 10:37 AM
To: "Olga Shern"<olgas at mellanox.com>; "dev"<dev at dpdk.org>;
Subject: Re: [dpdk-dev] DPDK v2.0 testpmd with Mellanox ConnectX-3problem!
3KS, Olga,
My MLNX_OFED version 2.4-1.0.0 and libibverbs & libmlx4 download from
dpdk.org.
when i try to run ibdev2netdev , it don't output any message! :(
Here is my environment.
# uname -r
3.10.0-mlx
# lsmod
mlx4_en 92824 0 - Live 0xffffffffa010f000 (O)
mlx4_ib 135033 0 - Live 0xffffffffa00e3000 (O)
ib_sa 24721 1 mlx4_ib, Live 0xffffffffa00d7000 (O)
ib_mad 31664 2 mlx4_ib,ib_sa, Live 0xffffffffa00ca000 (O)
mlx4_core 226516 2 mlx4_en,mlx4_ib, Live 0xffffffffa007c000 (O)
ib_uverbs 43309 0 - Live 0xffffffffa003f000 (O)
ib_core 79534 4 mlx4_ib,ib_sa,ib_mad,ib_uverbs, Live 0xffffffffa001d000 (O)
ib_addr 4273 2 ib_uverbs,ib_core, Live 0xffffffffa0017000 (O)
compat 4948 8 mlx4_en,mlx4_ib,ib_sa,ib_mad,mlx4_core,ib_uverbs,ib_core,ib_addr,
Live 0xffffffffa0011000 (O)
# mstflint -d 02:00.0 q
Image type: FS2
FW Version: 2.33.5100
FW Release Date: 25.1.2015
Product Version: 02.33.51.00
Rom Info: type=PXE version=3.4.460 devid=4099
Device ID: 4099
Description: Node Port1 Port2 Sys image
GUIDs: 0002c90300faa480 0002c90300faa481 0002c90300faa482
0002c90300faa483
MACs: 0002c9faa481 0002c9faa482
VSD:
PSID: MT_1090110019
# ibv_devices
libibverbs: Warning: dlopen(NULL) failed, assuming static linking.
libibverbs: Warning: no userspace device-specific driver found for
/sys/class/infiniband_verbs/uverbs0
When linking libibverbs statically, driver must be statically linked
too.
device node GUID
------ ----------------
# ibv_devinfo -l
libibverbs: Warning: dlopen(NULL) failed, assuming static linking.
libibverbs: Warning: no userspace device-specific driver found for
/sys/class/infiniband_verbs/uverbs0
When linking libibverbs statically, driver must be statically linked
too.
0 HCAs found:
/sys/class/infiniband_verbs # ls
abi_version uverbs0
# find . -name "mlx4*"
./proc/irq/148/mlx4-32 at 0000:02:00.0
./proc/irq/147/mlx4-31 at 0000:02:00.0
./proc/irq/146/mlx4-30 at 0000:02:00.0
./proc/irq/145/mlx4-29 at 0000:02:00.0
./proc/irq/144/mlx4-28 at 0000:02:00.0
./proc/irq/143/mlx4-27 at 0000:02:00.0
./proc/irq/142/mlx4-26 at 0000:02:00.0
./proc/irq/141/mlx4-25 at 0000:02:00.0
./proc/irq/140/mlx4-24 at 0000:02:00.0
./proc/irq/139/mlx4-23 at 0000:02:00.0
./proc/irq/138/mlx4-22 at 0000:02:00.0
./proc/irq/137/mlx4-21 at 0000:02:00.0
./proc/irq/136/mlx4-20 at 0000:02:00.0
./proc/irq/135/mlx4-19 at 0000:02:00.0
./proc/irq/134/mlx4-18 at 0000:02:00.0
./proc/irq/133/mlx4-17 at 0000:02:00.0
./proc/irq/132/mlx4-16 at 0000:02:00.0
./proc/irq/131/mlx4-15 at 0000:02:00.0
./proc/irq/130/mlx4-14 at 0000:02:00.0
./proc/irq/129/mlx4-13 at 0000:02:00.0
./proc/irq/128/mlx4-12 at 0000:02:00.0
./proc/irq/127/mlx4-11 at 0000:02:00.0
./proc/irq/126/mlx4-10 at 0000:02:00.0
./proc/irq/125/mlx4-9 at 0000:02:00.0
./proc/irq/124/mlx4-8 at 0000:02:00.0
./proc/irq/123/mlx4-7 at 0000:02:00.0
./proc/irq/122/mlx4-6 at 0000:02:00.0
./proc/irq/121/mlx4-5 at 0000:02:00.0
./proc/irq/120/mlx4-4 at 0000:02:00.0
./proc/irq/119/mlx4-3 at 0000:02:00.0
./proc/irq/118/mlx4-2 at 0000:02:00.0
./proc/irq/117/mlx4-1 at 0000:02:00.0
./proc/irq/116/mlx4-async at pci:0000:02:00.0
./proc/driver/mlx4_ib
./sys/bus/pci/drivers/mlx4_core
./sys/devices/pci0000:00/0000:00:02.0/0000:02:00.0/mlx4_port1_mtu
./sys/devices/pci0000:00/0000:00:02.0/0000:02:00.0/mlx4_port2_mtu
./sys/devices/pci0000:00/0000:00:02.0/0000:02:00.0/infiniband/mlx4_0
./sys/devices/pci0000:00/0000:00:02.0/0000:02:00.0/mlx4_port1
./sys/devices/pci0000:00/0000:00:02.0/0000:02:00.0/mlx4_port2
./sys/class/infiniband/mlx4_0
./sys/module/ib_sa/holders/mlx4_ib
./sys/module/compat/holders/mlx4_en
./sys/module/compat/holders/mlx4_ib
./sys/module/compat/holders/mlx4_core
./sys/module/mlx4_en
./sys/module/mlx4_ib
./sys/module/ib_mad/holders/mlx4_ib
./sys/module/ib_core/holders/mlx4_ib
./sys/module/mlx4_core
./sys/module/mlx4_core/holders/mlx4_en
./sys/module/mlx4_core/holders/mlx4_ib
./lib/modules/3.10.0-mlx/extra/mlnx-ofa_kernel/drivers/net/ethernet/mellanox/mlx4
./lib/modules/3.10.0-mlx/extra/mlnx-ofa_kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko
./lib/modules/3.10.0-mlx/extra/mlnx-ofa_kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko
./lib/modules/3.10.0-mlx/extra/mlnx-ofa_kernel/drivers/infiniband/hw/mlx4
./lib/modules/3.10.0-mlx/extra/mlnx-ofa_kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko
./sdd/tmp/mlx4.driver
./sdd/tmp/mlx4
./sdd/tmp/mlx4/mlx4.hw
./sdd/tmp/mlx4/mlx4.config
./sdd/tmp/mlx4/mlx4.img.2.10.700
./sdd/tmp/etc/modprobe.d/mlx4.conf
./etc/libibverbs.d/mlx4.driver
./etc/modprobe.d/mlx4.conf
------------------ Original ------------------
From: "Olga Shern";<olgas at mellanox.com>;
Date: Sun, Apr 26, 2015 03:28 PM
To: "Arthas"<kangzy1982 at qq.com>; "dev"<dev at dpdk.org>;
Subject: RE: RE: [dpdk-dev] DPDK v2.0 testpmd with Mellanox ConnectX-3problem!
Hi Kang,
Sorry for the late response.
ConnectX-3 VPI card that you have is supported. So there shouldn?t be any
problem.
Please make sure to follow the instructions in
http://www.dpdk.org/doc/guides/nics/mlx4.html
What MLNX_OFED did you install?
What is the output of ibdev2netdev?
Make sure, mlx4_core parameter log_num_mgm_entry_size = -1
Best Regards,
Olga
From: Arthas [mailto:kangzy1...@qq.com]
Sent: Wednesday, April 22, 2015 6:56 AM
To: Olga Shern; dev
Subject: Re: RE: [dpdk-dev] DPDK v2.0 testpmd with Mellanox ConnectX-3 problem!
3KS, Olga,
I try you suggestion and it can't start yet. My Mellanox ConnectX-3 card is
"ConnectX?-3 VPI adapter card, dual-port QSFP, FDR IB (56Gb/s) and 40/56GbE,
PCIe3.0 x8 8GT/s, tall bracket,RoHS R6". May be the problem?
# mstflint -d 02:00.0 q
Image type: FS2
FW Version: 2.33.5100
FW Release Date: 25.1.2015
Product Version: 02.33.51.00
Rom Info: type=PXE version=3.4.460 devid=4099
Device ID: 4099
Description: Node Port1 Port2 Sys image
GUIDs: 0002c90300faa480 0002c90300faa481 0002c90300faa482
0002c90300faa483
MACs: 0002c9faa481 0002c9faa482
VSD:
PSID: MT_1090110019
Best Regards,
Kang
------------------ Original ------------------
From: "Olga Shern";<ol...@mellanox.com>;
Date: Tue, Apr 21, 2015 09:34 PM
To: "Arthas"<kangzy1982 at qq.com>; "dev"<dev at dpdk.org>;
Subject: RE: [dpdk-dev] DPDK v2.0 testpmd with Mellanox ConnectX-3 problem!
Hi Kang,
You probably compiled the downloaded libraries using dynamic linkage, prior to
the static one you specified here.
You need to remove them from /usr/local/libs and also please delete
/usr/local/include/infiniband
Let me know if it solves your problem.
Best Regards,
Olga
-----Original Message-----
From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of ZhanYing Kang
Sent: Tuesday, April 21, 2015 11:45 AM
To: dev
Subject: [dpdk-dev] DPDK v2.0 testpmd with Mellanox ConnectX-3 problem!
When I'm run testpmd with DPDK 2.0 and the Mellanox ConnectX-3. I get a error.
Here are my questions:
1> lspci |grep Mell
02:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]
2> compile libibverbs-1.1.7mlnx1 && libmlx4-1.0.5mlnx1 export
EXTRA_CFLAGS=-I/DPDK/mlx4/install/usr/local/include
export EXTRA_LDFLAGS=-L/DPDK/mlx4/install/usr/local/lib
3> compile dpdk v2.0 with x86_64-native-linuxapp-gcc config and
3> CONFIG_RTE_LIBRTE_MLX4_PMD=y
4> modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib lsmod mlx4_ib 135033 0 -
Live 0xffffffffa0215000 (O) ib_sa 24721 1 mlx4_ib, Live 0xffffffffa0209000 (O)
ib_mad 31664 2 mlx4_ib,ib_sa, Live 0xffffffffa01fc000 (O) mlx4_en 92808 0 -
Live 0xffffffffa01b1000 (O) mlx4_core 226516 2 mlx4_ib,mlx4_en, Live
0xffffffffa0163000 (O) ib_uverbs 43309 0 - Live 0xffffffffa0133000 (O) ib_core
79534 4 mlx4_ib,ib_sa,ib_mad,ib_uverbs, Live 0xffffffffa0100000 (O) ib_addr
4273 2 ib_uverbs,ib_core, Live 0xffffffffa00fa000 (O) compat 4948 8
mlx4_ib,ib_sa,ib_mad,mlx4_en,mlx4_core,ib_uverbs,ib_core,ib_addr, Live
0xffffffffa00ad000 (O) 5> run testpmd
# ./testpmd-mlx -c 0xff00 -n 4 -w 0000:02:00.0 -- --rxq=2 --txq=2 -i
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 0
EAL: Detected lcore 9 as core 1 on socket 0
EAL: Detected lcore 10 as core 2 on socket 0
EAL: Detected lcore 11 as core 3 on socket 0
EAL: Detected lcore 12 as core 4 on socket 0
EAL: Detected lcore 13 as core 5 on socket 0
EAL: Detected lcore 14 as core 6 on socket 0
EAL: Detected lcore 15 as core 7 on socket 0
EAL: Support maximum 32 logical core(s) by configuration.
EAL: Detected 16 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f5a2b800000 (size = 0x200000)
EAL: Ask a virtual area of 0x31000000 bytes
EAL: Virtual area found at 0x7f59fa600000 (size = 0x31000000)
EAL: Ask a virtual area of 0x7ec00000 bytes
EAL: Virtual area found at 0x7f597b800000 (size = 0x7ec00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f597b400000 (size = 0x200000)
EAL: Ask a virtual area of 0x250000000 bytes
EAL: Virtual area found at 0x7f572b200000 (size = 0x250000000)
EAL: Requesting 6144 pages of size 2MB from socket 0
EAL: TSC frequency is ~2394227 KHz
EAL: Master lcore 8 is ready (tid=2e1b7900;cpuset=[8])
EAL: lcore 11 is ready (tid=299b7700;cpuset=[11])
EAL: lcore 14 is ready (tid=281b4700;cpuset=[14])
EAL: lcore 15 is ready (tid=279b3700;cpuset=[15])
EAL: lcore 12 is ready (tid=291b6700;cpuset=[12])
EAL: lcore 9 is ready (tid=2caaf700;cpuset=[9])
EAL: lcore 10 is ready (tid=2c2ae700;cpuset=[10])
EAL: lcore 13 is ready (tid=289b5700;cpuset=[13])
EAL: PCI device 0000:02:00.0 on NUMA socket -1
EAL: probe driver: 15b3:1003 librte_pmd_mlx4
PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_0" (VF: false)
PMD: librte_pmd_mlx4: 2 port(s) detected
PMD: librte_pmd_mlx4: bad state for port 1: "down" (1)
PMD: librte_pmd_mlx4: port 1 MAC address is 02:02:c9:fa:a4:81
PMD: librte_pmd_mlx4: bad state for port 2: "down" (1)
PMD: librte_pmd_mlx4: port 2 MAC address is 00:02:c9:fa:a4:82 Interactive-mode
selected Configuring Port 0 (socket 0)
PMD: librte_pmd_mlx4: 0x7dffe0: TX queues number update: 0 -> 2
PMD: librte_pmd_mlx4: 0x7dffe0: RX queues number update: 0 -> 2
PMD: librte_pmd_mlx4: 0x7fffc81e67e0: flow configuration failed, errno=38:
Function not implemented
PMD: librte_pmd_mlx4: 0x7fffc81e67e0: flow configuration failed, errno=38:
Function not implemented
PMD: librte_pmd_mlx4: 0x7dffe0: QP flow attachment failed: Function not
implemented Fail to configure port 0
EAL: Error - exiting with code: 1
Cause: Start ports failed
Best,
Kang!
