[dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs
Just an update on the issue I was facing... I got what was wrong in the steps which I followed. I shouldn't have bound the VNICs to igb_uio driver after replacing vmxnet3 driver. The testpdm app worked fine after eliminating that step from the earlier sequence :) Sorry to bother you all. Regards, Sandesh From: Sandesh Sawant Sent: Friday, November 21, 2014 7:06 PM To: 'dev at dpdk.org' Subject: Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs Hello dpdk experts, I am following the documentation available on dpdk.org & the documentation published by Intel to learn how to use DPDK libraries. I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app provided with this release with 2 VMXNET3 vNICs on my setup. My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440. I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all commands run as a root user): cd /home/sandesh/dpdk/dpdk-1.7.1/ export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/ export RTE_TARGET=build modprobe uio insmod build/kmod/igb_uio.ko modprobe vfio-pci mkdir -p /mnt/huge mount -t hugetlbfs nodev /mnt/huge echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver Network devices using kernel driver === :02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active* :03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* :0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* Other network devices = Then I bind VMXNET3 vNICs to igb_uio driver: ../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1 ../dpdk-1.7.1/tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver Network devices using kernel driver === :02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active* :03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* :0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* Other network devices = Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work with VMXNET3. I got a few compilation errors for the same which I resolved and got the driver & .so built. (I belive these errors in driver code were due to its code not being uptodate wrt linux kernel 3.13). Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko as follows: cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/ rmmod vmxnet3 insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512 and then executed: ../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i --rxq=1 --rxd=512 --txd=512 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 1 on socket 0 EAL: Detected lcore 6 as core 2 on socket 0 EAL: Detected lcore 7 as core 3 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 8 lcore(s) EAL: unsupported IOMMU type! EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Ask a virtual area of 0x580 bytes EAL: Virtual area found at 0x7fb8f700 (size = 0x580) EAL: Ask a virtual area of 0x40 bytes EAL: Virtual area found at 0x7fb8f6a0 (size = 0x40) EAL: Ask a virtual area of 0x240 bytes EAL: Virtual area found at 0x7fb8f440 (size = 0x240) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~241 KHz EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so librte_pmd_vmxnet3 version 1.2 Copyright 2012-2014 6WIND S.A. provided without warranty. EAL: Master core 0 is ready (tid=fd716880) EAL: Core 7 is ready (tid=f09ed700) EAL: Core 6 is ready (tid=f11ee700) EAL: Core 5 is ready (tid=f19ef700) EAL: Core 4 is ready (tid=f21f0700) EAL: Core 3 is ready (tid=f29f1700) EAL: Core 2 is ready (tid=f31f2700) EAL: Core 1 is ready (tid=f39f3700) EAL: PCI device :02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: :02:01.0 not managed by UIO driver, skipping EAL: PCI device :03:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd PMD: eth_vmxnet3_dev_init(): No matching device found EAL: Error - exiting with code: 1 Cause: Requested device :03:00.0 cannot be used root at pnq1-v
[dpdk-dev] Need help with the correct steps to run testpmd app on Linux VM using VMXNET3 vNICs
Hello dpdk experts, I am following the documentation available on dpdk.org & the documentation published by Intel to learn how to use DPDK libraries. I am using dpdk v1.7.1 downloaded from dpdk.org. I wish to run testpmd app provided with this release with 2 VMXNET3 vNICs on my setup. My test setup is a 64-bit Ubunutu 14.04 (3.13.0) Linux VM running on a VMWare ESXi-5.5.0 hypervisor on Dell server with Intel Xeon E5-2440. I followed the following stpes to capture the vmxnet3 vNICs in DPDK (all commands run as a root user): cd /home/sandesh/dpdk/dpdk-1.7.1/ export RTE_SDK=/home/sandesh/dpdk/dpdk-1.7.1/ export RTE_TARGET=build modprobe uio insmod build/kmod/igb_uio.ko modprobe vfio-pci mkdir -p /mnt/huge mount -t hugetlbfs nodev /mnt/huge echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver Network devices using kernel driver === :02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active* :03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* :0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* Other network devices = Then I bind VMXNET3 vNICs to igb_uio driver: ../dpdk-1.7.1/tools/dpdk_nic_bind.py --bind=igb_uio eth0 eth1 ../dpdk-1.7.1/tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver Network devices using kernel driver === :02:01.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio,vfio-pci *Active* :03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* :0b:00.0 'VMXNET3 Ethernet Controller' if=eth1 drv=vmxnet3 unused=igb_uio,vfio-pci *Active* Other network devices = Note: I downloaded vmxnet3-usermap extension 1.2 source from dpdk.org to work with VMXNET3. I got a few compilation errors for the same which I resolved and got the driver & .so built. (I belive these errors in driver code were due to its code not being uptodate wrt linux kernel 3.13). Next I proceeded to replace the orignial vmxnet3 driver with vmxnet3-usermap.ko as follows: cd /home/sandesh/dpdk/vmxnet3-usermap-1.2/ rmmod vmxnet3 insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512 and then executed: ../dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -d librte_pmd_vmxnet3.so -- -i --rxq=1 --rxd=512 --txd=512 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 1 on socket 0 EAL: Detected lcore 6 as core 2 on socket 0 EAL: Detected lcore 7 as core 3 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 8 lcore(s) EAL: unsupported IOMMU type! EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Ask a virtual area of 0x580 bytes EAL: Virtual area found at 0x7fb8f700 (size = 0x580) EAL: Ask a virtual area of 0x40 bytes EAL: Virtual area found at 0x7fb8f6a0 (size = 0x40) EAL: Ask a virtual area of 0x240 bytes EAL: Virtual area found at 0x7fb8f440 (size = 0x240) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~241 KHz EAL: open shared lib /home/sandesh/dpdk/vmxnet3-usermap-1.2/librte_pmd_vmxnet3.so librte_pmd_vmxnet3 version 1.2 Copyright 2012-2014 6WIND S.A. provided without warranty. EAL: Master core 0 is ready (tid=fd716880) EAL: Core 7 is ready (tid=f09ed700) EAL: Core 6 is ready (tid=f11ee700) EAL: Core 5 is ready (tid=f19ef700) EAL: Core 4 is ready (tid=f21f0700) EAL: Core 3 is ready (tid=f29f1700) EAL: Core 2 is ready (tid=f31f2700) EAL: Core 1 is ready (tid=f39f3700) EAL: PCI device :02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: :02:01.0 not managed by UIO driver, skipping EAL: PCI device :03:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd PMD: eth_vmxnet3_dev_init(): No matching device found EAL: Error - exiting with code: 1 Cause: Requested device :03:00.0 cannot be used root at pnq1-vm-eng-dhcp13:/home/sandesh/dpdk/vmxnet3-usermap-1.2# Can someone please point out what I am missing here? Am I supposed to bind VMXNET3 vNICs to the vmxnet3-usermap driver instead of igb_uio driver here? Note: Earlier when I was not running testpmd without vmxnet3-usermap extension, when I was hitting the following error: dpdk-1.7.1/build/app/testpmd -c 0xff -n 1 -- -i . . . EAL: PCI device :0b:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd EAL: PCI
