Priyanka, Please, make sure you have "intel_iommu=on" and "iommu=pt" in your boot grub configuration.
Also, if you're using (as I presume but you didn't confirmed) a NUMA machine, remove any configuration about hugepages from the boot grub configuration and follow the instructions I gave you earlier. Then, reboot your machine and do not use uio_pci_generic but rather vfio-pci (edit the startup.conf: change the uio_driver used and modprobe vfio-pci instead of uio_pci_generic). Also, please, if things still do not work share: 1) VPP version 2) Content of "startup.conf" 3) Output of "cat /proc/cmdline" On Thu, 2018-09-06 at 13:12 +0530, ppnaik wrote: > Hi Marco, > > The NIC is binded to DPDK driver: devbing.py output: > > Network devices using DPDK-compatible driver > ============================================ > 0000:01:00.1 'I350 Gigabit Network Connection 1521' drv=uio_pci_generic > unused=vfio-pci > 0000:81:00.0 'Ethernet Controller 10-Gigabit X540-AT2 1528' > drv=uio_pci_generic unused=vfio-pci > 0000:81:00.1 'Ethernet Controller 10-Gigabit X540-AT2 1528' > drv=uio_pci_generic unused=vfio-pci > > Network devices using kernel driver > =================================== > 0000:01:00.0 'I350 Gigabit Network Connection 1521' if=enp1s0f0 drv=igb > unused=vfio-pci,uio_pci_generic *Active* > > Other Network devices > ===================== > <none> > > Crypto devices using DPDK-compatible driver > =========================================== > <none> > > Crypto devices using kernel driver > ================================== > <none> > > Other Crypto devices > ==================== > <none> > > Eventdev devices using DPDK-compatible driver > ============================================= > <none> > > Eventdev devices using kernel driver > ==================================== > <none> > > Other Eventdev devices > ====================== > <none> > > Mempool devices using DPDK-compatible driver > ============================================ > <none> > > Thanks, > Priyanka > > > On 2018-09-06 12:56, Marco Varlese wrote: > > On Thu, 2018-09-06 at 11:23 +0530, Avinash Dhar Dubey wrote: > > > Arijit, > > > Share the output of "cat /proc/meminfo | grep -i huge" and "free > > > -h". > > > You are using 1GB hugepages and you have not configured them. > > > Better use 2MB hugepages if you are running vpp just for learning. > > > > > > Also share the contents of "/etc/default/grub". Looks like you have > > > set 1GB as > > > default hugepage size. > > > > Actually the output log is the complete opposite!!!That log is simply > > stating > > that you have _NOT_ reserved 1GB hugepages but rather the 2MB ones. > > See below > > for the actual "problem"... > > > Regards, > > > Avinash > > > > > > On Thu, Sep 6, 2018 at 4:41 PM, Arijit Pramanik > > > <[email protected]> wrote: > > > > I have added the plugin path as /usr/lib/vpp/vpp_plugins and > > > > > > enabled only > > > > dpdk_plugin.so > > > > > > > > But I'm getting the error currently as > > > > > > > > Sep 06 16:30:54 turing-04 vpp[2339]: EAL: No free hugepages > > > > > > reported in > > > > hugepages-1048576kB > > > > Sep 06 16:30:54 turing-04 vpp[2339]: EAL: VFIO support initialized > > > > Sep 06 16:30:54 turing-04 vnet[2339]: EAL: VFIO support > > > > > > initialized > > > > Sep 06 16:30:54 turing-04 vpp[2339]: EAL: Requested device > > > > > > 0000:81:00.1 > > > > cannot be used > > > > Sep 06 16:30:54 turing-04 vnet[2339]: EAL: Requested device > > > > > > 0000:81:00.1 > > > > cannot be used > > > > Sep 06 16:30:55 turing-04 vnet[2339]: dpdk_pool_create:550: > > > > ioctl(VFIO_IOMMU_MAP_DMA) pool 'dpdk_mbuf_pool_socket0': Invalid > > > > > > argument > > > > (errno 22) > > > > Sep 06 16:30:55 turing-04 vnet[2339]: dpdk_pool_create: failed to > > > > > > create > > > > dpdk_mbuf_pool_socket1 > > > > Sep 06 16:30:55 turing-04 vnet[2339]: dpdk_buffer_pool_create:616: > > > > > > WARNING: > > > > Failed to allocate mempool for CPU socket 1. Threads running on > > > > > > socket 1 > > > > will use socket 0 mempool. > > > > I suppose you're running on a NUMA machine? Apparently the hugepages > > allocation > > has run out on one of the two socket and it's _warning_ you that > > memory will be > > taken from the other one. It's warning you because it's a performance > > hit to > > access remote-node memory...Again this is a _warning_ not an error. > > Having said > > all that, why don't you follow this page (DPDK) on how to reserve > > hugepages for > > NUMA > > machines?https://doc.dpdk.org/guides/linux_gsg/sys_reqs.htmlSomething > > like:echo 1024 > > > > > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages > > echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages- > > 2048kB/nr_hugepages > > > > Sep 06 16:30:55 turing-04 vnet[2339]: dpdk_ipsec_process:1018: not > > > > > > enough > > > > DPDK crypto resources, default to OpenSSL > > > > Sep 06 16:30:55 turing-04 vnet[2339]: dpdk_lib_init:230: DPDK > > > > > > drivers found > > > > no ports... > > > > You have not binded any port to be used by DPDK... again, take a look > > at this > > webpagehttps://doc.dpdk.org/guides/tools/devbind.htmlSomething like > > dpdk-devbind > > --bind=vfio-pci PCI_BAR_ADDRESSwill map your port (identified by > > PCI_BAR_ADDRESS) to the vfio-pci driver. > > > > Thanks, > > > > Arijit. > > > > - Marco > > > > > > > > On Thu, Sep 6, 2018 at 9:44 AM Avinash Dhar Dubey > > > > > > <[email protected] > > > > > wrote: > > > > > Arijit, > > > > > > > > > > > > > > > We are using Intel X540 NIC. > > > > > > > Intel X540 NIC is supported by DPDK. > > > > > > > > > > > > > > > > > > > > I'll try with one of them. Which one do you think is best suited > > > > > > for my > > > > > NIC, How do I ensure that the driver module is loaded in the > > > > > > kernel > > > > > beforehand? > > > > > > > > > > > > > > > > > You can use uio_pci_generic driver. To ensure that the driver > > > > > > is loaded > > > > > before you start vpp, you can run "modprobe uio_pci_generic". It > > > > > > will load > > > > > the driver if its already present as part of your kernel > > > > > > distribution. If > > > > > not found you can get it by installing > > > > > > linux-image-extra-<kernel-version>- > > > > > generic package. > > > > > > > > > > > > > > > > > > > > We want vpp with dpdk plugin. I've just installed vpp using > > > > > > apt-get > > > > > following the steps on this link : > > > > > > > > > > > https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages. > > > > > Does DPDK have to be spearely installed to be able to use the > > > > > > dpdk plugin? > > > > > > > No, when you compile vpp it creates dpdk_plugin.so which > > > > > > gets loaded > > > > > if you have specified the plugin_path in startup.conf > > > > > > > > > > Hope this helps. > > > > > > > > > > Avinash > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 6, 2018 at 1:05 AM, Arijit Pramanik > > > > > > <[email protected]> > > > > > wrote: > > > > > > Hi Marco, > > > > > > Thanks a lot for your prompt reply and assistance. > > > > > > > > > > > > We are using Intel X540 NIC. > > > > > > > > > > > > I'll try with one of them. Which one do you think is best > > > > > > suited for my > > > > > > NIC, How do I ensure that the driver module is loaded in the > > > > > > kernel > > > > > > beforehand? > > > > > > > > > > > > We want vpp with dpdk plugin. I've just installed vpp using > > > > > > apt-get > > > > > > following the steps on this link : > > > > > > > > > > > > https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages. > > > > > > Does DPDK have to be spearely installed to be able to use the > > > > > > dpdk > > > > > > plugin? > > > > > > > > > > > > Thanks a lot, > > > > > > Arijit. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 5, 2018 at 8:30 PM Marco Varlese > > > > > > <[email protected]> wrote: > > > > > > > Arijit, > > > > > > > > > > > > > > Few questions: > > > > > > > 1) What NIC is your 10G NIC? > > > > > > > 2) Your configuration is wrong: > > > > > > > ixgbe is not a uio-driver... you should choose between > > > > > > vfio-pci, uio- > > > > > > > pci-generic or igb_uio (and make sure to have that driver > > > > > > module > > > > > > > loaded in the kernel beforehand). > > > > > > > 3) I assume you're trying to use VPP with DPDK plugin > > > > > > enabled. Can you > > > > > > > confirm? > > > > > > > > > > > > > > > > > > > > > - Marco > > > > > > > > > > > > > > On Wed, 2018-09-05 at 23:18 +0530, Arijit Pramanik wrote: > > > > > > > > ng VPP with DPDK on our intel Xeon 24 core server. We have > > > > > > installed > > > > > > > > VPP and our 1G card was visible when > > > > > > > > $ sudo vppctl show interface > > > > > > > > > > > > > > -- > > > > > > > Marco V > > > > > > > > > > > > > > > > > > > > > SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, > > > > > > Graham Norton > > > > > > > HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg > > > > > > > > > > > > > > > > > > On Wed, Sep 5, 2018 at 8:30 PM Marco Varlese > > > > > > <[email protected]> wrote: > > > > > > > Arijit, > > > > > > > > > > > > > > Few questions: > > > > > > > 1) What NIC is your 10G NIC? > > > > > > > 2) Your configuration is wrong: > > > > > > > ixgbe is not a uio-driver... you should choose between > > > > > > vfio-pci, uio- > > > > > > > pci-generic or igb_uio (and make sure to have that driver > > > > > > module > > > > > > > loaded in the kernel beforehand). > > > > > > > 3) I assume you're trying to use VPP with DPDK plugin > > > > > > enabled. Can you > > > > > > > confirm? > > > > > > > > > > > > > > > > > > > > > - Marco > > > > > > > > > > > > > > On Wed, 2018-09-05 at 23:18 +0530, Arijit Pramanik wrote: > > > > > > > > ng VPP with DPDK on our intel Xeon 24 core server. We have > > > > > > installed > > > > > > > > VPP and our 1G card was visible when > > > > > > > > $ sudo vppctl show interface > > > > > > > > > > > > > > -- > > > > > > > Marco V > > > > > > > > > > > > > > > > > > > > > SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, > > > > > > Graham Norton > > > > > > > HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > Links: You receive all messages sent to this group. > > > > > > > > > > > > > > > > > > > > > > > > View/Reply Online (#10388): > > > > > > https://lists.fd.io/g/vpp-dev/message/10388 > > > > > > > > > > > > Mute This Topic: https://lists.fd.io/mt/25209728/675719 > > > > > > > > > > > > Group Owner: [email protected] > > > > > > > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub > > > > > > [[email protected]] > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > -- Marco V SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#10398): https://lists.fd.io/g/vpp-dev/message/10398 Mute This Topic: https://lists.fd.io/mt/25209728/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
