Re: [dpdk-users] Unable to bind intel NIC 82599ES to vfio-pci, uio_pci_generic
Thanks Ferruh, I figured my mistake immediately afterwards. I am able to work with uio_pci_generic and proceeding with my project pilot for now, have not spent much time on debugging vfio-pci issue. Here is one more try to bind vfio-pci, includes dmesg [root@unassigned dpdk-16.11]# dpdk-devbind --status Network devices using DPDK-compatible driver :01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=uio_pci_generic unused=ixgbe,igb_uio,vfio-pci Network devices using kernel driver === :02:00.0 'I210 Gigabit Network Connection' if=eno1 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic :03:00.0 'I210 Gigabit Network Connection' if=eno2 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic Other network devices = :01:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio,vfio-pci,uio_pci_generic Crypto devices using DPDK-compatible driver === Crypto devices using kernel driver == Other crypto devices [root@unassigned dpdk-16.11]# dpdk-devbind -b vfio-pci 01:00.1 Error: bind failed for :01:00.1 - Cannot bind to driver vfio-pci [root@unassigned dpdk-16.11]# dmesg | tail -n 10 [ 1989.451194] VFIO - User Level meta-driver version: 0.3 [16700.417269] VFIO - User Level meta-driver version: 0.3 [16708.620870] rte_kni: loading out-of-tree module taints kernel. [16708.621016] rte_kni: module verification failed: signature and/or required key missing - tainting kernel [16762.831711] igb_uio: Use MSIX interrupt by default [378754.897617] Intel(R) 10GbE PCI Express Linux Network Driver - version 4.5.4 [378754.897623] Copyright(c) 1999 - 2016 Intel Corporation. [491762.064211] Intel(R) 10GbE PCI Express Linux Network Driver - version 4.5.4 [491762.064213] Copyright(c) 1999 - 2016 Intel Corporation. [515594.211309] VFIO - User Level meta-driver version: 0.3 [root@unassigned dpdk-16.11]# ulimit -l unlimited dmesg for initialisation check [root@unassigned dpdk-16.11]# dmesg | grep DMAR [0.00] ACPI: DMAR 7d705e98 00070 (v01 INTEL SKL 0001 INTL 0001) [0.00] DMAR: IOMMU enabled [0.027917] DMAR: Host address width 39 [0.027918] DMAR: DRHD base: 0x00fed9 flags: 0x1 [0.027923] DMAR: dmar0: reg_base_addr fed9 ver 1:0 cap d2008c40660462 ecap f050da [0.027923] DMAR: RMRR base: 0x007d19 end: 0x007d1a [0.027925] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed9 IOMMU 0 [0.027926] DMAR-IR: HPET id 0 under DRHD base 0xfed9 [0.027927] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [0.027927] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [0.028288] DMAR-IR: Enabled IRQ remapping in xapic mode [0.511886] DMAR: No ATSR found [0.511942] DMAR: dmar0: Using Queued invalidation [0.511954] DMAR: Hardware identity mapping for device :00:00.0 [0.511955] DMAR: Hardware identity mapping for device :00:01.0 [0.511956] DMAR: Hardware identity mapping for device :00:14.0 [0.511957] DMAR: Hardware identity mapping for device :00:14.2 [0.511958] DMAR: Hardware identity mapping for device :00:16.0 [0.511959] DMAR: Hardware identity mapping for device :00:17.0 [0.511959] DMAR: Hardware identity mapping for device :00:1d.0 [0.511960] DMAR: Hardware identity mapping for device :00:1d.1 [0.511961] DMAR: Hardware identity mapping for device :00:1d.2 [0.511962] DMAR: Hardware identity mapping for device :00:1f.0 [0.511963] DMAR: Hardware identity mapping for device :00:1f.2 [0.511964] DMAR: Hardware identity mapping for device :00:1f.4 [0.511965] DMAR: Hardware identity mapping for device :01:00.0 [0.511966] DMAR: Hardware identity mapping for device :01:00.1 [0.511969] DMAR: Hardware identity mapping for device :02:00.0 [0.511971] DMAR: Hardware identity mapping for device :03:00.0 [0.511972] DMAR: Setting RMRR: [0.511973] DMAR: Ignoring identity map for HW passthrough device :00:14.0 [0x7d19 - 0x7d1a] [0.511974] DMAR: Prepare 0-16MiB unity mapping for LPC [0.511975] DMAR: Ignoring identity map for HW passthrough device :00:1f.0 [0x0 - 0xff] [0.511976] DMAR: Intel(R) Virtualization Technology for Directed I/O [root@unassigned dpdk-16.11]# On Mon, Feb 13, 2017 at 10:31 PM, Ferruh Yigit wrote: > On 2/8/2017 7:19 AM, Shyam Shrivastav wrote: > > Hi All > > > > I am not able to bind 82599 to either uio_pci_generic or vfio-pci > > successfully. Any help greatly appreciated, I am completely stuck at this > > initial step. > > > > *1) uio_pci_generic* : tools/dpdk-devbind.py script reports success but > it > > is not detected by EAL on initialisation, still ixgbe driver is detected. > > here is releva
Re: [dpdk-users] Is PF-only driver allowed to upstream into dpdk?
I am currently working on a non-nic mode user space PCI driver >> which processes custom packets from OCTEON 7xxx processor. As >> per the requirements, DPDK is the best way to achieve this. Though I have seen few designs and sources that talk about dpdk >> pf drivers, I am interested to know if such a driver with pf-only >> support and which doesn't involve virtual functions will be allowed >> to upstream into DPDK sources. There is no problem with PF only drivers, and there are already samples of it in the project. As long as driver is open source and for data path, it is good for upstream. Please feel free to upstream your driver when it is ready. The discussion about DPDK PF drivers was about if DPDK PF driver should be in control path or not, this is high level project scope discussion. It is about diverging with the Linux kernel's PF. It is not acceptable, from a VF point of view to have different behaviors. Then, about this specific request regarding Octeon's PF, your description is not clear enough to comment deeply about it. Best regards, Vincent
Re: [dpdk-users] Unable to bind intel NIC 82599ES to vfio-pci, uio_pci_generic
On 2/8/2017 7:19 AM, Shyam Shrivastav wrote: > Hi All > > I am not able to bind 82599 to either uio_pci_generic or vfio-pci > successfully. Any help greatly appreciated, I am completely stuck at this > initial step. > > *1) uio_pci_generic* : tools/dpdk-devbind.py script reports success but it > is not detected by EAL on initialisation, still ixgbe driver is detected. > here is relevant console output It is detected, but according following log ports are not enabled by proper application (not eal) command line argument: " Cause: All available ports are disabled. Please set portmask." ./examples/l2fwd/build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] -p PORTMASK: hexadecimal bitmask of ports to configure -q NQ: number of queue (=ports) per lcore (default is 1) -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default, 86400 maximum) --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) When enabled: - The source MAC address is replaced by the TX port MAC address - The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID <...> > > > > *2) vfio-pci :* Configured vfio permissions using setup.sh, bind script > reports error in this case as under Not sure about this one, below looks correct. Perhaps dmesg can tell more. <...> > > Thanks > Shyam >
[dpdk-users] KNI & SIOCSIFFLAGS permission?
Howdy, When using KNI, I'm trying to programmatically enable the port by bringing the interface up by calling ioctl with SIOCGIFFLAGS (thanks Dumitru Caera and Juniper's warp17 for great help getting KNI wrapped in PMD the right way!). However, kernel does not allow me to do this unless I am root or the executable has CAP_NET_ADMIN capability. Warp17 suggests to change user of the executable to root and give it SUI bit. Needless to say that configuring capabilities or running my process as root is a pain in the neck. So the question is how do you guys usually go about this? Also, since we already have access to rte_kni.ko, can we somehow add code there to enable us to bypass this kernel permission check just to make life easier? Thanks, Vlad ### The information contained in this communication is confidential, may be subject to legal privilege, and is intended only for the individual named. If you are not the named addressee, please notify the sender immediately and delete this email from your system. The views expressed in this email are the views of the sender only. Outgoing and incoming electronic communications to this address are electronically archived and subject to review and/or disclosure to someone other than the recipient. ###
[dpdk-users] Is Elastic Flow Distributor thread-safe?
Hi Everyone, Playing around with EFD and seeing some really bizarre behavior in a multi-core environment. Is it safe for multiple logical cores to be making EFD table updates while another logical core is performing lookups? Side note: the core which is making lookups is where the offline table resides. Intuition tells me this is probably not safe, but the docs make no mention of thread safety. Thanks, -Rob
Re: [dpdk-users] NIC NETXtreme II BCM57810 10GE
On 2/13/2017 10:03 AM, Avi Cohen (A) wrote: > HI, Hi Avi, > I don't find this in the list of supported NIC > So I get errors when setting it as dpdk type interface in the OVS > I understand that it is not supported by DPDK - correct ? List of supported NICs in DPDK: http://dpdk.org/doc/nics And for supported PCI dev IDs by Broadcom driver: http://dpdk.org/browse/dpdk/tree/drivers/net/bnxt/bnxt_ethdev.c#n95 Above I can't see the mentioned device, it looks like not supported as you state. Broadcom driver maintainer CC'ed for more accurate answer. Cc: Stephen Hurd > Regards avi >
Re: [dpdk-users] KNI drive without supported NICs?
On 2/6/2017 3:01 PM, Lazarenko, Vlad (WorldQuant) wrote: > Dumitru, > > This clear things up a big time! Much appreciated. Thank you! KNI PMD patch [1], which is not upstream yet, can be used to create any number of KNI interfaces independent from hardware. It is a virtual PMD implementation, so can be used via testpmd or any DPDK application. [1] http://dpdk.org/dev/patchwork/patch/20092/ > > - Vlad > >> -Original Message- >> From: Take Ceara [mailto:dumitru.ce...@gmail.com] >> Sent: Monday, February 06, 2017 6:05 AM >> To: Lazarenko, Vlad (WorldQuant) >> Cc: users@dpdk.org >> Subject: Re: [dpdk-users] KNI drive without supported NICs? >> >> Hi Vlad, >> >> On Sun, Feb 5, 2017 at 6:34 PM, Lazarenko, Vlad (WorldQuant) >> wrote: >>> Gentlemen and gentleladies, >>> >>> I am trying to get the KNI working on my dev box. So I've loaded the >> rte_kni.ko module and made sure /dev/kni has correct permissions. But >> when I am trying to run the KNI example, it fails with "No supported >> Ethernet device found" error: >>> >>> $ build/kni -n 4 -c 0xf0 -- -P -p 0x3 --config="(0,4,6),(1,5,7)" >>> EAL: Detected 24 lcore(s) >>> EAL: PCI device :01:00.0 on NUMA socket -1 >>> EAL: probe driver: 8086:10d3 net_e1000_em >>> EAL: Error - exiting with code: 1 >>> Cause: No supported Ethernet device found >>> >>> My dev box does not in fact have supported NICs. But my understanding >> about KNI was that it allows to access Linux network stack trough the KNI >> kernel module, where applications create a virtual Ethernet devices (i.e. >> /sys/class/net/vEth*), thus there is no need for a real supported NIC or >> taking over the entire NIC. >>> >>> On the other hand, the example application calls rte_eth_dev_count() >> first, which in my case returns 0, and exits. >>> >>> Is my understanding about what KNI actually is wrong? Or am I missing >> something else? >>> >> >> As far as I know the KNI sample app actually forwards traffic to/from the >> kernel and a physical port therefore the check for physical interfaces. If >> you >> want to use KNI for something else there's nothing stopping you from >> starting a DPDK app and adding a KNI interface without having a real NIC >> bound to DPDK. >> >> I don't have a better example right now but, if you want, you can take a look >> at how we do it in warp17 (using DPDK) when you start it without any >> physical interfaces: >> >> https://github.com/Juniper/warp17#using-kernel-network-interface-kni- >> interfaces >> >> If you look at the HTTP example there we actually start the application >> without any physical interfaces (only one KNI): >> >> ./build/warp17 -c FC3 -n 4 -m 32768 -- --kni-ifs 1 >> >> >>> Thanks, >>> Vlad >> >> Hope this helps.. >> >> Regards, >> Dumitru >> >>> >>> >>> >>> >> ### >> ### >>> # >>> >>> The information contained in this communication is confidential, may >>> be >>> >>> subject to legal privilege, and is intended only for the individual named. >>> >>> If you are not the named addressee, please notify the sender >>> immediately and >>> >>> delete this email from your system. The views expressed in this email >>> are >>> >>> the views of the sender only. Outgoing and incoming electronic >>> communications >>> >>> to this address are electronically archived and subject to review >>> and/or disclosure >>> >>> to someone other than the recipient. >>> >>> >> ### >> ### >>> # >> >> >> >> -- >> Dumitru Ceara > > > ### > > The information contained in this communication is confidential, may be > > subject to legal privilege, and is intended only for the individual named. > > If you are not the named addressee, please notify the sender immediately and > > delete this email from your system. The views expressed in this email are > > the views of the sender only. Outgoing and incoming electronic communications > > to this address are electronically archived and subject to review and/or > disclosure > > to someone other than the recipient. > > ### >
Re: [dpdk-users] Is PF-only driver allowed to upstream into dpdk?
On 2/13/2017 9:43 AM, Netala, Sameera wrote: > Hi, Hi Sameera, > > I am currently working on a non-nic mode user space PCI driver which > processes custom packets from OCTEON 7xxx processor. As per the requirements, > DPDK is the best way to achieve this. Can you please give more information about hardware? Is this an offload engine? > > Though I have seen few designs and sources that talk about dpdk pf drivers, I > am interested to know if such a driver with pf-only support and which doesn't > involve virtual functions will be allowed to upstream into DPDK sources. There is no problem with PF only drivers, and there are already samples of it in the project. As long as driver is open source and for data path, it is good for upstream. Please feel free to upstream your driver when it is ready. The discussion about DPDK PF drivers was about if DPDK PF driver should be in control path or not, this is high level project scope discussion. Thanks, ferruh > > Thanks, > Sameera >
[dpdk-users] NIC NETXtreme II BCM57810 10GE
HI, I don't find this in the list of supported NIC So I get errors when setting it as dpdk type interface in the OVS I understand that it is not supported by DPDK - correct ? Regards avi
[dpdk-users] Is PF-only driver allowed to upstream into dpdk?
Hi, I am currently working on a non-nic mode user space PCI driver which processes custom packets from OCTEON 7xxx processor. As per the requirements, DPDK is the best way to achieve this. Though I have seen few designs and sources that talk about dpdk pf drivers, I am interested to know if such a driver with pf-only support and which doesn't involve virtual functions will be allowed to upstream into DPDK sources. Thanks, Sameera
