Pavel, No, the patches to the driver are supplied by the Intel field teams that you buy these NIC's from. It's the only way that the enablement code is distributed.
Thanks for your understanding. Cheers, John > -----Original Message----- > From: Pavel Odintsov [mailto:pavel.odint...@gmail.com] > Sent: Friday, June 19, 2015 9:04 AM > To: Ronciak, John > Cc: Skidmore, Donald C; e1000-devel@lists.sourceforge.net > Subject: Re: [E1000-devel] VF inside VM with bypassed PCI-E ixgbe > > Hello, folks! > > Thanks for answer! Really important information! I will ask my equipment > provider. > > But could I ask. Do you have this drivers in open source somewhere? > > On Fri, Jun 19, 2015 at 6:42 PM, Ronciak, John <john.ronc...@intel.com> > wrote: > > In addition Pavel you should go back to the Intel Field person you got the > NIC's from to get the enabling code. Most people who buy these NIC's are > building their own kernels which have the modified driver which includes this > code. That code can't be pushed upstream as there is no generic > kernel/driver interface for such a feature. > > > > Cheers, > > John > > > >> -----Original Message----- > >> From: Skidmore, Donald C [mailto:donald.c.skidm...@intel.com] > >> Sent: Friday, June 19, 2015 8:31 AM > >> To: Pavel Odintsov; e1000-devel@lists.sourceforge.net > >> Subject: Re: [E1000-devel] VF inside VM with bypassed PCI-E ixgbe > >> > >> Hey Pavel, > >> > >> In order to support the bypass drive you need a special driver. I'm > >> surprised you weren't supplied with one when you bought the NIC. > >> Without it you can't communicate with the uC that handles all the bypass > functionality. > >> Likewise the SDP aren't connected to what the upstream ixgbe driver > >> thinks they should be so "undefined" things could happen. > >> > >> Thanks, > >> -Don Skidmore <donald.c.skidm...@intel.com> > >> > >> > >> > -----Original Message----- > >> > From: Pavel Odintsov [mailto:pavel.odint...@gmail.com] > >> > Sent: Friday, June 19, 2015 3:43 AM > >> > To: e1000-devel@lists.sourceforge.net > >> > Subject: Re: [E1000-devel] VF inside VM with bypassed PCI-E ixgbe > >> > > >> > I have finished my "home work". > >> > > >> > issue reproduced with 4.0.0 kernel and latest 4.1.1 ixgbe driver. > >> > > >> > Same error in dmesg: > >> > [ 221.676388] ixgbe 0000:00:05.0 0000:00:05.0 (uninitialized): > >> > Failed to enable PCI sriov: -38 > >> > > >> > On Fri, Jun 19, 2015 at 12:58 PM, Pavel Odintsov > >> > <pavel.odint...@gmail.com> wrote: > >> > > Hello! > >> > > > >> > > After some investigation of ixgbe and Linux kernel code. > >> > > > >> > > I found function which return this "error 38", it's sriov_enable > >> > > from Linux Kernel (http://lxr.free- > >> > electrons.com/source/drivers/pci/iov.c?v=3.16#L173). > >> > > > >> > > And error 38 means return of ENOSYS > >> > > (http://lxr.free-electrons.com/source/include/uapi/asm-generic/er > >> > > rno > >> > > .h > >> > > ?v=3.16#L9) from it. Which means "Function not implemented". > >> > > > >> > > Finally, Linux Kernel really sure about virtual nature of this bypassed > NIC. > >> > > > >> > > if (!dev->is_physfn) > >> > > return -ENOSYS; > >> > > > >> > > So, we need dig into kernel code. > >> > > > >> > > On Fri, Jun 19, 2015 at 12:05 PM, Pavel Odintsov > >> > > <pavel.odint...@gmail.com> wrote: > >> > >> Sorry, for the mess. It was very hard day and I post fe silly > >> > >> posts due to my inattention (and incorrectly selected ssh window). > >> > >> > >> > >> Finally, I have got only single issue with ixgbe driver there. > >> > >> > >> > >> I can't create VF inside KVM VM. Not with max_vfs param, not > >> > >> with echo > >> > >> 2 > > /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs. > >> > >> > >> > >> Could you help me? > >> > >> > >> > >> On Fri, Jun 19, 2015 at 11:58 AM, Pavel Odintsov > >> > >> <pavel.odint...@gmail.com> wrote: > >> > >>> Hello! > >> > >>> > >> > >>> Please ignre this: > >> > >>> echo 2 > > >> > >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs > >> > >>> -bash: echo: write error: Function not implemented > >> > >>> > >> > >>> echo 2 > > >> > >>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs > >> > >>> -bash: echo: write error: Function not implemented > >> > >>> > >> > >>> > >> > >>> Now it works fine. But broken name which refer to /proc/* > >> > >>> instead of > >> > >>> /sys/* still an issue. > >> > >>> > >> > >>> > >> > >>> On Fri, Jun 19, 2015 at 11:55 AM, Pavel Odintsov > >> > >>> <pavel.odint...@gmail.com> wrote: > >> > >>>> Hello! > >> > >>>> > >> > >>>> Some details. > >> > >>>> > >> > >>>> Error: > >> > >>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs > >> > >>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: > >> > >>>> Permission denied > >> > >>>> > >> > >>>> Definitely related with non existent proc fs path: > >> > >>>> find /proc/|grep sriov_numvfs|wc -l > >> > >>>> 0 > >> > >>>> > >> > >>>> It's definitely a bug. > >> > >>>> > >> > >>>> Because I could find this pseudo files in /sys: > >> > >>>> root@filter ~ # find /sys/|grep vfs > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalv > >> > >>>> fs > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalv > >> > >>>> fs root@filter ~ # find /proc/|grep vfs > >> > >>>> /proc/sys/vm/vfs_cache_pressure > >> > >>>> > >> > >>>> But I still can't use sriov_numvfs from there: > >> > >>>> > >> > >>>> cat > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_totalv > >> > >>>> fs > >> > >>>> 63 > >> > >>>> > >> > >>>> cat > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_totalv > >> > >>>> fs > >> > >>>> 63 > >> > >>>> > >> > >>>> echo 2 > > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.0/sriov_numvfs > >> > >>>> -bash: echo: write error: Function not implemented > >> > >>>> > >> > >>>> echo 2 > > >> > >>>> /sys/devices/pci0000:00/0000:00:02.0/0000:03:00.1/sriov_numvfs > >> > >>>> -bash: echo: write error: Function not implemented > >> > >>>> > >> > >>>> dmesg|tail -n 100 > >> > >>>> [3004270.392052] ixgbe: Intel(R) 10 Gigabit PCI Express > >> > >>>> Network Driver > >> > >>>> - version 3.19.1-k > >> > >>>> [3004270.392316] ixgbe: Copyright (c) 1999-2014 Intel Corporation. > >> > >>>> [3004295.119478] vfio-pci 0000:03:00.0: Driver doesn't support > >> > >>>> SRIOV configuration via sysfs [3004299.761902] vfio-pci > >> > >>>> 0000:03:00.1: Driver doesn't support SRIOV configuration via > >> > >>>> sysfs > >> > >>>> > >> > >>>> :( > >> > >>>> > >> > >>>> On Fri, Jun 19, 2015 at 11:37 AM, Pavel Odintsov > >> > >>>> <pavel.odint...@gmail.com> wrote: > >> > >>>>> Hello, folks! > >> > >>>>> > >> > >>>>> I have KVM VM with bypassed PCI-E 82599 two port NIC. > >> > >>>>> > >> > >>>>> Driver version: 4.0.3 at Debian Jessie 3.16 kernel. > >> > >>>>> > >> > >>>>> lspci -v|grep Eth > >> > >>>>> 00:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > >> > >>>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 20) > >> > >>>>> 00:05.0 Ethernet controller: Intel Corporation 82599ES > >> > >>>>> 10-Gigabit SFI/SFP+ Network Connection (rev 01) > >> > >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > >> > >>>>> 00:06.0 Ethernet controller: Intel Corporation 82599ES > >> > >>>>> 10-Gigabit SFI/SFP+ Network Connection (rev 01) > >> > >>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > >> > >>>>> > >> > >>>>> And I want to use VF inside this KVM VM. > >> > >>>>> > >> > >>>>> And I doing following: > >> > >>>>> rmmod ixgbe ixgbevf > >> > >>>>> modprobe ixgbe max_vfs=2,2 > >> > >>>>> modprobe ixgbevf > >> > >>>>> > >> > >>>>> But after this operation I haven't saw virtual functions in > >> > >>>>> lspci > >> output: > >> > >>>>> lspci -tv > >> > >>>>> -[0000:00]-+-00.0 Intel Corporation 440FX - 82441FX PMC > [Natoma] > >> > >>>>> +-01.0 Intel Corporation 82371SB PIIX3 ISA > >> > >>>>> [Natoma/Triton > II] > >> > >>>>> +-01.1 Intel Corporation 82371SB PIIX3 IDE > >> > >>>>> [Natoma/Triton > II] > >> > >>>>> +-01.3 Intel Corporation 82371AB/EB/MB PIIX4 ACPI > >> > >>>>> +-02.0 Realtek Semiconductor Co., Ltd. > >> > >>>>> RTL-8100/8101L/8139 PCI Fast Ethernet Adapter > >> > >>>>> +-03.0 Intel Corporation 82801I (ICH9 Family) USB > >> > >>>>> UHCI > >> > Controller #1 > >> > >>>>> +-03.1 Intel Corporation 82801I (ICH9 Family) USB > >> > >>>>> UHCI > >> > Controller #2 > >> > >>>>> +-03.2 Intel Corporation 82801I (ICH9 Family) USB > >> > >>>>> UHCI > >> > Controller #3 > >> > >>>>> +-03.7 Intel Corporation 82801I (ICH9 Family) > >> > >>>>> USB2 EHCI Controller #1 > >> > >>>>> +-04.0 Red Hat, Inc Virtio memory balloon > >> > >>>>> +-05.0 Intel Corporation 82599ES 10-Gigabit > >> > >>>>> SFI/SFP+ Network Connection > >> > >>>>> +-06.0 Intel Corporation 82599ES 10-Gigabit > >> > >>>>> SFI/SFP+ Network Connection > >> > >>>>> \-07.0 Cirrus Logic GD 5446 > >> > >>>>> > >> > >>>>> And haven't saw any additional interfaces: > >> > >>>>> > >> > >>>>> ip link show > >> > >>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue > state > >> > >>>>> UNKNOWN mode DEFAULT group default > >> > >>>>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > >> > >>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 > qdisc > >> > >>>>> pfifo_fast state UP mode DEFAULT group default qlen 1000 > >> > >>>>> link/ether 52:54:00:ec:be:23 brd ff:ff:ff:ff:ff:ff > >> > >>>>> 19: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state > >> > DOWN > >> > >>>>> mode DEFAULT group default qlen 1000 > >> > >>>>> link/ether 90:e2:ba:83:3f:24 brd ff:ff:ff:ff:ff:ff > >> > >>>>> 20: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 > qdisc > >> > mq > >> > >>>>> state UP mode DEFAULT group default qlen 1000 > >> > >>>>> link/ether 90:e2:ba:83:3f:25 brd ff:ff:ff:ff:ff:ff > >> > >>>>> > >> > >>>>> > >> > >>>>> I saw this error in dmesg: > >> > >>>>> > >> > >>>>> [848550.356391] ixgbe 0000:00:06.0: Enabling SR-IOV VFs using > >> > >>>>> the max_vfs module parameter is deprecated. > >> > >>>>> [848550.356398] ixgbe 0000:00:06.0: Please use the pci sysfs > >> > >>>>> interface instead. Ex: > >> > >>>>> [848550.356403] ixgbe 0000:00:06.0: echo '2' > > >> > >>>>> /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs > >> > >>>>> [848705.672383] ixgbe 0000:00:06.0 (unregistered net_device): > >> > >>>>> Failed to enable PCI sriov: -38 <------- COULD BE IMPORTANT > >> > >>>>> > >> > >>>>> And try another approach: > >> > >>>>> rmmod ixgbe ixgbevf > >> > >>>>> > >> > >>>>> modprobe ixgbe > >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs > >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:05.0/sriov_numvfs > >> > >>>>> > >> > >>>>> And it's not working at all: > >> > >>>>> echo '2' > /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs > >> > >>>>> -bash: /sys/bus/pci/devices/0000:00:06.0/sriov_numvfs: > >> > >>>>> Permission denied > >> > >>>>> > >> > >>>>> Is it possible to use VF inside KVM VM? > >> > >>>>> > >> > >>>>> -- > >> > >>>>> Sincerely yours, Pavel Odintsov > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> -- > >> > >>>> Sincerely yours, Pavel Odintsov > >> > >>> > >> > >>> > >> > >>> > >> > >>> -- > >> > >>> Sincerely yours, Pavel Odintsov > >> > >> > >> > >> > >> > >> > >> > >> -- > >> > >> Sincerely yours, Pavel Odintsov > >> > > > >> > > > >> > > > >> > > -- > >> > > Sincerely yours, Pavel Odintsov > >> > > >> > > >> > > >> > -- > >> > Sincerely yours, Pavel Odintsov > >> > > >> > ------------------------------------------------------------------- > >> > --- > >> > -------- _______________________________________________ > >> > E1000-devel mailing list > >> > E1000-devel@lists.sourceforge.net > >> > https://lists.sourceforge.net/lists/listinfo/e1000-devel > >> > To learn more about Intel® Ethernet, visit > >> > http://communities.intel.com/community/wired > >> > >> --------------------------------------------------------------------- > >> --------- _______________________________________________ > >> E1000-devel mailing list > >> E1000-devel@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/e1000-devel > >> To learn more about Intel® Ethernet, visit > >> http://communities.intel.com/community/wired > > > > -- > Sincerely yours, Pavel Odintsov ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
