Sorry, previous dmesg was greped on my VMXNET3 adapter, not the i40evf. Correct dmesg:
jmoore@test:~$ dmesg | grep 0000:13:00.0 [ 0.259249] pci 0000:13:00.0: [8086:154c] type 00 class 0x020000 [ 0.261432] pci 0000:13:00.0: reg 0x10: [mem 0xe7af0000-0xe7afffff 64bit pref] [ 0.266767] pci 0000:13:00.0: reg 0x1c: [mem 0xe7aec000-0xe7aeffff 64bit pref] [ 0.272845] pci 0000:13:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force' [ 1.179790] iommu: Adding device 0000:13:00.0 to group 8 [ 2.196529] i40evf 0000:13:00.0: Multiqueue Enabled: Queue pair count = 4 [ 2.196799] i40evf 0000:13:00.0: MAC address: 00:0c:29:58:7f:b5 [ 2.196865] i40evf 0000:13:00.0: GRO is enabled [ 2.510262] i40evf 0000:13:00.0 ens224: renamed from eth2 On Mon, Sep 28, 2020 at 4:57 PM Joshua Moore via lists.fd.io <j= jcm...@lists.fd.io> wrote: > Hi Damjan, > > I am running Ubuntu 18.04 LTS with kernel 4.15.0-118-generic > > > See below dmesg output. > > jmoore@test:~$ dmesg | grep 0000:03:00.0 > [ 0.223459] pci 0000:03:00.0: [15ad:07b0] type 00 class 0x020000 > [ 0.225126] pci 0000:03:00.0: reg 0x10: [mem 0xfd4fc000-0xfd4fcfff] > [ 0.227304] pci 0000:03:00.0: reg 0x14: [mem 0xfd4fd000-0xfd4fdfff] > [ 0.229121] pci 0000:03:00.0: reg 0x18: [mem 0xfd4fe000-0xfd4fffff] > [ 0.231298] pci 0000:03:00.0: reg 0x1c: [io 0x4000-0x400f] > [ 0.237119] pci 0000:03:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref] > [ 0.237550] pci 0000:03:00.0: supports D1 D2 > [ 0.237551] pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold > [ 0.237774] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device. > You can enable it with 'pcie_aspm=force' > [ 0.353290] pci 0000:03:00.0: BAR 6: assigned [mem > 0xfd400000-0xfd40ffff pref] > [ 1.179463] iommu: Adding device 0000:03:00.0 to group 6 > [ 2.108455] vmxnet3 0000:03:00.0: # of Tx queues : 8, # of Rx queues : 8 > [ 2.110321] vmxnet3 0000:03:00.0 eth0: NIC Link is Up 10000 Mbps > [ 2.471328] vmxnet3 0000:03:00.0 ens160: renamed from eth0 > > > > On Mon, Sep 28, 2020 at 1:02 PM Damjan Marion <dmar...@me.com> wrote: > >> >> What message do you see in dmesg? What is the kernel version? >> >> On 28.09.2020., at 19:47, Joshua Moore <j...@jcm.me> wrote: >> >> Sorry, I'm still hitting an issue where I cannot create the interface in >> VPP: >> >> vpp# create interface avf 0000:13:00.0 >> create interface avf: device not bound to 'vfio-pci' or 'uio_pci_generic' >> kernel module >> >> >> >> So I tried to bind the NIC to vfio-pci: >> >> root@test:~# modprobe vfio-pci >> root@test:~# /usr/local/vpp/vpp-config/scripts/dpdk-devbind.py -s >> >> Network devices using DPDK-compatible driver >> ============================================ >> <none> >> >> Network devices using kernel driver >> =================================== >> 0000:13:00.0 'Ethernet Virtual Function 700 Series' if=ens224 drv=i40evf >> unused= >> >> root@test:~# /usr/local/vpp/vpp-config/scripts/dpdk-devbind.py --bind >> vfio-pci 13:00.0 >> Error - no supported modules(DPDK driver) are loaded >> >> >> >> Thoughts? >> >> On Mon, Sep 28, 2020 at 11:43 AM Benoit Ganne (bganne) <bga...@cisco.com> >> wrote: >> >>> Hi Johsua, >>> >>> Your understanding is correct, however you do not need to setup the VFs >>> if it is already correctly setup by ESXI. >>> Just create the AVF interface directly by specifying the VF PCI address. >>> >>> ben >>> >>> > -----Original Message----- >>> > From: Joshua Moore <j...@jcm.me> >>> > Sent: lundi 28 septembre 2020 17:48 >>> > To: Benoit Ganne (bganne) <bga...@cisco.com> >>> > Cc: vpp-dev@lists.fd.io >>> > Subject: Re: [vpp-dev] VPP on ESXI with i40evf (SR-IOV Passthrough) >>> Driver >>> > >>> > Hello Benoit, >>> > >>> > Looking at the script for AVF, it states: >>> > >>> > # Setup one VF on PF 0000:3b:00.0 and assign MAC address >>> > setup 0000:3b:00.0 00:11:22:33:44:00 >>> > # Setup one VF on PF 0000:3b:00.1 and assign MAC address >>> > setup 0000:3b:00.1 00:11:22:33:44:01 >>> > >>> > This seems to assume the entire PF NIC is exposed to the VM and the VM >>> is >>> > responsible for owning the configuration of the WHOLE PF to setup the >>> VF. >>> > This also makes sense to me considering that the script is looking for >>> > i40en driver (physical) and not i40evf driver (virtual). My >>> understanding >>> > is that this will not work with my ESXI setup as ESXI owns the >>> > configuration of the PF (physical NIC) and is assigning the VFs from >>> the >>> > NIC end is exposing just the VF to the VM. >>> > >>> > Does this make sense or am I misunderstanding something? >>> > >>> > If so, then how can the AVF plugin/driver consume just the VF NIC >>> already >>> > assigned to the VM and not try to setup a new VF? >>> > >>> > >>> > Thanks! >>> > >>> > -Josh >>> > >>> > On Mon, Sep 28, 2020 at 2:40 AM Benoit Ganne (bganne) < >>> bga...@cisco.com >>> > <mailto:bga...@cisco.com> > wrote: >>> > >>> > >>> > Hi, >>> > >>> > It should work with AVF as it is using VFs, not PF, see >>> > https://docs.fd.io/vpp/21.01/d1/def/avf_plugin_doc.html >>> > You should bind the VF with vfio-pci 1st though, so that it is >>> > usable by userspace drivers such as VPP AVF plugin. >>> > If your system crashes when doing so it is a bug with your >>> system. >>> > >>> > Best >>> > ben >>> > >>> > > -----Original Message----- >>> > > From: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> <vpp- >>> > d...@lists.fd.io <mailto:vpp-dev@lists.fd.io> > On Behalf Of j...@jcm.me >>> > <mailto:j...@jcm.me> >>> > > Sent: lundi 28 septembre 2020 01:29 >>> > > To: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> >>> > > Subject: [vpp-dev] VPP on ESXI with i40evf (SR-IOV Passthrough) >>> > Driver >>> > > >>> > > Hello, >>> > > >>> > > Is there any support for VPP to talk directly to ESXI-assigned >>> VFs >>> > via SR- >>> > > IOV? I saw the AVF plugin but I don't want VPP to control the >>> > whole PF >>> > > (physical NIC) but rather would like to have ESXI control the >>> > mapping of >>> > > VFs (SR-IOV) and VPP (or DPDK) consume the VF natively in the >>> VM >>> > so that I >>> > > can run multiple VMs on the same physical NIC while benefiting >>> > from >>> > > bypassing the vSwitch in ESXI. Right now I'm running VPP on a >>> > Ubuntu 18.04 >>> > > VM and I see the SR-IOV NIC as an i40evf driver. >>> > > >>> > > I tried binding the SR-IOV NIC to the vfio driver but this >>> causes >>> > the CPU >>> > > of the VM to skyrocket and crash. I don't think using vfio is >>> the >>> > right >>> > > approach and feel like the solution here is really simple. Any >>> > > suggestions? >>> > >>> >>> >> >> >> >> > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17567): https://lists.fd.io/g/vpp-dev/message/17567 Mute This Topic: https://lists.fd.io/mt/77164974/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
