Hi Damjan, I am running Ubuntu 18.04 LTS with kernel 4.15.0-118-generic
See below dmesg output. jmoore@test:~$ dmesg | grep 0000:03:00.0 [ 0.223459] pci 0000:03:00.0: [15ad:07b0] type 00 class 0x020000 [ 0.225126] pci 0000:03:00.0: reg 0x10: [mem 0xfd4fc000-0xfd4fcfff] [ 0.227304] pci 0000:03:00.0: reg 0x14: [mem 0xfd4fd000-0xfd4fdfff] [ 0.229121] pci 0000:03:00.0: reg 0x18: [mem 0xfd4fe000-0xfd4fffff] [ 0.231298] pci 0000:03:00.0: reg 0x1c: [io 0x4000-0x400f] [ 0.237119] pci 0000:03:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref] [ 0.237550] pci 0000:03:00.0: supports D1 D2 [ 0.237551] pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.237774] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force' [ 0.353290] pci 0000:03:00.0: BAR 6: assigned [mem 0xfd400000-0xfd40ffff pref] [ 1.179463] iommu: Adding device 0000:03:00.0 to group 6 [ 2.108455] vmxnet3 0000:03:00.0: # of Tx queues : 8, # of Rx queues : 8 [ 2.110321] vmxnet3 0000:03:00.0 eth0: NIC Link is Up 10000 Mbps [ 2.471328] vmxnet3 0000:03:00.0 ens160: renamed from eth0 On Mon, Sep 28, 2020 at 1:02 PM Damjan Marion <dmar...@me.com> wrote: > > What message do you see in dmesg? What is the kernel version? > > On 28.09.2020., at 19:47, Joshua Moore <j...@jcm.me> wrote: > > Sorry, I'm still hitting an issue where I cannot create the interface in > VPP: > > vpp# create interface avf 0000:13:00.0 > create interface avf: device not bound to 'vfio-pci' or 'uio_pci_generic' > kernel module > > > > So I tried to bind the NIC to vfio-pci: > > root@test:~# modprobe vfio-pci > root@test:~# /usr/local/vpp/vpp-config/scripts/dpdk-devbind.py -s > > Network devices using DPDK-compatible driver > ============================================ > <none> > > Network devices using kernel driver > =================================== > 0000:13:00.0 'Ethernet Virtual Function 700 Series' if=ens224 drv=i40evf > unused= > > root@test:~# /usr/local/vpp/vpp-config/scripts/dpdk-devbind.py --bind > vfio-pci 13:00.0 > Error - no supported modules(DPDK driver) are loaded > > > > Thoughts? > > On Mon, Sep 28, 2020 at 11:43 AM Benoit Ganne (bganne) <bga...@cisco.com> > wrote: > >> Hi Johsua, >> >> Your understanding is correct, however you do not need to setup the VFs >> if it is already correctly setup by ESXI. >> Just create the AVF interface directly by specifying the VF PCI address. >> >> ben >> >> > -----Original Message----- >> > From: Joshua Moore <j...@jcm.me> >> > Sent: lundi 28 septembre 2020 17:48 >> > To: Benoit Ganne (bganne) <bga...@cisco.com> >> > Cc: vpp-dev@lists.fd.io >> > Subject: Re: [vpp-dev] VPP on ESXI with i40evf (SR-IOV Passthrough) >> Driver >> > >> > Hello Benoit, >> > >> > Looking at the script for AVF, it states: >> > >> > # Setup one VF on PF 0000:3b:00.0 and assign MAC address >> > setup 0000:3b:00.0 00:11:22:33:44:00 >> > # Setup one VF on PF 0000:3b:00.1 and assign MAC address >> > setup 0000:3b:00.1 00:11:22:33:44:01 >> > >> > This seems to assume the entire PF NIC is exposed to the VM and the VM >> is >> > responsible for owning the configuration of the WHOLE PF to setup the >> VF. >> > This also makes sense to me considering that the script is looking for >> > i40en driver (physical) and not i40evf driver (virtual). My >> understanding >> > is that this will not work with my ESXI setup as ESXI owns the >> > configuration of the PF (physical NIC) and is assigning the VFs from the >> > NIC end is exposing just the VF to the VM. >> > >> > Does this make sense or am I misunderstanding something? >> > >> > If so, then how can the AVF plugin/driver consume just the VF NIC >> already >> > assigned to the VM and not try to setup a new VF? >> > >> > >> > Thanks! >> > >> > -Josh >> > >> > On Mon, Sep 28, 2020 at 2:40 AM Benoit Ganne (bganne) <bga...@cisco.com >> > <mailto:bga...@cisco.com> > wrote: >> > >> > >> > Hi, >> > >> > It should work with AVF as it is using VFs, not PF, see >> > https://docs.fd.io/vpp/21.01/d1/def/avf_plugin_doc.html >> > You should bind the VF with vfio-pci 1st though, so that it is >> > usable by userspace drivers such as VPP AVF plugin. >> > If your system crashes when doing so it is a bug with your system. >> > >> > Best >> > ben >> > >> > > -----Original Message----- >> > > From: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> <vpp- >> > d...@lists.fd.io <mailto:vpp-dev@lists.fd.io> > On Behalf Of j...@jcm.me >> > <mailto:j...@jcm.me> >> > > Sent: lundi 28 septembre 2020 01:29 >> > > To: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> >> > > Subject: [vpp-dev] VPP on ESXI with i40evf (SR-IOV Passthrough) >> > Driver >> > > >> > > Hello, >> > > >> > > Is there any support for VPP to talk directly to ESXI-assigned >> VFs >> > via SR- >> > > IOV? I saw the AVF plugin but I don't want VPP to control the >> > whole PF >> > > (physical NIC) but rather would like to have ESXI control the >> > mapping of >> > > VFs (SR-IOV) and VPP (or DPDK) consume the VF natively in the VM >> > so that I >> > > can run multiple VMs on the same physical NIC while benefiting >> > from >> > > bypassing the vSwitch in ESXI. Right now I'm running VPP on a >> > Ubuntu 18.04 >> > > VM and I see the SR-IOV NIC as an i40evf driver. >> > > >> > > I tried binding the SR-IOV NIC to the vfio driver but this >> causes >> > the CPU >> > > of the VM to skyrocket and crash. I don't think using vfio is >> the >> > right >> > > approach and feel like the solution here is really simple. Any >> > > suggestions? >> > >> >> > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#17566): https://lists.fd.io/g/vpp-dev/message/17566 Mute This Topic: https://lists.fd.io/mt/77164974/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
