On Thu, Jun 07, 2018 at 09:17:42AM -0700, Stephen Hemminger wrote: > On Thu, 7 Jun 2018 18:41:31 +0300 > "Michael S. Tsirkin" <m...@redhat.com> wrote: > > > > > Why would DPDK care what we do in the kernel? Isn't it just slapping > > > > vfio-pci on the netdevs it sees? > > > > > > Alex, you are correct for Intel devices; but DPDK on Azure is not Intel > > > based.,. > > > The DPDK support uses: > > > * Mellanox MLX5 which uses the Infinband hooks to do DMA directly to > > > userspace. This means VF netdev device must exist and be visible. > > > * Slow path using kernel netvsc device, TAP and BPF to get exception > > > path packets to userspace. > > > * A autodiscovery mechanism that to set all this up that relies on > > > 2 device model and sysfs. > > > > Could you describe what does it look for exactly? What will break if > > instead of MLX5 being a child of the PV, it's a child of the failover > > device? > > So in DPDK there is an internal four device model: > 1. failsafe is like failover in your model > 2. TAP is used like netvsc in kernel > 3. MLX5 is the VF > 4. vdev_netvsc is a pseudo device whose only reason to exist > is to glue everything together. > > Digging deeper inside... > > Vdev_netvsc does: > * driver is started in a convuluted way off device arguments > * probe routine for driver runs > - scans list of kernel interfaces in sysfs > - matches those using VMBUS
Could you tell a bit more what does this step entail? > - skip netvsc devices that have an IPV4 route > * scan for PCI devices that have same MAC address as kernel netvsc > devices discovered in previous step > * add these interfaces to arguments to failsafe > > Then failsafe configures based on arguments on device > > The code works but is specific to the Azure hardware model, and exposes lots > of things to application that it should not have to care about. > > If you try and walk through this code in DPDK, you will see why I have > developed > a dislike for high levels of indirection. > > > Thanks that was helpful! I'll try to poke at it next week. Just from the description it seems the kernel is merely used to locate the MAC address through sysfs and that for this DPDK code to keep working the hidden device must be hidden from it in sysfs - is that a fair summary? -- MST
