On Wed, Jan 25, 2017 at 5:24 PM, Ed Warnicke <hagb...@gmail.com> wrote:
> Looping in the deb_dpdk and rpm_dpdk guys to see if they have thoughts on > how these things play out in deployment. > Thanks Ed. Adding my 2 cents inline below just to provide another POV ... > On Tue, Jan 24, 2017 at 9:05 PM, Burt Silverman <bur...@gmail.com> wrote: > >> Hi Damjan, >> >> My understanding is that CONFIG_VFIO_NOIOMMU will never be set in a stock >> kernel, and you will need to build a custom kernel for that. >> > Well, to Enable the config itself isn't bad until enable_unsafe_noiommu_modeis set (which is default off). See https://patchwork.kernel.org/patch/7906971/ for a few more Details. IMHO that means it can be safely in a default kernel with support, and it is on the administrator to enable that mode. But that is like giving wrong people root, or missetting something else - not as bad as building an own kernel. In fact there was a guy with my name who got that enabled in the next Ubuntu Release Zesty :-) https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1634050 For vfio (as the default) there is another thing much more in the way. People will likely want iommu support if possible, but in 9/10 cases I've seen I'd say that some oddity forced them to be unable to use it. Certain Firmware rendered iommu useless, Unable to set up the iommu group assignment correctly, ... I happened to get the feeling that it might only work in just one special Lab setup somebody has out there. The noiommu config might make that simpler, but then it throws away a lot that vfio should provide. So I'd only use it for test/devel but would not like it too much for production use. That said IMHO I would not suggest vfio as a default these days even with the kernel option enabled. > I understand that with this option, the kernel cannot guarantee that >> applications are prevented from creating bugs that normally the kernel can >> guarantee will not occur (outside of a kernel bug.) It therefore violates >> the fundamental Linux system design. That being said, you may wish to >> accept the risk for performance reasons and build a custom kernel. The >> other strange thing would be that MSI or MSI-X style interrupts are not >> needed for performance. The people who developed them have made a lot of >> noise about how they came about for performance reasons. I have no direct >> experience, but to learn that they are not important is a shock. >> > I don't have good insight on the related performance impact for the question "igb_uio -> uio_pci_generic". But then (whatever it is worth) I usually go with uio_pci_generic and most people I talked to did so as well (on the wider DPDK and not just VPP stage). But here the circle closes referring to some former statements that "compiling the kernel on your own is a no-go" If I'm not overlooking something - at least in all environments I touched - igb_uio is a dpdk provided kernel module and uio_pci_generic is in the kernel. At least http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html#loading-modules-to-enable-userspace-io-for-dpdk backs my memory up. That said - loading a custom module is just as bad as building your own kernel in terms of applicability in a real case. So to add my unimportant opinion to this thread about what could/should be the default: 1. uio_pci_generic (default) 2. vfio (as a better option for those who want/can set it up correctly) 3. igb_uio (fallback for special cases where it is needed) As I said I can't participate in the performance fight between uip_pci_generic and igb_uio >> It seems to me that the Ubuntu 14.04 issue is really a separate one from >> all of this, although I would imagine that the conclusion to stop >> supporting it does not change. >> >> Burt >> >> On Tue, Jan 24, 2017 at 1:07 PM, Damjan Marion (damarion) < >> damar...@cisco.com> wrote: >> >>> >>> > On 24 Jan 2017, at 18:40, Damjan Marion (damarion) <damar...@cisco.com> >>> wrote: >>> > >>> >> >>> >> On 24 Jan 2017, at 18:26, Stephen Hemminger < >>> step...@networkplumber.org> wrote: >>> >> >>> >> On Tue, 24 Jan 2017 17:14:42 +0000 >>> >> "Damjan Marion (damarion)" <damar...@cisco.com> wrote: >>> >> >>> >>> Is anybody aware of any valid reason why we cannot switch to >>> uio_pci_generic >>> >>> as default PCI uio driver in ubuntu packages? >>> >>> >>> >>> I think generally people don’t like out-of-tree modules, so as long >>> as we are getting >>> >>> the same service from uio_pci_generic we should use it… >>> >>> >>> >>> Thanks, >>> >>> >>> >>> Damjan >>> > >>> >> uio_pci_generic does not support MSI or MSI-X interrupts, only legacy >>> INTX. >>> > >>> > I know but do we really care? >>> > >>> >> >>> >> The preference should always be to use VFIO. Even on systems without >>> IOMMU. >>> > >>> > What is the perf impact? >>> > >>> > Also, I just tried with kernel 4.8 on rangeley ATOM, and i got: >>> > >>> > [536030.250072] vfio-pci: probe of 0000:00:14.0 failed with error -22 >>> > [536030.253271] vfio-pci: probe of 0000:00:14.0 failed with error -22 >>> > >>> > I guess I’m doing something wrong…. >>> >>> This explains: >>> >>> grep VFIO_NOIO /boot/config-4.8.0-34-generic >>> # CONFIG_VFIO_NOIOMMU is not set >>> >>> So vfio is out of the game of being default choice, people can still >>> switch simply with one line change in /etc/vpp/startup.conf. >>> >> -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
