[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
@Juerg Hello, I have a new patchset. It has all the necessary fixes to stabilize nvlink2 passthrough, which required more patches backported. I'm attaching the tar.gz here. You can also find it at https://github.com/jrziviani/linux-devel/tree/nv2_scratch_181203_bionic It's built on top of ubuntu-bionic/master-next (Ubuntu-4.15.0-43.46). Patches from 1 to 91 are upstream, others can be found at https://github.com/aik/linux/tree/nv2. Thank you very much! ** Attachment added: "linux-patches-bionic.tar.gz" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+attachment/5219785/+files/linux-patches-bionic.tar.gz -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
@Christian, hello! I completed the backport to QEMU v2.11.2, but more patches were required in order to make it work. The patchset is attached here but it can be found also at https://github.com/jrziviani/qemu-devel/tree/nv2_scratch_181204_bionic. Patches from 1 to 34 are already upstream, from 34 to 45 are under review. Thank you very much! ** Attachment added: "qemu-patches-v2.11.2.tar.gz" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+attachment/5219808/+files/qemu-patches-v2.11.2.tar.gz -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Hi Jose, I'm puzzled that you update that series here. AFAIK per discussions this project will run it's own custom qemu, so for I'm not considering the backports for the main Archive (for everybody). ** Changed in: qemu (Ubuntu Bionic) Status: New => Won't Fix ** Changed in: qemu (Ubuntu) Importance: Undecided => Wishlist -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Hello Christian! That's a good point. :-) As far as I know the project should be based on top of v2.11.2 (at least, this is what we've been testing). I'll try to confirm it here. Thank you -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Hello Juerg, Thanks for the packages provided. I made a initial setup and it looks good. However, a bug was identified and a new patch should be applied on top of my previous patchset. Would you mind to include it? The patch is: https://github.com/jrziviani/linux-devel/commit/b466f257ada3b408be3ca475ce6253b80efc252c (also attached). Thank you very much, Jose ** Patch added: "0001-powerpc-powernv-npu-Fault-user-page-into-the-hypervi.patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+attachment/5231782/+files/0001-powerpc-powernv-npu-Fault-user-page-into-the-hypervi.patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Won't Fix Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Jose, I'm confused. I never provided a kernel with the patches from comment #8 yet you did some testing and need a follow-on patch? What kernel did you test? Also, this ticket is closed as "won't fix" so you should not add new requests to it. And since the patches are for the ibm-gt kernel (which is a private IBM kernel) all communication should go through SalesForce not LaunchPad. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Won't Fix Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Juerg, sorry, my bad. I tested my own build. I was sure I had 'apt-get' that IBM-cloud, which didn't happen. Anyway, I was not aware of that salesforce process. I saw this bug marked as "won't fix" without any further explanation, how could I know that? Anyway, how do I access the salesforce to make that request? Thanks -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Won't Fix Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
** Summary changed: - Add support to NVIDIA GPU passthrough + Add support for NVIDIA GPU passthrough -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
** Also affects: qemu (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: New Status in linux source package in Bionic: New Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
** Changed in: linux (Ubuntu Bionic) Status: New => Fix Released ** Changed in: linux (Ubuntu) Status: Incomplete => Fix Released ** Changed in: linux (Ubuntu) Status: Fix Released => Won't Fix ** Changed in: linux (Ubuntu Bionic) Status: Fix Released => Won't Fix -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: New Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
The kernel patches were delivered via a custom kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: New Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Hi, we can certainly help trying to make a test PPA available for your testing backporting what you provided here. I wonder about the git tree at [1]. It is already further than what was reported a few days before. It is based on qemu v3.1.0-rc0 at the moment and has 11 patches . Some of the series listed here, some new ones. In any case it does not seem complete or fully upstreamed yet. But eventually these things should be upstream in qemu before being picked into the current Ubuntu development release (19.04) and then planning SRUs from there. Since I don't know about the interim state of the repo I'll pick the listed changes for now. I was told that your target is Ubuntu 18.04 which means qemu 2.11 has to be the target unless you are fine with e.g. Ubuntu Cloud Archive Stein [3] which would eventually if the current plan holds true be on qemu 3.1 at least - much closer for the backports. Or if it will be a PPA based solution anyway something like qemu 3.1 + changes in that PPA. The backports of that series provided here to qemu 2.11 which would be needed for a real SRU release into Ubuntu 18.04: OK 0001-qdev-Use-string-for-QOM-string-properties.patch NOISE in 0002-ppc-spapr-Receive-and-store-device-tree-blob-from-SL.patch spapr_machine_reset was still ppc_spapr_reset Also this changes vmstate which I think would create a migration issue, so we'd need an extra machine type I think. This always as a big NONO for real SRUs - this makes this even more an only-PPA solution IMHO. OK 0003-DBG-store-fdt.patch Minor Noise 0004-vfio-spapr-Fix-indirect-levels-calculation.patch Noise 0005-headers-update.patch Noise 0006-pci-Move-NVIDIA-vendor-id-to-the-rest-of-ids.patch Noise 0007-RFC-vfio-nvidia-v100-Disable-VBIOS-update.patch Noise 0008-spapr-iommu-Always-advertise-the-maximum-possible-DM.patch The noise for all these is not too much form a code level, the context can be found. But I get the feeling I'll see this as a compile fail missing some other changes it depends on. OK 0009-FIXME-vfio-Do-not-replay-IOMMU-mappings.patch Minor Noise 0010-vfio-Make-vfio_get_region_info_cap-public.patch OK - 0011-vfio-spapr-Try-allocating-less-levels-if-failed-with.patch Noise - 0012-spapr-Add-NVLink2-memory-to-PHB-placement.patch OK - 0013-spapr-Create-ibm-gpu-and-ibm-npu-cross-links-in-the-.patch Noise - 0014-spapr-vfio-Map-GPU-RAM-and-advertise-to-the-guest.patch Noise - 0015-debug-Disables-KVM-TCE-acceleration-and-direct-ATSD-.patch In general this is WIP still, because that makes not much sense: 6 cap_spapr_vfio = kvm_vm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO); 7 +cap_spapr_vfio = false; With the related kernel being a special kernel I can't SRU it generally anyway. And all the problems outlined above emphasize that. So I wonder what is the actual delivery mechanism and content expected here? Would a qemu 3.1 in a PP alongside the kernel do it - there the backport distance certainly would be much less? That could eventually be replaced in the PPA with the more polished final qmeu 3.1 we will create for Ubuntu 19.04. I have pushed a packetized and backported version of this at [4] but I'm 98% sure it will fail to build [5] as it is right now. TL;DR: - From what I see in the code this will most likely never be SRUable to 18.04 - as-is this won't go further on 18.04 (Bionic) qemu - I need more details about the required delivery content (minimum qemu x.y?) - I need more details about the required delivery content (version&state of patches git vs bug attachment) - I need more details about the required delivery path (PPA along kernel?) - I need more details about the required delivery path (Later update policy?) [1]: https://github.com/aik/qemu/tree/nv2 [2]: https://wiki.ubuntu.com/StableReleaseUpdates [3]: https://wiki.ubuntu.com/OpenStack/CloudArchive [4]: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+ref/test-backport-1800649-nvlink2-ppc [5]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3518/+packages ** Changed in: qemu (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: New Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launch
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Add support for NVIDIA GPU passthrough Is this just a duplicate of LP 1800649? (Now coming in via the ubuntu-power-system project.) https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649 or is it meant to be the libvirt part of LP 1800649, or is this a request to move this feature over from the custom to the generic package(s)? Please provide some more details - thx. --- Comment From mranw...@us.ibm.com 2018-12-10 10:15 EDT--- This is a dup of 1800649, it was meant to create an IBM side mirror it instead of a new bug. I'm not sure why it's that way, but asked if we can get it fixed. ** Tags added: architecture-ppc64le bugnameltc-173957 severity-medium targetmilestone-inin1804 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Incomplete Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
** Changed in: qemu (Ubuntu) Status: Incomplete => Won't Fix -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Won't Fix Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1800649] Re: Add support for NVIDIA GPU passthrough
Hi Christian, You're right, we're going to use our customized QEMU. Thank you!! Jose -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1800649 Title: Add support for NVIDIA GPU passthrough Status in linux package in Ubuntu: Won't Fix Status in qemu package in Ubuntu: Won't Fix Status in linux source package in Bionic: Won't Fix Status in qemu source package in Bionic: Won't Fix Bug description: This bug will keep track of an important feature that is being developed upstream and need to be backported to Ubuntu 18.04 PPC64. The feature allows QEMU/KVM guests to have NVIDIA GPUs passed-through. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1800649/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
