Public bug reported:
NVIDIA Vera-Rubin GPUs can operate as CXL Type-2 devices, exposing both
compute and coherent Host-managed Device Memory (HDM) over CXL. The
26.04_linux-nvidia-7.3-next tree has no support for either resetting such a
device or assigning one to a guest:
* There is no CXL Reset method. A PCIe FLR leaves CXL.mem state and HDM
contents intact (CXL r4.0 sec 9.5), so the only reset available to a
CXL-mode GPU does not clear device memory and does not restore the HDM
decoder. A secondary bus reset does decommit the decoder but nothing
restores it, leaving the device decoding ranges the CXL core considers
free.
* There is no way to pass a CXL Type-2 device through to a VM. vfio-pci
has no understanding of the HDM decoder, the CXL DVSEC, or the component
register block, so a CXL-mode GPU cannot be assigned to a guest.
* On arm64 there is no cache-maintenance provider for the host-side
invalidation that CXL Reset requires before a device reset.
[Fix]
Backport three posted series plus their upstream prerequisites, 56 commits
in total on top of 786e18af7433:
1. PCI/CXL: Add CXL reset support for Type 2 devices, v17 (19 patches)
Srirangan Madhavan
https://lore.kernel.org/linux-cxl/[email protected]/
2. arm64: Add SMCCC cache clean/invalidate support, v5 (2 patches)
Srirangan Madhavan
https://lore.kernel.org/lkml/[email protected]/
3. vfio/pci: Add CXL Type-2 device passthrough support, v6 (31 patches)
Manish Honap
https://lore.kernel.org/linux-cxl/[email protected]/
Plus:
* 2 upstream cherry-picks that series 2 depends on (see [Other Info])
* 1 new NVIDIA: SAUCE: patch to adapt series 3 onto series 1 (see below)
* 1 [Config] patch adding the required annotations
Upstream prerequisites cherry-picked for the SMCCC series (both from the
7.4 merge window, neither in a tagged release at the time of this port):
fe7f2a805443 firmware: smccc: Add an Arm SMCCC bus
d8263a131bb0 firmware: hwrng: arm_smccc_trng: Register as an SMCCC device
The vfio series (v6) was written against CXL reset v15. v17 reworked the
restore API, so the port carries one new NVIDIA: SAUCE: patch and modifies
two of Manish's patches. Details in the pull request.
** Affects: linux-nvidia-bos (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2174630
Title:
[26.04-[linux-nvidia-7.3] Enable CXL Type-2 device support: CXL Reset
and VFIO passthrough
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-nvidia-bos/+bug/2174630/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs