Public bug reported:
Since the 5.15.0-176 update, the amdgpu driver crashes during probe on
an AMD Picasso APU (Ryzen 5 3400G, Radeon Vega 11, 1002:15d8) when the
IOMMU is not available. The screen stays black, systemd-udevd and
systemd-journald hang in D state, and the system is unusable. 5.15.0-139
on the same machine boots fine: it logs "kfd ... NOT added due to
errors" and continues without KFD.
Seen on Ubuntu 20.04 linux-hwe-5.15 5.15.0-202.212~20.04.1. The same
code is in jammy linux 5.15.
ROOT CAUSE
The upstream 5.15.y backport of commit 80614c509810 ("drm/amdkfd: fix a
memory leak in device_queue_manager_init()"), stable commit
e11e8a29b304, first in v5.15.199, changed deallocate_hiq_sdma_mqd() to:
amdgpu_amdkfd_free_gtt_mem(dev->kgd, &mqd->gtt_mem);
In mainline, 6.1.y and 6.6.y, amdgpu_amdkfd_free_gtt_mem() takes "void
**mem_obj" since "drm/amdkfd: amdkfd_free_gtt_mem clear the correct
pointer". In 5.15.y it still takes "void *mem_obj", the buffer object
itself:
void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj)
{
struct amdgpu_bo *bo = (struct amdgpu_bo *) mem_obj;
amdgpu_bo_reserve(bo, true);
...
So in 5.15 the address of the gtt_mem field inside struct
device_queue_manager is used as a struct amdgpu_bo. Its reservation
object pointer reads as NULL and ww_mutex_lock() writes to address 0
(RDI=0 in the oops below). The compiler gives no warning because void **
converts to void * implicitly. The 5.10.y backport is correct; it still
passes mqd->gtt_mem.
Every call to device_queue_manager_uninit() on 5.15 hits this:
- Raven, Raven2 and Picasso APUs with the IOMMU disabled in the BIOS (often the
default). kfd_iommu_device_init() fails ("Error initializing iommuv2") and the
error path in kgd2kfd_device_init() calls device_queue_manager_uninit(). This
was handled cleanly before 5.15.0-176.
- Any 5.15 system with an AMD GPU when amdgpu is unbound or unloaded
(kgd2kfd_device_exit).
Affected users cannot move to newer kernels, including the upcoming fix
for CVE-2026-98432 (5.15.0-203).
SUGGESTED FIX (one character),
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c:
- amdgpu_amdkfd_free_gtt_mem(dev->kgd, &mqd->gtt_mem);
+ amdgpu_amdkfd_free_gtt_mem(dev->kgd, mqd->gtt_mem);
Workaround: enable the IOMMU in the BIOS, or boot an older kernel. I
only tested 5.15.0-139 (works) and 5.15.0-202 (crashes). 5.15.0-176 is
where the Ubuntu changelog lists the change.
ENVIRONMENT
- Ubuntu 20.04.6 LTS with Ubuntu Pro (ESM)
- Broken: linux-image-5.15.0-202-generic 5.15.0-202.212~20.04.1
- Working: linux-image-5.15.0-139-generic 5.15.0-139.149~20.04.1
- ASRock B450 Pro4, BIOS P4.20 (06/18/2020); IOMMU not exposed (no IVRS ACPI
table)
- AMD Ryzen 5 3400G (Picasso), PCI ID 1002:15d8
- /proc/version_signature on the working kernel: Ubuntu
5.15.0-139.149~20.04.1-generic 5.15.178 (5.15.0-202 does not boot far enough to
collect it, so ubuntu-bug/apport-collect cannot run on it)
- Attached: lspci -vnn (from 5.15.0-139)
KERNEL LOG (5.15.0-202, from /var/log/kern.log):
[ 4.830798] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[ 4.830904] kfd kfd: amdgpu: error getting iommu info. is the iommu enabled?
[ 4.830914] kfd kfd: amdgpu: Error initializing iommuv2
[ 4.830923] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 4.830930] #PF: supervisor write access in kernel mode
[ 4.830936] #PF: error_code(0x0002) - not-present page
[ 4.830942] PGD 0 P4D 0
[ 4.830950] Oops: 0002 [#1] SMP NOPTI
[ 4.830957] CPU: 6 PID: 455 Comm: systemd-udevd Not tainted
5.15.0-202-generic #212~20.04.1-Ubuntu
[ 4.830967] Hardware name: To Be Filled By O.E.M. To Be Filled By
O.E.M./B450 Pro4, BIOS P4.20 06/18/2020
[ 4.830976] RIP: 0010:ww_mutex_lock+0x25/0xa0
[ 4.830986] Code: 00 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 49
89 f5 41 54 49 89 fc e8 b6 e4 ff ff 31 c0 65 48 8b 14 25 c0 fb 01 00 <f0> 49 0f
b1 14 24 75 4f 4d 85 ed 74 18 41 83 45 10 01 4d 89 6c 24
[ 4.831000] RSP: 0018:ffffcee9011e77a8 EFLAGS: 00010246
[ 4.831007] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000016f
[ 4.831015] RDX: ffff8d0d91bb3300 RSI: 0000000000000000 RDI: 0000000000000000
[ 4.831022] RBP: ffffcee9011e77c0 R08: 0000000000000001 R09: ffffffffc13fa100
[ 4.831030] R10: 0000000000000001 R11: 000000000000001f R12: 0000000000000000
[ 4.831037] R13: 0000000000000000 R14: 0000000000000008 R15: ffff8d0d8e960080
[ 4.831044] FS: 00007f25b1bbb880(0000) GS:ffff8d1420780000(0000)
knlGS:0000000000000000
[ 4.831053] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.831060] CR2: 0000000000000000 CR3: 0000000107c1a000 CR4: 00000000003506e0
[ 4.831067] Call Trace:
[ 4.831073] <TASK>
[ 4.831082] amdgpu_amdkfd_free_gtt_mem+0x3b/0xe0 [amdgpu]
[ 4.831294] device_queue_manager_uninit+0x39/0x60 [amdgpu]
[ 4.831494] kgd2kfd_device_init.cold+0x28c/0x537 [amdgpu]
[ 4.831721] ? vprintk_default+0x1d/0x30
[ 4.831731] amdgpu_amdkfd_device_init+0x13b/0x180 [amdgpu]
[ 4.831929] amdgpu_device_ip_init+0x6e8/0x706 [amdgpu]
[ 4.832116] amdgpu_device_init.cold+0x64d/0xc9c [amdgpu]
[ 4.832296] ? srso_return_thunk+0x5/0x10
[ 4.832303] ? pci_read_config_word+0x27/0x40
[ 4.832310] ? srso_return_thunk+0x5/0x10
[ 4.832318] amdgpu_driver_load_kms+0x1e/0x2b0 [amdgpu]
[ 4.832454] amdgpu_pci_probe+0x1cb/0x290 [amdgpu]
[ 4.832589] local_pci_probe+0x4b/0x90
[ 4.832595] ? srso_return_thunk+0x5/0x10
[ 4.832602] pci_device_probe+0x191/0x200
[ 4.832610] really_probe.part.0+0xcb/0x380
[ 4.832617] ? srso_return_thunk+0x5/0x10
[ 4.832623] really_probe+0x40/0x80
[ 4.832629] __driver_probe_device+0xff/0x180
[ 4.832636] driver_probe_device+0x23/0xb0
[ 4.832643] __driver_attach+0xc5/0x180
[ 4.832648] ? __device_attach_driver+0x140/0x140
[ 4.832655] bus_for_each_dev+0x7e/0xd0
[ 4.832662] driver_attach+0x1e/0x30
[ 4.832668] bus_add_driver+0x178/0x220
[ 4.832675] driver_register+0x74/0xe0
[ 4.832681] __pci_register_driver+0x68/0x70
[ 4.832688] amdgpu_init+0x7c/0x1000 [amdgpu]
[ 4.832822] ? 0xffffffffc0a05000
[ 4.832827] do_one_initcall+0x48/0x1e0
[ 4.832835] ? __cond_resched+0x1a/0x50
[ 4.832840] ? srso_return_thunk+0x5/0x10
[ 4.832845] ? kmem_cache_alloc_trace+0x15a/0x420
[ 4.832853] ? srso_return_thunk+0x5/0x10
[ 4.832860] do_init_module+0x52/0x230
[ 4.832867] load_module+0x10d1/0x1230
[ 4.832878] __do_sys_finit_module+0xbf/0x120
[ 4.832883] ? srso_return_thunk+0x5/0x10
[ 4.832888] ? __do_sys_finit_module+0xbf/0x120
[ 4.832899] __x64_sys_finit_module+0x1a/0x20
[ 4.832904] x64_sys_call+0x1ac3/0x1fa0
[ 4.832910] do_syscall_64+0x5c/0xb0
[ 4.832917] ? srso_return_thunk+0x5/0x10
[ 4.832922] ? syscall_exit_to_user_mode+0x2c/0x50
[ 4.832928] ? x64_sys_call+0x3b9/0x1fa0
[ 4.832933] ? srso_return_thunk+0x5/0x10
[ 4.832939] ? do_syscall_64+0x69/0xb0
[ 4.832944] ? exit_to_user_mode_prepare+0x49/0x220
[ 4.832952] ? srso_return_thunk+0x5/0x10
[ 4.832957] ? syscall_exit_to_user_mode+0x2c/0x50
[ 4.832963] ? x64_sys_call+0x1e1d/0x1fa0
[ 4.832968] ? srso_return_thunk+0x5/0x10
[ 4.832974] ? do_syscall_64+0x69/0xb0
[ 4.832979] ? do_syscall_64+0x69/0xb0
[ 4.832985] entry_SYSCALL_64_after_hwframe+0x6c/0xd6
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Tags: amdgpu focal jammy regression-update
** Attachment added: "lspci -vnn (5.15.0-139, non-root)"
https://bugs.launchpad.net/bugs/2172819/+attachment/6014644/+files/lspci-vnn.log
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2172819
Title:
amdgpu: NULL pointer dereference in amdgpu_amdkfd_free_gtt_mem() from
device_queue_manager_uninit() on 5.15 (bad backport), black screen on
Picasso APU without IOMMU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2172819/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
