A compute partition switch tears down the KFD device through amdgpu_amdkfd_device_fini_sw() without suspending KFD first, while kgd2kfd_device_exit() drops a suspend reference. Every switch leaves kfd_locked one below zero, so on a hive the counter ends up deeply negative.
kfd_locked gates the eviction sweep on "++kfd_locked == 1". While it is negative a GPU reset never evicts user processes, nothing restores them afterwards, and page tables torn down with VRAM are never rebuilt, so the first queue mapped after the reset takes a VM fault. Take the reference in amdgpu_xcp_pre_partition_switch() so it pairs with the one kgd2kfd_device_exit() drops, and only drop it for a device that finished initialisation, matching the early return in kgd2kfd_suspend(). Signed-off-by: Xiang Liu <[email protected]> --- drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c | 5 ++++- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 6 +++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c index b4ebc157fd06..e2e7d101b3c4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c @@ -698,8 +698,11 @@ int amdgpu_xcp_pre_partition_switch(struct amdgpu_xcp_mgr *xcp_mgr, u32 flags) * Stop user queues and threads, and make sure GPU is empty of work. */ - if (flags & AMDGPU_XCP_OPS_KFD) + /* kgd2kfd_device_exit() drops the suspend reference taken here */ + if (flags & AMDGPU_XCP_OPS_KFD) { + amdgpu_amdkfd_suspend(xcp_mgr->adev, true); amdgpu_amdkfd_device_fini_sw(xcp_mgr->adev); + } return 0; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 729b35841e3b..d76920adc466 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -1014,12 +1014,12 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd) kfd_gtt_sa_fini(kfd); amdgpu_amdkfd_free_kernel_mem(kfd->adev, &kfd->gtt_mem); mutex_destroy(&kfd->profiler_lock); + + /* pairs with the reference taken in kgd2kfd_suspend() */ + kgd2kfd_unlock_kfd(NULL); } kfree(kfd); - - /* after remove a kfd device unlock kfd driver */ - kgd2kfd_unlock_kfd(NULL); } int kgd2kfd_pre_reset(struct kfd_dev *kfd, -- 2.34.1
