v2: Some legacy ASICs do not support amdgpu_virt_request_init_data(). On platform ESXi, the PF can access VF VRAM only after the VF enters full gpu access mode. Only move the full access request later when the required conditions are met
v1: Move the initialization of non-GPU resources out of the full GPU access region during AMDGPU device initialization background: After the amdgpu driver sends GPU_INIT_DATA to the host SR-IOV driver, the host dumps the IP discovery/VBIOS/PF-VF exchange data into the VF FB. The VF can then read these data blocks without requesting full GPU access. This is the purpose of the GPU_INIT_DATA event in the SR-IOV init flow. However, during ip_early_init, TTM is not initialized yet. The current amdgpu driver therefore falls back to reading these data through MM_INDEX/MM_DATA. That path requires register access, which still needs full GPU access. As a result, even though amdgpu sends GPU_INIT_DATA and the host prepares the init data early, the full GPU access window is not reduced because the guest still needs full GPU access to copy the data. To fix this, use amdgpu_device_read_fb_via_bar0() to copy the init data from VF FB before TTM is ready. For this early copy, the guest no longer needs full GPU access so the full GPU access request can be moved later. Signed-off-by: chong li <[email protected]> Co-authored-by: Cursor <[email protected]> --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 32 +++++++++++++++++++--- drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c | 4 +-- 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 610d82b79de3..ec353f4dd0d2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -38,6 +38,10 @@ #include <linux/apple-gmux.h> #include <linux/nospec.h> +#ifdef CONFIG_X86 +#include <asm/hypervisor.h> +#endif + #include <drm/drm_atomic_helper.h> #include <drm/drm_client_event.h> #include <drm/drm_crtc_helper.h> @@ -1973,6 +1977,17 @@ static struct pci_dev *amdgpu_device_find_parent(struct amdgpu_device *adev) return parent; } +static bool amdgpu_device_delay_full_gpu_access(struct amdgpu_device *adev) +{ +#ifdef CONFIG_X86 + return adev->virt.req_init_data_ver > 0 && + (hypervisor_is_type(X86_HYPER_VMWARE) || + hypervisor_is_type(X86_HYPER_MS_HYPERV)); +#else + return false; +#endif +} + /** * amdgpu_device_ip_early_init - run early init for hardware IPs * @@ -1987,16 +2002,19 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) { struct amdgpu_ip_block *ip_block; struct pci_dev *parent; - bool total, skip_bios; + bool total, skip_bios, delay_full_gpu_access = false; uint32_t bios_flags; int i, r; amdgpu_device_enable_virtual_display(adev); if (amdgpu_sriov_vf(adev)) { - r = amdgpu_virt_request_full_gpu(adev, true); - if (r) - return r; + delay_full_gpu_access = amdgpu_device_delay_full_gpu_access(adev); + if (!delay_full_gpu_access) { + r = amdgpu_virt_request_full_gpu(adev, true); + if (r) + return r; + } r = amdgpu_virt_init_critical_region(adev); if (r) @@ -2159,6 +2177,12 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) if (!total) return -ENODEV; + if (amdgpu_sriov_vf(adev) && delay_full_gpu_access) { + r = amdgpu_virt_request_full_gpu(adev, true); + if (r) + return r; + } + if (adev->gmc.xgmi.supported) amdgpu_xgmi_early_init(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c index 9a40107a0869..340703d89d6b 100644 --- a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c +++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c @@ -185,8 +185,8 @@ static int xgpu_ai_send_access_requests(struct amdgpu_device *adev, } else if (req == IDH_REQ_GPU_INIT_DATA){ /* Dummy REQ_GPU_INIT_DATA handling */ r = xgpu_ai_poll_msg(adev, IDH_REQ_GPU_INIT_DATA_READY); - /* version set to 0 since dummy */ - adev->virt.req_init_data_ver = 0; + /* Version is set to 1 since GPU_CRIT_REGION_V1 */ + adev->virt.req_init_data_ver = GPU_CRIT_REGION_V1; } return 0; -- 2.48.1 >Re: [PATCH 2/2] drm/amdgpu: improve the amdgpu device init progress in sriov >mode >On 7/1/26 07:53, chong li wrote: >> Move the initialization of non-GPU resources out of the full GPU >> access region during AMDGPU device initialization > >As far as I can see that won't work like this. > >There are a lot of steps which require full GPU access. > >What exactly is the justification of the change? > >Regards, >Christian. > >> >> Signed-off-by: chong li <[email protected]> >> Co-authored-by: Cursor <[email protected]> >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++---- >> 1 file changed, 6 insertions(+), 4 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> index 610d82b79de3..c2ce4659ddc7 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> @@ -1994,10 +1994,6 @@ static int amdgpu_device_ip_early_init(struct >> amdgpu_device *adev) >> amdgpu_device_enable_virtual_display(adev); >> >> if (amdgpu_sriov_vf(adev)) { >> - r = amdgpu_virt_request_full_gpu(adev, true); >> - if (r) >> - return r; >> - >> r = amdgpu_virt_init_critical_region(adev); >> if (r) >> return r; >> @@ -2159,6 +2155,12 @@ static int amdgpu_device_ip_early_init(struct >> amdgpu_device *adev) >> if (!total) >> return -ENODEV; >> >> + if (amdgpu_sriov_vf(adev)) { >> + r = amdgpu_virt_request_full_gpu(adev, true); >> + if (r) >> + return r; >> + } >> + >> if (adev->gmc.xgmi.supported) >> amdgpu_xgmi_early_init(adev); >>
