Re: [PATCH v2 2/2] drm/amdgpu: improve the amdgpu device init progress in sriov mode

[Lazar, Lijo]

On 07-Jul-26 5:25 PM, Christian König wrote:
On 7/7/26 13:30, chong li wrote:
v2:
Some legacy ASICs do not support amdgpu_virt_request_init_data().
On platform ESXi, the PF can access VF VRAM only after the VF
enters full gpu access mode.
Only move the full access request later when the required conditions are met

This is adjusting the driver to the hypervisor and not the hypervisor to the 
driver.

If the existing init order doesn't work on ESXi then that is an ESXi problem 
and needs to be fixed there.

So still absolutely clear NAK.


v1:
Move the initialization of non-GPU resources
out of the full GPU access region during AMDGPU device initialization

background:

After the amdgpu driver sends GPU_INIT_DATA to the host SR-IOV driver,
the host dumps the IP discovery/VBIOS/PF-VF exchange data into the VF FB.
The VF can then read these data blocks without requesting full GPU access.
This is the purpose of the GPU_INIT_DATA event in the SR-IOV init flow.

However, during ip_early_init, TTM is not initialized yet.
The current amdgpu driver therefore falls back to reading these data
through MM_INDEX/MM_DATA. That path requires register access,
which still needs full GPU access.

As a result, even though amdgpu sends GPU_INIT_DATA and
the host prepares the init data early,
the full GPU access window is not reduced because
the guest still needs full GPU access to copy the data.

To fix this, use amdgpu_device_read_fb_via_bar0() to copy
the init data from VF FB before TTM is ready.
For this early copy, the guest no longer needs full GPU access
so the full GPU access request can be moved later.

You are just explaining the chicken and egg problem here instead of giving an 
actual justification.


I guess there is no other technical reason to disallow accessing FB 
using visible BAR range. The only reason of going through HDP indirect 
path could be because the offsets could be outside of visible range. In 
SRIOV case, if host driver guarantees discovery offsets in visible FB 
range, there is no reason to disallow that.

Thanks,
Lijo


Regards,
Christian.


Signed-off-by: chong li <chong...@amd.com>
Co-authored-by: Cursor <cursorag...@cursor.com>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 32 +++++++++++++++++++---
  drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c      |  4 +--
  2 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 610d82b79de3..ec353f4dd0d2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -38,6 +38,10 @@
  #include <linux/apple-gmux.h>
  #include <linux/nospec.h>
  
+#ifdef CONFIG_X86
+#include <asm/hypervisor.h>
+#endif
+
  #include <drm/drm_atomic_helper.h>
  #include <drm/drm_client_event.h>
  #include <drm/drm_crtc_helper.h>
@@ -1973,6 +1977,17 @@ static struct pci_dev *amdgpu_device_find_parent(struct 
amdgpu_device *adev)
        return parent;
  }
  
+static bool amdgpu_device_delay_full_gpu_access(struct amdgpu_device *adev)
+{
+#ifdef CONFIG_X86
+       return adev->virt.req_init_data_ver > 0 &&
+               (hypervisor_is_type(X86_HYPER_VMWARE) ||
+               hypervisor_is_type(X86_HYPER_MS_HYPERV));
+#else
+       return false;
+#endif
+}
+
  /**
   * amdgpu_device_ip_early_init - run early init for hardware IPs
   *
@@ -1987,16 +2002,19 @@ static int amdgpu_device_ip_early_init(struct 
amdgpu_device *adev)
  {
        struct amdgpu_ip_block *ip_block;
        struct pci_dev *parent;
-       bool total, skip_bios;
+       bool total, skip_bios, delay_full_gpu_access = false;
        uint32_t bios_flags;
        int i, r;
  
  	amdgpu_device_enable_virtual_display(adev);
  
  	if (amdgpu_sriov_vf(adev)) {
-               r = amdgpu_virt_request_full_gpu(adev, true);
-               if (r)
-                       return r;
+               delay_full_gpu_access = 
amdgpu_device_delay_full_gpu_access(adev);
+               if (!delay_full_gpu_access) {
+                       r = amdgpu_virt_request_full_gpu(adev, true);
+                       if (r)
+                               return r;
+               }
  
  		r = amdgpu_virt_init_critical_region(adev);
                if (r)
@@ -2159,6 +2177,12 @@ static int amdgpu_device_ip_early_init(struct 
amdgpu_device *adev)
        if (!total)
                return -ENODEV;
  
+	if (amdgpu_sriov_vf(adev) && delay_full_gpu_access) {
+               r = amdgpu_virt_request_full_gpu(adev, true);
+               if (r)
+                       return r;
+       }
+
        if (adev->gmc.xgmi.supported)
                amdgpu_xgmi_early_init(adev);
  
diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
index 9a40107a0869..340703d89d6b 100644
--- a/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
+++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c
@@ -185,8 +185,8 @@ static int xgpu_ai_send_access_requests(struct 
amdgpu_device *adev,
        } else if (req == IDH_REQ_GPU_INIT_DATA){
                /* Dummy REQ_GPU_INIT_DATA handling */
                r = xgpu_ai_poll_msg(adev, IDH_REQ_GPU_INIT_DATA_READY);
-               /* version set to 0 since dummy */
-               adev->virt.req_init_data_ver = 0;    
+               /* Version is set to 1 since GPU_CRIT_REGION_V1 */
+               adev->virt.req_init_data_ver = GPU_CRIT_REGION_V1;
        }
  
  	return 0;