On Fri, Apr 03, 2020 at 06:05:55PM +0800, Huang Rui wrote:
> On Fri, Apr 03, 2020 at 05:22:28PM +0800, Liang, Prike wrote:
> >
> > > -----Original Message-----
> > > From: Huang, Ray <ray.hu...@amd.com>
> > > Sent: Friday, April 3, 2020 2:27 PM
> > > To: Liang, Prike <prike.li...@amd.com>
> > > Cc: amd-gfx@lists.freedesktop.org; Quan, Evan <evan.q...@amd.com>;
> > > Deucher, Alexander <alexander.deuc...@amd.com>; Kuehling, Felix
> > > <felix.kuehl...@amd.com>
> > > Subject: Re: [PATCH] drm/amdgpu: fix gfx hang during suspend with video
> > > playback
> > >
> > > (+ Felix)
> > >
> > > On Fri, Apr 03, 2020 at 12:07:53PM +0800, Liang, Prike wrote:
> > > > The system will be hang up during S3 as SMU is pending at GC not
> > > > respose the register CP_HQD_ACTIVE access request and this issue can
> > > > be fixed by adding RLC safe mode guard before each HQD map/unmap
> > > > retrive opt.
> > >
> > > We need more information for the issue, does the map/unmap is required
> > > for MAP_QUEUES/UNMAP_QUEUES packets or writing with MMIO or both?
> > >
> > [Prike] The issue hang up at MP1 was trying to read register
> > RSMU_RESIDENCY_COUNTER_GC
> > but did not get response from GFX, since GFX was busy at reading register
> > CP_HQD_ACTIVE.
> > Moreover, when disabled GFXOFF this issue also can't see so there is likely
> > to perform
> > register accessed at GFXOFF CGPG/CGCG enter stage. As for only this
> > issue, that seems just
> > MMIO access failed case which occurred under QUEUE map/unmap status check.
> >
>
> While we start to do S3, we will disable gfxoff at start of suspend. Then
> in this point, the gfx should be always in "on" state.
>
> > > From your patch, you just protect the kernel kiq and user queue. What
> > > about
> > > other kernel compute queues? HIQ?
> > >
> > [Prike] So far just find the KIQ/CPQ/DIQ map/unmap will inquire the
> > CP_HQD_ACTIVE status by MMIO accessing,
> > therefore just guard the KIQ and some type user queue now. Regarding HIQ
> > map and ummap which used the method of submitting configuration packet.
> >
>
> KIQ itself init/unit should be always under gfx on state. Can you give a
> check the result if not add enter/exit rlc safe mode around it?
Wait... In your case, the system didn't load any user queues because no
ROCm based application is running. So the issue is probably caused by KIQ
itself init/unit, can you confirm?
Thanks,
Ray
>
> Hi Felix, maybe we need to use packets with kiq to map all user queues.
>
> Thanks,
> Ray
>
> > > Thanks,
> > > Ray
> > >
> > > >
> > > > Signed-off-by: Prike Liang <prike.li...@amd.com>
> > > > Tested-by: Mengbing Wang <mengbing.w...@amd.com>
> > > > ---
> > > > drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c | 6 ++++++
> > > > drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c | 4 ++++
> > > > 2 files changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
> > > > b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
> > > > index df841c2..e265063 100644
> > > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
> > > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c
> > > > @@ -232,6 +232,7 @@ int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void
> > > *mqd, uint32_t pipe_id,
> > > > uint32_t *mqd_hqd;
> > > > uint32_t reg, hqd_base, data;
> > > >
> > > > + amdgpu_gfx_rlc_enter_safe_mode(adev);
> > > > m = get_mqd(mqd);
> > > >
> > > > acquire_queue(kgd, pipe_id, queue_id); @@ -299,6 +300,7 @@ int
> > > > kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
> > > >
> > > > release_queue(kgd);
> > > >
> > > > + amdgpu_gfx_rlc_exit_safe_mode(adev);
> > > > return 0;
> > > > }
> > > >
> > > > @@ -497,6 +499,7 @@ bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev
> > > *kgd, uint64_t queue_address,
> > > > bool retval = false;
> > > > uint32_t low, high;
> > > >
> > > > + amdgpu_gfx_rlc_enter_safe_mode(adev);
> > > > acquire_queue(kgd, pipe_id, queue_id);
> > > > act = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE));
> > > > if (act) {
> > > > @@ -508,6 +511,7 @@ bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev
> > > *kgd, uint64_t queue_address,
> > > > retval = true;
> > > > }
> > > > release_queue(kgd);
> > > > + amdgpu_gfx_rlc_exit_safe_mode(adev);
> > > > return retval;
> > > > }
> > > >
> > > > @@ -541,6 +545,7 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd,
> > > void *mqd,
> > > > uint32_t temp;
> > > > struct v9_mqd *m = get_mqd(mqd);
> > > >
> > > > + amdgpu_gfx_rlc_enter_safe_mode(adev);
> > > > if (adev->in_gpu_reset)
> > > > return -EIO;
> > > >
> > > > @@ -577,6 +582,7 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd,
> > > void *mqd,
> > > > }
> > > >
> > > > release_queue(kgd);
> > > > + amdgpu_gfx_rlc_exit_safe_mode(adev);
> > > > return 0;
> > > > }
> > > >
> > > > diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
> > > > b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
> > > > index 1fea077..ee107d9 100644
> > > > --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
> > > > +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
> > > > @@ -3533,6 +3533,7 @@ static int gfx_v9_0_kiq_init_register(struct
> > > amdgpu_ring *ring)
> > > > struct v9_mqd *mqd = ring->mqd_ptr;
> > > > int j;
> > > >
> > > > + amdgpu_gfx_rlc_enter_safe_mode(adev);
> > > > /* disable wptr polling */
> > > > WREG32_FIELD15(GC, 0, CP_PQ_WPTR_POLL_CNTL, EN, 0);
> > > >
> > > > @@ -3629,6 +3630,7 @@ static int gfx_v9_0_kiq_init_register(struct
> > > amdgpu_ring *ring)
> > > > if (ring->use_doorbell)
> > > > WREG32_FIELD15(GC, 0, CP_PQ_STATUS, DOORBELL_ENABLE,
> > > 1);
> > > >
> > > > + amdgpu_gfx_rlc_exit_safe_mode(adev);
> > > > return 0;
> > > > }
> > > >
> > > > @@ -3637,6 +3639,7 @@ static int gfx_v9_0_kiq_fini_register(struct
> > > amdgpu_ring *ring)
> > > > struct amdgpu_device *adev = ring->adev;
> > > > int j;
> > > >
> > > > + amdgpu_gfx_rlc_enter_safe_mode(adev);
> > > > /* disable the queue if it's active */
> > > > if (RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE) & 1) {
> > > >
> > > > @@ -3668,6 +3671,7 @@ static int gfx_v9_0_kiq_fini_register(struct
> > > amdgpu_ring *ring)
> > > > WREG32_SOC15_RLC(GC, 0, mmCP_HQD_PQ_WPTR_HI, 0);
> > > > WREG32_SOC15_RLC(GC, 0, mmCP_HQD_PQ_WPTR_LO, 0);
> > > >
> > > > + amdgpu_gfx_rlc_exit_safe_mode(adev);
> > > > return 0;
> > > > }
> > > >
> > > > --
> > > > 2.7.4
> > > >
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=02%7C01%7Cray.huang%40amd.com%7C040563ff26374383ec6108d7d7b6a2bb%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637215053543776633&sdata=COMvG7W4%2Fl7aKDAV8Qgbl%2F3myW0HCSz7qk014OLUzrY%3D&reserved=0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
