Re: [PATCH hmm 00/15] Consolidate the mmu notifier interval_tree and locking

[Yang, Philip]

On 2019-10-17 4:54 a.m., Christian König wrote:
> Am 16.10.19 um 18:04 schrieb Jason Gunthorpe:
>> On Wed, Oct 16, 2019 at 10:58:02AM +0200, Christian König wrote:
>>> Am 15.10.19 um 20:12 schrieb Jason Gunthorpe:
 From: Jason Gunthorpe 

 8 of the mmu_notifier using drivers (i915_gem, radeon_mn, umem_odp, 
 hfi1,
 scif_dma, vhost, gntdev, hmm) drivers are using a common pattern where
 they only use invalidate_range_start/end and immediately check the
 invalidating range against some driver data structure to tell if the
 driver is interested. Half of them use an interval_tree, the others are
 simple linear search lists.

 Of the ones I checked they largely seem to have various kinds of races,
 bugs and poor implementation. This is a result of the complexity in how
 the notifier interacts with get_user_pages(). It is extremely 
 difficult to
 use it correctly.

 Consolidate all of this code together into the core mmu_notifier and
 provide a locking scheme similar to hmm_mirror that allows the user to
 safely use get_user_pages() and reliably know if the page list still
 matches the mm.
>>> That sounds really good, but could you outline for a moment how that is
>>> archived?
>> It uses the same basic scheme as hmm and rdma odp, outlined in the
>> revisions to hmm.rst later on.
>>
>> Basically,
>>
>>   seq = mmu_range_read_begin(&mrn);
>>
>>   // This is a speculative region
>>   .. get_user_pages()/hmm_range_fault() ..
> 
> How do we enforce that this get_user_pages()/hmm_range_fault() doesn't 
> see outdated page table information?
> 
> In other words how the the following race prevented:
> 
> CPU A CPU B
> invalidate_range_start()
>        mmu_range_read_begin()
>        get_user_pages()/hmm_range_fault()
> Updating the ptes
> invalidate_range_end()
> 
> 
> I mean get_user_pages() tries to circumvent this issue by grabbing a 
> reference to the pages in question, but that isn't sufficient for the 
> SVM use case.
> 
> That's the reason why we had this horrible solution with a r/w lock and 
> a linked list of BOs in an interval tree.
> 
> Regards,
> Christian.
get_user_pages/hmm_range_fault() and invalidate_range_start() both are 
called while holding mm->map_sem, so they are always serialized.

Philip
> 
>>   // Result cannot be derferenced
>>
>>   take_lock(driver->update);
>>   if (mmu_range_read_retry(&mrn, range.notifier_seq) {
>>  // collision! The results are not correct
>>  goto again
>>   }
>>
>>   // no collision, and now under lock. Now we can de-reference the 
>> pages/etc
>>   // program HW
>>   // Now the invalidate callback is responsible to synchronize against 
>> changes
>>   unlock(driver->update)
>>
>> Basically, anything that was using hmm_mirror correctly transisions
>> over fairly trivially, just with the modification to store a sequence
>> number to close that race described in the hmm commit.
>>
>> For something like AMD gpu I expect it to transition to use dma_fence
>> from the notifier for coherency right before it unlocks driver->update.
>>
>> Jason
>> ___
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> 
> ___
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: disable c-states on xgmi perfmons

[Yang, Philip]

I got compiler warnings after update this morning, because the variables 
are not initialized in df_v3_6_set_df_cstate() return failed path.

  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.o
   CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.o
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/df_v3_6.c:96:8:
 
warning: return type defaults to ‘int’ [-Wreturn-type]
  static df_v3_6_set_df_cstate(struct amdgpu_device *adev, int allow)
 ^
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/df_v3_6.c: 
In function ‘df_v3_6_pmc_get_count’:
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/df_v3_6.c:564:22:
 
warning: ‘hi_val’ may be used uninitialized in this function 
[-Wmaybe-uninitialized]
*count  = ((hi_val | 0ULL) << 32) | (lo_val | 0ULL);
   ^~~
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/df_v3_6.c:564:47:
 
warning: ‘lo_val’ may be used uninitialized in this function 
[-Wmaybe-uninitialized]
*count  = ((hi_val | 0ULL) << 32) | (lo_val | 0ULL);
^~~
   CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.o


Regards,
Philip

On 2019-10-16 10:54 p.m., Quan, Evan wrote:
> Reviewed-by: Evan Quan 
> 
>> -Original Message-
>> From: Kim, Jonathan 
>> Sent: 2019年10月17日 10:06
>> To: amd-gfx@lists.freedesktop.org
>> Cc: Kuehling, Felix ; Quan, Evan
>> ; Kim, Jonathan ; Kim,
>> Jonathan 
>> Subject: [PATCH] drm/amdgpu: disable c-states on xgmi perfmons
>>
>> read or writes to df registers when gpu df is in c-states will result in
>> hang.  df c-states should be disabled prior to read or writes then
>> re-enabled after read or writes.
>>
>> v2: use old powerplay routines for vega20
>>
>> Change-Id: I6d5a83e4fe13e29c73dfb03a94fe7c611e867fec
>> Signed-off-by: Jonathan Kim 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/df_v3_6.c | 36
>> +++-
>>   1 file changed, 35 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
>> b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
>> index 16fbd2bc8ad1..f403c62c944e 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
>> @@ -93,6 +93,21 @@ const struct attribute_group *df_v3_6_attr_groups[] =
>> {
>>  NULL
>>   };
>>
>> +static df_v3_6_set_df_cstate(struct amdgpu_device *adev, int allow)
>> +{
>> +int r = 0;
>> +
>> +if (is_support_sw_smu(adev)) {
>> +r = smu_set_df_cstate(&adev->smu, allow);
>> +} else if (adev->powerplay.pp_funcs
>> +&& adev->powerplay.pp_funcs->set_df_cstate) {
>> +r = adev->powerplay.pp_funcs->set_df_cstate(
>> +adev->powerplay.pp_handle, allow);
>> +}
>> +
>> +return r;
>> +}
>> +
>>   static uint64_t df_v3_6_get_fica(struct amdgpu_device *adev,
>>   uint32_t ficaa_val)
>>   {
>> @@ -102,6 +117,9 @@ static uint64_t df_v3_6_get_fica(struct
>> amdgpu_device *adev,
>>  address = adev->nbio.funcs->get_pcie_index_offset(adev);
>>  data = adev->nbio.funcs->get_pcie_data_offset(adev);
>>
>> +if (df_v3_6_set_df_cstate(adev, DF_CSTATE_DISALLOW))
>> +return 0x;
>> +
>>  spin_lock_irqsave(&adev->pcie_idx_lock, flags);
>>  WREG32(address,
>> smnDF_PIE_AON_FabricIndirectConfigAccessAddress3);
>>  WREG32(data, ficaa_val);
>> @@ -114,6 +132,8 @@ static uint64_t df_v3_6_get_fica(struct
>> amdgpu_device *adev,
>>
>>  spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
>>
>> +df_v3_6_set_df_cstate(adev, DF_CSTATE_ALLOW);
>> +
>>  return (((ficadh_val & 0x) << 32) | ficadl_val);
>>   }
>>
>> @@ -125,6 +145,9 @@ static void df_v3_6_set_fica(struct amdgpu_device
>> *adev, uint32_t ficaa_val,
>>  address = adev->nbio.funcs->get_pcie_index_offset(adev);
>>  data = adev->nbio.funcs->get_pcie_data_offset(adev);
>>
>> +if (df_v3_6_set_df_cstate(adev, DF_CSTATE_DISALLOW))
>> +return;
>> +
>>  spin_lock_irqsave(&adev->pcie_idx_lock, flags);
>>  WREG32(address,
>> smnDF_PIE_AON_FabricIndirectConfigAccessAddress3);
>>  WREG32(data, ficaa_val);
>> @@ -134,8 +157,9 @@ static void df_v3_6_set_fica(struct amdgpu_device
>> *adev, uint32_t ficaa_val,
>>
>>  WREG32(address,
>> smnDF_PIE_AON_FabricIndirectConfigAccessDataHi3);
>>  WREG32(data, ficadh_val);
>> -
>>  spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
>> +
>> +df_v3_6_set_df_cstate(adev, DF_CSTATE_ALLOW);
>>   }
>>
>>   /*
>> @@ -153,12 +177,17 @@ static void df_v3_6_perfmon_rreg(struct
>> amdgpu_device *adev,
>>  address = adev->nbio.funcs->get_pcie_index_offset(adev);
>>  data = adev->nbio.funcs->get_pcie_data_offset(adev);
>>
>> +if (df_v3_6_set_df_cstate(adev, DF_CSTATE_DISALLOW))
>> +return;
>> +
>>  spin_lock_irqsave(&adev->pcie_idx_lock, flags);
>>  WREG32(address, lo_addr);
>>  

Re: [PATCH] drm/amdgpu: fix compiler warnings for df perfmons

[Yang, Philip]

Reviewed-by: Philip Yang 

On 2019-10-17 1:56 p.m., Kim, Jonathan wrote:
> fixing compiler warnings in df v3.6 for c-state toggle and pmc count.
> 
> Change-Id: I74f8f1eafccf523a89d60d005e3549235f75c6b8
> Signed-off-by: Jonathan Kim 
> ---
>   drivers/gpu/drm/amd/amdgpu/df_v3_6.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c 
> b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
> index f403c62c944e..e1cf7e9c616a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
> +++ b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
> @@ -93,7 +93,7 @@ const struct attribute_group *df_v3_6_attr_groups[] = {
>   NULL
>   };
>   
> -static df_v3_6_set_df_cstate(struct amdgpu_device *adev, int allow)
> +static int df_v3_6_set_df_cstate(struct amdgpu_device *adev, int allow)
>   {
>   int r = 0;
>   
> @@ -546,7 +546,7 @@ static void df_v3_6_pmc_get_count(struct amdgpu_device 
> *adev,
> uint64_t config,
> uint64_t *count)
>   {
> - uint32_t lo_base_addr, hi_base_addr, lo_val, hi_val;
> + uint32_t lo_base_addr, hi_base_addr, lo_val = 0, hi_val = 0;
>   *count = 0;
>   
>   switch (adev->asic_type) {
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdkfd: kfd open return failed if device is locked

[Yang, Philip]

If device is locked for suspend and resume, kfd open should return
failed -EAGAIN without creating process, otherwise the application exit
to release the process will hang to wait for resume is done if the suspend
and resume is stuck somewhere. This is backtrace:

[Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
than 120 seconds.
[Thu Oct 17 16:43:37 2019]   Not tainted
5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
[Thu Oct 17 16:43:37 2019] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
0x8000
[Thu Oct 17 16:43:37 2019] Call Trace:
[Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
[Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
[Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
[Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
[Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
[Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]
kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
[Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
[amdgpu]
[Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
[Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
[Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
[Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
[Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
[Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
[Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
[Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
[Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
[Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
[Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index d9e36dbf13d5..40d75c39f08e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
*filep)
return -EPERM;
}
 
+   if (kfd_is_locked())
+   return -EAGAIN;
+
process = kfd_create_process(filep);
if (IS_ERR(process))
return PTR_ERR(process);
 
-   if (kfd_is_locked())
-   return -EAGAIN;
-
dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
process->pasid, process->is_32bit_user_mode);
 
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdkfd: kfd open return failed if device is locked

[Yang, Philip]

On 2019-10-18 11:40 a.m., Kuehling, Felix wrote:
> On 2019-10-18 10:27 a.m., Yang, Philip wrote:
>> If device is locked for suspend and resume, kfd open should return
>> failed -EAGAIN without creating process, otherwise the application exit
>> to release the process will hang to wait for resume is done if the suspend
>> and resume is stuck somewhere. This is backtrace:
> 
> This doesn't fix processes that were created before suspend/resume got
> stuck. They would still get stuck with the same backtrace. So this is
> jut a band-aid. The real underlying problem, that is not getting
> addressed, is suspend/resume getting stuck.
> 
> Am I missing something?
> 
This is to address application stuck to quit issue after suspend/resume 
got stuck. The real underlying suspend/resume issue should be addressed 
separately.

I will submit v2 patch to fix processes that were created before 
suspend/resume got stuck.

Philip

> Regards,
>     Felix
> 
> 
>>
>> [Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
>> than 120 seconds.
>> [Thu Oct 17 16:43:37 2019]   Not tainted
>> 5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
>> [Thu Oct 17 16:43:37 2019] "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> [Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
>> 0x8000
>> [Thu Oct 17 16:43:37 2019] Call Trace:
>> [Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
>> [Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
>> [Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
>> [Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
>> [Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
>> [Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]
>> kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
>> [Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
>> [Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
>> [Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
>> [Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
>> [Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
>> [Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
>> [Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
>> [Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
>> [Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
>> [Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>
>> Signed-off-by: Philip Yang 
>> ---
>>drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 6 +++---
>>1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> index d9e36dbf13d5..40d75c39f08e 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> @@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
>> *filep)
>>  return -EPERM;
>>  }
>>
>> +if (kfd_is_locked())
>> +return -EAGAIN;
>> +
>>  process = kfd_create_process(filep);
>>  if (IS_ERR(process))
>>  return PTR_ERR(process);
>>
>> -if (kfd_is_locked())
>> -return -EAGAIN;
>> -
>>  dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
>>  process->pasid, process->is_32bit_user_mode);
>>
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH v2] drm/amdkfd: kfd open return failed if device is locked

[Yang, Philip]

If device is locked for suspend and resume, kfd open should return
failed -EAGAIN without creating process, otherwise the application exit
to release the process will hang to wait for resume is done if the suspend
and resume is stuck somewhere. This is backtrace:

v2: fix processes that were created before suspend/resume got stuck

[Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
than 120 seconds.
[Thu Oct 17 16:43:37 2019]   Not tainted
5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
[Thu Oct 17 16:43:37 2019] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
0x8000
[Thu Oct 17 16:43:37 2019] Call Trace:
[Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
[Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
[Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
[Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
[Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
[Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]
kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
[Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
[amdgpu]
[Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
[Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
[Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
[Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
[Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
[Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
[Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
[Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
[Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
[Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
[Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c   | 6 +++---
 drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c | 6 ++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index d9e36dbf13d5..40d75c39f08e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
*filep)
return -EPERM;
}
 
+   if (kfd_is_locked())
+   return -EAGAIN;
+
process = kfd_create_process(filep);
if (IS_ERR(process))
return PTR_ERR(process);
 
-   if (kfd_is_locked())
-   return -EAGAIN;
-
dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
process->pasid, process->is_32bit_user_mode);
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 8509814a6ff0..3784013b92a0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -128,6 +128,12 @@ void kfd_process_dequeue_from_all_devices(struct 
kfd_process *p)
 {
struct kfd_process_device *pdd;
 
+   /* If suspend/resume got stuck, dqm_lock is hold,
+* skip process_termination_cpsch to avoid deadlock
+*/
+   if (kfd_is_locked())
+   return;
+
list_for_each_entry(pdd, &p->per_device_data, per_device_list)
kfd_process_dequeue_from_device(pdd);
 }
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

If device reset/suspend/resume failed for some reason, dqm lock is
hold forever and this causes deadlock. Below is a kernel backtrace when
application open kfd after suspend/resume failed.

Instead of holding dqm lock in pre_reset and releasing dqm lock in
post_reset, add dqm->device_stopped flag which is modified in
dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
because write/read are all inside dqm lock.

For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
device_stopped flag before sending the updated runlist.

Backtrace of dqm lock deadlock:

[Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
than 120 seconds.
[Thu Oct 17 16:43:37 2019]   Not tainted
5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
[Thu Oct 17 16:43:37 2019] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
0x8000
[Thu Oct 17 16:43:37 2019] Call Trace:
[Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
[Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
[Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
[Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
[Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
[Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]
kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
[Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
[amdgpu]
[Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
[Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
[Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
[Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
[Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
[Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
[Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
[Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
[Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
[Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
[Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Suggested-by: Felix Kuehling 
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c|  6 +++---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c |  5 -
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c   | 13 ++---
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h   |  1 +
 4 files changed, 14 insertions(+), 11 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index d9e36dbf13d5..40d75c39f08e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
*filep)
return -EPERM;
}
 
+   if (kfd_is_locked())
+   return -EAGAIN;
+
process = kfd_create_process(filep);
if (IS_ERR(process))
return PTR_ERR(process);
 
-   if (kfd_is_locked())
-   return -EAGAIN;
-
dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
process->pasid, process->is_32bit_user_mode);
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 8f4b24e84964..4fa8834ce7cb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -730,9 +730,6 @@ int kgd2kfd_pre_reset(struct kfd_dev *kfd)
return 0;
kgd2kfd_suspend(kfd);
 
-   /* hold dqm->lock to prevent further execution*/
-   dqm_lock(kfd->dqm);
-
kfd_signal_reset_event(kfd);
return 0;
 }
@@ -750,8 +747,6 @@ int kgd2kfd_post_reset(struct kfd_dev *kfd)
if (!kfd->init_complete)
return 0;
 
-   dqm_unlock(kfd->dqm);
-
ret = kfd_resume(kfd);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 81fb545cf42c..04a40fabe9d7 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -915,7 +915,8 @@ static int start_nocpsch(struct device_queue_manager *dqm)

if (dqm->dev->device_info->asic_family == CHIP_HAWAII)
return pm_init(&dqm->packets, dqm);
-   
+   dqm->device_stopped = false;
+
return 0;
 }
 
@@ -923,7 +924,8 @@ static int stop_nocpsch(struct device_queue_manager *dqm)
 {
if (dqm->dev->device_info->asic_family == CHIP_HAWAII)
pm_uninit(&dqm->packets);
-   
+   dqm->device_stopped = true;
+
return 0;
 }
 
@@ -1074,6 +1076,7 @@ static int start_cpsch(struct device_queue_manager *dqm)
dqm_lock(dqm);
/* clear hang status when driver try to start 

Re: [PATCH] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

On 2019-10-21 9:03 p.m., Kuehling, Felix wrote:
> 
> On 2019-10-21 5:04 p.m., Yang, Philip wrote:
>> If device reset/suspend/resume failed for some reason, dqm lock is
>> hold forever and this causes deadlock. Below is a kernel backtrace when
>> application open kfd after suspend/resume failed.
>>
>> Instead of holding dqm lock in pre_reset and releasing dqm lock in
>> post_reset, add dqm->device_stopped flag which is modified in
>> dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
>> because write/read are all inside dqm lock.
>>
>> For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
>> device_stopped flag before sending the updated runlist.
> 
> What about the non-HWS case?
> 
> In theory in non-HWS case new queues should be created in evicted state
> while the device (and all processes) are suspended. So we should never
> try to map or unmap queues to HQDs during suspend. But I'd feel better
> with a WARN_ON and error return in the right places to make sure we're
> not missing anything. Basically, we can't call any of the
> load/destroy_mqd functions while suspended.
> 
v2 patch add non-HWS case.

> That reminds me, we also have to add some checks in the debugfs code to
> avoid dumping HQDs of a DQM that's stopped.
>
Thanks, done in v2 patch

> Last comment: dqm->device_stopped must be initialized as true. It will
> get set to false when the device is first started. It may be easier to
> reverse the logic, something like dqm->sched_running that gets
> implicitly initialized as false.
> 
Change to dqm->sched_running in v2 patch.

Regards,
Philip

> Regards,
>     Felix
> 
> 
>>
>> Backtrace of dqm lock deadlock:
>>
>> [Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
>> than 120 seconds.
>> [Thu Oct 17 16:43:37 2019]   Not tainted
>> 5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
>> [Thu Oct 17 16:43:37 2019] "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> [Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
>> 0x8000
>> [Thu Oct 17 16:43:37 2019] Call Trace:
>> [Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
>> [Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
>> [Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
>> [Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
>> [Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
>> [Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]
>> kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
>> [Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
>> [Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
>> [Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
>> [Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
>> [Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
>> [Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
>> [Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
>> [Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
>> [Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
>> [Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>
>> Suggested-by: Felix Kuehling 
>> Signed-off-by: Philip Yang 
>> ---
>>drivers/gpu/drm/amd/amdkfd/kfd_chardev.c|  6 +++---
>>drivers/gpu/drm/amd/amdkfd/kfd_device.c |  5 -
>>.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c   | 13 ++---
>>.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h   |  1 +
>>4 files changed, 14 insertions(+), 11 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> index d9e36dbf13d5..40d75c39f08e 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> @@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
>> *filep)
>>  return -EPERM;
>>  }
>>
>> +if (kfd_is_locked())
>> +return -EAGAIN;
>> +
>>  process = kfd_create_process(filep);
>>  if (IS_ERR(process))
>>  return PTR_ERR(process);
>>
>> -if (kfd_is_locked())
>> -return -EA

[PATCH v2] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

If device reset/suspend/resume failed for some reason, dqm lock is
hold forever and this causes deadlock. Below is a kernel backtrace when
application open kfd after suspend/resume failed.

Instead of holding dqm lock in pre_reset and releasing dqm lock in
post_reset, add dqm->device_stopped flag which is modified in
dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
because write/read are all inside dqm lock.

For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
device_stopped flag before sending the updated runlist.

v2: For no-HWS case, when device is stopped, don't call
load/destroy_mqd for eviction, restore and create queue, and avoid
debugfs dump hdqs.

Backtrace of dqm lock deadlock:

[Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
than 120 seconds.
[Thu Oct 17 16:43:37 2019]   Not tainted
5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
[Thu Oct 17 16:43:37 2019] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
0x8000
[Thu Oct 17 16:43:37 2019] Call Trace:
[Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
[Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
[Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
[Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
[Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
[Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
[amdgpu]
[Thu Oct 17 16:43:37 2019]
kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
[Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
[amdgpu]
[Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
[Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
[Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
[Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
[Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
[Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
[Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
[Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
[Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
[Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
[Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Suggested-by: Felix Kuehling 
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c  |  6 +--
 drivers/gpu/drm/amd/amdkfd/kfd_device.c   |  5 --
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 47 +--
 .../drm/amd/amdkfd/kfd_device_queue_manager.h |  1 +
 4 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index d9e36dbf13d5..40d75c39f08e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
*filep)
return -EPERM;
}
 
+   if (kfd_is_locked())
+   return -EAGAIN;
+
process = kfd_create_process(filep);
if (IS_ERR(process))
return PTR_ERR(process);
 
-   if (kfd_is_locked())
-   return -EAGAIN;
-
dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
process->pasid, process->is_32bit_user_mode);
 
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 8f4b24e84964..4fa8834ce7cb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -730,9 +730,6 @@ int kgd2kfd_pre_reset(struct kfd_dev *kfd)
return 0;
kgd2kfd_suspend(kfd);
 
-   /* hold dqm->lock to prevent further execution*/
-   dqm_lock(kfd->dqm);
-
kfd_signal_reset_event(kfd);
return 0;
 }
@@ -750,8 +747,6 @@ int kgd2kfd_post_reset(struct kfd_dev *kfd)
if (!kfd->init_complete)
return 0;
 
-   dqm_unlock(kfd->dqm);
-
ret = kfd_resume(kfd);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 81fb545cf42c..82e1c6280d13 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -340,6 +340,10 @@ static int create_queue_nocpsch(struct 
device_queue_manager *dqm,
mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
if (q->properties.is_active) {
+   if (!dqm->sched_running) {
+   WARN_ONCE(1, "Load non-HWS mqd while stopped\n");
+   goto add_queue_to_list;
+   }
 
if (WARN(q->process->mm != current->mm,
"should only run in user thread

Re: [PATCH v2] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

On 2019-10-22 2:40 p.m., Grodzovsky, Andrey wrote:
> 
> On 10/22/19 2:38 PM, Grodzovsky, Andrey wrote:
>> On 10/22/19 2:28 PM, Yang, Philip wrote:
>>> If device reset/suspend/resume failed for some reason, dqm lock is
>>> hold forever and this causes deadlock. Below is a kernel backtrace when
>>> application open kfd after suspend/resume failed.
>>>
>>> Instead of holding dqm lock in pre_reset and releasing dqm lock in
>>> post_reset, add dqm->device_stopped flag which is modified in
>>> dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
>>> because write/read are all inside dqm lock.
>>>
>>> For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
>>> device_stopped flag before sending the updated runlist.
>>
>> Is there a chance of race condition here where dqm->device_stopped
>> returns true for some operation (e.g.map_queues_cpsch) but just as it
>> proceeds GPU reset starts  ?
>>
>> Andrey
> 
> 
> Correction -
> 
> dqm->device_stopped returns FALSE
> 
No race condition here, dqm->device_stopped is set to FALSE in 
kgd2kfd_post_reset -> dqm->ops.start(), which the last step of 
amdgpu_device_gpu_recover, so it's safe to do map_queue_cpsch.

Regards,
Philip

> Andrey
> 
>>
>>
>>> v2: For no-HWS case, when device is stopped, don't call
>>> load/destroy_mqd for eviction, restore and create queue, and avoid
>>> debugfs dump hdqs.
>>>
>>> Backtrace of dqm lock deadlock:
>>>
>>> [Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
>>> than 120 seconds.
>>> [Thu Oct 17 16:43:37 2019]   Not tainted
>>> 5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
>>> [Thu Oct 17 16:43:37 2019] "echo 0 >
>>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>> [Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
>>> 0x8000
>>> [Thu Oct 17 16:43:37 2019] Call Trace:
>>> [Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
>>> [Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
>>> [Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
>>> [Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
>>> [Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
>>> [Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
>>> [amdgpu]
>>> [Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
>>> [amdgpu]
>>> [Thu Oct 17 16:43:37 2019]
>>> kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
>>> [Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
>>> [amdgpu]
>>> [Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
>>> [Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
>>> [Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
>>> [Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
>>> [Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
>>> [Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
>>> [Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
>>> [Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
>>> [Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
>>> [Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
>>> [Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>
>>> Suggested-by: Felix Kuehling 
>>> Signed-off-by: Philip Yang 
>>> ---
>>> drivers/gpu/drm/amd/amdkfd/kfd_chardev.c  |  6 +--
>>> drivers/gpu/drm/amd/amdkfd/kfd_device.c   |  5 --
>>> .../drm/amd/amdkfd/kfd_device_queue_manager.c | 47 +--
>>> .../drm/amd/amdkfd/kfd_device_queue_manager.h |  1 +
>>> 4 files changed, 46 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
>>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> index d9e36dbf13d5..40d75c39f08e 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>>> @@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
>>> *filep)
>>> return -EPERM;
>>> }
>>> 
>>> +   if (kfd_is_locked())
>>> +   return -EAGAIN;
>>> +
>>> process = kfd_create_process(filep);
>>> if (IS_ERR(process))
>>> return PTR_ERR(process);
>>

Re: [PATCH v2] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

On 2019-10-22 2:44 p.m., Kuehling, Felix wrote:
> On 2019-10-22 14:28, Yang, Philip wrote:
>> If device reset/suspend/resume failed for some reason, dqm lock is
>> hold forever and this causes deadlock. Below is a kernel backtrace when
>> application open kfd after suspend/resume failed.
>>
>> Instead of holding dqm lock in pre_reset and releasing dqm lock in
>> post_reset, add dqm->device_stopped flag which is modified in
>> dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
>> because write/read are all inside dqm lock.
>>
>> For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
>> device_stopped flag before sending the updated runlist.
>>
>> v2: For no-HWS case, when device is stopped, don't call
>> load/destroy_mqd for eviction, restore and create queue, and avoid
>> debugfs dump hdqs.
>>
>> Backtrace of dqm lock deadlock:
>>
>> [Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
>> than 120 seconds.
>> [Thu Oct 17 16:43:37 2019]   Not tainted
>> 5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
>> [Thu Oct 17 16:43:37 2019] "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> [Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
>> 0x8000
>> [Thu Oct 17 16:43:37 2019] Call Trace:
>> [Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
>> [Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
>> [Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
>> [Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
>> [Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
>> [Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]
>> kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
>> [amdgpu]
>> [Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
>> [Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
>> [Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
>> [Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
>> [Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
>> [Thu Oct 17 16:43:37 2019]  do_exit+0x284/0xb20
>> [Thu Oct 17 16:43:37 2019]  ? handle_mm_fault+0xfa/0x200
>> [Thu Oct 17 16:43:37 2019]  do_group_exit+0x3a/0xa0
>> [Thu Oct 17 16:43:37 2019]  __x64_sys_exit_group+0x14/0x20
>> [Thu Oct 17 16:43:37 2019]  do_syscall_64+0x4f/0x100
>> [Thu Oct 17 16:43:37 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>
>> Suggested-by: Felix Kuehling 
>> Signed-off-by: Philip Yang 
> 
> Three more comments inline. With those comments addressed, this patch is
> 
> Reviewed-by: Felix Kuehling 
> 
> 
>> ---
>>drivers/gpu/drm/amd/amdkfd/kfd_chardev.c  |  6 +--
>>drivers/gpu/drm/amd/amdkfd/kfd_device.c   |  5 --
>>.../drm/amd/amdkfd/kfd_device_queue_manager.c | 47 +--
>>.../drm/amd/amdkfd/kfd_device_queue_manager.h |  1 +
>>4 files changed, 46 insertions(+), 13 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> index d9e36dbf13d5..40d75c39f08e 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
>> @@ -120,13 +120,13 @@ static int kfd_open(struct inode *inode, struct file 
>> *filep)
>>  return -EPERM;
>>  }
>>
>> +if (kfd_is_locked())
>> +return -EAGAIN;
>> +
>>  process = kfd_create_process(filep);
>>  if (IS_ERR(process))
>>  return PTR_ERR(process);
>>
>> -if (kfd_is_locked())
>> -return -EAGAIN;
>> -
> 
> Is this part of the change still needed? I remember that this sequence
> was a bit tricky with some potential race condition when Shaoyun was
> working on it. This may have unintended side effects.
> 
> 
Revert this change to be safe, it is not needed and not clear if there 
are side effects.

>>  dev_dbg(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
>>  process->pasid, process->is_32bit_user_mode);
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
>> index 8f4b24e84964..4fa8834ce7cb 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
>> +++ b/drivers/gpu/dr

Re: [PATCH v2] drm/amdkfd: don't use dqm lock during device reset/suspend/resume

[Yang, Philip]

On 2019-10-22 3:36 p.m., Grodzovsky, Andrey wrote:
> 
> On 10/22/19 3:19 PM, Yang, Philip wrote:
>>
>> On 2019-10-22 2:40 p.m., Grodzovsky, Andrey wrote:
>>> On 10/22/19 2:38 PM, Grodzovsky, Andrey wrote:
>>>> On 10/22/19 2:28 PM, Yang, Philip wrote:
>>>>> If device reset/suspend/resume failed for some reason, dqm lock is
>>>>> hold forever and this causes deadlock. Below is a kernel backtrace when
>>>>> application open kfd after suspend/resume failed.
>>>>>
>>>>> Instead of holding dqm lock in pre_reset and releasing dqm lock in
>>>>> post_reset, add dqm->device_stopped flag which is modified in
>>>>> dqm->ops.start and dqm->ops.stop. The flag doesn't need lock protection
>>>>> because write/read are all inside dqm lock.
>>>>>
>>>>> For HWS case, map_queues_cpsch and unmap_queues_cpsch checks
>>>>> device_stopped flag before sending the updated runlist.
>>>> Is there a chance of race condition here where dqm->device_stopped
>>>> returns true for some operation (e.g.map_queues_cpsch) but just as it
>>>> proceeds GPU reset starts  ?
>>>>
>>>> Andrey
>>>
>>> Correction -
>>>
>>> dqm->device_stopped returns FALSE
>>>
>> No race condition here, dqm->device_stopped is set to FALSE in
>> kgd2kfd_post_reset -> dqm->ops.start(), which the last step of
>> amdgpu_device_gpu_recover, so it's safe to do map_queue_cpsch.
>>
>> Regards,
>> Philip
> 
> Sorry - i was confused by the commit description vs. body - in the
> description it's called device_stopped flag while in the body it's
> sched_running - probably the description needs to be fixed.
> 
Thanks, commit description is updated.

> So i mean the switch true->false when GPU reset just begins - you get an
> IOCTL to map a queue (if i understood KFD code correctly), check
> dqm->sched_running == true and continue, what if right then GPU reset
> started due to some issue like job timeout or user triggered reset -
> dqm->sched_running becomes false but you already past that checkpoint in
> the IOCTL, no ?
>
map a queue is done under dqm lock, to send the updated runlist to HWS, 
then relase dqm lock. GPU reset start pre_reset will unmap all queues 
first under dqm lock, then set dqm->sched_running to false, release dqm 
lock, and then continue to reset HW. dqm lock guarantee the two 
operations are serialized.

Philip

> Andrey
> 
>>
>>> Andrey
>>>
>>>>
>>>>> v2: For no-HWS case, when device is stopped, don't call
>>>>> load/destroy_mqd for eviction, restore and create queue, and avoid
>>>>> debugfs dump hdqs.
>>>>>
>>>>> Backtrace of dqm lock deadlock:
>>>>>
>>>>> [Thu Oct 17 16:43:37 2019] INFO: task rocminfo:3024 blocked for more
>>>>> than 120 seconds.
>>>>> [Thu Oct 17 16:43:37 2019]   Not tainted
>>>>> 5.0.0-rc1-kfd-compute-rocm-dkms-no-npi-1131 #1
>>>>> [Thu Oct 17 16:43:37 2019] "echo 0 >
>>>>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>>>> [Thu Oct 17 16:43:37 2019] rocminfoD0  3024   2947
>>>>> 0x8000
>>>>> [Thu Oct 17 16:43:37 2019] Call Trace:
>>>>> [Thu Oct 17 16:43:37 2019]  ? __schedule+0x3d9/0x8a0
>>>>> [Thu Oct 17 16:43:37 2019]  schedule+0x32/0x70
>>>>> [Thu Oct 17 16:43:37 2019]  schedule_preempt_disabled+0xa/0x10
>>>>> [Thu Oct 17 16:43:37 2019]  __mutex_lock.isra.9+0x1e3/0x4e0
>>>>> [Thu Oct 17 16:43:37 2019]  ? __call_srcu+0x264/0x3b0
>>>>> [Thu Oct 17 16:43:37 2019]  ? process_termination_cpsch+0x24/0x2f0
>>>>> [amdgpu]
>>>>> [Thu Oct 17 16:43:37 2019]  process_termination_cpsch+0x24/0x2f0
>>>>> [amdgpu]
>>>>> [Thu Oct 17 16:43:37 2019]
>>>>> kfd_process_dequeue_from_all_devices+0x42/0x60 [amdgpu]
>>>>> [Thu Oct 17 16:43:37 2019]  kfd_process_notifier_release+0x1be/0x220
>>>>> [amdgpu]
>>>>> [Thu Oct 17 16:43:37 2019]  __mmu_notifier_release+0x3e/0xc0
>>>>> [Thu Oct 17 16:43:37 2019]  exit_mmap+0x160/0x1a0
>>>>> [Thu Oct 17 16:43:37 2019]  ? __handle_mm_fault+0xba3/0x1200
>>>>> [Thu Oct 17 16:43:37 2019]  ? exit_robust_list+0x5a/0x110
>>>>> [Thu Oct 17 16:43:37 2019]  mmput+0x4a/0x120
>>>>> [Thu Oct 17 

Re: [PATCH v2 14/15] drm/amdgpu: Use mmu_range_notifier instead of hmm_mirror

[Yang, Philip]

Hi Jason,

I did quick test after merging amd-staging-drm-next with the 
mmu_notifier branch, which includes this set changes. The test result 
has different failures, app stuck intermittently, GUI no display etc. I 
am understanding the changes and will try to figure out the cause.

Regards,
Philip

On 2019-10-28 4:10 p.m., Jason Gunthorpe wrote:
> From: Jason Gunthorpe 
> 
> Convert the collision-retry lock around hmm_range_fault to use the one now
> provided by the mmu_range notifier.
> 
> Although this driver does not seem to use the collision retry lock that
> hmm provides correctly, it can still be converted over to use the
> mmu_range_notifier api instead of hmm_mirror without too much trouble.
> 
> This also deletes another place where a driver is associating additional
> data (struct amdgpu_mn) with a mmu_struct.
> 
> Cc: Alex Deucher 
> Cc: Christian König 
> Cc: David (ChunMing) Zhou 
> Cc: amd-gfx@lists.freedesktop.org
> Signed-off-by: Jason Gunthorpe 
> ---
>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |   4 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c|  14 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c| 148 ++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|  49 --
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   |  76 -
>   5 files changed, 66 insertions(+), 225 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index 47700302a08b7f..1bcedb9b477dce 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -1738,6 +1738,10 @@ static int update_invalid_user_pages(struct 
> amdkfd_process_info *process_info,
>   return ret;
>   }
>   
> + /*
> +  * FIXME: Cannot ignore the return code, must hold
> +  * notifier_lock
> +  */
>   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>   
>   /* Mark the BO as valid unless it was invalidated
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 2e53feed40e230..76771f5f0b60ab 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -607,8 +607,6 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser 
> *p,
>   e->tv.num_shared = 2;
>   
>   amdgpu_bo_list_get_list(p->bo_list, &p->validated);
> - if (p->bo_list->first_userptr != p->bo_list->num_entries)
> - p->mn = amdgpu_mn_get(p->adev, AMDGPU_MN_TYPE_GFX);
>   
>   INIT_LIST_HEAD(&duplicates);
>   amdgpu_vm_get_pd_bo(&fpriv->vm, &p->validated, &p->vm_pd);
> @@ -1291,11 +1289,11 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser 
> *p,
>   if (r)
>   goto error_unlock;
>   
> - /* No memory allocation is allowed while holding the mn lock.
> -  * p->mn is hold until amdgpu_cs_submit is finished and fence is added
> -  * to BOs.
> + /* No memory allocation is allowed while holding the notifier lock.
> +  * The lock is held until amdgpu_cs_submit is finished and fence is
> +  * added to BOs.
>*/
> - amdgpu_mn_lock(p->mn);
> + mutex_lock(&p->adev->notifier_lock);
>   
>   /* If userptr are invalidated after amdgpu_cs_parser_bos(), return
>* -EAGAIN, drmIoctl in libdrm will restart the amdgpu_cs_ioctl.
> @@ -1338,13 +1336,13 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser 
> *p,
>   amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
>   
>   ttm_eu_fence_buffer_objects(&p->ticket, &p->validated, p->fence);
> - amdgpu_mn_unlock(p->mn);
> + mutex_unlock(&p->adev->notifier_lock);
>   
>   return 0;
>   
>   error_abort:
>   drm_sched_job_cleanup(&job->base);
> - amdgpu_mn_unlock(p->mn);
> + mutex_unlock(&p->adev->notifier_lock);
>   
>   error_unlock:
>   amdgpu_job_free(job);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
> index 4ffd7b90f4d907..cb718a064eb491 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
> @@ -50,28 +50,6 @@
>   #include "amdgpu.h"
>   #include "amdgpu_amdkfd.h"
>   
> -/**
> - * amdgpu_mn_lock - take the write side lock for this notifier
> - *
> - * @mn: our notifier
> - */
> -void amdgpu_mn_lock(struct amdgpu_mn *mn)
> -{
> - if (mn)
> - down_write(&mn->lock);
> -}
> -
> -/**
> - * amdgpu_mn_unlock - drop the write side lock for this notifier
> - *
> - * @mn: our notifier
> - */
> -void amdgpu_mn_unlock(struct amdgpu_mn *mn)
> -{
> - if (mn)
> - up_write(&mn->lock);
> -}
> -
>   /**
>* amdgpu_mn_invalidate_gfx - callback to notify about mm change
>*
> @@ -82,12 +60,19 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn)
>* potentially dirty.
>*/
>   static bool amdgpu_mn_invalidate_gfx(struc

Re: [PATCH v2 14/15] drm/amdgpu: Use mmu_range_notifier instead of hmm_mirror

[Yang, Philip]

On 2019-10-29 3:25 p.m., Jason Gunthorpe wrote:
> On Tue, Oct 29, 2019 at 07:22:37PM +0000, Yang, Philip wrote:
>> Hi Jason,
>>
>> I did quick test after merging amd-staging-drm-next with the
>> mmu_notifier branch, which includes this set changes. The test result
>> has different failures, app stuck intermittently, GUI no display etc. I
>> am understanding the changes and will try to figure out the cause.
> 
> Thanks! I'm not surprised by this given how difficult this patch was
> to make. Let me know if I can assist in any way
> 
> Please ensure to run with lockdep enabled.. Your symptops sounds sort
> of like deadlocking?
> 
Hi Jason,

Attached patch fix several issues in amdgpu driver, maybe you can squash 
this into patch 14. With this is done, patch 12, 13, 14 is Reviewed-by 
and Tested-by Philip Yang 

Regards,
Philip

> Regards,
> Jason
> 
From 5a0bd4d8cef8472fe2904550142d288feed8cd81 Mon Sep 17 00:00:00 2001
From: Philip Yang 
Date: Thu, 31 Oct 2019 09:10:30 -0400
Subject: [PATCH] drm/amdgpu: issues with new mmu_range_notifier api

put mmu_range_set_seq under the same lock which is used to call
mmu_range_read_retry.

fix amdgpu_ttm_tt_get_user_pages_done return value, because
mmu_range_read_retry means !hmm_range_valid

retry if hmm_range_fault return -EBUSY

fix false WARN for missing get_user_page_done, we should check all
pages not just the first page, don't understand why this issue is
triggered by this change.

Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c  | 32 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 37 +
 2 files changed, 36 insertions(+), 33 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index cb718a064eb4..c8bbd06f1009 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -67,21 +67,15 @@ static bool amdgpu_mn_invalidate_gfx(struct mmu_range_notifier *mrn,
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
 	long r;
 
-	/*
-	 * FIXME: Must hold some lock shared with
-	 * amdgpu_ttm_tt_get_user_pages_done()
-	 */
-	mmu_range_set_seq(mrn, cur_seq);
+	mutex_lock(&adev->notifier_lock);
 
-	/* FIXME: Is this necessary? */
-	if (!amdgpu_ttm_tt_affect_userptr(bo->tbo.ttm, range->start,
-	  range->end))
-		return true;
+	mmu_range_set_seq(mrn, cur_seq);
 
-	if (!mmu_notifier_range_blockable(range))
+	if (!mmu_notifier_range_blockable(range)) {
+		mutex_unlock(&adev->notifier_lock);
 		return false;
+	}
 
-	mutex_lock(&adev->notifier_lock);
 	r = dma_resv_wait_timeout_rcu(bo->tbo.base.resv, true, false,
   MAX_SCHEDULE_TIMEOUT);
 	mutex_unlock(&adev->notifier_lock);
@@ -110,21 +104,15 @@ static bool amdgpu_mn_invalidate_hsa(struct mmu_range_notifier *mrn,
 	struct amdgpu_bo *bo = container_of(mrn, struct amdgpu_bo, notifier);
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
 
-	/*
-	 * FIXME: Must hold some lock shared with
-	 * amdgpu_ttm_tt_get_user_pages_done()
-	 */
-	mmu_range_set_seq(mrn, cur_seq);
+	mutex_lock(&adev->notifier_lock);
 
-	/* FIXME: Is this necessary? */
-	if (!amdgpu_ttm_tt_affect_userptr(bo->tbo.ttm, range->start,
-	  range->end))
-		return true;
+	mmu_range_set_seq(mrn, cur_seq);
 
-	if (!mmu_notifier_range_blockable(range))
+	if (!mmu_notifier_range_blockable(range)) {
+		mutex_unlock(&adev->notifier_lock);
 		return false;
+	}
 
-	mutex_lock(&adev->notifier_lock);
 	amdgpu_amdkfd_evict_userptr(bo->kfd_bo, bo->notifier.mm);
 	mutex_unlock(&adev->notifier_lock);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index a38437fd290a..56fde43d5efa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -799,10 +799,11 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)
 {
 	struct ttm_tt *ttm = bo->tbo.ttm;
 	struct amdgpu_ttm_tt *gtt = (void *)ttm;
-	struct mm_struct *mm;
-	struct hmm_range *range;
 	unsigned long start = gtt->userptr;
 	struct vm_area_struct *vma;
+	struct hmm_range *range;
+	unsigned long timeout;
+	struct mm_struct *mm;
 	unsigned long i;
 	int r = 0;
 
@@ -841,8 +842,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)
 		goto out_free_ranges;
 	}
 
-	range->notifier_seq = mmu_range_read_begin(&bo->notifier);
-
 	down_read(&mm->mmap_sem);
 	vma = find_vma(mm, start);
 	if (unlikely(!vma || start < vma->vm_start)) {
@@ -854,12 +853,20 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)
 		r = -EPERM;
 		goto out_unlock;
 	}
+	up_read(&mm->mmap_sem);
+	timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+
+retry:
+	range->notifier_seq = mmu_range_read_

Re: [PATCH v2 14/15] drm/amdgpu: Use mmu_range_notifier instead of hmm_mirror

[Yang, Philip]

On 2019-11-01 11:12 a.m., Jason Gunthorpe wrote:
> On Fri, Nov 01, 2019 at 02:44:51PM +0000, Yang, Philip wrote:
>>
>>
>> On 2019-10-29 3:25 p.m., Jason Gunthorpe wrote:
>>> On Tue, Oct 29, 2019 at 07:22:37PM +, Yang, Philip wrote:
>>>> Hi Jason,
>>>>
>>>> I did quick test after merging amd-staging-drm-next with the
>>>> mmu_notifier branch, which includes this set changes. The test result
>>>> has different failures, app stuck intermittently, GUI no display etc. I
>>>> am understanding the changes and will try to figure out the cause.
>>>
>>> Thanks! I'm not surprised by this given how difficult this patch was
>>> to make. Let me know if I can assist in any way
>>>
>>> Please ensure to run with lockdep enabled.. Your symptops sounds sort
>>> of like deadlocking?
>>>
>> Hi Jason,
>>
>> Attached patch fix several issues in amdgpu driver, maybe you can squash
>> this into patch 14. With this is done, patch 12, 13, 14 is Reviewed-by
>> and Tested-by Philip Yang 
> 
> Wow, this is great thanks! Can you clarify what the problems you found
> were? Was the bug the 'return !r' below?
> 
Yes. return !r is critical one, and retry if hmm_range_fault return 
-EBUSY is needed too.

> I'll also add your signed off by
> 
> Here are some remarks:
> 
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> index cb718a064eb4..c8bbd06f1009 100644
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> @@ -67,21 +67,15 @@ static bool amdgpu_mn_invalidate_gfx(struct 
>> mmu_range_notifier *mrn,
>>  struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
>>  long r;
>>   
>> -/*
>> - * FIXME: Must hold some lock shared with
>> - * amdgpu_ttm_tt_get_user_pages_done()
>> - */
>> -mmu_range_set_seq(mrn, cur_seq);
>> +mutex_lock(&adev->notifier_lock);
>>   
>> -/* FIXME: Is this necessary? */
>> -if (!amdgpu_ttm_tt_affect_userptr(bo->tbo.ttm, range->start,
>> -  range->end))
>> -return true;
>> +mmu_range_set_seq(mrn, cur_seq);
>>   
>> -if (!mmu_notifier_range_blockable(range))
>> +if (!mmu_notifier_range_blockable(range)) {
>> +mutex_unlock(&adev->notifier_lock);
>>  return false;
> 
> This test for range_blockable should be before mutex_lock, I can move
> it up
> 
yes, thanks.
> Also, do you know if notifier_lock is held while calling
> amdgpu_ttm_tt_get_user_pages_done()? Can we add a 'lock assert held'
> to amdgpu_ttm_tt_get_user_pages_done()?
> 
gpu side hold notifier_lock but kfd side doesn't. kfd side doesn't check 
amdgpu_ttm_tt_get_user_pages_done/mmu_range_read_retry return value but 
check mem->invalid flag which is updated from invalidate callback. It 
takes more time to change, I will come to another patch to fix it later.

>> @@ -854,12 +853,20 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, 
>> struct page **pages)
>>  r = -EPERM;
>>  goto out_unlock;
>>  }
>> +up_read(&mm->mmap_sem);
>> +timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
>> +
>> +retry:
>> +range->notifier_seq = mmu_range_read_begin(&bo->notifier);
>>   
>> +down_read(&mm->mmap_sem);
>>  r = hmm_range_fault(range, 0);
>>  up_read(&mm->mmap_sem);
>> -
>> -if (unlikely(r < 0))
>> +if (unlikely(r <= 0)) {
>> +if ((r == 0 || r == -EBUSY) && !time_after(jiffies, timeout))
>> +goto retry;
>>  goto out_free_pfns;
>> +}
> 
> This isn't really right, a retry loop like this needs to go all the
> way to mmu_range_read_retry() and done under the notifier_lock. ie
> mmu_range_read_retry() can fail just as likely as hmm_range_fault()
> can, and drivers are supposed to retry in both cases, with a single
> timeout.
> 
For gpu, check mmu_range_read_retry return value under the notifier_lock 
to do retry is in seperate location, not in same retry loop.

> AFAICT it is a major bug that many places ignore the return code of
> amdgpu_ttm_tt_get_user_pages_done() ???
>
For kfd, explained above.

> However, this is all pre-existing bugs, so I'm OK go ahead with this
> patch as modified. I advise AMD to make a followup patch ..
> 
yes, I will.
> I'll add a FIXME note to this ef

Re: [PATCH v2 14/15] drm/amdgpu: Use mmu_range_notifier instead of hmm_mirror

[Yang, Philip]

On 2019-11-01 1:42 p.m., Jason Gunthorpe wrote:
> On Fri, Nov 01, 2019 at 03:59:26PM +0000, Yang, Philip wrote:
>>> This test for range_blockable should be before mutex_lock, I can move
>>> it up
>>>
>> yes, thanks.
> 
> Okay, I wrote it like this:
> 
>   if (mmu_notifier_range_blockable(range))
>   mutex_lock(&adev->notifier_lock);
>   else if (!mutex_trylock(&adev->notifier_lock))
>   return false;
> 
>>> Also, do you know if notifier_lock is held while calling
>>> amdgpu_ttm_tt_get_user_pages_done()? Can we add a 'lock assert held'
>>> to amdgpu_ttm_tt_get_user_pages_done()?
>>
>> gpu side hold notifier_lock but kfd side doesn't. kfd side doesn't check
>> amdgpu_ttm_tt_get_user_pages_done/mmu_range_read_retry return value but
>> check mem->invalid flag which is updated from invalidate callback. It
>> takes more time to change, I will come to another patch to fix it later.
> 
> Ah.. confusing, OK, I'll let you sort that
> 
>>> However, this is all pre-existing bugs, so I'm OK go ahead with this
>>> patch as modified. I advise AMD to make a followup patch ..
>>>
>> yes, I will.
> 
> While you are here, this is also wrong:
> 
> int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)
> {
>   down_read(&mm->mmap_sem);
>   r = hmm_range_fault(range, 0);
>   up_read(&mm->mmap_sem);
>   if (unlikely(r <= 0)) {
>   if ((r == 0 || r == -EBUSY) && !time_after(jiffies, timeout))
>   goto retry;
>   goto out_free_pfns;
>   }
> 
>   for (i = 0; i < ttm->num_pages; i++) {
>   pages[i] = hmm_device_entry_to_page(range, range->pfns[i]);
> 
> It is not allowed to read the results of hmm_range_fault() outside
> locking, and in particular, we can't convert to a struct page.
> 
> This must be done inside the notifier_lock, after checking
> mmu_range_read_retry(), all handling of the struct page must be
> structured like that.
> 
Below change will fix this, then driver will call mmu_range_read_retry 
second time using same range->notifier_seq to check if range is 
invalidated inside amdgpu_cs_submit, this looks ok for me.

@@ -868,6 +869,13 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo 
*bo, struct page **pages)
 goto out_free_pfns;
 }

+   mutex_lock(&adev->notifier_lock);
+
+   if (mmu_range_read_retry(&bo->notifier, range->notifier_seq)) {
+   mutex_unlock(&adev->notifier_lock);
+   goto retry;
+   }
+
 for (i = 0; i < ttm->num_pages; i++) {
 pages[i] = hmm_device_entry_to_page(range, range->pfns[i]);
 if (unlikely(!pages[i])) {
@@ -875,10 +883,12 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo 
*bo, struct page **pages)
i, range->pfns[i]);
 r = -ENOMEM;

+   mutex_unlock(&adev->notifier_lock);
 goto out_free_pfns;
 }
 }

+   mutex_unlock(&adev->notifier_lock);
 gtt->range = range;
 mmput(mm);

Philip

>>>> @@ -997,10 +1004,18 @@ static void amdgpu_ttm_tt_unpin_userptr(struct 
>>>> ttm_tt *ttm)
>>>>sg_free_table(ttm->sg);
>>>>
>>>>#if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
>>>> -  if (gtt->range &&
>>>> -  ttm->pages[0] == hmm_device_entry_to_page(gtt->range,
>>>> -gtt->range->pfns[0]))
>>>> -  WARN_ONCE(1, "Missing get_user_page_done\n");
>>>> +  if (gtt->range) {
>>>> +  unsigned long i;
>>>> +
>>>> +  for (i = 0; i < ttm->num_pages; i++) {
>>>> +  if (ttm->pages[i] !=
>>>> +  hmm_device_entry_to_page(gtt->range,
>>>> +gtt->range->pfns[i]))
>>>> +  break;
>>>> +  }
>>>> +
>>>> +  WARN((i == ttm->num_pages), "Missing get_user_page_done\n");
>>>> +  }
>>>
>>> Is this related/necessary? I can put it in another patch if it is just
>>> debugging improvement? Please advise
>>>
>> I see this WARN backtrace now, but I didn't see it before. This is
>> somehow related.
> 
> Hm, might be instructive to learn what is going on..
> 
> Thanks,
> Jason
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH v2 14/15] drm/amdgpu: Use mmu_range_notifier instead of hmm_mirror

[Yang, Philip]

Sorry, resend patch, the one in previous email missed couple of lines 
duo to copy/paste.

On 2019-11-01 3:45 p.m., Yang, Philip wrote:
> 
> 
> On 2019-11-01 1:42 p.m., Jason Gunthorpe wrote:
>> On Fri, Nov 01, 2019 at 03:59:26PM +0000, Yang, Philip wrote:
>>>> This test for range_blockable should be before mutex_lock, I can move
>>>> it up
>>>>
>>> yes, thanks.
>>
>> Okay, I wrote it like this:
>>
>>  if (mmu_notifier_range_blockable(range))
>>  mutex_lock(&adev->notifier_lock);
>>  else if (!mutex_trylock(&adev->notifier_lock))
>>  return false;
>>
>>>> Also, do you know if notifier_lock is held while calling
>>>> amdgpu_ttm_tt_get_user_pages_done()? Can we add a 'lock assert held'
>>>> to amdgpu_ttm_tt_get_user_pages_done()?
>>>
>>> gpu side hold notifier_lock but kfd side doesn't. kfd side doesn't check
>>> amdgpu_ttm_tt_get_user_pages_done/mmu_range_read_retry return value but
>>> check mem->invalid flag which is updated from invalidate callback. It
>>> takes more time to change, I will come to another patch to fix it later.
>>
>> Ah.. confusing, OK, I'll let you sort that
>>
>>>> However, this is all pre-existing bugs, so I'm OK go ahead with this
>>>> patch as modified. I advise AMD to make a followup patch ..
>>>>
>>> yes, I will.
>>
>> While you are here, this is also wrong:
>>
>> int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)
>> {
>>  down_read(&mm->mmap_sem);
>>  r = hmm_range_fault(range, 0);
>>  up_read(&mm->mmap_sem);
>>  if (unlikely(r <= 0)) {
>>  if ((r == 0 || r == -EBUSY) && !time_after(jiffies, timeout))
>>  goto retry;
>>  goto out_free_pfns;
>>  }
>>
>>  for (i = 0; i < ttm->num_pages; i++) {
>>  pages[i] = hmm_device_entry_to_page(range, range->pfns[i]);
>>
>> It is not allowed to read the results of hmm_range_fault() outside
>> locking, and in particular, we can't convert to a struct page.
>>
>> This must be done inside the notifier_lock, after checking
>> mmu_range_read_retry(), all handling of the struct page must be
>> structured like that.
>>
> Below change will fix this, then driver will call mmu_range_read_retry
> second time using same range->notifier_seq to check if range is
> invalidated inside amdgpu_cs_submit, this looks ok for me.
> 
@@ -797,6 +797,7 @@ static const uint64_t 
hmm_range_values[HMM_PFN_VALUE_MAX] = {
   */
  int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page 
**pages)
  {
+   struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
 struct ttm_tt *ttm = bo->tbo.ttm;
 struct amdgpu_ttm_tt *gtt = (void *)ttm;
 unsigned long start = gtt->userptr;
@@ -868,6 +869,13 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo 
*bo, struct page **pages)
 goto out_free_pfns;
 }

+   mutex_lock(&adev->notifier_lock);
+
+   if (mmu_range_read_retry(&bo->notifier, range->notifier_seq)) {
+   mutex_unlock(&adev->notifier_lock);
+   goto retry;
+   }
+
 for (i = 0; i < ttm->num_pages; i++) {
 pages[i] = hmm_device_entry_to_page(range, range->pfns[i]);
 if (unlikely(!pages[i])) {
@@ -875,10 +883,12 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo 
*bo, struct page **pages)
i, range->pfns[i]);
 r = -ENOMEM;

+   mutex_unlock(&adev->notifier_lock);
 goto out_free_pfns;
 }
 }

+   mutex_unlock(&adev->notifier_lock);
 gtt->range = range;
 mmput(mm);

> 
> Philip
> 
>>>>> @@ -997,10 +1004,18 @@ static void amdgpu_ttm_tt_unpin_userptr(struct 
>>>>> ttm_tt *ttm)
>>>>>   sg_free_table(ttm->sg);
>>>>> 
>>>>> #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
>>>>> - if (gtt->range &&
>>>>> - ttm->pages[0] == hmm_device_entry_to_page(gtt->range,
>>>>> -   gtt->range->pfns[0]))
>>>>> - WARN_ONCE(1, "Missing get_user_page_done\n");
>>>>> + if (gtt->range) {
>>>>> + unsigned long i;
>>>>> +
>>>>>

[PATCH] mm/hmm: hmm_range_fault handle pages swapped out

[Yang, Philip]

hmm_range_fault may return NULL pages because some of pfns are equal to
HMM_PFN_NONE. This happens randomly under memory pressure. The reason is
for swapped out page pte path, hmm_vma_handle_pte doesn't update fault
variable from cpu_flags, so it failed to call hmm_vam_do_fault to swap
the page in.

The fix is to call hmm_pte_need_fault to update fault variable.

Change-Id: I2e8611485563d11d938881c18b7935fa1e7c91ee
Signed-off-by: Philip Yang 
---
 mm/hmm.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/hmm.c b/mm/hmm.c
index 9f22562e2c43..7ca4fb39d3d8 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -544,6 +544,9 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, 
unsigned long addr,
swp_entry_t entry = pte_to_swp_entry(pte);
 
if (!non_swap_entry(entry)) {
+   cpu_flags = pte_to_hmm_pfn_flags(range, pte);
+   hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags,
+  &fault, &write_fault);
if (fault || write_fault)
goto fault;
return 0;
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] mm/hmm: hmm_range_fault handle pages swapped out

[Yang, Philip]

On 2019-08-15 8:54 p.m., Jason Gunthorpe wrote:
> On Thu, Aug 15, 2019 at 08:52:56PM +0000, Yang, Philip wrote:
>> hmm_range_fault may return NULL pages because some of pfns are equal to
>> HMM_PFN_NONE. This happens randomly under memory pressure. The reason is
>> for swapped out page pte path, hmm_vma_handle_pte doesn't update fault
>> variable from cpu_flags, so it failed to call hmm_vam_do_fault to swap
>> the page in.
>>
>> The fix is to call hmm_pte_need_fault to update fault variable.
> 
>> Change-Id: I2e8611485563d11d938881c18b7935fa1e7c91ee
> 
> I'll fix it for you but please be careful not to send Change-id's to
> the public lists.
> 
Thanks, the change-id was added by our Gerrit hook, I need generate 
patch files, remove change-id line and then send out modified patch 
files in future.

> Also what is the Fixes line for this?
> 
This fixes the issue found by the internal rocrtst, the 
rocrtstFunc.Memory_Max_Mem evicted some user buffers, and then following 
test restore those user buffers failed because the buffers are swapped 
out and application doesn't touch the buffers to swap it in.

>> Signed-off-by: Philip Yang 
>>   mm/hmm.c | 3 +++
>>   1 file changed, 3 insertions(+)
> 
> Ralph has also been looking at this area also so I'll give him a bit
> to chime in, otherwise with Jerome's review this looks OK to go to
> linux-next
>
Ok, thanks for helping push this to hmm branch at 
https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git

> Thanks,
> Jason
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 9/9] drm/amdgpu: add graceful VM fault handling v2

[Yang, Philip]

This series looks nice and clear for me, two questions embedded below.

Are we going to use dedicated sdma page queue for direct VM update path 
during a fault?

Thanks,
Philip

On 2019-09-04 11:02 a.m., Christian König wrote:
> Next step towards HMM support. For now just silence the retry fault and
> optionally redirect the request to the dummy page.
> 
> v2: make sure the VM is not destroyed while we handle the fault.
> 
> Signed-off-by: Christian König 
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 74 ++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  2 +
>   drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c  |  4 ++
>   3 files changed, 80 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 951608fc1925..410d89966a66 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -3142,3 +3142,77 @@ void amdgpu_vm_set_task_info(struct amdgpu_vm *vm)
>   }
>   }
>   }
> +
> +/**
> + * amdgpu_vm_handle_fault - graceful handling of VM faults.
> + * @adev: amdgpu device pointer
> + * @pasid: PASID of the VM
> + * @addr: Address of the fault
> + *
> + * Try to gracefully handle a VM fault. Return true if the fault was handled 
> and
> + * shouldn't be reported any more.
> + */
> +bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, unsigned int pasid,
> + uint64_t addr)
> +{
> + struct amdgpu_ring *ring = &adev->sdma.instance[0].page;
> + struct amdgpu_bo *root;
> + uint64_t value, flags;
> + struct amdgpu_vm *vm;
> + long r;
> +
> + if (!ring->sched.ready)
> + return false;
> +
> + spin_lock(&adev->vm_manager.pasid_lock);
> + vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
> + if (vm)
> + root = amdgpu_bo_ref(vm->root.base.bo);
> + else
> + root = NULL;
> + spin_unlock(&adev->vm_manager.pasid_lock);
> +
> + if (!root)
> + return false;
> +
> + r = amdgpu_bo_reserve(root, true);
> + if (r)
> + goto error_unref;
> +
> + spin_lock(&adev->vm_manager.pasid_lock);
> + vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
> + spin_unlock(&adev->vm_manager.pasid_lock);
> +
Here get vm from pasid second time, and check if PD bo is changed, is 
this to handle vm fault race with vm destory?

> + if (!vm || vm->root.base.bo != root)
> + goto error_unlock;
> +
> + addr /= AMDGPU_GPU_PAGE_SIZE;
> + flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> + AMDGPU_PTE_SYSTEM;
> +
> + if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
> + /* Redirect the access to the dummy page */
> + value = adev->dummy_page_addr;
> + flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
> + AMDGPU_PTE_WRITEABLE;
> + } else {
> + value = 0;
> + }
> +
> + r = amdgpu_vm_bo_update_mapping(adev, vm, true, NULL, addr, addr + 1,
> + flags, value, NULL, NULL);
> + if (r)
> + goto error_unlock;
> +
After fault address redirect to dummy page, will the fault recover and 
retry continue to execute? Is this dangerous to update PTE to use system 
memory address 0?

> + r = amdgpu_vm_update_pdes(adev, vm, true);
> +
> +error_unlock:
> + amdgpu_bo_unreserve(root);
> + if (r < 0)
> + DRM_ERROR("Can't handle page fault (%ld)\n", r);
> +
> +error_unref:
> + amdgpu_bo_unref(&root);
> +
> + return false;
> +}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> index 0a97dc839f3b..4dbbe1b6b413 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> @@ -413,6 +413,8 @@ void amdgpu_vm_check_compute_bug(struct amdgpu_device 
> *adev);
>   
>   void amdgpu_vm_get_task_info(struct amdgpu_device *adev, unsigned int pasid,
>struct amdgpu_task_info *task_info);
> +bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, unsigned int pasid,
> + uint64_t addr);
>   
>   void amdgpu_vm_set_task_info(struct amdgpu_vm *vm);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> index 9d15679df6e0..15a1ce51befa 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> @@ -353,6 +353,10 @@ static int gmc_v9_0_process_interrupt(struct 
> amdgpu_device *adev,
>   }
>   
>   /* If it's the first fault for this address, process it normally */
> + if (retry_fault && !in_interrupt() &&
> + amdgpu_vm_handle_fault(adev, entry->pasid, addr))
> + return 1; /* This also prevents sending it to KFD */
> +
>   if (!amdgpu_sriov_vf(adev)) {
>   /*
>* Issue a dummy read to wait for the status regis

Re: [PATCH 1/1] drm/amdgpu: Disable retry faults in VMID0

[Yang, Philip]

VMID0 init path was missed when enabling amdgpu_noretry option. Good 
catch and fix.

Reviewed-by: Philip Yang 

On 2019-09-04 7:31 p.m., Kuehling, Felix wrote:
> There is no point retrying page faults in VMID0. Those faults are
> always fatal.
> 
> Signed-off-by: Felix Kuehling 
> ---
>   drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 2 ++
>   drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 2 ++
>   drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  | 2 ++
>   drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c  | 2 ++
>   drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c  | 2 ++
>   5 files changed, 10 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> index 6ce37ce77d14..9ec4297e61e5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> @@ -178,6 +178,8 @@ static void gfxhub_v1_0_enable_system_domain(struct 
> amdgpu_device *adev)
>   tmp = RREG32_SOC15(GC, 0, mmVM_CONTEXT0_CNTL);
>   tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
>   tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> + tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL,
> + RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>   WREG32_SOC15(GC, 0, mmVM_CONTEXT0_CNTL, tmp);
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> index 8b789f750b72..a9238735d361 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> @@ -166,6 +166,8 @@ static void gfxhub_v2_0_enable_system_domain(struct 
> amdgpu_device *adev)
>   tmp = RREG32_SOC15(GC, 0, mmGCVM_CONTEXT0_CNTL);
>   tmp = REG_SET_FIELD(tmp, GCVM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
>   tmp = REG_SET_FIELD(tmp, GCVM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> + tmp = REG_SET_FIELD(tmp, GCVM_CONTEXT0_CNTL,
> + RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>   WREG32_SOC15(GC, 0, mmGCVM_CONTEXT0_CNTL, tmp);
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c 
> b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
> index b9d6c0bfa594..4c7e8c64a94e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
> @@ -207,6 +207,8 @@ static void mmhub_v1_0_enable_system_domain(struct 
> amdgpu_device *adev)
>   tmp = RREG32_SOC15(MMHUB, 0, mmVM_CONTEXT0_CNTL);
>   tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
>   tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> + tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL,
> + RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>   WREG32_SOC15(MMHUB, 0, mmVM_CONTEXT0_CNTL, tmp);
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c 
> b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> index 3542c203c3c8..86ed8cb915a8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> @@ -152,6 +152,8 @@ static void mmhub_v2_0_enable_system_domain(struct 
> amdgpu_device *adev)
>   tmp = RREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL);
>   tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
>   tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> + tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL,
> + RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>   WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL, tmp);
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c 
> b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
> index 0cf7ef44b4b5..657970f9ebfb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
> @@ -240,6 +240,8 @@ static void mmhub_v9_4_enable_system_domain(struct 
> amdgpu_device *adev,
> hubid * MMHUB_INSTANCE_REGISTER_OFFSET);
>   tmp = REG_SET_FIELD(tmp, VML2VC0_VM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
>   tmp = REG_SET_FIELD(tmp, VML2VC0_VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> + tmp = REG_SET_FIELD(tmp, VML2VC0_VM_CONTEXT0_CNTL,
> + RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
>   WREG32_SOC15_OFFSET(MMHUB, 0, mmVML2VC0_VM_CONTEXT0_CNTL,
>   hubid * MMHUB_INSTANCE_REGISTER_OFFSET, tmp);
>   }
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: check if nbio->ras_if exist

[Yang, Philip]

To avoid NULL function pointer access. This happens on VG10, reboot
command hangs and have to power off/on to reboot the machine. This is
serial console log:

[  OK  ] Reached target Unmount All Filesystems.
[  OK  ] Reached target Final Step.
 Starting Reboot...
[  305.696271] systemd-shutdown[1]: Syncing filesystems and block
devices.
[  306.947328] systemd-shutdown[1]: Sending SIGTERM to remaining
processes...
[  306.963920] systemd-journald[1722]: Received SIGTERM from PID 1
(systemd-shutdow).
[  307.322717] systemd-shutdown[1]: Sending SIGKILL to remaining
processes...
[  307.336472] systemd-shutdown[1]: Unmounting file systems.
[  307.454202] EXT4-fs (sda2): re-mounted. Opts: errors=remount-ro
[  307.480523] systemd-shutdown[1]: All filesystems unmounted.
[  307.486537] systemd-shutdown[1]: Deactivating swaps.
[  307.491962] systemd-shutdown[1]: All swaps deactivated.
[  307.497624] systemd-shutdown[1]: Detaching loop devices.
[  307.504418] systemd-shutdown[1]: All loop devices detached.
[  307.510418] systemd-shutdown[1]: Detaching DM devices.
[  307.565907] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[  307.731313] BUG: kernel NULL pointer dereference, address:

[  307.738802] #PF: supervisor read access in kernel mode
[  307.744326] #PF: error_code(0x) - not-present page
[  307.749850] PGD 0 P4D 0
[  307.752568] Oops:  [#1] SMP PTI
[  307.756314] CPU: 3 PID: 1 Comm: systemd-shutdow Not tainted
5.2.0-rc1-kfd-yangp #453
[  307.764644] Hardware name: ASUS All Series/Z97-PRO(Wi-Fi ac)/USB 3.1,
BIOS 9001 03/07/2016
[  307.773580] RIP: 0010:soc15_common_hw_fini+0x33/0xc0 [amdgpu]
[  307.779760] Code: 89 fb e8 60 f5 ff ff f6 83 50 df 01 00 04 75 3d 48
8b b3 90 7d 00 00 48 c7 c7 17 b8 530
[  307.799967] RSP: 0018:ac9483153d40 EFLAGS: 00010286
[  307.805585] RAX:  RBX: 9eb299da RCX:
0006
[  307.813261] RDX:  RSI: 9eb29e3508a0 RDI:
9eb29e35
[  307.820935] RBP: 9eb299da R08:  R09:

[  307.828609] R10:  R11:  R12:
9eb299dbd1f8
[  307.836284] R13: c04f8368 R14: 9eb29cebd130 R15:

[  307.843959] FS:  7f06721c9940() GS:9eb2a18c()
knlGS:
[  307.852663] CS:  0010 DS:  ES:  CR0: 80050033
[  307.858842] CR2:  CR3: 00081d798005 CR4:
001606e0
[  307.866516] Call Trace:
[  307.869169]  amdgpu_device_ip_suspend_phase2+0x80/0x110 [amdgpu]
[  307.875654]  ? amdgpu_device_ip_suspend_phase1+0x4d/0xd0 [amdgpu]
[  307.882230]  amdgpu_device_ip_suspend+0x2e/0x60 [amdgpu]
[  307.887966]  amdgpu_pci_shutdown+0x2f/0x40 [amdgpu]
[  307.893211]  pci_device_shutdown+0x31/0x60
[  307.897613]  device_shutdown+0x14c/0x1f0
[  307.901829]  kernel_restart+0xe/0x50
[  307.905669]  __do_sys_reboot+0x1df/0x210
[  307.909884]  ? task_work_run+0x73/0xb0
[  307.913914]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  307.918970]  do_syscall_64+0x4a/0x1c0
[  307.922904]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  307.928336] RIP: 0033:0x7f0671cf8373
[  307.932176] Code: 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00
00 0f 1f 44 00 00 89 fa be 69 19 128
[  307.952384] RSP: 002b:7ffdd1723d68 EFLAGS: 0202 ORIG_RAX:
00a9
[  307.960527] RAX: ffda RBX: 01234567 RCX:
7f0671cf8373
[  307.968201] RDX: 01234567 RSI: 28121969 RDI:
fee1dead
[  307.975875] RBP: 7ffdd1723dd0 R08:  R09:

[  307.983550] R10: 0002 R11: 0202 R12:
7ffdd1723dd8
[  307.991224] R13:  R14: 001b0004 R15:
7ffdd17240c8
[  307.998901] Modules linked in: xt_MASQUERADE nfnetlink iptable_nat
xt_addrtype xt_conntrack nf_nat nf_cos
[  308.026505] CR2: 
[  308.039998] RIP: 0010:soc15_common_hw_fini+0x33/0xc0 [amdgpu]
[  308.046180] Code: 89 fb e8 60 f5 ff ff f6 83 50 df 01 00 04 75 3d 48
8b b3 90 7d 00 00 48 c7 c7 17 b8 530
[  308.066392] RSP: 0018:ac9483153d40 EFLAGS: 00010286
[  308.072013] RAX:  RBX: 9eb299da RCX:
0006
[  308.079689] RDX:  RSI: 9eb29e3508a0 RDI:
9eb29e35
[  308.087366] RBP: 9eb299da R08:  R09:

[  308.095042] R10:  R11:  R12:
9eb299dbd1f8
[  308.102717] R13: c04f8368 R14: 9eb29cebd130 R15:

[  308.110394] FS:  7f06721c9940() GS:9eb2a18c()
knlGS:
[  308.119099] CS:  0010 DS:  ES:  CR0: 80050033
[  308.125280] CR2:  CR3: 00081d798005 CR4:
001606e0
[  308.135304] printk: systemd-shutdow: 3 output lines suppressed due to
ratelimiting
[  308.143518] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0009
[  308.151798] Kernel Offset: 0x1500 from 0x8100

Re: [PATCH 9/9] drm/amdgpu: add graceful VM fault handling v2

[Yang, Philip]

On 2019-09-09 8:03 a.m., Christian König wrote:
> Am 04.09.19 um 22:12 schrieb Yang, Philip:
>> This series looks nice and clear for me, two questions embedded below.
>>
>> Are we going to use dedicated sdma page queue for direct VM update path
>> during a fault?
>>
>> Thanks,
>> Philip
>>
>> On 2019-09-04 11:02 a.m., Christian König wrote:
>>> Next step towards HMM support. For now just silence the retry fault and
>>> optionally redirect the request to the dummy page.
>>>
>>> v2: make sure the VM is not destroyed while we handle the fault.
>>>
>>> Signed-off-by: Christian König 
>>> ---
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 74 
>>> ++
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  2 +
>>>    drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c  |  4 ++
>>>    3 files changed, 80 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> index 951608fc1925..410d89966a66 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> @@ -3142,3 +3142,77 @@ void amdgpu_vm_set_task_info(struct amdgpu_vm 
>>> *vm)
>>>    }
>>>    }
>>>    }
>>> +
>>> +/**
>>> + * amdgpu_vm_handle_fault - graceful handling of VM faults.
>>> + * @adev: amdgpu device pointer
>>> + * @pasid: PASID of the VM
>>> + * @addr: Address of the fault
>>> + *
>>> + * Try to gracefully handle a VM fault. Return true if the fault was 
>>> handled and
>>> + * shouldn't be reported any more.
>>> + */
>>> +bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, unsigned int 
>>> pasid,
>>> +    uint64_t addr)
>>> +{
>>> +    struct amdgpu_ring *ring = &adev->sdma.instance[0].page;
>>> +    struct amdgpu_bo *root;
>>> +    uint64_t value, flags;
>>> +    struct amdgpu_vm *vm;
>>> +    long r;
>>> +
>>> +    if (!ring->sched.ready)
>>> +    return false;
>>> +
>>> +    spin_lock(&adev->vm_manager.pasid_lock);
>>> +    vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
>>> +    if (vm)
>>> +    root = amdgpu_bo_ref(vm->root.base.bo);
>>> +    else
>>> +    root = NULL;
>>> +    spin_unlock(&adev->vm_manager.pasid_lock);
>>> +
>>> +    if (!root)
>>> +    return false;
>>> +
>>> +    r = amdgpu_bo_reserve(root, true);
>>> +    if (r)
>>> +    goto error_unref;
>>> +
>>> +    spin_lock(&adev->vm_manager.pasid_lock);
>>> +    vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
>>> +    spin_unlock(&adev->vm_manager.pasid_lock);
>>> +
>> Here get vm from pasid second time, and check if PD bo is changed, is
>> this to handle vm fault race with vm destory?
> 
> Yes, exactly.
> 
>>
>>> +    if (!vm || vm->root.base.bo != root)
>>> +    goto error_unlock;
>>> +
>>> +    addr /= AMDGPU_GPU_PAGE_SIZE;
>>> +    flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
>>> +    AMDGPU_PTE_SYSTEM;
>>> +
>>> +    if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
>>> +    /* Redirect the access to the dummy page */
>>> +    value = adev->dummy_page_addr;
>>> +    flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
>>> +    AMDGPU_PTE_WRITEABLE;
>>> +    } else {
>>> +    value = 0;
>>> +    }
>>> +
>>> +    r = amdgpu_vm_bo_update_mapping(adev, vm, true, NULL, addr, addr 
>>> + 1,
>>> +    flags, value, NULL, NULL);
>>> +    if (r)
>>> +    goto error_unlock;
>>> +
>> After fault address redirect to dummy page, will the fault recover and
>> retry continue to execute?
> 
> Yes, the read/write operation will just retry and use the value from the 
> dummy page instead.
> 
>> Is this dangerous to update PTE to use system
>> memory address 0?
> 
> What are you talking about? The dummy page is a page allocate by TTM 
> where we redirect faulty accesses to.
> 
For amdgpu_vm_fault_stop equals to AMDGPU_VM_FAULT_STOP_FIRST/ALWAYS 
case, value is 0, this will redirect to system memory 0. Maybe redirect 
is only needed for AMDGPU_VM_FAULT_STOP_NEVER?

Regards,
Philip

>

[PATCH] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

user_pages array should be freed regardless if user pages are
invalidated after bo is created because HMM change to always allocate
user pages array to get user pages while parsing user page bo.

Don't need to to get user pages while creating bo because user pages
will only be used after parsing user page bo.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 23 +--
 2 files changed, 2 insertions(+), 23 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 49b767b7238f..e861de259def 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -498,7 +498,7 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser 
*p,
if (r)
return r;
 
-   if (binding_userptr) {
+   if (lobj->user_pages) {
kvfree(lobj->user_pages);
lobj->user_pages = NULL;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index a828e3d0bfbd..3ccd61d69964 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -283,7 +283,6 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void 
*data,
 int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
 struct drm_file *filp)
 {
-   struct ttm_operation_ctx ctx = { true, false };
struct amdgpu_device *adev = dev->dev_private;
struct drm_amdgpu_gem_userptr *args = data;
struct drm_gem_object *gobj;
@@ -326,32 +325,12 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void 
*data,
goto release_object;
}
 
-   if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {
-   r = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages);
-   if (r)
-   goto release_object;
-
-   r = amdgpu_bo_reserve(bo, true);
-   if (r)
-   goto user_pages_done;
-
-   amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
-   r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
-   amdgpu_bo_unreserve(bo);
-   if (r)
-   goto user_pages_done;
-   }
-
r = drm_gem_handle_create(filp, gobj, &handle);
if (r)
-   goto user_pages_done;
+   goto release_object;
 
args->handle = handle;
 
-user_pages_done:
-   if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)
-   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
-
 release_object:
drm_gem_object_put_unlocked(gobj);
 
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

user_pages array should always be freed after validation regardless if
user pages are changed after bo is created because with HMM change parse
bo always allocate user pages array to get user pages for userptr bo.

Don't need to get user pages while creating uerptr bo because user pages
will only be used while validating after parsing userptr bo.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

v2: remove unused local variable and amend commit

Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  |  4 +---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 23 +--
 2 files changed, 2 insertions(+), 25 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 49b767b7238f..961186e7113e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -474,7 +474,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser 
*p,
 
list_for_each_entry(lobj, validated, tv.head) {
struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-   bool binding_userptr = false;
struct mm_struct *usermm;
 
usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
@@ -491,14 +490,13 @@ static int amdgpu_cs_list_validate(struct 
amdgpu_cs_parser *p,
 
amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
 lobj->user_pages);
-   binding_userptr = true;
}
 
r = amdgpu_cs_validate(p, bo);
if (r)
return r;
 
-   if (binding_userptr) {
+   if (lobj->user_pages) {
kvfree(lobj->user_pages);
lobj->user_pages = NULL;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index a828e3d0bfbd..3ccd61d69964 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -283,7 +283,6 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void 
*data,
 int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
 struct drm_file *filp)
 {
-   struct ttm_operation_ctx ctx = { true, false };
struct amdgpu_device *adev = dev->dev_private;
struct drm_amdgpu_gem_userptr *args = data;
struct drm_gem_object *gobj;
@@ -326,32 +325,12 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void 
*data,
goto release_object;
}
 
-   if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {
-   r = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages);
-   if (r)
-   goto release_object;
-
-   r = amdgpu_bo_reserve(bo, true);
-   if (r)
-   goto user_pages_done;
-
-   amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
-   r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
-   amdgpu_bo_unreserve(bo);
-   if (r)
-   goto user_pages_done;
-   }
-
r = drm_gem_handle_create(filp, gobj, &handle);
if (r)
-   goto user_pages_done;
+   goto release_object;
 
args->handle = handle;
 
-user_pages_done:
-   if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)
-   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
-
 release_object:
drm_gem_object_put_unlocked(gobj);
 
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

Thanks Joe for the test, I will add your Tested-by.

Hi Christian,

May you help review? The change removes the get user pages from 
gem_userptr_ioctl, this was done if flags AMDGPU_GEM_USERPTR_VALIDATE is 
set, and delay the get user pages to amdgpu_cs_parser_bos, and check if 
user pages are invalidated when amdgpu_cs_submit. I don't find issue for 
overnight test, but not sure if there is potential side effect.

Thanks,
Philip

On 2019-10-03 3:44 p.m., Yang, Philip wrote:
> user_pages array should always be freed after validation regardless if
> user pages are changed after bo is created because with HMM change parse
> bo always allocate user pages array to get user pages for userptr bo.
> 
> Don't need to get user pages while creating uerptr bo because user pages
> will only be used while validating after parsing userptr bo.
> 
> Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962
> 
> v2: remove unused local variable and amend commit
> 
> Signed-off-by: Philip Yang 
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  |  4 +---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 23 +--
>   2 files changed, 2 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 49b767b7238f..961186e7113e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -474,7 +474,6 @@ static int amdgpu_cs_list_validate(struct 
> amdgpu_cs_parser *p,
>   
>   list_for_each_entry(lobj, validated, tv.head) {
>   struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
> - bool binding_userptr = false;
>   struct mm_struct *usermm;
>   
>   usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
> @@ -491,14 +490,13 @@ static int amdgpu_cs_list_validate(struct 
> amdgpu_cs_parser *p,
>   
>   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
>lobj->user_pages);
> - binding_userptr = true;
>   }
>   
>   r = amdgpu_cs_validate(p, bo);
>   if (r)
>   return r;
>   
> - if (binding_userptr) {
> + if (lobj->user_pages) {
>   kvfree(lobj->user_pages);
>   lobj->user_pages = NULL;
>   }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index a828e3d0bfbd..3ccd61d69964 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -283,7 +283,6 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void 
> *data,
>   int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
>struct drm_file *filp)
>   {
> - struct ttm_operation_ctx ctx = { true, false };
>   struct amdgpu_device *adev = dev->dev_private;
>   struct drm_amdgpu_gem_userptr *args = data;
>   struct drm_gem_object *gobj;
> @@ -326,32 +325,12 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, 
> void *data,
>   goto release_object;
>   }
>   
> - if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {
> - r = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages);
> - if (r)
> - goto release_object;
> -
> - r = amdgpu_bo_reserve(bo, true);
> - if (r)
> - goto user_pages_done;
> -
> - amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
> - r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
> - amdgpu_bo_unreserve(bo);
> - if (r)
> - goto user_pages_done;
> - }
> -
>   r = drm_gem_handle_create(filp, gobj, &handle);
>   if (r)
> - goto user_pages_done;
> + goto release_object;
>   
>   args->handle = handle;
>   
> -user_pages_done:
> - if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)
> - amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
> -
>   release_object:
>   drm_gem_object_put_unlocked(gobj);
>   
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

On 2019-10-11 4:40 a.m., Christian König wrote:
> Am 03.10.19 um 21:44 schrieb Yang, Philip:
>> user_pages array should always be freed after validation regardless if
>> user pages are changed after bo is created because with HMM change parse
>> bo always allocate user pages array to get user pages for userptr bo.
>>
>> Don't need to get user pages while creating uerptr bo because user pages
>> will only be used while validating after parsing userptr bo.
>>
>> Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962
>>
>> v2: remove unused local variable and amend commit
>>
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  |  4 +---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 23 +--
>>   2 files changed, 2 insertions(+), 25 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> index 49b767b7238f..961186e7113e 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> @@ -474,7 +474,6 @@ static int amdgpu_cs_list_validate(struct 
>> amdgpu_cs_parser *p,
>>   list_for_each_entry(lobj, validated, tv.head) {
>>   struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
>> -    bool binding_userptr = false;
>>   struct mm_struct *usermm;
>>   usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
>> @@ -491,14 +490,13 @@ static int amdgpu_cs_list_validate(struct 
>> amdgpu_cs_parser *p,
>>   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
>>    lobj->user_pages);
>> -    binding_userptr = true;
>>   }
>>   r = amdgpu_cs_validate(p, bo);
>>   if (r)
>>   return r;
>> -    if (binding_userptr) {
>> +    if (lobj->user_pages) {
>>   kvfree(lobj->user_pages);
>>   lobj->user_pages = NULL;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>> index a828e3d0bfbd..3ccd61d69964 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>> @@ -283,7 +283,6 @@ int amdgpu_gem_create_ioctl(struct drm_device 
>> *dev, void *data,
>>   int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
>>    struct drm_file *filp)
>>   {
>> -    struct ttm_operation_ctx ctx = { true, false };
>>   struct amdgpu_device *adev = dev->dev_private;
>>   struct drm_amdgpu_gem_userptr *args = data;
>>   struct drm_gem_object *gobj;
>> @@ -326,32 +325,12 @@ int amdgpu_gem_userptr_ioctl(struct drm_device 
>> *dev, void *data,
>>   goto release_object;
>>   }
>> -    if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {
> 
> We can't drop that handling here, it is mandatory to detect an 
> application bug where an application tries to user an userptr with a VMA 
> which is not anonymous memory.
> 
> This must be detected and rejected as invalid here.
> 
> I suggest that we allocate a local pages array similar to how we do it 
> during CS and release that after the function is done.
> 
Thanks for this, we can use bo->tbo.ttm->pages array here to avoid extra 
alloc/free of pages array because CS uses local pages array and update 
to bo->tbo.ttm->pages if user pages are moved. I will submit patch v3 
for review.

> Regards,
> Christian.
> 
>> -    r = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages);
>> -    if (r)
>> -    goto release_object;
>> -
>> -    r = amdgpu_bo_reserve(bo, true);
>> -    if (r)
>> -    goto user_pages_done;
>> -
>> -    amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
>> -    r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
>> -    amdgpu_bo_unreserve(bo);
>> -    if (r)
>> -    goto user_pages_done;
>> -    }
>> -
>>   r = drm_gem_handle_create(filp, gobj, &handle);
>>   if (r)
>> -    goto user_pages_done;
>> +    goto release_object;
>>   args->handle = handle;
>> -user_pages_done:
>> -    if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)
>> -    amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>> -
>>   release_object:
>>   drm_gem_object_put_unlocked(gobj);
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH v3] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

user_pages array should always be freed after validation regardless if
user pages are changed after bo is created because with HMM change parse
bo always allocate user pages array to get user pages for userptr bo.

v2: remove unused local variable and amend commit

v3: add back get user pages in gem_userptr_ioctl, to detect application
bug where an userptr VMA is not ananymous memory and reject it.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

Signed-off-by: Philip Yang 
Tested-by: Joe Barnett 
Reviewed-by: Christian König 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index c18a153b3d2a..e7b39daa22f6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser 
*p,
 
list_for_each_entry(lobj, validated, tv.head) {
struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-   bool binding_userptr = false;
struct mm_struct *usermm;
 
usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
@@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct 
amdgpu_cs_parser *p,
 
amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
 lobj->user_pages);
-   binding_userptr = true;
}
 
r = amdgpu_cs_validate(p, bo);
if (r)
return r;
 
-   if (binding_userptr) {
+   if (lobj->user_pages) {
kvfree(lobj->user_pages);
lobj->user_pages = NULL;
}
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH v3] drm/amdgpu: user pages array memory leak fix

[Yang, Philip]

On 2019-10-11 1:33 p.m., Kuehling, Felix wrote:
> On 2019-10-11 10:36 a.m., Yang, Philip wrote:
>> user_pages array should always be freed after validation regardless if
>> user pages are changed after bo is created because with HMM change parse
>> bo always allocate user pages array to get user pages for userptr bo.
>>
>> v2: remove unused local variable and amend commit
>>
>> v3: add back get user pages in gem_userptr_ioctl, to detect application
>> bug where an userptr VMA is not ananymous memory and reject it.
>>
>> Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962
>>
>> Signed-off-by: Philip Yang 
>> Tested-by: Joe Barnett 
>> Reviewed-by: Christian König 
>> ---
>>drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
>>1 file changed, 1 insertion(+), 3 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> index c18a153b3d2a..e7b39daa22f6 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>> @@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct 
>> amdgpu_cs_parser *p,
>>
>>  list_for_each_entry(lobj, validated, tv.head) {
>>  struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
>> -bool binding_userptr = false;
>>  struct mm_struct *usermm;
>>
>>  usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
>> @@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct 
>> amdgpu_cs_parser *p,
>>
>>  amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
>>   lobj->user_pages);
>> -binding_userptr = true;
>>  }
>>
>>  r = amdgpu_cs_validate(p, bo);
>>  if (r)
>>  return r;
>>
>> -if (binding_userptr) {
>> +if (lobj->user_pages) {
> 
> This if is not needed. kvfree should be able to handle NULL pointers,
> and unconditionally setting the pointer to NULL afterwards is not a
> problem either. With that fixed, this commit is
> 
> Reviewed-by: Felix Kuehling 
> 
> However, I don't think this should be the final solution. My concern
> with this solution is, that you end up freeing and regenerating the
> user_pages arrays more frequently than necessary: On every command
> submission, even if there was no MMU notifier since the last command
> submission. I was hoping we could get back to a solution where we can
> maintain the same user_pages array across command submissions, since MMU
> notifiers are rare. That should reduce overhead from doing all thos page
> table walks in HMM on every command submissions when using userptrs.
> 
Yes, I will have another patch to address this using hmm_range_valid, 
the idea is to allow hmm range tracking cross gem_userptr_ioctl and 
cs_ioctl.

Thanks,
Philip

> Regards,
>     Felix
> 
> 
>>  kvfree(lobj->user_pages);
>>  lobj->user_pages = NULL;
>>  }
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/2] drm/amdgpu: use HMM mirror callback to replace mmu notifier v5

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

This depends on several HMM patchset from Jérôme Glisse queued for
upstream.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 124 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 57 insertions(+), 77 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e54..960a633 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index 138cb78..c1e5d43 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -171,7 +171,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index e55508b..56595b3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -87,6 +86,9 @@ struct amdgpu_mn {
struct rb_root_cached   objects;
struct mutexread_lock;
atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +105,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -237,21 +237,19 @@ static void amdgpu_mn_invalidate_node(struct 
amdgpu_mn_node *node,
 /**
  * amdgpu_mn_invalidate_range_start_gfx - callback to notify about mm change
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
- * @start: start of updated range
- * @end: end of updated range
+ * @mirror: the hmm_mirror (mm) is about to update
+ * @update: the up

[PATCH 2/2] drm/amdgpu: replace get_user_pages with HMM address mirror helpers

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

To avoid circular lock dependency, the locking order is:
mmap_sem -> amdgpu_mn_lock(p->mn) -> bo::reserve
mmap_sem -> bo::reserve

HMM simplify the CPU page table concurrently update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM doesnot pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: Iffd5f855cc9ce402cdfca167f68f83fe39ac56f9
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 101 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c  |   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h  |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c   | 171 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c  |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c   |  34 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h   |   7 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c  | 164 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h  |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c   |   1 -
 10 files changed, 252 insertions(+), 248 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index df0a059..3fd0340 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -615,8 +615,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 free_out:
kvfree(mem->user_pages);
mem->user_pages = NULL;
@@ -678,7 +677,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -742,7 +740,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1311,9 +1308,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
/* Free user pages if necessary */
if (mem->user_pages) {
pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
kvfree(mem->user_pages);
}
 
@@ -1739,8 +1733,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
   __func__);
return -ENOMEM;
}
-   } else if (mem->user_pages[0]) {
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
}
 
/* Get updated user pages */
@@ -1756,12 +1748,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
 * stalled user mode queues.
 */
}
-
-   /* Mark the BO as valid unless it was invalidated
-* again concurrently
-*/
-   if (atomic_cmpxchg(&mem->invalid, invalid, 0) != invalid)
-   return -EAGAIN;
}
 
return 0;
@@ -1854,14 +1840,10 @@ static int validate_invalid_user_pages(struct 
amdkfd_process_info *process_info)
}
 
/* Validate succeeded, now the BO owns the pages, free
-* our copy of the pointer array. Put this BO back on
-* the userptr_valid_list. If we need to revalidate
-* it, we need to start from scratch.
+* our copy of the pointer array.
 */
kvfree(mem->user_pages);
mem->user_pages = NULL;
-   list_move_tail(&mem->validate_list.head,
-  &process_info->userptr_valid_list);
 
/* Update mapping. If the BO was not validated
 * (because we couldn't get user pages), this will
@@ -1902,6 +1884,70 @@ static int

Re: [PATCH 2/2] drm/amdgpu: replace get_user_pages with HMM address mirror helpers

[Yang, Philip]

On 2018-10-17 04:15 AM, Christian König wrote:
> Am 17.10.18 um 04:56 schrieb Yang, Philip:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> To avoid circular lock dependency, the locking order is:
>> mmap_sem -> amdgpu_mn_lock(p->mn) -> bo::reserve
>> mmap_sem -> bo::reserve
>
> I'm not sure that this will work, we used to have some dependencies on 
> bo::reserve -> mmap_sem.
>
> See the following patch as well:
>
> commit 2f568dbd6b944c2e8c0c54b53c2211c23995e6a4
> Author: Christian König 
> Date:   Tue Feb 23 12:36:59 2016 +0100
>
>     drm/amdgpu: move get_user_pages out of amdgpu_ttm_tt_pin_userptr v6
>
>     That avoids lock inversion between the BO reservation lock
>     and the anon_vma lock.
>
> A whole bunch of more problems below.
>
Thanks for the review.

I will remove the nested lock between mmap_sem and bo::reserve, and 
change the kfd side locking order
to bo::reserve -> amdgpu_mn_lock(p->mn) as the original gfx side locking 
order.

Will submit patch v2 with other changes.

Philip
>>
>> HMM simplify the CPU page table concurrently update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM doesnot pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: Iffd5f855cc9ce402cdfca167f68f83fe39ac56f9
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 101 ++---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c  |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h  |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c   | 171 
>> +--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c  |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c   |  34 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h   |   7 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c  | 164 
>> +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h  |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c   |   1 -
>>   10 files changed, 252 insertions(+), 248 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index df0a059..3fd0340 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -615,8 +615,7 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   amdgpu_bo_unreserve(bo);
>>     release_out:
>> -    if (ret)
>> -    release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> +    amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>>   free_out:
>>   kvfree(mem->user_pages);
>>   mem->user_pages = NULL;
>> @@ -678,7 +677,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
>>   ctx->kfd_bo.priority = 0;
>>   ctx->kfd_bo.tv.bo = &bo->tbo;
>>   ctx->kfd_bo.tv.shared = true;
>> -    ctx->kfd_bo.user_pages = NULL;
>>   list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>>     amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
>> @@ -742,7 +740,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem 
>> *mem,
>>   ctx->kfd_bo.priority = 0;
>>   ctx->kfd_bo.tv.bo = &bo->tbo;
>>   ctx->kfd_bo.tv.shared = true;
>> -    ctx->kfd_bo.user_pages = NULL;
>>   list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>>     i = 0;
>> @@ -1311,9 +1308,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
>>   /* Free user pages if necessary */
>>   if (mem->user_pages) {
>>   pr_debug("%s: Freeing user_pages array\n", __func__);
>> -    if (mem->user_pages[0])
>> -    release_pages(mem->user_pages,
>> -    mem->bo->tbo.ttm->num_pages);
>>   kvfree(mem->user_pages);
>>   }
>>   @@ -1739,8 +1733,6 @@ static int update_invalid_user_pages(struct 
>> amdkfd_proc

[PATCH 2/2] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v2

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

To avoid circular lock dependency, no nested locking between mmap_sem
and bo::reserve. The locking order is:
bo::reserve -> amdgpu_mn_lock(p->mn)

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

v2:
* Remove nested locking between mmap_sem and bo::reserve
* Change locking order to bo::reserve -> amdgpu_mn_lock()
* Use dynamic allocation to replace VLA in kernel stack

Change-Id: Iffd5f855cc9ce402cdfca167f68f83fe39ac56f9
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c   | 101 ---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c|   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h|   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 188 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c|  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c |  34 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   7 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c| 164 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h|   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c |   1 -
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c  |  67 
 11 files changed, 312 insertions(+), 272 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index df0a059..3fd0340 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -615,8 +615,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 free_out:
kvfree(mem->user_pages);
mem->user_pages = NULL;
@@ -678,7 +677,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -742,7 +740,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1311,9 +1308,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
/* Free user pages if necessary */
if (mem->user_pages) {
pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
kvfree(mem->user_pages);
}
 
@@ -1739,8 +1733,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
   __func__);
return -ENOMEM;
}
-   } else if (mem->user_pages[0]) {
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
}
 
/* Get updated user pages */
@@ -1756,12 +1748,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
 * stalled user mode queues.
 */
}
-
-   /* Mark the BO as valid unless it was invalidated
-* again concurrently
-*/
-   if (atomic_cmpxchg(&mem->invalid, invalid, 0) != invalid)
-   return -EAGAIN;
}
 
return 0;
@@ -1854,14 +1840,10 @@ static int validate_invalid_user_pages(struct 
amdkfd_process_info *process_info)
}
 
/* Validate succeeded, now the BO owns the pages, free
-* our copy of the pointer array. Put this BO back on
-* the userptr_valid_list. If we need to revalidate
-* it, we need to start from scratch.
+* our copy of the pointer array.
 */
kvfree(mem->user_pages);
mem->user_pages = NULL;
-   list

[PATCH] drm/amdgpu: fix sdma v4 ring is disabled accidently

[Yang, Philip]

For sdma v4, there is bug caused by
commit d4e869b6b5d6 ("drm/amdgpu: add ring test for page queue")'

local variable ring is reused and changed, so 
amdgpu_ttm_set_buffer_funcs_status(adev, true)
is skipped accidently. As a result, amdgpu_fill_buffer() will fail, kernel 
message:

[drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory with ring 
turned off.
[   25.260444] [drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory 
with ring turned off.
[   25.260627] [drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory 
with ring turned off.
[   25.290119] [drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory 
with ring turned off.
[   25.290370] [drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory 
with ring turned off.
[   25.319971] [drm:amdgpu_fill_buffer [amdgpu]] *ERROR* Trying to clear memory 
with ring turned off.
[   25.320486] amdgpu :19:00.0: [mmhub] VMC page fault (src_id:0 ring:154 
vmid:8 pasid:32768, for process  pid 0 thread  pid 0)
[   25.320533] amdgpu :19:00.0:   in page starting at address 
0x from 18
[   25.320563] amdgpu :19:00.0: VM_L2_PROTECTION_FAULT_STATUS:0x00800134

Change-Id: Idacdf8e60557edb0a4a499aa4051b75d87ce4091
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index ede149a..cd368ac 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -1151,10 +1151,11 @@ static int sdma_v4_0_start(struct amdgpu_device *adev)
}
 
if (adev->sdma.has_page_queue) {
-   ring = &adev->sdma.instance[i].page;
-   r = amdgpu_ring_test_ring(ring);
+   struct amdgpu_ring *page = &adev->sdma.instance[i].page;
+
+   r = amdgpu_ring_test_ring(page);
if (r) {
-   ring->ready = false;
+   page->ready = false;
return r;
}
}
-- 
2.7.4

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: use module parameter noretry for gfx and kfd

[Yang, Philip]

Currently kfd uses noretry module parameter but gfx uses hardcode
setting. Change both to use same noretry module parameter.

Set default value to 1, to disable retry for better performance.

Export noretry value to kfd userspace runtime through topology.

The permission is 0644, means it can be changed at runtime, the dynamic
change only works for kfd because kfd uses the value while user queue
is created by application. But gfx only uses the value once when doing
gfx hw init.

Change-Id: I368bafeba6b0aa792d5d5680ba4ef5c8b8f48087
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h|  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c| 22 +++---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c  |  4 
 .../drm/amd/amdkfd/kfd_device_queue_manager_v9.c   |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h  |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_topology.c  |  1 +
 6 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index c773ea1..c84db7c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -152,6 +152,7 @@ extern int amdgpu_gpu_recovery;
 extern int amdgpu_emu_mode;
 extern uint amdgpu_smu_memory_pool_size;
 extern struct amdgpu_mgpu_info mgpu_info;
+extern int amdgpu_noretry;
 
 #ifdef CONFIG_DRM_AMDGPU_SI
 extern int amdgpu_si_support;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 27528ee..f4ca1b5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -130,6 +130,7 @@ uint amdgpu_smu_memory_pool_size = 0;
 struct amdgpu_mgpu_info mgpu_info = {
.mutex = __MUTEX_INITIALIZER(mgpu_info.mutex),
 };
+int amdgpu_noretry = 1;
 
 /**
  * DOC: vramlimit (int)
@@ -535,6 +536,16 @@ MODULE_PARM_DESC(smu_memory_pool_size,
"0x1 = 256Mbyte, 0x2 = 512Mbyte, 0x4 = 1 Gbyte, 0x8 = 2GByte");
 module_param_named(smu_memory_pool_size, amdgpu_smu_memory_pool_size, uint, 
0444);
 
+/**
+ * DOC: noretry (int)
+ * This parameter sets sh_mem_config.retry_disable. Default value, 1, disable 
retry.
+ * Setting 0 enables retry.
+ * Retry is needed for recoverable page faults.
+ */
+module_param_named(noretry, amdgpu_noretry, int, 0644);
+MODULE_PARM_DESC(noretry,
+   "Set sh_mem_config.retry_disable on Vega10 (0 = retry enabled 
(default), 1 = retry disabled)");
+
 #ifdef CONFIG_HSA_AMD
 /**
  * DOC: sched_policy (int)
@@ -611,17 +622,6 @@ MODULE_PARM_DESC(ignore_crat,
"Ignore CRAT table during KFD initialization (0 = use CRAT (default), 1 
= ignore CRAT)");
 
 /**
- * DOC: noretry (int)
- * This parameter sets sh_mem_config.retry_disable. Default value, 0, enables 
retry.
- * Setting 1 disables retry.
- * Retry is needed for recoverable page faults.
- */
-int noretry;
-module_param(noretry, int, 0644);
-MODULE_PARM_DESC(noretry,
-   "Set sh_mem_config.retry_disable on Vega10 (0 = retry enabled 
(default), 1 = retry disabled)");
-
-/**
  * DOC: halt_if_hws_hang (int)
  * Halt if HWS hang is detected. Default value, 0, disables the halt on hang.
  * Setting 1 enables halt on hang.
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c 
b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index 4281a37..f00f04e 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -1980,11 +1980,15 @@ static void gfx_v9_0_constants_init(struct 
amdgpu_device *adev)
if (i == 0) {
tmp = REG_SET_FIELD(0, SH_MEM_CONFIG, ALIGNMENT_MODE,
SH_MEM_ALIGNMENT_MODE_UNALIGNED);
+   tmp = REG_SET_FIELD(tmp, SH_MEM_CONFIG, RETRY_DISABLE,
+   amdgpu_noretry);
WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, tmp);
WREG32_SOC15(GC, 0, mmSH_MEM_BASES, 0);
} else {
tmp = REG_SET_FIELD(0, SH_MEM_CONFIG, ALIGNMENT_MODE,
SH_MEM_ALIGNMENT_MODE_UNALIGNED);
+   tmp = REG_SET_FIELD(tmp, SH_MEM_CONFIG, RETRY_DISABLE,
+   amdgpu_noretry);
WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, tmp);
tmp = REG_SET_FIELD(0, SH_MEM_BASES, PRIVATE_BASE,
(adev->gmc.private_aperture_start >> 48));
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
index 4175153..ae1fdb7 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c
@@ -60,7 +60,7 @@ static int update_qpd_v9(struct device_queue_manager *dqm,
qpd->sh_mem_config =
SH_MEM_ALIGNMENT_MODE_UN

Re: [PATCH] drm/amdgpu: disable page queue on Vega10 SR-IOV VF

[Yang, Philip]

On 2018-11-07 12:53 p.m., Kuehling, Felix wrote:
> [+Philip]
>
> On 2018-11-07 12:25 a.m., Zhang, Jerry(Junwei) wrote:
>> On 11/7/18 1:15 PM, Trigger Huang wrote:
>>> Currently, SDMA page queue is not used under SR-IOV VF, and this
>>> queue will
>>> cause ring test failure in amdgpu module reload case. So just disable
>>> it.
>>>
>>> Signed-off-by: Trigger Huang 
>> Looks we ran into several issues about it on vega.
>> kfd also disabled vega10 for development.(but not sure the detail
>> issue for them)
>>
>> Thus, we may disable it for vega10 as well?
>> any comment? Alex, Christian, Flex.
> We ran into a regression with the page queue in a specific KFDTest that
> runs user mode SDMA in two processes. The SDMA engine would stall for
> about 6 seconds after one of the processes terminates (and destroys its
> queues). We don't have a root cause. Suspect an SDMA firmware issue.
>
> Regards,
>    Felix
The SDMA firmware has root cause the bug, I have tested one SDMA 
firmware, that fixed
the KFDIPCTest.BasicTest issue. I am waiting for SDMA firmware check in 
then enable paging
queue for Vega 10. I has asked SDAM firmware if Vega12/Vega20 paging 
queue issue has
the same root cause.

Philip
>
>> Regards,
>> Jerry
>>> ---
>>>    drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 4 +++-
>>>    1 file changed, 3 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> index e39a09eb0f..4edc848 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> @@ -1451,7 +1451,9 @@ static int sdma_v4_0_early_init(void *handle)
>>>    adev->sdma.has_page_queue = false;
>>>    } else {
>>>    adev->sdma.num_instances = 2;
>>> -    if (adev->asic_type != CHIP_VEGA20 &&
>>> +    if ((adev->asic_type == CHIP_VEGA10) &&
>>> amdgpu_sriov_vf((adev)))
>>> +    adev->sdma.has_page_queue = false;
>>> +    else if (adev->asic_type != CHIP_VEGA20 &&
>>>    adev->asic_type != CHIP_VEGA12)
>>>    adev->sdma.has_page_queue = true;
>>>    }
>> ___
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: fix bug with IH ring setup

[Yang, Philip]

The bug limits the IH ring wptr address to 40bit. When the system memory
is bigger than 1TB, the bus address is more than 40bit, this causes the
interrupt cannot be handled and cleared correctly.

Change-Id: I3cd1b8ad046b38945372f2fd1a2d225624893e28
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index a99f717..a0fda6f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -129,7 +129,7 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
else
wptr_off = adev->wb.gpu_addr + (adev->irq.ih.wptr_offs * 4);
WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_ADDR_LO, lower_32_bits(wptr_off));
-   WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_ADDR_HI, upper_32_bits(wptr_off) & 
0xFF);
+   WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_ADDR_HI, upper_32_bits(wptr_off) & 
0x);
 
/* set rptr, wptr to 0 */
WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR, 0);
-- 
2.7.4

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: enable paging queue doorbell support

[Yang, Philip]

paging queues doorbell index use existing assignment sDMA_HI_PRI_ENGINE0/1
index, and increase SDMA_DOORBELL_RANGE size from 2 dwords to 4 dwords to
enable the new doorbell index.

Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 25 +
 3 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c 
b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
index 6f9c54978cc1..0eb42c29ecac 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
@@ -80,7 +80,7 @@ static void nbio_v6_1_sdma_doorbell_range(struct 
amdgpu_device *adev, int instan
 
if (use_doorbell) {
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
-   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
+   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
} else
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c 
b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
index f8cee95d61cc..9342ee03d7d4 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
@@ -76,7 +76,7 @@ static void nbio_v7_4_sdma_doorbell_range(struct 
amdgpu_device *adev, int instan
 
if (use_doorbell) {
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
-   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
+   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
} else
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index f4490cdd9804..96c9e83204b7 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
*adev, unsigned int i)
OFFSET, ring->doorbell_index);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
-   /* TODO: enable doorbell support */
-   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
- ring->doorbell_index);*/
 
-   sdma_v4_0_ring_set_wptr(ring);
+   /* paging queue doorbell index is already enabled at 
sdma_v4_0_gfx_resume */
+   sdma_v4_0_page_ring_set_wptr(ring);
 
/* set minor_ptr_update to 0 after wptr programed */
WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
@@ -1504,9 +1502,6 @@ static int sdma_v4_0_sw_init(void *handle)
ring->ring_obj = NULL;
ring->use_doorbell = true;
 
-   DRM_INFO("use_doorbell being set to: [%s]\n",
-   ring->use_doorbell?"true":"false");
-
if (adev->asic_type == CHIP_VEGA10)
ring->doorbell_index = (i == 0) ?
(AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
//get DWORD offset
@@ -1516,6 +1511,8 @@ static int sdma_v4_0_sw_init(void *handle)
(AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get 
DWORD offset
: (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get 
DWORD offset
 
+   DRM_INFO("use_doorbell being set to: [%s] doorbell index %d\n",
+   ring->use_doorbell?"true":"false", 
ring->doorbell_index);
 
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
@@ -1529,7 +1526,19 @@ static int sdma_v4_0_sw_init(void *handle)
if (adev->sdma.has_page_queue) {
ring = &adev->sdma.instance[i].page;
ring->ring_obj = NULL;
-   ring->use_doorbell = false;
+   ring->use_doorbell = true;
+
+   if (adev->asic_type == CHIP_VEGA10)
+   ring->doorbell_index = (i == 0) ?
+   
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE0 << 1)
+   : 
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE1 << 1);
+   else
+   ring->doorbell_index = (i == 0) ?
+   
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE0 << 1)
+   : 
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE1 << 1)

Re: [PATCH] drm/amdgpu: enable paging queue doorbell support

[Yang, Philip]

On 2018-11-15 11:43 a.m., Alex Deucher wrote:
> On Thu, Nov 15, 2018 at 11:08 AM Yang, Philip  wrote:
>> paging queues doorbell index use existing assignment sDMA_HI_PRI_ENGINE0/1
>> index, and increase SDMA_DOORBELL_RANGE size from 2 dwords to 4 dwords to
>> enable the new doorbell index.
>>
>> Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
>> Signed-off-by: Philip Yang 
> Is there a specific fw version requirement for this?  If so, we need
> to add a check.  Also will this break SR-IOV due to the doorbell
> mapping requirements for other OSes?  Have we resolved that yet?
>
Yes, new FW is required to enable paging queue, it's good idea to check 
FW version
when we enable paging queue later after new FW is checked in. paging 
queue is not enabled yet.
This change will not take effect until paging queue is enabled.

This does not change doorbell mapping so it will not break SR-IOV. My 
understanding is Doorbell
mapping is good for vega10/12/20, but need update for future chip.
>> ---
>>   drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c |  2 +-
>>   drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c |  2 +-
>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 25 +
>>   3 files changed, 19 insertions(+), 10 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c 
>> b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>> index 6f9c54978cc1..0eb42c29ecac 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>> @@ -80,7 +80,7 @@ static void nbio_v6_1_sdma_doorbell_range(struct 
>> amdgpu_device *adev, int instan
>>
>>  if (use_doorbell) {
>>  doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
>> -   doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
>> +   doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
>>  } else
>>  doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c 
>> b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> index f8cee95d61cc..9342ee03d7d4 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> @@ -76,7 +76,7 @@ static void nbio_v7_4_sdma_doorbell_range(struct 
>> amdgpu_device *adev, int instan
>>
>>  if (use_doorbell) {
>>  doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
>> -   doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
>> +   doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
>>  } else
>>  doorbell_range = REG_SET_FIELD(doorbell_range, 
>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> index f4490cdd9804..96c9e83204b7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> @@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
>> *adev, unsigned int i)
>>  OFFSET, ring->doorbell_index);
>>  WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
>>  WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
>> -   /* TODO: enable doorbell support */
>> -   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
>> - ring->doorbell_index);*/
>>
>> -   sdma_v4_0_ring_set_wptr(ring);
>> +   /* paging queue doorbell index is already enabled at 
>> sdma_v4_0_gfx_resume */
>> +   sdma_v4_0_page_ring_set_wptr(ring);
>>
>>  /* set minor_ptr_update to 0 after wptr programed */
>>  WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
>> @@ -1504,9 +1502,6 @@ static int sdma_v4_0_sw_init(void *handle)
>>  ring->ring_obj = NULL;
>>  ring->use_doorbell = true;
>>
>> -   DRM_INFO("use_doorbell being set to: [%s]\n",
>> -   ring->use_doorbell?"true":"false");
>> -
>>  if (adev->asic_type == CHIP_VEGA10)
>>  ring->doorbell_index = (i == 0) ?
>>   

Re: [PATCH] drm/amdgpu: enable paging queue doorbell support

[Yang, Philip]

Thanks, I will disable paging queue doorbell on Vega10 and Vega12 with 
SRIOV.

Philip

On 2018-11-15 1:01 p.m., Kuehling, Felix wrote:
> You changed the doorbell routing in NBIO. I think that won't work for SR-IOV, 
> because it's not controlled by the guest OS there. We may need to disable 
> paging queue doorbell on Vega10 and Vega12 with SRIOV. For Vega20 we plan to 
> change the doorbell layout before it goes to production (Oak started working 
> on that). So on Vega20 we should be able to enable the doorbell for the 
> paging queue.
>
> Regards,
>Felix
>
> -Original Message-----
> From: amd-gfx  On Behalf Of Yang, 
> Philip
> Sent: Thursday, November 15, 2018 12:54 PM
> To: Alex Deucher 
> Cc: amd-gfx list 
> Subject: Re: [PATCH] drm/amdgpu: enable paging queue doorbell support
>
> On 2018-11-15 11:43 a.m., Alex Deucher wrote:
>> On Thu, Nov 15, 2018 at 11:08 AM Yang, Philip  wrote:
>>> paging queues doorbell index use existing assignment
>>> sDMA_HI_PRI_ENGINE0/1 index, and increase SDMA_DOORBELL_RANGE size
>>> from 2 dwords to 4 dwords to enable the new doorbell index.
>>>
>>> Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
>>> Signed-off-by: Philip Yang 
>> Is there a specific fw version requirement for this?  If so, we need
>> to add a check.  Also will this break SR-IOV due to the doorbell
>> mapping requirements for other OSes?  Have we resolved that yet?
>>
> Yes, new FW is required to enable paging queue, it's good idea to check FW 
> version when we enable paging queue later after new FW is checked in. paging 
> queue is not enabled yet.
> This change will not take effect until paging queue is enabled.
>
> This does not change doorbell mapping so it will not break SR-IOV. My 
> understanding is Doorbell mapping is good for vega10/12/20, but need update 
> for future chip.
>>> ---
>>>drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c |  2 +-
>>>drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c |  2 +-
>>>drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 25 +
>>>3 files changed, 19 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>>> b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>>> index 6f9c54978cc1..0eb42c29ecac 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
>>> @@ -80,7 +80,7 @@ static void nbio_v6_1_sdma_doorbell_range(struct
>>> amdgpu_device *adev, int instan
>>>
>>>   if (use_doorbell) {
>>>   doorbell_range = REG_SET_FIELD(doorbell_range, 
>>> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
>>> -   doorbell_range = REG_SET_FIELD(doorbell_range, 
>>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
>>> +   doorbell_range = REG_SET_FIELD(doorbell_range,
>>> + BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
>>>   } else
>>>   doorbell_range = REG_SET_FIELD(doorbell_range,
>>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>>> b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>>> index f8cee95d61cc..9342ee03d7d4 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>>> @@ -76,7 +76,7 @@ static void nbio_v7_4_sdma_doorbell_range(struct
>>> amdgpu_device *adev, int instan
>>>
>>>   if (use_doorbell) {
>>>   doorbell_range = REG_SET_FIELD(doorbell_range, 
>>> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
>>> -   doorbell_range = REG_SET_FIELD(doorbell_range, 
>>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
>>> +   doorbell_range = REG_SET_FIELD(doorbell_range,
>>> + BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
>>>   } else
>>>   doorbell_range = REG_SET_FIELD(doorbell_range,
>>> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> index f4490cdd9804..96c9e83204b7 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>>> @@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
>>> *adev, unsigned int i)
>>>   OFFSET, ring->doorbell_index);
>>>   WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
>>>

[PATCH] drm/amdgpu: enable paging queue doorbell support v2

[Yang, Philip]

paging queues doorbell index use existing assignment sDMA_HI_PRI_ENGINE0/1
index, and increase SDMA_DOORBELL_RANGE size from 2 dwords to 4 dwords to
enable the new doorbell index.

v2: disable paging queue doorbell on Vega10 and Vega12 with SRIOV

Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 26 ++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c 
b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
index 6f9c54978cc1..0eb42c29ecac 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
@@ -80,7 +80,7 @@ static void nbio_v6_1_sdma_doorbell_range(struct 
amdgpu_device *adev, int instan
 
if (use_doorbell) {
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
-   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
+   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
} else
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c 
b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
index f8cee95d61cc..9342ee03d7d4 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
@@ -76,7 +76,7 @@ static void nbio_v7_4_sdma_doorbell_range(struct 
amdgpu_device *adev, int instan
 
if (use_doorbell) {
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
-   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
+   doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 4);
} else
doorbell_range = REG_SET_FIELD(doorbell_range, 
BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index f4490cdd9804..1f19f15bb171 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
*adev, unsigned int i)
OFFSET, ring->doorbell_index);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
-   /* TODO: enable doorbell support */
-   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
- ring->doorbell_index);*/
 
-   sdma_v4_0_ring_set_wptr(ring);
+   /* paging queue doorbell index is already enabled at 
sdma_v4_0_gfx_resume */
+   sdma_v4_0_page_ring_set_wptr(ring);
 
/* set minor_ptr_update to 0 after wptr programed */
WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
@@ -1504,9 +1502,6 @@ static int sdma_v4_0_sw_init(void *handle)
ring->ring_obj = NULL;
ring->use_doorbell = true;
 
-   DRM_INFO("use_doorbell being set to: [%s]\n",
-   ring->use_doorbell?"true":"false");
-
if (adev->asic_type == CHIP_VEGA10)
ring->doorbell_index = (i == 0) ?
(AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
//get DWORD offset
@@ -1516,6 +1511,8 @@ static int sdma_v4_0_sw_init(void *handle)
(AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get 
DWORD offset
: (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get 
DWORD offset
 
+   DRM_DEBUG("use_doorbell being set to: [%s] doorbell index %d\n",
+   ring->use_doorbell?"true":"false", 
ring->doorbell_index);
 
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
@@ -1529,7 +1526,20 @@ static int sdma_v4_0_sw_init(void *handle)
if (adev->sdma.has_page_queue) {
ring = &adev->sdma.instance[i].page;
ring->ring_obj = NULL;
-   ring->use_doorbell = false;
+   if (!amdgpu_sriov_vf(adev) || adev->asic_type == 
CHIP_VEGA20)
+   ring->use_doorbell = true;
+
+   if (adev->asic_type == CHIP_VEGA10)
+   ring->doorbell_index = (i == 0) ?
+   
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE0 << 1)
+   : 
(AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE1 << 1);
+   else
+   ring->doorbell_index = (i == 0) ?
+  

[PATCH] drm/amdgpu: enable paging queue doorbell support v3

[Yang, Philip]

Because increase SDMA_DOORBELL_RANGE to add new SDMA doorbell for paging queue 
will
break SRIOV, instead we can reserve and map two doorbell pages for amdgpu, 
paging
queues doorbell index use same index as SDMA gfx queues index but on second 
page.

For Vega20, after we change doorbell layout to increase SDMA doorbell for 8 
SDMA RLC
queues later, we could use new doorbell index for paging queue.

Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 13 +
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 33 +-
 2 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 590588a82471..7a54760591ae 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -534,6 +534,19 @@ static int amdgpu_device_doorbell_init(struct 
amdgpu_device *adev)
if (adev->doorbell.num_doorbells == 0)
return -EINVAL;
 
+   /* For Vega, reserve and map two pages on doorbell BAR since SDMA
+* paging queue doorbell use the second page
+*/
+   switch (adev->asic_type) {
+   case CHIP_VEGA10:
+   case CHIP_VEGA12:
+   case CHIP_VEGA20:
+   adev->doorbell.num_doorbells *= 2;
+   break;
+   default:
+   break;
+   }
+
adev->doorbell.ptr = ioremap(adev->doorbell.base,
 adev->doorbell.num_doorbells *
 sizeof(u32));
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index f4490cdd9804..db5a71db9f10 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
*adev, unsigned int i)
OFFSET, ring->doorbell_index);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
-   /* TODO: enable doorbell support */
-   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
- ring->doorbell_index);*/
 
-   sdma_v4_0_ring_set_wptr(ring);
+   /* paging queue doorbell index is already enabled at 
sdma_v4_0_gfx_resume */
+   sdma_v4_0_page_ring_set_wptr(ring);
 
/* set minor_ptr_update to 0 after wptr programed */
WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
@@ -1504,18 +1502,14 @@ static int sdma_v4_0_sw_init(void *handle)
ring->ring_obj = NULL;
ring->use_doorbell = true;
 
-   DRM_INFO("use_doorbell being set to: [%s]\n",
-   ring->use_doorbell?"true":"false");
-
if (adev->asic_type == CHIP_VEGA10)
ring->doorbell_index = (i == 0) ?
-   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
//get DWORD offset
-   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1); 
// get DWORD offset
+   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) // 
doorbell size 2 dwords
+   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1); 
// doorbell size 2 dwords
else
ring->doorbell_index = (i == 0) ?
-   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get 
DWORD offset
-   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get 
DWORD offset
-
+   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) // 
doorbell size 2 dwords
+   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // 
doorbell size 2 dwords
 
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
@@ -1529,7 +1523,20 @@ static int sdma_v4_0_sw_init(void *handle)
if (adev->sdma.has_page_queue) {
ring = &adev->sdma.instance[i].page;
ring->ring_obj = NULL;
-   ring->use_doorbell = false;
+   ring->use_doorbell = true;
+
+   /* paging queue use same doorbell index/routing as gfx 
queue
+* with 0x400 (4096 dwords) offset on second doorbell 
page
+*/
+   if (adev->asic_type == CHIP_VEGA10)
+   ring->doorbell_index = (i == 0) ?
+   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 
<< 1)
+   : 
(AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1);
+   else
+   ring->doorbell_index = (i == 0) ?
+   (AMDGPU_DOORBELL64_sDM

Re: [PATCH] drm/amdgpu: enable paging queue doorbell support v3

[Yang, Philip]

On 2018-11-16 4:35 p.m., Alex Deucher wrote:
> On Fri, Nov 16, 2018 at 2:08 PM Yang, Philip  wrote:
>> Because increase SDMA_DOORBELL_RANGE to add new SDMA doorbell for paging 
>> queue will
>> break SRIOV, instead we can reserve and map two doorbell pages for amdgpu, 
>> paging
>> queues doorbell index use same index as SDMA gfx queues index but on second 
>> page.
>>
>> For Vega20, after we change doorbell layout to increase SDMA doorbell for 8 
>> SDMA RLC
>> queues later, we could use new doorbell index for paging queue.
>>
>> Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 13 +
>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 33 +-
>>   2 files changed, 33 insertions(+), 13 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 590588a82471..7a54760591ae 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -534,6 +534,19 @@ static int amdgpu_device_doorbell_init(struct 
>> amdgpu_device *adev)
>>  if (adev->doorbell.num_doorbells == 0)
>>  return -EINVAL;
>>
>> +   /* For Vega, reserve and map two pages on doorbell BAR since SDMA
>> +* paging queue doorbell use the second page
>> +*/
>> +   switch (adev->asic_type) {
>> +   case CHIP_VEGA10:
>> +   case CHIP_VEGA12:
>> +   case CHIP_VEGA20:
>> +   adev->doorbell.num_doorbells *= 2;
>> +   break;
>> +   default:
>> +   break;
>> +   }
>> +
> To future proof this, maybe we should just change this to:
> if (asic_type >= VEGA10)
>number_doorbells *= 2;
>
OK
>>  adev->doorbell.ptr = ioremap(adev->doorbell.base,
>>   adev->doorbell.num_doorbells *
>>   sizeof(u32));
>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> index f4490cdd9804..db5a71db9f10 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> @@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
>> *adev, unsigned int i)
>>  OFFSET, ring->doorbell_index);
>>  WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
>>  WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
>> -   /* TODO: enable doorbell support */
>> -   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
>> - ring->doorbell_index);*/
>>
>> -   sdma_v4_0_ring_set_wptr(ring);
>> +   /* paging queue doorbell index is already enabled at 
>> sdma_v4_0_gfx_resume */
>> +   sdma_v4_0_page_ring_set_wptr(ring);
> This looks like a bug fix that should be a separate patch.
OK
>>  /* set minor_ptr_update to 0 after wptr programed */
>>  WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
>> @@ -1504,18 +1502,14 @@ static int sdma_v4_0_sw_init(void *handle)
>>  ring->ring_obj = NULL;
>>  ring->use_doorbell = true;
>>
>> -   DRM_INFO("use_doorbell being set to: [%s]\n",
>> -   ring->use_doorbell?"true":"false");
>> -
>>  if (adev->asic_type == CHIP_VEGA10)
>>  ring->doorbell_index = (i == 0) ?
>> -   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
>> //get DWORD offset
>> -   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 
>> 1); // get DWORD offset
>> +   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
>> // doorbell size 2 dwords
>> +   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 
>> 1); // doorbell size 2 dwords
>>  else
>>  ring->doorbell_index = (i == 0) ?
>> -   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get 
>> DWORD offset
>> -   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // 
>> get DWORD offset
>> -
>> +   (AMDGPU_DOORBELL64_sD

[PATCH 3/3] drm/amdgpu: enable paging queue based on FW version

[Yang, Philip]

Based SDMA fw version to enable has_page_queue support. Have to move
sdma_v4_0_init_microcode from sw_init to early_init, to load firmware
and init fw_version before set_ring/buffer/vm_pte_funcs use it.

Change-Id: Ife5d4659d28bc2a7012b48947b27e929749d87c1
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 46 +-
 1 file changed, 30 insertions(+), 16 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index 4d873fd3242c..0a3b68dd49a0 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -1447,23 +1447,44 @@ static void sdma_v4_0_ring_emit_reg_wait(struct 
amdgpu_ring *ring, uint32_t reg,
sdma_v4_0_wait_reg_mem(ring, 0, 0, reg, 0, val, mask, 10);
 }
 
+static bool sdma_v4_0_fw_support_paging_queue(struct amdgpu_device *adev)
+{
+   uint fw_version = adev->sdma.instance[0].fw_version;
+
+   switch (adev->asic_type) {
+   case CHIP_VEGA10:
+   return fw_version >= 430;
+   case CHIP_VEGA12:
+   return fw_version >= 31;
+   case CHIP_VEGA20:
+   return fw_version >= 115;
+   default:
+   return false;
+   }
+}
+
 static int sdma_v4_0_early_init(void *handle)
 {
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+   int r;
 
-   if (adev->asic_type == CHIP_RAVEN) {
+   if (adev->asic_type == CHIP_RAVEN)
adev->sdma.num_instances = 1;
-   adev->sdma.has_page_queue = false;
-   } else {
+   else
adev->sdma.num_instances = 2;
-   /* TODO: Page queue breaks driver reload under SRIOV */
-   if ((adev->asic_type == CHIP_VEGA10) && amdgpu_sriov_vf((adev)))
-   adev->sdma.has_page_queue = false;
-   else if (adev->asic_type != CHIP_VEGA20 &&
-   adev->asic_type != CHIP_VEGA12)
-   adev->sdma.has_page_queue = true;
+
+   r = sdma_v4_0_init_microcode(adev);
+   if (r) {
+   DRM_ERROR("Failed to load sdma firmware!\n");
+   return r;
}
 
+   /* TODO: Page queue breaks driver reload under SRIOV */
+   if ((adev->asic_type == CHIP_VEGA10) && amdgpu_sriov_vf((adev)))
+   adev->sdma.has_page_queue = false;
+   else if (sdma_v4_0_fw_support_paging_queue(adev))
+   adev->sdma.has_page_queue = true;
+
sdma_v4_0_set_ring_funcs(adev);
sdma_v4_0_set_buffer_funcs(adev);
sdma_v4_0_set_vm_pte_funcs(adev);
@@ -1472,7 +1493,6 @@ static int sdma_v4_0_early_init(void *handle)
return 0;
 }
 
-
 static int sdma_v4_0_sw_init(void *handle)
 {
struct amdgpu_ring *ring;
@@ -1491,12 +1511,6 @@ static int sdma_v4_0_sw_init(void *handle)
if (r)
return r;
 
-   r = sdma_v4_0_init_microcode(adev);
-   if (r) {
-   DRM_ERROR("Failed to load sdma firmware!\n");
-   return r;
-   }
-
for (i = 0; i < adev->sdma.num_instances; i++) {
ring = &adev->sdma.instance[i].ring;
ring->ring_obj = NULL;
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/3] drm/amdgpu: enable paging queue doorbell support v4

[Yang, Philip]

Because increase SDMA_DOORBELL_RANGE to add new SDMA doorbell for paging queue 
will
break SRIOV, instead we can reserve and map two doorbell pages for amdgpu, 
paging
queues doorbell index use same index as SDMA gfx queues index but on second 
page.

For Vega20, after we change doorbell layout to increase SDMA doorbell for 8 
SDMA RLC
queues later, we could use new doorbell index for paging queue.

Change-Id: I9adb965f16ee4089d261d9a22231337739184e49
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  6 +
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 28 +++---
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 590588a82471..cb06e6883fad 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -534,6 +534,12 @@ static int amdgpu_device_doorbell_init(struct 
amdgpu_device *adev)
if (adev->doorbell.num_doorbells == 0)
return -EINVAL;
 
+   /* For Vega, reserve and map two pages on doorbell BAR since SDMA
+* paging queue doorbell use the second page
+*/
+   if (adev->asic_type >= CHIP_VEGA10)
+   adev->doorbell.num_doorbells *= 2;
+
adev->doorbell.ptr = ioremap(adev->doorbell.base,
 adev->doorbell.num_doorbells *
 sizeof(u32));
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index 3f6b7882dbd2..4d873fd3242c 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -1502,18 +1502,15 @@ static int sdma_v4_0_sw_init(void *handle)
ring->ring_obj = NULL;
ring->use_doorbell = true;
 
-   DRM_INFO("use_doorbell being set to: [%s]\n",
-   ring->use_doorbell?"true":"false");
-
+   /* doorbell size is 2 dwords, get DWORD offset */
if (adev->asic_type == CHIP_VEGA10)
ring->doorbell_index = (i == 0) ?
-   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) 
//get DWORD offset
-   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1); 
// get DWORD offset
+   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1)
+   : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1);
else
ring->doorbell_index = (i == 0) ?
-   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get 
DWORD offset
-   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get 
DWORD offset
-
+   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1)
+   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1);
 
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
@@ -1527,7 +1524,20 @@ static int sdma_v4_0_sw_init(void *handle)
if (adev->sdma.has_page_queue) {
ring = &adev->sdma.instance[i].page;
ring->ring_obj = NULL;
-   ring->use_doorbell = false;
+   ring->use_doorbell = true;
+
+   /* paging queue use same doorbell index/routing as gfx 
queue
+* with 0x400 (4096 dwords) offset on second doorbell 
page
+*/
+   if (adev->asic_type == CHIP_VEGA10)
+   ring->doorbell_index = (i == 0) ?
+   (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 
<< 1)
+   : 
(AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1);
+   else
+   ring->doorbell_index = (i == 0) ?
+   (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1)
+   : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1);
+   ring->doorbell_index += 0x400;
 
sprintf(ring->name, "page%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/3] drm/amdgpu: fix typo in function sdma_v4_0_page_resume

[Yang, Philip]

This looks like copy paste typo

Change-Id: Iee3fd3a551650ec9199bc030a7886e92000b02e7
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index f4490cdd9804..3f6b7882dbd2 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -925,11 +925,9 @@ static void sdma_v4_0_page_resume(struct amdgpu_device 
*adev, unsigned int i)
OFFSET, ring->doorbell_index);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL, doorbell);
WREG32_SDMA(i, mmSDMA0_PAGE_DOORBELL_OFFSET, doorbell_offset);
-   /* TODO: enable doorbell support */
-   /*adev->nbio_funcs->sdma_doorbell_range(adev, i, ring->use_doorbell,
- ring->doorbell_index);*/
 
-   sdma_v4_0_ring_set_wptr(ring);
+   /* paging queue doorbell range is setup at sdma_v4_0_gfx_resume */
+   sdma_v4_0_page_ring_set_wptr(ring);
 
/* set minor_ptr_update to 0 after wptr programed */
WREG32_SDMA(i, mmSDMA0_PAGE_MINOR_PTR_UPDATE, 0);
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/3] drm/amdgpu: enable paging queue based on FW version

[Yang, Philip]

On 2018-11-19 3:57 p.m., Deucher, Alexander wrote:
>> -Original Message-
>> From: amd-gfx  On Behalf Of
>> Yang, Philip
>> Sent: Monday, November 19, 2018 3:20 PM
>> To: amd-gfx@lists.freedesktop.org
>> Cc: Yang, Philip 
>> Subject: [PATCH 3/3] drm/amdgpu: enable paging queue based on FW
>> version
>>
>> Based SDMA fw version to enable has_page_queue support. Have to move
>> sdma_v4_0_init_microcode from sw_init to early_init, to load firmware and
>> init fw_version before set_ring/buffer/vm_pte_funcs use it.
>>
>> Change-Id: Ife5d4659d28bc2a7012b48947b27e929749d87c1
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 46 +
>> -
>>   1 file changed, 30 insertions(+), 16 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> index 4d873fd3242c..0a3b68dd49a0 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
>> @@ -1447,23 +1447,44 @@ static void
>> sdma_v4_0_ring_emit_reg_wait(struct amdgpu_ring *ring, uint32_t reg,
>>  sdma_v4_0_wait_reg_mem(ring, 0, 0, reg, 0, val, mask, 10);  }
>>
>> +static bool sdma_v4_0_fw_support_paging_queue(struct amdgpu_device
>> +*adev) {
>> +uint fw_version = adev->sdma.instance[0].fw_version;
>> +
>> +switch (adev->asic_type) {
>> +case CHIP_VEGA10:
>> +return fw_version >= 430;
>> +case CHIP_VEGA12:
>> +return fw_version >= 31;
>> +case CHIP_VEGA20:
>> +return fw_version >= 115;
>> +default:
>> +return false;
>> +}
>> +}
>> +
>>   static int sdma_v4_0_early_init(void *handle)  {
>>  struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>> +int r;
>>
>> -if (adev->asic_type == CHIP_RAVEN) {
>> +if (adev->asic_type == CHIP_RAVEN)
>>  adev->sdma.num_instances = 1;
>> -adev->sdma.has_page_queue = false;
>> -} else {
>> +else
>>  adev->sdma.num_instances = 2;
>> -/* TODO: Page queue breaks driver reload under SRIOV */
>> -if ((adev->asic_type == CHIP_VEGA10) &&
>> amdgpu_sriov_vf((adev)))
>> -adev->sdma.has_page_queue = false;
>> -else if (adev->asic_type != CHIP_VEGA20 &&
>> -adev->asic_type != CHIP_VEGA12)
>> -adev->sdma.has_page_queue = true;
>> +
>> +r = sdma_v4_0_init_microcode(adev);
>> +if (r) {
>> +DRM_ERROR("Failed to load sdma firmware!\n");
>> +return r;
> I think this should be ok.  As long as you've verified that 
> sdam_v4_0_init_microcode() doesn't depend on any other init from another 
> module like psp.  I took a quick look at the code and it seems like we should 
> be ok.
> Acked-by: Alex Deucher 
sdma_v4_0_init_microcode() reads fw binary file and setup fw version etc 
data structure. It doesn't depend on other init so it is fine to move it 
from sw_init() to early_init(). sdma_v4_0_load_microcode() will start fw 
from sdma_v4_0_start() which depends on psp, gmc etc. This happens after 
early_init()->sw_init()->hw_init().

Philip
>
>>  }
>>
>> +/* TODO: Page queue breaks driver reload under SRIOV */
>> +if ((adev->asic_type == CHIP_VEGA10) &&
>> amdgpu_sriov_vf((adev)))
>> +adev->sdma.has_page_queue = false;
>> +else if (sdma_v4_0_fw_support_paging_queue(adev))
>> +adev->sdma.has_page_queue = true;
>> +
>>  sdma_v4_0_set_ring_funcs(adev);
>>  sdma_v4_0_set_buffer_funcs(adev);
>>  sdma_v4_0_set_vm_pte_funcs(adev);
>> @@ -1472,7 +1493,6 @@ static int sdma_v4_0_early_init(void *handle)
>>  return 0;
>>   }
>>
>> -
>>   static int sdma_v4_0_sw_init(void *handle)  {
>>  struct amdgpu_ring *ring;
>> @@ -1491,12 +1511,6 @@ static int sdma_v4_0_sw_init(void *handle)
>>  if (r)
>>  return r;
>>
>> -r = sdma_v4_0_init_microcode(adev);
>> -if (r) {
>> -DRM_ERROR("Failed to load sdma firmware!\n");
>> -return r;
>> -}
>> -
>>  for (i = 0; i < adev->sdma.num_instances; i++) {
>>  ring = &adev->sdma.instance[i].ring;
>>  ring->ring_obj = NULL;
>> --
>> 2.17.1
>>
>> ___
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: disable page queue support for Vega12

[Yang, Philip]

Reviewed-by: Philip Yang 


On 2018-11-21 9:54 a.m., Alex Deucher wrote:
> Keep it disabled until we confirm it's ready.
>
> Signed-off-by: Alex Deucher 
> ---
>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c 
> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
> index e6cb2c399957..a973dea7b242 100644
> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
> @@ -1455,9 +1455,10 @@ static bool sdma_v4_0_fw_support_paging_queue(struct 
> amdgpu_device *adev)
>   case CHIP_VEGA10:
>   return fw_version >= 430;
>   case CHIP_VEGA12:
> - return fw_version >= 31;
> + /*return fw_version >= 31;*/
> + return false;
>   case CHIP_VEGA20:
> - //return fw_version >= 115;
> + /*return fw_version >= 115;*/
>   return false;
>   default:
>   return false;

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/2] drm/amdgpu: use HMM mirror callback to replace mmu notifier v5

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

The depdent HMM patchsets from Jérôme Glisse are all merged into 4.20.0
kernel now.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 124 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 57 insertions(+), 77 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index f76bcb9c45e4..05ca6dc381e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index e55508b39496..56595b3d90d2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -87,6 +86,9 @@ struct amdgpu_mn {
struct rb_root_cached   objects;
struct mutexread_lock;
atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +105,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -237,21 +237,19 @@ static void amdgpu_mn_invalidate_node(struct 
amdgpu_mn_node *node,
 /**
  * amdgpu_mn_invalidate_range_start_gfx - callback to notify about mm change
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
- * @start: start of updated range
- * @end: end of updated range
+ * @mirror: the hmm_mirror (mm) is about 

[PATCH 2/2] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v2

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

To avoid circular lock dependency, no nested locking between mmap_sem
and bo::reserve. The locking order is:
bo::reserve -> amdgpu_mn_lock(p->mn)

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

v2:
* Remove nested locking between mmap_sem and bo::reserve
* Change locking order to bo::reserve -> amdgpu_mn_lock()
* Use dynamic allocation to replace VLA in kernel stack

Change-Id: Iffd5f855cc9ce402cdfca167f68f83fe39ac56f9
Signed-off-by: Philip Yang 
---
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 101 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 188 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  34 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   7 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 164 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |  67 ---
 11 files changed, 312 insertions(+), 272 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index f3129b912714..5ce6ba24fc72 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -614,8 +614,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 free_out:
kvfree(mem->user_pages);
mem->user_pages = NULL;
@@ -677,7 +676,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -741,7 +739,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.shared = true;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1332,9 +1329,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
/* Free user pages if necessary */
if (mem->user_pages) {
pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
kvfree(mem->user_pages);
}
 
@@ -1761,8 +1755,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
   __func__);
return -ENOMEM;
}
-   } else if (mem->user_pages[0]) {
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
}
 
/* Get updated user pages */
@@ -1778,12 +1770,6 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
 * stalled user mode queues.
 */
}
-
-   /* Mark the BO as valid unless it was invalidated
-* again concurrently
-*/
-   if (atomic_cmpxchg(&mem->invalid, invalid, 0) != invalid)
-   return -EAGAIN;
}
 
return 0;
@@ -1876,14 +1862,10 @@ static int validate_invalid_user_pages(struct 
amdkfd_process_info *process_info)
}
 
/* Validate succeeded, now the BO owns the pages, free
-* our copy of the pointer array. Put this BO back on
-* the userptr_valid_list. If we need to revalidate
-* it, we need to start from scratch.
+* our copy of the pointer array.
 */
kvfree(mem->user_pages);
mem->user_pages = NULL;
-   list_move_tail(&mem->validate_list.head,
-   

Re: [PATCH 1/2] drm/amdgpu: use HMM mirror callback to replace mmu notifier v5

[Yang, Philip]

On 2018-12-04 4:06 a.m., Christian König wrote:
> Am 03.12.18 um 21:19 schrieb Yang, Philip:
>> Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
>> callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
>> DRM_AMDGPU_USERPTR Kconfig.
>>
>> It supports both KFD userptr and gfx userptr paths.
>>
>> The depdent HMM patchsets from Jérôme Glisse are all merged into 4.20.0
>> kernel now.
>>
>> Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
>>   drivers/gpu/drm/amd/amdgpu/Makefile    |   2 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 124 +++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
>>   4 files changed, 57 insertions(+), 77 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
>> b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> index 9221e5489069..960a63355705 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Kconfig
>> +++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> @@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
>>   config DRM_AMDGPU_USERPTR
>>   bool "Always enable userptr write support"
>>   depends on DRM_AMDGPU
>> -    select MMU_NOTIFIER
>> +    select HMM_MIRROR
>>   help
>> -  This option selects CONFIG_MMU_NOTIFIER if it isn't already
>> -  selected to enabled full userptr support.
>> +  This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
>> +  isn't already selected to enabled full userptr support.
>>     config DRM_AMDGPU_GART_DEBUGFS
>>   bool "Allow GART access through debugfs"
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
>> b/drivers/gpu/drm/amd/amdgpu/Makefile
>> index f76bcb9c45e4..05ca6dc381e0 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Makefile
>> +++ b/drivers/gpu/drm/amd/amdgpu/Makefile
>> @@ -172,7 +172,7 @@ endif
>>   amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
>>   amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
>>   amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
>> -amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
>> +amdgpu-$(CONFIG_HMM) += amdgpu_mn.o
>
> This probably need to be CONFIG_HMM_MIRROR.
Yes, CONFIG_HMM_MIRROR selects CONFIG_HMM, but the reverse is not true.
>
>>     include $(FULL_AMD_PATH)/powerplay/Makefile
>>   diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> index e55508b39496..56595b3d90d2 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> @@ -45,7 +45,7 @@
>>     #include 
>>   #include 
>> -#include 
>> +#include 
>>   #include 
>>   #include 
>>   #include 
>> @@ -58,7 +58,6 @@
>>    *
>>    * @adev: amdgpu device pointer
>>    * @mm: process address space
>> - * @mn: MMU notifier structure
>>    * @type: type of MMU notifier
>>    * @work: destruction work item
>>    * @node: hash table node to find structure by adev and mn
>> @@ -66,6 +65,7 @@
>>    * @objects: interval tree containing amdgpu_mn_nodes
>>    * @read_lock: mutex for recursive locking of @lock
>>    * @recursion: depth of recursion
>> + * @mirror: HMM mirror function support
>>    *
>>    * Data for each amdgpu device and process address space.
>>    */
>> @@ -73,7 +73,6 @@ struct amdgpu_mn {
>>   /* constant after initialisation */
>>   struct amdgpu_device    *adev;
>>   struct mm_struct    *mm;
>> -    struct mmu_notifier    mn;
>>   enum amdgpu_mn_type    type;
>>     /* only used on destruction */
>> @@ -87,6 +86,9 @@ struct amdgpu_mn {
>>   struct rb_root_cached    objects;
>>   struct mutex    read_lock;
>>   atomic_t    recursion;
>> +
>> +    /* HMM mirror */
>> +    struct hmm_mirror    mirror;
>>   };
>>     /**
>> @@ -103,7 +105,7 @@ struct amdgpu_mn_node {
>>   };
>>     /**
>> - * amdgpu_mn_destroy - destroy the MMU notifier
>> + * amdgpu_mn_destroy - destroy the HMM mirror
>>    *
>>    * @work: previously sheduled work item
>>    *
>> @@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct 
>> work_struct *work)
>>   }
>>   up_write(&amn->lock);
>>   mutex_unlock(&adev->mn_lock);
>> -    mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
>> +
>> +    hmm_mirror_unregister(&amn->mirror);
>>   kfre

Re: [PATCH 2/2] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v2

[Yang, Philip]

On 2018-12-03 8:52 p.m., Kuehling, Felix wrote:
> See comments inline. I didn't review the amdgpu_cs and amdgpu_gem parts
> as I don't know them very well.
>
> On 2018-12-03 3:19 p.m., Yang, Philip wrote:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> To avoid circular lock dependency, no nested locking between mmap_sem
>> and bo::reserve. The locking order is:
>> bo::reserve -> amdgpu_mn_lock(p->mn)
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> v2:
>> * Remove nested locking between mmap_sem and bo::reserve
>> * Change locking order to bo::reserve -> amdgpu_mn_lock()
>> * Use dynamic allocation to replace VLA in kernel stack
>>
>> Change-Id: Iffd5f855cc9ce402cdfca167f68f83fe39ac56f9
>> Signed-off-by: Philip Yang 
>> ---
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 101 --
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 188 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  34 +++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   7 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 164 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
>>   .../drm/amd/amdkfd/kfd_device_queue_manager.c |  67 ---
>>   11 files changed, 312 insertions(+), 272 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index f3129b912714..5ce6ba24fc72 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -614,8 +614,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  amdgpu_bo_unreserve(bo);
>>   
>>   release_out:
>> -if (ret)
>> -release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> +amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>>   free_out:
>>  kvfree(mem->user_pages);
>>  mem->user_pages = NULL;
>> @@ -677,7 +676,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
>>  ctx->kfd_bo.priority = 0;
>>  ctx->kfd_bo.tv.bo = &bo->tbo;
>>  ctx->kfd_bo.tv.shared = true;
>> -ctx->kfd_bo.user_pages = NULL;
>>  list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>>   
>>  amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
>> @@ -741,7 +739,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
>>  ctx->kfd_bo.priority = 0;
>>  ctx->kfd_bo.tv.bo = &bo->tbo;
>>  ctx->kfd_bo.tv.shared = true;
>> -ctx->kfd_bo.user_pages = NULL;
>>  list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>>   
>>  i = 0;
>> @@ -1332,9 +1329,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
>>  /* Free user pages if necessary */
>>  if (mem->user_pages) {
>>  pr_debug("%s: Freeing user_pages array\n", __func__);
>> -if (mem->user_pages[0])
>> -release_pages(mem->user_pages,
>> -mem->bo->tbo.ttm->num_pages);
>>  kvfree(mem->user_pages);
>>  }
>>   
>> @@ -1761,8 +1755,6 @@ static int update_invalid_user_pages(struct 
>> amdkfd_process_info *process_info,
>> __func__);
>>  return -ENOMEM;
>>  }
>> -} else if (mem->user_pages[0]) {
>> -release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>>  }
>>   
>>

[PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v6

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

The depdent HMM patchset from Jérôme Glisse are all merged into 4.20.0
kernel now.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 122 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 55 insertions(+), 77 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index f76bcb9c45e4..675efc850ff4 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index e55508b39496..5d518d2bb9be 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -87,6 +86,9 @@ struct amdgpu_mn {
struct rb_root_cached   objects;
struct mutexread_lock;
atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +105,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -237,21 +237,19 @@ static void amdgpu_mn_invalidate_node(struct 
amdgpu_mn_node *node,
 /**
  * amdgpu_mn_invalidate_range_start_gfx - callback to notify about mm change
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
- * @start: start of updated range
- * @end: end of updated range
+ * @mirror: the hmm_mirror (mm) is 

[PATCH 2/3] drm/amdkfd: avoid HMM change cause circular lock dependency

[Yang, Philip]

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Change-Id: I2ec09a47571f6b4c8eaef93f22c0a600f5f70153
Signed-off-by: Philip Yang 
---
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 67 ++-
 1 file changed, 36 insertions(+), 31 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 8372556b52eb..fe120cc0930c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1158,6 +1158,33 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
 
retval = 0;
 
+   /* Do init_mqd before dqm_lock(dqm) to avoid circular locking order:
+* lock(dqm) -> bo::reserve
+*/
+   mqd_mgr = dqm->ops.get_mqd_manager(dqm,
+   get_mqd_type_from_queue_type(q->properties.type));
+
+   if (!mqd_mgr) {
+   retval = -ENOMEM;
+   goto out;
+   }
+
+   /*
+* Eviction state logic: we only mark active queues as evicted
+* to avoid the overhead of restoring inactive queues later
+*/
+   if (qpd->evicted)
+   q->properties.is_evicted = (q->properties.queue_size > 0 &&
+   q->properties.queue_percent > 0 &&
+   q->properties.queue_address != 0);
+   dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
+   q->properties.tba_addr = qpd->tba_addr;
+   q->properties.tma_addr = qpd->tma_addr;
+   retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
+   &q->gart_mqd_addr, &q->properties);
+   if (retval)
+   goto out;
+
dqm_lock(dqm);
 
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
@@ -1181,30 +1208,6 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
if (retval)
goto out_deallocate_sdma_queue;
 
-   mqd_mgr = dqm->ops.get_mqd_manager(dqm,
-   get_mqd_type_from_queue_type(q->properties.type));
-
-   if (!mqd_mgr) {
-   retval = -ENOMEM;
-   goto out_deallocate_doorbell;
-   }
-   /*
-* Eviction state logic: we only mark active queues as evicted
-* to avoid the overhead of restoring inactive queues later
-*/
-   if (qpd->evicted)
-   q->properties.is_evicted = (q->properties.queue_size > 0 &&
-   q->properties.queue_percent > 0 &&
-   q->properties.queue_address != 0);
-
-   dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
-
-   q->properties.tba_addr = qpd->tba_addr;
-   q->properties.tma_addr = qpd->tma_addr;
-   retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
-   &q->gart_mqd_addr, &q->properties);
-   if (retval)
-   goto out_deallocate_doorbell;
 
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
@@ -1228,14 +1231,12 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
dqm_unlock(dqm);
return retval;
 
-out_deallocate_doorbell:
-   deallocate_doorbell(qpd, q);
 out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
deallocate_sdma_queue(dqm, q->sdma_id);
 out_unlock:
dqm_unlock(dqm);
-
+out:
return retval;
 }
 
@@ -1398,8 +1399,6 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = true;
}
 
-   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
-
/*
 * Unconditionally decrement this counter, regardless of the queue's
 * type
@@ -1410,6 +1409,9 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
 
dqm_unlock(dqm);
 
+   /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
+   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+
return retval;
 
 failed:
@@ -1631,7 +1633,11 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = false;
}
 
-   /* lastly, free mqd resources */
+   dqm_unlock(dqm);
+
+   /* Lastly, free mqd resources.
+* Do uninit_mqd() after dqm_unlock to avoid circular locking.
+*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
@@ -1645,7 +1651,6 @@ static int process_term

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v3

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: Id2d3130378b44a774e0d77156102a20a203b5036
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  81 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 188 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  34 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   7 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 160 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
 11 files changed, 206 insertions(+), 289 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index bcf587b4ba98..b492dd9e541a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -62,7 +62,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index b29ef088fa14..75833b40507b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -580,28 +580,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -614,11 +598,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -677,7 +657,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -741,7 +720,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1329,15 +1307,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_co

Re: [PATCH] drm/amdgpu: bypass RLC init under sriov for Tonga

[Yang, Philip]

I got this compilation error message after I rebase this morning, do I miss 
anything?

/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c: 
In function ‘gfx_v8_0_rlc_resume’:
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c:4071:6:
 error: implicit declaration of function ‘amdgpu_sriov’; did you mean 
‘amdgpu_sriov_vf’? [-Werror=implicit-function-declaration]
  if (amdgpu_sriov(adev)) {
  ^~~~
  amdgpu_sriov_vf
cc1: some warnings being treated as errors
/home/yangp/git/compute_staging/kernel/scripts/Makefile.build:293: recipe for 
target 'drivers/gpu/drm/amd/amdgpu/gfx_v8_0.o' failed
make[10]: *** [drivers/gpu/drm/amd/amdgpu/gfx_v8_0.o] Error 1

Philip

On 2018-12-07 10:33 a.m., Deucher, Alexander wrote:

Acked-by: Alex Deucher 



From: amd-gfx 

 on behalf of Tiecheng Zhou 

Sent: Thursday, December 6, 2018 9:11:49 PM
To: amd-gfx@lists.freedesktop.org
Cc: Zhou, Tiecheng
Subject: [PATCH] drm/amdgpu: bypass RLC init under sriov for Tonga

SWDEV-173384: vm-mix reboot (4 VMs) fail on Tonga under sriov

Phenomena: there is compute_1.3.1 ring test fail on one VM
   when it starts to do hw_init after it is rebooted

Root cause: RLC will go wrong in soft_reset under sriov

Workaround: init RLC csb, and skip RLC stop, reset, start
this is because GIM has already done
full initialization on RLC

refer to 'commit cfee05bc9057 ("drm/amdgpu:bypass RLC init for SRIOV")'
 and 'commit f840cc5f8447 ("drm/amdgpu/sriov:init csb for gfxv9")'

Signed-off-by: Tiecehng Zhou 

---
 drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c 
b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 1454fc3..a9c853a 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -4068,6 +4068,11 @@ static void gfx_v8_0_rlc_start(struct amdgpu_device 
*adev)

 static int gfx_v8_0_rlc_resume(struct amdgpu_device *adev)
 {
+   if (amdgpu_sriov(adev)) {
+   gfx_v8_0_init_csb(adev);
+   return 0;
+   }
+
 adev->gfx.rlc.funcs->stop(adev);
 adev->gfx.rlc.funcs->reset(adev);
 gfx_v8_0_init_pg(adev);
--
2.7.4

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx



___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx


___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: bypass RLC init under sriov for Tonga

[Yang, Philip]

Never mind, just saw the patch to fix the typo.

On 2018-12-10 1:07 p.m., Yang, Philip wrote:
I got this compilation error message after I rebase this morning, do I miss 
anything?

/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c: 
In function ‘gfx_v8_0_rlc_resume’:
/home/yangp/git/compute_staging/kernel/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c:4071:6:
 error: implicit declaration of function ‘amdgpu_sriov’; did you mean 
‘amdgpu_sriov_vf’? [-Werror=implicit-function-declaration]
  if (amdgpu_sriov(adev)) {
  ^~~~
  amdgpu_sriov_vf
cc1: some warnings being treated as errors
/home/yangp/git/compute_staging/kernel/scripts/Makefile.build:293: recipe for 
target 'drivers/gpu/drm/amd/amdgpu/gfx_v8_0.o' failed
make[10]: *** [drivers/gpu/drm/amd/amdgpu/gfx_v8_0.o] Error 1

Philip

On 2018-12-07 10:33 a.m., Deucher, Alexander wrote:

Acked-by: Alex Deucher 
<mailto:alexander.deuc...@amd.com>


From: amd-gfx 
<mailto:amd-gfx-boun...@lists.freedesktop.org>
 on behalf of Tiecheng Zhou 
<mailto:tiecheng.z...@amd.com>
Sent: Thursday, December 6, 2018 9:11:49 PM
To: amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>
Cc: Zhou, Tiecheng
Subject: [PATCH] drm/amdgpu: bypass RLC init under sriov for Tonga

SWDEV-173384: vm-mix reboot (4 VMs) fail on Tonga under sriov

Phenomena: there is compute_1.3.1 ring test fail on one VM
   when it starts to do hw_init after it is rebooted

Root cause: RLC will go wrong in soft_reset under sriov

Workaround: init RLC csb, and skip RLC stop, reset, start
this is because GIM has already done
full initialization on RLC

refer to 'commit cfee05bc9057 ("drm/amdgpu:bypass RLC init for SRIOV")'
 and 'commit f840cc5f8447 ("drm/amdgpu/sriov:init csb for gfxv9")'

Signed-off-by: Tiecehng Zhou 
<mailto:tiecheng.z...@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c 
b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 1454fc3..a9c853a 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -4068,6 +4068,11 @@ static void gfx_v8_0_rlc_start(struct amdgpu_device 
*adev)

 static int gfx_v8_0_rlc_resume(struct amdgpu_device *adev)
 {
+   if (amdgpu_sriov(adev)) {
+   gfx_v8_0_init_csb(adev);
+   return 0;
+   }
+
 adev->gfx.rlc.funcs->stop(adev);
 adev->gfx.rlc.funcs->reset(adev);
 gfx_v8_0_init_pg(adev);
--
2.7.4

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx



___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx





___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org<mailto:amd-gfx@lists.freedesktop.org>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx


___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v3

[Yang, Philip]

On 2018-12-10 7:12 p.m., Kuehling, Felix wrote:
> This is a nice improvement from the last version. I still see some
> potential problems. See inline ...
>
> I'm skipping over the CS and GEM parts. I hope Christian can review
> those parts.
>
> On 2018-12-06 4:02 p.m., Yang, Philip wrote:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: Id2d3130378b44a774e0d77156102a20a203b5036
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  81 ++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 188 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  34 +++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   7 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 160 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
>>   11 files changed, 206 insertions(+), 289 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index bcf587b4ba98..b492dd9e541a 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -62,7 +62,6 @@ struct kgd_mem {
>>   
>>  atomic_t invalid;
>>  struct amdkfd_process_info *process_info;
>> -struct page **user_pages;
>>   
>>  struct amdgpu_sync sync;
>>   
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index b29ef088fa14..75833b40507b 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -580,28 +580,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  goto out;
>>  }
>>   
>> -/* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -if (!mem->user_pages) {
>> -pr_err("%s: Failed to allocate pages array\n", __func__);
>> -ret = -ENOMEM;
>> -goto unregister_out;
>> -}
>> -
>> -ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>  if (ret) {
>>  pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
>> -goto free_out;
>> +goto unregister_out;
>>  }
>>   
>> -amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
>> -
>>  ret = amdgpu_bo_reserve(bo, true);
>>  if (ret) {
>>  pr_err("%s: Failed to reserve BO\n", __func__);
>> @@ -614,11 +598,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  amdgpu_bo_unreserve(bo);
>>   
>>   release_out:
>> -if (ret)
>> -release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> -free_out:
>> -kvfree(mem->user_pages);
>> -mem->user_pages = NULL;
>> +amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>>   unregister_out:
>>  if (ret)
>>  amdgpu_mn

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v4

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 185 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  27 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 166 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
 11 files changed, 208 insertions(+), 294 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 70429f7aa9a8..717791d4fa45 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -62,7 +62,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index be1ab43473c6..2897793600f7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_c

Re: [PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v6

[Yang, Philip]

I thought to change the filename to amdgpu_hmm.c/h, then change the data 
structure,
variables, function name from amdgpu_mn_* to amdgpu_hmm_*, seems too 
many changes,
so I gave up that change.

Philip

On 2018-12-07 7:00 a.m., Zhou, David(ChunMing) wrote:
> Even you should rename amdgpu_mn.c/h to amdgpu_hmm.c/h.
>
> -David
>
>> -Original Message-
>> From: amd-gfx  On Behalf Of Yang,
>> Philip
>> Sent: Friday, December 07, 2018 5:03 AM
>> To: amd-gfx@lists.freedesktop.org
>> Cc: Yang, Philip 
>> Subject: [PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu
>> notifier v6
>>
>> Replace our MMU notifier with
>> hmm_mirror_ops.sync_cpu_device_pagetables
>> callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a
>> dependency in DRM_AMDGPU_USERPTR Kconfig.
>>
>> It supports both KFD userptr and gfx userptr paths.
>>
>> The depdent HMM patchset from Jérôme Glisse are all merged into 4.20.0
>> kernel now.
>>
>> Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
>>   drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 122 ++-
>> --
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
>>   4 files changed, 55 insertions(+), 77 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig
>> b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> index 9221e5489069..960a63355705 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Kconfig
>> +++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> @@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK  config
>> DRM_AMDGPU_USERPTR
>>  bool "Always enable userptr write support"
>>  depends on DRM_AMDGPU
>> -select MMU_NOTIFIER
>> +select HMM_MIRROR
>>  help
>> -  This option selects CONFIG_MMU_NOTIFIER if it isn't already
>> -  selected to enabled full userptr support.
>> +  This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
>> +  isn't already selected to enabled full userptr support.
>>
>>   config DRM_AMDGPU_GART_DEBUGFS
>>  bool "Allow GART access through debugfs"
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile
>> b/drivers/gpu/drm/amd/amdgpu/Makefile
>> index f76bcb9c45e4..675efc850ff4 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Makefile
>> +++ b/drivers/gpu/drm/amd/amdgpu/Makefile
>> @@ -172,7 +172,7 @@ endif
>>   amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
>>   amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
>>   amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
>> -amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
>> +amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
>>
>>   include $(FULL_AMD_PATH)/powerplay/Makefile
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> index e55508b39496..5d518d2bb9be 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> @@ -45,7 +45,7 @@
>>
>>   #include 
>>   #include 
>> -#include 
>> +#include 
>>   #include 
>>   #include 
>>   #include 
>> @@ -58,7 +58,6 @@
>>*
>>* @adev: amdgpu device pointer
>>* @mm: process address space
>> - * @mn: MMU notifier structure
>>* @type: type of MMU notifier
>>* @work: destruction work item
>>* @node: hash table node to find structure by adev and mn @@ -66,6 +65,7
>> @@
>>* @objects: interval tree containing amdgpu_mn_nodes
>>* @read_lock: mutex for recursive locking of @lock
>>* @recursion: depth of recursion
>> + * @mirror: HMM mirror function support
>>*
>>* Data for each amdgpu device and process address space.
>>*/
>> @@ -73,7 +73,6 @@ struct amdgpu_mn {
>>  /* constant after initialisation */
>>  struct amdgpu_device*adev;
>>  struct mm_struct*mm;
>> -struct mmu_notifier mn;
>>  enum amdgpu_mn_type type;
>>
>>  /* only used on destruction */
>> @@ -87,6 +86,9 @@ struct amdgpu_mn {
>>  struct rb_root_cached   objects;
>>  struct mutexread_lock;
>>  atomic_trecursion;
>> +
>> +/* HMM mirror */
>> +struct hmm_mirror   mirror;
>>   };
>>
>>   /**
>> @@ -103,7 +105,7 @@ struct amdgpu_mn_node {  };
>>
>>   /**
>> - * amdgpu_mn_destroy - destroy the MMU notifier
>> 

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v4

[Yang, Philip]

Hi Felix,

Thanks, I will submit another patch to fix those KFD part issues.

Philip

On 2018-12-14 2:54 p.m., Kuehling, Felix wrote:
> Looks good for the KFD part. One nit-pick inline. And two more
> suggestions about amdgpu_ttm.c to make our handling of
> get_user_pages_done more robust.
> 
> Christian, would you review the CS and GEM parts? And maybe take a look
> you see nothing wrong with the amdgpu_ttm changes either.
> 
> On 2018-12-13 4:01 p.m., Yang, Philip wrote:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 185 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  27 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 166 ++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
>>   11 files changed, 208 insertions(+), 294 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index 70429f7aa9a8..717791d4fa45 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -62,7 +62,6 @@ struct kgd_mem {
>>   
>>  atomic_t invalid;
>>  struct amdkfd_process_info *process_info;
>> -struct page **user_pages;
>>   
>>  struct amdgpu_sync sync;
>>   
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index be1ab43473c6..2897793600f7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  goto out;
>>  }
>>   
>> -/* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -if (!mem->user_pages) {
>> -pr_err("%s: Failed to allocate pages array\n", __func__);
>> -ret = -ENOMEM;
>> -goto unregister_out;
>> -}
>> -
>> -ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>  if (ret) {
>>  pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
>> -goto free_out;
>> +goto unregister_out;
>>  }
>>   
>> -amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
>> -
>>  ret = amdgpu_bo_reserve(bo, true);
>>  if (ret) {
>>  pr_err("%s: Failed to reserve BO\n", __func__);
>> @@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  amdgpu_bo_unreserve(bo);
>>   
>>   release_out:
>> -if (ret)
>> -release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> -free_out:
>> -kvfree(mem->user_pages);
>> -mem-&g

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v5

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 185 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 168 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
 11 files changed, 209 insertions(+), 293 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 70429f7aa9a8..717791d4fa45 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -62,7 +62,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index be1ab43473c6..2897793600f7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_c

Re: [PATCH 1/2] drm/amdgpu: Add per device sdma_doorbell_range field

[Yang, Philip]

On 2018-12-17 9:12 p.m., Zeng, Oak wrote:
> Different ASIC has different sdma doorbell range. Add
> a per device sdma_doorbell_range field and initialize
> it.
> 
> Change-Id: Idd980db1a72cfb373e24ac23ba3e48bb329ed4ad
> Signed-off-by: Oak Zeng 
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h | 2 ++
>   drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c | 1 +
>   drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c | 1 +
>   3 files changed, 4 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
> index 35a0c05..1cfec06 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
> @@ -72,6 +72,8 @@ struct amdgpu_doorbell_index {
>   } uvd_vce;
>   };
>   uint32_t max_assignment;
> + /* Per engine SDMA doorbell size in dword */
> + uint32_t sdma_doorbell_range;
>   };
>   
>   typedef enum _AMDGPU_DOORBELL_ASSIGNMENT
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c 
> b/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
> index b75d17b..4b5d60e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
> @@ -83,5 +83,6 @@ void vega10_doorbell_index_init(struct amdgpu_device *adev)
>   adev->doorbell_index.uvd_vce.vce_ring6_7 = 
> AMDGPU_DOORBELL64_VCE_RING6_7;
>   /* In unit of dword doorbell */
>   adev->doorbell_index.max_assignment = AMDGPU_DOORBELL64_MAX_ASSIGNMENT 
> << 1;
> + adev->doorbell_index.sdma_doorbell_range = 4;
Vega10 doorbell range was 2 dwords (one 64bit doorbell), change to 4 
dwords may not work under SRIOV
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c 
> b/drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c
> index 63c542c..53716c5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c
> @@ -86,5 +86,6 @@ void vega20_doorbell_index_init(struct amdgpu_device *adev)
>   adev->doorbell_index.uvd_vce.vce_ring4_5 = 
> AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5;
>   adev->doorbell_index.uvd_vce.vce_ring6_7 = 
> AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7;
>   adev->doorbell_index.max_assignment = 
> AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT << 1;
> + adev->doorbell_index.sdma_doorbell_range = 20;
>   }
>   
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 1/2] drm/amdgpu: Add per device sdma_doorbell_range field

[Yang, Philip]

Thanks Oak for the clarification for Vega10.

For Vega20, 8 user space sdma queues should set doorbell range to 16, 
maybe you add gfx and page queue doorbell together to become 20? But gfx 
and page queue doorbell is on kernel space, user space sdma queues 
doorbell mapping is on user space, they can overlap and share same 
doorbell range.

Regards,
Philip

On 2018-12-18 11:43 a.m., Zeng, Oak wrote:
> Thank you Dhiren. So we will also set it to 4 in Linux device driver.
> 
> BTW, patch 2 set the doorbell range for vega20 from 2 to 20. With previous 
> setting of 2, I believe only the first two user space sdma queues work. The 
> rest 6 queues won't work because the doorbell ringing won't be routed to sdma 
> engine due to previous bif doorbell range setting.
> 
> Regards,
> Oak
> 
> -Original Message-
> From: Partap Singh Rana, Dhirendra 
> Sent: Tuesday, December 18, 2018 11:34 AM
> To: Zeng, Oak ; Yang, Philip ; 
> amd-gfx@lists.freedesktop.org; Deng, Emily 
> Subject: RE: [PATCH 1/2] drm/amdgpu: Add per device sdma_doorbell_range field
> 
> Hi Oak,
> 
> Windows will set 4 dwords for both sdma0 and 1.
> 
> Thanks,
> Dhiren
> 
> -Original Message-
> From: Zeng, Oak
> Sent: Tuesday, December 18, 2018 10:52 AM
> To: Yang, Philip ; amd-gfx@lists.freedesktop.org; Partap 
> Singh Rana, Dhirendra ; Deng, Emily 
> 
> Subject: RE: [PATCH 1/2] drm/amdgpu: Add per device sdma_doorbell_range field
> 
> Thanks Philip.
> 
> For vega10, sdma doorbells are defined to be windows sriov compatible. Two 
> qwords doorbell are defined for each sdma engine. See amdgpu_doorbell.h line 
> 192. So program nbio sdma doorbell routing range to 4 (dwords) is correct. I 
> am not sure why previously the doorbell range was programmed to 2, even 
> though it is defined 4. @Partap Singh Rana, Dhirendra what value windows does 
> host driver program BIF_SDMA0_DOORBELL_RANGE.SIZE for vega10?
> 
> @Deng, Emily I saw previously you changed the vega10 doorbell assignment for 
> sriov. Did you copy the assignment from windows host driver? If yes, I assume 
> windows host should set the bif doorbell range to 4 dwords, according to the 
> assignment. Does changing bif doorbell range to 4 conflict with windows host 
> driver?
> 
> People might wonder why programming it to 2 also worked before. The reason 
> is, currently amdgpu only use one sdma ring per sdma engine. So it never used 
> the "HI_PRI" ring and doorbell. Kfd uses two sdma rings per sdma engine, but 
> for the second ring, it uses a doorbell with the same in page offset (0xf0) 
> but mapped to the second doorbell page of the process.
> 
> So my understanding is, setting the doorbell range to 4 (see patch 2 of this 
> series) is more consistent and no harm for current usage model. If we use it 
> in the future, it should be fine. But let's see what is the windows host 
> driver setting.
> 
> Regards,
> Oak
> 
> -Original Message-
> From: amd-gfx  On Behalf Of Yang, 
> Philip
> Sent: Tuesday, December 18, 2018 9:57 AM
> To: Zeng, Oak ; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 1/2] drm/amdgpu: Add per device sdma_doorbell_range field
> 
> 
> 
> On 2018-12-17 9:12 p.m., Zeng, Oak wrote:
>> Different ASIC has different sdma doorbell range. Add a per device
>> sdma_doorbell_range field and initialize it.
>>
>> Change-Id: Idd980db1a72cfb373e24ac23ba3e48bb329ed4ad
>> Signed-off-by: Oak Zeng 
>> ---
>>drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h | 2 ++
>>drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c | 1 +
>>drivers/gpu/drm/amd/amdgpu/vega20_reg_init.c | 1 +
>>3 files changed, 4 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
>> index 35a0c05..1cfec06 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h
>> @@ -72,6 +72,8 @@ struct amdgpu_doorbell_index {
>>  } uvd_vce;
>>  };
>>  uint32_t max_assignment;
>> +/* Per engine SDMA doorbell size in dword */
>> +uint32_t sdma_doorbell_range;
>>};
>>
>>typedef enum _AMDGPU_DOORBELL_ASSIGNMENT diff --git
>> a/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
>> b/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
>> index b75d17b..4b5d60e 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_reg_init.c
>> @@ -83,5 +83,6 @@ void vega10_doorbell_index_init(struct amdgpu_device *adev)
>>  adev->doorbell_index.uvd_vce.vce_ring6_7 = 
>> AMDGPU_DOORBEL

Re: [PATCH 2/2] drm/amdgpu: Fix sdma doorbell range setting

[Yang, Philip]

The series is Reviewed-by: Philip Yang 

On 2018-12-17 9:12 p.m., Zeng, Oak wrote:
> Different ASIC has different SDMA queues so different
> SDMA doorbell range. Introduce an extra parameter
> to sdma_doorbell_range function and set sdma doorbell
> range correctly.
> 
> Change-Id: I9b8d75b04f5a47ef1c6fd7cc1caaefd98dd2ff2b
> Signed-off-by: Oak Zeng 
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h| 2 +-
>   drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c | 4 ++--
>   drivers/gpu/drm/amd/amdgpu/nbio_v7_0.c | 4 ++--
>   drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c | 4 ++--
>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c | 3 ++-
>   5 files changed, 9 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index bcef6ea..5a7907b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -634,7 +634,7 @@ struct amdgpu_nbio_funcs {
>   void (*hdp_flush)(struct amdgpu_device *adev, struct amdgpu_ring *ring);
>   u32 (*get_memsize)(struct amdgpu_device *adev);
>   void (*sdma_doorbell_range)(struct amdgpu_device *adev, int instance,
> - bool use_doorbell, int doorbell_index);
> + bool use_doorbell, int doorbell_index, int 
> doorbell_size);
>   void (*enable_doorbell_aperture)(struct amdgpu_device *adev,
>bool enable);
>   void (*enable_doorbell_selfring_aperture)(struct amdgpu_device *adev,
> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c 
> b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
> index 6f9c549..1eeb318 100644
> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
> @@ -71,7 +71,7 @@ static u32 nbio_v6_1_get_memsize(struct amdgpu_device *adev)
>   }
>   
>   static void nbio_v6_1_sdma_doorbell_range(struct amdgpu_device *adev, int 
> instance,
> -   bool use_doorbell, int doorbell_index)
> + bool use_doorbell, int doorbell_index, int 
> doorbell_size)
>   {
>   u32 reg = instance == 0 ? SOC15_REG_OFFSET(NBIO, 0, 
> mmBIF_SDMA0_DOORBELL_RANGE) :
>   SOC15_REG_OFFSET(NBIO, 0, mmBIF_SDMA1_DOORBELL_RANGE);
> @@ -80,7 +80,7 @@ static void nbio_v6_1_sdma_doorbell_range(struct 
> amdgpu_device *adev, int instan
>   
>   if (use_doorbell) {
>   doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
> - doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
> + doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, doorbell_size);
>   } else
>   doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_0.c 
> b/drivers/gpu/drm/amd/amdgpu/nbio_v7_0.c
> index df34dc7..d80413d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_0.c
> @@ -69,7 +69,7 @@ static u32 nbio_v7_0_get_memsize(struct amdgpu_device *adev)
>   }
>   
>   static void nbio_v7_0_sdma_doorbell_range(struct amdgpu_device *adev, int 
> instance,
> -   bool use_doorbell, int doorbell_index)
> + bool use_doorbell, int doorbell_index, int 
> doorbell_size)
>   {
>   u32 reg = instance == 0 ? SOC15_REG_OFFSET(NBIO, 0, 
> mmBIF_SDMA0_DOORBELL_RANGE) :
>   SOC15_REG_OFFSET(NBIO, 0, mmBIF_SDMA1_DOORBELL_RANGE);
> @@ -78,7 +78,7 @@ static void nbio_v7_0_sdma_doorbell_range(struct 
> amdgpu_device *adev, int instan
>   
>   if (use_doorbell) {
>   doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, OFFSET, doorbell_index);
> - doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, 2);
> + doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, doorbell_size);
>   } else
>   doorbell_range = REG_SET_FIELD(doorbell_range, 
> BIF_SDMA0_DOORBELL_RANGE, SIZE, 0);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c 
> b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
> index f8cee95..27976b3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
> @@ -67,7 +67,7 @@ static u32 nbio_v7_4_get_memsize(struct amdgpu_device *adev)
>   }
>   
>   static void nbio_v7_4_sdma_doorbell_range(struct amdgpu_device *adev, int 
> instance,
> -   bool use_doorbell, int doorbell_index)
> + bool use_doorbell, int doorbell_index, int 
> doorbell_size)
>   {
>   u32 reg = instance == 0 ? SOC15_REG_OFFSET(NBIO, 0, 
> mmBIF_SDMA0_DOORBELL_RANGE) :
>   SOC15_REG_OFFSET(NBIO, 0, mmBIF_SDMA1_DOORBELL_RANGE);
> @@ -76,7 +76,7 @@ static void nbio_v7_4_sdma_doorbell_rang

Re: [PATCH 2/2] drm/amdgpu/nbio7.4: add hw bug workaround for vega20

[Yang, Philip]

This series are verified on 4 Vega20 by: Philip Yang 

On 2018-12-20 10:39 a.m., Kasiviswanathan, Harish wrote:
> This patch set Reviewed-by: Harish Kasiviswanathan
> 
> 
> On 2018-12-19 6:09 p.m., Alex Deucher wrote:
>> Configure PCIE_CI_CNTL to work around a hw bug that affects
>> some multi-GPU compute workloads.
>>
>> Signed-off-by: Alex Deucher 
>> ---
>>drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c | 7 +++
>>1 file changed, 7 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c 
>> b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> index f8cee95d61cc..4cd31a276dcd 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
>> @@ -31,6 +31,7 @@
>>
>>#define smnCPM_CONTROL
>>   0x11180460
>>#define smnPCIE_CNTL2 
>>   0x11180070
>> +#define smnPCIE_CI_CNTL 
>> 0x11180080
>>
>>static u32 nbio_v7_4_get_rev_id(struct amdgpu_device *adev)
>>{
>> @@ -222,7 +223,13 @@ static void nbio_v7_4_detect_hw_virt(struct 
>> amdgpu_device *adev)
>>
>>static void nbio_v7_4_init_registers(struct amdgpu_device *adev)
>>{
>> +uint32_t def, data;
>> +
>> +def = data = RREG32_PCIE(smnPCIE_CI_CNTL);
>> +data = REG_SET_FIELD(data, PCIE_CI_CNTL, CI_SLV_ORDERING_DIS, 1);
>>
>> +if (def != data)
>> +WREG32_PCIE(smnPCIE_CI_CNTL, data);
>>}
>>
>>const struct amdgpu_nbio_funcs nbio_v7_4_funcs = {
> ___
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v5

[Yang, Philip]

Hi Christian,

May you help review the CS parts related changes? I tested it using 
libdrm Userptr Test under X. Do you know other test applications which 
can stress the CS userptr path?

Thanks,
Philip

On 2018-12-14 4:25 p.m., Kuehling, Felix wrote:
> Except for the GEM and CS parts, the series is Reviewed-by: Felix
> Kuehling 
> 
> Regards,
>    Felix
> 
> On 2018-12-14 4:10 p.m., Yang, Philip wrote:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 185 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 168 ++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   1 -
>>   11 files changed, 209 insertions(+), 293 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index 70429f7aa9a8..717791d4fa45 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -62,7 +62,6 @@ struct kgd_mem {
>>   
>>  atomic_t invalid;
>>  struct amdkfd_process_info *process_info;
>> -struct page **user_pages;
>>   
>>  struct amdgpu_sync sync;
>>   
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index be1ab43473c6..2897793600f7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  goto out;
>>  }
>>   
>> -/* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -if (!mem->user_pages) {
>> -pr_err("%s: Failed to allocate pages array\n", __func__);
>> -ret = -ENOMEM;
>> -goto unregister_out;
>> -}
>> -
>> -ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>  if (ret) {
>>  pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
>> -goto free_out;
>> +goto unregister_out;
>>  }
>>   
>> -amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
>> -
>>  ret = amdgpu_bo_reserve(bo, true);
>>  if (ret) {
>>  pr_err("%s: Failed to reserve BO\n", __func__);
>> @@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
>> mm_struct *mm,
>>  amdgpu_bo_unreserve(bo);
>>   
>>   release_out:
>> -if (ret)
>> -release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> -free_out:
>> -kvfree(mem->user_pages);
>> -mem->user_pages = NULL;
>> +amdgpu_ttm_tt_get_user_pages_done

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v5

[Yang, Philip]

On 2019-01-07 9:21 a.m., Christian König wrote:
> Am 14.12.18 um 22:10 schrieb Yang, Philip:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c   |   2 -
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   3 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c    | 185 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c    |  25 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h    |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 168 ++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c    |   1 -
>>   11 files changed, 209 insertions(+), 293 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index 70429f7aa9a8..717791d4fa45 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -62,7 +62,6 @@ struct kgd_mem {
>>   atomic_t invalid;
>>   struct amdkfd_process_info *process_info;
>> -    struct page **user_pages;
>>   struct amdgpu_sync sync;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index be1ab43473c6..2897793600f7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   goto out;
>>   }
>> -    /* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -    WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -    mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -    if (!mem->user_pages) {
>> -    pr_err("%s: Failed to allocate pages array\n", __func__);
>> -    ret = -ENOMEM;
>> -    goto unregister_out;
>> -    }
>> -
>> -    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>   if (ret) {
>>   pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
>> -    goto free_out;
>> +    goto unregister_out;
>>   }
>> -    amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
>> -
>>   ret = amdgpu_bo_reserve(bo, true);
>>   if (ret) {
>>   pr_err("%s: Failed to reserve BO\n", __func__);
>> @@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   amdgpu_bo_unreserve(bo);
>>   release_out:
>> -    if (ret)
>> -    release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> -free_out:
>> -    kvfree(mem->user_pages);
>> -    mem->user_pages = NULL;
>> +    amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>>   unregister_out:
>>   if (ret)
>>   amdgpu_mn_unregister(bo);
>> @@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
>>   ctx->kfd_bo.priority = 0;
>>   ctx->kfd_bo.tv.bo = &bo->tbo;
>>   ctx->kfd_bo.tv.num_shared = 1;
>> -    ctx->kfd_bo.user_pages = NULL;
>>   list_add(&ctx->kfd_bo.tv.head, &ctx->li

[PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v6

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

The depdent HMM patchset from Jérôme Glisse are all merged into 4.20.0
kernel now.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 122 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 55 insertions(+), 77 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index f76bcb9c45e4..675efc850ff4 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index e55508b39496..5d518d2bb9be 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -87,6 +86,9 @@ struct amdgpu_mn {
struct rb_root_cached   objects;
struct mutexread_lock;
atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +105,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -237,21 +237,19 @@ static void amdgpu_mn_invalidate_node(struct 
amdgpu_mn_node *node,
 /**
  * amdgpu_mn_invalidate_range_start_gfx - callback to notify about mm change
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
- * @start: start of updated range
- * @end: end of updated range
+ * @mirror: the hmm_mirror (mm) is 

[PATCH 2/3] drm/amdkfd: avoid HMM change cause circular lock dependency v2

[Yang, Philip]

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Change-Id: I2ec09a47571f6b4c8eaef93f22c0a600f5f70153
Signed-off-by: Philip Yang 
---
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 32 ++-
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 8372556b52eb..efe0d3c0215b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1156,21 +1156,17 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
int retval;
struct mqd_manager *mqd_mgr;
 
-   retval = 0;
-
-   dqm_lock(dqm);
-
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
pr_warn("Can't create new usermode queue because %d queues were 
already created\n",
dqm->total_queue_count);
retval = -EPERM;
-   goto out_unlock;
+   goto out;
}
 
if (q->properties.type == KFD_QUEUE_TYPE_SDMA) {
retval = allocate_sdma_queue(dqm, &q->sdma_id);
if (retval)
-   goto out_unlock;
+   goto out;
q->properties.sdma_queue_id =
q->sdma_id / get_num_sdma_engines(dqm);
q->properties.sdma_engine_id =
@@ -1181,6 +1177,9 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_sdma_queue;
 
+   /* Do init_mqd before dqm_lock(dqm) to avoid circular locking order:
+* lock(dqm) -> bo::reserve
+*/
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
 
@@ -1188,6 +1187,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
retval = -ENOMEM;
goto out_deallocate_doorbell;
}
+
/*
 * Eviction state logic: we only mark active queues as evicted
 * to avoid the overhead of restoring inactive queues later
@@ -1196,9 +1196,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
q->properties.is_evicted = (q->properties.queue_size > 0 &&
q->properties.queue_percent > 0 &&
q->properties.queue_address != 0);
-
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
-
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
@@ -1206,6 +1204,8 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_doorbell;
 
+   dqm_lock(dqm);
+
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
if (q->properties.is_active) {
@@ -1233,9 +1233,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
 out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
deallocate_sdma_queue(dqm, q->sdma_id);
-out_unlock:
-   dqm_unlock(dqm);
-
+out:
return retval;
 }
 
@@ -1398,8 +1396,6 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = true;
}
 
-   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
-
/*
 * Unconditionally decrement this counter, regardless of the queue's
 * type
@@ -1410,6 +1406,9 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
 
dqm_unlock(dqm);
 
+   /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
+   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+
return retval;
 
 failed:
@@ -1631,7 +1630,11 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = false;
}
 
-   /* lastly, free mqd resources */
+   dqm_unlock(dqm);
+
+   /* Lastly, free mqd resources.
+* Do uninit_mqd() after dqm_unlock to avoid circular locking.
+*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
@@ -1645,7 +1648,6 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
}
 
 out:
-   dqm_unlock(dqm);
return retval;
 }
 
-- 
2.17.1

_

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 9 files changed, 198 insertions(+), 277 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0b31a1859023..0e1711a75b68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -61,7 +61,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d7b10d79f1de..ae2d838d31ea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
return ret;
@@ -1855,25 +1824,

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Ping Christian, any comments for the GEM and CS part changes?

Thanks. Philip

On 2019-01-10 12:02 p.m., Yang, Philip wrote:
> Use HMM helper function hmm_vma_fault() to get physical pages backing
> userptr and start CPU page table update track of those pages. Then use
> hmm_vma_range_done() to check if those pages are updated before
> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
> 
> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
> from scratch, for kfd, restore worker is rescheduled to retry.
> 
> HMM simplify the CPU page table concurrent update check, so remove
> guptasklock, mmu_invalidations, last_set_pages fields from
> amdgpu_ttm_tt struct.
> 
> HMM does not pin the page (increase page ref count), so remove related
> operations like release_pages(), put_page(), mark_page_dirty().
> 
> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
> Signed-off-by: Philip Yang 
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
>   9 files changed, 198 insertions(+), 277 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> index 0b31a1859023..0e1711a75b68 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
> @@ -61,7 +61,6 @@ struct kgd_mem {
>   
>   atomic_t invalid;
>   struct amdkfd_process_info *process_info;
> - struct page **user_pages;
>   
>   struct amdgpu_sync sync;
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index d7b10d79f1de..ae2d838d31ea 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
> mm_struct *mm,
>   goto out;
>   }
>   
> - /* If no restore worker is running concurrently, user_pages
> -  * should not be allocated
> -  */
> - WARN(mem->user_pages, "Leaking user_pages array");
> -
> - mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
> -sizeof(struct page *),
> -GFP_KERNEL | __GFP_ZERO);
> - if (!mem->user_pages) {
> - pr_err("%s: Failed to allocate pages array\n", __func__);
> - ret = -ENOMEM;
> - goto unregister_out;
> - }
> -
> - ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
> + ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>   if (ret) {
>   pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
> - goto free_out;
> + goto unregister_out;
>   }
>   
> - amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
> -
>   ret = amdgpu_bo_reserve(bo, true);
>   if (ret) {
>   pr_err("%s: Failed to reserve BO\n", __func__);
> @@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
> mm_struct *mm,
>   amdgpu_bo_unreserve(bo);
>   
>   release_out:
> - if (ret)
> - release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
> -free_out:
> - kvfree(mem->user_pages);
> - mem->user_pages = NULL;
> + amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>   unregister_out:
>   if (ret)
>   amdgpu_mn_unregister(bo);
> @@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
>   ctx->kfd_bo.priority = 0;
>   ctx->kfd_bo.tv.bo = &bo->tbo;
>   ctx->kfd_bo.tv.num_shared = 1;
> - ctx->kfd_bo.user_pages = NULL;
>   list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>   
>   amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
> @@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
>   ctx->kfd_bo.priority = 0;
>   ctx->kfd_bo.tv.bo = &bo->tbo;
>   ctx->kfd_bo.tv.num_shared = 1;
> - ctx->kf

Re: Yet another RX Vega hang with another kernel panic signature. WARNING: inconsistent lock state

[Yang, Philip]

I found same issue while debugging, I will submit patch to fix this shortly.

Philip

On 2019-01-30 10:35 p.m., Mikhail Gavrilov wrote:
> Hi folks.
> Yet another kernel panic happens while GPU again is hang:
> 
> [ 1469.906798] 
> [ 1469.906799] WARNING: inconsistent lock state
> [ 1469.906801] 5.0.0-0.rc4.git2.2.fc30.x86_64 #1 Tainted: G C
> [ 1469.906802] 
> [ 1469.906804] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
> [ 1469.906806] kworker/12:3/681 [HC0[0]:SC0[0]:HE1:SE1] takes:
> [ 1469.906807] d591b82b
> (&(&adev->vm_manager.pasid_lock)->rlock){?...}, at:
> amdgpu_vm_get_task_info+0x23/0x80 [amdgpu]
> [ 1469.906851] {IN-HARDIRQ-W} state was registered at:
> [ 1469.906855]   _raw_spin_lock+0x31/0x80
> [ 1469.906893]   amdgpu_vm_get_task_info+0x23/0x80 [amdgpu]
> [ 1469.906936]   gmc_v9_0_process_interrupt+0x198/0x2b0 [amdgpu]
> [ 1469.906978]   amdgpu_irq_dispatch+0x90/0x1f0 [amdgpu]
> [ 1469.907018]   amdgpu_irq_callback+0x4a/0x70 [amdgpu]
> [ 1469.907061]   amdgpu_ih_process+0x89/0x100 [amdgpu]
> [ 1469.907103]   amdgpu_irq_handler+0x22/0x50 [amdgpu]
> [ 1469.907106]   __handle_irq_event_percpu+0x3f/0x290
> [ 1469.907108]   handle_irq_event_percpu+0x31/0x80
> [ 1469.907109]   handle_irq_event+0x34/0x51
> [ 1469.907111]   handle_edge_irq+0x7c/0x1a0
> [ 1469.907114]   handle_irq+0xbf/0x100
> [ 1469.907116]   do_IRQ+0x61/0x120
> [ 1469.907118]   ret_from_intr+0x0/0x22
> [ 1469.907121]   cpuidle_enter_state+0xbf/0x470
> [ 1469.907123]   do_idle+0x1ec/0x280
> [ 1469.907125]   cpu_startup_entry+0x19/0x20
> [ 1469.907127]   start_secondary+0x1b3/0x200
> [ 1469.907129]   secondary_startup_64+0xa4/0xb0
> [ 1469.907131] irq event stamp: 5546749
> [ 1469.907133] hardirqs last  enabled at (5546749):
> [] ktime_get+0xfa/0x130
> [ 1469.907135] hardirqs last disabled at (5546748):
> [] ktime_get+0x2b/0x130
> [ 1469.907137] softirqs last  enabled at (5498318):
> [] __do_softirq+0x35f/0x46a
> [ 1469.907140] softirqs last disabled at (5497393):
> [] irq_exit+0x119/0x120
> [ 1469.907141]
> other info that might help us debug this:
> [ 1469.907142]  Possible unsafe locking scenario:
> 
> [ 1469.907143]CPU0
> [ 1469.907144]
> [ 1469.907144]   lock(&(&adev->vm_manager.pasid_lock)->rlock);
> [ 1469.907146]   
> [ 1469.907147] lock(&(&adev->vm_manager.pasid_lock)->rlock);
> [ 1469.907148]
>  *** DEADLOCK ***
> 
> [ 1469.907150] 2 locks held by kworker/12:3/681:
> [ 1469.907152]  #0: 953235a7 ((wq_completion)"events"){+.+.},
> at: process_one_work+0x1e9/0x5d0
> [ 1469.907157]  #1: 71a3d218
> ((work_completion)(&(&sched->work_tdr)->work)){+.+.}, at:
> process_one_work+0x1e9/0x5d0
> [ 1469.907160]
> stack backtrace:
> [ 1469.907163] CPU: 12 PID: 681 Comm: kworker/12:3 Tainted: G
> C5.0.0-0.rc4.git2.2.fc30.x86_64 #1
> [ 1469.907165] Hardware name: System manufacturer System Product
> Name/ROG STRIX X470-I GAMING, BIOS 1103 11/16/2018
> [ 1469.907169] Workqueue: events drm_sched_job_timedout [gpu_sched]
> [ 1469.907171] Call Trace:
> [ 1469.907176]  dump_stack+0x85/0xc0
> [ 1469.907180]  print_usage_bug.cold+0x1ae/0x1e8
> [ 1469.907183]  ? print_shortest_lock_dependencies+0x40/0x40
> [ 1469.907185]  mark_lock+0x50a/0x600
> [ 1469.907186]  ? print_shortest_lock_dependencies+0x40/0x40
> [ 1469.907189]  __lock_acquire+0x544/0x1660
> [ 1469.907191]  ? mark_held_locks+0x57/0x80
> [ 1469.907193]  ? trace_hardirqs_on_thunk+0x1a/0x1c
> [ 1469.907195]  ? lockdep_hardirqs_on+0xed/0x180
> [ 1469.907197]  ? trace_hardirqs_on_thunk+0x1a/0x1c
> [ 1469.907200]  ? retint_kernel+0x10/0x10
> [ 1469.907202]  lock_acquire+0xa2/0x1b0
> [ 1469.907242]  ? amdgpu_vm_get_task_info+0x23/0x80 [amdgpu]
> [ 1469.907245]  _raw_spin_lock+0x31/0x80
> [ 1469.907283]  ? amdgpu_vm_get_task_info+0x23/0x80 [amdgpu]
> [ 1469.907323]  amdgpu_vm_get_task_info+0x23/0x80 [amdgpu]
> [ 1469.907324] [ cut here ]
> 
> 
> My kernel commit is: 62967898789d
> 
> 
> 
> --
> Best Regards,
> Mike Gavrilov.
> 
> 
> ___
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: use spin_lock_irqsave to protect vm_manager.pasid_idr

[Yang, Philip]

amdgpu_vm_get_task_info is called from interrupt handler and sched timeout
workqueue, so it is needed to use irq version spin_lock to avoid deadlock.

Change-Id: Ifedd4b97535bf0b5d3936edd2d9688957020efd4
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 8f394a20a9eb..bfeb9007e100 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -3386,14 +3386,15 @@ void amdgpu_vm_get_task_info(struct amdgpu_device 
*adev, unsigned int pasid,
 struct amdgpu_task_info *task_info)
 {
struct amdgpu_vm *vm;
+   unsigned long flags;
 
-   spin_lock(&adev->vm_manager.pasid_lock);
+   spin_lock_irqsave(&adev->vm_manager.pasid_lock, flags);
 
vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
if (vm)
*task_info = vm->task_info;
 
-   spin_unlock(&adev->vm_manager.pasid_lock);
+   spin_unlock_irqrestore(&adev->vm_manager.pasid_lock, flags);
 }
 
 /**
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 0/3] Use HMM to replace get_user_pages

[Yang, Philip]

Hi Christian,

This patch is rebased to lastest HMM. Please review the GEM and CS part changes
in patch 3/3.

Regards,

Philip Yang (3):
  drm/amdgpu: use HMM mirror callback to replace mmu notifier v6
  drm/amdkfd: avoid HMM change cause circular lock dependency v2
  drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

 drivers/gpu/drm/amd/amdgpu/Kconfig|   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c| 164 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |  32 ++--
 12 files changed, 282 insertions(+), 374 deletions(-)

-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v6

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

The depdent HMM patchset from Jérôme Glisse are all merged into 4.20.0
kernel now.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 139 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 67 insertions(+), 82 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index 466da5954a68..851001ced5e8 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index 3e6823fdd939..5d518d2bb9be 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -87,6 +86,9 @@ struct amdgpu_mn {
struct rb_root_cached   objects;
struct mutexread_lock;
atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +105,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +131,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -237,141 +237,126 @@ static void amdgpu_mn_invalidate_node(struct 
amdgpu_mn_node *node,
 /**
  * amdgpu_mn_invalidate_range_start_gfx - callback to notify about mm change
  *
- * @mn: our notifier
- * @range: mmu notifier context
+ * @mirror: the hmm_mirror (mm) is about to update
+ * @update: the update start, end address
  *
  * Blo

[PATCH 2/3] drm/amdkfd: avoid HMM change cause circular lock dependency v2

[Yang, Philip]

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Change-Id: I2ec09a47571f6b4c8eaef93f22c0a600f5f70153
Signed-off-by: Philip Yang 
---
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 32 ++-
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 8372556b52eb..efe0d3c0215b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1156,21 +1156,17 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
int retval;
struct mqd_manager *mqd_mgr;
 
-   retval = 0;
-
-   dqm_lock(dqm);
-
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
pr_warn("Can't create new usermode queue because %d queues were 
already created\n",
dqm->total_queue_count);
retval = -EPERM;
-   goto out_unlock;
+   goto out;
}
 
if (q->properties.type == KFD_QUEUE_TYPE_SDMA) {
retval = allocate_sdma_queue(dqm, &q->sdma_id);
if (retval)
-   goto out_unlock;
+   goto out;
q->properties.sdma_queue_id =
q->sdma_id / get_num_sdma_engines(dqm);
q->properties.sdma_engine_id =
@@ -1181,6 +1177,9 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_sdma_queue;
 
+   /* Do init_mqd before dqm_lock(dqm) to avoid circular locking order:
+* lock(dqm) -> bo::reserve
+*/
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
 
@@ -1188,6 +1187,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
retval = -ENOMEM;
goto out_deallocate_doorbell;
}
+
/*
 * Eviction state logic: we only mark active queues as evicted
 * to avoid the overhead of restoring inactive queues later
@@ -1196,9 +1196,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
q->properties.is_evicted = (q->properties.queue_size > 0 &&
q->properties.queue_percent > 0 &&
q->properties.queue_address != 0);
-
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
-
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
@@ -1206,6 +1204,8 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_doorbell;
 
+   dqm_lock(dqm);
+
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
if (q->properties.is_active) {
@@ -1233,9 +1233,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
 out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
deallocate_sdma_queue(dqm, q->sdma_id);
-out_unlock:
-   dqm_unlock(dqm);
-
+out:
return retval;
 }
 
@@ -1398,8 +1396,6 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = true;
}
 
-   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
-
/*
 * Unconditionally decrement this counter, regardless of the queue's
 * type
@@ -1410,6 +1406,9 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
 
dqm_unlock(dqm);
 
+   /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
+   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+
return retval;
 
 failed:
@@ -1631,7 +1630,11 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = false;
}
 
-   /* lastly, free mqd resources */
+   dqm_unlock(dqm);
+
+   /* Lastly, free mqd resources.
+* Do uninit_mqd() after dqm_unlock to avoid circular locking.
+*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
@@ -1645,7 +1648,6 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
}
 
 out:
-   dqm_unlock(dqm);
return retval;
 }
 
-- 
2.17.1

_

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 9 files changed, 198 insertions(+), 277 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0b31a1859023..0e1711a75b68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -61,7 +61,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d7b10d79f1de..ae2d838d31ea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
return ret;
@@ -1855,25 +1824,

Re: [PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v6

[Yang, Philip]

On 2019-02-04 10:18 a.m., Christian König wrote:
> Am 04.02.19 um 16:06 schrieb Yang, Philip:
>> Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
>> callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
>> DRM_AMDGPU_USERPTR Kconfig.
>>
>> It supports both KFD userptr and gfx userptr paths.
>>
>> The depdent HMM patchset from Jérôme Glisse are all merged into 4.20.0
>> kernel now.
>>
>> Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
>>   drivers/gpu/drm/amd/amdgpu/Makefile    |   2 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 139 +++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
>>   4 files changed, 67 insertions(+), 82 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
>> b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> index 9221e5489069..960a63355705 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Kconfig
>> +++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
>> @@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
>>   config DRM_AMDGPU_USERPTR
>>   bool "Always enable userptr write support"
>>   depends on DRM_AMDGPU
>> -    select MMU_NOTIFIER
>> +    select HMM_MIRROR
>>   help
>> -  This option selects CONFIG_MMU_NOTIFIER if it isn't already
>> -  selected to enabled full userptr support.
>> +  This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
>> +  isn't already selected to enabled full userptr support.
>>   config DRM_AMDGPU_GART_DEBUGFS
>>   bool "Allow GART access through debugfs"
>> diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
>> b/drivers/gpu/drm/amd/amdgpu/Makefile
>> index 466da5954a68..851001ced5e8 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/Makefile
>> +++ b/drivers/gpu/drm/amd/amdgpu/Makefile
>> @@ -172,7 +172,7 @@ endif
>>   amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
>>   amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
>>   amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
>> -amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
>> +amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
>>   include $(FULL_AMD_PATH)/powerplay/Makefile
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> index 3e6823fdd939..5d518d2bb9be 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
>> @@ -45,7 +45,7 @@
>>   #include 
>>   #include 
>> -#include 
>> +#include 
>>   #include 
>>   #include 
>>   #include 
>> @@ -58,7 +58,6 @@
>>    *
>>    * @adev: amdgpu device pointer
>>    * @mm: process address space
>> - * @mn: MMU notifier structure
>>    * @type: type of MMU notifier
>>    * @work: destruction work item
>>    * @node: hash table node to find structure by adev and mn
>> @@ -66,6 +65,7 @@
>>    * @objects: interval tree containing amdgpu_mn_nodes
>>    * @read_lock: mutex for recursive locking of @lock
>>    * @recursion: depth of recursion
>> + * @mirror: HMM mirror function support
>>    *
>>    * Data for each amdgpu device and process address space.
>>    */
>> @@ -73,7 +73,6 @@ struct amdgpu_mn {
>>   /* constant after initialisation */
>>   struct amdgpu_device    *adev;
>>   struct mm_struct    *mm;
>> -    struct mmu_notifier    mn;
>>   enum amdgpu_mn_type    type;
>>   /* only used on destruction */
>> @@ -87,6 +86,9 @@ struct amdgpu_mn {
>>   struct rb_root_cached    objects;
>>   struct mutex    read_lock;
> 
>>   atomic_t    recursion;
> 
> With HMM we don't need this any more. Please remove it and simplify 
> amdgpu_mn_read_lock() and amdgpu_mn_read_unlock().
> 
Thanks, this makes sense because HMM uses hmm->mirror_sem to serialize 
invalidate range.

amn->read_lock is also not needed anymore, this was used to protect 
atomic operations for atomic_inc_return(amn->recursion) and 
down_read(amn->lock).

Just one amn->lock is needed to sync amdgpu_cs_submit and userptr 
update. I will submit another patch.

Philip

> Apart from that looks good to me,
> Christian.
> 
>> +
>> +    /* HMM mirror */
>> +    struct hmm_mirror    mirror;
>>   };
>>   /**
>> @@ -103,7 +105,7 @@ struct amdgpu_mn_node {
>>   };
>>   /**
>> - * amdgpu_mn_destroy - destroy the MMU notifier
>> + * amdgpu_mn_destroy - destroy the HMM mirror
>>    

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 9 files changed, 198 insertions(+), 277 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0b31a1859023..0e1711a75b68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -61,7 +61,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d7b10d79f1de..ae2d838d31ea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
return ret;
@@ -1855,25 +1824,

[PATCH 2/3] drm/amdkfd: avoid HMM change cause circular lock dependency v2

[Yang, Philip]

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Change-Id: I2ec09a47571f6b4c8eaef93f22c0a600f5f70153
Signed-off-by: Philip Yang 
---
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 32 ++-
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 8372556b52eb..efe0d3c0215b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1156,21 +1156,17 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
int retval;
struct mqd_manager *mqd_mgr;
 
-   retval = 0;
-
-   dqm_lock(dqm);
-
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
pr_warn("Can't create new usermode queue because %d queues were 
already created\n",
dqm->total_queue_count);
retval = -EPERM;
-   goto out_unlock;
+   goto out;
}
 
if (q->properties.type == KFD_QUEUE_TYPE_SDMA) {
retval = allocate_sdma_queue(dqm, &q->sdma_id);
if (retval)
-   goto out_unlock;
+   goto out;
q->properties.sdma_queue_id =
q->sdma_id / get_num_sdma_engines(dqm);
q->properties.sdma_engine_id =
@@ -1181,6 +1177,9 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_sdma_queue;
 
+   /* Do init_mqd before dqm_lock(dqm) to avoid circular locking order:
+* lock(dqm) -> bo::reserve
+*/
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
 
@@ -1188,6 +1187,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
retval = -ENOMEM;
goto out_deallocate_doorbell;
}
+
/*
 * Eviction state logic: we only mark active queues as evicted
 * to avoid the overhead of restoring inactive queues later
@@ -1196,9 +1196,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
q->properties.is_evicted = (q->properties.queue_size > 0 &&
q->properties.queue_percent > 0 &&
q->properties.queue_address != 0);
-
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
-
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
@@ -1206,6 +1204,8 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_doorbell;
 
+   dqm_lock(dqm);
+
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
if (q->properties.is_active) {
@@ -1233,9 +1233,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
 out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
deallocate_sdma_queue(dqm, q->sdma_id);
-out_unlock:
-   dqm_unlock(dqm);
-
+out:
return retval;
 }
 
@@ -1398,8 +1396,6 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = true;
}
 
-   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
-
/*
 * Unconditionally decrement this counter, regardless of the queue's
 * type
@@ -1410,6 +1406,9 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
 
dqm_unlock(dqm);
 
+   /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
+   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+
return retval;
 
 failed:
@@ -1631,7 +1630,11 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = false;
}
 
-   /* lastly, free mqd resources */
+   dqm_unlock(dqm);
+
+   /* Lastly, free mqd resources.
+* Do uninit_mqd() after dqm_unlock to avoid circular locking.
+*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
@@ -1645,7 +1648,6 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
}
 
 out:
-   dqm_unlock(dqm);
return retval;
 }
 
-- 
2.17.1

_

[PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v7

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 154 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 70 insertions(+), 94 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index 466da5954a68..851001ced5e8 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index 3e6823fdd939..e356867d2308 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -85,8 +84,9 @@ struct amdgpu_mn {
/* objects protected by lock */
struct rw_semaphore lock;
struct rb_root_cached   objects;
-   struct mutexread_lock;
-   atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +103,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +129,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -181,14 +179,10 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn)
 static int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable)
 {
if (blockable)
-   mutex_lock(&amn->read_lock);
-   else if (!mutex_trylock(&amn->read_lock))
+   down_read(&amn->lock);
+   else if (!down_read_trylock(&amn->lock))

[PATCH 0/3] Use HMM to replace get_user_pages

[Yang, Philip]

Hi Christian,

This patch is rebased to lastest HMM. Please review the GEM and CS part changes
in patch 3/3.

Thanks,

Philip Yang (3):
  drm/amdgpu: use HMM mirror callback to replace mmu notifier v7
  drm/amdkfd: avoid HMM change cause circular lock dependency v2
  drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

 drivers/gpu/drm/amd/amdgpu/Kconfig|   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c| 179 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |  32 ++--
 12 files changed, 285 insertions(+), 386 deletions(-)

-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Hi Christian,

My comments are embedded below. I will submit another patch to address 
those.

Thanks,
Philip

On 2019-02-05 6:52 a.m., Christian König wrote:
> Am 04.02.19 um 19:23 schrieb Yang, Philip:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c    | 158 +++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c    |  25 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h    |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 173 --
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
>>   9 files changed, 198 insertions(+), 277 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index 0b31a1859023..0e1711a75b68 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -61,7 +61,6 @@ struct kgd_mem {
>>   atomic_t invalid;
>>   struct amdkfd_process_info *process_info;
>> -    struct page **user_pages;
>>   struct amdgpu_sync sync;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index d7b10d79f1de..ae2d838d31ea 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   goto out;
>>   }
>> -    /* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -    WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -    mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -    if (!mem->user_pages) {
>> -    pr_err("%s: Failed to allocate pages array\n", __func__);
>> -    ret = -ENOMEM;
>> -    goto unregister_out;
>> -    }
>> -
>> -    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>   if (ret) {
>>   pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
>> -    goto free_out;
>> +    goto unregister_out;
>>   }
>> -    amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
>> -
>>   ret = amdgpu_bo_reserve(bo, true);
>>   if (ret) {
>>   pr_err("%s: Failed to reserve BO\n", __func__);
>> @@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   amdgpu_bo_unreserve(bo);
>>   release_out:
>> -    if (ret)
>> -    release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
>> -free_out:
>> -    kvfree(mem->user_pages);
>> -    mem->user_pages = NULL;
>> +    amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
>>   unregister_out:
>>   if (ret)
>>   amdgpu_mn_unregister(bo);
>> @@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
>>   ctx->kfd_bo.priority = 0;
>>   ctx->kfd_bo.tv.bo = &bo->tbo;
>>   ctx->kfd_bo.tv.num_shared = 1;
>> -    ctx->kfd_bo.user_pages = NULL;
>>   list_add(&ctx->kfd_bo.tv.head, &ctx->list);
>>

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v6

[Yang, Philip]

Hi Christian,

I will submit new patch for review, my comments embedded inline below.

Thanks,
Philip

On 2019-02-05 1:09 p.m., Koenig, Christian wrote:
> Am 05.02.19 um 18:25 schrieb Yang, Philip:
>> [SNIP]+
>>>> +    if (r == -ERESTARTSYS) {
>>>> +    if (!--tries) {
>>>> +    DRM_ERROR("Possible deadlock? Retry too many times\n");
>>>> +    return -EDEADLK;
>>>> +    }
>>>> +    goto restart;
>>> You really need to restart the IOCTL to potentially handle signals.
>>>
>>> But since the calling code correctly handles this already you can just
>>> drop this change as far as I can see.
>>>
>> I agree that we should return -ERESTARTSYS to upper layer to handle signals.
>>
>> But I do not find upper layers handle -ERESTARTSYS in the entire calling
>> path, ksys_ioctl -> do_vfs_ioctl -> amdgpu_drm_ioctl -> drm->ioctl ->
>> drm_ioctl_kernel -> amdgpu_cs_ioctl. The error code returns to
>> application. I confirm it, libdrm userptr test application calling
>> amdgpu_cs_ioctl return code is -512, which is -ERESTARTSYS.
>>
>> So application should handle -ERESTARTSYS to restart the ioctl, but
>> libdrm userptr test application doesn't handle this. This causes the
>> test failed.
> 
> This is a bug in the test cases then.
> 
> -ERESTARTSYS can happen at any time during interruptible waiting and it
> is mandatory for the upper layer to handle it correctly.
> 
-ERESTARTSYS can be returned only when signal is pending, signal handler 
will translate ERESTARTSYS to EINTR, drmIoctl in libdrm does handle 
EINTR and restart the ioctl. The test cases are ok.

Driver fail path should not return ERESTARTSYS to user space. The new 
patch, I change amdgpu_cs_submit to return -EAGAIN if userptr is 
updated, and amdgpu_cs_ioctl redo the ioctl only if error code is 
-EAGAIN. ERESTARTSYS error code returns to user space for signal handle 
as before.

>>
>> Below are details of userptr path difference. For the previous path,
>> libdrm test always goes to step 2, step 3 never trigger. So it never
>> return -ERESTARTSYS, but in theory, this could happen.
>>
>> For HMM path, the test always goes to step 3, we have to handle this
>> case inside amdgpu_cs_ioctl. Maybe I can change amdgpu_cs_submit to
>> return -EBUSY, then restart the ioctl inside amdgpu_cs_ioctl. I will
>> submit new patch.
> 
> Clearly a NAK, this won't work correctly.
> 
I don't understand your concern, may you explain the reason?

> Christian.
> 
>>
>> The previous userptr path:
>> 1. gem_userptr_ioctl to register userptr
>> 2. amdgpu_cs_parser_bos, check if userptr is invalidated, then update
>> userptr
>> 3. amdgpu_cs_submit, hold p->mn lock, check if userptr is invalidated,
>> return -ERESTARTSYS
>>
>> The new HMM userptr path:
>> 1. gem_userptr_ioctl to register userptr
>> 2. amdgpu_cs_parser_bos, start HMM to track userptr update
>> 3. amdgpu_cs_submit, hold p->mn lock, check HMM if userptr is
>> invalidated, return -ERESTARTSYS
>>
>>
>>>> +    }
>>>> +
>>>>     return r;
>>>>     }
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>>>> index d21dd2f369da..555285e329ed 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
>>>> @@ -329,26 +329,24 @@ int amdgpu_gem_userptr_ioctl(struct drm_device
>>>> *dev, void *data,
>>>>     r = amdgpu_bo_reserve(bo, true);
>>>>     if (r)
>>>> -    goto free_pages;
>>>> +    goto user_pages_done;
>>>>     amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT);
>>>>     r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
>>>>     amdgpu_bo_unreserve(bo);
>>>>     if (r)
>>>> -    goto free_pages;
>>>> +    goto user_pages_done;
>>>>     }
>>>>     r = drm_gem_handle_create(filp, gobj, &handle);
>>>> -    /* drop reference from allocate - handle holds it now */
>>>> -    drm_gem_object_put_unlocked(gobj);
>>>>     if (r)
>>>> -    return r;
>>>> +    goto user_pages_done;
>>>>     args->handle = handle;
>>>> -    return 0;
>>>> -free_pages:
>>>> -    release_pages

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v7

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 158 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 178 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 9 files changed, 198 insertions(+), 282 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0b31a1859023..0e1711a75b68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -61,7 +61,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d7b10d79f1de..ae2d838d31ea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
return ret;
@@ -1855,25 +1824,

Re: [PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v7

[Yang, Philip]

 >> +/* If userptr are updated after amdgpu_cs_parser_bos(), restart
 >> cs */
 >>   amdgpu_bo_list_for_each_userptr_entry(e, p->bo_list) {
 >>   struct amdgpu_bo *bo = ttm_to_amdgpu_bo(e->tv.bo);
 >> -if (amdgpu_ttm_tt_userptr_needs_pages(bo->tbo.ttm)) {
 >> -r = -ERESTARTSYS;
 >> -goto error_abort;
 >> -}
 >> +r |= !amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 >
 > Just abort when you see the first one with a problem here.
 >
 > There is no value in checking all of them.
 >
No, amdgpu_ttm_tt_get_user_pages_done calls hmm_vma_range_done to stop 
HMM track and free range structure memory, this needs go through all 
userptr BOs.

 >> +
 >> +if (r == -EAGAIN) {
 >> +if (!--tries) {
 >> +DRM_ERROR("Possible deadlock? Retry too many times\n");
 >> +return -EDEADLK;
 >> +}
 >> +goto restart;
 >> +}
 >> +
 >
 > I would still say to better just return to userspace here.
 >
 > Because of the way HMM works 10 retries might not be sufficient any more.
 >
Yes, it looks better to handle retry from user space. The extra sys call 
overhead can be ignored because this does not happen all the time. I 
will submit new patch for review.

Thanks,
Philip

On 2019-02-06 4:20 a.m., Christian König wrote:
> Am 05.02.19 um 23:00 schrieb Yang, Philip:
>> Use HMM helper function hmm_vma_fault() to get physical pages backing
>> userptr and start CPU page table update track of those pages. Then use
>> hmm_vma_range_done() to check if those pages are updated before
>> amdgpu_cs_submit for gfx or before user queues are resumed for kfd.
>>
>> If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
>> from scratch, for kfd, restore worker is rescheduled to retry.
>>
>> HMM simplify the CPU page table concurrent update check, so remove
>> guptasklock, mmu_invalidations, last_set_pages fields from
>> amdgpu_ttm_tt struct.
>>
>> HMM does not pin the page (increase page ref count), so remove related
>> operations like release_pages(), put_page(), mark_page_dirty().
>>
>> Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
>> Signed-off-by: Philip Yang 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   1 -
>>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c    | 158 +++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c    |  25 ++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h    |   4 +-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 178 +++---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
>>   9 files changed, 198 insertions(+), 282 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> index 0b31a1859023..0e1711a75b68 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
>> @@ -61,7 +61,6 @@ struct kgd_mem {
>>   atomic_t invalid;
>>   struct amdkfd_process_info *process_info;
>> -    struct page **user_pages;
>>   struct amdgpu_sync sync;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> index d7b10d79f1de..ae2d838d31ea 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
>> @@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, 
>> struct mm_struct *mm,
>>   goto out;
>>   }
>> -    /* If no restore worker is running concurrently, user_pages
>> - * should not be allocated
>> - */
>> -    WARN(mem->user_pages, "Leaking user_pages array");
>> -
>> -    mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
>> -   sizeof(struct page *),
>> -   GFP_KERNEL | __GFP_ZERO);
>> -    if (!mem->user_pages) {
>> -    pr_err("%s: Failed to allocate pages array\n", __func__);
>> -    ret = -ENOMEM;
>> -    goto unregister_out;
>> -    }
>> -
>> -    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
>> +    ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
>>   if (ret) {
>>

[PATCH 3/3] drm/amdgpu: replace get_user_pages with HMM address mirror helpers v8

[Yang, Philip]

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Change-Id: I2e8c0c6f0d8c21e5596a32d7fc91762778bc9e67
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 138 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|  25 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 178 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 9 files changed, 182 insertions(+), 278 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0b31a1859023..0e1711a75b68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -61,7 +61,6 @@ struct kgd_mem {
 
atomic_t invalid;
struct amdkfd_process_info *process_info;
-   struct page **user_pages;
 
struct amdgpu_sync sync;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d7b10d79f1de..ae2d838d31ea 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -582,28 +582,12 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
goto out;
}
 
-   /* If no restore worker is running concurrently, user_pages
-* should not be allocated
-*/
-   WARN(mem->user_pages, "Leaking user_pages array");
-
-   mem->user_pages = kvmalloc_array(bo->tbo.ttm->num_pages,
-  sizeof(struct page *),
-  GFP_KERNEL | __GFP_ZERO);
-   if (!mem->user_pages) {
-   pr_err("%s: Failed to allocate pages array\n", __func__);
-   ret = -ENOMEM;
-   goto unregister_out;
-   }
-
-   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, mem->user_pages);
+   ret = amdgpu_ttm_tt_get_user_pages(bo->tbo.ttm, bo->tbo.ttm->pages);
if (ret) {
pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
-   goto free_out;
+   goto unregister_out;
}
 
-   amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, mem->user_pages);
-
ret = amdgpu_bo_reserve(bo, true);
if (ret) {
pr_err("%s: Failed to reserve BO\n", __func__);
@@ -616,11 +600,7 @@ static int init_user_pages(struct kgd_mem *mem, struct 
mm_struct *mm,
amdgpu_bo_unreserve(bo);
 
 release_out:
-   if (ret)
-   release_pages(mem->user_pages, bo->tbo.ttm->num_pages);
-free_out:
-   kvfree(mem->user_pages);
-   mem->user_pages = NULL;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
 unregister_out:
if (ret)
amdgpu_mn_unregister(bo);
@@ -679,7 +659,6 @@ static int reserve_bo_and_vm(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]);
@@ -743,7 +722,6 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem,
ctx->kfd_bo.priority = 0;
ctx->kfd_bo.tv.bo = &bo->tbo;
ctx->kfd_bo.tv.num_shared = 1;
-   ctx->kfd_bo.user_pages = NULL;
list_add(&ctx->kfd_bo.tv.head, &ctx->list);
 
i = 0;
@@ -1371,15 +1349,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
list_del(&bo_list_entry->head);
mutex_unlock(&process_info->lock);
 
-   /* Free user pages if necessary */
-   if (mem->user_pages) {
-   pr_debug("%s: Freeing user_pages array\n", __func__);
-   if (mem->user_pages[0])
-   release_pages(mem->user_pages,
-   mem->bo->tbo.ttm->num_pages);
-   kvfree(mem->user_pages);
-   }
-
ret = reserve_bo_and_cond_vms(mem, NULL, BO_VM_ALL, &ctx);
if (unlikely(ret))
return ret;
@@ -1855,25 +1824,11

[PATCH 0/3] Use HMM to replace get_user_pages

[Yang, Philip]

Hi Christian,

Resend patch 1/3, 2/3, added Reviewed-by in comments.

Change in patch 3/3, amdgpu_cs_submit, amdgpu_cs_ioctl return -EAGAIN
to user space to retry cs_ioctl.

Regards,
Philip

Philip Yang (3):
  drm/amdgpu: use HMM mirror callback to replace mmu notifier v7
  drm/amdkfd: avoid HMM change cause circular lock dependency v2
  drm/amdgpu: replace get_user_pages with HMM address mirror helpers v8

 drivers/gpu/drm/amd/amdgpu/Kconfig|   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h|   1 -
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  |  95 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c| 138 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |  14 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c| 179 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h|   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 178 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h   |   3 +-
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |  32 ++--
 12 files changed, 269 insertions(+), 387 deletions(-)

-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/3] drm/amdgpu: use HMM mirror callback to replace mmu notifier v7

[Yang, Philip]

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

Change-Id: Ie62c3c5e3c5b8521ab3b438d1eff2aa2a003835e
Signed-off-by: Philip Yang 
Reviewed-by: Felix Kuehling 
Reviewed-by: Christian König 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig |   6 +-
 drivers/gpu/drm/amd/amdgpu/Makefile|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 154 +++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h |   2 +-
 4 files changed, 70 insertions(+), 94 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9221e5489069..960a63355705 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,10 +26,10 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
-   select MMU_NOTIFIER
+   select HMM_MIRROR
help
- This option selects CONFIG_MMU_NOTIFIER if it isn't already
- selected to enabled full userptr support.
+ This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
+ isn't already selected to enabled full userptr support.
 
 config DRM_AMDGPU_GART_DEBUGFS
bool "Allow GART access through debugfs"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile 
b/drivers/gpu/drm/amd/amdgpu/Makefile
index 466da5954a68..851001ced5e8 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -172,7 +172,7 @@ endif
 amdgpu-$(CONFIG_COMPAT) += amdgpu_ioc32.o
 amdgpu-$(CONFIG_VGA_SWITCHEROO) += amdgpu_atpx_handler.o
 amdgpu-$(CONFIG_ACPI) += amdgpu_acpi.o
-amdgpu-$(CONFIG_MMU_NOTIFIER) += amdgpu_mn.o
+amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_mn.o
 
 include $(FULL_AMD_PATH)/powerplay/Makefile
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index 3e6823fdd939..e356867d2308 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -45,7 +45,7 @@
 
 #include 
 #include 
-#include 
+#include 
 #include 
 #include 
 #include 
@@ -58,7 +58,6 @@
  *
  * @adev: amdgpu device pointer
  * @mm: process address space
- * @mn: MMU notifier structure
  * @type: type of MMU notifier
  * @work: destruction work item
  * @node: hash table node to find structure by adev and mn
@@ -66,6 +65,7 @@
  * @objects: interval tree containing amdgpu_mn_nodes
  * @read_lock: mutex for recursive locking of @lock
  * @recursion: depth of recursion
+ * @mirror: HMM mirror function support
  *
  * Data for each amdgpu device and process address space.
  */
@@ -73,7 +73,6 @@ struct amdgpu_mn {
/* constant after initialisation */
struct amdgpu_device*adev;
struct mm_struct*mm;
-   struct mmu_notifier mn;
enum amdgpu_mn_type type;
 
/* only used on destruction */
@@ -85,8 +84,9 @@ struct amdgpu_mn {
/* objects protected by lock */
struct rw_semaphore lock;
struct rb_root_cached   objects;
-   struct mutexread_lock;
-   atomic_trecursion;
+
+   /* HMM mirror */
+   struct hmm_mirror   mirror;
 };
 
 /**
@@ -103,7 +103,7 @@ struct amdgpu_mn_node {
 };
 
 /**
- * amdgpu_mn_destroy - destroy the MMU notifier
+ * amdgpu_mn_destroy - destroy the HMM mirror
  *
  * @work: previously sheduled work item
  *
@@ -129,28 +129,26 @@ static void amdgpu_mn_destroy(struct work_struct *work)
}
up_write(&amn->lock);
mutex_unlock(&adev->mn_lock);
-   mmu_notifier_unregister_no_release(&amn->mn, amn->mm);
+
+   hmm_mirror_unregister(&amn->mirror);
kfree(amn);
 }
 
 /**
- * amdgpu_mn_release - callback to notify about mm destruction
+ * amdgpu_hmm_mirror_release - callback to notify about mm destruction
  *
- * @mn: our notifier
- * @mm: the mm this callback is about
+ * @mirror: the HMM mirror (mm) this callback is about
  *
- * Shedule a work item to lazy destroy our notifier.
+ * Shedule a work item to lazy destroy HMM mirror.
  */
-static void amdgpu_mn_release(struct mmu_notifier *mn,
- struct mm_struct *mm)
+static void amdgpu_hmm_mirror_release(struct hmm_mirror *mirror)
 {
-   struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn);
+   struct amdgpu_mn *amn = container_of(mirror, struct amdgpu_mn, mirror);
 
INIT_WORK(&amn->work, amdgpu_mn_destroy);
schedule_work(&amn->work);
 }
 
-
 /**
  * amdgpu_mn_lock - take the write side lock for this notifier
  *
@@ -181,14 +179,10 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn)
 static int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable)
 {
if (blockable)
-   mutex_lock(&amn->read_lock);
-   else if (!mutex_trylock(&amn->read_lock))
+   down_read(&am

[PATCH 2/3] drm/amdkfd: avoid HMM change cause circular lock dependency v2

[Yang, Philip]

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Change-Id: I2ec09a47571f6b4c8eaef93f22c0a600f5f70153
Signed-off-by: Philip Yang 
Reviewed-by: Felix Kuehling 
Acked-by: Christian König 
---
 .../drm/amd/amdkfd/kfd_device_queue_manager.c | 32 ++-
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 8372556b52eb..efe0d3c0215b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1156,21 +1156,17 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
int retval;
struct mqd_manager *mqd_mgr;
 
-   retval = 0;
-
-   dqm_lock(dqm);
-
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
pr_warn("Can't create new usermode queue because %d queues were 
already created\n",
dqm->total_queue_count);
retval = -EPERM;
-   goto out_unlock;
+   goto out;
}
 
if (q->properties.type == KFD_QUEUE_TYPE_SDMA) {
retval = allocate_sdma_queue(dqm, &q->sdma_id);
if (retval)
-   goto out_unlock;
+   goto out;
q->properties.sdma_queue_id =
q->sdma_id / get_num_sdma_engines(dqm);
q->properties.sdma_engine_id =
@@ -1181,6 +1177,9 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_sdma_queue;
 
+   /* Do init_mqd before dqm_lock(dqm) to avoid circular locking order:
+* lock(dqm) -> bo::reserve
+*/
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
 
@@ -1188,6 +1187,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
retval = -ENOMEM;
goto out_deallocate_doorbell;
}
+
/*
 * Eviction state logic: we only mark active queues as evicted
 * to avoid the overhead of restoring inactive queues later
@@ -1196,9 +1196,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
q->properties.is_evicted = (q->properties.queue_size > 0 &&
q->properties.queue_percent > 0 &&
q->properties.queue_address != 0);
-
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
-
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
@@ -1206,6 +1204,8 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
if (retval)
goto out_deallocate_doorbell;
 
+   dqm_lock(dqm);
+
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
if (q->properties.is_active) {
@@ -1233,9 +1233,7 @@ static int create_queue_cpsch(struct device_queue_manager 
*dqm, struct queue *q,
 out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
deallocate_sdma_queue(dqm, q->sdma_id);
-out_unlock:
-   dqm_unlock(dqm);
-
+out:
return retval;
 }
 
@@ -1398,8 +1396,6 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = true;
}
 
-   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
-
/*
 * Unconditionally decrement this counter, regardless of the queue's
 * type
@@ -1410,6 +1406,9 @@ static int destroy_queue_cpsch(struct 
device_queue_manager *dqm,
 
dqm_unlock(dqm);
 
+   /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
+   mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+
return retval;
 
 failed:
@@ -1631,7 +1630,11 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
qpd->reset_wavefronts = false;
}
 
-   /* lastly, free mqd resources */
+   dqm_unlock(dqm);
+
+   /* Lastly, free mqd resources.
+* Do uninit_mqd() after dqm_unlock to avoid circular locking.
+*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->ops.get_mqd_manager(dqm,
get_mqd_type_from_queue_type(q->properties.type));
@@ -1645,7 +1648,6 @@ static int process_termination_cpsch(struct 
device_queue_manager *dqm,
}
 
 out:
-   dqm_unlock

[PATCH] drm/amdgpu: select ARCH_HAS_HMM and ZONE_DEVICE option

[Yang, Philip]

Those options are needed to support HMM

Change-Id: Ieb7bb3bcec07245d79a02793e6728228decc400a
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 960a63355705..63f0542bc34b 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,7 +26,9 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
+   select ARCH_HAS_HMM
select HMM_MIRROR
+   select ZONE_DEVICE
help
  This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
  isn't already selected to enabled full userptr support.
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: select ARCH_HAS_HMM and ZONE_DEVICE option

[Yang, Philip]

Thanks Jerome for the the correct HMM config option, only select 
HMM_MIRROR is not good enough because CONFIG_HMM option maybe missing, 
add depends on ARCH_HAS_HMM will solve the issue.

I will submit new patch to fix the compilation error if HMM_MIRROR 
config is missing and the HMM config dependency issue.

Philip

On 2019-02-20 7:25 p.m., Jerome Glisse wrote:
> On Thu, Feb 21, 2019 at 12:17:33AM +, Kuehling, Felix wrote:
>> On 2019-02-20 6:34 p.m., Jerome Glisse wrote:
>>> On Wed, Feb 20, 2019 at 10:39:49PM +, Kuehling, Felix wrote:
 On 2019-02-20 5:12 p.m., Jerome Glisse wrote:
> On Wed, Feb 20, 2019 at 07:18:17PM +, Kuehling, Felix wrote:
>> [+Jerome]
>>
>> Why to we need ZONE_DEVICE. I didn't think this was needed for mirroring
>> CPU page tables to device page tables.
>>
>> ARCH_HAS_HMM depends on (X86_64 || PPC64). Do we have some alternative
>> for ARM support?
>>
>> Also, the name ARCH_HAS_HMM looks like it's meant to be selected by the
>> CPU architecture rather than any driver. Jerome, do you have any advice?
> This patch is wrong you need to depend on ARCH_HAS_HMM and
 Who selects ARCH_HAS_HMM? Currently I don't see this selected anywhere.
 So any config option that depends on it will be invisible in menuconfig.
 Do we need ARCH_HAS_HMM somewhere in the arch/x86/Kconfig and
 arch/powerpc/Kconfig?

 Also, ARCH_HAS_HMM does not currently support ARM. Does that mean we
 can't have ARM support in AMDGPU if we start using HMM?
>>> ARCH_HAS_HMM is defined by architecture that support HMM. So par x86
>>> and PPC. It should not be hard to add it to ARM (i can not remember if
>>> ARM has DAX yet or not, if ARM does not have DAX then you need to add
>>> that first).
>>
>> Not having ARM support is a bummer. I just enabled KFD on ARM a few
>> weeks ago. Now depending on HMM makes KFD unusable on ARM. [+Mark FYI] I
>> hope this is only a temporary setback.
> 
> It should not be hard to add in fact all it might need is a Kconfig
> patch. I have no easy access to ARM with PCIE so i have not tackle
> this yet.
> 
>>
>>
 Finally, ARCH_HAS_HMM has a bunch of dependencies. If they are not met,
 I guess it can't be enabled. Should those be "select"s instead?
>>> No they should not be selected, people configuring their system need
>>> to have the freedom of doing so. All those option are selected in all
>>> the big distribution.
>> As far as I can tell, the arch/x86/Kconfig doesn't select ARCH_HAS_HMM.
>> Its default is "y", so it should be enabled on anything that meets the
>> dependencies. But ZONE_DEVICE was not enabled by default. I think that's
>> what broke our kernel configs.
>>
>> We'll fix our own kernel configs to enable ZONE_DEVICE and ARCH_HAS_HMM
>> to get our internal builds to work again.
> 
> You seem to be doing weird thing with your kconfig ...
> 
>>
>> I suspect other users with their own kernel configs will stumble over
>> this and wonder why KFD and userptr support are disabled in their builds.
> 
> Patch to improve kconfig are welcome but they should not force select
> thing. Configuration is there to give user freedom to select fewature
> they want to give up.
> 
> Maybe following would help:
> ARCH_HAS_HMM
> - bool
> - default y
> + def_bool y
> 
> Cheers,
> Jérôme
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: fix HMM config dependency issue

[Yang, Philip]

Only select HMM_MIRROR will get kernel config dependency warnings
if CONFIG_HMM is missing in the config. Add depends on HMM will
solve the issue.

Add conditional compilation to fix compilation errors if HMM_MIRROR
is not enabled as HMM config is not enabled.

Change-Id: I1b44a0b5285bbef5e98bfb045d1d82c167af1cb8
Signed-off-by: Philip Yang 
---
 drivers/gpu/drm/amd/amdgpu/Kconfig  |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c |  6 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 12 
 3 files changed, 19 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig 
b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 960a63355705..67553effb649 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -26,6 +26,7 @@ config DRM_AMDGPU_CIK
 config DRM_AMDGPU_USERPTR
bool "Always enable userptr write support"
depends on DRM_AMDGPU
+   depends on ARCH_HAS_HMM
select HMM_MIRROR
help
  This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 1e675048f790..c1dbca14dce5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -712,7 +712,9 @@ struct amdgpu_ttm_tt {
uint64_tuserptr;
struct task_struct  *usertask;
uint32_tuserflags;
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
struct hmm_rangerange;
+#endif
 };
 
 /**
@@ -722,6 +724,7 @@ struct amdgpu_ttm_tt {
  * Calling function must call amdgpu_ttm_tt_userptr_range_done() once and only
  * once afterwards to stop HMM tracking
  */
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
 int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages)
 {
struct amdgpu_ttm_tt *gtt = (void *)ttm;
@@ -804,6 +807,7 @@ bool amdgpu_ttm_tt_get_user_pages_done(struct ttm_tt *ttm)
 
return r;
 }
+#endif
 
 /**
  * amdgpu_ttm_tt_set_user_pages - Copy pages in, putting old pages as 
necessary.
@@ -904,9 +908,11 @@ static void amdgpu_ttm_tt_unpin_userptr(struct ttm_tt *ttm)
 
sg_free_table(ttm->sg);
 
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
if (gtt->range.pfns &&
ttm->pages[0] == hmm_pfn_to_page(>t->range, gtt->range.pfns[0]))
WARN_ONCE(1, "Missing get_user_page_done\n");
+#endif
 }
 
 int amdgpu_ttm_gart_bind(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index 8988c87fff9d..c9d87271a4cb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -101,8 +101,20 @@ int amdgpu_mmap(struct file *filp, struct vm_area_struct 
*vma);
 int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo);
 int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo);
 
+#if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
 int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages);
 bool amdgpu_ttm_tt_get_user_pages_done(struct ttm_tt *ttm);
+#else
+static inline int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page 
**pages)
+{
+   return -EPERM;
+}
+static inline bool amdgpu_ttm_tt_get_user_pages_done(struct ttm_tt *ttm)
+{
+   return false;
+}
+#endif
+
 void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages);
 void amdgpu_ttm_tt_mark_user_pages(struct ttm_tt *ttm);
 int amdgpu_ttm_tt_set_userptr(struct ttm_tt *ttm, uint64_t addr,
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: KASAN caught amdgpu / HMM use-after-free

[Yang, Philip]

Hi Michel,

Yes, I found the same issue and the bug has been fixed by Jerome:

876b462120aa mm/hmm: use reference counting for HMM struct

The fix is on hmm-for-5.1 branch, I cherry-pick it into my local branch 
to workaround the issue.

Regards,
Philip

On 2019-02-27 12:02 p.m., Michel Dänzer wrote:
> 
> See the attached dmesg excerpt. I've hit this a few times running piglit
> with amd-staging-drm-next, first on February 22nd.
> 
> The memory was freed after calling hmm_mirror_unregister in
> amdgpu_mn_destroy.
> 
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: KASAN caught amdgpu / HMM use-after-free

[Yang, Philip]

amd-staging-drm-next will rebase to kernel 5.1 to pickup this fix 
automatically. As a short-term workaround, please cherry-pick this fix 
into your local repository.

Regards,
Philip

On 2019-02-27 12:33 p.m., Michel Dänzer wrote:
> On 2019-02-27 6:14 p.m., Yang, Philip wrote:
>> Hi Michel,
>>
>> Yes, I found the same issue and the bug has been fixed by Jerome:
>>
>> 876b462120aa mm/hmm: use reference counting for HMM struct
>>
>> The fix is on hmm-for-5.1 branch, I cherry-pick it into my local branch
>> to workaround the issue.
> 
> Please push it to amd-staging-drm-next, so that others don't run into
> the issue as well.
> 
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: KASAN caught amdgpu / HMM use-after-free

[Yang, Philip]

Hi Alex,

Pushed, thanks.

mm/hmm: use reference counting for HMM struct

Philip

On 2019-02-27 1:32 p.m., Deucher, Alexander wrote:
> Go ahead an apply it to amd-staging-drm-next.  It'll naturally fall out 
> when I rebase it.
> 
> Alex
> 
> *From:* amd-gfx  on behalf of 
> Yang, Philip 
> *Sent:* Wednesday, February 27, 2019 1:05 PM
> *To:* Michel Dänzer; Jérôme Glisse
> *Cc:* linux...@kvack.org; amd-gfx@lists.freedesktop.org
> *Subject:* Re: KASAN caught amdgpu / HMM use-after-free
> amd-staging-drm-next will rebase to kernel 5.1 to pickup this fix
> automatically. As a short-term workaround, please cherry-pick this fix
> into your local repository.
> 
> Regards,
> Philip
> 
> On 2019-02-27 12:33 p.m., Michel Dänzer wrote:
>> On 2019-02-27 6:14 p.m., Yang, Philip wrote:
>>> Hi Michel,
>>>
>>> Yes, I found the same issue and the bug has been fixed by Jerome:
>>>
>>> 876b462120aa mm/hmm: use reference counting for HMM struct
>>>
>>> The fix is on hmm-for-5.1 branch, I cherry-pick it into my local branch
>>> to workaround the issue.
>> 
>> Please push it to amd-staging-drm-next, so that others don't run into
>> the issue as well.
>> 
>> 
> ___
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: KASAN caught amdgpu / HMM use-after-free

[Yang, Philip]

Hi Alex,

May you help take a look? It is not merged into amd-staging-drm-next 
yet, maybe missing code-review+2, it was done automatically after code 
review for other patch.

http://git.amd.com:8080/c/brahma/ec/linux/+/206711

Regards,
Philip

On 2019-02-28 6:51 a.m., Michel Dänzer wrote:
> 
> [ Dropping Jérôme and the linux-mm list ]
> 
> On 2019-02-27 7:48 p.m., Yang, Philip wrote:
>> Hi Alex,
>>
>> Pushed, thanks.
>>
>> mm/hmm: use reference counting for HMM struct
> 
> Thanks, but I'm not seeing it yet. Maybe it needs some special
> treatment, because it's not a DRM code change?
> 
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu: handle userptr corner cases with HMM path

[Yang, Philip]

Those corner cases are found by kfdtest.KFDIPCTest.

userptr may cross two vmas if the forked child process (not call exec
after fork) malloc buffer, then free it, and then malloc larger size
buf, kerenl will create new vma adjacent to old vma which was cloned
from parent process, some pages of userptr are in the first vma, the
rest pages are in the second vma. HMM expects range only have one vma,
we have to use two ranges to handle this case. See is_mergeable_anon_vma
in mm/mmap.c for details.

kfd userptr restore may have concurrent userptr invalidation, reschedule
to restore and then needs call hmm_vma_range_done to remove range from
hmm->ranges list, otherwise hmm_vma_fault add same range to the list
will cause loop in the list because range->next point to range itself.

Change-Id: I641ba7406c32bd8b7ae715f52bd896d53fe56801
Signed-off-by: Philip Yang 
---
 .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 28 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 73 +--
 2 files changed, 71 insertions(+), 30 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index f8104760f1e6..179af9d3ab19 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1738,6 +1738,23 @@ static int update_invalid_user_pages(struct 
amdkfd_process_info *process_info,
return 0;
 }
 
+/* Untrack invalid userptr BOs
+ *
+ * Stop HMM track the userptr update
+ */
+static void untrack_invalid_user_pages(struct amdkfd_process_info 
*process_info)
+{
+   struct kgd_mem *mem, *tmp_mem;
+   struct amdgpu_bo *bo;
+
+   list_for_each_entry_safe(mem, tmp_mem,
+&process_info->userptr_inval_list,
+validate_list.head) {
+   bo = mem->bo;
+   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
+   }
+}
+
 /* Validate invalid userptr BOs
  *
  * Validates BOs on the userptr_inval_list, and moves them back to the
@@ -1855,12 +1872,7 @@ static int validate_invalid_user_pages(struct 
amdkfd_process_info *process_info)
 out_free:
kfree(pd_bo_list_entries);
 out_no_mem:
-   list_for_each_entry_safe(mem, tmp_mem,
-&process_info->userptr_inval_list,
-validate_list.head) {
-   bo = mem->bo;
-   amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
-   }
+   untrack_invalid_user_pages(process_info);
 
return ret;
 }
@@ -1904,8 +1916,10 @@ static void amdgpu_amdkfd_restore_userptr_worker(struct 
work_struct *work)
 * and we can just restart the queues.
 */
if (!list_empty(&process_info->userptr_inval_list)) {
-   if (atomic_read(&process_info->evicted_bos) != evicted_bos)
+   if (atomic_read(&process_info->evicted_bos) != evicted_bos) {
+   untrack_invalid_user_pages(process_info);
goto unlock_out; /* Concurrent eviction, try again */
+   }
 
if (validate_invalid_user_pages(process_info))
goto unlock_out;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index cd0ccfbbcb84..e5736225f513 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -711,7 +711,7 @@ struct amdgpu_ttm_tt {
struct task_struct  *usertask;
uint32_tuserflags;
 #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
-   struct hmm_rangerange;
+   struct hmm_rangerange, range2;
 #endif
 };
 
@@ -727,58 +727,81 @@ int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, 
struct page **pages)
 {
struct amdgpu_ttm_tt *gtt = (void *)ttm;
struct mm_struct *mm = gtt->usertask->mm;
-   unsigned long end = gtt->userptr + ttm->num_pages * PAGE_SIZE;
+   unsigned long start = gtt->userptr;
+   unsigned long end = start + ttm->num_pages * PAGE_SIZE;
struct hmm_range *range = >t->range;
+   struct hmm_range *range2 = >t->range2;
+   struct vm_area_struct *vma, *vma_next = NULL;
+   uint64_t *pfns, f;
int r = 0, i;
 
if (!mm) /* Happens during process shutdown */
return -ESRCH;
 
-   amdgpu_hmm_init_range(range);
-
down_read(&mm->mmap_sem);
 
-   range->vma = find_vma(mm, gtt->userptr);
-   if (!range_in_vma(range->vma, gtt->userptr, end))
-   r = -EFAULT;
-   else if ((gtt->userflags & AMDGPU_GEM_USERPTR_ANONONLY) &&
-   range->vma->vm_file)
+   /* user pages may cross vma bound */
+   vma = find_vma(mm, start);
+   if (unlikely(!range_in_vma(vma, start, end))) {
+   vma_next = find_extend_vma(mm, end);
+   if (unlikely(!vma_next)) {
+   r = -EFAULT;
+   go

Re: [PATCH] drm/amdgpu: handle userptr corner cases with HMM path

[Yang, Philip]

Hi Felix,

I will split this and submit two patches for review.

I didn't realize mprotect can have userptr with more than 2 vmas. For 
malloc, it will always be one or two vmas. I will loop over all VMAs to 
call hmm_vma_fault to get userptr pages.

Thanks,
Philip

On 2019-03-01 7:46 p.m., Kuehling, Felix wrote:
> Since you're addressing two distinct bugs, please split this into two patches.
> 
> For the multiple VMAs, should we generalize that to handle any number of 
> VMAs? It's not a typical case, but you could easily construct situations with 
> mprotect where different parts of the same buffer have different VMAs and 
> then register that as a single user pointer. Or you could user MAP_FIXED to 
> map multiple files to adjacent virtual addresses.
> 
> There may be two ways to handle this:
> 1. If the userptr address range spans more than one VMA, fail
> 2. Loop over all the VMAs in the address range
> 
> Thanks,
>Felix
> 
> -----Original Message-
> From: amd-gfx  On Behalf Of Yang, 
> Philip
> Sent: Friday, March 01, 2019 12:30 PM
> To: amd-gfx@lists.freedesktop.org
> Cc: Yang, Philip 
> Subject: [PATCH] drm/amdgpu: handle userptr corner cases with HMM path
> 
> Those corner cases are found by kfdtest.KFDIPCTest.
> 
> userptr may cross two vmas if the forked child process (not call exec
> after fork) malloc buffer, then free it, and then malloc larger size
> buf, kerenl will create new vma adjacent to old vma which was cloned
> from parent process, some pages of userptr are in the first vma, the
> rest pages are in the second vma. HMM expects range only have one vma,
> we have to use two ranges to handle this case. See is_mergeable_anon_vma
> in mm/mmap.c for details.
> 
> kfd userptr restore may have concurrent userptr invalidation, reschedule
> to restore and then needs call hmm_vma_range_done to remove range from
> hmm->ranges list, otherwise hmm_vma_fault add same range to the list
> will cause loop in the list because range->next point to range itself.
> 
> Change-Id: I641ba7406c32bd8b7ae715f52bd896d53fe56801
> Signed-off-by: Philip Yang 
> ---
>   .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c  | 28 +--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c   | 73 +--
>   2 files changed, 71 insertions(+), 30 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index f8104760f1e6..179af9d3ab19 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -1738,6 +1738,23 @@ static int update_invalid_user_pages(struct 
> amdkfd_process_info *process_info,
>   return 0;
>   }
>   
> +/* Untrack invalid userptr BOs
> + *
> + * Stop HMM track the userptr update
> + */
> +static void untrack_invalid_user_pages(struct amdkfd_process_info 
> *process_info)
> +{
> + struct kgd_mem *mem, *tmp_mem;
> + struct amdgpu_bo *bo;
> +
> + list_for_each_entry_safe(mem, tmp_mem,
> +  &process_info->userptr_inval_list,
> +  validate_list.head) {
> + bo = mem->bo;
> + amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
> + }
> +}
> +
>   /* Validate invalid userptr BOs
>*
>* Validates BOs on the userptr_inval_list, and moves them back to the
> @@ -1855,12 +1872,7 @@ static int validate_invalid_user_pages(struct 
> amdkfd_process_info *process_info)
>   out_free:
>   kfree(pd_bo_list_entries);
>   out_no_mem:
> - list_for_each_entry_safe(mem, tmp_mem,
> -  &process_info->userptr_inval_list,
> -  validate_list.head) {
> - bo = mem->bo;
> - amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm);
> - }
> + untrack_invalid_user_pages(process_info);
>   
>   return ret;
>   }
> @@ -1904,8 +1916,10 @@ static void 
> amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work)
>* and we can just restart the queues.
>*/
>   if (!list_empty(&process_info->userptr_inval_list)) {
> - if (atomic_read(&process_info->evicted_bos) != evicted_bos)
> + if (atomic_read(&process_info->evicted_bos) != evicted_bos) {
> + untrack_invalid_user_pages(process_info);
>   goto unlock_out; /* Concurrent eviction, try again */
> + }
>   
>   if (validate_invalid_user_pages(process_info))
>   goto unlock_out;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> b/driver