Re: [PATCH 2/2] drm/amdgpu: use ring buffer for fault handling on GMC 9
Am 01.02.19 um 17:42 schrieb Kuehling, Felix: > On 2019-02-01 8:23 a.m., Christian König wrote: >> Hi Felix, >> >> Am 31.01.19 um 22:38 schrieb Kuehling, Felix: >>> I don't see how this solves anything. You'll still need to remove >>> entries from this fault ring buffer. Otherwise you'll ignore faults for >>> addresses that have faulted recently, although it may be a new fault. >>> >>> Also, this reintroduces interrupt storms and repeated processing of the >>> same fault in cases where we have more than 64 distinct addresses >>> faulting at once. >> Yeah, this is not meant to be a perfect solution. It just works for now. >> >> Key point is that from my experience the hardware never faults more >> than 26 different addresses at the same time (and not the slightest >> idea where this restriction comes from). > That could be specific to the test you're running. If it only uses 26 or > less pages per draw operation, you won't get more addresses faulting at > the same time. I'm almost certain compute apps can create more > outstanding page faults with large data sets. My test uses a compute submission which clears a 4MB big texture and so causes exactly 1024 different faults. It was actually one of your comments which pushed me towards this solution. Since the hardware is only limited capable of switching to other threads the number of faults you can have in the hardware queue is actually quite limited. But I totally agree that 64 is most likely not sufficient. The real value is probably something between 256 and 1024. >> So with 64 distinct addresses we should be on the safe side. > I think Philip has seen more faults at the same time with his tests. > Philip, can you confirm. > > >> What can obviously happen is that hardware faults address A, we serve >> that one, OS evicts page at address A, but address A is still in the >> fault handling and we never notice that it is faulted once more. > Exactly. That's my concern. Yeah, but that's handle able. In other words when the OS is invalidating a page we need to flush the TLB anyway, flushing those lockup entries on top is trivial. >>> I think the solution should be different. We do need a way to ignore >>> duplicate faults for the same address. The problem you ran into was that >>> after clearing an entry, you get additional interrupts for the same >>> address. This could be handled by keeping track of the age of IH ring >>> entries. After we update the page table and flush TLBs, there may still >>> be stale entries in the IH ring. But no new interrupts should be >>> generated (until that PTE is invalidated again). So we can check the >>> current IH wptr. When IH processing (rptr) has caught up to that value, >>> all the stale retry interrupts have been flushed, and we can remove the >>> entry from the hash table (or ring buffer). Any subsequent interrupts >>> for the same address are new and should be handled again. >> That's what I thought initially as well, but the problem is this has >> absolutely nothing todo with the IH ring buffer size. >> >> See the IH ring is only 32 entries, but in my tests I've sometimes got >>> 200 stale faults from the hardware. >> The explanation is that I recently learned from the hardware guys that >> the VA fifo of the SDMA is 256 entries long! And I suspect that the >> GFX fifo is not shorter. >> >> To make the situation even worse I have a nice log where a rendering >> operation completed (with interrupt and everything finished), >> according to the registers the GFX block is completely idle and you >> are still getting hundreds of stale faults for this operation. >> >> Nice isn't it? So we can certainly not tie this to the VM or we are >> going to need some grace period before destroying VMs or something >> like this. > Damn. There is the aspect of how to handle VM destruction. But also the > question of when to remove entries from the hash table or a ring buffer. > We know when we invalidate page table entries, so that could be a good > time to remove entries and allow new page faults for the address in > question. But lets take a step back here. > > The consequence of those unpredictable stale page faults is, an > interrupt means that there may or may not be a valid reason to update > the page table. Whether or not the driver needs to do anything will need > to be determined not just based on the interrupt, but based on the > driver's knowledge of the current state of the page table. Well to be honest I don't think this is even necessary. See on the CPU side it is perfectly valid to get multiple faults for the same address. So the fault handling on the OS is perfectly capable of handling this. This means that as long as we code the driver in the same way handling the same fault twice is just a matter of improving performance. In other words when we have 1 million faults and handle maybe 100 twice I honestly don't care that we write the same value to a PTE multiple times :) > If there is a > valid
Re: [PATCH 2/2] drm/amdgpu: use ring buffer for fault handling on GMC 9
On 2019-02-01 8:23 a.m., Christian König wrote: > Hi Felix, > > Am 31.01.19 um 22:38 schrieb Kuehling, Felix: >> I don't see how this solves anything. You'll still need to remove >> entries from this fault ring buffer. Otherwise you'll ignore faults for >> addresses that have faulted recently, although it may be a new fault. >> >> Also, this reintroduces interrupt storms and repeated processing of the >> same fault in cases where we have more than 64 distinct addresses >> faulting at once. > > Yeah, this is not meant to be a perfect solution. It just works for now. > > Key point is that from my experience the hardware never faults more > than 26 different addresses at the same time (and not the slightest > idea where this restriction comes from). That could be specific to the test you're running. If it only uses 26 or less pages per draw operation, you won't get more addresses faulting at the same time. I'm almost certain compute apps can create more outstanding page faults with large data sets. > So with 64 distinct addresses we should be on the safe side. I think Philip has seen more faults at the same time with his tests. Philip, can you confirm. > > What can obviously happen is that hardware faults address A, we serve > that one, OS evicts page at address A, but address A is still in the > fault handling and we never notice that it is faulted once more. Exactly. That's my concern. > >> I think the solution should be different. We do need a way to ignore >> duplicate faults for the same address. The problem you ran into was that >> after clearing an entry, you get additional interrupts for the same >> address. This could be handled by keeping track of the age of IH ring >> entries. After we update the page table and flush TLBs, there may still >> be stale entries in the IH ring. But no new interrupts should be >> generated (until that PTE is invalidated again). So we can check the >> current IH wptr. When IH processing (rptr) has caught up to that value, >> all the stale retry interrupts have been flushed, and we can remove the >> entry from the hash table (or ring buffer). Any subsequent interrupts >> for the same address are new and should be handled again. > > That's what I thought initially as well, but the problem is this has > absolutely nothing todo with the IH ring buffer size. > > See the IH ring is only 32 entries, but in my tests I've sometimes got > >200 stale faults from the hardware. > > The explanation is that I recently learned from the hardware guys that > the VA fifo of the SDMA is 256 entries long! And I suspect that the > GFX fifo is not shorter. > > To make the situation even worse I have a nice log where a rendering > operation completed (with interrupt and everything finished), > according to the registers the GFX block is completely idle and you > are still getting hundreds of stale faults for this operation. > > Nice isn't it? So we can certainly not tie this to the VM or we are > going to need some grace period before destroying VMs or something > like this. Damn. There is the aspect of how to handle VM destruction. But also the question of when to remove entries from the hash table or a ring buffer. We know when we invalidate page table entries, so that could be a good time to remove entries and allow new page faults for the address in question. But lets take a step back here. The consequence of those unpredictable stale page faults is, an interrupt means that there may or may not be a valid reason to update the page table. Whether or not the driver needs to do anything will need to be determined not just based on the interrupt, but based on the driver's knowledge of the current state of the page table. If there is a valid entry with the right permissions, we don't need to do anything. The driver will need to track enough information to make that determination quickly. If we can do that, we probably don't need a hash table or a ring buffer. We do need a fast path to ignore redundant interrupts to handle the interrupt storm. The hash table may not be the right approach. The ring buffer you added is definitely not the right approach. A quick lookup of the state of the page table at a given address seems like the right approach. The page table in VRAM is too slow to access. We'll need to shadow the valid and permissions bits of all allocated PTs in the driver. When we add page migration, we'll also need to track information about the location of the mapped memory (same GPU, remote GPU, system memory). I think one byte per PTE should cover it. For handling VM teardown, we need to ignore page faults from VMs that don't exist any more. We may need some mechanism to avoid reuse of PASIDs until we can be sure there will be no more stale page faults from that PASID. Then we can simply ignore page faults with PASIDs that aren't allocated. I don't have a good idea how long we need to quarantine PASIDs after destroying VMs yet. We'll
Re: [PATCH 2/2] drm/amdgpu: use ring buffer for fault handling on GMC 9
Hi Felix, Am 31.01.19 um 22:38 schrieb Kuehling, Felix: I don't see how this solves anything. You'll still need to remove entries from this fault ring buffer. Otherwise you'll ignore faults for addresses that have faulted recently, although it may be a new fault. Also, this reintroduces interrupt storms and repeated processing of the same fault in cases where we have more than 64 distinct addresses faulting at once. Yeah, this is not meant to be a perfect solution. It just works for now. Key point is that from my experience the hardware never faults more than 26 different addresses at the same time (and not the slightest idea where this restriction comes from). So with 64 distinct addresses we should be on the safe side. What can obviously happen is that hardware faults address A, we serve that one, OS evicts page at address A, but address A is still in the fault handling and we never notice that it is faulted once more. I think the solution should be different. We do need a way to ignore duplicate faults for the same address. The problem you ran into was that after clearing an entry, you get additional interrupts for the same address. This could be handled by keeping track of the age of IH ring entries. After we update the page table and flush TLBs, there may still be stale entries in the IH ring. But no new interrupts should be generated (until that PTE is invalidated again). So we can check the current IH wptr. When IH processing (rptr) has caught up to that value, all the stale retry interrupts have been flushed, and we can remove the entry from the hash table (or ring buffer). Any subsequent interrupts for the same address are new and should be handled again. That's what I thought initially as well, but the problem is this has absolutely nothing todo with the IH ring buffer size. See the IH ring is only 32 entries, but in my tests I've sometimes got >200 stale faults from the hardware. The explanation is that I recently learned from the hardware guys that the VA fifo of the SDMA is 256 entries long! And I suspect that the GFX fifo is not shorter. To make the situation even worse I have a nice log where a rendering operation completed (with interrupt and everything finished), according to the registers the GFX block is completely idle and you are still getting hundreds of stale faults for this operation. Nice isn't it? So we can certainly not tie this to the VM or we are going to need some grace period before destroying VMs or something like this. Regards, Christian. Regards, Felix On 2019-01-24 7:52 a.m., Christian König wrote: Further testing showed that the idea with the chash doesn't work as expected. Especially we can't predict when we can remove the entries from the hash again. So replace the chash with a simple ring buffer for now to filter out the already handled faults. As long as we see less than 64 distinct addresses we still report them only once to the following handling. Signed-off-by: Christian König --- drivers/gpu/drm/Kconfig | 2 - drivers/gpu/drm/Makefile | 1 - drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 8 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c| 105 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 14 - drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 40 +- drivers/gpu/drm/amd/include/linux/chash.h | 366 - drivers/gpu/drm/amd/lib/Kconfig | 28 - drivers/gpu/drm/amd/lib/Makefile | 32 -- drivers/gpu/drm/amd/lib/chash.c | 638 -- 10 files changed, 16 insertions(+), 1218 deletions(-) delete mode 100644 drivers/gpu/drm/amd/include/linux/chash.h delete mode 100644 drivers/gpu/drm/amd/lib/Kconfig delete mode 100644 drivers/gpu/drm/amd/lib/Makefile delete mode 100644 drivers/gpu/drm/amd/lib/chash.c diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index 4385f00e1d05..98b033e675db 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -229,8 +229,6 @@ config DRM_AMDGPU source "drivers/gpu/drm/amd/amdgpu/Kconfig" -source "drivers/gpu/drm/amd/lib/Kconfig" - source "drivers/gpu/drm/nouveau/Kconfig" source "drivers/gpu/drm/i915/Kconfig" diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index 7f3be3506057..ac6213575691 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -56,7 +56,6 @@ obj-$(CONFIG_DRM_TTM) += ttm/ obj-$(CONFIG_DRM_SCHED) += scheduler/ obj-$(CONFIG_DRM_TDFX) += tdfx/ obj-$(CONFIG_DRM_R128) += r128/ -obj-y += amd/lib/ obj-$(CONFIG_HSA_AMD) += amd/amdkfd/ obj-$(CONFIG_DRM_RADEON)+= radeon/ obj-$(CONFIG_DRM_AMDGPU)+= amd/amdgpu/ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h index 81e6070d255b..83c415eb40f4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
Re: [PATCH 2/2] drm/amdgpu: use ring buffer for fault handling on GMC 9
I don't see how this solves anything. You'll still need to remove
entries from this fault ring buffer. Otherwise you'll ignore faults for
addresses that have faulted recently, although it may be a new fault.
Also, this reintroduces interrupt storms and repeated processing of the
same fault in cases where we have more than 64 distinct addresses
faulting at once.
I think the solution should be different. We do need a way to ignore
duplicate faults for the same address. The problem you ran into was that
after clearing an entry, you get additional interrupts for the same
address. This could be handled by keeping track of the age of IH ring
entries. After we update the page table and flush TLBs, there may still
be stale entries in the IH ring. But no new interrupts should be
generated (until that PTE is invalidated again). So we can check the
current IH wptr. When IH processing (rptr) has caught up to that value,
all the stale retry interrupts have been flushed, and we can remove the
entry from the hash table (or ring buffer). Any subsequent interrupts
for the same address are new and should be handled again.
Regards,
Felix
On 2019-01-24 7:52 a.m., Christian König wrote:
> Further testing showed that the idea with the chash doesn't work as expected.
> Especially we can't predict when we can remove the entries from the hash
> again.
>
> So replace the chash with a simple ring buffer for now to filter out the
> already handled faults. As long as we see less than 64 distinct addresses we
> still report them only once to the following handling.
>
> Signed-off-by: Christian König
> ---
> drivers/gpu/drm/Kconfig | 2 -
> drivers/gpu/drm/Makefile | 1 -
> drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 8 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c| 105
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 14 -
> drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 40 +-
> drivers/gpu/drm/amd/include/linux/chash.h | 366 -
> drivers/gpu/drm/amd/lib/Kconfig | 28 -
> drivers/gpu/drm/amd/lib/Makefile | 32 --
> drivers/gpu/drm/amd/lib/chash.c | 638 --
> 10 files changed, 16 insertions(+), 1218 deletions(-)
> delete mode 100644 drivers/gpu/drm/amd/include/linux/chash.h
> delete mode 100644 drivers/gpu/drm/amd/lib/Kconfig
> delete mode 100644 drivers/gpu/drm/amd/lib/Makefile
> delete mode 100644 drivers/gpu/drm/amd/lib/chash.c
>
> diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
> index 4385f00e1d05..98b033e675db 100644
> --- a/drivers/gpu/drm/Kconfig
> +++ b/drivers/gpu/drm/Kconfig
> @@ -229,8 +229,6 @@ config DRM_AMDGPU
>
> source "drivers/gpu/drm/amd/amdgpu/Kconfig"
>
> -source "drivers/gpu/drm/amd/lib/Kconfig"
> -
> source "drivers/gpu/drm/nouveau/Kconfig"
>
> source "drivers/gpu/drm/i915/Kconfig"
> diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
> index 7f3be3506057..ac6213575691 100644
> --- a/drivers/gpu/drm/Makefile
> +++ b/drivers/gpu/drm/Makefile
> @@ -56,7 +56,6 @@ obj-$(CONFIG_DRM_TTM) += ttm/
> obj-$(CONFIG_DRM_SCHED) += scheduler/
> obj-$(CONFIG_DRM_TDFX) += tdfx/
> obj-$(CONFIG_DRM_R128) += r128/
> -obj-y+= amd/lib/
> obj-$(CONFIG_HSA_AMD) += amd/amdkfd/
> obj-$(CONFIG_DRM_RADEON)+= radeon/
> obj-$(CONFIG_DRM_AMDGPU)+= amd/amdgpu/
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> index 81e6070d255b..83c415eb40f4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
> @@ -43,6 +43,11 @@
>*/
> #define AMDGPU_GMC_HOLE_MASK0xULL
>
> +/*
> + * Number of entries for the log of recent VM faults
> + */
> +#define AMDGPU_GMC_NUM_VM_FAULTS 64
> +
> struct firmware;
>
> /*
> @@ -147,6 +152,9 @@ struct amdgpu_gmc {
> struct kfd_vm_fault_info *vm_fault_info;
> atomic_tvm_fault_info_updated;
>
> + uint64_tvm_faults[AMDGPU_GMC_NUM_VM_FAULTS];
> + int vm_fault_idx;
> +
> const struct amdgpu_gmc_funcs *gmc_funcs;
>
> struct amdgpu_xgmi xgmi;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 0bc6f553dc08..3041d72c5478 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -2972,22 +2972,6 @@ void amdgpu_vm_adjust_size(struct amdgpu_device *adev,
> uint32_t min_vm_size,
>adev->vm_manager.fragment_size);
> }
>
> -static struct amdgpu_retryfault_hashtable *init_fault_hash(void)
> -{
> - struct amdgpu_retryfault_hashtable *fault_hash;
> -
> - fault_hash = kmalloc(sizeof(*fault_hash), GFP_KERNEL);
> - if (!fault_hash)
> - return fault_hash;
> -
> - INIT_CHASH_TABLE(fault_hash->hash,
> -
[PATCH 2/2] drm/amdgpu: use ring buffer for fault handling on GMC 9
Further testing showed that the idea with the chash doesn't work as expected.
Especially we can't predict when we can remove the entries from the hash again.
So replace the chash with a simple ring buffer for now to filter out the
already handled faults. As long as we see less than 64 distinct addresses we
still report them only once to the following handling.
Signed-off-by: Christian König
---
drivers/gpu/drm/Kconfig | 2 -
drivers/gpu/drm/Makefile | 1 -
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 8 +
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c| 105
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 14 -
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 40 +-
drivers/gpu/drm/amd/include/linux/chash.h | 366 -
drivers/gpu/drm/amd/lib/Kconfig | 28 -
drivers/gpu/drm/amd/lib/Makefile | 32 --
drivers/gpu/drm/amd/lib/chash.c | 638 --
10 files changed, 16 insertions(+), 1218 deletions(-)
delete mode 100644 drivers/gpu/drm/amd/include/linux/chash.h
delete mode 100644 drivers/gpu/drm/amd/lib/Kconfig
delete mode 100644 drivers/gpu/drm/amd/lib/Makefile
delete mode 100644 drivers/gpu/drm/amd/lib/chash.c
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index 4385f00e1d05..98b033e675db 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -229,8 +229,6 @@ config DRM_AMDGPU
source "drivers/gpu/drm/amd/amdgpu/Kconfig"
-source "drivers/gpu/drm/amd/lib/Kconfig"
-
source "drivers/gpu/drm/nouveau/Kconfig"
source "drivers/gpu/drm/i915/Kconfig"
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 7f3be3506057..ac6213575691 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -56,7 +56,6 @@ obj-$(CONFIG_DRM_TTM) += ttm/
obj-$(CONFIG_DRM_SCHED)+= scheduler/
obj-$(CONFIG_DRM_TDFX) += tdfx/
obj-$(CONFIG_DRM_R128) += r128/
-obj-y += amd/lib/
obj-$(CONFIG_HSA_AMD) += amd/amdkfd/
obj-$(CONFIG_DRM_RADEON)+= radeon/
obj-$(CONFIG_DRM_AMDGPU)+= amd/amdgpu/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
index 81e6070d255b..83c415eb40f4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
@@ -43,6 +43,11 @@
*/
#define AMDGPU_GMC_HOLE_MASK 0xULL
+/*
+ * Number of entries for the log of recent VM faults
+ */
+#define AMDGPU_GMC_NUM_VM_FAULTS 64
+
struct firmware;
/*
@@ -147,6 +152,9 @@ struct amdgpu_gmc {
struct kfd_vm_fault_info *vm_fault_info;
atomic_tvm_fault_info_updated;
+ uint64_tvm_faults[AMDGPU_GMC_NUM_VM_FAULTS];
+ int vm_fault_idx;
+
const struct amdgpu_gmc_funcs *gmc_funcs;
struct amdgpu_xgmi xgmi;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 0bc6f553dc08..3041d72c5478 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2972,22 +2972,6 @@ void amdgpu_vm_adjust_size(struct amdgpu_device *adev,
uint32_t min_vm_size,
adev->vm_manager.fragment_size);
}
-static struct amdgpu_retryfault_hashtable *init_fault_hash(void)
-{
- struct amdgpu_retryfault_hashtable *fault_hash;
-
- fault_hash = kmalloc(sizeof(*fault_hash), GFP_KERNEL);
- if (!fault_hash)
- return fault_hash;
-
- INIT_CHASH_TABLE(fault_hash->hash,
- AMDGPU_PAGEFAULT_HASH_BITS, 8, 0);
- spin_lock_init(_hash->lock);
- fault_hash->count = 0;
-
- return fault_hash;
-}
-
/**
* amdgpu_vm_init - initialize a vm instance
*
@@ -3080,12 +3064,6 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct
amdgpu_vm *vm,
vm->pasid = pasid;
}
- vm->fault_hash = init_fault_hash();
- if (!vm->fault_hash) {
- r = -ENOMEM;
- goto error_free_root;
- }
-
INIT_KFIFO(vm->faults);
return 0;
@@ -3241,15 +3219,10 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct
amdgpu_vm *vm)
struct amdgpu_bo_va_mapping *mapping, *tmp;
bool prt_fini_needed = !!adev->gmc.gmc_funcs->set_prt;
struct amdgpu_bo *root;
- u64 fault;
int i, r;
amdgpu_amdkfd_gpuvm_destroy_cb(adev, vm);
- /* Clear pending page faults from IH when the VM is destroyed */
- while (kfifo_get(>faults, ))
- amdgpu_vm_clear_fault(vm->fault_hash, fault);
-
if (vm->pasid) {
unsigned long flags;
@@ -3258,9 +3231,6 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct
amdgpu_vm *vm)
spin_unlock_irqrestore(>vm_manager.pasid_lock, flags);
}
- kfree(vm->fault_hash);
- vm->fault_hash = NULL;
-
drm_sched_entity_destroy(>entity);
if
