Re: [PATCH] drm/buddy: fix issue that force_merge cannot free all roots
On 13/08/2024 10:44, Lin.Cao wrote: If buddy manager have more than one roots and each root have sub-block need to be free. When drm_buddy_fini called, the first loop of force_merge will merge and free all of the sub block of first root, which offset is 0x0 and size is biggest(more than have of the mm size). In subsequent force_merge rounds, if we use 0 as start and use remaining mm size as end, the block of other roots will be skipped in __force_merge function. It will cause the other roots can not be freed. Solution: use roots' offset as the start could fix this issue. Were you able to take a look at the test side for this? See previous reply: https://lore.kernel.org/dri-devel/3fdd9175-832a-4113-8aaa-6039925c5...@intel.com/ Patch itself is good, but would be good to understand why the test for this is not failing, and try to improve that also. Signed-off-by: Lin.Cao --- drivers/gpu/drm/drm_buddy.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..b3f0dd652088 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -324,7 +324,7 @@ EXPORT_SYMBOL(drm_buddy_init); */ void drm_buddy_fini(struct drm_buddy *mm) { - u64 root_size, size; + u64 root_size, size, start; unsigned int order; int i; @@ -332,7 +332,8 @@ void drm_buddy_fini(struct drm_buddy *mm) for (i = 0; i < mm->n_roots; ++i) { order = ilog2(size) - ilog2(mm->chunk_size); - __force_merge(mm, 0, size, order); + start = drm_buddy_block_offset(mm->roots[i]); + __force_merge(mm, start, start + size, order); WARN_ON(!drm_buddy_block_is_free(mm->roots[i])); drm_block_free(mm, mm->roots[i]);
Re: [PATCH] drm/buddy: fix issue that force_merge cannot free all roots
Hi, On 08/08/2024 07:38, Lin.Cao wrote: If buddy manager have more than one roots and each root have sub-block need to be free. When drm_buddy_fini called, the first loop of force_merge will merge and free all of the sub block of first root, which offset is 0x0 and size is biggest(more than have of the mm size). In subsequent force_merge rounds, if we use 0 as start and use remaining mm size as end, the block of other roots will be skipped in __force_merge function. It will cause the other roots can not be freed. Solution: use roots' offset as the start could fix this issue. Signed-off-by: Lin.Cao Nice catch. --- drivers/gpu/drm/drm_buddy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..5379687552bc 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -327,12 +327,14 @@ void drm_buddy_fini(struct drm_buddy *mm) u64 root_size, size; unsigned int order; int i; + u64 start = 0; Nit: We could maybe move this into root_size, size or even into the loop body below? Also no need to init. size = mm->size; for (i = 0; i < mm->n_roots; ++i) { order = ilog2(size) - ilog2(mm->chunk_size); - __force_merge(mm, 0, size, order); + start = drm_buddy_block_offset(mm->roots[i]); + __force_merge(mm, start, start + size, order); WARN_ON(!drm_buddy_block_is_free(mm->roots[i])); We do seem to have a testcase for this at the bottom of drm_test_buddy_alloc_clear(), so either it is not triggering the WARN_ON() here in which case we should maybe improve that. Or it is, but kunit doesn't treat that as a test failure? Maybe we can call something like kunit_fail_current_test() here if that WARN_ON is triggered? For reference our CI is just running all drm selftests with: /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig drm_block_free(mm, mm->roots[i]);
Re: [PATCH v7 1/2] drm/buddy: Add start address support to trim function
On 23/07/2024 14:25, Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Acked-by: Alex Deucher Acked-by: Christian König Given the comment from Marek that this is about non power-of-two alignment, this makes sense (although might be good to mention that somewhere in the commit message to make that clearer), Reviewed-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 6a8e45e9d0ec..103c185bb1c8 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, +u64 *start, u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block); + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { @@ -1066,7 +1085,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } while (1); /* Trim the allocated block to the required size */ - if (original_size != size) { + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && + original_size != size) { struct list_head *trim_list; LIST_HEAD(temp); u64 trim_size; @@ -1083,6 +1103,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } drm_buddy_block_trim(mm, +NULL, trim_size, trim_list); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..423b261ea743 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -150,7 +150,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, } while (remaining_size); if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { - if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks)) + if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) size = vres->base.size; } diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 2a74fa9d0ce5..9689a7c5dd36 100644 --- a/in
Re: [PATCH v7 1/2] drm/buddy: Add start address support to trim function
On 24/07/2024 02:35, Marek Olšák wrote: The reason is that our DCC requires 768K alignment in some cases. I haven't read this patch series, but one way to do that is to align to 256K, overallocate by 512K, and then not use either 0, 256K, or 512K at the beginning to get to 768K alignment. Ah, so we need a non power-of-two alignment. That makes sense, thanks. Marek On Tue, Jul 23, 2024, 11:04 Matthew Auld <mailto:matthew.a...@intel.com>> wrote: On 23/07/2024 14:43, Paneer Selvam, Arunpravin wrote: > Hi Matthew, > > Can we push this version for now as we need to mainline the DCC changes > ASAP, > while we continue our discussion and proceed to implement the permanent > solution > for address alignment? Yeah, we can always merge now and circle back around later, if this for sure helps your usecase and is needed asap. I just didn't fully get the idea for needing this interface, but likely I am missing something. > > Thanks, > Arun. > > On 7/23/2024 6:55 PM, Arunpravin Paneer Selvam wrote: >> - Add a new start parameter in trim function to specify exact >> address from where to start the trimming. This would help us >> in situations like if drivers would like to do address alignment >> for specific requirements. >> >> - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this >> flag to disable the allocator trimming part. This patch enables >> the drivers control trimming and they can do it themselves >> based on the application requirements. >> >> v1:(Matthew) >> - check new_start alignment with min chunk_size >> - use range_overflows() >> >> Signed-off-by: Arunpravin Paneer Selvam mailto:arunpravin.paneersel...@amd.com>> >> Acked-by: Alex Deucher mailto:alexander.deuc...@amd.com>> >> Acked-by: Christian König mailto:christian.koe...@amd.com>> >> --- >> drivers/gpu/drm/drm_buddy.c | 25 +++-- >> drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- >> include/drm/drm_buddy.h | 2 ++ >> 3 files changed, 26 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c >> index 6a8e45e9d0ec..103c185bb1c8 100644 >> --- a/drivers/gpu/drm/drm_buddy.c >> +++ b/drivers/gpu/drm/drm_buddy.c >> @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct >> drm_buddy *mm, >> * drm_buddy_block_trim - free unused pages >> * >> * @mm: DRM buddy manager >> + * @start: start address to begin the trimming. >> * @new_size: original size requested >> * @blocks: Input and output list of allocated blocks. >> * MUST contain single block as input to be trimmed. >> @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct >> drm_buddy *mm, >> * 0 on success, error code on failure. >> */ >> int drm_buddy_block_trim(struct drm_buddy *mm, >> + u64 *start, >> u64 new_size, >> struct list_head *blocks) >> { >> struct drm_buddy_block *parent; >> struct drm_buddy_block *block; >> + u64 block_start, block_end; >> LIST_HEAD(dfs); >> u64 new_start; >> int err; >> @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, >> struct drm_buddy_block, >> link); >> + block_start = drm_buddy_block_offset(block); >> + block_end = block_start + drm_buddy_block_size(mm, block); >> + >> if (WARN_ON(!drm_buddy_block_is_allocated(block))) >> return -EINVAL; >> @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, >> if (new_size == drm_buddy_block_size(mm, block)) >> return 0; >> + new_start = block_start; >> + if (start) { >> + new_start = *start; >> + >> + if (new_start < block_start) >> + return -EINVAL; >> + >> + if (!IS_ALIGNED(new_start, mm->chunk_size)) >> + return -EINVAL; >> + >> + if (range_overflows(new_start, new_size, block_end)) >> + return -EINVAL; >> +
Re: [PATCH v7 1/2] drm/buddy: Add start address support to trim function
On 23/07/2024 14:43, Paneer Selvam, Arunpravin wrote: Hi Matthew, Can we push this version for now as we need to mainline the DCC changes ASAP, while we continue our discussion and proceed to implement the permanent solution for address alignment? Yeah, we can always merge now and circle back around later, if this for sure helps your usecase and is needed asap. I just didn't fully get the idea for needing this interface, but likely I am missing something. Thanks, Arun. On 7/23/2024 6:55 PM, Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Acked-by: Alex Deucher Acked-by: Christian König --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 6a8e45e9d0ec..103c185bb1c8 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, + u64 *start, u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block); + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { @@ -1066,7 +1085,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } while (1); /* Trim the allocated block to the required size */ - if (original_size != size) { + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && + original_size != size) { struct list_head *trim_list; LIST_HEAD(temp); u64 trim_size; @@ -1083,6 +1103,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } drm_buddy_block_trim(mm, + NULL, trim_size, trim_list); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..423b261ea743 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -150,7 +150,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, } while (remaining_size); if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { - if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks)) + if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) size = vres->base.size; } diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 2a74fa9d0ce5..9689a7c5dd36 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -27,6 +27,7 @@ #define DRM_BUDDY_CONTIGUOUS_ALLOCATION
Re: [PATCH] drm/buddy: Add start address support to trim function
Hi, On 22/07/2024 12:41, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 7/19/2024 4:01 PM, Matthew Auld wrote: On 17/07/2024 16:02, Paneer Selvam, Arunpravin wrote: On 7/16/2024 3:34 PM, Matthew Auld wrote: On 16/07/2024 10:50, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 7/10/2024 6:20 PM, Matthew Auld wrote: On 10/07/2024 07:03, Paneer Selvam, Arunpravin wrote: Thanks Alex. Hi Matthew, Any comments? Do we not pass the required address alignment when allocating the pages in the first place? If address alignment is really useful, we can add that in the drm_buddy_alloc_blocks() function. I mean don't we already pass the min page size, which should give us matching physical address alignment? I think we don't need to align the address to the passed min_block_size value for all the contiguous buffers, so I thought that decision we can leave it to the drivers and they can achieve that through trim function in this kind of a specific request. I would have assumed it would be simpler to use min_block_size and then trim the size, if it's too big? That would then also take care of the try_harder case? For example, if the required contiguous size is 1MiB and min_block_size is 256KiB, to perform the address alignment of 256KiB, we might need to over-allocate at least to the min_block_size (say 256KiB). Now the size becomes 1280KiB and If we have 1M contig request then it should already be aligned to 256K and every other power-of-two < 1M. VRAM should start at offset zero, so 1M block will have 1M address alignment, and so should also be aligned to 256K, right? Or does "address alignment of 256KiB" mean something else here? To me it just means IS_ALIGNED(block_start, 256K). since the contiguous flag is enabled, we will round up the size to the next power of two and the size value becomes 2MiB. Next, in trimming we should round up the block start address to the min_block_size. May be we can keep the above mentioned operations under the flag combination DRM_BUDDY_CONTIGUOUS_ALLOCATION && DRM_BUDDY_ADDRESS_ALIGNMENT?. At the moment, we cannot support address alignment for try_harder allocations since in case of try_harder allocations we first traverse RHS to allocate the maximum possible and traverse LHS (here we align the LHS size to min_block_size) to allocate the remaining size. May be in case of DRM_BUDDY_ADDRESS_ALIGNMENT, we should first allocate LHS satisfying the address alignment requirement and then traverse RHS to allocate the remaining size if required? Also how are we dealing with the multi-block try_harder case? AFAICT we only allow trimming single block atm, or is it not possible to trigger that path here? Or are we handling that somehow? not possible to trigger that path here. only when we either over-allocate the LHS size and pass the multiple blocks to the trim function or implement the above mentioned method. https://patchwork.freedesktop.org/series/136150/ We are getting this sparse error from the Intel CI. Do you think these errors are introduced with this patches? I think it's safe to ignore, there seem to be other series with the same thing. Thanks. Thanks, Arun. Thanks, Arun. Thanks, Arun. On 7/9/2024 1:42 AM, Alex Deucher wrote: On Thu, Jul 4, 2024 at 4:40 AM Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Series is: Acked-by: Alex Deucher I'd like to take this series through the amdgpu tree if there are no objections as it's required for display buffers on some chips and I'd like to make sure it lands in 6.11. Thanks, Alex --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..8cebe1fa4e9d 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block
Re: [PATCH] drm/buddy: Add start address support to trim function
On 17/07/2024 16:02, Paneer Selvam, Arunpravin wrote: On 7/16/2024 3:34 PM, Matthew Auld wrote: On 16/07/2024 10:50, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 7/10/2024 6:20 PM, Matthew Auld wrote: On 10/07/2024 07:03, Paneer Selvam, Arunpravin wrote: Thanks Alex. Hi Matthew, Any comments? Do we not pass the required address alignment when allocating the pages in the first place? If address alignment is really useful, we can add that in the drm_buddy_alloc_blocks() function. I mean don't we already pass the min page size, which should give us matching physical address alignment? I think we don't need to align the address to the passed min_block_size value for all the contiguous buffers, so I thought that decision we can leave it to the drivers and they can achieve that through trim function in this kind of a specific request. I would have assumed it would be simpler to use min_block_size and then trim the size, if it's too big? That would then also take care of the try_harder case? Also how are we dealing with the multi-block try_harder case? AFAICT we only allow trimming single block atm, or is it not possible to trigger that path here? Or are we handling that somehow? https://patchwork.freedesktop.org/series/136150/ We are getting this sparse error from the Intel CI. Do you think these errors are introduced with this patches? I think it's safe to ignore, there seem to be other series with the same thing. Thanks, Arun. Thanks, Arun. Thanks, Arun. On 7/9/2024 1:42 AM, Alex Deucher wrote: On Thu, Jul 4, 2024 at 4:40 AM Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Series is: Acked-by: Alex Deucher I'd like to take this series through the amdgpu tree if there are no objections as it's required for display buffers on some chips and I'd like to make sure it lands in 6.11. Thanks, Alex --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..8cebe1fa4e9d 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, + u64 *start, u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block); + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_budd
Re: [PATCH] drm/buddy: Add start address support to trim function
On 16/07/2024 10:50, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 7/10/2024 6:20 PM, Matthew Auld wrote: On 10/07/2024 07:03, Paneer Selvam, Arunpravin wrote: Thanks Alex. Hi Matthew, Any comments? Do we not pass the required address alignment when allocating the pages in the first place? If address alignment is really useful, we can add that in the drm_buddy_alloc_blocks() function. I mean don't we already pass the min page size, which should give us matching physical address alignment? Thanks, Arun. Thanks, Arun. On 7/9/2024 1:42 AM, Alex Deucher wrote: On Thu, Jul 4, 2024 at 4:40 AM Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Series is: Acked-by: Alex Deucher I'd like to take this series through the amdgpu tree if there are no objections as it's required for display buffers on some chips and I'd like to make sure it lands in 6.11. Thanks, Alex --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..8cebe1fa4e9d 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, + u64 *start, u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block); + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { @@ -1066,7 +1085,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } while (1); /* Trim the allocated block to the required size */ - if (original_size != size) { + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && + original_size != size) { struct list_head *trim_list; LIST_HEAD(temp); u64 trim_size; @@ -1083,6 +1103,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } drm_buddy_block_trim(mm, + NULL, trim_size, trim_list); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..423b261ea743 100644 --- a/drivers/gp
Re: [PATCH] drm/buddy: Add start address support to trim function
On 10/07/2024 07:03, Paneer Selvam, Arunpravin wrote: Thanks Alex. Hi Matthew, Any comments? Do we not pass the required address alignment when allocating the pages in the first place? Thanks, Arun. On 7/9/2024 1:42 AM, Alex Deucher wrote: On Thu, Jul 4, 2024 at 4:40 AM Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. v1:(Matthew) - check new_start alignment with min chunk_size - use range_overflows() Signed-off-by: Arunpravin Paneer Selvam Series is: Acked-by: Alex Deucher I'd like to take this series through the amdgpu tree if there are no objections as it's required for display buffers on some chips and I'd like to make sure it lands in 6.11. Thanks, Alex --- drivers/gpu/drm/drm_buddy.c | 25 +++-- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc293..8cebe1fa4e9d 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, + u64 *start, u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block); + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { @@ -1066,7 +1085,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } while (1); /* Trim the allocated block to the required size */ - if (original_size != size) { + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && + original_size != size) { struct list_head *trim_list; LIST_HEAD(temp); u64 trim_size; @@ -1083,6 +1103,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } drm_buddy_block_trim(mm, + NULL, trim_size, trim_list); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..423b261ea743 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -150,7 +150,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, } while (remaining_size); if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { - if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks)) + if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blo
Re: [PATCH] drm/buddy: Add start address support to trim function
Hi, On 21/06/2024 06:29, Arunpravin Paneer Selvam wrote: - Add a new start parameter in trim function to specify exact address from where to start the trimming. This would help us in situations like if drivers would like to do address alignment for specific requirements. - Add a new flag DRM_BUDDY_TRIM_DISABLE. Drivers can use this flag to disable the allocator trimming part. This patch enables the drivers control trimming and they can do it themselves based on the application requirements. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 22 -- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 2 +- include/drm/drm_buddy.h | 2 ++ 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 6a8e45e9d0ec..287b6acb1637 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * drm_buddy_block_trim - free unused pages * * @mm: DRM buddy manager + * @start: start address to begin the trimming. * @new_size: original size requested * @blocks: Input and output list of allocated blocks. * MUST contain single block as input to be trimmed. @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, * 0 on success, error code on failure. */ int drm_buddy_block_trim(struct drm_buddy *mm, +u64 *start, I guess just wondering if this should be offset within or address. If it offset then zero be the valid default giving the existing behaviour. But hard to say without seeing the user for this. Are there some more patches to give some context for this usecase? u64 new_size, struct list_head *blocks) { struct drm_buddy_block *parent; struct drm_buddy_block *block; + u64 block_start, block_end; LIST_HEAD(dfs); u64 new_start; int err; @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, struct drm_buddy_block, link); + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + if (WARN_ON(!drm_buddy_block_is_allocated(block))) return -EINVAL; @@ -894,6 +900,17 @@ int drm_buddy_block_trim(struct drm_buddy *mm, if (new_size == drm_buddy_block_size(mm, block)) return 0; + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; In addition should check that the alignment of new_start is at least compatible with the min chunk_size. Otherwise I think bad stuff can happen. + + if ((new_start + new_size) > block_end) range_overflows() ? + return -EINVAL; + } + list_del(&block->link); mark_free(mm, block); mm->avail += drm_buddy_block_size(mm, block); @@ -904,7 +921,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, parent = block->parent; block->parent = NULL; - new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { @@ -1066,7 +1082,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } while (1); /* Trim the allocated block to the required size */ - if (original_size != size) { + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && + original_size != size) { struct list_head *trim_list; LIST_HEAD(temp); u64 trim_size; @@ -1083,6 +1100,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, } drm_buddy_block_trim(mm, +NULL, trim_size, trim_list); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..423b261ea743 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -150,7 +150,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, } while (remaining_size); if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { - if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks)) + if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) size = vres->base.size; } diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 2a74fa9d0ce5..9689a7c5dd36 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -27,6 +27,7 @@ #define DRM_BUDDY_CONTIGUOUS_ALLOCATION BIT(
Re: [PATCH v2] drm/buddy: Fix the warn on's during force merge
On 17/05/2024 14:50, Arunpravin Paneer Selvam wrote: Move the fallback and block incompatible checks above, so that we dont unnecessarily split the blocks and leaving the unmerged. This resolves the unnecessary warn on's thrown during force_merge call. v2:(Matthew) - Move the fallback and block incompatible checks above the contains check. Signed-off-by: Arunpravin Paneer Selvam Fixes: 96950929eb23 ("drm/buddy: Implement tracking clear page feature") Reviewed-by: Matthew Auld A follow up unit test to catch this edge case would be lovely. --- drivers/gpu/drm/drm_buddy.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 1daf778cf6fa..94f8c34fc293 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -524,11 +524,11 @@ __alloc_range_bias(struct drm_buddy *mm, continue; } + if (!fallback && block_incompatible(block, flags)) + continue; + if (contains(start, end, block_start, block_end) && order == drm_buddy_block_order(block)) { - if (!fallback && block_incompatible(block, flags)) - continue; - /* * Find the free block within the range. */
Re: [PATCH] drm/buddy: Merge back blocks in bias range function
On 17/05/2024 13:38, Arunpravin Paneer Selvam wrote: In bias range allocation, when we don't find the required blocks (i.e) on returning the -ENOSPC, we should merge back the split blocks. Otherwise, during force_merge we are flooded with warn on's due to block and its buddy are in same clear state (dirty or clear). Hence, renamed the force_merge with merge_blocks and passed a force_merge as a bool function parameter. Based on the requirement, say, in any normal situation we can call the merge_blocks passing the force_merge variable as false. And, in any memory cruch situation, we can call the merge_blocks passing the force_merge as true. This resolves the unnecessary warn on's thrown during force_merge call. Signed-off-by: Arunpravin Paneer Selvam Fixes: 96950929eb23 ("drm/buddy: Implement tracking clear page feature") --- drivers/gpu/drm/drm_buddy.c | 32 ++-- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 1daf778cf6fa..111f602f1359 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -161,10 +161,11 @@ static unsigned int __drm_buddy_free(struct drm_buddy *mm, return order; } -static int __force_merge(struct drm_buddy *mm, -u64 start, -u64 end, -unsigned int min_order) +static int __merge_blocks(struct drm_buddy *mm, + u64 start, + u64 end, + unsigned int min_order, + bool force_merge) { unsigned int order; int i; @@ -195,8 +196,9 @@ static int __force_merge(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) continue; - WARN_ON(drm_buddy_block_is_clear(block) == - drm_buddy_block_is_clear(buddy)); + if (force_merge) + WARN_ON(drm_buddy_block_is_clear(block) == + drm_buddy_block_is_clear(buddy)); /* * If the prev block is same as buddy, don't access the @@ -210,7 +212,7 @@ static int __force_merge(struct drm_buddy *mm, if (drm_buddy_block_is_clear(block)) mm->clear_avail -= drm_buddy_block_size(mm, block); - order = __drm_buddy_free(mm, block, true); + order = __drm_buddy_free(mm, block, force_merge); if (order >= min_order) return 0; } @@ -332,7 +334,7 @@ void drm_buddy_fini(struct drm_buddy *mm) for (i = 0; i < mm->n_roots; ++i) { order = ilog2(size) - ilog2(mm->chunk_size); - __force_merge(mm, 0, size, order); + __merge_blocks(mm, 0, size, order, true); WARN_ON(!drm_buddy_block_is_free(mm->roots[i])); drm_block_free(mm, mm->roots[i]); @@ -479,7 +481,7 @@ __alloc_range_bias(struct drm_buddy *mm, unsigned long flags, bool fallback) { - u64 req_size = mm->chunk_size << order; + u64 size, root_size, req_size = mm->chunk_size << order; struct drm_buddy_block *block; struct drm_buddy_block *buddy; LIST_HEAD(dfs); @@ -487,6 +489,7 @@ __alloc_range_bias(struct drm_buddy *mm, int i; end = end - 1; + size = mm->size; for (i = 0; i < mm->n_roots; ++i) list_add_tail(&mm->roots[i]->tmp_link, &dfs); @@ -548,6 +551,15 @@ __alloc_range_bias(struct drm_buddy *mm, list_add(&block->left->tmp_link, &dfs); } while (1); + /* Merge back the split blocks */ + for (i = 0; i < mm->n_roots; ++i) { + order = ilog2(size) - ilog2(mm->chunk_size); + __merge_blocks(mm, start, end, order, false); + + root_size = mm->chunk_size << order; + size -= root_size; + } Hmm, can't we just not split a given block if it is incompatible? Like say we are looking for cleared, there is not much point in splitting blocks that are dirty on this pass, right? What about moving the incompatible check earlier like: if (!fallback && block_incompatible(block) continue; Would that not fix the issue? + return ERR_PTR(-ENOSPC); err_undo: @@ -1026,7 +1038,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, if (order-- == min_order) { /* Try allocation through force merge method */ if (mm->clear_avail && - !__force_merge(mm, start, end, min_order)) { + !__merge_blocks(mm, start, end, min_order, true)) { block = __drm_buddy_alloc_blocks(mm, star
Re: [PATCH v2 2/2] drm/tests: Add a unit test for range bias allocation
On 13/05/2024 16:11, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 5/13/2024 1:49 PM, Matthew Auld wrote: On 12/05/2024 08:59, Arunpravin Paneer Selvam wrote: Allocate cleared blocks in the bias range when the DRM buddy's clear avail is zero. This will validate the bias range allocation in scenarios like system boot when no cleared blocks are available and exercise the fallback path too. The resulting blocks should always be dirty. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/tests/drm_buddy_test.c | 35 ++ 1 file changed, 35 insertions(+) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index e3b50e240d36..a194f271bc55 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -26,6 +26,8 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test) u32 mm_size, ps, bias_size, bias_start, bias_end, bias_rem; DRM_RND_STATE(prng, random_seed); unsigned int i, count, *order; + struct drm_buddy_block *block; + unsigned long flags; struct drm_buddy mm; LIST_HEAD(allocated); @@ -222,6 +224,39 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test) drm_buddy_free_list(&mm, &allocated, 0); drm_buddy_fini(&mm); + + /* + * Allocate cleared blocks in the bias range when the DRM buddy's clear avail is + * zero. This will validate the bias range allocation in scenarios like system boot + * when no cleared blocks are available and exercise the fallback path too. The resulting + * blocks should always be dirty. + */ + + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps), + "buddy_init failed\n"); + mm.clear_avail = 0; Should already be zero, right? Maybe make this an assert instead? No, since the mm declared as a local variable in the test case, mm.clear_avail is not zero. That sounds like a bug IMO. The init() should initialize it, like it does for mm.avail and everything else. + + bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); + bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); + bias_end = max(bias_end, bias_start + ps); + bias_rem = bias_end - bias_start; + + flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION; + u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); u32 declaration should be moved to above? Sure. Thanks, Arun. Otherwise, Reviewed-by: Matthew Auld + + KUNIT_ASSERT_FALSE_MSG(test, + drm_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &allocated, + flags), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + + list_for_each_entry(block, &allocated, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); + + drm_buddy_free_list(&mm, &allocated, 0); + drm_buddy_fini(&mm); } static void drm_test_buddy_alloc_clear(struct kunit *test)
Re: [PATCH v2 2/2] drm/tests: Add a unit test for range bias allocation
On 12/05/2024 08:59, Arunpravin Paneer Selvam wrote: Allocate cleared blocks in the bias range when the DRM buddy's clear avail is zero. This will validate the bias range allocation in scenarios like system boot when no cleared blocks are available and exercise the fallback path too. The resulting blocks should always be dirty. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/tests/drm_buddy_test.c | 35 ++ 1 file changed, 35 insertions(+) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index e3b50e240d36..a194f271bc55 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -26,6 +26,8 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test) u32 mm_size, ps, bias_size, bias_start, bias_end, bias_rem; DRM_RND_STATE(prng, random_seed); unsigned int i, count, *order; + struct drm_buddy_block *block; + unsigned long flags; struct drm_buddy mm; LIST_HEAD(allocated); @@ -222,6 +224,39 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test) drm_buddy_free_list(&mm, &allocated, 0); drm_buddy_fini(&mm); + + /* +* Allocate cleared blocks in the bias range when the DRM buddy's clear avail is +* zero. This will validate the bias range allocation in scenarios like system boot +* when no cleared blocks are available and exercise the fallback path too. The resulting +* blocks should always be dirty. +*/ + + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps), + "buddy_init failed\n"); + mm.clear_avail = 0; Should already be zero, right? Maybe make this an assert instead? + + bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); + bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); + bias_end = max(bias_end, bias_start + ps); + bias_rem = bias_end - bias_start; + + flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION; + u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); u32 declaration should be moved to above? Otherwise, Reviewed-by: Matthew Auld + + KUNIT_ASSERT_FALSE_MSG(test, + drm_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &allocated, + flags), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + + list_for_each_entry(block, &allocated, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); + + drm_buddy_free_list(&mm, &allocated, 0); + drm_buddy_fini(&mm); } static void drm_test_buddy_alloc_clear(struct kunit *test)
[PATCH 02/20] drm/amdgpu: don't trample pdev drvdata
The driver release callback is called when a particular drm_device goes away, just like with drmm, so here we should never nuke the pdev drvdata pointer, since that could already be pointing to a new drvdata. For example something hotunplugs the device, for which we have an open driver fd, keeping the drm_device alive, and in the meantime the same physical device is re-attached to a new drm_device therefore setting drvdata again. Once the original drm_device goes away, we might then call the release which then incorrectly tramples the drvdata. The driver core will already nuke the pointer for us when the pci device is removed, so should be safe to simply drop. Alternative would be to move to the driver pci remove callback. Signed-off-by: Matthew Auld Cc: Christian König Cc: Daniel Vetter Cc: amd-gfx@lists.freedesktop.org --- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index a0ea6fe8d060..d5fed007c698 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -1450,7 +1450,6 @@ void amdgpu_driver_release_kms(struct drm_device *dev) struct amdgpu_device *adev = drm_to_adev(dev); amdgpu_device_fini_sw(adev); - pci_set_drvdata(adev->pdev, NULL); } /* -- 2.45.0
Re: [PATCH] drm/buddy: Fix the range bias clear memory allocation issue
On 06/05/2024 14:38, Arunpravin Paneer Selvam wrote: Problem statement: During the system boot time, an application request for the bulk volume of cleared range bias memory when the clear_avail is zero, we dont fallback into normal allocation method as we had an unnecessary clear_avail check which prevents the fallback method leads to fb allocation failure following system goes into unresponsive state. Solution: Remove the unnecessary clear_avail check in the range bias allocation function. Signed-off-by: Arunpravin Paneer Selvam Fixes: 96950929eb23 ("drm/buddy: Implement tracking clear page feature") Reviewed-by: Matthew Auld
Re: [PATCH v10 1/3] drm/buddy: Implement tracking clear page feature
On 08/04/2024 16:16, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) v4: - rename the function drm_buddy_defrag with __force_merge. - Include __force_merge directly in drm buddy file and remove the defrag use in amdgpu driver. - Remove list_empty() check(Matthew) - Remove unnecessary space, headers and placement of new variables(Matthew) - Add a unit test case(Matthew) v5: - remove force merge support to actual range allocation and not to bail out when contains && split(Matthew) - add range support to force merge function. Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 430 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 28 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 16 +- 6 files changed, 368 insertions(+), 122 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 5ebdd6f8f36e..83dbe252f727 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,8 +38,8 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } -static void list_insert_sorted(struct drm_buddy *mm, - struct drm_buddy_block *block) +static void list_insert(struct drm_buddy *mm, + struct drm_buddy_block *block)
Re: [PATCH v10 3/3] drm/tests: Add a test case for drm buddy clear allocation
On 08/04/2024 16:16, Arunpravin Paneer Selvam wrote: Add a new test case for the drm buddy clear and dirty allocation. v2:(Matthew) - make size as u32 - rename PAGE_SIZE with SZ_4K - dont fragment the address space for all the order allocation iterations. we can do it once and just increment and allocate the size. - create new mm with non power-of-two size to ensure the multi-root force_merge during fini. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/tests/drm_buddy_test.c | 141 + 1 file changed, 141 insertions(+) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index 4621a860cb05..b07f132f2835 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -224,6 +224,146 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test) drm_buddy_fini(&mm); } +static void drm_test_buddy_alloc_clear(struct kunit *test) +{ + unsigned long n_pages, total, i = 0; + const unsigned long ps = SZ_4K; + struct drm_buddy_block *block; + const int max_order = 12; + LIST_HEAD(allocated); + struct drm_buddy mm; + unsigned int order; + u32 mm_size, size; + LIST_HEAD(dirty); + LIST_HEAD(clean); + + mm_size = SZ_4K << max_order; + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + /* +* Idea is to allocate and free some random portion of the address space, +* returning those pages as non-dirty and randomly alternate between +* requesting dirty and non-dirty pages (not going over the limit +* we freed as non-dirty), putting that into two separate lists. +* Loop over both lists at the end checking that the dirty list +* is indeed all dirty pages and vice versa. Free it all again, +* keeping the dirty/clear status. +*/ + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 5 * ps, ps, &allocated, + DRM_BUDDY_TOPDOWN_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 5 * ps); + drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); + + n_pages = 10; + do { + unsigned long flags; + struct list_head *list; + int slot = i % 2; + + if (slot == 0) { + list = &dirty; + flags = 0; + } else { + list = &clean; + flags = DRM_BUDDY_CLEAR_ALLOCATION; + } + + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, + flags), + "buddy_alloc hit an error size=%lu\n", ps); + } while (++i < n_pages); + + list_for_each_entry(block, &clean, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true); + + list_for_each_entry(block, &dirty, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); + + drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); + + /* +* Trying to go over the clear limit for some allocation. +* The allocation should never fail with reasonable page-size. +*/ + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 10 * ps, ps, &clean, + DRM_BUDDY_CLEAR_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 10 * ps); + + drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); + drm_buddy_free_list(&mm, &dirty, 0); + drm_buddy_fini(&mm); + + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); + + /* +* Create a new mm. Intentionally fragment the address space by creating +* two alternating lists. Free both lists, one as dirty the other as clean. +* Try to allocate double the previous size with matching min_page_size. The +* allocation should never fail as it calls the force_merge. Also check that +* the page is always dirty after force_merge. Free the page as dirty, then +* repeat the whole thing, increment the order until we hit the max_order. +*/ + + i = 0; + n_pages = mm_size / ps; + do { + struct list_head *list; +
Re: [PATCH v9 1/3] drm/buddy: Implement tracking clear page feature
On 01/04/2024 12:07, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 3/28/2024 10:18 PM, Matthew Auld wrote: On 28/03/2024 16:07, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 3/26/2024 11:39 PM, Matthew Auld wrote: On 18/03/2024 21:40, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) v4: - rename the function drm_buddy_defrag with __force_merge. - Include __force_merge directly in drm buddy file and remove the defrag use in amdgpu driver. - Remove list_empty() check(Matthew) - Remove unnecessary space, headers and placement of new variables(Matthew) - Add a unit test case(Matthew) Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 427 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c | 18 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 16 +- 6 files changed, 360 insertions(+), 117 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index c4222b886db7..625a30a6b855 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,8 +38,8 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } -static void list_insert_sorted(struct drm_buddy *mm, - struct drm_buddy_block *block) +static void list_insert(struct drm_buddy *mm, + struct drm_buddy_block *block)
Re: [PATCH v9 1/3] drm/buddy: Implement tracking clear page feature
On 28/03/2024 16:07, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 3/26/2024 11:39 PM, Matthew Auld wrote: On 18/03/2024 21:40, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) v4: - rename the function drm_buddy_defrag with __force_merge. - Include __force_merge directly in drm buddy file and remove the defrag use in amdgpu driver. - Remove list_empty() check(Matthew) - Remove unnecessary space, headers and placement of new variables(Matthew) - Add a unit test case(Matthew) Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 427 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c | 18 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 16 +- 6 files changed, 360 insertions(+), 117 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index c4222b886db7..625a30a6b855 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,8 +38,8 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } -static void list_insert_sorted(struct drm_buddy *mm, - struct drm_buddy_block *block) +static void list_insert(struct drm_buddy *mm, + struct drm_buddy_block *block) { struct drm_buddy_block *node; struct list_head *head; @@ -57,6 +57,16 @@ static void list_insert_sorte
Re: [PATCH v9 1/3] drm/buddy: Implement tracking clear page feature
On 18/03/2024 21:40, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) v4: - rename the function drm_buddy_defrag with __force_merge. - Include __force_merge directly in drm buddy file and remove the defrag use in amdgpu driver. - Remove list_empty() check(Matthew) - Remove unnecessary space, headers and placement of new variables(Matthew) - Add a unit test case(Matthew) Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 427 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 18 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 16 +- 6 files changed, 360 insertions(+), 117 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index c4222b886db7..625a30a6b855 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,8 +38,8 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } -static void list_insert_sorted(struct drm_buddy *mm, - struct drm_buddy_block *block) +static void list_insert(struct drm_buddy *mm, + struct drm_buddy_block *block) { struct drm_buddy_block *node; struct list_head *head; @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy *mm, __list_add(&bl
Re: [PATCH v9 3/3] drm/tests: Add a test case for drm buddy clear allocation
On 18/03/2024 21:40, Arunpravin Paneer Selvam wrote: Add a new test case for the drm buddy clear and dirty allocation. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/tests/drm_buddy_test.c | 127 + 1 file changed, 127 insertions(+) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index 454ad9952f56..d355a6e61893 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -19,6 +19,132 @@ static inline u64 get_size(int order, u64 chunk_size) return (1 << order) * chunk_size; } +static void drm_test_buddy_alloc_clear(struct kunit *test) +{ + unsigned long n_pages, total, i = 0; + const unsigned long ps = SZ_4K; + struct drm_buddy_block *block; + const int max_order = 12; + LIST_HEAD(allocated); + struct drm_buddy mm; + unsigned int order; + u64 mm_size, size; Maybe just make these two u32 or unsigned long. That should be big enough, plus avoids any kind of 32b compilation bugs below. + LIST_HEAD(dirty); + LIST_HEAD(clean); + + mm_size = PAGE_SIZE << max_order; s/PAGE_SIZE/SZ_4K/ below also. + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + /** Drop the extra *, since is not actual kernel-doc. Below also. +* Idea is to allocate and free some random portion of the address space, +* returning those pages as non-dirty and randomly alternate between +* requesting dirty and non-dirty pages (not going over the limit +* we freed as non-dirty), putting that into two separate lists. +* Loop over both lists at the end checking that the dirty list +* is indeed all dirty pages and vice versa. Free it all again, +* keeping the dirty/clear status. +*/ + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 5 * ps, ps, &allocated, + DRM_BUDDY_TOPDOWN_ALLOCATION), + "buddy_alloc hit an error size=%u\n", 5 * ps); + drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); + + n_pages = 10; + do { + unsigned long flags; + struct list_head *list; + int slot = i % 2; + + if (slot == 0) { + list = &dirty; + flags = 0; + } else if (slot == 1) { Could just be else { + list = &clean; + flags = DRM_BUDDY_CLEAR_ALLOCATION; + } + + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, + flags), + "buddy_alloc hit an error size=%u\n", ps); + } while (++i < n_pages); + + list_for_each_entry(block, &clean, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true); + + list_for_each_entry(block, &dirty, link) + KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); + + drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); + + /** +* Trying to go over the clear limit for some allocation. +* The allocation should never fail with reasonable page-size. +*/ + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 10 * ps, ps, &clean, + DRM_BUDDY_CLEAR_ALLOCATION), + "buddy_alloc hit an error size=%u\n", 10 * ps); + + drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); + drm_buddy_free_list(&mm, &dirty, 0); + drm_buddy_fini(&mm); + + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); + + /** +* Create a new mm. Intentionally fragment the address space by creating +* two alternating lists. Free both lists, one as dirty the other as clean. +* Try to allocate double the previous size with matching min_page_size. The +* allocation should never fail as it calls the force_merge. Also check that +* the page is always dirty after force_merge. Free the page as dirty, then +* repeat the whole thing, increment the order until we hit the max_order. +*/ + + order = 1; + do { + size = PAGE_SIZE << order; + i = 0; + n_pages = mm_size / ps; +
Re: [PATCH v8 1/3] drm/buddy: Implement tracking clear page feature
On 07/03/2024 12:25, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 3/6/2024 11:19 PM, Matthew Auld wrote: On 04/03/2024 16:32, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld Is there a unit test for this? What about maybe something roughly like: - Pick small random mm_size which is not always power-of-two. - Allocate and free some random portion of the address space, returning those pages as non-dirty. Then do another cycle and randomly alternate between requesting dirty and non-dirty pages (not going over the limit you freed as non-dirty), putting that into two separate lists. Loop over both lists at the end checking that the dirty list is indeed all dirty pages and vice versa. Free it all again, keeping the dirty/clear status. - Also try to go over the clear limit for some allocation. The allocation should never fail with reasonable page-size. - Test the defrag/force_merge interface. Clean the mm or create new one. Intentionally fragment the address space, by creating two alternating lists. Free both lists, one as dirty the other as clean. Try to allocate double the previous size with matching min_page_size. Should fail. Call force_merge. Should now succeed. Also check that the page is always dirty after force_merge. Free the page as dirty, then repeat the whole thing, doubling the page-size until you hit max_order. - Make sure we also call fini() with some part of the address space left as non-dirty. Should not trigger any warnings. I think would be good to consider, but not a blocker or anything. Some comments below, otherwise I think looks good. Yes. It is good to have a unit test for this feature. I will send the patch. --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 294 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c | 18 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 22 +- 6 files changed, 290 insertions(+), 60 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_
Re: [PATCH v8 1/3] drm/buddy: Implement tracking clear page feature
On 04/03/2024 16:32, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. - Add a function to support defragmentation. v1: - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list(Christian) - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy)(Christian) - Defragment the memory beginning from min_order till the required memory space is available. v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. v3: - fix Gitlab user reported lockup issue. - Keep DRM_BUDDY_HEADER_CLEAR define sorted(Matthew) - modify to pass the root order instead max_order in fini() function(Matthew) - change bool 1 to true(Matthew) - add check if min_block_size is power of 2(Matthew) - modify the min_block_size datatype to u64(Matthew) Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Suggested-by: Matthew Auld Is there a unit test for this? What about maybe something roughly like: - Pick small random mm_size which is not always power-of-two. - Allocate and free some random portion of the address space, returning those pages as non-dirty. Then do another cycle and randomly alternate between requesting dirty and non-dirty pages (not going over the limit you freed as non-dirty), putting that into two separate lists. Loop over both lists at the end checking that the dirty list is indeed all dirty pages and vice versa. Free it all again, keeping the dirty/clear status. - Also try to go over the clear limit for some allocation. The allocation should never fail with reasonable page-size. - Test the defrag/force_merge interface. Clean the mm or create new one. Intentionally fragment the address space, by creating two alternating lists. Free both lists, one as dirty the other as clean. Try to allocate double the previous size with matching min_page_size. Should fail. Call force_merge. Should now succeed. Also check that the page is always dirty after force_merge. Free the page as dirty, then repeat the whole thing, doubling the page-size until you hit max_order. - Make sure we also call fini() with some part of the address space left as non-dirty. Should not trigger any warnings. I think would be good to consider, but not a blocker or anything. Some comments below, otherwise I think looks good. --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 294 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 18 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 22 +- 6 files changed, 290 insertions(+), 60 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +
Re: [PATCH v7 3/3] drm/buddy: Add defragmentation support
On 04/03/2024 12:22, Paneer Selvam, Arunpravin wrote: Hi Matthew, On 2/22/2024 12:12 AM, Matthew Auld wrote: On 21/02/2024 12:18, Arunpravin Paneer Selvam wrote: Add a function to support defragmentation. v1: - Defragment the memory beginning from min_order till the required memory space is available. v2(Matthew): - add amdgpu user for defragmentation - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 17 +++- drivers/gpu/drm/drm_buddy.c | 93 +--- include/drm/drm_buddy.h | 3 + 3 files changed, 97 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index e494f5bf136a..cff8a526c622 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -533,8 +533,21 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, min_block_size, &vres->blocks, vres->flags); - if (unlikely(r)) - goto error_free_blocks; + if (unlikely(r)) { + if (r == -ENOSPC) { + drm_buddy_defrag(mm, min_block_size); + r = drm_buddy_alloc_blocks(mm, fpfn, + lpfn, + size, + min_block_size, + &vres->blocks, + vres->flags); + if (unlikely(r)) + goto error_free_blocks; + } else { + goto error_free_blocks; + } + } if (size > remaining_size) remaining_size = 0; diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 18e004fa39d3..56bd1560fbcd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -203,6 +203,8 @@ void drm_buddy_fini(struct drm_buddy *mm) drm_block_free(mm, mm->roots[i]); } + drm_buddy_defrag(mm, mm->chunk_size << mm->max_order); I think this needs to be called higher up, otherwise we blow up with the WARN, plus we just freed the root(s). There is also the case with non-power-of-two VRAM size, in which case you get multiple roots and max_order is just the largest root and not entire address space. I guess do this in the loop above and use the root order instead? Also this should be done as part of the first patch and then in this patch it is just a case of exporting it. Every commit should ideally be functional by itself. You mean we move the above change in drm_buddy_fini function and drm_buddy_defrag function as part of first patch. And just we add export function and add amdgpu user in this patch. Is my understanding correct? Yeah, I think that makes sense. Thanks, Arun. + WARN_ON(mm->avail != mm->size); kfree(mm->roots); @@ -276,25 +278,39 @@ drm_get_buddy(struct drm_buddy_block *block) } EXPORT_SYMBOL(drm_get_buddy); -static void __drm_buddy_free(struct drm_buddy *mm, - struct drm_buddy_block *block) +static unsigned int __drm_buddy_free(struct drm_buddy *mm, + struct drm_buddy_block *block, + bool defrag) { + unsigned int order, block_order; struct drm_buddy_block *parent; + block_order = drm_buddy_block_order(block); + while ((parent = block->parent)) { - struct drm_buddy_block *buddy; + struct drm_buddy_block *buddy = NULL; buddy = __get_buddy(block); if (!drm_buddy_block_is_free(buddy)) break; - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; + if (!defrag) { + /* + * Check the block and its buddy clear state and exit + * the loop if they both have the dissimilar state. + */ + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + } + + WARN_ON(defrag && + (drm_buddy_block_is_clear(block) == + drm_buddy_block_is_clear(buddy))); list_del(&buddy->link); @@ -304,8 +320,57 @@ static void __drm_buddy_free(struct drm_buddy *mm, block = parent;
Re: [PATCH v7 3/3] drm/buddy: Add defragmentation support
On 21/02/2024 12:18, Arunpravin Paneer Selvam wrote: Add a function to support defragmentation. v1: - Defragment the memory beginning from min_order till the required memory space is available. v2(Matthew): - add amdgpu user for defragmentation - add a warning if the two blocks are incompatible on defragmentation - call full defragmentation in the fini() function - place a condition to test if min_order is equal to 0 - replace the list with safe_reverse() variant as we might remove the block from the list. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 17 +++- drivers/gpu/drm/drm_buddy.c | 93 +--- include/drm/drm_buddy.h | 3 + 3 files changed, 97 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index e494f5bf136a..cff8a526c622 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -533,8 +533,21 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, min_block_size, &vres->blocks, vres->flags); - if (unlikely(r)) - goto error_free_blocks; + if (unlikely(r)) { + if (r == -ENOSPC) { + drm_buddy_defrag(mm, min_block_size); + r = drm_buddy_alloc_blocks(mm, fpfn, + lpfn, + size, + min_block_size, + &vres->blocks, + vres->flags); + if (unlikely(r)) + goto error_free_blocks; + } else { + goto error_free_blocks; + } + } if (size > remaining_size) remaining_size = 0; diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 18e004fa39d3..56bd1560fbcd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -203,6 +203,8 @@ void drm_buddy_fini(struct drm_buddy *mm) drm_block_free(mm, mm->roots[i]); } + drm_buddy_defrag(mm, mm->chunk_size << mm->max_order); I think this needs to be called higher up, otherwise we blow up with the WARN, plus we just freed the root(s). There is also the case with non-power-of-two VRAM size, in which case you get multiple roots and max_order is just the largest root and not entire address space. I guess do this in the loop above and use the root order instead? Also this should be done as part of the first patch and then in this patch it is just a case of exporting it. Every commit should ideally be functional by itself. + WARN_ON(mm->avail != mm->size); kfree(mm->roots); @@ -276,25 +278,39 @@ drm_get_buddy(struct drm_buddy_block *block) } EXPORT_SYMBOL(drm_get_buddy); -static void __drm_buddy_free(struct drm_buddy *mm, -struct drm_buddy_block *block) +static unsigned int __drm_buddy_free(struct drm_buddy *mm, +struct drm_buddy_block *block, +bool defrag) { + unsigned int order, block_order; struct drm_buddy_block *parent; + block_order = drm_buddy_block_order(block); + while ((parent = block->parent)) { - struct drm_buddy_block *buddy; + struct drm_buddy_block *buddy = NULL; buddy = __get_buddy(block); if (!drm_buddy_block_is_free(buddy)) break; - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; + if (!defrag) { + /* +* Check the block and its buddy clear state and exit +* the loop if they both have the dissimilar state. +*/ + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + } + + WARN_ON(defrag && + (drm_buddy_block_is_clear(block)
Re: [PATCH v6 1/3] drm/buddy: Implement tracking clear page feature
On 21/02/2024 12:40, Paneer Selvam, Arunpravin wrote: On 2/16/2024 5:33 PM, Matthew Auld wrote: On 08/02/2024 15:49, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. v1: (Christian) - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list. - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy) v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Probably needs a new unit test. I think we are missing something to forcefully re-merge everything at fini()? In theory we can just call the defrag routine. Otherwise we might trigger various warnings since the root(s) might still be split. Also one nit below. Otherwise I think looks good. --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 192 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c | 10 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 18 +- 6 files changed, 187 insertions(+), 49 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..33ad0cfbd54c 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy *mm, __list_add(&block->link, node->link.prev, &node->link); } +static void clear_reset(struct drm_buddy_block *block) +{ + block->header &= ~DRM_BUDDY_HEADER_CLEAR; +} + +static void mark_cleared(struct drm_buddy_block *block) +{ + block->header |= DRM_BUDDY_HEADER_CLEAR; +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -223,6 +233,12 @@ static int split_block(struct drm_buddy *mm, mark_free(mm, block->left); mark_free(mm, block->right); + if (drm_buddy_block_is_clear(block)) { + mark_cleared(block->left); + mark_cleared(block->right); + clear_reset(block); + } + mark_split(block); return 0; @@ -273,6 +289,13 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; + + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + list_del(&buddy->link); drm_block_free(m
Re: [PATCH v6 3/3] drm/buddy: Add defragmentation support
On 16/02/2024 14:02, Christian König wrote: Am 16.02.24 um 14:21 schrieb Matthew Auld: On 16/02/2024 12:33, Christian König wrote: Am 16.02.24 um 13:23 schrieb Matthew Auld: On 08/02/2024 15:50, Arunpravin Paneer Selvam wrote: Add a function to support defragmentation. v1: Defragment the memory beginning from min_order till the required memory space is available. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 67 +++-- include/drm/drm_buddy.h | 3 ++ No users? Other question is how can a buddy allocator fragment in the first place? The fragmentation is due to pages now being tracked as dirty/clear. Should the allocator merge together a page that is dirty with a page that is cleared? When should it do that? User wants to be able to keep the two separate if possible. For example, freeing one single dirty page can dirty a huge swathe of your already cleared pages if they are merged together. Or do you have some some other ideas here? Sorry, that was not what I meant. I should probably have been clearer. That dirty and clean pages are now kept separated is obvious, but why do you need to de-fragment them at some point? Ah, right. At the very least we need to do something similar to this at fini(), just to ensure we properly merge everything back together so we can correctly tear down the mm. Outside of that the thinking was that it might be useful to call when allocating larger min page-sizes. You might now be failing the allocation due to fragmentation, and so in some cases might be better off running some kind of defrag step first, instead of failing the allocation and trying to evict stuff. Anyway, if that is not a concern for amdgpu, then we just need to handle the fini() case and can keep this internal. Christian. Christian. 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 33ad0cfbd54c..fac423d2cb73 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -276,10 +276,12 @@ drm_get_buddy(struct drm_buddy_block *block) } EXPORT_SYMBOL(drm_get_buddy); -static void __drm_buddy_free(struct drm_buddy *mm, - struct drm_buddy_block *block) +static unsigned int __drm_buddy_free(struct drm_buddy *mm, + struct drm_buddy_block *block, + bool defrag) { struct drm_buddy_block *parent; + unsigned int order; while ((parent = block->parent)) { struct drm_buddy_block *buddy; @@ -289,12 +291,14 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; + if (!defrag) { + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + } Maybe check if the two blocks are incompatible and chuck a warn if they are not? Main thing is not to hide issues with split blocks that should have been merged before. list_del(&buddy->link); @@ -304,8 +308,49 @@ static void __drm_buddy_free(struct drm_buddy *mm, block = parent; } + order = drm_buddy_block_order(block); mark_free(mm, block); + + return order; +} + +/** + * drm_buddy_defrag - Defragmentation routine + * + * @mm: DRM buddy manager + * @min_order: minimum order in the freelist to begin + * the defragmentation process + * + * Driver calls the defragmentation function when the + * requested memory allocation returns -ENOSPC. + */ +void drm_buddy_defrag(struct drm_buddy *mm, + unsigned int min_order) Just wondering if we need "full defag" also? We would probably need to call this at fini() anyway. +{ + struct drm_buddy_block *block; + struct list_head *list; + unsigned int order; + int i; + + if (min_order > mm->max_order) + return; + + for (i = min_order - 1; i >= 0; i--) { Need to be careful with min_order = 0 ? + list = &mm->free_list[i]; + if (list_empty(list)) + continue; + + list_for_each_entry_reverse(block, list, link) { Don't we need the safe_reverse() variant here, since this is removing from the list? + if (!block->parent) + continue; + + order = __drm_buddy_free(mm, block, 1); + if (order >= min_order) + return; + } + } } +EXPORT_SYMBOL(drm_buddy_defrag); /** * drm_buddy_free_block - free a block @@ -321,7 +366,7 @@ void drm_buddy_free_block(struct drm_budd
Re: [PATCH v6 3/3] drm/buddy: Add defragmentation support
On 16/02/2024 12:33, Christian König wrote: Am 16.02.24 um 13:23 schrieb Matthew Auld: On 08/02/2024 15:50, Arunpravin Paneer Selvam wrote: Add a function to support defragmentation. v1: Defragment the memory beginning from min_order till the required memory space is available. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 67 +++-- include/drm/drm_buddy.h | 3 ++ No users? Other question is how can a buddy allocator fragment in the first place? The fragmentation is due to pages now being tracked as dirty/clear. Should the allocator merge together a page that is dirty with a page that is cleared? When should it do that? User wants to be able to keep the two separate if possible. For example, freeing one single dirty page can dirty a huge swathe of your already cleared pages if they are merged together. Or do you have some some other ideas here? Christian. 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 33ad0cfbd54c..fac423d2cb73 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -276,10 +276,12 @@ drm_get_buddy(struct drm_buddy_block *block) } EXPORT_SYMBOL(drm_get_buddy); -static void __drm_buddy_free(struct drm_buddy *mm, - struct drm_buddy_block *block) +static unsigned int __drm_buddy_free(struct drm_buddy *mm, + struct drm_buddy_block *block, + bool defrag) { struct drm_buddy_block *parent; + unsigned int order; while ((parent = block->parent)) { struct drm_buddy_block *buddy; @@ -289,12 +291,14 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; + if (!defrag) { + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + } Maybe check if the two blocks are incompatible and chuck a warn if they are not? Main thing is not to hide issues with split blocks that should have been merged before. list_del(&buddy->link); @@ -304,8 +308,49 @@ static void __drm_buddy_free(struct drm_buddy *mm, block = parent; } + order = drm_buddy_block_order(block); mark_free(mm, block); + + return order; +} + +/** + * drm_buddy_defrag - Defragmentation routine + * + * @mm: DRM buddy manager + * @min_order: minimum order in the freelist to begin + * the defragmentation process + * + * Driver calls the defragmentation function when the + * requested memory allocation returns -ENOSPC. + */ +void drm_buddy_defrag(struct drm_buddy *mm, + unsigned int min_order) Just wondering if we need "full defag" also? We would probably need to call this at fini() anyway. +{ + struct drm_buddy_block *block; + struct list_head *list; + unsigned int order; + int i; + + if (min_order > mm->max_order) + return; + + for (i = min_order - 1; i >= 0; i--) { Need to be careful with min_order = 0 ? + list = &mm->free_list[i]; + if (list_empty(list)) + continue; + + list_for_each_entry_reverse(block, list, link) { Don't we need the safe_reverse() variant here, since this is removing from the list? + if (!block->parent) + continue; + + order = __drm_buddy_free(mm, block, 1); + if (order >= min_order) + return; + } + } } +EXPORT_SYMBOL(drm_buddy_defrag); /** * drm_buddy_free_block - free a block @@ -321,7 +366,7 @@ void drm_buddy_free_block(struct drm_buddy *mm, if (drm_buddy_block_is_clear(block)) mm->clear_avail += drm_buddy_block_size(mm, block); - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); } EXPORT_SYMBOL(drm_buddy_free_block); @@ -470,7 +515,7 @@ __alloc_range_bias(struct drm_buddy *mm, if (buddy && (drm_buddy_block_is_free(block) && drm_buddy_block_is_free(buddy))) - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); return ERR_PTR(err); } @@ -588,7 +633,7 @@ alloc_from_freelist(struct drm_buddy *mm, err_undo: if (tmp != order) - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); return ERR_PTR(err); } @@ -668,7 +713,7 @@ static int __alloc_range(struct drm_buddy *mm, if (buddy && (drm_buddy_block_is_free(block) &&am
Re: [PATCH v6 3/3] drm/buddy: Add defragmentation support
On 08/02/2024 15:50, Arunpravin Paneer Selvam wrote: Add a function to support defragmentation. v1: Defragment the memory beginning from min_order till the required memory space is available. Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld --- drivers/gpu/drm/drm_buddy.c | 67 +++-- include/drm/drm_buddy.h | 3 ++ No users? 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 33ad0cfbd54c..fac423d2cb73 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -276,10 +276,12 @@ drm_get_buddy(struct drm_buddy_block *block) } EXPORT_SYMBOL(drm_get_buddy); -static void __drm_buddy_free(struct drm_buddy *mm, -struct drm_buddy_block *block) +static unsigned int __drm_buddy_free(struct drm_buddy *mm, +struct drm_buddy_block *block, +bool defrag) { struct drm_buddy_block *parent; + unsigned int order; while ((parent = block->parent)) { struct drm_buddy_block *buddy; @@ -289,12 +291,14 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; + if (!defrag) { + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + } Maybe check if the two blocks are incompatible and chuck a warn if they are not? Main thing is not to hide issues with split blocks that should have been merged before. list_del(&buddy->link); @@ -304,8 +308,49 @@ static void __drm_buddy_free(struct drm_buddy *mm, block = parent; } + order = drm_buddy_block_order(block); mark_free(mm, block); + + return order; +} + +/** + * drm_buddy_defrag - Defragmentation routine + * + * @mm: DRM buddy manager + * @min_order: minimum order in the freelist to begin + * the defragmentation process + * + * Driver calls the defragmentation function when the + * requested memory allocation returns -ENOSPC. + */ +void drm_buddy_defrag(struct drm_buddy *mm, + unsigned int min_order) Just wondering if we need "full defag" also? We would probably need to call this at fini() anyway. +{ + struct drm_buddy_block *block; + struct list_head *list; + unsigned int order; + int i; + + if (min_order > mm->max_order) + return; + + for (i = min_order - 1; i >= 0; i--) { Need to be careful with min_order = 0 ? + list = &mm->free_list[i]; + if (list_empty(list)) + continue; + + list_for_each_entry_reverse(block, list, link) { Don't we need the safe_reverse() variant here, since this is removing from the list? + if (!block->parent) + continue; + + order = __drm_buddy_free(mm, block, 1); + if (order >= min_order) + return; + } + } } +EXPORT_SYMBOL(drm_buddy_defrag); /** * drm_buddy_free_block - free a block @@ -321,7 +366,7 @@ void drm_buddy_free_block(struct drm_buddy *mm, if (drm_buddy_block_is_clear(block)) mm->clear_avail += drm_buddy_block_size(mm, block); - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); } EXPORT_SYMBOL(drm_buddy_free_block); @@ -470,7 +515,7 @@ __alloc_range_bias(struct drm_buddy *mm, if (buddy && (drm_buddy_block_is_free(block) && drm_buddy_block_is_free(buddy))) - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); return ERR_PTR(err); } @@ -588,7 +633,7 @@ alloc_from_freelist(struct drm_buddy *mm, err_undo: if (tmp != order) - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); return ERR_PTR(err); } @@ -668,7 +713,7 @@ static int __alloc_range(struct drm_buddy *mm, if (buddy && (drm_buddy_block_is_free(block) && drm_buddy_block_is_free(buddy))) - __drm_buddy_free(mm, block); + __drm_buddy_free(mm, block, 0); err_free: if (err == -ENOSPC && total_allocated_on_err) { diff --git a
Re: [PATCH v6 1/3] drm/buddy: Implement tracking clear page feature
On 08/02/2024 15:49, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. v1: (Christian) - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list. - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy) v2: (Matthew) - Add a wrapper drm_buddy_free_list_internal for the freeing of blocks operation within drm buddy. - Write a macro block_incompatible() to allocate the required blocks. - Update the xe driver for the drm_buddy_free_list change in arguments. Signed-off-by: Arunpravin Paneer Selvam Signed-off-by: Matthew Auld Suggested-by: Christian König Probably needs a new unit test. I think we are missing something to forcefully re-merge everything at fini()? In theory we can just call the defrag routine. Otherwise we might trigger various warnings since the root(s) might still be split. Also one nit below. Otherwise I think looks good. --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 192 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 10 +- drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 4 +- include/drm/drm_buddy.h | 18 +- 6 files changed, 187 insertions(+), 49 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 8db880244324..c0c851409241 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -571,7 +571,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -604,7 +604,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -912,7 +912,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..33ad0cfbd54c 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy *mm, __list_add(&block->link, node->link.prev, &node->link); } +static void clear_reset(struct drm_buddy_block *block) +{ + block->header &= ~DRM_BUDDY_HEADER_CLEAR; +} + +static void mark_cleared(struct drm_buddy_block *block) +{ + block->header |= DRM_BUDDY_HEADER_CLEAR; +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -223,6 +233,12 @@ static int split_block(struct drm_buddy *mm, mark_free(mm, block->left); mark_free(mm, block->right); + if (drm_buddy_block_is_clear(block)) { + mark_cleared(block->left); + mark_cleared(block->right); + clear_reset(block); + } + mark_split(block); return 0; @@ -273,6 +289,13 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; + + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + list_del(&buddy->link); drm_block_free
Re: [PATCH] drm/buddy: Modify duplicate list_splice_tail call
On 16/02/2024 10:00, Arunpravin Paneer Selvam wrote: Remove the duplicate list_splice_tail call when the total_allocated < size condition is true. Cc: # 6.7+ Fixes: 8746c6c9dfa3 ("drm/buddy: Fix alloc_range() error handling code") Reported-by: Bert Karwatzki Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index c1a99bf4dffd..c4222b886db7 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -538,13 +538,13 @@ static int __alloc_range(struct drm_buddy *mm, list_add(&block->left->tmp_link, dfs); } while (1); - list_splice_tail(&allocated, blocks); - if (total_allocated < size) { err = -ENOSPC; goto err_free; } + list_splice_tail(&allocated, blocks); Sigh. Can we extend the unit test(s) to catch this? Reviewed-by: Matthew Auld + return 0; err_undo: base-commit: a64056bb5a3215bd31c8ce17d609ba0f4d5c55ea
Re: [PATCH 2/2] drm/tests/drm_buddy: add alloc_contiguous test
On 13/02/2024 13:52, Arunpravin Paneer Selvam wrote: Sanity check DRM_BUDDY_CONTIGUOUS_ALLOCATION. References: https://gitlab.freedesktop.org/drm/amd/-/issues/3097 Signed-off-by: Matthew Auld Reviewed-by: Arunpravin Paneer Selvam It looks like you changed the patch authorship here. Cc: Arunpravin Paneer Selvam Cc: Limonciello Cc: Christian König Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/tests/drm_buddy_test.c | 89 ++ 1 file changed, 89 insertions(+) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index ea2af6bd9abe..fee6bec757d1 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -8,6 +8,7 @@ #include #include +#include #include @@ -18,6 +19,93 @@ static inline u64 get_size(int order, u64 chunk_size) return (1 << order) * chunk_size; } +static void drm_test_buddy_alloc_contiguous(struct kunit *test) +{ + u64 mm_size, ps = SZ_4K, i, n_pages, total; + struct drm_buddy_block *block; + struct drm_buddy mm; + LIST_HEAD(left); + LIST_HEAD(middle); + LIST_HEAD(right); + LIST_HEAD(allocated); + + mm_size = 16 * 3 * SZ_4K; + + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); + + /* +* Idea is to fragment the address space by alternating block +* allocations between three different lists; one for left, middle and +* right. We can then free a list to simulate fragmentation. In +* particular we want to exercise the DRM_BUDDY_CONTIGUOUS_ALLOCATION, +* including the try_harder path. +*/ + + i = 0; + n_pages = mm_size / ps; + do { + struct list_head *list; + int slot = i % 3; + + if (slot == 0) + list = &left; + else if (slot == 1) + list = &middle; + else + list = &right; + KUNIT_ASSERT_FALSE_MSG(test, + drm_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, 0), + "buddy_alloc hit an error size=%d\n", + ps); + } while (++i < n_pages); + + KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%d\n", 3 * ps); + + drm_buddy_free_list(&mm, &middle); + KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%llu\n", 3 * ps); + KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 2 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%llu\n", 2 * ps); + + drm_buddy_free_list(&mm, &right); + KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%llu\n", 3 * ps); + /* +* At this point we should have enough contiguous space for 2 blocks, +* however they are never buddies (since we freed middle and right) so +* will require the try_harder logic to find them. +*/ + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 2 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc hit an error size=%d\n", 2 * ps); + + drm_buddy_free_list(&mm, &left); + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + DRM_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc hit an error size=%d\n", 3 * p
Re: [PATCH] drm/buddy: Fix alloc_range() error handling code
On 08/02/2024 14:17, Matthew Auld wrote: On 08/02/2024 13:47, Arunpravin Paneer Selvam wrote: Hi Matthew, On 2/8/2024 7:00 PM, Matthew Auld wrote: On 07/02/2024 17:44, Arunpravin Paneer Selvam wrote: Few users have observed display corruption when they boot the machine to KDE Plasma or playing games. We have root caused the problem that whenever alloc_range() couldn't find the required memory blocks the function was returning SUCCESS in some of the corner cases. Can you please give an example here? In the try hard contiguous allocation, for example the requested memory is 1024 pages, it might go and pick the highest and last block (of size 512 pages) in the freelist where there are no more space exist in the total address range. In this kind of corner case, alloc_range was returning success though the allocated size is less than the requested size. Hence in try_hard_contiguous_allocation, we will not proceed to the LHS allocation and we return only with the RHS allocation having only the 512 pages of allocation. This leads to display corruption in many use cases (I think mainly when requested for contiguous huge buffer) mainly on APU platforms. Ok, I guess other thing is doing: lhs_offset = drm_buddy_block_offset(block) - lhs_size; I presume it's possible for block_offset < lhs_size here, which might be funny? I think would also be good to add some basic unit test here: https://patchwork.freedesktop.org/patch/577497/?series=129671&rev=1 Test passes with your patch, and ofc fails without it. Just the question of the lhs_offset above, Reviewed-by: Matthew Auld Thanks, Arun. The right approach would be if the total allocated size is less than the required size, the function should return -ENOSPC. Gitlab ticket link - https://gitlab.freedesktop.org/drm/amd/-/issues/3097 Fixes: 0a1844bf0b53 ("drm/buddy: Improve contiguous memory allocation") Signed-off-by: Arunpravin Paneer Selvam Tested-by: Mario Limonciello --- drivers/gpu/drm/drm_buddy.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..c1a99bf4dffd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -539,6 +539,12 @@ static int __alloc_range(struct drm_buddy *mm, } while (1); list_splice_tail(&allocated, blocks); + + if (total_allocated < size) { + err = -ENOSPC; + goto err_free; + } + return 0; err_undo:
Re: [PATCH] drm/buddy: Fix alloc_range() error handling code
On 08/02/2024 13:47, Arunpravin Paneer Selvam wrote: Hi Matthew, On 2/8/2024 7:00 PM, Matthew Auld wrote: On 07/02/2024 17:44, Arunpravin Paneer Selvam wrote: Few users have observed display corruption when they boot the machine to KDE Plasma or playing games. We have root caused the problem that whenever alloc_range() couldn't find the required memory blocks the function was returning SUCCESS in some of the corner cases. Can you please give an example here? In the try hard contiguous allocation, for example the requested memory is 1024 pages, it might go and pick the highest and last block (of size 512 pages) in the freelist where there are no more space exist in the total address range. In this kind of corner case, alloc_range was returning success though the allocated size is less than the requested size. Hence in try_hard_contiguous_allocation, we will not proceed to the LHS allocation and we return only with the RHS allocation having only the 512 pages of allocation. This leads to display corruption in many use cases (I think mainly when requested for contiguous huge buffer) mainly on APU platforms. Ok, I guess other thing is doing: lhs_offset = drm_buddy_block_offset(block) - lhs_size; I presume it's possible for block_offset < lhs_size here, which might be funny? Thanks, Arun. The right approach would be if the total allocated size is less than the required size, the function should return -ENOSPC. Gitlab ticket link - https://gitlab.freedesktop.org/drm/amd/-/issues/3097 Fixes: 0a1844bf0b53 ("drm/buddy: Improve contiguous memory allocation") Signed-off-by: Arunpravin Paneer Selvam Tested-by: Mario Limonciello --- drivers/gpu/drm/drm_buddy.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..c1a99bf4dffd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -539,6 +539,12 @@ static int __alloc_range(struct drm_buddy *mm, } while (1); list_splice_tail(&allocated, blocks); + + if (total_allocated < size) { + err = -ENOSPC; + goto err_free; + } + return 0; err_undo:
Re: [PATCH] drm/buddy: Fix alloc_range() error handling code
On 07/02/2024 17:44, Arunpravin Paneer Selvam wrote: Few users have observed display corruption when they boot the machine to KDE Plasma or playing games. We have root caused the problem that whenever alloc_range() couldn't find the required memory blocks the function was returning SUCCESS in some of the corner cases. Can you please give an example here? The right approach would be if the total allocated size is less than the required size, the function should return -ENOSPC. Gitlab ticket link - https://gitlab.freedesktop.org/drm/amd/-/issues/3097 Fixes: 0a1844bf0b53 ("drm/buddy: Improve contiguous memory allocation") Signed-off-by: Arunpravin Paneer Selvam Tested-by: Mario Limonciello --- drivers/gpu/drm/drm_buddy.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..c1a99bf4dffd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -539,6 +539,12 @@ static int __alloc_range(struct drm_buddy *mm, } while (1); list_splice_tail(&allocated, blocks); + + if (total_allocated < size) { + err = -ENOSPC; + goto err_free; + } + return 0; err_undo:
Re: [PATCH v3 1/2] drm/buddy: Implement tracking clear page feature
On 30/01/2024 20:30, Arunpravin Paneer Selvam wrote: Hi Matthew, On 12/21/2023 12:51 AM, Matthew Auld wrote: Hi, On 14/12/2023 13:42, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. I was not involved, but it looks like we have also tried enabling the clear-on-free idea for VRAM in i915 and then also tracking that in the allocator, however that work unfortunately is not upstream. The code is open source though: https://github.com/intel-gpu/intel-gpu-i915-backports/blob/backport/main/drivers/gpu/drm/i915/i915_buddy.c#L300 It looks like some of the design differences there are having two separate free lists, so mm->clean and mm->dirty (sounds reasonable to me). And also the inclusion of a de-fragmentation routine, since buddy blocks are now not always merged back, we might choose to run the defrag in some cases, which also sounds reasonable. IIRC in amdgpu userspace can control the page-size for an allocation, so perhaps you would want to run it first if the allocation fails, before trying to evict stuff? I checked the clear-on-free idea implemented in i915. In amdgpu version, we are clearing all the blocks in amdgpu free routine and DRM buddy expects only the DRM_BUDDY_CLEARED flag. Basically, we are keeping the cleared blocks ready to be allocated when the user request for the cleared memory. We observed that this improves the performance on games and resolves the stutter issues as well. I see i915 active fences part does the same job for i915. Could we move this part into i915 free routine and set the DRM_BUDDY_CLEARED flag. On de-fragmentation , I have included a function which can be called at places where we get -ENOSPC. This routine will merge back the clear and dirty blocks together to form a larger block of requested size. I am wondering where we could use this routine as for the non-contiguous memory we have the fallback method and for the contiguous memory we have the try harder method which searches through the tree. Don't you also want to call it from your vram manager when the requested page size is something large, before trying to evict stuff? That could now fail due to fragmention IIUC. Or am I misreading mdgpu_vram_mgr_new()? I agree we can have 2 lists (clear list and dirty list) and this would reduce the search iterations. But we need to handle the 2 lists design in all the functions which might require more time for testing on all platforms. Could we just go ahead with 1 list (free list) for now and I am going to take up this work as my next task. Sounds good. Thanks, Arun. v1: (Christian) - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list. - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy) Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 169 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c | 10 +- include/drm/drm_buddy.h | 18 +- 5 files changed, 168 insertions(+), 41 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 08916538a615..d0e199cc8f17 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -556,7 +556,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -589,7 +589,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -897,7 +897,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
Re: [PATCH v5 1/3] drm/buddy: Implement tracking clear page feature
On 30/01/2024 19:48, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. v1: (Christian) - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list. - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy) Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 169 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 10 +- include/drm/drm_buddy.h | 18 +- 5 files changed, 168 insertions(+), 41 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 08916538a615..d0e199cc8f17 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -556,7 +556,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -589,7 +589,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -897,7 +897,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..d44172f23f05 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy *mm, __list_add(&block->link, node->link.prev, &node->link); } +static void clear_reset(struct drm_buddy_block *block) +{ + block->header &= ~DRM_BUDDY_HEADER_CLEAR; +} + +static void mark_cleared(struct drm_buddy_block *block) +{ + block->header |= DRM_BUDDY_HEADER_CLEAR; +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -223,6 +233,12 @@ static int split_block(struct drm_buddy *mm, mark_free(mm, block->left); mark_free(mm, block->right); + if (drm_buddy_block_is_clear(block)) { + mark_cleared(block->left); + mark_cleared(block->right); + clear_reset(block); + } + mark_split(block); return 0; @@ -273,6 +289,13 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; + + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); + list_del(&buddy->link); drm_block_free(mm, block); @@ -295,6 +318,9 @@ void drm_buddy_free_block(struct drm_buddy *mm, { BUG_ON(!drm_buddy_block_is_allocated(block)); mm->avail += drm_buddy_block_size(mm, block); + if (drm_buddy_block_is_clear(block)) + mm->clear_avail += drm_buddy_block_size(mm, block); + __drm_buddy_free(mm, block); } EXPORT_SYMBOL(drm_buddy_free_block); @@ -305,10 +331,20 @@ EXPORT_SYMBOL(drm_buddy_free_block); * @mm: DRM buddy manager * @objects: input list head to free blocks */ -void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects) +void drm_buddy_free_list(struct drm_buddy *mm, +struct list_head *objects, +unsigned long flags) { struct drm_buddy_block *block, *on; + if (flags &
Re: [PATCH v3 1/2] drm/buddy: Implement tracking clear page feature
Hi, On 14/12/2023 13:42, Arunpravin Paneer Selvam wrote: - Add tracking clear page feature. - Driver should enable the DRM_BUDDY_CLEARED flag if it successfully clears the blocks in the free path. On the otherhand, DRM buddy marks each block as cleared. - Track the available cleared pages size - If driver requests cleared memory we prefer cleared memory but fallback to uncleared if we can't find the cleared blocks. when driver requests uncleared memory we try to use uncleared but fallback to cleared memory if necessary. - When a block gets freed we clear it and mark the freed block as cleared, when there are buddies which are cleared as well we can merge them. Otherwise, we prefer to keep the blocks as separated. I was not involved, but it looks like we have also tried enabling the clear-on-free idea for VRAM in i915 and then also tracking that in the allocator, however that work unfortunately is not upstream. The code is open source though: https://github.com/intel-gpu/intel-gpu-i915-backports/blob/backport/main/drivers/gpu/drm/i915/i915_buddy.c#L300 It looks like some of the design differences there are having two separate free lists, so mm->clean and mm->dirty (sounds reasonable to me). And also the inclusion of a de-fragmentation routine, since buddy blocks are now not always merged back, we might choose to run the defrag in some cases, which also sounds reasonable. IIRC in amdgpu userspace can control the page-size for an allocation, so perhaps you would want to run it first if the allocation fails, before trying to evict stuff? v1: (Christian) - Depends on the flag check DRM_BUDDY_CLEARED, enable the block as cleared. Else, reset the clear flag for each block in the list. - For merging the 2 cleared blocks compare as below, drm_buddy_is_clear(block) != drm_buddy_is_clear(buddy) Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 +- drivers/gpu/drm/drm_buddy.c | 169 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 6 +- drivers/gpu/drm/tests/drm_buddy_test.c| 10 +- include/drm/drm_buddy.h | 18 +- 5 files changed, 168 insertions(+), 41 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 08916538a615..d0e199cc8f17 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -556,7 +556,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -589,7 +589,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, amdgpu_vram_mgr_do_reserve(man); - drm_buddy_free_list(mm, &vres->blocks); + drm_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); atomic64_sub(vis_usage, &mgr->vis_usage); @@ -897,7 +897,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated); + drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index f57e6d74fb0e..d44172f23f05 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -57,6 +57,16 @@ static void list_insert_sorted(struct drm_buddy *mm, __list_add(&block->link, node->link.prev, &node->link); } +static void clear_reset(struct drm_buddy_block *block) +{ + block->header &= ~DRM_BUDDY_HEADER_CLEAR; +} + +static void mark_cleared(struct drm_buddy_block *block) +{ + block->header |= DRM_BUDDY_HEADER_CLEAR; +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -223,6 +233,12 @@ static int split_block(struct drm_buddy *mm, mark_free(mm, block->left); mark_free(mm, block->right); + if (drm_buddy_block_is_clear(block)) { + mark_cleared(block->left); + mark_cleared(block->right); + clear_reset(block); + } + mark_split(block); return 0; @@ -273,6 +289,13 @@ static void __drm_buddy_free(struct drm_buddy *mm, if (!drm_buddy_block_is_free(buddy)) break; + if (drm_buddy_block_is_clear(block) != + drm_buddy_block_is_clear(buddy)) + break; + + if (drm_buddy_block_is_clear(block)) + mark_cleared(parent); +
Re: [PATCH v2 1/3] drm/buddy: Improve contiguous memory allocation
s, blocks); + return 0; I guess we could attempt to trim this also (could tweak the trim to work on multiple nodes)? But I guess in practice should be pretty meh, given that the extra rhs is hopefully not too big in the corner case where the alignment doesn't fit the min_block_size? Anyway, for patches 1-3, Reviewed-by: Matthew Auld + } else if (err != -ENOSPC) { + drm_buddy_free_list(mm, blocks); + return err; + } + /* Free blocks for the next iteration */ + drm_buddy_free_list(mm, blocks); + } + + return -ENOSPC; } /** @@ -626,7 +691,7 @@ int drm_buddy_block_trim(struct drm_buddy *mm, new_start = drm_buddy_block_offset(block); list_add(&block->tmp_link, &dfs); - err = __alloc_range(mm, &dfs, new_start, new_size, blocks); + err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); if (err) { mark_allocated(block); mm->avail -= drm_buddy_block_size(mm, block); @@ -645,7 +710,7 @@ EXPORT_SYMBOL(drm_buddy_block_trim); * @start: start of the allowed range for this block * @end: end of the allowed range for this block * @size: size of the allocation - * @min_page_size: alignment of the allocation + * @min_block_size: alignment of the allocation * @blocks: output list head to add allocated blocks * @flags: DRM_BUDDY_*_ALLOCATION flags * @@ -660,23 +725,24 @@ EXPORT_SYMBOL(drm_buddy_block_trim); */ int drm_buddy_alloc_blocks(struct drm_buddy *mm, u64 start, u64 end, u64 size, - u64 min_page_size, + u64 min_block_size, struct list_head *blocks, unsigned long flags) { struct drm_buddy_block *block = NULL; + u64 original_size, original_min_size; unsigned int min_order, order; - unsigned long pages; LIST_HEAD(allocated); + unsigned long pages; int err; if (size < mm->chunk_size) return -EINVAL; - if (min_page_size < mm->chunk_size) + if (min_block_size < mm->chunk_size) return -EINVAL; - if (!is_power_of_2(min_page_size)) + if (!is_power_of_2(min_block_size)) return -EINVAL; if (!IS_ALIGNED(start | end | size, mm->chunk_size)) @@ -690,14 +756,23 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, /* Actual range allocation */ if (start + size == end) - return __drm_buddy_alloc_range(mm, start, size, blocks); - - if (!IS_ALIGNED(size, min_page_size)) - return -EINVAL; + return __drm_buddy_alloc_range(mm, start, size, NULL, blocks); + + original_size = size; + original_min_size = min_block_size; + + /* Roundup the size to power of 2 */ + if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) { + size = roundup_pow_of_two(size); + min_block_size = size; + /* Align size value to min_block_size */ + } else if (!IS_ALIGNED(size, min_block_size)) { + size = round_up(size, min_block_size); + } pages = size >> ilog2(mm->chunk_size); order = fls(pages) - 1; - min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + min_order = ilog2(min_block_size) - ilog2(mm->chunk_size); do { order = min(order, (unsigned int)fls(pages) - 1); @@ -716,6 +791,16 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, break; if (order-- == min_order) { + if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION && + !(flags & DRM_BUDDY_RANGE_ALLOCATION)) + /* +* Try contiguous block allocation through +* try harder method +*/ + return __alloc_contig_try_harder(mm, + original_size, + original_min_size, + blocks); err = -ENOSPC; goto err_free; } @@ -732,6 +817,31 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, break; } while (1); + /* Trim the allocated block to the required size */ + if (original_size != size) { + struct list_head *trim_list; + LIST_HEAD(temp); + u64 trim_size; + + trim_list = &am
Re: [PATCH 1/3] drm/buddy: Fix contiguous memory allocation issues
Hi, On 21/08/2023 11:14, Arunpravin Paneer Selvam wrote: The way now contiguous requests are implemented such that the size rounded up to power of 2 and the corresponding order block picked from the freelist. In addition to the older method, the new method will rounddown the size to power of 2 and the corresponding order block picked from the freelist. And for the remaining size we traverse the tree and try to allocate either from the freelist block's buddy or from the peer block. If the remaining size from peer/buddy block is not free, we pick the next freelist block and repeat the same method. Moved contiguous/alignment size computation part and trim function to the drm buddy manager. I think we should also mention somewhere what issue this is trying to solve. IIUC the roundup_power_of_two() might in some cases trigger -ENOSPC even though there might be enough free space, and so to help with that we introduce a try harder mechanism. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 253 ++-- include/drm/drm_buddy.h | 6 +- 2 files changed, 248 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 7098f125b54a..220f60c08a03 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -569,6 +569,197 @@ static int __drm_buddy_alloc_range(struct drm_buddy *mm, return __alloc_range(mm, &dfs, start, size, blocks); } +static int __alloc_contiguous_block_from_buddy(struct drm_buddy *mm, + u64 size, + u64 min_block_size, + struct drm_buddy_block *block, + struct list_head *blocks) +{ + struct drm_buddy_block *buddy, *parent = NULL; + u64 start, offset = 0; + LIST_HEAD(dfs); + int err; + + if (!block) + return -EINVAL; + + buddy = __get_buddy(block); + if (!buddy) + return -ENOSPC; + + if (drm_buddy_block_is_allocated(buddy)) + return -ENOSPC; + + parent = block->parent; + if (!parent) + return -ENOSPC; + + if (block->parent->right == block) { + u64 remaining; + + /* Compute the leftover size for allocation */ + remaining = max((size - drm_buddy_block_size(mm, buddy)), + min_block_size); + if (!IS_ALIGNED(remaining, min_block_size)) + remaining = round_up(remaining, min_block_size); + + /* Check if remaining size is greater than buddy block size */ + if (drm_buddy_block_size(mm, buddy) < remaining) + return -ENOSPC; + + offset = drm_buddy_block_size(mm, buddy) - remaining; + } + + list_add(&parent->tmp_link, &dfs); + start = drm_buddy_block_offset(parent) + offset; + + err = __alloc_range(mm, &dfs, start, size, blocks); + if (err) + return -ENOSPC; + + return 0; +} + +static int __alloc_contiguous_block_from_peer(struct drm_buddy *mm, + u64 size, + u64 min_block_size, + struct drm_buddy_block *block, + struct list_head *blocks) +{ + struct drm_buddy_block *first, *peer, *tmp; + struct drm_buddy_block *parent = NULL; + u64 start, offset = 0; + unsigned int order; + LIST_HEAD(dfs); + int err; + + if (!block) + return -EINVAL; + + order = drm_buddy_block_order(block); + /* Add freelist block to dfs list */ + list_add(&block->tmp_link, &dfs); + + tmp = block; + parent = block->parent; + while (parent) { + if (block->parent->left == block) { + if (parent->left != tmp) { + peer = parent->left; + break; + } + } else { + if (parent->right != tmp) { + peer = parent->right; + break; + } + } + + tmp = parent; + parent = tmp->parent; + } + + if (!parent) + return -ENOSPC; + + do { + if (drm_buddy_block_is_allocated(peer)) + return -ENOSPC; + /* Exit loop if peer block order is equal to block order */ + if (drm_buddy_block_order(peer) == order) + break; + + if (drm_buddy_block_is_split(peer)) { + /* Traverse down to the block order level */ +
Re: [PATCH v2] drm/ttm: fix one use-after-free
On Wed, 5 Jul 2023 at 11:08, Lang Yu wrote: > > bo->kref is increased once(kref_init()) in ttm_bo_release, > but decreased twice(ttm_bo_put()) respectively in > ttm_bo_delayed_delete and ttm_bo_cleanup_refs, > which is unbalanced. > > Just clean up bo resource in one place for a delayed deleted bo. > > Fixes: 9bff18d13473 ("drm/ttm: use per BO cleanup workers") > > [ 67.399887] refcount_t: underflow; use-after-free. > [ 67.399901] WARNING: CPU: 0 PID: 3172 at lib/refcount.c:28 > refcount_warn_saturate+0xc2/0x110 > [ 67.400124] RIP: 0010:refcount_warn_saturate+0xc2/0x110 > [ 67.400173] Call Trace: > [ 67.400176] > [ 67.400181] ttm_mem_evict_first+0x4fe/0x5b0 [ttm] > [ 67.400216] ttm_bo_mem_space+0x1e3/0x240 [ttm] > [ 67.400239] ttm_bo_validate+0xc7/0x190 [ttm] > [ 67.400253] ? ww_mutex_trylock+0x1b1/0x390 > [ 67.400266] ttm_bo_init_reserved+0x183/0x1c0 [ttm] > [ 67.400280] ? __rwlock_init+0x3d/0x70 > [ 67.400292] amdgpu_bo_create+0x1cd/0x4f0 [amdgpu] > [ 67.400607] ? __pfx_amdgpu_bo_user_destroy+0x10/0x10 [amdgpu] > [ 67.400980] amdgpu_bo_create_user+0x38/0x70 [amdgpu] > [ 67.401291] amdgpu_gem_object_create+0x77/0xb0 [amdgpu] > [ 67.401641] ? __pfx_amdgpu_bo_user_destroy+0x10/0x10 [amdgpu] > [ 67.401958] amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x228/0xa30 [amdgpu] > [ 67.402433] kfd_ioctl_alloc_memory_of_gpu+0x14e/0x390 [amdgpu] > [ 67.402824] ? lock_release+0x13f/0x290 > [ 67.402838] kfd_ioctl+0x1e0/0x640 [amdgpu] > [ 67.403205] ? __pfx_kfd_ioctl_alloc_memory_of_gpu+0x10/0x10 [amdgpu] > [ 67.403579] ? tomoyo_file_ioctl+0x19/0x20 > [ 67.403590] __x64_sys_ioctl+0x95/0xd0 > [ 67.403601] do_syscall_64+0x3b/0x90 > [ 67.403609] entry_SYSCALL_64_after_hwframe+0x72/0xdc > > Signed-off-by: Lang Yu > --- > drivers/gpu/drm/ttm/ttm_bo.c | 89 > 1 file changed, 10 insertions(+), 79 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index 326a3d13a829..1e073dfb1332 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -224,82 +224,6 @@ static void ttm_bo_flush_all_fences(struct > ttm_buffer_object *bo) > dma_resv_iter_end(&cursor); > } > > -/** > - * ttm_bo_cleanup_refs > - * If bo idle, remove from lru lists, and unref. > - * If not idle, block if possible. > - * > - * Must be called with lru_lock and reservation held, this function > - * will drop the lru lock and optionally the reservation lock before > returning. > - * > - * @bo:The buffer object to clean-up > - * @interruptible: Any sleeps should occur interruptibly. > - * @no_wait_gpu: Never wait for gpu. Return -EBUSY instead. > - * @unlock_resv: Unlock the reservation lock as well. > - */ > - > -static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo, > - bool interruptible, bool no_wait_gpu, > - bool unlock_resv) > -{ > - struct dma_resv *resv = &bo->base._resv; > - int ret; > - > - if (dma_resv_test_signaled(resv, DMA_RESV_USAGE_BOOKKEEP)) > - ret = 0; > - else > - ret = -EBUSY; > - > - if (ret && !no_wait_gpu) { > - long lret; > - > - if (unlock_resv) > - dma_resv_unlock(bo->base.resv); > - spin_unlock(&bo->bdev->lru_lock); > - > - lret = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP, > -interruptible, > -30 * HZ); > - > - if (lret < 0) > - return lret; > - else if (lret == 0) > - return -EBUSY; > - > - spin_lock(&bo->bdev->lru_lock); > - if (unlock_resv && !dma_resv_trylock(bo->base.resv)) { > - /* > -* We raced, and lost, someone else holds the > reservation now, > -* and is probably busy in ttm_bo_cleanup_memtype_use. > -* > -* Even if it's not the case, because we finished > waiting any > -* delayed destruction would succeed, so just return > success > -* here. > -*/ > - spin_unlock(&bo->bdev->lru_lock); > - return 0; > - } > - ret = 0; > - } > - > - if (ret) { > - if (unlock_resv) > - dma_resv_unlock(bo->base.resv); > - spin_unlock(&bo->bdev->lru_lock); > - return ret; > - } > - > - spin_unlock(&bo->bdev->lru_lock); > - ttm_bo_cleanup_memtype_use(bo); > - > - if (unlock_resv) > - dma_resv_unlock(bo->base.resv); > - > - ttm_bo_put(bo); The put() here
Re: [Intel-gfx] [BUG 6.3-rc1] Bad lock in ttm_bo_delayed_delete()
On Wed, 15 Mar 2023 at 18:41, Christian König wrote: > > Am 08.03.23 um 13:43 schrieb Steven Rostedt: > > On Wed, 8 Mar 2023 07:17:38 +0100 > > Christian König wrote: > > > >> What test case/environment do you run to trigger this? > > I'm running a 32bit x86 qemu instance. Attached is the config. > > > > The libvirt xml file is here: https://rostedt.org/vm-images/tracetest-32.xml > > and the VM image itself is here: > > https://rostedt.org/vm-images/tracetest-32.qcow2.bz2 > > I've started to download that, but it will take about an hour. So I > tried to avoid that for now. > > But looks like there isn't any other way to reproduce this, the code > seems to work with both amdgpu and radeon. > > My suspicion is that we just have a reference count issue in qxl or ttm > which was never noticed because it didn't caused any problems (except > for a minor memory corruption). Why does ttm_bo_cleanup_refs() do a bo_put() at the end? It doesn't make sense to me. Say if the BO is in the process of being delay freed (bo->deleted = true), and we just did the kref_init() in ttm_bo_release(), it might drop that ref hitting ttm_bo_release() yet again, this time doing the actual bo->destroy(), which frees the object. The worker then fires at some later point calling ttm_bo_delayed_delete(), but the BO has already been freed. > > Now you get a rain of warnings because we try to grab the lock in the > delete worker. > > Christian. > > > > > It happened again in another test (it's not 100% reproducible). > > > > [ 23.234838] [ cut here ] > > [ 23.236391] DEBUG_LOCKS_WARN_ON(lock->magic != lock) > > [ 23.236429] WARNING: CPU: 0 PID: 61 at kernel/locking/mutex.c:582 > > __ww_mutex_lock.constprop.0+0x566/0xfec > > [ 23.240990] Modules linked in: > > [ 23.242368] CPU: 0 PID: 61 Comm: kworker/0:1H Not tainted > > 6.3.0-rc1-test-1-ga98bd42762ed-dirty #972 > > [ 23.245106] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS > > 1.16.0-debian-1.16.0-5 04/01/2014 > > [ 23.247900] Workqueue: ttm ttm_bo_delayed_delete > > [ 23.249642] EIP: __ww_mutex_lock.constprop.0+0x566/0xfec > > [ 23.251563] Code: e8 2b 5a 95 ff 85 c0 0f 84 25 fb ff ff 8b 0d 18 71 3b > > c8 85 c9 0f 85 17 fb ff ff 68 c0 58 07 c8 68 07 77 05 c8 e8 e6 0a 40 ff > > <0f> 0b 58 5a e9 ff fa ff ff e8 f8 59 95 ff 85 c0 74 0e 8b 0d 18 71 > > [ 23.256901] EAX: 0028 EBX: ECX: c1847dd8 EDX: 0002 > > [ 23.258849] ESI: EDI: c12958bc EBP: c1847f00 ESP: c1847eac > > [ 23.260786] DS: 007b ES: 007b FS: 00d8 GS: SS: 0068 EFLAGS: 00010286 > > [ 23.262840] CR0: 80050033 CR2: ffbff000 CR3: 0850e000 CR4: 00150ef0 > > [ 23.264781] Call Trace: > > [ 23.265899] ? lock_is_held_type+0xbe/0x10c > > [ 23.267434] ? ttm_bo_delayed_delete+0x30/0x94 > > [ 23.268971] ww_mutex_lock+0x32/0x94 > > [ 23.270327] ttm_bo_delayed_delete+0x30/0x94 > > [ 23.271818] process_one_work+0x21a/0x538 > > [ 23.273242] worker_thread+0x146/0x398 > > [ 23.274616] kthread+0xea/0x10c > > [ 23.275859] ? process_one_work+0x538/0x538 > > [ 23.277312] ? kthread_complete_and_exit+0x1c/0x1c > > [ 23.278899] ret_from_fork+0x1c/0x28 > > [ 23.280223] irq event stamp: 33 > > [ 23.281440] hardirqs last enabled at (33): [] > > _raw_spin_unlock_irqrestore+0x2d/0x58 > > [ 23.283860] hardirqs last disabled at (32): [] > > kvfree_call_rcu+0x155/0x2ec > > [ 23.286066] softirqs last enabled at (0): [] > > copy_process+0x989/0x2368 > > [ 23.288220] softirqs last disabled at (0): [<>] 0x0 > > [ 23.289952] ---[ end trace ]--- > > [ 23.291501] [ cut here ] > > [ 23.293027] refcount_t: underflow; use-after-free. > > [ 23.294644] WARNING: CPU: 0 PID: 61 at lib/refcount.c:28 > > refcount_warn_saturate+0xb6/0xfc > > [ 23.296959] Modules linked in: > > [ 23.298168] CPU: 0 PID: 61 Comm: kworker/0:1H Tainted: GW > > 6.3.0-rc1-test-1-ga98bd42762ed-dirty #972 > > [ 23.301073] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS > > 1.16.0-debian-1.16.0-5 04/01/2014 > > [ 23.303642] Workqueue: ttm ttm_bo_delayed_delete > > [ 23.305190] EIP: refcount_warn_saturate+0xb6/0xfc > > [ 23.306767] Code: 68 70 e1 0c c8 e8 f6 d6 a9 ff 0f 0b 58 c9 c3 90 80 3d > > 8a 78 38 c8 00 75 8a c6 05 8a 78 38 c8 01 68 9c e1 0c c8 e8 d6 d6 a9 ff > > <0f> 0b 59 c9 c3 80 3d 88 78 38 c8 00 0f 85 67 ff ff ff c6 05 88 78 > > [ 23.311935] EAX: 0026 EBX: c1295950 ECX: c1847e40 EDX: 0002 > > [ 23.313884] ESI: c12958bc EDI: f7591100 EBP: c1847f18 ESP: c1847f14 > > [ 23.315840] DS: 007b ES: 007b FS: 00d8 GS: SS: 0068 EFLAGS: 00010246 > > [ 23.317887] CR0: 80050033 CR2: ffbff000 CR3: 0850e000 CR4: 00150ef0 > > [ 23.319859] Call Trace: > > [ 23.320978] ttm_bo_delayed_delete+0x8c/0x94 > > [ 23.322492] process_one_work+0x21a/0x538 > > [ 23.323959] worker_thread+0x146/0x398 > > [ 23.325353] kthread+0xea/0x10c > > [
Re: [Intel-gfx] [PATCH 7/7] drm/ttm: cleanup ttm_range_mgr_node
On Fri, 17 Feb 2023 at 12:23, Christian König wrote: > > We don't need multiple drm_mm nodes any more. Clean that up and remove > the extra complexity. > > Signed-off-by: Christian König Reviewed-by: Matthew Auld
Re: [PATCH 5/7] drm/gem: Remove BUG_ON in drm_gem_private_object_init
On Fri, 17 Feb 2023 at 12:23, Christian König wrote: > > From: Somalapuram Amaranath > > ttm_resource can allocate size in bytes to support less than page size. > > Signed-off-by: Somalapuram Amaranath > Reviewed-by: Christian König > Signed-off-by: Christian König > Link: > https://patchwork.freedesktop.org/patch/msgid/20230208090106.9659-1-amaranath.somalapu...@amd.com > --- > drivers/gpu/drm/drm_gem.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c > index aa15c52ae182..5a3ca3363f82 100644 > --- a/drivers/gpu/drm/drm_gem.c > +++ b/drivers/gpu/drm/drm_gem.c > @@ -152,8 +152,6 @@ EXPORT_SYMBOL(drm_gem_object_init); > void drm_gem_private_object_init(struct drm_device *dev, > struct drm_gem_object *obj, size_t size) > { > - BUG_ON((size & (PAGE_SIZE - 1)) != 0); > - There are also some comments in drm_gem_{get, put}_pages referring to this exact BUG_ON(), which could do with updating now. > obj->dev = dev; > obj->filp = NULL; > > -- > 2.34.1 >
Re: [Intel-gfx] [PATCH 3/3] drm/ttm: Change the meaning of the fields in the drm_mm_nodes structure from pfn to bytes v2
On Tue, 14 Feb 2023 at 07:43, Christian König wrote: > > From: Somalapuram Amaranath > > Change the ttm_range_man_alloc() allocation from pages to size in bytes. > Fix the dependent drm_mm_nodes start and size from pages to bytes. > > v2 (chk): Change the drm_mm_node usage in amdgpu as well. re-order the > patch to be independent of the resource->start change. > > Signed-off-by: Somalapuram Amaranath > Reviewed-by: Christian König > Signed-off-by: Christian König > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c| 15 --- > drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 8 > drivers/gpu/drm/i915/i915_scatterlist.c| 6 +++--- > drivers/gpu/drm/ttm/ttm_range_manager.c| 17 - > 4 files changed, 23 insertions(+), 23 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c > index 44367f03316f..c90423cd1292 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c > @@ -116,7 +116,6 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager > *man, > struct ttm_resource **res) > { > struct amdgpu_gtt_mgr *mgr = to_gtt_mgr(man); > - uint32_t num_pages = PFN_UP(tbo->base.size); > struct ttm_range_mgr_node *node; > int r; > > @@ -134,17 +133,19 @@ static int amdgpu_gtt_mgr_new(struct > ttm_resource_manager *man, > if (place->lpfn) { > spin_lock(&mgr->lock); > r = drm_mm_insert_node_in_range(&mgr->mm, &node->mm_nodes[0], > - num_pages, > tbo->page_alignment, > - 0, place->fpfn, place->lpfn, > + tbo->base.size, > + tbo->page_alignment << > PAGE_SHIFT, 0, > + place->fpfn << PAGE_SHIFT, > + place->lpfn << PAGE_SHIFT, > DRM_MM_INSERT_BEST); > spin_unlock(&mgr->lock); > if (unlikely(r)) > goto err_free; > > - node->base.start = node->mm_nodes[0].start; > + node->base.start = node->mm_nodes[0].start >> PAGE_SHIFT; > } else { > node->mm_nodes[0].start = 0; > - node->mm_nodes[0].size = PFN_UP(node->base.size); > + node->mm_nodes[0].size = node->base.size; > node->base.start = AMDGPU_BO_INVALID_OFFSET; > } > > @@ -285,8 +286,8 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, > uint64_t gtt_size) > > ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size); > > - start = AMDGPU_GTT_MAX_TRANSFER_SIZE * > AMDGPU_GTT_NUM_TRANSFER_WINDOWS; > - size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; > + start = (AMDGPU_GTT_MAX_TRANSFER_SIZE * > AMDGPU_GTT_NUM_TRANSFER_WINDOWS) << PAGE_SHIFT; > + size = adev->gmc.gart_size - start; > drm_mm_init(&mgr->mm, start, size); > spin_lock_init(&mgr->lock); > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h > b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h > index 5c4f93ee0c57..5c78f0b09351 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h > @@ -94,8 +94,8 @@ static inline void amdgpu_res_first(struct ttm_resource > *res, > while (start >= node->size << PAGE_SHIFT) > start -= node++->size << PAGE_SHIFT; > > - cur->start = (node->start << PAGE_SHIFT) + start; > - cur->size = min((node->size << PAGE_SHIFT) - start, size); > + cur->start = node->start + start; > + cur->size = min(node->size - start, size); > cur->remaining = size; > cur->node = node; > break; > @@ -155,8 +155,8 @@ static inline void amdgpu_res_next(struct > amdgpu_res_cursor *cur, uint64_t size) > node = cur->node; > > cur->node = ++node; > - cur->start = node->start << PAGE_SHIFT; > - cur->size = min(node->size << PAGE_SHIFT, cur->remaining); > + cur->start = node->start; > + cur->size = min(node->size, cur->remaining); > break; > default: > return; > diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c > b/drivers/gpu/drm/i915/i915_scatterlist.c > index 756289e43dff..7defda1219d0 100644 > --- a/drivers/gpu/drm/i915/i915_scatterlist.c > +++ b/drivers/gpu/drm/i915/i915_scatterlist.c > @@ -94,7 +94,7 @@ struct i915_refct_sgt *i915_rsgt_from_mm_node(const struct > drm_mm_node *node, > if (!rsgt) > return ERR_PTR(-ENOMEM); > >
Re: [PATCH] drm: Alloc high address for drm buddy topdown flag
On 10/01/2023 12:02, Matthew Auld wrote: On 07/01/2023 15:15, Arunpravin Paneer Selvam wrote: As we are observing low numbers in viewperf graphics benchmark, we are strictly not allowing the top down flag enabled allocations to steal the memory space from cpu visible region. The approach is, we are sorting each order list entries in ascending order and compare the last entry of each order list in the freelist and return the max block. Did you also run the selftests? Does everything still pass and complete in a reasonable amount of time? This patch improves the viewperf 3D benchmark scores. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 81 - 1 file changed, 54 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 11bb59399471..50916b2f2fc5 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,6 +38,25 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } +static void list_insert_sorted(struct drm_buddy *mm, + struct drm_buddy_block *block) +{ + struct drm_buddy_block *node; + struct list_head *head; + + head = &mm->free_list[drm_buddy_block_order(block)]; + if (list_empty(head)) { + list_add(&block->link, head); + return; + } + + list_for_each_entry(node, head, link) + if (drm_buddy_block_offset(block) < drm_buddy_block_offset(node)) + break; + + __list_add(&block->link, node->link.prev, &node->link); +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -52,8 +71,7 @@ static void mark_free(struct drm_buddy *mm, block->header &= ~DRM_BUDDY_HEADER_STATE; block->header |= DRM_BUDDY_FREE; - list_add(&block->link, - &mm->free_list[drm_buddy_block_order(block)]); + list_insert_sorted(mm, block); One advantage of not sorting is when splitting down a large block. Previously the most-recently-split would be at the start of the list for the next order down, where potentially the next allocation could use it. So perhaps less fragmentation if it's all part of one BO. Otherwise I don't see any other downsides, other than the extra overhead of sorting. } static void mark_split(struct drm_buddy_block *block) @@ -387,20 +405,26 @@ alloc_range_bias(struct drm_buddy *mm, } static struct drm_buddy_block * -get_maxblock(struct list_head *head) +get_maxblock(struct drm_buddy *mm, unsigned int order) { struct drm_buddy_block *max_block = NULL, *node; + unsigned int i; - max_block = list_first_entry_or_null(head, - struct drm_buddy_block, - link); - if (!max_block) - return NULL; + for (i = order; i <= mm->max_order; ++i) { + if (!list_empty(&mm->free_list[i])) { + node = list_last_entry(&mm->free_list[i], + struct drm_buddy_block, + link); + if (!max_block) { + max_block = node; + continue; + } - list_for_each_entry(node, head, link) { - if (drm_buddy_block_offset(node) > - drm_buddy_block_offset(max_block)) - max_block = node; + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) { Formatting doesn't look right here. Going to test this today with some workloads with small-bar and i915 just to see if this improves/impacts anything for us. No surprises, and as advertised seems to lead to reduced utilisation of the mappable part for buffers that don't explicitly need it (TOPDOWN). Assuming the selftests are still happy, Reviewed-by: Matthew Auld + max_block = node; + } + } } return max_block; @@ -412,20 +436,23 @@ alloc_from_freelist(struct drm_buddy *mm, unsigned long flags) { struct drm_buddy_block *block = NULL; - unsigned int i; + unsigned int tmp; int err; - for (i = order; i <= mm->max_order; ++i) { - if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { - block = get_maxblock(&mm->free_list[i]); - if (block) - break; - } else { - block = list_first_entry_or_null(&mm->free_list[i], - struct drm_buddy_block, - link); - if (block) - break; + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(mm, order); + if (block) + /* Store the obtained block order */ + tmp = drm_buddy_block_order(block); + } else { + for (tmp = order; tmp
Re: [PATCH] drm: Alloc high address for drm buddy topdown flag
On 07/01/2023 15:15, Arunpravin Paneer Selvam wrote: As we are observing low numbers in viewperf graphics benchmark, we are strictly not allowing the top down flag enabled allocations to steal the memory space from cpu visible region. The approach is, we are sorting each order list entries in ascending order and compare the last entry of each order list in the freelist and return the max block. Did you also run the selftests? Does everything still pass and complete in a reasonable amount of time? This patch improves the viewperf 3D benchmark scores. Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 81 - 1 file changed, 54 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 11bb59399471..50916b2f2fc5 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -38,6 +38,25 @@ static void drm_block_free(struct drm_buddy *mm, kmem_cache_free(slab_blocks, block); } +static void list_insert_sorted(struct drm_buddy *mm, + struct drm_buddy_block *block) +{ + struct drm_buddy_block *node; + struct list_head *head; + + head = &mm->free_list[drm_buddy_block_order(block)]; + if (list_empty(head)) { + list_add(&block->link, head); + return; + } + + list_for_each_entry(node, head, link) + if (drm_buddy_block_offset(block) < drm_buddy_block_offset(node)) + break; + + __list_add(&block->link, node->link.prev, &node->link); +} + static void mark_allocated(struct drm_buddy_block *block) { block->header &= ~DRM_BUDDY_HEADER_STATE; @@ -52,8 +71,7 @@ static void mark_free(struct drm_buddy *mm, block->header &= ~DRM_BUDDY_HEADER_STATE; block->header |= DRM_BUDDY_FREE; - list_add(&block->link, -&mm->free_list[drm_buddy_block_order(block)]); + list_insert_sorted(mm, block); One advantage of not sorting is when splitting down a large block. Previously the most-recently-split would be at the start of the list for the next order down, where potentially the next allocation could use it. So perhaps less fragmentation if it's all part of one BO. Otherwise I don't see any other downsides, other than the extra overhead of sorting. } static void mark_split(struct drm_buddy_block *block) @@ -387,20 +405,26 @@ alloc_range_bias(struct drm_buddy *mm, } static struct drm_buddy_block * -get_maxblock(struct list_head *head) +get_maxblock(struct drm_buddy *mm, unsigned int order) { struct drm_buddy_block *max_block = NULL, *node; + unsigned int i; - max_block = list_first_entry_or_null(head, -struct drm_buddy_block, -link); - if (!max_block) - return NULL; + for (i = order; i <= mm->max_order; ++i) { + if (!list_empty(&mm->free_list[i])) { + node = list_last_entry(&mm->free_list[i], + struct drm_buddy_block, + link); + if (!max_block) { + max_block = node; + continue; + } - list_for_each_entry(node, head, link) { - if (drm_buddy_block_offset(node) > - drm_buddy_block_offset(max_block)) - max_block = node; + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) { Formatting doesn't look right here. Going to test this today with some workloads with small-bar and i915 just to see if this improves/impacts anything for us. + max_block = node; + } + } } return max_block; @@ -412,20 +436,23 @@ alloc_from_freelist(struct drm_buddy *mm, unsigned long flags) { struct drm_buddy_block *block = NULL; - unsigned int i; + unsigned int tmp; int err; - for (i = order; i <= mm->max_order; ++i) { - if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { - block = get_maxblock(&mm->free_list[i]); - if (block) - break; - } else { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); - if (block) - break; + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(mm, order); + if (block) + /* Store the obt
Re: [Intel-gfx] [PATCH 7/9] drm/i915: stop using ttm_bo_wait
On 05/12/2022 19:58, Christian König wrote: Am 30.11.22 um 15:06 schrieb Daniel Vetter: On Wed, 30 Nov 2022 at 14:03, Tvrtko Ursulin wrote: On 29/11/2022 18:05, Matthew Auld wrote: On Fri, 25 Nov 2022 at 11:14, Tvrtko Ursulin wrote: + Matt On 25/11/2022 10:21, Christian König wrote: TTM is just wrapping core DMA functionality here, remove the mid-layer. No functional change. Signed-off-by: Christian König --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 5247d88b3c13..d409a77449a3 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -599,13 +599,16 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, static int i915_ttm_truncate(struct drm_i915_gem_object *obj) { struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); - int err; + long err; WARN_ON_ONCE(obj->mm.madv == I915_MADV_WILLNEED); - err = ttm_bo_wait(bo, true, false); - if (err) + err = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, + true, 15 * HZ); This 15 second stuck out a bit for me and then on a slightly deeper look it seems this timeout will "leak" into a few of i915 code paths. If we look at the difference between the legacy shmem and ttm backend I am not sure if the legacy one is blocking or not - but if it can block I don't think it would have an arbitrary timeout like this. Matt your thoughts? Not sure what is meant by leak here, but the legacy shmem must also wait/block when unbinding each VMA, before calling truncate. It's the By "leak" I meant if 15s timeout propagates into some code paths visible from userspace which with a legacy backend instead have an indefinite wait. If we have that it's probably not very good to have this inconsistency, or to apply an arbitrary timeout to those path to start with. same story for the ttm backend, except slightly more complicated in that there might be no currently bound VMA, and yet the GPU could still be accessing the pages due to async unbinds, kernel moves etc, which the wait here (and in i915_ttm_shrink) is meant to protect against. If the wait times out it should just fail gracefully. I guess we could just use MAX_SCHEDULE_TIMEOUT here? Not sure if it really matters though. Right, depends if it can leak or not to userspace and diverge between backends. Generally lock_timeout() is a design bug. It's either lock_interruptible (or maybe lock_killable) or try_lock, but lock_timeout is just duct-tape. I haven't dug in to figure out what should be here, but it smells fishy. Independent of this discussion could I get an rb for removing ttm_bo_wait() from i915? Exactly hiding this timeout inside TTM is what always made me quite nervous here. There are also a few places in i915 calling bo_wait_ctx(), which appears to just wrap ttm_bo_wait(). I guess that should also be converted to dma_resv_wait_timeout()? Or what is the story with that? Anyway, Reviewed-by: Matthew Auld Regards, Christian. -Daniel
Re: [Intel-gfx] [PATCH 7/9] drm/i915: stop using ttm_bo_wait
On Fri, 25 Nov 2022 at 11:14, Tvrtko Ursulin wrote: > > > + Matt > > On 25/11/2022 10:21, Christian König wrote: > > TTM is just wrapping core DMA functionality here, remove the mid-layer. > > No functional change. > > > > Signed-off-by: Christian König > > --- > > drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 9 ++--- > > 1 file changed, 6 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c > > b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c > > index 5247d88b3c13..d409a77449a3 100644 > > --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c > > +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c > > @@ -599,13 +599,16 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object > > *obj, > > static int i915_ttm_truncate(struct drm_i915_gem_object *obj) > > { > > struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); > > - int err; > > + long err; > > > > WARN_ON_ONCE(obj->mm.madv == I915_MADV_WILLNEED); > > > > - err = ttm_bo_wait(bo, true, false); > > - if (err) > > + err = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, > > + true, 15 * HZ); > > This 15 second stuck out a bit for me and then on a slightly deeper look > it seems this timeout will "leak" into a few of i915 code paths. If we > look at the difference between the legacy shmem and ttm backend I am not > sure if the legacy one is blocking or not - but if it can block I don't > think it would have an arbitrary timeout like this. Matt your thoughts? Not sure what is meant by leak here, but the legacy shmem must also wait/block when unbinding each VMA, before calling truncate. It's the same story for the ttm backend, except slightly more complicated in that there might be no currently bound VMA, and yet the GPU could still be accessing the pages due to async unbinds, kernel moves etc, which the wait here (and in i915_ttm_shrink) is meant to protect against. If the wait times out it should just fail gracefully. I guess we could just use MAX_SCHEDULE_TIMEOUT here? Not sure if it really matters though. > > Regards, > > Tvrtko > > > + if (err < 0) > > return err; > > + if (err == 0) > > + return -EBUSY; > > > > err = i915_ttm_move_notify(bo); > > if (err)
Re: [PATCH 1/3] drm/ttm: remove ttm_bo_(un)lock_delayed_workqueue
On Thu, 24 Nov 2022 at 10:03, Christian König wrote: > > Those functions never worked correctly since it is still perfectly > possible that a buffer object is released and the background worker > restarted even after calling them. > > Signed-off-by: Christian König I know you usually do, but just a friendly reminder to Cc: intel-gfx on the next revision or before merging, just so our CI can give the series a quick test. Thanks.
Re: [Intel-gfx] [PATCH v2] drm/ttm: rework on ttm_resource to use size_t type
On Tue, 25 Oct 2022 at 16:51, Somalapuram Amaranath wrote: > > Change ttm_resource structure from num_pages to size_t size in bytes. > v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages > v1 -> v2: change bo->resource->size to bo->base.size at some places > v1 -> v2: remove the local variable > v1 -> v2: cleanup cmp_size_smaller_first() > > Signed-off-by: Somalapuram Amaranath > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c| 2 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++- > drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 4 ++-- > drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h | 2 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c| 6 +++--- > drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 8 > drivers/gpu/drm/i915/gem/i915_gem_ttm.c| 2 +- > drivers/gpu/drm/i915/i915_scatterlist.c| 4 ++-- > drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 12 ++-- > drivers/gpu/drm/i915/intel_region_ttm.c| 2 +- > drivers/gpu/drm/nouveau/nouveau_bo.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_bo0039.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_bo5039.c | 2 +- > drivers/gpu/drm/nouveau/nouveau_bo74c1.c | 2 +- > drivers/gpu/drm/nouveau/nouveau_bo85b5.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_bo9039.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_bo90b5.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_boa0b5.c | 2 +- > drivers/gpu/drm/nouveau/nouveau_gem.c | 5 ++--- > drivers/gpu/drm/nouveau/nouveau_mem.c | 4 ++-- > drivers/gpu/drm/nouveau/nouveau_ttm.c | 2 +- > drivers/gpu/drm/radeon/radeon_cs.c | 7 +-- > drivers/gpu/drm/radeon/radeon_object.c | 4 ++-- > drivers/gpu/drm/radeon/radeon_trace.h | 2 +- > drivers/gpu/drm/radeon/radeon_ttm.c| 4 ++-- > drivers/gpu/drm/ttm/ttm_bo.c | 3 --- > drivers/gpu/drm/ttm/ttm_bo_util.c | 6 +++--- > drivers/gpu/drm/ttm/ttm_bo_vm.c| 4 ++-- > drivers/gpu/drm/ttm/ttm_range_manager.c| 2 +- > drivers/gpu/drm/ttm/ttm_resource.c | 14 ++ > drivers/gpu/drm/vmwgfx/vmwgfx_blit.c | 4 ++-- > drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 6 +++--- > drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c| 2 +- > drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c| 2 +- > drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c | 6 +++--- > drivers/gpu/drm/vmwgfx/vmwgfx_kms.c| 2 +- > drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c | 6 +++--- > include/drm/ttm/ttm_resource.h | 4 ++-- > 38 files changed, 79 insertions(+), 81 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c > index 38119311284d..f86dc92965bb 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c > +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c > @@ -217,7 +217,7 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, > page_last = vma_pages(vma) + vma->vm_pgoff - > drm_vma_node_start(&bo->base.vma_node); > > - if (unlikely(page_offset >= bo->resource->num_pages)) > + if (unlikely(page_offset >= bo->base.size)) At a glance it looks like we are missing PFN_UP(bo->base.size) for this one?
Re: [PATCH v2] drm/ttm: update bulk move object of ghost BO
On Tue, 6 Sept 2022 at 09:54, Christian König wrote: > > Am 06.09.22 um 10:46 schrieb ZhenGuo Yin: > > [Why] > > Ghost BO is released with non-empty bulk move object. There is a > > warning trace: > > WARNING: CPU: 19 PID: 1582 at ttm/ttm_bo.c:366 ttm_bo_release+0x2e1/0x2f0 > > [amdttm] > > Call Trace: > >amddma_resv_reserve_fences+0x10d/0x1f0 [amdkcl] > >amdttm_bo_put+0x28/0x30 [amdttm] > >amdttm_bo_move_accel_cleanup+0x126/0x200 [amdttm] > >amdgpu_bo_move+0x1a8/0x770 [amdgpu] > >ttm_bo_handle_move_mem+0xb0/0x140 [amdttm] > >amdttm_bo_validate+0xbf/0x100 [amdttm] > > > > [How] > > The resource of ghost BO should be moved to LRU directly, instead of > > using bulk move. The bulk move object of ghost BO should set to NULL > > before function ttm_bo_move_to_lru_tail_unlocked. > > > > v2: set bulk move to NULL manually if no resource associated with ghost BO > > > > Fixed: 5b951e487fd6bf5f ("drm/ttm: fix bulk move handling v2") > > Signed-off-by: ZhenGuo Yin > > Reviewed-by: Christian König > > Going to push that to drm-misc-fixes in a minute. > > Thanks, > Christian. > > > --- > > drivers/gpu/drm/ttm/ttm_bo_util.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c > > b/drivers/gpu/drm/ttm/ttm_bo_util.c > > index 1cbfb00c1d65..57a27847206f 100644 > > --- a/drivers/gpu/drm/ttm/ttm_bo_util.c > > +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c > > @@ -239,6 +239,9 @@ static int ttm_buffer_object_transfer(struct > > ttm_buffer_object *bo, > > if (fbo->base.resource) { > > ttm_resource_set_bo(fbo->base.resource, &fbo->base); > > bo->resource = NULL; > > + ttm_bo_set_bulk_move(&fbo->base, NULL); This appears to blow up quite badly in i915. See here for an example trace: https://gitlab.freedesktop.org/drm/intel/-/issues/6744 Do you know if amdgpu is also hitting this, or is this somehow i915 specific? > > + } else { > > + fbo->base.bulk_move = NULL; > > } > > > > dma_resv_init(&fbo->base.base._resv); >
Re: [PATCH v4 4/6] drm/i915: Implement intersect/compatible functions
On 04/08/2022 09:59, Arunpravin Paneer Selvam wrote: Implemented a new intersect and compatible callback function fetching start offset from drm buddy allocator. v3: move the bits that are specific to buddy_man (Matthew) v4: consider the block size /range (Matthew) Signed-off-by: Christian König Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 41 +-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 73 +++ 2 files changed, 74 insertions(+), 40 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 70e2ed4e99df..bf5fd6886ca0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -379,7 +379,6 @@ static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo, const struct ttm_place *place) { struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); - struct ttm_resource *res = bo->resource; if (!obj) return false; @@ -396,45 +395,7 @@ static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo, if (!i915_gem_object_evictable(obj)) return false; - switch (res->mem_type) { - case I915_PL_LMEM0: { - struct ttm_resource_manager *man = - ttm_manager_type(bo->bdev, res->mem_type); - struct i915_ttm_buddy_resource *bman_res = - to_ttm_buddy_resource(res); - struct drm_buddy *mm = bman_res->mm; - struct drm_buddy_block *block; - - if (!place->fpfn && !place->lpfn) - return true; - - GEM_BUG_ON(!place->lpfn); - - /* -* If we just want something mappable then we can quickly check -* if the current victim resource is using any of the CPU -* visible portion. -*/ - if (!place->fpfn && - place->lpfn == i915_ttm_buddy_man_visible_size(man)) - return bman_res->used_visible_size > 0; - - /* Real range allocation */ - list_for_each_entry(block, &bman_res->blocks, link) { - unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; - unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); - - if (place->fpfn < lpfn && place->lpfn > fpfn) - return true; - } - return false; - } default: - break; - } - - return true; + return ttm_bo_eviction_valuable(bo, place); } static void i915_ttm_evict_flags(struct ttm_buffer_object *bo, diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index a5109548abc0..9def01d5f368 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -178,6 +178,77 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man, kfree(bman_res); } +static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man, + struct ttm_resource *res, + const struct ttm_place *place, + size_t size) +{ + struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); + struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); + struct drm_buddy *mm = &bman->mm; + struct drm_buddy_block *block; + + if (!place->fpfn && !place->lpfn) + return true; + + GEM_BUG_ON(!place->lpfn); + + /* +* If we just want something mappable then we can quickly check +* if the current victim resource is using any of the CP Nit: s/CP/CPU/ Reviewed-by: Matthew Auld +* visible portion. +*/ + if (!place->fpfn && + place->lpfn == i915_ttm_buddy_man_visible_size(man)) + return bman_res->used_visible_size > 0; + + /* Check each drm buddy block individually */ + list_for_each_entry(block, &bman_res->blocks, link) { + unsigned long fpfn = + drm_buddy_block_offset(block) >> PAGE_SHIFT; + unsigned long lpfn = fpfn + + (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + + if (place->fpfn < lpfn && place->lpfn > fpfn) + return true; + } + + return false; +} + +static bool i915_ttm_buddy_man_compatible
Re: [PATCH v3 6/6] drm/ttm: Switch to using the new res callback
On 28/07/2022 15:33, Arunpravin Paneer Selvam wrote: Apply new intersect and compatible callback instead of having a generic placement range verfications. v2: Added a separate callback for compatiblilty checks (Christian) Signed-off-by: Christian König Signed-off-by: Arunpravin Paneer Selvam There is also some code at the bottom of i915_ttm_buddy_man_alloc() playing games with res->start, which I think can be safely deleted with this series (now that we have proper ->compatible() hook). Also, is the plan to remove res->start completely, or does that still have a use? --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++-- drivers/gpu/drm/ttm/ttm_bo.c| 9 +++-- drivers/gpu/drm/ttm/ttm_resource.c | 5 +-- 3 files changed, 20 insertions(+), 39 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 170935c294f5..7d25a10395c0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1328,11 +1328,12 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, const struct ttm_place *place) { - unsigned long num_pages = bo->resource->num_pages; struct dma_resv_iter resv_cursor; - struct amdgpu_res_cursor cursor; struct dma_fence *f; + if (!amdgpu_bo_is_amdgpu_bo(bo)) + return ttm_bo_eviction_valuable(bo, place); + /* Swapout? */ if (bo->resource->mem_type == TTM_PL_SYSTEM) return true; @@ -1351,40 +1352,20 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, return false; } - switch (bo->resource->mem_type) { - case AMDGPU_PL_PREEMPT: - /* Preemptible BOs don't own system resources managed by the -* driver (pages, VRAM, GART space). They point to resources -* owned by someone else (e.g. pageable memory in user mode -* or a DMABuf). They are used in a preemptible context so we -* can guarantee no deadlocks and good QoS in case of MMU -* notifiers or DMABuf move notifiers from the resource owner. -*/ + /* Preemptible BOs don't own system resources managed by the +* driver (pages, VRAM, GART space). They point to resources +* owned by someone else (e.g. pageable memory in user mode +* or a DMABuf). They are used in a preemptible context so we +* can guarantee no deadlocks and good QoS in case of MMU +* notifiers or DMABuf move notifiers from the resource owner. +*/ + if (bo->resource->mem_type == AMDGPU_PL_PREEMPT) return false; - case TTM_PL_TT: - if (amdgpu_bo_is_amdgpu_bo(bo) && - amdgpu_bo_encrypted(ttm_to_amdgpu_bo(bo))) - return false; - return true; - case TTM_PL_VRAM: - /* Check each drm MM node individually */ - amdgpu_res_first(bo->resource, 0, (u64)num_pages << PAGE_SHIFT, -&cursor); - while (cursor.remaining) { - if (place->fpfn < PFN_DOWN(cursor.start + cursor.size) - && !(place->lpfn && -place->lpfn <= PFN_DOWN(cursor.start))) - return true; - - amdgpu_res_next(&cursor, cursor.size); - } + if (bo->resource->mem_type == TTM_PL_TT && + amdgpu_bo_encrypted(ttm_to_amdgpu_bo(bo))) return false; - default: - break; - } - return ttm_bo_eviction_valuable(bo, place); } diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index c1bd006a5525..03409409e43e 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -518,6 +518,9 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, const struct ttm_place *place) { + struct ttm_resource *res = bo->resource; + struct ttm_device *bdev = bo->bdev; + dma_resv_assert_held(bo->base.resv); if (bo->resource->mem_type == TTM_PL_SYSTEM) return true; @@ -525,11 +528,7 @@ bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, /* Don't evict this BO if it's outside of the * requested placement range */ - if (place->fpfn >= (bo->resource->start + bo->resource->num_pages) || - (place->lpfn && place->lpfn <= bo->resource->start)) - return false; - - return true; + return ttm_resource_intersect(bdev, res, place, bo->base.size); } E
Re: [PATCH v3 4/6] drm/i915: Implement intersect/compatible functions
On 28/07/2022 15:33, Arunpravin Paneer Selvam wrote: Implemented a new intersect and compatible callback function fetching start offset from drm buddy allocator. v2: move the bits that are specific to buddy_man (Matthew) Signed-off-by: Christian König Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 39 +--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 62 +++ 2 files changed, 64 insertions(+), 37 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 70e2ed4e99df..54eead15d74b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -396,43 +396,8 @@ static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo, if (!i915_gem_object_evictable(obj)) return false; - switch (res->mem_type) { - case I915_PL_LMEM0: { - struct ttm_resource_manager *man = - ttm_manager_type(bo->bdev, res->mem_type); - struct i915_ttm_buddy_resource *bman_res = - to_ttm_buddy_resource(res); - struct drm_buddy *mm = bman_res->mm; - struct drm_buddy_block *block; - - if (!place->fpfn && !place->lpfn) - return true; - - GEM_BUG_ON(!place->lpfn); - - /* -* If we just want something mappable then we can quickly check -* if the current victim resource is using any of the CPU -* visible portion. -*/ - if (!place->fpfn && - place->lpfn == i915_ttm_buddy_man_visible_size(man)) - return bman_res->used_visible_size > 0; - - /* Real range allocation */ - list_for_each_entry(block, &bman_res->blocks, link) { - unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; - unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); - - if (place->fpfn < lpfn && place->lpfn > fpfn) - return true; - } - return false; - } default: - break; - } + if (res->mem_type == I915_PL_LMEM0) + return ttm_bo_eviction_valuable(bo, place); We should be able to drop the mem_type == I915_PL_LMEM0 check here I think, and just unconditionally do: return ttm_bo_eviction_valuable(bo, place); return true; } diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index a5109548abc0..9d2a31154d58 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -178,6 +178,66 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man, kfree(bman_res); } +static bool i915_ttm_buddy_man_intersect(struct ttm_resource_manager *man, Nit: intersects +struct ttm_resource *res, +const struct ttm_place *place, +size_t size) +{ + struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); + u32 start, num_pages = PFN_UP(size); + struct drm_buddy_block *block; + + if (!place->fpfn && !place->lpfn) + return true; + + /* +* If we just want something mappable then we can quickly check +* if the current victim resource is using any of the CP +* visible portion. +*/ + if (!place->fpfn && + place->lpfn == i915_ttm_buddy_man_visible_size(man)) + return bman_res->used_visible_size > 0; + + /* Check each drm buddy block individually */ + list_for_each_entry(block, &bman_res->blocks, link) { + start = drm_buddy_block_offset(block) >> PAGE_SHIFT; + /* Don't evict BOs outside of the requested placement range */ + if (place->fpfn >= (start + num_pages) || + (place->lpfn && place->lpfn <= start)) + return false; + } + + return true; We need to account for the block size somewhere. Also same bug in the amdgpu patch it seems. We also need to do this the other way around and keep checking until we find something that overlaps, for example if the first block doesn't intersect/overlap we will incorrectly return false here, even if one of the other blocks does intersect. list_for_each_entry() { fpfn = drm_buddy_block_size(mm, block) >> PAGE_SHIFT; lpfn = fpfn + drm_buddy_block_size(mm, block) >> PAGE_SHIFT); if (place->fpfn < lpfn && place->lpfn > fpfn) return true; } return false; +} + +static bool i915_ttm_bu
Re: [PATCH v2 4/6] drm/i915: Implement intersect/compatible functions
On 25/07/2022 12:42, Arunpravin Paneer Selvam wrote: Implemented a new intersect and compatible callback function fetching start offset from drm buddy allocator. Signed-off-by: Christian König Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 43 +++ 1 file changed, 43 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index a5109548abc0..b5801c05bd41 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -178,6 +178,47 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man, kfree(bman_res); } +static bool i915_ttm_buddy_man_intersect(struct ttm_resource_manager *man, +struct ttm_resource *res, +const struct ttm_place *place, +size_t size) +{ + struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); + u32 start, num_pages = PFN_UP(size); + struct drm_buddy_block *block; + + /* Check each drm buddy block individually */ + list_for_each_entry(block, &bman_res->blocks, link) { + start = drm_buddy_block_offset(block) >> PAGE_SHIFT; + /* Don't evict BOs outside of the requested placement range */ + if (place->fpfn >= (start + num_pages) || + (place->lpfn && place->lpfn <= start)) + return false; + } + + return true; +} This looks like a nice idea. We should be able to clean up i915_ttm_eviction_valuable() a fair bit I think, if we now call ttm_bo_eviction_valuable() at the end (like in amdgpu), and move the bits that are specific to buddy_man here? So something like: if (!place->fpfn && !place->lpfn) return true; if (!place->fpfn && place->lpfn == i915_buddy_man_visible_size(man)) return bman_res->used_visible_size > 0; /* Check each drm buddy block individually */ + +static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man, + struct ttm_resource *res, + const struct ttm_place *place, + size_t size) Is it not possible to derive the size from res->num_pages? +{ + struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); + u32 start, num_pages = PFN_UP(size); + struct drm_buddy_block *block; + + /* Check each drm buddy block individually */ + list_for_each_entry(block, &bman_res->blocks, link) { + start = drm_buddy_block_offset(block) >> PAGE_SHIFT; + if (start < place->fpfn || + (place->lpfn && (start + num_pages) > place->lpfn)) + return false; + } if (!place->fpfn && !place->lpfn) return true; if (!place->fpfn && place->lpfn == i915_buddy_man_visible_size(man)) return bman_res->used_visible_size == res->num_pages; /* Check each drm buddy block individually */ ... + + return true; +} + static void i915_ttm_buddy_man_debug(struct ttm_resource_manager *man, struct drm_printer *printer) { @@ -205,6 +246,8 @@ static void i915_ttm_buddy_man_debug(struct ttm_resource_manager *man, static const struct ttm_resource_manager_func i915_ttm_buddy_manager_func = { .alloc = i915_ttm_buddy_man_alloc, .free = i915_ttm_buddy_man_free, + .intersect = i915_ttm_buddy_man_intersect, s/intersect/intersects/ ? + .compatible = i915_ttm_buddy_man_compatible, .debug = i915_ttm_buddy_man_debug, };
Re: [igt-dev] [PATCH i-g-t v2] tests/drm_buddy: Add drm buddy test cases
On Mon, 11 Apr 2022 at 19:51, Arunpravin Paneer Selvam wrote: > > Add a set of drm buddy test cases to validate the > drm/drm_buddy.c memory allocator. > > v2: sorted in alphabetical order > > Signed-off-by: Arunpravin Paneer Selvam > Reviewed-by: Matthew Auld Tests look to be passing in CI sharded runs. Pushed. Thanks. > --- > tests/drm_buddy.c | 14 ++ > tests/meson.build | 1 + > 2 files changed, 15 insertions(+) > create mode 100644 tests/drm_buddy.c > > diff --git a/tests/drm_buddy.c b/tests/drm_buddy.c > new file mode 100644 > index ..06876e0c > --- /dev/null > +++ b/tests/drm_buddy.c > @@ -0,0 +1,14 @@ > +// SPDX-License-Identifier: MIT > +/* > + * Copyright © 2019 Intel Corporation > + */ > + > +#include "igt.h" > +#include "igt_kmod.h" > + > +IGT_TEST_DESCRIPTION("Basic sanity check of DRM's buddy allocator (struct > drm_buddy)"); > + > +igt_main > +{ > + igt_kselftests("test-drm_buddy", NULL, NULL, NULL); > +} > diff --git a/tests/meson.build b/tests/meson.build > index b0eab3d6..7261e9aa 100644 > --- a/tests/meson.build > +++ b/tests/meson.build > @@ -8,6 +8,7 @@ test_progs = [ > 'debugfs_test', > 'dmabuf', > 'device_reset', > + 'drm_buddy', > 'drm_import_export', > 'drm_mm', > 'drm_read', > -- > 2.25.1 >
Re: [PATCH v3] drm: add a check to verify the size alignment
On 11/04/2022 13:42, Christian König wrote: Am 11.04.22 um 11:47 schrieb Matthew Auld: On 11/04/2022 08:38, Arunpravin Paneer Selvam wrote: Add a simple check to reject any size not aligned to the min_page_size. when size is not aligned to min_page_size, driver module should handle in their own way either to round_up() the size value to min_page_size or just to enable WARN_ON(). If we dont handle the alignment properly, we may hit the following bug, Unigine Heaven has allocation requests for example required pages are 257 and alignment request is 256. To allocate the left over 1 page, continues the iteration to find the order value which is 0 and when it compares with min_order = 8, triggers the BUG_ON(order < min_order). v2: add more commit description v3: remove WARN_ON() Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld Reviewed-by: Matthew Auld Question here is who will be pushing that to drm-misc-next? Should I take care of that? Yes, please do. I think it's time that Arun should request push permission for drm-misc-next. Thanks, Christian.
Re: [PATCH v3] drm: add a check to verify the size alignment
On 11/04/2022 08:38, Arunpravin Paneer Selvam wrote: Add a simple check to reject any size not aligned to the min_page_size. when size is not aligned to min_page_size, driver module should handle in their own way either to round_up() the size value to min_page_size or just to enable WARN_ON(). If we dont handle the alignment properly, we may hit the following bug, Unigine Heaven has allocation requests for example required pages are 257 and alignment request is 256. To allocate the left over 1 page, continues the iteration to find the order value which is 0 and when it compares with min_order = 8, triggers the BUG_ON(order < min_order). v2: add more commit description v3: remove WARN_ON() Signed-off-by: Arunpravin Paneer Selvam Suggested-by: Matthew Auld Reviewed-by: Matthew Auld
Re: [PATCH] tests/drm_buddy: Add drm buddy test cases
On 11/04/2022 08:28, Arunpravin Paneer Selvam wrote: Add a set of drm buddy test cases to validate the drm/drm_buddy.c memory allocator. Signed-off-by: Arunpravin Paneer Selvam --- tests/drm_buddy.c | 14 ++ tests/meson.build | 1 + 2 files changed, 15 insertions(+) create mode 100644 tests/drm_buddy.c diff --git a/tests/drm_buddy.c b/tests/drm_buddy.c new file mode 100644 index ..06876e0c --- /dev/null +++ b/tests/drm_buddy.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2019 Intel Corporation + */ + +#include "igt.h" +#include "igt_kmod.h" + +IGT_TEST_DESCRIPTION("Basic sanity check of DRM's buddy allocator (struct drm_buddy)"); + +igt_main +{ + igt_kselftests("test-drm_buddy", NULL, NULL, NULL); +} diff --git a/tests/meson.build b/tests/meson.build index b0eab3d6..4ed8e610 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -10,6 +10,7 @@ test_progs = [ 'device_reset', 'drm_import_export', 'drm_mm', + 'drm_buddy', Nit: Should be kept sorted. No need to resend though. Reviewed-by: Matthew Auld 'drm_read', 'fbdev', 'feature_discovery', base-commit: 016a7169e66b710a6720ed8ff94815a7e8076541
Re: [PATCH v2] drm: add a check to verify the size alignment
On Tue, 29 Mar 2022 at 12:17, Arunpravin Paneer Selvam wrote: > > > > On 23/03/22 1:15 pm, Christian König wrote: > > Am 23.03.22 um 08:34 schrieb Arunpravin Paneer Selvam: > >> Add a simple check to reject any size not aligned to the > >> min_page_size. > >> > >> handle instances when size is not aligned with the min_page_size. > >> Unigine Heaven has allocation requests for example required pages > >> are 257 and alignment request is 256. To allocate the left over 1 > >> page, continues the iteration to find the order value which is 0 > >> and when it compares with min_order = 8, triggers the BUG_ON(order > >> < min_order). To avoid this problem, we added a simple check to > >> return -EINVAL if size is not aligned to the min_page_size. > >> > >> v2: Added more details to the commit description > >> > >> Signed-off-by: Arunpravin Paneer Selvam > >> Suggested-by: Matthew Auld > >> --- > >> drivers/gpu/drm/drm_buddy.c | 3 +++ > >> 1 file changed, 3 insertions(+) > >> > >> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c > >> index 72f52f293249..b503c88786b0 100644 > >> --- a/drivers/gpu/drm/drm_buddy.c > >> +++ b/drivers/gpu/drm/drm_buddy.c > >> @@ -661,6 +661,9 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, > >> if (range_overflows(start, size, mm->size)) > >> return -EINVAL; > >> > >> +if (WARN_ON(!IS_ALIGNED(size, min_page_size))) > >> +return -EINVAL; > >> + > > > > I'm not that happy with the handling here. > > > > See a minimum page size larger than the requested size is perfectly > > valid, it just means that the remaining pages needs to be trimmed. > > > > In other words when the request is to allocate 1 page with an alignment > > of 256 we just need to give the remaining 255 pages back to the allocator. > > In one of the previous mail Matthew explained that i915 expects to > return -EINVAL error code if size is not aligned to min_page_size. We could also move the WARN_ON() into i915 as a separate patch, and just change the default buddy behaviour to transparently handle the rounding + trim, if you prefer. I don't have a strong opinion. > > can we just modify in amdgpu code where we round_up the size to the > min_page_size value and keep this error handling in drm_buddy.c? > > > > Regards, > > Christian. > > > >> /* Actual range allocation */ > >> if (start + size == end) > >> return __drm_buddy_alloc_range(mm, start, size, blocks); > >> > >> base-commit: 056d47eaf6ea753fa2e21da31f9cbd8b721bbb7b > >
Re: [PATCH v2] drm: Fix a infinite loop condition when order becomes 0
On 16/03/2022 06:34, Arunpravin Paneer Selvam wrote: handle a situation in the condition order-- == min_order, when order = 0 and min_order = 0, leading to order = -1, it now won't exit the loop. To avoid this problem, added a order check in the same condition, (i.e) when order is 0, we return -ENOSPC v2: use full name in email program and in Signed-off tag Signed-off-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/drm_buddy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 72f52f293249..5ab66aaf2bbd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -685,7 +685,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, if (!IS_ERR(block)) break; - if (order-- == min_order) { + if (!order || order-- == min_order) { It shouldn't be possible to enter an infinite loop here, without first tripping up the BUG_ON(order < min_order) further up, and for that, as we discussed here[1], it sounded like the conclusion was to rather add a simple check somewhere in drm_buddy_alloc_blocks() to reject any size not aligned to the min_page_size? [1] https://patchwork.freedesktop.org/patch/477414/?series=101108&rev=1 err = -ENOSPC; goto err_free; } base-commit: 3bd60c0259406c5ca3ce5cdc958fb910ad4b8175
Re: [PATCH] drm: Fix a infinite loop condition when order becomes 0
On 14/03/2022 19:40, Arunpravin wrote: handle a situation in the condition order-- == min_order, when order = 0, leading to order = -1, it now won't exit the loop. To avoid this problem, added a order check in the same condition, (i.e) when order is 0, we return -ENOSPC Signed-off-by: Arunpravin Hmm, it sounded like we were instead going to go with the round_up(size, min_page_size), or check and bail if the size is misaligned, in which case we don't need this, AFAICT. --- drivers/gpu/drm/drm_buddy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 72f52f293249..5ab66aaf2bbd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -685,7 +685,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, if (!IS_ERR(block)) break; - if (order-- == min_order) { + if (!order || order-- == min_order) { err = -ENOSPC; goto err_free; }
Re: [Intel-gfx] [PATCH] drm/i915: round_up the size to the alignment value
On Mon, 14 Mar 2022 at 19:32, Arunpravin wrote: > > handle instances when size is not aligned with the min_page_size. > Unigine Heaven has allocation requests for example required pages > are 161 and alignment request is 128. To allocate the left over > 33 pages, continues the iteration to find the order value which > is 5 and 0 and when it compares with min_order = 7, triggers the > BUG_ON((order < min_order). To avoid this problem, round_up the > size to the alignment and enable the is_contiguous variable and > the block trimmed to the original size. > > Signed-off-by: Arunpravin > --- > drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 13 +++-- > 1 file changed, 11 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c > b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c > index 129f668f21ff..318aa731de5b 100644 > --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c > +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c > @@ -40,6 +40,7 @@ static int i915_ttm_buddy_man_alloc(struct > ttm_resource_manager *man, > struct i915_ttm_buddy_resource *bman_res; > struct drm_buddy *mm = &bman->mm; > unsigned long n_pages, lpfn; > + bool is_contiguous = 0; > u64 min_page_size; > u64 size; > int err; > @@ -48,6 +49,9 @@ static int i915_ttm_buddy_man_alloc(struct > ttm_resource_manager *man, > if (!lpfn) > lpfn = man->size; > > + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) > + is_contiguous = 1; > + > bman_res = kzalloc(sizeof(*bman_res), GFP_KERNEL); > if (!bman_res) > return -ENOMEM; > @@ -71,7 +75,12 @@ static int i915_ttm_buddy_man_alloc(struct > ttm_resource_manager *man, > > GEM_BUG_ON(min_page_size < mm->chunk_size); > > - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { > + if (!is_contiguous && !IS_ALIGNED(size, min_page_size)) { > + size = round_up(size, min_page_size); > + is_contiguous = 1; > + } > + > + if (is_contiguous) { This looks like it will also do roundup_power_of_two()? I assume we instead just want to feed the list_last_entry() block for trimming? Anway, we should be able to just make this: if (WARN_ON(!IS_ALIGNED(size, min_page_size)) return -EINVAL; That's at least the currently expected behaviour in i915, just that we were previously not verifying it here. > unsigned long pages; > > size = roundup_pow_of_two(size); > @@ -106,7 +115,7 @@ static int i915_ttm_buddy_man_alloc(struct > ttm_resource_manager *man, > if (unlikely(err)) > goto err_free_blocks; > > - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { > + if (is_contiguous) { > u64 original_size = (u64)bman_res->base.num_pages << > PAGE_SHIFT; > > mutex_lock(&bman->lock); > > base-commit: b37605de46fef48555bf0cf463cccf355c51fac0 > -- > 2.25.1 >
Re: [PATCH] drm: remove min_order BUG_ON check
On 10/03/2022 14:47, Arunpravin wrote: On 08/03/22 10:31 pm, Matthew Auld wrote: On 08/03/2022 13:59, Arunpravin wrote: On 07/03/22 10:11 pm, Matthew Auld wrote: On 07/03/2022 14:37, Arunpravin wrote: place BUG_ON(order < min_order) outside do..while loop as it fails Unigine Heaven benchmark. Unigine Heaven has buffer allocation requests for example required pages are 161 and alignment request is 128. To allocate the remaining 33 pages, continues the iteration to find the order value which is 5 and when it compares with min_order = 7, enables the BUG_ON(). To avoid this problem, placed the BUG_ON check outside of do..while loop. Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 72f52f293249..ed94c56b720f 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -669,10 +669,11 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, order = fls(pages) - 1; min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + BUG_ON(order < min_order); Isn't the issue that we are allowing a size that is not aligned to the requested min_page_size? Should we not fix the caller(and throw a normal error here), or perhaps add the round_up() here instead? CASE 1: when size is not aligned to the requested min_page_size, for instance, required size = 161 pages, min_page_size = 128 pages, here we have 3 possible options, a. AFAIK,This kind of situation is common in any workload,the first allocation (i.e) 128 pages is aligned to min_page_size, Should we just allocate the left over 33 pages (2 pow 5, 2 pow 0) since the caller does know the left over pages are not in min_page_size alignment? So IIUC looking at amdgpu_gem_create_ioctl(), userspace can specify some arbitrary physical alignment for an object? Is that not meant to apply to every page/chunk? The above example would only have the correct physical alignment guaranteed for the first chunk, or so, is this the expected ABI behaviour? I gone through the function amdgpu_gem_create_ioctl(), it reads the physical alignment in bytes from userspace, does i915 round up the size value to the alignment or does i915 fails the allocation request if size is not aligned with min_page_size? If not, I think running unigine heaven or similar benchmark triggers BUG_ON() on current version of drm buddy i915 will always round_up the obj->base.size as per the default_page_size. But in our case the default_page_size is selected by the kernel, which is always either PAGE_SIZE, or 64K on some platforms, due to the HW having some minimum GPU page-size for mapping VRAM pages. We don't currently have anything similar to amdgpu_gem_create_in.alignment, where userspace can request some arbitrary physical alignment. Also looking at this some more, the other related bug here is the order-- == min_order check, since it now won't bail when order == 0, leading to order = -1, if we are unlucky... will add a fix Originally, if asking for min_page_size > chunk_size, then the allocation was meant to fail if it can't fill the resource request with pages of at least that size(and also alignment). Or at least that was the original meaning in i915 IIRC. we can follow the same here too, failing the allocation request if size is not aligned with min_page_size? Yeah, seems reasonable to me. I added a debug print for requested num_pages from userspace and its alignment request and executed unigine heaven, I see many such instances where min_page_size is not aligned to the size, how i915 handles such requests? b. There are many such instances in unigine heaven workload (there would be many such workloads), throwing a normal error would lower the FPS? is it possible to fix at caller application? c. adding the round_up() is possible, but in every such instances we end up allocating extra unused memory. For example, if required pages = 1028 and min_page_size = 1024 pages, we end up round up of left over 4 pages to the min_page_size, so the total size would be 2048 pages. i.e if someone does: alloc_blocks(mm, 0, end, 4096, 1<<16, &blocks, flags); CASE 2: I think this case should be detected (i.e) when min_page_size > size, should we return -EINVAL? This will still trigger the BUG_ON() even if we move it out of the loop, AFAICT. Should we just allow the CASE 1 proceed for the allocation and return -EINVAL for the CASE 2? + do { order = min(order, (unsigned int)fls(pages) - 1); BUG_ON(order > mm->max_order); - BUG_ON(order < min_order); do { if (flags & DRM_BUDDY_RANGE_ALLOCATION) base-commit: 8025c79350b90e5a8029234d433578f12abbae2b
Re: [PATCH] drm: remove min_order BUG_ON check
On 08/03/2022 13:59, Arunpravin wrote: On 07/03/22 10:11 pm, Matthew Auld wrote: On 07/03/2022 14:37, Arunpravin wrote: place BUG_ON(order < min_order) outside do..while loop as it fails Unigine Heaven benchmark. Unigine Heaven has buffer allocation requests for example required pages are 161 and alignment request is 128. To allocate the remaining 33 pages, continues the iteration to find the order value which is 5 and when it compares with min_order = 7, enables the BUG_ON(). To avoid this problem, placed the BUG_ON check outside of do..while loop. Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 72f52f293249..ed94c56b720f 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -669,10 +669,11 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, order = fls(pages) - 1; min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + BUG_ON(order < min_order); Isn't the issue that we are allowing a size that is not aligned to the requested min_page_size? Should we not fix the caller(and throw a normal error here), or perhaps add the round_up() here instead? CASE 1: when size is not aligned to the requested min_page_size, for instance, required size = 161 pages, min_page_size = 128 pages, here we have 3 possible options, a. AFAIK,This kind of situation is common in any workload,the first allocation (i.e) 128 pages is aligned to min_page_size, Should we just allocate the left over 33 pages (2 pow 5, 2 pow 0) since the caller does know the left over pages are not in min_page_size alignment? So IIUC looking at amdgpu_gem_create_ioctl(), userspace can specify some arbitrary physical alignment for an object? Is that not meant to apply to every page/chunk? The above example would only have the correct physical alignment guaranteed for the first chunk, or so, is this the expected ABI behaviour? Also looking at this some more, the other related bug here is the order-- == min_order check, since it now won't bail when order == 0, leading to order = -1, if we are unlucky... Originally, if asking for min_page_size > chunk_size, then the allocation was meant to fail if it can't fill the resource request with pages of at least that size(and also alignment). Or at least that was the original meaning in i915 IIRC. b. There are many such instances in unigine heaven workload (there would be many such workloads), throwing a normal error would lower the FPS? is it possible to fix at caller application? c. adding the round_up() is possible, but in every such instances we end up allocating extra unused memory. For example, if required pages = 1028 and min_page_size = 1024 pages, we end up round up of left over 4 pages to the min_page_size, so the total size would be 2048 pages. i.e if someone does: alloc_blocks(mm, 0, end, 4096, 1<<16, &blocks, flags); CASE 2: I think this case should be detected (i.e) when min_page_size > size, should we return -EINVAL? This will still trigger the BUG_ON() even if we move it out of the loop, AFAICT. Should we just allow the CASE 1 proceed for the allocation and return -EINVAL for the CASE 2? + do { order = min(order, (unsigned int)fls(pages) - 1); BUG_ON(order > mm->max_order); - BUG_ON(order < min_order); do { if (flags & DRM_BUDDY_RANGE_ALLOCATION) base-commit: 8025c79350b90e5a8029234d433578f12abbae2b
Re: [PATCH] drm: remove min_order BUG_ON check
On 07/03/2022 14:37, Arunpravin wrote: place BUG_ON(order < min_order) outside do..while loop as it fails Unigine Heaven benchmark. Unigine Heaven has buffer allocation requests for example required pages are 161 and alignment request is 128. To allocate the remaining 33 pages, continues the iteration to find the order value which is 5 and when it compares with min_order = 7, enables the BUG_ON(). To avoid this problem, placed the BUG_ON check outside of do..while loop. Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 72f52f293249..ed94c56b720f 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -669,10 +669,11 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, order = fls(pages) - 1; min_order = ilog2(min_page_size) - ilog2(mm->chunk_size); + BUG_ON(order < min_order); Isn't the issue that we are allowing a size that is not aligned to the requested min_page_size? Should we not fix the caller(and throw a normal error here), or perhaps add the round_up() here instead? i.e if someone does: alloc_blocks(mm, 0, end, 4096, 1<<16, &blocks, flags); This will still trigger the BUG_ON() even if we move it out of the loop, AFAICT. + do { order = min(order, (unsigned int)fls(pages) - 1); BUG_ON(order > mm->max_order); - BUG_ON(order < min_order); do { if (flags & DRM_BUDDY_RANGE_ALLOCATION) base-commit: 8025c79350b90e5a8029234d433578f12abbae2b
Re: [PATCH v12 1/5] drm: improve drm_buddy_alloc function
On Mon, 14 Feb 2022 at 06:32, Christian König wrote: > > Am 13.02.22 um 09:52 schrieb Arunpravin: > > - Make drm_buddy_alloc a single function to handle > >range allocation and non-range allocation demands > > > > - Implemented a new function alloc_range() which allocates > >the requested power-of-two block comply with range limitations > > > > - Moved order computation and memory alignment logic from > >i915 driver to drm buddy > > > > v2: > >merged below changes to keep the build unbroken > > - drm_buddy_alloc_range() becomes obsolete and may be removed > > - enable ttm range allocation (fpfn / lpfn) support in i915 driver > > - apply enhanced drm_buddy_alloc() function to i915 driver > > > > v3(Matthew Auld): > >- Fix alignment issues and remove unnecessary list_empty check > >- add more validation checks for input arguments > >- make alloc_range() block allocations as bottom-up > > - optimize order computation logic > >- replace uint64_t with u64, which is preferred in the kernel > > > > v4(Matthew Auld): > >- keep drm_buddy_alloc_range() function implementation for generic > > actual range allocations > >- keep alloc_range() implementation for end bias allocations > > > > v5(Matthew Auld): > >- modify drm_buddy_alloc() passing argument place->lpfn to lpfn > > as place->lpfn will currently always be zero for i915 > > > > v6(Matthew Auld): > >- fixup potential uaf - If we are unlucky and can't allocate > > enough memory when splitting blocks, where we temporarily > > end up with the given block and its buddy on the respective > > free list, then we need to ensure we delete both blocks, > > and no just the buddy, before potentially freeing them > > > >- fix warnings reported by kernel test robot > > > > v7(Matthew Auld): > >- revert fixup potential uaf > >- keep __alloc_range() add node to the list logic same as > > drm_buddy_alloc_blocks() by having a temporary list variable > >- at drm_buddy_alloc_blocks() keep i915 range_overflows macro > > and add a new check for end variable > > > > v8: > >- fix warnings reported by kernel test robot > > > > v9(Matthew Auld): > >- remove DRM_BUDDY_RANGE_ALLOCATION flag > >- remove unnecessary function description > > > > Signed-off-by: Arunpravin > > Reviewed-by: Matthew Auld > > As long as nobody objects I'm going to push patches 1-3 to drm-misc-next > in the next hour or so: As part of this could you also push https://patchwork.freedesktop.org/series/99842/ ? > > Then going to take a deeper look into patches 4 and 5 to get them reviewed. > > Thanks, > Christian. > > > --- > > drivers/gpu/drm/drm_buddy.c | 292 +- > > drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 63 ++-- > > drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + > > include/drm/drm_buddy.h | 11 +- > > 4 files changed, 250 insertions(+), 118 deletions(-) > > > > diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c > > index d60878bc9c20..e0c0d786a572 100644 > > --- a/drivers/gpu/drm/drm_buddy.c > > +++ b/drivers/gpu/drm/drm_buddy.c > > @@ -282,23 +282,97 @@ void drm_buddy_free_list(struct drm_buddy *mm, struct > > list_head *objects) > > } > > EXPORT_SYMBOL(drm_buddy_free_list); > > > > -/** > > - * drm_buddy_alloc_blocks - allocate power-of-two blocks > > - * > > - * @mm: DRM buddy manager to allocate from > > - * @order: size of the allocation > > - * > > - * The order value here translates to: > > - * > > - * 0 = 2^0 * mm->chunk_size > > - * 1 = 2^1 * mm->chunk_size > > - * 2 = 2^2 * mm->chunk_size > > - * > > - * Returns: > > - * allocated ptr to the &drm_buddy_block on success > > - */ > > -struct drm_buddy_block * > > -drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order) > > +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) > > +{ > > + return s1 <= e2 && e1 >= s2; > > +} > > + > > +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) > > +{ > > + return s1 <= s2 && e1 >= e2; > > +} > > + > > +static struct drm_buddy_block * > > +alloc_range_bias(struct drm_buddy *mm, > > + u64 start, u64 end, > > + unsigned
Re: [PATCH v11 5/5] drm/amdgpu: add drm buddy support to amdgpu
On 08/02/2022 11:20, Arunpravin wrote: On 04/02/22 6:53 pm, Christian König wrote: Am 04.02.22 um 12:22 schrieb Arunpravin: On 28/01/22 7:48 pm, Matthew Auld wrote: On Thu, 27 Jan 2022 at 14:11, Arunpravin wrote: - Remove drm_mm references and replace with drm buddy functionalities - Add res cursor support for drm buddy v2(Matthew Auld): - replace spinlock with mutex as we call kmem_cache_zalloc (..., GFP_KERNEL) in drm_buddy_alloc() function - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - fix warnings reported by kernel test robot v5: - fix merge conflict issue v6: - fix warnings reported by kernel test robot Signed-off-by: Arunpravin --- drivers/gpu/drm/Kconfig | 1 + .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h| 97 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 7 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 259 ++ 4 files changed, 231 insertions(+), 133 deletions(-) -/** - * amdgpu_vram_mgr_virt_start - update virtual start address - * - * @mem: ttm_resource to update - * @node: just allocated node - * - * Calculate a virtual BO start address to easily check if everything is CPU - * accessible. - */ -static void amdgpu_vram_mgr_virt_start(struct ttm_resource *mem, - struct drm_mm_node *node) -{ - unsigned long start; - - start = node->start + node->size; - if (start > mem->num_pages) - start -= mem->num_pages; - else - start = 0; - mem->start = max(mem->start, start); -} - /** * amdgpu_vram_mgr_new - allocate new ranges * @@ -366,13 +357,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, const struct ttm_place *place, struct ttm_resource **res) { - unsigned long lpfn, num_nodes, pages_per_node, pages_left, pages; + unsigned long lpfn, pages_per_node, pages_left, pages, n_pages; + u64 vis_usage = 0, mem_bytes, max_bytes, min_page_size; struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - uint64_t vis_usage = 0, mem_bytes, max_bytes; - struct ttm_range_mgr_node *node; - struct drm_mm *mm = &mgr->mm; - enum drm_mm_insert_mode mode; + struct amdgpu_vram_mgr_node *node; + struct drm_buddy *mm = &mgr->mm; + struct drm_buddy_block *block; unsigned i; int r; @@ -391,10 +382,9 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, goto error_sub; } - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) pages_per_node = ~0ul; - num_nodes = 1; - } else { + else { #ifdef CONFIG_TRANSPARENT_HUGEPAGE pages_per_node = HPAGE_PMD_NR; #else @@ -403,11 +393,9 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, #endif pages_per_node = max_t(uint32_t, pages_per_node, tbo->page_alignment); - num_nodes = DIV_ROUND_UP_ULL(PFN_UP(mem_bytes), pages_per_node); } - node = kvmalloc(struct_size(node, mm_nodes, num_nodes), - GFP_KERNEL | __GFP_ZERO); + node = kzalloc(sizeof(*node), GFP_KERNEL); if (!node) { r = -ENOMEM; goto error_sub; @@ -415,9 +403,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, ttm_resource_init(tbo, place, &node->base); - mode = DRM_MM_INSERT_BEST; + INIT_LIST_HEAD(&node->blocks); + if (place->flags & TTM_PL_FLAG_TOPDOWN) - mode = DRM_MM_INSERT_HIGH; + node->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + + if (place->fpfn || lpfn != man->size) + /* Allocate blocks in desired range */ + node->flags |= DRM_BUDDY_RANGE_ALLOCATION; + + min_page_size = mgr->default_page_size; + BUG_ON(min_page_size < mm->chunk_size); pages_left = node->base.num_pages; @@ -425,36 +421,61 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, pages = min(pages_left, 2UL << (30 - PAGE_SHIFT)); i = 0; - spin_lock(&mgr->lock); while (pages_left) { - uint32_t alignment = tbo->page_alignment; - if (pages >= pages_per_node) - alignment = pages_per_node; - - r = drm_mm_insert_no
Re: [PATCH v11 1/5] drm: improve drm_buddy_alloc function
On 27/01/2022 14:11, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them - fix warnings reported by kernel test robot v7(Matthew Auld): - revert fixup potential uaf - keep __alloc_range() add node to the list logic same as drm_buddy_alloc_blocks() by having a temporary list variable - at drm_buddy_alloc_blocks() keep i915 range_overflows macro and add a new check for end variable v8: - fix warnings reported by kernel test robot Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 315 +- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 13 +- 4 files changed, 280 insertions(+), 117 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index d60878bc9c20..cfc160a1ef1a 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -282,23 +282,97 @@ void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc_blocks - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order) +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= s2 && e1 >= e2; +} + +static struct drm_buddy_block * +alloc_range_bias(struct drm_buddy *mm, +u64 start, u64 end, +unsigned int order) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (drm_buddy_block_order(block) < order) + continue; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (drm_buddy_block_is_allocated(block)) + continue; + + if (contains(start, end, block_start, block_end) && + order == drm_buddy_block_order(block)) { + /* +* Find the free block within the range. +*/ + if (drm_buddy_block_is_free(block)) + return block; + + continue; +
Re: [PATCH 2/9] drm/ttm: move the LRU into resource handling v3
On Wed, 9 Feb 2022 at 08:41, Christian König wrote: > > This way we finally fix the problem that new resource are > not immediately evict-able after allocation. > > That has caused numerous problems including OOM on GDS handling > and not being able to use TTM as general resource manager. > > v2: stop assuming in ttm_resource_fini that res->bo is still valid. > v3: cleanup kerneldoc, add more lockdep annotation > > Signed-off-by: Christian König > Tested-by: Bas Nieuwenhuizen > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 +- > drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 2 +- > drivers/gpu/drm/ttm/ttm_bo.c| 115 ++ > drivers/gpu/drm/ttm/ttm_bo_util.c | 1 - > drivers/gpu/drm/ttm/ttm_device.c| 64 ++--- > drivers/gpu/drm/ttm/ttm_resource.c | 122 +++- > include/drm/ttm/ttm_bo_api.h| 16 > include/drm/ttm/ttm_bo_driver.h | 29 +- > include/drm/ttm/ttm_resource.h | 35 +++ > 9 files changed, 197 insertions(+), 195 deletions(-) > /** > * ttm_resource_init - resource object constructure > * @bo: buffer object this resources is allocated for > @@ -52,10 +156,12 @@ void ttm_resource_init(struct ttm_buffer_object *bo, > res->bus.is_iomem = false; > res->bus.caching = ttm_cached; > res->bo = bo; > + INIT_LIST_HEAD(&res->lru); > > man = ttm_manager_type(bo->bdev, place->mem_type); > spin_lock(&bo->bdev->lru_lock); > man->usage += bo->base.size; > + ttm_resource_move_to_lru_tail(res, NULL); > spin_unlock(&bo->bdev->lru_lock); > } > EXPORT_SYMBOL(ttm_resource_init); > @@ -66,15 +172,21 @@ EXPORT_SYMBOL(ttm_resource_init); > * @res: the resource to clean up > * > * Should be used by resource manager backends to clean up the TTM resource > - * objects before freeing the underlying structure. Counterpart of > - * &ttm_resource_init > + * objects before freeing the underlying structure. Makes sure the resource > is > + * removed from the LRU before destruction. > + * Counterpart of &ttm_resource_init. > */ > void ttm_resource_fini(struct ttm_resource_manager *man, >struct ttm_resource *res) > { > - spin_lock(&man->bdev->lru_lock); > - man->usage -= res->bo->base.size; > - spin_unlock(&man->bdev->lru_lock); > + struct ttm_device *bdev = man->bdev; > + > + spin_lock(&bdev->lru_lock); > + list_del_init(&res->lru); > + if (res->bo && bdev->funcs->del_from_lru_notify) > + bdev->funcs->del_from_lru_notify(res->bo); > + man->usage -= res->num_pages << PAGE_SHIFT; Above we are using the bo->base.size for tracking usage, but here we are using num_pages. Is it guaranteed that bo->base.size is always page aligned? Do we need some kind of WARN_ON()? Perhaps also sanity checking that usage == 0 when tearing down the man?
Re: [PATCH 6/9] drm/amdgpu: remove VRAM accounting
On Wed, 9 Feb 2022 at 08:41, Christian König wrote: > > This is provided by TTM now. > > Also switch man->size to bytes instead of pages and fix the double > printing of size and usage in debugfs. > > Signed-off-by: Christian König > Tested-by: Bas Nieuwenhuizen > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 6 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 - > drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 6 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 58 +++- > 6 files changed, 31 insertions(+), 45 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > index e8440d306496..025748e9c772 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > @@ -314,7 +314,7 @@ static void amdgpu_cs_get_threshold_for_moves(struct > amdgpu_device *adev, > } > > total_vram = adev->gmc.real_vram_size - > atomic64_read(&adev->vram_pin_size); > - used_vram = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); > + used_vram = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); > free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram; > > spin_lock(&adev->mm_stats.lock); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > index 9ff4aced5da7..0beab961b18b 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > @@ -678,7 +678,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, > struct drm_file *filp) > ui64 = atomic64_read(&adev->num_vram_cpu_page_faults); > return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; > case AMDGPU_INFO_VRAM_USAGE: > - ui64 = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); > + ui64 = > ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); > return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; > case AMDGPU_INFO_VIS_VRAM_USAGE: > ui64 = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr); > @@ -717,6 +717,8 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, > struct drm_file *filp) > struct drm_amdgpu_memory_info mem; > struct ttm_resource_manager *gtt_man = > &adev->mman.gtt_mgr.manager; > + struct ttm_resource_manager *vram_man = > + &adev->mman.vram_mgr.manager; > > memset(&mem, 0, sizeof(mem)); > mem.vram.total_heap_size = adev->gmc.real_vram_size; > @@ -724,7 +726,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, > struct drm_file *filp) > atomic64_read(&adev->vram_pin_size) - > AMDGPU_VM_RESERVED_VRAM; > mem.vram.heap_usage = > - amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); > + ttm_resource_manager_usage(vram_man); > mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; > > mem.cpu_accessible_vram.total_heap_size = > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > index d178fbec7048..5859ed0552a4 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > @@ -1884,7 +1884,7 @@ void amdgpu_ttm_set_buffer_funcs_status(struct > amdgpu_device *adev, bool enable) > size = adev->gmc.real_vram_size; > else > size = adev->gmc.visible_vram_size; > - man->size = size >> PAGE_SHIFT; > + man->size = size; > adev->mman.buffer_funcs_enabled = enable; > } > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h > b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h > index 120b69ec9885..cbee84a77331 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h > @@ -44,7 +44,6 @@ struct amdgpu_vram_mgr { > spinlock_t lock; > struct list_head reservations_pending; > struct list_head reserved_pages; > - atomic64_t usage; > atomic64_t vis_usage; > }; > > @@ -127,7 +126,6 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, > void amdgpu_vram_mgr_free_sgt(struct device *dev, > enum dma_data_direction dir, > struct sg_table *sgt); > -uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr); > uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr); > int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, > uint64_t start, uint64_t size); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
Re: [PATCH 1/7] drm/selftests: Move i915 buddy selftests into drm
On 03/02/2022 13:32, Arunpravin wrote: - move i915 buddy selftests into drm selftests folder - add Makefile and Kconfig support - add sanitycheck testcase Prerequisites - These series of selftests patches are created on top of drm buddy series - Enable kselftests for DRM as a module in .config Signed-off-by: Arunpravin At some point I guess we also want some IGT that picks this up? Like we do in tests/drm_mm.c? That way this can get picked up by CI? Acked-by: Matthew Auld --- drivers/gpu/drm/Kconfig | 1 + drivers/gpu/drm/selftests/Makefile| 3 +- .../gpu/drm/selftests/drm_buddy_selftests.h | 9 drivers/gpu/drm/selftests/test-drm_buddy.c| 49 +++ 4 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/selftests/drm_buddy_selftests.h create mode 100644 drivers/gpu/drm/selftests/test-drm_buddy.c diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index eb5a57ae3c5c..ff856df3f97f 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -71,6 +71,7 @@ config DRM_DEBUG_SELFTEST select DRM_DP_HELPER select DRM_LIB_RANDOM select DRM_KMS_HELPER + select DRM_BUDDY select DRM_EXPORT_FOR_TESTS if m default n help diff --git a/drivers/gpu/drm/selftests/Makefile b/drivers/gpu/drm/selftests/Makefile index 0856e4b12f70..5ba5f9138c95 100644 --- a/drivers/gpu/drm/selftests/Makefile +++ b/drivers/gpu/drm/selftests/Makefile @@ -4,4 +4,5 @@ test-drm_modeset-y := test-drm_modeset_common.o test-drm_plane_helper.o \ test-drm_damage_helper.o test-drm_dp_mst_helper.o \ test-drm_rect.o -obj-$(CONFIG_DRM_DEBUG_SELFTEST) += test-drm_mm.o test-drm_modeset.o test-drm_cmdline_parser.o +obj-$(CONFIG_DRM_DEBUG_SELFTEST) += test-drm_mm.o test-drm_modeset.o test-drm_cmdline_parser.o \ + test-drm_buddy.o diff --git a/drivers/gpu/drm/selftests/drm_buddy_selftests.h b/drivers/gpu/drm/selftests/drm_buddy_selftests.h new file mode 100644 index ..a4bcf3a6dfe3 --- /dev/null +++ b/drivers/gpu/drm/selftests/drm_buddy_selftests.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* List each unit test as selftest(name, function) + * + * The name is used as both an enum and expanded as igt__name to create + * a module parameter. It must be unique and legal for a C identifier. + * + * Tests are executed in order by igt/drm_buddy + */ +selftest(sanitycheck, igt_sanitycheck) /* keep first (selfcheck for igt) */ diff --git a/drivers/gpu/drm/selftests/test-drm_buddy.c b/drivers/gpu/drm/selftests/test-drm_buddy.c new file mode 100644 index ..51e4d393d22c --- /dev/null +++ b/drivers/gpu/drm/selftests/test-drm_buddy.c @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2019 Intel Corporation + */ + +#define pr_fmt(fmt) "drm_buddy: " fmt + +#include + +#include + +#include "../lib/drm_random.h" + +#define TESTS "drm_buddy_selftests.h" +#include "drm_selftest.h" + +static unsigned int random_seed; + +static int igt_sanitycheck(void *ignored) +{ + pr_info("%s - ok!\n", __func__); + return 0; +} + +#include "drm_selftest.c" + +static int __init test_drm_buddy_init(void) +{ + int err; + + while (!random_seed) + random_seed = get_random_int(); + + pr_info("Testing DRM buddy manager (struct drm_buddy), with random_seed=0x%x\n", + random_seed); + err = run_selftests(selftests, ARRAY_SIZE(selftests), NULL); + + return err > 0 ? 0 : err; +} + +static void __exit test_drm_buddy_exit(void) +{ +} + +module_init(test_drm_buddy_init); +module_exit(test_drm_buddy_exit); + +MODULE_AUTHOR("Intel Corporation"); +MODULE_LICENSE("GPL");
Re: [PATCH 7/7] drm/selftests: add drm buddy pathological testcase
On 03/02/2022 13:32, Arunpravin wrote: create a pot-sized mm, then allocate one of each possible order within. This should leave the mm with exactly one page left. Free the largest block, then whittle down again. Eventually we will have a fully 50% fragmented mm. Signed-off-by: Arunpravin --- .../gpu/drm/selftests/drm_buddy_selftests.h | 1 + drivers/gpu/drm/selftests/test-drm_buddy.c| 136 ++ 2 files changed, 137 insertions(+) diff --git a/drivers/gpu/drm/selftests/drm_buddy_selftests.h b/drivers/gpu/drm/selftests/drm_buddy_selftests.h index 411d072cbfc5..455b756c4ae5 100644 --- a/drivers/gpu/drm/selftests/drm_buddy_selftests.h +++ b/drivers/gpu/drm/selftests/drm_buddy_selftests.h @@ -12,3 +12,4 @@ selftest(buddy_alloc_range, igt_buddy_alloc_range) selftest(buddy_alloc_optimistic, igt_buddy_alloc_optimistic) selftest(buddy_alloc_pessimistic, igt_buddy_alloc_pessimistic) selftest(buddy_alloc_smoke, igt_buddy_alloc_smoke) +selftest(buddy_alloc_pathological, igt_buddy_alloc_pathological) diff --git a/drivers/gpu/drm/selftests/test-drm_buddy.c b/drivers/gpu/drm/selftests/test-drm_buddy.c index 2074e8c050a4..b2d0313a4bc5 100644 --- a/drivers/gpu/drm/selftests/test-drm_buddy.c +++ b/drivers/gpu/drm/selftests/test-drm_buddy.c @@ -338,6 +338,142 @@ static void igt_mm_config(u64 *size, u64 *chunk_size) *size = (u64)s << 12; } +static int igt_buddy_alloc_pathological(void *arg) +{ + u64 mm_size, size, min_page_size, start = 0; + struct drm_buddy_block *block; + const int max_order = 3; + unsigned long flags = 0; + int order, top, err; + struct drm_buddy mm; + LIST_HEAD(blocks); + LIST_HEAD(holes); + LIST_HEAD(tmp); + + /* +* Create a pot-sized mm, then allocate one of each possible +* order within. This should leave the mm with exactly one +* page left. Free the largest block, then whittle down again. +* Eventually we will have a fully 50% fragmented mm. +*/ + + mm_size = PAGE_SIZE << max_order; + err = drm_buddy_init(&mm, mm_size, PAGE_SIZE); + if (err) { + pr_err("buddy_init failed(%d)\n", err); + return err; + } + BUG_ON(mm.max_order != max_order); + + for (top = max_order; top; top--) { + /* Make room by freeing the largest allocated block */ + block = list_first_entry_or_null(&blocks, typeof(*block), link); + if (block) { + list_del(&block->link); + drm_buddy_free_block(&mm, block); + } + + for (order = top; order--; ) { + size = min_page_size = get_size(order, PAGE_SIZE); + err = drm_buddy_alloc_blocks(&mm, start, mm_size, size, +min_page_size, &tmp, flags); + if (err) { + pr_info("buddy_alloc hit -ENOMEM with order=%d, top=%d\n", + order, top); + goto err; + } + + block = list_first_entry_or_null(&tmp, +struct drm_buddy_block, +link); + if (!block) { + pr_err("alloc_blocks has no blocks\n"); + err = -EINVAL; + goto err; + } + + list_del(&block->link); + list_add_tail(&block->link, &blocks); + } + + /* There should be one final page for this sub-allocation */ + size = min_page_size = get_size(0, PAGE_SIZE); + err = drm_buddy_alloc_blocks(&mm, start, mm_size, size, min_page_size, &tmp, flags); + if (err) { + pr_info("buddy_alloc hit -ENOME for hole\n"); ENOMEM Reviewed-by: Matthew Auld + goto err; + } + + block = list_first_entry_or_null(&tmp, +struct drm_buddy_block, +link); + if (!block) { + pr_err("alloc_blocks has no blocks\n"); + err = -EINVAL; + goto err; + } + + list_del(&block->link); + list_add_tail(&block->link, &holes); + + size = min_page_size = get_size(top, PAGE_SIZE); + err = drm_buddy_alloc_blocks(&mm, start, mm_size, size, min_page_size, &tmp, flags); + if (!err) { +
Re: [PATCH 6/7] drm/selftests: add drm buddy smoke testcase
; + } + + list_del(&block->link); + list_add_tail(&block->link, &blocks); Could just make this list_move_tail()? Elsewhere also. Anyway, Reviewed-by: Matthew Auld + + if (drm_buddy_block_order(block) != order) { + pr_err("buddy_alloc order mismatch\n"); + err = -EINVAL; + break; + } + + total += drm_buddy_block_size(&mm, block); + + if (__igt_timeout(end_time, NULL)) { + timeout = true; + break; + } + } while (total < mm.size); + + if (!err) + err = igt_check_blocks(&mm, &blocks, total, false); + + drm_buddy_free_list(&mm, &blocks); + + if (!err) { + err = igt_check_mm(&mm); + if (err) + pr_err("post-mm check failed\n"); + } + + if (err || timeout) + break; + + cond_resched(); + } + + if (err == -ENOMEM) + err = 0; + + if (!err) + pr_info("%s - succeeded\n", __func__); + + kfree(order); +out_fini: + drm_buddy_fini(&mm); + + return err; +} + static int igt_buddy_alloc_pessimistic(void *arg) { u64 mm_size, size, min_page_size, start = 0;
Re: [PATCH 5/7] drm/selftests: add drm buddy pessimistic testcase
ntry(&blocks, typeof(*block), link); + list_del(&block->link); + drm_buddy_free_block(&mm, block); + + /* As we free in increasing size, we make available larger blocks */ + order = 1; + list_for_each_entry_safe(block, bn, &blocks, link) { + list_del(&block->link); + drm_buddy_free_block(&mm, block); + + size = min_page_size = get_size(order, PAGE_SIZE); + err = drm_buddy_alloc_blocks(&mm, start, mm_size, size, min_page_size, &tmp, flags); + if (err) { + pr_info("buddy_alloc (realloc) hit -ENOMEM with order=%d\n", + order); + goto err; + } + + block = list_first_entry_or_null(&tmp, +struct drm_buddy_block, +link); + if (!block) { + pr_err("alloc_blocks has no blocks\n"); + err = -EINVAL; + goto err; + } + + list_del(&block->link); + drm_buddy_free_block(&mm, block); + order++; + } + + /* To confirm, now the whole mm should be available */ + size = min_page_size = get_size(max_order, PAGE_SIZE); + err = drm_buddy_alloc_blocks(&mm, start, mm_size, size, min_page_size, &tmp, flags); + if (err) { + pr_info("buddy_alloc (realloc) hit -ENOMEM with order=%d\n", + max_order); + goto err; + } + + block = list_first_entry_or_null(&tmp, +struct drm_buddy_block, +link); + if (!block) { + pr_err("alloc_blocks has no blocks\n"); + err = -EINVAL; + goto err; + } + + list_del(&block->link); + drm_buddy_free_block(&mm, block); + + if (!err) Always true? Reviewed-by: Matthew Auld + pr_info("%s - succeeded\n", __func__); + +err: + drm_buddy_free_list(&mm, &blocks); + drm_buddy_fini(&mm); + return err; +} + static int igt_buddy_alloc_optimistic(void *arg) { u64 mm_size, size, min_page_size, start = 0;
Re: [PATCH 4/7] drm/selftests: add drm buddy optimistic testcase
On 03/02/2022 13:32, Arunpravin wrote: create a mm with one block of each order available, and try to allocate them all. Signed-off-by: Arunpravin Reviewed-by: Matthew Auld
Re: [PATCH 3/7] drm/selftests: add drm buddy alloc range testcase
On 03/02/2022 13:32, Arunpravin wrote: - add a test to check the range allocation - export get_buddy() function in drm_buddy.c - export drm_prandom_u32_max_state() in lib/drm_random.c - include helper functions - include prime number header file Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 20 +- drivers/gpu/drm/lib/drm_random.c | 3 +- drivers/gpu/drm/lib/drm_random.h | 2 + .../gpu/drm/selftests/drm_buddy_selftests.h | 1 + drivers/gpu/drm/selftests/test-drm_buddy.c| 390 ++ include/drm/drm_buddy.h | 3 + 6 files changed, 414 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 4845ef784b5e..501229d843c4 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -211,7 +211,7 @@ static int split_block(struct drm_buddy *mm, } static struct drm_buddy_block * -get_buddy(struct drm_buddy_block *block) +__get_buddy(struct drm_buddy_block *block) { struct drm_buddy_block *parent; @@ -225,6 +225,18 @@ get_buddy(struct drm_buddy_block *block) return parent->left; } +/** + * drm_get_buddy - get buddy address Maybe add some more info here: "Return the corresponding buddy block for @block, or NULL if this is a root block and can't be merged further. Requires some kind of locking to protect against any concurrent allocate and free operations." ? Anyway, Reviewed-by: Matthew Auld + * + * @block: DRM buddy block + */ +struct drm_buddy_block * +drm_get_buddy(struct drm_buddy_block *block) +{ + return __get_buddy(block); +} +EXPORT_SYMBOL(drm_get_buddy); + static void __drm_buddy_free(struct drm_buddy *mm, struct drm_buddy_block *block) { @@ -233,7 +245,7 @@ static void __drm_buddy_free(struct drm_buddy *mm, while ((parent = block->parent)) { struct drm_buddy_block *buddy; - buddy = get_buddy(block); + buddy = __get_buddy(block); if (!drm_buddy_block_is_free(buddy)) break; @@ -361,7 +373,7 @@ alloc_range_bias(struct drm_buddy *mm, * bigger is better, so make sure we merge everything back before we * free the allocated blocks. */ - buddy = get_buddy(block); + buddy = __get_buddy(block); if (buddy && (drm_buddy_block_is_free(block) && drm_buddy_block_is_free(buddy))) @@ -500,7 +512,7 @@ static int __alloc_range(struct drm_buddy *mm, * bigger is better, so make sure we merge everything back before we * free the allocated blocks. */ - buddy = get_buddy(block); + buddy = __get_buddy(block); if (buddy && (drm_buddy_block_is_free(block) && drm_buddy_block_is_free(buddy))) diff --git a/drivers/gpu/drm/lib/drm_random.c b/drivers/gpu/drm/lib/drm_random.c index eeb155826d27..31b5a3e21911 100644 --- a/drivers/gpu/drm/lib/drm_random.c +++ b/drivers/gpu/drm/lib/drm_random.c @@ -7,10 +7,11 @@ #include "drm_random.h" -static inline u32 drm_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state) +u32 drm_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state) { return upper_32_bits((u64)prandom_u32_state(state) * ep_ro); } +EXPORT_SYMBOL(drm_prandom_u32_max_state); void drm_random_reorder(unsigned int *order, unsigned int count, struct rnd_state *state) diff --git a/drivers/gpu/drm/lib/drm_random.h b/drivers/gpu/drm/lib/drm_random.h index 4a3e94dfa0c0..5543bf0474bc 100644 --- a/drivers/gpu/drm/lib/drm_random.h +++ b/drivers/gpu/drm/lib/drm_random.h @@ -22,5 +22,7 @@ unsigned int *drm_random_order(unsigned int count, void drm_random_reorder(unsigned int *order, unsigned int count, struct rnd_state *state); +u32 drm_prandom_u32_max_state(u32 ep_ro, + struct rnd_state *state); #endif /* !__DRM_RANDOM_H__ */ diff --git a/drivers/gpu/drm/selftests/drm_buddy_selftests.h b/drivers/gpu/drm/selftests/drm_buddy_selftests.h index ebe16162762f..3230bfd2770b 100644 --- a/drivers/gpu/drm/selftests/drm_buddy_selftests.h +++ b/drivers/gpu/drm/selftests/drm_buddy_selftests.h @@ -8,3 +8,4 @@ */ selftest(sanitycheck, igt_sanitycheck) /* keep first (selfcheck for igt) */ selftest(buddy_alloc_limit, igt_buddy_alloc_limit) +selftest(buddy_alloc_range, igt_buddy_alloc_range) diff --git a/drivers/gpu/drm/selftests/test-drm_buddy.c b/drivers/gpu/drm/selftests/test-drm_buddy.c index fd7d1a112458..e347060c05a2 100644 --- a/drivers/gpu/drm/selftests/test-drm_buddy.c +++ b/drivers/gpu/drm/selftests/test-drm_buddy.c @@ -6,6 +6,7 @@ #define pr_fmt(fmt) "drm_buddy: " fmt #include +#include #include @@ -16,6 +17,395 @@ st
Re: [PATCH 2/7] drm/selftests: add drm buddy alloc limit testcase
On 03/02/2022 13:32, Arunpravin wrote: add a test to check the maximum allocation limit Signed-off-by: Arunpravin --- .../gpu/drm/selftests/drm_buddy_selftests.h | 1 + drivers/gpu/drm/selftests/test-drm_buddy.c| 60 +++ 2 files changed, 61 insertions(+) diff --git a/drivers/gpu/drm/selftests/drm_buddy_selftests.h b/drivers/gpu/drm/selftests/drm_buddy_selftests.h index a4bcf3a6dfe3..ebe16162762f 100644 --- a/drivers/gpu/drm/selftests/drm_buddy_selftests.h +++ b/drivers/gpu/drm/selftests/drm_buddy_selftests.h @@ -7,3 +7,4 @@ * Tests are executed in order by igt/drm_buddy */ selftest(sanitycheck, igt_sanitycheck) /* keep first (selfcheck for igt) */ +selftest(buddy_alloc_limit, igt_buddy_alloc_limit) diff --git a/drivers/gpu/drm/selftests/test-drm_buddy.c b/drivers/gpu/drm/selftests/test-drm_buddy.c index 51e4d393d22c..fd7d1a112458 100644 --- a/drivers/gpu/drm/selftests/test-drm_buddy.c +++ b/drivers/gpu/drm/selftests/test-drm_buddy.c @@ -16,6 +16,66 @@ static unsigned int random_seed; +static int igt_buddy_alloc_limit(void *arg) +{ + u64 end, size = U64_MAX, start = 0; + struct drm_buddy_block *block; + unsigned long flags = 0; + LIST_HEAD(allocated); + struct drm_buddy mm; + int err; + + size = end = round_down(size, 4096); + err = drm_buddy_init(&mm, size, PAGE_SIZE); + if (err) + return err; + + if (mm.max_order != DRM_BUDDY_MAX_ORDER) { + pr_err("mm.max_order(%d) != %d\n", + mm.max_order, DRM_BUDDY_MAX_ORDER); + err = -EINVAL; + goto out_fini; + } + + err = drm_buddy_alloc_blocks(&mm, start, end, size, +PAGE_SIZE, &allocated, flags); + + if (unlikely(err)) + goto out_free; + + block = list_first_entry_or_null(&allocated, +struct drm_buddy_block, +link); + + if (!block) err = -EINVAL; + goto out_fini; + + if (drm_buddy_block_order(block) != mm.max_order) { + pr_err("block order(%d) != %d\n", + drm_buddy_block_order(block), mm.max_order); + err = -EINVAL; + goto out_free; + } + + if (drm_buddy_block_size(&mm, block) != + BIT_ULL(mm.max_order) * PAGE_SIZE) { + pr_err("block size(%llu) != %llu\n", + drm_buddy_block_size(&mm, block), + BIT_ULL(mm.max_order) * PAGE_SIZE); + err = -EINVAL; + goto out_free; + } + + if (!err) Always true AFAICT? + pr_info("%s - succeeded\n", __func__); I guess this could be made part of the run_selftests()? It looks like it already prints the current test, perhaps that is already enough? With the err = -EINVAL change, feel free to add, Reviewed-by: Matthew Auld + +out_free: + drm_buddy_free_list(&mm, &allocated); +out_fini: + drm_buddy_fini(&mm); + return err; +} + static int igt_sanitycheck(void *ignored) { pr_info("%s - ok!\n", __func__);
Re: [PATCH v11 5/5] drm/amdgpu: add drm buddy support to amdgpu
On Thu, 27 Jan 2022 at 14:11, Arunpravin wrote: > > - Remove drm_mm references and replace with drm buddy functionalities > - Add res cursor support for drm buddy > > v2(Matthew Auld): > - replace spinlock with mutex as we call kmem_cache_zalloc > (..., GFP_KERNEL) in drm_buddy_alloc() function > > - lock drm_buddy_block_trim() function as it calls > mark_free/mark_split are all globally visible > > v3(Matthew Auld): > - remove trim method error handling as we address the failure case > at drm_buddy_block_trim() function > > v4: > - fix warnings reported by kernel test robot > > v5: > - fix merge conflict issue > > v6: > - fix warnings reported by kernel test robot > > Signed-off-by: Arunpravin > --- > drivers/gpu/drm/Kconfig | 1 + > .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h| 97 +-- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 7 +- > drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 259 ++ > 4 files changed, 231 insertions(+), 133 deletions(-) > > -/** > - * amdgpu_vram_mgr_virt_start - update virtual start address > - * > - * @mem: ttm_resource to update > - * @node: just allocated node > - * > - * Calculate a virtual BO start address to easily check if everything is CPU > - * accessible. > - */ > -static void amdgpu_vram_mgr_virt_start(struct ttm_resource *mem, > - struct drm_mm_node *node) > -{ > - unsigned long start; > - > - start = node->start + node->size; > - if (start > mem->num_pages) > - start -= mem->num_pages; > - else > - start = 0; > - mem->start = max(mem->start, start); > -} > - > /** > * amdgpu_vram_mgr_new - allocate new ranges > * > @@ -366,13 +357,13 @@ static int amdgpu_vram_mgr_new(struct > ttm_resource_manager *man, >const struct ttm_place *place, >struct ttm_resource **res) > { > - unsigned long lpfn, num_nodes, pages_per_node, pages_left, pages; > + unsigned long lpfn, pages_per_node, pages_left, pages, n_pages; > + u64 vis_usage = 0, mem_bytes, max_bytes, min_page_size; > struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); > struct amdgpu_device *adev = to_amdgpu_device(mgr); > - uint64_t vis_usage = 0, mem_bytes, max_bytes; > - struct ttm_range_mgr_node *node; > - struct drm_mm *mm = &mgr->mm; > - enum drm_mm_insert_mode mode; > + struct amdgpu_vram_mgr_node *node; > + struct drm_buddy *mm = &mgr->mm; > + struct drm_buddy_block *block; > unsigned i; > int r; > > @@ -391,10 +382,9 @@ static int amdgpu_vram_mgr_new(struct > ttm_resource_manager *man, > goto error_sub; > } > > - if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { > + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) > pages_per_node = ~0ul; > - num_nodes = 1; > - } else { > + else { > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > pages_per_node = HPAGE_PMD_NR; > #else > @@ -403,11 +393,9 @@ static int amdgpu_vram_mgr_new(struct > ttm_resource_manager *man, > #endif > pages_per_node = max_t(uint32_t, pages_per_node, >tbo->page_alignment); > - num_nodes = DIV_ROUND_UP_ULL(PFN_UP(mem_bytes), > pages_per_node); > } > > - node = kvmalloc(struct_size(node, mm_nodes, num_nodes), > - GFP_KERNEL | __GFP_ZERO); > + node = kzalloc(sizeof(*node), GFP_KERNEL); > if (!node) { > r = -ENOMEM; > goto error_sub; > @@ -415,9 +403,17 @@ static int amdgpu_vram_mgr_new(struct > ttm_resource_manager *man, > > ttm_resource_init(tbo, place, &node->base); > > - mode = DRM_MM_INSERT_BEST; > + INIT_LIST_HEAD(&node->blocks); > + > if (place->flags & TTM_PL_FLAG_TOPDOWN) > - mode = DRM_MM_INSERT_HIGH; > + node->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; > + > + if (place->fpfn || lpfn != man->size) > + /* Allocate blocks in desired range */ > + node->flags |= DRM_BUDDY_RANGE_ALLOCATION; > + > + min_page_size = mgr->default_page_size; > + BUG_ON(min_page_size < mm->chunk_size); > > pages_left = node->base.num_pages; > > @@ -425,36 +421,61 @@ static int amdgpu_vram_mgr_new(struct > ttm_resource
Re: [PATCH v9 3/6] drm: implement top-down allocation method
On 19/01/2022 11:37, Arunpravin wrote: Implemented a function which walk through the order list, compares the offset and returns the maximum offset block, this method is unpredictable in obtaining the high range address blocks which depends on allocation and deallocation. for instance, if driver requests address at a low specific range, allocator traverses from the root block and splits the larger blocks until it reaches the specific block and in the process of splitting, lower orders in the freelist are occupied with low range address blocks and for the subsequent TOPDOWN memory request we may return the low range blocks.To overcome this issue, we may go with the below approach. The other approach, sorting each order list entries in ascending order and compares the last entry of each order list in the freelist and return the max block. This creates sorting overhead on every drm_buddy_free() request and split up of larger blocks for a single page request. v2: - Fix alignment issues(Matthew Auld) - Remove unnecessary list_empty check(Matthew Auld) - merged the below patch to see the feature in action - add top-down alloc support to i915 driver Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 36 --- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 3 ++ include/drm/drm_buddy.h | 1 + 3 files changed, 35 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 954e31962c74..6aa5c1ce25bf 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -371,6 +371,26 @@ alloc_range_bias(struct drm_buddy *mm, return ERR_PTR(err); } +static struct drm_buddy_block * +get_maxblock(struct list_head *head) +{ + struct drm_buddy_block *max_block = NULL, *node; + + max_block = list_first_entry_or_null(head, +struct drm_buddy_block, +link); + if (!max_block) + return NULL; + + list_for_each_entry(node, head, link) { + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) + max_block = node; + } If we feed in the knowledge of the visible_size(or perhaps implement that generically as "zones"), I think this can be done more efficiently. It could also be useful to track directly in the allocator how much of the visible_size is still available, rather than having to do that in the upper levels by scanning the entire list. But hopefully in practice this should be good enough for our needs, Reviewed-by: Matthew Auld + + return max_block; +} + static struct drm_buddy_block * alloc_from_freelist(struct drm_buddy *mm, unsigned int order, @@ -381,11 +401,17 @@ alloc_from_freelist(struct drm_buddy *mm, int err; for (i = order; i <= mm->max_order; ++i) { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); - if (block) - break; + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(&mm->free_list[i]); + if (block) + break; + } else { + block = list_first_entry_or_null(&mm->free_list[i], +struct drm_buddy_block, +link); + if (block) + break; + } } if (!block) diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 1411f4cf1f21..3662434b64bb 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -53,6 +53,9 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, INIT_LIST_HEAD(&bman_res->blocks); bman_res->mm = mm; + if (place->flags & TTM_PL_FLAG_TOPDOWN) + bman_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + if (place->fpfn || lpfn != man->size) bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 865664b90a8a..424fc443115e 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -28,6 +28,7 @@ }) #define DRM_BUDDY_RANGE_ALLOCATION (1 << 0) +#define DRM_BUDDY_TOPDOWN_ALLOCATION (1 << 1) struct drm_buddy_block { #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
Re: [PATCH v9 2/6] drm: improve drm_buddy_alloc function
On 19/01/2022 11:37, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them Hmm, not sure we really want to squash existing bug fixes into this patch. Perhaps bring in [1] to the start of your series? i915_buddy is gone now. Alternatively I can resend such that it applies on top drm_buddy. Your choice. [1] https://patchwork.freedesktop.org/patch/469806/?series=98953&rev=1 - fix warnings reported by kernel test robot Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 326 +- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 293 insertions(+), 124 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index d60878bc9c20..954e31962c74 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -282,23 +282,99 @@ void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc_blocks - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc_blocks(struct drm_buddy *mm, unsigned int order) +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= s2 && e1 >= e2; +} + +static struct drm_buddy_block * +alloc_range_bias(struct drm_buddy *mm, +u64 start, u64 end, +unsigned int order) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (drm_buddy_block_order(block) < order) + continue; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (drm_buddy_block_is_allocated(block)) + continue; + + if (contains(start, end, block_start, block_end) && + order == drm_buddy_block_order(block)) { + /* +* Find the free block within the range. +*/ + if (drm_buddy_block_is_free(block)) + return block; + + continue; + } + +
Re: [PATCH v9 4/6] drm: implement a method to free unused pages
On 19/01/2022 11:37, Arunpravin wrote: On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - in case of trim, at __alloc_range() split_block failure path marks the block as free and removes it from the original list, potentially also freeing it, to overcome this problem, we turn the drm_buddy_block_trim() input node into a temporary node to prevent recursively freeing itself, but still retain the un-splitting/freeing of the other nodes(Matthew Auld) - modify the drm_buddy_block_trim() function return type v5(Matthew Auld): - revert drm_buddy_block_trim() function return type changes in v4 - modify drm_buddy_block_trim() passing argument n_pages to original_size as n_pages has already been rounded up to the next power-of-two and passing n_pages results noop v6: - fix warnings reported by kernel test robot Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 65 +++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 10 +++ include/drm/drm_buddy.h | 4 ++ 3 files changed, 79 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 6aa5c1ce25bf..c5902a81b8c5 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -546,6 +546,71 @@ static int __drm_buddy_alloc_range(struct drm_buddy *mm, return __alloc_range(mm, &dfs, start, size, blocks); } +/** + * drm_buddy_block_trim - free unused pages + * + * @mm: DRM buddy manager + * @new_size: original size requested + * @blocks: output list head to add allocated blocks @blocks: Input and output list of allocated blocks. MUST contain single block as input to be trimmed. On success will contain the newly allocated blocks making up the @new_size. Blocks always appear in ascending order. ? + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and it can be freed. so remaining portions are unused and can be optionally freed with this function. ? + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_block_trim(struct drm_buddy *mm, +u64 new_size, +struct list_head *blocks) +{ + struct drm_buddy_block *parent; + struct drm_buddy_block *block; + LIST_HEAD(dfs); + u64 new_start; + int err; + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_entry(blocks, +struct drm_buddy_block, +link); + + if (!drm_buddy_block_is_allocated(block)) Maybe: if (WARN_ON(!drm_buddy_block_is_allocated())) AFAIK it should be normally impossible to be handed such non-allocated block, and so should be treated as a serious programmer error. ? + return -EINVAL; + + if (new_size > drm_buddy_block_size(mm, block)) + return -EINVAL; + + if (!new_size && !IS_ALIGNED(new_size, mm->chunk_size)) + return -EINVAL; I assume that's a typo: if (!new_size || ...) Otherwise I think looks good. Some unit tests for this would be nice, but not a blocker. And this does at least pass the igt_mock_contiguous selftest, and I didn't see anything nasty when running on DG1, which does make use of TTM_PL_FLAG_CONTIGUOUS, Reviewed-by: Matthew Auld + + if (new_size == drm_buddy_block_size(mm, block)) + return 0; + + list_del(&block->link); + mark_free(mm, block); + mm->avail += drm_buddy_block_size(mm, block); + + /* Prevent recursively freeing this node */ + parent = block->parent; + block->parent = NULL; + + new_start = drm_buddy_block_offset(block); + list_add(&block->tmp_link, &dfs); + err = __alloc_range(mm, &dfs, new_start, new_size, blocks); + if (err) { + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + list_add(&block->link, blocks); +
Re: [PATCH v6 1/6] drm: move the buddy allocator from i915 into common drm
On 26/12/2021 22:24, Arunpravin wrote: Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order(Thomas) - merged changes listed in the body section into a single patch to keep the build intact(Christian, Jani) v3: - make drm buddy a separate module(Thomas, Christian) v4: - Fix build error reported by kernel test robot - removed i915 buddy selftest from i915_mock_selftests.h to avoid build error - removed selftests/i915_buddy.c file as we create a new set of buddy test cases in drm/selftests folder v5: - Fix merge conflict issue Signed-off-by: Arunpravin +int drm_buddy_init(struct drm_buddy_mm *mm, u64 size, u64 chunk_size) +{ + unsigned int i; + u64 offset; + + if (size < chunk_size) + return -EINVAL; + + if (chunk_size < PAGE_SIZE) + return -EINVAL; + + if (!is_power_of_2(chunk_size)) + return -EINVAL; + + size = round_down(size, chunk_size); + + mm->size = size; + mm->avail = size; + mm->chunk_size = chunk_size; + mm->max_order = ilog2(size) - ilog2(chunk_size); + + BUG_ON(mm->max_order > DRM_BUDDY_MAX_ORDER); + + mm->slab_blocks = KMEM_CACHE(drm_buddy_block, 0); + if (!mm->slab_blocks) + return -ENOMEM; It looks like every KMEM_CACHE() also creates a debugfs entry? See the error here[1]. I guess because we end with multiple instances in i915. If so, is it possible to have a single KMEM_CACHE() as part of the buddy module, similar to what i915 was doing previously? [1] https://intel-gfx-ci.01.org/tree/drm-tip/Trybot_8217/shard-skl4/igt@i915_selftest@mock@memory_region.html
Re: [PATCH v6 4/6] drm: implement a method to free unused pages
On 26/12/2021 22:24, Arunpravin wrote: On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - in case of trim, at __alloc_range() split_block failure path marks the block as free and removes it from the original list, potentially also freeing it, to overcome this problem, we turn the drm_buddy_block_trim() input node into a temporary node to prevent recursively freeing itself, but still retain the un-splitting/freeing of the other nodes(Matthew Auld) - modify the drm_buddy_block_trim() function return type Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 61 +++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 8 +++ include/drm/drm_buddy.h | 4 ++ 3 files changed, 73 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index eddc1eeda02e..855afcaf7edd 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -538,6 +538,67 @@ static int __drm_buddy_alloc_range(struct drm_buddy_mm *mm, return __alloc_range(mm, &dfs, start, size, blocks); } +/** + * drm_buddy_block_trim - free unused pages + * + * @mm: DRM buddy manager + * @new_size: original size requested + * @blocks: output list head to add allocated blocks + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and it can be freed. + */ +void drm_buddy_block_trim(struct drm_buddy_mm *mm, + u64 new_size, + struct list_head *blocks) It might be better to just return the error, and let the user decide if they want to ignore it? Also we might want some kind of unit test for this, so having an actual return value might be useful there. +{ + struct drm_buddy_block *parent; + struct drm_buddy_block *block; + LIST_HEAD(dfs); + u64 new_start; + int err; + + if (!list_is_singular(blocks)) + return; + + block = list_first_entry(blocks, +struct drm_buddy_block, +link); + + if (!drm_buddy_block_is_allocated(block)) + return; + + if (new_size > drm_buddy_block_size(mm, block)) + return; + + if (!new_size && !IS_ALIGNED(new_size, mm->chunk_size)) + return; + + if (new_size == drm_buddy_block_size(mm, block)) + return; + + list_del(&block->link); + mark_free(mm, block); + mm->avail += drm_buddy_block_size(mm, block); + + /* Prevent recursively freeing this node */ + parent = block->parent; + block->parent = NULL; + + new_start = drm_buddy_block_offset(block); + list_add(&block->tmp_link, &dfs); + err = __alloc_range(mm, &dfs, new_start, new_size, blocks); + if (err) { + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + list_add(&block->link, blocks); + } + + block->parent = parent; +} +EXPORT_SYMBOL(drm_buddy_block_trim); + /** * drm_buddy_alloc - allocate power-of-two blocks * diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 7c58efb60dba..05f924f32e96 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -97,6 +97,14 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, if (unlikely(err)) goto err_free_blocks; + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + mutex_lock(&bman->lock); + drm_buddy_block_trim(mm, + (u64)n_pages << PAGE_SHIFT, AFAIK, n_pages has already been rounded up to the next power-of-two here, so this becomes a noop. I assume we need to use the "original" size here? + &bman_res->blocks); + mutex_unlock(&bman->lock); + } + *res
Re: [PATCH v6 2/6] drm: improve drm_buddy_alloc function
On 26/12/2021 22:24, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations Signed-off-by: Arunpravin @@ -73,34 +83,16 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, n_pages = size >> ilog2(mm->chunk_size); - do { - struct drm_buddy_block *block; - unsigned int order; - - order = fls(n_pages) - 1; - GEM_BUG_ON(order > mm->max_order); - GEM_BUG_ON(order < min_order); - - do { - mutex_lock(&bman->lock); - block = drm_buddy_alloc(mm, order); - mutex_unlock(&bman->lock); - if (!IS_ERR(block)) - break; - - if (order-- == min_order) { - err = -ENOSPC; - goto err_free_blocks; - } - } while (1); - - n_pages -= BIT(order); - - list_add_tail(&block->link, &bman_res->blocks); - - if (!n_pages) - break; - } while (1); + mutex_lock(&bman->lock); + err = drm_buddy_alloc(mm, (u64)place->fpfn << PAGE_SHIFT, + (u64)place->lpfn << PAGE_SHIFT, place->lpfn will currently always be zero for i915, AFAIK. I assume here we want s/place->lpfn/lpfn/? Also something in this series is preventing i915 from loading on discrete devices, according to CI. Hopefully that is just the lpfn issue...which might explain seeing -EINVAL here[1] when allocating some vram for the firmware. [1] https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21904/bat-dg1-6/boot0.txt + (u64)n_pages << PAGE_SHIFT, +min_page_size, +&bman_res->blocks, +bman_res->flags); + mutex_unlock(&bman->lock); + if (unlikely(err)) + goto err_free_blocks; *res = &bman_res->base; return 0; @@ -266,10 +258,17 @@ int i915_ttm_buddy_man_reserve(struct ttm_resource_manager *man, { struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); struct drm_buddy_mm *mm = &bman->mm; + unsigned long flags = 0; int ret; + flags |= DRM_BUDDY_RANGE_ALLOCATION; + mutex_lock(&bman->lock); - ret = drm_buddy_alloc_range(mm, &bman->reserved, start, size); + ret = drm_buddy_alloc(mm, start, + start + size, + size, mm->chunk_size, + &bman->reserved, + flags); mutex_unlock(&bman->lock); return ret; diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h index fa644b512c2e..5ba490875f66 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h @@ -20,6 +20,7 @@ struct drm_buddy_mm; * * @base: struct ttm_resource base class we extend * @blocks: the list of struct i915_buddy_block for this resource/allocation + * @flags: DRM_BUDDY_*_ALLOCATION flags * @mm: the struct i915_buddy_mm for this resource * * Extends the struct ttm_resource to manage an address space allocation with @@ -28,6 +29,7 @@ struct drm_buddy_mm; struct i915_ttm_buddy_resource { struct ttm_resource base; struct list_head blocks; + unsigned long flags; struct drm_buddy_mm *mm; }; diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 09d73328c268..4368acaad222 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -13,15 +13,22 @@ #include -#define range_overflows(start, size, max) ({ \ +#define check_range_overflow(start, end, size, ma
Re: [PATCH v4 2/6] drm: improve drm_buddy_alloc function
On 15/12/2021 20:46, Arunpravin wrote: On 14/12/21 12:29 am, Matthew Auld wrote: On 09/12/2021 15:47, Paneer Selvam, Arunpravin wrote: [AMD Official Use Only] Hi Matthew, Ping on this? No new comments from me :) I guess just a question of what we should do with the selftests, and then ofc at some point being able to throw this at CI, or at least test locally, once the series builds. sure :) I think we should rewrite the i915 buddy selftests since now we have a single function for range and non-range requirements. I will rewrite the i915 buddy selftests and move to drm selftests folder? so for the time being, I remove the i915_buddy_mock_selftest() from i915_mock_selftests.h list to avoid build errors? Yeah, whatever is easiest. Regards, Arun -Original Message- From: amd-gfx On Behalf Of Arunpravin Sent: Wednesday, December 1, 2021 10:10 PM To: dri-de...@lists.freedesktop.org; intel-...@lists.freedesktop.org; amd-gfx@lists.freedesktop.org Cc: dan...@ffwll.ch; Paneer Selvam, Arunpravin ; jani.nik...@linux.intel.com; matthew.a...@intel.com; tzimmerm...@suse.de; Deucher, Alexander ; Koenig, Christian Subject: [PATCH v4 2/6] drm: improve drm_buddy_alloc function - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 316 +- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 285 insertions(+), 122 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 9340a4b61c5a..7f47632821f4 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -280,23 +280,97 @@ void drm_buddy_free_list(struct drm_buddy_mm *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc(struct drm_buddy_mm *mm, unsigned int order) +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) { + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) { + return s1 <= s2 && e1 >= e2; +} + +static struct drm_buddy_block * +alloc_range_bias(struct drm_buddy_mm *mm, +u64 start, u64 end, +unsigned int order) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (drm_buddy_block_order(block) < order) + continue; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (drm_buddy_block_is_allocated(block)) + continue; + + if (contains(start, end, block_start, block_end) && + order == drm_bud
Re: [PATCH v4 2/6] drm: improve drm_buddy_alloc function
On 09/12/2021 15:47, Paneer Selvam, Arunpravin wrote: [AMD Official Use Only] Hi Matthew, Ping on this? No new comments from me :) I guess just a question of what we should do with the selftests, and then ofc at some point being able to throw this at CI, or at least test locally, once the series builds. Regards, Arun -Original Message- From: amd-gfx On Behalf Of Arunpravin Sent: Wednesday, December 1, 2021 10:10 PM To: dri-de...@lists.freedesktop.org; intel-...@lists.freedesktop.org; amd-gfx@lists.freedesktop.org Cc: dan...@ffwll.ch; Paneer Selvam, Arunpravin ; jani.nik...@linux.intel.com; matthew.a...@intel.com; tzimmerm...@suse.de; Deucher, Alexander ; Koenig, Christian Subject: [PATCH v4 2/6] drm: improve drm_buddy_alloc function - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 316 +- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++-- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 285 insertions(+), 122 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 9340a4b61c5a..7f47632821f4 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -280,23 +280,97 @@ void drm_buddy_free_list(struct drm_buddy_mm *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc(struct drm_buddy_mm *mm, unsigned int order) +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) { + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) { + return s1 <= s2 && e1 >= e2; +} + +static struct drm_buddy_block * +alloc_range_bias(struct drm_buddy_mm *mm, +u64 start, u64 end, +unsigned int order) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (drm_buddy_block_order(block) < order) + continue; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (drm_buddy_block_is_allocated(block)) + continue; + + if (contains(start, end, block_start, block_end) && + order == drm_buddy_block_order(block)) { + /* +* Find the free block within the range. +*/ + if (drm_buddy_block_is_free(block)) + return block; + + continue; + } + + if (!drm_buddy_block_is_split(block)) { + err = split_block(mm, block); +
Re: [PATCH v4 4/6] drm: implement a method to free unused pages
On 01/12/2021 16:39, Arunpravin wrote: On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3: - remove drm_buddy_block_trim() error handling and print a warn message if it fails Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 72 ++- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 10 +++ include/drm/drm_buddy.h | 4 ++ 3 files changed, 83 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index eddc1eeda02e..707efc82216d 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -434,7 +434,8 @@ alloc_from_freelist(struct drm_buddy_mm *mm, static int __alloc_range(struct drm_buddy_mm *mm, struct list_head *dfs, u64 start, u64 size, -struct list_head *blocks) +struct list_head *blocks, +bool trim_path) { struct drm_buddy_block *block; struct drm_buddy_block *buddy; @@ -480,8 +481,20 @@ static int __alloc_range(struct drm_buddy_mm *mm, if (!drm_buddy_block_is_split(block)) { err = split_block(mm, block); - if (unlikely(err)) + if (unlikely(err)) { + if (trim_path) + /* +* Here in case of trim, we return and dont goto +* split failure path as it removes from the +* original list and potentially also freeing +* the block. so we could leave as it is, +* worse case we get some internal fragmentation +* and leave the decision to the user +*/ + return err; Hmm, ideally we don't want to leave around blocks where both buddies are free without then also merging them back(not sure if that trips some BUG_ON). Also IIUC, if we hit this failure path, depending on where the split_block() fails we might be allocating something less than new_size? Also if it's the first split_block() that fails then the user just gets an empty list? Could we perhaps just turn this node into a temporary root node to prevent recursively freeing itself, but still retain the un-splitting/freeing of the other nodes i.e something like: list_del(&block->link); mark_free(mm, block); mm->avail += ...; /* Prevent recursively freeing this node */ parent = block->parent; block->parent = NULL; list_add(&block->tmp_link, &dfs); ret = _alloc_range(mm, &dfs, new_start, new_size, blocks); if (ret) { mem->avail -= ...; mark_allocated(block); list_add(&block->link, blocks); } block->parent = parent; return ret; That way we can also drop the special trim_path handling. Thoughts? + goto err_undo; + } } list_add(&block->right->tmp_link, dfs); @@ -535,8 +548,61 @@ static int __drm_buddy_alloc_range(struct drm_buddy_mm *mm, for (i = 0; i < mm->n_roots; ++i) list_add_tail(&mm->roots[i]->tmp_link, &dfs); - return __alloc_range(mm, &dfs, start, size, blocks); + return __alloc_range(mm, &dfs, start, size, blocks, 0); +} + +/** + * drm_buddy_block_trim - free unused pages + * + * @mm: DRM buddy manager + * @new_size: original size requested + * @blocks: output list head to add allocated blocks + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and it can be freed. + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_block_trim(struct drm_buddy_mm *mm, +u64 new_size, +struct list_head *blocks) +{ + struct drm_buddy_block *block; + u64 new_start; + LIST_HEAD(dfs); + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_e
Re: [PATCH v3 2/6] drm: improve drm_buddy_alloc function
On 23/11/2021 22:39, Arunpravin wrote: On 18/11/21 12:09 am, Matthew Auld wrote: On 16/11/2021 20:18, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 259 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 69 ++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 203 insertions(+), 149 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 39eb1d224bec..c9b18a29f8d1 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -274,63 +274,6 @@ void drm_buddy_free_list(struct drm_buddy_mm *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc(struct drm_buddy_mm *mm, unsigned int order) -{ - struct drm_buddy_block *block = NULL; - unsigned int i; - int err; - - for (i = order; i <= mm->max_order; ++i) { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); - if (block) - break; - } - - if (!block) - return ERR_PTR(-ENOSPC); - - BUG_ON(!drm_buddy_block_is_free(block)); - - while (i != order) { - err = split_block(mm, block); - if (unlikely(err)) - goto out_free; - - /* Go low */ - block = block->left; - i--; - } - - mark_allocated(block); - mm->avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); - return block; - -out_free: - if (i != order) - __drm_buddy_free(mm, block); - return ERR_PTR(err); -} -EXPORT_SYMBOL(drm_buddy_alloc); - static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) { return s1 <= e2 && e1 >= s2; @@ -341,52 +284,22 @@ static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) return s1 <= s2 && e1 >= e2; } -/** - * drm_buddy_alloc_range - allocate range - * - * @mm: DRM buddy manager to allocate from - * @blocks: output list head to add allocated blocks - * @start: start of the allowed range for this block - * @size: size of the allocation - * - * Intended for pre-allocating portions of the address space, for example to - * reserve a block for the initial framebuffer or similar, hence the expectation - * here is that drm_buddy_alloc() is still the main vehicle for - * allocations, so if that's not the case then the drm_mm range allocator is - * probably a much better fit, and so you should probably go use that instead. - * - * Note that it's safe to chain together multiple alloc_ranges - * with the same blocks list - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_alloc_range(struct drm_buddy_mm *mm, - struct list_head *blocks, - u64 start, u64 size) +static struct drm_buddy_block * +alloc_range(struct drm_buddy_mm *mm, + u64 start, u64 end, + unsigned int order) { struct drm_buddy_block *block; struct drm_buddy_block *buddy; - LIST_HEAD(allocated); LIST_HEAD(dfs); - u64 end; int err; int i; - if (size < mm->chunk_size) - return -EINVAL; - - if (!IS_ALIGNED(size | start, mm->chunk_size)) - return -EINVAL; - - if (range_overflows(start, size, mm->size)) -
Re: [PATCH v3 4/6] drm: implement a method to free unused pages
On 16/11/2021 20:18, Arunpravin wrote: On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 108 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 10 ++ include/drm/drm_buddy.h | 4 + 3 files changed, 122 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 0a9db2981188..943fe2ad27bf 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -284,6 +284,114 @@ static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) return s1 <= s2 && e1 >= e2; } +/** + * drm_buddy_block_trim - free unused pages + * + * @mm: DRM buddy manager + * @new_size: original size requested + * @blocks: output list head to add allocated blocks + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and it can be freed. + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_block_trim(struct drm_buddy_mm *mm, +u64 new_size, +struct list_head *blocks) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + u64 new_start; + u64 new_end; + LIST_HEAD(dfs); + u64 count = 0; + int err; + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_entry(blocks, +struct drm_buddy_block, +link); + + if (!drm_buddy_block_is_allocated(block)) + return -EINVAL; + + if (new_size > drm_buddy_block_size(mm, block)) + return -EINVAL; + + if (!new_size && !IS_ALIGNED(new_size, mm->chunk_size)) + return -EINVAL; + + if (new_size == drm_buddy_block_size(mm, block)) + return 0; + + list_del(&block->link); + + new_start = drm_buddy_block_offset(block); + new_end = new_start + new_size - 1; + + mark_free(mm, block); + + list_add(&block->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (count == new_size) + return 0; + + block_start = drm_buddy_block_offset(block); + block_end = block_start + drm_buddy_block_size(mm, block) - 1; + + if (!overlaps(new_start, new_end, block_start, block_end)) + continue; + + if (contains(new_start, new_end, block_start, block_end)) { + BUG_ON(!drm_buddy_block_is_free(block)); + + /* Allocate only required blocks */ + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + list_add_tail(&block->link, blocks); + count += drm_buddy_block_size(mm, block); + continue; + } + + if (!drm_buddy_block_is_split(block)) { Should always be true, right? But I guess depends if we want to re-use this for generic range allocation... + err = split_block(mm, block); + if (unlikely(err)) + goto err_undo; + } + + list_add(&block->right->tmp_link, &dfs); + list_add(&block->left->tmp_link, &dfs); + } while (1); + + return -ENOSPC; + +err_undo: + buddy = get_buddy(block); + if (buddy && + (drm_buddy_block_is_free(block) && +drm_buddy_block_is_free(buddy))) + __drm_buddy_free(mm, block); + return err; Looking at the split_block failure path. The user allocated some block, and then tried to trim it, but this then marks it as free and removes it
Re: [PATCH v3 2/6] drm: improve drm_buddy_alloc function
On 16/11/2021 20:18, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 259 ++ drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 69 ++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 203 insertions(+), 149 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 39eb1d224bec..c9b18a29f8d1 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -274,63 +274,6 @@ void drm_buddy_free_list(struct drm_buddy_mm *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc(struct drm_buddy_mm *mm, unsigned int order) -{ - struct drm_buddy_block *block = NULL; - unsigned int i; - int err; - - for (i = order; i <= mm->max_order; ++i) { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); - if (block) - break; - } - - if (!block) - return ERR_PTR(-ENOSPC); - - BUG_ON(!drm_buddy_block_is_free(block)); - - while (i != order) { - err = split_block(mm, block); - if (unlikely(err)) - goto out_free; - - /* Go low */ - block = block->left; - i--; - } - - mark_allocated(block); - mm->avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); - return block; - -out_free: - if (i != order) - __drm_buddy_free(mm, block); - return ERR_PTR(err); -} -EXPORT_SYMBOL(drm_buddy_alloc); - static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) { return s1 <= e2 && e1 >= s2; @@ -341,52 +284,22 @@ static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) return s1 <= s2 && e1 >= e2; } -/** - * drm_buddy_alloc_range - allocate range - * - * @mm: DRM buddy manager to allocate from - * @blocks: output list head to add allocated blocks - * @start: start of the allowed range for this block - * @size: size of the allocation - * - * Intended for pre-allocating portions of the address space, for example to - * reserve a block for the initial framebuffer or similar, hence the expectation - * here is that drm_buddy_alloc() is still the main vehicle for - * allocations, so if that's not the case then the drm_mm range allocator is - * probably a much better fit, and so you should probably go use that instead. - * - * Note that it's safe to chain together multiple alloc_ranges - * with the same blocks list - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_alloc_range(struct drm_buddy_mm *mm, - struct list_head *blocks, - u64 start, u64 size) +static struct drm_buddy_block * +alloc_range(struct drm_buddy_mm *mm, + u64 start, u64 end, + unsigned int order) { struct drm_buddy_block *block; struct drm_buddy_block *buddy; - LIST_HEAD(allocated); LIST_HEAD(dfs); - u64 end; int err; int i; - if (size < mm->chunk_size) - return -EINVAL; - - if (!IS_ALIGNED(size | start, mm->chunk_size)) - return -EINVAL; - - if (range_overflows(start, size, mm->size)) - return -EINVAL; + end = end - 1; for (i = 0; i < mm->n_roots; ++i) list_add_tail(&mm-
Re: [PATCH v3 1/6] drm: move the buddy allocator from i915 into common drm
On 16/11/2021 20:18, Arunpravin wrote: Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order (Thomas) - merged changes listed in the body section into a single patch to keep the build intact (Christian, Jani) Signed-off-by: Arunpravin Any ideas for what to do with the existing selftests? Currently this series doesn't build yet for i915 due to this, and prevents throwing the series at CI.
Re: [PATCH 8/8] drm/amdgpu: add drm buddy support to amdgpu
On 04/11/2021 07:34, Christian König wrote: Am 03.11.21 um 20:25 schrieb Matthew Auld: On 25/10/2021 14:00, Arunpravin wrote: - Remove drm_mm references and replace with drm buddy functionalities - Add res cursor support for drm buddy Signed-off-by: Arunpravin + spin_lock(&mgr->lock); + r = drm_buddy_alloc(mm, (uint64_t)place->fpfn << PAGE_SHIFT, + (uint64_t)lpfn << PAGE_SHIFT, + (uint64_t)n_pages << PAGE_SHIFT, + min_page_size, &node->blocks, + node->flags); Is spinlock + GFP_KERNEL allowed? Nope it isn't, but does that function really calls kmalloc()? It calls kmem_cache_zalloc(..., GFP_KERNEL) Christian. + spin_unlock(&mgr->lock); + + if (unlikely(r)) + goto error_free_blocks; + pages_left -= pages; ++i; if (pages > pages_left) pages = pages_left; } - spin_unlock(&mgr->lock); + + /* Free unused pages for contiguous allocation */ + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + uint64_t actual_size = (uint64_t)node->base.num_pages << PAGE_SHIFT; + + r = drm_buddy_free_unused_pages(mm, + actual_size, + &node->blocks); Needs some locking.
Re: [PATCH 8/8] drm/amdgpu: add drm buddy support to amdgpu
On 25/10/2021 14:00, Arunpravin wrote: - Remove drm_mm references and replace with drm buddy functionalities - Add res cursor support for drm buddy Signed-off-by: Arunpravin + spin_lock(&mgr->lock); + r = drm_buddy_alloc(mm, (uint64_t)place->fpfn << PAGE_SHIFT, + (uint64_t)lpfn << PAGE_SHIFT, + (uint64_t)n_pages << PAGE_SHIFT, +min_page_size, &node->blocks, +node->flags); Is spinlock + GFP_KERNEL allowed? + spin_unlock(&mgr->lock); + + if (unlikely(r)) + goto error_free_blocks; + pages_left -= pages; ++i; if (pages > pages_left) pages = pages_left; } - spin_unlock(&mgr->lock); + + /* Free unused pages for contiguous allocation */ + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + uint64_t actual_size = (uint64_t)node->base.num_pages << PAGE_SHIFT; + + r = drm_buddy_free_unused_pages(mm, + actual_size, + &node->blocks); Needs some locking.
Re: [PATCH 6/8] drm/i915: add free_unused_pages support to i915
On 25/10/2021 14:00, Arunpravin wrote: add drm_buddy_free_unused_pages() support on contiguous allocation Signed-off-by: Arunpravin --- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 8 1 file changed, 8 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 963468228392..162947af8e04 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -98,6 +98,14 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, if (unlikely(err)) goto err_free_blocks; + if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { + err = drm_buddy_free_unused_pages(mm, (uint64_t)n_pages << PAGE_SHIFT, + &bman_res->blocks); + + if (unlikely(err)) + goto err_free_blocks; That needs some locking, mark_free/mark_split are all globally visible. Some concurrent user might steal the block, or worse. + } + *res = &bman_res->base; return 0;
Re: [PATCH 5/8] drm: Implement method to free unused pages
On 25/10/2021 14:00, Arunpravin wrote: On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. Signed-off-by: Arunpravin Ideally this gets added with some user, so we can see it in action? Maybe squash the next patch here? --- drivers/gpu/drm/drm_buddy.c | 103 include/drm/drm_buddy.h | 4 ++ 2 files changed, 107 insertions(+) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 9d3547bcc5da..0da8510736eb 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -284,6 +284,109 @@ static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) return s1 <= s2 && e1 >= e2; } +/** + * drm_buddy_free_unused_pages - free unused pages + * + * @mm: DRM buddy manager + * @actual_size: original size requested + * @blocks: output list head to add allocated blocks + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and it can be freed. + * + * Returns: + * 0 on success, error code on failure. + */ +int drm_buddy_free_unused_pages(struct drm_buddy_mm *mm, drm_buddy_block_trim? + u64 actual_size, new_size? + struct list_head *blocks) +{ + struct drm_buddy_block *block; + struct drm_buddy_block *buddy; + u64 actual_start; + u64 actual_end; + LIST_HEAD(dfs); + u64 count = 0; + int err; + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_entry_or_null(blocks, +struct drm_buddy_block, +link); + + if (!block) + return -EINVAL; list_is_singular() already ensures that I guess? + + if (actual_size > drm_buddy_block_size(mm, block)) + return -EINVAL; + + if (actual_size == drm_buddy_block_size(mm, block)) + return 0; Probably need to check the alignment of the actual_size, and also check that it is non-zero? + + list_del(&block->link); + + actual_start = drm_buddy_block_offset(block); + actual_end = actual_start + actual_size - 1; + + if (drm_buddy_block_is_allocated(block)) That should rather be a programmer error. + mark_free(mm, block); + + list_add(&block->tmp_link, &dfs); + + while (1) { + block = list_first_entry_or_null(&dfs, +struct drm_buddy_block, +tmp_link); + + if (!block) + break; + + list_del(&block->tmp_link); + + if (count == actual_size) + return 0; Check for overlaps somewhere here to avoid needless searching and splitting? + + if (contains(actual_start, actual_end, drm_buddy_block_offset(block), + (drm_buddy_block_offset(block) + drm_buddy_block_size(mm, block) - 1))) { Could maybe record the start/end for better readability? + BUG_ON(!drm_buddy_block_is_free(block)); + + /* Allocate only required blocks */ + mark_allocated(block); + mm->avail -= drm_buddy_block_size(mm, block); + list_add_tail(&block->link, blocks); + count += drm_buddy_block_size(mm, block); + continue; + } + + if (drm_buddy_block_order(block) == 0) + continue; Should be impossible with overlaps check added. + + if (!drm_buddy_block_is_split(block)) { That should always be true. + err = split_block(mm, block); + + if (unlikely(err)) + goto err_undo; + } + + list_add(&block->right->tmp_link, &dfs); + list_add(&block->left->tmp_link, &dfs); + } + + return -ENOSPC; Would it make sense to factor out part of the alloc_range for this? It looks roughly the same. + +err_undo: + buddy = get_buddy(block); + if (buddy && + (drm_buddy_block_is_free(block) && +drm_buddy_block_is_free(buddy))) + __drm_buddy_free(mm, block); + return err; Where do we add the block back to the original list? Did we not just leak it? +} +EXPORT_SYMBOL(drm_buddy_free_unused_pages); + static struct drm_buddy_block * alloc_range(struct drm_buddy_mm *mm, u64 start, u64 end, diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index cd8021d2d6e7..1dfc80c88e1f 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@
Re: [PATCH 3/8] drm: implement top-down allocation method
On 25/10/2021 14:00, Arunpravin wrote: Implemented a function which walk through the order list, compares the offset and returns the maximum offset block, this method is unpredictable in obtaining the high range address blocks which depends on allocation and deallocation. for instance, if driver requests address at a low specific range, allocator traverses from the root block and splits the larger blocks until it reaches the specific block and in the process of splitting, lower orders in the freelist are occupied with low range address blocks and for the subsequent TOPDOWN memory request we may return the low range blocks.To overcome this issue, we may go with the below approach. The other approach, sorting each order list entries in ascending order and compares the last entry of each order list in the freelist and return the max block. This creates sorting overhead on every drm_buddy_free() request and split up of larger blocks for a single page request. Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 42 +++-- include/drm/drm_buddy.h | 1 + 2 files changed, 37 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 406e3d521903..9d3547bcc5da 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -362,6 +362,27 @@ alloc_range(struct drm_buddy_mm *mm, return ERR_PTR(err); } +static struct drm_buddy_block * +get_maxblock(struct list_head *head) +{ + struct drm_buddy_block *max_block = NULL, *node; + + max_block = list_first_entry_or_null(head, +struct drm_buddy_block, +link); + + if (!max_block) + return NULL; + + list_for_each_entry(node, head, link) { + if (drm_buddy_block_offset(node) > + drm_buddy_block_offset(max_block)) Alignment. + max_block = node; + } I suppose there will be pathological cases where this will unnecessarily steal the mappable portion? But in practice maybe this is good enough? + + return max_block; +} + static struct drm_buddy_block * alloc_from_freelist(struct drm_buddy_mm *mm, unsigned int order, @@ -372,13 +393,22 @@ alloc_from_freelist(struct drm_buddy_mm *mm, int err; for (i = order; i <= mm->max_order; ++i) { - if (!list_empty(&mm->free_list[i])) { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); + if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { + if (!list_empty(&mm->free_list[i])) { AFAIK no need to keep checking list_empty(), below also. + block = get_maxblock(&mm->free_list[i]); - if (block) - break; + if (block) + break; + } + } else { + if (!list_empty(&mm->free_list[i])) { + block = list_first_entry_or_null(&mm->free_list[i], +struct drm_buddy_block, +link); + + if (block) + break; + } } } diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index c7bb5509a7ad..cd8021d2d6e7 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -28,6 +28,7 @@ }) #define DRM_BUDDY_RANGE_ALLOCATION (1 << 0) +#define DRM_BUDDY_TOPDOWN_ALLOCATION (1 << 1) struct drm_buddy_block { #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
Re: [PATCH v2 2/8] drm: improve drm_buddy_alloc function
On 25/10/2021 14:00, Arunpravin wrote: - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy V2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver Signed-off-by: Arunpravin --- drivers/gpu/drm/drm_buddy.c | 265 +++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 67 ++--- drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 2 + include/drm/drm_buddy.h | 22 +- 4 files changed, 207 insertions(+), 149 deletions(-) diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 39eb1d224bec..406e3d521903 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -274,63 +274,6 @@ void drm_buddy_free_list(struct drm_buddy_mm *mm, struct list_head *objects) } EXPORT_SYMBOL(drm_buddy_free_list); -/** - * drm_buddy_alloc - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @order: size of the allocation - * - * The order value here translates to: - * - * 0 = 2^0 * mm->chunk_size - * 1 = 2^1 * mm->chunk_size - * 2 = 2^2 * mm->chunk_size - * - * Returns: - * allocated ptr to the &drm_buddy_block on success - */ -struct drm_buddy_block * -drm_buddy_alloc(struct drm_buddy_mm *mm, unsigned int order) -{ - struct drm_buddy_block *block = NULL; - unsigned int i; - int err; - - for (i = order; i <= mm->max_order; ++i) { - block = list_first_entry_or_null(&mm->free_list[i], -struct drm_buddy_block, -link); - if (block) - break; - } - - if (!block) - return ERR_PTR(-ENOSPC); - - BUG_ON(!drm_buddy_block_is_free(block)); - - while (i != order) { - err = split_block(mm, block); - if (unlikely(err)) - goto out_free; - - /* Go low */ - block = block->left; - i--; - } - - mark_allocated(block); - mm->avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); - return block; - -out_free: - if (i != order) - __drm_buddy_free(mm, block); - return ERR_PTR(err); -} -EXPORT_SYMBOL(drm_buddy_alloc); - static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) { return s1 <= e2 && e1 >= s2; @@ -341,52 +284,22 @@ static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) return s1 <= s2 && e1 >= e2; } -/** - * drm_buddy_alloc_range - allocate range - * - * @mm: DRM buddy manager to allocate from - * @blocks: output list head to add allocated blocks - * @start: start of the allowed range for this block - * @size: size of the allocation - * - * Intended for pre-allocating portions of the address space, for example to - * reserve a block for the initial framebuffer or similar, hence the expectation - * here is that drm_buddy_alloc() is still the main vehicle for - * allocations, so if that's not the case then the drm_mm range allocator is - * probably a much better fit, and so you should probably go use that instead. - * - * Note that it's safe to chain together multiple alloc_ranges - * with the same blocks list - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_alloc_range(struct drm_buddy_mm *mm, - struct list_head *blocks, - u64 start, u64 size) +static struct drm_buddy_block * +alloc_range(struct drm_buddy_mm *mm, + u64 start, u64 end, + unsigned int order) { struct drm_buddy_block *block; struct drm_buddy_block *buddy; - LIST_HEAD(allocated); LIST_HEAD(dfs); - u64 end; int err; int i; - if (size < mm->chunk_size) - return -EINVAL; - - if (!IS_ALIGNED(size | start, mm->chunk_size)) - return -EINVAL; - - if (range_overflows(start, size, mm->size)) - return -EINVAL; + end = end - 1; for (i = 0; i < mm->n_roots; ++i) list_add_tail(&mm->roots[i]->tmp_link, &dfs); - end = start + size - 1; - do { u64 block_start; u64 block_end; @@ -394,31 +307,32 @@ int drm_buddy_alloc_range(struct drm_buddy_mm *mm, block = list_first_entry_or_null(&dfs, struct drm_buddy_block, tmp_link); + No need