Re: [PATCH 2/9] mm/page_alloc: Add a bulk page allocator

On Mon, Apr 12, 2021 at 11:59:38AM +0100, Mel Gorman wrote: > > I don't understand this comment. Only alloc_flags_nofragment() sets this > > flag > > and we don't use it here? > > > > It's there as a reminder that there are non-obvious consequences > to ALLOC_NOFRAGMENT that may affect the bulk

[PATCH] mm/page_alloc: Add a bulk page allocator -fix -fix -fix

Vlastimil Babka noted that a comment is wrong, fix it. This is the third fix to the mmotm patch mm-page_alloc-add-a-bulk-page-allocator.patch. Signed-off-by: Mel Gorman --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c

Re: [PATCH 2/9] mm/page_alloc: Add a bulk page allocator

On Mon, Apr 12, 2021 at 12:21:42PM +0200, Vlastimil Babka wrote: > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index 8a3e13277e22..eb547470a7e4 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -4965,6 +4965,124 @@ static inline bool prepare_alloc_pages(gfp_t > >

Re: [PATCH 2/9] mm/page_alloc: Add a bulk page allocator

On 3/25/21 12:42 PM, Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > to be allocated and added to a list. > > The API is not guaranteed to return the requested number of pages and >

Re: [PATCH] mm/page_alloc: Add a bulk page allocator -fix -fix

On 30/03/2021 12:48, Mel Gorman wrote: > Colin Ian King reported the following problem (slightly edited) > > Author: Mel Gorman > Date: Mon Mar 29 11:12:24 2021 +1100 > > mm/page_alloc: add a bulk page allocator > > ... > >

[PATCH] mm/page_alloc: Add a bulk page allocator -fix -fix

Colin Ian King reported the following problem (slightly edited) Author: Mel Gorman Date: Mon Mar 29 11:12:24 2021 +1100 mm/page_alloc: add a bulk page allocator ... Static analysis on linux-next with Coverity has found a potential

Re: mm/page_alloc: add a bulk page allocator

b51b2da > Author: Mel Gorman > Date: Mon Mar 29 11:12:24 2021 +1100 > > mm/page_alloc: add a bulk page allocator > > The analysis is as follows: > > > > > 5050if (nr_pages - nr_populated == 1) > 5051goto failed; > 5

re: mm/page_alloc: add a bulk page allocator

Hi, Static analysis on linux-next with Coverity has found a potential uninitialized variable issue in function __alloc_pages_bulk with the following commit: commit b0e0a469733fa571ddd8fe147247c9561b51b2da Author: Mel Gorman Date: Mon Mar 29 11:12:24 2021 +1100 mm/page_alloc: add a bulk

Re: [PATCH 2/9] mm/page_alloc: Add a bulk page allocator

it matches alloc_contig_range even though free_contig_range takes unsigned int *sigh* > > > > +/* > > If you could make that "/**" instead ... > I decided not to until we're reasonably sure the semantics are not going to change. ---8<--- mm

Re: [PATCH 2/9] mm/page_alloc: Add a bulk page allocator

On Thu, Mar 25, 2021 at 11:42:21AM +, Mel Gorman wrote: > +int __alloc_pages_bulk(gfp_t gfp, int preferred_nid, > + nodemask_t *nodemask, int nr_pages, > + struct list_head *list); > + > +/* Bulk allocate order-0 pages */ > +static inline

[PATCH 2/9] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. The API is not guaranteed to return the requested number of pages and may fail if the preferred allocation zone has

Re: [PATCH 2/3] mm/page_alloc: Add a bulk page allocator

On Tue, Mar 23, 2021 at 05:00:08PM +0100, Jesper Dangaard Brouer wrote: > > + /* > > +* If there are no allowed local zones that meets the watermarks then > > +* try to allocate a single page and reclaim if necessary. > > +*/ > > + if (!zone) > > + goto failed; > > + > >

Re: [PATCH 2/3] mm/page_alloc: Add a bulk page allocator

On Mon, 22 Mar 2021 09:18:44 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > to be allocated and added to a list. > > The API is not guaranteed to return the requested number

[PATCH 2/3] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. The API is not guaranteed to return the requested number of pages and may fail if the preferred allocation zone has

Re: [PATCH 3/7] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 19, 2021 at 07:18:32PM +0100, Vlastimil Babka wrote: > On 3/12/21 4:43 PM, Mel Gorman wrote: > > This patch adds a new page allocator interface via alloc_pages_bulk, > > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > > to be allocated and added to a list. They

Re: [PATCH 3/7] mm/page_alloc: Add a bulk page allocator

On 3/12/21 4:43 PM, Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > to be allocated and added to a list. They can be freed in bulk using > free_pages_bulk(). > > The API is not

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Sun, 14 Mar 2021 12:52:32 + Mel Gorman wrote: > mm/page_alloc: Add an array-based interface to the bulk page allocator > > The existing callers for the bulk allocator are storing the pages in > arrays. This patch adds an array-based interface to the API to avoid > multiple list

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Mon, 15 Mar 2021 10:42:05 + Mel Gorman wrote: > On Sun, Mar 14, 2021 at 03:22:02PM +, Chuck Lever III wrote: > > >> Anyway, I'm not arguing against a bulk allocator, nor even saying this > > >> is a bad interface. It just maybe could be better. > > >> > > > > > > I think it puts

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Sun, Mar 14, 2021 at 03:22:02PM +, Chuck Lever III wrote: > >> Anyway, I'm not arguing against a bulk allocator, nor even saying this > >> is a bad interface. It just maybe could be better. > >> > > > > I think it puts more responsibility on the caller to use the API correctly > > but I

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

> On Mar 14, 2021, at 8:52 AM, Mel Gorman wrote: > > On Sat, Mar 13, 2021 at 07:33:43PM +, Matthew Wilcox wrote: >> On Sat, Mar 13, 2021 at 04:56:31PM +, Chuck Lever III wrote: >>> IME lists are indeed less CPU-efficient, but I wonder if that >>> expense is insignificant compared to

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Sat, Mar 13, 2021 at 07:33:43PM +, Matthew Wilcox wrote: > On Sat, Mar 13, 2021 at 04:56:31PM +, Chuck Lever III wrote: > > IME lists are indeed less CPU-efficient, but I wonder if that > > expense is insignificant compared to serialization primitives like > > disabling and re-enabling

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Sat, Mar 13, 2021 at 04:56:31PM +, Chuck Lever III wrote: > IME lists are indeed less CPU-efficient, but I wonder if that > expense is insignificant compared to serialization primitives like > disabling and re-enabling IRQs, which we are avoiding by using > bulk page allocation. Cache

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

> On Mar 13, 2021, at 11:39 AM, Matthew Wilcox wrote: > > On Sat, Mar 13, 2021 at 01:16:48PM +, Mel Gorman wrote: >>> I'm not claiming the pagevec is definitely a win, but it's very >>> unclear which tradeoff is actually going to lead to better performance. >>> Hopefully Jesper or Chuck

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Sat, Mar 13, 2021 at 01:16:48PM +, Mel Gorman wrote: > > I'm not claiming the pagevec is definitely a win, but it's very > > unclear which tradeoff is actually going to lead to better performance. > > Hopefully Jesper or Chuck can do some tests and figure out what actually > > works better

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 09:08:23PM +, Matthew Wilcox wrote: > > > > The result of the API is to deliver pages as a double-linked list via > > > > LRU (page->lru member). If you are planning to use llist, then how to > > > > handle this API change later? > > > > > > > > Have you notice that

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 04:03:50PM +, Mel Gorman wrote: > On Fri, Mar 12, 2021 at 02:58:14PM +, Matthew Wilcox wrote: > > On Fri, Mar 12, 2021 at 12:46:09PM +0100, Jesper Dangaard Brouer wrote: > > > In my page_pool patch I'm bulk allocating 64 pages. I wanted to ask if > > > this is too

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 02:58:14PM +, Matthew Wilcox wrote: > On Fri, Mar 12, 2021 at 12:46:09PM +0100, Jesper Dangaard Brouer wrote: > > In my page_pool patch I'm bulk allocating 64 pages. I wanted to ask if > > this is too much? (PP_ALLOC_CACHE_REFILL=64). > > > > The mlx5 driver have a

[PATCH 3/7] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). The API is not guaranteed to return the requested number of pages and

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 12:46:09PM +0100, Jesper Dangaard Brouer wrote: > In my page_pool patch I'm bulk allocating 64 pages. I wanted to ask if > this is too much? (PP_ALLOC_CACHE_REFILL=64). > > The mlx5 driver have a while loop for allocation 64 pages, which it > used in this case, that is why

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 12:43:31PM +, Matthew Wilcox wrote: > On Wed, Mar 10, 2021 at 10:46:15AM +, Mel Gorman wrote: > > +int __alloc_pages_bulk_nodemask(gfp_t gfp_mask, int preferred_nid, > > + nodemask_t *nodemask, int nr_pages, > > +

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Fri, Mar 12, 2021 at 12:46:09PM +0100, Jesper Dangaard Brouer wrote: > > > > > > > > + if (!zone) > > > > + return 0; > > > > + > > > > + /* Attempt the batch allocation */ > > > > + local_irq_save(flags); > > > > + pcp = _cpu_ptr(zone->pageset)->pcp; > >

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Wed, Mar 10, 2021 at 10:46:15AM +, Mel Gorman wrote: > +int __alloc_pages_bulk_nodemask(gfp_t gfp_mask, int preferred_nid, > + nodemask_t *nodemask, int nr_pages, > + struct list_head *list); For the next revision, can you ditch the

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Wed, 10 Mar 2021 11:38:36 + Mel Gorman wrote: > On Wed, Mar 10, 2021 at 01:04:17PM +0200, Shay Agroskin wrote: > > > > Mel Gorman writes: > > > > > > > > diff --git a/include/linux/gfp.h b/include/linux/gfp.h > > > index 8572a1474e16..4903d1cc48dc 100644 > > > ---

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Thu, 11 Mar 2021 08:42:00 + Mel Gorman wrote: > On Wed, Mar 10, 2021 at 03:46:50PM -0800, Andrew Morton wrote: > > On Wed, 10 Mar 2021 10:46:15 + Mel Gorman > > wrote: > > > > > This patch adds a new page allocator interface via alloc_pages_bulk, > > > and

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Thu, Mar 11, 2021 at 08:42:16AM -0800, Alexander Duyck wrote: > > @@ -4919,6 +4934,9 @@ static inline bool prepare_alloc_pages(gfp_t > > gfp_mask, unsigned int order, > > struct alloc_context *ac, gfp_t *alloc_mask, > > unsigned int *alloc_flags) > > { > > +

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Thu, Mar 11, 2021 at 3:49 AM Mel Gorman wrote: > > This patch adds a new page allocator interface via alloc_pages_bulk, > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > to be allocated and added to a list. They can be freed in bulk using > free_pages_bulk(). > > The API

[PATCH 2/5] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). The API is not guaranteed to return the requested number of pages and

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Wed, Mar 10, 2021 at 03:46:50PM -0800, Andrew Morton wrote: > On Wed, 10 Mar 2021 10:46:15 + Mel Gorman > wrote: > > > This patch adds a new page allocator interface via alloc_pages_bulk, > > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > > to be allocated and

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Wed, 10 Mar 2021 10:46:15 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > and __alloc_pages_bulk_nodemask. A caller requests a number of pages > to be allocated and added to a list. They can be freed in bulk using > free_pages_bulk(). Why am

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Wed, Mar 10, 2021 at 01:04:17PM +0200, Shay Agroskin wrote: > > Mel Gorman writes: > > > > > diff --git a/include/linux/gfp.h b/include/linux/gfp.h > > index 8572a1474e16..4903d1cc48dc 100644 > > --- a/include/linux/gfp.h > > +++ b/include/linux/gfp.h > > @@ -515,6 +515,10 @@ static inline

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

Mel Gorman writes: diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 8572a1474e16..4903d1cc48dc 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -515,6 +515,10 @@ static inline int arch_make_page_accessible(struct page *page) } #endif +int

[PATCH 2/5] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). The API is not guaranteed to return the requested number of pages and

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

On Tue, Mar 09, 2021 at 05:12:30PM +, Christoph Hellwig wrote: > Would vmalloc be another good user of this API? > > > + /* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */ > > + if (!prepare_alloc_pages(gfp_mask, 0, preferred_nid, nodemask, , > > _mask, _flags)) > > This

Re: [PATCH 2/5] mm/page_alloc: Add a bulk page allocator

Would vmalloc be another good user of this API? > + /* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */ > + if (!prepare_alloc_pages(gfp_mask, 0, preferred_nid, nodemask, , > _mask, _flags)) This crazy long line is really hard to follow. > + return 0; > +

[PATCH 2/5] mm/page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). The API is not guaranteed to return the requested number of pages and

[PATCH 2/3] mm, page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). The API is not guaranteed to return the requested number of pages and

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Mon, Jan 16, 2017 at 03:25:18PM +0100, Jesper Dangaard Brouer wrote: > On Mon, 9 Jan 2017 16:35:18 + > Mel Gorman wrote: > > > This patch adds a new page allocator interface via alloc_pages_bulk, > > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Mon, Jan 16, 2017 at 03:25:18PM +0100, Jesper Dangaard Brouer wrote: > On Mon, 9 Jan 2017 16:35:18 + > Mel Gorman wrote: > > > This patch adds a new page allocator interface via alloc_pages_bulk, > > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a > > number of

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Mon, 9 Jan 2017 16:35:18 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a > number of pages to be allocated and added to a list. They can be

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Mon, 9 Jan 2017 16:35:18 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a > number of pages to be allocated and added to a list. They can be freed in > bulk using

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Tue, Jan 10, 2017 at 12:00:27PM +0800, Hillf Danton wrote: > > It shows a roughly 50-60% reduction in the cost of allocating pages. > > The free paths are not improved as much but relatively little can be batched > > there. It's not quite as fast as it could be but taking further shortcuts > >

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Tue, Jan 10, 2017 at 12:00:27PM +0800, Hillf Danton wrote: > > It shows a roughly 50-60% reduction in the cost of allocating pages. > > The free paths are not improved as much but relatively little can be batched > > there. It's not quite as fast as it could be but taking further shortcuts > >

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Tuesday, January 10, 2017 12:35 AM Mel Gorman wrote: > > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a > number of pages to be allocated and added to a list. They can be freed in > bulk using

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Tuesday, January 10, 2017 12:35 AM Mel Gorman wrote: > > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a > number of pages to be allocated and added to a list. They can be freed in > bulk using

[PATCH 4/4] mm, page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). Note that it would theoretically be possible to use

[PATCH 4/4] mm, page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_pages_bulk(). Note that it would theoretically be possible to use

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Wed, Jan 04, 2017 at 02:48:44PM +0100, Jesper Dangaard Brouer wrote: > On Wed, 4 Jan 2017 11:10:49 + > > The API is not guaranteed to return the requested number of pages and > > may fail if the preferred allocation zone has limited free memory, > > the cpuset changes during the allocation

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Wed, Jan 04, 2017 at 02:48:44PM +0100, Jesper Dangaard Brouer wrote: > On Wed, 4 Jan 2017 11:10:49 + > > The API is not guaranteed to return the requested number of pages and > > may fail if the preferred allocation zone has limited free memory, > > the cpuset changes during the allocation

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Wed, 4 Jan 2017 11:10:49 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests > a number of pages to be allocated and added to a list. They can be >

Re: [PATCH 4/4] mm, page_alloc: Add a bulk page allocator

On Wed, 4 Jan 2017 11:10:49 + Mel Gorman wrote: > This patch adds a new page allocator interface via alloc_pages_bulk, > __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests > a number of pages to be allocated and added to a list. They can be > freed in bulk using

[PATCH 4/4] mm, page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_hot_cold_page_list. The API is not guaranteed to return the

[PATCH 4/4] mm, page_alloc: Add a bulk page allocator

This patch adds a new page allocator interface via alloc_pages_bulk, __alloc_pages_bulk and __alloc_pages_bulk_nodemask. A caller requests a number of pages to be allocated and added to a list. They can be freed in bulk using free_hot_cold_page_list. The API is not guaranteed to return the