Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Mon, Jan 05, 2015 at 08:01:45PM -0800, Gregory Fong wrote:
> +linux-mm and linux-kernel (not sure how those got removed from cc,
> sorry about that)
>
> On Mon, Jan 5, 2015 at 7:58 PM, Gregory Fong wrote:
> > Hi Joonsoo,
> >
> > On Wed, May 28, 2014 at 12:04 AM, Joonsoo Kim
> > wrote:
> >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> >> index 674ade7..ca678b6 100644
> >> --- a/mm/page_alloc.c
> >> +++ b/mm/page_alloc.c
> >> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page,
> >> unsigned int order)
> >> }
> >>
> >> #ifdef CONFIG_CMA
> >> +void adjust_managed_cma_page_count(struct zone *zone, long count)
> >> +{
> >> + unsigned long flags;
> >> + long total, cma, movable;
> >> +
> >> + spin_lock_irqsave(&zone->lock, flags);
> >> + zone->managed_cma_pages += count;
> >> +
> >> + total = zone->managed_pages;
> >> + cma = zone->managed_cma_pages;
> >> + movable = total - cma - high_wmark_pages(zone);
> >> +
> >> + /* No cma pages, so do only movable allocation */
> >> + if (cma <= 0) {
> >> + zone->max_try_movable = pageblock_nr_pages;
> >> + zone->max_try_cma = 0;
> >> + goto out;
> >> + }
> >> +
> >> + /*
> >> +* We want to consume cma pages with well balanced ratio so that
> >> +* we have consumed enough cma pages before the reclaim. For this
> >> +* purpose, we can use the ratio, movable : cma. And we doesn't
> >> +* want to switch too frequently, because it prevent allocated
> >> pages
> >> +* from beging successive and it is bad for some sorts of devices.
> >> +* I choose pageblock_nr_pages for the minimum amount of successive
> >> +* allocation because it is the size of a huge page and
> >> fragmentation
> >> +* avoidance is implemented based on this size.
> >> +*
> >> +* To meet above criteria, I derive following equation.
> >> +*
> >> +* if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> >> +* else (movable <= cma) then; movable : cma = pageblock_nr_pages
> >> : X
> >> +*/
> >> + if (movable > cma) {
> >> + zone->max_try_movable =
> >> + (movable * pageblock_nr_pages) / cma;
> >> + zone->max_try_cma = pageblock_nr_pages;
> >> + } else {
> >> + zone->max_try_movable = pageblock_nr_pages;
> >> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> >
> > I don't know if anyone's already pointed this out (didn't see anything
> > when searching lkml), but while testing this, I noticed this can
> > result in a div by zero under memory pressure (movable becomes 0).
> > This is not unlikely when the majority of pages are in CMA regions
> > (this may seem pathological but we do actually do this right now).
Hello,
Yes, you are right. Thanks for pointing this out.
I will fix it on next version.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
+linux-mm and linux-kernel (not sure how those got removed from cc,
sorry about that)
On Mon, Jan 5, 2015 at 7:58 PM, Gregory Fong wrote:
> Hi Joonsoo,
>
> On Wed, May 28, 2014 at 12:04 AM, Joonsoo Kim wrote:
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 674ade7..ca678b6 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page,
>> unsigned int order)
>> }
>>
>> #ifdef CONFIG_CMA
>> +void adjust_managed_cma_page_count(struct zone *zone, long count)
>> +{
>> + unsigned long flags;
>> + long total, cma, movable;
>> +
>> + spin_lock_irqsave(&zone->lock, flags);
>> + zone->managed_cma_pages += count;
>> +
>> + total = zone->managed_pages;
>> + cma = zone->managed_cma_pages;
>> + movable = total - cma - high_wmark_pages(zone);
>> +
>> + /* No cma pages, so do only movable allocation */
>> + if (cma <= 0) {
>> + zone->max_try_movable = pageblock_nr_pages;
>> + zone->max_try_cma = 0;
>> + goto out;
>> + }
>> +
>> + /*
>> +* We want to consume cma pages with well balanced ratio so that
>> +* we have consumed enough cma pages before the reclaim. For this
>> +* purpose, we can use the ratio, movable : cma. And we doesn't
>> +* want to switch too frequently, because it prevent allocated pages
>> +* from beging successive and it is bad for some sorts of devices.
>> +* I choose pageblock_nr_pages for the minimum amount of successive
>> +* allocation because it is the size of a huge page and fragmentation
>> +* avoidance is implemented based on this size.
>> +*
>> +* To meet above criteria, I derive following equation.
>> +*
>> +* if (movable > cma) then; movable : cma = X : pageblock_nr_pages
>> +* else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
>> +*/
>> + if (movable > cma) {
>> + zone->max_try_movable =
>> + (movable * pageblock_nr_pages) / cma;
>> + zone->max_try_cma = pageblock_nr_pages;
>> + } else {
>> + zone->max_try_movable = pageblock_nr_pages;
>> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
>
> I don't know if anyone's already pointed this out (didn't see anything
> when searching lkml), but while testing this, I noticed this can
> result in a div by zero under memory pressure (movable becomes 0).
> This is not unlikely when the majority of pages are in CMA regions
> (this may seem pathological but we do actually do this right now).
>
> [0.249674] Division by zero in kernel.
> [0.249682] CPU: 2 PID: 1 Comm: swapper/0 Not tainted
> 3.14.13-1.3pre-00368-g4d90957-dirty #10
> [0.249710] [] (unwind_backtrace) from []
> (show_stack+0x10/0x14)
> [0.249725] [] (show_stack) from []
> (dump_stack+0x80/0x90)
> [0.249740] [] (dump_stack) from [] (Ldiv0+0x8/0x10)
> [0.249751] [] (Ldiv0) from []
> (adjust_managed_cma_page_count+0x64/0xd8)
> [0.249762] [] (adjust_managed_cma_page_count) from
> [] (cma_release+0xa8/0xe0)
> [0.249776] [] (cma_release) from []
> (cma_drvr_probe+0x378/0x470)
> [0.249787] [] (cma_drvr_probe) from []
> (platform_drv_probe+0x18/0x48)
> [0.249799] [] (platform_drv_probe) from []
> (driver_probe_device+0xac/0x3a4)
> [0.249808] [] (driver_probe_device) from []
> (__driver_attach+0x8c/0x90)
> [0.249817] [] (__driver_attach) from []
> (bus_for_each_dev+0x60/0x94)
> [0.249825] [] (bus_for_each_dev) from []
> (bus_add_driver+0x15c/0x218)
> [0.249834] [] (bus_add_driver) from []
> (driver_register+0x78/0xf8)
> [0.249841] [] (driver_register) from []
> (platform_driver_probe+0x20/0xa4)
> [0.249849] [] (platform_driver_probe) from []
> (do_one_initcall+0xd4/0x17c)
> [0.249857] [] (do_one_initcall) from []
> (kernel_init_freeable+0x13c/0x1dc)
> [0.249864] [] (kernel_init_freeable) from []
> (kernel_init+0x8/0xe8)
> [0.249873] [] (kernel_init) from []
> (ret_from_fork+0x14/0x3c)
>
> Could probably just add something above similar to the "no cma pages" case,
> like
>
> /* No movable pages, so only do CMA allocation */
> if (movable <= 0) {
> zone->max_try_cma = pageblock_nr_pages;
> zone->max_try_movable = 0;
> goto out;
> }
>
>> + }
>> +
>> +out:
>> + zone->nr_try_movable = zone->max_try_movable;
>> + zone->nr_try_cma = zone->max_try_cma;
>> +
>> + spin_unlock_irqrestore(&zone->lock, flags);
>> +}
>> +
>
> Best regards,
> Gregory
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Wed, May 28, 2014 at 3:04 PM, Joonsoo Kim wrote: > CMA is introduced to provide physically contiguous pages at runtime. > For this purpose, it reserves memory at boot time. Although it reserve > memory, this reserved memory can be used for movable memory allocation > request. This usecase is beneficial to the system that needs this CMA > reserved memory infrequently and it is one of main purpose of > introducing CMA. > > But, there is a problem in current implementation. The problem is that > it works like as just reserved memory approach. The pages on cma reserved > memory are hardly used for movable memory allocation. This is caused by > combination of allocation and reclaim policy. > > The pages on cma reserved memory are allocated if there is no movable > memory, that is, as fallback allocation. So the time this fallback > allocation is started is under heavy memory pressure. Although it is under > memory pressure, movable allocation easily succeed, since there would be > many pages on cma reserved memory. But this is not the case for unmovable > and reclaimable allocation, because they can't use the pages on cma > reserved memory. These allocations regard system's free memory as > (free pages - free cma pages) on watermark checking, that is, free > unmovable pages + free reclaimable pages + free movable pages. Because > we already exhausted movable pages, only free pages we have are unmovable > and reclaimable types and this would be really small amount. So watermark > checking would be failed. It will wake up kswapd to make enough free > memory for unmovable and reclaimable allocation and kswapd will do. > So before we fully utilize pages on cma reserved memory, kswapd start to > reclaim memory and try to make free memory over the high watermark. This > watermark checking by kswapd doesn't take care free cma pages so many > movable pages would be reclaimed. After then, we have a lot of movable > pages again, so fallback allocation doesn't happen again. To conclude, > amount of free memory on meminfo which includes free CMA pages is moving > around 512 MB if I reserve 512 MB memory for CMA. > > I found this problem on following experiment. > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > make -j16 > > CMA reserve:0 MB512 MB > Elapsed-time: 225.2 472.5 > Average-MemFree:322490 KB 630839 KB > > To solve this problem, I can think following 2 possible solutions. > 1. allocate the pages on cma reserved memory first, and if they are >exhausted, allocate movable pages. > 2. interleaved allocation: try to allocate specific amounts of memory >from cma reserved memory and then allocate from free movable memory. > > I tested #1 approach and found the problem. Although free memory on > meminfo can move around low watermark, there is large fluctuation on free > memory, because too many pages are reclaimed when kswapd is invoked. > Reason for this behaviour is that successive allocated CMA pages are > on the LRU list in that order and kswapd reclaim them in same order. > These memory doesn't help watermark checking from kwapd, so too many > pages are reclaimed, I guess. Could you send more information about this part? I want to do some test around it. I use this way in my patch. Thanks, Hui > > So, I implement #2 approach. > One thing I should note is that we should not change allocation target > (movable list or cma) on each allocation attempt, since this prevent > allocated pages to be in physically succession, so some I/O devices can > be hurt their performance. To solve this, I keep allocation target > in at least pageblock_nr_pages attempts and make this number reflect > ratio, free pages without free cma pages to free cma pages. With this > approach, system works very smoothly and fully utilize the pages on > cma reserved memory. > > Following is the experimental result of this patch. > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > make -j16 > > > CMA reserve:0 MB512 MB > Elapsed-time: 225.2 472.5 > Average-MemFree:322490 KB 630839 KB > nr_free_cma:0 131068 > pswpin: 0 261666 > pswpout:75 1241363 > > > CMA reserve:0 MB512 MB > Elapsed-time: 222.7 224 > Average-MemFree:325595 KB 393033 KB > nr_free_cma:0 61001 > pswpin: 0 6 > pswpout:44 502 > > There is no difference if we don't have cma reserved memory (0 MB case). > But, with cma reserved memory (512 MB case), we fully utilize these > reserved memory through this patch and the system behaves like as > it doesn't reserve any memory. > > With this patch, we aggressively allocate the pages on cma reserved memory > so latency of CMA can arise. Below is the experimental result about > latency. > > 4 CPUs, 1024 MB, VIRTUAL MACH
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
I'm not sure what I'm doing wrong.
These are my code.
770 #ifdef CONFIG_CMA
771 void adjust_managed_cma_page_count(struct zone *zone, long count)
772 {
773 unsigned long flags;
774 long total, cma, movable;
775
776 spin_lock_irqsave(&zone->lock, flags);
777
778 zone->managed_cma_pages += count;
779
780 total = zone->managed_pages;
781 cma = zone->managed_cma_pages;
782 movable = total - cma - high_wmark_pages(zone);
783
784 printk("count=%ld total=%ld cma=%ld movable=%ld\n",
785count, total, cma, movable);
786
2014-06-02 오후 3:23, Joonsoo Kim 쓴 글:
On Mon, Jun 02, 2014 at 02:54:30PM +0900, Gioh Kim wrote:
I found 2 problems at my platform.
1st is occured when I set CMA size 528MB and total memory is 960MB.
I print some values in adjust_managed_cma_page_count(),
the total value becomes 105439 and cma value 131072.
Finally movable value becomes negative value.
The total value 105439 means 411MB.
Is the zone->managed_pages value pages amount except the CMA?
I think zone->managed_pages value is including CMA size but it's value is
strange.
Hmm...
zone->managed_pages includes nr of CMA pages.
Is there any mistake about your printk?
2nd is a kernel panic at __netdev_alloc_skb().
I'm not sure it is caused by the CMA.
I'm checking it again and going to send you another report with detail
call-stacks.
Okay.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Mon, Jun 02, 2014 at 02:54:30PM +0900, Gioh Kim wrote: > I found 2 problems at my platform. > > 1st is occured when I set CMA size 528MB and total memory is 960MB. > I print some values in adjust_managed_cma_page_count(), > the total value becomes 105439 and cma value 131072. > Finally movable value becomes negative value. > > The total value 105439 means 411MB. > Is the zone->managed_pages value pages amount except the CMA? > I think zone->managed_pages value is including CMA size but it's value is > strange. Hmm... zone->managed_pages includes nr of CMA pages. Is there any mistake about your printk? > > 2nd is a kernel panic at __netdev_alloc_skb(). > I'm not sure it is caused by the CMA. > I'm checking it again and going to send you another report with detail > call-stacks. Okay. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Sat, May 31, 2014 at 09:02:51AM +0900, Michal Nazarewicz wrote: > > On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote: > >> Is IS_ENABLED(CONFIG_CMA) necessary? > >> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ? > > On Fri, May 30 2014, Joonsoo Kim wrote: > > Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so > > removing IS_ENABLE(CONFIG_CMA) would break the build. > > That statement makes no sense. If zone->managed_cma_pages not being > defined is the problem, what you need is: > > +#ifdef CONFIG_CMA > + if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) > + page = __rmqueue_cma(zone, order); > +#endif > > If you use IS_ENABLED, zone-managed_cma_pages has to be defined > regardless of result of state of CONFIG_CMA. Hello, Oops. I totally misunderstand how IS_ENABLED works. Thanks for spotting this. Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
I found 2 problems at my platform.
1st is occured when I set CMA size 528MB and total memory is 960MB.
I print some values in adjust_managed_cma_page_count(),
the total value becomes 105439 and cma value 131072.
Finally movable value becomes negative value.
The total value 105439 means 411MB.
Is the zone->managed_pages value pages amount except the CMA?
I think zone->managed_pages value is including CMA size but it's value is
strange.
2nd is a kernel panic at __netdev_alloc_skb().
I'm not sure it is caused by the CMA.
I'm checking it again and going to send you another report with detail
call-stacks.
2014-05-30 오후 11:23, Joonsoo Kim 쓴 글:
2014-05-30 16:53 GMT+09:00 Gioh Kim :
Joonsoo,
I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
I didn't test fully but my board is turned on and working well if no frequent
memory allocations.
I'm sorry to send not-tested code.
I just want to report this during your working hour ;-)
I'm testing this this evening and reporting next week.
Have a nice weekend!
Thanks Gioh. :)
-- 8<
-
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7f97767..9ced736 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE,
MIGRATE_R
#ifdef CONFIG_CMA
[MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE,
MIGRATE_U
- [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
+ [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE,
MIGRATE_U
I don't want to use __rmqueue_fallback() for CMA.
__rmqueue_fallback() takes big order page rather than small order page
in order to steal large amount of pages and continue to use them in
next allocation attempts.
We can use CMA pages on limited cases, so stealing some pages from
other migrate type
to CMA type isn't good idea to me.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Wed, May 28 2014, Joonsoo Kim wrote:
> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int
> start_migratetype)
> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> int migratetype)
> {
> - struct page *page;
> + struct page *page = NULL;
> +
> + if (IS_ENABLED(CONFIG_CMA) &&
> + migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
Come to think of it, I would consider:
if (…) {
page = __rmqueue_cma(zone, order);
if (page)
goto done
}
…
done:
trace_mm_page_alloc_zone_locked(page, order, migratetype);
return page;
>
> retry_reserve:
> - page = __rmqueue_smallest(zone, order, migratetype);
> + if (!page)
> + page = __rmqueue_smallest(zone, order, migratetype);
>
The above would allow this if statement to go away.
> if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
> page = __rmqueue_fallback(zone, order, migratetype);
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz(o o)
ooo +--ooO--(_)--Ooo--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
> On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote: >> Is IS_ENABLED(CONFIG_CMA) necessary? >> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ? On Fri, May 30 2014, Joonsoo Kim wrote: > Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so > removing IS_ENABLE(CONFIG_CMA) would break the build. That statement makes no sense. If zone->managed_cma_pages not being defined is the problem, what you need is: +#ifdef CONFIG_CMA + if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) + page = __rmqueue_cma(zone, order); +#endif If you use IS_ENABLED, zone-managed_cma_pages has to be defined regardless of result of state of CONFIG_CMA. -- Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michał “mina86” Nazarewicz(o o) ooo +--ooO--(_)--Ooo-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-30 16:53 GMT+09:00 Gioh Kim :
> Joonsoo,
>
> I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
> I didn't test fully but my board is turned on and working well if no frequent
> memory allocations.
>
> I'm sorry to send not-tested code.
> I just want to report this during your working hour ;-)
>
> I'm testing this this evening and reporting next week.
> Have a nice weekend!
Thanks Gioh. :)
> -- 8<
> -
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 7f97767..9ced736 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE,
> MIGRATE_R
> #ifdef CONFIG_CMA
> [MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE,
> MIGRATE_U
> - [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
> + [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE,
> MIGRATE_U
I don't want to use __rmqueue_fallback() for CMA.
__rmqueue_fallback() takes big order page rather than small order page
in order to steal large amount of pages and continue to use them in
next allocation attempts.
We can use CMA pages on limited cases, so stealing some pages from
other migrate type
to CMA type isn't good idea to me.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
Joonsoo,
I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
I didn't test fully but my board is turned on and working well if no frequent
memory allocations.
I'm sorry to send not-tested code.
I just want to report this during your working hour ;-)
I'm testing this this evening and reporting next week.
Have a nice weekend!
-- 8<
-
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7f97767..9ced736 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE,
MIGRATE_R
#ifdef CONFIG_CMA
[MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE,
MIGRATE_U
- [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
+ [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE,
MIGRATE_U
#else
[MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE,
MIGRATE_R
#endif
@@ -1170,9 +1170,22 @@ static struct page *__rmqueue(struct zone *zone,
unsigned int
int migratetype)
{
struct page *page;
+ long free, free_cma, free_wmark;
retry_reserve:
- page = __rmqueue_smallest(zone, order, migratetype);
+ if (IS_ENABLED(CONFIG_CMA) && migratetype == MIGRATE_MOVABLE) {
+ if (zone->nr_try_movable) {
+ zone->nr_try_movable -= 1 << order;
+ } else if (zone->nr_try_cma) {
+ zone->nr_try_cma -= 1 << order;
+ migratetype = MIGRATE_CMA;
+ } else {
+ zone->nr_try_movable = zone->max_try_movable;
+ zone->nr_try_movable -= 1 << order;
+ zone->nr_try_cma = zone->max_try_cma;
+ }
+ }
+ page = __rmqueue_smallest(zone, order, migratetype);
if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
page = __rmqueue_fallback(zone, order, migratetype);
2014-05-28 오후 4:04, Joonsoo Kim 쓴 글:
> CMA is introduced to provide physically contiguous pages at runtime.
> For this purpose, it reserves memory at boot time. Although it reserve
> memory, this reserved memory can be used for movable memory allocation
> request. This usecase is beneficial to the system that needs this CMA
> reserved memory infrequently and it is one of main purpose of
> introducing CMA.
>
> But, there is a problem in current implementation. The problem is that
> it works like as just reserved memory approach. The pages on cma reserved
> memory are hardly used for movable memory allocation. This is caused by
> combination of allocation and reclaim policy.
>
> The pages on cma reserved memory are allocated if there is no movable
> memory, that is, as fallback allocation. So the time this fallback
> allocation is started is under heavy memory pressure. Although it is under
> memory pressure, movable allocation easily succeed, since there would be
> many pages on cma reserved memory. But this is not the case for unmovable
> and reclaimable allocation, because they can't use the pages on cma
> reserved memory. These allocations regard system's free memory as
> (free pages - free cma pages) on watermark checking, that is, free
> unmovable pages + free reclaimable pages + free movable pages. Because
> we already exhausted movable pages, only free pages we have are unmovable
> and reclaimable types and this would be really small amount. So watermark
> checking would be failed. It will wake up kswapd to make enough free
> memory for unmovable and reclaimable allocation and kswapd will do.
> So before we fully utilize pages on cma reserved memory, kswapd start to
> reclaim memory and try to make free memory over the high watermark. This
> watermark checking by kswapd doesn't take care free cma pages so many
> movable pages would be reclaimed. After then, we have a lot of movable
> pages again, so fallback allocation doesn't happen again. To conclude,
> amount of free memory on meminfo which includes free CMA pages is moving
> around 512 MB if I reserve 512 MB memory for CMA.
>
> I found this problem on following experiment.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> CMA reserve:0 MB512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree:322490 KB 630839 KB
>
> To solve this problem, I can think following 2 possible solutions.
> 1. allocate the pages on cma reserved memory first, and if they are
> exhausted, allocate movable pages.
> 2. interleaved allocation: try to allocate specific amounts of memory
> from cma reserved memory and then allocate from free movable memory.
>
> I tested #1 approach and found the problem. Although free memory on
> meminfo can move around low watermark, there is large fluctuation on free
>
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote:
>
> >>>+
> >>> /*
> >>>* Do the hard work of removing an element from the buddy allocator.
> >>>* Call me with the zone->lock already held.
> >>>@@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order,
> >>>int start_migratetype)
> >>> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> >>> int migratetype)
> >>> {
> >>>- struct page *page;
> >>>+ struct page *page = NULL;
> >>>+
> >>>+ if (IS_ENABLED(CONFIG_CMA) &&
> >>
> >>You might know that CONFIG_CMA is enabled and there is no CMA memory,
> >>because CONFIG_CMA_SIZE_MBYTES can be zero.
> >>Is IS_ENABLED(CONFIG_CMA) alright in that case?
> >
> >next line checks whether zone->managed_cma_pages is positive or not.
> >If there is no CMA memory, zone->managed_cma_pages will be zero and
> >we will skip to call __rmqueue_cma().
>
> Is IS_ENABLED(CONFIG_CMA) necessary?
> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so
removing IS_ENABLE(CONFIG_CMA) would break the build.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
+
/*
* Do the hard work of removing an element from the buddy allocator.
* Call me with the zone->lock already held.
@@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int
start_migratetype)
static struct page *__rmqueue(struct zone *zone, unsigned int order,
int migratetype)
{
- struct page *page;
+ struct page *page = NULL;
+
+ if (IS_ENABLED(CONFIG_CMA) &&
You might know that CONFIG_CMA is enabled and there is no CMA memory, because
CONFIG_CMA_SIZE_MBYTES can be zero.
Is IS_ENABLED(CONFIG_CMA) alright in that case?
next line checks whether zone->managed_cma_pages is positive or not.
If there is no CMA memory, zone->managed_cma_pages will be zero and
we will skip to call __rmqueue_cma().
Is IS_ENABLED(CONFIG_CMA) necessary?
What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
Thanks for review!!!
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
On Thu, May 29, 2014 at 04:24:58PM +0900, Gioh Kim wrote: > I've not understand your code fully. Please let me ask some silly questions. > > 2014-05-28 오후 4:04, Joonsoo Kim 쓴 글: > > CMA is introduced to provide physically contiguous pages at runtime. > > For this purpose, it reserves memory at boot time. Although it reserve > > memory, this reserved memory can be used for movable memory allocation > > request. This usecase is beneficial to the system that needs this CMA > > reserved memory infrequently and it is one of main purpose of > > introducing CMA. > > > > But, there is a problem in current implementation. The problem is that > > it works like as just reserved memory approach. The pages on cma reserved > > memory are hardly used for movable memory allocation. This is caused by > > combination of allocation and reclaim policy. > > > > The pages on cma reserved memory are allocated if there is no movable > > memory, that is, as fallback allocation. So the time this fallback > > allocation is started is under heavy memory pressure. Although it is under > > memory pressure, movable allocation easily succeed, since there would be > > many pages on cma reserved memory. But this is not the case for unmovable > > and reclaimable allocation, because they can't use the pages on cma > > reserved memory. These allocations regard system's free memory as > > (free pages - free cma pages) on watermark checking, that is, free > > unmovable pages + free reclaimable pages + free movable pages. Because > > we already exhausted movable pages, only free pages we have are unmovable > > and reclaimable types and this would be really small amount. So watermark > > checking would be failed. It will wake up kswapd to make enough free > > memory for unmovable and reclaimable allocation and kswapd will do. > > So before we fully utilize pages on cma reserved memory, kswapd start to > > reclaim memory and try to make free memory over the high watermark. This > > watermark checking by kswapd doesn't take care free cma pages so many > > movable pages would be reclaimed. After then, we have a lot of movable > > pages again, so fallback allocation doesn't happen again. To conclude, > > amount of free memory on meminfo which includes free CMA pages is moving > > around 512 MB if I reserve 512 MB memory for CMA. > > > > I found this problem on following experiment. > > > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > > make -j16 > > > > CMA reserve:0 MB512 MB > > Elapsed-time: 225.2 472.5 > > Average-MemFree:322490 KB 630839 KB > > > > To solve this problem, I can think following 2 possible solutions. > > 1. allocate the pages on cma reserved memory first, and if they are > > exhausted, allocate movable pages. > > 2. interleaved allocation: try to allocate specific amounts of memory > > from cma reserved memory and then allocate from free movable memory. > > > > I tested #1 approach and found the problem. Although free memory on > > meminfo can move around low watermark, there is large fluctuation on free > > memory, because too many pages are reclaimed when kswapd is invoked. > > Reason for this behaviour is that successive allocated CMA pages are > > on the LRU list in that order and kswapd reclaim them in same order. > > These memory doesn't help watermark checking from kwapd, so too many > > pages are reclaimed, I guess. > > > > So, I implement #2 approach. > > One thing I should note is that we should not change allocation target > > (movable list or cma) on each allocation attempt, since this prevent > > allocated pages to be in physically succession, so some I/O devices can > > be hurt their performance. To solve this, I keep allocation target > > in at least pageblock_nr_pages attempts and make this number reflect > > ratio, free pages without free cma pages to free cma pages. With this > > approach, system works very smoothly and fully utilize the pages on > > cma reserved memory. > > > > Following is the experimental result of this patch. > > > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > > make -j16 > > > > > > CMA reserve:0 MB512 MB > > Elapsed-time: 225.2 472.5 > > Average-MemFree:322490 KB 630839 KB > > nr_free_cma:0 131068 > > pswpin: 0 261666 > > pswpout:75 1241363 > > > > > > CMA reserve:0 MB512 MB > > Elapsed-time: 222.7 224 > > Average-MemFree:325595 KB 393033 KB > > nr_free_cma:0 61001 > > pswpin: 0 6 > > pswpout:44 502 > > > > There is no difference if we don't have cma reserved memory (0 MB case). > > But, with cma reserved memory (512 MB case), we fully utilize these > > reserved memory through this patch and the system behaves like as > > it doesn't reserve any memor
Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
I've not understand your code fully. Please let me ask some silly questions. 2014-05-28 오후 4:04, Joonsoo Kim 쓴 글: > CMA is introduced to provide physically contiguous pages at runtime. > For this purpose, it reserves memory at boot time. Although it reserve > memory, this reserved memory can be used for movable memory allocation > request. This usecase is beneficial to the system that needs this CMA > reserved memory infrequently and it is one of main purpose of > introducing CMA. > > But, there is a problem in current implementation. The problem is that > it works like as just reserved memory approach. The pages on cma reserved > memory are hardly used for movable memory allocation. This is caused by > combination of allocation and reclaim policy. > > The pages on cma reserved memory are allocated if there is no movable > memory, that is, as fallback allocation. So the time this fallback > allocation is started is under heavy memory pressure. Although it is under > memory pressure, movable allocation easily succeed, since there would be > many pages on cma reserved memory. But this is not the case for unmovable > and reclaimable allocation, because they can't use the pages on cma > reserved memory. These allocations regard system's free memory as > (free pages - free cma pages) on watermark checking, that is, free > unmovable pages + free reclaimable pages + free movable pages. Because > we already exhausted movable pages, only free pages we have are unmovable > and reclaimable types and this would be really small amount. So watermark > checking would be failed. It will wake up kswapd to make enough free > memory for unmovable and reclaimable allocation and kswapd will do. > So before we fully utilize pages on cma reserved memory, kswapd start to > reclaim memory and try to make free memory over the high watermark. This > watermark checking by kswapd doesn't take care free cma pages so many > movable pages would be reclaimed. After then, we have a lot of movable > pages again, so fallback allocation doesn't happen again. To conclude, > amount of free memory on meminfo which includes free CMA pages is moving > around 512 MB if I reserve 512 MB memory for CMA. > > I found this problem on following experiment. > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > make -j16 > > CMA reserve:0 MB512 MB > Elapsed-time: 225.2 472.5 > Average-MemFree:322490 KB 630839 KB > > To solve this problem, I can think following 2 possible solutions. > 1. allocate the pages on cma reserved memory first, and if they are > exhausted, allocate movable pages. > 2. interleaved allocation: try to allocate specific amounts of memory > from cma reserved memory and then allocate from free movable memory. > > I tested #1 approach and found the problem. Although free memory on > meminfo can move around low watermark, there is large fluctuation on free > memory, because too many pages are reclaimed when kswapd is invoked. > Reason for this behaviour is that successive allocated CMA pages are > on the LRU list in that order and kswapd reclaim them in same order. > These memory doesn't help watermark checking from kwapd, so too many > pages are reclaimed, I guess. > > So, I implement #2 approach. > One thing I should note is that we should not change allocation target > (movable list or cma) on each allocation attempt, since this prevent > allocated pages to be in physically succession, so some I/O devices can > be hurt their performance. To solve this, I keep allocation target > in at least pageblock_nr_pages attempts and make this number reflect > ratio, free pages without free cma pages to free cma pages. With this > approach, system works very smoothly and fully utilize the pages on > cma reserved memory. > > Following is the experimental result of this patch. > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > make -j16 > > > CMA reserve:0 MB512 MB > Elapsed-time: 225.2 472.5 > Average-MemFree:322490 KB 630839 KB > nr_free_cma:0 131068 > pswpin: 0 261666 > pswpout:75 1241363 > > > CMA reserve:0 MB512 MB > Elapsed-time: 222.7 224 > Average-MemFree:325595 KB 393033 KB > nr_free_cma:0 61001 > pswpin: 0 6 > pswpout:44 502 > > There is no difference if we don't have cma reserved memory (0 MB case). > But, with cma reserved memory (512 MB case), we fully utilize these > reserved memory through this patch and the system behaves like as > it doesn't reserve any memory. > > With this patch, we aggressively allocate the pages on cma reserved memory > so latency of CMA can arise. Below is the experimental result about > latency. > > 4 CPUs, 1024 MB, VIRTUAL MACHINE > CMA reserve: 512 MB > Backgound Workload:
[PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
CMA is introduced to provide physically contiguous pages at runtime. For this purpose, it reserves memory at boot time. Although it reserve memory, this reserved memory can be used for movable memory allocation request. This usecase is beneficial to the system that needs this CMA reserved memory infrequently and it is one of main purpose of introducing CMA. But, there is a problem in current implementation. The problem is that it works like as just reserved memory approach. The pages on cma reserved memory are hardly used for movable memory allocation. This is caused by combination of allocation and reclaim policy. The pages on cma reserved memory are allocated if there is no movable memory, that is, as fallback allocation. So the time this fallback allocation is started is under heavy memory pressure. Although it is under memory pressure, movable allocation easily succeed, since there would be many pages on cma reserved memory. But this is not the case for unmovable and reclaimable allocation, because they can't use the pages on cma reserved memory. These allocations regard system's free memory as (free pages - free cma pages) on watermark checking, that is, free unmovable pages + free reclaimable pages + free movable pages. Because we already exhausted movable pages, only free pages we have are unmovable and reclaimable types and this would be really small amount. So watermark checking would be failed. It will wake up kswapd to make enough free memory for unmovable and reclaimable allocation and kswapd will do. So before we fully utilize pages on cma reserved memory, kswapd start to reclaim memory and try to make free memory over the high watermark. This watermark checking by kswapd doesn't take care free cma pages so many movable pages would be reclaimed. After then, we have a lot of movable pages again, so fallback allocation doesn't happen again. To conclude, amount of free memory on meminfo which includes free CMA pages is moving around 512 MB if I reserve 512 MB memory for CMA. I found this problem on following experiment. 4 CPUs, 1024 MB, VIRTUAL MACHINE make -j16 CMA reserve:0 MB512 MB Elapsed-time: 225.2 472.5 Average-MemFree:322490 KB 630839 KB To solve this problem, I can think following 2 possible solutions. 1. allocate the pages on cma reserved memory first, and if they are exhausted, allocate movable pages. 2. interleaved allocation: try to allocate specific amounts of memory from cma reserved memory and then allocate from free movable memory. I tested #1 approach and found the problem. Although free memory on meminfo can move around low watermark, there is large fluctuation on free memory, because too many pages are reclaimed when kswapd is invoked. Reason for this behaviour is that successive allocated CMA pages are on the LRU list in that order and kswapd reclaim them in same order. These memory doesn't help watermark checking from kwapd, so too many pages are reclaimed, I guess. So, I implement #2 approach. One thing I should note is that we should not change allocation target (movable list or cma) on each allocation attempt, since this prevent allocated pages to be in physically succession, so some I/O devices can be hurt their performance. To solve this, I keep allocation target in at least pageblock_nr_pages attempts and make this number reflect ratio, free pages without free cma pages to free cma pages. With this approach, system works very smoothly and fully utilize the pages on cma reserved memory. Following is the experimental result of this patch. 4 CPUs, 1024 MB, VIRTUAL MACHINE make -j16 CMA reserve:0 MB512 MB Elapsed-time: 225.2 472.5 Average-MemFree:322490 KB 630839 KB nr_free_cma:0 131068 pswpin: 0 261666 pswpout:75 1241363 CMA reserve:0 MB512 MB Elapsed-time: 222.7 224 Average-MemFree:325595 KB 393033 KB nr_free_cma:0 61001 pswpin: 0 6 pswpout:44 502 There is no difference if we don't have cma reserved memory (0 MB case). But, with cma reserved memory (512 MB case), we fully utilize these reserved memory through this patch and the system behaves like as it doesn't reserve any memory. With this patch, we aggressively allocate the pages on cma reserved memory so latency of CMA can arise. Below is the experimental result about latency. 4 CPUs, 1024 MB, VIRTUAL MACHINE CMA reserve: 512 MB Backgound Workload: make -jN Real Workload: 8 MB CMA allocation/free 20 times with 5 sec interval N:14 816 Elapsed-time(Before): 4309.75 9511.09 12276.1 77103.5 Elapsed-time(After): 5391.69 16114.1 19380.3 34879.2 So generally we can see latency increase. Ratio of this increase is rath
