Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Jörn Engel
On Fri, Jul 24, 2015 at 12:49:14PM -0700, David Rientjes wrote: > > I don't see the cond_resched() you propose to add, but the need for it is > obvious with a large user-written nr_hugepages in the above loop. > > The suggestion is to check the conditional, reschedule if needed (and if > so,

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Davidlohr Bueso
On Fri, 2015-07-24 at 10:12 -0700, Jörn Engel wrote: > On Fri, Jul 24, 2015 at 08:59:59AM +0200, Michal Hocko wrote: > > On Thu 23-07-15 14:54:31, Spencer Baugh wrote: > > > From: Joern Engel > > > > > > ~150ms scheduler latency for both observed in the wild. > > > > This is way to vague. Could

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread David Rientjes
On Thu, 23 Jul 2015, Jörn Engel wrote: > > The loop is dropping the lock simply to do the allocation and it needs to > > compare with the user-written number of hugepages to allocate. > > And at this point the existing code is racy. Page allocation might > block for minutes trying to free some

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Jörn Engel
On Fri, Jul 24, 2015 at 08:59:59AM +0200, Michal Hocko wrote: > On Thu 23-07-15 14:54:31, Spencer Baugh wrote: > > From: Joern Engel > > > > ~150ms scheduler latency for both observed in the wild. > > This is way to vague. Could you describe your problem somehow more, > please? > There are

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Michal Hocko
On Thu 23-07-15 14:54:31, Spencer Baugh wrote: > From: Joern Engel > > ~150ms scheduler latency for both observed in the wild. This is way to vague. Could you describe your problem somehow more, please? There are schduling points in the page allocator (when it triggers the reclaim), why are

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Michal Hocko
On Thu 23-07-15 14:54:31, Spencer Baugh wrote: From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. This is way to vague. Could you describe your problem somehow more, please? There are schduling points in the page allocator (when it triggers the reclaim),

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Jörn Engel
On Fri, Jul 24, 2015 at 12:49:14PM -0700, David Rientjes wrote: I don't see the cond_resched() you propose to add, but the need for it is obvious with a large user-written nr_hugepages in the above loop. The suggestion is to check the conditional, reschedule if needed (and if so, recheck

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Davidlohr Bueso
On Fri, 2015-07-24 at 10:12 -0700, Jörn Engel wrote: On Fri, Jul 24, 2015 at 08:59:59AM +0200, Michal Hocko wrote: On Thu 23-07-15 14:54:31, Spencer Baugh wrote: From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. This is way to vague. Could

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread David Rientjes
On Thu, 23 Jul 2015, Jörn Engel wrote: The loop is dropping the lock simply to do the allocation and it needs to compare with the user-written number of hugepages to allocate. And at this point the existing code is racy. Page allocation might block for minutes trying to free some

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-24 Thread Jörn Engel
On Fri, Jul 24, 2015 at 08:59:59AM +0200, Michal Hocko wrote: On Thu 23-07-15 14:54:31, Spencer Baugh wrote: From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. This is way to vague. Could you describe your problem somehow more, please? There are

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Jörn Engel
On Thu, Jul 23, 2015 at 03:54:43PM -0700, David Rientjes wrote: > On Thu, 23 Jul 2015, Jörn Engel wrote: > > > > This is wrong, you'd want to do any cond_resched() before the page > > > allocation to avoid racing with an update to h->nr_huge_pages or > > > h->surplus_huge_pages while

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread David Rientjes
On Thu, 23 Jul 2015, Jörn Engel wrote: > > This is wrong, you'd want to do any cond_resched() before the page > > allocation to avoid racing with an update to h->nr_huge_pages or > > h->surplus_huge_pages while hugetlb_lock was dropped that would result in > > the page having been uselessly

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Jörn Engel
On Thu, Jul 23, 2015 at 03:08:58PM -0700, David Rientjes wrote: > On Thu, 23 Jul 2015, Spencer Baugh wrote: > > From: Joern Engel > > > > ~150ms scheduler latency for both observed in the wild. > > > > Signed-off-by: Joern Engel > > Signed-off-by: Spencer Baugh > > --- > > mm/hugetlb.c | 2

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread David Rientjes
On Thu, 23 Jul 2015, Spencer Baugh wrote: > From: Joern Engel > > ~150ms scheduler latency for both observed in the wild. > > Signed-off-by: Joern Engel > Signed-off-by: Spencer Baugh > --- > mm/hugetlb.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/mm/hugetlb.c

[PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Spencer Baugh
From: Joern Engel ~150ms scheduler latency for both observed in the wild. Signed-off-by: Joern Engel Signed-off-by: Spencer Baugh --- mm/hugetlb.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a8c3087..2eb6919 100644 --- a/mm/hugetlb.c +++

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread David Rientjes
On Thu, 23 Jul 2015, Jörn Engel wrote: This is wrong, you'd want to do any cond_resched() before the page allocation to avoid racing with an update to h-nr_huge_pages or h-surplus_huge_pages while hugetlb_lock was dropped that would result in the page having been uselessly allocated.

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Jörn Engel
On Thu, Jul 23, 2015 at 03:54:43PM -0700, David Rientjes wrote: On Thu, 23 Jul 2015, Jörn Engel wrote: This is wrong, you'd want to do any cond_resched() before the page allocation to avoid racing with an update to h-nr_huge_pages or h-surplus_huge_pages while hugetlb_lock was

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Jörn Engel
On Thu, Jul 23, 2015 at 03:08:58PM -0700, David Rientjes wrote: On Thu, 23 Jul 2015, Spencer Baugh wrote: From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. Signed-off-by: Joern Engel jo...@logfs.org Signed-off-by: Spencer Baugh

[PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread Spencer Baugh
From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. Signed-off-by: Joern Engel jo...@logfs.org Signed-off-by: Spencer Baugh sba...@catern.com --- mm/hugetlb.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index

Re: [PATCH] hugetlb: cond_resched for set_max_huge_pages and follow_hugetlb_page

2015-07-23 Thread David Rientjes
On Thu, 23 Jul 2015, Spencer Baugh wrote: From: Joern Engel jo...@logfs.org ~150ms scheduler latency for both observed in the wild. Signed-off-by: Joern Engel jo...@logfs.org Signed-off-by: Spencer Baugh sba...@catern.com --- mm/hugetlb.c | 2 ++ 1 file changed, 2 insertions(+)