RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Odzioba, Lukasz
On Thu 16-06-16 08:19 PM, Michal Hocko wrote: > > On Thu 16-06-16 18:08:57, Odzioba, Lukasz wrote: > I am not able to find clear reasons why we shouldn't do it for the rest. > Ok so what do we do now? I'll send v2 with proposed changes. > Then do we still want to have stats on those pvecs? > In

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Odzioba, Lukasz
On Thu 16-06-16 08:19 PM, Michal Hocko wrote: > > On Thu 16-06-16 18:08:57, Odzioba, Lukasz wrote: > I am not able to find clear reasons why we shouldn't do it for the rest. > Ok so what do we do now? I'll send v2 with proposed changes. > Then do we still want to have stats on those pvecs? > In

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Michal Hocko
On Thu 16-06-16 18:08:57, Odzioba, Lukasz wrote: > On Thru 09-06-16 02:22 PM Michal Hocko wrote: > > I agree it would be better to do the same for others as well. Even if > > this is not an immediate problem for those. > > I am not able to find clear reasons why we shouldn't do it for the rest. >

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Michal Hocko
On Thu 16-06-16 18:08:57, Odzioba, Lukasz wrote: > On Thru 09-06-16 02:22 PM Michal Hocko wrote: > > I agree it would be better to do the same for others as well. Even if > > this is not an immediate problem for those. > > I am not able to find clear reasons why we shouldn't do it for the rest. >

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Odzioba, Lukasz
On Thru 09-06-16 02:22 PM Michal Hocko wrote: > I agree it would be better to do the same for others as well. Even if > this is not an immediate problem for those. I am not able to find clear reasons why we shouldn't do it for the rest. Ok so what do we do now? I'll send v2 with proposed changes.

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-16 Thread Odzioba, Lukasz
On Thru 09-06-16 02:22 PM Michal Hocko wrote: > I agree it would be better to do the same for others as well. Even if > this is not an immediate problem for those. I am not able to find clear reasons why we shouldn't do it for the rest. Ok so what do we do now? I'll send v2 with proposed changes.

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-13 Thread Odzioba, Lukasz
On 09-06-16 17:42:00, Dave Hansen wrote: > Does your workload put large pages in and out of those pvecs, though? > If your system doesn't have any activity, then all we've shown is that > they're not a problem when not in use. But what about when we use them? It doesn't. To use them extensively

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-13 Thread Odzioba, Lukasz
On 09-06-16 17:42:00, Dave Hansen wrote: > Does your workload put large pages in and out of those pvecs, though? > If your system doesn't have any activity, then all we've shown is that > they're not a problem when not in use. But what about when we use them? It doesn't. To use them extensively

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Dave Hansen
On 06/09/2016 01:50 AM, Odzioba, Lukasz wrote: > On 08-06-16 17:31:00, Dave Hansen wrote: >> Do we have any statistics that tell us how many pages are sitting the >> lru pvecs? Although this helps the problem overall, don't we still have >> a problem with memory being held in such an opaque

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Dave Hansen
On 06/09/2016 01:50 AM, Odzioba, Lukasz wrote: > On 08-06-16 17:31:00, Dave Hansen wrote: >> Do we have any statistics that tell us how many pages are sitting the >> lru pvecs? Although this helps the problem overall, don't we still have >> a problem with memory being held in such an opaque

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Michal Hocko
On Wed 08-06-16 09:34:01, Dave Hansen wrote: > On 06/08/2016 09:06 AM, Michal Hocko wrote: > >> > Do we have any statistics that tell us how many pages are sitting the > >> > lru pvecs? Although this helps the problem overall, don't we still have > >> > a problem with memory being held in such an

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Michal Hocko
On Wed 08-06-16 09:34:01, Dave Hansen wrote: > On 06/08/2016 09:06 AM, Michal Hocko wrote: > >> > Do we have any statistics that tell us how many pages are sitting the > >> > lru pvecs? Although this helps the problem overall, don't we still have > >> > a problem with memory being held in such an

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Odzioba, Lukasz
On 08-06-16 17:31:00, Dave Hansen wrote: > Do we have any statistics that tell us how many pages are sitting the > lru pvecs? Although this helps the problem overall, don't we still have > a problem with memory being held in such an opaque place? >From what I observed the problem is mainly with

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Odzioba, Lukasz
On 08-06-16 17:31:00, Dave Hansen wrote: > Do we have any statistics that tell us how many pages are sitting the > lru pvecs? Although this helps the problem overall, don't we still have > a problem with memory being held in such an opaque place? >From what I observed the problem is mainly with

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Odzioba, Lukasz
On Wed 08-07-16 17:04:00, Michal Hocko wrote: > I do not see how a SIGTERM would make any difference. But see below. This is how we encounter this problem initially, by hitting ctr-c while running parallel memory intensive workload, which ended up not calling munmap on allocated memory. > Is

RE: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-09 Thread Odzioba, Lukasz
On Wed 08-07-16 17:04:00, Michal Hocko wrote: > I do not see how a SIGTERM would make any difference. But see below. This is how we encounter this problem initially, by hitting ctr-c while running parallel memory intensive workload, which ended up not calling munmap on allocated memory. > Is

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Dave Hansen
On 06/08/2016 09:06 AM, Michal Hocko wrote: >> > Do we have any statistics that tell us how many pages are sitting the >> > lru pvecs? Although this helps the problem overall, don't we still have >> > a problem with memory being held in such an opaque place? > Is it really worth bothering when we

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Dave Hansen
On 06/08/2016 09:06 AM, Michal Hocko wrote: >> > Do we have any statistics that tell us how many pages are sitting the >> > lru pvecs? Although this helps the problem overall, don't we still have >> > a problem with memory being held in such an opaque place? > Is it really worth bothering when we

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Michal Hocko
On Wed 08-06-16 08:31:21, Dave Hansen wrote: > On 06/08/2016 07:35 AM, Lukasz Odzioba wrote: > > diff --git a/mm/swap.c b/mm/swap.c > > index 9591614..3fe4f18 100644 > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -391,9 +391,8 @@ static void __lru_cache_add(struct page *page) > > struct

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Michal Hocko
On Wed 08-06-16 08:31:21, Dave Hansen wrote: > On 06/08/2016 07:35 AM, Lukasz Odzioba wrote: > > diff --git a/mm/swap.c b/mm/swap.c > > index 9591614..3fe4f18 100644 > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -391,9 +391,8 @@ static void __lru_cache_add(struct page *page) > > struct

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Dave Hansen
On 06/08/2016 07:35 AM, Lukasz Odzioba wrote: > diff --git a/mm/swap.c b/mm/swap.c > index 9591614..3fe4f18 100644 > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -391,9 +391,8 @@ static void __lru_cache_add(struct page *page) > struct pagevec *pvec = _cpu_var(lru_add_pvec); > >

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Dave Hansen
On 06/08/2016 07:35 AM, Lukasz Odzioba wrote: > diff --git a/mm/swap.c b/mm/swap.c > index 9591614..3fe4f18 100644 > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -391,9 +391,8 @@ static void __lru_cache_add(struct page *page) > struct pagevec *pvec = _cpu_var(lru_add_pvec); > >

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Michal Hocko
On Wed 08-06-16 16:35:37, Lukasz Odzioba wrote: > When the application does not exit cleanly (i.e. SIGTERM) we might I do not see how a SIGTERM would make any difference. But see below. > end up with some pages in lru_add_pvec, which is ok. With THP > enabled huge pages may also end up on per

Re: [PATCH 1/1] mm/swap.c: flush lru_add pvecs on compound page arrival

2016-06-08 Thread Michal Hocko
On Wed 08-06-16 16:35:37, Lukasz Odzioba wrote: > When the application does not exit cleanly (i.e. SIGTERM) we might I do not see how a SIGTERM would make any difference. But see below. > end up with some pages in lru_add_pvec, which is ok. With THP > enabled huge pages may also end up on per