Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-12-05 Thread Jerome Marchand
On 11/26/2014 06:35 PM, Michal Hocko wrote: > On Tue 25-11-14 16:00:06, Michal Hocko wrote: >> On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: >>> On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko wrote: On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: > On Thu, Nov 20, 2014 at

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-12-05 Thread Jerome Marchand
On 11/26/2014 06:35 PM, Michal Hocko wrote: On Tue 25-11-14 16:00:06, Michal Hocko wrote: On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko mho...@suse.cz wrote: On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: On Thu, Nov 20, 2014 at

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-26 Thread Michal Hocko
On Tue 25-11-14 16:00:06, Michal Hocko wrote: > On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: > > On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko wrote: > > > On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: > > >> On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov > > >>

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-26 Thread Michal Hocko
On Tue 25-11-14 16:00:06, Michal Hocko wrote: On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko mho...@suse.cz wrote: On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Michal Hocko
On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: > On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko wrote: > > On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: > >> On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov > >> wrote: > >> > On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Konstantin Khlebnikov
On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko wrote: > On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: >> On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov >> wrote: >> > On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel wrote: >> >> -BEGIN PGP SIGNED MESSAGE- >> >> Hash: SHA1

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Michal Hocko
On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: > On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov > wrote: > > On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel wrote: > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA1 > >> > >> On 11/20/2014 09:42 AM, Konstantin Khlebnikov

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Michal Hocko
On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov koc...@gmail.com wrote: On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel r...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 09:42 AM, Konstantin

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Konstantin Khlebnikov
On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko mho...@suse.cz wrote: On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov koc...@gmail.com wrote: On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel r...@redhat.com wrote: -BEGIN PGP

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-25 Thread Michal Hocko
On Tue 25-11-14 16:13:16, Konstantin Khlebnikov wrote: On Tue, Nov 25, 2014 at 1:59 PM, Michal Hocko mho...@suse.cz wrote: On Mon 24-11-14 11:09:40, Konstantin Khlebnikov wrote: On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov koc...@gmail.com wrote: On Thu, Nov 20, 2014 at 5:50

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-23 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov wrote: > On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: >> >>> I'm thinking about limitation for reusing anon_vmas which

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-23 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 6:03 PM, Konstantin Khlebnikov koc...@gmail.com wrote: On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel r...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: I'm thinking about limitation for reusing

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Michel Lespinasse
On Thu, Nov 20, 2014 at 3:42 PM, Konstantin Khlebnikov wrote: > On Thu, Nov 20, 2014 at 2:14 AM, Michel Lespinasse wrote: >> On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov >> wrote: >>> On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka wrote: Also from reading

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: > >> I'm thinking about limitation for reusing anon_vmas which might >> increase performance without breaking asymptotic estimation of >>

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: > I'm thinking about limitation for reusing anon_vmas which might > increase performance without breaking asymptotic estimation of > count anon_vma in the worst case. For example this heuristic:

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 2:14 AM, Michel Lespinasse wrote: > On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov > wrote: >> On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka wrote: >>> Also from reading http://lwn.net/Articles/383162/ I understand that >>> correctness >>> also depends on

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 2:14 AM, Michel Lespinasse wal...@google.com wrote: On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka vba...@suse.cz wrote: Also from reading http://lwn.net/Articles/383162/ I understand that

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: I'm thinking about limitation for reusing anon_vmas which might increase performance without breaking asymptotic estimation of count anon_vma in the worst case. For example this heuristic: allow

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Konstantin Khlebnikov
On Thu, Nov 20, 2014 at 5:50 PM, Rik van Riel r...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 09:42 AM, Konstantin Khlebnikov wrote: I'm thinking about limitation for reusing anon_vmas which might increase performance without breaking asymptotic estimation

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-20 Thread Michel Lespinasse
On Thu, Nov 20, 2014 at 3:42 PM, Konstantin Khlebnikov koc...@gmail.com wrote: On Thu, Nov 20, 2014 at 2:14 AM, Michel Lespinasse wal...@google.com wrote: On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Michel Lespinasse
On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov wrote: > On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka wrote: >> Also from reading http://lwn.net/Articles/383162/ I understand that >> correctness >> also depends on the hierarchy and I wonder if there's a danger of >> reintroducing >>

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka wrote: > On 11/19/2014 03:36 PM, Konstantin Khlebnikov wrote: >> On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka wrote: >>> On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Vlastimil Babka
On 11/19/2014 03:36 PM, Konstantin Khlebnikov wrote: > On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka wrote: >> On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: >>> On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov >>> wrote: On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka wrote: > On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: >> On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov >> wrote: >>> On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton >>> wrote: On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka vba...@suse.cz wrote: On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton a...@linux-foundation.org wrote: On Mon,

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Vlastimil Babka
On 11/19/2014 03:36 PM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka vba...@suse.cz wrote: On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Tue, Nov 18, 2014 at 11:19 PM,

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka vba...@suse.cz wrote: On 11/19/2014 03:36 PM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka vba...@suse.cz wrote: On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 1:15 AM, Konstantin

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-19 Thread Michel Lespinasse
On Wed, Nov 19, 2014 at 8:58 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Wed, Nov 19, 2014 at 7:09 PM, Vlastimil Babka vba...@suse.cz wrote: Also from reading http://lwn.net/Articles/383162/ I understand that correctness also depends on the hierarchy and I wonder if there's a danger

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/18/2014 03:19 PM, Andrew Morton wrote: > On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel > wrote: > >> That way people can understand what the code does simply by >> looking at the changelog - no need to go find old linux-kernel >> mailing

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Vlastimil Babka
On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: > On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov > wrote: >> On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton >> wrote: >>> On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel wrote: >>> > Because of the serial forking there does

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov wrote: > On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton > wrote: >> On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel wrote: >> >>> > Because of the serial forking there does indeed end up being an >>> > infinite number of vmas. The initial

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Konstantin Khlebnikov
On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton wrote: > On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel wrote: > >> > Because of the serial forking there does indeed end up being an >> > infinite number of vmas. The initial vma can never be deleted >> > (even though the initial parent process

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Andrew Morton
On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel wrote: > > Because of the serial forking there does indeed end up being an > > infinite number of vmas. The initial vma can never be deleted > > (even though the initial parent process has long since terminated) > > because the initial vma is

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Andrew Morton
On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel r...@redhat.com wrote: Because of the serial forking there does indeed end up being an infinite number of vmas. The initial vma can never be deleted (even though the initial parent process has long since terminated) because the initial vma

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Konstantin Khlebnikov
On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton a...@linux-foundation.org wrote: On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel r...@redhat.com wrote: Because of the serial forking there does indeed end up being an infinite number of vmas. The initial vma can never be deleted (even

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Konstantin Khlebnikov
On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton a...@linux-foundation.org wrote: On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel r...@redhat.com wrote: Because of the serial forking there does indeed end up being

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Vlastimil Babka
On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote: On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov koc...@gmail.com wrote: On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton a...@linux-foundation.org wrote: On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel r...@redhat.com wrote:

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-18 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/18/2014 03:19 PM, Andrew Morton wrote: On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel r...@redhat.com wrote: That way people can understand what the code does simply by looking at the changelog - no need to go find old linux-kernel

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/17/2014 08:41 PM, Daniel Forrest wrote: > On Mon, Nov 17, 2014 at 04:02:12PM -0800, Andrew Morton wrote: >> On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest >> wrote: >> >>> There have been a couple of inquiries about the status of this >>>

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Daniel Forrest
On Mon, Nov 17, 2014 at 04:02:12PM -0800, Andrew Morton wrote: > On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest > wrote: > > > There have been a couple of inquiries about the status of this patch > > over the last few months, so I am going to try pushing it out. > > > > Andrea Arcangeli has

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Andrew Morton
On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest wrote: > There have been a couple of inquiries about the status of this patch > over the last few months, so I am going to try pushing it out. > > Andrea Arcangeli has commented: > > > Agreed. The only thing I don't like about this patch is

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Andrew Morton
On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest dan.forr...@ssec.wisc.edu wrote: There have been a couple of inquiries about the status of this patch over the last few months, so I am going to try pushing it out. Andrea Arcangeli has commented: Agreed. The only thing I don't like

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Daniel Forrest
On Mon, Nov 17, 2014 at 04:02:12PM -0800, Andrew Morton wrote: On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest dan.forr...@ssec.wisc.edu wrote: There have been a couple of inquiries about the status of this patch over the last few months, so I am going to try pushing it out. Andrea

Re: [PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-17 Thread Rik van Riel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/17/2014 08:41 PM, Daniel Forrest wrote: On Mon, Nov 17, 2014 at 04:02:12PM -0800, Andrew Morton wrote: On Fri, 14 Nov 2014 10:30:53 -0600 Daniel Forrest dan.forr...@ssec.wisc.edu wrote: There have been a couple of inquiries about the

[PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-14 Thread Daniel Forrest
There have been a couple of inquiries about the status of this patch over the last few months, so I am going to try pushing it out. Andrea Arcangeli has commented: > Agreed. The only thing I don't like about this patch is the hardcoding > of number 5: could we make it a variable to tweak with

[PATCH] Repeated fork() causes SLAB to grow without bound

2014-11-14 Thread Daniel Forrest
There have been a couple of inquiries about the status of this patch over the last few months, so I am going to try pushing it out. Andrea Arcangeli has commented: Agreed. The only thing I don't like about this patch is the hardcoding of number 5: could we make it a variable to tweak with

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-05 Thread Andrea Arcangeli
On Tue, Jun 04, 2013 at 06:37:25AM -0400, Rik van Riel wrote: > On 06/03/2013 03:50 PM, Daniel Forrest wrote: > > On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: > >> On 08/21/2012 11:20 PM, Michel Lespinasse wrote: > >>> On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-05 Thread Andrea Arcangeli
On Tue, Jun 04, 2013 at 06:37:25AM -0400, Rik van Riel wrote: On 06/03/2013 03:50 PM, Daniel Forrest wrote: On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote:

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-04 Thread Rik van Riel
On 06/03/2013 03:50 PM, Daniel Forrest wrote: On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-04 Thread Rik van Riel
On 06/03/2013 03:50 PM, Daniel Forrest wrote: On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-03 Thread Daniel Forrest
On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: > On 08/21/2012 11:20 PM, Michel Lespinasse wrote: > >On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: > >>Instead of adding an atomic count for page references, we could limit > >>the anon_vma stacking depth. In fork,

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2013-06-03 Thread Daniel Forrest
On Tue, Aug 21, 2012 at 11:29:54PM -0400, Rik van Riel wrote: On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit the anon_vma stacking depth. In fork, we would

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2012-08-21 Thread Rik van Riel
On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit the anon_vma stacking depth. In fork, we would only clone anon_vmas that have a low enough generation count.

[RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2012-08-21 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: > Instead of adding an atomic count for page references, we could limit > the anon_vma stacking depth. In fork, we would only clone anon_vmas > that have a low enough generation count. I think that's not great > (adds a special

[RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2012-08-21 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit the anon_vma stacking depth. In fork, we would only clone anon_vmas that have a low enough generation count. I think that's not great (adds a special case

Re: [RFC PATCH] Re: Repeated fork() causes SLAB to grow without bound

2012-08-21 Thread Rik van Riel
On 08/21/2012 11:20 PM, Michel Lespinasse wrote: On Mon, Aug 20, 2012 at 02:39:26AM -0700, Michel Lespinasse wrote: Instead of adding an atomic count for page references, we could limit the anon_vma stacking depth. In fork, we would only clone anon_vmas that have a low enough generation count.

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 4:53 AM, Michel Lespinasse wrote: > I wonder if it might help to add the child VMA onto the parent's > anon_vma only at the first child COW event. That way it would at least > be possible (with userspace changes) for any forking servers to > separate the areas they want to

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 4:17 AM, Rik van Riel wrote: > Without the anon_vma_chains, we end up scanning every single > one of the child processes (and the parent) for every COWed > page, which can be a real issue when the VM runs into 1000 > such pages, for 1000 child processes. > > Unfortunately,

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Rik van Riel
On 08/20/2012 05:39 AM, Michel Lespinasse wrote: I would still prefer if we could just remove the anon_vma_chain stuff, though. If only we could. That simply replaces a medium issue at fork time, with the potential for a catastrophic issue at page reclaim time, in any workload with heavily

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Andi Kleen
Michel Lespinasse writes: > > I would still prefer if we could just remove the anon_vma_chain stuff, though. Would probably help with the fork locking problems too. We never really recovered from that regression. -Andi -- a...@linux.intel.com -- Speaking for myself only -- To unsubscribe

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 1:00 AM, Hugh Dickins wrote: > On Fri, 17 Aug 2012, Rik van Riel wrote: >> Of course, that leaves the big question: do we want the >> overhead of having the atomic addition and decrement for >> every anonymous memory page, or is it easier to fix this >> issue in userspace?

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Hugh Dickins
On Fri, 17 Aug 2012, Rik van Riel wrote: > On 08/17/2012 08:03 PM, Daniel Forrest wrote: > > > Based on your comments, I came up with the following patch. It boots > > and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know > > if I've overlooked something. I'm not a kernel

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Hugh Dickins
On Fri, 17 Aug 2012, Rik van Riel wrote: On 08/17/2012 08:03 PM, Daniel Forrest wrote: Based on your comments, I came up with the following patch. It boots and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know if I've overlooked something. I'm not a kernel hacker.

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 1:00 AM, Hugh Dickins hu...@google.com wrote: On Fri, 17 Aug 2012, Rik van Riel wrote: Of course, that leaves the big question: do we want the overhead of having the atomic addition and decrement for every anonymous memory page, or is it easier to fix this issue in

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Andi Kleen
Michel Lespinasse wal...@google.com writes: I would still prefer if we could just remove the anon_vma_chain stuff, though. Would probably help with the fork locking problems too. We never really recovered from that regression. -Andi -- a...@linux.intel.com -- Speaking for myself only -- To

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Rik van Riel
On 08/20/2012 05:39 AM, Michel Lespinasse wrote: I would still prefer if we could just remove the anon_vma_chain stuff, though. If only we could. That simply replaces a medium issue at fork time, with the potential for a catastrophic issue at page reclaim time, in any workload with heavily

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 4:17 AM, Rik van Riel r...@redhat.com wrote: Without the anon_vma_chains, we end up scanning every single one of the child processes (and the parent) for every COWed page, which can be a real issue when the VM runs into 1000 such pages, for 1000 child processes.

Re: Repeated fork() causes SLAB to grow without bound

2012-08-20 Thread Michel Lespinasse
On Mon, Aug 20, 2012 at 4:53 AM, Michel Lespinasse wal...@google.com wrote: I wonder if it might help to add the child VMA onto the parent's anon_vma only at the first child COW event. That way it would at least be possible (with userspace changes) for any forking servers to separate the areas

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Rik van Riel
On 08/18/2012 12:07 AM, Daniel Forrest wrote: I was being careful since I wasn't certain about the locking. Does the test need to be protected by "lock_anon_vma_root"? That's why I chose the overhead of the possible wasted "anon_vma_chain_alloc". The function anon_vma_clone is being called

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Daniel Forrest
On Fri, Aug 17, 2012 at 11:46:18PM -0400, Rik van Riel wrote: > On 08/17/2012 08:03 PM, Daniel Forrest wrote: > > >Based on your comments, I came up with the following patch. It boots > >and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know > >if I've overlooked something. I'm

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Rik van Riel
On 08/17/2012 08:03 PM, Daniel Forrest wrote: Based on your comments, I came up with the following patch. It boots and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know if I've overlooked something. I'm not a kernel hacker. The patch looks reasonable to me. There is one

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Daniel Forrest
On Thu, Aug 16, 2012 at 02:58:45PM -0400, Rik van Riel wrote: > Oh dear. > > Basically, what happens is that at fork time, a new > "level" is created for the anon_vma hierarchy. This > works great for normal forking daemons, since the > parent process just keeps running, and forking off >

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Daniel Forrest
On Thu, Aug 16, 2012 at 02:58:45PM -0400, Rik van Riel wrote: Oh dear. Basically, what happens is that at fork time, a new level is created for the anon_vma hierarchy. This works great for normal forking daemons, since the parent process just keeps running, and forking off children.

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Rik van Riel
On 08/17/2012 08:03 PM, Daniel Forrest wrote: Based on your comments, I came up with the following patch. It boots and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know if I've overlooked something. I'm not a kernel hacker. The patch looks reasonable to me. There is one

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Daniel Forrest
On Fri, Aug 17, 2012 at 11:46:18PM -0400, Rik van Riel wrote: On 08/17/2012 08:03 PM, Daniel Forrest wrote: Based on your comments, I came up with the following patch. It boots and the anon_vma/anon_vma_chain SLAB usage is stable, but I don't know if I've overlooked something. I'm not a

Re: Repeated fork() causes SLAB to grow without bound

2012-08-17 Thread Rik van Riel
On 08/18/2012 12:07 AM, Daniel Forrest wrote: I was being careful since I wasn't certain about the locking. Does the test need to be protected by lock_anon_vma_root? That's why I chose the overhead of the possible wasted anon_vma_chain_alloc. The function anon_vma_clone is being called from

Re: Repeated fork() causes SLAB to grow without bound

2012-08-16 Thread Rik van Riel
On 08/15/2012 10:46 PM, Daniel Forrest wrote: I'm hoping someone has seen this before... I've been trying to track down a performance problem with Linux 3.0.4. The symptom is system-mode load increasing over time while user-mode load remains constant while running a data ingest/processing

Re: Repeated fork() causes SLAB to grow without bound

2012-08-16 Thread Rik van Riel
On 08/15/2012 10:46 PM, Daniel Forrest wrote: I'm hoping someone has seen this before... I've been trying to track down a performance problem with Linux 3.0.4. The symptom is system-mode load increasing over time while user-mode load remains constant while running a data ingest/processing

Repeated fork() causes SLAB to grow without bound

2012-08-15 Thread Daniel Forrest
I'm hoping someone has seen this before... I've been trying to track down a performance problem with Linux 3.0.4. The symptom is system-mode load increasing over time while user-mode load remains constant while running a data ingest/processing program. Looking at /proc/meminfo I noticed

Repeated fork() causes SLAB to grow without bound

2012-08-15 Thread Daniel Forrest
I'm hoping someone has seen this before... I've been trying to track down a performance problem with Linux 3.0.4. The symptom is system-mode load increasing over time while user-mode load remains constant while running a data ingest/processing program. Looking at /proc/meminfo I noticed