Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Mon, Apr 01, 2019 at 10:12:01PM +0200, Michal Hocko wrote: > On Fri 29-03-19 16:16:38, Catalin Marinas wrote: > > On Fri, Mar 29, 2019 at 01:02:37PM +0100, Michal Hocko wrote: > > > On Thu 28-03-19 14:59:17, Catalin Marinas wrote: > > > [...] > > > > >From 09eba8f0235eb16409931e6aad77a45a12bedc8

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Fri 29-03-19 16:16:38, Catalin Marinas wrote: > On Fri, Mar 29, 2019 at 01:02:37PM +0100, Michal Hocko wrote: > > On Thu 28-03-19 14:59:17, Catalin Marinas wrote: > > [...] > > > >From 09eba8f0235eb16409931e6aad77a45a12bedc82 Mon Sep 17 00:00:00 2001 > > > From: Catalin Marinas > > > Date: Thu,

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Fri, Mar 29, 2019 at 01:02:37PM +0100, Michal Hocko wrote: > On Thu 28-03-19 14:59:17, Catalin Marinas wrote: > [...] > > >From 09eba8f0235eb16409931e6aad77a45a12bedc82 Mon Sep 17 00:00:00 2001 > > From: Catalin Marinas > > Date: Thu, 28 Mar 2019 13:26:07 + > > Subject: [PATCH] mm: kmemleak

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Thu 28-03-19 14:59:17, Catalin Marinas wrote: [...] > >From 09eba8f0235eb16409931e6aad77a45a12bedc82 Mon Sep 17 00:00:00 2001 > From: Catalin Marinas > Date: Thu, 28 Mar 2019 13:26:07 + > Subject: [PATCH] mm: kmemleak: Use mempool allocations for kmemleak objects > > This patch adds mempoo

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Thu, 2019-03-28 at 15:05 +, Catalin Marinas wrote: > > It takes 2 runs of LTP oom01 tests to disable kmemleak. > > What configuration are you using (number of CPUs, RAM)? I tried this on > an arm64 guest under kvm with 4 CPUs and 512MB of RAM, together with > fault injection on kmemleak_obj

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Thu, Mar 28, 2019 at 01:50:29PM +0200, Pekka Enberg wrote: > On 27/03/2019 2.59, Qian Cai wrote: > > > > Unless there is a brave soul to reimplement the kmemleak to embed it's > > > > metadata into the tracked memory itself in a foreseeable future, this > > > > provides a good balance between en

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Thu, 2019-03-28 at 14:59 +, Catalin Marinas wrote: 2 > +/* minimum memory pool sizes */ > +#define MIN_OBJECT_POOL (NR_CPUS * 4) > +#define MIN_SCAN_AREA_POOL (NR_CPUS * 1) I am thinking about making those are tunable, so people could have a big pool depends on

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed, Mar 27, 2019 at 02:02:27PM -0400, Qian Cai wrote: > On 3/27/19 1:29 PM, Catalin Marinas wrote: > > From dc4194539f8191bb754901cea74c86e7960886f8 Mon Sep 17 00:00:00 2001 > > From: Catalin Marinas > > Date: Wed, 27 Mar 2019 17:20:57 + > > Subject: [PATCH] mm: kmemleak: Add an emergency

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed, Mar 27, 2019 at 11:21:58AM -0700, Matthew Wilcox wrote: > On Wed, Mar 27, 2019 at 05:29:57PM +, Catalin Marinas wrote: > > On Wed, Mar 27, 2019 at 09:44:32AM +0100, Michal Hocko wrote: > > > As long as there is an implicit __GFP_NOFAIL then kmemleak is simply > > > broken no matter what

Re: [PATCH v4] kmemleak: survive in a low-memory situation

Hi Catalin, On 27/03/2019 2.59, Qian Cai wrote: Unless there is a brave soul to reimplement the kmemleak to embed it's metadata into the tracked memory itself in a foreseeable future, this provides a good balance between enabling kmemleak in a low-memory situation and not introducing too much ha

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Thu, Mar 28, 2019 at 08:05:31AM +0200, Pekka Enberg wrote: > On 27/03/2019 2.59, Qian Cai wrote: > > Unless there is a brave soul to reimplement the kmemleak to embed it's > > metadata into the tracked memory itself in a foreseeable future, this > > provides a good balance between enabling kmeml

Re: [PATCH v4] kmemleak: survive in a low-memory situation

Hi, On 27/03/2019 2.59, Qian Cai wrote: Unless there is a brave soul to reimplement the kmemleak to embed it's metadata into the tracked memory itself in a foreseeable future, this provides a good balance between enabling kmemleak in a low-memory situation and not introducing too much hackiness

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On 3/27/19 1:29 PM, Catalin Marinas wrote: > From dc4194539f8191bb754901cea74c86e7960886f8 Mon Sep 17 00:00:00 2001 > From: Catalin Marinas > Date: Wed, 27 Mar 2019 17:20:57 + > Subject: [PATCH] mm: kmemleak: Add an emergency allocation pool for kmemleak > objects > > This patch adds an emer

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed 27-03-19 17:29:57, Catalin Marinas wrote: [...] > Quick attempt below and it needs some more testing (pretty random pick > of the EMERGENCY_POOL_SIZE value). Also, with __GFP_NOFAIL removed, are > the other flags safe or we should trim them further? I would be still careful about __GFP_NORE

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed, Mar 27, 2019 at 05:29:57PM +, Catalin Marinas wrote: > On Wed, Mar 27, 2019 at 09:44:32AM +0100, Michal Hocko wrote: > > As long as there is an implicit __GFP_NOFAIL then kmemleak is simply > > broken no matter what other gfp flags you play with. Has anybody looked > > at some sort of p

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed, Mar 27, 2019 at 09:44:32AM +0100, Michal Hocko wrote: > On Tue 26-03-19 20:59:48, Qian Cai wrote: > [...] > > Unless there is a brave soul to reimplement the kmemleak to embed it's > > metadata into the tracked memory itself in a foreseeable future, Revisiting the kmemleak memory scanning

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed 27-03-19 09:05:31, Qian Cai wrote: > On 3/27/19 7:44 AM, Michal Hocko wrote> What? Normal spin lock implementation > doesn't disable interrupts. So > > either I misunderstand what you are saying or you seem to be confused. > > the thing is that in_atomic relies on preempt_count to work prope

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On 3/27/19 7:44 AM, Michal Hocko wrote> What? Normal spin lock implementation doesn't disable interrupts. So > either I misunderstand what you are saying or you seem to be confused. > the thing is that in_atomic relies on preempt_count to work properly and > if you have CONFIG_PREEMPT_COUNT=n then

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Wed 27-03-19 07:34:32, Qian Cai wrote: > On 3/27/19 4:44 AM, Michal Hocko wrote: > >> diff --git a/mm/kmemleak.c b/mm/kmemleak.c > >> index a2d894d3de07..7f4545ab1f84 100644 > >> --- a/mm/kmemleak.c > >> +++ b/mm/kmemleak.c > >> @@ -580,7 +580,16 @@ static struct kmemleak_object *create_object(u

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On 3/27/19 4:44 AM, Michal Hocko wrote: >> diff --git a/mm/kmemleak.c b/mm/kmemleak.c >> index a2d894d3de07..7f4545ab1f84 100644 >> --- a/mm/kmemleak.c >> +++ b/mm/kmemleak.c >> @@ -580,7 +580,16 @@ static struct kmemleak_object *create_object(unsigned >> long ptr, size_t size, >> struct rb_n

Re: [PATCH v4] kmemleak: survive in a low-memory situation

On Tue 26-03-19 20:59:48, Qian Cai wrote: [...] > Unless there is a brave soul to reimplement the kmemleak to embed it's > metadata into the tracked memory itself in a foreseeable future, this > provides a good balance between enabling kmemleak in a low-memory > situation and not introducing too mu

[PATCH v4] kmemleak: survive in a low-memory situation

Kmemleak could quickly fail to allocate an object structure and then disable itself below in a low-memory situation. For example, running a mmap() workload triggering swapping and OOM. This is especially problematic for running things like LTP testsuite where one OOM test case would disable the who