Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-23 Thread Michal Hocko
On Sat 17-06-17 22:30:31, Tetsuo Handa wrote: > Michal Hocko wrote: [...] > > What does this dissassemble to on your kernel? Care to post addr2line? > [...] > The __oom_reap_task_mm+0xa1/0x160 is __oom_reap_task_mm at mm/oom_kill.c:472 > which is "struct vm_area_struct *vma;" line in __oom_reap_ta

Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-17 Thread Tetsuo Handa
Michal Hocko wrote: > On Fri 16-06-17 23:26:20, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Fri 16-06-17 19:27:19, Tetsuo Handa wrote: > > > > Michal Hocko wrote: > > > > > On Fri 16-06-17 09:54:34, Tetsuo Handa wrote: > > > > > [...] > > > > > > And the patch you proposed is broken. > > >

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Michal Hocko
On Fri 16-06-17 21:22:20, Tetsuo Handa wrote: > Michal Hocko wrote: > > OK, could you play with the patch/idea suggested in > > http://lkml.kernel.org/r/20170615122031.gl1...@dhcp22.suse.cz? > > I think we don't need to worry about mmap_sem dependency inside __mmput(). > Since the OOM killer check

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Tetsuo Handa
Michal Hocko wrote: > OK, could you play with the patch/idea suggested in > http://lkml.kernel.org/r/20170615122031.gl1...@dhcp22.suse.cz? I think we don't need to worry about mmap_sem dependency inside __mmput(). Since the OOM killer checks for !MMF_OOM_SKIP mm rather than TIF_MEMDIE thread, we c

Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Michal Hocko
On Fri 16-06-17 19:27:19, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Fri 16-06-17 09:54:34, Tetsuo Handa wrote: > > [...] > > > And the patch you proposed is broken. > > > > Thanks for your testing! > > > > > -- > > > [ 161.846202] Out of memory: Kill process 6331 (a.out) score 99

Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Tetsuo Handa
Michal Hocko wrote: > On Fri 16-06-17 09:54:34, Tetsuo Handa wrote: > [...] > > And the patch you proposed is broken. > > Thanks for your testing! > > > -- > > [ 161.846202] Out of memory: Kill process 6331 (a.out) score 999 or > > sacrifice child > > [ 161.850327] Killed process 6331

Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Michal Hocko
On Fri 16-06-17 09:54:34, Tetsuo Handa wrote: [...] > And the patch you proposed is broken. Thanks for your testing! > -- > [ 161.846202] Out of memory: Kill process 6331 (a.out) score 999 or > sacrifice child > [ 161.850327] Killed process 6331 (a.out) total-vm:4172kB, anon-rss:84kB,

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-16 Thread Michal Hocko
On Thu 15-06-17 15:42:23, David Rientjes wrote: > On Fri, 16 Jun 2017, Michal Hocko wrote: > > > I am sorry but I have really hard to make the oom reaper a reliable way > > to stop all the potential oom lockups go away. I do not want to > > reintroduce another potential lockup now. > > Please sho

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Tetsuo Handa wrote: > and clarify in your patch that there is no possibility > of waiting for direct/indirect memory allocation inside free_pgtables(), > in addition to fixing the bug above. Oops, this part was wrong, for __oom_reap_task_mm() will give up after waiting for one second because down_

Re: Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Michal Hocko wrote: > On Thu 15-06-17 15:03:17, David Rientjes wrote: > > On Thu, 15 Jun 2017, Michal Hocko wrote: > > > > > > Yes, quite a bit in testing. > > > > > > > > One oom kill shows the system to be oom: > > > > > > > > [22999.488705] Node 0 Normal free:90484kB min:90500kB ... > > > > [

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread David Rientjes
On Fri, 16 Jun 2017, Michal Hocko wrote: > I am sorry but I have really hard to make the oom reaper a reliable way > to stop all the potential oom lockups go away. I do not want to > reintroduce another potential lockup now. Please show where this "potential lockup" ever existed in a bug report o

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 15:03:17, David Rientjes wrote: > On Thu, 15 Jun 2017, Michal Hocko wrote: > > > > Yes, quite a bit in testing. > > > > > > One oom kill shows the system to be oom: > > > > > > [22999.488705] Node 0 Normal free:90484kB min:90500kB ... > > > [22999.488711] Node 1 Normal free:91536

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread David Rientjes
On Thu, 15 Jun 2017, Michal Hocko wrote: > > Yes, quite a bit in testing. > > > > One oom kill shows the system to be oom: > > > > [22999.488705] Node 0 Normal free:90484kB min:90500kB ... > > [22999.488711] Node 1 Normal free:91536kB min:91948kB ... > > > > followed up by one or more unnecessa

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Michal Hocko wrote: > On Thu 15-06-17 22:01:33, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Thu 15-06-17 14:03:35, Michal Hocko wrote: > > > > On Thu 15-06-17 20:32:39, Tetsuo Handa wrote: > > > > > @@ -556,25 +553,21 @@ static void oom_reap_task(struct task_struct > > > > > *tsk) > > > >

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 14:26:26, David Rientjes wrote: > On Thu, 15 Jun 2017, Michal Hocko wrote: > > > > If mm->mm_users is not incremented because it is already zero by the oom > > > reaper, meaning the final refcount has been dropped, do not set > > > MMF_OOM_SKIP prematurely. > > > > > > __mmput() m

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread David Rientjes
On Thu, 15 Jun 2017, Tetsuo Handa wrote: > David is trying to avoid setting MMF_OOM_SKIP when the OOM reaper found that > mm->users == 0. Yes, because MMF_OOM_SKIP enables the oom killer to select another process to kill and will do so without the original victim's mm being able to undergo exit

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread David Rientjes
On Thu, 15 Jun 2017, Michal Hocko wrote: > > If mm->mm_users is not incremented because it is already zero by the oom > > reaper, meaning the final refcount has been dropped, do not set > > MMF_OOM_SKIP prematurely. > > > > __mmput() may not have had a chance to do exit_mmap() yet, so memory from

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 22:01:33, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Thu 15-06-17 14:03:35, Michal Hocko wrote: > > > On Thu 15-06-17 20:32:39, Tetsuo Handa wrote: > > > > @@ -556,25 +553,21 @@ static void oom_reap_task(struct task_struct *tsk) > > > > struct mm_struct *mm = tsk->sign

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Michal Hocko wrote: > On Thu 15-06-17 14:03:35, Michal Hocko wrote: > > On Thu 15-06-17 20:32:39, Tetsuo Handa wrote: > > > @@ -556,25 +553,21 @@ static void oom_reap_task(struct task_struct *tsk) > > > struct mm_struct *mm = tsk->signal->oom_mm; > > > > > > /* Retry the down_read_trylock(mma

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 13:01:19, Michal Hocko wrote: > On Thu 15-06-17 19:53:24, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Wed 14-06-17 16:43:03, David Rientjes wrote: > > > > If mm->mm_users is not incremented because it is already zero by the oom > > > > reaper, meaning the final refcount has

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 14:03:35, Michal Hocko wrote: > On Thu 15-06-17 20:32:39, Tetsuo Handa wrote: > > Michal Hocko wrote: > [...] > > > An alternative would be to allow reaping and exit_mmap race. The unmap > > > part should just work I guess. We just have to be careful to not race > > > with free_pgta

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 20:32:39, Tetsuo Handa wrote: > Michal Hocko wrote: [...] > > An alternative would be to allow reaping and exit_mmap race. The unmap > > part should just work I guess. We just have to be careful to not race > > with free_pgtables and that shouldn't be too hard to implement (e.g. > >

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Michal Hocko wrote: > On Thu 15-06-17 19:53:24, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Wed 14-06-17 16:43:03, David Rientjes wrote: > > > > If mm->mm_users is not incremented because it is already zero by the oom > > > > reaper, meaning the final refcount has been dropped, do not set

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Thu 15-06-17 19:53:24, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Wed 14-06-17 16:43:03, David Rientjes wrote: > > > If mm->mm_users is not incremented because it is already zero by the oom > > > reaper, meaning the final refcount has been dropped, do not set > > > MMF_OOM_SKIP prematurely

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Tetsuo Handa
Michal Hocko wrote: > On Wed 14-06-17 16:43:03, David Rientjes wrote: > > If mm->mm_users is not incremented because it is already zero by the oom > > reaper, meaning the final refcount has been dropped, do not set > > MMF_OOM_SKIP prematurely. > > > > __mmput() may not have had a chance to do exi

Re: [patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-15 Thread Michal Hocko
On Wed 14-06-17 16:43:03, David Rientjes wrote: > If mm->mm_users is not incremented because it is already zero by the oom > reaper, meaning the final refcount has been dropped, do not set > MMF_OOM_SKIP prematurely. > > __mmput() may not have had a chance to do exit_mmap() yet, so memory from > a

[patch] mm, oom: prevent additional oom kills before memory is freed

2017-06-14 Thread David Rientjes
If mm->mm_users is not incremented because it is already zero by the oom reaper, meaning the final refcount has been dropped, do not set MMF_OOM_SKIP prematurely. __mmput() may not have had a chance to do exit_mmap() yet, so memory from a previous oom victim is still mapped. __mput() naturally re