Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, 26 Jul 2017, Andrea Arcangeli wrote: > From 3d9001490ee1a71f39c7bfaf19e96821f9d3ff16 Mon Sep 17 00:00:00 2001 > From: Andrea Arcangeli > Date: Tue, 25 Jul 2017 20:02:27 +0200 > Subject: [PATCH 1/1] mm: oom: let oom_reap_task and exit_mmap to run > concurrently > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, 26 Jul 2017, Andrea Arcangeli wrote: > From 3d9001490ee1a71f39c7bfaf19e96821f9d3ff16 Mon Sep 17 00:00:00 2001 > From: Andrea Arcangeli > Date: Tue, 25 Jul 2017 20:02:27 +0200 > Subject: [PATCH 1/1] mm: oom: let oom_reap_task and exit_mmap to run > concurrently > > This is purely

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 10-08-17 20:51:38, Michal Hocko wrote: [...] > OK, let's agree to disagree. As I've said I like when the critical > section is explicit and we _know_ what it protects. In this case it is > clear that we have to protect from the page tables tear down and the > vma destructions. But as I've

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 10-08-17 20:51:38, Michal Hocko wrote: [...] > OK, let's agree to disagree. As I've said I like when the critical > section is explicit and we _know_ what it protects. In this case it is > clear that we have to protect from the page tables tear down and the > vma destructions. But as I've

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 10-08-17 20:05:54, Andrea Arcangeli wrote: > On Thu, Aug 10, 2017 at 10:16:32AM +0200, Michal Hocko wrote: > > Andrea has proposed and alternative solution [4] which should be > > equivalent functionally similar to {ksm,khugepaged}_exit. I have to > > confess I really don't like that

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 10-08-17 20:05:54, Andrea Arcangeli wrote: > On Thu, Aug 10, 2017 at 10:16:32AM +0200, Michal Hocko wrote: > > Andrea has proposed and alternative solution [4] which should be > > equivalent functionally similar to {ksm,khugepaged}_exit. I have to > > confess I really don't like that

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, Aug 10, 2017 at 10:16:32AM +0200, Michal Hocko wrote: > Andrea has proposed and alternative solution [4] which should be > equivalent functionally similar to {ksm,khugepaged}_exit. I have to > confess I really don't like that approach but I can live with it if > that is a preferred way (to

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, Aug 10, 2017 at 10:16:32AM +0200, Michal Hocko wrote: > Andrea has proposed and alternative solution [4] which should be > equivalent functionally similar to {ksm,khugepaged}_exit. I have to > confess I really don't like that approach but I can live with it if > that is a preferred way (to

[PATCH] mm, oom: allow oom reaper to race with exit_mmap

From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too early when mm->mm_users dropped down to 0. The race is as follows

[PATCH] mm, oom: allow oom reaper to race with exit_mmap

From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too early when mm->mm_users dropped down to 0. The race is as follows oom_reap_task

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 27-07-17 16:55:59, Andrea Arcangeli wrote: > On Thu, Jul 27, 2017 at 08:50:24AM +0200, Michal Hocko wrote: > > Yes this will work and it won't depend on the oom_lock. But isn't it > > just more ugly than simply doing > > > > if (tsk_is_oom_victim) { > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 27-07-17 16:55:59, Andrea Arcangeli wrote: > On Thu, Jul 27, 2017 at 08:50:24AM +0200, Michal Hocko wrote: > > Yes this will work and it won't depend on the oom_lock. But isn't it > > just more ugly than simply doing > > > > if (tsk_is_oom_victim) { > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, Jul 27, 2017 at 08:50:24AM +0200, Michal Hocko wrote: > Yes this will work and it won't depend on the oom_lock. But isn't it > just more ugly than simply doing > > if (tsk_is_oom_victim) { > down_write(>mmap_sem); > locked = true; > } >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, Jul 27, 2017 at 08:50:24AM +0200, Michal Hocko wrote: > Yes this will work and it won't depend on the oom_lock. But isn't it > just more ugly than simply doing > > if (tsk_is_oom_victim) { > down_write(>mmap_sem); > locked = true; > } >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 27-07-17 13:59:09, Manish Jaggi wrote: [...] > With 4.11.6 I was getting random kernel panics (Out of memory - No process > left to kill), > when running LTP oom01 /oom02 ltp tests on our arm64 hardware with ~256G > memory and high core count. > The issue experienced was as follows >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 27-07-17 13:59:09, Manish Jaggi wrote: [...] > With 4.11.6 I was getting random kernel panics (Out of memory - No process > left to kill), > when running LTP oom01 /oom02 ltp tests on our arm64 hardware with ~256G > memory and high core count. > The issue experienced was as follows >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Hi Michal, On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Hi Michal, On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too early when

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

if (locked) down_up(>mmap_sem); in general I do not like empty locked sections much, to be honest. Now with the conditional locking my patch looks as follows. It should be pretty much equivalent to your patch. Would that be acceptable for you or do you think there is a strong reaso

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

if (locked) down_up(>mmap_sem); in general I do not like empty locked sections much, to be honest. Now with the conditional locking my patch looks as follows. It should be pretty much equivalent to your patch. Would that be acceptable for you or do you think there is a stron

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed 26-07-17 18:39:28, Andrea Arcangeli wrote: > On Wed, Jul 26, 2017 at 07:45:33AM +0200, Michal Hocko wrote: > > Yes, exit_aio is the only blocking call I know of currently. But I would > > like this to be as robust as possible and so I do not want to rely on > > the current implementation.

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed 26-07-17 18:39:28, Andrea Arcangeli wrote: > On Wed, Jul 26, 2017 at 07:45:33AM +0200, Michal Hocko wrote: > > Yes, exit_aio is the only blocking call I know of currently. But I would > > like this to be as robust as possible and so I do not want to rely on > > the current implementation.

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 06:29:12PM +0200, Andrea Arcangeli wrote: > From 3d9001490ee1a71f39c7bfaf19e96821f9d3ff16 Mon Sep 17 00:00:00 2001 > From: Andrea Arcangeli > Date: Tue, 25 Jul 2017 20:02:27 +0200 > Subject: [PATCH 1/1] mm: oom: let oom_reap_task and exit_mmap to run >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 06:29:12PM +0200, Andrea Arcangeli wrote: > From 3d9001490ee1a71f39c7bfaf19e96821f9d3ff16 Mon Sep 17 00:00:00 2001 > From: Andrea Arcangeli > Date: Tue, 25 Jul 2017 20:02:27 +0200 > Subject: [PATCH 1/1] mm: oom: let oom_reap_task and exit_mmap to run > concurrently This

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 07:45:33AM +0200, Michal Hocko wrote: > Yes, exit_aio is the only blocking call I know of currently. But I would > like this to be as robust as possible and so I do not want to rely on > the current implementation. This can change in future and I can > guarantee that nobody

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 07:45:33AM +0200, Michal Hocko wrote: > Yes, exit_aio is the only blocking call I know of currently. But I would > like this to be as robust as possible and so I do not want to rely on > the current implementation. This can change in future and I can > guarantee that nobody

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 07:45:57AM +0200, Michal Hocko wrote: > On Tue 25-07-17 21:19:52, Andrea Arcangeli wrote: > > On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > > > - down_write(>mmap_sem); > > > + if (tsk_is_oom_victim(current)) > > > + down_write(>mmap_sem); > > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, Jul 26, 2017 at 07:45:57AM +0200, Michal Hocko wrote: > On Tue 25-07-17 21:19:52, Andrea Arcangeli wrote: > > On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > > > - down_write(>mmap_sem); > > > + if (tsk_is_oom_victim(current)) > > > + down_write(>mmap_sem); > > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 21:19:52, Andrea Arcangeli wrote: > On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > > - down_write(>mmap_sem); > > + if (tsk_is_oom_victim(current)) > > + down_write(>mmap_sem); > > free_pgtables(, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING); >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 21:19:52, Andrea Arcangeli wrote: > On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > > - down_write(>mmap_sem); > > + if (tsk_is_oom_victim(current)) > > + down_write(>mmap_sem); > > free_pgtables(, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING); >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 20:26:19, Andrea Arcangeli wrote: > On Tue, Jul 25, 2017 at 05:45:14PM +0200, Michal Hocko wrote: > > That problem is real though as reported by David. > > I'm not against fixing it, I just think it's not a major concern, and > the solution doesn't seem optimal as measured by

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 20:26:19, Andrea Arcangeli wrote: > On Tue, Jul 25, 2017 at 05:45:14PM +0200, Michal Hocko wrote: > > That problem is real though as reported by David. > > I'm not against fixing it, I just think it's not a major concern, and > the solution doesn't seem optimal as measured by

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > - down_write(>mmap_sem); > + if (tsk_is_oom_victim(current)) > + down_write(>mmap_sem); > free_pgtables(, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING); > tlb_finish_mmu(, 0, -1); > > @@ -3012,7

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 06:04:00PM +0200, Michal Hocko wrote: > - down_write(>mmap_sem); > + if (tsk_is_oom_victim(current)) > + down_write(>mmap_sem); > free_pgtables(, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING); > tlb_finish_mmu(, 0, -1); > > @@ -3012,7

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 05:45:14PM +0200, Michal Hocko wrote: > That problem is real though as reported by David. I'm not against fixing it, I just think it's not a major concern, and the solution doesn't seem optimal as measured by Kirill. I'm just skeptical it's the best to solve that tiny

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 05:45:14PM +0200, Michal Hocko wrote: > That problem is real though as reported by David. I'm not against fixing it, I just think it's not a major concern, and the solution doesn't seem optimal as measured by Kirill. I'm just skeptical it's the best to solve that tiny

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:31:10, Kirill A. Shutemov wrote: > On Tue, Jul 25, 2017 at 05:23:00PM +0200, Michal Hocko wrote: > > what is stdev? > > Updated tables: > > 3 runs before the patch: >Min. 1st Qu. MedianMean 3rd Qu.Max. Stdev > 177200 205000 212900 217800 223700 2377000

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:31:10, Kirill A. Shutemov wrote: > On Tue, Jul 25, 2017 at 05:23:00PM +0200, Michal Hocko wrote: > > what is stdev? > > Updated tables: > > 3 runs before the patch: >Min. 1st Qu. MedianMean 3rd Qu.Max. Stdev > 177200 205000 212900 217800 223700 2377000

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 17:26:39, Andrea Arcangeli wrote: > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > David has noticed that the oom killer might kill additional tasks while > > the exiting oom victim hasn't terminated yet because the

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 17:26:39, Andrea Arcangeli wrote: > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > David has noticed that the oom killer might kill additional tasks while > > the exiting oom victim hasn't terminated yet because the oom_reaper marks >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 05:23:00PM +0200, Michal Hocko wrote: > what is stdev? Updated tables: 3 runs before the patch: Min. 1st Qu. MedianMean 3rd Qu.Max. Stdev 177200 205000 212900 217800 223700 2377000 32868 172400 201700 209700 214300 220600 1343000 31191 175700

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 05:23:00PM +0200, Michal Hocko wrote: > what is stdev? Updated tables: 3 runs before the patch: Min. 1st Qu. MedianMean 3rd Qu.Max. Stdev 177200 205000 212900 217800 223700 2377000 32868 172400 201700 209700 214300 220600 1343000 31191 175700

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the exiting oom victim hasn't terminated yet because the oom_reaper marks > the curent victim MMF_OOM_SKIP too early

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the exiting oom victim hasn't terminated yet because the oom_reaper marks > the curent victim MMF_OOM_SKIP too early when mm->mm_users

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:17:54, Kirill A. Shutemov wrote: > > before the patch > > min: 306300.00 max: 6731916.00 avg: 437962.07 std: 92898.30 nr: 10 > > > > after > > min: 303196.00 max: 5728080.00 avg: 436081.87 std: 96165.98 nr: 10 > > > > The results are well withing noise as I would

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:17:54, Kirill A. Shutemov wrote: > > before the patch > > min: 306300.00 max: 6731916.00 avg: 437962.07 std: 92898.30 nr: 10 > > > > after > > min: 303196.00 max: 5728080.00 avg: 436081.87 std: 96165.98 nr: 10 > > > > The results are well withing noise as I would

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 04:26:26PM +0200, Michal Hocko wrote: > On Mon 24-07-17 18:11:46, Michal Hocko wrote: > > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > > [...] > > > > What kind of scalability implication you have

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 04:26:26PM +0200, Michal Hocko wrote: > On Mon 24-07-17 18:11:46, Michal Hocko wrote: > > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > > [...] > > > > What kind of scalability implication you have

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:07:19, Kirill A. Shutemov wrote: > On Tue, Jul 25, 2017 at 04:26:17PM +0200, Michal Hocko wrote: [...] > > Thanks for retesting Kirill. Are those numbers stable over runs? E.g. > > the run without the patch has ~3% variance while the one with the patch > > has it smaller. This

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 18:07:19, Kirill A. Shutemov wrote: > On Tue, Jul 25, 2017 at 04:26:17PM +0200, Michal Hocko wrote: [...] > > Thanks for retesting Kirill. Are those numbers stable over runs? E.g. > > the run without the patch has ~3% variance while the one with the patch > > has it smaller. This

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 04:26:17PM +0200, Michal Hocko wrote: > On Tue 25-07-17 17:17:23, Kirill A. Shutemov wrote: > [...] > > Below are numbers for the same test case, but from bigger machine (48 > > threads, 64GiB of RAM). > > > > v4.13-rc2: > > > > Performance counter stats for './a.sh

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, Jul 25, 2017 at 04:26:17PM +0200, Michal Hocko wrote: > On Tue 25-07-17 17:17:23, Kirill A. Shutemov wrote: > [...] > > Below are numbers for the same test case, but from bigger machine (48 > > threads, 64GiB of RAM). > > > > v4.13-rc2: > > > > Performance counter stats for './a.sh

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 17:17:23, Kirill A. Shutemov wrote: [...] > Below are numbers for the same test case, but from bigger machine (48 > threads, 64GiB of RAM). > > v4.13-rc2: > > Performance counter stats for './a.sh 10' (5 runs): > > 159857.233790 task-clock:u (msec) #

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 17:17:23, Kirill A. Shutemov wrote: [...] > Below are numbers for the same test case, but from bigger machine (48 > threads, 64GiB of RAM). > > v4.13-rc2: > > Performance counter stats for './a.sh 10' (5 runs): > > 159857.233790 task-clock:u (msec) #

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 18:11:46, Michal Hocko wrote: > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > [...] > > > What kind of scalability implication you have in mind? There is > > > basically a zero contention on the mmap_sem that

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 18:11:46, Michal Hocko wrote: > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > [...] > > > What kind of scalability implication you have in mind? There is > > > basically a zero contention on the mmap_sem that

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 06:11:47PM +0200, Michal Hocko wrote: > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > [...] > > > What kind of scalability implication you have in mind? There is > > > basically a zero contention on

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 06:11:47PM +0200, Michal Hocko wrote: > On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > [...] > > > What kind of scalability implication you have in mind? There is > > > basically a zero contention on

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 00:42:05, kbuild test robot wrote: > Hi Michal, > > [auto build test ERROR on mmotm/master] > [also build test ERROR on v4.13-rc2 next-20170724] > [if your patch is applied to the wrong git tree, please drop us a note to > help improve the system] > > url: >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 25-07-17 00:42:05, kbuild test robot wrote: > Hi Michal, > > [auto build test ERROR on mmotm/master] > [also build test ERROR on v4.13-rc2 next-20170724] > [if your patch is applied to the wrong git tree, please drop us a note to > help improve the system] > > url: >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Hi Michal, [auto build test ERROR on mmotm/master] [also build test ERROR on v4.13-rc2 next-20170724] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Hi Michal, [auto build test ERROR on mmotm/master] [also build test ERROR on v4.13-rc2 next-20170724] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: [...] > > What kind of scalability implication you have in mind? There is > > basically a zero contention on the mmap_sem that late in the exit path > > so this should be pretty much

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 17:51:42, Kirill A. Shutemov wrote: > On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: [...] > > What kind of scalability implication you have in mind? There is > > basically a zero contention on the mmap_sem that late in the exit path > > so this should be pretty much

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Dohh, the full conflict resolution didn't make it into the commit. The full patch is below --- >From be69b355a56649167dce901d24c2296ef3a3f7ec Mon Sep 17 00:00:00 2001 From: Michal Hocko <mho...@suse.com> Date: Mon, 24 Jul 2017 09:23:32 +0200 Subject: [PATCH] mm, oom: allow oom reape

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

Dohh, the full conflict resolution didn't make it into the commit. The full patch is below --- >From be69b355a56649167dce901d24c2296ef3a3f7ec Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Mon, 24 Jul 2017 09:23:32 +0200 Subject: [PATCH] mm, oom: allow oom reaper to race with exit_mmap Da

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > On Mon 24-07-17 17:00:08, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > David has noticed that the oom killer might kill additional

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 04:15:26PM +0200, Michal Hocko wrote: > On Mon 24-07-17 17:00:08, Kirill A. Shutemov wrote: > > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > David has noticed that the oom killer might kill additional tasks while > > >

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 17:00:08, Kirill A. Shutemov wrote: > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > David has noticed that the oom killer might kill additional tasks while > > the exiting oom victim hasn't terminated yet because the

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 24-07-17 17:00:08, Kirill A. Shutemov wrote: > On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > David has noticed that the oom killer might kill additional tasks while > > the exiting oom victim hasn't terminated yet because the oom_reaper marks

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the exiting oom victim hasn't terminated yet because the oom_reaper marks > the curent victim MMF_OOM_SKIP too early

Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, Jul 24, 2017 at 09:23:32AM +0200, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the exiting oom victim hasn't terminated yet because the oom_reaper marks > the curent victim MMF_OOM_SKIP too early when mm->mm_users

[PATCH] mm, oom: allow oom reaper to race with exit_mmap

From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too early when mm->mm_users dropped down to 0. The race is as follows

[PATCH] mm, oom: allow oom reaper to race with exit_mmap

From: Michal Hocko David has noticed that the oom killer might kill additional tasks while the exiting oom victim hasn't terminated yet because the oom_reaper marks the curent victim MMF_OOM_SKIP too early when mm->mm_users dropped down to 0. The race is as follows oom_reap_task

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, 20 Jul 2017, Michal Hocko wrote: > On Wed 19-07-17 18:18:27, Hugh Dickins wrote: > > > > But I haven't looked at the oom_kill or oom_reaper end of it at all, > > perhaps you have an overriding argument on the placement from that end. > > Well, the main problem here is that the oom_reaper

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, 20 Jul 2017, Michal Hocko wrote: > On Wed 19-07-17 18:18:27, Hugh Dickins wrote: > > > > But I haven't looked at the oom_kill or oom_reaper end of it at all, > > perhaps you have an overriding argument on the placement from that end. > > Well, the main problem here is that the oom_reaper

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed 19-07-17 18:18:27, Hugh Dickins wrote: > On Wed, 19 Jul 2017, Michal Hocko wrote: > > On Thu 29-06-17 10:46:21, Michal Hocko wrote: > > > Forgot to CC Hugh. > > > > > > Hugh, Andrew, do you see this could cause any problem wrt. > > > ksm/khugepaged exit path? > > > > ping. I would really

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed 19-07-17 18:18:27, Hugh Dickins wrote: > On Wed, 19 Jul 2017, Michal Hocko wrote: > > On Thu 29-06-17 10:46:21, Michal Hocko wrote: > > > Forgot to CC Hugh. > > > > > > Hugh, Andrew, do you see this could cause any problem wrt. > > > ksm/khugepaged exit path? > > > > ping. I would really

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, 19 Jul 2017, Michal Hocko wrote: > On Thu 29-06-17 10:46:21, Michal Hocko wrote: > > Forgot to CC Hugh. > > > > Hugh, Andrew, do you see this could cause any problem wrt. > > ksm/khugepaged exit path? > > ping. I would really appreciate some help here. I would like to resend > the patch

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, 19 Jul 2017, Michal Hocko wrote: > On Thu 29-06-17 10:46:21, Michal Hocko wrote: > > Forgot to CC Hugh. > > > > Hugh, Andrew, do you see this could cause any problem wrt. > > ksm/khugepaged exit path? > > ping. I would really appreciate some help here. I would like to resend > the patch

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 29-06-17 10:46:21, Michal Hocko wrote: > Forgot to CC Hugh. > > Hugh, Andrew, do you see this could cause any problem wrt. > ksm/khugepaged exit path? ping. I would really appreciate some help here. I would like to resend the patch soon. > On Mon 26-06-17 15:03:46, Michal Hocko wrote: >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu 29-06-17 10:46:21, Michal Hocko wrote: > Forgot to CC Hugh. > > Hugh, Andrew, do you see this could cause any problem wrt. > ksm/khugepaged exit path? ping. I would really appreciate some help here. I would like to resend the patch soon. > On Mon 26-06-17 15:03:46, Michal Hocko wrote: >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 11-07-17 13:40:04, David Rientjes wrote: > On Tue, 11 Jul 2017, Michal Hocko wrote: > > > This? > > --- > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > index 5dc0ff22d567..e155d1d8064f 100644 > > --- a/mm/oom_kill.c > > +++ b/mm/oom_kill.c > > @@ -470,11 +470,14 @@ static bool

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 11-07-17 13:40:04, David Rientjes wrote: > On Tue, 11 Jul 2017, Michal Hocko wrote: > > > This? > > --- > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > index 5dc0ff22d567..e155d1d8064f 100644 > > --- a/mm/oom_kill.c > > +++ b/mm/oom_kill.c > > @@ -470,11 +470,14 @@ static bool

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, 11 Jul 2017, Michal Hocko wrote: > This? > --- > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > index 5dc0ff22d567..e155d1d8064f 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -470,11 +470,14 @@ static bool __oom_reap_task_mm(struct task_struct *tsk, > struct mm_struct *mm) > { >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, 11 Jul 2017, Michal Hocko wrote: > This? > --- > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > index 5dc0ff22d567..e155d1d8064f 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -470,11 +470,14 @@ static bool __oom_reap_task_mm(struct task_struct *tsk, > struct mm_struct *mm) > { >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 10-07-17 16:55:22, David Rientjes wrote: > On Mon, 26 Jun 2017, Michal Hocko wrote: > > > diff --git a/mm/mmap.c b/mm/mmap.c > > index 3bd5ecd20d4d..253808e716dc 100644 > > --- a/mm/mmap.c > > +++ b/mm/mmap.c > > @@ -2962,6 +2962,11 @@ void exit_mmap(struct mm_struct *mm) > > /* Use -1

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon 10-07-17 16:55:22, David Rientjes wrote: > On Mon, 26 Jun 2017, Michal Hocko wrote: > > > diff --git a/mm/mmap.c b/mm/mmap.c > > index 3bd5ecd20d4d..253808e716dc 100644 > > --- a/mm/mmap.c > > +++ b/mm/mmap.c > > @@ -2962,6 +2962,11 @@ void exit_mmap(struct mm_struct *mm) > > /* Use -1

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, 27 Jun 2017, Tetsuo Handa wrote: > I wonder why you prefer timeout based approach. Your patch will after all > set MMF_OOM_SKIP if operations between down_write() and up_write() took > more than one second. lock_anon_vma_root() from unlink_anon_vmas() from > free_pgtables() for example

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, 27 Jun 2017, Tetsuo Handa wrote: > I wonder why you prefer timeout based approach. Your patch will after all > set MMF_OOM_SKIP if operations between down_write() and up_write() took > more than one second. lock_anon_vma_root() from unlink_anon_vmas() from > free_pgtables() for example

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, 26 Jun 2017, Michal Hocko wrote: > diff --git a/mm/mmap.c b/mm/mmap.c > index 3bd5ecd20d4d..253808e716dc 100644 > --- a/mm/mmap.c > +++ b/mm/mmap.c > @@ -2962,6 +2962,11 @@ void exit_mmap(struct mm_struct *mm) > /* Use -1 here to ensure all VMAs in the mm are unmapped */ >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Mon, 26 Jun 2017, Michal Hocko wrote: > diff --git a/mm/mmap.c b/mm/mmap.c > index 3bd5ecd20d4d..253808e716dc 100644 > --- a/mm/mmap.c > +++ b/mm/mmap.c > @@ -2962,6 +2962,11 @@ void exit_mmap(struct mm_struct *mm) > /* Use -1 here to ensure all VMAs in the mm are unmapped */ >

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

Forgot to CC Hugh. Hugh, Andrew, do you see this could cause any problem wrt. ksm/khugepaged exit path? On Mon 26-06-17 15:03:46, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the existing victim hasn't

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

Forgot to CC Hugh. Hugh, Andrew, do you see this could cause any problem wrt. ksm/khugepaged exit path? On Mon 26-06-17 15:03:46, Michal Hocko wrote: > From: Michal Hocko > > David has noticed that the oom killer might kill additional tasks while > the existing victim hasn't terminated yet

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 27-06-17 23:26:22, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: [...] > > > shouldn't we try __oom_reap_task_mm() before calling these down_write() > > > if mm is OOM victim's? > > > > This is what we try. We simply try to get mmap_sem for

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 27-06-17 23:26:22, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: [...] > > > shouldn't we try __oom_reap_task_mm() before calling these down_write() > > > if mm is OOM victim's? > > > > This is what we try. We simply try to get mmap_sem for

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

Michal Hocko wrote: > On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Tue 27-06-17 20:39:28, Tetsuo Handa wrote: > > > > Michal Hocko wrote: > > > > > > I wonder why you prefer timeout based approach. Your patch will > > > > > > after all > > > > > > set

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

Michal Hocko wrote: > On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: > > Michal Hocko wrote: > > > On Tue 27-06-17 20:39:28, Tetsuo Handa wrote: > > > > Michal Hocko wrote: > > > > > > I wonder why you prefer timeout based approach. Your patch will > > > > > > after all > > > > > > set

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Tue 27-06-17 20:39:28, Tetsuo Handa wrote: > > > Michal Hocko wrote: > > > > > I wonder why you prefer timeout based approach. Your patch will after > > > > > all > > > > > set MMF_OOM_SKIP if operations between

Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue 27-06-17 22:31:58, Tetsuo Handa wrote: > Michal Hocko wrote: > > On Tue 27-06-17 20:39:28, Tetsuo Handa wrote: > > > Michal Hocko wrote: > > > > > I wonder why you prefer timeout based approach. Your patch will after > > > > > all > > > > > set MMF_OOM_SKIP if operations between

  1   2   >