Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-24 Thread Suren Baghdasaryan
On Fri, Aug 21, 2020 at 11:53 AM Suren Baghdasaryan wrote: > > On Fri, Aug 21, 2020 at 11:00 AM Oleg Nesterov wrote: > > > > On 08/21, Oleg Nesterov wrote: > > > > > > On 08/21, Suren Baghdasaryan wrote: > > > > > > > > On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > > > > > > > > >

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Suren Baghdasaryan
On Fri, Aug 21, 2020 at 11:00 AM Oleg Nesterov wrote: > > On 08/21, Oleg Nesterov wrote: > > > > On 08/21, Suren Baghdasaryan wrote: > > > > > > On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > > > > > > > bool probably_has_other_mm_users(tsk) > > > > { > > > >

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Oleg Nesterov
On 08/21, Oleg Nesterov wrote: > > On 08/21, Suren Baghdasaryan wrote: > > > > On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > > > > > bool probably_has_other_mm_users(tsk) > > > { > > > return atomic_read_acquire(&tsk->mm->mm_users) > > > >

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Suren Baghdasaryan
On Fri, Aug 21, 2020 at 9:37 AM Oleg Nesterov wrote: > > again, don't really understand... > > On 08/21, Suren Baghdasaryan wrote: > > > > Actually, reviewing again and considering where list_add_tail_rcu is > > happening, maybe the race with clone(CLONE_VM) does not introduce > > false negatives.

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Oleg Nesterov
again, don't really understand... On 08/21, Suren Baghdasaryan wrote: > > Actually, reviewing again and considering where list_add_tail_rcu is > happening, maybe the race with clone(CLONE_VM) does not introduce > false negatives. I think it does... Whatever we check, mm_users or MMF_PROC_SHARED,

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Oleg Nesterov
On 08/21, Suren Baghdasaryan wrote: > > On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > > > bool probably_has_other_mm_users(tsk) > > { > > return atomic_read_acquire(&tsk->mm->mm_users) > > > atomic_read(&tsk->signal->live); > >

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Suren Baghdasaryan
On Fri, Aug 21, 2020 at 8:28 AM Suren Baghdasaryan wrote: > > On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > > > On 08/20, Eric W. Biederman wrote: > > > > > > That said if we are going for a small change why not: > > > > > > /* > > >* Make sure we will check other process

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Suren Baghdasaryan
On Fri, Aug 21, 2020 at 4:16 AM Oleg Nesterov wrote: > > On 08/20, Eric W. Biederman wrote: > > > > That said if we are going for a small change why not: > > > > /* > >* Make sure we will check other processes sharing the mm if this is > >* not vfrok which wants its own oom_s

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Oleg Nesterov
On 08/20, Eric W. Biederman wrote: > > That said if we are going for a small change why not: > > /* >* Make sure we will check other processes sharing the mm if this is >* not vfrok which wants its own oom_score_adj. >* pin the mm so it doesn't go away and get reused a

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-21 Thread Michal Hocko
On Thu 20-08-20 23:39:25, Eric W. Biederman wrote: > Michal Hocko writes: > > > On Thu 20-08-20 08:56:53, Suren Baghdasaryan wrote: > > [...] > >> Catching up on the discussion which was going on while I was asleep... > >> So it sounds like there is a consensus that oom_adj should be moved to > >

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Michal Hocko writes: > On Thu 20-08-20 08:56:53, Suren Baghdasaryan wrote: > [...] >> Catching up on the discussion which was going on while I was asleep... >> So it sounds like there is a consensus that oom_adj should be moved to >> mm_struct rather than trying to synchronize it among tasks shar

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Suren Baghdasaryan
On Thu, Aug 20, 2020 at 9:29 AM Christian Brauner wrote: > > On Thu, Aug 20, 2020 at 06:26:45PM +0200, Michal Hocko wrote: > > On Thu 20-08-20 08:56:53, Suren Baghdasaryan wrote: > > [...] > > > Catching up on the discussion which was going on while I was asleep... > > > So it sounds like there is

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 06:26:45PM +0200, Michal Hocko wrote: > On Thu 20-08-20 08:56:53, Suren Baghdasaryan wrote: > [...] > > Catching up on the discussion which was going on while I was asleep... > > So it sounds like there is a consensus that oom_adj should be moved to > > mm_struct rather than

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 08:56:53, Suren Baghdasaryan wrote: [...] > Catching up on the discussion which was going on while I was asleep... > So it sounds like there is a consensus that oom_adj should be moved to > mm_struct rather than trying to synchronize it among tasks sharing mm. > That sounds reasonab

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Suren Baghdasaryan
On Thu, Aug 20, 2020 at 7:53 AM Eric W. Biederman wrote: > > Tetsuo Handa writes: > > > On 2020/08/20 23:00, Christian Brauner wrote: > >> On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: > >>> On 2020/08/20 22:34, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 03:26:31PM +0

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Oleg Nesterov writes: > On 08/20, Oleg Nesterov wrote: >> >> On 08/20, Eric W. Biederman wrote: >> > >> > --- a/fs/exec.c >> > +++ b/fs/exec.c >> > @@ -1139,6 +1139,10 @@ static int exec_mmap(struct mm_struct *mm) >> >vmacache_flush(tsk); >> >task_unlock(tsk); >> >if (old_mm) { >> > +

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 09:49:11AM -0500, Eric W. Biederman wrote: > Tetsuo Handa writes: > > > On 2020/08/20 23:00, Christian Brauner wrote: > >> On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: > >>> On 2020/08/20 22:34, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 03:26

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Tetsuo Handa writes: > On 2020/08/20 23:00, Christian Brauner wrote: >> On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: >>> On 2020/08/20 22:34, Christian Brauner wrote: On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: > If you can handle vfork by other means t

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Oleg Nesterov writes: > On 08/20, Eric W. Biederman wrote: >> >> --- a/fs/exec.c >> +++ b/fs/exec.c >> @@ -1139,6 +1139,10 @@ static int exec_mmap(struct mm_struct *mm) >> vmacache_flush(tsk); >> task_unlock(tsk); >> if (old_mm) { >> +mm->oom_score_adj = old_mm->oom_sco

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Oleg Nesterov
On 08/20, Oleg Nesterov wrote: > > On 08/20, Eric W. Biederman wrote: > > > > --- a/fs/exec.c > > +++ b/fs/exec.c > > @@ -1139,6 +1139,10 @@ static int exec_mmap(struct mm_struct *mm) > > vmacache_flush(tsk); > > task_unlock(tsk); > > if (old_mm) { > > + mm->oom_score_adj = ol

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 23:26:29, Tetsuo Handa wrote: > On 2020/08/20 23:15, Michal Hocko wrote: > > I would tend to agree that from the userspace POV it is nice to look at > > oom tuning per process but fundamentaly the oom killer operates on the > > address space much more than other resources bound to a

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Tetsuo Handa
On 2020/08/20 23:15, Michal Hocko wrote: > I would tend to agree that from the userspace POV it is nice to look at > oom tuning per process but fundamentaly the oom killer operates on the > address space much more than other resources bound to a process because > it is usually the address space hog

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Tetsuo Handa
On 2020/08/20 23:00, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: >> On 2020/08/20 22:34, Christian Brauner wrote: >>> On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: If you can handle vfork by other means then I am all for it. There we

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 16:00:54, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: > > On 2020/08/20 22:34, Christian Brauner wrote: > > > On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: > > >> If you can handle vfork by other means then I am all for it

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 08:41:18, Eric W. Biederman wrote: [...] > I expect something like this completley untested patch will work. Neat. I think you will need to duplicate oom_score_adj_min as well. I am not familiar with vfork specifics to review this in depth but if this is correct then it is much mor

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Oleg Nesterov
On 08/20, Eric W. Biederman wrote: > > --- a/fs/exec.c > +++ b/fs/exec.c > @@ -1139,6 +1139,10 @@ static int exec_mmap(struct mm_struct *mm) > vmacache_flush(tsk); > task_unlock(tsk); > if (old_mm) { > + mm->oom_score_adj = old_mm->oom_score_adj; > + mm->oo

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 10:48:43PM +0900, Tetsuo Handa wrote: > On 2020/08/20 22:34, Christian Brauner wrote: > > On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: > >> If you can handle vfork by other means then I am all for it. There were > >> no patches in that regard proposed yet. M

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Tetsuo Handa
On 2020/08/20 22:34, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: >> If you can handle vfork by other means then I am all for it. There were >> no patches in that regard proposed yet. Maybe it will turn out simpler >> then the heavy lifting we have to do

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 02:41:09PM +0200, Michal Hocko wrote: > On Thu 20-08-20 13:42:56, Michal Hocko wrote: > > On Thu 20-08-20 13:30:23, Christian Brauner wrote: > [...] > > > trying to rely on set_bit() and test_bit() in copy_mm() being atomic and > > > then calling it where Oleg said after the

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Michal Hocko writes: > On Thu 20-08-20 07:54:44, Eric W. Biederman wrote: >> ebied...@xmission.com (Eric W. Biederman) writes: >> >> 2> Michal Hocko writes: >> > >> >> On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: >> >>> Suren Baghdasaryan writes: >> >>> >> >>> > Currently __set_oom_adj

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 03:26:31PM +0200, Michal Hocko wrote: > On Thu 20-08-20 07:54:44, Eric W. Biederman wrote: > > ebied...@xmission.com (Eric W. Biederman) writes: > > > > 2> Michal Hocko writes: > > > > > >> On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: > > >>> Suren Baghdasaryan writ

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 07:54:44, Eric W. Biederman wrote: > ebied...@xmission.com (Eric W. Biederman) writes: > > 2> Michal Hocko writes: > > > >> On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: > >>> Suren Baghdasaryan writes: > >>> > >>> > Currently __set_oom_adj loops through all processes in th

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
ebied...@xmission.com (Eric W. Biederman) writes: 2> Michal Hocko writes: > >> On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: >>> Suren Baghdasaryan writes: >>> >>> > Currently __set_oom_adj loops through all processes in the system to >>> > keep oom_score_adj and oom_score_adj_min in sync

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Michal Hocko writes: > On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: >> Suren Baghdasaryan writes: >> >> > Currently __set_oom_adj loops through all processes in the system to >> > keep oom_score_adj and oom_score_adj_min in sync between processes >> > sharing their mm. This is done for an

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 07:34:41, Eric W. Biederman wrote: > Suren Baghdasaryan writes: > > > Currently __set_oom_adj loops through all processes in the system to > > keep oom_score_adj and oom_score_adj_min in sync between processes > > sharing their mm. This is done for any task with more that one mm_u

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 13:42:56, Michal Hocko wrote: > On Thu 20-08-20 13:30:23, Christian Brauner wrote: [...] > > trying to rely on set_bit() and test_bit() in copy_mm() being atomic and > > then calling it where Oleg said after the point of no return. > > No objections. Would something like the follo

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Eric W. Biederman
Suren Baghdasaryan writes: > Currently __set_oom_adj loops through all processes in the system to > keep oom_score_adj and oom_score_adj_min in sync between processes > sharing their mm. This is done for any task with more that one mm_users, > which includes processes with multiple threads (shari

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 01:29:32PM +0200, Michal Hocko wrote: > On Thu 20-08-20 13:13:55, Michal Hocko wrote: > > On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > > > On 08/19, Suren Baghdasaryan wrote: > > > > > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > > > used the

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 13:30:23, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 01:13:49PM +0200, Michal Hocko wrote: > > On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > > > On 08/19, Suren Baghdasaryan wrote: > > > > > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > > > used

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Oleg Nesterov
On 08/20, Michal Hocko wrote: > > On Thu 20-08-20 13:13:55, Michal Hocko wrote: > > On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > > > On 08/19, Suren Baghdasaryan wrote: > > > > > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > > > used the additional mutex lock in that

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 01:13:49PM +0200, Michal Hocko wrote: > On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > > On 08/19, Suren Baghdasaryan wrote: > > > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > > used the additional mutex lock in that path of the clone() syscall

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 13:13:55, Michal Hocko wrote: > On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > > On 08/19, Suren Baghdasaryan wrote: > > > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > > used the additional mutex lock in that path of the clone() syscall should > > > not

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 12:55:56, Oleg Nesterov wrote: > On 08/19, Suren Baghdasaryan wrote: > > > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > > used the additional mutex lock in that path of the clone() syscall should > > not affect its overall performance. Clearing the MMF_PROC_SHA

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 12:32:48, Christian Brauner wrote: > On Thu, Aug 20, 2020 at 11:09:01AM +0200, Michal Hocko wrote: > > On Thu 20-08-20 10:46:54, Christian Brauner wrote: [...] > > > > which includes processes with multiple threads (sharing mm and signals). > > > > However for such processes the loo

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Oleg Nesterov
On 08/19, Suren Baghdasaryan wrote: > > Since the combination of CLONE_VM and !CLONE_SIGHAND is rarely > used the additional mutex lock in that path of the clone() syscall should > not affect its overall performance. Clearing the MMF_PROC_SHARED flag > (when the last process sharing the mm exits) i

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Thu, Aug 20, 2020 at 11:09:01AM +0200, Michal Hocko wrote: > On Thu 20-08-20 10:46:54, Christian Brauner wrote: > > On Wed, Aug 19, 2020 at 05:20:53PM -0700, Suren Baghdasaryan wrote: > > > Currently __set_oom_adj loops through all processes in the system to > > > keep oom_score_adj and oom_scor

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Michal Hocko
On Thu 20-08-20 10:46:54, Christian Brauner wrote: > On Wed, Aug 19, 2020 at 05:20:53PM -0700, Suren Baghdasaryan wrote: > > Currently __set_oom_adj loops through all processes in the system to > > keep oom_score_adj and oom_score_adj_min in sync between processes > > sharing their mm. This is done

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-20 Thread Christian Brauner
On Wed, Aug 19, 2020 at 05:20:53PM -0700, Suren Baghdasaryan wrote: > Currently __set_oom_adj loops through all processes in the system to > keep oom_score_adj and oom_score_adj_min in sync between processes > sharing their mm. This is done for any task with more that one mm_users, > which includes

Re: [PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-19 Thread Michal Hocko
On Wed 19-08-20 17:20:53, Suren Baghdasaryan wrote: > Currently __set_oom_adj loops through all processes in the system to > keep oom_score_adj and oom_score_adj_min in sync between processes > sharing their mm. This is done for any task with more that one mm_users, > which includes processes with

[PATCH 1/1] mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary

2020-08-19 Thread Suren Baghdasaryan
Currently __set_oom_adj loops through all processes in the system to keep oom_score_adj and oom_score_adj_min in sync between processes sharing their mm. This is done for any task with more that one mm_users, which includes processes with multiple threads (sharing mm and signals). However for such