Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-06 Thread Michal Hocko
On Tue 06-08-19 10:19:49, Konstantin Khlebnikov wrote: > On 8/6/19 10:07 AM, Michal Hocko wrote: > > On Fri 02-08-19 13:44:38, Michal Hocko wrote: > > [...] > > > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > > > > index ba9138a4a1de..53a35c526e43 100644 > > > > > --- a/mm/memcontrol.c >

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-06 Thread Konstantin Khlebnikov
On 8/6/19 10:07 AM, Michal Hocko wrote: On Fri 02-08-19 13:44:38, Michal Hocko wrote: [...] diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ba9138a4a1de..53a35c526e43 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2429,8 +2429,12 @@ static int try_charge(struct mem_cgroup *memcg,

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-06 Thread Michal Hocko
On Fri 02-08-19 13:44:38, Michal Hocko wrote: [...] > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > > index ba9138a4a1de..53a35c526e43 100644 > > > --- a/mm/memcontrol.c > > > +++ b/mm/memcontrol.c > > > @@ -2429,8 +2429,12 @@ static int try_charge(struct mem_cgroup *memcg, > > > gfp_t

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-06 Thread Michal Hocko
On Mon 05-08-19 20:28:40, Yang Shi wrote: > On Mon, Aug 5, 2019 at 7:32 AM Michal Hocko wrote: > > > > On Fri 02-08-19 11:56:28, Yang Shi wrote: > > > On Fri, Aug 2, 2019 at 2:35 AM Michal Hocko wrote: > > > > > > > > On Thu 01-08-19 14:00:51, Yang Shi wrote: > > > > > On Mon, Jul 29, 2019 at

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-05 Thread Yang Shi
On Mon, Aug 5, 2019 at 7:32 AM Michal Hocko wrote: > > On Fri 02-08-19 11:56:28, Yang Shi wrote: > > On Fri, Aug 2, 2019 at 2:35 AM Michal Hocko wrote: > > > > > > On Thu 01-08-19 14:00:51, Yang Shi wrote: > > > > On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > > > > > > > > > On Mon

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-05 Thread Shakeel Butt
On Mon, Aug 5, 2019 at 7:32 AM Michal Hocko wrote: > > On Fri 02-08-19 11:56:28, Yang Shi wrote: > > On Fri, Aug 2, 2019 at 2:35 AM Michal Hocko wrote: > > > > > > On Thu 01-08-19 14:00:51, Yang Shi wrote: > > > > On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > > > > > > > > > On Mon

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-05 Thread Michal Hocko
On Fri 02-08-19 11:56:28, Yang Shi wrote: > On Fri, Aug 2, 2019 at 2:35 AM Michal Hocko wrote: > > > > On Thu 01-08-19 14:00:51, Yang Shi wrote: > > > On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > > > > > > > On Mon 29-07-19 10:28:43, Yang Shi wrote: > > > > [...] > > > > > I don't

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-02 Thread Yang Shi
On Fri, Aug 2, 2019 at 2:35 AM Michal Hocko wrote: > > On Thu 01-08-19 14:00:51, Yang Shi wrote: > > On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > > > > > On Mon 29-07-19 10:28:43, Yang Shi wrote: > > > [...] > > > > I don't worry too much about scale since the scale issue is not

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-02 Thread Michal Hocko
On Fri 02-08-19 13:01:07, Konstantin Khlebnikov wrote: > > > On 02.08.2019 12:40, Michal Hocko wrote: > > On Mon 29-07-19 20:55:09, Michal Hocko wrote: > > > On Mon 29-07-19 11:49:52, Johannes Weiner wrote: > > > > On Sun, Jul 28, 2019 at 03:29:38PM +0300, Konstantin Khlebnikov wrote: > > > > >

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-02 Thread Michal Hocko
On Mon 29-07-19 20:55:09, Michal Hocko wrote: > On Mon 29-07-19 11:49:52, Johannes Weiner wrote: > > On Sun, Jul 28, 2019 at 03:29:38PM +0300, Konstantin Khlebnikov wrote: > > > --- a/mm/gup.c > > > +++ b/mm/gup.c > > > @@ -847,8 +847,11 @@ static long __get_user_pages(struct task_struct > > >

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-02 Thread Konstantin Khlebnikov
On 02.08.2019 12:40, Michal Hocko wrote: On Mon 29-07-19 20:55:09, Michal Hocko wrote: On Mon 29-07-19 11:49:52, Johannes Weiner wrote: On Sun, Jul 28, 2019 at 03:29:38PM +0300, Konstantin Khlebnikov wrote: --- a/mm/gup.c +++ b/mm/gup.c @@ -847,8 +847,11 @@ static long

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-02 Thread Michal Hocko
On Thu 01-08-19 14:00:51, Yang Shi wrote: > On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > > > On Mon 29-07-19 10:28:43, Yang Shi wrote: > > [...] > > > I don't worry too much about scale since the scale issue is not unique > > > to background reclaim, direct reclaim may run into the

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-08-01 Thread Yang Shi
On Mon, Jul 29, 2019 at 11:48 AM Michal Hocko wrote: > > On Mon 29-07-19 10:28:43, Yang Shi wrote: > [...] > > I don't worry too much about scale since the scale issue is not unique > > to background reclaim, direct reclaim may run into the same problem. > > Just to clarify. By scaling problem I

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Michal Hocko
On Mon 29-07-19 11:49:52, Johannes Weiner wrote: > On Sun, Jul 28, 2019 at 03:29:38PM +0300, Konstantin Khlebnikov wrote: > > --- a/mm/gup.c > > +++ b/mm/gup.c > > @@ -847,8 +847,11 @@ static long __get_user_pages(struct task_struct *tsk, > > struct mm_struct *mm, > > ret =

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Michal Hocko
On Mon 29-07-19 10:28:43, Yang Shi wrote: [...] > I don't worry too much about scale since the scale issue is not unique > to background reclaim, direct reclaim may run into the same problem. Just to clarify. By scaling problem I mean 1:1 kswapd thread to memcg. You can have thousands of memcgs

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Yang Shi
On Mon, Jul 29, 2019 at 3:33 AM Michal Hocko wrote: > > On Mon 29-07-19 12:40:29, Konstantin Khlebnikov wrote: > > On 29.07.2019 12:17, Michal Hocko wrote: > > > On Sun 28-07-19 15:29:38, Konstantin Khlebnikov wrote: > > > > High memory limit in memory cgroup allows to batch memory reclaiming and

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Johannes Weiner
On Sun, Jul 28, 2019 at 03:29:38PM +0300, Konstantin Khlebnikov wrote: > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -847,8 +847,11 @@ static long __get_user_pages(struct task_struct *tsk, > struct mm_struct *mm, > ret = -ERESTARTSYS; > goto out; >

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Konstantin Khlebnikov
On 29.07.2019 13:33, Michal Hocko wrote: On Mon 29-07-19 12:40:29, Konstantin Khlebnikov wrote: On 29.07.2019 12:17, Michal Hocko wrote: On Sun 28-07-19 15:29:38, Konstantin Khlebnikov wrote: High memory limit in memory cgroup allows to batch memory reclaiming and defer it until returning

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Michal Hocko
On Mon 29-07-19 12:40:29, Konstantin Khlebnikov wrote: > On 29.07.2019 12:17, Michal Hocko wrote: > > On Sun 28-07-19 15:29:38, Konstantin Khlebnikov wrote: > > > High memory limit in memory cgroup allows to batch memory reclaiming and > > > defer it until returning into userland. This moves it

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Konstantin Khlebnikov
On 29.07.2019 12:17, Michal Hocko wrote: On Sun 28-07-19 15:29:38, Konstantin Khlebnikov wrote: High memory limit in memory cgroup allows to batch memory reclaiming and defer it until returning into userland. This moves it out of any locks. Fixed gap between high and max limit works pretty

Re: [PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-29 Thread Michal Hocko
On Sun 28-07-19 15:29:38, Konstantin Khlebnikov wrote: > High memory limit in memory cgroup allows to batch memory reclaiming and > defer it until returning into userland. This moves it out of any locks. > > Fixed gap between high and max limit works pretty well (we are using > 64 * NR_CPUS

[PATCH RFC] mm/memcontrol: reclaim severe usage over high limit in get_user_pages loop

2019-07-28 Thread Konstantin Khlebnikov
High memory limit in memory cgroup allows to batch memory reclaiming and defer it until returning into userland. This moves it out of any locks. Fixed gap between high and max limit works pretty well (we are using 64 * NR_CPUS pages) except cases when one syscall allocates tons of memory. This