Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-09-09 Thread Michal Hocko
[Sorry, this slipped through cracks] On Mon 24-08-20 12:58:50, Johannes Weiner wrote: > On Fri, Aug 21, 2020 at 09:37:16PM +0200, Peter Zijlstra wrote: [...] > > Arguably seeing the rate drop to near 0 is a very good point to consider > > running cgroup-OOM. > > Agreed. In the past, that's actual

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-09-07 Thread Chris Down
Johannes Weiner writes: That all being said, the semantics of the new 'high' limit in cgroup2 have allowed us to move reclaim/limit enforcement out of the allocation context and into the userspace return path. See the call to mem_cgroup_handle_over_high() from tracehook_notify_resume(), and the

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-24 Thread Johannes Weiner
On Fri, Aug 21, 2020 at 09:37:16PM +0200, Peter Zijlstra wrote: > On Tue, Aug 18, 2020 at 09:49:00AM -0400, Johannes Weiner wrote: > > On Tue, Aug 18, 2020 at 12:18:44PM +0200, pet...@infradead.org wrote: > > > What you need is a feeback loop against the rate of freeing pages, and > > > when you ne

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-22 Thread Waiman Long
On 8/18/20 6:17 AM, Chris Down wrote: pet...@infradead.org writes: But then how can it run-away like Waiman suggested? Probably because he's not running with that commit at all. We and others use this to prevent runaway allocation on a huge range of production and desktop use cases and it wo

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-21 Thread Peter Zijlstra
On Tue, Aug 18, 2020 at 09:49:00AM -0400, Johannes Weiner wrote: > On Tue, Aug 18, 2020 at 12:18:44PM +0200, pet...@infradead.org wrote: > > What you need is a feeback loop against the rate of freeing pages, and > > when you near the saturation point, the allocation rate should exactly > > match th

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-19 Thread Dave Chinner
On Tue, Aug 18, 2020 at 01:55:59PM +0100, Matthew Wilcox wrote: > On Tue, Aug 18, 2020 at 12:04:44PM +0200, pet...@infradead.org wrote: > > On Tue, Aug 18, 2020 at 10:27:37AM +0100, Chris Down wrote: > > > pet...@infradead.org writes: > > > > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wr

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Waiman Long
On 8/18/20 5:27 AM, Chris Down wrote: pet...@infradead.org writes: On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "memory.high" in a v2 non-root memory cgroup, t

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Waiman Long
On 8/18/20 5:14 AM, pet...@infradead.org wrote: On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "memory.high" in a v2 non-root memory cgroup, the memory controller

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Waiman Long
On 8/17/20 3:26 PM, Michal Hocko wrote: On Mon 17-08-20 11:55:37, Waiman Long wrote: On 8/17/20 11:26 AM, Michal Hocko wrote: On Mon 17-08-20 10:08:23, Waiman Long wrote: Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "me

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Johannes Weiner
On Tue, Aug 18, 2020 at 12:18:44PM +0200, pet...@infradead.org wrote: > What you need is a feeback loop against the rate of freeing pages, and > when you near the saturation point, the allocation rate should exactly > match the freeing rate. IO throttling solves a slightly different problem. IO o

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Matthew Wilcox
On Tue, Aug 18, 2020 at 12:04:44PM +0200, pet...@infradead.org wrote: > On Tue, Aug 18, 2020 at 10:27:37AM +0100, Chris Down wrote: > > pet...@infradead.org writes: > > > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > > > > Memory controller can be used to control and limit the amou

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Tue, Aug 18, 2020 at 12:30:59PM +0200, Michal Hocko wrote: > The proposal also aims at much richer interface to define the > oom behavior. Oh yeah, I'm not defending any of that prctl() nonsense. Just saying that from a math / control theory point of view, the current thing is a abhorrent fail

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Chris Down
pet...@infradead.org writes: On Tue, Aug 18, 2020 at 11:17:56AM +0100, Chris Down wrote: I'd ask that you understand a bit more about the tradeoffs and intentions of the patch before rushing in to declare its failure, considering it works just fine :-) Clamping the maximal time allows the appl

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Michal Hocko
On Tue 18-08-20 12:18:44, Peter Zijlstra wrote: > On Tue, Aug 18, 2020 at 12:05:16PM +0200, Michal Hocko wrote: > > > But then how can it run-away like Waiman suggested? > > > > As Chris mentioned in other reply. This functionality is quite new. > > > > > /me goes look... and finds MEMCG_MAX_HIG

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Tue, Aug 18, 2020 at 11:17:56AM +0100, Chris Down wrote: > I'd ask that you understand a bit more about the tradeoffs and intentions of > the patch before rushing in to declare its failure, considering it works > just fine :-) > > Clamping the maximal time allows the application to take some a

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Tue, Aug 18, 2020 at 12:05:16PM +0200, Michal Hocko wrote: > > But then how can it run-away like Waiman suggested? > > As Chris mentioned in other reply. This functionality is quite new. > > > /me goes look... and finds MEMCG_MAX_HIGH_DELAY_JIFFIES. > > We can certainly tune a different back

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Chris Down
pet...@infradead.org writes: But then how can it run-away like Waiman suggested? Probably because he's not running with that commit at all. We and others use this to prevent runaway allocation on a huge range of production and desktop use cases and it works just fine. /me goes look... and

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Tue, Aug 18, 2020 at 10:27:37AM +0100, Chris Down wrote: > pet...@infradead.org writes: > > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > > > Memory controller can be used to control and limit the amount of > > > physical memory used by a task. When a limit is set in "memory.hig

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Michal Hocko
On Tue 18-08-20 11:59:10, Peter Zijlstra wrote: > On Tue, Aug 18, 2020 at 11:26:17AM +0200, Michal Hocko wrote: > > On Tue 18-08-20 11:14:53, Peter Zijlstra wrote: > > > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > > > > Memory controller can be used to control and limit the amoun

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Tue, Aug 18, 2020 at 11:26:17AM +0200, Michal Hocko wrote: > On Tue 18-08-20 11:14:53, Peter Zijlstra wrote: > > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > > > Memory controller can be used to control and limit the amount of > > > physical memory used by a task. When a limit

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Chris Down
pet...@infradead.org writes: On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "memory.high" in a v2 non-root memory cgroup, the memory controller will try to reclai

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread Michal Hocko
On Tue 18-08-20 11:14:53, Peter Zijlstra wrote: > On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > > Memory controller can be used to control and limit the amount of > > physical memory used by a task. When a limit is set in "memory.high" in > > a v2 non-root memory cgroup, the memory

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-18 Thread peterz
On Mon, Aug 17, 2020 at 10:08:23AM -0400, Waiman Long wrote: > Memory controller can be used to control and limit the amount of > physical memory used by a task. When a limit is set in "memory.high" in > a v2 non-root memory cgroup, the memory controller will try to reclaim > memory if the limit ha

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-17 Thread Michal Hocko
On Mon 17-08-20 10:08:23, Waiman Long wrote: > Memory controller can be used to control and limit the amount of > physical memory used by a task. When a limit is set in "memory.high" in > a v2 non-root memory cgroup, the memory controller will try to reclaim > memory if the limit has been exceeded.

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-17 Thread Michal Hocko
On Mon 17-08-20 11:55:37, Waiman Long wrote: > On 8/17/20 11:26 AM, Michal Hocko wrote: > > On Mon 17-08-20 10:08:23, Waiman Long wrote: > > > Memory controller can be used to control and limit the amount of > > > physical memory used by a task. When a limit is set in "memory.high" in > > > a v2 no

Re: [RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-17 Thread Waiman Long
On 8/17/20 11:26 AM, Michal Hocko wrote: On Mon 17-08-20 10:08:23, Waiman Long wrote: Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "memory.high" in a v2 non-root memory cgroup, the memory controller will try to reclaim me

[RFC PATCH 0/8] memcg: Enable fine-grained per process memory control

2020-08-17 Thread Waiman Long
Memory controller can be used to control and limit the amount of physical memory used by a task. When a limit is set in "memory.high" in a v2 non-root memory cgroup, the memory controller will try to reclaim memory if the limit has been exceeded. Normally, that will be enough to keep the physical m