Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy

2021-03-25 Thread Johannes Weiner
On Thu, Mar 25, 2021 at 10:02:28AM +0100, Michal Hocko wrote: > On Wed 24-03-21 15:49:15, Arjun Roy wrote: > > On Wed, Mar 24, 2021 at 2:24 PM Johannes Weiner wrote: > > > > > > On Wed, Mar 24, 2021 at 10:12:46AM +0100, Michal Hocko wrote: > > > > On

Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy

2021-03-24 Thread Johannes Weiner
On Wed, Mar 24, 2021 at 10:12:46AM +0100, Michal Hocko wrote: > On Tue 23-03-21 11:47:54, Arjun Roy wrote: > > On Tue, Mar 23, 2021 at 7:34 AM Michal Hocko wrote: > > > > > > On Wed 17-03-21 18:12:55, Johannes Weiner wrote: > > > [...] > >

Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy

2021-03-23 Thread Johannes Weiner
On Mon, Mar 22, 2021 at 02:35:11PM -0700, Arjun Roy wrote: > To make sure we're on the same page, then, here's a tentative > mechanism - I'd rather get buy in before spending too much time on > something that wouldn't pass muster afterwards. > > A) An opt-in mechanism, that a driver needs to expli

Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy

2021-03-17 Thread Johannes Weiner
On Tue, Mar 16, 2021 at 11:05:11PM -0700, Arjun Roy wrote: > On Tue, Mar 16, 2021 at 3:27 AM Johannes Weiner wrote: > > > > Hello, > > > > On Mon, Mar 15, 2021 at 09:16:45PM -0700, Arjun Roy wrote: > > > From: Arjun Roy > > > > > > TC

Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy

2021-03-16 Thread Johannes Weiner
Hello, On Mon, Mar 15, 2021 at 09:16:45PM -0700, Arjun Roy wrote: > From: Arjun Roy > > TCP zerocopy receive is used by high performance network applications > to further scale. For RX zerocopy, the memory containing the network > data filled by the network driver is directly mapped into the add

Re: [PATCH 2/5] mm: memcontrol: make page_memcg{_rcu} only applicable for non-kmem page

2021-03-01 Thread Johannes Weiner
Muchun, can you please reduce the CC list to mm/memcg folks only for the next submission? I think probably 80% of the current recipients don't care ;-) On Mon, Mar 01, 2021 at 10:11:45AM -0800, Shakeel Butt wrote: > On Sun, Feb 28, 2021 at 10:25 PM Muchun Song wrote: > > > > We want to reuse the

Re: [PATCH] mm: memcontrol: localize mem_cgroup_sockets_enabled() check

2020-10-12 Thread Johannes Weiner
On Sat, Oct 10, 2020 at 06:45:21PM +0800, Muchun Song wrote: > Move the mem_cgroup_sockets_enabled() checks into memcg socket charge > or uncharge functions, so the users don't have to explicitly check that > condition. > > This is purely code cleanup patch without any functional change. But > mov

Re: [PATCH 3/3] mm: memcontrol: consolidate cgroup socket tracking

2016-09-15 Thread Johannes Weiner
] > > [Check https://git-scm.com/docs/git-format-patch for more information] > > > > url: > > https://github.com/0day-ci/linux/commits/Johannes-Weiner/mm-memcontrol-make-per-cpu-charge-cache-IRQ-safe-for-socket-accounting/20160915-035634 > > config: m68k-sun3_defconfig (

[PATCH 3/3] mm: memcontrol: consolidate cgroup socket tracking

2016-09-14 Thread Johannes Weiner
. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c| 19 +++ net/core/sock.c| 6 +++--- net/ipv4/tcp.c | 2 -- net/ipv4/tcp_ipv4.c| 3 --- 5 files changed, 16 insertions(+), 18 deletions(-) diff

[PATCH 1/3] mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting

2016-09-14 Thread Johannes Weiner
From: Johannes Weiner During cgroup2 rollout into production, we started encountering css refcount underflows and css access crashes in the memory controller. Splitting the heavily shared css reference counter into logical users narrowed the imbalance down to the cgroup2 socket memory accounting

[PATCH 2/3] cgroup: duplicate cgroup reference when cloning sockets

2016-09-14 Thread Johannes Weiner
From: Johannes Weiner When a socket is cloned, the associated sock_cgroup_data is duplicated but not its reference on the cgroup. As a result, the cgroup reference count will underflow when both sockets are destroyed later on. Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup"

Re: [PATCH] net: sock: remove dead cgroup methods from struct proto

2016-01-21 Thread Johannes Weiner
On Thu, Jan 21, 2016 at 10:30:31PM +0300, Sergei Shtylyov wrote: > Hello. > > On 01/21/2016 10:01 PM, Johannes Weiner wrote: > > >The cgroup methods are no longer used after baac50b ("net: > >12-digit ID is now enforced by scripts/checkpatch.pl. Thanks for the

[PATCH] net: sock: remove dead cgroup methods from struct proto

2016-01-21 Thread Johannes Weiner
tcp_memcontrol: simplify linkage between socket and page counter") Signed-off-by: Johannes Weiner --- include/net/sock.h | 12 1 file changed, 12 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index b9e7b3d..f5ea148 100644 --- a/include/net/sock.h +++

Re: [PATCH] mm: memcontrol: only manage socket pressure for CONFIG_INET

2015-12-09 Thread Johannes Weiner
On Wed, Dec 09, 2015 at 02:28:36PM -0800, Andrew Morton wrote: > On Wed, 9 Dec 2015 13:58:58 -0500 Johannes Weiner wrote: > > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > > index 6faea81e66d7..73cd572167bb 100644 > > > --- a/mm/memcontrol.c > > &g

Re: [PATCH] mm: memcontrol: MEMCG no longer works with SLOB

2015-12-09 Thread Johannes Weiner
On Wed, Dec 09, 2015 at 05:32:39PM +0100, Arnd Bergmann wrote: > The change to move the kmem accounting into the normal memcg > code means we can no longer use memcg with slob, which lacks > the memcg_params member in its struct kmem_cache: > > ../mm/slab.h: In function 'is_root_cache': > ../mm/sl

Re: [PATCH] mm: memcontrol: only manage socket pressure for CONFIG_INET

2015-12-09 Thread Johannes Weiner
function `mem_cgroup_css_online': > :(.text+0x1c20e): undefined reference to `tcp_init_cgroup' > > This puts the code causing this in the same #ifdef that guards the > struct member and the TCP implementation. > > Signed-off-by: Arnd Bergmann > Fixes: 20cc40e66c

Re: [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND

2015-12-09 Thread Johannes Weiner
Hey Arnd! On Wed, Dec 09, 2015 at 05:31:38PM +0100, Arnd Bergmann wrote: > On Tuesday 08 December 2015 10:30:10 Johannes Weiner wrote: > > Hi Andrew, > > > > there was some build breakage in CONFIG_ combinations I hadn't tested > > in the last revision, so here i

Re: [PATCH -mm] net: drop tcp_memcontrol.c

2015-12-09 Thread Johannes Weiner
us to > reuse generic code for handling legacy memcg files. > > Signed-off-by: Vladimir Davydov This looks great, the legacy code really doesn't get in the way. I'm sure the network folks appreciate this as well. CC'd and full quote below. Acked-by: Johannes Weiner

[PATCH 09/14] mm: memcontrol: generalize the socket accounting jump label

2015-12-08 Thread Johannes Weiner
The unified hierarchy memory controller is going to use this jump label as well to control the networking callbacks. Move it to the memory controller code and give it a more generic name. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov Acked-by: David S

[PATCH 08/14] net: tcp_memcontrol: simplify linkage between socket and page counter

2015-12-08 Thread Johannes Weiner
There won't be any separate counters for socket memory consumed by protocols other than TCP in the future. Remove the indirection and link sockets directly to their owning memory cgroup. Signed-off-by: Johannes Weiner Reviewed-by: Vladimir Davydov Acked-by: David S. Miller --- include/

[PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND

2015-12-08 Thread Johannes Weiner
Hi Andrew, there was some build breakage in CONFIG_ combinations I hadn't tested in the last revision, so here is a fixed-up resend with minimal CC list. The only difference to the previous version is a section in memcontrol.h, but it accumulates throughout the series and would have been a pain to

[PATCH 01/14] mm: memcontrol: export root_mem_cgroup

2015-12-08 Thread Johannes Weiner
A later patch will need this symbol in files other than memcontrol.c, so export it now and replace mem_cgroup_root_css at the same time. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 3 ++- mm

[PATCH 13/14] mm: memcontrol: hook up vmpressure to socket pressure

2015-12-08 Thread Johannes Weiner
cur before letting the socket code return to normal. This will likely need finetuning for a wider variety of workloads, but for now stick to the vmpressure presets and keep hysteresis simple. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- inc

[PATCH 14/14] mm: memcontrol: switch to the updated jump-label API

2015-12-08 Thread Johannes Weiner
According to the direct use of struct static_key is deprecated. Update the socket and slab accounting code accordingly. Reported-by: Jason Baron Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 8 mm/memcontrol.c| 12 ++-- net/ipv4/tcp_memcontrol.c

[PATCH 02/14] net: tcp_memcontrol: properly detect ancestor socket pressure

2015-12-08 Thread Johannes Weiner
kless page counters") Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/net/sock.h | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 7f89e4b..8133c71 100644 --- a/include/n

[PATCH 03/14] net: tcp_memcontrol: remove bogus hierarchy pressure propagation

2015-12-08 Thread Johannes Weiner
: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/net/sock.h | 19 --- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 8133c71..e27a8bb 100644 --- a/include/net/sock.h +++ b/include/net

[PATCH 05/14] net: tcp_memcontrol: remove dead per-memcg count of allocated sockets

2015-12-08 Thread Johannes Weiner
code. Remove it. As this is the last user of parent_cg_proto(), remove that too. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 1 - include/net/sock.h | 39 +++ net/ipv4

[PATCH 04/14] net: tcp_memcontrol: protect all tcp_memcontrol calls by jump-label

2015-12-08 Thread Johannes Weiner
. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 9 mm/memcontrol.c| 56 +- net/core/sock.c| 9 ++-- net/ipv4/tcp.c | 3 ++- net

[PATCH 07/14] net: tcp_memcontrol: sanitize tcp memory accounting callbacks

2015-12-08 Thread Johannes Weiner
ion is now checked on the per-memcg level and the global level separately. Likewise, memory pressure states are maintained on both the per-memcg level and the global level, and a socket is considered under pressure when either level asserts as much. Signed-off-by: Johannes Weiner Reviewed-by: Vladim

[PATCH 11/14] mm: memcontrol: move socket code for unified hierarchy accounting

2015-12-08 Thread Johannes Weiner
The unified hierarchy memory controller will account socket memory. Move the infrastructure functions accordingly. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov Acked-by: David S. Miller --- mm/memcontrol.c | 148

[PATCH 12/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-12-08 Thread Johannes Weiner
memory resource control. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- Documentation/kernel-parameters.txt | 4 ++ include/linux/memcontrol.h | 9 ++- mm/memcontrol.c | 122 +--- 3 files

[PATCH 10/14] mm: memcontrol: do not account memory+swap on unified hierarchy

2015-12-08 Thread Johannes Weiner
mory support in unified hierarchy, disable the counter altogether when in unified hierarchy mode. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov Acked-by: David S. Miller --- mm/memcontrol.c | 44 +--- 1 file changed

[PATCH 06/14] net: tcp_memcontrol: simplify the per-memcg limit access

2015-12-08 Thread Johannes Weiner
tcp_memcontrol replicates the global sysctl_mem limit array per cgroup, but it only ever sets these entries to the value of the memory_allocated page_counter limit. Use the latter directly. Signed-off-by: Johannes Weiner Reviewed-by: Vladimir Davydov Acked-by: David S. Miller --- include

Re: [PATCH 09/13] mm: memcontrol: generalize the socket accounting jump label

2015-11-30 Thread Johannes Weiner
On Mon, Nov 30, 2015 at 05:28:49PM -0500, Jason Baron wrote: > On 11/30/2015 04:50 PM, Johannes Weiner wrote: > > On Mon, Nov 30, 2015 at 04:08:18PM -0500, Jason Baron wrote: > >> We're trying to move to the updated API, so this should be: > >> static_branch_unl

Re: [PATCH 09/13] mm: memcontrol: generalize the socket accounting jump label

2015-11-30 Thread Johannes Weiner
in there as well. How about the following on top of this series? --- >From b784aa0323628d43272e13a67ead2a2ce0e93ea6 Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 30 Nov 2015 16:41:38 -0500 Subject: [PATCH] mm: memcontrol: switch to the updated jump-label API According to the direct

Re: [PATCH 13/13] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-30 Thread Johannes Weiner
On Mon, Nov 30, 2015 at 02:36:28PM +0300, Vladimir Davydov wrote: > Suppose we have the following cgroup configuration. > > A __ B > \_ C > > A is empty (which is natural for the unified hierarchy AFAIU). B has > some workload running in it, and C generates socket pressure. Due to the > socket

Re: [PATCH 12/13] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-30 Thread Johannes Weiner
On Mon, Nov 30, 2015 at 01:54:21PM +0300, Vladimir Davydov wrote: > On Tue, Nov 24, 2015 at 04:58:44PM -0500, Johannes Weiner wrote: > ... > > @@ -5520,15 +5557,30 @@ void sock_release_memcg(struct sock *sk) > > */ > > bool mem_cgroup_charge_skmem(struct mem_cgr

[PATCH 13/13] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-24 Thread Johannes Weiner
cur before letting the socket code return to normal. This will likely need finetuning for a wider variety of workloads, but for now stick to the vmpressure presets and keep hysteresis simple. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 32 --- include/linux/vm

[PATCH 12/13] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-24 Thread Johannes Weiner
memory resource control. Signed-off-by: Johannes Weiner --- Documentation/kernel-parameters.txt | 4 ++ include/linux/memcontrol.h | 11 +++- mm/memcontrol.c | 122 +--- 3 files changed, 111 insertions(+), 26 deletions(-) diff --git

[PATCH 00/13] mm: memcontrol: account socket memory in unified hierarchy v4

2015-11-24 Thread Johannes Weiner
Hi, this is version 4 of the patches to add socket memory accounting to the unified hierarchy memory controller. Andrew, absent any new showstoppers, please consider merging this series for v4.5. Thanks! Changes since v3 include: - Restored the local vmpressure reporting while preserving the

[PATCH 03/13] net: tcp_memcontrol: remove bogus hierarchy pressure propagation

2015-11-24 Thread Johannes Weiner
: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/net/sock.h | 19 --- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 8133c71..e27a8bb 100644 --- a/include/net/sock.h +++ b/include/net

[PATCH 05/13] net: tcp_memcontrol: remove dead per-memcg count of allocated sockets

2015-11-24 Thread Johannes Weiner
code. Remove it. As this is the last user of parent_cg_proto(), remove that too. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 1 - include/net/sock.h | 39 +++ net/ipv4

[PATCH 04/13] net: tcp_memcontrol: protect all tcp_memcontrol calls by jump-label

2015-11-24 Thread Johannes Weiner
. Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 9 +--- mm/memcontrol.c| 56 +- net/core/sock.c| 9 ++-- net/ipv4/tcp.c | 3 ++- net

[PATCH 07/13] net: tcp_memcontrol: sanitize tcp memory accounting callbacks

2015-11-24 Thread Johannes Weiner
ion is now checked on the per-memcg level and the global level separately. Likewise, memory pressure states are maintained on both the per-memcg level and the global level, and a socket is considered under pressure when either level asserts as much. Signed-off-by: Johannes Weiner Reviewed-by: Vladim

[PATCH 10/13] mm: memcontrol: do not account memory+swap on unified hierarchy

2015-11-24 Thread Johannes Weiner
mory support in unified hierarchy, disable the counter altogether when in unified hierarchy mode. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov --- mm/memcontrol.c | 44 +--- 1 file changed, 25 insertions(+), 19 del

[PATCH 06/13] net: tcp_memcontrol: simplify the per-memcg limit access

2015-11-24 Thread Johannes Weiner
tcp_memcontrol replicates the global sysctl_mem limit array per cgroup, but it only ever sets these entries to the value of the memory_allocated page_counter limit. Use the latter directly. Signed-off-by: Johannes Weiner Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 1

[PATCH 09/13] mm: memcontrol: generalize the socket accounting jump label

2015-11-24 Thread Johannes Weiner
The unified hierarchy memory controller is going to use this jump label as well to control the networking callbacks. Move it to the memory controller code and give it a more generic name. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov --- include/linux

[PATCH 08/13] net: tcp_memcontrol: simplify linkage between socket and page counter

2015-11-24 Thread Johannes Weiner
There won't be any separate counters for socket memory consumed by protocols other than TCP in the future. Remove the indirection and link sockets directly to their owning memory cgroup. Signed-off-by: Johannes Weiner Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h

[PATCH 11/13] mm: memcontrol: move socket code for unified hierarchy accounting

2015-11-24 Thread Johannes Weiner
The unified hierarchy memory controller will account socket memory. Move the infrastructure functions accordingly. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Vladimir Davydov --- mm/memcontrol.c | 148 1 file

[PATCH 02/13] net: tcp_memcontrol: properly detect ancestor socket pressure

2015-11-24 Thread Johannes Weiner
kless page counters") Signed-off-by: Johannes Weiner Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/net/sock.h | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 7f89e4b..8133c71 100644 --- a/include/n

[PATCH 01/13] mm: memcontrol: export root_mem_cgroup

2015-11-24 Thread Johannes Weiner
A later patch will need this symbol in files other than memcontrol.c, so export it now and replace mem_cgroup_root_css at the same time. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: David S. Miller Reviewed-by: Vladimir Davydov --- include/linux/memcontrol.h | 3 ++- mm

Re: [PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-23 Thread Johannes Weiner
On Mon, Nov 23, 2015 at 01:00:59PM +0300, Vladimir Davydov wrote: > I've another question regarding this socket_work: its reclaim target > always equals CHARGE_BATCH. Can't it result in a workload exceeding > memory.high in case there are a lot of allocations coming from different > cpus? In this c

Re: [PATCH 09/14] net: tcp_memcontrol: simplify linkage between socket and page counter

2015-11-23 Thread Johannes Weiner
On Mon, Nov 23, 2015 at 12:36:46PM +0300, Vladimir Davydov wrote: > On Fri, Nov 20, 2015 at 01:56:48PM -0500, Johannes Weiner wrote: > > I actually had all this at first, but then wondered if it makes more > > sense to keep the legacy code in isolation. Don't you think it wo

Re: [PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-20 Thread Johannes Weiner
On Fri, Nov 20, 2015 at 04:10:33PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:32PM -0500, Johannes Weiner wrote: > ... > > @@ -5514,16 +5550,43 @@ void sock_release_memcg(struct sock *sk) > > */ > > bool mem_cgroup_charge_skmem(struct mem_cgr

Re: [PATCH 09/14] net: tcp_memcontrol: simplify linkage between socket and page counter

2015-11-20 Thread Johannes Weiner
On Fri, Nov 20, 2015 at 03:42:16PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:28PM -0500, Johannes Weiner wrote: > > There won't be any separate counters for socket memory consumed by > > protocols other than TCP in the future. Remove the indirection and &

Re: [PATCH 08/14] net: tcp_memcontrol: sanitize tcp memory accounting callbacks

2015-11-20 Thread Johannes Weiner
On Fri, Nov 20, 2015 at 01:58:57PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:27PM -0500, Johannes Weiner wrote: > > There won't be a tcp control soft limit, so integrating the memcg code > > into the global skmem limiting scheme complicates things > &

Re: [PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-19 Thread Johannes Weiner
On Thu, Nov 19, 2015 at 02:50:24PM +0100, Michal Hocko wrote: > On Wed 18-11-15 16:48:22, Johannes Weiner wrote: > [...] > > So I ran perf record -g -a netperf -t TCP_STREAM multiple times inside > > a memory-controlled cgroup, but mostly mem_cgroup_charge_skmem() does >

Re: [PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-18 Thread Johannes Weiner
On Wed, Nov 18, 2015 at 05:22:56PM +0100, Michal Hocko wrote: > On Mon 16-11-15 13:18:10, Johannes Weiner wrote: > > What load would you test and what would be the baseline to compare it > > to? > > It seems like netperf with a stream load running in a memcg with no >

Re: [PATCH 14/14] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-18 Thread Johannes Weiner
On Wed, Nov 18, 2015 at 07:02:54PM +0300, Vladimir Davydov wrote: > On Tue, Nov 17, 2015 at 05:22:17PM -0500, Johannes Weiner wrote: > > On Tue, Nov 17, 2015 at 11:18:50PM +0300, Vladimir Davydov wrote: > > > And with this patch it will work this way, but only if sum limits

Re: [PATCH 14/14] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-17 Thread Johannes Weiner
On Tue, Nov 17, 2015 at 11:18:50PM +0300, Vladimir Davydov wrote: > AFAIK vmpressure was designed to allow userspace to tune hard limits of > cgroups in accordance with their demands, in which case the way how > vmpressure notifications work makes sense. You can still do that when the reporting ha

Re: [PATCH 14/14] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-16 Thread Johannes Weiner
On Sun, Nov 15, 2015 at 04:54:57PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:33PM -0500, Johannes Weiner wrote: > > Let the networking stack know when a memcg is under reclaim pressure > > so that it can clamp its transmit windows accordingly. > > >

Re: [PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-16 Thread Johannes Weiner
On Mon, Nov 16, 2015 at 04:59:25PM +0100, Michal Hocko wrote: > On Thu 12-11-15 18:41:32, Johannes Weiner wrote: > > Socket memory can be a significant share of overall memory consumed by > > common workloads. In order to provide reasonable resource isolation in > > the un

Re: [PATCH 05/14] net: tcp_memcontrol: protect all tcp_memcontrol calls by jump-label

2015-11-16 Thread Johannes Weiner
On Sat, Nov 14, 2015 at 07:33:10PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:24PM -0500, Johannes Weiner wrote: > > Move the jump-label from sock_update_memcg() and sock_release_memcg() > > to the callsite, and so eliminate those function calls when socket &g

Re: [PATCH 03/14] net: tcp_memcontrol: properly detect ancestor socket pressure

2015-11-14 Thread Johannes Weiner
On Sat, Nov 14, 2015 at 03:45:52PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:22PM -0500, Johannes Weiner wrote: > > When charging socket memory, the code currently checks only the local > > page counter for excess to determine whether the memcg is under sock

Re: [PATCH 02/14] mm: vmscan: simplify memcg vs. global shrinker invocation

2015-11-14 Thread Johannes Weiner
On Sat, Nov 14, 2015 at 03:36:50PM +0300, Vladimir Davydov wrote: > On Thu, Nov 12, 2015 at 06:41:21PM -0500, Johannes Weiner wrote: > > @@ -2432,20 +2447,6 @@ static bool shrink_zone(struct zone *zone, struct > > scan_control *sc, > > } > >

Re: [PATCH 08/14] net: tcp_memcontrol: sanitize tcp memory accounting callbacks

2015-11-12 Thread Johannes Weiner
On Thu, Nov 12, 2015 at 08:53:38PM -0800, Eric Dumazet wrote: > On Thu, 2015-11-12 at 18:41 -0500, Johannes Weiner wrote: > > @@ -711,6 +705,12 @@ static inline void mem_cgroup_wb_stats(struct > > bdi_writeback *wb, > > struct sock; > > void sock_update_memcg(

[PATCH 01/14] mm: memcontrol: export root_mem_cgroup

2015-11-12 Thread Johannes Weiner
A later patch will need this symbol in files other than memcontrol.c, so export it now and replace mem_cgroup_root_css at the same time. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- include/linux/memcontrol.h | 3 ++- mm/backing-dev.c | 2 +- mm/memcontrol.c

[PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy

2015-11-12 Thread Johannes Weiner
Hi, this is version 3 of the patches to add socket memory accounting to the unified hierarchy memory controller. Changes since v2 include: - Fixed an underflow bug in the mem+swap counter that came through the design of the per-cpu charge cache. To fix that, the unused mem+swap counter is now

[PATCH 02/14] mm: vmscan: simplify memcg vs. global shrinker invocation

2015-11-12 Thread Johannes Weiner
Letting shrink_slab() handle the root_mem_cgroup, and implicitely the !CONFIG_MEMCG case, allows shrink_zone() to invoke the shrinkers unconditionally from within the memcg iteration loop. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- include/linux/memcontrol.h | 2 ++ mm/vmscan.c

[PATCH 03/14] net: tcp_memcontrol: properly detect ancestor socket pressure

2015-11-12 Thread Johannes Weiner
pressure. This currently doesn't happen. Fix this by using page_counter_try_charge() first. If that fails, it means that either the local counter or one of the ancestors are in excess of their limit, and the child should enter socket pressure. Signed-off-by: Johannes Weiner --- include/net/s

[PATCH 04/14] net: tcp_memcontrol: remove bogus hierarchy pressure propagation

2015-11-12 Thread Johannes Weiner
: Johannes Weiner --- include/net/sock.h | 19 --- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index c4b33c9..6fc9147 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1152,14 +1152,8 @@ static inline void

[PATCH 05/14] net: tcp_memcontrol: protect all tcp_memcontrol calls by jump-label

2015-11-12 Thread Johannes Weiner
. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 9 +--- mm/memcontrol.c| 56 +- net/core/sock.c| 9 ++-- net/ipv4/tcp.c | 3 ++- net/ipv4/tcp_ipv4.c| 4 +++- 5 files changed, 33

[PATCH 06/14] net: tcp_memcontrol: remove dead per-memcg count of allocated sockets

2015-11-12 Thread Johannes Weiner
code. Remove it. As this is the last user of parent_cg_proto(), remove that too. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 1 - include/net/sock.h | 39 +++ net/ipv4/tcp_memcontrol.c | 3 --- 3 files changed, 3 insertions(+), 40

[PATCH 07/14] net: tcp_memcontrol: simplify the per-memcg limit access

2015-11-12 Thread Johannes Weiner
tcp_memcontrol replicates the global sysctl_mem limit array per cgroup, but it only ever sets these entries to the value of the memory_allocated page_counter limit. Use the latter directly. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 1 - include/net/sock.h | 8

[PATCH 08/14] net: tcp_memcontrol: sanitize tcp memory accounting callbacks

2015-11-12 Thread Johannes Weiner
ion is now checked on the per-memcg level and the global level separately. Likewise, memory pressure states are maintained on both the per-memcg level and the global level, and a socket is considered under pressure when either level asserts as much. Signed-off-by: Johannes Weiner --- include/linux/me

[PATCH 11/14] mm: memcontrol: do not account memory+swap on unified hierarchy

2015-11-12 Thread Johannes Weiner
mory support in unified hierarchy, disable the counter altogether when in unified hierarchy mode. Signed-off-by: Johannes Weiner --- mm/memcontrol.c | 44 +--- 1 file changed, 25 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.

[PATCH 10/14] mm: memcontrol: generalize the socket accounting jump label

2015-11-12 Thread Johannes Weiner
The unified hierarchy memory controller is going to use this jump label as well to control the networking callbacks. Move it to the memory controller code and give it a more generic name. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 4 include/net/sock.h | 7

[PATCH 09/14] net: tcp_memcontrol: simplify linkage between socket and page counter

2015-11-12 Thread Johannes Weiner
There won't be any separate counters for socket memory consumed by protocols other than TCP in the future. Remove the indirection and link sockets directly to their owning memory cgroup. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 18 +++- include/net/s

[PATCH 12/14] mm: memcontrol: move socket code for unified hierarchy accounting

2015-11-12 Thread Johannes Weiner
The unified hierarchy memory controller will account socket memory. Move the infrastructure functions accordingly. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- mm/memcontrol.c | 148 1 file changed, 74 insertions(+), 74

[PATCH 13/14] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-12 Thread Johannes Weiner
memory resource control. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 12 - mm/memcontrol.c| 131 + 2 files changed, 118 insertions(+), 25 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux

[PATCH 14/14] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-12 Thread Johannes Weiner
ormal. This will likely need finetuning for a wider variety of workloads, but for now stick to the vmpressure presets and keep hysteresis simple. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 29 + mm/memcontrol.c| 15 +-

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-12 Thread Johannes Weiner
On Thu, Nov 12, 2015 at 06:36:20PM +, Mel Gorman wrote: > Bottom line, there is legimate confusion over whether cgroup controllers > are going to be enabled by default or not in the future. If they are > enabled by default, there is a non-zero cost to that and a change in > semantics that peopl

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-06 Thread Johannes Weiner
On Fri, Nov 06, 2015 at 05:46:57PM +0100, Michal Hocko wrote: > The basic problem was that the Delegate feature has been backported to > our systemd package without further consideration and that has > invalidated a lot of performance testing because some resource > controllers have measurable effe

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-06 Thread Johannes Weiner
On Fri, Nov 06, 2015 at 12:05:55PM +0300, Vladimir Davydov wrote: > On Thu, Nov 05, 2015 at 03:55:22PM -0500, Johannes Weiner wrote: > > On Thu, Nov 05, 2015 at 03:40:02PM +0100, Michal Hocko wrote: > ... > > > 3) keep only some (safe) cache types enabled by def

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-06 Thread Johannes Weiner
On Fri, Nov 06, 2015 at 11:57:24AM +0100, Michal Hocko wrote: > On Thu 05-11-15 17:52:00, Johannes Weiner wrote: > > On Thu, Nov 05, 2015 at 03:55:22PM -0500, Johannes Weiner wrote: > > > On Thu, Nov 05, 2015 at 03:40:02PM +0100, Michal Hocko wrote: > > > > This wou

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-05 Thread Johannes Weiner
On Thu, Nov 05, 2015 at 03:55:22PM -0500, Johannes Weiner wrote: > On Thu, Nov 05, 2015 at 03:40:02PM +0100, Michal Hocko wrote: > > This would be true if they moved on to the new cgroup API intentionally. > > The reality is more complicated though. AFAIK sysmted is waiting for >

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-05 Thread Johannes Weiner
On Thu, Nov 05, 2015 at 05:28:03PM +0100, Michal Hocko wrote: > On Thu 05-11-15 11:16:09, David S. Miller wrote: > > From: Michal Hocko > > Date: Thu, 5 Nov 2015 15:40:02 +0100 > > > > > On Wed 04-11-15 14:50:37, Johannes Weiner wrote: > > > [...] > >

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-05 Thread Johannes Weiner
On Thu, Nov 05, 2015 at 03:40:02PM +0100, Michal Hocko wrote: > On Wed 04-11-15 14:50:37, Johannes Weiner wrote: > [...] > > Because it goes without saying that once the cgroupv2 interface is > > released, and people use it in production, there is no way we can then > > *

[PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy v2

2015-11-04 Thread Johannes Weiner
Hi, this is version 2 of the patches to add socket memory accounting to the unified hierarchy memory controller. Changes from v1 include: - No accounting overhead unless a dedicated cgroup is created and the memory controller instructed to track that group's memory footprint. Distribution ker

[PATCH 2/8] mm: vmscan: simplify memcg vs. global shrinker invocation

2015-11-04 Thread Johannes Weiner
Letting shrink_slab() handle the root_mem_cgroup, and implicitely the !CONFIG_MEMCG case, allows shrink_zone() to invoke the shrinkers unconditionally from within the memcg iteration loop. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- include/linux/memcontrol.h | 2 ++ mm/vmscan.c

[PATCH 4/8] net: tcp_memcontrol: remove bogus hierarchy pressure propagation

2015-11-04 Thread Johannes Weiner
* enter memory pressure mode. But this happens already, albeit lazily: if a parent limit is breached, siblings will enter memory pressure on their own once the next packet arrives for them. So no additional hierarchy code is needed. Remove the bogus stuff. Signed-off-by: Johannes Weiner --- include

[PATCH 1/8] mm: memcontrol: export root_mem_cgroup

2015-11-04 Thread Johannes Weiner
A later patch will need this symbol in files other than memcontrol.c, so export it now and replace mem_cgroup_root_css at the same time. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- include/linux/memcontrol.h | 3 ++- mm/backing-dev.c | 2 +- mm/memcontrol.c

[PATCH 6/8] mm: memcontrol: prepare for unified hierarchy socket accounting

2015-11-04 Thread Johannes Weiner
The unified hierarchy memory controller will account socket memory. Move the infrastructure functions accordingly. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- mm/memcontrol.c | 140 1 file changed, 70 insertions(+), 70

[PATCH 3/8] mm: page_counter: let page_counter_try_charge() return bool

2015-11-04 Thread Johannes Weiner
page_counter_try_charge() currently returns 0 on success and -ENOMEM on failure, which is surprising behavior given the function name. Make it follow the expected pattern of try_stuff() functions that return a boolean true to indicate success, or false for failure. Signed-off-by: Johannes Weiner

[PATCH 8/8] mm: memcontrol: hook up vmpressure to socket pressure

2015-11-04 Thread Johannes Weiner
ormal. This will likely need finetuning for a wider variety of workloads, but for now stick to the vmpressure presets and keep hysteresis simple. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 27 +-- mm/memcontrol.c| 15 +-

[PATCH 7/8] mm: memcontrol: account socket memory in unified hierarchy memory controller

2015-11-04 Thread Johannes Weiner
memory resource control. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 8 +++- mm/memcontrol.c| 110 + 2 files changed, 97 insertions(+), 21 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux

[PATCH 5/8] net: tcp_memcontrol: consolidate socket buffer tracking and accounting

2015-11-04 Thread Johannes Weiner
ely necessary. But this is overengineered: it was meant to protect the people who never use this feature in the first place. Simply enable the branches once when the first limit is set until the next reboot. Signed-off-by: Johannes Weiner --- include/linux/memcontrol.h | 60 ++

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-11-04 Thread Johannes Weiner
On Wed, Nov 04, 2015 at 11:42:40AM +0100, Michal Hocko wrote: > On Thu 29-10-15 09:10:09, Johannes Weiner wrote: > > On Thu, Oct 29, 2015 at 04:25:46PM +0100, Michal Hocko wrote: > > > On Tue 27-10-15 09:42:27, Johannes Weiner wrote: > [...] > > > > You careful

Re: [PATCH 0/8] mm: memcontrol: account socket memory in unified hierarchy

2015-10-29 Thread Johannes Weiner
On Thu, Oct 29, 2015 at 12:27:47PM +0300, Vladimir Davydov wrote: > On Wed, Oct 28, 2015 at 11:58:10AM -0700, Johannes Weiner wrote: > > Having the hard limit as a failsafe (or a minimum for other consumers) > > is one thing, and certainly something I'm open to for cgroupv2

Re: [PATCH 5/8] mm: memcontrol: account socket memory on unified hierarchy

2015-10-29 Thread Johannes Weiner
On Thu, Oct 29, 2015 at 04:25:46PM +0100, Michal Hocko wrote: > On Tue 27-10-15 09:42:27, Johannes Weiner wrote: > > On Tue, Oct 27, 2015 at 05:15:54PM +0100, Michal Hocko wrote: > > > On Tue 27-10-15 11:41:38, Johannes Weiner wrote: > > > > IMO that's an imp

  1   2   >