Re: [PATCH] mm: avoid bothering interrupted task when charge memcg in softirq

2018-07-14 Thread Shakeel Butt
On Sat, Jul 14, 2018 at 7:10 PM Yafang Shao wrote: > > On Sat, Jul 14, 2018 at 11:38 PM, Shakeel Butt wrote: > > On Sat, Jul 14, 2018 at 1:32 AM Yafang Shao wrote: > >> > >> try_charge maybe executed in packet receive path, which is in interrupt > &

Re: Showing /sys/fs/cgroup/memory/memory.stat very slow on some machines

2018-07-18 Thread Shakeel Butt
On Wed, Jul 18, 2018 at 7:29 AM Bruce Merry wrote: > > On 18 July 2018 at 12:42, Michal Hocko wrote: > > [CC some more people] > > > > On Tue 17-07-18 21:23:07, Andrew Morton wrote: > >> (cc linux-mm) > >> > >> On Tue, 3 Jul 2018 08:43:23 +0200 Bruce Merry wrote: > >> > >> > Hi > >> > > >> >

Re: Showing /sys/fs/cgroup/memory/memory.stat very slow on some machines

2018-07-18 Thread Shakeel Butt
On Wed, Jul 18, 2018 at 8:27 AM Bruce Merry wrote: > > On 18 July 2018 at 16:47, Michal Hocko wrote: > >> Thanks for looking into this. I'm not familiar with ftrace. Can you > >> give me a specific command line to run? Based on "perf record cat > >> /sys/fs/cgroup/memory/memory.stat"/"perf

Re: Showing /sys/fs/cgroup/memory/memory.stat very slow on some machines

2018-07-18 Thread Shakeel Butt
On Wed, Jul 18, 2018 at 8:37 AM Bruce Merry wrote: > > On 18 July 2018 at 17:26, Shakeel Butt wrote: > > On Wed, Jul 18, 2018 at 7:29 AM Bruce Merry wrote: > > It seems like you are using cgroup-v1. How many nodes are there in > > your memcg tree and also how many c

Re: Showing /sys/fs/cgroup/memory/memory.stat very slow on some machines

2018-07-18 Thread Shakeel Butt
On Wed, Jul 18, 2018 at 10:40 AM Bruce Merry wrote: > > On 18 July 2018 at 17:49, Shakeel Butt wrote: > > On Wed, Jul 18, 2018 at 8:37 AM Bruce Merry wrote: > >> That sounds promising. Is there any way to tell how many zombies there > >> are, and is there any way

Re: Showing /sys/fs/cgroup/memory/memory.stat very slow on some machines

2018-07-18 Thread Shakeel Butt
On Wed, Jul 18, 2018 at 10:58 AM Bruce Merry wrote: > > On 18 July 2018 at 19:48, Shakeel Butt wrote: > > On Wed, Jul 18, 2018 at 10:40 AM Bruce Merry wrote: > >> > Yes, very easy to produce zombies, though I don't think kernel > >> > provides an

Re: [PATCH] mm: avoid bothering interrupted task when charge memcg in softirq

2018-07-15 Thread Shakeel Butt
On Sun, Jul 15, 2018 at 1:02 AM Yafang Shao wrote: > > On Sun, Jul 15, 2018 at 2:34 PM, Shakeel Butt wrote: > > On Sat, Jul 14, 2018 at 10:26 PM Yafang Shao wrote: > >> > >> On Sun, Jul 15, 2018 at 12:25 PM, Shakeel Butt wrote: > >> > On Sat,

Re: [PATCH] mm: avoid bothering interrupted task when charge memcg in softirq

2018-07-15 Thread Shakeel Butt
On Sun, Jul 15, 2018 at 6:50 PM Yafang Shao wrote: > > On Sun, Jul 15, 2018 at 11:04 PM, Shakeel Butt wrote: > > On Sun, Jul 15, 2018 at 1:02 AM Yafang Shao wrote: > >> > >> On Sun, Jul 15, 2018 at 2:34 PM, Shakeel Butt wrote: > >> > On Sat, Jul 14, 2

Re: [PATCH] mm: memcg: fix use after free in mem_cgroup_iter()

2018-07-23 Thread Shakeel Butt
On Sun, Jul 22, 2018 at 11:44 PM Michal Hocko wrote: > > On Thu 19-07-18 09:23:10, Shakeel Butt wrote: > > On Thu, Jul 19, 2018 at 3:43 AM Michal Hocko wrote: > > > > > > [CC Andrew] > > > > > > On Thu 19-07-18 18:06:47, Jing Xia wrote: >

[PATCH] memcg: reduce memcg tree traversals for stats collection

2018-07-24 Thread Shakeel Butt
memcgs on cgroup-v1. The results are: Without the patch: $ time ./read-root-stat-1000-times real0m1.663s user0m0.000s sys 0m1.660s With the patch: $ time ./read-root-stat-1000-times real0m0.468s user0m0.000s sys 0m0.467s Signed-off-by: Shakeel Butt --- mm/memcontrol.c

Re: [PATCH] memcg: reduce memcg tree traversals for stats collection

2018-07-25 Thread Shakeel Butt
On Wed, Jul 25, 2018 at 4:26 AM Bruce Merry wrote: > > On 25 July 2018 at 00:46, Shakeel Butt wrote: > > I ran a simple benchmark which reads the root_mem_cgroup's stat file > > 1000 times in the presense of 2500 memcgs on cgroup-v1. The results are: > > > > Withou

Re: [PATCH] mm: memcg: fix use after free in mem_cgroup_iter()

2018-07-19 Thread Shakeel Butt
On Thu, Jul 19, 2018 at 3:43 AM Michal Hocko wrote: > > [CC Andrew] > > On Thu 19-07-18 18:06:47, Jing Xia wrote: > > It was reported that a kernel crash happened in mem_cgroup_iter(), > > which can be triggered if the legacy cgroup-v1 non-hierarchical > > mode is used. > > > > Unable to handle

Re: [PATCH] mm: Cleanup in do_shrink_slab()

2018-07-19 Thread Shakeel Butt
f-by: Kirill Tkhai Reviewed-by: Shakeel Butt > --- > mm/vmscan.c | 11 +++ > 1 file changed, 3 insertions(+), 8 deletions(-) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 9918bfc1d2f9..636657213b9b 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -445,16

Re: [PATCH] mm: avoid bothering interrupted task when charge memcg in softirq

2018-07-14 Thread Shakeel Butt
On Sat, Jul 14, 2018 at 1:32 AM Yafang Shao wrote: > > try_charge maybe executed in packet receive path, which is in interrupt > context. > In this situation, the 'current' is the interrupted task, which may has > no relation to the rx softirq, So it is nonsense to use 'current'. > Have you

Re: [PATCH v2] mm: slowly shrink slabs with a relatively small number of objects

2018-09-05 Thread Shakeel Butt
On Wed, Sep 5, 2018 at 2:23 PM Roman Gushchin wrote: > > On Wed, Sep 05, 2018 at 01:51:52PM -0700, Andrew Morton wrote: > > On Tue, 4 Sep 2018 15:47:07 -0700 Roman Gushchin wrote: > > > > > Commit 9092c71bb724 ("mm: use sc->priority for slab shrink targets") > > > changed the way how the target

Re: [PATCH v2 1/3] mm: rework memcg kernel stack accounting

2018-08-29 Thread Shakeel Butt
On Wed, Aug 29, 2018 at 2:24 PM Roman Gushchin wrote: > > On Tue, Aug 21, 2018 at 03:10:52PM -0700, Shakeel Butt wrote: > > On Tue, Aug 21, 2018 at 2:36 PM Roman Gushchin wrote: > > > > > > If CONFIG_VMAP_STACK is set, kernel stacks are allocated &g

Re: [PATCH v8 03/17] mm: Assign id to every memcg-aware shrinker

2018-07-03 Thread Shakeel Butt
On Tue, Jul 3, 2018 at 8:27 AM Matthew Wilcox wrote: > > On Tue, Jul 03, 2018 at 06:09:05PM +0300, Kirill Tkhai wrote: > > +++ b/mm/vmscan.c > > @@ -169,6 +169,49 @@ unsigned long vm_total_pages; > > static LIST_HEAD(shrinker_list); > > static DECLARE_RWSEM(shrinker_rwsem); > > > > +#ifdef

Re: [PATCH v8 03/17] mm: Assign id to every memcg-aware shrinker

2018-07-03 Thread Shakeel Butt
On Tue, Jul 3, 2018 at 9:17 AM Kirill Tkhai wrote: > > Hi, Shakeel, > > On 03.07.2018 18:46, Shakeel Butt wrote: > > On Tue, Jul 3, 2018 at 8:27 AM Matthew Wilcox wrote: > >> > >> On Tue, Jul 03, 2018 at 06:09:05PM +0300, Kirill Tkhai wrote: > >&g

[PATCH] block, mm: remove unnecessary __GFP_HIGH flag

2018-07-03 Thread Shakeel Butt
The flag GFP_ATOMIC already contains __GFP_HIGH. There is no need to explicitly or __GFP_HIGH again. So, just remove unnecessary __GFP_HIGH. Signed-off-by: Shakeel Butt --- block/blk-ioc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-ioc.c b/block/blk-ioc.c

Re: [PATCH v8 03/17] mm: Assign id to every memcg-aware shrinker

2018-07-03 Thread Shakeel Butt
On Tue, Jul 3, 2018 at 12:13 PM Kirill Tkhai wrote: > > On 03.07.2018 20:58, Matthew Wilcox wrote: > > On Tue, Jul 03, 2018 at 06:46:57PM +0300, Kirill Tkhai wrote: > >> shrinker_idr now contains only memcg-aware shrinkers, so all bits from > >> memcg map > >> may be potentially populated. In

Re: [PATCH v8 03/17] mm: Assign id to every memcg-aware shrinker

2018-07-03 Thread Shakeel Butt
On Tue, Jul 3, 2018 at 12:25 PM Matthew Wilcox wrote: > > On Tue, Jul 03, 2018 at 12:19:35PM -0700, Shakeel Butt wrote: > > On Tue, Jul 3, 2018 at 12:13 PM Kirill Tkhai wrote: > > > > Do we really have so very many !memcg-aware shrinkers? > > > > > &g

[PATCH] mm: memcg: expose mem_cgroup_put API

2018-03-07 Thread Shakeel Butt
This patch exports mem_cgroup_put API to put the refcnt of the memory cgroup. Signed-off-by: Shakeel Butt <shake...@google.com> --- include/linux/memcontrol.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index c46016

Re: mmotm 2018-03-07-16-19 uploaded (UML & memcg)

2018-03-07 Thread Shakeel Butt
On Wed, Mar 7, 2018 at 6:34 PM, Shakeel Butt <shake...@google.com> wrote: > On Wed, Mar 7, 2018 at 6:23 PM, Randy Dunlap <rdun...@infradead.org> wrote: >> On 03/07/2018 06:20 PM, Randy Dunlap wrote: >>> On 03/07/2018 04:20 PM, a...@linux-foundation.org wrote: >

Re: mmotm 2018-03-07-16-19 uploaded (UML & memcg)

2018-03-07 Thread Shakeel Butt
dom hopefully >>> more than once a week. >> >> UML on i386 and/or x86_64: >> >> defconfig, CONFIG_MEMCG is not set: >> >> ../fs/notify/group.c: In function 'fsnotify_final_destroy_group': >> ../fs/notify/group.c:41:24: error: dereferencing pointer to incomp

[PATCH] slab, slub: remove size disparity on debug kernel

2018-03-13 Thread Shakeel Butt
, the non-root slab's size is measured based on the root's object_size and thus the size will remain same for root and non-root slab. This patch makes slab's object_size the default base to measure the slab's size. Signed-off-by: Shakeel Butt <shake...@google.com> --- mm/slab_common.c | 9 --

Re: [PATCH] slab, slub: remove size disparity on debug kernel

2018-03-13 Thread Shakeel Butt
On Tue, Mar 13, 2018 at 10:19 AM, Christopher Lameter <c...@linux.com> wrote: > On Tue, 13 Mar 2018, Shakeel Butt wrote: > >> However for SLUB in debug kernel, the sizes were same. On further >> inspection it is found that SLUB always use kmem_cache.object_size to >&g

Re: [PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-03-13 Thread Shakeel Butt
On Tue, Mar 13, 2018 at 6:49 AM, Michal Hocko <mho...@kernel.org> wrote: > On Wed 21-02-18 14:37:56, Shakeel Butt wrote: > [...] >> +#ifdef CONFIG_MEMCG >> +static inline struct mem_cgroup *memalloc_memcg_save(struct mem_cgroup >> *memcg) >> +{ >> +

Re: [PATCH 1/6] mm/vmscan: Wake up flushers for legacy cgroups too

2018-03-15 Thread Shakeel Butt
total-vm:8484kB, anon-rss:1052kB, > file-rss:1720kB, shmem-rss:0kB > oom_reaper: reaped process 3876 (dd), now anon-rss:0kB, file-rss:0kB, > shmem-rss:0kB > > Wake up flushers in legacy cgroup reclaim too. > > Fixes: bbef938429f5 ("mm: vmscan: remove old flusher wak

Re: [PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-03-15 Thread Shakeel Butt
On Thu, Mar 15, 2018 at 10:49 AM, Michal Hocko <mho...@kernel.org> wrote: > On Tue 13-03-18 10:55:18, Shakeel Butt wrote: >> On Tue, Mar 13, 2018 at 6:49 AM, Michal Hocko <mho...@kernel.org> wrote: >> > On Wed 21-02-18 14:37:56, Shakeel Butt wrote: >>

Re: [PATCH] slab, slub: remove size disparity on debug kernel

2018-03-14 Thread Shakeel Butt
On Wed, Mar 14, 2018 at 1:43 AM, Vladimir Davydov <vdavydov@gmail.com> wrote: > On Tue, Mar 13, 2018 at 10:36:52AM -0700, Shakeel Butt wrote: >> On Tue, Mar 13, 2018 at 10:19 AM, Christopher Lameter <c...@linux.com> wrote: >> > On Tue, 13 Mar 2018, Shakeel

Re: [PATCH] mm: memcg: expose mem_cgroup_put API

2018-03-08 Thread Shakeel Butt
On Thu, Mar 8, 2018 at 3:45 PM, Andrew Morton <a...@linux-foundation.org> wrote: > On Wed, 7 Mar 2018 18:48:50 -0800 Shakeel Butt <shake...@google.com> wrote: > >> This patch exports mem_cgroup_put API to put the refcnt of the memory >> cgroup. > > OK, I rem

Re: [PATCH v2 3/4] mm/vmscan: Don't change pgdat state on base of a single LRU list state.

2018-04-05 Thread Shakeel Butt
t; > This change will not have any effect on a systems with all workload > concentrated in a single cgroup. > > Signed-off-by: Andrey Ryabinin <aryabi...@virtuozzo.com> Seems reasonable. Reviewed-by: Shakeel Butt <shake...@google.com> > --- > mm/vmscan.c | 124 &g

Re: [PATCH v2 4/4] mm/vmscan: Don't mess with pgdat->flags in memcg reclaim.

2018-04-05 Thread Shakeel Butt
On Fri, Mar 23, 2018 at 8:20 AM, Andrey Ryabinin wrote: > memcg reclaim may alter pgdat->flags based on the state of LRU lists > in cgroup and its children. PGDAT_WRITEBACK may force kswapd to sleep > congested_wait(), PGDAT_DIRTY may force kswapd to writeback filesystem

Re: [PATCH] mm-vmscan-dont-mess-with-pgdat-flags-in-memcg-reclaim-v2-fix

2018-04-06 Thread Shakeel Butt
On Fri, Apr 6, 2018 at 8:09 AM, Andrey Ryabinin <aryabi...@virtuozzo.com> wrote: > > > On 04/06/2018 05:37 PM, Shakeel Butt wrote: > >>> >>> @@ -2482,7 +2494,7 @@ static inline bool should_continue_reclaim(struct >>> pglist_data *pgdat, >>&g

Re: [PATCH v2 4/4] mm/vmscan: Don't mess with pgdat->flags in memcg reclaim.

2018-04-06 Thread Shakeel Butt
On Fri, Apr 6, 2018 at 4:44 AM, Andrey Ryabinin <aryabi...@virtuozzo.com> wrote: > On 04/06/2018 05:13 AM, Shakeel Butt wrote: >> On Fri, Mar 23, 2018 at 8:20 AM, Andrey Ryabinin >> <aryabi...@virtuozzo.com> wrote: >>> memcg reclaim may alter pgdat-&g

Re: [PATCH] mm-vmscan-dont-mess-with-pgdat-flags-in-memcg-reclaim-v2-fix

2018-04-06 Thread Shakeel Butt
On Fri, Apr 6, 2018 at 6:52 AM, Andrey Ryabinin <aryabi...@virtuozzo.com> wrote: > On 04/06/2018 05:13 AM, Shakeel Butt wrote: >> Question: Should this 'flags' be per-node? Is it ok for a congested >> memcg to call wait_iff_congested for all nodes? > > Indeed, congesti

[PATCH] Re: [PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-04-06 Thread Shakeel Butt
is simply hard to > justify. Maybe we can thing of something that would put the burden on > the charging context? What do you think of the following? Signed-off-by: Shakeel Butt <shake...@google.com> --- mm/memcontrol.c | 37 ++--- 1 file changed, 18

[PATCH v5 0/2] Directed kmem charging

2018-04-16 Thread Shakeel Butt
the target_memcg in task_struct in v3, added node variant for kmem allocation functions and rebased fsnotify patch over Jan's patches in v4 and in v5 fixed CONFIG_MEMCG=n build and removed the extra branch in the common case of memory allocation. Shakeel Butt (2): mm: memcg: remote memcg charging for kmem

[PATCH v5 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-04-16 Thread Shakeel Butt
instead of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Using kmalloc_memcg and kmem_cache_alloc_memcg implicitly assumes that the caller is requesting a __GFP_ACCOUNT allocation. Signed-off-by: Shakeel Butt <shake...@google.

[PATCH v5 2/2] fs: fsnotify: account fsnotify metadata to kmemcg

2018-04-16 Thread Shakeel Butt
the allocations to the listener's memcg. Thus we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel

[PATCH] slab, slub: skip unnecessary kasan_cache_shutdown()

2018-03-27 Thread Shakeel Butt
n there should not be any objects of this cache in the quarantine. Without the patch the script got stuck for couple of hours. With the patch the script completed within a second. Signed-off-by: Shakeel Butt <shake...@google.com> --- mm/kasan/kasan.c | 3 ++- mm/slab.c| 12 +

Re: [PATCH] slab, slub: skip unnecessary kasan_cache_shutdown()

2018-03-27 Thread Shakeel Butt
On Tue, Mar 27, 2018 at 5:16 PM, David Rientjes <rient...@google.com> wrote: > On Tue, 27 Mar 2018, Shakeel Butt wrote: > >> diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c >> index 49fffb0ca83b..135ce2838c89 100644 >> --- a/mm/kasan/kasan.c >> +++ b/mm/kasan

Re: [PATCH] mm, slab: eagerly delete inactive offlined SLABs

2018-03-26 Thread Shakeel Butt
> comments inline. > > On Wed, Mar 21, 2018 at 03:43:01PM -0700, Shakeel Butt wrote: >> With kmem cgroup support, high memcgs churn can leave behind a lot of >> empty kmem_caches. Usually such kmem_caches will be destroyed when the >> corresponding memcg gets released but th

SIGSEGV on OSPKE machine

2018-03-16 Thread Shakeel Butt
Hi all, The following simple program is producing SIGSEGV on machines which have X86_FEATURE_OSPKE feature on 4.15 kernel. #include int main(int argc, char *argv[]) { void *p = mmap(0, 4096, PROT_EXEC, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); mprotect(p, 4096,

[PATCH] mm, slab: memcg_link the SLAB's kmem_cache

2018-03-19 Thread Shakeel Butt
75c ("slab: implement slab_root_caches list") Signed-off-by: Shakeel Butt <shake...@google.com> --- mm/slab.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/slab.c b/mm/slab.c index 324446621b3e..9095c3945425 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1283,6 +1283,7 @@ void _

[PATCH] mm, slab: eagerly delete inactive offlined SLABs

2018-03-21 Thread Shakeel Butt
destroy such empty offlined kmem_caches. Signed-off-by: Shakeel Butt <shake...@google.com> --- mm/slab.c| 18 -- mm/slab.h| 15 +++ mm/slab_common.c | 2 +- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/mm/slab.c b/mm/slab.c

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-03-23 Thread Shakeel Butt
execute-only > permissions, but the VMA has the execute-only pkey. > > We had a check for PROT_READ/WRITE, but it did not work > for PROT_NONE. This entirely removes the PROT_* checks, > which ensures that PROT_NONE now works. > > Reported-by: Shakeel Butt <shake...@google.com> >

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-03-23 Thread Shakeel Butt
On Fri, Mar 23, 2018 at 12:23 PM, Dave Hansen <dave.han...@intel.com> wrote: > On 03/23/2018 12:15 PM, Shakeel Butt wrote: >>> We had a check for PROT_READ/WRITE, but it did not work >>> for PROT_NONE. This entirely removes the PROT_* checks, >>> wh

[PATCH v4 2/2] fs: fsnotify: account fsnotify metadata to kmemcg

2018-03-05 Thread Shakeel Butt
the allocations to the listener's memcg. Thus we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel Butt

[PATCH v4 0/2] Directed kmem charging

2018-03-05 Thread Shakeel Butt
the target_memcg in task_struct in v3 and in v4, added node variant for kmem allocation functions and rebased fsnotify patch over Jan's patches. Shakeel Butt (2): mm: memcg: remote memcg charging for kmem allocations fs: fsnotify: account fsnotify metadata to kmemcg fs/notify/dnotify/dnotify.c

[PATCH v4 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-03-05 Thread Shakeel Butt
instead of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Using kmalloc_memcg and kmem_cache_alloc_memcg implicitly assumes that the caller is requesting a __GFP_ACCOUNT allocation. Signed-off-by: Shakeel Butt <shake...@google.

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-04-25 Thread Shakeel Butt
execute-only > permissions, but the VMA has the execute-only pkey. > > We had a check for PROT_READ/WRITE, but it did not work > for PROT_NONE. This entirely removes the PROT_* checks, > which ensures that PROT_NONE now works. > > Reported-by: Shakeel Butt <shake...@google.com>

Re: [PATCH] slub: track number of slabs irrespective of CONFIG_SLUB_DEBUG

2018-06-21 Thread Shakeel Butt
On Wed, Jun 20, 2018 at 6:15 PM Christopher Lameter wrote: > > On Wed, 20 Jun 2018, Shakeel Butt wrote: > > > For !CONFIG_SLUB_DEBUG, SLUB does not maintain the number of slabs > > allocated per node for a kmem_cache. Thus, slabs_node() in > > __kmem_cache_e

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

2018-06-28 Thread Shakeel Butt
On Thu, Jun 28, 2018 at 12:03 PM Jan Kara wrote: > > On Wed 27-06-18 12:12:49, Shakeel Butt wrote: > > A lot of memory can be consumed by the events generated for the huge or > > unlimited queues if there is either no or slow listener. This can cause > > system level memo

[PATCH v2] kvm, mm: account shadow page tables to kmemcg

2018-06-29 Thread Shakeel Butt
The size of kvm's shadow page tables corresponds to the size of the guest virtual machines on the system. Large VMs can spend a significant amount of memory as shadow page tables which can not be left as system memory overhead. So, account shadow page tables to the kmemcg. Signed-off-by: Shakeel

Re: [PATCH] mm: handle no memcg case in memcg_kmem_charge() properly

2018-10-29 Thread Shakeel Butt
On Mon, Oct 29, 2018 at 2:52 PM Roman Gushchin wrote: > > Mike Galbraith reported a regression caused by the commit 9b6f7e163cd0 > ("mm: rework memcg kernel stack accounting") on a system with > "cgroup_disable=memory" boot option: the system panics with the > following stack trace: > >

Re: [PATCH] mm: handle no memcg case in memcg_kmem_charge() properly

2018-10-29 Thread Shakeel Butt
On Mon, Oct 29, 2018 at 6:01 PM Rik van Riel wrote: > > On Mon, 2018-10-29 at 17:50 -0700, Shakeel Butt wrote: > > On Mon, Oct 29, 2018 at 2:52 PM Roman Gushchin wrote: > > > > > > Mike Galbraith reported a regression caused by the commit > > > 9b6f7e163cd

Re: [PATCH] mm: handle no memcg case in memcg_kmem_charge() properly

2018-10-30 Thread Shakeel Butt
On Tue, Oct 30, 2018 at 8:56 AM Roman Gushchin wrote: > > On Tue, Oct 30, 2018 at 07:12:49AM +0100, Michal Hocko wrote: > > On Mon 29-10-18 21:51:55, Roman Gushchin wrote: > > > Mike Galbraith reported a regression caused by the commit 9b6f7e163cd0 > > > ("mm: rework memcg kernel stack

[PATCH] memcg: remove memcg_kmem_skip_account

2018-09-18 Thread Shakeel Butt
The flag memcg_kmem_skip_account was added during the era of opt-out kmem accounting. There is no need for such flag in the opt-in world as there aren't any __GFP_ACCOUNT allocations within memcg_create_cache_enqueue(). Signed-off-by: Shakeel Butt --- include/linux/sched.h | 3 --- mm

[PATCH] slab, slub: remove size disparity on debug kernel

2018-03-13 Thread Shakeel Butt
, the non-root slab's size is measured based on the root's object_size and thus the size will remain same for root and non-root slab. This patch makes slab's object_size the default base to measure the slab's size. Signed-off-by: Shakeel Butt --- mm/slab_common.c | 9 - 1 file changed, 4

Re: [PATCH] slab, slub: remove size disparity on debug kernel

2018-03-13 Thread Shakeel Butt
On Tue, Mar 13, 2018 at 10:19 AM, Christopher Lameter wrote: > On Tue, 13 Mar 2018, Shakeel Butt wrote: > >> However for SLUB in debug kernel, the sizes were same. On further >> inspection it is found that SLUB always use kmem_cache.object_size to >> measure the kmem_c

Re: [PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-03-13 Thread Shakeel Butt
On Tue, Mar 13, 2018 at 6:49 AM, Michal Hocko wrote: > On Wed 21-02-18 14:37:56, Shakeel Butt wrote: > [...] >> +#ifdef CONFIG_MEMCG >> +static inline struct mem_cgroup *memalloc_memcg_save(struct mem_cgroup >> *memcg) >> +{ >> + struct mem_cgrou

Re: [PATCH] slab, slub: remove size disparity on debug kernel

2018-03-14 Thread Shakeel Butt
On Wed, Mar 14, 2018 at 1:43 AM, Vladimir Davydov wrote: > On Tue, Mar 13, 2018 at 10:36:52AM -0700, Shakeel Butt wrote: >> On Tue, Mar 13, 2018 at 10:19 AM, Christopher Lameter wrote: >> > On Tue, 13 Mar 2018, Shakeel Butt wrote: >> > >> >> However for SL

Re: [PATCH] mm, slab: eagerly delete inactive offlined SLABs

2018-03-26 Thread Shakeel Butt
On Wed, Mar 21, 2018 at 03:43:01PM -0700, Shakeel Butt wrote: >> With kmem cgroup support, high memcgs churn can leave behind a lot of >> empty kmem_caches. Usually such kmem_caches will be destroyed when the >> corresponding memcg gets released but the memcg release can be >>

[PATCH] slab, slub: skip unnecessary kasan_cache_shutdown()

2018-03-27 Thread Shakeel Butt
n there should not be any objects of this cache in the quarantine. Without the patch the script got stuck for couple of hours. With the patch the script completed within a second. Signed-off-by: Shakeel Butt --- mm/kasan/kasan.c | 3 ++- mm/slab.c| 12 mm/slab.h| 1 + mm/

Re: [PATCH] slab, slub: skip unnecessary kasan_cache_shutdown()

2018-03-27 Thread Shakeel Butt
On Tue, Mar 27, 2018 at 5:16 PM, David Rientjes wrote: > On Tue, 27 Mar 2018, Shakeel Butt wrote: > >> diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c >> index 49fffb0ca83b..135ce2838c89 100644 >> --- a/mm/kasan/kasan.c >> +++ b/mm/kasan/kasan.c >> @@ -382

[RFC PATCH 0/3] Directed kmem charging

2018-02-13 Thread Shakeel Butt
series introduces kmem allocation functions where the caller can pass the pointer to the remote memcg. The remote memcg will be charged for the allocation instead of the memcg of the caller. Shakeel Butt (3): mm: memcg: plumbing memcg for kmem cache allocations mm: memcg: plumbing memcg for kmalloc

[RFC PATCH 3/3] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-13 Thread Shakeel Butt
we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel Butt --- fs/notify/dnotify/dnotify.c

[RFC PATCH 1/3] mm: memcg: plumbing memcg for kmem cache allocations

2018-02-13 Thread Shakeel Butt
of the caller. One such concrete use-case is the allocations for fsnotify event objects where the objects should be charged to the listener instead of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Signed-off-by: Shakeel Butt

[RFC PATCH 2/3] mm: memcg: plumbing memcg for kmalloc allocations

2018-02-13 Thread Shakeel Butt
to page allocator. So, for __GFP_ACCOUNT kmalloc allocations, the memcg of current task is charged. This patch introduces memcg variant of kmalloc functions to allow callers to provide memcg for charging. Signed-off-by: Shakeel Butt --- include/linux/memcontrol.h | 3 +- include/linux/slab.h

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-03-23 Thread Shakeel Butt
> > We had a check for PROT_READ/WRITE, but it did not work > for PROT_NONE. This entirely removes the PROT_* checks, > which ensures that PROT_NONE now works. > > Reported-by: Shakeel Butt > > Signed-off-by: Dave Hansen Should there be a 'Fixes' tag? Also should this patch go to

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-03-23 Thread Shakeel Butt
On Fri, Mar 23, 2018 at 12:23 PM, Dave Hansen wrote: > On 03/23/2018 12:15 PM, Shakeel Butt wrote: >>> We had a check for PROT_READ/WRITE, but it did not work >>> for PROT_NONE. This entirely removes the PROT_* checks, >>> which ensures that PROT_NONE now works. >

[PATCH] mm, slab: eagerly delete inactive offlined SLABs

2018-03-21 Thread Shakeel Butt
destroy such empty offlined kmem_caches. Signed-off-by: Shakeel Butt --- mm/slab.c| 18 -- mm/slab.h| 15 +++ mm/slab_common.c | 2 +- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 66f2db98f026

Re: [PATCH] mm: memcg: expose mem_cgroup_put API

2018-03-08 Thread Shakeel Butt
On Thu, Mar 8, 2018 at 3:45 PM, Andrew Morton wrote: > On Wed, 7 Mar 2018 18:48:50 -0800 Shakeel Butt wrote: > >> This patch exports mem_cgroup_put API to put the refcnt of the memory >> cgroup. > > OK, I remember now. This is intended to make > fs-fsnotif

Re: [PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-03-15 Thread Shakeel Butt
On Thu, Mar 15, 2018 at 10:49 AM, Michal Hocko wrote: > On Tue 13-03-18 10:55:18, Shakeel Butt wrote: >> On Tue, Mar 13, 2018 at 6:49 AM, Michal Hocko wrote: >> > On Wed 21-02-18 14:37:56, Shakeel Butt wrote: >> > [...] >> >> +#ifdef CONFIG_MEMCG

Re: [PATCH 1/6] mm/vmscan: Wake up flushers for legacy cgroups too

2018-03-15 Thread Shakeel Butt
1052kB, > file-rss:1720kB, shmem-rss:0kB > oom_reaper: reaped process 3876 (dd), now anon-rss:0kB, file-rss:0kB, > shmem-rss:0kB > > Wake up flushers in legacy cgroup reclaim too. > > Fixes: bbef938429f5 ("mm: vmscan: remove old flusher wakeup from direct > reclaim pa

SIGSEGV on OSPKE machine

2018-03-16 Thread Shakeel Butt
Hi all, The following simple program is producing SIGSEGV on machines which have X86_FEATURE_OSPKE feature on 4.15 kernel. #include int main(int argc, char *argv[]) { void *p = mmap(0, 4096, PROT_EXEC, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); mprotect(p, 4096,

Re: [PATCH v2] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-20 Thread Shakeel Butt
On Tue, Feb 20, 2018 at 4:43 AM, Jan Kara wrote: > On Mon 19-02-18 21:07:28, Amir Goldstein wrote: >> On Mon, Feb 19, 2018 at 3:50 PM, Jan Kara wrote: >> [...] >> > For fanotify without FAN_UNLIMITED_QUEUE the situation is similar as for >> > inotify - IMO low practical impact, apps should

[PATCH 3/3] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-20 Thread Shakeel Butt
the allocations to the listener's memcg. Thus we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel Butt

[PATCH 1/3] mm: memcg: plumbing memcg for kmem cache allocations

2018-02-20 Thread Shakeel Butt
of the caller. One such concrete use-case is the allocations for fsnotify event objects where the objects should be charged to the listener instead of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Signed-off-by: Shakeel Butt

[PATCH 2/3] mm: memcg: plumbing memcg for kmalloc allocations

2018-02-20 Thread Shakeel Butt
to page allocator. So, for __GFP_ACCOUNT kmalloc allocations, the memcg of current task is charged. This patch introduces memcg variant of kmalloc functions to allow callers to provide memcg for charging. Signed-off-by: Shakeel Butt --- include/linux/memcontrol.h | 3 +- include/linux/slab.h

[PATCH 0/3] Directed kmem charging

2018-02-20 Thread Shakeel Butt
series introduces kmem allocation functions where the caller can pass the pointer to the remote memcg. The remote memcg will be charged for the allocation instead of the memcg of the caller. However the caller must have a reference to the remote memcg. Shakeel Butt (3): mm: memcg: plumbing memcg

Re: [PATCH 3/3] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-20 Thread Shakeel Butt
On Tue, Feb 20, 2018 at 11:41 AM, Shakeel Butt wrote: > A lot of memory can be consumed by the events generated for the huge or > unlimited queues if there is either no or slow listener. This can cause > system level memory pressure or OOMs. So, it's better to account the > fsnotify

[PATCH v2 0/3] Directed kmem charging

2018-02-20 Thread Shakeel Butt
series introduces kmem allocation functions where the caller can pass the pointer to the remote memcg. The remote memcg will be charged for the allocation instead of the memcg of the caller. However the caller must have a reference to the remote memcg. Fixed the build for SLOB in v2. Shakeel Butt (3

[PATCH v2 3/3] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-20 Thread Shakeel Butt
the allocations to the listener's memcg. Thus we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel Butt

[PATCH v2 2/3] mm: memcg: plumbing memcg for kmalloc allocations

2018-02-20 Thread Shakeel Butt
to page allocator. So, for __GFP_ACCOUNT kmalloc allocations, the memcg of current task is charged. This patch introduces memcg variant of kmalloc functions to allow callers to provide memcg for charging. Signed-off-by: Shakeel Butt --- Changelog since v1: - Fixed build for SLOB include/linux

[PATCH v2 1/3] mm: memcg: plumbing memcg for kmem cache allocations

2018-02-20 Thread Shakeel Butt
of the caller. One such concrete use-case is the allocations for fsnotify event objects where the objects should be charged to the listener instead of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Signed-off-by: Shakeel Butt

Re: [PATCH v2 0/3] Directed kmem charging

2018-02-21 Thread Shakeel Butt
On Wed, Feb 21, 2018 at 8:09 AM, Christopher Lameter wrote: > Another way to solve this is to switch the user context right? > > Isnt it possible to avoid these patches if do the allocation in another > task context instead? > Sorry, can you please explain what you mean by 'switch the user

Re: [PATCH v2 0/3] Directed kmem charging

2018-02-21 Thread Shakeel Butt
On Wed, Feb 21, 2018 at 9:57 AM, Christopher Lameter wrote: > On Wed, 21 Feb 2018, Shakeel Butt wrote: > >> On Wed, Feb 21, 2018 at 8:09 AM, Christopher Lameter wrote: >> > Another way to solve this is to switch the user context right? >> > >> > Isnt i

[PATCH v3 1/2] mm: memcg: remote memcg charging for kmem allocations

2018-02-21 Thread Shakeel Butt
of the producer. One requirement to call these functions is that the caller must have the reference to the memcg. Using kmalloc_memcg and kmem_cache_alloc_memcg implicitly assumes that the caller is requesting a __GFP_ACCOUNT allocation. Signed-off-by: Shakeel Butt --- Changelog since v2: - Merge

[PATCH v3 0/2] Directed kmem charging

2018-02-21 Thread Shakeel Butt
the target_memcg in task_struct in v3. Shakeel Butt (2): mm: memcg: remote memcg charging for kmem allocations fs: fsnotify: account fsnotify metadata to kmemcg fs/notify/dnotify/dnotify.c | 5 +++-- fs/notify/fanotify/fanotify.c| 12 ++- fs/notify/fanotify/fanotify.h

[PATCH v3 2/2] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-21 Thread Shakeel Butt
the allocations to the listener's memcg. Thus we save the memcg reference in the fsnotify_group structure of the listener. This patch has also moved the members of fsnotify_group to keep the size same, at least for 64 bit build, even with additional member by filling the holes. Signed-off-by: Shakeel Butt

Re: [PATCH v2 3/3] fs: fsnotify: account fsnotify metadata to kmemcg

2018-02-22 Thread Shakeel Butt
On Thu, Feb 22, 2018 at 6:48 AM, Jan Kara wrote: > On Thu 22-02-18 14:49:44, Michal Hocko wrote: >> On Tue 20-02-18 19:01:01, Shakeel Butt wrote: >> > A lot of memory can be consumed by the events generated for the huge or >> > unlimited queues if there is either no

Re: [PATCH 4/9] x86, pkeys: override pkey when moving away from PROT_EXEC

2018-04-25 Thread Shakeel Butt
> > We had a check for PROT_READ/WRITE, but it did not work > for PROT_NONE. This entirely removes the PROT_* checks, > which ensures that PROT_NONE now works. > > Reported-by: Shakeel Butt > > Signed-off-by: Dave Hansen > Fixes: 62b5f7d013f ("mm/core, x86/mm/pkeys: Add

[PATCH] mm: memcontrol: drain stocks on resize limit

2018-05-04 Thread Shakeel Butt
Resizing the memcg limit for cgroup-v2 drains the stocks before triggering the memcg reclaim. Do the same for cgroup-v1 to make the behavior consistent. Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c

[PATCH] mm: memcontrol: drain memcg stock on force_empty

2018-05-07 Thread Shakeel Butt
-by: Shakeel Butt --- mm/memcontrol.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e2d33a37f971..2c3c69524b49 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2841,6 +2841,9 @@ static int mem_cgroup_force_empty(struct mem_cgroup *memcg

Re: [PATCH] mm: memcontrol: drain memcg stock on force_empty

2018-05-07 Thread Shakeel Butt
On Mon, May 7, 2018 at 1:16 PM Shakeel Butt wrote: > From: Junaid Shahid > The per-cpu memcg stock can retain a charge of upto 32 pages. On a > machine with large number of cpus, this can amount to a decent amount > of memory. Additionally force_empty interface might be triggerin

Re: [PATCH v4 01/13] mm: Assign id to every memcg-aware shrinker

2018-05-09 Thread Shakeel Butt
On Wed, May 9, 2018 at 3:55 PM Andrew Morton wrote: > On Wed, 09 May 2018 14:56:55 +0300 Kirill Tkhai wrote: > > The patch introduces shrinker::id number, which is used to enumerate > > memcg-aware shrinkers. The number start from 0, and the code tries > > to maintain it as small as possible.

[PATCH] mm: save two stranding bit in gfp_mask

2018-05-16 Thread Shakeel Butt
___GFP_COLD and ___GFP_OTHER_NODE were removed but their bits were stranded. Slide existing gfp masks to make those two bits available. Signed-off-by: Shakeel Butt --- include/linux/gfp.h | 42 +- 1 file changed, 21 insertions(+), 21 deletions(-) diff

[PATCH v2] mm: save two stranding bit in gfp_mask

2018-05-16 Thread Shakeel Butt
___GFP_COLD and ___GFP_OTHER_NODE were removed but their bits were stranded. Fill the gaps by moving the existing gfp masks around. Signed-off-by: Shakeel Butt Suggested-by: Vlastimil Babka Acked-by: Michal Hocko --- Changelog since v1: - Moved couple of gfp masks instead of sliding all

<    1   2   3   4   5   6   7   8   9   10   >