[PATCH] mm: memcontrol: account kernel stack per node

2020-06-29 Thread Shakeel Butt
-by: Shakeel Butt --- drivers/base/node.c| 4 ++-- fs/proc/meminfo.c | 4 ++-- include/linux/memcontrol.h | 2 -- include/linux/mmzone.h | 8 kernel/fork.c | 29 ++--- kernel/scs.c | 2 +- mm/memcontrol.c

Re: [PATCH] mm: memcontrol: account kernel stack per node

2020-06-30 Thread Shakeel Butt
On Mon, Jun 29, 2020 at 8:24 PM Roman Gushchin wrote: > > On Mon, Jun 29, 2020 at 05:44:13PM -0700, Shakeel Butt wrote: > > Currently the kernel stack is being accounted per-zone. There is no need > > to do that. In addition due to being per-zone, memcg has to k

[PATCH v2] mm: memcontrol: account kernel stack per node

2020-06-30 Thread Shakeel Butt
localize the kernel stack stats updates to account_kernel_stack(). Signed-off-by: Shakeel Butt --- Changes since v1: - Use lruvec based stat update functions based on Roman's suggestion. drivers/base/node.c| 4 +-- fs/proc/meminfo.c | 4 +-- include/linux/memcontrol.h

Re: [RFC][PATCH 0/8] Migrate Pages in lieu of discard

2020-06-30 Thread Shakeel Butt
On Mon, Jun 29, 2020 at 4:48 PM Dave Hansen wrote: > > I've been sitting on these for too long. Tha main purpose of this > post is to have a public discussion with the other folks who are > interested in this functionalty and converge on a single > implementation. > > This set directly incorporat

Re: [PATCH] mm: filemap: clear idle flag for writes

2020-06-24 Thread Shakeel Butt
clear idle flag for write operations. > > Fixes: bbddabe2e436 ("mm: filemap: only do access activations on reads") > Cc: Johannes Weiner > Cc: Rik van Riel > Cc: Shakeel Butt > Reported-by: Gang Deng > Signed-off-by: Yang Shi Reviewed-by: Shakeel Butt

Re: [PATCH] mm: filemap: clear idle flag for writes

2020-06-24 Thread Shakeel Butt
On Wed, Jun 24, 2020 at 12:18 PM Yang Shi wrote: > > > > On 6/24/20 11:53 AM, Andrew Morton wrote: > > On Thu, 25 Jun 2020 01:43:32 +0800 Yang Shi > > wrote: > > > >> Since commit bbddabe2e436aa7869b3ac5248df5c14ddde0cbf ("mm: filemap: > >> only do access activations on reads"), mark_page_access

Re: [RFC PROPOSAL] memcg: per-memcg user space reclaim interface

2020-07-07 Thread Shakeel Butt
On Mon, Jul 6, 2020 at 2:38 PM Roman Gushchin wrote: > > On Fri, Jul 03, 2020 at 09:27:19AM -0700, Shakeel Butt wrote: > > On Fri, Jul 3, 2020 at 8:50 AM Roman Gushchin wrote: > > > > > > On Fri, Jul 03, 2020 at 07:23:14AM -0700, Shakeel Butt wrote: > > > &

Re: [RFC PROPOSAL] memcg: per-memcg user space reclaim interface

2020-07-07 Thread Shakeel Butt
On Tue, Jul 7, 2020 at 5:14 AM Michal Hocko wrote: > > On Fri 03-07-20 07:23:14, Shakeel Butt wrote: > > On Thu, Jul 2, 2020 at 11:35 PM Michal Hocko wrote: > > > > > > On Thu 02-07-20 08:22:22, Shakeel Butt wrote: > &g

Re: [PATCH 1/3] mm: memcg/slab: remove unused argument by charge_slab_page()

2020-07-07 Thread Shakeel Butt
On Tue, Jul 7, 2020 at 10:36 AM Roman Gushchin wrote: > > charge_slab_page() is not using the gfp argument anymore, > remove it. > > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH 2/3] mm: slab: rename (un)charge_slab_page() to (un)account_slab_page()

2020-07-07 Thread Shakeel Butt
_page() respectively. > > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH 3/3] mm: kmem: switch to static_branch_likely() in memcg_kmem_enabled()

2020-07-07 Thread Shakeel Butt
s accounting significantly exceeds the > cost of a jump. However, the conversion makes the code look more > logically. > > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH v6 05/19] mm: memcontrol: decouple reference counting from page accounting

2020-06-18 Thread Shakeel Butt
Not sure if my email went through, so, re-sending. On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > From: Johannes Weiner > [...] > @@ -3003,13 +3004,16 @@ void __memcg_kmem_uncharge_page(struct page *page, > int order) > */ > void mem_cgroup_split_huge_fixup(struct page *head) > {

Re: [PATCH v6 05/19] mm: memcontrol: decouple reference counting from page accounting

2020-06-18 Thread Shakeel Butt
On Thu, Jun 18, 2020 at 6:08 PM Roman Gushchin wrote: > > On Thu, Jun 18, 2020 at 07:55:35AM -0700, Shakeel Butt wrote: > > Not sure if my email went through, so, re-sending. > > > > On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > >

Re: [PATCH v6 05/19] mm: memcontrol: decouple reference counting from page accounting

2020-06-18 Thread Shakeel Butt
ss_get() in >mem_cgroup_charge(), dropped mem_cgroup_try_charge() part > 2) I've reformatted commit references in the commit log to make >checkpatch.pl happy. > > Signed-off-by: Johannes Weiner > Signed-off-by: Roman Gushchin > Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH v6 06/19] mm: memcg/slab: obj_cgroup API

2020-06-19 Thread Shakeel Butt
On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > Obj_cgroup API provides an ability to account sub-page sized kernel > objects, which potentially outlive the original memory cgroup. > > The top-level API consists of the following functions: > bool obj_cgroup_tryget(struct obj_cgroup *obj

Re: [PATCH v6 12/19] mm: memcg/slab: use a single set of kmem_caches for all accounted allocations

2020-06-22 Thread Shakeel Butt
-off-by: Roman Gushchin > Reviewed-by: Vlastimil Babka This is a very satisfying patch. Reviewed-by: Shakeel Butt

Re: [PATCH v16 00/14] Introduce Data Access MONitor (DAMON)

2020-06-22 Thread Shakeel Butt
Hi SeongJae, On Mon, Jun 22, 2020 at 1:42 AM SeongJae Park wrote: > > Last week, this patchset received 5 'Reviewed-by' tags, but no further > comments > for changes. I updated the documentation, but the change is only small. For > the reason, I'm only asking more reviews rather than posting t

Re: [PATCH v6 10/19] mm: memcg/slab: deprecate memory.kmem.slabinfo

2020-06-22 Thread Shakeel Butt
On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > Deprecate memory.kmem.slabinfo. > > An empty file will be presented if corresponding config options are > enabled. > > The interface is implementation dependent, isn't present in cgroup v2, > and is generally useful only for core mm debuggin

Re: [PATCH v6 13/19] mm: memcg/slab: simplify memcg cache creation

2020-06-22 Thread Shakeel Butt
On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > Because the number of non-root kmem_caches doesn't depend on the > number of memory cgroups anymore and is generally not very big, > there is no more need for a dedicated workqueue. > > Also, as there is no more need to pass any arguments to

Re: [PATCH v6 16/19] mm: memcg/slab: remove redundant check in memcg_accumulate_slabinfo()

2020-06-22 Thread Shakeel Butt
hin > Reviewed-by: Vlastimil Babka Reviewed-by: Shakeel Butt

Re: [PATCH v6 15/19] mm: memcg/slab: deprecate slab_root_caches

2020-06-22 Thread Shakeel Butt
kmem_cache structure. > > Signed-off-by: Roman Gushchin > Reviewed-by: Vlastimil Babka Reviewed-by: Shakeel Butt

Re: [PATCH v6 13/19] mm: memcg/slab: simplify memcg cache creation

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 10:40 AM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 10:29:29AM -0700, Shakeel Butt wrote: > > On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > > > > > Because the number of non-root kmem_caches doesn't depend on the >

Re: [PATCH v6 10/19] mm: memcg/slab: deprecate memory.kmem.slabinfo

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 11:02 AM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 10:12:46AM -0700, Shakeel Butt wrote: > > On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > > > > > Deprecate memory.kmem.slabinfo. > > > > > > An empt

Re: [PATCH v6 10/19] mm: memcg/slab: deprecate memory.kmem.slabinfo

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 11:25 AM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 11:09:47AM -0700, Shakeel Butt wrote: > > On Mon, Jun 22, 2020 at 11:02 AM Roman Gushchin wrote: > > > > > > On Mon, Jun 22, 2020 at 10:12:46AM -0700, Shakeel Butt wrote: > >

Re: [PATCH v6 14/19] mm: memcg/slab: remove memcg_kmem_get_cache()

2020-06-22 Thread Shakeel Butt
to generate a better code. > > Signed-off-by: Roman Gushchin > Reviewed-by: Vlastimil Babka Reviewed-by: Shakeel Butt

Re: [PATCH v6 17/19] mm: memcg/slab: use a single set of kmem_caches for all allocations

2020-06-22 Thread Shakeel Butt
On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > Instead of having two sets of kmem_caches: one for system-wide and > non-accounted allocations and the second one shared by all accounted > allocations, we can use just one. > > The idea is simple: space for obj_cgroup metadata can be alloca

Re: [PATCH v6 17/19] mm: memcg/slab: use a single set of kmem_caches for all allocations

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 1:37 PM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 12:21:28PM -0700, Shakeel Butt wrote: > > On Mon, Jun 8, 2020 at 4:07 PM Roman Gushchin wrote: > > > > > > Instead of having two sets of kmem_caches: one for system-wide and > >

Re: [PATCH v6 17/19] mm: memcg/slab: use a single set of kmem_caches for all allocations

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 2:15 PM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 02:04:29PM -0700, Shakeel Butt wrote: > > On Mon, Jun 22, 2020 at 1:37 PM Roman Gushchin wrote: > > > > > > On Mon, Jun 22, 2020 at 12:21:28PM -0700, Shakeel Butt wrote: > > >

Re: [PATCH v6 17/19] mm: memcg/slab: use a single set of kmem_caches for all allocations

2020-06-22 Thread Shakeel Butt
On Mon, Jun 22, 2020 at 2:58 PM Roman Gushchin wrote: > > On Mon, Jun 22, 2020 at 02:28:54PM -0700, Shakeel Butt wrote: > > On Mon, Jun 22, 2020 at 2:15 PM Roman Gushchin wrote: > > > > > > On Mon, Jun 22, 2020 at 02:04:29PM -0700, Shakeel Butt wrote: > >

Re: [PATCH tip/core/rcu 02/26] mm/mmap.c: Add cond_resched() for exit_mmap() CPU stalls

2020-06-22 Thread Shakeel Butt
can run > for a very long time given a large process. This commit therefore adds > a cond_resched() to this loop, providing RCU any needed quiescent states. > > Cc: Andrew Morton > Cc: > Signed-off-by: Paul E. McKenney We have exactly the same change in our internal kernel s

Re: [PATCH v7 09/19] mm: memcg/slab: charge individual slab objects instead of pages

2020-06-22 Thread Shakeel Butt
_full_size(s))) { > + memcg_kmem_put_cache(cachep); I think you forgot to put obj_cgroup_put(*objcgp) here again. > + cachep = NULL; > + } > + > + return cachep; > +} > + After the above fix: Reviewed-by: Shakeel Butt

Re: [PATCH v7 17/19] mm: memcg/slab: use a single set of kmem_caches for all allocations

2020-06-22 Thread Shakeel Butt
> Suggested-by: Johannes Weiner > Signed-off-by: Roman Gushchin > Reviewed-by: Vlastimil Babka Reviewed-by: Shakeel Butt

Re: [PATCH v2 1/2] mm, memcg: reclaim more aggressively before high allocator throttling

2020-07-13 Thread Shakeel Butt
s the reclaim behaviour between > the two. > > There's precedent for this behaviour: we already do reclaim retries when > writing to memory.{high,max}, in max reclaim, and in the page allocator > itself. > > Signed-off-by: Chris Down > Cc: Andrew Morton > Cc: Johannes Weiner > Cc: Tejun Heo > Cc: Michal Hocko Reviewed-by: Shakeel Butt

Re: [PATCH] mm: memcontrol: avoid workload stalls when lowering memory.high

2020-07-14 Thread Shakeel Butt
On Tue, Jul 14, 2020 at 1:41 AM Michal Hocko wrote: > > On Fri 10-07-20 12:19:37, Shakeel Butt wrote: > > On Fri, Jul 10, 2020 at 11:42 AM Roman Gushchin wrote: > > > > > > On Fri, Jul 10, 2020 at 07:12:22AM -0700, Shakeel Butt wrote: > > > > On Fri, J

Re: [PATCH v5 1/4] mm: memcontrol: use helpers to read page's memcg data

2020-10-05 Thread Shakeel Butt
page's > mem_cgroup/obj_cgroups is converted to unsigned long memcg_data. > > Signed-off-by: Roman Gushchin > Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt

Re: [PATCH v5 2/4] mm: memcontrol/slab: use helpers to access slab page's memcg_data

2020-10-05 Thread Shakeel Butt
from a different thread. > > Signed-off-by: Roman Gushchin > Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt

Re: [PATCH v5 4/4] mm: convert page kmemcg type to a page memcg flag

2020-10-05 Thread Shakeel Butt
leted, as the whole memcg_data is zeroed at once. > > As a bonus, on !CONFIG_MEMCG build the PageMemcgKmem() check will > be compiled out. > > Signed-off-by: Roman Gushchin > Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt

Re: [PATCH] memcg: introduce per-memcg reclaim interface

2020-10-05 Thread Shakeel Butt
Hi Johannes, On Thu, Oct 1, 2020 at 8:12 AM Johannes Weiner wrote: > > Hello Shakeel, > > On Wed, Sep 30, 2020 at 08:26:26AM -0700, Shakeel Butt wrote: > > On Mon, Sep 28, 2020 at 2:03 PM Johannes Weiner wrote: > > > Workloads may not > > > allocate a

Re: [PATCH] memcg: introduce per-memcg reclaim interface

2020-10-06 Thread Shakeel Butt
On Thu, Oct 1, 2020 at 7:33 AM Johannes Weiner wrote: > [snip] > > >So instead of asking users for a target size whose suitability > > >heavily depends on the kernel's LRU implementation, the readahead > > >code, the IO device's capability and general load, why not directly > > >as

Re: [PATCH] memcg: introduce per-memcg reclaim interface

2020-09-30 Thread Shakeel Butt
comments below. > > On Wed, Sep 09, 2020 at 02:57:52PM -0700, Shakeel Butt wrote: > > Introduce an memcg interface to trigger memory reclaim on a memory cgroup. > > > > Use cases: > > -- > > > > 1) Per-memcg uswapd: > > > > Usually app

Re: [PATCH] memcg: introduce per-memcg reclaim interface

2020-09-30 Thread Shakeel Butt
On Tue, Sep 29, 2020 at 2:55 PM Johannes Weiner wrote: > > On Tue, Sep 29, 2020 at 05:04:44PM +0200, Michal Hocko wrote: > > On Mon 28-09-20 17:02:16, Johannes Weiner wrote: > > [...] > > > My take is that a proactive reclaim feature, whose goal is never to > > > thrash or punish but to keep the L

Re: [PATCH v3 3/4] mm: introduce page memcg flags

2020-09-30 Thread Shakeel Butt
it using page memcg flags, defined in enum > page_memcg_data_flags. > > Additional flags might be added later. > > Signed-off-by: Roman Gushchin One nit below: Reviewed-by: Shakeel Butt > --- > include/linux/memcontrol.h | 35 --- > 1 file c

Re: [PATCH bpf-next v4 02/30] bpf: memcg-based memory accounting for bpf progs

2020-08-25 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:01 AM Roman Gushchin wrote: > > Include memory used by bpf programs into the memcg-based accounting. > This includes the memory used by programs itself, auxiliary data > and statistics. > > Signed-off-by: Roman Gushchin > --- > kernel/bpf/core.c | 8 > 1 file c

Re: [PATCH bpf-next v4 03/30] bpf: memcg-based memory accounting for bpf maps

2020-08-25 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:01 AM Roman Gushchin wrote: > > This patch enables memcg-based memory accounting for memory allocated > by __bpf_map_area_alloc(), which is used by most map types for > large allocations. > > If a map is updated from an interrupt context, and the update > results in memor

Re: [PATCH bpf-next v4 04/30] bpf: refine memcg-based memory accounting for arraymap maps

2020-08-26 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:11 AM Roman Gushchin wrote: > > Include percpu arrays and auxiliary data into the memcg-based memory > accounting. > > Signed-off-by: Roman Gushchin > Acked-by: Song Liu > --- > kernel/bpf/arraymap.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > dif

Re: [PATCH bpf-next v4 05/30] bpf: refine memcg-based memory accounting for cpumap maps

2020-08-26 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:11 AM Roman Gushchin wrote: > > Include metadata and percpu data into the memcg-based memory accounting. > > Signed-off-by: Roman Gushchin > Acked-by: Song Liu Reviewed-by: Shakeel Butt

Re: [PATCH bpf-next v4 06/30] bpf: memcg-based memory accounting for cgroup storage maps

2020-08-26 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:17 AM Roman Gushchin wrote: > > Account memory used by cgroup storage maps including the percpu memory > for the percpu flavor of cgroup storage and map metadata. > > Signed-off-by: Roman Gushchin > Acked-by: Song Liu Reviewed-by: Shakeel Butt

Re: [PATCH bpf-next v4 07/30] bpf: refine memcg-based memory accounting for devmap maps

2020-08-26 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:18 AM Roman Gushchin wrote: > > Include map metadata and the node size (struct bpf_dtab_netdev) on > element update into the accounting. > > Signed-off-by: Roman Gushchin > Acked-by: Song Liu Reviewed-by: Shakeel Butt

Re: [PATCH] memcg: introduce per-memcg reclaim interface

2020-10-08 Thread Shakeel Butt
On Thu, Oct 8, 2020 at 7:55 AM Johannes Weiner wrote: > > On Tue, Oct 06, 2020 at 09:55:43AM -0700, Shakeel Butt wrote: > > On Thu, Oct 1, 2020 at 7:33 AM Johannes Weiner wrote: > > > > > [snip] > > > > >So instead of asking users for a target s

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 9:19 AM Ming Lei wrote: > > On Fri, Sep 25, 2020 at 03:31:45PM +0800, Ming Lei wrote: > > On Thu, Sep 24, 2020 at 09:13:11PM -0400, Theodore Y. Ts'o wrote: > > > On Thu, Sep 24, 2020 at 10:33:45AM -0400, Theodore Y. Ts'o wrote: > > > > HOWEVER, thanks to a hint from a colle

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 9:32 AM Shakeel Butt wrote: > > On Fri, Sep 25, 2020 at 9:19 AM Ming Lei wrote: > > > > On Fri, Sep 25, 2020 at 03:31:45PM +0800, Ming Lei wrote: > > > On Thu, Sep 24, 2020 at 09:13:11PM -0400, Theodore Y. Ts'o wrote: > > > &

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 10:17 AM Linus Torvalds wrote: > > On Fri, Sep 25, 2020 at 9:19 AM Ming Lei wrote: > > > > git bisect shows the first bad commit: > > > > [10befea91b61c4e2c2d1df06a2e978d182fcf792] mm: memcg/slab: use a > > single set of > > kmem_caches for all all

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 10:22 AM Shakeel Butt wrote: > > On Fri, Sep 25, 2020 at 10:17 AM Linus Torvalds > wrote: > > > > On Fri, Sep 25, 2020 at 9:19 AM Ming Lei wrote: > > > > > > git bisect shows the first bad commit: > > > > > >

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 10:48 AM Roman Gushchin wrote: > > On Fri, Sep 25, 2020 at 10:35:03AM -0700, Shakeel Butt wrote: > > On Fri, Sep 25, 2020 at 10:22 AM Shakeel Butt wrote: > > > > > > On Fri, Sep 25, 2020 at 10:17 AM Linus Torvalds > > > wrote: >

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 10:58 AM Shakeel Butt wrote: > [snip] > > I don't think you can ignore the flushing. The __free_once() in > ___cache_free() assumes there is a space available. > > BTW do_drain() also have the same issue. > > Why not move slabs_destroy()

Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag

2020-09-25 Thread Shakeel Butt
On Fri, Sep 25, 2020 at 1:56 PM Roman Gushchin wrote: > > On Fri, Sep 25, 2020 at 12:19:02PM -0700, Shakeel Butt wrote: > > On Fri, Sep 25, 2020 at 10:58 AM Shakeel Butt > > wrote: > > > > > [snip] > > > > > > I don't think you can ignor

[PATCH] mm: slab: fix potential double free in ___cache_free

2020-09-26 Thread Shakeel Butt
local CPU array_cache cache before calling slabs_destroy(). Fixes: 10befea91b61 ("mm: memcg/slab: use a single set of kmem_caches for all allocations") Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin Tested-by: Ming Lei Reported-by: kernel test robot --- mm/slab.c | 8 ++-

Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling

2020-05-28 Thread Shakeel Butt
I haven't gone through the whole email-chain, so, I might be asking some repetitive questions. I will go through the email-chain later. On Wed, May 20, 2020 at 7:37 AM Chris Down wrote: > > In Facebook production, we've seen cases where cgroups have been put > into allocator throttling even when

Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling

2020-05-28 Thread Shakeel Butt
On Thu, May 28, 2020 at 1:30 PM Johannes Weiner wrote: > > On Thu, May 28, 2020 at 08:48:31PM +0100, Chris Down wrote: > > Shakeel Butt writes: > > > What was the initial reason to have different behavior in the first place? > > > > This differing behaviour

[PATCH 1/3] mm: swap: fix vmstats for huge pages

2020-05-08 Thread Shakeel Butt
Many of the callbacks called by pagevec_lru_move_fn() do not correctly update the vmstats for huge pages. Fix that. Also __pagevec_lru_add_fn() use the irq-unsafe alternative to update the stat as the irqs are already disabled. Signed-off-by: Shakeel Butt --- mm/swap.c | 14 -- 1

[PATCH 2/3] mm: swap: memcg: fix memcg stats for huge pages

2020-05-08 Thread Shakeel Butt
ready disabled. Fixes: 2262185c5b28 ("mm: per-cgroup memory reclaim stats") Signed-off-by: Shakeel Butt --- mm/swap.c | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 3dbef6517cac..4eb179ee0b72 100644 --- a/mm/swap.c +++ b/mm

[PATCH 3/3] mm: swap: fix update_page_reclaim_stat for huge pages

2020-05-08 Thread Shakeel Butt
Currently update_page_reclaim_stat() updates the lruvec.reclaim_stats just once for a page irrespective if a page is huge or not. Fix that by passing the hpage_nr_pages(page) to it. Signed-off-by: Shakeel Butt --- mm/swap.c | 20 ++-- 1 file changed, 10 insertions(+), 10

Re: [PATCH 3/3] mm: swap: fix update_page_reclaim_stat for huge pages

2020-05-09 Thread Shakeel Butt
On Fri, May 8, 2020 at 2:51 PM Johannes Weiner wrote: > > On Fri, May 08, 2020 at 02:22:15PM -0700, Shakeel Butt wrote: > > Currently update_page_reclaim_stat() updates the lruvec.reclaim_stats > > just once for a page irrespective if a page is huge or not. Fix that

Re: [PATCH] memcg: expose root cgroup's memory.stat

2020-05-09 Thread Shakeel Butt
On Fri, May 8, 2020 at 2:44 PM Johannes Weiner wrote: > > On Fri, May 08, 2020 at 10:06:30AM -0700, Shakeel Butt wrote: > > One way to measure the efficiency of memory reclaim is to look at the > > ratio (pgscan+pfrefill)/pgsteal. However at the moment these stats are > >

[PATCH] mm: fix LRU balancing effect of new transparent huge pages

2020-05-09 Thread Shakeel Butt
fixup from the splitting path. Signed-off-by: Johannes Weiner Signed-off-by: Shakeel Butt --- Revived the patch from https://lore.kernel.org/patchwork/patch/685703/ mm/swap.c | 23 +-- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index

Re: [PATCH] memcg: oom: ignore oom warnings from memory.max

2020-05-04 Thread Shakeel Butt
On Mon, May 4, 2020 at 9:06 AM Michal Hocko wrote: > > On Mon 04-05-20 08:35:57, Shakeel Butt wrote: > > On Mon, May 4, 2020 at 8:00 AM Michal Hocko wrote: > > > > > > On Mon 04-05-20 07:53:01, Shakeel Butt wrote: > [...] > > > > I am trying to see

Re: [PATCH v5 0/4] Charge loop device i/o to issuing cgroup

2020-05-05 Thread Shakeel Butt
On Mon, May 4, 2020 at 11:30 PM Dave Chinner wrote: > > On Tue, Apr 28, 2020 at 10:27:32PM -0400, Johannes Weiner wrote: > > On Wed, Apr 29, 2020 at 07:47:34AM +1000, Dave Chinner wrote: > > > On Tue, Apr 28, 2020 at 12:13:46PM -0400, Dan Schatzberg wrote: > > > > This patch series does some > > >

Re: [PATCH] memcg: oom: ignore oom warnings from memory.max

2020-05-05 Thread Shakeel Butt
On Tue, May 5, 2020 at 12:13 AM Michal Hocko wrote: > > On Mon 04-05-20 12:23:51, Shakeel Butt wrote: > [...] > > *Potentially* useful for debugging versus actually beneficial for > > "sweep before tear down" use-case. > > I definitely do not want to preve

Re: [PATCH] memcg: oom: ignore oom warnings from memory.max

2020-05-05 Thread Shakeel Butt
On Tue, May 5, 2020 at 8:27 AM Johannes Weiner wrote: > > On Mon, May 04, 2020 at 12:23:51PM -0700, Shakeel Butt wrote: > > On Mon, May 4, 2020 at 9:06 AM Michal Hocko wrote: > > > I really hate to repeat myself but this is no different from a regular > > > oom s

Re: [PATCH] mm, memcg: unify reclaim retry limits with page allocator

2020-05-22 Thread Shakeel Butt
_RETRIES with MAX_RECLAIM_RETRIES, making the page > allocator and memcg internals more similar in semantics when reclaim > fails to produce results, avoiding premature OOMs or throttling. > > Signed-off-by: Chris Down Reviewed-by: Shakeel Butt

Re: [PATCH] memcg: expose root cgroup's memory.stat

2020-05-15 Thread Shakeel Butt
On Fri, May 15, 2020 at 6:24 AM Johannes Weiner wrote: > > On Fri, May 15, 2020 at 10:29:55AM +0200, Michal Hocko wrote: > > On Sat 09-05-20 07:06:38, Shakeel Butt wrote: > > > On Fri, May 8, 2020 at 2:44 PM Johannes Weiner wrote: > > > > > > > > On F

Re: [PATCH] memcg: expose root cgroup's memory.stat

2020-05-15 Thread Shakeel Butt
On Fri, May 15, 2020 at 8:00 AM Roman Gushchin wrote: > > On Fri, May 15, 2020 at 06:44:44AM -0700, Shakeel Butt wrote: > > On Fri, May 15, 2020 at 6:24 AM Johannes Weiner wrote: > > > > > > On Fri, May 15, 2020 at 10:29:55AM +0200, Michal Hocko wrote: > > &

Re: [PATCH] memcg: expose root cgroup's memory.stat

2020-05-15 Thread Shakeel Butt
On Fri, May 15, 2020 at 11:09 AM Roman Gushchin wrote: > > On Fri, May 15, 2020 at 10:49:22AM -0700, Shakeel Butt wrote: > > On Fri, May 15, 2020 at 8:00 AM Roman Gushchin wrote: > > > > > > On Fri, May 15, 2020 at 06:44:44AM -0700, Shakeel Butt wrote: > >

Re: [PATCH] memcg: expose root cgroup's memory.stat

2020-05-15 Thread Shakeel Butt
On Fri, May 15, 2020 at 11:09 AM Johannes Weiner wrote: > > On Fri, May 15, 2020 at 10:49:22AM -0700, Shakeel Butt wrote: > > On Fri, May 15, 2020 at 8:00 AM Roman Gushchin wrote: > > > On Fri, May 15, 2020 at 06:44:44AM -0700, Shakeel Butt wrote: > > > > On Fri,

[PATCH] net/packet: simply allocations in alloc_one_pg_vec_page

2020-05-15 Thread Shakeel Butt
GFP_KERNEL for each individual page allocation and thus there is no need to have any fallback after vzalloc. Signed-off-by: Shakeel Butt --- net/packet/af_packet.c | 15 --- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c

Re: [PATCH RFC 1/4] mm: kmem: move memcg_kmem_bypass() calls to get_mem/obj_cgroup_from_current()

2020-08-27 Thread Shakeel Butt
call sites and allows further code simplifications. > > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH RFC 2/4] mm: kmem: remove redundant checks from get_obj_cgroup_from_current()

2020-08-27 Thread Shakeel Butt
n Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH RFC 3/4] mm: kmem: prepare remote memcg charging infra for interrupt contexts

2020-08-27 Thread Shakeel Butt
es which were stuck due to radix tree nodes charges while the actual pages pointed by the those nodes/entries were in used by active jobs (shared file system and the kernel is older than the kmem reparenting). Reviewed-by: Shakeel Butt

Re: [PATCH RFC 4/4] mm: kmem: enable kernel memcg accounting from interrupt contexts

2020-08-27 Thread Shakeel Butt
c. > > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH bpf-next v4 08/30] bpf: refine memcg-based memory accounting for hashtab maps

2020-08-28 Thread Shakeel Butt
On Fri, Aug 21, 2020 at 8:23 AM Roman Gushchin wrote: > > Include percpu objects and the size of map metadata into the > accounting. > > Signed-off-by: Roman Gushchin > Acked-by: Song Liu Reviewed-by: Shakeel Butt

Re: kernel BUG at kernel/fork.c:LINE!

2020-08-23 Thread Shakeel Butt
(GCC) 10.1.0-syz 20200507 > > > > Unfortunately, I don't have any reproducer for this issue yet. > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > Reported-by: syzbot+3776ecd80aac504e6...@syzkaller.appspotmail.com > > I

Re: KASAN: use-after-free Write in page_counter_uncharge

2020-08-24 Thread Shakeel Butt
; memcg reference after it adds charges to the batch. If the current page > happens to be the last one holding the reference for its memcg then the > memcg is OK to go and the next page to be freed will trigger batched > uncharge which needs to access the memcg which is gone already. > > Fix the issue by taking a reference for the memcg in the current batch. > > Fixes: 1a3e1f40962c ("mm: memcontrol: decouple reference counting from page > accounting") > Reported-by: syzbot+b305848212deec86e...@syzkaller.appspotmail.com > Reported-by: syzbot+b5ea6fb6f139c8b94...@syzkaller.appspotmail.com > Signed-off-by: Michal Hocko Seems correct to me. Reviewed-by: Shakeel Butt

Re: [RFC PATCH 1/8] memcg: Enable fine-grained control of over memory.high action

2020-08-17 Thread Shakeel Butt
On Mon, Aug 17, 2020 at 7:11 AM Waiman Long wrote: > > Memory controller can be used to control and limit the amount of > physical memory used by a task. When a limit is set in "memory.high" > in a non-root memory cgroup, the memory controller will try to reclaim > memory if the limit has been exc

Re: [PATCH 1/5] ksm: reinstate memcg charge on copied pages

2020-08-31 Thread Shakeel Butt
es. > > Fixes: 4c6355b25e8b ("mm: memcontrol: charge swapin pages on instantiation") > Signed-off-by: Hugh Dickins > Cc: sta...@vger.kernel.org # v5.8 Reviewed-by: Shakeel Butt

Re: [PATCH 2/5] mm: migration of hugetlbfs page skip memcg

2020-08-31 Thread Shakeel Butt
ove_pages12 triggers the warning in Alex Shi's prospective commit > "mm/memcg: warning on !memcg after readahead page charged". > > Signed-off-by: Hugh Dickins Reviewed-by: Shakeel Butt

Re: [PATCH 3/5] shmem: shmem_writepage() split unlikely i915 THP

2020-08-31 Thread Shakeel Butt
is off). > > LRU page reclaim always splits the shmem huge page first: I'd prefer not > to demand that of i915, so check and split compound in shmem_writepage(). > > Fixes: 2d6692e642e7 ("drm/i915: Start writeback from the shrinker") > Signed-off-by: Hugh Dickins > Cc: sta...@vger.kernel.org # v5.3+ Reviewed-by: Shakeel Butt

Re: [PATCH 4/5] mm: fix check_move_unevictable_pages() on THP

2020-08-31 Thread Shakeel Butt
follow that precedent here. > > Do this in such a way that if mem_cgroup_page_lruvec() is made stricter > (to check page->mem_cgroup is always set), no problem: skip the tails > before calling it, and add thp_nr_pages() to vmstats on the head. > > Signed-off-by: Hugh Dickins Thanks for catching this. Reviewed-by: Shakeel Butt

Re: [PATCH 5/5] mlock: fix unevictable_pgs event counts on THP

2020-08-31 Thread Shakeel Butt
ixes: 5d91f31faf8e ("mm: swap: fix vmstats for huge page") > Signed-off-by: Hugh Dickins Reviewed-by: Shakeel Butt

Re: [PATCH v3] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-19 Thread Shakeel Butt
kernel/workqueue.c:2416) > [9.846034] ? process_one_work (kernel/workqueue.c:2358) > [9.846162] kthread (kernel/kthread.c:292) > [9.846271] ? __kthread_bind_mask (kernel/kthread.c:245) > [9.846420] ret_from_fork (arch/x86/entry/entry_64.S:300) > [9.846531] ---[ end

Re: [External] Re: [PATCH] mm: proc: add Sock to /proc/meminfo

2020-10-19 Thread Shakeel Butt
CCed: Paolo Bonzini On Fri, Oct 16, 2020 at 1:53 PM Minchan Kim wrote: [snip] > > And there might be others, and adding everything to /proc/meminfo is not > > feasible. I have once proposed adding a counter called "Unaccounted:" which > > would at least tell the user easily if a significant porti

Re: [RFC PATCH] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-20 Thread Shakeel Butt
On Mon, Oct 19, 2020 at 3:28 PM Roman Gushchin wrote: > > On Fri, Oct 16, 2020 at 07:15:02PM +0200, Michal Koutny wrote: > > On Fri, Oct 16, 2020 at 10:53:08AM -0400, Johannes Weiner > > wrote: > > > The central try_charge() function charges recursively all the way up > > > to and including the

Re: [PATCH v3] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-20 Thread Shakeel Butt
On Tue, Oct 20, 2020 at 6:49 AM Richard Palethorpe wrote: > > Hello, > > Richard Palethorpe writes: > > > Hello Shakeel, > > > > Shakeel Butt writes: > >>> > >>> V3: Handle common case where use_hierarchy=1 and update description. >

Re: [PATCH v4] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-22 Thread Shakeel Butt
ue.c:2358) > [ 9.846162] kthread (kernel/kthread.c:292) > [9.846271] ? __kthread_bind_mask (kernel/kthread.c:245) > [9.846420] ret_from_fork (arch/x86/entry/entry_64.S:300) > [9.846531] ---[ end trace 8b5647c1eba9d18a ]--- > > Reported-by: l...@lists.linux.it >

Re: [PATCH] mm: memcontrol: add file_thp, shmem_thp to memory.stat

2020-10-22 Thread Shakeel Butt
non_thp per cgroup. We already track file > THP and shmem THP per node, so making them per-cgroup is only a matter > of switching from node to lruvec counters. All callsites are in places > where the pages are charged and locked, so page->memcg is stable. > > Signed-off-by: Johannes Weiner Reviewed-by: Shakeel Butt

Re: [PATCH v4] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-22 Thread Shakeel Butt
On Thu, Oct 22, 2020 at 10:25 AM Roman Gushchin wrote: > [snip] > > > > Since bf4f059954dc ("mm: memcg/slab: obj_cgroup API") is in 5.9, I > > think we can take this patch for 5.9 and 5.10 but keep Roman's cleanup > > for 5.11. > > > > What does everyone think? > > I think we should use the link t

Re: [PATCH v4] mm: memcg/slab: Stop reparented obj_cgroups from charging root

2020-10-23 Thread Shakeel Butt
On Thu, Oct 22, 2020 at 5:40 PM Roman Gushchin wrote: > > On Thu, Oct 22, 2020 at 04:59:56PM -0700, Shakeel Butt wrote: > > On Thu, Oct 22, 2020 at 10:25 AM Roman Gushchin wrote: > > > > > [snip] > > > > > > > > Since bf4f059954dc ("mm: me

Re: [PATCH v2] mm: memcg/slab: Fix return child memcg objcg for root memcg

2020-10-29 Thread Shakeel Butt
On Tue, Oct 27, 2020 at 8:50 PM Muchun Song wrote: > > Consider the following memcg hierarchy. > > root >/\ > A B > > If we get the objcg of memcg A failed, Please fix the above statement. > the get_obj_cgroup_from_current > can

Re: [PATCH v2] mm: memcg/slab: Fix use after free in obj_cgroup_charge

2020-10-29 Thread Shakeel Butt
ab: obj_cgroup API") > Signed-off-by: Muchun Song > Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH v2] mm: memcg/slab: Rename *_lruvec_slab_state to *_lruvec_kmem_state

2020-10-29 Thread Shakeel Butt
mem. > > Signed-off-by: Muchun Song > Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt

Re: [PATCH v2] mm: memcontrol: Simplify the mem_cgroup_page_lruvec

2020-10-29 Thread Shakeel Butt
On Thu, Oct 29, 2020 at 2:08 AM Michal Hocko wrote: > > On Wed 28-10-20 11:50:13, Muchun Song wrote: > [...] > > -struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct > > pglist_data *pgdat) > > +static struct lruvec * > > +__mem_cgroup_node_lruvec(struct mem_cgroup *memcg, struct pgli

<    2   3   4   5   6   7   8   9   10   >