Re: [PATCH -next v6 0/2] Make memory reclamation measurable

2024-03-07 Thread Michal Hocko
track the time process has been preempted by other means, no? We have context switching tracepoints in place. Have you considered that option? -- Michal Hocko SUSE Labs

Re: [PATCH v2] mm: Update mark_victim tracepoints fields

2024-02-22 Thread Michal Hocko
On Wed 21-02-24 13:30:51, Carlos Galo wrote: > On Tue, Feb 20, 2024 at 11:55 PM Michal Hocko wrote: > > > > Hi, > > sorry I have missed this before. > > > > On Thu 11-01-24 21:05:30, Carlos Galo wrote: > > > The current implementation of the mark_victim

Re: [PATCH v2] mm: Update mark_victim tracepoints fields

2024-02-20 Thread Michal Hocko
pid_nr(victim), victim->comm, K(mm->total_vm), K(get_mm_counter(mm, MM_ANONPAGES)), K(get_mm_counter(mm, MM_FILEPAGES)), K(get_mm_counter(mm, MM_SHMEMPAGES)), from_kuid(_user_ns, task_uid(victim)), mm_pgtables_bytes(mm) >> 10, victim->signal->oom_score_adj); -- Michal Hocko SUSE Labs

Re: [PATCH -next v6 0/2] Make memory reclamation measurable

2024-02-20 Thread Michal Hocko
an add-hoc dynamic tracepoints or BPF for a very special situation is not sufficient? In other words, tell us more about the usecases and why is this generally useful. Thanks! -- Michal Hocko SUSE Labs

Re: [PATCH] module: print module name on refcount error

2023-09-14 Thread Michal Hocko
ually the interesting case at all. -- Michal Hocko SUSE Labs

Re: [PATCH v2] docs: proc.rst: meminfo: briefly describe gaps in memory accounting

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 15:57:08, Michal Hocko wrote: [...] > Usual memory consumption is usually something like LRU pages + Slab > memory + kernel stack + vmalloc used + pcp. > > > But I know that KernelStack is allocated through vmalloc these days, > > and I don't know whether

Re: [PATCH v2] docs: proc.rst: meminfo: briefly describe gaps in memory accounting

2021-04-20 Thread Michal Hocko
Mlocked a subset of Unevictable? > > There is some attempt at explaining how these numbers fit together, but > it's outdated, and doesn't include Mlocked, Unevictable or KernelStack Agreed there is a lot of tribal knowledge or even misconceptions flying around and it will take much more work to put everything into shape. This is only one tiny step forward. -- Michal Hocko SUSE Labs

Re: [PATCH v2] docs: proc.rst: meminfo: briefly describe gaps in memory accounting

2021-04-20 Thread Michal Hocko
usage. > > > > Signed-off-by: Mike Rapoport > > Ooops, forgot to add Michal's Ack, sorry. Let's make it more explicit Acked-by: Michal Hocko Thanks! -- Michal Hocko SUSE Labs

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 09:25:51, peter.enderb...@sony.com wrote: > On 4/20/21 11:12 AM, Michal Hocko wrote: > > On Tue 20-04-21 09:02:57, peter.enderb...@sony.com wrote: > >>>> But that isn't really system memory at all, it's just allocated device > >>>> memory. &g

Re: [PATCH v9 7/8] x86/Kconfig: Introduce ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE

2021-04-20 Thread Michal Hocko
On Fri 16-04-21 13:24:10, Oscar Salvador wrote: > Enable x86_64 platform to use the MHP_MEMMAP_ON_MEMORY feature. > > Signed-off-by: Oscar Salvador > Reviewed-by: David Hildenbrand Acked-by: Michal Hocko > --- > arch/x86/Kconfig | 3 +++ > 1 file changed, 3 insertio

Re: [PATCH v9 4/8] mm,memory_hotplug: Allocate memmap from the added memory range

2021-04-20 Thread Michal Hocko
return -EINVAL; > + } > + > + /* > + * Let remove_pmd_table->free_hugepage_table do the > + * right thing if we used vmem_altmap when hot-adding > + * the range. > + */ > + mhp_altmap.alloc = nr_vmemmap_pages; > + altmap = _altmap; > + } > + } > + > /* remove memmap entry */ > firmware_map_remove(start, start + size, "System RAM"); I have to say I still dislike this and I would just wrap it inside out and do the operation from within walk_memory_blocks but I will not insist. -- Michal Hocko SUSE Labs

Re: [PATCH v9 3/8] mm,memory_hotplug: Factor out adjusting present pages into adjust_present_page_count()

2021-04-20 Thread Michal Hocko
modify the number of present pages. > > Signed-off-by: David Hildenbrand > Signed-off-by: Oscar Salvador > Reviewed-by: Oscar Salvador Not sure self review counts ;) Acked-by: Michal Hocko Btw. I strongly suspect the resize lock is quite pointless here. Something for a follow up patch.

Re: [PATCH v9 2/8] mm,memory_hotplug: Relax fully spanned sections check

2021-04-20 Thread Michal Hocko
is a special case we want to allow." > Signed-off-by: Oscar Salvador > Reviewed-by: David Hildenbrand With the changelog extended and the comment clarification (se below) feel free to add Acked-by: Michal Hocko > --- > mm/memory_hotplug.c | 18 ++ > 1 file c

Re: [PATCH v9 1/8] drivers/base/memory: Introduce memory_block_{online,offline}

2021-04-20 Thread Michal Hocko
Reviewed-by: David Hildenbrand Acked-by: Michal Hocko > --- > drivers/base/memory.c | 33 + > 1 file changed, 21 insertions(+), 12 deletions(-) > > diff --git a/drivers/base/memory.c b/drivers/base/memory.c > index f35298425575..f209925a5d4e

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
a single counter without a wider context cannot be put into any reasonable context. There is no notion of the total amount of device memory usable for dma-buf. As Christian explained some of it can be RAM based. So a single number is rather pointless on its own in many cases. Or let me just ask. What can

Re: [PATCH] docs: proc.rst: meminfo: briefly describe gaps in memory accounting

2021-04-20 Thread Michal Hocko
a step into the right direction. Acked-by: Michal Hocko one nit below > --- > Documentation/filesystems/proc.rst | 11 +-- > 1 file changed, 9 insertions(+), 2 deletions(-) > > diff --git a/Documentation/filesystems/proc.rst > b/Documentation/filesystems/proc.rst > index

Re: [PATCH 0/2 V6]Add dma-buf counter

2021-04-20 Thread Michal Hocko
now replaced with dma-buf. ION had some overview metrics that was > similar. The discussion around the previous version is still not over and as it seems your proposed approach is not really viable. So please do not send new versions until that is sorted out. Thanks! -- Michal Hocko SUSE Labs

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 10:00:07, Christian König wrote: > Am 20.04.21 um 09:46 schrieb Michal Hocko: > > On Tue 20-04-21 09:32:14, Christian König wrote: > > > Am 20.04.21 um 09:04 schrieb Michal Hocko: > > > > On Mon 19-04-21 18:37:13, Christian König wrote: > > >

Re: [PATCH v4 1/5] mm/memcg: Move mod_objcg_state() to memcontrol.c

2021-04-20 Thread Michal Hocko
s own file makes sense to me as well. If the code is not conditional (e.g. like swap accounting and some others) then moving it would make memecontrol.c easier to navigate through. -- Michal Hocko SUSE Labs

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 10:20:43, Mike Rapoport wrote: > On Tue, Apr 20, 2021 at 09:04:51AM +0200, Michal Hocko wrote: > > On Mon 19-04-21 18:37:13, Christian König wrote: > > > Am 19.04.21 um 18:11 schrieb Michal Hocko: > > [...] > > > > The question is not

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 09:32:14, Christian König wrote: > Am 20.04.21 um 09:04 schrieb Michal Hocko: > > On Mon 19-04-21 18:37:13, Christian König wrote: > > > Am 19.04.21 um 18:11 schrieb Michal Hocko: [...] > > What I am trying to bring up with NUMA side is that the same probl

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Mon 19-04-21 18:37:13, Christian König wrote: > Am 19.04.21 um 18:11 schrieb Michal Hocko: [...] > > The question is not whether it is NUMA aware but whether it is useful to > > know per-numa data for the purpose the counter is supposed to serve. > > No, not at all. The

Re: [RFC] memory reserve for userspace oom-killer

2021-04-20 Thread Michal Hocko
itrary metrics and if that can be done without any > allocations. A kernel module or eBPF to implement oom decisions has already been discussed few years back. But I am afraid this would be hard to wire in for anything except for the victim selection. I am not sure it is maintainable to also control when the OOM handling should trigger. -- Michal Hocko SUSE Labs

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
On Mon 19-04-21 17:44:13, Christian König wrote: > Am 19.04.21 um 17:19 schrieb peter.enderb...@sony.com: > > On 4/19/21 5:00 PM, Michal Hocko wrote: > > > On Mon 19-04-21 12:41:58, peter.enderb...@sony.com wrote: > > > > On 4/19/21 2:16 PM, Michal Hocko wrote: >

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
On Mon 19-04-21 12:41:58, peter.enderb...@sony.com wrote: > On 4/19/21 2:16 PM, Michal Hocko wrote: > > On Sat 17-04-21 12:40:32, Peter Enderborg wrote: > >> This adds a total used dma-buf memory. Details > >> can be found in debugfs, however it is not for everyone &g

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
ter explanation and secondly is this information useful for OOM situations analysis? If yes then show_mem should dump the value as well. >From the implementation point of view, is there any reason why this hasn't used the existing global_node_page_state infrastructure? -- Michal Hocko SUSE Labs

Re: [PATCH 00/10] [v7][RESEND] Migrate Pages in lieu of discard

2021-04-16 Thread Michal Hocko
On Fri 16-04-21 07:26:43, Dave Hansen wrote: > On 4/16/21 5:35 AM, Michal Hocko wrote: > > I have to confess that I haven't grasped the initialization > > completely. There is a nice comment explaining a 2 socket system with > > 3 different NUMA nodes attached to i

Re: [PATCH 00/10] [v7][RESEND] Migrate Pages in lieu of discard

2021-04-16 Thread Michal Hocko
details but they do not seem that important. I am still trying to digest the whole thing but at least jamming node_reclaim logic into kswapd seems strange to me. Need to think more about that though. Btw. do you have any numbers from running this with some real work workload? -- Michal Hocko SUSE

Re: [RFC PATCH v1 00/11] Manage the top tier memory in a tiered memory

2021-04-16 Thread Michal Hocko
On Thu 15-04-21 15:31:46, Tim Chen wrote: > > > On 4/9/21 12:24 AM, Michal Hocko wrote: > > On Thu 08-04-21 13:29:08, Shakeel Butt wrote: > >> On Thu, Apr 8, 2021 at 11:01 AM Yang Shi wrote: > > [...] > >>> The low priority jobs should be able to be res

Re: [PATCH v2 5/8] mm: memcontrol: rename lruvec_holds_page_lru_lock to page_matches_lruvec

2021-04-16 Thread Michal Hocko
On Fri 16-04-21 13:14:04, Muchun Song wrote: > lruvec_holds_page_lru_lock() doesn't check anything about locking and is > used to check whether the page belongs to the lruvec. So rename it to > page_matches_lruvec(). > > Signed-off-by: Muchun Song Acked-by: Michal

Re: [PATCH v8 7/7] mm,page_alloc: Drop unnecessary checks from pfn_range_valid_contig

2021-04-15 Thread Michal Hocko
enbrand > Acked-by: Mike Kravetz Acked-by: Michal Hocko > --- > mm/page_alloc.c | 6 -- > 1 file changed, 6 deletions(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index b5a94de3cdde..c5338e912ace 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @

Re: [PATCH v8 3/7] mm,hugetlb: Drop clearing of flag from prep_new_huge_page

2021-04-15 Thread Michal Hocko
from 6c0371490140 ("hugetlb: convert PageHugeFreed to HPageFreed flag"). Previously the explicit clearing was necessary because compound allocations do not get this initialization (see prep_compound_page). > Signed-off-by: Oscar Salvador with that Acked-by: Michal Hocko > --- > mm/hug

Re: [External] Re: [PATCH 2/7] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm

2021-04-15 Thread Michal Hocko
On Thu 15-04-21 11:13:16, Muchun Song wrote: > On Wed, Apr 14, 2021 at 6:15 PM Michal Hocko wrote: > > > > On Wed 14-04-21 18:04:35, Muchun Song wrote: > > > On Wed, Apr 14, 2021 at 5:24 PM Michal Hocko wrote: > > > > > > > > On Tue 13-04-21 14:51:

Re: [External] Re: [PATCH 4/7] mm: memcontrol: simplify lruvec_holds_page_lru_lock

2021-04-15 Thread Michal Hocko
On Wed 14-04-21 13:49:56, Johannes Weiner wrote: > On Wed, Apr 14, 2021 at 06:00:42PM +0800, Muchun Song wrote: > > On Wed, Apr 14, 2021 at 5:44 PM Michal Hocko wrote: > > > > > > On Tue 13-04-21 14:51:50, Muchun Song wrote: > > > > We already have a help

Re: High kmalloc-32 slab cache consumption with 10k containers

2021-04-15 Thread Michal Hocko
> fs/super.c | 27 +++ > include/linux/fs_context.h | 2 ++ > mm/shmem.c | 1 + > 5 files changed, 24 insertions(+), 8 deletions(-) [...] -- Michal Hocko SUSE Labs

Re: [PATCH v4 11/13] mm/mempolicy: huge-page allocation for many preferred

2021-04-14 Thread Michal Hocko
gt;mode == MPOL_PREFERRED_MANY) { page = alloc_surplus_huge_page(h, (gfp_mask | __GFP_NOWARN) & ~(__GFP_DIRECT_RECLAIM), nodemask); if (page) goto got_page; /* fallback to all nodes */ nodemask = NULL;

Re: [PATCH v4 10/13] mm/mempolicy: VMA allocation for many preferred

2021-04-14 Thread Michal Hocko
And alloc_pages_policy doesn't really help I have to say. I would have expected that a dedicated alloc_pages_preferred and a general fallback to __alloc_pages_nodemask would have been much easier to follow. -- Michal Hocko SUSE Labs

Re: [PATCH v4 08/13] mm/mempolicy: Create a page allocator for policy

2021-04-14 Thread Michal Hocko
RRED_MANY's semantic is more like MPOL_PREFERRED > that it will first try the preferred node/nodes, and fallback to all > other nodes when first try fails. Thanks to Michal Hocko for suggestions > on this. > > For now, only interleaved policy will be used so there should be no > functional

Re: [PATCH v4 07/13] mm/mempolicy: handle MPOL_PREFERRED_MANY like BIND

2021-04-14 Thread Michal Hocko
; reuses BIND. No, this is a bug step back. I think we really want to treat this as PREFERRED. It doesn't have much to do with the BIND semantic at all. At this stage there should be 2 things remaining - syscalls plumbing and 2 pass allocation request (optimistic preferred nodes restricted and fa

Re: [PATCH v4 06/13] mm/mempolicy: kill v.preferred_nodes

2021-04-14 Thread Michal Hocko
to grep for preferred_nodes than nodes. -- Michal Hocko SUSE Labs

Re: [PATCH v4 05/13] mm/mempolicy: refactor rebind code for PREFERRED_MANY

2021-04-14 Thread Michal Hocko
ixed typos in commit message. (Ben) > Merged bits from other patches. (Ben) > annotate mpol_rebind_preferred_many as unused (Ben) I am giving up on the rebinding code for now until we clarify that in my earlier email. -- Michal Hocko SUSE Labs

Re: [PATCH v4 04/13] mm/mempolicy: allow preferred code to take a nodemask

2021-04-14 Thread Michal Hocko
gt; + if (nodes_empty(*nodes)) > + return -EINVAL; > + > + tmp = nodemask_of_node(first_node(*nodes)); > + return mpol_new_preferred_many(pol, ); > + } > + > + return mpol_new_preferred_many(pol, NULL); > +} > + > static int mpol_new_bind(struct mempolicy *pol, const nodemask_t *nodes) > { > if (nodes_empty(*nodes)) > -- > 2.7.4 -- Michal Hocko SUSE Labs

Re: [PATCH v4 03/13] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes

2021-04-14 Thread Michal Hocko
hing in it should really be as simple as node_isset check. > default: > BUG(); Besides that, this should really go! > @@ -3035,6 +3066,9 @@ void mpol_to_str(char *buffer, int maxlen, struct > mempolicy *pol) > switch (mode) { > case MPOL_DEFAULT: > break; > + case MPOL_PREFERRED_MANY: > + WARN_ON(flags & MPOL_F_LOCAL); Why WARN_ON here? > + fallthrough; > case MPOL_PREFERRED: > if (flags & MPOL_F_LOCAL) > mode = MPOL_LOCAL; > -- > 2.7.4 -- Michal Hocko SUSE Labs

Re: [PATCH v4 02/13] mm/mempolicy: convert single preferred_node to full nodemask

2021-04-14 Thread Michal Hocko
= tmp; > pol->w.cpuset_mems_allowed = *nodes; > } I have to say that I really disliked the original code (becasuse it fiddles with user provided input behind the back) I got lost here completely. What the heck is going on? a) why do we even care remaping a hint which is overriden by the cpuset at the page allocator level and b) why do we need to allocate _two_ potentially large temporary bitmaps for that here? I haven't spotted anything unexpected in the rest. -- Michal Hocko SUSE Labs

Re: [PATCH v4 00/13] Introduced multi-preference mempolicy

2021-04-14 Thread Michal Hocko
ode to full nodemask > mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes > mm/mempolicy: allow preferred code to take a nodemask > mm/mempolicy: refactor rebind code for PREFERRED_MANY > > Feng Tang (1): > mem/mempolicy: unify mpol_new_preferred() and > mpol_new_preferred_many() > > .../admin-guide/mm/numa_memory_policy.rst | 22 +- > include/linux/mempolicy.h | 6 +- > include/uapi/linux/mempolicy.h | 6 +- > mm/hugetlb.c | 26 +- > mm/mempolicy.c | 272 > ++--- > 5 files changed, 225 insertions(+), 107 deletions(-) > > -- > 2.7.4 -- Michal Hocko SUSE Labs

Re: [PATCH v7 3/7] mm,hugetlb: Clear HPageFreed outside of the lock

2021-04-14 Thread Michal Hocko
On Wed 14-04-21 12:49:53, Oscar Salvador wrote: > On Wed, Apr 14, 2021 at 12:32:58PM +0200, Michal Hocko wrote: [...] > > > I checked, and when we get there in __alloc_bootmem_huge_page, > > > page->private is > > > still zeroed, so I guess it should be safe to as

Re: [PATCH v7 3/7] mm,hugetlb: Clear HPageFreed outside of the lock

2021-04-14 Thread Michal Hocko
On Wed 14-04-21 12:01:47, Oscar Salvador wrote: > On Wed, Apr 14, 2021 at 10:28:33AM +0200, Michal Hocko wrote: > > You are right it doesn't do it there. But all struct pages, even those > > that are allocated by the bootmem allocator should initialize its struct > > pages. T

Re: [External] Re: [PATCH 2/7] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm

2021-04-14 Thread Michal Hocko
On Wed 14-04-21 18:04:35, Muchun Song wrote: > On Wed, Apr 14, 2021 at 5:24 PM Michal Hocko wrote: > > > > On Tue 13-04-21 14:51:48, Muchun Song wrote: > > > When mm is NULL, we do not need to hold rcu lock and call css_tryget for > > > the root memcg. And

Re: [PATCH 7/7] mm: vmscan: remove noinline_for_stack

2021-04-14 Thread Michal Hocko
nes Weiner > Acked-by: Roman Gushchin > Reviewed-by: Shakeel Butt Acked-by: Michal Hocko > --- > mm/vmscan.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 64bf07cc20f2..e40b21298d77 100644

Re: [PATCH 4/7] mm: memcontrol: simplify lruvec_holds_page_lru_lock

2021-04-14 Thread Michal Hocko
nd > CONFIG_MEMCG. Neat. While you are at it wouldn't it make sesne to rename the function as well. I do not want to bikeshed but this is really a misnomer. it doesn't check anything about locking. page_belongs_lruvec? > Signed-off-by: Muchun Song > Acked-by: Johannes Weiner Acked-b

Re: [PATCH 3/7] mm: memcontrol: remove the pgdata parameter of mem_cgroup_page_lruvec

2021-04-14 Thread Michal Hocko
his case it even doesn't give any advantage for most callers. Acked-by: Michal Hocko > --- > include/linux/memcontrol.h | 10 +- > mm/compaction.c| 2 +- > mm/memcontrol.c| 9 +++-- > mm/swap.c | 2 +- > mm/workingset.c

Re: [PATCH 2/7] mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm

2021-04-14 Thread Michal Hocko
dereference(mm->owner)); > - if (unlikely(!memcg)) > - memcg = root_mem_cgroup; > - } > } while (!css_tryget(>css)); > rcu_read_unlock(); > return memcg; > -- > 2.11.0 -- Michal Hocko SUSE Labs

Re: [PATCH 1/7] mm: memcontrol: fix page charging in page replacement

2021-04-14 Thread Michal Hocko
WARN_ON_ONCE in the page_counter_cancel(). Who knows if it > will trigger? So it is better to fix it. > > Signed-off-by: Muchun Song > Acked-by: Johannes Weiner > Reviewed-by: Shakeel Butt Acked-by: Michal Hocko > --- > mm/memcontrol.c | 8 +--- > 1 file changed, 5 ins

Re: [PATCH v7 3/7] mm,hugetlb: Clear HPageFreed outside of the lock

2021-04-14 Thread Michal Hocko
On Wed 14-04-21 09:41:32, Oscar Salvador wrote: > On Wed, Apr 14, 2021 at 08:04:21AM +0200, Michal Hocko wrote: > > On Tue 13-04-21 14:19:03, Mike Kravetz wrote: > > > On 4/13/21 6:23 AM, Michal Hocko wrote: > > > The only place where page->private may no

Re: [PATCH v7 3/7] mm,hugetlb: Clear HPageFreed outside of the lock

2021-04-14 Thread Michal Hocko
On Tue 13-04-21 14:19:03, Mike Kravetz wrote: > On 4/13/21 6:23 AM, Michal Hocko wrote: > > On Tue 13-04-21 12:47:43, Oscar Salvador wrote: [...] > > Or do we need it for giga pages which are not allocated by the page > > allocator? If yes then moving it to prep_compoun

Re: [PATCH v7 5/7] mm: Make alloc_contig_range handle free hugetlb pages

2021-04-13 Thread Michal Hocko
case above we retry as the window race is quite small and we have high > chances to succeed next time. > > With regard to the allocation, we restrict it to the node the page belongs > to with __GFP_THISNODE, meaning we do not fallback on other node's zones. > > Note that gigant

Re: [PATCH v7 4/7] mm,hugetlb: Split prep_new_huge_page functionality

2021-04-13 Thread Michal Hocko
t;nr_huge_pages_node[nid]++; > + __prep_account_new_huge_page(h, nid); > spin_unlock_irq(_lock); > } Any reason to decouple the locking from the accounting? > > -- > 2.16.3 -- Michal Hocko SUSE Labs

Re: [PATCH v7 4/7] mm,hugetlb: Split prep_new_huge_page functionality

2021-04-13 Thread Michal Hocko
On Tue 13-04-21 15:24:32, Michal Hocko wrote: > On Tue 13-04-21 12:47:44, Oscar Salvador wrote: > [...] > > +static void prep_new_huge_page(struct hstate *h, struct page *page, int > > nid) > > +{ > > + __prep_new_huge_page(page); > > spin_lock_

Re: [PATCH v7 3/7] mm,hugetlb: Clear HPageFreed outside of the lock

2021-04-13 Thread Michal Hocko
d(page); > spin_lock_irq(_lock); > h->nr_huge_pages++; > h->nr_huge_pages_node[nid]++; > - ClearHPageFreed(page); > spin_unlock_irq(_lock); > } > > -- > 2.16.3 -- Michal Hocko SUSE Labs

Re: [PATCH v2 resend] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-13 Thread Michal Hocko
Not sure this is worth it TBH. Even an idea of any pcp access synchronization with memory hotplug makes for a decent headache. -- Michal Hocko SUSE Labs

Re: [RFC PATCH v1 00/11] Manage the top tier memory in a tiered memory

2021-04-13 Thread Michal Hocko
On Fri 09-04-21 16:26:53, Tim Chen wrote: > > On 4/8/21 4:52 AM, Michal Hocko wrote: > > >> The top tier memory used is reported in > >> > >> memory.toptier_usage_in_bytes > >> > >> The amount of top tier memory usable by each cgr

Re: [PATCH] mm: optimize memory allocation

2021-04-13 Thread Michal Hocko
ly dislike kmem and LRU pages to be handled differently so for that reason Nacked-by: Michal Hocko If the optimization really can be provent then the patch would require to be much more invasive. > Signed-off-by: Chen Xiaoguang > Signed-off-by: Chen He > --- > incl

Re: [PATCH v2 resend] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-13 Thread Michal Hocko
:, then > gets stalled/scheduled out while hotremove rebuilds the zonelist and destroys > the pcplists, then the first task is resumed and proceeds with > rmqueue_pcplist(). > > So that's very rare thus not urgent, and this patch doesn't make it less rare > so > not a reason to block it. Completely agreed here. Not an urgent thing to work on but something to look into long term. -- Michal Hocko SUSE Labs

Re: [PATCH v2 resend] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-13 Thread Michal Hocko
h that an existing race was likely never observed. Acked-by: Michal Hocko Thanks! > Signed-off-by: Mel Gorman > --- > Resending for email address correction and adding lists > > Changelog since v1 > o Minimal fix > > mm/page_alloc.c | 4 > 1 file changed, 4 delet

Re: [PATCH] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-10 Thread Michal Hocko
OK. Let's do that for now and I will put a follow up on my todo list. Thanks! -- Michal Hocko SUSE Labs

Re: [PATCH] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-09 Thread Michal Hocko
On Fri 09-04-21 14:42:21, Mel Gorman wrote: > On Fri, Apr 09, 2021 at 02:48:12PM +0200, Michal Hocko wrote: > > On Fri 09-04-21 14:42:58, Michal Hocko wrote: > > > On Fri 09-04-21 13:09:57, Mel Gorman wrote: > > > > zone_pcp_reset allegedly protects against a race

Re: [PATCH] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-09 Thread Michal Hocko
On Fri 09-04-21 14:42:58, Michal Hocko wrote: > On Fri 09-04-21 13:09:57, Mel Gorman wrote: > > zone_pcp_reset allegedly protects against a race with drain_pages > > using local_irq_save but this is bogus. local_irq_save only operates > > on the local CPU. If memory hotp

Re: [PATCH] mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

2021-04-09 Thread Michal Hocko
reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it. -- Michal Hocko SUSE Labs

Re: [RFC PATCH v1 00/11] Manage the top tier memory in a tiered memory

2021-04-09 Thread Michal Hocko
memcg. The behavior of those limits would be quite tricky for OOM situations as well due to a lack of NUMA aware oom killer. -- Michal Hocko SUSE Labs

Re: [PATCH] mm: page_counter: mitigate consequences of a page_counter underflow

2021-04-08 Thread Michal Hocko
is already botched and counters cannot be trusted this is definitely better than a potentially completely unusable memcg. It would be nice to mention that in the above paragraph as a caveat. > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko > --- > mm/page_counter.c | 8 ++--

Re: [RFC PATCH v1 00/11] Manage the top tier memory in a tiered memory

2021-04-08 Thread Michal Hocko
On Wed 07-04-21 15:33:26, Tim Chen wrote: > > > On 4/6/21 2:08 AM, Michal Hocko wrote: > > On Mon 05-04-21 10:08:24, Tim Chen wrote: > > [...] > >> To make fine grain cgroup based management of the precious top tier > >> DRAM memory possible, this p

Re: High kmalloc-32 slab cache consumption with 10k containers

2021-04-07 Thread Michal Hocko
On Wed 07-04-21 19:13:42, Bharata B Rao wrote: > On Wed, Apr 07, 2021 at 01:54:48PM +0200, Michal Hocko wrote: > > On Mon 05-04-21 11:18:48, Bharata B Rao wrote: > > > Hi, > > > > > > When running 1 (more-or-less-empty-)containers on a bare-metal Power9

Re: High kmalloc-32 slab cache consumption with 10k containers

2021-04-07 Thread Michal Hocko
global memory reclaim iterating over 10k memcgs will likely be very visible. I do remember playing with similar setups few years back and the overhead was very high. -- Michal Hocko SUSE Labs

Re: [PATCH v4 7/8] hugetlb: make free_huge_page irq safe

2021-04-07 Thread Michal Hocko
nging spin_*lock calls to spin_*lock_irq* calls. > > - Make subpool lock irq safe in a similar manner. > > - Revert the !in_task check and workqueue handoff. > > > > [1] > > https://lore.kernel.org/linux-mm/f1c03b05bc43a...@google.com/ > > > >

Re: [PATCH v4 5/8] hugetlb: call update_and_free_page without hugetlb_lock

2021-04-07 Thread Michal Hocko
) { > > + update_and_free_page(h, page); > > + cond_resched(); > > + } > > + spin_lock(_lock); > > Can we get here with an empty list? An emoty page_list? If yes then sure, this can happen but list_for_each_entry_safe will simply not iterate. Or what do you mean? -- Michal Hocko SUSE Labs

Re: [PATCH v4 4/8] hugetlb: create remove_hugetlb_page() to separate functionality

2021-04-07 Thread Michal Hocko
On Tue 06-04-21 09:49:13, Mike Kravetz wrote: > On 4/6/21 2:56 AM, Michal Hocko wrote: > > On Mon 05-04-21 16:00:39, Mike Kravetz wrote: [...] > >> @@ -2298,6 +2312,7 @@ static int alloc_and_dissolve_huge_page(struct > >> hstat

Re: [PATCH] mm/vmscan.c: drop_slab_node with task's memcg

2021-04-06 Thread Michal Hocko
On Tue 06-04-21 23:12:34, Neil Sun wrote: > > > On 2021/4/6 22:39, Michal Hocko wrote: > > > > Have you considered using high limit for the pro-active memory reclaim? > > Thanks, Michal, do you mean the procfs interfaces? > We have set vm.vfs_cache_pressure=1000

Re: [PATCH] mm/vmscan.c: drop_slab_node with task's memcg

2021-04-06 Thread Michal Hocko
On Tue 06-04-21 22:34:02, Neil Sun wrote: > > > On 2021/4/6 19:39, Michal Hocko wrote: > > On Tue 06-04-21 19:30:22, Neil Sun wrote: > > > On 2021/4/6 15:21, Michal Hocko wrote: > > > > > > > > You are changing semantic of the existing user i

Re: [PATCH] mm/vmscan.c: drop_slab_node with task's memcg

2021-04-06 Thread Michal Hocko
On Tue 06-04-21 19:30:22, Neil Sun wrote: > On 2021/4/6 15:21, Michal Hocko wrote: > > > > You are changing semantic of the existing user interface. This knob has > > never been memcg aware and it is supposed to have a global impact. I do > > not think we can simply cha

Re: [PATCH v4 5/8] hugetlb: call update_and_free_page without hugetlb_lock

2021-04-06 Thread Michal Hocko
continue; > remove_hugetlb_page(h, page, false); > - update_and_free_page(h, page); > + list_add(>lru, _list); > } > } > + > +out: > + spin_unlock(_lock); > + list_for_each_entry_safe(page, next, _list, lru) { > + update_and_free_page(h, page); > + cond_resched(); > + } > + spin_lock(_lock); > } > #else > static inline void try_to_free_low(struct hstate *h, unsigned long count, > -- > 2.30.2 > -- Michal Hocko SUSE Labs

Re: [PATCH v4 4/8] hugetlb: create remove_hugetlb_page() to separate functionality

2021-04-06 Thread Michal Hocko
te_and_free_page(h, new_page); > spin_unlock(_lock); > if (!isolate_huge_page(old_page, list)) the page is not enqued anywhere here so remove_hugetlb_page would blow when linked list debugging is enabled. -- Michal Hocko SUSE Labs

Re: [PATCH v4 1/8] mm/cma: change cma mutex to irq safe spinlock

2021-04-06 Thread Michal Hocko
> Signed-off-by: Mike Kravetz I belive I have acked the previous version already. Anyway Acked-by: Michal Hocko > --- > mm/cma.c | 18 +- > mm/cma.h | 2 +- > mm/cma_debug.c | 8 > 3 files changed, 14 insertions(+), 14 deletions(-) > &

Re: [RFC PATCH v1 00/11] Manage the top tier memory in a tiered memory

2021-04-06 Thread Michal Hocko
rather alien concept to the existing memcg infrastructure IMO. It looks like it is fusing borders between memcg and cputset controllers. You also seem to be basing the interface on the very specific usecase. Can we expect that there will be many different tiers requiring their own balancing? -- Michal Hocko SUSE Labs

Re: [PATCH] mm/vmscan.c: drop_slab_node with task's memcg

2021-04-06 Thread Michal Hocko
g = mem_cgroup_iter(NULL, NULL, NULL); > + memcg = mem_cgroup_from_task(current); > do { > freed += shrink_slab(GFP_KERNEL, nid, memcg, 0); > } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); > -- > 2.7.4 -- Michal Hocko SUSE Labs

Re: [External] Re: [PATCH v2] writeback: fix obtain a reference to a freeing memcg css

2021-04-01 Thread Michal Hocko
On Thu 01-04-21 21:59:13, Muchun Song wrote: > On Thu, Apr 1, 2021 at 6:26 PM Michal Hocko wrote: [...] > > Even if the css ref count is not really necessary it shouldn't cause any > > harm and it makes the code easier to understand. At least a comment > > explaining why

Re: [PATCH v2] writeback: fix obtain a reference to a freeing memcg css

2021-04-01 Thread Michal Hocko
into it's own patch. With more explanation why NOIO is required. > Fixes: 682aa8e1a6a1 ("writeback: implement unlocked_inode_to_wb transaction > and use it for stat updates") > Signed-off-by: Muchun Song For the css part feel free to add Acked-by: Michal Hocko Even if the css ref

Re: [PATCH v3] sysfs: Unconditionally use vmalloc for buffer

2021-04-01 Thread Michal Hocko
rt = sysfs_kf_seq_start, > .seq_show = sysfs_kf_seq_show, > }; > > static const struct kernfs_ops sysfs_file_kfops_wo = { > + .seq_start = sysfs_kf_seq_start, > .write = sysfs_kf_write, > }; > > static const struct kernfs_ops sysfs_file_kfops_rw = { > + .seq_start = sysfs_kf_seq_start, > .seq_show = sysfs_kf_seq_show, > .write = sysfs_kf_write, > }; > -- > 2.25.1 -- Michal Hocko SUSE Labs

Re: [PATCH RFC 0/3] drivers/char: remove /dev/kmem for good

2021-03-31 Thread Michal Hocko
firm that they don't enable it. > > > > > > I can confirm that it's certainly not enabled on any of the machines I > > > have, but.. > > > > Debian has CONFIG_DEVKMEM disabled since 2.6.31. > > SLES, too. (but no idea since when exactly) 15-SP2 IIRC -- Michal Hocko SUSE Labs

Re: [PATCH] userfaultfd: Write protect when virtual memory range has no page table entry

2021-03-31 Thread Michal Hocko
On Mon 22-03-21 14:49:35, Michal Hocko wrote: > On Mon 22-03-21 15:00:37, Mike Rapoport wrote: > > On Mon, Mar 22, 2021 at 11:14:37AM +0100, Michal Hocko wrote: > > > Le'ts Andrea and Mike > > > > > > On Fri 19-03-21 22:24:28, Bui Quang Minh wrote: &g

Re: [PATCH v3 1/8] mm/cma: change cma mutex to irq safe spinlock

2021-03-31 Thread Michal Hocko
t; > Signed-off-by: Mike Kravetz Acked-by: Michal Hocko > --- > mm/cma.c | 18 +- > mm/cma.h | 2 +- > mm/cma_debug.c | 8 > 3 files changed, 14 insertions(+), 14 deletions(-) > > diff --git a/mm/cma.c b/mm/cma.c > index b2393b8

Re: [PATCH] linux/memcontrol.h: Remove duplicate struct declaration

2021-03-30 Thread Michal Hocko
MCG so the below one is not needed though. It would be great if the changelog mentioned that so that. > Signed-off-by: Wan Jiabing Acked-by: Michal Hocko > --- > include/linux/memcontrol.h | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/include/linux/memcontrol.h b/in

Re: [External] Re: [PATCH v2 1/8] mm/cma: change cma mutex to irq safe spinlock

2021-03-30 Thread Michal Hocko
On Tue 30-03-21 16:08:36, Muchun Song wrote: > On Tue, Mar 30, 2021 at 4:01 PM Michal Hocko wrote: > > > > On Mon 29-03-21 16:23:55, Mike Kravetz wrote: > > > Ideally, cma_release could be called from any context. However, that is > > > not possible because a

Re: [PATCH v2 6/8] hugetlb: change free_pool_huge_page to remove_pool_huge_page

2021-03-30 Thread Michal Hocko
accounting effectively > reverting the commit. > > Signed-off-by: Mike Kravetz Please drop INIT_LIST_HEAD which seems to be a left over from rebasing to use LIST_HEAD. Acked-by: Michal Hocko > --- > mm/hugetlb.c | 95 +--- >

Re: [PATCH v2 2/8] hugetlb: no need to drop hugetlb_lock to call cma_release

2021-03-30 Thread Michal Hocko
On Mon 29-03-21 16:23:56, Mike Kravetz wrote: > Now that cma_release is non-blocking and irq safe, there is no need to > drop hugetlb_lock before calling. > > Signed-off-by: Mike Kravetz Acked-by: Michal Hocko > --- > mm/hugetlb.c | 6 -- > 1 file changed, 6 deletio

Re: [PATCH v2 1/8] mm/cma: change cma mutex to irq safe spinlock

2021-03-30 Thread Michal Hocko
ma_bitmap_maxno(cma); > + unsigned long flags; > > - mutex_lock(>lock); > + spin_lock_irqsave(>lock, flags); > for (;;) { > start = find_next_zero_bit(cma->bitmap, bitmap_maxno, end); > if (start >= bitmap_maxno) > @@ -61,7 +63,7 @@ static int cma_maxchunk_get(void *data, u64 *val) > end = find_next_bit(cma->bitmap, bitmap_maxno, start); > maxchunk = max(end - start, maxchunk); > } > - mutex_unlock(>lock); > + spin_unlock_irqrestore(>lock, flags); > *val = (u64)maxchunk << cma->order_per_bit; > > return 0; and here. -- Michal Hocko SUSE Labs

Re: [External] [PATCH 7/8] hugetlb: make free_huge_page irq safe

2021-03-29 Thread Michal Hocko
know since when it is possible to use hugetlb in the networking context? Maybe this is possible since ever but I am wondering why the lockdep started complaining only now. Maybe just fuzzing finally started using this setup which nobody does normally. -- Michal Hocko SUSE Labs

Re: [PATCH 1/8] mm: cma: introduce cma_release_nowait()

2021-03-29 Thread Michal Hocko
could have missed something. If this is really a practical concern then we can try a more complex solution based on some data. -- Michal Hocko SUSE Labs

Re: [PATCH] mm/page_alloc: try oom if reclaim is unable to make forward progress

2021-03-26 Thread Michal Hocko
ld be focusing on the compaction retry logic and see whether we can have some "run away" scenarios there. Seeing so many retries without compaction bailing out sounds like a bug in that retry logic. Vlastimil is much more familiar with that. -- Michal Hocko SUSE Labs

Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added memory range

2021-03-26 Thread Michal Hocko
On Fri 26-03-21 15:53:41, David Hildenbrand wrote: > On 26.03.21 15:38, Michal Hocko wrote: > > On Fri 26-03-21 09:52:58, David Hildenbrand wrote: [...] > > > 2. We won't allocate kasan shadow memory. We most probably have to do it > > > explicitl

Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added memory range

2021-03-26 Thread Michal Hocko
lock, but not the memory hotplug lock. E.g., for get_online_mems(). Might > have to move that out online_pages. Could you be more explicit why this locking is needed? What it would protect from for vmemmap pages? -- Michal Hocko SUSE Labs

  1   2   3   4   5   6   7   8   9   10   >