Re: [v2 PATCH 6/7] mm: migrate: check mapcount for THP instead of ref count

2021-04-15 Thread Zi Yan
On 15 Apr 2021, at 2:45, Huang, Ying wrote: > "Zi Yan" writes: > >> On 13 Apr 2021, at 23:00, Huang, Ying wrote: >> >>> Yang Shi writes: >>> >>>> The generic migration path will check refcount, so no need check refcount >>>&g

Re: [PATCH] mm: Optimise nth_page for contiguous memmap

2021-04-14 Thread Zi Yan
MEM_VMEMMAP) > #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n)) > +#else > +#define nth_page(page,n) ((page) + (n)) > +#endif > > /* to align the pointer to the (next) page boundary */ > #define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE) > -- > 2.30.2 LGTM. Thanks. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [v2 PATCH 6/7] mm: migrate: check mapcount for THP instead of ref count

2021-04-14 Thread Zi Yan
On 13 Apr 2021, at 23:00, Huang, Ying wrote: > Yang Shi writes: > >> The generic migration path will check refcount, so no need check refcount >> here. >> But the old code actually prevents from migrating shared THP (mapped by >> multiple >> processes), so bail out early if mapcount is > 1 to

[PATCH v8 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-31 Thread Zi Yan
From: Zi Yan We did not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

[PATCH v8 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-31 Thread Zi Yan
From: Zi Yan Further extend /split_huge_pages to accept ",," for file-backed THP split tests since tmpfs may have file backed by THP that mapped nowhere. Update selftest program to test file-backed THP split too. Suggested-by: Kirill A. Shutemov Signed-off-by: Zi Yan Reviewed-by

Re: [PATCH v7 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-31 Thread Zi Yan
On 31 Mar 2021, at 12:44, Matthew Wilcox wrote: > On Mon, Mar 29, 2021 at 11:39:32AM -0400, Zi Yan wrote: >> +for (off_cur = off_start; off_cur < off_end;) { >> +struct page *fpage = pagecache_get_page(mapping, off_cur, >> +

[PATCH v7 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-29 Thread Zi Yan
From: Zi Yan Further extend /split_huge_pages to accept ",," for file-backed THP split tests since tmpfs may have file backed by THP that mapped nowhere. Update selftest program to test file-backed THP split too. Suggested-by: Kirill A. Shutemov Signed-off-by: Zi Yan Reviewed-by

[PATCH v7 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-29 Thread Zi Yan
From: Zi Yan We did not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

Re: [PATCH 0/3] Cleanup for khugepaged

2021-03-25 Thread Zi Yan
c | 14 -- > 1 file changed, 4 insertions(+), 10 deletions(-) All looks good to me. Thanks. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v6 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-24 Thread Zi Yan
On 24 Mar 2021, at 15:16, David Rientjes wrote: > On Mon, 22 Mar 2021, Zi Yan wrote: > >> From: Zi Yan >> >> We did not have a direct user interface of splitting the compound page >> backing a THP and there is no need unless we want to expose the THP >> i

[PATCH v6 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-22 Thread Zi Yan
From: Zi Yan Further extend /split_huge_pages to accept ",," for file-backed THP split tests since tmpfs may have file backed by THP that mapped nowhere. Update selftest program to test file-backed THP split too. Suggested-by: Kirill A. Shutemov Signed-off-by: Zi Yan Reviewed-by

[PATCH v6 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-22 Thread Zi Yan
From: Zi Yan We did not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

Re: [PATCH v5 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-21 Thread Zi Yan
On 19 Mar 2021, at 19:37, Yang Shi wrote: > On Thu, Mar 18, 2021 at 5:52 PM Zi Yan wrote: >> >> From: Zi Yan >> >> We did not have a direct user interface of splitting the compound page >> backing a THP and there is no need unless we want to expose the THP &

[PATCH v5 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-18 Thread Zi Yan
From: Zi Yan We did not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

[PATCH v5 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-18 Thread Zi Yan
From: Zi Yan Further extend /split_huge_pages to accept ",," for file-backed THP split tests since tmpfs may have file backed by THP that mapped nowhere. Update selftest program to test file-backed THP split too. Suggested-by: Kirill A. Shutemov Signed-off-by: Zi Yan --- mm/hug

Re: [PATCH v3 5/6] mm/huge_memory.c: remove unused macro TRANSPARENT_HUGEPAGE_DEBUG_COW_FLAG

2021-03-18 Thread Zi Yan
e changed, 3 deletions(-) LGTM. Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v3 3/6] mm/huge_memory.c: rework the function do_huge_pmd_numa_page() slightly

2021-03-18 Thread Zi Yan
mm/huge_memory.c | 11 +-- > 1 file changed, 5 insertions(+), 6 deletions(-) > LGTM. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v3 2/6] mm/huge_memory.c: make get_huge_zero_page() return bool

2021-03-18 Thread Zi Yan
ertions(+), 4 deletions(-) > LGTM. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v4 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-17 Thread Zi Yan
On 16 Mar 2021, at 19:18, Yang Shi wrote: > On Mon, Mar 15, 2021 at 1:34 PM Zi Yan wrote: >> >> From: Zi Yan >> >> Further extend /split_huge_pages to accept >> ",," for file-backed THP split tests since >> tmpfs may have file backed by THP t

Re: [PATCH v4 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-17 Thread Zi Yan
On 16 Mar 2021, at 18:23, Yang Shi wrote: > On Mon, Mar 15, 2021 at 1:34 PM Zi Yan wrote: >> >> From: Zi Yan >> >> We did not have a direct user interface of splitting the compound page >> backing a THP and there is no need unless we want to expose the THP &

Re: [PATCH v4 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-15 Thread Zi Yan
/master next-20210315] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch] > > url: > https://github.com/0day-ci/linux/commits/Zi-Yan/mm-huge_memor

[PATCH v4 1/2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-15 Thread Zi Yan
From: Zi Yan We did not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

[PATCH v4 2/2] mm: huge_memory: debugfs for file-backed THP split.

2021-03-15 Thread Zi Yan
From: Zi Yan Further extend /split_huge_pages to accept ",," for file-backed THP split tests since tmpfs may have file backed by THP that mapped nowhere. Update selftest program to test file-backed THP split too. Suggested-by: Kirill A. Shutemov Signed-off-by: Zi Yan --- mm/hug

Re: [PATCH v3] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-15 Thread Zi Yan
On 15 Mar 2021, at 8:07, Kirill A. Shutemov wrote: > On Thu, Mar 11, 2021 at 07:57:12PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> We do not have a direct user interface of splitting the compound page >> backing a THP > > But we do. You expand it. > &g

Re: [PATCH v1 06/14] mm, x86: support the access bit on non-leaf PMD entries

2021-03-14 Thread Zi Yan
On 14 Mar 2021, at 20:03, Yu Zhao wrote: > On Sun, Mar 14, 2021 at 10:51:03PM +, Matthew Wilcox wrote: >> On Sun, Mar 14, 2021 at 06:12:42PM -0400, Zi Yan wrote: >>> On 13 Mar 2021, at 2:57, Yu Zhao wrote: >>> >>>> Some architectures support t

Re: [PATCH v1 00/14] Multigenerational LRU

2021-03-14 Thread Zi Yan
On 13 Mar 2021, at 2:57, Yu Zhao wrote: > TLDR > > The current page reclaim is too expensive in terms of CPU usage and > often making poor choices about what to evict. We would like to offer > a performant, versatile and straightforward augment. > > Repo > > git fetch

Re: [PATCH v1 06/14] mm, x86: support the access bit on non-leaf PMD entries

2021-03-14 Thread Zi Yan
On 13 Mar 2021, at 2:57, Yu Zhao wrote: > Some architectures support the accessed bit on non-leaf PMD entries > (parents) in addition to leaf PTE entries (children) where pages are > mapped, e.g., x86_64 sets the accessed bit on a parent when using it > as part of linear-address translation [1].

[PATCH v3] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-11 Thread Zi Yan
From: Zi Yan We do not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Make /split_huge_pages accept a new command to do that. By writing ",," to /split_huge_pages, T

Re: [PATCH v2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-11 Thread Zi Yan
On 10 Mar 2021, at 20:12, Yang Shi wrote: > On Wed, Mar 10, 2021 at 7:36 AM Zi Yan wrote: >> >> From: Zi Yan >> >> We do not have a direct user interface of splitting the compound page >> backing a THP and there is no need unless we want to expose the THP &

Re: [RFC PATCH 0/3] hugetlb: add demote/split page functionality

2021-03-10 Thread Zi Yan
On 10 Mar 2021, at 12:05, Michal Hocko wrote: > On Wed 10-03-21 11:46:57, Zi Yan wrote: >> On 10 Mar 2021, at 11:23, Michal Hocko wrote: >> >>> On Mon 08-03-21 16:18:52, Mike Kravetz wrote: >>> [...] >>>> Converting larger to smaller hugetlb pages ca

Re: [RFC PATCH 0/3] hugetlb: add demote/split page functionality

2021-03-10 Thread Zi Yan
On 10 Mar 2021, at 11:23, Michal Hocko wrote: > On Mon 08-03-21 16:18:52, Mike Kravetz wrote: > [...] >> Converting larger to smaller hugetlb pages can be accomplished today by >> first freeing the larger page to the buddy allocator and then allocating >> the smaller pages. However, there are

[PATCH v2] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-10 Thread Zi Yan
From: Zi Yan We do not have a direct user interface of splitting the compound page backing a THP and there is no need unless we want to expose the THP implementation details to users. Adding an interface for debugging. By writing ",," to /split_huge_pages_in_range_pid, THPs within

Re: [PATCH v2 1/5] userfaultfd: support minor fault handling for shmem

2021-03-09 Thread Zi Yan
On 1 Mar 2021, at 19:01, Axel Rasmussen wrote: > Modify the userfaultfd register API to allow registering shmem VMAs in > minor mode. Modify the shmem mcopy implementation to support > UFFDIO_CONTINUE in order to resolve such faults. > > Combine the shmem mcopy handler functions into a single >

Re: [PATCH v4 3/3] x86/vmemmap: Handle unpopulated sub-pmd ranges

2021-03-09 Thread Zi Yan
On 1 Mar 2021, at 3:32, Oscar Salvador wrote: > When the size of a struct page is not multiple of 2MB, sections do > not span a PMD anymore and so when populating them some parts of the > PMD will remain unused. > Because of this, PMDs will be left behind when depopulating sections > since

Re: [PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-08 Thread Zi Yan
ter] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch] > > url: > https://github.com/0day-ci/linux/commits/Zi-Yan/mm-huge_memory-a-new-de

Re: [PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-08 Thread Zi Yan
On 8 Mar 2021, at 13:11, David Hildenbrand wrote: > On 08.03.21 18:49, Zi Yan wrote: >> On 8 Mar 2021, at 11:17, David Hildenbrand wrote: >> >>> On 08.03.21 16:22, Zi Yan wrote: >>>> From: Zi Yan >>>> >>>> By writing &quo

Re: [PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-08 Thread Zi Yan
+ the rest of cc back and move your reply inline. On 8 Mar 2021, at 12:47, Mika Penttilä wrote: >> >> >> On 8.3.2021 17.22, Zi Yan wrote: >>> From: Zi Yan >>> >>> By writing ",," to >>> /split_huge_pages_in_range_pid, THPs in the pr

Re: [PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-08 Thread Zi Yan
On 8 Mar 2021, at 11:17, David Hildenbrand wrote: > On 08.03.21 16:22, Zi Yan wrote: >> From: Zi Yan >> >> By writing ",," to >> /split_huge_pages_in_range_pid, THPs in the process with the >> given pid and virtual address range are split. It

[PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.

2021-03-08 Thread Zi Yan
From: Zi Yan By writing ",," to /split_huge_pages_in_range_pid, THPs in the process with the given pid and virtual address range are split. It is used to test split_huge_page function. In addition, a selftest program is added to tools/testing/selftests/vm to utilize the interface by

Re: [PATCH v3 1/5] mm,memory_hotplug: Allocate memmap from the added memory range

2021-03-07 Thread Zi Yan
On 4 Mar 2021, at 4:59, Oscar Salvador wrote: > Physical memory hotadd has to allocate a memmap (struct page array) for > the newly added memory section. Currently, alloc_pages_node() is used > for those allocations. > > This has some disadvantages: > a) an existing memory is consumed for that

Re: [PATCH v2 2/2] mm/memcg: set memcg when split page

2021-03-04 Thread Zi Yan
; order); > + split_page_memcg(page, 1 << order); > } > EXPORT_SYMBOL_GPL(split_page); > > -- > 2.25.0 LGTM. Thanks. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v2 1/2] mm/memcg: rename mem_cgroup_split_huge_fixup to split_page_memcg

2021-03-04 Thread Zi Yan
(memcg and flag) of > the memcg needs to be set to the tail pages. > > Signed-off-by: Zhou Guanghui > --- > include/linux/memcontrol.h | 6 ++ > mm/huge_memory.c | 2 +- > mm/memcontrol.c| 15 ++- > 3 files changed, 9 insertions(+), 14 deletions(

Re: [PATCH v3 4/8] mm/rmap: Split migration into its own function

2021-03-02 Thread Zi Yan
On 26 Feb 2021, at 2:18, Alistair Popple wrote: > Migration is currently implemented as a mode of operation for > try_to_unmap_one() generally specified by passing the TTU_MIGRATION flag > or in the case of splitting a huge anonymous page TTU_SPLIT_FREEZE. > > However it does not have much in

Re: [PATCH v3 01/25] mm: Introduce struct folio

2021-03-02 Thread Zi Yan
On 2 Mar 2021, at 8:22, Matthew Wilcox wrote: > On Mon, Mar 01, 2021 at 03:26:11PM -0500, Zi Yan wrote: >>> +static inline struct folio *next_folio(struct folio *folio) >>> +{ >>> + return folio + folio_nr_pages(folio); >> >> Are you planning t

Re: [PATCH] mm/memcg: set memcg when split pages

2021-03-02 Thread Zi Yan
On 2 Mar 2021, at 2:05, Zhouguanghui (OS Kernel) wrote: > 在 2021/3/2 10:00, Zi Yan 写道: >> On 1 Mar 2021, at 20:34, Zhou Guanghui wrote: >> >>> When split page, the memory cgroup info recorded in first page is >>> not copied to tail pages. In this case,

Re: [PATCH] mm/memcg: set memcg when split pages

2021-03-01 Thread Zi Yan
On 1 Mar 2021, at 20:34, Zhou Guanghui wrote: > When split page, the memory cgroup info recorded in first page is > not copied to tail pages. In this case, when the tail pages are > freed, the uncharge operation is not performed. As a result, the > usage of this memcg keeps increasing, and the

Re: [PATCH v3 06/25] mm: Add get_folio

2021-03-01 Thread Zi Yan
*/ > - VM_BUG_ON_PAGE(page_ref_zero_or_close_to_overflow(page), page); > - page_ref_inc(page); > + get_folio(page_folio(page)); > } > > bool __must_check try_grab_page(struct page *page, unsigned int flags); > -- > 2.29.2 LGTM. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v3 05/25] mm: Add put_folio

2021-03-01 Thread Zi Yan
evmap_managed_page(>page); > return; > } > > - if (put_page_testzero(page)) > - __put_page(page); > + put_folio(folio); > } > > /* > -- > 2.29.2 LGTM. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH v3 04/25] mm/debug: Add VM_BUG_ON_FOLIO and VM_WARN_ON_ONCE_FOLIO

2021-03-01 Thread Zi Yan
d, mm) VM_BUG_ON(cond) > #define VM_WARN_ON(cond) BUILD_BUG_ON_INVALID(cond) > #define VM_WARN_ON_ONCE(cond) BUILD_BUG_ON_INVALID(cond) > #define VM_WARN_ON_ONCE_PAGE(cond, page) BUILD_BUG_ON_INVALID(cond) > +#define VM_WARN_ON_ONCE_FOLIO(cond, folio) BUILD_BUG_ON_INVALID(cond) >

Re: [PATCH v3 03/25] mm/vmstat: Add folio stat wrappers

2021-03-01 Thread Zi Yan
On 28 Jan 2021, at 2:03, Matthew Wilcox (Oracle) wrote: > Allow page counters to be more readily modified by callers which have > a folio. Name these wrappers with 'stat' instead of 'state' as requested > by Linus here: >

Re: [PATCH v3 02/25] mm: Add folio_pgdat

2021-03-01 Thread Zi Yan
> return NODE_DATA(page_to_nid(page)); > } > > +static inline pg_data_t *folio_pgdat(const struct folio *folio) > +{ > + return page_pgdat(>page); > +} > + > #ifdef SECTION_IN_PAGE_FLAGS > static inline void set_page_section(struct page *page, unsigned long sectio

Re: [PATCH v3 01/25] mm: Introduce struct folio

2021-03-01 Thread Zi Yan
On 1 Mar 2021, at 15:53, Matthew Wilcox wrote: > On Mon, Mar 01, 2021 at 03:26:11PM -0500, Zi Yan wrote: >>> +static inline struct folio *next_folio(struct folio *folio) >>> +{ >>> + return folio + folio_nr_pages(folio); >> >> Are you planning to

Re: [PATCH v3 01/25] mm: Introduce struct folio

2021-03-01 Thread Zi Yan
On 28 Jan 2021, at 2:03, Matthew Wilcox (Oracle) wrote: > We have trouble keeping track of whether we've already called > compound_head() to ensure we're not operating on a tail page. Further, > it's never clear whether we intend a struct page to refer to PAGE_SIZE > bytes or

Re: [PATCH v1] mm/page_alloc: drop pr_info_ratelimited() in alloc_contig_range()

2021-03-01 Thread Zi Yan
eason why > alloc_contig_range() failed on specific pages. > > Cc: Andrew Morton > Cc: Minchan Kim > Cc: Oscar Salvador > Cc: Michal Hocko > Cc: Vlastimil Babka > Signed-off-by: David Hildenbrand > --- LGTM. I agree that the printout is not quite useful. Review

Re: [PATCH 1/2] hugetlb: fix update_and_free_page contig page struct assumption

2021-02-18 Thread Zi Yan
On 18 Feb 2021, at 12:51, Mike Kravetz wrote: > On 2/18/21 9:40 AM, Zi Yan wrote: >> On 18 Feb 2021, at 12:32, Jason Gunthorpe wrote: >> >>> On Thu, Feb 18, 2021 at 12:27:58PM -0500, Zi Yan wrote: >>>> On 18 Feb 2021, at 12:25, Jason Gunthorpe wrote: >>&

Re: [PATCH 1/2] hugetlb: fix update_and_free_page contig page struct assumption

2021-02-18 Thread Zi Yan
On 18 Feb 2021, at 12:32, Jason Gunthorpe wrote: > On Thu, Feb 18, 2021 at 12:27:58PM -0500, Zi Yan wrote: >> On 18 Feb 2021, at 12:25, Jason Gunthorpe wrote: >> >>> On Thu, Feb 18, 2021 at 02:45:54PM +, Matthew Wilcox wrote: >>>> On Wed, Feb 17, 2021 at 1

Re: [PATCH 1/2] hugetlb: fix update_and_free_page contig page struct assumption

2021-02-18 Thread Zi Yan
On 18 Feb 2021, at 12:25, Jason Gunthorpe wrote: > On Thu, Feb 18, 2021 at 02:45:54PM +, Matthew Wilcox wrote: >> On Wed, Feb 17, 2021 at 11:02:52AM -0800, Andrew Morton wrote: >>> On Wed, 17 Feb 2021 10:49:25 -0800 Mike Kravetz >>> wrote: page structs are not guaranteed to be

Re: [PATCH v2 2/2] mm/hugetlb: refactor subpage recording

2021-02-13 Thread Zi Yan
On 11 Feb 2021, at 18:44, Mike Kravetz wrote: > On 2/11/21 12:47 PM, Zi Yan wrote: >> On 28 Jan 2021, at 16:53, Mike Kravetz wrote: >> >>> On 1/28/21 10:26 AM, Joao Martins wrote: >>>> For a given hugepage backing a VA, there's a rather ineficient >>>&

Re: [PATCH v2 2/2] mm/hugetlb: refactor subpage recording

2021-02-11 Thread Zi Yan
On 28 Jan 2021, at 16:53, Mike Kravetz wrote: > On 1/28/21 10:26 AM, Joao Martins wrote: >> For a given hugepage backing a VA, there's a rather ineficient >> loop which is solely responsible for storing subpages in GUP >> @pages/@vmas array. For each subpage we check whether it's within >> range

Re: [PATCH V5] x86/mm: Tracking linear mapping split events

2021-01-28 Thread Zi Yan
On 28 Jan 2021, at 11:41, Dave Hansen wrote: > On 1/28/21 8:33 AM, Zi Yan wrote: >>> One of the many lasting (as we don't coalesce back) sources for >>> huge page splits is tracing as the granular page >>> attribute/permission changes would force the kernel to

Re: [PATCH V5] x86/mm: Tracking linear mapping split events

2021-01-28 Thread Zi Yan
On 28 Jan 2021, at 5:49, Saravanan D wrote: > To help with debugging the sluggishness caused by TLB miss/reload, > we introduce monotonic lifetime hugepage split event counts since > system state: SYSTEM_RUNNING to be displayed as part of > /proc/vmstat in x86 servers > > The lifetime split event

Re: [PATCH v1 1/2] mm/cma: expose all pages to the buddy if activation of an area fails

2021-01-27 Thread Zi Yan
alvador > Cc: Michal Hocko > Cc: Wei Yang > Signed-off-by: David Hildenbrand > --- > mm/cma.c | 43 +-- > 1 file changed, 21 insertions(+), 22 deletions(-) LGTM. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH 2/2] mm/hugetlb: refactor subpage recording

2021-01-26 Thread Zi Yan
for (i = 0; i < thp_nr_pages(page); i++) { >> mapcount = atomic_read([i]._mapcount) + 1; > > I think we are guaranteed this for transparent huge pages. At least > for now. Zi Yan may have some thoughts for his work on 1GB transhuge > pages ... It sho

[PATCH 6/7] mm: truncate: split thp to a non-zero order if possible.

2020-11-19 Thread Zi Yan
From: Zi Yan To minimize the number of pages after a truncation, when truncating a THP, we do not need to split it all the way down to order-0. The THP has at most three parts, the part before offset, the part to be truncated, the part left at the end. Use the non-zero minimum of them to decide

[PATCH 4/7] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-19 Thread Zi Yan
From: Zi Yan It adds a new_order parameter to set new page order in page owner and uses old_order instead of nr to make the parameters look consistent. It prepares for upcoming changes to support split huge page to any lower order. Signed-off-by: Zi Yan --- include/linux/page_owner.h | 10

[PATCH 2/7] mm: huge_memory: add new debugfs interface to trigger split huge page on any page range.

2020-11-19 Thread Zi Yan
From: Zi Yan Huge pages in the process with the given pid and virtual address range are split. It is used to test split huge page function. In addition, a testing program is added to tools/testing/selftests/vm to utilize the interface by splitting PMD THPs and PTE-mapped THPs. Signed-off-by: Zi

[PATCH 5/7] mm: thp: split huge page to any lower order pages.

2020-11-19 Thread Zi Yan
From: Zi Yan To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list

[PATCH 0/7] Split huge pages to any lower order pages and selftests.

2020-11-19 Thread Zi Yan
From: Zi Yan Hi all, With Matthew's THP in pagecache patches[1], we will be able to handle any size pagecache THPs, but currently split_huge_page can only split a THP to order-0 pages. This can easily erase the benefit of having pagecache THPs, when operations like truncate might want to keep

[PATCH 7/7] mm: huge_memory: enable debugfs to split huge pages to any order.

2020-11-19 Thread Zi Yan
From: Zi Yan It is used to test split_huge_page_to_list_to_order for pagecache THPs. Also add test cases for split_huge_page_to_list_to_order via both debugfs, truncating a file, and punching holes in a file. Signed-off-by: Zi Yan --- mm/huge_memory.c | 13

[PATCH 1/7] XArray: Fix splitting to non-zero orders

2020-11-19 Thread Zi Yan
From: "Matthew Wilcox (Oracle)" Splitting an order-4 entry into order-2 entries would leave the array containing pointers to 40008000c000 instead of . This is a one-character fix, but enhance the test suite to check this case. Reported-by: Zi Yan Signed-off-b

[PATCH 3/7] mm: memcg: make memcg huge page split support any order split.

2020-11-19 Thread Zi Yan
From: Zi Yan It sets memcg information for the pages after the split. A new parameter new_order is added to tell the new page order, always 0 for now. It prepares for upcoming changes to support split huge page to any lower order. Signed-off-by: Zi Yan Reviewed-by: Ralph Campbell Acked

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-17 Thread Zi Yan
On 17 Nov 2020, at 16:22, Matthew Wilcox wrote: > On Tue, Nov 17, 2020 at 04:12:03PM -0500, Zi Yan wrote: >> On 17 Nov 2020, at 16:05, Matthew Wilcox wrote: >> >>> On Fri, Nov 13, 2020 at 05:38:01PM -0800, Roman Gushchin wrote: >>>> On Fri, Nov 13, 20

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-17 Thread Zi Yan
On 17 Nov 2020, at 16:10, Matthew Wilcox wrote: > On Wed, Nov 11, 2020 at 03:40:05PM -0500, Zi Yan wrote: >> -for (i = 0; i < nr; i++) { >> +for (i = 0; i < nr; i += (1 << new_order)) { >> page_owner = get_page_owner(page_ext); >

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-17 Thread Zi Yan
On 17 Nov 2020, at 16:05, Matthew Wilcox wrote: > On Fri, Nov 13, 2020 at 05:38:01PM -0800, Roman Gushchin wrote: >> On Fri, Nov 13, 2020 at 08:08:58PM -0500, Zi Yan wrote: >>> Matthew recently converted split_page_owner to take nr instead of order.[1] >>> But I

Re: [PATCH] docs/vm: remove unused 3 items explanation for /proc/vmstat

2020-11-16 Thread Zi Yan
moved", > "compact_pages_moved", > "compact_pagemigrate_failed", > > Signed-off-by: Alex Shi > Cc: Jonathan Corbet > Cc: Andrew Morton > Cc: Yang Shi > Cc: "Kirill A. Shutemov" > Cc: David Rientjes > Cc: Zi Yan > Cc: linux-...@vger.kern

Re: [RFC PATCH 1/6] mm: huge_memory: add new debugfs interface to trigger split huge page on any page range.

2020-11-16 Thread Zi Yan
On 16 Nov 2020, at 11:06, Kirill A. Shutemov wrote: > On Wed, Nov 11, 2020 at 03:40:03PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> Huge pages in the process with the given pid and virtual address range >> are split. It is used to test split huge page function. In add

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-16 Thread Zi Yan
On 16 Nov 2020, at 11:25, Kirill A. Shutemov wrote: > On Wed, Nov 11, 2020 at 03:40:05PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> It adds a new_order parameter to set new page order in page owner. >> It prepares for upcoming changes to support split huge

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-13 Thread Zi Yan
On 13 Nov 2020, at 19:15, Roman Gushchin wrote: On Wed, Nov 11, 2020 at 03:40:05PM -0500, Zi Yan wrote: From: Zi Yan It adds a new_order parameter to set new page order in page owner. It prepares for upcoming changes to support split huge page to any lower order. Signed-off-by: Zi Yan

Re: [RFC PATCH 4/6] mm: thp: add support for split huge page to any lower order pages.

2020-11-13 Thread Zi Yan
On 13 Nov 2020, at 19:52, Roman Gushchin wrote: > On Wed, Nov 11, 2020 at 03:40:06PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> To split a THP to any lower order pages, we need to reform THPs on >> subpages at given order and add page refcount based on the new

Re: [RFC PATCH 2/6] mm: memcg: make memcg huge page split support any order split.

2020-11-13 Thread Zi Yan
On 13 Nov 2020, at 19:23, Roman Gushchin wrote: > On Wed, Nov 11, 2020 at 03:40:04PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> It reads thp_nr_pages and splits to provided new_nr. It prepares for >> upcoming changes to support split huge page to any lower order.

Re: [RFC PATCH 1/6] mm: huge_memory: add new debugfs interface to trigger split huge page on any page range.

2020-11-12 Thread Zi Yan
On 12 Nov 2020, at 17:22, Ralph Campbell wrote: > On 11/11/20 12:40 PM, Zi Yan wrote: >> From: Zi Yan >> >> Huge pages in the process with the given pid and virtual address range >> are split. It is used to test split huge page function. In addition, >> a

Re: [RFC PATCH 5/6] mm: truncate: split thp to a non-zero order if possible.

2020-11-12 Thread Zi Yan
On 12 Nov 2020, at 17:08, Ralph Campbell wrote: > On 11/11/20 12:40 PM, Zi Yan wrote: >> From: Zi Yan >> >> To minimize the number of pages after a truncation, when truncating a >> THP, we do not need to split it all the way down to order-0. The THP has >> at

Re: [RFC PATCH 4/6] mm: thp: add support for split huge page to any lower order pages.

2020-11-12 Thread Zi Yan
On 12 Nov 2020, at 17:01, Ralph Campbell wrote: > On 11/11/20 12:40 PM, Zi Yan wrote: >> From: Zi Yan >> >> To split a THP to any lower order pages, we need to reform THPs on >> subpages at given order and add page refcount based on the new page >> or

Re: [RFC PATCH 2/6] mm: memcg: make memcg huge page split support any order split.

2020-11-12 Thread Zi Yan
On 12 Nov 2020, at 12:58, Ralph Campbell wrote: > On 11/11/20 12:40 PM, Zi Yan wrote: >> From: Zi Yan >> >> It reads thp_nr_pages and splits to provided new_nr. It prepares for >> upcoming changes to support split huge page to any lower order. >> >> Si

Re: [RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-12 Thread Zi Yan
On 12 Nov 2020, at 12:57, Ralph Campbell wrote: > On 11/11/20 12:40 PM, Zi Yan wrote: >> From: Zi Yan >> >> It adds a new_order parameter to set new page order in page owner. >> It prepares for upcoming changes to support split huge page to any lower >> o

[RFC PATCH 2/6] mm: memcg: make memcg huge page split support any order split.

2020-11-11 Thread Zi Yan
From: Zi Yan It reads thp_nr_pages and splits to provided new_nr. It prepares for upcoming changes to support split huge page to any lower order. Signed-off-by: Zi Yan --- include/linux/memcontrol.h | 5 +++-- mm/huge_memory.c | 2 +- mm/memcontrol.c| 4 ++-- 3 files

[RFC PATCH 3/6] mm: page_owner: add support for splitting to any order in split page_owner.

2020-11-11 Thread Zi Yan
From: Zi Yan It adds a new_order parameter to set new page order in page owner. It prepares for upcoming changes to support split huge page to any lower order. Signed-off-by: Zi Yan --- include/linux/page_owner.h | 7 --- mm/huge_memory.c | 2 +- mm/page_alloc.c| 2

[RFC PATCH 6/6] mm: huge_memory: enable debugfs to split huge pages to any order.

2020-11-11 Thread Zi Yan
From: Zi Yan It is used to test split_huge_page_to_list_to_order for pagecache THPs. Also add test cases for split_huge_page_to_list_to_order via both debugfs and truncating a file. Signed-off-by: Zi Yan --- mm/huge_memory.c | 13 +-- .../selftests/vm

[RFC PATCH 0/6] Split huge pages to any lower order pages.

2020-11-11 Thread Zi Yan
From: Zi Yan Hi all, With Matthew's THP in pagecache patches[1], we will be able to handle any size pagecache THPs, but currently split_huge_page can only split a THP to order-0 pages. This can easily erase the benefit of having pagecache THPs, when operations like truncate might want to keep

[RFC PATCH 1/6] mm: huge_memory: add new debugfs interface to trigger split huge page on any page range.

2020-11-11 Thread Zi Yan
From: Zi Yan Huge pages in the process with the given pid and virtual address range are split. It is used to test split huge page function. In addition, a testing program is added to tools/testing/selftests/vm to utilize the interface by splitting PMD THPs. Signed-off-by: Zi Yan --- mm

[RFC PATCH 5/6] mm: truncate: split thp to a non-zero order if possible.

2020-11-11 Thread Zi Yan
From: Zi Yan To minimize the number of pages after a truncation, when truncating a THP, we do not need to split it all the way down to order-0. The THP has at most three parts, the part before offset, the part to be truncated, the part left at the end. Use the non-zero minimum of them to decide

[RFC PATCH 4/6] mm: thp: add support for split huge page to any lower order pages.

2020-11-11 Thread Zi Yan
From: Zi Yan To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list

Re: [PATCH 5/5] mm: migrate: return -ENOSYS if THP migration is unsupported

2020-11-06 Thread Zi Yan
On 3 Nov 2020, at 8:03, Yang Shi wrote: > In the current implementation unmap_and_move() would return -ENOMEM if > THP migration is unsupported, then the THP will be split. If split is > failed just exit without trying to migrate other pages. It doesn't make > too much sense since there may be

Re: [PATCH 4/5] mm: migrate: clean up migrate_prep{_local}

2020-11-06 Thread Zi Yan
migrate.c| 8 ++-- > 3 files changed, 6 insertions(+), 14 deletions(-) > LGTM. Thanks. Reviewed-by: Zi Yan — Best Regards, Yan Zi signature.asc Description: OpenPGP digital signature

Re: [PATCH 2/5] mm: migrate: simplify the logic for handling permanent failure

2020-11-06 Thread Zi Yan
less you really mean if and only if. :) Everything else looks good to me. Thanks for making the code cleaner. With the changes above, you can add Reviewed-by: Zi Yan . > was not so before. This has no impact to current existing callsites. > > Signed-off-by: Yang Shi > --- > mm/

Re: [PATCH] mm/compaction: count pages and stop correctly during page isolation.

2020-10-30 Thread Zi Yan
On 30 Oct 2020, at 14:33, Yang Shi wrote: > On Fri, Oct 30, 2020 at 6:36 AM Michal Hocko wrote: >> >> On Fri 30-10-20 08:20:50, Zi Yan wrote: >>> On 30 Oct 2020, at 5:43, Michal Hocko wrote: >>> >>>> [Cc Vlastimil] >>>> >&g

[PATCH v3 1/2] mm/compaction: count pages and stop correctly during page isolation.

2020-10-30 Thread Zi Yan
From: Zi Yan In isolate_migratepages_block, when cc->alloc_contig is true, we are able to isolate compound pages, nr_migratepages and nr_isolated did not count compound pages correctly, causing us to isolate more pages than we thought. Count compound pages as the number of base pa

[PATCH v3 2/2] mm/compaction: stop isolation if too many pages are isolated and we have pages to migrate.

2020-10-30 Thread Zi Yan
From: Zi Yan In isolate_migratepages_block, if we have too many isolated pages and nr_migratepages is not zero, we should try to migrate what we have without wasting time on isolating. Fixes: 1da2f328fa64 (“mm,thp,compaction,cma: allow THP migration for CMA allocations”) Suggested

Re: [PATCH v2 1/2] mm/compaction: count pages and stop correctly during page isolation.

2020-10-30 Thread Zi Yan
On 30 Oct 2020, at 14:12, Matthew Wilcox wrote: > On Fri, Oct 30, 2020 at 11:57:15AM -0400, Zi Yan wrote: >> In isolate_migratepages_block, when cc->alloc_contig is true, we are >> able to isolate compound pages, nr_migratepages and nr_isolated did not >> count compound

[PATCH v2 1/2] mm/compaction: count pages and stop correctly during page isolation.

2020-10-30 Thread Zi Yan
From: Zi Yan In isolate_migratepages_block, when cc->alloc_contig is true, we are able to isolate compound pages, nr_migratepages and nr_isolated did not count compound pages correctly, causing us to isolate more pages than we thought. Use thp_nr_pages to count pages. Otherwise, we mi

  1   2   3   4   5   6   7   8   >