Re: [PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

he other file systems. Thanks a lot! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vg

Re: [PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

ked Kirill command line, so I can have a apple-to-apple comparison then. I will update the patch with new results soon. Sorry about the mess-up previously. I should have asked Kirill about the test in the first place. Best wishes, -- Ning Qu On Mon, Mar 3, 2014 at 2:38 PM, Andrew Morton wrote: &g

Re: [PATCH 1/1] mm: implement ->map_pages for shmem/tmpfs

Thanks for the updates! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Mon, Mar 3, 2014 at 3:07 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> Btw, should we first check if page returned by radix_tree_deref_slot is NULL? > > Yes, we

Re: [PATCH 1/1] mm: implement ->map_pages for shmem/tmpfs

) break; repeat: page = radix_tree_deref_slot(slot); + if (unlikely(!page)) + continue; if (radix_tree_exception(page)) { if (radix_tree_deref_retry(page)) Best wishes, -- Ning Qu (曲宁) | Software

Re: [PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

/bin/time -a ./iozone —B s 8g -i 0 -i 2 Best wishes, -- Ning Qu On Fri, Feb 28, 2014 at 10:10 PM, Ning Qu wrote: > Yes, I am using the iozone -i 0 -i 1. Let me try the most simple test > as you mentioned. > Best wishes, > -- > Ning Qu > > > On Fri, Feb 28, 2014 at 5:41 PM

Re: [PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

Yes, I am using the iozone -i 0 -i 1. Let me try the most simple test as you mentioned. Best wishes, -- Ning Qu On Fri, Feb 28, 2014 at 5:41 PM, Andrew Morton wrote: > On Fri, 28 Feb 2014 16:35:16 -0800 Ning Qu wrote: > >> Sorry about my fault about the experiments, here is

Re: [PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

Baselinewith-patch 1 thread minor fault 8,389,315 6,423,386 time, seconds11.68 10.51 Best wishes, -- Ning Qu -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More

[PATCH 1/1] mm: implement ->map_pages for shmem/tmpfs

In shmem/tmpfs, we also use the generic filemap_map_pages, seems the additional checking is not worth a separate version of map_pages for it. Signed-off-by: Ning Qu --- mm/shmem.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/shmem.c b/mm/shmem.c index 1f18c9d..2ea4e89 100644 --- a/mm

[PATCH 0/1] mm, shmem: map few pages around fault address if they are in page cache

time, seconds 7.947.82 Random access 8GiB file Baselinewith-patch 1 thread minor fault 724 623 time, seconds 9.75 9.84 Ning Qu (1): mm: implement ->map_pages for shmem/tmpfs mm/shmem

[PATCHv2 12/13] mm, thp, tmpfs: enable thp page cache in tmpfs

>From inode, mark to enable thp in the page cache for tmpfs Signed-off-by: Ning Qu --- mm/Kconfig | 4 ++-- mm/shmem.c | 5 + 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index 562f12f..4d2f90f 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -42

[PATCHv2 13/13] mm, thp, tmpfs: misc fixes for thp tmpfs

1) get rid of the actor function pointer in shm as what Kirill did in generic file operations. 2) add kernel command line option to turn on/off the thp page cache support. Signed-off-by: Ning Qu --- mm/huge_memory.c | 27 +++ mm/shmem.c | 7 --- 2 files

[PATCHv2 11/13] mm, thp, tmpfs: only alloc small pages in shmem_file_splice_read

We just hope this is not a common case path. The huge page can't be added without completely refactoring the code. Signed-off-by: Ning Qu --- mm/shmem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 48b1d84..c42331a 100644 --- a/mm/shmem.c +++ b/mm/sh

[PATCHv2 09/13] mm, thp, tmpfs: huge page support in do_shmem_file_read

Support huge page in do_shmem_file_read when possible. Still have room to improve, since we re-search the page in page cache everytime, but for huge page, we might save some searches and reuse the huge page for the next read across page boundary. Signed-off-by: Ning Qu --- mm/shmem.c | 47

[PATCHv2 10/13] mm, thp, tmpfs: huge page support in shmem_fallocate

Try to allocate huge page if the range fits, otherwise, fall back to small pages. Signed-off-by: Ning Qu --- mm/shmem.c | 24 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 1764a29..48b1d84 100644 --- a/mm/shmem.c +++ b/mm

[PATCHv2 06/13] mm, thp, tmpfs: request huge page in shm_fault when needed

Add the function to request huge page in shm_fault when needed. And it will fall back to regular page if huge page can't be satisfied or allocated. If small page requested but huge page is found, the huge page will be splitted. Signed-off-by: Ning Qu --- mm/shmem.c

[PATCHv2 07/13] mm, thp, tmpfs: initial support for huge page in write_begin/write_end in tmpfs

For now we try to grab a huge cache page if the minimum requirements have been satisfied. Signed-off-by: Ning Qu --- mm/shmem.c | 30 +- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 0dd6689..af56731 100644 --- a/mm

[PATCHv2 08/13] mm, thp, tmpfs: handle huge page in shmem_undo_range for truncate

When comes to truncate file, add support to handle huge page in the truncate range. Signed-off-by: Ning Qu --- mm/shmem.c | 85 ++ 1 file changed, 74 insertions(+), 11 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index af56731

[PATCHv2 05/13] mm, thp, tmpfs: split huge page when moving from page cache to swap

n for each file system. Signed-off-by: Ning Qu --- include/linux/huge_mm.h | 2 ++ mm/shmem.c | 79 - 2 files changed, 61 insertions(+), 20 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 65f90db..58

[PATCHv2 04/13] mm, thp, tmpfs: handle huge page cases in shmem_getpage_gfp

plify the job of caller. Right now shmem_getpage does: 1) simply request a page, default as a small page 2) or caller specify a flag to request either a huge page or a small page, then leave the caller to decide how to use it Signed-off-by: Ning Qu --- mm/shmem.c

[PATCHv2 02/13] mm, thp, tmpfs: add function to alloc huge page for tmpfs

Add function to alloc huge page for tmpfs when needed. Signed-off-by: Ning Qu --- mm/shmem.c | 29 + 1 file changed, 29 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 8297623..a857ba8 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -949,6 +949,28 @@ static

[PATCHv2 01/13] mm, thp: extract the common code from add_to_page_cache_locked

Extract the common code from add_to_page_cache_locked so that it could be shared by shmem file system. Signed-off-by: Ning Qu --- include/linux/pagemap.h | 2 ++ mm/filemap.c| 91 - 2 files changed, 61 insertions(+), 32 deletions

[PATCHv2 03/13] mm, thp, tmpfs: support to add huge page into page cache for tmpfs

For replacing a page inside page cache, we assume the huge page has been splitted before getting here. For adding a new page to page cache, huge page support has been added. Also refactor the shm_add_to_page_cache function. Signed-off-by: Ning Qu --- mm/shmem.c | 73

[PATCHv2 00/13] Transparent huge page cache support on tmpfs

e design in the current patchset and where we should be heading. Ning Qu (13): mm, thp: extract the common code from add_to_page_cache_locked mm, thp, tmpfs: add function to alloc huge page for tmpfs mm, thp, tmpfs: support to add huge page into page cache for tmpfs mm, thp, tmpfs: handle hug

Re: [PATCH 11/12] mm, thp, tmpfs: enable thp page cache in tmpfs

! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Fri, Oct 18, 2013 at 11:24 AM, Ning Qu wrote: > I guess this is the last review I have for this around, but not sure > what's the best solution right now. > > Kirill, do you think it'

Re: [PATCH 11/12] mm, thp, tmpfs: enable thp page cache in tmpfs

anks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 11:42 AM, Ning Qu wrote: > I agree with this. It has been like this just for a quick proof, but I > need to address this problem as soon as possible. > > Thanks! > Best

Re: [PATCH 07/12] mm, thp, tmpfs: handle huge page in shmem_undo_range for truncate

Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed, Oct 16, 2013 at 5:09 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> > Again. Here and below ifdef is redundant: PageTransHugeCache() is zero >> > compile-time and thp case

Re: [PATCH 02/12] mm, thp, tmpfs: support to add huge page into page cache for tmpfs

ge_zone(page), NR_SHMEM, nr); radix_tree_preload_end(); spin_unlock_irq(&mapping->tree_lock); if (error) page_cache_release(page); return error; Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed

Re: [PATCH 07/12] mm, thp, tmpfs: handle huge page in shmem_undo_range for truncate

Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed, Oct 16, 2013 at 5:09 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> > Again. Here and below ifdef is redundant: PageTransHugeCache() is zero >> > compile-time and thp case

Re: [PATCH 05/12] mm, thp, tmpfs: request huge page in shm_fault when needed

Fixed. Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 11:49 AM, Ning Qu wrote: > Will fix this. > Best wishes, > -- > Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 > > > On Tue, Oc

Re: [PATCH 02/12] mm, thp, tmpfs: support to add huge page into page cache for tmpfs

Yes, I guess I can if I just put whatever inside the spin lock into a common function. Thanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed, Oct 16, 2013 at 5:26 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> Yes, I can try. Th

Re: [PATCH 03/12] mm, thp, tmpfs: handle huge page cases in shmem_getpage_gfp

Great! Thanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed, Oct 16, 2013 at 5:11 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> you mean something like this? If so, then fixed. >> >>

Re: [PATCH 04/12] mm, thp, tmpfs: split huge page when moving from page cache to swap

Let me take another look at that logic. Thanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 3:33 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> in shmem_writepage, we have to split the huge page when moving page

Re: [PATCH 03/12] mm, thp, tmpfs: handle huge page cases in shmem_getpage_gfp

Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 3:29 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> We don't support huge page when page is moved from page cache to swap. >> So in this function, we enable h

Re: [PATCH 05/12] mm, thp, tmpfs: request huge page in shm_fault when needed

Will fix this. Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 3:37 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> Add the function to request huge page in shm_fault when needed. >> And it will fall back to regul

Re: [PATCH 07/12] mm, thp, tmpfs: handle huge page in shmem_undo_range for truncate

Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 4:01 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> When comes to truncate file, add support to handle huge page in the >> truncate range. >> >

Re: [PATCH 11/12] mm, thp, tmpfs: enable thp page cache in tmpfs

I agree with this. It has been like this just for a quick proof, but I need to address this problem as soon as possible. Thanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Tue, Oct 15, 2013 at 4:09 AM, Kirill A. Shutemov wrote: > Ning Qu wr

Re: [PATCH 02/12] mm, thp, tmpfs: support to add huge page into page cache for tmpfs

: __mod_zone_page_state(page_zone(page), NR_SHMEM, nr); Which means we need to tell if it's coming from shm or not, is that OK to add additional parameter just for that? Or is there any other better way we can infer that information? Thanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@googl

[PATCH 12/12] mm, thp, tmpfs: misc fixes for thp tmpfs

1) get rid of the actor function pointer in shm as what Kirill did in generic file operations. 2) add kernel command line option to turn on/off the thp page cache support. Signed-off-by: Ning Qu --- mm/huge_memory.c | 27 +++ mm/shmem.c | 7 --- 2 files

[PATCH 11/12] mm, thp, tmpfs: enable thp page cache in tmpfs

Signed-off-by: Ning Qu --- mm/Kconfig | 4 ++-- mm/shmem.c | 5 + 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index 562f12f..4d2f90f 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -428,8 +428,8 @@ config TRANSPARENT_HUGEPAGE_PAGECACHE help

[PATCH 09/12] mm, thp, tmpfs: huge page support in shmem_fallocate

Try to allocate huge page if the range fits, otherwise, fall back to small pages. Signed-off-by: Ning Qu --- mm/shmem.c | 27 +++ 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 7065ae5..cbf01ce 100644 --- a/mm/shmem.c +++ b

[PATCH 10/12] mm, thp, tmpfs: only alloc small pages in shmem_file_splice_read

We just hope this is not a common case path. The huge page can't be added without completely refactoring the code. Signed-off-by: Ning Qu --- mm/shmem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index cbf01ce..75c0ac6 100644 --- a/mm/shmem.c +++ b/mm/sh

[PATCH 06/12] mm, thp, tmpfs: initial support for huge page in write_begin/write_end in tmpfs

For now we try to grab a huge cache page if the minimum requirements have been satisfied. Signed-off-by: Ning Qu --- mm/shmem.c | 31 ++- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 2fc450d..0a423a9 100644 --- a/mm

[PATCH 08/12] mm, thp, tmpfs: huge page support in do_shmem_file_read

Support huge page in do_shmem_file_read when possible. Still have room to improve, since we re-search the page in page cache everytime, but for huge page, we might save some searches and reuse the huge page for the next read across page boundary. Signed-off-by: Ning Qu --- mm/shmem.c | 52

[PATCH 04/12] mm, thp, tmpfs: split huge page when moving from page cache to swap

in shmem_writepage, we have to split the huge page when moving pages from page cache to swap because we don't support huge page in swap yet. Signed-off-by: Ning Qu --- mm/shmem.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 8f

[PATCH 07/12] mm, thp, tmpfs: handle huge page in shmem_undo_range for truncate

When comes to truncate file, add support to handle huge page in the truncate range. Signed-off-by: Ning Qu --- mm/shmem.c | 97 +++--- 1 file changed, 86 insertions(+), 11 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 0a423a9

[PATCH 05/12] mm, thp, tmpfs: request huge page in shm_fault when needed

Add the function to request huge page in shm_fault when needed. And it will fall back to regular page if huge page can't be satisfied or allocated. If small page requested but huge page is found, the huge page will be splitted. Signed-off-by: Ning Qu --- mm/shmem.c

[PATCH 02/12] mm, thp, tmpfs: support to add huge page into page cache for tmpfs

For replacing a page inside page cache, we assume the huge page has been splitted before getting here. For adding a new page to page cache, huge page support has been added. Also refactor the shm_add_to_page_cache function. Signed-off-by: Ning Qu --- mm/shmem.c | 97

[PATCH 03/12] mm, thp, tmpfs: handle huge page cases in shmem_getpage_gfp

plify the job of caller. Right now shmem_getpage does: 1) simply request a page, default as a small page 2) or caller specify a flag to request either a huge page or a small page, then leave the caller to decide how to use it Signed-off-by: Ning Qu --- mm/shmem.c

[PATCH 01/12] mm, thp, tmpfs: add function to alloc huge page for tmpfs

Add function to alloc huge page for tmpfs when needed. Signed-off-by: Ning Qu --- mm/shmem.c | 29 + 1 file changed, 29 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 8297623..a857ba8 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -949,6 +949,28 @@ static

[PATCH 00/12] Transparent huge page cache support on tmpfs

he current patchset and where we should be heading to. Ning Qu (12): mm, thp, tmpfs: add function to alloc huge page for tmpfs mm, thp, tmpfs: support to add huge page into page cache for tmpfs mm, thp, tmpfs: handle huge page cases in shmem_getpage_gfp mm, thp, tmpfs: split huge page w

Re: [PATCHv6 00/22] Transparent huge page cache: phase 1, everything but mmap()

which is the beauty of thp. Before that, we have been using hugetlbfs, then we have to reserve a huge amount of memory at boot time, no matter those memory will be used or not. It is working but no other major services could ever share the server resources anymore. Best wishes, -- Ning Q

Re: [PATCHv6 00/22] Transparent huge page cache: phase 1, everything but mmap()

I suppose sysv shm and tmpfs share the same code base now, so both of them will benefit from thp page cache? And for Kirill's previous patchset (till v4), it contains mmap support as well. I suppose the patchset got splitted into smaller group so it's easier to review Best wishes

Re: [PATCHv6 00/22] Transparent huge page cache: phase 1, everything but mmap()

Yes, I agree. For our case, we have tens of GB files and thp with page cache does improve the number as expected. And compared to hugetlbfs (static huge page), it's more flexible and beneficial to the system wide Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com

Re: [PATCHv6 10/22] thp, mm: rewrite delete_from_page_cache() to support huge pages

Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Mon, Sep 23, 2013 at 5:05 AM, Kirill A. Shutemov wrote: > > As with add_to_page_cache_locked() we handle HPAGE_CACHE_NR pages a > time. > > Signed-off-by: Kirill A. Shutemov > --- &g

Re: [PATCHv6 00/22] Transparent huge page cache: phase 1, everything but mmap()

Got you. THanks! Best wishes, -- Ning Qu (曲宁) | Software Engineer | qun...@google.com | +1-408-418-6066 On Wed, Sep 25, 2013 at 2:23 AM, Kirill A. Shutemov wrote: > Ning Qu wrote: >> Hi, Kirill, >> >> Seems you dropped one patch in v5, is that intentional? Just wonderin

[PATCH] thp: Fix deadlock situation in vma_adjust with huge page in page cache

this is causing deadlock situation. This fix is to move the vma_adjust_trans_huge before grab the lock for file, the same as what the function is currently doing for anonymous memory. Tested, everything works fine so far. Signed-off-by: Ning Qu --- mm/mmap.c | 4 ++-- 1 file changed, 2 insertions

[PATCH] thp: Fix deadlock situation in vma_adjust with huge page in page cache.

this is causing deadlock situation. This fix is to move the vma_adjust_trans_huge before grab the lock for file, the same as what the function is currently doing for anonymous memory. Tested, everything works fine so far. Signed-off-by: Ning Qu --- mm/mmap.c | 4 ++-- 1 file changed, 2