[PATCH -v2 10/10] mm, THP, swap: Delay splitting THP during swap out

2016-09-01 Thread Huang, Ying
From: Huang Ying In this patch, splitting huge page is delayed from almost the first step of swapping out to after allocating the swap space for the THP (Transparent Huge Page) and adding the THP into the swap cache. This will reduce lock acquiring/releasing for the locks used for the swap cache

[PATCH -v2 00/10] THP swap: Delay splitting THP during swapping out

2016-09-01 Thread Huang, Ying
From: Huang Ying This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [01/10

Re: [PATCH -v2 01/10] swap: Change SWAPFILE_CLUSTER to 512

2016-09-01 Thread Huang, Ying
Andrew Morton writes: > On Thu, 1 Sep 2016 08:16:54 -0700 "Huang, Ying" wrote: > >> From: Huang Ying >> >> In this patch, the size of the swap cluster is changed to that of the >> THP (Transparent Huge Page) on x86_64 architecture (512). This is for &

Re: [PATCH -v2 01/10] swap: Change SWAPFILE_CLUSTER to 512

2016-09-02 Thread Huang, Ying
Andrew Morton writes: > On Thu, 01 Sep 2016 16:04:57 -0700 "Huang\, Ying" > wrote: > >> >> } >> >> >> >> -#define SWAPFILE_CLUSTER 256 >> >> +#define SWAPFILE_CLUSTER 512 >> >> #define LATENCY_LIMIT

Re: [LKP] [lkp] [f2fs] ec795418c4: fsmark.files_per_sec -36.3% regression

2016-08-04 Thread Huang, Ying
Hi, Jaegeuk, "Huang, Ying" writes: > Hi, > > I checked the comparison result below and found this is a regression for > fsmark.files_per_sec, not fsmark.app_overhead. > > Best Regards, > Huang, Ying > > kernel test robot writes: > >> FYI, we notic

Re: [LKP] [lkp] [f2fs] ec795418c4: fsmark.files_per_sec -36.3% regression

2016-08-04 Thread Huang, Ying
Jaegeuk Kim writes: > On Thu, Aug 04, 2016 at 10:44:20AM -0700, Huang, Ying wrote: >> Jaegeuk Kim writes: >> >> > Hi Huang, >> > >> > On Thu, Aug 04, 2016 at 10:00:41AM -0700, Huang, Ying wrote: >> >> Hi, Jaegeuk, >> >> >

Re: [RFC 00/11] THP swap: Delay splitting THP during swapping out

2016-08-23 Thread Huang, Ying
016 at 02:33:08PM -0700, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Anyway, I hope [1/11] should be merged regardless of the patchset because >> > I believe anyone doesn't feel comfortable with cluser_info functions. ;-) >> >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

2016-08-24 Thread Huang, Ying
ramdisk on a Xeon E5 v3 machine, the swap out throughput improved 40.4%, from ~0.97GB/s to ~1.36GB/s. What's your plan for this patch? If it can be merged soon, that will be great! I found some issues in the original patch to work with swap cache. Below is my fixes to make it work for swap cac

Re: [RFC] mm: Don't use radix tree writeback tags for pages in swap cache

2016-08-24 Thread Huang, Ying
"Huang, Ying" writes: > Hi, Dave, > > Dave Hansen writes: > >> On 08/09/2016 09:17 AM, Huang, Ying wrote: >>> File pages uses a set of radix tags (DIRTY, TOWRITE, WRITEBACK) to >>> accelerate finding the pages with the specific tag in the the rad

Re: [LKP] [lkp] [f2fs] ec795418c4: fsmark.files_per_sec -36.3% regression

2016-08-24 Thread huang ying
Hi, Jaegeuk, On Thu, Aug 11, 2016 at 6:22 PM, Jaegeuk Kim wrote: > On Thu, Aug 11, 2016 at 03:49:41PM -0700, Huang, Ying wrote: >> Hi, Kim, >> >> "Huang, Ying" writes: >> >> >> >> [lkp] [f2fs] 3bdad3c7ee: aim7.jobs-per-min -25.3% regression

[PATCH] mm, swap: Add swap_cluster_list

2016-08-24 Thread Huang, Ying
From: Huang Ying This is a code clean up patch without functionality changes. The swap_cluster_list data structure and its operations are introduced to provide some better encapsulation for the free cluster and discard cluster list operations. This avoid some code duplication, improved the

Re: [RFC][PATCH 0/3] locking/mutex: Rewrite basic mutex

2016-08-25 Thread huang ying
Hi, Peter, Do you have a git tree branch for this patchset? We want to test it in 0day performance test. That will make it a little easier. Best Regards, Huang, Ying

[PATCH] mm: Don't use radix tree writeback tags for pages in swap cache

2016-08-25 Thread Huang, Ying
% perf-profile.cycles-pp._raw_spin_lock_irqsave.test_clear_page_writeback.end_page_writeback.page_endio.pmem_rw_page Cc: Hugh Dickins Cc: Shaohua Li Cc: Minchan Kim Cc: Rik van Riel Cc: Mel Gorman Cc: Tejun Heo Cc: Wu Fengguang Cc: Dave Hansen Signed-off-by: "Huang, Ying" --

[PATCH -mm] mm, swap: Fix race between swap count continuation operations

2017-10-17 Thread Huang, Ying
From: Huang Ying One page may store a set of entries of the sis->swap_map (swap_info_struct->swap_map) in multiple swap clusters. If some of the entries has sis->swap_map[offset] > SWAP_MAP_MAX, multiple pages will be used to store the set of entries of the sis->swap_map. An

[PATCH -mm] mm, pagemap: Fix soft dirty marking for PMD migration entry

2017-10-17 Thread Huang, Ying
From: Huang Ying Now, when the page table is walked in the implementation of /proc//pagemap, pmd_soft_dirty() is used for both the PMD huge page map and the PMD migration entries. That is wrong, pmd_swp_soft_dirty() should be used for the PMD migration entries instead because the different page

Re: [PATCH -mm] mm, swap: Fix race between swap count continuation operations

2017-10-17 Thread Huang, Ying
Michal Hocko writes: > On Tue 17-10-17 16:13:20, Huang, Ying wrote: >> From: Huang Ying >> >> One page may store a set of entries of the >> sis->swap_map (swap_info_struct->swap_map) in multiple swap clusters. >> If some of the entries has sis->sw

Re: [PATCH -mm] mm, pagemap: Fix soft dirty marking for PMD migration entry

2017-10-17 Thread Huang, Ying
Zi Yan writes: > Huang, Ying wrote: >> "Kirill A. Shutemov" writes: >> >>> On Tue, Oct 17, 2017 at 04:18:18PM +0800, Huang, Ying wrote: >>>> From: Huang Ying >>>> >>>> Now, when the page table is walked in the implementa

Re: [PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

2017-05-25 Thread Huang, Ying
Ming Lei writes: > On Thu, May 25, 2017 at 02:46:28PM +0800, Huang, Ying wrote: >> From: Huang Ying >> >> In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if >> CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256. This is to support >&

Re: [PATCH 1/2] mm:swap: clean up swap readahead

2017-10-31 Thread Huang, Ying
e) > - put_page(page); > + if (!pte_unmap_same(vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) > goto out; > - } The page table holding PTE may be unmapped in pte_unmap_same(), so is it safe for us to access page table after this in d

Re: [PATCH 1/2] mm:swap: clean up swap readahead

2017-10-31 Thread Huang, Ying
Minchan Kim writes: > Hi Huang, > > On Wed, Nov 01, 2017 at 01:41:00PM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> >> > When I see recent change of swap readahead, I am very unhappy >> > about current code structu

Re: [PATCH 2/2] mm:swap: unify cluster-based and vma-based swap readahead

2017-10-31 Thread Huang, Ying
Minchan Kim writes: > This patch makes do_swap_page no need to be aware of two different > swap readahead algorithm. Just unify cluster-based and vma-based > readahead function call. > > Signed-off-by: Minchan Kim > --- > include/linux/swap.h | 17 - > mm/memory.c | 11

Re: [PATCH] mm: extend reuse_swap_page range as much as possible

2017-11-01 Thread Huang, Ying
u need this? You saved copying one page from memory to memory (COW) now, at the cost of reading a page from disk to memory later? Best Regards, Huang, Ying > Signed-off-by: zhouxianrong > --- > mm/swapfile.c |9 +++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > &g

[PATCH -mm -V3] mm, swap: Fix false error message in __swp_swapcount()

2017-11-01 Thread Huang, Ying
From: Huang Ying When a page fault occurs for a swap entry, the physical swap readahead (not the VMA base swap readahead) may readahead several swap entries after the fault swap entry. The readahead algorithm calculates some of the swap entries to readahead via increasing the offset of the

Re: [PATCH -mm] mm, swap: Fix false error message in __swp_swapcount()

2017-10-24 Thread Huang, Ying
Hi, Michal, Michal Hocko writes: > On Tue 24-10-17 10:47:00, Huang, Ying wrote: >> From: Ying Huang >> >> __swp_swapcount() is used in __read_swap_cache_async(). Where the >> invalid swap entry (offset > max) may be supplied during swap >> readahead. Bu

Re: [PATCH -mm] mm, swap: Fix false error message in __swp_swapcount()

2017-10-24 Thread Huang, Ying
Michal Hocko writes: > On Tue 24-10-17 23:15:32, Huang, Ying wrote: >> Hi, Michal, >> >> Michal Hocko writes: >> >> > On Tue 24-10-17 10:47:00, Huang, Ying wrote: >> >> From: Ying Huang >> >> >> >> __swp_swapcount() i

Re: [PATCH -mm] mm, swap: Fix false error message in __swp_swapcount()

2017-10-24 Thread Huang, Ying
Minchan Kim writes: > On Tue, Oct 24, 2017 at 10:47:00AM +0800, Huang, Ying wrote: >> From: Ying Huang >> >> __swp_swapcount() is used in __read_swap_cache_async(). Where the >> invalid swap entry (offset > max) may be supplied during swap >> readahead. Bu

[PATCH -mm -V2] mm, swap: Fix false error message in __swp_swapcount()

2017-10-26 Thread Huang, Ying
From: Huang Ying When a page fault occurs for a swap entry, the physical swap readahead (not the VMA base swap readahead) may readahead several swap entries after the fault swap entry. The readahead algorithm calculates some of the swap entries to readahead via increasing the offset of the

[RFC -mm] mm, userfaultfd, THP: Avoid waiting when PMD under THP migration

2017-11-03 Thread Huang, Ying
From: Huang Ying If THP migration is enabled, the following situation is possible, - A THP is mapped at source address - Migration is started to move the THP to another node - Page fault occurs - The PMD (migration entry) is copied to the destination address in mremap That is, it is possible

Re: [RFC -mm] mm, userfaultfd, THP: Avoid waiting when PMD under THP migration

2017-11-04 Thread huang ying
On Fri, Nov 3, 2017 at 11:00 PM, Zi Yan wrote: > On 3 Nov 2017, at 3:52, Huang, Ying wrote: > >> From: Huang Ying >> >> If THP migration is enabled, the following situation is possible, >> >> - A THP is mapped at source address >> - Migration is started

[PATCH -mm -V2] mm, pagemap: Fix soft dirty marking for PMD migration entry

2017-10-19 Thread Huang, Ying
From: Huang Ying Now, when the page table is walked in the implementation of /proc//pagemap, pmd_soft_dirty() is used for both the PMD huge page map and the PMD migration entries. That is wrong, pmd_swp_soft_dirty() should be used for the PMD migration entries instead because the different page

[PATCH -mm -V2] mm, swap: Fix race between swap count continuation operations

2017-10-19 Thread Huang, Ying
From: Huang Ying One page may store a set of entries of the sis->swap_map (swap_info_struct->swap_map) in multiple swap clusters. If some of the entries has sis->swap_map[offset] > SWAP_MAP_MAX, multiple pages will be used to store the set of entries of the sis->swap_map. An

Re: [PATCH -mm -V2] mm, swap: Fix false error message in __swp_swapcount()

2017-10-30 Thread Huang, Ying
Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Fri, Oct 27, 2017 at 01:53:27PM +0800, Huang, Ying wrote: >> From: Huang Ying >> >> When a page fault occurs for a swap entry, the physical swap readahead >> (not the VMA base swap readahead) may readahead

Re: [PATCH -mm -V2] mm, swap: Fix false error message in __swp_swapcount()

2017-10-30 Thread Huang, Ying
Minchan Kim writes: > Hi Huang, > > On Tue, Oct 31, 2017 at 01:32:32PM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Oct 27, 2017 at 01:53:27PM +0800, Huang, Ying wrote: >&

Re: [lkp] [sched/fair] 98d8fd8126: -20.8% hackbench.throughput

2015-10-13 Thread Huang, Ying
12 19:27:20 /usr/bin/hackbench -g 8 --threads -l 6 > Running in threaded mode with 8 groups using 40 file descriptors each > (== 320 tasks) > Each sender will pass 6 messages of 100 bytes > ... > wait for background monitors: perf-profile uptime proc-vmstat proc-stat > mem

Re: [lkp] [x86/build] b2c51106c75: -18.1% will-it-scale.per_process_ops

2015-09-01 Thread Huang Ying
155259 ± 0% +0.0% 155259 ± 0% -42.2% 89723 ± 0% sched_debug.sysctl_sched.sysctl_sched_features Best Regards, Huang, Ying -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://v

Re: [PATCH] GHES: Fix cached error-status

2015-10-25 Thread Huang, Ying
in should be the first. So if we found one cache with too small (old) cache->time_in, we can say there are no cache with same contents and bigger (newer) cache->time_in, so that we can make decision (break) earlier. Best Regards, Huang, Ying -- To unsubscribe from this list: send the line

Re: [PATCH] GHES: Fix cached error-status

2015-10-25 Thread Huang, Ying
Borislav Petkov writes: > On Mon, Oct 26, 2015 at 11:20:35AM +0800, Huang, Ying wrote: >> In ghes_estatus_caches[], for caches with same contents, the cache with >> biggest (newest) cache->time_in should be the first. So if we found one >> cache with too small (old) ca

[PATCH] swap: Use __try_to_reclaim_swap() in free_swap_and_cache()

2018-08-03 Thread Huang Ying
swap entry can be reclaimed later eventually. Signed-off-by: "Huang, Ying" Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim --- mm/swapfile.c | 57 + 1 file changed, 25

Re: [lkp-robot] [Kbuild] 050e9baa9d: netperf.Throughput_total_tps -5.6% regression (FYI)

2018-06-21 Thread Huang, Ying
=y >> CONFIG_STACKPROTECTOR=y >> CONFIG_STACKPROTECTOR_STRONG=y >> CONFIG_CC_HAS_SANE_STACKPROTECTOR=y > > so you're testing the "no overhead" case to the "worst overhead" case. Do you have interest in some other comparison? Best Regards, Huang, Ying

[PATCH -mm -v4 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, during swapout, all PMD page mapping will be split and replaced with PTE swap mapping. And when clearing the SWAP_HAS_CACHE flag for the huge swap cluster in swapcache_free_cluster(), the huge swap cluster will be split. Now, during swapout, the PMD page mapping

[PATCH -mm -v4 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying It's unreasonable to optimize swapping for THP without basic swapping support. And this will cause build errors when THP_SWAP functions are defined in swapfile.c and called elsewhere. The comments are fixed too to reflect the latest progress. Signed-off-by: "Huang,

[PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-21 Thread Huang, Ying
From: Huang Ying Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21

[PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When a PMD swap mapping is removed from a huge swap cluster, for example, unmap a memory range mapped with PMD swap mapping, etc, free_swap_and_cache() will be called to decrease the reference count to the huge swap cluster. free_swap_and_cache() may also free or split the huge

[PATCH -mm -v4 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, the PMD swap operations are only enabled for CONFIG_ARCH_ENABLE_THP_MIGRATION. Because they are only used by the THP migration support. We will support PMD swap mapping to the huge swap cluster and swapin the THP as a whole. That will be enabled via

[PATCH -mm -v4 16/21] mm, THP, swap: Free PMD swap mapping when zap_huge_pmd()

2018-06-21 Thread Huang, Ying
From: Huang Ying For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov"

[PATCH -mm -v4 15/21] mm, THP, swap: Support to copy PMD swap mapping when fork()

2018-06-21 Thread Huang, Ying
From: Huang Ying During fork, the page table need to be copied from parent to child. A PMD swap mapping need to be copied too and the swap reference count need to be increased. When the huge swap cluster has been split already, we need to split the PMD swap mapping and fallback to PTE copying

[PATCH -mm -v4 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to

[PATCH -mm -v4 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin

2018-06-21 Thread Huang, Ying
From: Huang Ying Swapin a THP as a whole isn't desirable at some situations. For example, for random access pattern, swapin a THP as a whole will inflate the reading greatly. So a sysfs interface: /sys/kernel/mm/transparent_hugepage/swapin_enabled is added to configure it. Three optio

[PATCH -mm -v4 18/21] mm, THP, swap: Support PMD swap mapping in mincore()

2018-06-21 Thread Huang, Ying
From: Huang Ying During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov"

[PATCH -mm -v4 20/21] mm, THP, swap: create PMD swap mapping when unmap the THP

2018-06-21 Thread Huang, Ying
From: Huang Ying This is the final step of the THP swapin support. When reclaiming a anonymous THP, after allocating the huge swap cluster and add the THP into swap cache, the PMD page mapping will be changed to the mapping to the swap space. Previously, the PMD page mapping will be split

[PATCH -mm -v4 21/21] mm, THP: Avoid to split THP when reclaim MADV_FREE THP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, to reclaim MADV_FREE THP, the THP will be split firstly, then reclaim each sub-pages. This wastes cycles to split THP and unmap and free each sub-pages, and split THP even if it has been written since MADV_FREE. We have to do this because MADV_FREE THP reclaiming

[PATCH -mm -v4 19/21] mm, THP, swap: Support PMD swap mapping in common path

2018-06-21 Thread Huang, Ying
From: Huang Ying Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -mm -v4 12/21] mm, THP, swap: Support PMD swap mapping in swapoff

2018-06-21 Thread Huang, Ying
From: Huang Ying During swapoff, for a huge swap cluster, we need to allocate a THP, read its contents into the THP and unuse the PMD and PTE swap mappings to it. If failed to allocate a THP, the huge swap cluster will be split. During unuse, if it is found that the swap cluster mapped by a

[PATCH -mm -v4 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously the huge swap cluster will be split after the THP is swapout. Now, to support to swapin the THP as a whole, the huge swap cluster will not be split after the THP is reclaimed. So in memcg, we need to move the swap account for PMD swap mappings in the process's

[PATCH -mm -v4 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED

2018-06-21 Thread Huang, Ying
From: Huang Ying During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" Cc:

[PATCH -mm -v4 07/21] mm, THP, swap: Support PMD swap mapping in split_swap_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying When splitting a THP in swap cache or failing to allocate a THP when swapin a huge swap cluster, the huge swap cluster will be split. In addition to clear the huge flag of the swap cluster, the PMD swap mapping count recorded in cluster_count() will be set to 0. But we will

[PATCH -mm -v4 10/21] mm, THP, swap: Support to count THP swapin and its fallback

2018-06-21 Thread Huang, Ying
From: Huang Ying 2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device as a whole and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangel

[PATCH -mm -v4 09/21] mm, THP, swap: Swapin a THP as a whole

2018-06-21 Thread Huang, Ying
From: Huang Ying With this patch, when page fault handler find a PMD swap mapping, it will swap in a THP as a whole. This avoids the overhead of splitting/collapsing before/after the THP swapping. And improves the swap performance greatly for reduced page fault count etc

[PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-06-21 Thread Huang, Ying
From: Huang Ying To swapin a THP as a whole, we need to read a huge swap cluster from the swap device. This patch revised the __read_swap_cache_async() and its callers and callees to support this. If __read_swap_cache_async() find the swap cluster of the specified swap entry is huge, it will

[PATCH -mm -v4 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD

2018-06-21 Thread Huang, Ying
From: Huang Ying A huge PMD need to be split when zap a part of the PMD mapping etc. If the PMD mapping is a swap mapping, we need to split it too. This patch implemented the support for this. This is similar as splitting the PMD page mapping, except we need to decrease the PMD swap mapping

[PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-21 Thread Huang, Ying
From: Huang Ying To support to swapin the THP as a whole, we need to create PMD swap mapping during swapout, and maintain PMD swap mapping count. This patch implements the support to increase the PMD swap mapping count (for swapout, fork, etc.) and set SWAP_HAS_CACHE flag (for swapin, etc

[PATCH 0/3] swap: Code refactoring for some swap free related functions

2018-08-27 Thread Huang Ying
To improve the code readability. Some swap free related functions are refactored. This patchset is based on 8/23 HEAD of mmotm tree. Best Regards, Huang, Ying

[PATCH 1/3] swap: Use __try_to_reclaim_swap() in free_swap_and_cache()

2018-08-27 Thread Huang Ying
the swap entry can be reclaimed later eventually. Signed-off-by: "Huang, Ying" Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim --- mm/swapfile.c | 57 + 1 file c

[PATCH 2/3] swap: call free_swap_slot() in __swap_entry_free()

2018-08-27 Thread Huang Ying
code. Signed-off-by: "Huang, Ying" Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim --- mm/swapfile.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 40

[PATCH 3/3] swap: Clear si->swap_map[] in swap_free_cluster()

2018-08-27 Thread Huang Ying
is moved to swap_free_cluster() to avoid the downside. Signed-off-by: "Huang, Ying" Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim --- mm/swapfile.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/sw

Re: [PATCH v2] RFC: clear 1G pages with streaming stores on x86

2018-07-26 Thread Huang, Ying
; usually being 1 or 2 when idle, and only increasing to ~4-7 when stressed. >> > >> > >> > PAGES_BETWEEN_RESCHED state AVG stddev >> > 1 4 KiB idle36.086 1.920 >> > 16 64 KiB idle34.797 1.702 >> > 32 128 KiB idle35.104 1.752 >> > 64 256 KiB idle34.468 0.661 >> > 512 2048 KiBidle36.427 0.946 >> > 20488192 KiBidle34.988 2.406 >> > 262144 1048576 KiB idle36.792 0.193 >> > infin 512 GiB idle38.817 0.238 [causes softlockup] >> > 1 4 KiB stress 55.562 0.661 >> > 16 64 KiB stress 57.509 0.248 >> > 32 128 KiB stress 69.265 3.913 >> > 64 256 KiB stress 70.217 4.534 >> > 512 2048 KiBstress 68.474 1.708 >> > 20488192 KiBstress 70.806 1.068 >> > 262144 1048576 KiB stress 55.217 1.184 >> > infin 512 GiB stress 55.062 0.291 [causes softlockup] I think it may be good to separate the two optimization into 2 patches. This makes it easier to evaluate the benefit of individual optimization. Best Regards, Huang, Ying

Re: [PATCH v2 2/7] mm/swapfile.c: Replace some #ifdef with IS_ENABLED()

2018-07-18 Thread Huang, Ying
Dave Hansen writes: > On 07/17/2018 08:25 PM, Huang, Ying wrote: >>> Seriously, though, does it hurt us to add a comment or two to say >>> something like: >>> >>> /* >>> * Should not even be attempting cluster allocations when >>&

[PATCH v3 2/8] mm/swapfile.c: Replace some #ifdef with IS_ENABLED()

2018-07-19 Thread Huang Ying
is a public function with a stub implementation for CONFIG_THP_SWAP=n in swap.h. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Wi

[PATCH v3 6/8] swap, get_swap_pages: Use entry_size instead of cluster in parameter

2018-07-19 Thread Huang Ying
filename base 242152028 340 2658367d7 mm/swapfile.o head 241232004 340 264676763 mm/swapfile.o Signed-off-by: "Huang, Ying" Cc: Daniel Jordan Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH v3 0/8] swap: THP optimizing refactoring

2018-07-19 Thread Huang Ying
via merging huge/normal code path/functions if possible. One concern is that this may cause code size to dilate when !CONFIG_TRANSPARENT_HUGEPAGE. The data shows that most refactoring will only cause quite slight code size increase. Best Regards, Huang, Ying

[PATCH v3 3/8] swap: Use swap_count() in swap_page_trans_huge_swapped()

2018-07-19 Thread Huang Ying
nted out by Daniel, it is better to use "swap_count(map[i])" here, because it works for "map[i] == 0" case too. And this makes the implementation more consistent between normal and huge swap entry. Signed-off-by: "Huang, Ying" Suggested-and-reviewed-by: Daniel Jordan

[PATCH v3 1/8] swap: Add comments to lock_cluster_or_swap_info()

2018-07-19 Thread Huang Ying
To improve the code readability. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapfile.c | 6 ++ 1 file

[PATCH v3 5/8] swap: Unify normal/huge code path in put_swap_page()

2018-07-19 Thread Huang Ying
=n. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapf

[PATCH v3 4/8] swap: Unify normal/huge code path in swap_page_trans_huge_swapped()

2018-07-19 Thread Huang Ying
: "Huang, Ying" Suggested-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapfile.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/

[PATCH v3 7/8] swap: Add __swap_entry_free_locked()

2018-07-19 Thread Huang Ying
The part of __swap_entry_free() with lock held is separated into a new function __swap_entry_free_locked(). Because we want to reuse that piece of code in some other places. Just mechanical code refactoring, there is no any functional change in this function. Signed-off-by: "Huang,

[PATCH v3 8/8] swap, put_swap_page: Share more between huge/normal code path

2018-07-19 Thread Huang Ying
. More difference comes from the ORC unwinder segments: (1480 + 2220) - (1380 + 2070) = 250. If the frame pointer unwinder is used, this costs nothing. Signed-off-by: "Huang, Ying" Reviewed-by: Daniel Jordan Cc: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh D

Re: [PATCH v3 4/8] swap: Unify normal/huge code path in swap_page_trans_huge_swapped()

2018-07-19 Thread Huang, Ying
E; >> +else >> +return false; > > Nitpick: no need for an else after a return: > > if (IS_ENABLED(CONFIG_THP_SWAP)) > return info->flags & CLUSTER_FLAG_HUGE; > return false; Sure. Will change this in next version. Best Regards, Huang, Ying

Re: [PATCH v3 1/8] swap: Add comments to lock_cluster_or_swap_info()

2018-07-19 Thread Huang, Ying
Christoph Hellwig writes: > On Thu, Jul 19, 2018 at 04:48:35PM +0800, Huang Ying wrote: >> +/* >> + * Determine the locking method in use for this device. Return >> + * swap_cluster_info if SSD-style cluster-based locking is in place. >> + */ >> stati

[PATCH v4 0/8] swap: THP optimizing refactoring

2018-07-20 Thread Huang Ying
via merging huge/normal code path/functions if possible. One concern is that this may cause code size to dilate when !CONFIG_TRANSPARENT_HUGEPAGE. The data shows that most refactoring will only cause quite slight code size increase. Best Regards, Huang, Ying

[PATCH v4 1/8] swap: Add comments to lock_cluster_or_swap_info()

2018-07-20 Thread Huang Ying
To improve the code readability. Signed-off-by: "Huang, Ying" Suggested-and-acked-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapfile.c | 9 +++

[PATCH v4 4/8] swap: Unify normal/huge code path in swap_page_trans_huge_swapped()

2018-07-20 Thread Huang Ying
: "Huang, Ying" Suggested-and-acked-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapfile.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff

[PATCH v4 2/8] mm/swapfile.c: Replace some #ifdef with IS_ENABLED()

2018-07-20 Thread Huang Ying
is a public function with a stub implementation for CONFIG_THP_SWAP=n in swap.h. Signed-off-by: "Huang, Ying" Suggested-and-acked-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel C

[PATCH v4 8/8] swap, put_swap_page: Share more between huge/normal code path

2018-07-20 Thread Huang Ying
. More difference comes from the ORC unwinder segments: (1480 + 2220) - (1380 + 2070) = 250. If the frame pointer unwinder is used, this costs nothing. Signed-off-by: "Huang, Ying" Reviewed-by: Daniel Jordan Acked-by: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li

[PATCH v4 5/8] swap: Unify normal/huge code path in put_swap_page()

2018-07-20 Thread Huang Ying
=n. Signed-off-by: "Huang, Ying" Suggested-and-acked-by: Dave Hansen Reviewed-by: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dan Williams --- mm/swapf

[PATCH v4 7/8] swap: Add __swap_entry_free_locked()

2018-07-20 Thread Huang Ying
The part of __swap_entry_free() with lock held is separated into a new function __swap_entry_free_locked(). Because we want to reuse that piece of code in some other places. Just mechanical code refactoring, there is no any functional change in this function. Signed-off-by: "Huang,

[PATCH v4 6/8] swap, get_swap_pages: Use entry_size instead of cluster in parameter

2018-07-20 Thread Huang Ying
filename base 242152028 340 2658367d7 mm/swapfile.o head 241232004 340 264676763 mm/swapfile.o Signed-off-by: "Huang, Ying" Acked-by: Dave Hansen Cc: Daniel Jordan Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins C

[PATCH v4 3/8] swap: Use swap_count() in swap_page_trans_huge_swapped()

2018-07-20 Thread Huang Ying
nted out by Daniel, it is better to use "swap_count(map[i])" here, because it works for "map[i] == 0" case too. And this makes the implementation more consistent between normal and huge swap entry. Signed-off-by: "Huang, Ying" Suggested-and-reviewed-by: Daniel Jor

Re: [QUESTION] llist: Comment releasing 'must delete' restriction before traversing

2018-07-30 Thread Huang, Ying
rom the llist. Some other locks could be used to provide mutual exclusive between - llist add, llist traverse and - llist delete Is this your use case? Best Regards, Huang, Ying

Re: [QUESTION] llist: Comment releasing 'must delete' restriction before traversing

2018-07-30 Thread Huang, Ying
Byungchul Park writes: > On Tue, Jul 31, 2018 at 09:37:50AM +0800, Huang, Ying wrote: >> Byungchul Park writes: >> >> > Hello folks, >> > >> > I'm careful in saying.. and curious about.. >> > >> > In restrictive cases like only

Re: [QUESTION] llist: Comment releasing 'must delete' restriction before traversing

2018-07-31 Thread Huang, Ying
e more information about where is your use case? Is it a kernel driver? Then it is better to submit the patch together with its user. And I have the similar concern as Steven Rostedt. That is, if you are the only user forever, it's not necessary to change the common code. Best Regards, Huang, Ying

[PATCH 6/6] swap, put_swap_page: Share more between huge/normal code path

2018-07-12 Thread Huang, Ying
From: Huang Ying In this patch, locking related code is shared between huge/normal code path in put_swap_page() to reduce code duplication. And `free_entries == 0` case is merged into more general `free_entries != SWAPFILE_CLUSTER` case, because the new locking method makes it easy. The added

[PATCH 0/6] swap: THP optimizing refactoring

2018-07-12 Thread Huang, Ying
via merging huge/normal code path/functions if possible. One concern is that this may cause code size to dilate when !CONFIG_TRANSPARENT_HUGEPAGE. The data shows that most refactoring will only cause quite slight code size increase. Best Regards, Huang, Ying

[PATCH 2/6] mm/swapfile.c: Replace some #ifdef with IS_ENABLED()

2018-07-12 Thread Huang, Ying
From: Huang Ying In mm/swapfile.c, THP (Transparent Huge Page) swap specific code is enclosed by #ifdef CONFIG_THP_SWAP/#endif to avoid code dilating when THP isn't enabled. But #ifdef/#endif in .c file hurt the code readability, so Dave suggested to use IS_ENABLED(CONFIG_THP_SWAP) instea

[PATCH 1/6] swap: Add comments to lock_cluster_or_swap_info()

2018-07-12 Thread Huang, Ying
From: Huang Ying To improve the code readability. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Daniel Jordan Cc: Dan Williams --- mm/swapfile.c | 6

[PATCH 3/6] swap: Unify normal/huge code path in swap_page_trans_huge_swapped()

2018-07-12 Thread Huang, Ying
From: Huang Ying As suggested by Dave, we should unify the code path for normal and huge swap support if possible to avoid duplicated code, bugs, etc. and make it easier to review code. In this patch, the normal/huge code path in swap_page_trans_huge_swapped() is unified, the added and removed

[PATCH 4/6] swap: Unify normal/huge code path in put_swap_page()

2018-07-12 Thread Huang, Ying
From: Huang Ying In this patch, the normal/huge code path in put_swap_page() and several helper functions are unified to avoid duplicated code, bugs, etc. and make it easier to review the code. The removed lines are more than added lines. And the binary size is kept exactly same when

[PATCH 5/6] swap: Add __swap_entry_free_locked()

2018-07-12 Thread Huang, Ying
From: Huang Ying The part of __swap_entry_free() with lock held is separated into a new function __swap_entry_free_locked(). Because we want to reuse that piece of code in some other places. Just mechanical code refactoring, there is no any functional change in this function. Signed-off-by

[PATCH] mm, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-07-12 Thread Huang, Ying
From: Huang Ying CONFIG_THP_SWAP should depend on CONFIG_SWAP, because it's unreasonable to optimize swapping for THP (Transparent Huge Page) without basic swapping support. In original code, when CONFIG_SWAP=n and CONFIG_THP_SWAP=y, split_swap_cluster() will not be built because it

Re: [PATCH 1/6] swap: Add comments to lock_cluster_or_swap_info()

2018-07-13 Thread Huang, Ying
ster_info, and si->swap_map[], this is described in comments of struct swap_cluster_info. si->lock is used to protect other fields of si. If two locks need to be held, hold si->lock first. This is for non-HDD. For HDD, there are no cluster, so si->lock is used to protect si->swap_map[]. Best Regards, Huang, Ying

Re: [PATCH 3/6] swap: Unify normal/huge code path in swap_page_trans_huge_swapped()

2018-07-14 Thread Huang, Ying
Daniel Jordan writes: > On Fri, Jul 13, 2018 at 07:36:33AM +0800, Huang, Ying wrote: >> diff --git a/mm/swapfile.c b/mm/swapfile.c >> index 75c84aa763a3..160f78072667 100644 >> --- a/mm/swapfile.c >> +++ b/mm/swapfile.c >> @@ -270,7 +270,10 @@ static in

<    1   2   3   4   5   6   7   8   9   10   >