[PATCH -v4 RESEND 6/9] mm, THP, swap: Support to add/delete THP to/from swap cache

2016-10-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> With this patch, a THP (Transparent Huge Page) can be added/deleted to/from the swap cache as a set of (HPAGE_PMD_NR) sub-pages. This will be used for the THP (Transparent Huge Page) swap support. Where one THP may be added/delted to/from the swap

Re: [PATCH -v4 RESEND 6/9] mm, THP, swap: Support to add/delete THP to/from swap cache

2016-10-28 Thread Huang, Ying
Hi, Hillf, Hillf Danton <hillf...@alibaba-inc.com> writes: > On Friday, October 28, 2016 1:56 PM Huang, Ying wrote: >> >> @@ -109,9 +118,16 @@ int __add_to_swap_cache(struct page *page, swp_entry_t >> entry) >> * So add_to_swa

Re: [LKP] [lkp] [f2fs] ec795418c4: fsmark.files_per_sec -36.3% regression

2016-10-30 Thread Huang, Ying
Hi, Kim, Jaegeuk Kim <jaeg...@kernel.org> writes: > On Tue, Sep 27, 2016 at 08:50:02AM +0800, Huang, Ying wrote: >> Jaegeuk Kim <jaeg...@kernel.org> writes: >> >> > On Mon, Sep 26, 2016 at 02:26:06PM +0800, Huang, Ying wrote: >> >> Hi, Jaegeuk,

Re: [LKP] [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression

2016-12-12 Thread Huang, Ying
stics builtin, although it is an OS noise benchmark. ftq.noise.50% is calculated as below: There is a score for every sample of ftq. The lower the score, the higher the noises. ftq.noise.50% is the number (per 100 samples) of samples whose score is less than 50% of the mean score. Best Regards

Re: [LKP] [ext4] e2ae766c1b: BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/rwsem.c

2016-12-12 Thread Huang, Ying
name: Intel Corporation LH >> Pass/S4600LH, BIOS SE5C600.86B.99.02.1047.032320122259 03/23/2012 >> kern :warn : [ 325.608922] c9002c1f7be0 >> kern :warn : [ 325.608923] 81466af9 >> kern :warn : [ 325.608924] 880fea2425c0 > > I think this is actually a bug introduced by Ross' PMD support. Attached > patch should fix it. Ross, can you check it please? Hi, Jan Could you provide a git tree commit for me to test it? If you want it to be tested by 0day. Best Regards, Huang, Ying > Honza

Re: [LKP] [lkp-developer] [mm] 4db18597f8: invoked_oom-killer:gfp_mask=0x

2016-12-12 Thread Huang, Ying
misread some text in the original log. Best Regards, Huang, Ying

Re: [LKP] [ext4] e2ae766c1b: BUG:sleeping_function_called_from_invalid_context_at_kernel/locking/rwsem.c

2016-12-13 Thread Huang, Ying
Jan Kara <j...@suse.cz> writes: > On Tue 13-12-16 09:27:51, Huang, Ying wrote: >> Jan Kara <j...@suse.cz> writes: >> >> > On Mon 12-12-16 18:13:21, kernel test robot wrote: >> >> FYI, we noticed the following commit: >> >> >> &

Re: [PATCH v2] llist: Clarify comments about when locking is needed

2016-12-11 Thread Huang, Ying
Joel Fernandes <joe...@google.com> writes: > llist.h comments are a bit confusing about when locking is needed versus when > it isn't. Clarify these comments a bit more by being a bit more descriptive > about why locking is needed for llist_del_first. > > Cc: Huang Ying

Re: [RFC] llist: Fix code comments about llist_del_first locking

2016-12-08 Thread Huang, Ying
Joel Fernandes <joe...@google.com> writes: > Usage llist_del_first needs lock protection, however the table in the > comments of llist.h show a '-'. Correct this, and also add better > comments on top. > > Cc: Huang Ying <ying.hu...@intel.com> > Cc: Ingo

Re: [RFC] llist: Fix code comments about llist_del_first locking

2016-12-08 Thread Huang, Ying
Joel Fernandes <joe...@google.com> writes: > On Thu, Dec 8, 2016 at 6:12 PM, Huang, Ying <ying.hu...@intel.com> wrote: >> Joel Fernandes <joe...@google.com> writes: >> >>> On Thu, Dec 8, 2016 at 4:42 PM, Joel Fernandes <joe...@google.com> wrote:

Re: [RFC] llist: Fix code comments about llist_del_first locking

2016-12-08 Thread Huang, Ying
Joel Fernandes <joe...@google.com> writes: > On Thu, Dec 8, 2016 at 4:42 PM, Joel Fernandes <joe...@google.com> wrote: >> On Thu, Dec 8, 2016 at 4:35 PM, Huang, Ying <ying.hu...@intel.com> wrote: >>> Joel Fernandes <joe...@google.com> writes: >>&g

Re: [PATCH v5 2/9] mm/swap: Add cluster lock

2017-01-11 Thread Huang, Ying
Andrew Morton <a...@linux-foundation.org> writes: > On Thu, 12 Jan 2017 09:47:51 +0800 "Huang\, Ying" <ying.hu...@intel.com> > wrote: > >> >> > 1MB swap space, so for 1TB swap space, the total size will be 80M >> >> > compared with 8M

[Update][PATCH v5 2/9] mm/swap: Add cluster lock

2017-01-13 Thread Huang, Ying
s, which sequentially allocate and write to the anonymous pages until the RAM and part of the swap device is used. Signed-off-by: "Huang, Ying" <ying.hu...@intel.com> --- include/linux/swap.h | 6 ++ mm/swapfile.c| 211 +-- 2

Re: [PATCH v5 2/9] mm/swap: Add cluster lock

2017-01-11 Thread Huang, Ying
m, have you received it, copied below, From: huang ying <huang.ying.cari...@gmail.com> Subject: Re: [PATCH v2 2/8] mm/swap: Add cluster lock To: "Huang, Ying" <ying.hu...@intel.com> CC: Jonathan Corbet <cor...@lwn.net>, Tim Chen <tim.c.c...@linux.intel.com>,

Re: [PATCH v5 2/9] mm/swap: Add cluster lock

2017-01-11 Thread Huang, Ying
t size measuring result is: If we use bit_spin_lock, the size of cluster_swap_info will, - increased from 4 bytes to 8 bytes on 64 bit platform - keep as 4 bytes on 32 bit platform If we use normal spinlock (queue spinlock), the size of cluster_swap_info will, - increased from 4 bytes to 8 bytes on 64 bit platform - increased from 4 bytes to 8 bytes on 32 bit platform So the difference occurs on 32 bit platform. If the size increment on 32 bit platform is OK, then I think it should be good to use normal spinlock instead of bit_spin_lock. Personally, I am OK for that. But I don't know whether there will be some embedded world people don't like it. Best Regards, Huang, Ying

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2017-01-02 Thread Huang, Ying
to go this way and how much effort will be needed. In contrast, splitting swap cache is quite simple, for implementation and review. And the effect is good. Best Regards, Huang, Ying > Ccing Nick who might have an interest on lockless page cache. > > Thanks.

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2017-01-04 Thread Huang, Ying
owpath.__alloc_pages_nodemask.alloc_pages_vma": 2.56, "perf-profile.calltrace.cycles-pp._raw_spin_lock.get_swap_pages.get_swap_page.add_to_swap.shrink_page_list": 2.47, The swap out throughput is improved about 192% compared with the baseline. There are still some lock contention for swap_info_struct->lock, but the pressure begins to shift to buddy system now. Best Regards, Huang, Ying

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2017-01-04 Thread Huang, Ying
Minchan Kim <minc...@kernel.org> writes: > Hi, > > On Thu, Jan 05, 2017 at 09:33:55AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim <minc...@kernel.org> writes: >> [snip] >> > >> > The patchset has used several techniqu

Re: [LKP] [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression

2017-01-03 Thread Huang, Ying
Vincent Guittot <vincent.guit...@linaro.org> writes: > Hi Dietmar and Ying, > > Le Tuesday 03 Jan 2017 11:38:39 (+0100), Dietmar Eggemann a crit : >> Hi Vincent and Ying, >> >> On 01/02/2017 04:42 PM, Vincent Guittot wrote: >> >Hi Ying, >>

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2016-12-27 Thread Huang, Ying
Minchan Kim <minc...@kernel.org> writes: > On Wed, Dec 28, 2016 at 11:31:06AM +0800, Huang, Ying wrote: > > < snip > > >> >>> > Frankly speaking, although I'm huge user of bit_spin_lock(zram/zsmalloc >> >>> > have used it heavily

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2016-12-27 Thread Huang, Ying
"Huang, Ying" <ying.hu...@intel.com> writes: > Minchan Kim <minc...@kernel.org> writes: > >> Hi Huang, >> >> On Wed, Dec 28, 2016 at 09:54:27AM +0800, Huang, Ying wrote: >> >> < snip > >> >>> > The patchset has used

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2016-12-27 Thread Huang, Ying
Minchan Kim <minc...@kernel.org> writes: > Hi Huang, > > On Wed, Dec 28, 2016 at 09:54:27AM +0800, Huang, Ying wrote: > > < snip > > >> > The patchset has used several techniqueus to reduce lock contention, for >> > example, >> > batchi

Re: [PATCH v2 2/8] mm/swap: Add cluster lock

2016-12-27 Thread huang ying
Hi, Jonathan, On Tue, Oct 25, 2016 at 10:05 AM, Huang, Ying <ying.hu...@intel.com> wrote: > Hi, Jonathan, > > Thanks for review. > > Jonathan Corbet <cor...@lwn.net> writes: > >> On Thu, 20 Oct 2016 16:31:41 -0700 >> Tim Chen <tim.c.c...@linux.intel.co

Re: [LKP] [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression

2016-12-28 Thread Huang, Ying
Vincent Guittot <vincent.guit...@linaro.org> writes: > Le Tuesday 13 Dec 2016 . 09:47:30 (+0800), Huang, Ying a .crit : >> Hi, Vincent, >> >> Vincent Guittot <vincent.guit...@linaro.org> writes: >> >> > Hi Ying, >> > >>

Re: [PATCH v4 0/9] mm/swap: Regular page swap optimizations

2016-12-27 Thread Huang, Ying
kly speaking, although I'm huge user of bit_spin_lock(zram/zsmalloc > have used it heavily), I don't like swap subsystem uses it. > During zram development, it really hurts debugging due to losing lockdep. > The reason zram have used it is by size concern of embedded world but server > would

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-03-23 Thread Huang, Ying
John Hubbard <jhubb...@nvidia.com> writes: > On 03/23/2017 07:41 PM, Huang, Ying wrote: >> David Rientjes <rient...@google.com> writes: >> >>> On Mon, 20 Mar 2017, Huang, Ying wrote: >>> >>>> From: Huang Ying <ying.hu...@intel.com> &

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-03-23 Thread Huang, Ying
David Rientjes <rient...@google.com> writes: > On Mon, 20 Mar 2017, Huang, Ying wrote: > >> From: Huang Ying <ying.hu...@intel.com> >> >> Now vzalloc() is used in swap code to allocate various data >> structures, such as swap cache, swap slots cache,

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-03-24 Thread Huang, Ying
John Hubbard <jhubb...@nvidia.com> writes: > On 03/23/2017 09:52 PM, Huang, Ying wrote: >> John Hubbard <jhubb...@nvidia.com> writes: >> >>> On 03/23/2017 07:41 PM, Huang, Ying wrote: >>>> David Rientjes <rient...@google.com> wri

Re: [PATCH -mm -v7 4/9] mm, THP, swap: Add get_huge_swap_page()

2017-03-29 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Tue, Mar 28, 2017 at 01:32:04PM +0800, Huang, Ying wrote: >> @@ -527,6 +527,23 @@ static inline swp_entry_t get_swap_page(void) >> >> #endif /* CONFIG_SWAP */ >> >> +#ifdef CONFIG_THP_SWA

Re: [PATCH -mm -v7 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-29 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Tue, Mar 28, 2017 at 01:32:01PM +0800, Huang, Ying wrote: >> @@ -499,6 +499,19 @@ config FRONTSWAP >> >>If unsure, say Y to enable frontswap. >> >> +config ARCH_USES_THP_SWAP_CLUSTER >>

Re: [PATCH -mm -v7 9/9] mm, THP, swap: Delay splitting THP during swap out

2017-03-29 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Tue, Mar 28, 2017 at 01:32:09PM +0800, Huang, Ying wrote: >> @@ -183,12 +184,53 @@ void __delete_from_swap_cache(struct page *page) >> ADD_CACHE_INFO(del_total, nr); >> } >> >> +#ifdef CONFIG_THP_SWAP

Re: [PATCH -mm -v7 2/9] mm, memcg: Support to charge/uncharge multiple swap entries

2017-03-29 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Tue, Mar 28, 2017 at 01:32:02PM +0800, Huang, Ying wrote: >> @@ -5908,16 +5907,19 @@ void mem_cgroup_swapout(struct page *page, >> swp_entry_t entry) >> css_put(>css); >> } >> >

Re: [PATCH -mm -v7 0/9] THP swap: Delay splitting THP during swapping out

2017-03-29 Thread Huang, Ying
Hi, Andrew, Andrew Morton <a...@linux-foundation.org> writes: > On Tue, 28 Mar 2017 13:32:00 +0800 "Huang, Ying" <ying.hu...@intel.com> wrote: > >> Hi, Andrew, could you help me to check whether the overall design is >> reasonable? >> >> H

Re: [PATCH -mm -v7 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-28 Thread Huang, Ying
"Kirill A. Shutemov" <kir...@shutemov.name> writes: > On Tue, Mar 28, 2017 at 01:32:01PM +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> In this patch, the size of the swap cluster is changed to that of the >> THP (T

[PATCH -mm -v7 9/9] mm, THP, swap: Delay splitting THP during swap out

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, splitting huge page is delayed from almost the first step of swapping out to after allocating the swap space for the THP (Transparent Huge Page) and adding the THP into the swap cache. This will reduce lock acquiring/releasing for the

[PATCH -mm -v7 3/9] mm, THP, swap: Add swap cluster allocate/free functions

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> The swap cluster allocation/free functions are added based on the existing swap cluster management mechanism for SSD. These functions don't work for the rotating hard disks because the existing swap cluster management mechanism doesn't work fo

[PATCH -mm -v7 5/9] mm, THP, swap: Support to clear SWAP_HAS_CACHE for huge page

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> __swapcache_free() is added to support to clear the SWAP_HAS_CACHE flag for the huge page. This will free the specified swap cluster now. Because now this function will be called only in the error path to free the swap cluster just allocate

[PATCH -mm -v7 7/9] mm, THP: Add can_split_huge_page()

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Separates checking whether we can split the huge page from split_huge_page_to_list() into a function. This will help to check that before splitting the THP (Transparent Huge Page) really. This will be used for delaying splitting THP during swappi

[PATCH -mm -v7 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, the size of the swap cluster is changed to that of the THP (Transparent Huge Page) on x86_64 architecture (512). This is for the THP swap support on x86_64. Where one swap cluster will be used to hold the contents of each THP swapp

[PATCH -mm -v7 0/9] THP swap: Delay splitting THP during swapping out

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [1/9], [3/9], [4/9], [5/9], [6/9], [9/9]. Hi, Andrea could yo

[PATCH -mm -v7 6/9] mm, THP, swap: Support to add/delete THP to/from swap cache

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> With this patch, a THP (Transparent Huge Page) can be added/deleted to/from the swap cache as a set of (HPAGE_PMD_NR) sub-pages. This will be used for the THP (Transparent Huge Page) swap support. Where one THP may be added/delted to/from the swap

[PATCH -mm -v7 2/9] mm, memcg: Support to charge/uncharge multiple swap entries

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patch make it possible to charge or uncharge a set of continuous swap entries in the swap cgroup. The number of swap entries is specified via an added parameter. This will be used for the THP (Transparent Huge Page) swap support. Where a swap c

[PATCH -mm -v7 4/9] mm, THP, swap: Add get_huge_swap_page()

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> A variation of get_swap_page(), get_huge_swap_page(), is added to allocate a swap cluster (HPAGE_PMD_NR swap slots) based on the swap cluster allocation function. A fair simple algorithm is used, that is, only the first swap device in priorit

[PATCH -mm -v7 8/9] mm, THP, swap: Support to split THP in swap cache

2017-03-27 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patch enhanced the split_huge_page_to_list() to work properly for the THP (Transparent Huge Page) in the swap cache during swapping out. This is used for delaying splitting the THP during swapping out. Where for a THP to be swapped o

[PATCH] mm, swap: Remove WARN_ON_ONCE() in free_swap_slot()

2017-03-20 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Before commit 452b94b8c8c7 ("mm/swap: don't BUG_ON() due to uninitialized swap slot cache"), the following bug is reported, [ cut here ] kernel BUG at mm/swap_slots.c:270! invalid opcode: [#1] SMP CPU:

Re: kernel BUG at mm/swap_slots.c:270

2017-03-19 Thread Huang, Ying
and bail > out when the slot cache hasn't been initialized yet. Yes. The BUG_ON() is problematic. The initialization of swap slot cache may fail too, if so, we should still allow using swap without slot cache. Will send out a fixing patch ASAP. Best Regards, Huang, Ying

Re: [PATCH 1/5] mm, swap: Fix comment in __read_swap_cache_async

2017-03-19 Thread Huang, Ying
Hi, Rafeal, Rafael Aquini <aqu...@redhat.com> writes: > On Fri, Mar 17, 2017 at 02:46:19PM +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> The commit cbab0e4eec29 ("swap: avoid read_swap_cache_async() race to >> deadlock whi

Re: [PATCH -mm -v6 3/9] mm, THP, swap: Add swap cluster allocate/free functions

2017-03-16 Thread Huang, Ying
Tim Chen <tim.c.c...@linux.intel.com> writes: > On Wed, 2017-03-15 at 09:19 +0800, Huang, Ying wrote: >> Tim Chen <tim.c.c...@linux.intel.com> writes: >> >> > >> > On Wed, 2017-03-08 at 15:26 +0800, Huang, Ying wrote: >>

[PATCH -v2 2/2] mm, swap: Sort swap entries before free

2017-03-20 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To reduce the lock contention of swap_info_struct->lock when freeing swap entry. The freed swap entries will be collected in a per-CPU buffer firstly, and be really freed later in batch. During the batch freeing, if the consecutive swap entries i

[PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-03-20 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Now vzalloc() is used in swap code to allocate various data structures, such as swap cache, swap slots cache, cluster info, etc. Because the size may be too large on some system, so that normal kzalloc() may fail. But using kzalloc() has some adva

[PATCH 2/5] mm, swap: Improve readability via make spin_lock/unlock balanced

2017-03-17 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This is just a cleanup patch, no functionality change. In cluster_list_add_tail(), spin_lock_nested() is used to lock the cluster, while unlock_cluster() is used to unlock the cluster. To improve the code readability. Use spin_unlock() directly to

[PATCH 3/5] mm, swap: Avoid lock swap_avail_lock when held cluster lock

2017-03-17 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Cluster lock is used to protect the swap_cluster_info and corresponding elements in swap_info_struct->swap_map[]. But it is found that now in scan_swap_map_slots(), swap_avail_lock may be acquired when cluster lock is held. This does no good exce

[PATCH 5/5] mm, swap: Sort swap entries before free

2017-03-17 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To reduce the lock contention of swap_info_struct->lock when freeing swap entry. The freed swap entries will be collected in a per-CPU buffer firstly, and be really freed later in batch. During the batch freeing, if the consecutive swap entries i

[PATCH 1/5] mm, swap: Fix comment in __read_swap_cache_async

2017-03-17 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> The commit cbab0e4eec29 ("swap: avoid read_swap_cache_async() race to deadlock while waiting on discard I/O completion") fixed a deadlock in read_swap_cache_async(). Because at that time, in swap allocation path, a swap entry may be set a

[PATCH 4/5] mm, swap: Try kzalloc before vzalloc

2017-03-17 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Now vzalloc() is used in swap code to allocate various data structures, such as swap cache, swap slots cache, cluster info, etc. Because the size may be too large on some system, so that normal kzalloc() may fail. But using kzalloc() has some adva

Re: [PATCH -mm -v6 4/9] mm, THP, swap: Add get_huge_swap_page()

2017-03-14 Thread Huang, Ying
Tim Chen <tim.c.c...@linux.intel.com> writes: > On Wed, 2017-03-08 at 15:26 +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> A variation of get_swap_page(), get_huge_swap_page(), is added to >> allocate a swap cluster (HPA

Re: [PATCH -mm -v6 5/9] mm, THP, swap: Support to clear SWAP_HAS_CACHE for huge page

2017-03-14 Thread Huang, Ying
Tim Chen <tim.c.c...@linux.intel.com> writes: > On Wed, 2017-03-08 at 15:26 +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> __swapcache_free() is added to support to clear the SWAP_HAS_CACHE flag >> for the huge page.  This w

Re: [PATCH -mm -v6 3/9] mm, THP, swap: Add swap cluster allocate/free functions

2017-03-14 Thread Huang, Ying
Tim Chen <tim.c.c...@linux.intel.com> writes: > On Wed, 2017-03-08 at 15:26 +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> The swap cluster allocation/free functions are added based on the >> existing swap cluster managem

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-04-04 Thread Huang, Ying
Michal Hocko <mho...@kernel.org> writes: > On Sat 01-04-17 12:47:56, Huang, Ying wrote: >> Hi, Michal, >> >> Michal Hocko <mho...@kernel.org> writes: >> >> > On Fri 24-03-17 06:56:10, Dave Hansen wrote: >> >> On 03/24/2017 12:33 AM

[PATCH -mm -v2] mm, swap: Sort swap entries before free

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To reduce the lock contention of swap_info_struct->lock when freeing swap entry. The freed swap entries will be collected in a per-CPU buffer firstly, and be really freed later in batch. During the batch freeing, if the consecutive swap entries i

[PATCH -mm] mm, swap: Remove unused function prototype

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This is a code cleanup patch, no functionality changes. There are 2 unused function prototype in swap.h, they are removed. Signed-off-by: "Huang, Ying" <ying.hu...@intel.com> Cc: Tim Chen <tim.c.c...@linux.intel.com> --- inc

[PATCH -mm -v2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Now vzalloc() is used in swap code to allocate various data structures, such as swap cache, swap slots cache, cluster info, etc. Because the size may be too large on some system, so that normal kzalloc() may fail. But using kzalloc() has some adva

Re: [PATCH -mm -v7 9/9] mm, THP, swap: Delay splitting THP during swap out

2017-03-31 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Thu, Mar 30, 2017 at 12:15:13PM +0800, Huang, Ying wrote: >> Johannes Weiner <han...@cmpxchg.org> writes: >> > On Tue, Mar 28, 2017 at 01:32:09PM +0800, Huang, Ying wrote: >> >> @@ -198,6 +240,18 @@

Re: [PATCH -mm -v7 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-31 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Thu, Mar 30, 2017 at 08:45:56AM +0800, Huang, Ying wrote: >> Johannes Weiner <han...@cmpxchg.org> writes: >> >> > On Tue, Mar 28, 2017 at 01:32:01PM +0800, Huang, Ying wrote: >> >> @@ -499,6 +499,19

Re: [PATCH -mm -v7 4/9] mm, THP, swap: Add get_huge_swap_page()

2017-03-31 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Thu, Mar 30, 2017 at 12:28:17PM +0800, Huang, Ying wrote: >> Johannes Weiner <han...@cmpxchg.org> writes: >> > On Tue, Mar 28, 2017 at 01:32:04PM +0800, Huang, Ying wrote: >> >> @@ -527,6 +527,23 @@ st

Re: [PATCH -v2 1/2] mm, swap: Use kvzalloc to allocate some swap data structure

2017-03-31 Thread Huang, Ying
ssion. Besides that the swapon path usually happens early > during the boot where we should have those larger blocks available. Could I add your Acked-by for this patch? Best Regards, Huang, Ying

Re: [PATCH -mm -v2] mm, swap: Sort swap entries before free

2017-04-05 Thread Huang, Ying
Rik van Riel <r...@redhat.com> writes: > On Wed, 2017-04-05 at 15:10 +0800, Huang, Ying wrote: >> To solve the issue, the per-CPU buffer is sorted according to the >> swap >> device before freeing the swap entries.  Test shows that the time >> spent by swapcach

[PATCH -mm -v8 2/3] mm, THP, swap: Check whether THP can be split firstly

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In the original THP swapping out implementation, before splitting the THP (Transparent Huage Page), the swap cluster will be allocated and the THP will be added into the swap cache. But it is possible that the THP cannot be split, and we must

[PATCH -mm -v8 1/3] mm, THP, swap: Delay splitting THP during swap out

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, splitting huge page is delayed from almost the first step of swapping out to after allocating the swap space for the THP (Transparent Huge Page) and adding the THP into the swap cache. This will batch the corresponding operation, thus i

[PATCH -mm -v8 0/3] THP swap: Delay splitting THP during swapping out

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swa

[PATCH -mm -v8 3/3] mm, THP, swap: Enable THP swap optimization only if has compound map

2017-04-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> If there is no compound map for a THP (Transparent Huge Page), it is possible that the map count of some sub-pages of the THP is 0. So it is better to split the THP before swapping out. In this way, the sub-pages not mapped will be freed, and we can

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-11 Thread Huang, Ying
Andrew Morton <a...@linux-foundation.org> writes: > On Fri, 7 Apr 2017 14:49:01 +0800 "Huang, Ying" <ying.hu...@intel.com> wrote: > >> To reduce the lock contention of swap_info_struct->lock when freeing >> swap entry. The freed swap entries will

Re: [PATCH -mm -v8 1/3] mm, THP, swap: Delay splitting THP during swap out

2017-04-14 Thread Huang, Ying
cation. Is 'XXX' here intended. > + > + For selection by architectures with reasonable THP sizes. > + > config TRANSPARENT_HUGE_PAGECACHE > def_bool y > depends on TRANSPARENT_HUGEPAGE > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index d14dd961f626..4a5c1ca21894 100644 [...] Best Regards, Huang, Ying

Re: [PATCH -mm -v9 2/3] mm, THP, swap: Check whether THP can be split firstly

2017-04-20 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Thu, Apr 20, 2017 at 08:50:43AM +0800, Huang, Ying wrote: >> Johannes Weiner <han...@cmpxchg.org> writes: >> > On Wed, Apr 19, 2017 at 03:06:24PM +0800, Huang, Ying wrote: >> >> With the patchset, the swa

Re: [PATCH -mm -v9 1/3] mm, THP, swap: Delay splitting THP during swap out

2017-04-21 Thread Huang, Ying
Balbir Singh <bsinghar...@gmail.com> writes: > On Wed, 2017-04-19 at 15:06 +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> In this patch, splitting huge page is delayed from almost the first >> step of swapping out to after a

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-21 Thread Huang, Ying
"Huang, Ying" <ying.hu...@intel.com> writes: > Minchan Kim <minc...@kernel.org> writes: > >> On Wed, Apr 19, 2017 at 04:14:43PM +0800, Huang, Ying wrote: >>> Minchan Kim <minc...@kernel.org> writes: >>> >>> > Hi Huang, &

[PATCH -mm] mm, swap: Fix swap space leak in error path of swap_free_entries()

2017-04-21 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In swapcache_free_entries(), if swap_info_get_cont() return NULL, something wrong occurs for the swap entry. But we should still continue to free the following swap entries in the array instead of skip them to avoid swap space leak. This is just p

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-13 Thread Huang, Ying
Andrew Morton <a...@linux-foundation.org> writes: > On Fri, 7 Apr 2017 14:49:01 +0800 "Huang, Ying" <ying.hu...@intel.com> wrote: > >> To reduce the lock contention of swap_info_struct->lock when freeing >> swap entry. The freed swap entries will

Re: [PATCH -mm -v9 1/3] mm, THP, swap: Delay splitting THP during swap out

2017-04-19 Thread Huang, Ying
Hi, Johannes, Johannes Weiner <han...@cmpxchg.org> writes: > On Wed, Apr 19, 2017 at 03:06:23PM +0800, Huang, Ying wrote: >> @@ -206,17 +212,34 @@ int add_to_swap(struct page *page, struct list_head >> *list) >> */ >> er

Re: [PATCH -mm -v9 3/3] mm, THP, swap: Enable THP swap optimization only if has compound map

2017-04-19 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Wed, Apr 19, 2017 at 03:06:25PM +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> If there is no compound map for a THP (Transparent Huge Page), it is >> possible that the map count

Re: [PATCH -mm -v9 2/3] mm, THP, swap: Check whether THP can be split firstly

2017-04-19 Thread Huang, Ying
Johannes Weiner <han...@cmpxchg.org> writes: > On Wed, Apr 19, 2017 at 03:06:24PM +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >> >> To swap out THP (Transparent Huage Page), before splitting the THP, >> the swap cluster will

[PATCH -mm -v9 3/3] mm, THP, swap: Enable THP swap optimization only if has compound map

2017-04-19 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> If there is no compound map for a THP (Transparent Huge Page), it is possible that the map count of some sub-pages of the THP is 0. So it is better to split the THP before swapping out. In this way, the sub-pages not mapped will be freed, and we can

[PATCH -mm -v9 1/3] mm, THP, swap: Delay splitting THP during swap out

2017-04-19 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, splitting huge page is delayed from almost the first step of swapping out to after allocating the swap space for the THP (Transparent Huge Page) and adding the THP into the swap cache. This will batch the corresponding operation, thus i

[PATCH -mm -v9 2/3] mm, THP, swap: Check whether THP can be split firstly

2017-04-19 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To swap out THP (Transparent Huage Page), before splitting the THP, the swap cluster will be allocated and the THP will be added into the swap cache. But it is possible that the THP cannot be split, so that we must delete the THP from the swap

[PATCH -mm -v9 0/3] THP swap: Delay splitting THP during swapping out

2017-04-19 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Recently, the performance of the storage devices improved so fast that we cannot saturate the disk bandwidth with single logical CPU when do page swap out even on a hi

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-20 Thread Huang, Ying
Minchan Kim <minc...@kernel.org> writes: > On Wed, Apr 19, 2017 at 04:14:43PM +0800, Huang, Ying wrote: >> Minchan Kim <minc...@kernel.org> writes: >> >> > Hi Huang, >> > >> > On Fri, Apr 07, 2017 at 02:49:01PM +0800, Huang, Yi

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-23 Thread Huang, Ying
Tim Chen <tim.c.c...@linux.intel.com> writes: > On Fri, 2017-04-21 at 20:29 +0800, Huang, Ying wrote: >> "Huang, Ying" <ying.hu...@intel.com> writes: >> >> > >> > Minchan Kim <minc...@kernel.org> writes: >> > >

Re: [PATCH -mm -v3] mm, swap: Sort swap entries before free

2017-04-24 Thread Huang, Ying
Minchan Kim <minc...@kernel.org> writes: > On Fri, Apr 21, 2017 at 08:29:30PM +0800, Huang, Ying wrote: >> "Huang, Ying" <ying.hu...@intel.com> writes: >> >> > Minchan Kim <minc...@kernel.org> writes: >> > >> >> On Wed

Re: [PATCH] mm, swap: Fix a race in free_swap_and_cache()

2017-03-04 Thread huang ying
Hi, Andrew, Sorry, I clicked the wrong button in my mail client, so forgot Ccing mailing list. Sorry for duplicated mail. On Sat, Mar 4, 2017 at 6:43 AM, Andrew Morton <a...@linux-foundation.org> wrote: > On Wed, 1 Mar 2017 22:38:09 +0800 "Huang, Ying" <ying.hu...@intel.c

Re: [PATCH -mm -v6 2/9] mm, memcg: Support to charge/uncharge multiple swap entries

2017-03-08 Thread Huang, Ying
Balbir Singh <bsinghar...@gmail.com> writes: > On Wed, 2017-03-08 at 15:26 +0800, Huang, Ying wrote: >> From: Huang Ying <ying.hu...@intel.com> >>  >> This patch make it possible to charge or uncharge a set of continuous >> swap entries in the s

Re: [PATCH -mm -v6 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-08 Thread Huang, Ying
Hi, Matthew, Matthew Wilcox <wi...@infradead.org> writes: > On Wed, Mar 08, 2017 at 03:26:05PM +0800, Huang, Ying wrote: >> In this patch, the size of the swap cluster is changed to that of the >> THP (Transparent Huge Page) on x86_64 architecture (512). This is for &g

[PATCH] mm, swap: VMA based swap readahead

2017-03-14 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> The swap readahead is a important mechanism to reduce the swap in latency. But the original swap readahead algorithm has some issues. a) The original swap readahead algorithm does readahead based on the consecutive blocks in swap

[PATCH -mm -v6 1/9] mm, swap: Make swap cluster size same of THP size on x86_64

2017-03-08 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, the size of the swap cluster is changed to that of the THP (Transparent Huge Page) on x86_64 architecture (512). This is for the THP swap support on x86_64. Where one swap cluster will be used to hold the contents of each THP swapp

[RFC] mm, swap: VMA based swap readahead

2017-03-05 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> The swap readahead is a important mechanism to reduce the swap in latency. But the original swap readahead algorithm has some issues. a) The original swap readahead algorithm does readahead based on the consecutive blocks in swap

Re: [LKP] [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression

2017-02-27 Thread Huang, Ying
Vincent Guittot <vincent.guit...@linaro.org> writes: > Hi Ying, > > On 21 February 2017 at 03:40, Huang, Ying <ying.hu...@intel.com> wrote: >> Hi, Vincent, >> >> Vincent Guittot <vincent.guit...@linaro.org> writes: >

[PATCH -mm -v6 8/9] mm, THP, swap: Support to split THP in swap cache

2017-03-07 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patch enhanced the split_huge_page_to_list() to work properly for the THP (Transparent Huge Page) in the swap cache during swapping out. This is used for delaying splitting the THP during swapping out. Where for a THP to be swapped o

[PATCH -mm -v6 9/9] mm, THP, swap: Delay splitting THP during swap out

2017-03-07 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In this patch, splitting huge page is delayed from almost the first step of swapping out to after allocating the swap space for the THP (Transparent Huge Page) and adding the THP into the swap cache. This will reduce lock acquiring/releasing for the

[PATCH -mm -v6 2/9] mm, memcg: Support to charge/uncharge multiple swap entries

2017-03-07 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This patch make it possible to charge or uncharge a set of continuous swap entries in the swap cgroup. The number of swap entries is specified via an added parameter. This will be used for the THP (Transparent Huge Page) swap support. Where a swap c

[PATCH -mm -v6 7/9] mm, THP: Add can_split_huge_page()

2017-03-07 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Separates checking whether we can split the huge page from split_huge_page_to_list() into a function. This will help to check that before splitting the THP (Transparent Huge Page) really. This will be used for delaying splitting THP during swappi

[PATCH -v6 0/9] THP swap: Delay splitting THP during swapping out

2017-03-07 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [1/9], [3/9], [4/9], [5/9], [6/9], [9/9]. Hi, Andrea could yo

<    4   5   6   7   8   9   10   11   12   13   >