Re: [PATCH] mm: Fix a race during split THP

2020-10-09 Thread Huang, Ying
"Huang, Ying" writes: > From: Huang Ying > > It is reported that the following bug is triggered if the HDD is used as swap > device, > > [ 5758.157556] BUG: kernel NULL pointer dereference, address: 0007 > [ 5758.165331] #PF: supervisor write access

[PATCH] mm: Fix a race during split THP

2020-10-09 Thread Huang, Ying
From: Huang Ying It is reported that the following bug is triggered if the HDD is used as swap device, [ 5758.157556] BUG: kernel NULL pointer dereference, address: 0007 [ 5758.165331] #PF: supervisor write access in kernel mode [ 5758.171161] #PF: error_code(0x0002) - not-present

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-10-08 Thread Huang, Ying
Rafael Aquini writes: > On Thu, Oct 01, 2020 at 10:31:57AM -0400, Rafael Aquini wrote: >> On Fri, Sep 25, 2020 at 11:21:58AM +0800, Huang, Ying wrote: >> > Rafael Aquini writes: >> > >> Or, can you help to run the test with a debug kernel based on upstream &g

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-26 Thread Huang, Ying
Rafael Aquini writes: > On Fri, Sep 25, 2020 at 11:21:58AM +0800, Huang, Ying wrote: >> Rafael Aquini writes: >> >> Or, can you help to run the test with a debug kernel based on upstream >> >> kernel. I can provide some debug patch. >> >>

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-24 Thread Huang, Ying
success. Thanks! I found a race condition. During THP splitting, "head" may be unlocked before calling split_swap_cluster(), because head != page during deferred splitting. So we should call split_swap_cluster() before unlocking. The debug patch to do that is as below. Can you help t

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-24 Thread Huang, Ying
out") >> > > Signed-off-by: Rafael Aquini >> > >> > Did you consider cc:stable? >> > >> >> UGH! I missed adding it to my cc list. Shall I just forward it, now, or >> do you prefer a fresh repost? > > I added the cc:stable to my copy. Please don't merge this patch. This patch doesn't fix the bug, but hide the real bug. I will work with Rafael on root causing and fixing. Best Regards, Huang, Ying

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-24 Thread Huang, Ying
int count, mapcount, extra_pins, ret; unsigned long flags; pgoff_t end; VM_BUG_ON_PAGE(is_huge_zero_page(head), head); VM_BUG_ON_PAGE(!PageLocked(head), head); I found there's page lock checking in split_huge_page_to_list(). Best Regards, Huang, Ying

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-23 Thread Huang, Ying
ght, but you will get the > NULL pointer dereference. Do you have a script to reproduce the bug? > Regardless you find furhter bugs, or not, this patch is needed to correct a > blunt coding mistake. As above. I don't agree with that. Best Regards, Huang, Ying

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-23 Thread Huang, Ying
Rafael Aquini writes: > On Wed, Sep 23, 2020 at 01:13:49PM +0800, Huang, Ying wrote: >> Rafael Aquini writes: >> >> > On Wed, Sep 23, 2020 at 10:21:36AM +0800, Huang, Ying wrote: >> >> Hi, Rafael, >> >> >> >> Rafael Aquini writ

Re: [RFC -V2] autonuma: Migrate on fault among multiple bound nodes

2020-09-22 Thread Huang, Ying
Phil Auld writes: > Hi, > > On Tue, Sep 22, 2020 at 02:54:01PM +0800 Huang Ying wrote: >> Now, AutoNUMA can only optimize the page placement among the NUMA nodes if >> the >> default memory policy is used. Because the memory policy specified >> exp

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-22 Thread Huang, Ying
Rafael Aquini writes: > On Wed, Sep 23, 2020 at 10:21:36AM +0800, Huang, Ying wrote: >> Hi, Rafael, >> >> Rafael Aquini writes: >> >> > The swap area descriptor only gets struct swap_cluster_info *cluster_info >> > allocated if the swapfile i

Re: [PATCH] mm: swapfile: avoid split_swap_cluster() NULL pointer dereference

2020-09-22 Thread Huang, Ying
f the bug. Did you test with the latest upstream kernel? Can you help trace the return value of swap_alloc_cluster()? Can you share the swap device information? Best Regards, Huang, Ying > CONFIG_THP_SWAP exposes cluster_info infrastructure to a broader number of > use cases, and split_

[RFC -V2] autonuma: Migrate on fault among multiple bound nodes

2020-09-22 Thread Huang Ying
optimize the page placement if possible as long as AutoNUMA is enabled globally? The interface would look simpler with that. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Ingo Molnar Cc: Mel Gorman Cc: Rik van Riel Cc: Johannes Weiner Cc: "Matthew Wilcox (Oracle)" Cc: D

Re: [PATCH 2/2] mm,swap: skip swap readahead if page was obtained instantaneously

2020-09-21 Thread huang ying
set, gfp_mask, vma, addr, > true); > } > +skip_unplug: > blk_finish_plug(); > > lru_add_drain();/* Push any new pages onto the LRU now */ Best Regards, Huang, Ying

Re: [RFC] autonuma: Migrate on fault among multiple bound nodes

2020-09-16 Thread Huang, Ying
pet...@infradead.org writes: > On Wed, Sep 16, 2020 at 08:59:36AM +0800, Huang Ying wrote: >> +static bool mpol_may_mof(struct mempolicy *pol) >> +{ >> +/* May migrate among bound nodes for MPOL_BIND */ >> +return pol->flags & MPOL_F_MOF || >&

Re: [RFC][PATCH 4/9] mm/migrate: make migrate_pages() return nr_succeeded

2020-09-16 Thread Huang, Ying
ucceeded)++; I think now we should consider THP in counting now. Because later nr_succeeded will be used to counting the number of reclaimed pages, and THP is respected for that. Best Regards, Huang, Ying > break; > default: >

Re: [RFC] autonuma: Migrate on fault among multiple bound nodes

2020-09-16 Thread Huang, Ying
Hi, Peter, Thanks for comments! pet...@infradead.org writes: > On Wed, Sep 16, 2020 at 08:59:36AM +0800, Huang Ying wrote: > >> So in this patch, if MPOL_BIND is used to bind the memory of the >> application to multiple nodes, and in the hint page fault handler both >>

[PATCH] x86, fakenuma: Avoid too large emulated node

2020-09-08 Thread Huang Ying
e specified node size (960MB vs. 1024MB). But the overall results look more reasonable. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc:

[tip: x86/urgent] x86, fakenuma: Fix invalid starting node ID

2020-09-04 Thread tip-bot2 for Huang Ying
The following commit has been merged into the x86/urgent branch of tip: Commit-ID: ccae0f36d500aef727f98acd8d0601e6b262a513 Gitweb: https://git.kernel.org/tip/ccae0f36d500aef727f98acd8d0601e6b262a513 Author:Huang Ying AuthorDate:Fri, 04 Sep 2020 14:10:47 +08:00 Committer

[PATCH RESEND] x86, fakenuma: Fix invalid starting node ID

2020-09-04 Thread Huang Ying
c9aec03e58f ("x86/numa_emulation: Introduce uniform split capability") Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Dan Williams

[PATCH] x86, fakenuma: Fix invalid starting node ID

2020-08-28 Thread Huang Ying
c9aec03e58f ("x86/numa_emulation: Introduce uniform split capability") Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: x...@kernel.org Cc: "H. Peter Anvin"

[RFC -V3 0/5] autonuma: Optimize memory placement for memory tiering system

2020-08-24 Thread Huang Ying
cleanup. - Rebased on the latest page demotion patchset. v2: - Addressed comments for V1. - Rebased on v5.5. Best Regards, Huang, Ying

[RFC -V3 2/5] autonuma, memory tiering: Skip to scan fast memory

2020-08-24 Thread Huang Ying
. So that the page faults could be avoided too. In the test, if only the memory tiering AutoNUMA mode is enabled, the number of the AutoNUMA hint faults for the DRAM node is reduced to almost 0 with the patch. While the benchmark score doesn't change visibly. Signed-off-by: "Huang,

[RFC -V3 4/5] autonuma, memory tiering: Rate limit NUMA migration throughput

2020-08-24 Thread Huang Ying
MB/s to 105.3 MB/s) with the patch, while the benchmark score decreases only 1.1%. A new sysctl knob kernel.numa_balancing_rate_limit_mbps is added for the users to specify the limit. TODO: Add ABI document for new sysctl knob. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Michal

[RFC -V3 5/5] autonuma, memory tiering: Adjust hot threshold automatically

2020-08-24 Thread Huang Ying
page migrations on a 2 socket Intel server with Optance DC Persistent Memory. Because it improves the accuracy of the hot page selection. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Michal Hocko Cc: Rik van Riel Cc: Mel Gorman Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dave

[RFC -V3 3/5] autonuma, memory tiering: Hot page selection with hint page fault latency

2020-08-24 Thread Huang Ying
ant for system performance, the administrator can set a higher hot threshold. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Michal Hocko Cc: Rik van Riel Cc: Mel Gorman Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dave Hansen Cc: Dan Williams Cc: linux-kernel@vger.kernel.

[RFC -V3 1/5] autonuma: Optimize page placement for memory tiering system

2020-08-24 Thread Huang Ying
ocumentation/sysctl/kernel.txt Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Michal Hocko Cc: Rik van Riel Cc: Mel Gorman Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dave Hansen Cc: Dan Williams Cc: linux-kernel@vger.kernel.org Cc: linux...@kvack.org --- include/linux/sched/

Re: [PATCH v2] mm, THP, swap: fix allocating cluster for swapfile by mistake

2020-08-20 Thread Huang, Ying
: Don't allocate huge cluster for file backed >> swap device") >> >>So such thing would be considered as some new feature and need >>more testing at least. But for now I think we just need a quick >>fix to fix the commit f0eea189e8e9 to avoid regression and

Re: [RFC][PATCH 5/9] mm/migrate: demote pages during reclaim

2020-08-20 Thread Huang, Ying
Yang Shi writes: > On Thu, Aug 20, 2020 at 8:22 AM Dave Hansen wrote: >> >> On 8/20/20 1:06 AM, Huang, Ying wrote: >> >> +/* Migrate pages selected for demotion */ >> >> +nr_reclaimed += demote_page_list(_pages, _pages, pgdat, >>

Re: [RFC][PATCH 5/9] mm/migrate: demote pages during reclaim

2020-08-20 Thread Huang, Ying
es, pgdat, sc); > + > pgactivate = stat->nr_activate[0] + stat->nr_activate[1]; > > mem_cgroup_uncharge_list(_pages); > _ Generally, it's good to batch the page migration. But one side effect is that, if the pages are failed to be migrated, they will be placed back to the LRU list instead of falling back to be reclaimed really. This may cause some issue in some situation. For example, if there's no enough space in the PMEM (slow) node, so the page migration fails, OOM may be triggered, because the direct reclaiming on the DRAM (fast) node may make no progress, while it can reclaim some pages really before. Best Regards, Huang, Ying

Re: [PATCH v2] mm, THP, swap: fix allocating cluster for swapfile by mistake

2020-08-19 Thread Huang, Ying
atter too much as well > > Symptoms: > - FS corruption (e.g. checksum failure) > - memory corruption at: 0xd2808010 > - segfault > > Fixes: f0eea189e8e9 ("mm, THP, swap: Don't allocate huge cluster for file > backed swap device") > Fixes: 38d8b4e6bdc8 ("mm, THP, swap: delay splitting THP during swap out") > Cc: "Huang, Ying" > Cc: Yang Shi > Cc: Rafael Aquini > Cc: Dave Chinner > Cc: stable > Signed-off-by: Gao Xiang Thanks! Reviewed-by: "Huang, Ying" Best Regards, Huang, Ying

Re: [PATCH] mm, THP, swap: fix allocating cluster for swapfile by mistake

2020-08-19 Thread Huang, Ying
vice") > Fixes: 38d8b4e6bdc8 ("mm, THP, swap: delay splitting THP during swap out") > Cc: "Huang, Ying" > Cc: stable > Signed-off-by: Gao Xiang Good catch! The fix itself looks good me! Although the description is a little confusing. After some digging, it

Re: linux-next: not-present page at swap_vma_readahead()

2020-07-19 Thread Huang, Ying
Qian Cai writes: > On Mon, Jul 20, 2020 at 03:32:59AM +0000, Huang, Ying wrote: >> Thanks! Can you try the dbg patch attached? That will print more debugging >> information when abnormal PTE pointer is detected. > > Here with both of your patches applied, > > [ 183

RE: linux-next: not-present page at swap_vma_readahead()

2020-07-19 Thread Huang, Ying
Thanks! Can you try the dbg patch attached? That will print more debugging information when abnormal PTE pointer is detected. Best Regards, Huang, Ying From: Qian Cai [c...@lca.pw] Sent: Monday, July 20, 2020 10:12 AM To: Huang, Ying Cc: Linux-MM; LKML

RE: linux-next: not-present page at swap_vma_readahead()

2020-07-19 Thread Huang, Ying
Hi, Sorry for late reply. I found a problem in the swap readahead code. Can you help to check whether it can fix this? Best Regards, Huang, Ying From: Qian Cai [c...@lca.pw] Sent: Tuesday, June 16, 2020 9:13 AM To: Huang, Ying Cc: Linux-MM; LKML

Re: [mm] 4e2c82a409: ltp.overcommit_memory01.fail

2020-07-06 Thread Huang, Ying
eturned. But that's more complexity. > > One thought I had is to put this schedule_on_each_cpu() before > the proc_dointvec_minmax() to do the sync before sysctl_overcommit_memory > is really changed. But the window still exists, as the batch is > still the larger one. Can we change the batch firstly, then sync the global counter, finally change the overcommit policy? Best Regards, Huang, Ying

Re: [RFC][PATCH 8/8] mm/numa: new reclaim mode to enable reclaim-based migration

2020-07-03 Thread Huang, Ying
k; I think we need to call disable_all_migrate_targets()/set_migration_target_nodes() for CPU online/offline event too. Because that will influence node_state(nid, N_CPU). Which will influence node demotion relationship. > + } > + > + return notifier_from_errno(0); > } > + Best Regards, Huang, Ying

Re: [PATCH 1/3] mm/vmscan: restore zone_reclaim_mode ABI

2020-07-02 Thread Huang, Ying
tion here? > to writing out pages during > node reclaim after the commit in question. That's not great. > > Put the bits back the way they were and add a comment so something > like this is a bit harder to do again. Update the documentation to > make it clear that the first bit is ignored. > Best Regards, Huang, Ying

Re: [RFC][PATCH 3/8] mm/vmscan: Attempt to migrate page in lieu of discard

2020-07-01 Thread Huang, Ying
d any workaround for that requires some > hacks, such as mapping the page to cpuset (what is the right solution for > shared pages?) or adding NUMA locality handling to memcg. It sounds natural to me to add NUMA nodes restriction to memcg. Best Regards, Huang, Ying

Re: [RFC][PATCH 3/8] mm/vmscan: Attempt to migrate page in lieu of discard

2020-07-01 Thread Huang, Ying
y such app today should just be mlocking its cold data if it can't > handle overhead from reclaim? Yes. That's a way to solve the problem. A cpuset-level knob may be more flexible, because you don't need to change the application source code. Best Regards, Huang, Ying

Re: [RFC][PATCH 5/8] mm/numa: automatically generate node migration order

2020-07-01 Thread Huang, Ying
Dave Hansen writes: > On 6/30/20 1:22 AM, Huang, Ying wrote: >>> + /* >>> +* To avoid cycles in the migration "graph", ensure >>> +* that migration sources are not future targets by >>> +* setting them in 'used_targets'. >>

Re: [RFC][PATCH 3/8] mm/vmscan: Attempt to migrate page in lieu of discard

2020-07-01 Thread Huang, Ying
acted as a NUMA node > instead, interleave over nodes {0,2,3} or a cpuset.mems of {0,2,3} doesn't > make much sense. Why can not we just bind the memory of the application to node 0, 2, 3 via mbind() or cpuset.mems? Then the application can allocate memory directly from PMEM. And if we bind the memory of the application via mbind() to node 0, we can only allocate memory directly from DRAM. Best Regards, Huang, Ying

Re: [RFC][PATCH 3/8] mm/vmscan: Attempt to migrate page in lieu of discard

2020-06-30 Thread Huang, Ying
s a batch with migrate_pages(). This results in simpler code and >> has all the performance advantages of batching, but has the >> disadvantage that pages which fail to migrate never get swapped. >> >> #Signed-off-by: Keith Busch >> Signed-off

Re: [RFC][PATCH 8/8] mm/numa: new reclaim mode to enable reclaim-based migration

2020-06-30 Thread Huang, Ying
Yang Shi writes: > On 6/30/20 5:48 PM, Huang, Ying wrote: >> Hi, Yang, >> >> Yang Shi writes: >> >>>>> diff -puN mm/vmscan.c~enable-numa-demotion mm/vmscan.c >>>>> --- a/mm/vmscan.c~enable-numa-demotion2020-06-29 16:35:01.017312

Re: [RFC][PATCH 8/8] mm/numa: new reclaim mode to enable reclaim-based migration

2020-06-30 Thread Huang, Ying
de reclaim > to adopt the new mode. But why shouldn't we migrate in kswapd and direct reclaim? I think that we may need a way to control it, but shouldn't disable it unconditionally. > Please refer to > https://lore.kernel.org/linux-mm/1560468577-101178-6-git-send-email-yang@linux.alibaba.com/ > Best Regards, Huang, Ying

Re: [RFC][PATCH 5/8] mm/numa: automatically generate node migration order

2020-06-30 Thread Huang, Ying
* that migration sources are not future targets by > + * setting them in 'used_targets'. > + * > + * But, do this only once per pass so that multiple > + * source nodes can share a target node. establish_migrate_target() calls find_next_best_node(), which will set target_node

Re: [RFC][PATCH 8/8] mm/numa: new reclaim mode to enable reclaim-based migration

2020-06-30 Thread Huang, Ying
t; optimal, most glaringly that it does not even attempt to figure > out if nodes are actually coming or going. > > Signed-off-by: Dave Hansen > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > --- > > b/Documentation/

Re: [PATCH -V2] swap: Reduce lock contention on swap cache from swap slots allocation

2020-06-23 Thread Huang, Ying
"Huang, Ying" writes: > Andrew Morton writes: > >> On Wed, 20 May 2020 11:15:02 +0800 Huang Ying wrote: >> >>> In some swap scalability test, it is found that there are heavy lock >>> contention on swap cache even if we have split one swap cache ra

Re: [PATCH 13/16] mm: support THP migration to device private memory

2020-06-22 Thread Huang, Ying
> >> The patch adds THP migration support to device memory, but you need to >> handle migrate in (back to system memory) case correctly. The fault >> handling should look like THP CoW fault handling behavior (before >> 5.8): >> - if THP is enabled: allocate THP, f

[PATCH -V4] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-28 Thread Huang, Ying
From: Huang Ying In some swap scalability test, it is found that there are heavy lock contention on swap cache even if we have split one swap cache radix tree per swap device to one swap cache radix tree every 64 MB trunk in commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB t

Re: [PATCH -V3] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-28 Thread Huang, Ying
Daniel Jordan writes: > On Thu, May 28, 2020 at 01:32:40PM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> >> > On Mon, May 25, 2020 at 08:26:48AM +0800, Huang Ying wrote: >> >> diff --git a/mm/swapfile.c b/mm/swapfile.c >> >> index

Re: [PATCH -V3] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-27 Thread Huang, Ying
Daniel Jordan writes: > On Mon, May 25, 2020 at 08:26:48AM +0800, Huang Ying wrote: >> diff --git a/mm/swapfile.c b/mm/swapfile.c >> index 423c234aca15..0abd93d2a4fc 100644 >> --- a/mm/swapfile.c >> +++ b/mm/swapfile.c >> @@ -615,7 +615,8 @@ static bool sca

[PATCH -V3] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-24 Thread Huang Ying
put increases 18.5% from 2.99 GB/s to 3.54 GB/s. Signed-off-by: "Huang, Ying" Cc: Daniel Jordan Cc: Michal Hocko Cc: Minchan Kim Cc: Tim Chen Cc: Hugh Dickins --- Changelog: v3: - Fix cluster_next_cpu allocation and freeing. Thanks Daniel's comments! v2: - Rebased on late

Re: [PATCH -V2] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-21 Thread Huang, Ying
Daniel Jordan writes: > On Wed, May 20, 2020 at 11:15:02AM +0800, Huang Ying wrote: >> @@ -2827,6 +2865,11 @@ static struct swap_info_struct *alloc_swap_info(void) >> p = kvzalloc(struct_size(p, avail_lists, nr_node_ids), GFP_KERNEL); >> if (!p) >>

Re: [PATCH -V2] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-20 Thread Huang, Ying
Andrew Morton writes: > On Wed, 20 May 2020 11:15:02 +0800 Huang Ying wrote: > >> In some swap scalability test, it is found that there are heavy lock >> contention on swap cache even if we have split one swap cache radix >> tree per swap device to one swap cache radi

[PATCH -V2] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-19 Thread Huang Ying
put increases 18.5% from 2.99 GB/s to 3.54 GB/s. Signed-off-by: "Huang, Ying" Cc: Daniel Jordan Cc: Michal Hocko Cc: Minchan Kim Cc: Tim Chen Cc: Hugh Dickins --- Changelog: v2: - Rebased on latest mmotm tree (v5.7-rc5-mmots-2020-05-15-16-36), the mem cgroup change has influe

Re: [PATCH] swap: Add percpu cluster_next to reduce lock contention on swap cache

2020-05-18 Thread Huang, Ying
Daniel Jordan writes: > On Thu, May 14, 2020 at 03:04:24PM +0800, Huang Ying wrote: >> And the pmbench score increases 15.9%. > > What metric is that, and how long did you run the benchmark for? I run the benchmark for 1800s. The metric comes from the following output of t

Re: [PATCH] swap: Add percpu cluster_next to reduce lock contention on swap cache

2020-05-17 Thread Huang, Ying
Hi, Andrew, Andrew Morton writes: > On Thu, 14 May 2020 15:04:24 +0800 Huang Ying wrote: >> diff --git a/mm/swapfile.c b/mm/swapfile.c >> index 35be7a7271f4..9f1343b066c1 100644 >> --- a/mm/swapfile.c >> +++ b/mm/swapfile.c >> @@ -746,7 +746,16 @@ stat

[PATCH] swap: Add percpu cluster_next to reduce lock contention on swap cache

2020-05-14 Thread Huang Ying
ncreases 16.1% from 2.87 GB/s to 3.33 GB/s. Signed-off-by: "Huang, Ying" Cc: Michal Hocko Cc: Minchan Kim Cc: Tim Chen Cc: Hugh Dickins --- include/linux/swap.h | 1 + mm/swapfile.c| 27 +-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a

[PATCH -V2] mm, swap: Use prandom_u32_max()

2020-05-12 Thread Huang Ying
To improve the code readability and take advantage of the common implementation. Signed-off-by: "Huang, Ying" Acked-by: Michal Hocko Cc: Minchan Kim Cc: Tim Chen Cc: Hugh Dickins --- Changelog: v2: - Revise the patch description per Michal's comments. --- mm/swapfile.c | 2

Re: [PATCH] mm, swap: Use prandom_u32_max()

2020-05-12 Thread Huang, Ying
Michal Hocko writes: > On Tue 12-05-20 15:14:46, Huang, Ying wrote: >> Michal Hocko writes: >> >> > On Tue 12-05-20 14:41:46, Huang Ying wrote: >> >> To improve the code readability and get random number with higher >> >> quality. >> >

Re: [PATCH] mm, swap: Use prandom_u32_max()

2020-05-12 Thread Huang, Ying
Michal Hocko writes: > On Tue 12-05-20 14:41:46, Huang Ying wrote: >> To improve the code readability and get random number with higher >> quality. > > I understand the readability argument but why should prandom_u32_max > (which I was not aware of) provide a higher q

[PATCH] mm, swap: Use prandom_u32_max()

2020-05-12 Thread Huang Ying
To improve the code readability and get random number with higher quality. Signed-off-by: "Huang, Ying" Cc: Michal Hocko Cc: Minchan Kim Cc: Tim Chen Cc: Hugh Dickins --- mm/swapfile.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/swapfile.c b/mm/swapfi

Re: [PATCH 3/3] mm/swapfile.c: count won't be bigger than SWAP_MAP_MAX

2020-05-07 Thread Huang, Ying
Wei Yang writes: > On Wed, May 06, 2020 at 04:22:54PM +0800, Huang, Ying wrote: >>Wei Yang writes: >> >>> On Fri, May 01, 2020 at 03:48:53PM -0700, Andrew Morton wrote: >>>>On Fri, 1 May 2020 01:52:59 + Wei Yang >>>>wrote: >

Re: [PATCH 3/3] mm/swapfile.c: count won't be bigger than SWAP_MAP_MAX

2020-05-06 Thread Huang, Ying
o this. > > Take look into the code, I don't find a relationship between the swap count > and migration. Seems we just make a migration entry but not duplicate it. > If my understanding is correct. Per my understanding, one functionality of the error path is to catch the behavior that shouldn't happen at all. For example, if __swap_duplicate() is called for the migration entry because of some race condition. Best Regards, Huang, Ying

Re: [PATCH v2] mm/swapfile.c: simplify the scan loop in scan_swap_map_slots()

2020-04-28 Thread Huang, Ying
Wei Yang writes: > On Mon, Apr 27, 2020 at 08:55:33AM +0800, Huang, Ying wrote: >>Wei Yang writes: >> >>> On Sun, Apr 26, 2020 at 09:07:11AM +0800, Huang, Ying wrote: >>>>Wei Yang writes: >>>> >>>>> On Fri, Apr 24, 2020

Re: [PATCH RESEND] autonuma: Fix scan period updating

2019-07-29 Thread Huang, Ying
Mel Gorman writes: > On Mon, Jul 29, 2019 at 04:16:28PM +0800, Huang, Ying wrote: >> Srikar Dronamraju writes: >> >> >> >> >> >> >> if (lr_ratio >= NUMA_PERIOD_THRESHOLD) >> >> >> slow down scanning >> >>

Re: [PATCH RESEND] autonuma: Fix scan period updating

2019-07-29 Thread Huang, Ying
nd most memory accesses are shared. 7 threads runs on the node 0 and 2 threads runs on the node 1 based on CPU load balancing. Then the 2 threads on the node 1 will have "share" >> "private" and "remote" >> "local". But it doesn't help to speed up scanning. Best Regards, Huang, Ying

Re: [PATCH RESEND] autonuma: Fix scan period updating

2019-07-28 Thread Huang, Ying
Srikar Dronamraju writes: > * Huang, Ying [2019-07-26 15:45:39]: > >> Hi, Srikar, >> >> > >> > More Remote + Private page Accesses: >> > Most likely the Private accesses are going to be local accesses. >> > >> > In the unlikely eve

Re: [PATCH RESEND] autonuma: Fix scan period updating

2019-07-26 Thread Huang, Ying
Hi, Srikar, Srikar Dronamraju writes: > * Huang, Ying [2019-07-25 16:01:24]: > >> From: Huang Ying >> >> From the commit log and comments of commit 37ec97deb3a8 ("sched/numa: >> Slow down scan rate if shared faults dominate"), the autonuma scan

Re: kernel BUG at mm/swap_state.c:170!

2019-07-25 Thread Huang, Ying
Matthew Wilcox writes: > On Tue, Jul 23, 2019 at 01:08:42PM +0800, Huang, Ying wrote: >> @@ -2489,6 +2491,14 @@ static void __split_huge_page(struct page *page, >> struct list_head *list, >> /* complete memcg works before add pages to LRU */ >> mem_c

[PATCH RESEND] autonuma: Fix scan period updating

2019-07-25 Thread Huang, Ying
From: Huang Ying >From the commit log and comments of commit 37ec97deb3a8 ("sched/numa: Slow down scan rate if shared faults dominate"), the autonuma scan period should be increased (scanning is slowed down) if the majority of the page accesses are shared with other processes. B

Re: kernel BUG at mm/swap_state.c:170!

2019-07-25 Thread Huang, Ying
Mikhail Gavrilov writes: > On Tue, 23 Jul 2019 at 10:08, Huang, Ying wrote: >> >> Thanks! I have found another (easier way) to reproduce the panic. >> Could you try the below patch on top of v5.2-rc2? It can fix the panic >> for me. >> > > Thanks!

Re: kernel BUG at mm/swap_state.c:170!

2019-07-22 Thread Huang, Ying
Mikhail Gavrilov writes: > On Mon, 22 Jul 2019 at 12:53, Huang, Ying wrote: >> >> Yes. This is quite complex. Is the transparent huge page enabled in >> your system? You can check the output of >> >> $ cat /sys/kernel/mm/transparent_hugepage/e

Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression

2019-07-22 Thread Huang, Ying
"Huang, Ying" writes: > Rong Chen writes: > >> On 6/26/19 11:17 AM, Josef Bacik wrote: >>> On Wed, Jun 26, 2019 at 10:39:36AM +0800, Rong Chen wrote: >>>> On 6/25/19 10:22 PM, Josef Bacik wrote: >>>>> On Fri, Jun 21, 2019 at 08:48:0

Re: kernel BUG at mm/swap_state.c:170!

2019-07-22 Thread Huang, Ying
Mikhail Gavrilov writes: > On Mon, 22 Jul 2019 at 06:37, huang ying wrote: >> >> I am trying to reproduce this bug. Can you give me some information >> about your test case? > > It not easy, but I try to explain: > > 1. I have the system with 32Gb RAM, 6

Re: kernel BUG at mm/swap_state.c:170!

2019-07-21 Thread huang ying
kernel BUG at > mm/swap_state.c:170! I am trying to reproduce this bug. Can you give me some information about your test case? Best Regards, Huang, Ying

Re: [PATCH -mm] autonuma: Fix scan period updating

2019-07-15 Thread Huang, Ying
Mel Gorman writes: > On Fri, Jul 12, 2019 at 06:48:05PM +0800, Huang, Ying wrote: >> > Ordinarily I would hope that the patch was motivated by observed >> > behaviour so you have a metric for goodness. However, for NUMA balancing >> > I would typically run basic wor

Re: [PATCH -mm] autonuma: Fix scan period updating

2019-07-12 Thread Huang, Ying
Mel Gorman writes: > On Thu, Jul 04, 2019 at 08:32:06AM +0800, Huang, Ying wrote: >> Mel Gorman writes: >> >> > On Tue, Jun 25, 2019 at 09:23:22PM +0800, huang ying wrote: >> >> On Mon, Jun 24, 2019 at 10:25 PM Mel Gorman wrote: >> >> > >

Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression

2019-07-08 Thread Huang, Ying
Rong Chen writes: > On 6/26/19 11:17 AM, Josef Bacik wrote: >> On Wed, Jun 26, 2019 at 10:39:36AM +0800, Rong Chen wrote: >>> On 6/25/19 10:22 PM, Josef Bacik wrote: >>>> On Fri, Jun 21, 2019 at 08:48:03AM +0800, Huang, Ying wrote: >>>>> "Huang,

Re: [PATCH -mm] autonuma: Fix scan period updating

2019-07-03 Thread Huang, Ying
Mel Gorman writes: > On Tue, Jun 25, 2019 at 09:23:22PM +0800, huang ying wrote: >> On Mon, Jun 24, 2019 at 10:25 PM Mel Gorman wrote: >> > >> > On Mon, Jun 24, 2019 at 10:56:04AM +0800, Huang Ying wrote: >> > > The autonuma scan period shoul

Re: [PATCH -mm] autonuma: Fix scan period updating

2019-06-25 Thread huang ying
On Mon, Jun 24, 2019 at 10:25 PM Mel Gorman wrote: > > On Mon, Jun 24, 2019 at 10:56:04AM +0800, Huang Ying wrote: > > The autonuma scan period should be increased (scanning is slowed down) > > if the majority of the page accesses are shared with other processes. > > But i

[PATCH -mm -V2] mm, swap: Fix THP swap out

2019-06-24 Thread Huang, Ying
From: Huang Ying 0-Day test system reported some OOM regressions for several THP (Transparent Huge Page) swap test cases. These regressions are bisected to 6861428921b5 ("block: always define BIO_MAX_PAGES as 256"). In the commit, BIO_MAX_PAGES is set to 256 even when THP swap is en

Re: [PATCH -mm] mm, swap: Fix THP swap out

2019-06-24 Thread Huang, Ying
Ming Lei writes: > On Mon, Jun 24, 2019 at 12:44:41PM +0800, Huang, Ying wrote: >> Ming Lei writes: >> >> > Hi Huang Ying, >> > >> > On Mon, Jun 24, 2019 at 10:23:36AM +0800, Huang, Ying wrote: >> >> From: Huang Ying >> >>

Re: [PATCH -mm] mm, swap: Fix THP swap out

2019-06-23 Thread Huang, Ying
Ming Lei writes: > Hi Huang Ying, > > On Mon, Jun 24, 2019 at 10:23:36AM +0800, Huang, Ying wrote: >> From: Huang Ying >> >> 0-Day test system reported some OOM regressions for several >> THP (Transparent Huge Page) swap test cases. These regressions are &g

[PATCH -mm] autonuma: Fix scan period updating

2019-06-23 Thread Huang Ying
tracing the scan period changing and /proc/vmstat numa_pte_updates counter when running a multi-threaded memory accessing program (most memory areas are accessed by multiple threads). Fixes: 37ec97deb3a8 ("sched/numa: Slow down scan rate if shared faults dominate") Signed-off-by: "H

[PATCH -mm] mm, swap: Fix THP swap out

2019-06-23 Thread Huang, Ying
From: Huang Ying 0-Day test system reported some OOM regressions for several THP (Transparent Huge Page) swap test cases. These regressions are bisected to 6861428921b5 ("block: always define BIO_MAX_PAGES as 256"). In the commit, BIO_MAX_PAGES is set to 256 even when THP swap is en

Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression

2019-06-20 Thread Huang, Ying
"Huang, Ying" writes: > "Huang, Ying" writes: > >> Hi, Josef, >> >> kernel test robot writes: >> >>> Greeting, >>> >>> FYI, we noticed a -11.7% regression of aim7.jobs-per-min due to commit: >>> >>&

[PATCH -mm RESEND] mm: fix race between swapoff and mincore

2019-06-10 Thread Huang, Ying
From: Huang Ying Via commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trunks") on, after swapoff, the address_space associated with the swap device will be freed. So swap_address_space() users which touch the address_space need some kind of mechanism to prevent the address_

Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression

2019-06-10 Thread Huang, Ying
"Huang, Ying" writes: > Hi, Josef, > > kernel test robot writes: > >> Greeting, >> >> FYI, we noticed a -11.7% regression of aim7.jobs-per-min due to commit: >> >> >> commit: c8eaeac7b734347c3afba7008b7af62f37b9c140 ("btrfs: reser

Re: [PATCH -mm] mm, swap: Fix bad swap file entry warning

2019-05-31 Thread Huang, Ying
Michal Hocko writes: > On Fri 31-05-19 10:41:02, Huang, Ying wrote: >> From: Huang Ying >> >> Mike reported the following warning messages >> >> get_swap_device: Bad swap file entry 1401 >> >> This is produced by >>

Re: mmotm 2019-05-29-20-52 uploaded

2019-05-30 Thread Huang, Ying
"Huang, Ying" writes: > Hi, Mike, > > Mike Kravetz writes: > >> On 5/29/19 8:53 PM, a...@linux-foundation.org wrote: >>> The mm-of-the-moment snapshot 2019-05-29-20-52 has been uploaded to >>> >>>http://www.ozlabs.org/~akpm/mmotm/ &

[PATCH -mm] mm, swap: Fix bad swap file entry warning

2019-05-30 Thread Huang, Ying
From: Huang Ying Mike reported the following warning messages get_swap_device: Bad swap file entry 1401 This is produced by - total_swapcache_pages() - get_swap_device() Where get_swap_device() is used to check whether the swap device is valid and prevent it from being

Re: mmotm 2019-05-29-20-52 uploaded

2019-05-30 Thread Huang, Ying
r problem now. The reason is total_swapcache_pages() will call get_swap_device() for invalid swap device. So we need to find a way to silence the warning. I will post a fix ASAP. Best Regards, Huang, Ying

Re: [LKP] [btrfs] c8eaeac7b7: aim7.jobs-per-min -11.7% regression

2019-05-29 Thread Huang, Ying
Here's another regression, do you have time to take a look at this? Best Regards, Huang, Ying

Re: [v7 PATCH 2/2] mm: vmscan: correct some vmscan counters for THP swapout

2019-05-28 Thread Huang, Ying
gt; THP may get split during page reclaim, then a part of tail pages get > reclaimed instead of the whole 512 pages, but nr_scanned is accounted > by 512, particularly for direct reclaim. But, this should be not a > significant issue. > > Cc: "Huang, Ying" > Cc: Johannes We

[PATCH -mm] mm, swap: Simplify total_swapcache_pages() with get_swap_device()

2019-05-27 Thread Huang, Ying
From: Huang Ying total_swapcache_pages() may race with swapper_spaces[] allocation and freeing. Previously, this is protected with a swapper_spaces[] specific RCU mechanism. To simplify the logic/code complexity, it is replaced with get/put_swap_device(). The code line number is reduced too

Re: [v6 PATCH 2/2] mm: vmscan: correct some vmscan counters for THP swapout

2019-05-27 Thread Huang, Ying
Yang Shi writes: > On 5/27/19 3:06 PM, Huang, Ying wrote: >> Yang Shi writes: >> >>> Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after >>> swapped out"), THP can be swapped out in a whole. But, nr_reclaimed >>> and some o

Re: [v6 PATCH 2/2] mm: vmscan: correct some vmscan counters for THP swapout

2019-05-27 Thread Huang, Ying
gt; THP may get split during page reclaim, then a part of tail pages get > reclaimed instead of the whole 512 pages, but nr_scanned is accounted > by 512, particularly for direct reclaim. But, this should be not a > significant issue. > > Cc: "Huang, Ying" > Cc: Johannes W

Re: [RESEND v5 PATCH 2/2] mm: vmscan: correct some vmscan counters for THP swapout

2019-05-26 Thread Huang, Ying
Yang Shi writes: > On 5/27/19 10:11 AM, Huang, Ying wrote: >> Yang Shi writes: >> >>> Since commit bd4c82c22c36 ("mm, THP, swap: delay splitting THP after >>> swapped out"), THP can be swapped out in a whole. But, nr_reclaimed >>> and some o

<    1   2   3   4   5   6   7   8   9   10   >