Re: mm: BUG in page_move_anon_rmap

2016-07-12 Thread Hugh Dickins
On Mon, 11 Jul 2016, Dmitry Vyukov wrote: > On Fri, Jul 8, 2016 at 8:58 PM, Hugh Dickins wrote: > > Hi Dmitry, > > > > On Tue, 5 Jul 2016, Kirill A. Shutemov wrote: > >> On Mon, Jul 04, 2016 at 04:10:53PM -0700, Hugh Dickins wrote: > >> > On Fri, 1 Jul

[PATCH] tmpfs: fix regression hang in fallocate undo

2016-07-10 Thread Hugh Dickins
tmpfs: don't undo fallocate past its last page") Cc: sta...@vger.kernel.org Signed-off-by: Hugh Dickins --- mm/shmem.c |8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) --- 4.7-rc6/mm/shmem.c 2016-06-26 22:02:27.543373427 -0700 +++ linux/mm/shmem.c2016-07-10 15:

Re: mm: BUG in page_move_anon_rmap

2016-07-08 Thread Hugh Dickins
Hi Dmitry, On Tue, 5 Jul 2016, Kirill A. Shutemov wrote: > On Mon, Jul 04, 2016 at 04:10:53PM -0700, Hugh Dickins wrote: > > On Fri, 1 Jul 2016, Dmitry Vyukov wrote: > > > Hello, > > > > > > I am getting the following crash

Re: [PATCH 0/9] [REVIEW-REQUEST] [v4] System Calls for Memory Protection Keys

2016-07-08 Thread Hugh Dickins
On Thu, 7 Jul 2016, Dave Hansen wrote: > I'm resending these because Ingo has said that he'd "love to have > some high level MM review & ack for these syscall ABI extensions." > The only changes to the code in months have been in the selftests. > So, if anyone has been putting off taking a look at

Re: mm: BUG in page_move_anon_rmap

2016-07-04 Thread Hugh Dickins
On Fri, 1 Jul 2016, Dmitry Vyukov wrote: > Hello, > > I am getting the following crashes while running syzkaller fuzzer on > 00bf377d19ad3d80cbc7a036521279a86e397bfb (Jun 29). So far I did not > manage to reproduce it outside of fuzzer, but fuzzer hits it once per > hour or so. > > flags: 0xfffe0

Re: [PATCH v2] more mapcount page as kpage could reduce total replacement times than fewer mapcount one in probability.

2016-06-21 Thread Hugh Dickins
On Tue, 21 Jun 2016, zhouxianrong wrote: > hey hugh: > could you please give me some suggestion about this ? I must ask you to be more patient: everyone would like me to be quicker, but I cannot; and this does not appear to be urgent. Your idea makes sense to me; but if your patch seems obvi

Re: [BUG/REGRESSION] THP: broken page count after commit aa88b68c

2016-06-02 Thread Hugh Dickins
On Thu, 2 Jun 2016, Kirill A. Shutemov wrote: > On Thu, Jun 02, 2016 at 05:21:41PM +0200, Gerald Schaefer wrote: > > > > The following quick hack fixed the issue: > > > > diff --git a/mm/swap_state.c b/mm/swap_state.c > > index 0d457e7..c99463a 100644 > > --- a/mm/swap_state.c > > +++ b/mm/swap_s

Re: [patch] mm, migrate: increment fail count on ENOMEM

2016-05-23 Thread Hugh Dickins
On Mon, 23 May 2016, Andrew Morton wrote: > On Fri, 20 May 2016 15:31:21 +0200 Michal Hocko wrote: > > On Fri 20-05-16 15:19:12, Vlastimil Babka wrote: > > > On 05/20/2016 03:06 PM, Michal Hocko wrote: > > [...] > > > > Why don't we need also to count also retries? > > > > > > We could, but not l

Re: [PATCH] mm: compact: fix zoneindex in compact

2016-05-19 Thread Hugh Dickins
On Thu, 19 May 2016, Vlastimil Babka wrote: > On 05/19/2016 02:11 PM, Vlastimil Babka wrote: > > On 05/19/2016 01:58 PM, Chen Feng wrote: > >> While testing the kcompactd in my platform 3G MEM only DMA ZONE. > >> I found the kcompactd never wakeup. It seems the zoneindex > >> has already minus 1 be

Re: [PATCH] mm: unhide vmstat_text definition for CONFIG_SMP

2016-05-17 Thread Hugh Dickins
23...@dhcp22.suse.cz and > kbuild robot agrees > http://lkml.kernel.org/r/201605171333.anqjcwpy%fengguang...@intel.com Sorry for my noise, sorry for my silence, thanks to Arnd and everyone for chipping in. But now I try it, I find that even Michal's is not quite right: if you build

Re: [PATCH] ksm: fix conflict between mmput and scan_get_next_rmap_item

2016-05-06 Thread Hugh Dickins
On Fri, 6 May 2016, zhouchengming wrote: > On 2016/5/6 5:07, Andrew Morton wrote: > > On Thu, 5 May 2016 20:42:56 +0800 Zhou Chengming > > wrote: > > > > > A concurrency issue about KSM in the function scan_get_next_rmap_item. > > > > > > task A (ksmd):|task B (the mm'

Re: [Question] Missing data after DMA read transfer - mm issue with transparent huge page?

2016-05-02 Thread Hugh Dickins
On Fri, 29 Apr 2016, Nicolas Morey Chaisemartin wrote: > Hi everyone, > > This is a repost from a different address as it seems the previous one ended > in Gmail junk due to a domain error.. linux-kernel is a very high volume list which few are reading: that also will account for your lack of r

[PATCH mmotm] radix-tree: rewrite radix_tree_locate_item fix

2016-05-01 Thread Hugh Dickins
radix_tree_locate_item() is often returning the wrong index, causing swapoff of shmem to hang because it cannot find the swap entry there. __locate()'s use of base is bogus, it adds an offset twice into index. Signed-off-by: Hugh Dickins --- Fix to radix-tree-rewrite-radix_tree_locate_item.

Re: [BUG linux-next] Kernel panic found with linux-next-20160414

2016-04-27 Thread Hugh Dickins
On Wed, 20 Apr 2016, Shi, Yang wrote: > On 4/20/2016 1:01 AM, Hugh Dickins wrote: > > On Tue, 19 Apr 2016, Shi, Yang wrote: > > > Hi folks, > > > > > > When I ran ltp on linux-next-20160414 on my ARM64 machine, I got the > > > below > > > ker

Re: [BUG linux-next] kernel NULL pointer dereference on linux-next-20160420

2016-04-26 Thread Hugh Dickins
On Tue, 26 Apr 2016, Shi, Yang wrote: > On 4/21/2016 5:38 PM, Shi, Yang wrote: > > Hi folks, > > > > I did the below test with huge tmpfs on linux-next-20160420: > > > > # mount -t tmpfs huge=1 tmpfs /mnt > > # cd /mnt > > Then clone linux kernel > > > > Then I got the below bug, such test works

Re: linux-next: build failure after merge of the akpm-current tree

2016-04-21 Thread Hugh Dickins
On Thu, 21 Apr 2016, Stephen Rothwell wrote: > Hi Andrew, > > After merging the akpm-current tree, today's linux-next build (arm > multi_v7_defconfig) failed like this: > > mm/built-in.o: In function `handle_mm_fault': > frame_vector.c:(.text+0x29628): undefined reference to > `do_huge_pmd_anon

Re: [PATCH] mm: move huge_pmd_set_accessed out of huge_memory.c

2016-04-21 Thread Hugh Dickins
On Wed, 20 Apr 2016, Shi, Yang wrote: > Hi folks, > > I didn't realize pmd_* functions are protected by CONFIG_TRANSPARENT_HUGEPAGE > on the most architectures before I made this change. > > Before I fix all the affected architectures code, I want to check if you guys > think this change is wort

Re: [PATCHv7 00/29] THP-enabled tmpfs/shmem using compound pages

2016-04-20 Thread Hugh Dickins
On Mon, 18 Apr 2016, Shi, Yang wrote: > Hi Kirill, > > Finally, I got some time to look into and try yours and Hugh's patches, got Thank you. > two problems. > > 1. A quick boot up test on my ARM64 machine with your v7 tree shows some > unexpected error: > > systemd-journald[285]: Failed to s

Re: [BUG linux-next] Kernel panic found with linux-next-20160414

2016-04-20 Thread Hugh Dickins
On Tue, 19 Apr 2016, Shi, Yang wrote: > Hi folks, > > When I ran ltp on linux-next-20160414 on my ARM64 machine, I got the below > kernel panic: > > Unable to handle kernel paging request at virtual address ffc007846000 > pgd = ffc01e21d000 > [ffc007846000] *pgd=, *pud

Re: [PATCH 01/31] huge tmpfs: prepare counts in meminfo, vmstat and SysRq-m

2016-04-16 Thread Hugh Dickins
On Mon, 11 Apr 2016, Kirill A. Shutemov wrote: > On Tue, Apr 05, 2016 at 02:12:26PM -0700, Hugh Dickins wrote: > > ShmemFreeHoles will show the wastage from using huge pages for small, or > > sparsely occupied, or unrounded files: wastage not included in Shmem or > > MemFr

Re: [PATCH 03/31] huge tmpfs: huge=N mount option and /proc/sys/vm/shmem_huge

2016-04-16 Thread Hugh Dickins
On Mon, 11 Apr 2016, Kirill A. Shutemov wrote: > On Tue, Apr 05, 2016 at 02:15:05PM -0700, Hugh Dickins wrote: > > Plumb in a new "huge=1" or "huge=0" mount option to tmpfs: I don't > > want to get into a maze of boot options, madvises and fadvises at &

Re: [PATCH 09/31] huge tmpfs: avoid premature exposure of new pagetable

2016-04-16 Thread Hugh Dickins
On Mon, 11 Apr 2016, Kirill A. Shutemov wrote: > On Tue, Apr 05, 2016 at 02:24:23PM -0700, Hugh Dickins wrote: > > > > That itself is not a problem on x86_64, but there's plenty more: > > how about those places which use pte_offset_map_lock() - if that > > spi

Re: [PATCH mmotm 5/5] huge tmpfs: add shmem_pmd_fault()

2016-04-16 Thread Hugh Dickins
On Sun, 17 Apr 2016, Kirill A. Shutemov wrote: > On Sat, Apr 16, 2016 at 04:41:33PM -0700, Hugh Dickins wrote: > > The pmd_fault() method gives the filesystem an opportunity to place > > a trans huge pmd entry at *pmd, before any pagetable is exposed (and > > an opportuni

[PATCH mmotm 5/5] huge tmpfs: add shmem_pmd_fault()

2016-04-16 Thread Hugh Dickins
pfs should (like anon THP) or should not (like DAX) participate in deposit/withdraw protocol. Signed-off-by: Hugh Dickins --- I've been testing with this applied on top of mmotm plus 1-4/5, but I suppose the right place for it is immediately after huge-tmpfs-map-shmem-by-huge-page-pmd-

[PATCH mmotm 4/5] huge tmpfs: avoid premature exposure of new pagetable revert

2016-04-16 Thread Hugh Dickins
ly. Eventually arrived at the obvious: shmem should use the new pmd_fault(). Reported-by: kernel test robot Reported-by: Xiong Zhou Signed-off-by: Hugh Dickins --- mm/filemap.c | 10 -- mm/memory.c | 225 + 2 files changed, 101 insertions(+)

[PATCH mmotm 3/5] huge tmpfs recovery: tweak shmem_getpage_gfp to fill team fix

2016-04-16 Thread Hugh Dickins
PageTeam only after this check. But the check is guarding against a racing disband, which cannot happen before the head is published as PageTeam, plus we have an additional reference on the head which keeps it safe throughout: so very easily fixed. Reported-by: Mika Penttila Signed-off-by: Hugh Dickins

[PATCH mmotm 2/5] huge tmpfs: fix mlocked meminfo track huge unhuge mlocks fix

2016-04-16 Thread Hugh Dickins
ehave differently. Reported-by: kbuild test robot Signed-off-by: Hugh Dickins --- mm/rmap.c |6 +- 1 file changed, 5 insertions(+), 1 deletion(-) --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1445,8 +1445,12 @@ static int try_to_unmap_one(struct page */ if (!(flags & TTU_IGN

[PATCH mmotm 1/5] huge tmpfs: try to allocate huge pages split into a team fix

2016-04-16 Thread Hugh Dickins
seen on arm when putting together linux-next. Reported-by: Stephen Rothwell Signed-off-by: Hugh Dickins --- mm/shmem.c |1 - 1 file changed, 1 deletion(-) --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1744,7 +1744,6 @@ static inline struct page *shmem_hugetea static inline void shmem_disband_hug

Re: binary execution from DAX mount hang since next-20160407

2016-04-15 Thread Hugh Dickins
> cp hl /daxmnt > /daxmnt/hl > # cat hl.c > void main() > { > printf("ok\n"); > } > # cc hl.c -o hl > > Bisecting commits between 0406 and 0407 tag, points to this: > > d7c7d56ca61aec18e5e0cb3a64e50073c42195f7 is the first bad commit > commit d

Re: mmotm woes, mainly compaction

2016-04-14 Thread Hugh Dickins
On Fri, 15 Apr 2016, Vlastimil Babka wrote: > On 04/14/2016 10:15 PM, Hugh Dickins wrote: > > ... because the thing that was really wrong (and I spent far too long > > studying compaction.c before noticing in page_alloc.c) was this: > > > > /* > >

Re: mmotm woes, mainly compaction

2016-04-14 Thread Hugh Dickins
On Tue, 12 Apr 2016, Michal Hocko wrote: > On Tue 12-04-16 00:18:00, Hugh Dickins wrote: > > Michal, I'm sorry to say that I now find that I misinformed you. > > > > You'll remember when we were chasing the order=2 OOMs on two of my > > machines at the end

Re: [lkp] [huge tmpfs] d7c7d56ca6: vm-scalability.throughput -5.5% regression

2016-04-12 Thread Hugh Dickins
On Wed, 13 Apr 2016, kernel test robot wrote: > FYI, we noticed that vm-scalability.throughput -5.5% regression on > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master > commit d7c7d56ca61aec18e5e0cb3a64e50073c42195f7 ("huge tmpfs: avoid premature > exposure of new page

mmotm woes, mainly compaction

2016-04-12 Thread Hugh Dickins
ack the life of cc->migratepages if we don't assign it to a migratelist variable. 6. Remove unused bool success from kcompactd_do_work(). Signed-off-by: Hugh Dickins --- 4.6-rc2-mm1/mm/compaction.c 2016-04-10 09:43:20.314514944 -0700 +++ linux/mm/compaction.c 2016-04-

Re: linux-next: build failure after merge of the akpm-current tree

2016-04-06 Thread Hugh Dickins
On Thu, 7 Apr 2016, Stephen Rothwell wrote: > Hi Andrew, > > After merging the akpm-current tree, today's linux-next build (arm > multi_v7_defconfig) failed like this: > > In file included from include/linux/linkage.h:4:0, > from include/linux/fs.h:4, > from mm/

Re: [PATCH 12/31] huge tmpfs: extend get_user_pages_fast to shmem pmd

2016-04-06 Thread Hugh Dickins
On Wed, 6 Apr 2016, Ingo Molnar wrote: > * Hugh Dickins wrote: > > > --- > > Cc'ed to arch maintainers as an FYI: this patch is not expected to > > go into the tree in the next few weeks, and depends upon a PageTeam > > definition not yet available outside t

Re: [PATCH 23/31] huge tmpfs recovery: framework for reconstituting huge pages

2016-04-06 Thread Hugh Dickins
On Wed, 6 Apr 2016, Mika Penttila wrote: > On 04/06/2016 12:53 AM, Hugh Dickins wrote: > > +static void shmem_recovery_work(struct work_struct *work) ... > > + > > + if (head) { > > + /* We are resuming work from a previous partial recovery */ > &

Re: [PATCH 17/31] kvm: teach kvm to map page teams as huge pages.

2016-04-05 Thread Hugh Dickins
On Wed, 6 Apr 2016, Paolo Bonzini wrote: > On 05/04/2016 23:41, Hugh Dickins wrote: > > +/* > > + * We are holding kvm->mmu_lock, serializing against mmu notifiers. > > + * We have a ref on page. > > ... > > +static bool is_huge_tmpfs(struct kvm_vcpu *vcpu, >

Re: [PATCH 09/11] mm, compaction: Abstract compaction feedback to helpers

2016-04-05 Thread Hugh Dickins
On Tue, 5 Apr 2016, Andrew Morton wrote: > On Tue, 5 Apr 2016 13:25:31 +0200 Michal Hocko wrote: > > - if (is_thp_gfp_mask(gfp_mask)) { > > - /* > > -* If compaction is deferred for high-order allocations, it is > > -* because sync compaction recently failed. I

[PATCH 31/31] huge tmpfs: no kswapd by default on sync allocations

2016-04-05 Thread Hugh Dickins
From: Andres Lagar-Cavilla This triggers early compaction abort while in process context, to ameliorate mmap semaphore stalls. Suggested-by: David Rientjes Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- Documentation/filesystems/tmpfs.txt |5 +++-- mm/shmem.c

[PATCH 30/31] huge tmpfs: shmem_huge_gfpmask and shmem_recovery_gfpmask

2016-04-05 Thread Hugh Dickins
ed gfpmask with GFP_TRANSHUGE in a couple of places: which is appropriate for anonymous THP, but needed a little rework to extend it to huge tmpfs usage, when we're hoping to be able to tune behavior with these sysctls. Signed-off-by: Hugh Dickins --- Documentation/filesyst

[PATCH 29/31] huge tmpfs recovery: page migration call back into shmem

2016-04-05 Thread Hugh Dickins
(), so that the migrated page can be made PageTeam immediately. Which allows the SHMEM_RETRY_HUGE_PAGE hugehint to be reintroduced, for what little that's worth. Signed-off-by: Hugh Dickins --- include/linux/migrate_mode.h |2 include/linux/shmem_fs.h |6 + include/trace/events/m

[PATCH 28/31] huge tmpfs recovery: debugfs stats to complete this phase

2016-04-05 Thread Hugh Dickins
ad:6831300 work_already:43218374 work_queued:16221 work_too_late:2 work_too_many:0 Signed-off-by: Hugh Dickins --- Documentation/filesystems/tmpfs.txt |2 mm/shmem.c | 91 -- 2 files changed, 88 insertions(+), 5 deletions(-) --- a/Doc

[PATCH 27/31] huge tmpfs recovery: tweak shmem_getpage_gfp to fill team

2016-04-05 Thread Hugh Dickins
fp() knows what adjustments to make when we have allocated too far. Signed-off-by: Hugh Dickins --- mm/shmem.c | 56 --- 1 file changed, 49 insertions(+), 7 deletions(-) --- a/mm/shmem.c +++ b/mm/shmem.c @@ -464,6 +464,7 @@

[PATCH 26/31] huge tmpfs recovery: shmem_recovery_swapin to read from swap

2016-04-05 Thread Hugh Dickins
wapin() better not place that inside the huge page it is helping to build. Ifdef CONFIG_SWAP around it and its shmem_next_swap() helper because a couple of functions it calls are undeclared without CONFIG_SWAP. Signed-off-by: Hugh Dickins --- mm/shmem.c |

[PATCH 25/31] huge tmpfs recovery: shmem_recovery_remap & remap_team_by_pmd

2016-04-05 Thread Hugh Dickins
can be asked of remap_team_by_ptes(). Signed-off-by: Hugh Dickins --- include/linux/pageteam.h |2 mm/huge_memory.c | 87 + mm/shmem.c | 76 3 files changed, 165 insertions(+) --- a/include/l

[PATCH 24/31] huge tmpfs recovery: shmem_recovery_populate to fill huge page

2016-04-05 Thread Hugh Dickins
wards, but some comments on swap in this commit may anticipate that: sorry, a precise sequence of developing comments took too much trouble. Signed-off-by: Hugh Dickins --- include/linux/migrate.h|1 include/trace/events/migrate.h |3 mm/migrate.c | 15 +

[PATCH 23/31] huge tmpfs recovery: framework for reconstituting huge pages

2016-04-05 Thread Hugh Dickins
shmem_alloc_page()'s retry after shrinking is disabled: in early testing, the shrinker was too eager to undo the work of recovery. That was probably a side-effect of bugs at that time, but it still seems right to reduce the latency of shmem_fault() when it has a second chance. Signed-off-by: H

[PATCH 22/31] huge tmpfs: /proc//smaps show ShmemHugePages

2016-04-05 Thread Hugh Dickins
t() function to help get the PSS accounting right, now that compound pages are accounting correctly for ptes inside pmds; but nothing else needs that function, so keep it out of page_mapcount(). Signed-off-by: Hugh Dickins --- Documentation/filesystems/proc.txt | 10 +--- Documentation/filesy

[PATCH 21/31] huge tmpfs: show page team flag in pageflags

2016-04-05 Thread Hugh Dickins
From: Andres Lagar-Cavilla For debugging and testing. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- This patchset has been based on v4.6-rc2: here we get a clash with the addition of KPF_MOVABLE in current mmotm, not hard to fix up. Documentation/vm/pagemap.txt

[PATCH 20/31] huge tmpfs: mem_cgroup shmem_hugepages accounting

2016-04-05 Thread Hugh Dickins
tiated: the page lock is held on the page being instantiated, not on the team head, so we do still need the tree_lock to serialize them. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- Documentation/cgroup-v1/memory.txt |2 + Documentation/filesystems/tmpfs.txt

[PATCH 16/31] kvm: plumb return of hva when resolving page fault.

2016-04-05 Thread Hugh Dickins
From: Andres Lagar-Cavilla So we don't have to redo this work later. Note the hva is not racy, it is simple arithmetic based on the memslot. This will be used in the huge tmpfs commits. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- Cc'ed to k...@vger.kernel

[PATCH 19/31] huge tmpfs: mem_cgroup shmem_pmdmapped accounting

2016-04-05 Thread Hugh Dickins
trate better on the PageTeam locking issues which follow in the next patch. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- include/linux/memcontrol.h |2 ++ include/linux/pageteam.h | 16 mm/huge_memory.c |4 mm/memcontrol.c

[PATCH 18/31] huge tmpfs: mem_cgroup move charge on shmem huge pages

2016-04-05 Thread Hugh Dickins
me memcg, nowhere is that enforced - perhaps one day we shall need to enforce such a limitation, but not so far. Signed-off-by: Hugh Dickins --- mm/memcontrol.c | 103 +- 1 file changed, 58 insertions(+), 45 deletions(-) --- a/mm/memcontrol.c +

[PATCH 17/31] kvm: teach kvm to map page teams as huge pages.

2016-04-05 Thread Hugh Dickins
From: Andres Lagar-Cavilla Include a small treatise on the locking rules around page teams. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- Cc'ed to k...@vger.kernel.org as an FYI: this patch is not expected to go into the tree in the next few weeks, and depends u

[PATCH 15/31] huge tmpfs: fix Mapped meminfo, track huge & unhuge mappings

2016-04-05 Thread Hugh Dickins
APPED correctly for huge tmpfs, adjust vmscan's zone_unmapped_file_pages() to exclude NR_SHMEM_PMDMAPPED, which it clearly would not want included. Whereas minimum_image_size() in kernel/power/snapshot.c? I have not grasped the basis for that calculation, so leaving untouched. Signed-off-by: Hu

[PATCH 14/31] huge tmpfs: fix Mlocked meminfo, track huge & unhuge mlocks

2016-04-05 Thread Hugh Dickins
to check every page. Signed-off-by: Hugh Dickins --- include/linux/pageteam.h | 38 +++ mm/huge_memory.c | 15 ++- mm/internal.h| 26 +++-- mm/mlock.c | 181 + mm/rmap.c| 44 +--- 5 files change

[PATCH 13/31] huge tmpfs: use Unevictable lru with variable hpage_nr_pages

2016-04-05 Thread Hugh Dickins
he Unevictable LRU, assigning it weight again and giving it a new life on the Active and Inactive LRUs. As it is, I'm hoping PageReferenced gives a good enough hint as to whether a page should be retained, when shmem_evictify_hugetails() brings it back from Unevictable to Inactive.

[PATCH 12/31] huge tmpfs: extend get_user_pages_fast to shmem pmd

2016-04-05 Thread Hugh Dickins
that missed being propagated to the other architectures? No, see commit 8ee53820edfd ("thp: mmu_notifier_test_young"): it's a KVM GRU EPT thing, maybe not useful beyond x86. I've just followed the established practice in each architecture. Signed-off-by: Hugh Dickins --- Cc

[PATCH 11/31] huge tmpfs: disband split huge pmds on race or memory failure

2016-04-05 Thread Hugh Dickins
mappings(): it must recognize a hugely mapped team before concluding that the page is not mapped. (And still no support for soft_offline(), which will have to wait for page migration of teams.) Signed-off-by: Hugh Dickins --- mm/memory-failure.c |7 ++- mm/shmem.c | 30 ++

[PATCH 10/31] huge tmpfs: map shmem by huge page pmd or by page team ptes

2016-04-05 Thread Hugh Dickins
skips almost all of its complications, going to its own simpler handling. Kirill has a much better idea of what copy_huge_pmd() should do for pagecache: nothing, just as we don't copy shared file ptes. I shall adopt his idea in a future version, but for now show how to dup team. Signed-off-by:

[PATCH 09/31] huge tmpfs: avoid premature exposure of new pagetable

2016-04-05 Thread Hugh Dickins
t should certainly switch here to preallocating the pagetable outside the page lock. But I've not yet written and tested that change. Signed-off-by: Hugh Dickins --- mm/filemap.c | 10 +- mm/memory.c | 215 ++--- 2 files changed, 123 insertions(+),

[PATCH 08/31] huge tmpfs: try_to_unmap_one use page_check_address_transhuge

2016-04-05 Thread Hugh Dickins
n a later patch, and would no longer be appropriate here. Say "pmdval" as usual, instead of the "pmde" I made up for mm_find_pmd() before. Update the comment in mm_find_pmd() to generalise it away from just the anon_vma lock. Signed-off-by: Hugh Dickins --- includ

[PATCH 07/31] huge tmpfs: get_unmapped_area align & fault supply huge page

2016-04-05 Thread Hugh Dickins
s, asking for fallocate to be used explicitly for that. (hugetlbfs behaves even less standardly: its mmap extends the i_size of the object.) Signed-off-by: Hugh Dickins --- Documentation/filesystems/tmpfs.txt |8 + drivers/char/mem.c | 23 ++ include/linux/mm.h

[PATCH 06/31] huge tmpfs: shrinker to migrate and free underused holes

2016-04-05 Thread Hugh Dickins
tch, as the huge mapcount (offset by the HPAGE_PMD_NR occupancy) - it is never possible to map a sparsely occupied huge page, because that would expose stale data to the user. With this patch, the ShmemHugePages and ShmemFreeHoles lines of /proc/meminfo are shown correctly; but ShmemPmdMapped

[PATCH 05/31] huge tmpfs: avoid team pages in a few places

2016-04-05 Thread Hugh Dickins
BUG_ON, or hang on the non-existent ptlock; but there's much more to do later, to get mlock+munlock working properly. Signed-off-by: Hugh Dickins --- mm/compaction.c |5 + mm/memcontrol.c |4 ++-- mm/migrate.c| 15 ++- mm/mlock.c |2 +- mm/truncate.c

[PATCH 04/31] huge tmpfs: try to allocate huge pages, split into a team

2016-04-05 Thread Hugh Dickins
can never be mapped at this point. With this patch, the ShmemHugePages line of /proc/meminfo is shown, but it totals the amount of huge page memory allocated, not the amount fully used: so it may show ShmemHugePages exceeding Shmem. Signed-off-by: Hugh Dickins --- include/linux/page-flags

[PATCH 03/31] huge tmpfs: huge=N mount option and /proc/sys/vm/shmem_huge

2016-04-05 Thread Hugh Dickins
er-file huge/non-huge fcntl later. And allow shmem_huge two further values: -1 for use in emergencies, to force the huge option off from all mounts; and (currently) 2, to force the huge option on for all - very useful for testing. Signed-off-by: Hugh Dickins --- Documentation/filesystems/tmpfs.txt

[PATCH 02/31] huge tmpfs: include shmem freeholes in available memory

2016-04-05 Thread Hugh Dickins
is a case for including ShmemFreeHoles in the user-visible MemFree after all: I can see both sides of that argument, leaving it out so far. Signed-off-by: Hugh Dickins --- mm/page-writeback.c |2 ++ mm/page_alloc.c |6 ++ mm/util.c |1 + 3 files changed, 9 insertions

[PATCH 01/31] huge tmpfs: prepare counts in meminfo, vmstat and SysRq-m

2016-04-05 Thread Hugh Dickins
fs/proc/meminfo.c and drivers/base/node.c: the shorter names help. Clarify a comment in page_remove_rmap() to refer to "hugetlbfs pages" rather than hugepages generally. I left arch/tile/mm/pgtable.c's show_mem() unchanged: tile does not HAVE_ARCH_TRANSPARENT_HUGEPAGE. Signed-off-b

[PATCH 00/31] huge tmpfs: THPagecache implemented by teams

2016-04-05 Thread Hugh Dickins
Here is my "huge tmpfs" implementation of Transparent Huge Pagecache, rebased to v4.6-rc2 plus the "mm: easy preliminaries to THPagecache" series. The design is just the same as before, when I posted against v3.19: using a team of pagecache pages placed within a huge-order extent, instead of using

[PATCH 10/10] arch: fix has_transparent_hugepage()

2016-04-05 Thread Hugh Dickins
the first time it's called). Signed-off-by: Hugh Dickins --- arch/arc/include/asm/hugepage.h |2 - arch/arm/include/asm/pgtable-3level.h|5 arch/arm64/include/asm/pgtable.h |5 arch/mips/include/asm/pgtable.h |1 ar

[PATCH 09/10] huge pagecache: mmap_sem is unlocked when truncation splits pmd

2016-04-05 Thread Hugh Dickins
() check, as Kirill Shutemov has done. And let's get this in, without waiting for any particular huge pagecache implementation to reach the tree. Signed-off-by: Hugh Dickins --- mm/memory.c | 11 ++- 1 file changed, 2 insertions(+), 9 deletions(-) --- a/mm/memory.c +++ b/mm/memor

[PATCH 07/10] huge mm: move_huge_pmd does not need new_vma

2016-04-05 Thread Hugh Dickins
Remove move_huge_pmd()'s redundant new_vma arg: all it was used for was a VM_NOHUGEPAGE check on new_vma flags, but the new_vma is cloned from the old vma, so a trans_huge_pmd in the new_vma will be as acceptable as it was in the old vma, alignment and size permitting. Signed-off-by: Hugh Di

[PATCH 08/10] huge pagecache: extend mremap pmd rmap lockout to files

2016-04-05 Thread Hugh Dickins
move_ptes() and move_page_tables(), and delete the VM_BUG_ON_VMA which rejected vm_file and required anon_vma. Signed-off-by: Hugh Dickins --- mm/mremap.c | 42 ++ 1 file changed, 22 insertions(+), 20 deletions(-) --- a/mm/mremap.c +++ b/mm/mremap.

[PATCH 06/10] mm: /proc/sys/vm/stat_refresh to force vmstat update

2016-04-05 Thread Hugh Dickins
for the moment by just ignoring the anomaly on those. Signed-off-by: Hugh Dickins --- Documentation/sysctl/vm.txt | 14 include/linux/vmstat.h |4 ++ kernel/sysctl.c |7 mm/vmstat.c | 58 ++ 4 fil

[PATCH 05/10] tmpfs: mem_cgroup charge fault to vm_mm not current mm

2016-04-05 Thread Hugh Dickins
rs' way: change the common shmem_getpage() wrapper to hide fault_mm and fault_type as well as gfp. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- mm/shmem.c | 61 --- 1 file changed, 34 insertions(+), 27 deletions(-) --

[PATCH 04/10] tmpfs: preliminary minor tidyups

2016-04-05 Thread Hugh Dickins
quot; test jogged it back to reality. Signed-off-by: Hugh Dickins --- include/linux/mempolicy.h |6 +++ mm/shmem.c| 69 +--- 2 files changed, 32 insertions(+), 43 deletions(-) --- a/include/linux/mempolicy.h +++ b/include/linux/mempolicy.h @@ -22

[PATCH 03/10] mm: use __SetPageSwapBacked and dont ClearPageSwapBacked

2016-04-05 Thread Hugh Dickins
robably scope for further __SetPageFlags in other places, but SwapBacked is the one I'm interested in at the moment. Signed-off-by: Hugh Dickins --- Sorry, Mel did give Reviewed-by: Mel Gorman a year ago, but the kernel has moved on since then, so it feels slightly ruder to carry that forward without a

[PATCH 02/10] mm: update_lru_size do the __mod_zone_page_state

2016-04-05 Thread Hugh Dickins
peculiar to mem_cgroups: so better use the name update_lru_size, calls mem_cgroup_update_lru_size when CONFIG_MEMCG. Signed-off-by: Hugh Dickins --- include/linux/memcontrol.h |6 -- include/linux/mm_inline.h | 24 ++-- mm/memcontrol.c|2 ++ mm/vmscan.c

[PATCH 01/10] mm: update_lru_size warn and reset bad lru_size

2016-04-05 Thread Hugh Dickins
the particular bug which suggested this change was mine alone, and since fixed. Make it a WARN_ONCE: the first occurrence is the most informative, a flurry may follow, yet even when rate-limited little more is learnt. Signed-off-by: Hugh Dickins --- include/linux/mm_inline.h |2 +- mm

[PATCH 00/10] mm: easy preliminaries to THPagecache

2016-04-05 Thread Hugh Dickins
hes first. Signed-off-by: Hugh Dickins --- mm/filemap.c |4 ++-- mm/memory.c |6 +++--- mm/rmap.c|2 +- mm/shmem.c | 18 +- 4 files changed, 15 insertions(+), 15 deletions(-) --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2178,8 +2178,8 @@ repeat:

Re: 4.5 shmem lockdep/out-of-bound/list corruption disaster

2016-03-28 Thread Hugh Dickins
On Mon, 28 Mar 2016, Dave Jones wrote: > I hit this a few days ago. I'm not 100% what kernel it was running, > but I'm pretty sure it was a post 4.5 kernel from this merge window. Thanks for the report. All consistent with shmem_fallocate() somehow getting through its wake_up_all() without remo

Re: [PATCHv4 00/25] THP-enabled tmpfs/shmem

2016-03-28 Thread Hugh Dickins
On Mon, 28 Mar 2016, Kirill A. Shutemov wrote: > > I think I found it. I have refcounting screwed up in faultaround. > > This should fix the problem: Yes, this fixes it for me - thanks. Hugh > > diff --git a/mm/filemap.c b/mm/filemap.c > index 94c097ec08e7..1325bb4568d1 100644 > --- a/mm/file

Re: [PATCHv4 00/25] THP-enabled tmpfs/shmem

2016-03-25 Thread Hugh Dickins
On Fri, 25 Mar 2016, Kirill A. Shutemov wrote: > On Thu, Mar 24, 2016 at 12:08:55PM -0700, Hugh Dickins wrote: > > On Thu, 24 Mar 2016, Kirill A. Shutemov wrote: > > > On Wed, Mar 23, 2016 at 01:09:05PM -0700, Hugh Dickins wrote: > > > > The small files thing formed m

Re: [PATCHv4 00/25] THP-enabled tmpfs/shmem

2016-03-24 Thread Hugh Dickins
On Thu, 24 Mar 2016, Kirill A. Shutemov wrote: > On Wed, Mar 23, 2016 at 01:09:05PM -0700, Hugh Dickins wrote: > > The small files thing formed my first impression. My second > > impression was similar, when I tried mmap(NULL, size_of_RAM, > > PROT_READ|PROT_WRITE, MAP_ANONYM

Re: [PATCHv4 00/25] THP-enabled tmpfs/shmem

2016-03-23 Thread Hugh Dickins
On Sat, 12 Mar 2016, Kirill A. Shutemov wrote: > Here is an updated version of huge pages support implementation in > tmpfs/shmem. > > All known issues has been fixed. I'll continue with validation. > I will also send follow up patch with documentation update. > > Hugh, I would be glad to hear y

Re: [PATCH 2/9] mm, oom: introduce oom reaper

2016-03-22 Thread Hugh Dickins
tell the reclaim to cull the > page back and mlock it again. > > munlock_vma_pages_all is also safe to be called from the oom reaper > context because it doesn't sit on any locks but mmap_sem (for read). > > Signed-off-by: Michal Hocko > Cc: Andrea Argangeli > Acked-by:

Re: [PATCH 01/71] arc: get rid of PAGE_CACHE_* and page_cache_{get,release} macros

2016-03-20 Thread Hugh Dickins
On Sun, 20 Mar 2016, Linus Torvalds wrote: > On Sun, Mar 20, 2016 at 12:34 PM, Kirill A. Shutemov > wrote: > > > > Hm. Okay. Re-split this way would take some time. I'll post updated > > patchset tomorrow. > > Oh, I was assuming this was automated with coccinelle or at least some > simple shell s

Re: [PATCH 3/3] mm, oom: protect !costly allocations some more

2016-03-11 Thread Hugh Dickins
On Fri, 11 Mar 2016, Michal Hocko wrote: > On Fri 11-03-16 04:17:30, Hugh Dickins wrote: > > On Wed, 9 Mar 2016, Michal Hocko wrote: > > > Joonsoo has pointed out that this attempt is still not sufficient > > > becasuse we might have invoked only a single compaction ro

Re: [PATCH 3/3] mm, oom: protect !costly allocations some more

2016-03-11 Thread Hugh Dickins
On Wed, 9 Mar 2016, Michal Hocko wrote: > Joonsoo has pointed out that this attempt is still not sufficient > becasuse we might have invoked only a single compaction round which > is might be not enough. I fully agree with that. Here is my take on > that. It is again based on the number of retries

Re: [PATCH 2/5] oom reaper: handle mlocked pages

2016-03-08 Thread Hugh Dickins
On Tue, 8 Mar 2016, Michal Hocko wrote: > On Mon 29-02-16 14:41:39, Michal Hocko wrote: > > On Sun 28-02-16 19:19:11, Hugh Dickins wrote: > > > On Tue, 23 Feb 2016, Michal Hocko wrote: > > > > On Mon 22-02-16 17:36:07, David Rientjes wrote: > > >

Re: [PATCH] mm, oom: protect !costly allocations some more (was: Re: [PATCH 0/3] OOM detection rework v4)

2016-03-08 Thread Hugh Dickins
On Mon, 7 Mar 2016, Michal Hocko wrote: > On Mon 29-02-16 22:02:13, Michal Hocko wrote: > > Andrew, > > could you queue this one as well, please? This is more a band aid than a > > real solution which I will be working on as soon as I am able to > > reproduce the issue but the patch should help to

Re: THP-enabled filesystem vs. FALLOC_FL_PUNCH_HOLE

2016-03-04 Thread Hugh Dickins
On Fri, 4 Mar 2016, Dave Hansen wrote: > On 03/04/2016 03:26 AM, Kirill A. Shutemov wrote: > > On Thu, Mar 03, 2016 at 07:51:50PM +0300, Kirill A. Shutemov wrote: > >> Truncate and punch hole that only cover part of THP range is implemented > >> by zero out this part of THP. > >> > >> This have vis

Re: [PATCH 0/3] OOM detection rework v4

2016-03-03 Thread Hugh Dickins
On Thu, 3 Mar 2016, Michal Hocko wrote: > On Thu 03-03-16 01:54:43, Hugh Dickins wrote: > > On Tue, 1 Mar 2016, Michal Hocko wrote: > [...] > > > So I have tried the following: > > > diff --git a/mm/compaction.c b/mm/compaction.c > > > index 4d99e1f50

Re: [PATCH 0/3] OOM detection rework v4

2016-03-03 Thread Hugh Dickins
On Tue, 1 Mar 2016, Michal Hocko wrote: > [Adding Vlastimil and Joonsoo for compaction related things - this was a > large thread but the more interesting part starts with > http://lkml.kernel.org/r/alpine.LSU.2.11.1602241832160.15564@eggly.anvils] > > On Mon 29-02-16 23:29:06, Hug

Re: [PATCH 0/3] OOM detection rework v4

2016-02-29 Thread Hugh Dickins
On Mon, 29 Feb 2016, Michal Hocko wrote: > On Wed 24-02-16 19:47:06, Hugh Dickins wrote: > [...] > > Boot with mem=1G (or boot your usual way, and do something to occupy > > most of the memory: I think /proc/sys/vm/nr_hugepages provides a great > > way to gobble up most of

[PATCH] mm: __delete_from_page_cache show Bad page if mapped

2016-02-29 Thread Hugh Dickins
icted (rather than truncated), it won't have any vmas left, so it's safe(ish) to assume that the raised mapcount is erroneous, and we can discount it from page_count to avoid leaking the page (I'm less worried by leaking the occasional 4kB, than losing a potential 2MB page with each 4kB p

Re: [PATCH] mm: __delete_from_page_cache WARN_ON(page_mapped)

2016-02-29 Thread Hugh Dickins
On Mon, 29 Feb 2016, Kirill A. Shutemov wrote: > On Sun, Feb 28, 2016 at 08:49:10PM -0800, Hugh Dickins wrote: > > Commit e1534ae95004 ("mm: differentiate page_mapped() from page_mapcount() > > for compound pages") changed the famous BUG_ON(page_mapped(page)) in > >

[PATCH] mm: __delete_from_page_cache WARN_ON(page_mapped)

2016-02-28 Thread Hugh Dickins
x27;t have any vmas left, so it's safe(ish) to assume that the raised mapcount is erroneous, and we can discount it from page_count to avoid leaking the page (I'm less worried by leaking the occasional 4kB, than losing a potential 2MB page with each 4kB page leaked). Signed-off-by: Hugh Di

Re: [PATCH 2/5] oom reaper: handle mlocked pages

2016-02-28 Thread Hugh Dickins
On Tue, 23 Feb 2016, Michal Hocko wrote: > On Mon 22-02-16 17:36:07, David Rientjes wrote: > > > > Are we concerned about munlock_vma_pages_all() taking lock_page() and > > perhaps stalling forever, the same way it would stall in exit_mmap() for > > VM_LOCKED vmas, if another thread has locked t

<    1   2   3   4   5   6   7   8   9   10   >