Re: [PATCHv4 12/24] thp: PMD splitting without splitting compound page

2015-04-01 Thread Hugh Dickins
On Wed, 1 Apr 2015, Kirill A. Shutemov wrote: On Wed, Apr 01, 2015 at 12:08:35PM +0530, Aneesh Kumar K.V wrote: With this we now have pte mapping part of a compound page(). Now the gneric gup implementation does gup_pte_range() ptem = ptep = pte_offset_map(pmd, addr); do {

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-30 Thread Hugh Dickins
touched by the munmap() request should be unmapped. > > > > Please include > > Acked-by: Eric B Munson > > to the original patch. I would like to see the mmap man page adjusted > > to make note of this behavior as well. > > This is just a bug fix and I never

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-30 Thread Hugh Dickins
On Mon, 30 Mar 2015, Eric B Munson wrote: > On Sun, 29 Mar 2015, Hugh Dickins wrote: > > > > Eric, I apologize for bringing you in to the discussion, and then > > ignoring your input. I understand that you would like MAP_HUGETLB > > to behave more unders

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-30 Thread Hugh Dickins
On Mon, 30 Mar 2015, Eric B Munson wrote: On Sun, 29 Mar 2015, Hugh Dickins wrote: Eric, I apologize for bringing you in to the discussion, and then ignoring your input. I understand that you would like MAP_HUGETLB to behave more understandably. We can all agree that the existing

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-30 Thread Hugh Dickins
revert immediately if this patch arise some regression even if it's come from broken application code. Acked-by: KOSAKI Motohiro kosaki.motoh...@gmail.com and, without wishing to be confrontational, Nacked-by: Hugh Dickins hu...@google.com I agree with you that the risk on munmap is probably

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Mon, 30 Mar 2015, Xishi Qiu wrote: > On 2015/3/30 9:46, Xishi Qiu wrote: > > On 2015/3/30 8:43, Hugh Dickins wrote: > >> On Sat, 28 Mar 2015, Xishi Qiu wrote: > >>> On 2015/3/26 21:23, Xishi Qiu wrote: > >>> > >>>> Here are two

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-29 Thread Hugh Dickins
On Thu, 26 Mar 2015, David Rientjes wrote: > munmap(2) of hugetlb memory requires a length that is hugepage aligned, > otherwise it may fail. Add this to the documentation. Thanks for taking this on, David. But although munmap(2) is the one Davide called out, it goes beyond that, doesn't it?

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Sat, 28 Mar 2015, Xishi Qiu wrote: > On 2015/3/26 21:23, Xishi Qiu wrote: > > > Here are two panic logs from smart phone test, and the kernel version is > > v3.10. > > > > log1 is "Unable to handle kernel paging request at virtual address > > c0704da020", it should be ffc0704da020,

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Sat, 28 Mar 2015, Xishi Qiu wrote: On 2015/3/26 21:23, Xishi Qiu wrote: Here are two panic logs from smart phone test, and the kernel version is v3.10. log1 is Unable to handle kernel paging request at virtual address c0704da020, it should be ffc0704da020, right? That one

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-29 Thread Hugh Dickins
On Thu, 26 Mar 2015, David Rientjes wrote: munmap(2) of hugetlb memory requires a length that is hugepage aligned, otherwise it may fail. Add this to the documentation. Thanks for taking this on, David. But although munmap(2) is the one Davide called out, it goes beyond that, doesn't it? To

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Mon, 30 Mar 2015, Xishi Qiu wrote: On 2015/3/30 9:46, Xishi Qiu wrote: On 2015/3/30 8:43, Hugh Dickins wrote: On Sat, 28 Mar 2015, Xishi Qiu wrote: On 2015/3/26 21:23, Xishi Qiu wrote: Here are two panic logs from smart phone test, and the kernel version is v3.10. log1

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-25 Thread Hugh Dickins
On Wed, 22 Oct 2014, Davide Libenzi wrote: > [Resending with proper CC list suggested by Andrew] I have recently been reminded of this languishing in my inbox ;) (along with many others that I won't get to answer so quickly). And in turn it reminds me of an older from Joern, who was annoyed

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > On Wed, Mar 25, 2015 at 03:48:48PM -0700, Hugh Dickins wrote: > > On Wed, 25 Mar 2015, Hugh Dickins wrote: > > > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > > > > > > > > Since currently all THP

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Hugh Dickins wrote: > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > > > THP uses tail page refcounting to be able to split huge page at any > > time. Tail page refcounting is not needed for rest users of compound > > pages and it's harmful because

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
e about anything which can lead to > crash or other serious misbehaviour. > > Since currently all THP pages are anonymous and all drivers pages are > not, we can fix the __compound_tail_refcounted() check by requiring > PageAnon() to enable tail page refcounting. > > Signed-

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Kirill A. Shutemov wrote: > > We only need tail refcounting for THP, so I think this should fix the issue: > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 4a3a38522ab4..9ab432660adb 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -456,7

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Kirill A. Shutemov wrote: We only need tail refcounting for THP, so I think this should fix the issue: diff --git a/include/linux/mm.h b/include/linux/mm.h index 4a3a38522ab4..9ab432660adb 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -456,7 +456,7 @@

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Hugh Dickins wrote: On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: THP uses tail page refcounting to be able to split huge page at any time. Tail page refcounting is not needed for rest users of compound pages and it's harmful because of overhead. We try

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: On Wed, Mar 25, 2015 at 03:48:48PM -0700, Hugh Dickins wrote: On Wed, 25 Mar 2015, Hugh Dickins wrote: On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: Since currently all THP pages are anonymous and all drivers pages are not, we can

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
THP pages are anonymous and all drivers pages are not, we can fix the __compound_tail_refcounted() check by requiring PageAnon() to enable tail page refcounting. Signed-off-by: Kirill A. Shutemov kirill.shute...@linux.intel.com Acked-by: Hugh Dickins hu...@google.com --- include/linux/mm.h

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-25 Thread Hugh Dickins
On Wed, 22 Oct 2014, Davide Libenzi wrote: [Resending with proper CC list suggested by Andrew] I have recently been reminded of this languishing in my inbox ;) (along with many others that I won't get to answer so quickly). And in turn it reminds me of an older from Joern, who was annoyed that

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-24 Thread Hugh Dickins
On Tue, 24 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 09:40:02PM -0700, Hugh Dickins wrote: > > (I think Kirill has a problem of that kind in his page_remove_rmap scan). (And this one I mentioned to you at the conference :) > > > > It will be interesti

Re: [PATCH 00/16] Sanitize usage of ->flags and ->mapping for tail pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 05:28:47PM -0700, Hugh Dickins wrote: > > On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > > > > > Currently we take naive approach to page flags on compound -- we set the > > > flag

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 05:02:58PM -0700, Hugh Dickins wrote: > > On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > > > On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: > > > > On 03/19/2015 10:0

Re: [PATCH 00/16] Sanitize usage of -flags and -mapping for tail pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: On Sun, Mar 22, 2015 at 05:28:47PM -0700, Hugh Dickins wrote: On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: Currently we take naive approach to page flags on compound -- we set the flag on the page without consideration if the flag makes

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: On Sun, Mar 22, 2015 at 05:02:58PM -0700, Hugh Dickins wrote: On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: On 03/19/2015 10:08 AM, Kirill A. Shutemov wrote: The odd

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-24 Thread Hugh Dickins
On Tue, 24 Mar 2015, Kirill A. Shutemov wrote: On Sun, Mar 22, 2015 at 09:40:02PM -0700, Hugh Dickins wrote: (I think Kirill has a problem of that kind in his page_remove_rmap scan). (And this one I mentioned to you at the conference :) It will be interesting to see what Kirill does

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Konstantin Khlebnikov wrote: > On 21.02.2015 07:09, Hugh Dickins wrote: > > > > The "team_usage" field added to struct page (in union with "private") > > is somewhat vaguely named: because while the huge page is sparsely > &

Re: [PATCH 03/24] mm: use __SetPageSwapBacked and don't ClearPageSwapBacked

2015-03-22 Thread Hugh Dickins
On Wed, 25 Feb 2015, Mel Gorman wrote: > On Fri, Feb 20, 2015 at 07:56:15PM -0800, Hugh Dickins wrote: > > Commit 07a427884348 ("mm: shmem: avoid atomic operation during > > shmem_getpage_gfp") rightly replaced one instance of SetPageSwapBacked > > by __

Re: [PATCH 01/24] mm: update_lru_size warn and reset bad lru_size

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: > On Fri, Feb 20, 2015 at 07:51:16PM -0800, Hugh Dickins wrote: > > Though debug kernels have a VM_BUG_ON to help protect from misaccounting > > lru_size, non-debug kernels are liable to wrap it around: and then the > > vast uns

Re: [PATCH 00/24] huge tmpfs: an alternative approach to THPageCache

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: > > I scanned through the patches to get general idea on how it works. Thanks! > I'm not > sure that I will have time and will power to do proper code-digging before > the summit. I found few bugs in my patchset which I want to troubleshoot >

Re: [PATCH 00/16] Sanitize usage of ->flags and ->mapping for tail pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > Currently we take naive approach to page flags on compound -- we set the > flag on the page without consideration if the flag makes sense for tail > page or for compound page in general. This patchset try to sort this out > by defining per-flag

Re: [PATCH 02/16] page-flags: trivial cleanup for PageTrans* helpers

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > Use TESTPAGEFLAG_FALSE() to get it a bit cleaner. > > Signed-off-by: Kirill A. Shutemov Yeah, this is okay too. > --- > include/linux/page-flags.h | 18 +++--- > 1 file changed, 3 insertions(+), 15 deletions(-) > > diff --git

Re: [PATCH 01/16] mm: consolidate all page-flags helpers in

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > We have page-flags helper function declarations/definitions spread over > several header files. Let's consolidate them in . > > Signed-off-by: Kirill A. Shutemov Acked-by: Hugh Dickins I find this one helpful (assuming it

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: > > On 03/19/2015 10:08 AM, Kirill A. Shutemov wrote: > > > The odd exception is PG_dirty: sound uses compound pages and maps them > > > with PTEs. NO_COMPOUND triggers VM_BUG_ON() in

Re: [PATCH 02/16] page-flags: trivial cleanup for PageTrans* helpers

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: Use TESTPAGEFLAG_FALSE() to get it a bit cleaner. Signed-off-by: Kirill A. Shutemov kirill.shute...@linux.intel.com Yeah, this is okay too. --- include/linux/page-flags.h | 18 +++--- 1 file changed, 3 insertions(+), 15

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: On 03/19/2015 10:08 AM, Kirill A. Shutemov wrote: The odd exception is PG_dirty: sound uses compound pages and maps them with PTEs. NO_COMPOUND triggers VM_BUG_ON() in

Re: [PATCH 00/24] huge tmpfs: an alternative approach to THPageCache

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: I scanned through the patches to get general idea on how it works. Thanks! I'm not sure that I will have time and will power to do proper code-digging before the summit. I found few bugs in my patchset which I want to troubleshoot first.

Re: [PATCH 01/24] mm: update_lru_size warn and reset bad lru_size

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: On Fri, Feb 20, 2015 at 07:51:16PM -0800, Hugh Dickins wrote: Though debug kernels have a VM_BUG_ON to help protect from misaccounting lru_size, non-debug kernels are liable to wrap it around: and then the vast unsigned long size draws page

Re: [PATCH 03/24] mm: use __SetPageSwapBacked and don't ClearPageSwapBacked

2015-03-22 Thread Hugh Dickins
On Wed, 25 Feb 2015, Mel Gorman wrote: On Fri, Feb 20, 2015 at 07:56:15PM -0800, Hugh Dickins wrote: Commit 07a427884348 (mm: shmem: avoid atomic operation during shmem_getpage_gfp) rightly replaced one instance of SetPageSwapBacked by __SetPageSwapBacked, pointing out that the newly

Re: [PATCH 01/16] mm: consolidate all page-flags helpers in linux/page-flags.h

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: We have page-flags helper function declarations/definitions spread over several header files. Let's consolidate them in linux/page-flags.h. Signed-off-by: Kirill A. Shutemov kirill.shute...@linux.intel.com Acked-by: Hugh Dickins hu

Re: [PATCH 00/16] Sanitize usage of -flags and -mapping for tail pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: Currently we take naive approach to page flags on compound -- we set the flag on the page without consideration if the flag makes sense for tail page or for compound page in general. This patchset try to sort this out by defining per-flag policy

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Konstantin Khlebnikov wrote: On 21.02.2015 07:09, Hugh Dickins wrote: The team_usage field added to struct page (in union with private) is somewhat vaguely named: because while the huge page is sparsely occupied, it counts the occupancy; but once the huge page

Re: [PATCH] mm: shmem: check for mapping owner before dereferencing

2015-02-25 Thread Hugh Dickins
Thanks for the fix, Sasha. On Mon, 23 Feb 2015, Jens Axboe wrote: > On 02/23/2015 09:49 AM, Christoph Hellwig wrote: > > Looks good, > > > > Reviewed-by: Christoph Hellwig Acked-by: Hugh Dickins > > Shall I funnel this through for-linus? Please do, tha

Re: [PATCH] mm: shmem: check for mapping owner before dereferencing

2015-02-25 Thread Hugh Dickins
Thanks for the fix, Sasha. On Mon, 23 Feb 2015, Jens Axboe wrote: On 02/23/2015 09:49 AM, Christoph Hellwig wrote: Looks good, Reviewed-by: Christoph Hellwig h...@lst.de Acked-by: Hugh Dickins hu...@google.com Shall I funnel this through for-linus? Please do, thanks. -- Jens

[PATCH 24/24] kvm: teach kvm to map page teams as huge pages.

2015-02-20 Thread Hugh Dickins
From: Andres Lagar-Cavilla Include a small treatise on the locking rules around page teams. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- arch/x86/kvm/mmu.c | 155 +-- arch/x86/kvm/paging_tmpl.h |3 2 files changed, 132

[PATCH 23/24] kvm: plumb return of hva when resolving page fault.

2015-02-20 Thread Hugh Dickins
From: Andres Lagar-Cavilla So we don't have to redo this work later. Note the hva is not racy, it is simple arithmetic based on the memslot. This will be used in the huge tmpfs commits. Signed-off-by: Andres Lagar-Cavilla Signed-off-by: Hugh Dickins --- arch/x86/kvm/mmu.c | 16

[PATCH 22/24] huge tmpfs: fix Mapped meminfo, tracking huge and unhuge mappings

2015-02-20 Thread Hugh Dickins
correctly for huge tmpfs, adjust vmscan's zone_unmapped_file_pages() to exclude NR_SHMEM_PMDMAPPED, which it clearly would not want included. Whereas minimum_image_size() in kernel/power/snapshot.c? I have not grasped the basis for that calculation, so leaving untouched. Signed-off-by: Hugh Dickins

[PATCH 21/24] huge tmpfs: fix Mlocked meminfo, tracking huge and unhuge mlocks

2015-02-20 Thread Hugh Dickins
n. Signed-off-by: Hugh Dickins --- include/linux/pageteam.h | 38 +++ mm/huge_memory.c |6 + mm/internal.h| 25 +++-- mm/mlock.c | 181 ++--- mm/rmap.c| 34 -- 5 files changed, 193 insertions(+), 91

[PATCH 20/24] huge tmpfs: use Unevictable lru with variable hpage_nr_pages()

2015-02-20 Thread Hugh Dickins
. As it is, I'm hoping PageReferenced gives a good enough hint as to whether a page should be retained, when shmem_evictify_hugetails() brings it back from Unevictable to Inactive. Signed-off-by: Hugh Dickins --- include/linux/huge_mm.h | 13 +++ include/linux/pageteam.h | 48 ++- mm

[PATCH 19/24] huge tmpfs: disband split huge pmds on race or memory failure

2015-02-20 Thread Hugh Dickins
team before concluding that the page is not mapped. (And still no support for soft_offline(), which will have to wait for page migration of teams.) Signed-off-by: Hugh Dickins --- mm/memory-failure.c |8 +++- mm/shmem.c | 27 ++- 2 files changed, 33

[PATCH 18/24] huge tmpfs: mmap_sem is unlocked when truncation splits huge pmd

2015-02-20 Thread Hugh Dickins
d the issue. Signed-off-by: Hugh Dickins --- mm/memory.c | 13 ++--- 1 file changed, 2 insertions(+), 11 deletions(-) --- thpfs.orig/mm/memory.c 2015-02-20 19:34:48.083909034 -0800 +++ thpfs/mm/memory.c 2015-02-20 19:34:53.467896724 -0800 @@ -1219,18 +1219,9 @@ static

[PATCH 17/24] huge tmpfs: map shmem by huge page pmd or by page team ptes

2015-02-20 Thread Hugh Dickins
switches). Signed-off-by: Hugh Dickins --- include/linux/pageteam.h | 41 ++ mm/huge_memory.c | 238 ++--- mm/memory.c | 11 + 3 files changed, 273 insertions(+), 17 deletions(-) --- thpfs.orig/include/linux/pageteam.h 2015-02-20 19:34

[PATCH 16/24] huge tmpfs: fix problems from premature exposure of pagetable

2015-02-20 Thread Hugh Dickins
after, allowing huge pmd to be chosent first. Signed-off-by: Hugh Dickins --- mm/filemap.c | 10 +- mm/memory.c | 202 +++-- 2 files changed, 118 insertions(+), 94 deletions(-) --- thpfs.orig/mm/filemap.c 2015-02-08 18:54:22.0 -0

[PATCH 15/24] huge tmpfs: rework page_referenced_one and try_to_unmap_one

2015-02-20 Thread Hugh Dickins
But unlike page_referenced_one(), here we must allow for hugetlbfs pages (including non-pmd-based ones), so must still use huge_pte_offset instead of pmd_trans_huge for those. Signed-off-by: Hugh Dickins --- include/linux/pageteam.h |6 + mm/rmap.c| 158 +-

[PATCH 14/24] huge tmpfs: extend vma_adjust_trans_huge to shmem pmd

2015-02-20 Thread Hugh Dickins
urpose. And looking into the overhead of __vma_adjust_trans_huge(): it is silly for split_huge_page_pmd_mm() to be calling find_vma() followed by split_huge_page_pmd(), when it can check the pmd directly first, and usually avoid the find_vma() call. Signed-off-by: Hugh Dickins --- arch/powerpc/mm/pgtabl

[PATCH 13/24] huge tmpfs: extend get_user_pages_fast to shmem pmd

2015-02-20 Thread Hugh Dickins
Factor out one small part of the shmem pmd handling: the arch-specific get_user_pages_fast() has special code to cope with the peculiar refcounting on anonymous THP tail pages (and on hugetlbfs tail pages): which must be avoided in the straightforward shmem pmd case. Signed-off-by: Hugh Dickins

[PATCH 12/24] huge tmpfs: get_unmapped_area align and fault supply huge page

2015-02-20 Thread Hugh Dickins
but we might end up retracting this, asking for fallocate to be used explicitly for that. (hugetlbfs behaves even less standardly: its mmap extends the i_size of the object.) Signed-off-by: Hugh Dickins --- drivers/char/mem.c | 23 include/linux/mm.h |3 include/linux/shme

[PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-02-20 Thread Hugh Dickins
uge mapcount (offset by the HPAGE_PMD_NR occupancy) - it is never possible to map a sparsely occupied huge page, because that would expose stale data to the user. With this patch, the ShmemHugePages and ShmemFreeHoles lines of /proc/meminfo are shown correctly; but ShmemPmdMapped remains 0. Signed-

[PATCH 10/24] huge tmpfs: avoid team pages in a few places

2015-02-20 Thread Hugh Dickins
that PG_compound_lock can be removed altogether. Signed-off-by: Hugh Dickins --- include/linux/mm.h | 92 +-- mm/compaction.c|6 ++ mm/memcontrol.c|4 - mm/migrate.c | 12 + mm/truncate.c |2 mm/vmscan.c|2 6 files

[PATCH 09/24] huge tmpfs: try to allocate huge pages, split into a team

2015-02-20 Thread Hugh Dickins
E, PAGE_CACHE_SHIFT: enforcing that hypothetical distinction requires a different mindset, better left to a later exercise. Signed-off-by: Hugh Dickins --- include/linux/page-flags.h |6 include/linux/pageteam.h | 32 +++ mm/shmem.c | 357 +++

[PATCH 08/24] huge tmpfs: prepare huge=N mount option and /proc/sys/vm/shmem_huge

2015-02-20 Thread Hugh Dickins
cies, to force the huge option off from all mounts; and (currently) 2, to force the huge option on for all - very useful for testing. Signed-off-by: Hugh Dickins --- include/linux/shmem_fs.h | 16 ++ kernel/sysctl.c | 12 +++ mm/shmem.c | 59 +++

[PATCH 07/24] huge tmpfs: include shmem freeholes in available memory counts

2015-02-20 Thread Hugh Dickins
some notes), you may come up with a different list. I decided against max_sane_readahead(), because I suspect it's already too much; and left drivers/staging/android/lowmemorykiller.c out for now. Signed-off-by: Hugh Dickins --- fs/proc/meminfo.c |6 ++ mm/mmap.c |1 + mm

[PATCH 06/24] huge tmpfs: prepare counts in meminfo, vmstat and SysRq-m

2015-02-20 Thread Hugh Dickins
generally. I left arch/tile/mm/pgtable.c's show_mem() unchanged: tile does not HAVE_ARCH_TRANSPARENT_HUGEPAGE. Signed-off-by: Hugh Dickins --- drivers/base/node.c| 20 +++- fs/proc/meminfo.c | 11 --- include/linux/mmzone.h |5 - mm/huge_memory.c

[PATCH 05/24] tmpfs: preliminary minor tidyups

2015-02-20 Thread Hugh Dickins
to when shmem_getpage() returned the page unlocked): let's kill it and just do the necessary in do_shmem_file_read(). Signed-off-by: Hugh Dickins --- include/linux/mempolicy.h |6 ++ mm/shmem.c| 73 +--- 2 files changed, 34 insertions(+), 45

[PATCH 04/24] mm: make page migration's newpage handling more robust

2015-02-20 Thread Hugh Dickins
ate() to fit too. Finally, clean up __unmap_and_move()'s increasingly weird block "if (anon_vma) nothing; else if (PageSwapCache) nothing; else out;" while keeping its useful comment on unmapped swapcache. Signed-off-by: Hugh Dickins --- mm/balloon_compaction.c | 10 ---

[PATCH 03/24] mm: use __SetPageSwapBacked and don't ClearPageSwapBacked

2015-02-20 Thread Hugh Dickins
cache_page()'s error paths: it's not an error to free a page with PG_swapbacked set. (There's probably scope for further __SetPageFlags in other places, but SwapBacked is the one I'm interested in at the moment.) Signed-off-by: Hugh Dickins --- mm/migrate.c|6 +++--- mm/rmap.c |

[PATCH 02/24] mm: update_lru_size do the __mod_zone_page_state

2015-02-20 Thread Hugh Dickins
is not peculiar to mem_cgroups: so better use the name update_lru_size, calls mem_cgroup_update_lru_size when CONFIG_MEMCG. Signed-off-by: Hugh Dickins --- include/linux/memcontrol.h |6 -- include/linux/mm_inline.h | 24 ++-- mm/memcontrol.c|4 +++- mm/vmscan.c

[PATCH 01/24] mm: update_lru_size warn and reset bad lru_size

2015-02-20 Thread Hugh Dickins
. But the particular bug which suggested this change was mine alone, and since fixed. Signed-off-by: Hugh Dickins --- include/linux/mm_inline.h |2 +- mm/memcontrol.c | 24 2 files changed, 21 insertions(+), 5 deletions(-) --- thpfs.orig/include/linux

[PATCH 00/24] huge tmpfs: an alternative approach to THPageCache

2015-02-20 Thread Hugh Dickins
I warned last month that I have been working on "huge tmpfs": an implementation of Transparent Huge Page Cache in tmpfs, for those who are tired of the limitations of hugetlbfs. Here's a fully working patchset, against v3.19 so that you can give it a try against a stable base. I've not yet

[PATCH 05/24] tmpfs: preliminary minor tidyups

2015-02-20 Thread Hugh Dickins
to when shmem_getpage() returned the page unlocked): let's kill it and just do the necessary in do_shmem_file_read(). Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/mempolicy.h |6 ++ mm/shmem.c| 73 +--- 2 files changed, 34

[PATCH 17/24] huge tmpfs: map shmem by huge page pmd or by page team ptes

2015-02-20 Thread Hugh Dickins
switches). Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/pageteam.h | 41 ++ mm/huge_memory.c | 238 ++--- mm/memory.c | 11 + 3 files changed, 273 insertions(+), 17 deletions(-) --- thpfs.orig/include/linux/pageteam.h

[PATCH 18/24] huge tmpfs: mmap_sem is unlocked when truncation splits huge pmd

2015-02-20 Thread Hugh Dickins
-off-by: Hugh Dickins hu...@google.com --- mm/memory.c | 13 ++--- 1 file changed, 2 insertions(+), 11 deletions(-) --- thpfs.orig/mm/memory.c 2015-02-20 19:34:48.083909034 -0800 +++ thpfs/mm/memory.c 2015-02-20 19:34:53.467896724 -0800 @@ -1219,18 +1219,9 @@ static inline unsigned

[PATCH 03/24] mm: use __SetPageSwapBacked and don't ClearPageSwapBacked

2015-02-20 Thread Hugh Dickins
()'s error paths: it's not an error to free a page with PG_swapbacked set. (There's probably scope for further __SetPageFlags in other places, but SwapBacked is the one I'm interested in at the moment.) Signed-off-by: Hugh Dickins hu...@google.com --- mm/migrate.c|6 +++--- mm/rmap.c

[PATCH 12/24] huge tmpfs: get_unmapped_area align and fault supply huge page

2015-02-20 Thread Hugh Dickins
up retracting this, asking for fallocate to be used explicitly for that. (hugetlbfs behaves even less standardly: its mmap extends the i_size of the object.) Signed-off-by: Hugh Dickins hu...@google.com --- drivers/char/mem.c | 23 include/linux/mm.h |3 include/linux

[PATCH 14/24] huge tmpfs: extend vma_adjust_trans_huge to shmem pmd

2015-02-20 Thread Hugh Dickins
into the overhead of __vma_adjust_trans_huge(): it is silly for split_huge_page_pmd_mm() to be calling find_vma() followed by split_huge_page_pmd(), when it can check the pmd directly first, and usually avoid the find_vma() call. Signed-off-by: Hugh Dickins hu...@google.com --- arch/powerpc/mm

[PATCH 13/24] huge tmpfs: extend get_user_pages_fast to shmem pmd

2015-02-20 Thread Hugh Dickins
Factor out one small part of the shmem pmd handling: the arch-specific get_user_pages_fast() has special code to cope with the peculiar refcounting on anonymous THP tail pages (and on hugetlbfs tail pages): which must be avoided in the straightforward shmem pmd case. Signed-off-by: Hugh Dickins

[PATCH 21/24] huge tmpfs: fix Mlocked meminfo, tracking huge and unhuge mlocks

2015-02-20 Thread Hugh Dickins
-by: Hugh Dickins hu...@google.com --- include/linux/pageteam.h | 38 +++ mm/huge_memory.c |6 + mm/internal.h| 25 +++-- mm/mlock.c | 181 ++--- mm/rmap.c| 34 -- 5 files changed, 193 insertions

[PATCH 22/24] huge tmpfs: fix Mapped meminfo, tracking huge and unhuge mappings

2015-02-20 Thread Hugh Dickins
correctly for huge tmpfs, adjust vmscan's zone_unmapped_file_pages() to exclude NR_SHMEM_PMDMAPPED, which it clearly would not want included. Whereas minimum_image_size() in kernel/power/snapshot.c? I have not grasped the basis for that calculation, so leaving untouched. Signed-off-by: Hugh Dickins hu

[PATCH 08/24] huge tmpfs: prepare huge=N mount option and /proc/sys/vm/shmem_huge

2015-02-20 Thread Hugh Dickins
; and (currently) 2, to force the huge option on for all - very useful for testing. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/shmem_fs.h | 16 ++ kernel/sysctl.c | 12 +++ mm/shmem.c | 59 + 3 files changed

[PATCH 07/24] huge tmpfs: include shmem freeholes in available memory counts

2015-02-20 Thread Hugh Dickins
some notes), you may come up with a different list. I decided against max_sane_readahead(), because I suspect it's already too much; and left drivers/staging/android/lowmemorykiller.c out for now. Signed-off-by: Hugh Dickins hu...@google.com --- fs/proc/meminfo.c |6 ++ mm/mmap.c

[PATCH 15/24] huge tmpfs: rework page_referenced_one and try_to_unmap_one

2015-02-20 Thread Hugh Dickins
page_referenced_one(), here we must allow for hugetlbfs pages (including non-pmd-based ones), so must still use huge_pte_offset instead of pmd_trans_huge for those. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/pageteam.h |6 + mm/rmap.c| 158

[PATCH 16/24] huge tmpfs: fix problems from premature exposure of pagetable

2015-02-20 Thread Hugh Dickins
huge pmd to be chosent first. Signed-off-by: Hugh Dickins hu...@google.com --- mm/filemap.c | 10 +- mm/memory.c | 202 +++-- 2 files changed, 118 insertions(+), 94 deletions(-) --- thpfs.orig/mm/filemap.c 2015-02-08 18:54:22.0 -0800

[PATCH 00/24] huge tmpfs: an alternative approach to THPageCache

2015-02-20 Thread Hugh Dickins
I warned last month that I have been working on huge tmpfs: an implementation of Transparent Huge Page Cache in tmpfs, for those who are tired of the limitations of hugetlbfs. Here's a fully working patchset, against v3.19 so that you can give it a try against a stable base. I've not yet studied

[PATCH 01/24] mm: update_lru_size warn and reset bad lru_size

2015-02-20 Thread Hugh Dickins
. But the particular bug which suggested this change was mine alone, and since fixed. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/mm_inline.h |2 +- mm/memcontrol.c | 24 2 files changed, 21 insertions(+), 5 deletions(-) --- thpfs.orig

[PATCH 06/24] huge tmpfs: prepare counts in meminfo, vmstat and SysRq-m

2015-02-20 Thread Hugh Dickins
left arch/tile/mm/pgtable.c's show_mem() unchanged: tile does not HAVE_ARCH_TRANSPARENT_HUGEPAGE. Signed-off-by: Hugh Dickins hu...@google.com --- drivers/base/node.c| 20 +++- fs/proc/meminfo.c | 11 --- include/linux/mmzone.h |5 - mm/huge_memory.c

[PATCH 20/24] huge tmpfs: use Unevictable lru with variable hpage_nr_pages()

2015-02-20 Thread Hugh Dickins
. As it is, I'm hoping PageReferenced gives a good enough hint as to whether a page should be retained, when shmem_evictify_hugetails() brings it back from Unevictable to Inactive. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/huge_mm.h | 13 +++ include/linux/pageteam.h | 48

[PATCH 19/24] huge tmpfs: disband split huge pmds on race or memory failure

2015-02-20 Thread Hugh Dickins
team before concluding that the page is not mapped. (And still no support for soft_offline(), which will have to wait for page migration of teams.) Signed-off-by: Hugh Dickins hu...@google.com --- mm/memory-failure.c |8 +++- mm/shmem.c | 27 ++- 2 files

[PATCH 04/24] mm: make page migration's newpage handling more robust

2015-02-20 Thread Hugh Dickins
() to fit too. Finally, clean up __unmap_and_move()'s increasingly weird block if (anon_vma) nothing; else if (PageSwapCache) nothing; else out; while keeping its useful comment on unmapped swapcache. Signed-off-by: Hugh Dickins hu...@google.com --- mm/balloon_compaction.c | 10 --- mm/migrate.c

[PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-02-20 Thread Hugh Dickins
by the HPAGE_PMD_NR occupancy) - it is never possible to map a sparsely occupied huge page, because that would expose stale data to the user. With this patch, the ShmemHugePages and ShmemFreeHoles lines of /proc/meminfo are shown correctly; but ShmemPmdMapped remains 0. Signed-off-by: Hugh Dickins hu

[PATCH 23/24] kvm: plumb return of hva when resolving page fault.

2015-02-20 Thread Hugh Dickins
From: Andres Lagar-Cavilla andre...@google.com So we don't have to redo this work later. Note the hva is not racy, it is simple arithmetic based on the memslot. This will be used in the huge tmpfs commits. Signed-off-by: Andres Lagar-Cavilla andre...@google.com Signed-off-by: Hugh Dickins hu

[PATCH 02/24] mm: update_lru_size do the __mod_zone_page_state

2015-02-20 Thread Hugh Dickins
is not peculiar to mem_cgroups: so better use the name update_lru_size, calls mem_cgroup_update_lru_size when CONFIG_MEMCG. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/memcontrol.h |6 -- include/linux/mm_inline.h | 24 ++-- mm/memcontrol.c|4

[PATCH 09/24] huge tmpfs: try to allocate huge pages, split into a team

2015-02-20 Thread Hugh Dickins
that hypothetical distinction requires a different mindset, better left to a later exercise. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/page-flags.h |6 include/linux/pageteam.h | 32 +++ mm/shmem.c | 357 --- 3 files changed

[PATCH 10/24] huge tmpfs: avoid team pages in a few places

2015-02-20 Thread Hugh Dickins
that PG_compound_lock can be removed altogether. Signed-off-by: Hugh Dickins hu...@google.com --- include/linux/mm.h | 92 +-- mm/compaction.c|6 ++ mm/memcontrol.c|4 - mm/migrate.c | 12 + mm/truncate.c |2 mm/vmscan.c

[PATCH 24/24] kvm: teach kvm to map page teams as huge pages.

2015-02-20 Thread Hugh Dickins
From: Andres Lagar-Cavilla andre...@google.com Include a small treatise on the locking rules around page teams. Signed-off-by: Andres Lagar-Cavilla andre...@google.com Signed-off-by: Hugh Dickins hu...@google.com --- arch/x86/kvm/mmu.c | 155 +-- arch

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-12 Thread Hugh Dickins
On Mon, 12 Jan 2015, Kirill A. Shutemov wrote: > On Mon, Jan 12, 2015 at 11:01:46AM +0100, Jiri Slaby wrote: > > On 01/10/2015, 06:01 AM, Hugh Dickins wrote: > > > On Fri, 9 Jan 2015, Jiri Slaby wrote: > > > > > >> From: Hugh Dickins > > >>

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-12 Thread Hugh Dickins
On Mon, 12 Jan 2015, Kirill A. Shutemov wrote: On Mon, Jan 12, 2015 at 11:01:46AM +0100, Jiri Slaby wrote: On 01/10/2015, 06:01 AM, Hugh Dickins wrote: On Fri, 9 Jan 2015, Jiri Slaby wrote: From: Hugh Dickins hu...@google.com 3.12-stable review patch. If anyone has any

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-09 Thread Hugh Dickins
On Fri, 9 Jan 2015, Jiri Slaby wrote: > From: Hugh Dickins > > 3.12-stable review patch. If anyone has any objections, please let me know. > > === > > commit f72e7dcdd25229446b102e587ef2f826f76bff28 upstream. > > Trinity has reported: > >

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-09 Thread Hugh Dickins
On Fri, 9 Jan 2015, Jiri Slaby wrote: From: Hugh Dickins hu...@google.com 3.12-stable review patch. If anyone has any objections, please let me know. === commit f72e7dcdd25229446b102e587ef2f826f76bff28 upstream. Trinity has reported: BUG: unable to handle kernel

Re: [RFC] mm:change meminfo cached calculation

2015-01-06 Thread Hugh Dickins
On Tue, 6 Jan 2015, Andrew Morton wrote: > On Tue, 6 Jan 2015 17:04:33 -0800 (PST) Hugh Dickins wrote: > > On Tue, 6 Jan 2015, Andrew Morton wrote: > > > On Fri, 26 Dec 2014 19:56:49 +0800 "Wang, Yalin" > > > wrote: > > > > > > > This p

<    7   8   9   10   11   12   13   14   15   16   >