[PATCH v3 2/2] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-22 Thread Mike Kravetz
. Remove the dead code and expand comments to explain reasoning. Similarly, checks for races with truncation in the page fault path can be simplified and removed. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs/hugetlbfs/in

[PATCH v3 0/2] hugetlbfs: use i_mmap_rwsem for better synchronization

2018-12-22 Thread Mike Kravetz
Comments made by Naoya were addressed. Mike Kravetz (2): hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race fs/hugetlbfs/inode.c | 61 +++- mm/hugetl

[PATCH v3 1/2] hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization

2018-12-22 Thread Mike Kravetz
of huge_pte_alloc continue to hold the semaphore until finished with the ptep. - i_mmap_rwsem is held in write mode whenever huge_pmd_unshare is called. Cc: Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz --- mm/hugetlb.c

Re: [PATCH v2 2/2] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-21 Thread Mike Kravetz
On 12/21/18 12:21 PM, Kirill A. Shutemov wrote: > On Fri, Dec 21, 2018 at 10:28:25AM -0800, Mike Kravetz wrote: >> On 12/21/18 2:28 AM, Kirill A. Shutemov wrote: >>> On Tue, Dec 18, 2018 at 02:35:57PM -0800, Mike Kravetz wrote: >>>> Instead of writing the required

Re: [PATCH v2 2/2] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-21 Thread Mike Kravetz
On 12/21/18 2:28 AM, Kirill A. Shutemov wrote: > On Tue, Dec 18, 2018 at 02:35:57PM -0800, Mike Kravetz wrote: >> Instead of writing the required complicated code for this rare >> occurrence, just eliminate the race. i_mmap_rwsem is now held in read >> mode for the d

Re: [PATCH v2 1/2] hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization

2018-12-21 Thread Mike Kravetz
On 12/21/18 2:05 AM, Kirill A. Shutemov wrote: > On Tue, Dec 18, 2018 at 02:35:56PM -0800, Mike Kravetz wrote: >> While looking at BUGs associated with invalid huge page map counts, >> it was discovered and observed that a huge pte pointer could become >> 'invalid' and point t

[PATCH v2 2/2] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-18 Thread Mike Kravetz
. Remove the dead code and expand comments to explain reasoning. Similarly, checks for races with truncation in the page fault path can be simplified and removed. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs/hugetlbfs/in

[PATCH v2 0/2] hugetlbfs: use i_mmap_rwsem for better synchronization

2018-12-18 Thread Mike Kravetz
7-1-mike.krav...@oracle.com Comments made by Naoya were addressed. Mike Kravetz (2): hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race fs/hugetlbfs/inode.c | 50 + mm/hugetlb.c

[PATCH v2 1/2] hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization

2018-12-18 Thread Mike Kravetz
of huge_pte_alloc continue to hold the semaphore until finished with the ptep. - i_mmap_rwsem is held in write mode whenever huge_pmd_unshare is called. Cc: Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz --- mm/hugetlb.c

Re: [PATCH 2/3] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-18 Thread Mike Kravetz
On 12/18/18 2:10 PM, Andrew Morton wrote: > On Mon, 17 Dec 2018 16:17:52 -0800 Mike Kravetz > wrote: > >> ... >> >>> As you suggested in a comment to the subsequent patch, it would be better to >>> combine the patches and remove the dead code

Re: [PATCH 2/3] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-17 Thread Mike Kravetz
On 12/17/18 10:42 AM, Mike Kravetz wrote: > On 12/17/18 2:25 AM, Aneesh Kumar K.V wrote: >> On 12/4/18 1:38 AM, Mike Kravetz wrote: >>> >>> Instead of writing the required complicated code for this rare >>> occurrence, just eliminate the race. i_mma

Re: [PATCH] mm: Remove __hugepage_set_anon_rmap()

2018-12-17 Thread Mike Kravetz
ill Tkhai Thanks for cleaning this up! Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH 2/3] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-17 Thread Mike Kravetz
On 12/17/18 2:25 AM, Aneesh Kumar K.V wrote: > On 12/4/18 1:38 AM, Mike Kravetz wrote: >> hugetlbfs page faults can race with truncate and hole punch operations. >> Current code in the page fault path attempts to handle this by 'backing >> out' operations if we encounter t

[PATCH 0/3] hugetlbfs: use i_mmap_rwsem for better synchronization

2018-12-03 Thread Mike Kravetz
o separate patches addressing each issue. Hopefully, this is easier to understand/review. Mike Kravetz (3): hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race hugetlbfs: remove unnecessary code after i_mmap_rwsem synchroniz

[PATCH 0/3] hugetlbfs: use i_mmap_rwsem for better synchronization

2018-12-03 Thread Mike Kravetz
o separate patches addressing each issue. Hopefully, this is easier to understand/review. Mike Kravetz (3): hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race hugetlbfs: remove unnecessary code after i_mmap_rwsem synchroniz

[PATCH 3/3] hugetlbfs: remove unnecessary code after i_mmap_rwsem synchronization

2018-12-03 Thread Mike Kravetz
After expanding i_mmap_rwsem use for better shared pmd and page fault/ truncation synchronization, remove code that is no longer necessary. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs/hugetlbfs/in

[PATCH 3/3] hugetlbfs: remove unnecessary code after i_mmap_rwsem synchronization

2018-12-03 Thread Mike Kravetz
After expanding i_mmap_rwsem use for better shared pmd and page fault/ truncation synchronization, remove code that is no longer necessary. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs/hugetlbfs/in

[PATCH 2/3] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-03 Thread Mike Kravetz
. i_mmap_rwsem is now held in read mode for the duration of page fault processing. Hold i_mmap_rwsem longer in truncation and hold punch code to cover the call to remove_inode_hugepages. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs

[PATCH 2/3] hugetlbfs: Use i_mmap_rwsem to fix page fault/truncate race

2018-12-03 Thread Mike Kravetz
. i_mmap_rwsem is now held in read mode for the duration of page fault processing. Hold i_mmap_rwsem longer in truncation and hold punch code to cover the call to remove_inode_hugepages. Cc: Fixes: ebed4bfc8da8 ("hugetlb: fix absurd HugePages_Rsvd") Signed-off-by: Mike Kravetz --- fs

[PATCH 1/3] hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization

2018-12-03 Thread Mike Kravetz
of huge_pte_alloc continue to hold the semaphore until finished with the ptep. - i_mmap_rwsem is held in write mode whenever huge_pmd_unshare is called. Cc: Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz --- mm/hugetlb.c

[PATCH 1/3] hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization

2018-12-03 Thread Mike Kravetz
of huge_pte_alloc continue to hold the semaphore until finished with the ptep. - i_mmap_rwsem is held in write mode whenever huge_pmd_unshare is called. Cc: Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz --- mm/hugetlb.c

Re: [PATCH] hugetlbfs: Call VM_BUG_ON_PAGE earlier in free_huge_page

2018-11-29 Thread Mike Kravetz
gt; >> Signed-off-by: Yongkai Wu >> Acked-by: Michal Hocko >> Acked-by: Mike Kravetz Thank you for fixing the formatting and commit message. Adding Andrew on so he can add to his tree as appropriatre. Also Cc'ing Michal. >> --- >> mm/hugetlb.c | 5 +++-- >>

Re: [PATCH] hugetlbfs: Call VM_BUG_ON_PAGE earlier in free_huge_page

2018-11-29 Thread Mike Kravetz
gt; >> Signed-off-by: Yongkai Wu >> Acked-by: Michal Hocko >> Acked-by: Mike Kravetz Thank you for fixing the formatting and commit message. Adding Andrew on so he can add to his tree as appropriatre. Also Cc'ing Michal. >> --- >> mm/hugetlb.c | 5 +++-- >>

Re: [PATCH] mm/hugetl.c: keep the page mapping info when free_huge_page() hit the VM_BUG_ON_PAGE

2018-11-13 Thread Mike Kravetz
ation to the change log (commit message) and fix the formatting of the patch. Can you tell us more about the root cause of your issue? What was the issue? How could you reproduce it? How did you solve it? -- Mike Kravetz

Re: [PATCH] mm/hugetl.c: keep the page mapping info when free_huge_page() hit the VM_BUG_ON_PAGE

2018-11-13 Thread Mike Kravetz
ation to the change log (commit message) and fix the formatting of the patch. Can you tell us more about the root cause of your issue? What was the issue? How could you reproduce it? How did you solve it? -- Mike Kravetz

Re: [PATCH] hugetlbfs: fix kernel BUG at fs/hugetlbfs/inode.c:444!

2018-11-05 Thread Mike Kravetz
On 11/5/18 1:30 PM, Andrew Morton wrote: > On Mon, 5 Nov 2018 13:23:15 -0800 Mike Kravetz > wrote: > >> This bug has been experienced several times by Oracle DB team. >> The BUG is in the routine remove_inode_hugepages() as follows: >> /* >> *

Re: [PATCH] hugetlbfs: fix kernel BUG at fs/hugetlbfs/inode.c:444!

2018-11-05 Thread Mike Kravetz
On 11/5/18 1:30 PM, Andrew Morton wrote: > On Mon, 5 Nov 2018 13:23:15 -0800 Mike Kravetz > wrote: > >> This bug has been experienced several times by Oracle DB team. >> The BUG is in the routine remove_inode_hugepages() as follows: >> /* >> *

[PATCH] hugetlbfs: fix kernel BUG at fs/hugetlbfs/inode.c:444!

2018-11-05 Thread Mike Kravetz
how we end up with an elevated map count. To solve, check the dst_pte entry for huge_pte_none. If !none, this implies PMD sharing so do not copy. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 23 +++ 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.

[PATCH] hugetlbfs: fix kernel BUG at fs/hugetlbfs/inode.c:444!

2018-11-05 Thread Mike Kravetz
how we end up with an elevated map count. To solve, check the dst_pte entry for huge_pte_none. If !none, this implies PMD sharing so do not copy. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 23 +++ 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.

Re: [PATCH RFC v2 1/1] hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate/fault sync

2018-10-30 Thread Mike Kravetz
On 10/25/18 5:42 PM, Naoya Horiguchi wrote: > Hi Mike, > > On Tue, Oct 23, 2018 at 09:50:53PM -0700, Mike Kravetz wrote: >> Now, anopther task truncates the hugetlbfs file. As part of truncation, >> it unmaps everyone who has the file mapped. If a task has a shared p

Re: [PATCH RFC v2 1/1] hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate/fault sync

2018-10-30 Thread Mike Kravetz
On 10/25/18 5:42 PM, Naoya Horiguchi wrote: > Hi Mike, > > On Tue, Oct 23, 2018 at 09:50:53PM -0700, Mike Kravetz wrote: >> Now, anopther task truncates the hugetlbfs file. As part of truncation, >> it unmaps everyone who has the file mapped. If a task has a shared p

[PATCH RFC v2 0/1] hugetlbfs: Use i_mmap_rwsem for pmd share and fault/trunc

2018-10-23 Thread Mike Kravetz
. This leads to bad things such as incorrect page map/reference counts or invaid memory references. Fix this all by modifying the usage of i_mmap_rwsem to cover fault/truncate races as well as handling of shared pmds Mike Kravetz (1): hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate

[PATCH RFC v2 0/1] hugetlbfs: Use i_mmap_rwsem for pmd share and fault/trunc

2018-10-23 Thread Mike Kravetz
. This leads to bad things such as incorrect page map/reference counts or invaid memory references. Fix this all by modifying the usage of i_mmap_rwsem to cover fault/truncate races as well as handling of shared pmds Mike Kravetz (1): hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate

[PATCH RFC v2 1/1] hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate/fault sync

2018-10-23 Thread Mike Kravetz
huge_pte_alloc, until the caller is finished with the returned ptep. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c | 21 ++ mm/hugetlb.c | 65 +--- mm/rmap.c| 10 +++ mm/userfaultfd.c | 11 ++-- 4 files

[PATCH RFC v2 1/1] hugetlbfs: use i_mmap_rwsem for pmd sharing and truncate/fault sync

2018-10-23 Thread Mike Kravetz
huge_pte_alloc, until the caller is finished with the returned ptep. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c | 21 ++ mm/hugetlb.c | 65 +--- mm/rmap.c| 10 +++ mm/userfaultfd.c | 11 ++-- 4 files

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-23 Thread Mike Kravetz
On 10/23/18 12:43 AM, Michal Hocko wrote: > On Wed 17-10-18 21:10:22, Mike Kravetz wrote: >> Some test systems were experiencing negative huge page reserve >> counts and incorrect file block counts. This was traced to >> /proc/sys/vm/drop_caches removing clean pages f

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-23 Thread Mike Kravetz
On 10/23/18 12:43 AM, Michal Hocko wrote: > On Wed 17-10-18 21:10:22, Mike Kravetz wrote: >> Some test systems were experiencing negative huge page reserve >> counts and incorrect file block counts. This was traced to >> /proc/sys/vm/drop_caches removing clean pages f

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-18 Thread Mike Kravetz
On 10/18/18 6:47 PM, Andrew Morton wrote: > On Thu, 18 Oct 2018 20:46:21 -0400 Andrea Arcangeli > wrote: > >> On Thu, Oct 18, 2018 at 04:16:40PM -0700, Mike Kravetz wrote: >>> I was not sure about this, and expected someone could come up with >>>

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-18 Thread Mike Kravetz
On 10/18/18 6:47 PM, Andrew Morton wrote: > On Thu, 18 Oct 2018 20:46:21 -0400 Andrea Arcangeli > wrote: > >> On Thu, Oct 18, 2018 at 04:16:40PM -0700, Mike Kravetz wrote: >>> I was not sure about this, and expected someone could come up with >>>

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-18 Thread Mike Kravetz
On 10/18/18 4:08 PM, Andrew Morton wrote: > On Wed, 17 Oct 2018 21:10:22 -0700 Mike Kravetz > wrote: > >> Some test systems were experiencing negative huge page reserve >> counts and incorrect file block counts. This was traced to >> /proc/sys/vm/drop_caches removing

Re: [PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-18 Thread Mike Kravetz
On 10/18/18 4:08 PM, Andrew Morton wrote: > On Wed, 17 Oct 2018 21:10:22 -0700 Mike Kravetz > wrote: > >> Some test systems were experiencing negative huge page reserve >> counts and incorrect file block counts. This was traced to >> /proc/sys/vm/drop_caches removing

[PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-17 Thread Mike Kravetz
tle sense to even try to drop hugetlbfs pagecache pages, so disable calls to these filesystems in drop_caches code. Fixes: 70c3547e36f5 ("hugetlbfs: add hugetlbfs_fallocate()") Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- fs/drop_caches.c | 7 +++ mm/hugetlb.c | 6 ++

[PATCH] hugetlbfs: dirty pages as they are added to pagecache

2018-10-17 Thread Mike Kravetz
tle sense to even try to drop hugetlbfs pagecache pages, so disable calls to these filesystems in drop_caches code. Fixes: 70c3547e36f5 ("hugetlbfs: add hugetlbfs_fallocate()") Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- fs/drop_caches.c | 7 +++ mm/hugetlb.c | 6 ++

Re: [PATCH RFC 1/1] hugetlbfs: introduce truncation/fault mutex to avoid races

2018-10-08 Thread Mike Kravetz
On 10/8/18 1:03 AM, Kirill A. Shutemov wrote: > On Sun, Oct 07, 2018 at 04:38:48PM -0700, Mike Kravetz wrote: >> The following hugetlbfs truncate/page fault race can be recreated >> with programs doing something like the following. >> >> A huegtlbfs file is mmap(MAP_SHA

Re: [PATCH RFC 1/1] hugetlbfs: introduce truncation/fault mutex to avoid races

2018-10-08 Thread Mike Kravetz
On 10/8/18 1:03 AM, Kirill A. Shutemov wrote: > On Sun, Oct 07, 2018 at 04:38:48PM -0700, Mike Kravetz wrote: >> The following hugetlbfs truncate/page fault race can be recreated >> with programs doing something like the following. >> >> A huegtlbfs file is mmap(MAP_SHA

[PATCH RFC 0/1] hugetlbfs: fix truncate/fault races

2018-10-07 Thread Mike Kravetz
wing patch describes the current race in detail and adds the mutex to prevent truncate/fault races. Mike Kravetz (1): hugetlbfs: introduce truncation/fault mutex to avoid races fs/hugetlbfs/inode.c| 24 include/linux/hugetlb.h | 1 + mm/hugetlb.c| 25 +

[PATCH RFC 1/1] hugetlbfs: introduce truncation/fault mutex to avoid races

2018-10-07 Thread Mike Kravetz
mode. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 24 include/linux/hugetlb.h | 1 + mm/hugetlb.c| 25 +++-- mm/userfaultfd.c| 8 +++- 4 files changed, 47 insertions(+), 11 deletions(-) diff --git a/fs/hugetlbfs

[PATCH RFC 0/1] hugetlbfs: fix truncate/fault races

2018-10-07 Thread Mike Kravetz
wing patch describes the current race in detail and adds the mutex to prevent truncate/fault races. Mike Kravetz (1): hugetlbfs: introduce truncation/fault mutex to avoid races fs/hugetlbfs/inode.c| 24 include/linux/hugetlb.h | 1 + mm/hugetlb.c| 25 +

[PATCH RFC 1/1] hugetlbfs: introduce truncation/fault mutex to avoid races

2018-10-07 Thread Mike Kravetz
mode. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 24 include/linux/hugetlb.h | 1 + mm/hugetlb.c| 25 +++-- mm/userfaultfd.c| 8 +++- 4 files changed, 47 insertions(+), 11 deletions(-) diff --git a/fs/hugetlbfs

Re: [PATCH V2] mm/hugetlb: Add mmap() encodings for 32MB and 512MB page sizes

2018-09-25 Thread Mike Kravetz
nitions per Mike Thanks Anshuman, Acked-by: Mike Kravetz -- Mike Kravetz > > include/uapi/asm-generic/hugetlb_encode.h | 2 ++ > include/uapi/linux/memfd.h| 2 ++ > include/uapi/linux/mman.h | 2 ++ > include/uapi/linux/shm.h

Re: [PATCH V2] mm/hugetlb: Add mmap() encodings for 32MB and 512MB page sizes

2018-09-25 Thread Mike Kravetz
nitions per Mike Thanks Anshuman, Acked-by: Mike Kravetz -- Mike Kravetz > > include/uapi/asm-generic/hugetlb_encode.h | 2 ++ > include/uapi/linux/memfd.h| 2 ++ > include/uapi/linux/mman.h | 2 ++ > include/uapi/linux/shm.h

Re: [PATCH] mm/hugetlb: Add mmap() encodings for 32MB and 512MB page sizes

2018-09-24 Thread Mike Kravetz
; include/uapi/linux/mman.h | 2 ++ > 2 files changed, 4 insertions(+) Thanks Anshuman, However, I think we should also add similar definitions in: uapi/linux/memfd.h uapi/linux/shm.h -- Mike Kravetz

Re: [PATCH] mm/hugetlb: Add mmap() encodings for 32MB and 512MB page sizes

2018-09-24 Thread Mike Kravetz
; include/uapi/linux/mman.h | 2 ++ > 2 files changed, 4 insertions(+) Thanks Anshuman, However, I think we should also add similar definitions in: uapi/linux/memfd.h uapi/linux/shm.h -- Mike Kravetz

Re: Plumbers 2018 - Performance and Scalability Microconference

2018-09-06 Thread Mike Kravetz
>> Can we come up with a 2M base page VM or something? We have possible >> memory sizes of a couple TB now. That should give us a million or so 2M >> pages to work with. > > That sounds a good idea. Don't know whether someone has tried this. IIRC, Hugh Dickins and some others at Google tried going down this path. There was a brief discussion at LSF/MM. It is something I too would like to explore in my spare time. -- Mike Kravetz

Re: Plumbers 2018 - Performance and Scalability Microconference

2018-09-06 Thread Mike Kravetz
>> Can we come up with a 2M base page VM or something? We have possible >> memory sizes of a couple TB now. That should give us a million or so 2M >> pages to work with. > > That sounds a good idea. Don't know whether someone has tried this. IIRC, Hugh Dickins and some others at Google tried going down this path. There was a brief discussion at LSF/MM. It is something I too would like to explore in my spare time. -- Mike Kravetz

Re: [RFC PATCH] mm/hugetlb: make hugetlb_lock irq safe

2018-09-05 Thread Mike Kravetz
On 09/05/2018 04:07 PM, Matthew Wilcox wrote: > On Wed, Sep 05, 2018 at 03:00:08PM -0700, Andrew Morton wrote: >> On Wed, 5 Sep 2018 14:35:11 -0700 Mike Kravetz >> wrote: >> >>>>so perhaps we could put some >&

Re: [RFC PATCH] mm/hugetlb: make hugetlb_lock irq safe

2018-09-05 Thread Mike Kravetz
On 09/05/2018 04:07 PM, Matthew Wilcox wrote: > On Wed, Sep 05, 2018 at 03:00:08PM -0700, Andrew Morton wrote: >> On Wed, 5 Sep 2018 14:35:11 -0700 Mike Kravetz >> wrote: >> >>>>so perhaps we could put some >&

Re: [RFC PATCH] mm/hugetlb: make hugetlb_lock irq safe

2018-09-05 Thread Mike Kravetz
d, I doubt this was taken into account. I would be afraid of someone adding put_page from hardirq context. -- Mike Kravetz > And attention will need to be paid to -stable backporting. How long > has mm_iommu_free() existed, and been doing this?

Re: [RFC PATCH] mm/hugetlb: make hugetlb_lock irq safe

2018-09-05 Thread Mike Kravetz
d, I doubt this was taken into account. I would be afraid of someone adding put_page from hardirq context. -- Mike Kravetz > And attention will need to be paid to -stable backporting. How long > has mm_iommu_free() existed, and been doing this?

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-09-04 Thread Mike Kravetz
ed/want a separate patch. We could just add the notifiers to the shared pmd patch. Back porting the shared pmd patch will also require some fixup. Either would work. I'll admit I do not know what stable maintainers would prefer. -- Mike Kravetz

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-09-04 Thread Mike Kravetz
ed/want a separate patch. We could just add the notifiers to the shared pmd patch. Back porting the shared pmd patch will also require some fixup. Either would work. I'll admit I do not know what stable maintainers would prefer. -- Mike Kravetz

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-30 Thread Mike Kravetz
se wrote: >>>>> On Wed, Aug 29, 2018 at 08:39:06PM +0200, Michal Hocko wrote: >>>>>> On Wed 29-08-18 14:14:25, Jerome Glisse wrote: >>>>>>> On Wed, Aug 29, 2018 at 10:24:44AM -0700, Mike Kravetz wrote: >>>>>> [...] >

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-30 Thread Mike Kravetz
se wrote: >>>>> On Wed, Aug 29, 2018 at 08:39:06PM +0200, Michal Hocko wrote: >>>>>> On Wed 29-08-18 14:14:25, Jerome Glisse wrote: >>>>>>> On Wed, Aug 29, 2018 at 10:24:44AM -0700, Mike Kravetz wrote: >>>>>> [...] >

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-29 Thread Mike Kravetz
On 08/29/2018 02:11 PM, Jerome Glisse wrote: > On Wed, Aug 29, 2018 at 08:39:06PM +0200, Michal Hocko wrote: >> On Wed 29-08-18 14:14:25, Jerome Glisse wrote: >>> On Wed, Aug 29, 2018 at 10:24:44AM -0700, Mike Kravetz wrote: >> [...] >>>> What would be the best

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-29 Thread Mike Kravetz
On 08/29/2018 02:11 PM, Jerome Glisse wrote: > On Wed, Aug 29, 2018 at 08:39:06PM +0200, Michal Hocko wrote: >> On Wed 29-08-18 14:14:25, Jerome Glisse wrote: >>> On Wed, Aug 29, 2018 at 10:24:44AM -0700, Mike Kravetz wrote: >> [...] >>>> What would be the best

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-29 Thread Mike Kravetz
On 08/27/2018 06:46 AM, Jerome Glisse wrote: > On Mon, Aug 27, 2018 at 09:46:45AM +0200, Michal Hocko wrote: >> On Fri 24-08-18 11:08:24, Mike Kravetz wrote: >>> Here is an updated patch which does as you suggest above. >> [...] >>> @@ -1409,6 +1419,32 @@ static

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-29 Thread Mike Kravetz
On 08/27/2018 06:46 AM, Jerome Glisse wrote: > On Mon, Aug 27, 2018 at 09:46:45AM +0200, Michal Hocko wrote: >> On Fri 24-08-18 11:08:24, Mike Kravetz wrote: >>> Here is an updated patch which does as you suggest above. >> [...] >>> @@ -1409,6 +1419,32 @@ static

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-27 Thread Mike Kravetz
On 08/27/2018 12:46 AM, Michal Hocko wrote: > On Fri 24-08-18 11:08:24, Mike Kravetz wrote: >> On 08/24/2018 01:41 AM, Michal Hocko wrote: >>> On Thu 23-08-18 13:59:16, Mike Kravetz wrote: >>> >>> Acked-by: Michal Hocko >>> >>> One nit be

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-27 Thread Mike Kravetz
On 08/27/2018 12:46 AM, Michal Hocko wrote: > On Fri 24-08-18 11:08:24, Mike Kravetz wrote: >> On 08/24/2018 01:41 AM, Michal Hocko wrote: >>> On Thu 23-08-18 13:59:16, Mike Kravetz wrote: >>> >>> Acked-by: Michal Hocko >>> >>> One nit be

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-24 Thread Mike Kravetz
On 08/24/2018 01:41 AM, Michal Hocko wrote: > On Thu 23-08-18 13:59:16, Mike Kravetz wrote: > > Acked-by: Michal Hocko > > One nit below. > > [...] >> diff --git a/mm/hugetlb.c b/mm/hugetlb.c >> index 3103099f64fd..a73c5728e961 100644 >> --- a/mm/hugetl

Re: [PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-24 Thread Mike Kravetz
On 08/24/2018 01:41 AM, Michal Hocko wrote: > On Thu 23-08-18 13:59:16, Mike Kravetz wrote: > > Acked-by: Michal Hocko > > One nit below. > > [...] >> diff --git a/mm/hugetlb.c b/mm/hugetlb.c >> index 3103099f64fd..a73c5728e961 100644 >> --- a/mm/hugetl

[PATCH v6 2/2] hugetlb: take PMD sharing into account when flushing tlb/caches

2018-08-23 Thread Mike Kravetz
that this range is flushed if huge_pmd_unshare succeeds and unmaps a PUD_SUZE area. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 53 +++- 1 file changed, 44 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index

[PATCH v6 0/2] huge_pmd_unshare migration and flushing

2018-08-23 Thread Mike Kravetz
as suggested by Kirill. v3-v5: Address build errors if !CONFIG_HUGETLB_PAGE and !CONFIG_ARCH_WANT_HUGE_PMD_SHARE Mike Kravetz (2): mm: migration: fix migration of huge PMD shared pages hugetlb: take PMD sharing into account when flushing tlb/caches include/linux/hugetlb.h | 14

[PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
;) Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 14 ++ mm/hugetlb.c| 40 +-- mm/rmap.c | 42 ++--- 3 files changed, 91 insertions(+), 5 deletions(-)

[PATCH v6 2/2] hugetlb: take PMD sharing into account when flushing tlb/caches

2018-08-23 Thread Mike Kravetz
that this range is flushed if huge_pmd_unshare succeeds and unmaps a PUD_SUZE area. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 53 +++- 1 file changed, 44 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index

[PATCH v6 0/2] huge_pmd_unshare migration and flushing

2018-08-23 Thread Mike Kravetz
as suggested by Kirill. v3-v5: Address build errors if !CONFIG_HUGETLB_PAGE and !CONFIG_ARCH_WANT_HUGE_PMD_SHARE Mike Kravetz (2): mm: migration: fix migration of huge PMD shared pages hugetlb: take PMD sharing into account when flushing tlb/caches include/linux/hugetlb.h | 14

[PATCH v6 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
;) Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 14 ++ mm/hugetlb.c| 40 +-- mm/rmap.c | 42 ++--- 3 files changed, 91 insertions(+), 5 deletions(-)

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 10:01 AM, Mike Kravetz wrote: > On 08/23/2018 05:48 AM, Michal Hocko wrote: >> On Tue 21-08-18 18:10:42, Mike Kravetz wrote: >> [...] >> >> OK, after burning myself when trying to be clever here it seems like >> your proposed solutio

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 10:01 AM, Mike Kravetz wrote: > On 08/23/2018 05:48 AM, Michal Hocko wrote: >> On Tue 21-08-18 18:10:42, Mike Kravetz wrote: >> [...] >> >> OK, after burning myself when trying to be clever here it seems like >> your proposed solutio

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 05:48 AM, Michal Hocko wrote: > On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > [...] > > OK, after burning myself when trying to be clever here it seems like > your proposed solution is indeed simpler. > >> +bool huge_pmd_sharing_possible(st

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 05:48 AM, Michal Hocko wrote: > On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > [...] > > OK, after burning myself when trying to be clever here it seems like > your proposed solution is indeed simpler. > >> +bool huge_pmd_sharing_possible(st

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 01:21 AM, Kirill A. Shutemov wrote: > On Thu, Aug 23, 2018 at 09:30:35AM +0200, Michal Hocko wrote: >> On Wed 22-08-18 09:48:16, Mike Kravetz wrote: >>> On 08/22/2018 05:28 AM, Michal Hocko wrote: >>>> On Tue 21-08-18 18:10:42, Mike Kravetz wrote: &g

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-23 Thread Mike Kravetz
On 08/23/2018 01:21 AM, Kirill A. Shutemov wrote: > On Thu, Aug 23, 2018 at 09:30:35AM +0200, Michal Hocko wrote: >> On Wed 22-08-18 09:48:16, Mike Kravetz wrote: >>> On 08/22/2018 05:28 AM, Michal Hocko wrote: >>>> On Tue 21-08-18 18:10:42, Mike Kravetz wrote: &g

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-22 Thread Mike Kravetz
On 08/22/2018 02:05 PM, Kirill A. Shutemov wrote: > On Tue, Aug 21, 2018 at 06:10:42PM -0700, Mike Kravetz wrote: >> diff --git a/mm/hugetlb.c b/mm/hugetlb.c >> index 3103099f64fd..f085019a4724 100644 >> --- a/mm/hugetlb.c >> +++ b/mm/hugetlb.c >> @@ -4555,6 +45

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-22 Thread Mike Kravetz
On 08/22/2018 02:05 PM, Kirill A. Shutemov wrote: > On Tue, Aug 21, 2018 at 06:10:42PM -0700, Mike Kravetz wrote: >> diff --git a/mm/hugetlb.c b/mm/hugetlb.c >> index 3103099f64fd..f085019a4724 100644 >> --- a/mm/hugetlb.c >> +++ b/mm/hugetlb.c >> @@ -4555,6 +45

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-22 Thread Mike Kravetz
On 08/22/2018 05:28 AM, Michal Hocko wrote: > On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > [...] >> diff --git a/mm/rmap.c b/mm/rmap.c >> index eb477809a5c0..8cf853a4b093 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1362,11 +1362,21 @@ static boo

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-22 Thread Mike Kravetz
On 08/22/2018 05:28 AM, Michal Hocko wrote: > On Tue 21-08-18 18:10:42, Mike Kravetz wrote: > [...] >> diff --git a/mm/rmap.c b/mm/rmap.c >> index eb477809a5c0..8cf853a4b093 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1362,11 +1362,21 @@ static boo

Re: [PATCH v2 0/2] mm: soft-offline: fix race against page allocation

2018-08-21 Thread Mike Kravetz
a Tested-by :). Just wanted to point out that it was pretty easy to hit this issue. It was easier than the issue I am working. And, the issue I am trying to address was seen in a real customer environment. So, I would not be surprised to see this issue in real customer environments as well. If you (or others) think we should go forward with these patches, I can spend some time doing a review. Already did a 'quick look' some time back. -- Mike Kravetz

Re: [PATCH v2 0/2] mm: soft-offline: fix race against page allocation

2018-08-21 Thread Mike Kravetz
a Tested-by :). Just wanted to point out that it was pretty easy to hit this issue. It was easier than the issue I am working. And, the issue I am trying to address was seen in a real customer environment. So, I would not be surprised to see this issue in real customer environments as well. If you (or others) think we should go forward with these patches, I can spend some time doing a review. Already did a 'quick look' some time back. -- Mike Kravetz

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
te to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/Mike-Kravetz/huge_pmd_unshare-migration-and-flushing/20180822-050255 > config: sparc64-allyesconfig (attached as .config) > compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
te to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/Mike-Kravetz/huge_pmd_unshare-migration-and-flushing/20180822-050255 > config: sparc64-allyesconfig (attached as .config) > compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
te to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/Mike-Kravetz/huge_pmd_unshare-migration-and-flushing/20180822-050255 > config: i386-tinyconfig (attached as .config) > compiler: gcc-7 (Debian 7.3.0-16) 7.3.0 > reproduce: > #

Re: [PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
te to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/Mike-Kravetz/huge_pmd_unshare-migration-and-flushing/20180822-050255 > config: i386-tinyconfig (attached as .config) > compiler: gcc-7 (Debian 7.3.0-16) 7.3.0 > reproduce: > #

[PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
;) Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 14 ++ mm/hugetlb.c| 40 +++ mm/rmap.c | 42 ++--- 3 files changed, 93 insertions(+), 3 deletions(-)

[PATCH v3 0/2] huge_pmd_unshare migration and flushing

2018-08-21 Thread Mike Kravetz
advantage of the new routine huge_pmd_sharing_possible() to adjust flushing ranges in the cases where huge PMD sharing is possible. There is no copy to stable for this patch as it has not been reported as an issue and discovered only via code inspection. Mike Kravetz (2): mm: migration: fix migration

[PATCH v3 1/2] mm: migration: fix migration of huge PMD shared pages

2018-08-21 Thread Mike Kravetz
;) Cc: sta...@vger.kernel.org Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 14 ++ mm/hugetlb.c| 40 +++ mm/rmap.c | 42 ++--- 3 files changed, 93 insertions(+), 3 deletions(-)

[PATCH v3 0/2] huge_pmd_unshare migration and flushing

2018-08-21 Thread Mike Kravetz
advantage of the new routine huge_pmd_sharing_possible() to adjust flushing ranges in the cases where huge PMD sharing is possible. There is no copy to stable for this patch as it has not been reported as an issue and discovered only via code inspection. Mike Kravetz (2): mm: migration: fix migration

[PATCH v3 2/2] hugetlb: take PMD sharing into account when flushing tlb/caches

2018-08-21 Thread Mike Kravetz
that this range is flushed if huge_pmd_unshare succeeds and unmaps a PUD_SUZE area. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 53 +++- 1 file changed, 44 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fd155dc52117

[PATCH v3 2/2] hugetlb: take PMD sharing into account when flushing tlb/caches

2018-08-21 Thread Mike Kravetz
that this range is flushed if huge_pmd_unshare succeeds and unmaps a PUD_SUZE area. Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 53 +++- 1 file changed, 44 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fd155dc52117

Re: [PATCH] mm: migration: fix migration of huge PMD shared pages

2018-08-14 Thread Mike Kravetz
On 08/14/2018 01:48 AM, Kirill A. Shutemov wrote: > On Mon, Aug 13, 2018 at 11:21:41PM +0000, Mike Kravetz wrote: >> On 08/13/2018 03:58 AM, Kirill A. Shutemov wrote: >>> On Sun, Aug 12, 2018 at 08:41:08PM -0700, Mike Kravetz wrote: >>>> I am not %100 sure on the req

<    3   4   5   6   7   8   9   10   11   12   >