[PATCH v3 18/20] perf tools: Add support for the SPF perf event

Add support for the new speculative faults event. Signed-off-by: Laurent Dufour --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.c| 4 tools/perf/util/parse-events.l| 1 + tools/perf/util/python.c

[PATCH v3 15/20] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 5e98259c7ac0..18b39f930ce1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[PATCH v3 15/20] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 5e98259c7ac0..18b39f930ce1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2432,7 +2432,8 @@ static bool

[PATCH v3 14/20] mm: Provide speculative fault infrastructure

e can't support MPOL_INTERLEAVE case due to the processing done in mpol_misplaced()] [Don't support VMA growing up or down] [Move check on vm_sequence just before calling handle_pte_fault()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/hugetlb_inline.h | 2 +-

[PATCH v3 14/20] mm: Provide speculative fault infrastructure

the processing done in mpol_misplaced()] [Don't support VMA growing up or down] [Move check on vm_sequence just before calling handle_pte_fault()] Signed-off-by: Laurent Dufour --- include/linux/hugetlb_inline.h | 2 +- include/linux/mm.h | 5 + include/linux/pagemap.h|

[PATCH v3 11/20] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2

[PATCH v3 07/20] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v3 11/20] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2 files changed, 10 insertions(+), 5

[PATCH v3 07/20] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v3 19/20] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/include/asm/pgtable_types.h | 7 ++

[PATCH v3 17/20] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h

[PATCH v3 19/20] x86/mm: Add speculative pagefault handling

()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour --- arch/x86/include/asm/pgtable_types.h | 7 +++ arch/x86/mm/fault.c | 19 +++ 2 files changed, 26

[PATCH v3 17/20] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h index 140ae638cfd6..101e509ee39b 100644

[PATCH v3 08/20] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v3 08/20] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memor

[PATCH v3 10/20] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h

[PATCH v3 10/20] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 8a807292037f..9b4dbb98af89

[PATCH v3 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 68be41b3

[PATCH v3 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE

fail in case we find the VMA changed since we started the fault. Signed-off-by: Peter Zijlstra (Intel) [Port to 4.12 kernel] [Remove the comment about the fault_env structure which has been implemented as the vm_fault structure in the kernel] Signed-off-by: Laurent Dufour --- include/linux/mm.h

[PATCH v3 00/20] Speculative page faults

8d8ae8c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in

[PATCH v3 00/20] Speculative page faults

8d8ae8c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in

Re: [PATCH v3 00/20] Speculative page faults

On 08/09/2017 19:32, Laurent Dufour wrote: > This is a port on kernel 4.13 of the work done by Peter Zijlstra to > handle page fault without holding the mm semaphore [1]. Sorry for the noise, I got trouble sending the whole series through this email. I will try again. Cheers, Laurent.

Re: [PATCH v3 00/20] Speculative page faults

On 08/09/2017 19:32, Laurent Dufour wrote: > This is a port on kernel 4.13 of the work done by Peter Zijlstra to > handle page fault without holding the mm semaphore [1]. Sorry for the noise, I got trouble sending the whole series through this email. I will try again. Cheers, Laurent.

[PATCH v3 01/20] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter

[PATCH v3 01/20] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter Zijlstra (Intel) ---

[PATCH v3 00/20] Speculative page faults

8d8ae8c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in

[PATCH v3 00/20] Speculative page faults

8d8ae8c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in

Re: [PATCH v2 00/20] Speculative page faults

On 21/08/2017 04:26, Sergey Senozhatsky wrote: > Hello, > > On (08/18/17 00:04), Laurent Dufour wrote: >> This is a port on kernel 4.13 of the work done by Peter Zijlstra to >> handle page fault without holding the mm semaphore [1]. >> >> The idea is to try

Re: [PATCH v2 00/20] Speculative page faults

On 21/08/2017 04:26, Sergey Senozhatsky wrote: > Hello, > > On (08/18/17 00:04), Laurent Dufour wrote: >> This is a port on kernel 4.13 of the work done by Peter Zijlstra to >> handle page fault without holding the mm semaphore [1]. >> >> The idea is to try

Re: [PATCH] powerpc/mm: Fix missing mmap_sem release

On 07/09/2017 22:51, Davidlohr Bueso wrote: > On Thu, 07 Sep 2017, Laurent Dufour wrote: > >> The commit b5c8f0fd595d ("powerpc/mm: Rework mm_fault_error()") reviewed >> the way the error path is managed in __do_page_fault() but it was a bit too >> agressive

Re: [PATCH] powerpc/mm: Fix missing mmap_sem release

On 07/09/2017 22:51, Davidlohr Bueso wrote: > On Thu, 07 Sep 2017, Laurent Dufour wrote: > >> The commit b5c8f0fd595d ("powerpc/mm: Rework mm_fault_error()") reviewed >> the way the error path is managed in __do_page_fault() but it was a bit too >> agressive

[PATCH] powerpc/mm: Fix missing mmap_sem release

ap_sem as mm is set to current->mm. Fixes: b5c8f0fd595d ("powerpc/mm: Rework mm_fault_error()") Cc: Benjamin Herrenschmidt <b...@kernel.crashing.org> Signed-off-by: Laurent Dufour <laurent@free.fr> --- arch/powerpc/mm/fault.c | 3 ++- 1 file changed, 2 insertions(+),

[PATCH] powerpc/mm: Fix missing mmap_sem release

ap_sem as mm is set to current->mm. Fixes: b5c8f0fd595d ("powerpc/mm: Rework mm_fault_error()") Cc: Benjamin Herrenschmidt Signed-off-by: Laurent Dufour --- arch/powerpc/mm/fault.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/fault.c

Re: [HMM-v25 10/19] mm/memcontrol: support MEMORY_DEVICE_PRIVATE v4

On 17/08/2017 02:05, Jérôme Glisse wrote: > HMM pages (private or public device pages) are ZONE_DEVICE page and > thus need special handling when it comes to lru or refcount. This > patch make sure that memcontrol properly handle those when it face > them. Those pages are use like regular pages in

Re: [HMM-v25 10/19] mm/memcontrol: support MEMORY_DEVICE_PRIVATE v4

On 17/08/2017 02:05, Jérôme Glisse wrote: > HMM pages (private or public device pages) are ZONE_DEVICE page and > thus need special handling when it comes to lru or refcount. This > patch make sure that memcontrol properly handle those when it face > them. Those pages are use like regular pages in

Re: [PATCH] mm: Fix mem_cgroup_oom_disable() call missing

On 05/09/2017 17:46, Kirill A. Shutemov wrote: > On Tue, Sep 05, 2017 at 05:30:39PM +0200, Laurent Dufour wrote: >> Seen while reading the code, in handle_mm_fault(), in the case >> arch_vma_access_permitted() is failing the call to mem_cgroup_oom_disable() >> is not made. &g

Re: [PATCH] mm: Fix mem_cgroup_oom_disable() call missing

On 05/09/2017 17:46, Kirill A. Shutemov wrote: > On Tue, Sep 05, 2017 at 05:30:39PM +0200, Laurent Dufour wrote: >> Seen while reading the code, in handle_mm_fault(), in the case >> arch_vma_access_permitted() is failing the call to mem_cgroup_oom_disable() >> is not made. &g

[PATCH] mm: Fix mem_cgroup_oom_disable() call missing

. Fixes: bae473a423f6 ("mm: introduce fault_env") Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 56e48e4593cb..274547075486 100644 --- a/mm/m

[PATCH] mm: Fix mem_cgroup_oom_disable() call missing

. Fixes: bae473a423f6 ("mm: introduce fault_env") Signed-off-by: Laurent Dufour --- mm/memory.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 56e48e4593cb..274547075486 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3888,6 +3888

[PATCH] x86/mm: Fix fault error path using unsafe vma pointer

ave Hansen <dave.han...@linux.intel.com> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/mm/fault.c | 47 --- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c

[PATCH] x86/mm: Fix fault error path using unsafe vma pointer

e Hansen Signed-off-by: Laurent Dufour --- arch/x86/mm/fault.c | 47 --- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 2a1fa10c6a98..c18e737c5f9b 100644 --- a/arch/x86/mm/fault.c +++ b

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 30/08/2017 07:03, Anshuman Khandual wrote: > On 08/29/2017 07:15 PM, Peter Zijlstra wrote: >> On Tue, Aug 29, 2017 at 03:18:25PM +0200, Laurent Dufour wrote: >>> On 29/08/2017 14:04, Peter Zijlstra wrote: >>>> On Tue, Aug 29, 2017 at 09:59:30AM +0200, Laurent Duf

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 30/08/2017 07:03, Anshuman Khandual wrote: > On 08/29/2017 07:15 PM, Peter Zijlstra wrote: >> On Tue, Aug 29, 2017 at 03:18:25PM +0200, Laurent Dufour wrote: >>> On 29/08/2017 14:04, Peter Zijlstra wrote: >>>> On Tue, Aug 29, 2017 at 09:59:30AM +0200, Laurent Duf

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 30/08/2017 07:58, Peter Zijlstra wrote: > On Wed, Aug 30, 2017 at 10:33:50AM +0530, Anshuman Khandual wrote: >> diff --git a/mm/filemap.c b/mm/filemap.c >> index a497024..08f3042 100644 >> --- a/mm/filemap.c >> +++ b/mm/filemap.c >> @@ -1181,6 +1181,18 @@ int __lock_page_killable(struct page

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 30/08/2017 07:58, Peter Zijlstra wrote: > On Wed, Aug 30, 2017 at 10:33:50AM +0530, Anshuman Khandual wrote: >> diff --git a/mm/filemap.c b/mm/filemap.c >> index a497024..08f3042 100644 >> --- a/mm/filemap.c >> +++ b/mm/filemap.c >> @@ -1181,6 +1181,18 @@ int __lock_page_killable(struct page

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 27/08/2017 02:18, Kirill A. Shutemov wrote: > On Fri, Aug 18, 2017 at 12:05:13AM +0200, Laurent Dufour wrote: >> +/* >> + * vm_normal_page() adds some processing which should be done while >> + * hodling the mmap_sem. >> + */ >> +int handle_speculative_fault(st

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 27/08/2017 02:18, Kirill A. Shutemov wrote: > On Fri, Aug 18, 2017 at 12:05:13AM +0200, Laurent Dufour wrote: >> +/* >> + * vm_normal_page() adds some processing which should be done while >> + * hodling the mmap_sem. >> + */ >> +int handle_speculative_fault(st

Re: [PATCH v2 20/20] powerpc/mm: Add speculative page fault

On 21/08/2017 08:58, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> This patch enable the speculative page fault on the PowerPC >> architecture. >> >> This will try a speculative page fault without holding the mmap_sem, >>

Re: [PATCH v2 20/20] powerpc/mm: Add speculative page fault

On 21/08/2017 08:58, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> This patch enable the speculative page fault on the PowerPC >> architecture. >> >> This will try a speculative page fault without holding the mmap_sem, >>

Re: [PATCH v2 19/20] x86/mm: Add speculative pagefault handling

On 29/08/2017 16:50, Laurent Dufour wrote: > On 21/08/2017 09:29, Anshuman Khandual wrote: >> On 08/18/2017 03:35 AM, Laurent Dufour wrote: >>> From: Peter Zijlstra <pet...@infradead.org> >>> >>> Try a speculative fault before acquiring mmap_sem, if i

Re: [PATCH v2 19/20] x86/mm: Add speculative pagefault handling

On 29/08/2017 16:50, Laurent Dufour wrote: > On 21/08/2017 09:29, Anshuman Khandual wrote: >> On 08/18/2017 03:35 AM, Laurent Dufour wrote: >>> From: Peter Zijlstra >>> >>> Try a speculative fault before acquiring mmap_sem, if it returns with >>&

Re: [PATCH v2 19/20] x86/mm: Add speculative pagefault handling

On 21/08/2017 09:29, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> From: Peter Zijlstra <pet...@infradead.org> >> >> Try a speculative fault before acquiring mmap_sem, if it returns with >> VM_FAULT_RETRY continue wit

Re: [PATCH v2 19/20] x86/mm: Add speculative pagefault handling

On 21/08/2017 09:29, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> From: Peter Zijlstra >> >> Try a speculative fault before acquiring mmap_sem, if it returns with >> VM_FAULT_RETRY continue with the mmap_sem acquisition and do the >>

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 29/08/2017 14:04, Peter Zijlstra wrote: > On Tue, Aug 29, 2017 at 09:59:30AM +0200, Laurent Dufour wrote: >> On 27/08/2017 02:18, Kirill A. Shutemov wrote: >>>> + >>>> + if (unlikely(!vma->anon_vma)) >>>> + goto unlock; >>&g

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 29/08/2017 14:04, Peter Zijlstra wrote: > On Tue, Aug 29, 2017 at 09:59:30AM +0200, Laurent Dufour wrote: >> On 27/08/2017 02:18, Kirill A. Shutemov wrote: >>>> + >>>> + if (unlikely(!vma->anon_vma)) >>>> + goto unlock; >>&g

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 27/08/2017 02:18, Kirill A. Shutemov wrote: > On Fri, Aug 18, 2017 at 12:05:13AM +0200, Laurent Dufour wrote: >> +/* >> + * vm_normal_page() adds some processing which should be done while >> + * hodling the mmap_sem. >> + */ >> +int handle_speculative_fault(st

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 27/08/2017 02:18, Kirill A. Shutemov wrote: > On Fri, Aug 18, 2017 at 12:05:13AM +0200, Laurent Dufour wrote: >> +/* >> + * vm_normal_page() adds some processing which should be done while >> + * hodling the mmap_sem. >> + */ >> +int handle_speculative_fault(st

Re: [PATCH v2 00/20] Speculative page faults

On 21/08/2017 08:28, Anshuman Khandual wrote: > On 08/18/2017 03:34 AM, Laurent Dufour wrote: >> This is a port on kernel 4.13 of the work done by Peter Zijlstra to >> handle page fault without holding the mm semaphore [1]. >> >> The idea is to try to handle user space

Re: [PATCH v2 00/20] Speculative page faults

On 21/08/2017 08:28, Anshuman Khandual wrote: > On 08/18/2017 03:34 AM, Laurent Dufour wrote: >> This is a port on kernel 4.13 of the work done by Peter Zijlstra to >> handle page fault without holding the mm semaphore [1]. >> >> The idea is to try to handle user space

Re: [PATCH v2 18/20] perf tools: Add support for the SPF perf event

On 21/08/2017 10:48, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> Add support for the new speculative faults event. >> >> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> >> --- >> tools/include/uapi/linux/perf_ev

Re: [PATCH v2 18/20] perf tools: Add support for the SPF perf event

On 21/08/2017 10:48, Anshuman Khandual wrote: > On 08/18/2017 03:35 AM, Laurent Dufour wrote: >> Add support for the new speculative faults event. >> >> Signed-off-by: Laurent Dufour >> --- >> tools/include/uapi/linux/perf_event.h | 1 + >> tools/perf/uti

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 20/08/2017 14:11, Sergey Senozhatsky wrote: > On (08/18/17 00:05), Laurent Dufour wrote: > [..] >> +/* >> + * MPOL_INTERLEAVE implies additional check in mpol_misplaced() which >> + * are not compatible with the speculative page fault processing.

Re: [PATCH v2 14/20] mm: Provide speculative fault infrastructure

On 20/08/2017 14:11, Sergey Senozhatsky wrote: > On (08/18/17 00:05), Laurent Dufour wrote: > [..] >> +/* >> + * MPOL_INTERLEAVE implies additional check in mpol_misplaced() which >> + * are not compatible with the speculative page fault processing.

[PATCH v2 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 46b9ac5e

[PATCH v2 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE

fail in case we find the VMA changed since we started the fault. Signed-off-by: Peter Zijlstra (Intel) [Port to 4.12 kernel] [Remove the comment about the fault_env structure which has been implemented as the vm_fault structure in the kernel] Signed-off-by: Laurent Dufour --- include/linux/mm.h

[PATCH v2 04/20] mm: VMA sequence count

4.12 kernel] [Fix lock dependency between mapping->i_mmap_rwsem and vma->vm_sequence] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 1 + mm/memory.c | 2 ++ mm/mmap.c| 21 ++--- 3 files chan

[PATCH v2 04/20] mm: VMA sequence count

->i_mmap_rwsem and vma->vm_sequence] Signed-off-by: Laurent Dufour --- include/linux/mm_types.h | 1 + mm/memory.c | 2 ++ mm/mmap.c| 21 ++--- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/include/linux/mm_types.h b/include

[PATCH v2 06/20] mm: RCU free VMAs

Rename vma_is_dead() to vma_has_changed() and move its adding to the next patch] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 2 + kernel/fork.c| 1 + mm/init-mm.c | 1 + mm/internal.h| 5 +++ mm/mma

[PATCH v2 06/20] mm: RCU free VMAs

e next patch] Signed-off-by: Laurent Dufour --- include/linux/mm_types.h | 2 + kernel/fork.c| 1 + mm/init-mm.c | 1 + mm/internal.h| 5 +++ mm/mmap.c| 100 +++ 5 files changed, 83 insertions(

[PATCH v2 08/20] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git

[PATCH v2 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()

migrate_misplaced_page() is only called during the page fault handling so it's better to pass the pointer to the struct vm_fault instead of the vma. This way during the speculative page fault path the saved vma->vm_flags could be used. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.i

[PATCH v2 08/20] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour --- mm/memory.c | 4 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memor

[PATCH v2 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()

migrate_misplaced_page() is only called during the page fault handling so it's better to pass the pointer to the struct vm_fault instead of the vma. This way during the speculative page fault path the saved vma->vm_flags could be used. Signed-off-by: Laurent Dufour --- include/linux/migrat

[PATCH v2 12/20] mm: Introduce __vm_normal_page()

value as parameter. Note: The speculative path is turned on for architecture providing support for special PTE flag. So only the first block of vm_normal_page is used during the speculative path. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.

[PATCH v2 12/20] mm: Introduce __vm_normal_page()

value as parameter. Note: The speculative path is turned on for architecture providing support for special PTE flag. So only the first block of vm_normal_page is used during the speculative path. Signed-off-by: Laurent Dufour --- mm/memory.c | 25 + 1 file changed, 17

[PATCH v2 10/20] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h

[PATCH v2 10/20] mm: Introduce __lru_cache_add_active_or_unevictable

pointer. Signed-off-by: Laurent Dufour --- include/linux/swap.h | 11 +-- mm/memory.c | 8 mm/swap.c| 12 ++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index d83d28e53e62..fdea932fe10f

[PATCH v2 14/20] mm: Provide speculative fault infrastructure

e can't support MPOL_INTERLEAVE case due to the processing done in mpol_misplaced()] [Don't support VMA growing up or down] [Move check on vm_sequence just before calling handle_pte_fault()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/hugetlb_inline.h | 2 +-

[PATCH v2 14/20] mm: Provide speculative fault infrastructure

the processing done in mpol_misplaced()] [Don't support VMA growing up or down] [Move check on vm_sequence just before calling handle_pte_fault()] Signed-off-by: Laurent Dufour --- include/linux/hugetlb_inline.h | 2 +- include/linux/mm.h | 5 + include/linux/pagemap.h|

[PATCH v2 15/20] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0ba14a5797b2..8c701e4f59d3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[PATCH v2 15/20] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0ba14a5797b2..8c701e4f59d3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2300,7 +2300,8 @@ static bool

[PATCH v2 17/20] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h

[PATCH v2 17/20] perf: Add a speculative page fault sw event

Add a new software event to count succeeded speculative page faults. Signed-off-by: Laurent Dufour --- include/uapi/linux/perf_event.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h index b1c0b187acfe..3043ec0988e9 100644

[PATCH v2 19/20] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/include/asm/pgtable_types.h | 7 ++

[PATCH v2 19/20] x86/mm: Add speculative pagefault handling

()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour --- arch/x86/include/asm/pgtable_types.h | 7 +++ arch/x86/mm/fault.c | 19 +++ 2 files changed, 26

[PATCH v2 16/20] mm: Adding speculative page fault failure trace events

This patch a set of new trace events to collect the speculative page fault event failures. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/trace/events/pagefault.h | 87 mm/memory.c

[PATCH v2 16/20] mm: Adding speculative page fault failure trace events

This patch a set of new trace events to collect the speculative page fault event failures. Signed-off-by: Laurent Dufour --- include/trace/events/pagefault.h | 87 mm/memory.c | 68 --- 2 files changed

[PATCH v2 20/20] powerpc/mm: Add speculative page fault

currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/include/asm/boo

[PATCH v2 18/20] perf tools: Add support for the SPF perf event

Add support for the new speculative faults event. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.c| 4 tools/perf/util/parse-events.l

[PATCH v2 20/20] powerpc/mm: Add speculative page fault

currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour --- arch/powerpc/include/asm/book3s/64/pgtable.h | 5 + arch

[PATCH v2 18/20] perf tools: Add support for the SPF perf event

Add support for the new speculative faults event. Signed-off-by: Laurent Dufour --- tools/include/uapi/linux/perf_event.h | 1 + tools/perf/util/evsel.c | 1 + tools/perf/util/parse-events.c| 4 tools/perf/util/parse-events.l| 1 + tools/perf/util/python.c

[PATCH v2 07/20] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v2 07/20] mm: Cache some VMA fields in the vm_fault structure

hanges. This patch also set the fields in hugetlb_no_page() and __collapse_huge_page_swapin even if it is not need for the callee. Signed-off-by: Laurent Dufour --- include/linux/mm.h | 6 ++ mm/hugetlb.c | 2 ++ mm/khugepaged.c| 2 ++ mm/memory.c

[PATCH v2 13/20] mm: Introduce __page_add_new_anon_rmap()

still valid as explained above. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/rmap.h | 12 ++-- mm/memory.c | 8 mm/rmap.c| 5 ++--- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/

[PATCH v2 13/20] mm: Introduce __page_add_new_anon_rmap()

still valid as explained above. Signed-off-by: Laurent Dufour --- include/linux/rmap.h | 12 ++-- mm/memory.c | 8 mm/rmap.c| 5 ++--- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 43ef2c30cb0f..

[PATCH v2 11/20] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2

[PATCH v2 11/20] mm: Introduce __maybe_mkwrite()

() service which can be called by passing the value of the vm_flags field. There is no change functional changes expected for the other callers of maybe_mkwrite(). Signed-off-by: Laurent Dufour --- include/linux/mm.h | 9 +++-- mm/memory.c| 6 +++--- 2 files changed, 10 insertions(+), 5

[PATCH v2 05/20] mm: Protect VMA modifications using VMA sequence count

using WRITE_ONCE to prevent write to be split and intermediate values to be pushed to other CPUs. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 5 - fs/userfaultfd.c | 17 + mm/khugepaged.c| 3 +++ mm/madvise.c | 6

[PATCH v2 05/20] mm: Protect VMA modifications using VMA sequence count

using WRITE_ONCE to prevent write to be split and intermediate values to be pushed to other CPUs. Signed-off-by: Laurent Dufour --- fs/proc/task_mmu.c | 5 - fs/userfaultfd.c | 17 + mm/khugepaged.c| 3 +++ mm/madvise.c | 6 +- mm/mempolicy.c | 51

[PATCH v2 01/20] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter

<    4   5   6   7   8   9   10   11   12   13   >