[PATCH v2 01/20] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter Zijlstra (Intel) ---

[PATCH v2 03/20] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[PATCH v2 03/20] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour --- mm/memory.c

[PATCH v2 00/20] Speculative page faults

tps://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in the vm_fault structure mm: Protect SPF handler against anon_vma changes mm/migrate: Pass vm_

[PATCH v2 00/20] Speculative page faults

tps://lwn.net/Articles/725607/ Laurent Dufour (14): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm: Cache some VMA fields in the vm_fault structure mm: Protect SPF handler against anon_vma changes mm/migrate: Pass vm_

[PATCH] mm: Remove useless vma parameter to offset_il_node

While reading the code I found that offset_il_node() has a vm_area_struct pointer parameter which is unused. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/mempolicy.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempo

[PATCH] mm: Remove useless vma parameter to offset_il_node

While reading the code I found that offset_il_node() has a vm_area_struct pointer parameter which is unused. Signed-off-by: Laurent Dufour --- mm/mempolicy.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index d911fa5cb2a7

Re: possible circular locking dependency mmap_sem/cpu_hotplug_lock.rw_sem

On 16/08/2017 17:36, Artem Savkov wrote: > On Wed, Aug 16, 2017 at 03:39:14PM +0200, Laurent Dufour wrote: >> On 15/08/2017 21:01, Paul E. McKenney wrote: >>> On Mon, Aug 07, 2017 at 04:09:47PM +0200, Artem Savkov wrote: >>>> Hello, >>>> >>>&

Re: possible circular locking dependency mmap_sem/cpu_hotplug_lock.rw_sem

On 16/08/2017 17:36, Artem Savkov wrote: > On Wed, Aug 16, 2017 at 03:39:14PM +0200, Laurent Dufour wrote: >> On 15/08/2017 21:01, Paul E. McKenney wrote: >>> On Mon, Aug 07, 2017 at 04:09:47PM +0200, Artem Savkov wrote: >>>> Hello, >>>> >>>&

Re: possible circular locking dependency mmap_sem/cpu_hotplug_lock.rw_sem

On 15/08/2017 21:01, Paul E. McKenney wrote: > On Mon, Aug 07, 2017 at 04:09:47PM +0200, Artem Savkov wrote: >> Hello, >> >> After commit fc8dffd "cpu/hotplug: Convert hotplug locking to percpu rwsem" >> the following lockdep splat started showing up on some systems while running >> ltp's

Re: possible circular locking dependency mmap_sem/cpu_hotplug_lock.rw_sem

On 15/08/2017 21:01, Paul E. McKenney wrote: > On Mon, Aug 07, 2017 at 04:09:47PM +0200, Artem Savkov wrote: >> Hello, >> >> After commit fc8dffd "cpu/hotplug: Convert hotplug locking to percpu rwsem" >> the following lockdep splat started showing up on some systems while running >> ltp's

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 10/08/2017 15:43, Kirill A. Shutemov wrote: > On Thu, Aug 10, 2017 at 10:27:50AM +0200, Laurent Dufour wrote: >> On 10/08/2017 02:58, Kirill A. Shutemov wrote: >>> On Wed, Aug 09, 2017 at 12:43:33PM +0200, Laurent Dufour wrote: >>>> On 09/08/2017 12:12, Kirill A

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 10/08/2017 15:43, Kirill A. Shutemov wrote: > On Thu, Aug 10, 2017 at 10:27:50AM +0200, Laurent Dufour wrote: >> On 10/08/2017 02:58, Kirill A. Shutemov wrote: >>> On Wed, Aug 09, 2017 at 12:43:33PM +0200, Laurent Dufour wrote: >>>> On 09/08/2017 12:12, Kirill A

Re: [rcu:mmap_sem.2017.08.09a 6/16] mm/internal.h:59:26: error: 'struct vm_fault' has no member named 'sequence'

On 09/08/2017 23:47, kbuild test robot wrote: > tree: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git > mmap_sem.2017.08.09a > head: 39782fbd9252a3ace9b49a55f4dd2a41a6ced31f > commit: 1b801585f6b03276db9722dc725a50ca11439467 [6/16] mm: RCU free VMAs > config:

Re: [rcu:mmap_sem.2017.08.09a 6/16] mm/internal.h:59:26: error: 'struct vm_fault' has no member named 'sequence'

On 09/08/2017 23:47, kbuild test robot wrote: > tree: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git > mmap_sem.2017.08.09a > head: 39782fbd9252a3ace9b49a55f4dd2a41a6ced31f > commit: 1b801585f6b03276db9722dc725a50ca11439467 [6/16] mm: RCU free VMAs > config:

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 10/08/2017 02:58, Kirill A. Shutemov wrote: > On Wed, Aug 09, 2017 at 12:43:33PM +0200, Laurent Dufour wrote: >> On 09/08/2017 12:12, Kirill A. Shutemov wrote: >>> On Tue, Aug 08, 2017 at 04:35:38PM +0200, Laurent Dufour wrote: >>>> The VMA sequence count has

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 10/08/2017 02:58, Kirill A. Shutemov wrote: > On Wed, Aug 09, 2017 at 12:43:33PM +0200, Laurent Dufour wrote: >> On 09/08/2017 12:12, Kirill A. Shutemov wrote: >>> On Tue, Aug 08, 2017 at 04:35:38PM +0200, Laurent Dufour wrote: >>>> The VMA sequence count has

Re: [PATCH 13/16] perf: Add a speculative page fault sw events

On 09/08/2017 15:18, Michael Ellerman wrote: > Laurent Dufour <lduf...@linux.vnet.ibm.com> writes: > >> Add new software events to count succeeded and failed speculative page >> faults. >> >> Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> >

Re: [PATCH 13/16] perf: Add a speculative page fault sw events

On 09/08/2017 15:18, Michael Ellerman wrote: > Laurent Dufour writes: > >> Add new software events to count succeeded and failed speculative page >> faults. >> >> Signed-off-by: Laurent Dufour >> --- >> include/uapi/linux/perf_event.h | 2 ++ >>

Re: [PATCH 02/16] mm: Prepare for FAULT_FLAG_SPECULATIVE

On 09/08/2017 12:08, Kirill A. Shutemov wrote: > On Tue, Aug 08, 2017 at 04:35:35PM +0200, Laurent Dufour wrote: >> @@ -2295,7 +2302,11 @@ static int wp_page_copy(struct vm_fault *vmf) >> /* >> * Re-check the pte - we dropped the lock >> */ >> -

Re: [PATCH 02/16] mm: Prepare for FAULT_FLAG_SPECULATIVE

On 09/08/2017 12:08, Kirill A. Shutemov wrote: > On Tue, Aug 08, 2017 at 04:35:35PM +0200, Laurent Dufour wrote: >> @@ -2295,7 +2302,11 @@ static int wp_page_copy(struct vm_fault *vmf) >> /* >> * Re-check the pte - we dropped the lock >> */ >> -

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 09/08/2017 12:12, Kirill A. Shutemov wrote: > On Tue, Aug 08, 2017 at 04:35:38PM +0200, Laurent Dufour wrote: >> The VMA sequence count has been introduced to allow fast detection of >> VMA modification when running a page fault handler without holding >> the mmap_sem. >

Re: [PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

On 09/08/2017 12:12, Kirill A. Shutemov wrote: > On Tue, Aug 08, 2017 at 04:35:38PM +0200, Laurent Dufour wrote: >> The VMA sequence count has been introduced to allow fast detection of >> VMA modification when running a page fault handler without holding >> the mmap_sem. >

[PATCH 01/16] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter

[PATCH 01/16] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the reliance on the pte pointer. Signed-off-by: Peter Zijlstra (Intel) ---

[PATCH 04/16] mm: VMA sequence count

4.12 kernel] [Fix lock dependency between mapping->i_mmap_rwsem and vma->vm_sequence] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 1 + mm/memory.c | 2 ++ mm/mmap.c| 21 ++--- 3 files chan

[PATCH 04/16] mm: VMA sequence count

->i_mmap_rwsem and vma->vm_sequence] Signed-off-by: Laurent Dufour --- include/linux/mm_types.h | 1 + mm/memory.c | 2 ++ mm/mmap.c| 21 ++--- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/include/linux/mm_types.h b/include

[PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

() - vma_replace_policy() - change_prot_numa() - mlock(), munlock() - mprotect() - mmap_region() - collapse_huge_page() Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 2 ++ mm/khugepaged.c| 3 +++ mm/mad

[PATCH 05/16] mm: Protect VMA modifications using VMA sequence count

() - vma_replace_policy() - change_prot_numa() - mlock(), munlock() - mprotect() - mmap_region() - collapse_huge_page() Signed-off-by: Laurent Dufour --- fs/proc/task_mmu.c | 2 ++ mm/khugepaged.c| 3 +++ mm/madvise.c | 4 mm/mempolicy.c

[PATCH 03/16] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[PATCH 03/16] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour --- mm/memory.c

[PATCH 06/16] mm: RCU free VMAs

Rename vma_is_dead() to vma_has_changed()] [Pass vm_fault structure pointer instead of 2 arguments to vmf_has_changed() ] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 2 + kernel/fork.c| 1 + mm/init-mm.c | 1 + mm/int

[PATCH 06/16] mm: RCU free VMAs

ucture pointer instead of 2 arguments to vmf_has_changed() ] Signed-off-by: Laurent Dufour --- include/linux/mm_types.h | 2 + kernel/fork.c| 1 + mm/init-mm.c | 1 + mm/internal.h| 19 + mm/mmap.c

[PATCH 08/16] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 14236d98a5c5..519c28507a93 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[PATCH 08/16] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour --- mm/memory.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 14236d98a5c5..519c28507a93 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2259,7 +2259,8 @@ static bool

[PATCH 07/16] mm: Provide speculative fault infrastructure

_do_fault() calls vma->vm_ops->fault() which may want to release mmap_sem] [Only vm_fault pointer argument for vma_has_changed()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 3 + mm/memory.c| 183 +

[PATCH 07/16] mm: Provide speculative fault infrastructure

want to release mmap_sem] [Only vm_fault pointer argument for vma_has_changed()] Signed-off-by: Laurent Dufour --- include/linux/mm.h | 3 + mm/memory.c| 183 - 2 files changed, 183 insertions(+), 3 deletions(-) diff --git a/include/linu

[PATCH 09/16] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/mm/fault.c | 14 ++

[PATCH 09/16] x86/mm: Add speculative pagefault handling

()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour --- arch/x86/mm/fault.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c

[PATCH 11/16] mm: Introduce __page_add_new_anon_rmap()

one which do the check. Currently __page_add_new_anon_rmap() is only called during the speculative page fault path. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/rmap.h | 12 ++-- mm/rmap.c| 5 ++--- 2 files changed, 12 insertions(+), 5

[PATCH 11/16] mm: Introduce __page_add_new_anon_rmap()

one which do the check. Currently __page_add_new_anon_rmap() is only called during the speculative page fault path. Signed-off-by: Laurent Dufour --- include/linux/rmap.h | 12 ++-- mm/rmap.c| 5 ++--- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/include/li

[PATCH 10/16] powerpc/mm: Add speculative page fault

This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Signed-off-by: Laurent Dufour <l

[PATCH 13/16] perf: Add a speculative page fault sw events

Add new software events to count succeeded and failed speculative page faults. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/uapi/linux/perf_event.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_e

[PATCH 10/16] powerpc/mm: Add speculative page fault

This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Signed-off-by: Laurent Dufour --- arch

[PATCH 13/16] perf: Add a speculative page fault sw events

Add new software events to count succeeded and failed speculative page faults. Signed-off-by: Laurent Dufour --- include/uapi/linux/perf_event.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h index b1c0b187acfe

[PATCH 14/16] x86/mm: Add support for SPF events

Add support for the new speculative page faults software events. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/mm/fault.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 46fb9c2a832d..17985f

[PATCH 14/16] x86/mm: Add support for SPF events

Add support for the new speculative page faults software events. Signed-off-by: Laurent Dufour --- arch/x86/mm/fault.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 46fb9c2a832d..17985f11b9da 100644 --- a/arch/x86/mm

[PATCH 15/16] powerpc/mm: Add support for SPF events

Add support for the new speculative page faults software events. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/powerpc/mm/fault.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index c6cd40

[PATCH 15/16] powerpc/mm: Add support for SPF events

Add support for the new speculative page faults software events. Signed-off-by: Laurent Dufour --- arch/powerpc/mm/fault.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index c6cd40901dd0..112c4bc9da70 100644

[PATCH 12/16] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 13 ++--- 1 file changed, 10 insertions(

[PATCH 12/16] mm: Protect SPF handler against anon_vma changes

sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour --- mm/memory.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/mm/m

[PATCH 16/16] perf tools: Add support for SPF events

Add support for the new speculative faults events. Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- tools/include/uapi/linux/perf_event.h | 2 ++ tools/perf/util/evsel.c | 2 ++ tools/perf/util/parse-events.c| 8 tools/perf/util/parse-ev

[PATCH 16/16] perf tools: Add support for SPF events

Add support for the new speculative faults events. Signed-off-by: Laurent Dufour --- tools/include/uapi/linux/perf_event.h | 2 ++ tools/perf/util/evsel.c | 2 ++ tools/perf/util/parse-events.c| 8 tools/perf/util/parse-events.l| 2 ++ tools/perf/util

[PATCH 00/16] Speculative page faults

c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lkml.org/lkml/2017/7/24/829 [6] https://lwn.net/Articles/725607/ Laurent Dufour (10): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm:

[PATCH 00/16] Speculative page faults

c03e [3] http://ebizzy.sourceforge.net/ [4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ [5] https://lkml.org/lkml/2017/7/24/829 [6] https://lwn.net/Articles/725607/ Laurent Dufour (10): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: Protect VMA modifications using VMA sequence count mm:

[PATCH 02/16] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 46b9ac5e

[PATCH 02/16] mm: Prepare for FAULT_FLAG_SPECULATIVE

fail in case we find the VMA changed since we started the fault. Signed-off-by: Peter Zijlstra (Intel) [Port to 4.12 kernel] [Remove the comment about the fault_env structure which has been implemented as the vm_fault structure in the kernel] Signed-off-by: Laurent Dufour --- include/linux/mm.h

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 15:15, Peter Zijlstra wrote: > On Tue, Aug 08, 2017 at 02:20:23PM +0200, Laurent Dufour wrote: >> This is an option, but the previous one was signed by Peter, and I'd prefer >> to keep his unchanged and add this new one to fix that. >> Again this is to ease the

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 15:15, Peter Zijlstra wrote: > On Tue, Aug 08, 2017 at 02:20:23PM +0200, Laurent Dufour wrote: >> This is an option, but the previous one was signed by Peter, and I'd prefer >> to keep his unchanged and add this new one to fix that. >> Again this is to ease the

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 14:49, Jan Kara wrote: > On Tue 08-08-17 14:20:23, Laurent Dufour wrote: >> On 08/08/2017 13:17, Anshuman Khandual wrote: >>> On 06/16/2017 11:22 PM, Laurent Dufour wrote: >>>> kworker/32:1/819 is trying to acquire lock: >>>> (>vm_sequenc

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 14:49, Jan Kara wrote: > On Tue 08-08-17 14:20:23, Laurent Dufour wrote: >> On 08/08/2017 13:17, Anshuman Khandual wrote: >>> On 06/16/2017 11:22 PM, Laurent Dufour wrote: >>>> kworker/32:1/819 is trying to acquire lock: >>>> (>vm_sequenc

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 13:17, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> kworker/32:1/819 is trying to acquire lock: >> (>vm_sequence){+.+...}, at: [] >> zap_page_range_single+0xd0/0x1a0 >> >> but task is alre

Re: [RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

On 08/08/2017 13:17, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> kworker/32:1/819 is trying to acquire lock: >> (>vm_sequence){+.+...}, at: [] >> zap_page_range_single+0xd0/0x1a0 >> >> but task is alre

Re: [RFC v5 03/11] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

On 08/08/2017 12:35, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> When handling page fault without holding the mmap_sem the fetch of the >> pte lock pointer and the locking will have to be done while ensuring >> that the VMA is not touched in o

Re: [RFC v5 03/11] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

On 08/08/2017 12:35, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> When handling page fault without holding the mmap_sem the fetch of the >> pte lock pointer and the locking will have to be done while ensuring >> that the VMA is not touched in o

Re: [RFC v5 01/11] mm: Dont assume page-table invariance during faults

On 08/08/2017 11:45, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> From: Peter Zijlstra <pet...@infradead.org> >> >> One of the side effects of speculating on faults (without holding >> mmap_sem) is that we can race with free_pgta

Re: [RFC v5 01/11] mm: Dont assume page-table invariance during faults

On 08/08/2017 11:45, Anshuman Khandual wrote: > On 06/16/2017 11:22 PM, Laurent Dufour wrote: >> From: Peter Zijlstra >> >> One of the side effects of speculating on faults (without holding >> mmap_sem) is that we can race with free_pgtables() and therefore we >&

Re: [RFC v5 01/11] mm: Dont assume page-table invariance during faults

On 07/07/2017 09:07, Balbir Singh wrote: > On Fri, 2017-06-16 at 19:52 +0200, Laurent Dufour wrote: >> From: Peter Zijlstra <pet...@infradead.org> >> >> One of the side effects of speculating on faults (without holding >> mmap_sem) is that we can race with

Re: [RFC v5 01/11] mm: Dont assume page-table invariance during faults

On 07/07/2017 09:07, Balbir Singh wrote: > On Fri, 2017-06-16 at 19:52 +0200, Laurent Dufour wrote: >> From: Peter Zijlstra >> >> One of the side effects of speculating on faults (without holding >> mmap_sem) is that we can race with free_pgtables() and therefore w

Re: [RFC v5 09/11] mm: Try spin lock in speculative path

On 06/07/2017 16:48, Peter Zijlstra wrote: > On Thu, Jul 06, 2017 at 03:46:59PM +0200, Laurent Dufour wrote: >> On 05/07/2017 20:50, Peter Zijlstra wrote: >>> On Fri, Jun 16, 2017 at 07:52:33PM +0200, Laurent Dufour wrote: >>>> @@ -2294,8 +2295,19 @@ static bool pte

Re: [RFC v5 09/11] mm: Try spin lock in speculative path

On 06/07/2017 16:48, Peter Zijlstra wrote: > On Thu, Jul 06, 2017 at 03:46:59PM +0200, Laurent Dufour wrote: >> On 05/07/2017 20:50, Peter Zijlstra wrote: >>> On Fri, Jun 16, 2017 at 07:52:33PM +0200, Laurent Dufour wrote: >>>> @@ -2294,8 +2295,19 @@ static bool pte

Re: [RFC v5 09/11] mm: Try spin lock in speculative path

On 05/07/2017 20:50, Peter Zijlstra wrote: > On Fri, Jun 16, 2017 at 07:52:33PM +0200, Laurent Dufour wrote: >> @@ -2294,8 +2295,19 @@ static bool pte_map_lock(struct vm_fault *vmf) >> if (vma_has_changed(vmf->vma, vmf->sequence)) >>

Re: [RFC v5 09/11] mm: Try spin lock in speculative path

On 05/07/2017 20:50, Peter Zijlstra wrote: > On Fri, Jun 16, 2017 at 07:52:33PM +0200, Laurent Dufour wrote: >> @@ -2294,8 +2295,19 @@ static bool pte_map_lock(struct vm_fault *vmf) >> if (vma_has_changed(vmf->vma, vmf->sequence)) >>

Re: [RFC v5 00/11] Speculative page faults

On 16/06/2017 19:52, Laurent Dufour wrote: > This is a port on kernel 4.12 of the work done by Peter Zijlstra to > handle page fault without holding the mm semaphore [1]. > > The idea is to try to handle user space page faults without holding the > mmap_sem. This should allow bet

Re: [RFC v5 00/11] Speculative page faults

On 16/06/2017 19:52, Laurent Dufour wrote: > This is a port on kernel 4.12 of the work done by Peter Zijlstra to > handle page fault without holding the mm semaphore [1]. > > The idea is to try to handle user space page faults without holding the > mmap_sem. This should allow bet

[RFC v5 01/11] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the relyance on the pte pointer. Signed-off-by: Peter

[RFC v5 01/11] mm: Dont assume page-table invariance during faults

From: Peter Zijlstra One of the side effects of speculating on faults (without holding mmap_sem) is that we can race with free_pgtables() and therefore we cannot assume the page-tables will stick around. Remove the relyance on the pte pointer. Signed-off-by: Peter Zijlstra (Intel) ---

[RFC v5 04/11] mm: VMA sequence count

4.12 kernel] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 1 + mm/memory.c | 2 ++ mm/mmap.c| 13 + 3 files changed, 16 insertions(+) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h

[RFC v5 04/11] mm: VMA sequence count

are still valid. The flip side is that we cannot distinguish between a vma_adjust() and the unmap_page_range() -- where with the former we could have re-checked the vma bounds against the address. Signed-off-by: Peter Zijlstra (Intel) [port to 4.12 kernel] Signed-off-by: Laurent Dufour

[RFC v5 02/11] mm: Prepare for FAULT_FLAG_SPECULATIVE

] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm.h | 1 + mm/memory.c| 55 ++ 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index b892e95d

[RFC v5 02/11] mm: Prepare for FAULT_FLAG_SPECULATIVE

fail in case we find the VMA changed since we started the fault. Signed-off-by: Peter Zijlstra (Intel) [Port to 4.12 kernel] [Remove the comment about the fault_env structure which has been implemented as the vm_fault structure in the kernel] Signed-off-by: Laurent Dufour --- include/linux/mm.h

[RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

o: CPU0CPU1 lock(>i_mmap_rwsem); lock(>vm_sequence/1); lock(>i_mmap_rwsem); lock(>vm_sequence); *** DEADLOCK *** To fix that we must grab the vm_sequence lock after any mapping one in _

[RFC v5 06/11] mm: Protect VMA modifications using VMA sequence count

() - vma_replace_policy() - change_prot_numa() - mlock(), munlock() - mprotect() - mmap_region() Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- fs/proc/task_mmu.c | 2 ++ mm/madvise.c | 4 mm/mempolicy.c | 10 +- mm/m

[RFC v5 05/11] mm: fix lock dependency against mapping->i_mmap_rwsem

o: CPU0CPU1 lock(>i_mmap_rwsem); lock(>vm_sequence/1); lock(>i_mmap_rwsem); lock(>vm_sequence); *** DEADLOCK *** To fix that we must grab the vm_sequence lock after any mapping one in __vma_adjus

[RFC v5 06/11] mm: Protect VMA modifications using VMA sequence count

() - vma_replace_policy() - change_prot_numa() - mlock(), munlock() - mprotect() - mmap_region() Signed-off-by: Laurent Dufour --- fs/proc/task_mmu.c | 2 ++ mm/madvise.c | 4 mm/mempolicy.c | 10 +- mm/mlock.c | 9 ++--- mm

[RFC v5 07/11] mm: RCU free VMAs

Rename vma_is_dead() to vma_has_changed()] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- include/linux/mm_types.h | 2 + kernel/fork.c| 1 + mm/init-mm.c | 1 + mm/internal.h| 20 ++ mm/mma

[RFC v5 07/11] mm: RCU free VMAs

parate the tree removal and clearing the node. Provide find_vma_srcu() which wraps the required magic. Signed-off-by: Peter Zijlstra (Intel) [Remove the warnings in description about the SRCU global lock which has been removed now] [Rename vma_is_dead() to vma_has_changed()] Signed-off-by: Laurent

[RFC v5 11/11] powerpc/mm: Add speculative page fault

This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Signed-off-by: Laurent Dufour <l

[RFC v5 11/11] powerpc/mm: Add speculative page fault

This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Signed-off-by: Laurent Dufour --- arch

[RFC v5 08/11] mm: Provide speculative fault infrastructure

_ALLOW_RETRY in handle_speculative_fault()] [Don't set vmf->pte and vmf->ptl if pte_map_lock() failed] [Remove warning comment about waiting for !seq&1 since we don't want to wait] [Remove warning about no huge page support, mention it explictly] Signed-off-by: Laurent Dufour <lduf...@linux.v

[RFC v5 08/11] mm: Provide speculative fault infrastructure

pte and vmf->ptl if pte_map_lock() failed] [Remove warning comment about waiting for !seq&1 since we don't want to wait] [Remove warning about no huge page support, mention it explictly] Signed-off-by: Laurent Dufour --- include/linux/mm.h | 3 + mm/memory.c

[RFC v5 09/11] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- mm/memory.c | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0645cb21155f..a9ea3cc2d255 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -

[RFC v5 09/11] mm: Try spin lock in speculative path

async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour --- mm/memory.c | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0645cb21155f..a9ea3cc2d255 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2258,7 +2258,8 @@ static bool

[RFC v5 03/11] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour <l

[RFC v5 03/11] mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE

When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour --- mm/memory.c

[RFC v5 10/11] x86/mm: Add speculative pagefault handling

_ALLOW_RETRY is now done in handle_speculative_fault()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour <lduf...@linux.vnet.ibm.com> --- arch/x86/mm/fault.c | 14 ++

[RFC v5 10/11] x86/mm: Add speculative pagefault handling

()] [Retry with usual fault path in the case VM_ERROR is returned by handle_speculative_fault(). This allows signal to be delivered] Signed-off-by: Laurent Dufour --- arch/x86/mm/fault.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c

[RFC v5 00/11] Speculative page faults

[4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ Laurent Dufour (5): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: fix lock dependency against mapping->i_mmap_rwsem mm: Protect VMA modifications using VMA sequence count mm: Try spin lock in speculative path powerpc/m

[RFC v5 00/11] Speculative page faults

[4] http://ck.kolivas.org/apps/kernbench/kernbench-0.50/ Laurent Dufour (5): mm: Introduce pte_spinlock for FAULT_FLAG_SPECULATIVE mm: fix lock dependency against mapping->i_mmap_rwsem mm: Protect VMA modifications using VMA sequence count mm: Try spin lock in speculative path powerpc/m

Re: [RFC v4 00/20] Speculative page faults

On 12/06/2017 12:20, Jan Kara wrote: > Hello, > > On Fri 09-06-17 16:20:49, Laurent Dufour wrote: >> This is a port on kernel 4.12 of the work done by Peter Zijlstra to >> handle page fault without holding the mm semaphore. >> >> http://linux-kernel.2935.n7.na

<    5   6   7   8   9   10   11   12   13   14   >