When a syscall returns to userspace with pti_disable set, it means the current mm is configured to disable page table isolation (PTI). In this case, returns from kernel to user will not switch the CR3, leaving it to the kernel one which already maps both user and kernel pages. This avoids a TLB flush, and saves another one on next entry.
Thanks to these changes, haproxy running under KVM went back from 12700 conn/s (without PCID) or 19700 (with PCID) to 23100 once loaded after calling prctl(), indicating that PTI has no measurable impact on this workload. Signed-off-by: Willy Tarreau <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Borislav Petkov <[email protected]> Cc: Brian Gerst <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Cc: Josh Poimboeuf <[email protected]> Cc: "H. Peter Anvin" <[email protected]> Cc: Greg Kroah-Hartman <[email protected]> Cc: Kees Cook <[email protected]> v2: - use pti_disable instead of task flag --- arch/x86/entry/calling.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 2c0d3b5..5361a10 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -229,6 +229,11 @@ .macro SWITCH_TO_USER_CR3_NOSTACK scratch_reg:req scratch_reg2:req ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI + + /* The "pti_disable" mm attribute is mirrored into this per-cpu var */ + cmpb $0, PER_CPU_VAR(pti_disable) + jne .Lend_\@ + mov %cr3, \scratch_reg ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID -- 1.7.12.1
