[RFC PATCH v2 06/12] KVM: x86: Provide paravirtualized flush_tlb_multi()

to that of flush_tlb_others(). Suggested-by: Paolo Bonzini Cc: Paolo Bonzini Cc: k...@vger.kernel.org Signed-off-by: Nadav Amit --- arch/x86/kernel/kvm.c | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index c1c2b88ea3f1

[RFC PATCH v2 12/12] x86/mm/tlb: Reverting the removal of flush_tlb_info from stack

ore likely to be cached than a global variable. Cc: Peter Zijlstra Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: H. Peter Anvin Cc: Rik van Riel Cc: Thomas Gleixner Signed-off-by: Nadav Amit --- arch/x86/mm/tlb.c | 46 +--

[RFC PATCH v2 01/12] smp: Remove smp_call_function() and on_each_cpu() return values

The return value is fixed. Remove it and amend the callers. Cc: Richard Henderson Cc: Ivan Kokshaysky Cc: Matt Turner Cc: Tony Luck Cc: Fenghua Yu Cc: Andrew Morton Signed-off-by: Nadav Amit --- arch/alpha/kernel/smp.c | 19 +-- arch/alpha/oprofile/common.c | 6

[RFC PATCH v2 00/12] x86: Flush remote TLBs concurrently and async

ved a patch which did not improve performance - Patches 6-8: false-sharing and atomic operation optimizations - Patches 9-12: asynchronous TLB flushes Nadav Amit (12): smp: Remove smp_call_function() and on_each_cpu() return values smp: Run functions concurrently in smp_call_function_many()

Re: [RFC PATCH 4/6] x86/mm/tlb: Refactor common code into flush_tlb_on_cpus()

> On May 27, 2019, at 2:24 AM, Peter Zijlstra wrote: > > On Sat, May 25, 2019 at 01:22:01AM -0700, Nadav Amit wrote: > >> There is one functional change, which should not affect correctness: >> flush_tlb_mm_range compared loaded_mm and the mm to figure out if local >

Re: [RFC PATCH 5/6] x86/mm/tlb: Flush remote and local TLBs concurrently

> On May 27, 2019, at 2:47 AM, Peter Zijlstra wrote: > > On Sat, May 25, 2019 at 10:54:50AM +0200, Juergen Gross wrote: >> On 25/05/2019 10:22, Nadav Amit wrote: > >>> diff --git a/arch/x86/include/asm/paravirt_types.h >>> b/arch/x86/include/asm/par

Re: [RFC PATCH 3/6] smp: Run functions concurrently in smp_call_function_many()

> On May 27, 2019, at 2:15 AM, Peter Zijlstra wrote: > >> +/* >> + * Choose the most efficient way to send an IPI. Note that the >> + * number of CPUs might be zero due to concurrent changes to the >> + * provided mask or cpu_online_mask. >> +

Re: [RFC PATCH 2/6] cpumask: Purify cpumask_next()

> On May 27, 2019, at 1:30 AM, Peter Zijlstra wrote: > > On Sat, May 25, 2019 at 01:21:59AM -0700, Nadav Amit wrote: >> cpumask_next() has no side-effects. Mark it as pure. > > It would be good to have a few word on why... because apparently you > found this ma

Re: [RFC PATCH 5/6] x86/mm/tlb: Flush remote and local TLBs concurrently

> On May 25, 2019, at 1:22 AM, Nadav Amit wrote: > > To improve TLB shootdown performance, flush the remote and local TLBs > concurrently. Introduce flush_tlb_multi() that does so. The current > flush_tlb_others() interface is kept, since paravirtual interfaces need > to be ad

[RFC PATCH 4/6] x86/mm/tlb: Refactor common code into flush_tlb_on_cpus()

Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: x...@kernel.org Signed-off-by: Nadav Amit --- arch/x86/mm/tlb.c | 55 ++- 1 file changed, 26 insertions(+), 29 deletions(-) diff --git a/arch/x86/

[RFC PATCH 2/6] cpumask: Purify cpumask_next()

cpumask_next() has no side-effects. Mark it as pure. Cc: "David S. Miller" Signed-off-by: Nadav Amit --- include/linux/cpumask.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 147bdec42215..20df46705

[RFC PATCH 1/6] smp: Remove smp_call_function() and on_each_cpu() return values

The return value is fixed. Remove it and amend the callers. Cc: Richard Henderson Cc: Ivan Kokshaysky Cc: Matt Turner Cc: Tony Luck Cc: Fenghua Yu Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Andrew Morton Signed-off-by: Nadav Amit --- arch/alpha/kernel/smp.c | 19

[RFC PATCH 6/6] x86/mm/tlb: Optimize local TLB flushes

: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: x...@kernel.org Signed-off-by: Nadav Amit --- arch/x86/mm/tlb.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 0ec2bfca7581..3f3f983e224e 100644

[RFC PATCH 3/6] smp: Run functions concurrently in smp_call_function_many()

Cc: Rik van Riel Cc: Thomas Gleixner Cc: Andy Lutomirski Cc: Josh Poimboeuf Signed-off-by: Nadav Amit --- include/linux/smp.h | 27 ++--- kernel/smp.c| 133 +--- 2 files changed, 83 insertions(+), 77 deletions(-) diff --git a/include/linux

[RFC PATCH 5/6] x86/mm/tlb: Flush remote and local TLBs concurrently

org Signed-off-by: Nadav Amit --- arch/x86/hyperv/mmu.c | 2 + arch/x86/include/asm/paravirt.h | 8 +++ arch/x86/include/asm/paravirt_types.h | 6 ++ arch/x86/include/asm/tlbflush.h | 6 ++ arch/x86/kernel/kvm.c | 1 + arch/x86/kernel/paravirt.c

[RFC PATCH 0/6] x86/mm: Flush remote and local TLBs concurrently

was hurt by the optimization, to be as good as it was before these changes by introducing a fast-pass for this specific case. Nadav Amit (6): smp: Remove smp_call_function() and on_each_cpu() return values cpumask: Purify cpumask_next() smp: Run functions concurrently

Re: [RFC] x86: Speculative execution warnings

> On May 14, 2019, at 10:15 AM, Andy Lutomirski wrote: > > > > On May 14, 2019, at 10:00 AM, Nadav Amit wrote: > >>> On May 14, 2019, at 1:00 AM, Paul Turner wrote: >>> >>> From: Nadav Amit >>> Date: Fri, May 10, 2019 at 7:45 P

Re: [RFC] x86: Speculative execution warnings

> On May 14, 2019, at 1:00 AM, Paul Turner wrote: > > From: Nadav Amit > Date: Fri, May 10, 2019 at 7:45 PM > To: > Cc: Borislav Petkov, , Nadav Amit, Andy > Lutomirsky, Ingo Molnar, Peter Zijlstra, Thomas Gleixner, Jann Horn > >> It may be useful to ch

Re: [v2 PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 14, 2019, at 12:15 AM, Jan Stancek wrote: > > > - Original Message - >> On May 13, 2019 4:01 PM, Yang Shi wrote: >> >> >> On 5/13/19 9:38 AM, Will Deacon wrote: >>> On Fri, May 10, 2019 at 07:26:54AM +0800, Yang Shi wrote: diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 13, 2019, at 4:27 AM, Peter Zijlstra wrote: > > On Mon, May 13, 2019 at 09:21:01AM +0000, Nadav Amit wrote: >>> On May 13, 2019, at 2:12 AM, Peter Zijlstra wrote: > >>>> The other thing I was thinking of is trying to detect overlap through >>>

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 13, 2019, at 9:37 AM, Will Deacon wrote: > > On Mon, May 13, 2019 at 09:11:38AM +0000, Nadav Amit wrote: >>> On May 13, 2019, at 1:36 AM, Peter Zijlstra wrote: >>> >>> On Thu, May 09, 2019 at 09:21:35PM +, Nadav Amit wrote: >>> >>

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 13, 2019, at 2:12 AM, Peter Zijlstra wrote: > > On Mon, May 13, 2019 at 10:36:06AM +0200, Peter Zijlstra wrote: >> On Thu, May 09, 2019 at 09:21:35PM +0000, Nadav Amit wrote: >>> It may be possible to avoid false-positive nesting indications (when the &g

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 13, 2019, at 1:36 AM, Peter Zijlstra wrote: > > On Thu, May 09, 2019 at 09:21:35PM +0000, Nadav Amit wrote: > >>>>> And we can fix that by having tlb_finish_mmu() sync up. Never let a >>>>> concurrent tlb_finish_mmu() complete until all co

[RFC] x86: Speculative execution warnings

ata dependencies). I would appreciate your inputs. Cc: Andy Lutomirsky Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Jann Horn Signed-off-by: Nadav Amit --- arch/x86/Kconfig | 4 + arch/x86/include/asm/nospec-branch.h | 30 + arch/x86/kernel/Makefi

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

[ Restoring the recipients after mistakenly pressing reply instead of reply-all ] > On May 9, 2019, at 12:11 PM, Peter Zijlstra wrote: > > On Thu, May 09, 2019 at 06:50:00PM +0000, Nadav Amit wrote: >>> On May 9, 2019, at 11:24 AM, Peter Zijlstra wrote: >>> >&g

Re: [PATCH] mm: mmu_gather: remove __tlb_reset_range() for force flush

> On May 9, 2019, at 3:38 AM, Peter Zijlstra wrote: > > On Thu, May 09, 2019 at 09:37:26AM +0100, Will Deacon wrote: >> Hi all, [+Peter] > > Right, mm/mmu_gather.c has a MAINTAINERS entry; use it. > > Also added Nadav and Minchan who've poked at this issue before. And Mel, > because he loves

[tip:x86/mm] x86/mm: Initialize PGD cache during mm initialization

Commit-ID: caa841360134f863987f2d4f77b8dc2fbb7596f8 Gitweb: https://git.kernel.org/tip/caa841360134f863987f2d4f77b8dc2fbb7596f8 Author: Nadav Amit AuthorDate: Sat, 4 May 2019 18:11:24 -0700 Committer: Ingo Molnar CommitDate: Sun, 5 May 2019 20:32:46 +0200 x86/mm: Initialize PGD cache

Re: [tip:x86/mm 36/36] init/main.c:540:2: error: implicit declaration of function 'pgd_cache_init'

is is not arch-specific code. I don’t see the latest commit in the x86/mm tree, so I assume you can squash the following on top? -- >8 -- Subject: [PATCH] x86/mm: Fix breakage due to missing pgd_cache_init() Set pgd_cache_init() as a weak symbol. Signed-off-by: Nadav Amit --- arch/x86/

[tip:x86/mm] x86/mm: Initialize PGD cache during mm initialization

Commit-ID: ef5f22b4e5caf7e5ac12b28d4c9566c95d709ba5 Gitweb: https://git.kernel.org/tip/ef5f22b4e5caf7e5ac12b28d4c9566c95d709ba5 Author: Nadav Amit AuthorDate: Sat, 4 May 2019 18:11:24 -0700 Committer: Ingo Molnar CommitDate: Sun, 5 May 2019 12:43:13 +0200 x86/mm: Initialize PGD cache

[PATCH] x86/mm: Initialize pgd cache during mm initialization

by: kernel test robot Signed-off-by: Nadav Amit --- arch/x86/include/asm/pgtable.h | 1 + arch/x86/mm/pgtable.c | 10 ++ init/main.c| 1 + 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm

Re: [x86/alternatives] 4fc19708b1: kernel_BUG_at_arch/x86/mm/init.c

> On May 4, 2019, at 11:25 PM, kernel test robot wrote: > > FYI, we noticed the following commit (built with gcc-7): > > commit: 4fc19708b165c1c152fa1f12f6600e66184b7786 ("x86/alternatives: > Initialize temporary mm for patching") >

Re: [tip:x86/mm 14/35] kernel/trace/bpf_trace.c:179:16: error: implicit declaration of function 'nmi_uaccess_okay'; did you mean '__access_ok'?

this issue “cleanly” is hard due to the dependencies between the header files. The best I managed to do without over-complicating the solution is the following, which might not be super clean. Let me know whether to submit a separate patch (on top or instead of the current one). -- >8 -- From b

[tip:x86/mm] x86/alternatives: Add comment about module removal races

Commit-ID: 3950746d9d8ef981c1cb842384e0e86e8d1aad76 Gitweb: https://git.kernel.org/tip/3950746d9d8ef981c1cb842384e0e86e8d1aad76 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:41 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:38:01 +0200 x86/alternatives: Add

[tip:x86/mm] x86/alternatives: Remove the return value of text_poke_*()

Commit-ID: 0a203df5cf0eb709be4f190314e262b72d7e5b76 Gitweb: https://git.kernel.org/tip/0a203df5cf0eb709be4f190314e262b72d7e5b76 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:33 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:56 +0200 x86/alternatives: Remove

[tip:x86/mm] x86/jump-label: Remove support for custom text poker

Commit-ID: bb0a008d6a2c543efc11313b448d2f26f91dc4f8 Gitweb: https://git.kernel.org/tip/bb0a008d6a2c543efc11313b448d2f26f91dc4f8 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:32 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:55 +0200 x86/jump-label: Remove

[tip:x86/mm] x86/modules: Avoid breaking W^X while loading modules

Commit-ID: f2c65fb3221adc6b73b0549fc7ba892022db9797 Gitweb: https://git.kernel.org/tip/f2c65fb3221adc6b73b0549fc7ba892022db9797 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:31 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:55 +0200 x86/modules: Avoid breaking

[tip:x86/mm] x86/kprobes: Set instruction page as executable

Commit-ID: 7298e24f904224fa79eb8fd7e0fbd78950ccf2db Gitweb: https://git.kernel.org/tip/7298e24f904224fa79eb8fd7e0fbd78950ccf2db Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:30 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:54 +0200 x86/kprobes: Set

[tip:x86/mm] x86/ftrace: Set trampoline pages as executable

Commit-ID: 3c0dab44e22782359a0a706cbce72de99a22aa75 Gitweb: https://git.kernel.org/tip/3c0dab44e22782359a0a706cbce72de99a22aa75 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:29 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:53 +0200 x86/ftrace: Set trampoline

[tip:x86/mm] x86/kgdb: Avoid redundant comparison of patched code

Commit-ID: 86a22057127d1c0462a18901421bf1ff89491392 Gitweb: https://git.kernel.org/tip/86a22057127d1c0462a18901421bf1ff89491392 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:28 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:53 +0200 x86/kgdb: Avoid redundant

[tip:x86/mm] x86/alternatives: Use temporary mm for text poking

Commit-ID: b3fd8e83ada0d51b71a84297480187e2d40e5ded Gitweb: https://git.kernel.org/tip/b3fd8e83ada0d51b71a84297480187e2d40e5ded Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:27 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:52 +0200 x86/alternatives: Use

[tip:x86/mm] fork: Provide a function for copying init_mm

Commit-ID: 13585fa0668c724efab9635aaeef6ec390217415 Gitweb: https://git.kernel.org/tip/13585fa0668c724efab9635aaeef6ec390217415 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:25 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:51 +0200 fork: Provide a function

[tip:x86/mm] x86/alternatives: Initialize temporary mm for patching

Commit-ID: 4fc19708b165c1c152fa1f12f6600e66184b7786 Gitweb: https://git.kernel.org/tip/4fc19708b165c1c152fa1f12f6600e66184b7786 Author: Nadav Amit AuthorDate: Fri, 26 Apr 2019 16:22:46 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:52 +0200 x86/alternatives

[tip:x86/mm] uprobes: Initialize uprobes earlier

Commit-ID: aad42dd44db086c79ca3f470ad563d2ac4ac218d Gitweb: https://git.kernel.org/tip/aad42dd44db086c79ca3f470ad563d2ac4ac218d Author: Nadav Amit AuthorDate: Fri, 26 Apr 2019 16:22:44 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:51 +0200 uprobes: Initialize uprobes

[tip:x86/mm] bpf: Fail bpf_probe_write_user() while mm is switched

Commit-ID: c7b6f29b6257532792fc722b68fcc0e00b5a856c Gitweb: https://git.kernel.org/tip/c7b6f29b6257532792fc722b68fcc0e00b5a856c Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:43 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:48 +0200 bpf: Fail

[tip:x86/mm] x86/mm: Save debug registers when loading a temporary mm

Commit-ID: d97080ebed7811a53c931032a284166ee46b9565 Gitweb: https://git.kernel.org/tip/d97080ebed7811a53c931032a284166ee46b9565 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:24 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:50 +0200 x86/mm: Save debug

[tip:x86/mm] mm/tlb: Provide default nmi_uaccess_okay()

Commit-ID: 5932c9fd19e6e5ac84756c5c32fe5155d9a6b458 Gitweb: https://git.kernel.org/tip/5932c9fd19e6e5ac84756c5c32fe5155d9a6b458 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:42 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:48 +0200 mm/tlb: Provide default

[tip:x86/mm] x86/jump_label: Use text_poke_early() during early init

Commit-ID: 1fd8de46d01d95f875c12684a6a03559831e8b4c Gitweb: https://git.kernel.org/tip/1fd8de46d01d95f875c12684a6a03559831e8b4c Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:22 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:49 +0200 x86/jump_label: Use

[tip:x86/mm] x86/alternatives: Add text_poke_kgdb() to not assert the lock when debugging

Commit-ID: e836673c9b4966bc78e38aeda25f7022c57f0e90 Gitweb: https://git.kernel.org/tip/e836673c9b4966bc78e38aeda25f7022c57f0e90 Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 17:11:21 -0700 Committer: Ingo Molnar CommitDate: Tue, 30 Apr 2019 12:37:47 +0200 x86/alternatives: Add

[PATCH v6 08/24] x86/alternative: Use temporary mm for text poking

From: Nadav Amit text_poke() can potentially compromise security as it sets temporary PTEs in the fixmap. These PTEs might be used to rewrite the kernel code from other cores accidentally or maliciously, if an attacker gains the ability to write onto kernel memory. Moreover, since remote TLBs

[PATCH v6 11/24] x86/kprobes: Set instruction page as executable

From: Nadav Amit Set the page as executable after allocation. This patch is a preparatory patch for a following patch that makes module allocated pages non-executable. While at it, do some small cleanup of what appears to be unnecessary masking. Acked-by: Masami Hiramatsu Signed-off

[PATCH v6 06/24] fork: Provide a function for copying init_mm

From: Nadav Amit Provide a function for copying init_mm. This function will be later used for setting a temporary mm. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Nadav Amit

[PATCH v6 05/24] uprobes: Initialize uprobes earlier

From: Nadav Amit In order to have a separate address space for text poking, we need to duplicate init_mm early during start_kernel(). This, however, introduces a problem since uprobes functions are called from dup_mmap(), but uprobes is still not initialized in this early stage. Since uprobes

[PATCH v6 14/24] x86/alternative: Remove the return value of text_poke_*()

From: Nadav Amit The return value of text_poke_early() and text_poke_bp() is useless. Remove it. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text

[PATCH v6 09/24] x86/kgdb: Avoid redundant comparison of patched code

From: Nadav Amit text_poke() already ensures that the written value is the correct one and fails if that is not the case. There is no need for an additional comparison. Remove it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel

[PATCH v6 02/24] x86/jump_label: Use text_poke_early() during early init

From: Nadav Amit There is no apparent reason not to use text_poke_early() during early-init, since no patching of code that might be on the stack is done and only a single core is running. This is required for the next patches that would set a temporary mm for text poking, and this mm is only

[PATCH v6 13/24] x86/jump-label: Remove support for custom poker

From: Nadav Amit There are only two types of poking: early and breakpoint based. The use of a function pointer to perform poking complicates the code and is probably inefficient due to the use of indirect branches. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked

[PATCH v6 15/24] x86/mm/cpa: Add set_direct_map_ functions

From: Rick Edgecombe Add two new functions set_direct_map_default_noflush() and set_direct_map_invalid_noflush() for setting the direct map alias for the page to its default valid permissions and to an invalid state that cannot be cached in a TLB, respectively. These functions do not flush the

[PATCH v6 18/24] modules: Use vmalloc special flag

From: Rick Edgecombe Use new flag for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Since freeing of VM_FLUSH_RESET_PERMS memory is not supported in an interrupt by vmalloc, the freeing of init

[PATCH v6 12/24] x86/module: Avoid breaking W^X while loading modules

From: Nadav Amit When modules and BPF filters are loaded, there is a time window in which some memory is both writable and executable. An attacker that has already found another vulnerability (e.g., a dangling pointer) might be able to exploit this behavior to overwrite kernel code. Prevent

[PATCH v6 16/24] mm: Make hibernate handle unmapped pages

From: Rick Edgecombe Make hibernate handle unmapped pages on the direct map when CONFIG_ARCH_HAS_SET_ALIAS is set. These functions allow for setting pages to invalid configurations, so now hibernate should check if the pages have valid mappings and handle if they are unmapped when doing a

[PATCH v6 17/24] vmalloc: Add flag for free of special permsissions

From: Rick Edgecombe Add a new flag VM_FLUSH_RESET_PERMS, for enabling vfree operations to immediately clear executable TLB entries before freeing pages, and handle resetting permissions on the directmap. This flag is useful for any kind of memory with elevated permissions, or where there can be

[PATCH v6 23/24] mm/tlb: Provide default nmi_uaccess_okay()

From: Nadav Amit x86 has an nmi_uaccess_okay(), but other architectures do not. Arch-independent code might need to know whether access to user addresses is ok in an NMI context or in other code whose execution context is unknown. Specifically, this function is needed for bpf_probe_write_user

[PATCH v6 19/24] bpf: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Don't track if the memory is RO anymore because it is now tracked in vmalloc. Cc: Daniel

[PATCH v6 20/24] x86/ftrace: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Acked-by: Steven Rostedt (VMware) Tested-by: Steven Rostedt (VMware) Signed-off-by:

[PATCH v6 10/24] x86/ftrace: Set trampoline pages as executable

From: Nadav Amit Since alloc_module() will not set the pages as executable soon, set ftrace trampoline pages as executable after they are allocated. For the time being, do not change ftrace to use the text_poke() interface. As a result, ftrace still breaks W^X. Reviewed-by: Steven Rostedt

[PATCH v6 24/24] bpf: Fail bpf_probe_write_user() while mm is switched

From: Nadav Amit When using a temporary mm, bpf_probe_write_user() should not be able to write to user memory, since user memory addresses may be used to map kernel memory. Detect these cases and fail bpf_probe_write_user() in such cases. Cc: Daniel Borkmann Cc: Alexei Starovoitov Reported

[PATCH v6 21/24] x86/kprobes: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Masami Hiramatsu Signed-off-by: Rick Edgecombe ---

[PATCH v6 22/24] x86/alternative: Comment about module removal races

From: Nadav Amit Add a comment to clarify that users of text_poke() must ensure that no races with module removal take place. Cc: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 5 + 1 file changed, 5 insertions(+) diff --git

[PATCH v6 03/24] x86/mm: Introduce temporary mm structs

be disabled. The first use-case for temporary mm struct, which will follow, is for poking the kernel text. [ Commit message was written by Nadav Amit ] Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Andy

[PATCH v6 00/24] x86: text_poke() fixes and executable lockdowns

From: Nadav Amit * * This version fixes failed boots on 32-bit that were reported by 0day. * Patch 5 is added to initialize uprobes during fork initialization. * Patch 7 (which was 6 in the previous version) is updated - the code is * moved to common mm-init code with no further changes

[PATCH v6 01/24] Fix "x86/alternatives: Lockdep-enforce text_mutex in text_poke*()"

From: Nadav Amit text_mutex is currently expected to be held before text_poke() is called, but kgdb does not take the mutex, and instead *supposedly* ensures the lock is not taken and will not be acquired by any other core while text_poke() is running. The reason for the "supposedly&quo

[PATCH v6 07/24] x86/alternative: Initialize temporary mm for patching

From: Nadav Amit To prevent improper use of the PTEs that are used for text patching, the next patches will use a temporary mm struct. Initailize it by copying the init mm. The address that will be used for patching is taken from the lower area that is usually used for the task memory. Doing so

[PATCH v6 04/24] x86/mm: Save debug registers when loading a temporary mm

From: Nadav Amit Prevent user watchpoints from mistakenly firing while the temporary mm is being used. As the addresses of the temporary mm might overlap those of the user-process, this is necessary to prevent wrong signals or worse things from happening. Cc: Andy Lutomirski Acked-by: Peter

Re: [PATCH v5 14/23] x86/mm/cpa: Add set_direct_map_ functions

> On Apr 26, 2019, at 9:40 AM, Linus Torvalds > wrote: > > Nadav, > > I get > > dmarc=fail (p=QUARANTINE sp=NONE dis=QUARANTINE) header.from=vmware.com > > for these emails, because they lack the vmware DKIM signature. > > It clearly did go through some vmware mail servers, but

[tip:x86/mm] x86/mm/tlb: Remove 'struct flush_tlb_info' from the stack

Commit-ID: 3db6d5a5ecaf0a778d721ccf9809248350d4bfaf Gitweb: https://git.kernel.org/tip/3db6d5a5ecaf0a778d721ccf9809248350d4bfaf Author: Nadav Amit AuthorDate: Thu, 25 Apr 2019 16:01:43 -0700 Committer: Ingo Molnar CommitDate: Fri, 26 Apr 2019 12:01:45 +0200 x86/mm/tlb: Remove 'struct

Re: [PATCH v2] x86/mm/tlb: Remove flush_tlb_info from the stack

> On Apr 26, 2019, at 12:53 AM, Peter Zijlstra wrote: > > On Thu, Apr 25, 2019 at 09:20:24PM +0000, Nadav Amit wrote: > >>> I think that's a bug and PeterZ is fixing those. >> >> This would be quite surprising. > > I need to get back to that percpu seri

[PATCH v5 02/23] x86/jump_label: Use text_poke_early() during early init

some static-keys are enabled/disabled. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion

[PATCH v5 04/23] x86/mm: Save debug registers when loading a temporary mm

) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/mmu_context.h | 23 +++ 1 file changed, 23 insertions(+) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 24dc3b810970..93dff1963337 100644 --- a/arch/x86

[PATCH v5 07/23] x86/alternative: Use temporary mm for text poking

not conservative enough when mapping pages, as it always tries to map 2 pages, even when a single one is sufficient. So try to be more conservative, and do not map more than needed. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav

[PATCH v5 22/23] mm/tlb: Provide default nmi_uaccess_okay()

implementation of nmi_uaccess_okay() for architectures that do not have such a function. Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/tlbflush.h | 2 ++ include/asm-generic/tlb.h | 9 + 2 files changed, 11 insertions(+) diff --git a/arch/x86

[PATCH v5 20/23] x86/kprobes: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Masami Hiramatsu Signed-off-by: Rick Edgecombe ---

[PATCH v5 12/23] x86/jump-label: Remove support for custom poker

(Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 26 ++ 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index e7d8c636b228..e631c358f7f4 100644

[PATCH v5 11/23] x86/module: Avoid breaking W^X while loading modules

: Kees Cook Cc: Peter Zijlstra Cc: Dave Hansen Cc: Masami Hiramatsu Cc: Jessica Yu Suggested-by: Thomas Gleixner Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 28 +--- arch/x86/kernel

[PATCH v5 03/23] x86/mm: Introduce temporary mm structs

be disabled. The first use-case for temporary mm struct, which will follow, is for poking the kernel text. [ Commit message was written by Nadav Amit ] Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Andy

[PATCH v5 01/23] Fix "x86/alternatives: Lockdep-enforce text_mutex in text_poke*()"

x in text_poke*()") Suggested-by: Peter Zijlstra Acked-by: Jiri Kosina Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c| 52 ++

[PATCH v5 15/23] mm: Make hibernate handle unmapped pages

From: Rick Edgecombe Make hibernate handle unmapped pages on the direct map when CONFIG_ARCH_HAS_SET_ALIAS is set. These functions allow for setting pages to invalid configurations, so now hibernate should check if the pages have valid mappings and handle if they are unmapped when doing a

[PATCH v5 18/23] bpf: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Don't track if the memory is RO anymore because it is now tracked in vmalloc. Cc: Daniel

[PATCH v5 09/23] x86/ftrace: Set trampoline pages as executable

-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/ftrace.c | 8 1 file changed, 8 insertions(+) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index ef49517f6bb2..53ba1aa3a01f 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -730,6

[PATCH v5 16/23] vmalloc: Add flag for free of special permsissions

From: Rick Edgecombe Add a new flag VM_FLUSH_RESET_PERMS, for enabling vfree operations to immediately clear executable TLB entries before freeing pages, and handle resetting permissions on the directmap. This flag is useful for any kind of memory with elevated permissions, or where there can be

[PATCH v5 06/23] x86/alternative: Initialize temporary mm for patching

) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c| 3 +++ arch/x86/mm

[PATCH v5 17/23] modules: Use vmalloc special flag

From: Rick Edgecombe Use new flag for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Since freeing of VM_FLUSH_RESET_PERMS memory is not supported in an interrupt by vmalloc, the freeing of init

[PATCH v5 19/23] x86/ftrace: Use vmalloc special flag

From: Rick Edgecombe Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Acked-by: Steven Rostedt (VMware) Tested-by: Steven Rostedt (VMware) Signed-off-by:

[PATCH v5 21/23] x86/alternative: Comment about module removal races

Add a comment to clarify that users of text_poke() must ensure that no races with module removal take place. Cc: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel

[PATCH v5 23/23] bpf: Fail bpf_probe_write_user() while mm is switched

Suggested-by: Jann Horn Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- kernel/trace/bpf_trace.c | 8 1 file changed, 8 insertions(+) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index d64c00afceb5..94b0e37d90ef 100644 --- a/kernel/trace/bpf_trace.c +++ b

[PATCH v5 10/23] x86/kprobes: Set instruction page as executable

Set the page as executable after allocation. This patch is a preparatory patch for a following patch that makes module allocated pages non-executable. While at it, do some small cleanup of what appears to be unnecessary masking. Acked-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off

[PATCH v5 08/23] x86/kgdb: Avoid redundant comparison of patched code

text_poke() already ensures that the written value is the correct one and fails if that is not the case. There is no need for an additional comparison. Remove it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kgdb.c | 14

[PATCH v5 13/23] x86/alternative: Remove the return value of text_poke_*()

The return value of text_poke_early() and text_poke_bp() is useless. Remove it. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text-patching.h | 4

[PATCH v5 14/23] x86/mm/cpa: Add set_direct_map_ functions

From: Rick Edgecombe Add two new functions set_direct_map_default_noflush() and set_direct_map_invalid_noflush() for setting the direct map alias for the page to its default valid permissions and to an invalid state that cannot be cached in a TLB, respectively. These functions do not flush the

[PATCH v5 00/23] x86: text_poke() fixes and executable lockdowns

and ftrace - More arch generic names for set_pages functions (Ard Biesheuvel) - Fix for TLB not always flushing the directmap (Nadav Amit) Changes from "x86/alternative: text_poke() enhancements v7" to v1 - Fix build failure on CONFIG_RANDOMIZE_BASE=n (Rick) - Remove text_poke usage

[PATCH v5 05/23] fork: Provide a function for copying init_mm

Provide a function for copying init_mm. This function will be later used for setting a temporary mm. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick

<    1   2   3   4   5   6   7   8   9   10   >