To prepare the introduction of PROT_SHSTK and be consistent with other
architectures, move arch_vm_get_page_prot() and arch_calc_vm_prot_bits() to
arch/x86/include/asm/mman.h.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Kirill A. Shutemov
---
arch/x86/include/asm/mman.h | 30
.
ENDBR is a noop when IBT is unsupported or disabled. Most ENDBR
instructions are inserted automatically by the compiler, but branch
targets written in assembly must have ENDBR added manually.
Add ENDBR to __vdso_sgx_enter_enclave() branch targets.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Kees Cook
---
arch/x86/entry/vdso/vdso32/system_call.S | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/entry/vdso/vdso32/system_call.S
b/arch/x86/entry/vdso/vdso32/system_call.S
index de1fff7188aa..7793dc221726 100644
--- a/arch/x86/entry/v
for the proper one. Introduce ENDBR macro,
which equals the compiler macro when enabled, otherwise nothing.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Borislav Petkov
Cc: Dave Hansen
Cc: Jarkko Sakkinen
Cc: Peter Zijlstra
---
v25:
- Change from using the compiler's cet.h back to just
off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Kees Cook
---
v24:
- Replace CONFIG_X86_CET with CONFIG_X86_IBT to reflect splitting of shadow
stack and ibt.
arch/x86/entry/vdso/Makefile | 4
1 file changed, 4 insertions(+)
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vds
An ELF file's .note.gnu.property indicates features the file supports.
The property is parsed at loading time and passed to arch_setup_elf_
property(). Update it for Indirect Branch Tracking.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for splitting shadow stack and ibt.
arch
From: "H.J. Lu"
Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch
Tracking.
Signed-off-by: H.J. Lu
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
v24:
- Update for function name changes from splitting shadow stack and ibt.
arch/x86/kernel/cet_p
machine is described in Intel SDM Vol. 1, Sec. 18.3.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Move the addition of sc_ext.wait_endbr from an earlier shadow stack
patch to here.
- Change X86_FEATURE_CET to X86_FEATURE_SHSTK.
- Change wrmsrl() to wrmsrl_safe() and handle error.
v24
would be using this as a bypass to shadow stack protection. However, the
attacker would have to get to the syscall first.
[1] https://lore.kernel.org/lkml/20200828121624.108243-1-hjl.to...@gmail.com/
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Update
("arm64: mte: Add PROT_MTE support to mmap() and
mprotect()"),
Signed-off-by: Yu-cheng Yu
Cc: Catalin Marinas
Cc: Kees Cook
Cc: Kirill A. Shutemov
Cc: Vincenzo Frascino
Cc: Will Deacon
---
arch/arm64/include/asm/mman.h | 4 ++--
arch/sparc/include/asm/mman.h | 4 ++--
include/li
protection.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Make CONFIG_X86_IBT depend on CONFIG_X86_SHADOW_STACK.
arch/x86/Kconfig | 19 +++
arch/x86/include/asm/disabled-features.h | 8 +++-
2 files changed, 26 insertions(+), 1 deletion
do_arch_prctl_common()'s parameter 'cpuid_enabled' to
'arg2', as it is now also passed to prctl_cet().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
v25:
- Change CONFIG_X86_CET to CONFIG_X86_SHADOW_STACK.
- Change X86_FEATURE_CET to X86_FEATURE_SHSTK.
v24:
- Update #ifdef placement
Introduce user-mode Indirect Branch Tracking (IBT) support. Add routines
for the setup/disable of IBT.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Move IBT routines to a separate ibt.c, update related areas accordingly.
arch/x86/include/asm/cet.h | 9 ++
arch/x86/kernel/Makefile
arch_prctl functions for Indirect Branch Tracking
x86/vdso: Insert endbr32/endbr64 to vDSO
x86/vdso/32: Add ENDBR to __kernel_vsyscall entry point
Yu-cheng Yu (6):
x86/cet/ibt: Add Kconfig option for Indirect Branch Tracking
x86/cet/ibt: Add user-mode Indirect Branch Tracking support
x
of this function is Shadow
Stack.
ARM64 is the other arch that has ARCH_USE_GNU_PROPERTY and arch_parse_elf_
property(). Add arch_setup_elf_property() for it.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Mark Brown
Cc: Catalin Marinas
Cc: Dave Martin
---
v24:
- Change cet_setup_shstk
(),
- restore_extra_state_to_xregs().
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Update commit log/comments for the sc_ext struct.
- Use restorer address already calculated.
- Change CONFIG_X86_CET to CONFIG_X86_SHADOW_STACK.
- Change X86_FEATURE_CET to X86_FEATURE_SHSTK.
- Eliminate writing to MSR_IA32_U_CET
setup and verify routines. Also introduce WRUSS, which is
a kernel-mode instruction but writes directly to user shadow stack. It is
used to construct user signal stack as described above.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Update inline assembly syntax, use %[].
- Change token
). A compat-mode thread shadow stack
size is further reduced to 1/4. This allows more threads to run in a 32-
bit address space.
Signed-off-by: Yu-cheng Yu
---
arch/x86/include/asm/cet.h | 5 +++
arch/x86/include/asm/mmu_context.h | 3 ++
arch/x86/kernel/process.c | 15
Thus, re-introduce vm_flags to do_mmap().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Peter Collingbourne
Reviewed-by: Kees Cook
Cc: Andrew Morton
Cc: Oleg Nesterov
Cc: linux...@kvack.org
---
v24:
- Change VM_SHSTK to VM_SHADOW_STACK.
- Update commit log.
fs/aio.c | 2 +-
include/linux
Introduce basic shadow stack enabling/disabling/allocation routines.
A task's shadow stack is allocated from memory with VM_SHADOW_STACK flag
and has a fixed size of min(RLIMIT_STACK, 4GB).
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Change CONFIG_X86_CET to CONFIG_X86_SHADOW_STACK
to preserve it.
Exclude shadow stack from preserve_write test, and apply the same change to
change_huge_pmd().
Signed-off-by: Yu-cheng Yu
Cc: Kirill A. Shutemov
---
v25:
- Move is_shadow_stack_mapping() to a separate line.
v24:
- Change arch_shadow_stack_mapping() to is_shadow_stack_mapping
, in the
can_follow_write_pte() check, it belongs to the writable page case and
should be excluded from the read-only page pte_dirty() check. Apply
the same changes to can_follow_write_pmd().
While at it, also split the long line into smaller ones.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
Account shadow stack pages to stack memory.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v25:
- Remove #ifdef CONFIG_ARCH_HAS_SHADOW_STACK for is_shadow_stack_mapping().
v24:
- Change arch_shadow_stack_mapping() to is_shadow_stack_mapping().
- Change VM_SHSTK
, putting a gap page on both ends of a shadow stack prevents INCSSP,
CALL, and RET from going beyond.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
Cc: Kees Cook
---
v25:
- Move SHADOW_STACK_GUARD_GAP to arch/x86/mm/mmap.c.
v24:
- Instead changing vm_*_gap(), create x86-specific
().
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v25:
- Apply same changes to do_huge_pmd_numa_page() as to do_numa_page().
mm/huge_memory.c | 2 +-
mm/memory.c | 5 ++---
mm/migrate.c | 3 +--
mm/mprotect.c| 2 +-
4 files changed, 5 insertions(+), 7 deletions
and the shadow stack
page is writable again.
Introduce an x86 version of maybe_mkwrite(), which sets proper PTE bits
according to VM flags.
Apply the same changes to maybe_pmd_mkwrite().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
Cc: Kees Cook
---
v24:
- Instead of doing
clearing
_PAGE_DIRTY (vs. _PAGE_RW) is used to trigger the fault, shadow stack read
fault and shadow stack write fault are not differentiated and both are
handled as a write access.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Reviewed-by: Kirill A. Shutemov
---
v24:
- Change VM_SHSTK
A shadow stack PTE must be read-only and have _PAGE_DIRTY set. However,
read-only and Dirty PTEs also exist for copy-on-write (COW) pages. These
two cases are handled differently for page faults. Introduce
VM_SHADOW_STACK to track shadow stack VMAs.
Signed-off-by: Yu-cheng Yu
Reviewed
non-
atomically, a transient shadow stack PTE can be created as a result.
Thus, prevent that with cmpxchg.
Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many
insights to the issue. Jann Horn provided the cmpxchg solution.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
, and
pte_*() are updated.
Pte_modify() changes a PTE to 'newprot', but it doesn't use the pte_*().
Introduce fixup_dirty_pte(), which sets a dirty PTE, based on _PAGE_RW,
to either _PAGE_DIRTY or _PAGE_COW.
Apply the same changes to pmd_modify().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill
.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
---
v24:
- Replace CONFIG_X86_CET with CONFIG_X86_SHADOW_STACK.
arch/x86/include/asm/pgtable.h | 195 ---
arch/x86/include/asm/pgtable_types.h | 42 +-
2 files changed, 216 insertions(+), 21
After the introduction of _PAGE_COW, a modified page's PTE can have either
_PAGE_DIRTY or _PAGE_COW. Change _PAGE_DIRTY to _PAGE_DIRTY_BITS.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Reviewed-by: Kirill A. Shutemov
Cc: David Airlie
Cc: Joonas Lahtinen
Cc: Jani Nikula
Cc: Daniel
To prepare the introduction of _PAGE_COW, move pmd_write() and
pud_write() up in the file, so that they can be used by other
helpers below.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
---
arch/x86/include/asm/pgtable.h | 24
1 file changed, 12 insertions
. This results in ambiguity between shadow stack and
kernel read-only pages. To resolve this, removed Dirty from kernel read-
only pages.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
Cc: "H. Peter Anvin"
Cc: Kees Cook
Cc: Thomas Gleixner
Cc: Dave Hansen
Cc: Christoph Hellwig
, arrives at a non-ENDBR opcode.
The control-protection fault handler works in a similar way as the general
protection fault handler. It provides the si_code SEGV_CPERR to the signal
handler.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Michael Kerrisk
---
v25:
- Change CONFIG_X86_CET
Introduce CPU setup and boot option parsing for CET features.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Remove software-defined X86_FEATURE_CET.
v24:
- Update #ifdef placement to reflect Kconfig changes of splitting shadow stack
and ibt.
arch/x86/include/uapi/asm/processor-flags.h
20236-1-yu-cheng...@intel.com/
[5] The kernel ptrace patch is tested with an Intel-internal updated GDB.
I am holding off the kernel ptrace patch to re-test it with my earlier
patch for fixing regset holes.
Yu-cheng Yu (30):
Documentation/x86: Add CET description
x86/cet/shstk: A
-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Update xsave_cpuid_features[]. Now CET XSAVES features depend on
X86_FEATURE_SHSTK (vs. the software-defined X86_FEATURE_CET).
arch/x86/include/asm/fpu/types.h | 23 +--
arch/x86/include/asm/fpu/xstate.h | 6 --
arch/x86/include
Add CPU feature flags for Control-flow Enforcement Technology (CET).
CPUID.(EAX=7,ECX=0):ECX[bit 7] Shadow stack
CPUID.(EAX=7,ECX=0):EDX[bit 20] Indirect Branch Tracking
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Make X86_FEATURE_IBT depend on X86_FEATURE_SHSTK.
v24:
- Update
-Shadow Stack applications continue to work, but without
protection.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v25:
- Remove X86_CET and use X86_SHADOW_STACK directly.
v24:
- Update for the splitting X86_CET to X86_SHADOW_STACK and X86_IBT.
arch/x86/Kconfig | 22
Explain no_user_shstk/no_user_ibt kernel parameters, and introduce a new
document on Control-flow Enforcement Technology (CET).
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for Kconfig changes from X86_CET to X86_SHADOW_STACK, X86_IBT.
- Update for the change of VM_SHSTK
for the proper one. Introduce ENDBR macro,
which equals the compiler macro when enabled, otherwise nothing.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Borislav Petkov
Cc: Dave Hansen
Cc: Jarkko Sakkinen
Cc: Peter Zijlstra
---
arch/x86/entry/vdso/Makefile | 1 +
arch/x86/include/asm
off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Kees Cook
---
v24:
- Replace CONFIG_X86_CET with CONFIG_X86_IBT to reflect splitting of shadow
stack and ibt.
arch/x86/entry/vdso/Makefile | 4
1 file changed, 4 insertions(+)
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vds
.
ENDBR is a noop when IBT is unsupported or disabled. Most ENDBR
instructions are inserted automatically by the compiler, but branch
targets written in assembly must have ENDBR added manually.
Add ENDBR to __vdso_sgx_enter_enclave() branch targets.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
From: "H.J. Lu"
Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch
Tracking.
Signed-off-by: H.J. Lu
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
v24:
- Update for function name changes introduced from splitting shadow stack and
ibt.
arch/
An ELF file's .note.gnu.property indicates features the file supports.
The property is parsed at loading time and passed to arch_setup_elf_
property(). Update it for Indirect Branch Tracking.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for changes introduced from splitting
off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Kees Cook
---
arch/x86/entry/vdso/vdso32/system_call.S | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/entry/vdso/vdso32/system_call.S
b/arch/x86/entry/vdso/vdso32/system_call.S
index de1fff7188aa..c962e7e4f7e3 100644
--- a/arch/x86/entry/v
machine is described in Intel SDM Vol. 1, Sec. 18.3.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for changes from splitting shadow stack and ibt.
arch/x86/kernel/fpu/signal.c | 30 +++---
1 file changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/x86
protection.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
arch/x86/Kconfig | 20
arch/x86/include/asm/disabled-features.h | 8 +++-
2 files changed, 27 insertions(+), 1 deletion(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index
Introduce user-mode Indirect Branch Tracking (IBT) support. Add routines
for the setup/disable of IBT.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Move IBT routines to a separate ibt.c, update related areas accordingly.
arch/x86/include/asm/cet.h | 9 ++
arch/x86/kernel/Makefile
1-yu-cheng...@intel.com/
H.J. Lu (3):
x86/cet/ibt: Update arch_prctl functions for Indirect Branch Tracking
x86/vdso: Insert endbr32/endbr64 to vDSO
x86/vdso/32: Add ENDBR to __kernel_vsyscall entry point
Yu-cheng Yu (6):
x86/cet/ibt: Add Kconfig option for Indirect Branch Tracking
x86/cet/ibt: Add
To prepare the introduction of PROT_SHSTK and be consistent with other
architectures, move arch_vm_get_page_prot() and arch_calc_vm_prot_bits() to
arch/x86/include/asm/mman.h.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Kirill A. Shutemov
---
arch/x86/include/asm/mman.h | 30
would be using this as a bypass to shadow stack protection. However, the
attacker would have to get to the syscall first.
[1] https://lore.kernel.org/lkml/20200828121624.108243-1-hjl.to...@gmail.com/
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Update
("arm64: mte: Add PROT_MTE support to mmap() and
mprotect()"),
Signed-off-by: Yu-cheng Yu
Cc: Catalin Marinas
Cc: Kees Cook
Cc: Kirill A. Shutemov
Cc: Vincenzo Frascino
Cc: Will Deacon
---
arch/arm64/include/asm/mman.h | 4 ++--
arch/sparc/include/asm/mman.h | 4 ++--
include/li
do_arch_prctl_common()'s parameter 'cpuid_enabled' to
'arg2', as it is now also passed to prctl_cet().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
v24:
- Update #ifdef placement relating to shadow stack and ibt split.
- Update function names.
arch/x86/include/asm/cet.h| 7
of this function is Shadow
Stack.
ARM64 is the other arch that has ARCH_USE_GNU_PROPERTY and arch_parse_elf_
property(). Add arch_setup_elf_property() for it.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Mark Brown
Cc: Catalin Marinas
Cc: Dave Martin
---
v24:
- Change cet_setup_shstk
(),
- restore_extra_state().
[1] WAIT_ENDBR will be introduced later in the Indirect Branch Tracking
series, but add that into sc_ext now to keep the struct stable in case
the IBT series is applied later.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Split out shadow stack token routines
setup and verify routines. Also introduce WRUSS, which is
a kernel-mode instruction but writes directly to user shadow stack. It is
used to construct user signal stack as described above.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
arch/x86/include/asm/cet.h | 9 ++
arch/x86
). A compat-mode thread shadow stack
size is further reduced to 1/4. This allows more threads to run in a 32-
bit address space.
Signed-off-by: Yu-cheng Yu
---
arch/x86/include/asm/cet.h | 5 +++
arch/x86/include/asm/mmu_context.h | 3 ++
arch/x86/kernel/process.c | 15 ++--
arch
Introduce basic shadow stack enabling/disabling/allocation routines.
A task's shadow stack is allocated from memory with VM_SHADOW_STACK flag
and has a fixed size of min(RLIMIT_STACK, 4GB).
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Rename cet.c to shstk.c, update related areas
Thus, re-introduce vm_flags to do_mmap().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Peter Collingbourne
Reviewed-by: Kees Cook
Cc: Andrew Morton
Cc: Oleg Nesterov
Cc: linux...@kvack.org
---
v24:
- Change VM_SHSTK to VM_SHADOW_STACK.
- Update commit log.
fs/aio.c | 2 +-
include/linux
, and
pte_*() are updated.
Pte_modify() changes a PTE to 'newprot', but it doesn't use the pte_*().
Introduce fixup_dirty_pte(), which sets a dirty PTE, based on _PAGE_RW,
to either _PAGE_DIRTY or _PAGE_COW.
Apply the same changes to pmd_modify().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill
flags, and handled accordingly in maybe_mkwrite().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Kirill A. Shutemov
---
mm/memory.c | 5 ++---
mm/migrate.c | 3 +--
mm/mprotect.c | 2 +-
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/mm/memory.c b/mm/memory.c
index
to preserve it.
Exclude shadow stack from preserve_write test, and apply the same change to
change_huge_pmd().
Signed-off-by: Yu-cheng Yu
Cc: Kirill A. Shutemov
---
v24:
- Change arch_shadow_stack_mapping() to is_shadow_stack_mapping().
mm/huge_memory.c | 7 ++-
mm/mprotect.c| 9
Account shadow stack pages to stack memory.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Change arch_shadow_stack_mapping() to is_shadow_stack_mapping().
- Change VM_SHSTK to VM_SHADOW_STACK.
arch/x86/mm/pgtable.c | 7 +++
include/linux/pgtable.h | 11
and the shadow stack
page is writable again.
Introduce an x86 version of maybe_mkwrite(), which sets proper PTE bits
according to VM flags.
Apply the same changes to maybe_pmd_mkwrite().
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Instead of doing arch_maybe_mkwrite
, in the
can_follow_write_pte() check, it belongs to the writable page case and
should be excluded from the read-only page pte_dirty() check. Apply
the same changes to can_follow_write_pmd().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Change arch_shadow_stack_mapping
, putting a gap page on both ends of a shadow stack prevents INCSSP,
CALL, and RET from going beyond.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Instead changing vm_*_gap(), create x86-specific versions.
arch/x86/include/asm/page_types.h | 17 +++
arch
non-
atomically, a transient shadow stack PTE can be created as a result.
Thus, prevent that with cmpxchg.
Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many
insights to the issue. Jann Horn provided the cmpxchg solution.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc
After the introduction of _PAGE_COW, a modified page's PTE can have either
_PAGE_DIRTY or _PAGE_COW. Change _PAGE_DIRTY to _PAGE_DIRTY_BITS.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Reviewed-by: Kirill A. Shutemov
Cc: David Airlie
Cc: Joonas Lahtinen
Cc: Jani Nikula
Cc: Daniel
A shadow stack PTE must be read-only and have _PAGE_DIRTY set. However,
read-only and Dirty PTEs also exist for copy-on-write (COW) pages. These
two cases are handled differently for page faults. Introduce
VM_SHADOW_STACK to track shadow stack VMAs.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
To prepare the introduction of _PAGE_COW, move pmd_write() and
pud_write() up in the file, so that they can be used by other
helpers below.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
---
arch/x86/include/asm/pgtable.h | 24
1 file changed, 12 insertions
.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
---
v24:
- Replace CONFIG_X86_CET with CONFIG_X86_SHADOW_STACK to reflect the Kconfig
changes.
arch/x86/include/asm/pgtable.h | 195 ---
arch/x86/include/asm/pgtable_types.h | 42 +-
2 files
. This results in ambiguity between shadow stack and
kernel read-only pages. To resolve this, removed Dirty from kernel read-
only pages.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kirill A. Shutemov
Cc: "H. Peter Anvin"
Cc: Kees Cook
Cc: Thomas Gleixner
Cc: Dave Hansen
Cc: Christoph Hellwig
-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/fpu/types.h | 23 +--
arch/x86/include/asm/fpu/xstate.h | 6 --
arch/x86/include/asm/msr-index.h | 19 +++
arch/x86/kernel/fpu/xstate.c | 10 +-
4 files changed, 53 insertions(+), 5
clearing
_PAGE_DIRTY (vs. _PAGE_RW) is used to trigger the fault, shadow stack read
fault and shadow stack write fault are not differentiated and both are
handled as a write access.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Kirill A. Shutemov
---
v24:
- Change VM_SHSTK
, arrives at a non-ENDBR opcode.
The control-protection fault handler works in a similar way as the general
protection fault handler. It provides the si_code SEGV_CPERR to the signal
handler.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
Cc: Michael Kerrisk
---
arch/x86/include/asm/idtentry.h| 4
Add CPU feature flags for Control-flow Enforcement Technology (CET).
CPUID.(EAX=7,ECX=0):ECX[bit 7] Shadow stack
CPUID.(EAX=7,ECX=0):EDX[bit 20] Indirect Branch Tracking
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for splitting CONFIG_X86_CET to CONFIG_X86_SHADOW_STACK
-Shadow Stack applications continue to work, but without
protection.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for the splitting X86_CET to X86_SHADOW_STACK and X86_IBT.
arch/x86/Kconfig | 26 ++
arch/x86/Kconfig.assembler | 5 +
2 files
Introduce a software-defined X86_FEATURE_CET, which indicates either Shadow
Stack or Indirect Branch Tracking (or both) is present. Also introduce
related cpu init/setup functions.
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update #ifdef placement to reflect Kconfig changes
Explain no_user_shstk/no_user_ibt kernel parameters, and introduce a new
document on Control-flow Enforcement Technology (CET).
Signed-off-by: Yu-cheng Yu
Cc: Kees Cook
---
v24:
- Update for Kconfig changes from X86_CET to X86_SHADOW_STACK, X86_IBT.
- Update for the change of VM_SHSTK
ing regset holes.
Yu-cheng Yu (30):
Documentation/x86: Add CET description
x86/cet/shstk: Add Kconfig option for Shadow Stack
x86/cpufeatures: Add CET CPU feature flags for Control-flow
Enforcement Technology (CET)
x86/cpufeatures: Introduce X86_FEATURE_CET and setup functions
x86/
.
ENDBR is a noop when IBT is unsupported or disabled. Most ENDBR
instructions are inserted automatically by the compiler, but branch
targets written in assembly must have ENDBR added manually.
Add ENDBR to __vdso_sgx_enter_enclave() branch targets.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
off-by: Yu-cheng Yu
Acked-by: Andy Lutomirski
Reviewed-by: Kees Cook
---
arch/x86/entry/vdso/Makefile | 4
1 file changed, 4 insertions(+)
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 05c4abc2fdfd..c9eccbc06e8c 100644
--- a/arch/x86/entry/vdso/Makefile
off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Kees Cook
---
arch/x86/entry/vdso/vdso32/system_call.S | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/entry/vdso/vdso32/system_call.S
b/arch/x86/entry/vdso/vdso32/system_call.S
index de1fff7188aa..adbe948c1a81 100644
--- a/arch/x86/entry/v
at call sites.
Signed-off-by: Yu-cheng Yu
Cc: Andy Lutomirski
Cc: Borislav Petkov
Cc: Dave Hansen
Cc: Jarkko Sakkinen
Cc: Peter Zijlstra
---
arch/x86/entry/calling.h | 18 ++
1 file changed, 18 insertions(+)
diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
machine is described in Intel SDM Vol. 1, Sec. 18.3.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/kernel/cet.c| 26 --
arch/x86/kernel/fpu/signal.c | 8 +---
2 files changed, 29 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/cet.c
From: "H.J. Lu"
Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch
Tracking.
Signed-off-by: H.J. Lu
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/kernel/cet_prctl.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/arch/x86/kernel/cet
bt: Update arch_prctl functions for Indirect Branch Tracking
x86/vdso/32: Add ENDBR to __kernel_vsyscall entry point
x86/vdso: Insert endbr32/endbr64 to vDSO
Yu-cheng Yu (6):
x86/cet/ibt: Update Kconfig for user-mode Indirect Branch Tracking
x86/cet/ibt: User-mode Indirect Branch Tracki
Introduce user-mode Indirect Branch Tracking (IBT) support. Add routines
for the setup/disable of IBT.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/cet.h | 3 +++
arch/x86/kernel/cet.c | 33 +
2 files changed, 36 insertions
An ELF file's .note.gnu.property indicates features the file supports.
The property is parsed at loading time and passed to arch_setup_elf_
property(). Update it for Indirect Branch Tracking.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/kernel/process_64.c | 8
1
the compiler is up-to-date at config time.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 2c93178262f5..96000ed48469 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1953,6
be using this as a
bypass to shadow stack protection. However, the attacker would have to get
to the syscall first.
[1] https://lore.kernel.org/lkml/20200828121624.108243-1-hjl.to...@gmail.com/
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/mman.h | 57
do_arch_prctl_common()'s parameter 'cpuid_enabled' to
'arg2', as it is now also passed to prctl_cet().
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/cet.h| 3 ++
arch/x86/include/uapi/asm/prctl.h | 4 +++
arch/x86/kernel/Makefile | 2 +-
arch/x86
A shadow stack PTE must be read-only and have _PAGE_DIRTY set. However,
read-only and Dirty PTEs also exist for copy-on-write (COW) pages. These
two cases are handled differently for page faults. Introduce VM_SHSTK to
track shadow stack VMAs.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
To prepare changes to arch_calc_vm_prot_bits() in the next patch, and be
consistent with other architectures, move arch_vm_get_page_prot() and
arch_calc_vm_prot_bits() to arch/x86/include/asm/mman.h.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/mman.h | 30
). A compat-mode thread shadow stack
size is further reduced to 1/4. This allows more threads to run in a 32-
bit address space.
Signed-off-by: Yu-cheng Yu
---
arch/x86/include/asm/cet.h | 5 +++
arch/x86/include/asm/mmu_context.h | 3 ++
arch/x86/kernel/cet.c | 49
of this function is Shadow
Stack.
ARM64 is the other arch that has ARCH_USE_GNU_PROPERTY and arch_parse_elf_
property(). Add arch_setup_elf_property() for it.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
Cc: Mark Brown
Cc: Catalin Marinas
Cc: Dave Martin
---
arch/arm64/include/asm/elf.h | 5
Account shadow stack pages to stack memory.
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/mm/pgtable.c | 7 +++
include/linux/pgtable.h | 11 +++
mm/mmap.c | 5 +
3 files changed, 23 insertions(+)
diff --git a/arch/x86/mm/pgtable.c b/arch/x86
Introduce basic shadow stack enabling/disabling/allocation routines.
A task's shadow stack is allocated from memory with VM_SHSTK flag and has
a fixed size of min(RLIMIT_STACK, 4GB).
Signed-off-by: Yu-cheng Yu
Reviewed-by: Kees Cook
---
arch/x86/include/asm/cet.h | 28 ++
arch/x86
1 - 100 of 1099 matches
Mail list logo
