Re: [PATCHv3 2/2] arch: Rename CONFIG_DEBUG_RODATA and CONFIG_DEBUG_MODULE_RONX
On Thu, Feb 16, 2017 at 2:25 PM, Pavel Machek wrote: > Hi! > >> >> -config DEBUG_RODATA >> +config STRICT_KERNEL_RWX >> bool "Make kernel text and rodata read-only" if >> ARCH_OPTIONAL_KERNEL_RWX >> depends on ARCH_HAS_STRICT_KERNEL_RWX >> default !ARCH_OPTIONAL_KERNEL_RWX || > > Debug features are expected to have runtime cost, so kconfig help is > silent about those. But there are runtime costs, right? It would be > nice to mention them in the help text... It depends on the architecture. The prior help text for arm said: The tradeoff is that each region is padded to section-size (1MiB) boundaries (because their permissions are different and splitting the 1M pages into 4K ones causes TLB performance problems), which can waste memory. parisc (somewhat inaccurately) said: This option may have a slight performance impact because a portion of the kernel code won't be covered by a TLB anymore. IIUC, arm64 does what parisc is hinting at: mappings at the end are broken down to PAGE_SIZE. On x86, IIUC, there's actually no change to TLB performance due to how the mappings are already set up. I'm not sure the best way to express this in the new help text. Do you have some suggestions on wording? Personally, I don't really think it's worth mentioning this in Kconfig help, which, in theory, is supposed to limit how technical it gets. And I think the performance impact is almost entirely negligible compared to the risks addressed. -Kees -- Kees Cook Pixel Security -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 1/4] x86/mm: Adapt MODULES_END based on Fixmap section size
Hi Thomas,
[auto build test ERROR on next-20170216]
[also build test ERROR on v4.10-rc8]
[cannot apply to tip/x86/core kvm/linux-next tip/auto-latest v4.9-rc8 v4.9-rc7
v4.9-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system]
url:
https://github.com/0day-ci/linux/commits/Thomas-Garnier/x86-mm-Adapt-MODULES_END-based-on-Fixmap-section-size/20170217-072759
config: x86_64-randconfig-s4-02170325 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
In file included from arch/x86/include/asm/pgtable_types.h:240:0,
from arch/x86/include/asm/paravirt_types.h:44,
from arch/x86/include/asm/ptrace.h:71,
from arch/x86/include/asm/math_emu.h:4,
from arch/x86/include/asm/processor.h:11,
from arch/x86/include/asm/cpufeature.h:4,
from arch/x86/include/asm/thread_info.h:52,
from include/linux/thread_info.h:25,
from arch/x86/include/asm/preempt.h:6,
from include/linux/preempt.h:59,
from include/linux/spinlock.h:50,
from include/linux/wait.h:8,
from include/linux/fs.h:5,
from include/linux/debugfs.h:18,
from arch/x86/mm/dump_pagetables.c:15:
>> arch/x86/include/asm/pgtable_64_types.h:71:23: error: implicit declaration
>> of function '__fix_to_virt' [-Werror=implicit-function-declaration]
#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1)
^
arch/x86/mm/dump_pagetables.c:87:4: note: in expansion of macro 'MODULES_END'
{ MODULES_END, "End Modules" },
^~~
arch/x86/include/asm/pgtable_64_types.h:71:37: error:
'__end_of_fixed_addresses' undeclared here (not in a function)
#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1)
^
arch/x86/mm/dump_pagetables.c:87:4: note: in expansion of macro 'MODULES_END'
{ MODULES_END, "End Modules" },
^~~
cc1: some warnings being treated as errors
vim +/__fix_to_virt +71 arch/x86/include/asm/pgtable_64_types.h
65 #define VMALLOC_START __VMALLOC_BASE
66 #define VMEMMAP_START __VMEMMAP_BASE
67 #endif /* CONFIG_RANDOMIZE_MEMORY */
68 #define VMALLOC_END (VMALLOC_START + _AC((VMALLOC_SIZE_TB << 40) -
1, UL))
69 #define MODULES_VADDR(__START_KERNEL_map + KERNEL_IMAGE_SIZE)
70 /* The module sections ends with the start of the fixmap */
> 71 #define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1)
72 #define MODULES_LEN (MODULES_END - MODULES_VADDR)
73 #define ESPFIX_PGD_ENTRY _AC(-2, UL)
74 #define ESPFIX_BASE_ADDR (ESPFIX_PGD_ENTRY << PGDIR_SHIFT)
---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
.config.gz
Description: application/gzip
Re: [PATCH v4 1/4] x86/mm: Adapt MODULES_END based on Fixmap section size
Hi Thomas,
[auto build test ERROR on next-20170216]
[also build test ERROR on v4.10-rc8]
[cannot apply to tip/x86/core kvm/linux-next tip/auto-latest v4.9-rc8 v4.9-rc7
v4.9-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system]
url:
https://github.com/0day-ci/linux/commits/Thomas-Garnier/x86-mm-Adapt-MODULES_END-based-on-Fixmap-section-size/20170217-072759
config: x86_64-randconfig-h0-02170640 (attached as .config)
compiler: gcc-4.9 (Debian 4.9.4-2) 4.9.4
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All error/warnings (new ones prefixed by >>):
>> arch/x86/mm/dump_pagetables.c:87:2: error: implicit declaration of function
>> '__fix_to_virt' [-Werror=implicit-function-declaration]
{ MODULES_END, "End Modules" },
^
In file included from arch/x86/include/asm/pgtable_types.h:240:0,
from arch/x86/include/asm/paravirt_types.h:44,
from arch/x86/include/asm/ptrace.h:71,
from arch/x86/include/asm/math_emu.h:4,
from arch/x86/include/asm/processor.h:11,
from arch/x86/include/asm/cpufeature.h:4,
from arch/x86/include/asm/thread_info.h:52,
from include/linux/thread_info.h:25,
from arch/x86/include/asm/preempt.h:6,
from include/linux/preempt.h:59,
from include/linux/spinlock.h:50,
from include/linux/wait.h:8,
from include/linux/fs.h:5,
from include/linux/debugfs.h:18,
from arch/x86/mm/dump_pagetables.c:15:
>> arch/x86/include/asm/pgtable_64_types.h:71:37: error:
>> '__end_of_fixed_addresses' undeclared here (not in a function)
#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1)
^
>> arch/x86/mm/dump_pagetables.c:87:4: note: in expansion of macro 'MODULES_END'
{ MODULES_END, "End Modules" },
^
cc1: some warnings being treated as errors
vim +/__fix_to_virt +87 arch/x86/mm/dump_pagetables.c
926e5392b Arjan van de Ven 2008-04-17 9 * This program is free software;
you can redistribute it and/or
926e5392b Arjan van de Ven 2008-04-17 10 * modify it under the terms of the
GNU General Public License
926e5392b Arjan van de Ven 2008-04-17 11 * as published by the Free Software
Foundation; version 2
926e5392b Arjan van de Ven 2008-04-17 12 * of the License.
926e5392b Arjan van de Ven 2008-04-17 13 */
926e5392b Arjan van de Ven 2008-04-17 14
fe770bf03 H. Peter Anvin 2008-04-17 @15 #include
fe770bf03 H. Peter Anvin 2008-04-17 16 #include
84e629b66 Paul Gortmaker 2016-07-13 17 #include
146fbb766 Andrey Ryabinin 2017-02-10 18 #include
926e5392b Arjan van de Ven 2008-04-17 19 #include
926e5392b Arjan van de Ven 2008-04-17 20
926e5392b Arjan van de Ven 2008-04-17 21 #include
926e5392b Arjan van de Ven 2008-04-17 22
926e5392b Arjan van de Ven 2008-04-17 23 /*
926e5392b Arjan van de Ven 2008-04-17 24 * The dumper groups pagetable
entries of the same type into one, and for
926e5392b Arjan van de Ven 2008-04-17 25 * that it needs to keep some state
when walking, and flush this state
926e5392b Arjan van de Ven 2008-04-17 26 * when a "break" in the continuity
is found.
926e5392b Arjan van de Ven 2008-04-17 27 */
926e5392b Arjan van de Ven 2008-04-17 28 struct pg_state {
926e5392b Arjan van de Ven 2008-04-17 29 int level;
926e5392b Arjan van de Ven 2008-04-17 30 pgprot_t current_prot;
926e5392b Arjan van de Ven 2008-04-17 31 unsigned long start_address;
926e5392b Arjan van de Ven 2008-04-17 32 unsigned long current_address;
fe770bf03 H. Peter Anvin 2008-04-17 33 const struct addr_marker
*marker;
3891a04aa H. Peter Anvin 2014-04-29 34 unsigned long lines;
ef6bea6dd Borislav Petkov 2014-01-18 35 bool to_dmesg;
e1a58320a Stephen Smalley 2015-10-05 36 bool check_wx;
e1a58320a Stephen Smalley 2015-10-05 37 unsigned long wx_pages;
926e5392b Arjan van de Ven 2008-04-17 38 };
926e5392b Arjan van de Ven 2008-04-17 39
fe770bf03 H. Peter Anvin 2008-04-17 40 struct addr_marker {
fe770bf03 H. Peter Anvin 2008-04-17 41 unsigned long start_address;
fe770bf03 H. Peter Anvin 2008-04-17 42 const char *name;
3891a04aa H. Peter Anvin 2014-04-29 43 unsigned long max_lines;
fe770bf03 H. Peter Anvin 2008-04-17 44 };
926e5392b Arjan van de Ven 2008-04-17 45
92851e2fc Andres Salomon 2010-07-20 46 /* indices for address_markers; keep
sync'd w/ address_markers below */
92851e2fc Andres Salomon 2010-07-20 47 enum address_markers_idx {
92851e2fc Andres Salomon 2010-07-20 48 USER_SPACE_NR
Re: [PATCHv3 2/2] arch: Rename CONFIG_DEBUG_RODATA and CONFIG_DEBUG_MODULE_RONX
Hi! > > -config DEBUG_RODATA > +config STRICT_KERNEL_RWX > bool "Make kernel text and rodata read-only" if ARCH_OPTIONAL_KERNEL_RWX > depends on ARCH_HAS_STRICT_KERNEL_RWX > default !ARCH_OPTIONAL_KERNEL_RWX || Debug features are expected to have runtime cost, so kconfig help is silent about those. But there are runtime costs, right? It would be nice to mention them in the help text... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
[PATCH v4 2/4] x86: Remap GDT tables in the Fixmap section
Each processor holds a GDT in its per-cpu structure. The sgdt
instruction gives the base address of the current GDT. This address can
be used to bypass KASLR memory randomization. With another bug, an
attacker could target other per-cpu structures or deduce the base of
the main memory section (PAGE_OFFSET).
This patch relocates the GDT table for each processor inside the
Fixmap section. The space is reserved based on number of supported
processors.
For consistency, the remapping is done by default on 32 and 64-bit.
Each processor switches to its remapped GDT at the end of
initialization. For hibernation, the main processor returns with the
original GDT and switches back to the remapping at completion.
This patch was tested on both architectures. Hibernation and KVM were
both tested specially for their usage of the GDT.
Signed-off-by: Thomas Garnier
---
Based on next-20170213
---
arch/x86/entry/vdso/vma.c | 2 +-
arch/x86/include/asm/desc.h | 33 +
arch/x86/include/asm/fixmap.h | 4
arch/x86/include/asm/processor.h | 1 +
arch/x86/include/asm/stackprotector.h | 2 +-
arch/x86/kernel/acpi/sleep.c | 2 +-
arch/x86/kernel/apm_32.c | 6 +++---
arch/x86/kernel/cpu/common.c | 26 --
arch/x86/kernel/setup_percpu.c| 2 +-
arch/x86/kernel/smpboot.c | 2 +-
arch/x86/platform/efi/efi_32.c| 4 ++--
arch/x86/power/cpu.c | 7 +--
arch/x86/xen/enlighten.c | 2 +-
arch/x86/xen/smp.c| 2 +-
drivers/lguest/x86/core.c | 6 +++---
drivers/pnp/pnpbios/bioscalls.c | 10 +-
16 files changed, 83 insertions(+), 28 deletions(-)
diff --git a/arch/x86/entry/vdso/vma.c b/arch/x86/entry/vdso/vma.c
index 572cee3fccff..9c8bd4cfcc6e 100644
--- a/arch/x86/entry/vdso/vma.c
+++ b/arch/x86/entry/vdso/vma.c
@@ -353,7 +353,7 @@ static void vgetcpu_cpu_init(void *arg)
d.p = 1;/* Present */
d.d = 1;/* 32-bit */
- write_gdt_entry(get_cpu_gdt_table(cpu), GDT_ENTRY_PER_CPU, &d,
DESCTYPE_S);
+ write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_PER_CPU, &d, DESCTYPE_S);
}
static int vgetcpu_online(unsigned int cpu)
diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h
index 12080d87da3b..5d4ba1311737 100644
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -4,6 +4,7 @@
#include
#include
#include
+#include
#include
#include
@@ -45,11 +46,35 @@ struct gdt_page {
DECLARE_PER_CPU_PAGE_ALIGNED(struct gdt_page, gdt_page);
-static inline struct desc_struct *get_cpu_gdt_table(unsigned int cpu)
+/* Provide the original GDT */
+static inline struct desc_struct *get_cpu_gdt_rw(unsigned int cpu)
{
return per_cpu(gdt_page, cpu).gdt;
}
+static inline unsigned long get_cpu_gdt_rw_vaddr(unsigned int cpu)
+{
+ return (unsigned long)get_cpu_gdt_rw(cpu);
+}
+
+/* Get the fixmap index for a specific processor */
+static inline unsigned int get_cpu_gdt_ro_index(int cpu)
+{
+ return FIX_GDT_REMAP_BEGIN + cpu;
+}
+
+/* Provide the fixmap address of the remapped GDT */
+static inline struct desc_struct *get_cpu_gdt_ro(int cpu)
+{
+ unsigned int idx = get_cpu_gdt_ro_index(cpu);
+ return (struct desc_struct *)__fix_to_virt(idx);
+}
+
+static inline unsigned long get_cpu_gdt_ro_vaddr(int cpu)
+{
+ return (unsigned long)get_cpu_gdt_ro(cpu);
+}
+
#ifdef CONFIG_X86_64
static inline void pack_gate(gate_desc *gate, unsigned type, unsigned long
func,
@@ -174,7 +199,7 @@ static inline void set_tssldt_descriptor(void *d, unsigned
long addr, unsigned t
static inline void __set_tss_desc(unsigned cpu, unsigned int entry, void *addr)
{
- struct desc_struct *d = get_cpu_gdt_table(cpu);
+ struct desc_struct *d = get_cpu_gdt_rw(cpu);
tss_desc tss;
/*
@@ -202,7 +227,7 @@ static inline void native_set_ldt(const void *addr,
unsigned int entries)
set_tssldt_descriptor(&ldt, (unsigned long)addr, DESC_LDT,
entries * LDT_ENTRY_SIZE - 1);
- write_gdt_entry(get_cpu_gdt_table(cpu), GDT_ENTRY_LDT,
+ write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_LDT,
&ldt, DESC_LDT);
asm volatile("lldt %w0"::"q" (GDT_ENTRY_LDT*8));
}
@@ -244,7 +269,7 @@ static inline unsigned long native_store_tr(void)
static inline void native_load_tls(struct thread_struct *t, unsigned int cpu)
{
- struct desc_struct *gdt = get_cpu_gdt_table(cpu);
+ struct desc_struct *gdt = get_cpu_gdt_rw(cpu);
unsigned int i;
for (i = 0; i < GDT_ENTRY_TLS_ENTRIES; i++)
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 8554f960e21b..b65155cc3760 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86
[PATCH v4 1/4] x86/mm: Adapt MODULES_END based on Fixmap section size
This patch aligns MODULES_END to the beginning of the Fixmap section.
It optimizes the space available for both sections. The address is
pre-computed based on the number of pages required by the Fixmap
section.
It will allow GDT remapping in the Fixmap section. The current
MODULES_END static address does not provide enough space for the kernel
to support a large number of processors.
Signed-off-by: Thomas Garnier
---
Based on next-20170213
---
Documentation/x86/x86_64/mm.txt | 5 -
arch/x86/include/asm/pgtable_64_types.h | 3 ++-
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt
index 5724092db811..ee3f9c30957c 100644
--- a/Documentation/x86/x86_64/mm.txt
+++ b/Documentation/x86/x86_64/mm.txt
@@ -19,7 +19,7 @@ ff00 - ff7f (=39 bits) %esp fixup
stacks
ffef - fffe (=64 GB) EFI region mapping space
... unused hole ...
8000 - 9fff (=512 MB) kernel text mapping, from phys 0
-a000 - ff5f (=1526 MB) module mapping space
+a000 - ff5f (=1526 MB) module mapping space (variable)
ff60 - ffdf (=8 MB) vsyscalls
ffe0 - (=2 MB) unused hole
@@ -39,6 +39,9 @@ memory window (this size is arbitrary, it can be raised later
if needed).
The mappings are not part of any other kernel PGD and are only available
during EFI runtime calls.
+The module mapping space size changes based on the CONFIG requirements for the
+following fixmap section.
+
Note that if CONFIG_RANDOMIZE_MEMORY is enabled, the direct mapping of all
physical memory, vmalloc/ioremap space and virtual memory map are randomized.
Their order is preserved but their base will be offset early at boot time.
diff --git a/arch/x86/include/asm/pgtable_64_types.h
b/arch/x86/include/asm/pgtable_64_types.h
index 3a264200c62f..bb05e21cf3c7 100644
--- a/arch/x86/include/asm/pgtable_64_types.h
+++ b/arch/x86/include/asm/pgtable_64_types.h
@@ -67,7 +67,8 @@ typedef struct { pteval_t pte; } pte_t;
#endif /* CONFIG_RANDOMIZE_MEMORY */
#define VMALLOC_END(VMALLOC_START + _AC((VMALLOC_SIZE_TB << 40) - 1, UL))
#define MODULES_VADDR(__START_KERNEL_map + KERNEL_IMAGE_SIZE)
-#define MODULES_END _AC(0xff00, UL)
+/* The module sections ends with the start of the fixmap */
+#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1)
#define MODULES_LEN (MODULES_END - MODULES_VADDR)
#define ESPFIX_PGD_ENTRY _AC(-2, UL)
#define ESPFIX_BASE_ADDR (ESPFIX_PGD_ENTRY << PGDIR_SHIFT)
--
2.11.0.483.g087da7b7c-goog
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 3/4] x86: Make the GDT remapping read-only on 64-bit
This patch makes the GDT remapped pages read-only to prevent corruption.
This change is done only on 64-bit.
The native_load_tr_desc function was adapted to correctly handle a
read-only GDT. The LTR instruction always writes to the GDT TSS entry.
This generates a page fault if the GDT is read-only. This change checks
if the current GDT is a remap and swap GDTs as needed. This function was
tested by booting multiple machines and checking hibernation works
properly.
KVM SVM and VMX were adapted to use the writeable GDT. On VMX, the
per-cpu variable was removed for functions to fetch the original GDT.
Instead of reloading the previous GDT, VMX will reload the fixmap GDT as
expected. For testing, VMs were started and restored on multiple
configurations.
Signed-off-by: Thomas Garnier
---
Based on next-20170213
---
arch/x86/include/asm/desc.h | 51
arch/x86/include/asm/processor.h | 1 +
arch/x86/kernel/cpu/common.c | 28 +-
arch/x86/kvm/svm.c | 4 +---
arch/x86/kvm/vmx.c | 15
5 files changed, 75 insertions(+), 24 deletions(-)
diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h
index 5d4ba1311737..15b2a86c9267 100644
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -57,6 +57,17 @@ static inline unsigned long get_cpu_gdt_rw_vaddr(unsigned
int cpu)
return (unsigned long)get_cpu_gdt_rw(cpu);
}
+/* Provide the current original GDT */
+static inline struct desc_struct *get_current_gdt_rw(void)
+{
+ return this_cpu_ptr(&gdt_page)->gdt;
+}
+
+static inline unsigned long get_current_gdt_rw_vaddr(void)
+{
+ return (unsigned long)get_current_gdt_rw();
+}
+
/* Get the fixmap index for a specific processor */
static inline unsigned int get_cpu_gdt_ro_index(int cpu)
{
@@ -233,11 +244,6 @@ static inline void native_set_ldt(const void *addr,
unsigned int entries)
}
}
-static inline void native_load_tr_desc(void)
-{
- asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8));
-}
-
static inline void native_load_gdt(const struct desc_ptr *dtr)
{
asm volatile("lgdt %0"::"m" (*dtr));
@@ -258,6 +264,41 @@ static inline void native_store_idt(struct desc_ptr *dtr)
asm volatile("sidt %0":"=m" (*dtr));
}
+/*
+ * The LTR instruction marks the TSS GDT entry as busy. On 64-bit, the GDT is
+ * a read-only remapping. To prevent a page fault, the GDT is switched to the
+ * original writeable version when needed.
+ */
+#ifdef CONFIG_X86_64
+static inline void native_load_tr_desc(void)
+{
+ struct desc_ptr gdt;
+ int cpu = raw_smp_processor_id();
+ bool restore = 0;
+ struct desc_struct *fixmap_gdt;
+
+ native_store_gdt(&gdt);
+ fixmap_gdt = get_cpu_gdt_ro(cpu);
+
+ /*
+* If the current GDT is the read-only fixmap, swap to the original
+* writeable version. Swap back at the end.
+*/
+ if (gdt.address == (unsigned long)fixmap_gdt) {
+ load_direct_gdt(cpu);
+ restore = 1;
+ }
+ asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8));
+ if (restore)
+ load_fixmap_gdt(cpu);
+}
+#else
+static inline void native_load_tr_desc(void)
+{
+ asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8));
+}
+#endif
+
static inline unsigned long native_store_tr(void)
{
unsigned long tr;
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index c441d1f7e275..6ea9e419a856 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -706,6 +706,7 @@ extern struct desc_ptr early_gdt_descr;
extern void cpu_set_gdt(int);
extern void switch_to_new_gdt(int);
+extern void load_direct_gdt(int);
extern void load_fixmap_gdt(int);
extern void load_percpu_segment(int);
extern void cpu_init(void);
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 2853a42ded2d..bdf521383900 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -444,13 +444,31 @@ void load_percpu_segment(int cpu)
load_stack_canary_segment();
}
+/* On 64-bit the GDT remapping is read-only */
+#ifdef CONFIG_X86_64
+#define PAGE_FIXMAP_GDT PAGE_KERNEL_RO
+#else
+#define PAGE_FIXMAP_GDT PAGE_KERNEL
+#endif
+
/* Setup the fixmap mapping only once per-processor */
static inline void setup_fixmap_gdt(int cpu)
{
__set_fixmap(get_cpu_gdt_ro_index(cpu),
-__pa(get_cpu_gdt_rw(cpu)), PAGE_KERNEL);
+__pa(get_cpu_gdt_rw(cpu)), PAGE_FIXMAP_GDT);
}
+/* Load the original GDT from the per-cpu structure */
+void load_direct_gdt(int cpu)
+{
+ struct desc_ptr gdt_descr;
+
+ gdt_descr.address = (long)get_cpu_gdt_rw(cpu);
+ gdt_descr.size = GDT_SIZE - 1;
+ load_gdt(&gdt_descr);
+}
+EXPORT_SYMBOL_GPL(load_direct_gdt);
+
/* Load a fixmap remapping of the per-cpu GDT */
void load_fi
[PATCH v4 4/4] KVM: VMX: Simplify segment_base
The KVM segment_base function is confusing. This patch replaces integers
with appropriate flags, simplify constructs and add comments.
Signed-off-by: Thomas Garnier
---
Based on next-20170213
---
arch/x86/kvm/vmx.c | 30 --
1 file changed, 20 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 99167f20bc34..91e619269128 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2060,27 +2060,37 @@ static bool update_transition_efer(struct vcpu_vmx
*vmx, int efer_offset)
static unsigned long segment_base(u16 selector)
{
struct desc_struct *d;
- unsigned long table_base;
+ struct desc_struct *table_base;
unsigned long v;
+ u32 high32;
- if (!(selector & ~3))
+ if (!(selector & ~SEGMENT_RPL_MASK))
return 0;
- table_base = get_current_gdt_rw_vaddr();
-
- if (selector & 4) { /* from ldt */
+ /* LDT selector */
+ if ((selector & SEGMENT_TI_MASK) == SEGMENT_LDT) {
u16 ldt_selector = kvm_read_ldt();
- if (!(ldt_selector & ~3))
+ if (!(ldt_selector & ~SEGMENT_RPL_MASK))
return 0;
- table_base = segment_base(ldt_selector);
+ table_base = (struct desc_struct *)segment_base(ldt_selector);
+ } else {
+ table_base = get_current_gdt_rw();
}
- d = (struct desc_struct *)(table_base + (selector & ~7));
+
+ d = table_base + (selector >> 3);
v = get_desc_base(d);
#ifdef CONFIG_X86_64
- if (d->s == 0 && (d->type == 2 || d->type == 9 || d->type == 11))
- v |= ((unsigned long)((struct ldttss_desc64 *)d)->base3) << 32;
+ /*
+* Extend the virtual address if we have a system descriptor entry for
+* LDT or TSS (available or busy).
+*/
+ if (d->s == 0 && (d->type == DESC_LDT || d->type == DESC_TSS ||
+ d->type == 11/*Busy TSS */)) {
+ high32 = ((struct ldttss_desc64 *)d)->base3;
+ v |= (u64)high32 << 32;
+ }
#endif
return v;
}
--
2.11.0.483.g087da7b7c-goog
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature
On Thu, Feb 16, 2017 at 01:42:13PM -0600, Tom Lendacky wrote: > I realize it's a bit more code and expands the changes but I thought it > would be a bit clearer as to what was going on this way. And then the > follow on patch for the physical address reduction goes in nicely, too. Well, the code from the next patch should go to AMD-specific place like arch/x86/kernel/cpu/amd.c anyway, where you don't have to do vendor checks. > If you prefer I stay with the scattered feature approach and then clear > the bit based on the MSR at the end of init_amd() I can do that. I'm > not attached to either method. Yes please. We should keep the shole X86_FEATURE machinery from exploding in size. Especially if CPUID_0x801f is not a leaf we're going to be adding the majority of its bits, to warrant a separate ->x86_capability array element. [If it does later, we can always move it to a separate element. ] Thanks. -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)
On 02/16/2017 11:56 AM, Borislav Petkov wrote: > Ok, this time detailed review :-) > > On Thu, Feb 16, 2017 at 09:42:11AM -0600, Tom Lendacky wrote: >> This patch adds a Documenation entry to decribe the AMD Secure Memory >> Encryption (SME) feature. > > Please introduce a spellchecker into your patch creation workflow. I see > two typos in one line. > > Also, never start patch commit messages with "This patch" - we know it > is this patch. Always write a doer-sentences explaining the why, not the > what. Something like: > > "Add a SME and mem_encrypt= kernel parameter documentation." > > for example. Ok, will do. > >> Signed-off-by: Tom Lendacky >> --- >> Documentation/admin-guide/kernel-parameters.txt | 11 >> Documentation/x86/amd-memory-encryption.txt | 57 >> +++ >> 2 files changed, 68 insertions(+) >> create mode 100644 Documentation/x86/amd-memory-encryption.txt >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index 110745e..91c40fa 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -2145,6 +2145,17 @@ >> memory contents and reserves bad memory >> regions that are detected. >> >> +mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control >> +Valid arguments: on, off >> +Default (depends on kernel configuration option): >> + on (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) >> + off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n) >> +mem_encrypt=on: Activate SME >> +mem_encrypt=off:Do not activate SME >> + >> +Refer to the SME documentation for details on when > > "Refer to Documentation/x86/amd-memory-encryption.txt .." Ok. > >> +memory encryption can be activated. >> + >> mem_sleep_default= [SUSPEND] Default system suspend mode: >> s2idle - Suspend-To-Idle >> shallow - Power-On Suspend or equivalent (if supported) >> diff --git a/Documentation/x86/amd-memory-encryption.txt >> b/Documentation/x86/amd-memory-encryption.txt >> new file mode 100644 >> index 000..0938e89 >> --- /dev/null >> +++ b/Documentation/x86/amd-memory-encryption.txt >> @@ -0,0 +1,57 @@ >> +Secure Memory Encryption (SME) is a feature found on AMD processors. >> + >> +SME provides the ability to mark individual pages of memory as encrypted >> using >> +the standard x86 page tables. A page that is marked encrypted will be >> +automatically decrypted when read from DRAM and encrypted when written to >> +DRAM. SME can therefore be used to protect the contents of DRAM from >> physical >> +attacks on the system. >> + >> +A page is encrypted when a page table entry has the encryption bit set (see >> +below how to determine the position of the bit). The encryption bit can be > > "... how to determine its position)." Ok. > >> +specified in the cr3 register, allowing the PGD table to be encrypted. Each >> +successive level of page tables can also be encrypted. >> + >> +Support for SME can be determined through the CPUID instruction. The CPUID >> +function 0x801f reports information related to SME: >> + >> +0x801f[eax]: >> +Bit[0] indicates support for SME >> +0x801f[ebx]: >> +Bit[5:0] pagetable bit number used to activate memory >> + encryption > > s/Bit/Bits/ Ok. > >> +Bit[11:6] reduction in physical address space, in bits, when > > Ditto. > >> + memory encryption is enabled (this only affects system >> + physical addresses, not guest physical addresses) >> + >> +If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to > > Let's use the kernel's define name MSR_K8_SYSCFG to avoid ambiguity. Will do. > >> +determine if SME is enabled and/or to enable memory encryption: >> + >> +0xc0010010: >> +Bit[23] 0 = memory encryption features are disabled >> + 1 = memory encryption features are enabled >> + >> +Linux relies on BIOS to set this bit if BIOS has determined that the >> reduction >> +in the physical address space as a result of enabling memory encryption (see >> +CPUID information above) will not conflict with the address space resource >> +requirements for the system. If this bit is not set upon Linux startup then >> +Linux itself will not set it and memory encryption will not be possible. >> + >> +The state of SME in the Linux kernel can be documented as follows: >> +- Supported: >> + The CPU supports SME (determined through CPUID instruction). >> + >> +- Enabled: >> + Supported and bit 23 of the SYS_CFG MSR is set. > > Ditto. > >> + >> +- Ac
Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature
On 02/16/2017 12:13 PM, Borislav Petkov wrote: > On Thu, Feb 16, 2017 at 09:42:36AM -0600, Tom Lendacky wrote: >> Update the CPU features to include identifying and reporting on the >> Secure Memory Encryption (SME) feature. SME is identified by CPUID >> 0x801f, but requires BIOS support to enable it (set bit 23 of >> SYS_CFG MSR). Only show the SME feature as available if reported by >> CPUID and enabled by BIOS. >> >> Signed-off-by: Tom Lendacky >> --- >> arch/x86/include/asm/cpufeature.h|7 +-- >> arch/x86/include/asm/cpufeatures.h |5 - >> arch/x86/include/asm/disabled-features.h |3 ++- >> arch/x86/include/asm/msr-index.h |2 ++ >> arch/x86/include/asm/required-features.h |3 ++- >> arch/x86/kernel/cpu/common.c | 19 +++ >> 6 files changed, 34 insertions(+), 5 deletions(-) > > What happened here? > > You had it already: > > https://lkml.kernel.org/r/20161110003459.3280.25796.st...@tlendack-t1.amdoffice.net > > The bit in get_cpu_cap() with checking the MSR you can add at the end of > init_amd() for example. I realize it's a bit more code and expands the changes but I thought it would be a bit clearer as to what was going on this way. And then the follow on patch for the physical address reduction goes in nicely, too. If you prefer I stay with the scattered feature approach and then clear the bit based on the MSR at the end of init_amd() I can do that. I'm not attached to either method. Thanks, Tom > -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] mm/maccess.c: Fix up kernel doc notation
One argument was incorrect, two functions weren't showing the brief description, and the docs for strncpy_from_unsafe() had a colon attached to it. Fix it up. Cc: Signed-off-by: Stephen Boyd --- mm/maccess.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/maccess.c b/mm/maccess.c index 78f9274dd49d..ee305aa22535 100644 --- a/mm/maccess.c +++ b/mm/maccess.c @@ -6,7 +6,7 @@ #include /** - * probe_kernel_read(): safely attempt to read from a location + * probe_kernel_read() - safely attempt to read from a location. * @dst: pointer to the buffer that shall take the data * @src: address to read from * @size: size of the data chunk @@ -40,7 +40,7 @@ long __probe_kernel_read(void *dst, const void *src, size_t size) EXPORT_SYMBOL_GPL(probe_kernel_read); /** - * probe_kernel_write(): safely attempt to write to a location + * probe_kernel_write() - safely attempt to write to a location. * @dst: address to write to * @src: pointer to the data that shall be written * @size: size of the data chunk @@ -67,10 +67,10 @@ long __probe_kernel_write(void *dst, const void *src, size_t size) EXPORT_SYMBOL_GPL(probe_kernel_write); /** - * strncpy_from_unsafe: - Copy a NUL terminated string from unsafe address. + * strncpy_from_unsafe() - Copy a NUL terminated string from unsafe address. * @dst: Destination address, in kernel space. This buffer must be at * least @count bytes long. - * @src: Unsafe address. + * @unsafe_addr: Unsafe address. * @count: Maximum number of bytes to copy, including the trailing NUL. * * Copies a NUL-terminated string from unsafe address to kernel buffer. -- 2.10.0.297.gf6727b0 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature
On Thu, Feb 16, 2017 at 09:42:36AM -0600, Tom Lendacky wrote: > Update the CPU features to include identifying and reporting on the > Secure Memory Encryption (SME) feature. SME is identified by CPUID > 0x801f, but requires BIOS support to enable it (set bit 23 of > SYS_CFG MSR). Only show the SME feature as available if reported by > CPUID and enabled by BIOS. > > Signed-off-by: Tom Lendacky > --- > arch/x86/include/asm/cpufeature.h|7 +-- > arch/x86/include/asm/cpufeatures.h |5 - > arch/x86/include/asm/disabled-features.h |3 ++- > arch/x86/include/asm/msr-index.h |2 ++ > arch/x86/include/asm/required-features.h |3 ++- > arch/x86/kernel/cpu/common.c | 19 +++ > 6 files changed, 34 insertions(+), 5 deletions(-) What happened here? You had it already: https://lkml.kernel.org/r/20161110003459.3280.25796.st...@tlendack-t1.amdoffice.net The bit in get_cpu_cap() with checking the MSR you can add at the end of init_amd() for example. -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)
Ok, this time detailed review :-) On Thu, Feb 16, 2017 at 09:42:11AM -0600, Tom Lendacky wrote: > This patch adds a Documenation entry to decribe the AMD Secure Memory > Encryption (SME) feature. Please introduce a spellchecker into your patch creation workflow. I see two typos in one line. Also, never start patch commit messages with "This patch" - we know it is this patch. Always write a doer-sentences explaining the why, not the what. Something like: "Add a SME and mem_encrypt= kernel parameter documentation." for example. > Signed-off-by: Tom Lendacky > --- > Documentation/admin-guide/kernel-parameters.txt | 11 > Documentation/x86/amd-memory-encryption.txt | 57 > +++ > 2 files changed, 68 insertions(+) > create mode 100644 Documentation/x86/amd-memory-encryption.txt > > diff --git a/Documentation/admin-guide/kernel-parameters.txt > b/Documentation/admin-guide/kernel-parameters.txt > index 110745e..91c40fa 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -2145,6 +2145,17 @@ > memory contents and reserves bad memory > regions that are detected. > > + mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control > + Valid arguments: on, off > + Default (depends on kernel configuration option): > + on (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) > + off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n) > + mem_encrypt=on: Activate SME > + mem_encrypt=off:Do not activate SME > + > + Refer to the SME documentation for details on when "Refer to Documentation/x86/amd-memory-encryption.txt .." > + memory encryption can be activated. > + > mem_sleep_default= [SUSPEND] Default system suspend mode: > s2idle - Suspend-To-Idle > shallow - Power-On Suspend or equivalent (if supported) > diff --git a/Documentation/x86/amd-memory-encryption.txt > b/Documentation/x86/amd-memory-encryption.txt > new file mode 100644 > index 000..0938e89 > --- /dev/null > +++ b/Documentation/x86/amd-memory-encryption.txt > @@ -0,0 +1,57 @@ > +Secure Memory Encryption (SME) is a feature found on AMD processors. > + > +SME provides the ability to mark individual pages of memory as encrypted > using > +the standard x86 page tables. A page that is marked encrypted will be > +automatically decrypted when read from DRAM and encrypted when written to > +DRAM. SME can therefore be used to protect the contents of DRAM from > physical > +attacks on the system. > + > +A page is encrypted when a page table entry has the encryption bit set (see > +below how to determine the position of the bit). The encryption bit can be "... how to determine its position)." > +specified in the cr3 register, allowing the PGD table to be encrypted. Each > +successive level of page tables can also be encrypted. > + > +Support for SME can be determined through the CPUID instruction. The CPUID > +function 0x801f reports information related to SME: > + > + 0x801f[eax]: > + Bit[0] indicates support for SME > + 0x801f[ebx]: > + Bit[5:0] pagetable bit number used to activate memory > + encryption s/Bit/Bits/ > + Bit[11:6] reduction in physical address space, in bits, when Ditto. > + memory encryption is enabled (this only affects system > + physical addresses, not guest physical addresses) > + > +If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to Let's use the kernel's define name MSR_K8_SYSCFG to avoid ambiguity. > +determine if SME is enabled and/or to enable memory encryption: > + > + 0xc0010010: > + Bit[23] 0 = memory encryption features are disabled > + 1 = memory encryption features are enabled > + > +Linux relies on BIOS to set this bit if BIOS has determined that the > reduction > +in the physical address space as a result of enabling memory encryption (see > +CPUID information above) will not conflict with the address space resource > +requirements for the system. If this bit is not set upon Linux startup then > +Linux itself will not set it and memory encryption will not be possible. > + > +The state of SME in the Linux kernel can be documented as follows: > + - Supported: > + The CPU supports SME (determined through CPUID instruction). > + > + - Enabled: > + Supported and bit 23 of the SYS_CFG MSR is set. Ditto. > + > + - Active: > + Supported, Enabled and the Linux kernel is actively applying > + the encryption bit to page table entries (the SME mask in the > + kernel is non-zero). > + > +SME can also be enabled and
[RFC PATCH v4 26/28] x86: Allow kexec to be used with SME
Provide support so that kexec can be used to boot a kernel when SME is
enabled.
Support is needed to allocate pages for kexec without encryption. This
is needed in order to be able to reboot in the kernel in the same manner
as originally booted.
Additionally, when shutting down all of the CPUs we need to be sure to
disable caches, flush the caches and then halt. This is needed when booting
from a state where SME was not active into a state where SME is active.
Without these steps, it is possible for cache lines to exist for the same
physical location but tagged both with and without the encryption bit. This
can cause random memory corruption when caches are flushed depending on
which cacheline is written last.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/cacheflush.h|2 ++
arch/x86/include/asm/init.h |1 +
arch/x86/include/asm/mem_encrypt.h | 10
arch/x86/include/asm/pgtable_types.h |1 +
arch/x86/kernel/machine_kexec_64.c |3 ++
arch/x86/kernel/process.c| 43 +-
arch/x86/kernel/smp.c|4 ++-
arch/x86/mm/ident_map.c |6 +++--
arch/x86/mm/pageattr.c |2 ++
include/linux/mem_encrypt.h | 10
kernel/kexec_core.c | 24 +++
11 files changed, 100 insertions(+), 6 deletions(-)
diff --git a/arch/x86/include/asm/cacheflush.h
b/arch/x86/include/asm/cacheflush.h
index 33ae60a..2180cd5 100644
--- a/arch/x86/include/asm/cacheflush.h
+++ b/arch/x86/include/asm/cacheflush.h
@@ -48,8 +48,10 @@
int set_memory_rw(unsigned long addr, int numpages);
int set_memory_np(unsigned long addr, int numpages);
int set_memory_4k(unsigned long addr, int numpages);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
int set_memory_encrypted(unsigned long addr, int numpages);
int set_memory_decrypted(unsigned long addr, int numpages);
+#endif
int set_memory_array_uc(unsigned long *addr, int addrinarray);
int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index 737da62..b2ec511 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -6,6 +6,7 @@ struct x86_mapping_info {
void *context; /* context for alloc_pgt_page */
unsigned long pmd_flag; /* page flag for PMD entry */
unsigned long offset;/* ident mapping offset */
+ unsigned long kernpg_flag; /* kernel pagetable flag override */
};
int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index 5a17f1b..1fd5426 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -64,6 +64,16 @@ static inline u64 sme_dma_mask(void)
return 0ULL;
}
+static inline int set_memory_encrypted(unsigned long vaddr, int numpages)
+{
+ return 0;
+}
+
+static inline int set_memory_decrypted(unsigned long vaddr, int numpages)
+{
+ return 0;
+}
+
#endif
static inline void __init sme_early_encrypt(resource_size_t paddr,
diff --git a/arch/x86/include/asm/pgtable_types.h
b/arch/x86/include/asm/pgtable_types.h
index f00e70f..456c5cc 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -213,6 +213,7 @@ enum page_cache_mode {
#define PAGE_KERNEL__pgprot(__PAGE_KERNEL | _PAGE_ENC)
#define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
#define PAGE_KERNEL_EXEC __pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC_NOENC __pgprot(__PAGE_KERNEL_EXEC)
#define PAGE_KERNEL_RX __pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
#define PAGE_KERNEL_NOCACHE__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
#define PAGE_KERNEL_LARGE __pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
diff --git a/arch/x86/kernel/machine_kexec_64.c
b/arch/x86/kernel/machine_kexec_64.c
index 307b1f4..b01648c 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -76,7 +76,7 @@ static int init_transition_pgtable(struct kimage *image,
pgd_t *pgd)
set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
}
pte = pte_offset_kernel(pmd, vaddr);
- set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
+ set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
return 0;
err:
free_transition_pgtable(image);
@@ -104,6 +104,7 @@ static int init_pgtable(struct kimage *image, unsigned long
start_pgtable)
.alloc_pgt_page = alloc_pgt_page,
.context= image,
.pmd_flag = __PAGE_KERNEL_LARGE_EXEC,
+ .kernpg_flag= _KERNPG_TABLE_NOENC,
};
unsigned long mstart, mend;
pgd_t *level4p;
diff --git a/arch/x86/kernel/process.c b/arch/x86/ker
[RFC PATCH v4 01/28] x86: Documentation for AMD Secure Memory Encryption (SME)
This patch adds a Documenation entry to decribe the AMD Secure Memory Encryption (SME) feature. Signed-off-by: Tom Lendacky --- Documentation/admin-guide/kernel-parameters.txt | 11 Documentation/x86/amd-memory-encryption.txt | 57 +++ 2 files changed, 68 insertions(+) create mode 100644 Documentation/x86/amd-memory-encryption.txt diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 110745e..91c40fa 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2145,6 +2145,17 @@ memory contents and reserves bad memory regions that are detected. + mem_encrypt=[X86-64] AMD Secure Memory Encryption (SME) control + Valid arguments: on, off + Default (depends on kernel configuration option): + on (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) + off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n) + mem_encrypt=on: Activate SME + mem_encrypt=off:Do not activate SME + + Refer to the SME documentation for details on when + memory encryption can be activated. + mem_sleep_default= [SUSPEND] Default system suspend mode: s2idle - Suspend-To-Idle shallow - Power-On Suspend or equivalent (if supported) diff --git a/Documentation/x86/amd-memory-encryption.txt b/Documentation/x86/amd-memory-encryption.txt new file mode 100644 index 000..0938e89 --- /dev/null +++ b/Documentation/x86/amd-memory-encryption.txt @@ -0,0 +1,57 @@ +Secure Memory Encryption (SME) is a feature found on AMD processors. + +SME provides the ability to mark individual pages of memory as encrypted using +the standard x86 page tables. A page that is marked encrypted will be +automatically decrypted when read from DRAM and encrypted when written to +DRAM. SME can therefore be used to protect the contents of DRAM from physical +attacks on the system. + +A page is encrypted when a page table entry has the encryption bit set (see +below how to determine the position of the bit). The encryption bit can be +specified in the cr3 register, allowing the PGD table to be encrypted. Each +successive level of page tables can also be encrypted. + +Support for SME can be determined through the CPUID instruction. The CPUID +function 0x801f reports information related to SME: + + 0x801f[eax]: + Bit[0] indicates support for SME + 0x801f[ebx]: + Bit[5:0] pagetable bit number used to activate memory + encryption + Bit[11:6] reduction in physical address space, in bits, when + memory encryption is enabled (this only affects system + physical addresses, not guest physical addresses) + +If support for SME is present, MSR 0xc00100010 (SYS_CFG) can be used to +determine if SME is enabled and/or to enable memory encryption: + + 0xc0010010: + Bit[23] 0 = memory encryption features are disabled + 1 = memory encryption features are enabled + +Linux relies on BIOS to set this bit if BIOS has determined that the reduction +in the physical address space as a result of enabling memory encryption (see +CPUID information above) will not conflict with the address space resource +requirements for the system. If this bit is not set upon Linux startup then +Linux itself will not set it and memory encryption will not be possible. + +The state of SME in the Linux kernel can be documented as follows: + - Supported: + The CPU supports SME (determined through CPUID instruction). + + - Enabled: + Supported and bit 23 of the SYS_CFG MSR is set. + + - Active: + Supported, Enabled and the Linux kernel is actively applying + the encryption bit to page table entries (the SME mask in the + kernel is non-zero). + +SME can also be enabled and activated in the BIOS. If SME is enabled and +activated in the BIOS, then all memory accesses will be encrypted and it will +not be necessary to activate the Linux memory encryption support. If the BIOS +merely enables SME (sets bit 23 of the SYS_CFG MSR), then Linux can activate +memory encryption. However, if BIOS does not enable SME, then Linux will not +attempt to activate memory encryption, even if configured to do so by default +or the mem_encrypt=on command line parameter is specified. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 04/28] x86: Handle reduction in physical address size with SME
When System Memory Encryption (SME) is enabled, the physical address space is reduced. Adjust the x86_phys_bits value to reflect this reduction. Signed-off-by: Tom Lendacky --- arch/x86/kernel/cpu/common.c | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index b33bc06..358208d7 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -771,11 +771,15 @@ void get_cpu_cap(struct cpuinfo_x86 *c) u64 msr; /* -* For SME, BIOS support is required. If BIOS has not -* enabled SME don't advertise the feature. +* For SME, BIOS support is required. If BIOS has +* enabled SME adjust x86_phys_bits by the SME +* physical address space reduction value. If BIOS +* has not enabled SME don't advertise the feature. */ rdmsrl(MSR_K8_SYSCFG, msr); - if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT)) + if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT) + c->x86_phys_bits -= (ebx >> 6) & 0x3f; + else eax &= ~0x01; } -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 08/28] x86: Extend the early_memremap support with additional attrs
Add to the early_memremap support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.
Signed-off-by: Tom Lendacky
---
arch/x86/Kconfig |4 +++
arch/x86/include/asm/fixmap.h| 13 ++
arch/x86/include/asm/pgtable_types.h |8 ++
arch/x86/mm/ioremap.c| 44 ++
include/asm-generic/early_ioremap.h |2 ++
mm/early_ioremap.c | 10
6 files changed, 81 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index a3b8c71..581eae4 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1417,6 +1417,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
If set to N, then the encryption of system memory can be
activated with the mem_encrypt=on command line option.
+config ARCH_USE_MEMREMAP_PROT
+ def_bool y
+ depends on AMD_MEM_ENCRYPT
+
# Common NUMA Features
config NUMA
bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 83e91f0..8233373 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -160,6 +160,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
*/
#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+ unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+ unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+unsigned long size);
+
#include
#define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h
b/arch/x86/include/asm/pgtable_types.h
index 500fc60..f00e70f 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
#define _PAGE_CACHE_MASK (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
#define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP (cachemode2protval(_PAGE_CACHE_MODE_WP))
#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
#define PAGE_SHARED__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
#define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER)
#define __PAGE_KERNEL_LARGE(__PAGE_KERNEL | _PAGE_PSE)
#define __PAGE_KERNEL_LARGE_EXEC (__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP (__PAGE_KERNEL | _PAGE_CACHE_WP)
#define __PAGE_KERNEL_IO (__PAGE_KERNEL)
#define __PAGE_KERNEL_IO_NOCACHE (__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
#define _KERNPG_TABLE (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |\
_PAGE_DIRTY | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC (__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP (__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP (__PAGE_KERNEL_WP)
+
#define PAGE_KERNEL__pgprot(__PAGE_KERNEL | _PAGE_ENC)
#define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
#define PAGE_KERNEL_EXEC __pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index c43b6b3..2385e70 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -419,6 +419,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
}
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+ unsigned long size)
+{
+ return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+unsigned long size)
+{
+ /* Be sure the write-protect PAT entry is set for write-protect */
+ if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+
[RFC PATCH v4 07/28] x86: Provide general kernel support for memory encryption
Adding general kernel support for memory encryption includes:
- Modify and create some page table macros to include the Secure Memory
Encryption (SME) memory encryption mask
- Modify and create some macros for calculating physical and virtual
memory addresses
- Provide an SME initialization routine to update the protection map with
the memory encryption mask so that it is used by default
- #undef CONFIG_AMD_MEM_ENCRYPT in the compressed boot path
Signed-off-by: Tom Lendacky
---
arch/x86/boot/compressed/pagetable.c |7 +
arch/x86/include/asm/fixmap.h|7 +
arch/x86/include/asm/mem_encrypt.h | 14 +++
arch/x86/include/asm/page.h |4 ++-
arch/x86/include/asm/pgtable.h | 26 ++--
arch/x86/include/asm/pgtable_types.h | 45 ++
arch/x86/include/asm/processor.h |3 ++
arch/x86/kernel/espfix_64.c |2 +-
arch/x86/kernel/head64.c | 12 -
arch/x86/kernel/head_64.S| 18 +++---
arch/x86/mm/kasan_init_64.c |4 ++-
arch/x86/mm/mem_encrypt.c| 20 +++
arch/x86/mm/pageattr.c |3 ++
include/asm-generic/pgtable.h|8 ++
14 files changed, 133 insertions(+), 40 deletions(-)
diff --git a/arch/x86/boot/compressed/pagetable.c
b/arch/x86/boot/compressed/pagetable.c
index 56589d0..411c443 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
#define __pa(x) ((unsigned long)(x))
#define __va(x) ((void *)((unsigned long)(x)))
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
#include "misc.h"
/* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 8554f96..83e91f0 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -153,6 +153,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
}
#endif
+/*
+ * Fixmap settings used with memory encryption
+ * - FIXMAP_PAGE_NOCACHE is used for MMIO so make sure the memory
+ * encryption mask is not part of the page attributes
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
#include
#define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index ccc53b0..547989d 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,6 +15,8 @@
#ifndef __ASSEMBLY__
+#include
+
#ifdef CONFIG_AMD_MEM_ENCRYPT
extern unsigned long sme_me_mask;
@@ -24,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
}
+void __init sme_early_init(void);
+
+#define __sme_pa(x)(__pa((x)) | sme_me_mask)
+#define __sme_pa_nodebug(x)(__pa_nodebug((x)) | sme_me_mask)
+
#else /* !CONFIG_AMD_MEM_ENCRYPT */
#ifndef sme_me_mask
@@ -35,6 +42,13 @@ static inline bool sme_active(void)
}
#endif
+static inline void __init sme_early_init(void)
+{
+}
+
+#define __sme_pa __pa
+#define __sme_pa_nodebug __pa_nodebug
+
#endif /* CONFIG_AMD_MEM_ENCRYPT */
#endif /* __ASSEMBLY__ */
diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h
index cf8f619..b1f7bf6 100644
--- a/arch/x86/include/asm/page.h
+++ b/arch/x86/include/asm/page.h
@@ -15,6 +15,8 @@
#ifndef __ASSEMBLY__
+#include
+
struct page;
#include
@@ -55,7 +57,7 @@ static inline void copy_user_page(void *to, void *from,
unsigned long vaddr,
__phys_addr_symbol(__phys_reloc_hide((unsigned long)(x)))
#ifndef __va
-#define __va(x)((void *)((unsigned
long)(x)+PAGE_OFFSET))
+#define __va(x)((void *)(((unsigned long)(x) &
~sme_me_mask) + PAGE_OFFSET))
#endif
#define __boot_va(x) __va(x)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 2d81161..b41caab 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -3,6 +3,7 @@
#include
#include
+#include
/*
* Macro to mark a page protection value as UC-
@@ -13,6 +14,12 @@
cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \
: (prot))
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot) __pgprot(pgprot_val(prot) | sme_me_mask)
+#define pgprot_decrypted(prot) __pgprot(pgprot_val(prot) & ~sme_me_mask)
+
#ifndef __ASSEMBLY__
#include
@@ -153,17 +160,22 @@ static inline int pte_special(pte_t pte)
static inline unsigned long pte_pfn(pte_t pte)
{
- return (pte_val(pte) & PTE_PFN_MASK) >> PAGE_SHIFT;
+ r
[RFC PATCH v4 06/28] x86: Add support to enable SME during early boot processing
This patch adds support to the early boot code to use Secure Memory
Encryption (SME). Support is added to update the early pagetables with
the memory encryption mask and to encrypt the kernel in place.
The routines to set the encryption mask and perform the encryption are
stub routines for now with full function to be added in a later patch.
A new file, arch/x86/kernel/mem_encrypt_init.c, is introduced to avoid
adding #ifdefs within arch/x86/kernel/head_64.S and allow
arch/x86/mm/mem_encrypt.c to be removed from the build if SME is not
configured. The mem_encrypt_init.c file will contain the necessary #ifdefs
to allow head_64.S to successfully build and call the SME routines.
Signed-off-by: Tom Lendacky
---
arch/x86/kernel/Makefile |2 +
arch/x86/kernel/head_64.S | 46 -
arch/x86/kernel/mem_encrypt_init.c | 50
3 files changed, 96 insertions(+), 2 deletions(-)
create mode 100644 arch/x86/kernel/mem_encrypt_init.c
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index bdcdb3b..33af80a 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -140,4 +140,6 @@ ifeq ($(CONFIG_X86_64),y)
obj-$(CONFIG_PCI_MMCONFIG) += mmconf-fam10h_64.o
obj-y += vsmp_64.o
+
+ obj-y += mem_encrypt_init.o
endif
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index b467b14..4f8201b 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -91,6 +91,23 @@ startup_64:
jnz bad_address
/*
+* Enable Secure Memory Encryption (SME), if supported and enabled.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+* Save the returned mask in %r12 for later use.
+*/
+ push%rsi
+ callsme_enable
+ pop %rsi
+ movq%rax, %r12
+
+ /*
+* Add the memory encryption mask to %rbp to include it in the page
+* table fixups.
+*/
+ addq%r12, %rbp
+
+ /*
* Fixup the physical addresses in the page table
*/
addq%rbp, early_level4_pgt + (L4_START_KERNEL*8)(%rip)
@@ -113,6 +130,7 @@ startup_64:
shrq$PGDIR_SHIFT, %rax
leaq(PAGE_SIZE + _KERNPG_TABLE)(%rbx), %rdx
+ addq%r12, %rdx
movq%rdx, 0(%rbx,%rax,8)
movq%rdx, 8(%rbx,%rax,8)
@@ -129,6 +147,7 @@ startup_64:
movq%rdi, %rax
shrq$PMD_SHIFT, %rdi
addq$(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL), %rax
+ addq%r12, %rax
leaq(_end - 1)(%rip), %rcx
shrq$PMD_SHIFT, %rcx
subq%rdi, %rcx
@@ -162,11 +181,25 @@ startup_64:
cmp %r8, %rdi
jne 1b
- /* Fixup phys_base */
+ /*
+* Fixup phys_base - remove the memory encryption mask from %rbp
+* to obtain the true physical address.
+*/
+ subq%r12, %rbp
addq%rbp, phys_base(%rip)
+ /*
+* Encrypt the kernel if SME is active.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+*/
+ push%rsi
+ callsme_encrypt_kernel
+ pop %rsi
+
.Lskip_fixup:
movq$(early_level4_pgt - __START_KERNEL_map), %rax
+ addq%r12, %rax
jmp 1f
ENTRY(secondary_startup_64)
/*
@@ -186,7 +219,16 @@ ENTRY(secondary_startup_64)
/* Sanitize CPU configuration */
call verify_cpu
- movq$(init_level4_pgt - __START_KERNEL_map), %rax
+ /*
+* Get the SME encryption mask.
+* The real_mode_data address is in %rsi and that register can be
+* clobbered by the called function so be sure to save it.
+*/
+ push%rsi
+ callsme_get_me_mask
+ pop %rsi
+
+ addq$(init_level4_pgt - __START_KERNEL_map), %rax
1:
/* Enable PAE mode and PGE */
diff --git a/arch/x86/kernel/mem_encrypt_init.c
b/arch/x86/kernel/mem_encrypt_init.c
new file mode 100644
index 000..25af15d
--- /dev/null
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -0,0 +1,50 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include
+#include
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+#include
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+unsigned long __init sme_get_me_mask(void)
+{
+ return sme_me_mask;
+}
+
+unsigned long __init sme_enable(void)
+{
+ return sme_me_mask;
+}
+
+#else /* !
[RFC PATCH v4 03/28] x86: Add the Secure Memory Encryption CPU feature
Update the CPU features to include identifying and reporting on the Secure Memory Encryption (SME) feature. SME is identified by CPUID 0x801f, but requires BIOS support to enable it (set bit 23 of SYS_CFG MSR). Only show the SME feature as available if reported by CPUID and enabled by BIOS. Signed-off-by: Tom Lendacky --- arch/x86/include/asm/cpufeature.h|7 +-- arch/x86/include/asm/cpufeatures.h |5 - arch/x86/include/asm/disabled-features.h |3 ++- arch/x86/include/asm/msr-index.h |2 ++ arch/x86/include/asm/required-features.h |3 ++- arch/x86/kernel/cpu/common.c | 19 +++ 6 files changed, 34 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h index d59c15c..ea2de6a 100644 --- a/arch/x86/include/asm/cpufeature.h +++ b/arch/x86/include/asm/cpufeature.h @@ -28,6 +28,7 @@ enum cpuid_leafs CPUID_8000_000A_EDX, CPUID_7_ECX, CPUID_8000_0007_EBX, + CPUID_8000_001F_EAX, }; #ifdef CONFIG_X86_FEATURE_NAMES @@ -78,8 +79,9 @@ enum cpuid_leafs CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 15, feature_bit) ||\ CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 16, feature_bit) ||\ CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 17, feature_bit) ||\ + CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 18, feature_bit) ||\ REQUIRED_MASK_CHECK||\ - BUILD_BUG_ON_ZERO(NCAPINTS != 18)) + BUILD_BUG_ON_ZERO(NCAPINTS != 19)) #define DISABLED_MASK_BIT_SET(feature_bit) \ ( CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 0, feature_bit) ||\ @@ -100,8 +102,9 @@ enum cpuid_leafs CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 15, feature_bit) ||\ CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 16, feature_bit) ||\ CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 17, feature_bit) ||\ + CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 18, feature_bit) ||\ DISABLED_MASK_CHECK||\ - BUILD_BUG_ON_ZERO(NCAPINTS != 18)) + BUILD_BUG_ON_ZERO(NCAPINTS != 19)) #define cpu_has(c, bit) \ (__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 : \ diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index d45ab4b..331fb81 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -12,7 +12,7 @@ /* * Defines x86 CPU feature bits */ -#define NCAPINTS 18 /* N 32-bit words worth of info */ +#define NCAPINTS 19 /* N 32-bit words worth of info */ #define NBUGINTS 1 /* N 32-bit bug flags */ /* @@ -296,6 +296,9 @@ #define X86_FEATURE_SUCCOR (17*32+1) /* Uncorrectable error containment and recovery */ #define X86_FEATURE_SMCA (17*32+3) /* Scalable MCA */ +/* AMD-defined CPU features, CPUID level 0x801f (eax), word 18 */ +#define X86_FEATURE_SME(18*32+0) /* Secure Memory Encryption */ + /* * BUG word(s) */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 85599ad..8b45e08 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -57,6 +57,7 @@ #define DISABLED_MASK150 #define DISABLED_MASK16(DISABLE_PKU|DISABLE_OSPKE) #define DISABLED_MASK170 -#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18) +#define DISABLED_MASK180 +#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19) #endif /* _ASM_X86_DISABLED_FEATURES_H */ diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 00293a9..e2d0503 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -339,6 +339,8 @@ #define MSR_K8_TOP_MEM10xc001001a #define MSR_K8_TOP_MEM20xc001001d #define MSR_K8_SYSCFG 0xc0010010 +#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT 23 +#define MSR_K8_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT) #define MSR_K8_INT_PENDING_MSG 0xc0010055 /* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK0x1800 diff --git a/arch/x86/include/asm/required-features.h b/arch/x86/include/asm/required-features.h index fac9a5c..6847d85 100644 --- a/arch/x86/include/asm/required-features.h +++ b/arch/x86/include/asm/required-features.h @@ -100,6 +100,7 @@ #define REQUIRED_MASK150 #define REQUIRED_MASK160 #define REQUIRED_MASK170 -#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 18) +#define REQUIRED_MASK180 +#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19) #endif /* _ASM_X86_REQUIRED_FEATURES_H */ diff -
[RFC PATCH v4 05/28] x86: Add Secure Memory Encryption (SME) support
Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.
Signed-off-by: Tom Lendacky
---
arch/x86/Kconfig | 22 +++
arch/x86/include/asm/mem_encrypt.h | 42
arch/x86/mm/Makefile |1 +
arch/x86/mm/mem_encrypt.c | 21 ++
include/linux/mem_encrypt.h| 37
5 files changed, 123 insertions(+)
create mode 100644 arch/x86/include/asm/mem_encrypt.h
create mode 100644 arch/x86/mm/mem_encrypt.c
create mode 100644 include/linux/mem_encrypt.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f8fbfc5..a3b8c71 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1395,6 +1395,28 @@ config X86_DIRECT_GBPAGES
supports them), so don't confuse the user by printing
that we have them enabled.
+config AMD_MEM_ENCRYPT
+ bool "AMD Secure Memory Encryption (SME) support"
+ depends on X86_64 && CPU_SUP_AMD
+ ---help---
+ Say yes to enable support for the encryption of system memory.
+ This requires an AMD processor that supports Secure Memory
+ Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+ bool "Activate AMD Secure Memory Encryption (SME) by default"
+ default y
+ depends on AMD_MEM_ENCRYPT
+ ---help---
+ Say yes to have system memory encrypted by default if running on
+ an AMD processor that supports Secure Memory Encryption (SME).
+
+ If set to Y, then the encryption of system memory can be
+ deactivated with the mem_encrypt=off command line option.
+
+ If set to N, then the encryption of system memory can be
+ activated with the mem_encrypt=on command line option.
+
# Common NUMA Features
config NUMA
bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 000..ccc53b0
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,42 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+static inline bool sme_active(void)
+{
+ return (sme_me_mask) ? true : false;
+}
+
+#else /* !CONFIG_AMD_MEM_ENCRYPT */
+
+#ifndef sme_me_mask
+#define sme_me_mask0UL
+
+static inline bool sme_active(void)
+{
+ return false;
+}
+#endif
+
+#endif /* CONFIG_AMD_MEM_ENCRYPT */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 96d2b84..44d4d21 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX) += mpx.o
obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 000..14a7b9f
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,37 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+#include
+
+#else /* !CONFIG_AMD_MEM_ENCRYPT */
+
+#ifndef sme_me_mask
+#define sme_me_mask0UL
+
+static inline boo
[RFC PATCH v4 09/28] x86: Add support for early encryption/decryption of memory
Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).
The early_memmap support is enhanced to specify encrypted and decrypted
mappings with and without write-protection. The use of write-protection is
necessary when encrypting data "in place". The write-protect attribute is
considered cacheable for loads, but not stores. This implies that the
hardware will never give the core a dirty line with this memtype.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/mem_encrypt.h | 15 +++
arch/x86/mm/mem_encrypt.c | 79
2 files changed, 94 insertions(+)
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index 547989d..3c9052c 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
}
+void __init sme_early_encrypt(resource_size_t paddr,
+ unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+ unsigned long size);
+
void __init sme_early_init(void);
#define __sme_pa(x)(__pa((x)) | sme_me_mask)
@@ -42,6 +47,16 @@ static inline bool sme_active(void)
}
#endif
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+ unsigned long size)
+{
+}
+
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+ unsigned long size)
+{
+}
+
static inline void __init sme_early_init(void)
{
}
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index d71df97..ac3565c 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -14,6 +14,9 @@
#include
#include
+#include
+#include
+
extern pmdval_t early_pmd_flags;
/*
@@ -24,6 +27,82 @@
unsigned long sme_me_mask __section(.data) = 0;
EXPORT_SYMBOL_GPL(sme_me_mask);
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired stated.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+ unsigned long size, bool enc)
+{
+ void *src, *dst;
+ size_t len;
+
+ if (!sme_me_mask)
+ return;
+
+ local_flush_tlb();
+ wbinvd();
+
+ /*
+* There are limited number of early mapping slots, so map (at most)
+* one page at time.
+*/
+ while (size) {
+ len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+ /*
+* Create write protected mappings for the current format
+* of the memory.
+*/
+ src = enc ? early_memremap_decrypted_wp(paddr, len) :
+ early_memremap_encrypted_wp(paddr, len);
+
+ /*
+* Create mappings for the desired format of the memory.
+*/
+ dst = enc ? early_memremap_encrypted(paddr, len) :
+ early_memremap_decrypted(paddr, len);
+
+ /*
+* If a mapping can't be obtained to perform the operation,
+* then eventual access of that area will in the desired
+* mode will cause a crash.
+*/
+ BUG_ON(!src || !dst);
+
+ /*
+* Use a temporary buffer, of cache-line multiple size, to
+* avoid data corruption as documented in the APM.
+*/
+ memcpy(sme_early_buffer, src, len);
+ memcpy(dst, sme_early_buffer, len);
+
+ early_memunmap(dst, len);
+ early_memunmap(src, len);
+
+ paddr += len;
+ size -= len;
+ }
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+ __sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+ __sme_early_enc_dec(paddr, size, false);
+}
+
void __init sme_early_init(void)
{
uns
[RFC PATCH v4 13/28] efi: Update efi_mem_type() to return defined EFI mem types
Update the efi_mem_type() to return EFI_RESERVED_TYPE instead of a
hardcoded 0.
Signed-off-by: Tom Lendacky
---
arch/x86/platform/efi/efi.c |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index a15cf81..6407103 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1037,7 +1037,7 @@ u32 efi_mem_type(unsigned long phys_addr)
efi_memory_desc_t *md;
if (!efi_enabled(EFI_MEMMAP))
- return 0;
+ return EFI_RESERVED_TYPE;
for_each_efi_memory_desc(md) {
if ((md->phys_addr <= phys_addr) &&
@@ -1045,7 +1045,7 @@ u32 efi_mem_type(unsigned long phys_addr)
(md->num_pages << EFI_PAGE_SHIFT
return md->type;
}
- return 0;
+ return EFI_RESERVED_TYPE;
}
static int __init arch_parse_efi_cmdline(char *str)
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 12/28] efi: Add an EFI table address match function
This patch adds support that will determine if a supplied physical address
matches the address of an EFI table.
Signed-off-by: Tom Lendacky
---
drivers/firmware/efi/efi.c | 33 +
include/linux/efi.h|7 +++
2 files changed, 40 insertions(+)
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index e7d4040..e8dbcdf 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
};
EXPORT_SYMBOL(efi);
+static unsigned long *efi_tables[] = {
+ &efi.mps,
+ &efi.acpi,
+ &efi.acpi20,
+ &efi.smbios,
+ &efi.smbios3,
+ &efi.sal_systab,
+ &efi.boot_info,
+ &efi.hcdp,
+ &efi.uga,
+ &efi.uv_systab,
+ &efi.fw_vendor,
+ &efi.runtime,
+ &efi.config_table,
+ &efi.esrt,
+ &efi.properties_table,
+ &efi.mem_attr_table,
+};
+
static bool disable_runtime;
static int __init setup_noefi(char *arg)
{
@@ -855,6 +874,20 @@ int efi_status_to_err(efi_status_t status)
return err;
}
+bool efi_table_address_match(unsigned long phys_addr)
+{
+ unsigned int i;
+
+ if (phys_addr == EFI_INVALID_TABLE_ADDR)
+ return false;
+
+ for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+ if (*(efi_tables[i]) == phys_addr)
+ return true;
+
+ return false;
+}
+
#ifdef CONFIG_KEXEC
static int update_efi_random_seed(struct notifier_block *nb,
unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 94d34e0..7694b23 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1079,6 +1079,8 @@ static inline bool efi_enabled(int feature)
return test_bit(feature, &efi.flags) != 0;
}
extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_table_address_match(unsigned long phys_addr);
#else
static inline bool efi_enabled(int feature)
{
@@ -1092,6 +1094,11 @@ static inline bool efi_enabled(int feature)
{
return false;
}
+
+static inline bool efi_table_address_match(unsigned long phys_addr)
+{
+ return false;
+}
#endif
extern int efi_status_to_err(efi_status_t status);
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 10/28] x86: Insure that boot memory areas are mapped properly
The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process. The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.
For the initrd, encrypt this data in place. Since the future mapping of the
initrd area will be mapped as encrypted the data will be accessed properly.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/mem_encrypt.h | 11 +
arch/x86/kernel/head64.c | 34 +++--
arch/x86/kernel/setup.c| 10 +
arch/x86/mm/mem_encrypt.c | 74
4 files changed, 126 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index 3c9052c..e2b7364 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -31,6 +31,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
void __init sme_early_decrypt(resource_size_t paddr,
unsigned long size);
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
void __init sme_early_init(void);
#define __sme_pa(x)(__pa((x)) | sme_me_mask)
@@ -57,6 +60,14 @@ static inline void __init sme_early_decrypt(resource_size_t
paddr,
{
}
+static inline void __init sme_map_bootdata(char *real_mode_data)
+{
+}
+
+static inline void __init sme_unmap_bootdata(char *real_mode_data)
+{
+}
+
static inline void __init sme_early_init(void)
{
}
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 182a4c7..03f8e74 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -46,13 +46,18 @@ static void __init reset_early_page_tables(void)
write_cr3(__sme_pa_nodebug(early_level4_pgt));
}
+void __init __early_pgtable_flush(void)
+{
+ write_cr3(__sme_pa_nodebug(early_level4_pgt));
+}
+
/* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
{
unsigned long physaddr = address - __PAGE_OFFSET;
pgdval_t pgd, *pgd_p;
pudval_t pud, *pud_p;
- pmdval_t pmd, *pmd_p;
+ pmdval_t *pmd_p;
/* Invalid address or early pgt is done ? */
if (physaddr >= MAXMEM || read_cr3() !=
__sme_pa_nodebug(early_level4_pgt))
@@ -94,12 +99,21 @@ int __init early_make_pgtable(unsigned long address)
memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base +
_KERNPG_TABLE;
}
- pmd = (physaddr & PMD_MASK) + early_pmd_flags;
pmd_p[pmd_index(address)] = pmd;
return 0;
}
+int __init early_make_pgtable(unsigned long address)
+{
+ unsigned long physaddr = address - __PAGE_OFFSET;
+ pmdval_t pmd;
+
+ pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+ return __early_make_pgtable(address, pmd);
+}
+
/* Don't add a printk in there. printk relies on the PDA which is not
initialized
yet. */
static void __init clear_bss(void)
@@ -122,6 +136,12 @@ static void __init copy_bootdata(char *real_mode_data)
char * command_line;
unsigned long cmd_line_ptr;
+ /*
+* If SME is active, this will create decrypted mappings of the
+* boot data in advance of the copy operations.
+*/
+ sme_map_bootdata(real_mode_data);
+
memcpy(&boot_params, real_mode_data, sizeof boot_params);
sanitize_boot_params(&boot_params);
cmd_line_ptr = get_cmd_line_ptr();
@@ -129,6 +149,14 @@ static void __init copy_bootdata(char *real_mode_data)
command_line = __va(cmd_line_ptr);
memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
}
+
+ /*
+* The old boot data is no longer needed and won't be reserved,
+* freeing up that memory for use by the system. If SME is active,
+* we need to remove the mappings that were created so that the
+* memory doesn't remain mapped as decrypted.
+*/
+ sme_unmap_bootdata(real_mode_data);
}
asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index cab13f7..bd5b9a7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -114,6 +114,7 @@
#include
#include
#include
+#include
/*
* max_low_pfn_mapped: highest direct mapped pfn under 4GB
@@ -376,6 +377,15 @@ static void __init reserve_initrd(void)
!ramdisk_image || !ramdisk_size)
return; /* No initrd provided by bootloader */
+ /*
+* If SME is active, this memory will be marked encrypted by the
+* kernel when it is accessed (including relocation). However
[RFC PATCH v4 14/28] Add support to access boot related data in the clear
Boot data (such as EFI related data) is not encrypted when the system is
booted and needs to be mapped decrypted. Add support to apply the proper
attributes to the EFI page tables and to the early_memremap and memremap
APIs to identify the type of data being accessed so that the proper
encryption attribute can be applied.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/io.h |3 +
arch/x86/include/asm/setup.h |8 +++
arch/x86/kernel/setup.c| 33
arch/x86/mm/ioremap.c | 111
arch/x86/platform/efi/efi_64.c | 16 --
kernel/memremap.c | 11
mm/early_ioremap.c | 18 +-
7 files changed, 192 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..833f7cc 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,7 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
#define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
#endif
+extern bool arch_memremap_do_ram_remap(resource_size_t offset, size_t size);
+#define arch_memremap_do_ram_remap arch_memremap_do_ram_remap
+
#endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h
index ac1d5da..8d9 100644
--- a/arch/x86/include/asm/setup.h
+++ b/arch/x86/include/asm/setup.h
@@ -63,6 +63,14 @@ static inline void x86_ce4100_early_setup(void) { }
#include
#include
+struct setup_data_attrs {
+ u64 paddr;
+ unsigned long size;
+};
+
+extern struct setup_data_attrs setup_data_list[];
+extern unsigned int setup_data_list_count;
+
/*
* This is set up by the setup-routine at boot-time
*/
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index bd5b9a7..d2234bf 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -148,6 +148,9 @@ int default_check_phys_apicid_present(int phys_apicid)
struct boot_params boot_params;
+struct setup_data_attrs setup_data_list[32];
+unsigned int setup_data_list_count;
+
/*
* Machine setup..
*/
@@ -419,6 +422,32 @@ static void __init reserve_initrd(void)
}
#endif /* CONFIG_BLK_DEV_INITRD */
+static void __init update_setup_data_list(u64 pa_data, unsigned long size)
+{
+ unsigned int i;
+
+ for (i = 0; i < setup_data_list_count; i++) {
+ if (setup_data_list[i].paddr != pa_data)
+ continue;
+
+ setup_data_list[i].size = size;
+ break;
+ }
+}
+
+static void __init add_to_setup_data_list(u64 pa_data, unsigned long size)
+{
+ if (!sme_active())
+ return;
+
+ if (!WARN(setup_data_list_count == ARRAY_SIZE(setup_data_list),
+ "exceeded maximum setup data list slots")) {
+ setup_data_list[setup_data_list_count].paddr = pa_data;
+ setup_data_list[setup_data_list_count].size = size;
+ setup_data_list_count++;
+ }
+}
+
static void __init parse_setup_data(void)
{
struct setup_data *data;
@@ -428,12 +457,16 @@ static void __init parse_setup_data(void)
while (pa_data) {
u32 data_len, data_type;
+ add_to_setup_data_list(pa_data, sizeof(*data));
+
data = early_memremap(pa_data, sizeof(*data));
data_len = data->len + sizeof(struct setup_data);
data_type = data->type;
pa_next = data->next;
early_memunmap(data, sizeof(*data));
+ update_setup_data_list(pa_data, data_len);
+
switch (data_type) {
case SETUP_E820_EXT:
e820__memory_setup_extended(pa_data, data_len);
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 2385e70..b0ff6bc 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
#include
#include
#include
+#include
#include
#include
@@ -21,6 +22,7 @@
#include
#include
#include
+#include
#include "physaddr.h"
@@ -419,6 +421,115 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
}
+/*
+ * Examine the physical address to determine if it is boot data. Check
+ * it against the boot params structure and EFI tables.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+ unsigned long size)
+{
+ unsigned int i;
+ u64 paddr;
+
+ for (i = 0; i < setup_data_list_count; i++) {
+ if (phys_addr < setup_data_list[i].paddr)
+ continue;
+
+ if (phys_addr >= (setup_data_list[i].paddr +
+ setup_data_list[i].size))
+ continue;
+
+ /* Address is within setup data range */
+ return true;
+ }
+
+ paddr = boot_params.efi_info
[RFC PATCH v4 18/28] x86: DMA support for memory encryption
Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/dma-mapping.h |5 ++-
arch/x86/include/asm/mem_encrypt.h |5 +++
arch/x86/kernel/pci-dma.c | 11 +--
arch/x86/kernel/pci-nommu.c|2 +
arch/x86/kernel/pci-swiotlb.c |8 -
arch/x86/mm/mem_encrypt.c | 22 ++
include/linux/swiotlb.h|1 +
init/main.c| 13
lib/swiotlb.c | 56 +++-
9 files changed, 106 insertions(+), 17 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h
b/arch/x86/include/asm/dma-mapping.h
index 4446162..c9cdcae 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
#include
#include
#include
+#include
#ifdef CONFIG_ISA
# define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -69,12 +70,12 @@ static inline bool dma_capable(struct device *dev,
dma_addr_t addr, size_t size)
static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
{
- return paddr;
+ return paddr | sme_me_mask;
}
static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
{
- return daddr;
+ return daddr & ~sme_me_mask;
}
#endif /* CONFIG_X86_DMA_REMAP */
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index e2b7364..87e816f 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -36,6 +36,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
void __init sme_early_init(void);
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
#define __sme_pa(x)(__pa((x)) | sme_me_mask)
#define __sme_pa_nodebug(x)(__pa_nodebug((x)) | sme_me_mask)
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index d30c377..0ce28df 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -92,9 +92,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t
size,
/* CMA can be used only in the context which permits sleeping */
if (gfpflags_allow_blocking(flag)) {
page = dma_alloc_from_contiguous(dev, count, get_order(size));
- if (page && page_to_phys(page) + size > dma_mask) {
- dma_release_from_contiguous(dev, page, count);
- page = NULL;
+ if (page) {
+ addr = phys_to_dma(dev, page_to_phys(page));
+ if (addr + size > dma_mask) {
+ dma_release_from_contiguous(dev, page, count);
+ page = NULL;
+ }
}
}
/* fallback */
@@ -103,7 +106,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t
size,
if (!page)
return NULL;
- addr = page_to_phys(page);
+ addr = phys_to_dma(dev, page_to_phys(page));
if (addr + size > dma_mask) {
__free_pages(page, get_order(size));
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index 00e71ce..922c10d 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct
page *page,
enum dma_data_direction dir,
unsigned long attrs)
{
- dma_addr_t bus = page_to_phys(page) + offset;
+ dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
WARN_ON(size == 0);
if (!check_addr("map_single", dev, bus, size))
return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 410efb2..a0677a9 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+
int swiotlb __read_mostly;
void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -64,11 +66,13 @@ void x86_swiotlb_free_coherent(struct device *dev, size_t
size,
* pci_swiotlb_detect_override - set swiotlb to 1 if necessary
*
* This returns non-zero if we are forced to use swiotlb (by the boot
- * option).
+ * option). If memory encryption is enabled then swiotlb will be set
+ * to 1 so that bounce buffers are allocated and used for devices that
+ * do not support the addressing range required for the encryption mask.
*/
int __init pci_swiotlb_detect_override(void)
{
- if (swiotlb_force == SWIOTLB_FORCE)
[RFC PATCH v4 20/28] iommu/amd: Disable AMD IOMMU if memory encryption is active
For now, disable the AMD IOMMU if memory encryption is active. A future
patch will re-enable the function with full memory encryption support.
Signed-off-by: Tom Lendacky
---
drivers/iommu/amd_iommu_init.c |7 +++
1 file changed, 7 insertions(+)
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 6799cf9..6df2dd5 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -2544,6 +2545,12 @@ int __init amd_iommu_detect(void)
if (amd_iommu_disabled)
return -ENODEV;
+ /* For now, disable the IOMMU if SME is active */
+ if (sme_active()) {
+ pr_notice("AMD-Vi: SME is active, disabling the IOMMU\n");
+ return -ENODEV;
+ }
+
ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
if (ret)
return ret;
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 28/28] x86: Add support to make use of Secure Memory Encryption
This patch adds the support to check if SME has been enabled and if
memory encryption should be activated (checking of command line option
based on the configuration of the default state). If memory encryption
is to be activated, then the encryption mask is set and the kernel is
encrypted "in place."
Signed-off-by: Tom Lendacky
---
arch/x86/kernel/head_64.S |1 +
arch/x86/kernel/mem_encrypt_init.c | 71 +++-
arch/x86/mm/mem_encrypt.c |2 +
3 files changed, 73 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index edd2f14..e6820e7 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -97,6 +97,7 @@ startup_64:
* Save the returned mask in %r12 for later use.
*/
push%rsi
+ movq%rsi, %rdi
callsme_enable
pop %rsi
movq%rax, %r12
diff --git a/arch/x86/kernel/mem_encrypt_init.c
b/arch/x86/kernel/mem_encrypt_init.c
index 07cbb90..35c5e3d 100644
--- a/arch/x86/kernel/mem_encrypt_init.c
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -19,6 +19,12 @@
#include
#include
+#include
+#include
+#include
+
+static char sme_cmdline_arg_on[] __initdata = "mem_encrypt=on";
+static char sme_cmdline_arg_off[] __initdata = "mem_encrypt=off";
extern void sme_encrypt_execute(unsigned long, unsigned long, unsigned long,
void *, pgd_t *);
@@ -217,8 +223,71 @@ unsigned long __init sme_get_me_mask(void)
return sme_me_mask;
}
-unsigned long __init sme_enable(void)
+unsigned long __init sme_enable(void *boot_data)
{
+ struct boot_params *bp = boot_data;
+ unsigned int eax, ebx, ecx, edx;
+ unsigned long cmdline_ptr;
+ bool enable_if_found;
+ void *cmdline_arg;
+ u64 msr;
+
+ /* Check for an AMD processor */
+ eax = 0;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if ((ebx != 0x68747541) || (edx != 0x69746e65) || (ecx != 0x444d4163))
+ goto out;
+
+ /* Check for the SME support leaf */
+ eax = 0x8000;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (eax < 0x801f)
+ goto out;
+
+ /*
+* Check for the SME feature:
+* CPUID Fn8000_001F[EAX] - Bit 0
+* Secure Memory Encryption support
+* CPUID Fn8000_001F[EBX] - Bits 5:0
+* Pagetable bit position used to indicate encryption
+*/
+ eax = 0x801f;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (!(eax & 1))
+ goto out;
+
+ /* Check if SME is enabled */
+ msr = native_read_msr(MSR_K8_SYSCFG);
+ if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+ goto out;
+
+ /*
+* Fixups have not been to applied phys_base yet, so we must obtain
+* the address to the SME command line option in the following way.
+*/
+ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT)) {
+ asm ("lea sme_cmdline_arg_off(%%rip), %0"
+: "=r" (cmdline_arg)
+: "p" (sme_cmdline_arg_off));
+ enable_if_found = false;
+ } else {
+ asm ("lea sme_cmdline_arg_on(%%rip), %0"
+: "=r" (cmdline_arg)
+: "p" (sme_cmdline_arg_on));
+ enable_if_found = true;
+ }
+
+ cmdline_ptr = bp->hdr.cmd_line_ptr | ((u64)bp->ext_cmd_line_ptr << 32);
+
+ if (cmdline_find_option_bool((char *)cmdline_ptr, cmdline_arg))
+ sme_me_mask = enable_if_found ? 1UL << (ebx & 0x3f) : 0;
+ else
+ sme_me_mask = enable_if_found ? 0 : 1UL << (ebx & 0x3f);
+
+out:
return sme_me_mask;
}
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index a46bcf4..c5062e1 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -204,6 +204,8 @@ void __init mem_encrypt_init(void)
/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
swiotlb_update_mem_attributes();
+
+ pr_info("AMD Secure Memory Encryption (SME) active\n");
}
void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 22/28] x86: Do not specify encrypted memory for video mappings
Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/vga.h | 13 +
drivers/gpu/drm/drm_gem.c|2 ++
drivers/gpu/drm/drm_vm.c |4
drivers/gpu/drm/ttm/ttm_bo_vm.c |7 +--
drivers/gpu/drm/udl/udl_fb.c |4
drivers/video/fbdev/core/fbmem.c | 12
6 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..5c7567a 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,25 @@
#ifndef _ASM_X86_VGA_H
#define _ASM_X86_VGA_H
+#include
+
/*
* On the PC, we can just recalculate addresses and then
* access the videoram directly without any black magic.
+ * To support memory encryption however, we need to access
+ * the videoram as decrypted memory.
*/
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+#define VGA_MAP_MEM(x, s) \
+({ \
+ unsigned long start = (unsigned long)phys_to_virt(x); \
+ set_memory_decrypted(start, (s) >> PAGE_SHIFT); \
+ start; \
+})
+#else
#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#endif
#define vga_readb(x) (*(x))
#define vga_writeb(x, y) (*(y) = (x))
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 465bacd..f9b3be0 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned
long obj_size,
vma->vm_ops = dev->driver->gem_vm_ops;
vma->vm_private_data = obj;
vma->vm_page_prot =
pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+ vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
/* Take a ref for this mapping of the object, so that the fault
* handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index bd311c7..f0fc52a 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
#include
#include
#endif
+#include
#include
#include "drm_internal.h"
#include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
{
pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
+ /* We don't want graphics memory to be mapped encrypted */
+ tmp = pgprot_decrypted(tmp);
+
#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 68ef993..09f2e73 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
#include
#include
#include
+#include
#define TTM_BO_VM_NUM_PREFAULT 16
@@ -218,9 +219,11 @@ static int ttm_bo_vm_fault(struct vm_area_struct *vma,
struct vm_fault *vmf)
* first page.
*/
for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
- if (bo->mem.bus.is_iomem)
+ if (bo->mem.bus.is_iomem) {
+ /* Iomem should not be marked encrypted */
+ cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
pfn = ((bo->mem.bus.base + bo->mem.bus.offset) >>
PAGE_SHIFT) + page_offset;
- else {
+ } else {
page = ttm->pages[page_offset];
if (unlikely(!page && i == 0)) {
retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 167f42c..2207ec0 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
#include
#include
#include
+#include
#include
#include
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct
vm_area_struct *vma)
pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
pos, size);
+ /* We don't want the framebuffer to be mapped encrypted */
+ vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
while (size > 0) {
page = vmalloc_to_pfn((void *)pos);
if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 76c1ad9..b895e60 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
#include
#include
#include
+#include
#include
@@ -1405,6 +1406,12 @@ s
[RFC PATCH v4 00/28] x86: Secure Memory Encryption (AMD)
This RFC patch series provides support for AMD's new Secure Memory
Encryption (SME) feature.
SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.
The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.
The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.
This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.
The following links provide additional detail:
AMD Memory Encryption whitepaper:
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
AMD64 Architecture Programmer's Manual:
http://support.amd.com/TechDocs/24593.pdf
SME is section 7.10
SEV is section 15.34
This patch series is based off of the master branch of tip.
Commit a27cb9e1b2b4 ("Merge branch 'WIP.sched/core'")
---
Still to do: IOMMU enablement support
Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
SME if it is supported/enabled
- Added support for (re)booting with kexec
Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
standard include file
- Conversion of assembler routines to C where possible (not everything
could be converted, e.g. the routine that does the actual encryption
needs to be copied into a safe location and it is difficult to
determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
how it needs to be programmed relative to accessing physical memory
Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
boot data or kernel data. The idea being that boot data will have
been placed in memory as un-encrypted data and would need to be accessed
as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it. The setting of the
MemEncryptionModeEn bit results in a reduction of physical address size
of the processor. It is possible that BIOS could have configured resources
resources into a range that will now not be addressable. To prevent this,
rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
encryption support in the kernel.
Tom Lendacky (28):
x86: Documentation for AMD Secure Memory Encryption (SME)
x86: Set the write-protect cache mode for full PAT support
x86: Add the Secure Memory Encryption CPU feature
x86: Handle reduction in physical address size with SME
x86: Add Secure Memory Encryption (SME) support
x86: Add support to enable SME during early boot processing
x86: Provide general kernel support for memory encryption
x86: Extend the early_memremap support with additional attrs
x86: Add support for
[RFC PATCH v4 27/28] x86: Add support to encrypt the kernel in-place
This patch adds the support to encrypt the kernel in-place. This is done by creating new page mappings for the kernel - a decrypted write-protected mapping and an encrypted mapping. The kernel is encyrpted by copying the kernel through a temporary buffer. Signed-off-by: Tom Lendacky --- arch/x86/kernel/Makefile |1 arch/x86/kernel/mem_encrypt_boot.S | 156 + arch/x86/kernel/mem_encrypt_init.c | 191 3 files changed, 348 insertions(+) create mode 100644 arch/x86/kernel/mem_encrypt_boot.S diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 33af80a..dc3ed84 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -142,4 +142,5 @@ ifeq ($(CONFIG_X86_64),y) obj-y += vsmp_64.o obj-y += mem_encrypt_init.o + obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o endif diff --git a/arch/x86/kernel/mem_encrypt_boot.S b/arch/x86/kernel/mem_encrypt_boot.S new file mode 100644 index 000..58e1756 --- /dev/null +++ b/arch/x86/kernel/mem_encrypt_boot.S @@ -0,0 +1,156 @@ +/* + * AMD Memory Encryption Support + * + * Copyright (C) 2016 Advanced Micro Devices, Inc. + * + * Author: Tom Lendacky + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include +#include +#include +#include + + .text + .code64 +ENTRY(sme_encrypt_execute) + +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* +* Entry parameters: +* RDI - virtual address for the encrypted kernel mapping +* RSI - virtual address for the decrypted kernel mapping +* RDX - length of kernel +* RCX - address of the encryption workarea +* - stack page (PAGE_SIZE) +* - encryption routine page (PAGE_SIZE) +* - intermediate copy buffer (PMD_PAGE_SIZE) +*R8 - address of the pagetables to use for encryption +*/ + + /* Set up a one page stack in the non-encrypted memory area */ + movq%rcx, %rax + addq$PAGE_SIZE, %rax + movq%rsp, %rbp + movq%rax, %rsp + push%rbp + + push%r12 + push%r13 + + movq%rdi, %r10 + movq%rsi, %r11 + movq%rdx, %r12 + movq%rcx, %r13 + + /* Copy encryption routine into the workarea */ + movq%rax, %rdi + leaq.Lencrypt_start(%rip), %rsi + movq$(.Lencrypt_stop - .Lencrypt_start), %rcx + rep movsb + + /* Setup registers for call */ + movq%r10, %rdi + movq%r11, %rsi + movq%r8, %rdx + movq%r12, %rcx + movq%rax, %r8 + addq$PAGE_SIZE, %r8 + + /* Call the encryption routine */ + call*%rax + + pop %r13 + pop %r12 + + pop %rsp/* Restore original stack pointer */ +.Lencrypt_exit: +#endif /* CONFIG_AMD_MEM_ENCRYPT */ + + ret +ENDPROC(sme_encrypt_execute) + +#ifdef CONFIG_AMD_MEM_ENCRYPT +/* + * Routine used to encrypt kernel. + * This routine must be run outside of the kernel proper since + * the kernel will be encrypted during the process. So this + * routine is defined here and then copied to an area outside + * of the kernel where it will remain and run decrypted + * during execution. + * + * On entry the registers must be: + * RDI - virtual address for the encrypted kernel mapping + * RSI - virtual address for the decrypted kernel mapping + * RDX - address of the pagetables to use for encryption + * RCX - length of kernel + * R8 - intermediate copy buffer + * + * RAX - points to this routine + * + * The kernel will be encrypted by copying from the non-encrypted + * kernel space to an intermediate buffer and then copying from the + * intermediate buffer back to the encrypted kernel space. The physical + * addresses of the two kernel space mappings are the same which + * results in the kernel being encrypted "in place". + */ +.Lencrypt_start: + /* Enable the new page tables */ + mov %rdx, %cr3 + + /* Flush any global TLBs */ + mov %cr4, %rdx + andq$~X86_CR4_PGE, %rdx + mov %rdx, %cr4 + orq $X86_CR4_PGE, %rdx + mov %rdx, %cr4 + + /* Set the PAT register PA5 entry to write-protect */ + push%rcx + movl$MSR_IA32_CR_PAT, %ecx + rdmsr + push%rdx/* Save original PAT value */ + andl$0x00ff, %edx /* Clear PA5 */ + orl $0x0500, %edx /* Set PA5 to WP */ + wrmsr + pop %rdx/* RDX contains original PAT value */ + pop %rcx + + movq%rcx, %r9 /* Save len
[RFC PATCH v4 25/28] x86: Access the setup data through sysfs decrypted
Use memremap() to map the setup data. This will make the appropriate
decision as to whether a RAM remapping can be done or if a fallback to
ioremap_cache() is needed (similar to the setup data debugfs support).
Signed-off-by: Tom Lendacky
---
arch/x86/kernel/ksysfs.c | 27 ++-
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..d653b3e 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,6 +16,7 @@
#include
#include
#include
+#include
#include
#include
@@ -79,12 +80,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
*paddr = pa_data;
return 0;
}
- data = ioremap_cache(pa_data, sizeof(*data));
+ data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
pa_data = data->next;
- iounmap(data);
+ memunmap(data);
i++;
}
return -EINVAL;
@@ -97,17 +98,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
u64 pa_data = boot_params.hdr.setup_data;
while (pa_data) {
- data = ioremap_cache(pa_data, sizeof(*data));
+ data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
if (nr == i) {
*size = data->len;
- iounmap(data);
+ memunmap(data);
return 0;
}
pa_data = data->next;
- iounmap(data);
+ memunmap(data);
i++;
}
return -EINVAL;
@@ -127,12 +128,12 @@ static ssize_t type_show(struct kobject *kobj,
ret = get_setup_data_paddr(nr, &paddr);
if (ret)
return ret;
- data = ioremap_cache(paddr, sizeof(*data));
+ data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
ret = sprintf(buf, "0x%x\n", data->type);
- iounmap(data);
+ memunmap(data);
return ret;
}
@@ -154,7 +155,7 @@ static ssize_t setup_data_data_read(struct file *fp,
ret = get_setup_data_paddr(nr, &paddr);
if (ret)
return ret;
- data = ioremap_cache(paddr, sizeof(*data));
+ data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
if (!data)
return -ENOMEM;
@@ -170,15 +171,15 @@ static ssize_t setup_data_data_read(struct file *fp,
goto out;
ret = count;
- p = ioremap_cache(paddr + sizeof(*data), data->len);
+ p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
if (!p) {
ret = -ENOMEM;
goto out;
}
memcpy(buf, p + off, count);
- iounmap(p);
+ memunmap(p);
out:
- iounmap(data);
+ memunmap(data);
return ret;
}
@@ -250,13 +251,13 @@ static int __init get_setup_data_total_num(u64 pa_data,
int *nr)
*nr = 0;
while (pa_data) {
*nr += 1;
- data = ioremap_cache(pa_data, sizeof(*data));
+ data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
if (!data) {
ret = -ENOMEM;
goto out;
}
pa_data = data->next;
- iounmap(data);
+ memunmap(data);
}
out:
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 24/28] x86: Access the setup data through debugfs decrypted
Use memremap() to map the setup data. This simplifies the code and will
make the appropriate decision as to whether a RAM remapping can be done
or if a fallback to ioremap_cache() is needed (which includes checking
PageHighMem).
Signed-off-by: Tom Lendacky
---
arch/x86/kernel/kdebugfs.c | 30 +++---
1 file changed, 11 insertions(+), 19 deletions(-)
diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index bdb83e4..c3d354d 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -48,17 +48,13 @@ static ssize_t setup_data_read(struct file *file, char
__user *user_buf,
pa = node->paddr + sizeof(struct setup_data) + pos;
pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
- if (PageHighMem(pg)) {
- p = ioremap_cache(pa, count);
- if (!p)
- return -ENXIO;
- } else
- p = __va(pa);
+ p = memremap(pa, count, MEMREMAP_WB);
+ if (!p)
+ return -ENXIO;
remain = copy_to_user(user_buf, p, count);
- if (PageHighMem(pg))
- iounmap(p);
+ memunmap(p);
if (remain)
return -EFAULT;
@@ -127,15 +123,12 @@ static int __init create_setup_data_nodes(struct dentry
*parent)
}
pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
- if (PageHighMem(pg)) {
- data = ioremap_cache(pa_data, sizeof(*data));
- if (!data) {
- kfree(node);
- error = -ENXIO;
- goto err_dir;
- }
- } else
- data = __va(pa_data);
+ data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+ if (!data) {
+ kfree(node);
+ error = -ENXIO;
+ goto err_dir;
+ }
node->paddr = pa_data;
node->type = data->type;
@@ -143,8 +136,7 @@ static int __init create_setup_data_nodes(struct dentry
*parent)
error = create_setup_data_node(d, no, node);
pa_data = data->next;
- if (PageHighMem(pg))
- iounmap(data);
+ memunmap(data);
if (error)
goto err_dir;
no++;
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 23/28] x86/kvm: Enable Secure Memory Encryption of nested page tables
Update the KVM support to include the memory encryption mask when creating
and using nested page tables.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/kvm_host.h |3 ++-
arch/x86/kvm/mmu.c |8 ++--
arch/x86/kvm/vmx.c |3 ++-
arch/x86/kvm/x86.c |3 ++-
4 files changed, 12 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index a7066dc..37326b5 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1050,7 +1050,8 @@ struct kvm_arch_async_pf {
void kvm_mmu_init_vm(struct kvm *kvm);
void kvm_mmu_uninit_vm(struct kvm *kvm);
void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
- u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask);
+ u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
+ u64 me_mask);
void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index d8d235b..46f246c 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -123,7 +123,7 @@ enum {
* PT32_LEVEL_BITS))) - 1))
#define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
- | shadow_x_mask | shadow_nx_mask)
+ | shadow_x_mask | shadow_nx_mask | shadow_me_mask)
#define ACC_EXEC_MASK1
#define ACC_WRITE_MASK PT_WRITABLE_MASK
@@ -178,6 +178,7 @@ struct kvm_shadow_walk_iterator {
static u64 __read_mostly shadow_dirty_mask;
static u64 __read_mostly shadow_mmio_mask;
static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
static void mmu_spte_set(u64 *sptep, u64 spte);
static void mmu_free_roots(struct kvm_vcpu *vcpu);
@@ -285,7 +286,8 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
}
void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
- u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask)
+ u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
+ u64 me_mask)
{
shadow_user_mask = user_mask;
shadow_accessed_mask = accessed_mask;
@@ -293,6 +295,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
shadow_nx_mask = nx_mask;
shadow_x_mask = x_mask;
shadow_present_mask = p_mask;
+ shadow_me_mask = me_mask;
}
EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
@@ -2546,6 +2549,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
pte_access &= ~ACC_WRITE_MASK;
spte |= (u64)pfn << PAGE_SHIFT;
+ spte |= shadow_me_mask;
if (pte_access & ACC_WRITE_MASK) {
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a236dec..fac3c27 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6703,7 +6703,8 @@ static __init int hardware_setup(void)
(enable_ept_ad_bits) ? VMX_EPT_DIRTY_BIT : 0ull,
0ull, VMX_EPT_EXECUTABLE_MASK,
cpu_has_vmx_ept_execute_only() ?
- 0ull : VMX_EPT_READABLE_MASK);
+ 0ull : VMX_EPT_READABLE_MASK,
+ 0ull);
ept_set_mmio_spte_mask();
kvm_enable_tdp();
} else
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a719783..9e6a593 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -67,6 +67,7 @@
#include
#include
#include
+#include
#define CREATE_TRACE_POINTS
#include "trace.h"
@@ -6027,7 +6028,7 @@ int kvm_arch_init(void *opaque)
kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
PT_DIRTY_MASK, PT64_NX_MASK, 0,
- PT_PRESENT_MASK);
+ PT_PRESENT_MASK, sme_me_mask);
kvm_timer_init();
perf_register_guest_info_callbacks(&kvm_guest_cbs);
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 21/28] x86: Check for memory encryption on the APs
Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the SYS_CFG MSR bit to enable
memory encryption on that AP and allow the AP to continue start up.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/realmode.h | 12
arch/x86/realmode/init.c |4
arch/x86/realmode/rm/trampoline_64.S | 17 +
3 files changed, 33 insertions(+)
diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..4f7ef53 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
#ifndef _ARCH_X86_REALMODE_H
#define _ARCH_X86_REALMODE_H
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * int the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT0
+#define TH_FLAGS_SME_ACTIVEBIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
#include
#include
@@ -38,6 +47,7 @@ struct trampoline_header {
u64 start;
u64 efer;
u32 cr4;
+ u32 flags;
#endif
};
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
void set_real_mode_mem(phys_addr_t mem, size_t size);
void reserve_real_mode(void);
+#endif /* __ASSEMBLY__ */
+
#endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 21d7506..5010089 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -102,6 +102,10 @@ static void __init setup_real_mode(void)
trampoline_cr4_features = &trampoline_header->cr4;
*trampoline_cr4_features = mmu_cr4_features;
+ trampoline_header->flags = 0;
+ if (sme_active())
+ trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
trampoline_pgd[0] = trampoline_pgd_entry.pgd;
trampoline_pgd[511] = init_level4_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S
b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..a88c3d1 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
#include
#include
#include
+#include
#include "realmode.h"
.text
@@ -92,6 +93,21 @@ ENTRY(startup_32)
movl%edx, %fs
movl%edx, %gs
+ /* Check for memory encryption support */
+ bt $TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+ jnc .Ldone
+ movl$MSR_K8_SYSCFG, %ecx
+ rdmsr
+ bts $MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+ jc .Ldone
+
+ /*
+* Memory encryption is enabled but the SME enable bit for this
+* CPU has has not been set. It is safe to set it, so do so.
+*/
+ wrmsr
+.Ldone:
+
movlpa_tr_cr4, %eax
movl%eax, %cr4 # Enable PAE mode
@@ -147,6 +163,7 @@ GLOBAL(trampoline_header)
tr_start: .space 8
GLOBAL(tr_efer) .space 8
GLOBAL(tr_cr4) .space 4
+ GLOBAL(tr_flags).space 4
END(trampoline_header)
#include "trampoline_common.S"
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 19/28] swiotlb: Add warnings for use of bounce buffers with SME
Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active. Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/mem_encrypt.h | 11 +++
include/linux/dma-mapping.h| 11 +++
include/linux/mem_encrypt.h|6 ++
lib/swiotlb.c |3 +++
4 files changed, 31 insertions(+)
diff --git a/arch/x86/include/asm/mem_encrypt.h
b/arch/x86/include/asm/mem_encrypt.h
index 87e816f..5a17f1b 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,11 @@ static inline bool sme_active(void)
return (sme_me_mask) ? true : false;
}
+static inline u64 sme_dma_mask(void)
+{
+ return ((u64)sme_me_mask << 1) - 1;
+}
+
void __init sme_early_encrypt(resource_size_t paddr,
unsigned long size);
void __init sme_early_decrypt(resource_size_t paddr,
@@ -53,6 +58,12 @@ static inline bool sme_active(void)
{
return false;
}
+
+static inline u64 sme_dma_mask(void)
+{
+ return 0ULL;
+}
+
#endif
static inline void __init sme_early_encrypt(resource_size_t paddr,
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 10c5a17..130bef7 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
/**
* List of possible attributes associated with a DMA mapping. The semantics
@@ -557,6 +558,11 @@ static inline int dma_set_mask(struct device *dev, u64
mask)
if (!dev->dma_mask || !dma_supported(dev, mask))
return -EIO;
+
+ if (sme_active() && (mask < sme_dma_mask()))
+ dev_warn(dev,
+"SME is active, device will require DMA bounce
buffers\n");
+
*dev->dma_mask = mask;
return 0;
}
@@ -576,6 +582,11 @@ static inline int dma_set_coherent_mask(struct device
*dev, u64 mask)
{
if (!dma_supported(dev, mask))
return -EIO;
+
+ if (sme_active() && (mask < sme_dma_mask()))
+ dev_warn(dev,
+"SME is active, device will require DMA bounce
buffers\n");
+
dev->coherent_dma_mask = mask;
return 0;
}
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 14a7b9f..6829ff1 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -28,6 +28,12 @@ static inline bool sme_active(void)
{
return false;
}
+
+static inline u64 sme_dma_mask(void)
+{
+ return 0ULL;
+}
+
#endif
#endif /* CONFIG_AMD_MEM_ENCRYPT */
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index c463067..aff9353 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -509,6 +509,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
if (no_iotlb_memory)
panic("Can not allocate SWIOTLB buffer earlier and can't now
provide you with the DMA bounce buffer");
+ WARN_ONCE(sme_active(),
+ "SME is active and system is using DMA bounce buffers\n");
+
mask = dma_get_seg_boundary(hwdev);
tbl_dma_addr &= mask;
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 17/28] x86: Decrypt trampoline area if memory encryption is active
When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.
Signed-off-by: Tom Lendacky
---
arch/x86/realmode/init.c | 12
1 file changed, 12 insertions(+)
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 5db706f1..21d7506 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -6,6 +6,8 @@
#include
#include
#include
+#include
+#include
struct real_mode_header *real_mode_header;
u32 *trampoline_cr4_features;
@@ -130,6 +132,16 @@ static void __init set_real_mode_permissions(void)
unsigned long text_start =
(unsigned long) __va(real_mode_header->text_start);
+ /*
+* If SME is active, the trampoline area will need to be in
+* decrypted memory in order to bring up other processors
+* successfully.
+*/
+ if (sme_active()) {
+ sme_early_decrypt(__pa(base), size);
+ set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+ }
+
set_memory_nx((unsigned long) base, size >> PAGE_SHIFT);
set_memory_ro((unsigned long) base, ro_size >> PAGE_SHIFT);
set_memory_x((unsigned long) text_start, text_size >> PAGE_SHIFT);
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 15/28] Add support to access persistent memory in the clear
Persistent memory is expected to persist across reboots. The encryption key used by SME will change across reboots which will result in corrupted persistent memory. Persistent memory is handed out by block devices through memory remapping functions, so be sure not to map this memory as encrypted. Signed-off-by: Tom Lendacky --- arch/x86/mm/ioremap.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index b0ff6bc..c6cb921 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -498,6 +498,8 @@ static bool memremap_should_map_encrypted(resource_size_t phys_addr, case E820_TYPE_ACPI: case E820_TYPE_NVS: case E820_TYPE_UNUSABLE: + case E820_TYPE_PMEM: + case E820_TYPE_PRAM: return false; default: break; -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 16/28] x86: Add support for changing memory encryption attribute
Add support for changing the memory encryption attribute for one or more
memory pages.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/cacheflush.h |3 ++
arch/x86/mm/pageattr.c| 66 +
2 files changed, 69 insertions(+)
diff --git a/arch/x86/include/asm/cacheflush.h
b/arch/x86/include/asm/cacheflush.h
index 872877d..33ae60a 100644
--- a/arch/x86/include/asm/cacheflush.h
+++ b/arch/x86/include/asm/cacheflush.h
@@ -12,6 +12,7 @@
* Executability : eXeutable, NoteXecutable
* Read/Write: ReadOnly, ReadWrite
* Presence : NotPresent
+ * Encryption: Encrypted, Decrypted
*
* Within a category, the attributes are mutually exclusive.
*
@@ -47,6 +48,8 @@
int set_memory_rw(unsigned long addr, int numpages);
int set_memory_np(unsigned long addr, int numpages);
int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
int set_memory_array_uc(unsigned long *addr, int addrinarray);
int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 91c5c63..9710f5c 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1742,6 +1742,72 @@ int set_memory_4k(unsigned long addr, int numpages)
__pgprot(0), 1, 0, NULL);
}
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+ struct cpa_data cpa;
+ unsigned long start;
+ int ret;
+
+ /* Nothing to do if the _PAGE_ENC attribute is zero */
+ if (_PAGE_ENC == 0)
+ return 0;
+
+ /* Save original start address since it will be modified */
+ start = addr;
+
+ memset(&cpa, 0, sizeof(cpa));
+ cpa.vaddr = &addr;
+ cpa.numpages = numpages;
+ cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+ cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+ cpa.pgd = init_mm.pgd;
+
+ /* Should not be working on unaligned addresses */
+ if (WARN_ONCE(*cpa.vaddr & ~PAGE_MASK,
+ "misaligned address: %#lx\n", *cpa.vaddr))
+ *cpa.vaddr &= PAGE_MASK;
+
+ /* Must avoid aliasing mappings in the highmem code */
+ kmap_flush_unused();
+ vm_unmap_aliases();
+
+ /*
+* Before changing the encryption attribute, we need to flush caches.
+*/
+ if (static_cpu_has(X86_FEATURE_CLFLUSH))
+ cpa_flush_range(start, numpages, 1);
+ else
+ cpa_flush_all(1);
+
+ ret = __change_page_attr_set_clr(&cpa, 1);
+
+ /*
+* After changing the encryption attribute, we need to flush TLBs
+* again in case any speculative TLB caching occurred (but no need
+* to flush caches again). We could just use cpa_flush_all(), but
+* in case TLB flushing gets optimized in the cpa_flush_range()
+* path use the same logic as above.
+*/
+ if (static_cpu_has(X86_FEATURE_CLFLUSH))
+ cpa_flush_range(start, numpages, 0);
+ else
+ cpa_flush_all(0);
+
+ return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+ return __set_memory_enc_dec(addr, numpages, true);
+}
+EXPORT_SYMBOL(set_memory_encrypted);
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+ return __set_memory_enc_dec(addr, numpages, false);
+}
+EXPORT_SYMBOL(set_memory_decrypted);
+
int set_pages_uc(struct page *page, int numpages)
{
unsigned long addr = (unsigned long)page_address(page);
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 11/28] x86: Add support to determine the E820 type of an address
This patch adds support to return the E820 type associated with an address
range.
Signed-off-by: Tom Lendacky
---
arch/x86/include/asm/e820/api.h |2 ++
arch/x86/include/asm/e820/types.h |2 ++
arch/x86/kernel/e820.c| 26 +++---
3 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..7c1bdc9 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
extern void e820__reallocate_tables(void);
extern void e820__register_nosave_regions(unsigned long limit_pfn);
+extern enum e820_type e820__get_entry_type(u64 start, u64 end);
+
/*
* Returns true iff the specified range [start,end) is completely contained
inside
* the ISA region.
diff --git a/arch/x86/include/asm/e820/types.h
b/arch/x86/include/asm/e820/types.h
index 4adeed0..bf49591 100644
--- a/arch/x86/include/asm/e820/types.h
+++ b/arch/x86/include/asm/e820/types.h
@@ -7,6 +7,8 @@
* These are the E820 types known to the kernel:
*/
enum e820_type {
+ E820_TYPE_INVALID = 0,
+
E820_TYPE_RAM = 1,
E820_TYPE_RESERVED = 2,
E820_TYPE_ACPI = 3,
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index 6e9b26f..2ee7ee2 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
* Note: this function only works correctly once the E820 table is sorted and
* not-overlapping (at least for the range specified), which is the case
normally.
*/
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+enum e820_type type)
{
int i;
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum
e820_type type)
* coverage of the desired range exists:
*/
if (start >= end)
- return 1;
+ return entry;
}
- return 0;
+
+ return NULL;
+}
+
+/*
+ * This function checks if the entire range is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+ return __e820__mapped_all(start, end, type) ? 1 : 0;
+}
+
+/*
+ * This function returns the type associated with the range .
+ */
+enum e820_type e820__get_entry_type(u64 start, u64 end)
+{
+ struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+ return entry ? entry->type : E820_TYPE_INVALID;
}
/*
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v4 02/28] x86: Set the write-protect cache mode for full PAT support
For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).
Acked-by: Borislav Petkov
Signed-off-by: Tom Lendacky
---
arch/x86/mm/pat.c |6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
* pat_init - Initialize PAT MSR and PAT table
*
* This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
*
* This function must be called on all CPUs using the specific sequence of
* operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
* 0102UC-: _PAGE_CACHE_MODE_UC_MINUS
* 0113UC : _PAGE_CACHE_MODE_UC
* 1004WB : Reserved
-* 1015WC : Reserved
+* 1015WP : _PAGE_CACHE_MODE_WP
* 1106UC-: Reserved
* 1117WT : _PAGE_CACHE_MODE_WT
*
@@ -364,7 +364,7 @@ void pat_init(void)
* corresponding types in the presence of PAT errata.
*/
pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
- PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+ PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
}
if (!boot_cpu_done) {
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v7 00/20] ILP32 for ARM64
> On Feb 12, 2017, at 4:07 PM, Andrew Pinski wrote: > > On Mon, Jan 9, 2017 at 3:29 AM, Yury Norov wrote: >> This series enables aarch64 with ilp32 mode. >> ... > > For folks concerned about performance, here is what we get for SPEC > CPU 2006 on ThunderX 2 CN99xx. > Positive means ILP32 is faster than LP64. This core does not have > AARCH32 so I can't compare that. > Also my LP64 scores don't change with and without the patches. > > Options: > -Ofast -flto=32 -mcpu=native -fno-aggressive-loop-optimizations > -funroll-loops -fprefetch-loop-arrays > GCC 7.0.1 r245361 with ilp32 multi-arch patch applied. > 4.10rc2 Plus ILP32 patches > > SPEC CPU 2006 INT ILP32/LP64 > 400.perlbench 5.23% > 401.bzip2 7.83% > 403.gcc 6.22% > 429.mcf 14.25% > 445.gobmk -1.33% > 456.hmmer -0.61% > 458.sjeng0.00% > 462.libquantum-7.38% > 464.h264ref 10.86% > 471.omnetpp 13.53% > 473.astar 1.38% > 483.xalancbmk 3.73% > Score4.29% > > Rate (32): > 400.perlbench 6.10% > 401.bzip2 7.10% > 403.gcc 6.71% > 429.mcf 57.29% > 445.gobmk-0.87% > 456.hmmer-0.19% > 458.sjeng 0.22% > 462.libquantum 0.00% > 464.h264ref 11.19% > 471.omnetpp11.80% > 473.astar -0.29% > 483.xalancbmk 8.87% > Score 8.12% These are good numbers and show that ILP32 has performance advantage over LP64. SPEC CPU2006 is a user-land benchmark and spends almost no time in the kernel (by design). Similar results for a kernel-focused benchmark would be highly interesting too, and kernel reviewers have asked for these a couple of times. Do you plan to run kernel benchmarks on the hardware you have? Thanks, -- Maxim Kuvyrkov www.linaro.org -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 37/37] ARM: dts: DRA7: Add pcie1 dt node for EP mode
Hi Tony, On Saturday 21 January 2017 12:00 AM, Tony Lindgren wrote: > * Kishon Vijay Abraham I [170112 02:34]: >> Add pcie1 dt node in order for the controller to operate in >> endpoint mode. However since none of the dra7 based boards have >> slots configured to operate in endpoint mode, keep EP mode >> disabled. > > Can this be merged separately later on without breaking anything? yes, I'll resend this once rest of the EP support gets in. Thanks Kishon -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 30/37] dt-bindings: PCI: dra7xx: Add dt bindings to enable legacy mode
Hi, On Thursday 19 January 2017 03:16 AM, Rob Herring wrote: > On Thu, Jan 12, 2017 at 03:56:19PM +0530, Kishon Vijay Abraham I wrote: >> Update device tree binding documentation of TI's dra7xx PCI >> controller to include property for enabling legacy mode. >> >> Signed-off-by: Kishon Vijay Abraham I >> --- >> Documentation/devicetree/bindings/pci/ti-pci.txt |4 >> 1 file changed, 4 insertions(+) >> >> diff --git a/Documentation/devicetree/bindings/pci/ti-pci.txt >> b/Documentation/devicetree/bindings/pci/ti-pci.txt >> index 62f5f59..ed85e8e 100644 >> --- a/Documentation/devicetree/bindings/pci/ti-pci.txt >> +++ b/Documentation/devicetree/bindings/pci/ti-pci.txt >> @@ -39,6 +39,10 @@ DEVICE MODE >> - interrupts : one interrupt entries must be specified for main interrupt. >> - num-ib-windows : number of inbound address translation windows >> - num-ob-windows : number of outbound address translation windows >> + - syscon-legacy-mode: phandle to the syscon dt node. The 1st argument >> should >> + contain the register offset within syscon and the 2nd >> + argument should contain the bit field for setting the >> + legacy mode > > What's legacy mode? It's the name of the bit-field defined in TRM used to add workaround for errata id i870 (See [PATCH 29/37] PCI: dwc: dra7xx: Workaround for errata id i870 of this series). I'm not sure what it does internally. Thanks Kishon -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
