[tip: x86/cleanups] x86/mm: Remove duplicate definition of _PAGE_PAT_LARGE
The following commit has been merged into the x86/cleanups branch of tip:
Commit-ID: 4af0e6e39b7ed77796a41537db91d717fedd0ac3
Gitweb:
https://git.kernel.org/tip/4af0e6e39b7ed77796a41537db91d717fedd0ac3
Author:Arvind Sankar
AuthorDate:Wed, 11 Nov 2020 11:09:46 -05:00
Committer: Borislav Petkov
CommitterDate: Fri, 08 Jan 2021 22:04:51 +01:00
x86/mm: Remove duplicate definition of _PAGE_PAT_LARGE
_PAGE_PAT_LARGE is already defined next to _PAGE_PAT. Remove the
duplicate.
Fixes: 4efb56649132 ("x86/mm: Tabulate the page table encoding definitions")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Acked-by: Dave Hansen
Link: https://lkml.kernel.org/r/2020160946.147341-2-nived...@alum.mit.edu
---
arch/x86/include/asm/pgtable_types.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/x86/include/asm/pgtable_types.h
b/arch/x86/include/asm/pgtable_types.h
index 394757e..f24d7ef 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -177,8 +177,6 @@ enum page_cache_mode {
#define __pgprot(x)((pgprot_t) { (x) } )
#define __pg(x)__pgprot(x)
-#define _PAGE_PAT_LARGE(_AT(pteval_t, 1) <<
_PAGE_BIT_PAT_LARGE)
-
#define PAGE_NONE __pg( 0| 0| 0|___A| 0| 0| 0|___G)
#define PAGE_SHARED __pg(__PP|__RW|_USR|___A|__NX| 0| 0| 0)
#define PAGE_SHARED_EXEC __pg(__PP|__RW|_USR|___A| 0| 0| 0| 0)
[tip: x86/urgent] x86/mm/mem_encrypt: Fix definition of PMD_FLAGS_DEC_WP
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 29ac40cbed2bc06fa218ca25d7f5e280d3d08a25
Gitweb:
https://git.kernel.org/tip/29ac40cbed2bc06fa218ca25d7f5e280d3d08a25
Author:Arvind Sankar
AuthorDate:Wed, 11 Nov 2020 11:09:45 -05:00
Committer: Borislav Petkov
CommitterDate: Thu, 10 Dec 2020 12:28:06 +01:00
x86/mm/mem_encrypt: Fix definition of PMD_FLAGS_DEC_WP
The PAT bit is in different locations for 4k and 2M/1G page table
entries.
Add a definition for _PAGE_LARGE_CACHE_MASK to represent the three
caching bits (PWT, PCD, PAT), similar to _PAGE_CACHE_MASK for 4k pages,
and use it in the definition of PMD_FLAGS_DEC_WP to get the correct PAT
index for write-protected pages.
Fixes: 6ebcb060713f ("x86/mm: Add support to encrypt the kernel in-place")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Tested-by: Tom Lendacky
Cc: sta...@vger.kernel.org
Link: https://lkml.kernel.org/r/2020160946.147341-1-nived...@alum.mit.edu
---
arch/x86/include/asm/pgtable_types.h | 1 +
arch/x86/mm/mem_encrypt_identity.c | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/pgtable_types.h
b/arch/x86/include/asm/pgtable_types.h
index 816b31c..394757e 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -155,6 +155,7 @@ enum page_cache_mode {
#define _PAGE_ENC (_AT(pteval_t, sme_me_mask))
#define _PAGE_CACHE_MASK (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)
+#define _PAGE_LARGE_CACHE_MASK (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT_LARGE)
#define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC))
#define _PAGE_CACHE_WP (cachemode2protval(_PAGE_CACHE_MODE_WP))
diff --git a/arch/x86/mm/mem_encrypt_identity.c
b/arch/x86/mm/mem_encrypt_identity.c
index 733b983..6c5eb6f 100644
--- a/arch/x86/mm/mem_encrypt_identity.c
+++ b/arch/x86/mm/mem_encrypt_identity.c
@@ -45,8 +45,8 @@
#define PMD_FLAGS_LARGE(__PAGE_KERNEL_LARGE_EXEC &
~_PAGE_GLOBAL)
#define PMD_FLAGS_DEC PMD_FLAGS_LARGE
-#define PMD_FLAGS_DEC_WP ((PMD_FLAGS_DEC & ~_PAGE_CACHE_MASK) | \
-(_PAGE_PAT | _PAGE_PWT))
+#define PMD_FLAGS_DEC_WP ((PMD_FLAGS_DEC & ~_PAGE_LARGE_CACHE_MASK) | \
+(_PAGE_PAT_LARGE | _PAGE_PWT))
#define PMD_FLAGS_ENC (PMD_FLAGS_LARGE | _PAGE_ENC)
[tip: x86/cpu] x86/cpu/amd: Remove dead code for TSEG region remapping
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: 262bd5724afdefd4c48a260d6100e78cc43ee06b
Gitweb:
https://git.kernel.org/tip/262bd5724afdefd4c48a260d6100e78cc43ee06b
Author:Arvind Sankar
AuthorDate:Fri, 27 Nov 2020 12:13:24 -05:00
Committer: Borislav Petkov
CommitterDate: Tue, 08 Dec 2020 18:45:21 +01:00
x86/cpu/amd: Remove dead code for TSEG region remapping
Commit
26bfa5f89486 ("x86, amd: Cleanup init_amd")
moved the code that remaps the TSEG region using 4k pages from
init_amd() to bsp_init_amd().
However, bsp_init_amd() is executed well before the direct mapping is
actually created:
setup_arch()
-> early_cpu_init()
-> early_identify_cpu()
-> this_cpu->c_bsp_init()
-> bsp_init_amd()
...
-> init_mem_mapping()
So the change effectively disabled the 4k remapping, because
pfn_range_is_mapped() is always false at this point.
It has been over six years since the commit, and no-one seems to have
noticed this, so just remove the code. The original code was also
incomplete, since it doesn't check how large the TSEG address range
actually is, so it might remap only part of it in any case.
Hygon has copied the incorrect version, so the code has never run on it
since the cpu support was added two years ago. Remove it from there as
well.
Committer notes:
This workaround is incomplete anyway:
1. The code must check MSRC001_0113.TValid (SMM TSeg Mask MSR) first, to
check whether the TSeg address range is enabled.
2. The code must check whether the range is not 2M aligned - if it is,
there's nothing to work around.
3. In all the BIOSes tested, the TSeg range is in a e820 reserved area
and those are not mapped anymore, after
66520ebc2df3 ("x86, mm: Only direct map addresses that are marked as
E820_RAM")
which means, there's nothing to be worked around either.
So let's rip it out.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20201127171324.1846019-1-nived...@alum.mit.edu
---
arch/x86/kernel/cpu/amd.c | 21 -
arch/x86/kernel/cpu/hygon.c | 20
2 files changed, 41 deletions(-)
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 1f71c76..f8ca66f 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -23,7 +23,6 @@
#ifdef CONFIG_X86_64
# include
-# include
#endif
#include "cpu.h"
@@ -509,26 +508,6 @@ static void early_init_amd_mc(struct cpuinfo_x86 *c)
static void bsp_init_amd(struct cpuinfo_x86 *c)
{
-
-#ifdef CONFIG_X86_64
- if (c->x86 >= 0xf) {
- unsigned long long tseg;
-
- /*
-* Split up direct mapping around the TSEG SMM area.
-* Don't do it for gbpages because there seems very little
-* benefit in doing so.
-*/
- if (!rdmsrl_safe(MSR_K8_TSEG_ADDR, )) {
- unsigned long pfn = tseg >> PAGE_SHIFT;
-
- pr_debug("tseg: %010llx\n", tseg);
- if (pfn_range_is_mapped(pfn, pfn + 1))
- set_memory_4k((unsigned long)__va(tseg), 1);
- }
- }
-#endif
-
if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) {
if (c->x86 > 0x10 ||
diff --git a/arch/x86/kernel/cpu/hygon.c b/arch/x86/kernel/cpu/hygon.c
index dc0840a..ae59115 100644
--- a/arch/x86/kernel/cpu/hygon.c
+++ b/arch/x86/kernel/cpu/hygon.c
@@ -14,9 +14,6 @@
#include
#include
#include
-#ifdef CONFIG_X86_64
-# include
-#endif
#include "cpu.h"
@@ -203,23 +200,6 @@ static void early_init_hygon_mc(struct cpuinfo_x86 *c)
static void bsp_init_hygon(struct cpuinfo_x86 *c)
{
-#ifdef CONFIG_X86_64
- unsigned long long tseg;
-
- /*
-* Split up direct mapping around the TSEG SMM area.
-* Don't do it for gbpages because there seems very little
-* benefit in doing so.
-*/
- if (!rdmsrl_safe(MSR_K8_TSEG_ADDR, )) {
- unsigned long pfn = tseg >> PAGE_SHIFT;
-
- pr_debug("tseg: %010llx\n", tseg);
- if (pfn_range_is_mapped(pfn, pfn + 1))
- set_memory_4k((unsigned long)__va(tseg), 1);
- }
-#endif
-
if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) {
u64 val;
[tip: x86/cleanups] x86/boot: Remove unused finalize_identity_maps()
The following commit has been merged into the x86/cleanups branch of tip:
Commit-ID: 0ac317e89791b76055ef11b952625ef77a1d2eba
Gitweb:
https://git.kernel.org/tip/0ac317e89791b76055ef11b952625ef77a1d2eba
Author:Arvind Sankar
AuthorDate:Mon, 05 Oct 2020 11:12:07 -04:00
Committer: Borislav Petkov
CommitterDate: Wed, 18 Nov 2020 16:04:23 +01:00
x86/boot: Remove unused finalize_identity_maps()
Commit
8570978ea030 ("x86/boot/compressed/64: Don't pre-map memory in KASLR code")
removed all the references to finalize_identity_maps(), but neglected to
delete the actual function. Remove it.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20201005151208.2212886-2-nived...@alum.mit.edu
---
arch/x86/boot/compressed/ident_map_64.c | 10 --
1 file changed, 10 deletions(-)
diff --git a/arch/x86/boot/compressed/ident_map_64.c
b/arch/x86/boot/compressed/ident_map_64.c
index a5e5db6..6bf2022 100644
--- a/arch/x86/boot/compressed/ident_map_64.c
+++ b/arch/x86/boot/compressed/ident_map_64.c
@@ -167,16 +167,6 @@ void initialize_identity_maps(void *rmode)
write_cr3(top_level_pgt);
}
-/*
- * This switches the page tables to the new level4 that has been built
- * via calls to add_identity_map() above. If booted via startup_32(),
- * this is effectively a no-op.
- */
-void finalize_identity_maps(void)
-{
- write_cr3(top_level_pgt);
-}
-
static pte_t *split_large_pmd(struct x86_mapping_info *info,
pmd_t *pmdp, unsigned long __address)
{
[tip: x86/cleanups] x86/head/64: Remove unused GET_CR2_INTO() macro
The following commit has been merged into the x86/cleanups branch of tip:
Commit-ID: 31d8546033053b98de00846ede8088bdbe38651d
Gitweb:
https://git.kernel.org/tip/31d8546033053b98de00846ede8088bdbe38651d
Author:Arvind Sankar
AuthorDate:Mon, 05 Oct 2020 11:12:08 -04:00
Committer: Borislav Petkov
CommitterDate: Wed, 18 Nov 2020 18:09:38 +01:00
x86/head/64: Remove unused GET_CR2_INTO() macro
Commit
4b47cdbda6f1 ("x86/head/64: Move early exception dispatch to C code")
removed the usage of GET_CR2_INTO().
Drop the definition as well, and related definitions in paravirt.h and
asm-offsets.h
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20201005151208.2212886-3-nived...@alum.mit.edu
---
arch/x86/include/asm/paravirt.h | 11 ---
arch/x86/kernel/asm-offsets.c | 1 -
arch/x86/kernel/head_64.S | 9 -
3 files changed, 21 deletions(-)
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index d25cc68..f8dce11 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -812,17 +812,6 @@ extern void default_banner(void);
#endif /* CONFIG_PARAVIRT_XXL */
#endif /* CONFIG_X86_64 */
-#ifdef CONFIG_PARAVIRT_XXL
-
-#define GET_CR2_INTO_AX
\
- PARA_SITE(PARA_PATCH(PV_MMU_read_cr2), \
- ANNOTATE_RETPOLINE_SAFE; \
- call PARA_INDIRECT(pv_ops+PV_MMU_read_cr2); \
-)
-
-#endif /* CONFIG_PARAVIRT_XXL */
-
-
#endif /* __ASSEMBLY__ */
#else /* CONFIG_PARAVIRT */
# define default_banner x86_init_noop
diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
index 70b7154..60b9f42 100644
--- a/arch/x86/kernel/asm-offsets.c
+++ b/arch/x86/kernel/asm-offsets.c
@@ -66,7 +66,6 @@ static void __used common(void)
OFFSET(PV_IRQ_irq_disable, paravirt_patch_template, irq.irq_disable);
OFFSET(PV_IRQ_irq_enable, paravirt_patch_template, irq.irq_enable);
OFFSET(PV_CPU_iret, paravirt_patch_template, cpu.iret);
- OFFSET(PV_MMU_read_cr2, paravirt_patch_template, mmu.read_cr2);
#endif
#ifdef CONFIG_XEN
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 7eb2a1c..2215d4c 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -26,15 +26,6 @@
#include
#include
-#ifdef CONFIG_PARAVIRT_XXL
-#include
-#include
-#define GET_CR2_INTO(reg) GET_CR2_INTO_AX ; _ASM_MOV %_ASM_AX, reg
-#else
-#define INTERRUPT_RETURN iretq
-#define GET_CR2_INTO(reg) _ASM_MOV %cr2, reg
-#endif
-
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
* because we need identity-mapped pages.
[tip: efi/core] efi/x86: Only copy the compressed kernel image in efi_relocate_kernel()
The following commit has been merged into the efi/core branch of tip:
Commit-ID: 688eb28211abdf82a3f51e8997f1c8137947227d
Gitweb:
https://git.kernel.org/tip/688eb28211abdf82a3f51e8997f1c8137947227d
Author:Arvind Sankar
AuthorDate:Sun, 11 Oct 2020 10:20:12 -04:00
Committer: Ard Biesheuvel
CommitterDate: Mon, 26 Oct 2020 08:06:36 +01:00
efi/x86: Only copy the compressed kernel image in efi_relocate_kernel()
The image_size argument to efi_relocate_kernel() is currently specified
as init_size, but this is unnecessarily large. The compressed kernel is
much smaller, in fact, its image only extends up to the start of _bss,
since at this point, the .bss section is still uninitialized.
Depending on compression level, this can reduce the amount of data
copied by 4-5x.
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20201011142012.96493-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/x86-stub.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/firmware/efi/libstub/x86-stub.c
b/drivers/firmware/efi/libstub/x86-stub.c
index 3672539..f14c4ff 100644
--- a/drivers/firmware/efi/libstub/x86-stub.c
+++ b/drivers/firmware/efi/libstub/x86-stub.c
@@ -715,8 +715,11 @@ unsigned long efi_main(efi_handle_t handle,
(IS_ENABLED(CONFIG_X86_32) && buffer_end > KERNEL_IMAGE_SIZE)||
(IS_ENABLED(CONFIG_X86_64) && buffer_end > MAXMEM_X86_64_4LEVEL) ||
(image_offset == 0)) {
+ extern char _bss[];
+
status = efi_relocate_kernel(_addr,
-hdr->init_size, hdr->init_size,
+(unsigned long)_bss - bzimage_addr,
+hdr->init_size,
hdr->pref_address,
hdr->kernel_alignment,
LOAD_PHYSICAL_ADDR);
[tip: efi/urgent] efi/x86: Free efi_pgd with free_pages()
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: c2fe61d8be491ff8188edaf22e838f81146b
Gitweb:
https://git.kernel.org/tip/c2fe61d8be491ff8188edaf22e838f81146b
Author:Arvind Sankar
AuthorDate:Tue, 10 Nov 2020 11:39:19 -05:00
Committer: Ard Biesheuvel
CommitterDate: Tue, 10 Nov 2020 19:18:11 +01:00
efi/x86: Free efi_pgd with free_pages()
Commit
d9e9a6418065 ("x86/mm/pti: Allocate a separate user PGD")
changed the PGD allocation to allocate PGD_ALLOCATION_ORDER pages, so in
the error path it should be freed using free_pages() rather than
free_page().
Commit
06ace26f4e6f ("x86/efi: Free efi_pgd with free_pages()")
fixed one instance of this, but missed another.
Move the freeing out-of-line to avoid code duplication and fix this bug.
Fixes: d9e9a6418065 ("x86/mm/pti: Allocate a separate user PGD")
Link: https://lore.kernel.org/r/20201110163919.1134431-1-nived...@alum.mit.edu
Signed-off-by: Arvind Sankar
Signed-off-by: Ard Biesheuvel
---
arch/x86/platform/efi/efi_64.c | 24 +---
1 file changed, 13 insertions(+), 11 deletions(-)
diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 8f5759d..e1e8d4e 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -78,28 +78,30 @@ int __init efi_alloc_page_tables(void)
gfp_mask = GFP_KERNEL | __GFP_ZERO;
efi_pgd = (pgd_t *)__get_free_pages(gfp_mask, PGD_ALLOCATION_ORDER);
if (!efi_pgd)
- return -ENOMEM;
+ goto fail;
pgd = efi_pgd + pgd_index(EFI_VA_END);
p4d = p4d_alloc(_mm, pgd, EFI_VA_END);
- if (!p4d) {
- free_page((unsigned long)efi_pgd);
- return -ENOMEM;
- }
+ if (!p4d)
+ goto free_pgd;
pud = pud_alloc(_mm, p4d, EFI_VA_END);
- if (!pud) {
- if (pgtable_l5_enabled())
- free_page((unsigned long) pgd_page_vaddr(*pgd));
- free_pages((unsigned long)efi_pgd, PGD_ALLOCATION_ORDER);
- return -ENOMEM;
- }
+ if (!pud)
+ goto free_p4d;
efi_mm.pgd = efi_pgd;
mm_init_cpumask(_mm);
init_new_context(NULL, _mm);
return 0;
+
+free_p4d:
+ if (pgtable_l5_enabled())
+ free_page((unsigned long)pgd_page_vaddr(*pgd));
+free_pgd:
+ free_pages((unsigned long)efi_pgd, PGD_ALLOCATION_ORDER);
+fail:
+ return -ENOMEM;
}
/*
[tip: x86/build] x86/build: Fix vmlinux size check on 64-bit
The following commit has been merged into the x86/build branch of tip:
Commit-ID: ea3186b9572a1b0299448697cfc44920061872cf
Gitweb:
https://git.kernel.org/tip/ea3186b9572a1b0299448697cfc44920061872cf
Author:Arvind Sankar
AuthorDate:Thu, 29 Oct 2020 12:19:03 -04:00
Committer: Borislav Petkov
CommitterDate: Thu, 29 Oct 2020 21:54:35 +01:00
x86/build: Fix vmlinux size check on 64-bit
Commit
b4e0409a36f4 ("x86: check vmlinux limits, 64-bit")
added a check that the size of the 64-bit kernel is less than
KERNEL_IMAGE_SIZE.
The check uses (_end - _text), but this is not enough. The initial
PMD used in startup_64() (level2_kernel_pgt) can only map upto
KERNEL_IMAGE_SIZE from __START_KERNEL_map, not from _text, and the
modules area (MODULES_VADDR) starts at KERNEL_IMAGE_SIZE.
The correct check is what is currently done for 32-bit, since
LOAD_OFFSET is defined appropriately for the two architectures. Just
check (_end - LOAD_OFFSET) against KERNEL_IMAGE_SIZE unconditionally.
Note that on 32-bit, the limit is not strict: KERNEL_IMAGE_SIZE is not
really used by the main kernel. The higher the kernel is located, the
less the space available for the vmalloc area. However, it is used by
KASLR in the compressed stub to limit the maximum address of the kernel
to a safe value.
Clean up various comments to clarify that despite the name,
KERNEL_IMAGE_SIZE is not a limit on the size of the kernel image, but a
limit on the maximum virtual address that the image can occupy.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20201029161903.2553528-1-nived...@alum.mit.edu
---
arch/x86/include/asm/page_32_types.h | 8 +++-
arch/x86/include/asm/page_64_types.h | 6 --
arch/x86/include/asm/pgtable_32.h| 18 ++
arch/x86/kernel/head_64.S| 20 +---
arch/x86/kernel/vmlinux.lds.S| 12 +++-
5 files changed, 29 insertions(+), 35 deletions(-)
diff --git a/arch/x86/include/asm/page_32_types.h
b/arch/x86/include/asm/page_32_types.h
index f462895..faf9cc1 100644
--- a/arch/x86/include/asm/page_32_types.h
+++ b/arch/x86/include/asm/page_32_types.h
@@ -53,7 +53,13 @@
#define STACK_TOP_MAX STACK_TOP
/*
- * Kernel image size is limited to 512 MB (see in arch/x86/kernel/head_32.S)
+ * In spite of the name, KERNEL_IMAGE_SIZE is a limit on the maximum virtual
+ * address for the kernel image, rather than the limit on the size itself. On
+ * 32-bit, this is not a strict limit, but this value is used to limit the
+ * link-time virtual address range of the kernel, and by KASLR to limit the
+ * randomized address from which the kernel is executed. A relocatable kernel
+ * can be loaded somewhat higher than KERNEL_IMAGE_SIZE as long as enough space
+ * remains for the vmalloc area.
*/
#define KERNEL_IMAGE_SIZE (512 * 1024 * 1024)
diff --git a/arch/x86/include/asm/page_64_types.h
b/arch/x86/include/asm/page_64_types.h
index 3f49dac..645bd1d 100644
--- a/arch/x86/include/asm/page_64_types.h
+++ b/arch/x86/include/asm/page_64_types.h
@@ -98,8 +98,10 @@
#define STACK_TOP_MAX TASK_SIZE_MAX
/*
- * Maximum kernel image size is limited to 1 GiB, due to the fixmap living
- * in the next 1 GiB (see level2_kernel_pgt in arch/x86/kernel/head_64.S).
+ * In spite of the name, KERNEL_IMAGE_SIZE is a limit on the maximum virtual
+ * address for the kernel image, rather than the limit on the size itself.
+ * This can be at most 1 GiB, due to the fixmap living in the next 1 GiB (see
+ * level2_kernel_pgt in arch/x86/kernel/head_64.S).
*
* On KASLR use 1 GiB by default, leaving 1 GiB for modules once the
* page tables are fully set up.
diff --git a/arch/x86/include/asm/pgtable_32.h
b/arch/x86/include/asm/pgtable_32.h
index d7acae4..7c9c968 100644
--- a/arch/x86/include/asm/pgtable_32.h
+++ b/arch/x86/include/asm/pgtable_32.h
@@ -57,19 +57,13 @@ do {\
#endif
/*
- * This is how much memory in addition to the memory covered up to
- * and including _end we need mapped initially.
- * We need:
- * (KERNEL_IMAGE_SIZE/4096) / 1024 pages (worst case, non PAE)
- * (KERNEL_IMAGE_SIZE/4096) / 512 + 4 pages (worst case for PAE)
+ * This is used to calculate the .brk reservation for initial pagetables.
+ * Enough space is reserved to allocate pagetables sufficient to cover all
+ * of LOWMEM_PAGES, which is an upper bound on the size of the direct map of
+ * lowmem.
*
- * Modulo rounding, each megabyte assigned here requires a kilobyte of
- * memory, which is currently unreclaimed.
- *
- * This should be a multiple of a page.
- *
- * KERNEL_IMAGE_SIZE should be greater than pa(_end)
- * and small than max_low_pfn, otherwise will waste some page table entries
+ * With PAE paging (PTRS_PER_PMD > 1), we allocate PTRS_PER_PGD == 4 pages for
+ * the PMD's in addition to the pages required for the last level pagetables.
*/
#if
[tip: x86/mm] x86/mm/ident_map: Check for errors from ident_pud_init()
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 1fcd009102ee02e217f2e7635ab65517d785da8e
Gitweb:
https://git.kernel.org/tip/1fcd009102ee02e217f2e7635ab65517d785da8e
Author:Arvind Sankar
AuthorDate:Tue, 27 Oct 2020 19:06:48 -04:00
Committer: Borislav Petkov
CommitterDate: Wed, 28 Oct 2020 14:48:30 +01:00
x86/mm/ident_map: Check for errors from ident_pud_init()
Commit
ea3b5e60ce80 ("x86/mm/ident_map: Add 5-level paging support")
added ident_p4d_init() to support 5-level paging, but this function
doesn't check and return errors from ident_pud_init().
For example, the decompressor stub uses this code to create an identity
mapping. If it runs out of pages while trying to allocate a PMD
pagetable, the error will be currently ignored.
Fix this to propagate errors.
[ bp: Space out statements for better readability. ]
Fixes: ea3b5e60ce80 ("x86/mm/ident_map: Add 5-level paging support")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Joerg Roedel
Acked-by: Kirill A. Shutemov
Link: https://lkml.kernel.org/r/20201027230648.1885111-1-nived...@alum.mit.edu
---
arch/x86/mm/ident_map.c | 12 ++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c
index fe7a125..968d700 100644
--- a/arch/x86/mm/ident_map.c
+++ b/arch/x86/mm/ident_map.c
@@ -62,6 +62,7 @@ static int ident_p4d_init(struct x86_mapping_info *info,
p4d_t *p4d_page,
unsigned long addr, unsigned long end)
{
unsigned long next;
+ int result;
for (; addr < end; addr = next) {
p4d_t *p4d = p4d_page + p4d_index(addr);
@@ -73,13 +74,20 @@ static int ident_p4d_init(struct x86_mapping_info *info,
p4d_t *p4d_page,
if (p4d_present(*p4d)) {
pud = pud_offset(p4d, 0);
- ident_pud_init(info, pud, addr, next);
+ result = ident_pud_init(info, pud, addr, next);
+ if (result)
+ return result;
+
continue;
}
pud = (pud_t *)info->alloc_pgt_page(info->context);
if (!pud)
return -ENOMEM;
- ident_pud_init(info, pud, addr, next);
+
+ result = ident_pud_init(info, pud, addr, next);
+ if (result)
+ return result;
+
set_p4d(p4d, __p4d(__pa(pud) | info->kernpg_flag));
}
[tip: x86/seves] x86/head/64: Disable stack protection for head$(BITS).o
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 103a4908ad4da9decdf9bc7216ec5a4861edf703
Gitweb:
https://git.kernel.org/tip/103a4908ad4da9decdf9bc7216ec5a4861edf703
Author:Arvind Sankar
AuthorDate:Thu, 08 Oct 2020 15:16:23 -04:00
Committer: Borislav Petkov
CommitterDate: Mon, 19 Oct 2020 13:11:00 +02:00
x86/head/64: Disable stack protection for head$(BITS).o
On 64-bit, the startup_64_setup_env() function added in
866b556efa12 ("x86/head/64: Install startup GDT")
has stack protection enabled because of set_bringup_idt_handler().
This happens when CONFIG_STACKPROTECTOR_STRONG is enabled. It
also currently needs CONFIG_AMD_MEM_ENCRYPT enabled because then
set_bringup_idt_handler() is not an empty stub but that might change in
the future, when the other vendor adds their similar technology.
At this point, %gs is not yet initialized, and this doesn't cause a
crash only because the #PF handler from the decompressor stub is still
installed and handles the page fault.
Disable stack protection for the whole file, and do it on 32-bit as
well to avoid surprises.
[ bp: Extend commit message with the exact explanation how it happens. ]
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Joerg Roedel
Link: https://lkml.kernel.org/r/20201008191623.2881677-6-nived...@alum.mit.edu
---
arch/x86/kernel/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 04ceea8..68608bd 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -47,6 +47,8 @@ endif
# non-deterministic coverage.
KCOV_INSTRUMENT:= n
+CFLAGS_head$(BITS).o += -fno-stack-protector
+
CFLAGS_irq.o := -I $(srctree)/$(src)/../include/asm/trace
obj-y := process_$(BITS).o signal.o
[tip: x86/seves] x86/boot/64: Explicitly map boot_params and command line
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: b17a45b6e53f6613118b2e5cfc4a992cc50deb2c
Gitweb:
https://git.kernel.org/tip/b17a45b6e53f6613118b2e5cfc4a992cc50deb2c
Author:Arvind Sankar
AuthorDate:Fri, 16 Oct 2020 16:04:01 -04:00
Committer: Borislav Petkov
CommitterDate: Mon, 19 Oct 2020 19:39:50 +02:00
x86/boot/64: Explicitly map boot_params and command line
Commits
ca0e22d4f011 ("x86/boot/compressed/64: Always switch to own page table")
8570978ea030 ("x86/boot/compressed/64: Don't pre-map memory in KASLR code")
set up a new page table in the decompressor stub, but without explicit
mappings for boot_params and the kernel command line, relying on the #PF
handler instead.
This is fragile, as boot_params and the command line mappings are
required for the main kernel. If EARLY_PRINTK and RANDOMIZE_BASE are
disabled, a QEMU/OVMF boot never accesses the command line in the
decompressor stub, and so it never gets mapped. The main kernel accesses
it from the identity mapping if AMD_MEM_ENCRYPT is enabled, and will
crash.
Fix this by adding back the explicit mapping of boot_params and the
command line.
Note: the changes also removed the explicit mapping of the main kernel,
with the result that .bss and .brk may not be in the identity mapping,
but those don't get accessed by the main kernel before it switches to
its own page tables.
[ bp: Pass boot_params with a MOV %rsp... instead of PUSH/POP. Use
block formatting for the comment. ]
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Joerg Roedel
Link: https://lkml.kernel.org/r/20201016200404.1615994-1-nived...@alum.mit.edu
---
arch/x86/boot/compressed/head_64.S | 3 +++
arch/x86/boot/compressed/ident_map_64.c | 23 ---
2 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/arch/x86/boot/compressed/head_64.S
b/arch/x86/boot/compressed/head_64.S
index 1c80f17..017de6c 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -544,6 +544,9 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated)
pushq %rsi
callset_sev_encryption_mask
callload_stage2_idt
+
+ /* Pass boot_params to initialize_identity_maps() */
+ movq(%rsp), %rdi
callinitialize_identity_maps
popq%rsi
diff --git a/arch/x86/boot/compressed/ident_map_64.c
b/arch/x86/boot/compressed/ident_map_64.c
index c6f7aef..a5e5db6 100644
--- a/arch/x86/boot/compressed/ident_map_64.c
+++ b/arch/x86/boot/compressed/ident_map_64.c
@@ -33,6 +33,12 @@
#define __PAGE_OFFSET __PAGE_OFFSET_BASE
#include "../../mm/ident_map.c"
+#define _SETUP
+#include /* For COMMAND_LINE_SIZE */
+#undef _SETUP
+
+extern unsigned long get_cmd_line_ptr(void);
+
/* Used by PAGE_KERN* macros: */
pteval_t __default_kernel_pte_mask __read_mostly = ~0;
@@ -101,8 +107,10 @@ static void add_identity_map(unsigned long start, unsigned
long end)
}
/* Locates and clears a region for a new top level page table. */
-void initialize_identity_maps(void)
+void initialize_identity_maps(void *rmode)
{
+ unsigned long cmdline;
+
/* Exclude the encryption mask from __PHYSICAL_MASK */
physical_mask &= ~sme_me_mask;
@@ -143,10 +151,19 @@ void initialize_identity_maps(void)
}
/*
-* New page-table is set up - map the kernel image and load it
-* into cr3.
+* New page-table is set up - map the kernel image, boot_params and the
+* command line. The uncompressed kernel requires boot_params and the
+* command line to be mapped in the identity mapping. Map them
+* explicitly here in case the compressed kernel does not touch them,
+* or does not touch all the pages covering them.
*/
add_identity_map((unsigned long)_head, (unsigned long)_end);
+ boot_params = rmode;
+ add_identity_map((unsigned long)boot_params, (unsigned
long)(boot_params + 1));
+ cmdline = get_cmd_line_ptr();
+ add_identity_map(cmdline, cmdline + COMMAND_LINE_SIZE);
+
+ /* Load the new page-table. */
write_cr3(top_level_pgt);
}
[tip: x86/seves] x86/boot/64: Initialize 5-level paging variables earlier
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: e5ceb9a02402b984feecb95a82239be151c9f4e2
Gitweb:
https://git.kernel.org/tip/e5ceb9a02402b984feecb95a82239be151c9f4e2
Author:Arvind Sankar
AuthorDate:Sat, 10 Oct 2020 15:11:10 -04:00
Committer: Borislav Petkov
CommitterDate: Mon, 19 Oct 2020 12:47:21 +02:00
x86/boot/64: Initialize 5-level paging variables earlier
Commit
ca0e22d4f011 ("x86/boot/compressed/64: Always switch to own page table")
started using a new set of pagetables even without KASLR.
After that commit, initialize_identity_maps() is called before the
5-level paging variables are setup in choose_random_location(), which
will not work if 5-level paging is actually enabled.
Fix this by moving the initialization of __pgtable_l5_enabled,
pgdir_shift and ptrs_per_p4d into cleanup_trampoline(), which is called
immediately after the finalization of whether the kernel is executing
with 4- or 5-level paging. This will be earlier than anything that might
require those variables, and keeps the 4- vs 5-level paging code all in
one place.
Fixes: ca0e22d4f011 ("x86/boot/compressed/64: Always switch to own page table")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Joerg Roedel
Tested-by: Joerg Roedel
Tested-by: Kirill A. Shutemov
Link: https://lkml.kernel.org/r/20201010191110.4060905-1-nived...@alum.mit.edu
---
arch/x86/boot/compressed/ident_map_64.c | 6 --
arch/x86/boot/compressed/kaslr.c| 8
arch/x86/boot/compressed/pgtable_64.c | 16
3 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/arch/x86/boot/compressed/ident_map_64.c
b/arch/x86/boot/compressed/ident_map_64.c
index 063a60e..c6f7aef 100644
--- a/arch/x86/boot/compressed/ident_map_64.c
+++ b/arch/x86/boot/compressed/ident_map_64.c
@@ -33,12 +33,6 @@
#define __PAGE_OFFSET __PAGE_OFFSET_BASE
#include "../../mm/ident_map.c"
-#ifdef CONFIG_X86_5LEVEL
-unsigned int __pgtable_l5_enabled;
-unsigned int pgdir_shift = 39;
-unsigned int ptrs_per_p4d = 1;
-#endif
-
/* Used by PAGE_KERN* macros: */
pteval_t __default_kernel_pte_mask __read_mostly = ~0;
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index b59547c..b92fffb 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -840,14 +840,6 @@ void choose_random_location(unsigned long input,
return;
}
-#ifdef CONFIG_X86_5LEVEL
- if (__read_cr4() & X86_CR4_LA57) {
- __pgtable_l5_enabled = 1;
- pgdir_shift = 48;
- ptrs_per_p4d = 512;
- }
-#endif
-
boot_params->hdr.loadflags |= KASLR_FLAG;
if (IS_ENABLED(CONFIG_X86_32))
diff --git a/arch/x86/boot/compressed/pgtable_64.c
b/arch/x86/boot/compressed/pgtable_64.c
index 7d0394f..5def167 100644
--- a/arch/x86/boot/compressed/pgtable_64.c
+++ b/arch/x86/boot/compressed/pgtable_64.c
@@ -8,6 +8,13 @@
#define BIOS_START_MIN 0x2U/* 128K, less than this is
insane */
#define BIOS_START_MAX 0x9f000U/* 640K, absolute maximum */
+#ifdef CONFIG_X86_5LEVEL
+/* __pgtable_l5_enabled needs to be in .data to avoid being cleared along with
.bss */
+unsigned int __section(.data) __pgtable_l5_enabled;
+unsigned int __section(.data) pgdir_shift = 39;
+unsigned int __section(.data) ptrs_per_p4d = 1;
+#endif
+
struct paging_config {
unsigned long trampoline_start;
unsigned long l5_required;
@@ -198,4 +205,13 @@ void cleanup_trampoline(void *pgtable)
/* Restore trampoline memory */
memcpy(trampoline_32bit, trampoline_save, TRAMPOLINE_32BIT_SIZE);
+
+ /* Initialize variables for 5-level paging */
+#ifdef CONFIG_X86_5LEVEL
+ if (__read_cr4() & X86_CR4_LA57) {
+ __pgtable_l5_enabled = 1;
+ pgdir_shift = 48;
+ ptrs_per_p4d = 512;
+ }
+#endif
}
[tip: x86/asm] x86/asm: Replace __force_order with a memory clobber
The following commit has been merged into the x86/asm branch of tip:
Commit-ID: 3e626682046e30282979f7d71e054cd4c00069a7
Gitweb:
https://git.kernel.org/tip/3e626682046e30282979f7d71e054cd4c00069a7
Author:Arvind Sankar
AuthorDate:Wed, 02 Sep 2020 19:21:52 -04:00
Committer: Borislav Petkov
CommitterDate: Tue, 13 Oct 2020 11:23:15 +02:00
x86/asm: Replace __force_order with a memory clobber
The CRn accessor functions use __force_order as a dummy operand to
prevent the compiler from reordering CRn reads/writes with respect to
each other.
The fact that the asm is volatile should be enough to prevent this:
volatile asm statements should be executed in program order. However GCC
4.9.x and 5.x have a bug that might result in reordering. This was fixed
in 8.1, 7.3 and 6.5. Versions prior to these, including 5.x and 4.9.x,
may reorder volatile asm statements with respect to each other.
There are some issues with __force_order as implemented:
- It is used only as an input operand for the write functions, and hence
doesn't do anything additional to prevent reordering writes.
- It allows memory accesses to be cached/reordered across write
functions, but CRn writes affect the semantics of memory accesses, so
this could be dangerous.
- __force_order is not actually defined in the kernel proper, but the
LLVM toolchain can in some cases require a definition: LLVM (as well
as GCC 4.9) requires it for PIE code, which is why the compressed
kernel has a definition, but also the clang integrated assembler may
consider the address of __force_order to be significant, resulting in
a reference that requires a definition.
Fix this by:
- Using a memory clobber for the write functions to additionally prevent
caching/reordering memory accesses across CRn writes.
- Using a dummy input operand with an arbitrary constant address for the
read functions, instead of a global variable. This will prevent reads
from being reordered across writes, while allowing memory loads to be
cached/reordered across CRn reads, which should be safe.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Kees Cook
Reviewed-by: Miguel Ojeda
Tested-by: Nathan Chancellor
Tested-by: Sedat Dilek
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82602
Link: https://lore.kernel.org/lkml/20200527135329.1172644-1-a...@arndb.de/
Link: https://lkml.kernel.org/r/20200902232152.3709896-1-nived...@alum.mit.edu
---
arch/x86/boot/compressed/pgtable_64.c | 9 +
arch/x86/include/asm/special_insns.h | 28 +-
arch/x86/kernel/cpu/common.c | 4 ++--
3 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/arch/x86/boot/compressed/pgtable_64.c
b/arch/x86/boot/compressed/pgtable_64.c
index c886269..7d0394f 100644
--- a/arch/x86/boot/compressed/pgtable_64.c
+++ b/arch/x86/boot/compressed/pgtable_64.c
@@ -5,15 +5,6 @@
#include "pgtable.h"
#include "../string.h"
-/*
- * __force_order is used by special_insns.h asm code to force instruction
- * serialization.
- *
- * It is not referenced from the code, but GCC < 5 with -fPIE would fail
- * due to an undefined symbol. Define it to make these ancient GCCs work.
- */
-unsigned long __force_order;
-
#define BIOS_START_MIN 0x2U/* 128K, less than this is
insane */
#define BIOS_START_MAX 0x9f000U/* 640K, absolute maximum */
diff --git a/arch/x86/include/asm/special_insns.h
b/arch/x86/include/asm/special_insns.h
index 59a3e13..d6e3bb9 100644
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -11,45 +11,47 @@
#include
/*
- * Volatile isn't enough to prevent the compiler from reordering the
- * read/write functions for the control registers and messing everything up.
- * A memory clobber would solve the problem, but would prevent reordering of
- * all loads stores around it, which can hurt performance. Solution is to
- * use a variable and mimic reads and writes to it to enforce serialization
+ * The compiler should not reorder volatile asm statements with respect to each
+ * other: they should execute in program order. However GCC 4.9.x and 5.x have
+ * a bug (which was fixed in 8.1, 7.3 and 6.5) where they might reorder
+ * volatile asm. The write functions are not affected since they have memory
+ * clobbers preventing reordering. To prevent reads from being reordered with
+ * respect to writes, use a dummy memory operand.
*/
-extern unsigned long __force_order;
+
+#define __FORCE_ORDER "m"(*(unsigned int *)0x1000UL)
void native_write_cr0(unsigned long val);
static inline unsigned long native_read_cr0(void)
{
unsigned long val;
- asm volatile("mov %%cr0,%0\n\t" : "=r" (val), "=m" (__force_order));
+ asm volatile("mov %%cr0,%0\n\t" : "=r" (val) : __FORCE_ORDER);
return val;
}
static __always_inline unsigned long native_read_cr2(void)
{
unsigned long val;
- asm
[tip: x86/asm] x86/asm: Replace __force_order with a memory clobber
The following commit has been merged into the x86/asm branch of tip:
Commit-ID: aa5cacdc29d76a005cbbee018a47faa6e724dd2d
Gitweb:
https://git.kernel.org/tip/aa5cacdc29d76a005cbbee018a47faa6e724dd2d
Author:Arvind Sankar
AuthorDate:Wed, 02 Sep 2020 19:21:52 -04:00
Committer: Borislav Petkov
CommitterDate: Thu, 01 Oct 2020 10:31:48 +02:00
x86/asm: Replace __force_order with a memory clobber
The CRn accessor functions use __force_order as a dummy operand to
prevent the compiler from reordering CRn reads/writes with respect to
each other.
The fact that the asm is volatile should be enough to prevent this:
volatile asm statements should be executed in program order. However GCC
4.9.x and 5.x have a bug that might result in reordering. This was fixed
in 8.1, 7.3 and 6.5. Versions prior to these, including 5.x and 4.9.x,
may reorder volatile asm statements with respect to each other.
There are some issues with __force_order as implemented:
- It is used only as an input operand for the write functions, and hence
doesn't do anything additional to prevent reordering writes.
- It allows memory accesses to be cached/reordered across write
functions, but CRn writes affect the semantics of memory accesses, so
this could be dangerous.
- __force_order is not actually defined in the kernel proper, but the
LLVM toolchain can in some cases require a definition: LLVM (as well
as GCC 4.9) requires it for PIE code, which is why the compressed
kernel has a definition, but also the clang integrated assembler may
consider the address of __force_order to be significant, resulting in
a reference that requires a definition.
Fix this by:
- Using a memory clobber for the write functions to additionally prevent
caching/reordering memory accesses across CRn writes.
- Using a dummy input operand with an arbitrary constant address for the
read functions, instead of a global variable. This will prevent reads
from being reordered across writes, while allowing memory loads to be
cached/reordered across CRn reads, which should be safe.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Reviewed-by: Kees Cook
Reviewed-by: Miguel Ojeda
Tested-by: Nathan Chancellor
Tested-by: Sedat Dilek
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82602
Link: https://lore.kernel.org/lkml/20200527135329.1172644-1-a...@arndb.de/
Link: https://lkml.kernel.org/r/20200902232152.3709896-1-nived...@alum.mit.edu
---
arch/x86/boot/compressed/pgtable_64.c | 9 +
arch/x86/include/asm/special_insns.h | 28 +-
arch/x86/kernel/cpu/common.c | 4 ++--
3 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/arch/x86/boot/compressed/pgtable_64.c
b/arch/x86/boot/compressed/pgtable_64.c
index c886269..7d0394f 100644
--- a/arch/x86/boot/compressed/pgtable_64.c
+++ b/arch/x86/boot/compressed/pgtable_64.c
@@ -5,15 +5,6 @@
#include "pgtable.h"
#include "../string.h"
-/*
- * __force_order is used by special_insns.h asm code to force instruction
- * serialization.
- *
- * It is not referenced from the code, but GCC < 5 with -fPIE would fail
- * due to an undefined symbol. Define it to make these ancient GCCs work.
- */
-unsigned long __force_order;
-
#define BIOS_START_MIN 0x2U/* 128K, less than this is
insane */
#define BIOS_START_MAX 0x9f000U/* 640K, absolute maximum */
diff --git a/arch/x86/include/asm/special_insns.h
b/arch/x86/include/asm/special_insns.h
index 59a3e13..d6e3bb9 100644
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -11,45 +11,47 @@
#include
/*
- * Volatile isn't enough to prevent the compiler from reordering the
- * read/write functions for the control registers and messing everything up.
- * A memory clobber would solve the problem, but would prevent reordering of
- * all loads stores around it, which can hurt performance. Solution is to
- * use a variable and mimic reads and writes to it to enforce serialization
+ * The compiler should not reorder volatile asm statements with respect to each
+ * other: they should execute in program order. However GCC 4.9.x and 5.x have
+ * a bug (which was fixed in 8.1, 7.3 and 6.5) where they might reorder
+ * volatile asm. The write functions are not affected since they have memory
+ * clobbers preventing reordering. To prevent reads from being reordered with
+ * respect to writes, use a dummy memory operand.
*/
-extern unsigned long __force_order;
+
+#define __FORCE_ORDER "m"(*(unsigned int *)0x1000UL)
void native_write_cr0(unsigned long val);
static inline unsigned long native_read_cr0(void)
{
unsigned long val;
- asm volatile("mov %%cr0,%0\n\t" : "=r" (val), "=m" (__force_order));
+ asm volatile("mov %%cr0,%0\n\t" : "=r" (val) : __FORCE_ORDER);
return val;
}
static __always_inline unsigned long native_read_cr2(void)
{
unsigned long val;
- asm
[tip: efi/core] efi/x86: Add a quirk to support command line arguments on Dell EFI firmware
The following commit has been merged into the efi/core branch of tip:
Commit-ID: 4a568ce29d3f48df95919f82a80e4b9be7ea0dc1
Gitweb:
https://git.kernel.org/tip/4a568ce29d3f48df95919f82a80e4b9be7ea0dc1
Author:Arvind Sankar
AuthorDate:Mon, 14 Sep 2020 17:35:35 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 17 Sep 2020 10:19:53 +03:00
efi/x86: Add a quirk to support command line arguments on Dell EFI firmware
At least some versions of Dell EFI firmware pass the entire
EFI_LOAD_OPTION descriptor, rather than just the OptionalData part, to
the loaded image. This was verified with firmware revision 2.15.0 on a
Dell Precision T3620 by Jacobo Pantoja.
To handle this, add a quirk to check if the options look like a valid
EFI_LOAD_OPTION descriptor, and if so, use the OptionalData part as the
command line.
Signed-off-by: Arvind Sankar
Reported-by: Jacobo Pantoja
Link:
https://lore.kernel.org/linux-efi/20200907170021.ga2284...@rani.riverdale.lan/
Link: https://lore.kernel.org/r/20200914213535.933454-2-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 101 +++-
drivers/firmware/efi/libstub/efistub.h | 31 +-
drivers/firmware/efi/libstub/file.c| 5 +-
3 files changed, 135 insertions(+), 2 deletions(-)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
b/drivers/firmware/efi/libstub/efi-stub-helper.c
index 6bca70b..4dbf04c 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -231,6 +231,102 @@ efi_status_t efi_parse_options(char const *cmdline)
}
/*
+ * The EFI_LOAD_OPTION descriptor has the following layout:
+ * u32 Attributes;
+ * u16 FilePathListLength;
+ * u16 Description[];
+ * efi_device_path_protocol_t FilePathList[];
+ * u8 OptionalData[];
+ *
+ * This function validates and unpacks the variable-size data fields.
+ */
+static
+bool efi_load_option_unpack(efi_load_option_unpacked_t *dest,
+ const efi_load_option_t *src, size_t size)
+{
+ const void *pos;
+ u16 c;
+ efi_device_path_protocol_t header;
+ const efi_char16_t *description;
+ const efi_device_path_protocol_t *file_path_list;
+
+ if (size < offsetof(efi_load_option_t, variable_data))
+ return false;
+ pos = src->variable_data;
+ size -= offsetof(efi_load_option_t, variable_data);
+
+ if ((src->attributes & ~EFI_LOAD_OPTION_MASK) != 0)
+ return false;
+
+ /* Scan description. */
+ description = pos;
+ do {
+ if (size < sizeof(c))
+ return false;
+ c = *(const u16 *)pos;
+ pos += sizeof(c);
+ size -= sizeof(c);
+ } while (c != L'\0');
+
+ /* Scan file_path_list. */
+ file_path_list = pos;
+ do {
+ if (size < sizeof(header))
+ return false;
+ header = *(const efi_device_path_protocol_t *)pos;
+ if (header.length < sizeof(header))
+ return false;
+ if (size < header.length)
+ return false;
+ pos += header.length;
+ size -= header.length;
+ } while ((header.type != EFI_DEV_END_PATH && header.type !=
EFI_DEV_END_PATH2) ||
+(header.sub_type != EFI_DEV_END_ENTIRE));
+ if (pos != (const void *)file_path_list + src->file_path_list_length)
+ return false;
+
+ dest->attributes = src->attributes;
+ dest->file_path_list_length = src->file_path_list_length;
+ dest->description = description;
+ dest->file_path_list = file_path_list;
+ dest->optional_data_size = size;
+ dest->optional_data = size ? pos : NULL;
+
+ return true;
+}
+
+/*
+ * At least some versions of Dell firmware pass the entire contents of the
+ * Boot variable, i.e. the EFI_LOAD_OPTION descriptor, rather than just the
+ * OptionalData field.
+ *
+ * Detect this case and extract OptionalData.
+ */
+void efi_apply_loadoptions_quirk(const void **load_options, int
*load_options_size)
+{
+ const efi_load_option_t *load_option = *load_options;
+ efi_load_option_unpacked_t load_option_unpacked;
+
+ if (!IS_ENABLED(CONFIG_X86))
+ return;
+ if (!load_option)
+ return;
+ if (*load_options_size < sizeof(*load_option))
+ return;
+ if ((load_option->attributes & ~EFI_LOAD_OPTION_BOOT_MASK) != 0)
+ return;
+
+ if (!efi_load_option_unpack(_option_unpacked, load_option,
*load_options_size))
+ return;
+
+ efi_warn_once(FW_BUG "LoadOptions is an EFI_LOAD_OPTION descriptor\n");
+ efi_warn_once(FW_BUG "Using OptionalData as a workaround\n");
+
+ *load_options = load_option_unpacked.optional_data;
+
[tip: efi/core] efi/libstub: Add efi_warn and *_once logging helpers
The following commit has been merged into the efi/core branch of tip:
Commit-ID: c1df5e0c5796f775e60d1aec0b52f6d03d66ccd4
Gitweb:
https://git.kernel.org/tip/c1df5e0c5796f775e60d1aec0b52f6d03d66ccd4
Author:Arvind Sankar
AuthorDate:Mon, 14 Sep 2020 17:35:34 -04:00
Committer: Ard Biesheuvel
CommitterDate: Wed, 16 Sep 2020 18:53:42 +03:00
efi/libstub: Add efi_warn and *_once logging helpers
Add an efi_warn logging helper for warnings, and implement an analog of
printk_once for once-only logging.
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200914213535.933454-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efistub.h | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/firmware/efi/libstub/efistub.h
b/drivers/firmware/efi/libstub/efistub.h
index 27cdcb1..9ea87a2 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -52,11 +52,34 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
#define efi_info(fmt, ...) \
efi_printk(KERN_INFO fmt, ##__VA_ARGS__)
+#define efi_warn(fmt, ...) \
+ efi_printk(KERN_WARNING "WARNING: " fmt, ##__VA_ARGS__)
#define efi_err(fmt, ...) \
efi_printk(KERN_ERR "ERROR: " fmt, ##__VA_ARGS__)
#define efi_debug(fmt, ...) \
efi_printk(KERN_DEBUG "DEBUG: " fmt, ##__VA_ARGS__)
+#define efi_printk_once(fmt, ...) \
+({ \
+ static bool __print_once; \
+ bool __ret_print_once = !__print_once; \
+ \
+ if (!__print_once) {\
+ __print_once = true;\
+ efi_printk(fmt, ##__VA_ARGS__); \
+ } \
+ __ret_print_once; \
+})
+
+#define efi_info_once(fmt, ...) \
+ efi_printk_once(KERN_INFO fmt, ##__VA_ARGS__)
+#define efi_warn_once(fmt, ...) \
+ efi_printk_once(KERN_WARNING "WARNING: " fmt, ##__VA_ARGS__)
+#define efi_err_once(fmt, ...) \
+ efi_printk_once(KERN_ERR "ERROR: " fmt, ##__VA_ARGS__)
+#define efi_debug_once(fmt, ...) \
+ efi_printk_once(KERN_DEBUG "DEBUG: " fmt, ##__VA_ARGS__)
+
/* Helper macros for the usual case of using simple C variables: */
#ifndef fdt_setprop_inplace_var
#define fdt_setprop_inplace_var(fdt, node_offset, name, var) \
[tip: efi/core] efi/x86: Add a quirk to support command line arguments on Dell EFI firmware
The following commit has been merged into the efi/core branch of tip:
Commit-ID: d2778b4891fb600e38c910cb4fe3b4d6e00981e7
Gitweb:
https://git.kernel.org/tip/d2778b4891fb600e38c910cb4fe3b4d6e00981e7
Author:Arvind Sankar
AuthorDate:Mon, 14 Sep 2020 17:35:35 -04:00
Committer: Ard Biesheuvel
CommitterDate: Wed, 16 Sep 2020 18:53:42 +03:00
efi/x86: Add a quirk to support command line arguments on Dell EFI firmware
At least some versions of Dell EFI firmware pass the entire
EFI_LOAD_OPTION descriptor, rather than just the OptionalData part, to
the loaded image. This was verified with firmware revision 2.15.0 on a
Dell Precision T3620 by Jacobo Pontaja.
To handle this, add a quirk to check if the options look like a valid
EFI_LOAD_OPTION descriptor, and if so, use the OptionalData part as the
command line.
Signed-off-by: Arvind Sankar
Reported-by: Jacobo Pantoja
Link:
https://lore.kernel.org/linux-efi/20200907170021.ga2284...@rani.riverdale.lan/
Link: https://lore.kernel.org/r/20200914213535.933454-2-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 101 +++-
drivers/firmware/efi/libstub/efistub.h | 31 +-
drivers/firmware/efi/libstub/file.c| 5 +-
3 files changed, 135 insertions(+), 2 deletions(-)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
b/drivers/firmware/efi/libstub/efi-stub-helper.c
index 6bca70b..4dbf04c 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -231,6 +231,102 @@ efi_status_t efi_parse_options(char const *cmdline)
}
/*
+ * The EFI_LOAD_OPTION descriptor has the following layout:
+ * u32 Attributes;
+ * u16 FilePathListLength;
+ * u16 Description[];
+ * efi_device_path_protocol_t FilePathList[];
+ * u8 OptionalData[];
+ *
+ * This function validates and unpacks the variable-size data fields.
+ */
+static
+bool efi_load_option_unpack(efi_load_option_unpacked_t *dest,
+ const efi_load_option_t *src, size_t size)
+{
+ const void *pos;
+ u16 c;
+ efi_device_path_protocol_t header;
+ const efi_char16_t *description;
+ const efi_device_path_protocol_t *file_path_list;
+
+ if (size < offsetof(efi_load_option_t, variable_data))
+ return false;
+ pos = src->variable_data;
+ size -= offsetof(efi_load_option_t, variable_data);
+
+ if ((src->attributes & ~EFI_LOAD_OPTION_MASK) != 0)
+ return false;
+
+ /* Scan description. */
+ description = pos;
+ do {
+ if (size < sizeof(c))
+ return false;
+ c = *(const u16 *)pos;
+ pos += sizeof(c);
+ size -= sizeof(c);
+ } while (c != L'\0');
+
+ /* Scan file_path_list. */
+ file_path_list = pos;
+ do {
+ if (size < sizeof(header))
+ return false;
+ header = *(const efi_device_path_protocol_t *)pos;
+ if (header.length < sizeof(header))
+ return false;
+ if (size < header.length)
+ return false;
+ pos += header.length;
+ size -= header.length;
+ } while ((header.type != EFI_DEV_END_PATH && header.type !=
EFI_DEV_END_PATH2) ||
+(header.sub_type != EFI_DEV_END_ENTIRE));
+ if (pos != (const void *)file_path_list + src->file_path_list_length)
+ return false;
+
+ dest->attributes = src->attributes;
+ dest->file_path_list_length = src->file_path_list_length;
+ dest->description = description;
+ dest->file_path_list = file_path_list;
+ dest->optional_data_size = size;
+ dest->optional_data = size ? pos : NULL;
+
+ return true;
+}
+
+/*
+ * At least some versions of Dell firmware pass the entire contents of the
+ * Boot variable, i.e. the EFI_LOAD_OPTION descriptor, rather than just the
+ * OptionalData field.
+ *
+ * Detect this case and extract OptionalData.
+ */
+void efi_apply_loadoptions_quirk(const void **load_options, int
*load_options_size)
+{
+ const efi_load_option_t *load_option = *load_options;
+ efi_load_option_unpacked_t load_option_unpacked;
+
+ if (!IS_ENABLED(CONFIG_X86))
+ return;
+ if (!load_option)
+ return;
+ if (*load_options_size < sizeof(*load_option))
+ return;
+ if ((load_option->attributes & ~EFI_LOAD_OPTION_BOOT_MASK) != 0)
+ return;
+
+ if (!efi_load_option_unpack(_option_unpacked, load_option,
*load_options_size))
+ return;
+
+ efi_warn_once(FW_BUG "LoadOptions is an EFI_LOAD_OPTION descriptor\n");
+ efi_warn_once(FW_BUG "Using OptionalData as a workaround\n");
+
+ *load_options = load_option_unpacked.optional_data;
+
[tip: x86/urgent] x86/boot/compressed: Disable relocation relaxation
The following commit has been merged into the x86/urgent branch of tip: Commit-ID: 09e43968db40c33a73e9ddbfd937f46d5c334924 Gitweb: https://git.kernel.org/tip/09e43968db40c33a73e9ddbfd937f46d5c334924 Author:Arvind Sankar AuthorDate:Tue, 11 Aug 2020 20:43:08 -04:00 Committer: Ingo Molnar CommitterDate: Mon, 14 Sep 2020 11:14:45 +02:00 x86/boot/compressed: Disable relocation relaxation The x86-64 psABI [0] specifies special relocation types (R_X86_64_[REX_]GOTPCRELX) for indirection through the Global Offset Table, semantically equivalent to R_X86_64_GOTPCREL, which the linker can take advantage of for optimization (relaxation) at link time. This is supported by LLD and binutils versions 2.26 onwards. The compressed kernel is position-independent code, however, when using LLD or binutils versions before 2.27, it must be linked without the -pie option. In this case, the linker may optimize certain instructions into a non-position-independent form, by converting foo@GOTPCREL(%rip) to $foo. This potential issue has been present with LLD and binutils-2.26 for a long time, but it has never manifested itself before now: - LLD and binutils-2.26 only relax movqfoo@GOTPCREL(%rip), %reg to leaqfoo(%rip), %reg which is still position-independent, rather than mov $foo, %reg which is permitted by the psABI when -pie is not enabled. - GCC happens to only generate GOTPCREL relocations on mov instructions. - CLang does generate GOTPCREL relocations on non-mov instructions, but when building the compressed kernel, it uses its integrated assembler (due to the redefinition of KBUILD_CFLAGS dropping -no-integrated-as), which has so far defaulted to not generating the GOTPCRELX relocations. Nick Desaulniers reports [1,2]: "A recent change [3] to a default value of configuration variable (ENABLE_X86_RELAX_RELOCATIONS OFF -> ON) in LLVM now causes Clang's integrated assembler to emit R_X86_64_GOTPCRELX/R_X86_64_REX_GOTPCRELX relocations. LLD will relax instructions with these relocations based on whether the image is being linked as position independent or not. When not, then LLD will relax these instructions to use absolute addressing mode (R_RELAX_GOT_PC_NOPIC). This causes kernels built with Clang and linked with LLD to fail to boot." Patch series [4] is a solution to allow the compressed kernel to be linked with -pie unconditionally, but even if merged is unlikely to be backported. As a simple solution that can be applied to stable as well, prevent the assembler from generating the relaxed relocation types using the -mrelax-relocations=no option. For ease of backporting, do this unconditionally. [0] https://gitlab.com/x86-psABIs/x86-64-ABI/-/blob/master/x86-64-ABI/linker-optimization.tex#L65 [1] https://lore.kernel.org/lkml/20200807194100.3570838-1-ndesaulni...@google.com/ [2] https://github.com/ClangBuiltLinux/linux/issues/1121 [3] https://reviews.llvm.org/rGc41a18cf61790fc898dcda1055c3efbf442c14c0 [4] https://lore.kernel.org/lkml/20200731202738.2577854-1-nived...@alum.mit.edu/ Reported-by: Nick Desaulniers Signed-off-by: Arvind Sankar Signed-off-by: Ingo Molnar Tested-by: Nick Desaulniers Tested-by: Sedat Dilek Acked-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Cc: sta...@vger.kernel.org Link: https://lore.kernel.org/r/20200812004308.1448603-1-nived...@alum.mit.edu --- arch/x86/boot/compressed/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 3962f59..ff7894f 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -43,6 +43,8 @@ KBUILD_CFLAGS += -Wno-pointer-sign KBUILD_CFLAGS += $(call cc-option,-fmacro-prefix-map=$(srctree)/=) KBUILD_CFLAGS += -fno-asynchronous-unwind-tables KBUILD_CFLAGS += -D__DISABLE_EXPORTS +# Disable relocation relaxation in case the link is not PIE. +KBUILD_CFLAGS += $(call as-option,-Wa$(comma)-mrelax-relocations=no) KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__ GCOV_PROFILE := n
[tip: x86/fpu] x86/fpu: Allow multiple bits in clearcpuid= parameter
The following commit has been merged into the x86/fpu branch of tip:
Commit-ID: 0a4bb5e5507a585532cc413125b921c8546fc39f
Gitweb:
https://git.kernel.org/tip/0a4bb5e5507a585532cc413125b921c8546fc39f
Author:Arvind Sankar
AuthorDate:Mon, 07 Sep 2020 17:39:19 -04:00
Committer: Borislav Petkov
CommitterDate: Thu, 10 Sep 2020 18:32:05 +02:00
x86/fpu: Allow multiple bits in clearcpuid= parameter
Commit
0c2a3913d6f5 ("x86/fpu: Parse clearcpuid= as early XSAVE argument")
changed clearcpuid parsing from __setup() to cmdline_find_option().
While the __setup() function would have been called for each clearcpuid=
parameter on the command line, cmdline_find_option() will only return
the last one, so the change effectively made it impossible to disable
more than one bit.
Allow a comma-separated list of bit numbers as the argument for
clearcpuid to allow multiple bits to be disabled again. Log the bits
being disabled for informational purposes.
Also fix the check on the return value of cmdline_find_option(). It
returns -1 when the option is not found, so testing as a boolean is
incorrect.
Fixes: 0c2a3913d6f5 ("x86/fpu: Parse clearcpuid= as early XSAVE argument")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20200907213919.2423441-1-nived...@alum.mit.edu
---
Documentation/admin-guide/kernel-parameters.txt | 2 +-
arch/x86/kernel/fpu/init.c | 30 +++-
2 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt
b/Documentation/admin-guide/kernel-parameters.txt
index a106874..ffe8643 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -577,7 +577,7 @@
loops can be debugged more effectively on production
systems.
- clearcpuid=BITNUM [X86]
+ clearcpuid=BITNUM[,BITNUM...] [X86]
Disable CPUID feature X for the kernel. See
arch/x86/include/asm/cpufeatures.h for the valid bit
numbers. Note the Linux specific bits are not
necessarily
diff --git a/arch/x86/kernel/fpu/init.c b/arch/x86/kernel/fpu/init.c
index 61ddc3a..f8ff895 100644
--- a/arch/x86/kernel/fpu/init.c
+++ b/arch/x86/kernel/fpu/init.c
@@ -243,9 +243,9 @@ static void __init fpu__init_system_ctx_switch(void)
*/
static void __init fpu__init_parse_early_param(void)
{
- char arg[32];
+ char arg[128];
char *argptr = arg;
- int bit;
+ int arglen, res, bit;
#ifdef CONFIG_X86_32
if (cmdline_find_option_bool(boot_command_line, "no387"))
@@ -268,12 +268,26 @@ static void __init fpu__init_parse_early_param(void)
if (cmdline_find_option_bool(boot_command_line, "noxsaves"))
setup_clear_cpu_cap(X86_FEATURE_XSAVES);
- if (cmdline_find_option(boot_command_line, "clearcpuid", arg,
- sizeof(arg)) &&
- get_option(, ) &&
- bit >= 0 &&
- bit < NCAPINTS * 32)
- setup_clear_cpu_cap(bit);
+ arglen = cmdline_find_option(boot_command_line, "clearcpuid", arg,
sizeof(arg));
+ if (arglen <= 0)
+ return;
+
+ pr_info("Clearing CPUID bits:");
+ do {
+ res = get_option(, );
+ if (res == 0 || res == 3)
+ break;
+
+ /* If the argument was too long, the last bit may be cut off */
+ if (res == 1 && arglen >= sizeof(arg))
+ break;
+
+ if (bit >= 0 && bit < NCAPINTS * 32) {
+ pr_cont(" " X86_CAP_FMT, x86_cap_flag(bit));
+ setup_clear_cpu_cap(bit);
+ }
+ } while (res == 2);
+ pr_cont("\n");
}
/*
[tip: x86/urgent] x86/cmdline: Disable jump tables for cmdline.c
The following commit has been merged into the x86/urgent branch of tip: Commit-ID: aef0148f3606117352053c015cb33734e9ee7397 Gitweb: https://git.kernel.org/tip/aef0148f3606117352053c015cb33734e9ee7397 Author:Arvind Sankar AuthorDate:Wed, 02 Sep 2020 22:30:56 -04:00 Committer: Ingo Molnar CommitterDate: Thu, 03 Sep 2020 10:59:16 +02:00 x86/cmdline: Disable jump tables for cmdline.c When CONFIG_RETPOLINE is disabled, Clang uses a jump table for the switch statement in cmdline_find_option (jump tables are disabled when CONFIG_RETPOLINE is enabled). This function is called very early in boot from sme_enable() if CONFIG_AMD_MEM_ENCRYPT is enabled. At this time, the kernel is still executing out of the identity mapping, but the jump table will contain virtual addresses. Fix this by disabling jump tables for cmdline.c when AMD_MEM_ENCRYPT is enabled. Signed-off-by: Arvind Sankar Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20200903023056.3914690-1-nived...@alum.mit.edu --- arch/x86/lib/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index d46fff1..aa06785 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -24,7 +24,7 @@ ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_cmdline.o = -pg endif -CFLAGS_cmdline.o := -fno-stack-protector +CFLAGS_cmdline.o := -fno-stack-protector -fno-jump-tables endif inat_tables_script = $(srctree)/arch/x86/tools/gen-insn-attr-x86.awk
[tip: efi/urgent] efi/libstub: Handle NULL cmdline
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: a37ca6a2af9df2972372b918f09390c9303acfbd
Gitweb:
https://git.kernel.org/tip/a37ca6a2af9df2972372b918f09390c9303acfbd
Author:Arvind Sankar
AuthorDate:Wed, 29 Jul 2020 15:33:00 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 20 Aug 2020 11:18:55 +02:00
efi/libstub: Handle NULL cmdline
Treat a NULL cmdline the same as empty. Although this is unlikely to
happen in practice, the x86 kernel entry does check for NULL cmdline and
handles it, so do it here as well.
Cc:
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200729193300.598448-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
b/drivers/firmware/efi/libstub/efi-stub-helper.c
index 37ff34e..f53652a 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -187,10 +187,14 @@ int efi_printk(const char *fmt, ...)
*/
efi_status_t efi_parse_options(char const *cmdline)
{
- size_t len = strlen(cmdline) + 1;
+ size_t len;
efi_status_t status;
char *str, *buf;
+ if (!cmdline)
+ return EFI_SUCCESS;
+
+ len = strlen(cmdline) + 1;
status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, len, (void
**));
if (status != EFI_SUCCESS)
return status;
[tip: efi/urgent] efi/libstub: Handle unterminated cmdline
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: 8a8a3237a78cbc0557f0eb16a89f16d616323e99
Gitweb:
https://git.kernel.org/tip/8a8a3237a78cbc0557f0eb16a89f16d616323e99
Author:Arvind Sankar
AuthorDate:Thu, 13 Aug 2020 14:58:11 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 20 Aug 2020 11:18:58 +02:00
efi/libstub: Handle unterminated cmdline
Make the command line parsing more robust, by handling the case it is
not NUL-terminated.
Use strnlen instead of strlen, and make sure that the temporary copy is
NUL-terminated before parsing.
Cc:
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200813185811.554051-4-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
b/drivers/firmware/efi/libstub/efi-stub-helper.c
index f53652a..f735db5 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -194,12 +194,14 @@ efi_status_t efi_parse_options(char const *cmdline)
if (!cmdline)
return EFI_SUCCESS;
- len = strlen(cmdline) + 1;
+ len = strnlen(cmdline, COMMAND_LINE_SIZE - 1) + 1;
status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, len, (void
**));
if (status != EFI_SUCCESS)
return status;
- str = skip_spaces(memcpy(buf, cmdline, len));
+ memcpy(buf, cmdline, len - 1);
+ buf[len - 1] = '\0';
+ str = skip_spaces(buf);
while (*str) {
char *param, *val;
[tip: efi/urgent] efi/libstub: Stop parsing arguments at "--"
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: 1fd9717d75df68e3c3509b8e7b1138ca63472f88
Gitweb:
https://git.kernel.org/tip/1fd9717d75df68e3c3509b8e7b1138ca63472f88
Author:Arvind Sankar
AuthorDate:Sat, 25 Jul 2020 11:59:16 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 20 Aug 2020 11:18:52 +02:00
efi/libstub: Stop parsing arguments at "--"
Arguments after "--" are arguments for init, not for the kernel.
Cc:
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200725155916.1376773-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/efi-stub-helper.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c
b/drivers/firmware/efi/libstub/efi-stub-helper.c
index 6bca70b..37ff34e 100644
--- a/drivers/firmware/efi/libstub/efi-stub-helper.c
+++ b/drivers/firmware/efi/libstub/efi-stub-helper.c
@@ -201,6 +201,8 @@ efi_status_t efi_parse_options(char const *cmdline)
char *param, *val;
str = next_arg(str, , );
+ if (!val && !strcmp(param, "--"))
+ break;
if (!strcmp(param, "nokaslr")) {
efi_nokaslr = true;
[tip: efi/urgent] efi/x86: Mark kernel rodata non-executable for mixed mode
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: c8502eb2d43b6b9b1dc382299a4d37031be63876
Gitweb:
https://git.kernel.org/tip/c8502eb2d43b6b9b1dc382299a4d37031be63876
Author:Arvind Sankar
AuthorDate:Fri, 17 Jul 2020 15:45:26 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 20 Aug 2020 11:18:36 +02:00
efi/x86: Mark kernel rodata non-executable for mixed mode
When remapping the kernel rodata section RO in the EFI pagetables, the
protection flags that were used for the text section are being reused,
but the rodata section should not be marked executable.
Cc:
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200717194526.3452089-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
arch/x86/platform/efi/efi_64.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 413583f..6af4da1 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -259,6 +259,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap,
unsigned num_pages)
npages = (__end_rodata - __start_rodata) >> PAGE_SHIFT;
rodata = __pa(__start_rodata);
pfn = rodata >> PAGE_SHIFT;
+
+ pf = _PAGE_NX | _PAGE_ENC;
if (kernel_map_pages_in_pgd(pgd, pfn, rodata, npages, pf)) {
pr_err("Failed to map kernel rodata 1:1\n");
return 1;
[tip: x86/boot] x86/boot: Add .text.* to setup.ld
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 2e7a858ba843d2e6ceab1ba996805411de51b340
Gitweb:
https://git.kernel.org/tip/2e7a858ba843d2e6ceab1ba996805411de51b340
Author:Arvind Sankar
AuthorDate:Fri, 31 Jul 2020 16:07:48 -07:00
Committer: Ingo Molnar
CommitterDate: Fri, 14 Aug 2020 12:52:35 +02:00
x86/boot: Add .text.* to setup.ld
GCC puts the main function into .text.startup when compiled with -Os (or
-O2). This results in arch/x86/boot/main.c having a .text.startup
section which is currently not included explicitly in the linker script
setup.ld in the same directory.
The BFD linker places this orphan section immediately after .text, so
this still works. However, LLD git, since [1], is choosing to place it
immediately after the .bstext section instead (this is the first code
section). This plays havoc with the section layout that setup.elf
requires to create the setup header, for eg on 64-bit:
LD arch/x86/boot/setup.elf
ld.lld: error: section .text.startup file range overlaps with .header
>>> .text.startup range is [0x200040, 0x2001FE]
>>> .header range is [0x2001EF, 0x20026B]
ld.lld: error: section .header file range overlaps with .bsdata
>>> .header range is [0x2001EF, 0x20026B]
>>> .bsdata range is [0x2001FF, 0x200398]
ld.lld: error: section .bsdata file range overlaps with .entrytext
>>> .bsdata range is [0x2001FF, 0x200398]
>>> .entrytext range is [0x20026C, 0x2002D3]
ld.lld: error: section .text.startup virtual address range overlaps
with .header
>>> .text.startup range is [0x40, 0x1FE]
>>> .header range is [0x1EF, 0x26B]
ld.lld: error: section .header virtual address range overlaps with
.bsdata
>>> .header range is [0x1EF, 0x26B]
>>> .bsdata range is [0x1FF, 0x398]
ld.lld: error: section .bsdata virtual address range overlaps with
.entrytext
>>> .bsdata range is [0x1FF, 0x398]
>>> .entrytext range is [0x26C, 0x2D3]
ld.lld: error: section .text.startup load address range overlaps with
.header
>>> .text.startup range is [0x40, 0x1FE]
>>> .header range is [0x1EF, 0x26B]
ld.lld: error: section .header load address range overlaps with
.bsdata
>>> .header range is [0x1EF, 0x26B]
>>> .bsdata range is [0x1FF, 0x398]
ld.lld: error: section .bsdata load address range overlaps with
.entrytext
>>> .bsdata range is [0x1FF, 0x398]
>>> .entrytext range is [0x26C, 0x2D3]
Add .text.* to the .text output section to fix this, and also prevent
any future surprises if the compiler decides to create other such
sections.
[1] https://reviews.llvm.org/D75225
Signed-off-by: Arvind Sankar
Signed-off-by: Kees Cook
Signed-off-by: Ingo Molnar
Tested-by: Nick Desaulniers
Tested-by: Sedat Dilek
Reviewed-by: Kees Cook
Reviewed-by: Ard Biesheuvel
Reviewed-by: Fangrui Song
Link: https://lore.kernel.org/r/20200731230820.1742553-5-keesc...@chromium.org
---
arch/x86/boot/setup.ld | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/boot/setup.ld b/arch/x86/boot/setup.ld
index 24c9552..49546c2 100644
--- a/arch/x86/boot/setup.ld
+++ b/arch/x86/boot/setup.ld
@@ -20,7 +20,7 @@ SECTIONS
.initdata : { *(.initdata) }
__end_init = .;
- .text : { *(.text) }
+ .text : { *(.text .text.*) }
.text32 : { *(.text32) }
. = ALIGN(16);
[tip: x86/boot] x86/boot: Check that there are no run-time relocations
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 527afc212231ea9d585b7709c0ab73263ecf0c85
Gitweb:
https://git.kernel.org/tip/527afc212231ea9d585b7709c0ab73263ecf0c85
Author:Arvind Sankar
AuthorDate:Fri, 31 Jul 2020 16:07:51 -07:00
Committer: Ingo Molnar
CommitterDate: Fri, 14 Aug 2020 12:52:35 +02:00
x86/boot: Check that there are no run-time relocations
Add a linker script check that there are no run-time relocations, and
remove the old one that tries to check via looking for specially-named
sections in the object files.
Drop the tests for -fPIE compiler option and -pie linker option, as they
are available in all supported gcc and binutils versions (as well as
clang and lld).
Signed-off-by: Arvind Sankar
Signed-off-by: Kees Cook
Signed-off-by: Ingo Molnar
Tested-by: Nick Desaulniers
Tested-by: Sedat Dilek
Reviewed-by: Kees Cook
Reviewed-by: Ard Biesheuvel
Reviewed-by: Fangrui Song
Reviewed-by: Sedat Dilek
Link: https://lore.kernel.org/r/20200731230820.1742553-8-keesc...@chromium.org
---
arch/x86/boot/compressed/Makefile | 28 ++---
arch/x86/boot/compressed/vmlinux.lds.S | 8 +++-
2 files changed, 11 insertions(+), 25 deletions(-)
diff --git a/arch/x86/boot/compressed/Makefile
b/arch/x86/boot/compressed/Makefile
index 7d25089..753d572 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -29,7 +29,7 @@ targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2
vmlinux.bin.lzma \
vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 vmlinux.bin.zst
KBUILD_CFLAGS := -m$(BITS) -O2
-KBUILD_CFLAGS += -fno-strict-aliasing $(call cc-option, -fPIE, -fPIC)
+KBUILD_CFLAGS += -fno-strict-aliasing -fPIE
KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
cflags-$(CONFIG_X86_32) := -march=i386
cflags-$(CONFIG_X86_64) := -mcmodel=small
@@ -52,7 +52,7 @@ UBSAN_SANITIZE :=n
KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
# Compressed kernel should be built as PIE since it may be loaded at any
# address by the bootloader.
-LDFLAGS_vmlinux := $(call ld-option, -pie) $(call ld-option,
--no-dynamic-linker)
+LDFLAGS_vmlinux := -pie $(call ld-option, --no-dynamic-linker)
LDFLAGS_vmlinux += -T
hostprogs := mkpiggy
@@ -87,30 +87,8 @@ vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o
vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o
efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
-# The compressed kernel is built with -fPIC/-fPIE so that a boot loader
-# can place it anywhere in memory and it will still run. However, since
-# it is executed as-is without any ELF relocation processing performed
-# (and has already had all relocation sections stripped from the binary),
-# none of the code can use data relocations (e.g. static assignments of
-# pointer values), since they will be meaningless at runtime. This check
-# will refuse to link the vmlinux if any of these relocations are found.
-quiet_cmd_check_data_rel = DATAREL $@
-define cmd_check_data_rel
- for obj in $(filter %.o,$^); do \
- $(READELF) -S $$obj | grep -qF .rel.local && { \
- echo "error: $$obj has data relocations!" >&2; \
- exit 1; \
- } || true; \
- done
-endef
-
-# We need to run two commands under "if_changed", so merge them into a
-# single invocation.
-quiet_cmd_check-and-link-vmlinux = LD $@
- cmd_check-and-link-vmlinux = $(cmd_check_data_rel); $(cmd_ld)
-
$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
- $(call if_changed,check-and-link-vmlinux)
+ $(call if_changed,ld)
OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
$(obj)/vmlinux.bin: vmlinux FORCE
diff --git a/arch/x86/boot/compressed/vmlinux.lds.S
b/arch/x86/boot/compressed/vmlinux.lds.S
index a4a4a59..29df99b 100644
--- a/arch/x86/boot/compressed/vmlinux.lds.S
+++ b/arch/x86/boot/compressed/vmlinux.lds.S
@@ -42,6 +42,12 @@ SECTIONS
*(.rodata.*)
_erodata = . ;
}
+ .rel.dyn : {
+ *(.rel.*)
+ }
+ .rela.dyn : {
+ *(.rela.*)
+ }
.got : {
*(.got)
}
@@ -85,3 +91,5 @@ ASSERT(SIZEOF(.got.plt) == 0 || SIZEOF(.got.plt) == 0x18,
"Unexpected GOT/PLT en
#else
ASSERT(SIZEOF(.got.plt) == 0 || SIZEOF(.got.plt) == 0xc, "Unexpected GOT/PLT
entries detected!")
#endif
+
+ASSERT(SIZEOF(.rel.dyn) == 0 && SIZEOF(.rela.dyn) == 0, "Unexpected run-time
relocations detected!")
[tip: x86/boot] x86/boot: Remove run-time relocations from .head.text code
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: a2c4fc4d4e2c40b07534094810d915c7354d84a7
Gitweb:
https://git.kernel.org/tip/a2c4fc4d4e2c40b07534094810d915c7354d84a7
Author:Arvind Sankar
AuthorDate:Fri, 31 Jul 2020 16:07:49 -07:00
Committer: Ingo Molnar
CommitterDate: Fri, 14 Aug 2020 12:52:35 +02:00
x86/boot: Remove run-time relocations from .head.text code
The assembly code in head_{32,64}.S, while meant to be
position-independent, generates run-time relocations because it uses
instructions such as:
lealgdt(%edx), %eax
which make the assembler and linker think that the code is using %edx as
an index into gdt, and hence gdt needs to be relocated to its run-time
address.
On 32-bit, with lld Dmitry Golovin reports that this results in a
link-time error with default options (i.e. unless -z notext is
explicitly passed):
LD arch/x86/boot/compressed/vmlinux
ld.lld: error: can't create dynamic relocation R_386_32 against local
symbol in readonly segment; recompile object files with -fPIC or pass
'-Wl,-z,notext' to allow text relocations in the output
With the BFD linker, this generates a warning during the build, if
--warn-shared-textrel is enabled, which at least Gentoo enables by
default:
LD arch/x86/boot/compressed/vmlinux
ld: arch/x86/boot/compressed/head_32.o: warning: relocation in read-only
section `.head.text'
ld: warning: creating a DT_TEXTREL in object
On 64-bit, it is not possible to link the kernel as -pie with lld, and
it is only possible with a BFD linker that supports -z noreloc-overflow,
i.e. versions >2.26. This is because these instructions cannot really be
relocated: the displacement field is only 32-bits wide, and thus cannot
be relocated for a 64-bit load address. The -z noreloc-overflow option
simply overrides the linker error, and results in R_X86_64_RELATIVE
relocations that apply a 64-bit relocation to a 32-bit field anyway.
This happens to work because nothing will process these run-time
relocations.
Start fixing this by removing relocations from .head.text:
- On 32-bit, use a base register that holds the address of the GOT and
reference symbol addresses using @GOTOFF, i.e.
lealgdt@GOTOFF(%edx), %eax
- On 64-bit, most of the code can (and already does) use %rip-relative
addressing, however the .code32 bits can't, and the 64-bit code also
needs to reference symbol addresses as they will be after moving the
compressed kernel to the end of the decompression buffer.
For these cases, reference the symbols as an offset to startup_32 to
avoid creating relocations, i.e.:
leal(gdt-startup_32)(%bp), %eax
This only works in .head.text as the subtraction cannot be represented
as a PC-relative relocation unless startup_32 is in the same section
as the code. Move efi32_pe_entry into .head.text so that it can use
the same method to avoid relocations.
Reported-by: Dmitry Golovin
Signed-off-by: Arvind Sankar
Signed-off-by: Kees Cook
Signed-off-by: Ingo Molnar
Tested-by: Nick Desaulniers
Tested-by: Sedat Dilek
Reviewed-by: Kees Cook
Reviewed-by: Ard Biesheuvel
Reviewed-by: Fangrui Song
Link: https://lore.kernel.org/r/20200731230820.1742553-6-keesc...@chromium.org
---
arch/x86/boot/compressed/head_32.S | 64 ++---
arch/x86/boot/compressed/head_64.S | 104 +---
2 files changed, 90 insertions(+), 78 deletions(-)
diff --git a/arch/x86/boot/compressed/head_32.S
b/arch/x86/boot/compressed/head_32.S
index 39f0bb4..8c1a4f5 100644
--- a/arch/x86/boot/compressed/head_32.S
+++ b/arch/x86/boot/compressed/head_32.S
@@ -33,26 +33,10 @@
#include
/*
- * The 32-bit x86 assembler in binutils 2.26 will generate R_386_GOT32X
- * relocation to get the symbol address in PIC. When the compressed x86
- * kernel isn't built as PIC, the linker optimizes R_386_GOT32X
- * relocations to their fixed symbol addresses. However, when the
- * compressed x86 kernel is loaded at a different address, it leads
- * to the following load failure:
- *
- * Failed to allocate space for phdrs
- *
- * during the decompression stage.
- *
- * If the compressed x86 kernel is relocatable at run-time, it should be
- * compiled with -fPIE, instead of -fPIC, if possible and should be built as
- * Position Independent Executable (PIE) so that linker won't optimize
- * R_386_GOT32X relocation to its fixed symbol address. Older
- * linkers generate R_386_32 relocations against locally defined symbols,
- * _bss, _ebss and _end, in PIE. It isn't wrong, just less optimal than
- * R_386_RELATIVE. But the x86 kernel fails to properly handle R_386_32
- * relocations when relocating the kernel. To generate R_386_RELATIVE
- * relocations, we mark _bss, _ebss and _end as hidden:
+ * These symbols needed to be marked as .hidden to prevent the BFD linker from
+ * generating R_386_32 (rather than R_386_RELATIVE) relocations for them when
+ * the
[tip: x86/boot] x86/boot: Remove run-time relocations from head_{32,64}.S
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 3f086189cd3641d212949ff044d8e4486c93d55e
Gitweb:
https://git.kernel.org/tip/3f086189cd3641d212949ff044d8e4486c93d55e
Author:Arvind Sankar
AuthorDate:Fri, 31 Jul 2020 16:07:50 -07:00
Committer: Ingo Molnar
CommitterDate: Fri, 14 Aug 2020 12:52:35 +02:00
x86/boot: Remove run-time relocations from head_{32,64}.S
The BFD linker generates run-time relocations for z_input_len and
z_output_len, even though they are absolute symbols.
This is fixed for binutils-2.35 [1]. Work around this for earlier
versions by defining two variables input_len and output_len in addition
to the symbols, and use them via position-independent references.
This eliminates the last two run-time relocations in the head code and
allows us to drop the -z noreloc-overflow flag to the linker.
Move the -pie and --no-dynamic-linker LDFLAGS to LDFLAGS_vmlinux instead
of KBUILD_LDFLAGS. There shouldn't be anything else getting linked, but
this is the more logical location for these flags, and modversions might
call the linker if an EXPORT_SYMBOL is left over accidentally in one of
the decompressors.
[1] https://sourceware.org/bugzilla/show_bug.cgi?id=25754
Signed-off-by: Arvind Sankar
Signed-off-by: Kees Cook
Signed-off-by: Ingo Molnar
Tested-by: Nick Desaulniers
Tested-by: Sedat Dilek
Reviewed-by: Kees Cook
Reviewed-by: Ard Biesheuvel
Reviewed-by: Fangrui Song
Link: https://lore.kernel.org/r/20200731230820.1742553-7-keesc...@chromium.org
---
arch/x86/boot/compressed/Makefile | 12 ++--
arch/x86/boot/compressed/head_32.S | 17 -
arch/x86/boot/compressed/head_64.S | 4 ++--
arch/x86/boot/compressed/mkpiggy.c | 6 ++
4 files changed, 18 insertions(+), 21 deletions(-)
diff --git a/arch/x86/boot/compressed/Makefile
b/arch/x86/boot/compressed/Makefile
index 7c687a7..7d25089 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -52,16 +52,8 @@ UBSAN_SANITIZE :=n
KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
# Compressed kernel should be built as PIE since it may be loaded at any
# address by the bootloader.
-ifeq ($(CONFIG_X86_32),y)
-KBUILD_LDFLAGS += $(call ld-option, -pie) $(call ld-option,
--no-dynamic-linker)
-else
-# To build 64-bit compressed kernel as PIE, we disable relocation
-# overflow check to avoid relocation overflow error with a new linker
-# command-line option, -z noreloc-overflow.
-KBUILD_LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z noreloc-overflow" \
- && echo "-z noreloc-overflow -pie --no-dynamic-linker")
-endif
-LDFLAGS_vmlinux := -T
+LDFLAGS_vmlinux := $(call ld-option, -pie) $(call ld-option,
--no-dynamic-linker)
+LDFLAGS_vmlinux += -T
hostprogs := mkpiggy
HOST_EXTRACFLAGS += -I$(srctree)/tools/include
diff --git a/arch/x86/boot/compressed/head_32.S
b/arch/x86/boot/compressed/head_32.S
index 8c1a4f5..659fad5 100644
--- a/arch/x86/boot/compressed/head_32.S
+++ b/arch/x86/boot/compressed/head_32.S
@@ -178,18 +178,17 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated)
/*
* Do the extraction, and jump to the new kernel..
*/
- /* push arguments for extract_kernel: */
- pushl $z_output_len /* decompressed length, end of relocs */
+ /* push arguments for extract_kernel: */
- pushl %ebp/* output address */
-
- pushl $z_input_len/* input_len */
+ pushl output_len@GOTOFF(%ebx) /* decompressed length, end of relocs */
+ pushl %ebp/* output address */
+ pushl input_len@GOTOFF(%ebx) /* input_len */
lealinput_data@GOTOFF(%ebx), %eax
- pushl %eax/* input_data */
+ pushl %eax/* input_data */
lealboot_heap@GOTOFF(%ebx), %eax
- pushl %eax/* heap area */
- pushl %esi/* real mode pointer */
- callextract_kernel /* returns kernel location in %eax */
+ pushl %eax/* heap area */
+ pushl %esi/* real mode pointer */
+ callextract_kernel /* returns kernel location in %eax */
addl$24, %esp
/*
diff --git a/arch/x86/boot/compressed/head_64.S
b/arch/x86/boot/compressed/head_64.S
index 1142909..9e46729 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -534,9 +534,9 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated)
movq%rsi, %rdi /* real mode address */
leaqboot_heap(%rip), %rsi /* malloc area for uncompression */
leaqinput_data(%rip), %rdx /* input_data */
- movl$z_input_len, %ecx /* input_len */
+ movlinput_len(%rip), %ecx /* input_len */
movq%rbp, %r8 /* output target address */
- movl$z_output_len, %r9d /* decompressed length, end of relocs */
+
[tip: x86/kaslr] x86/kaslr: Add a check that the random address is in range
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: f49236ae424d499d02ee3ce35fb9130ddf95b03f
Gitweb:
https://git.kernel.org/tip/f49236ae424d499d02ee3ce35fb9130ddf95b03f
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:22 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Add a check that the random address is in range
Check in find_random_phys_addr() that the chosen address is inside the
range that was required.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-22-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 80cdd20..735fcb2 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -803,6 +803,8 @@ static void process_e820_entries(unsigned long minimum,
static unsigned long find_random_phys_addr(unsigned long minimum,
unsigned long image_size)
{
+ u64 phys_addr;
+
/* Bail out early if it's impossible to succeed. */
if (minimum + image_size > mem_limit)
return 0;
@@ -816,7 +818,15 @@ static unsigned long find_random_phys_addr(unsigned long
minimum,
if (!process_efi_entries(minimum, image_size))
process_e820_entries(minimum, image_size);
- return slots_fetch_random();
+ phys_addr = slots_fetch_random();
+
+ /* Perform a final check to make sure the address is in range. */
+ if (phys_addr < minimum || phys_addr + image_size > mem_limit) {
+ warn("Invalid physical address chosen!\n");
+ return 0;
+ }
+
+ return (unsigned long)phys_addr;
}
static unsigned long find_random_virt_addr(unsigned long minimum,
[tip: x86/kaslr] x86/kaslr: Make local variables 64-bit
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 0eb1a8af01d6264cf948d67c8bff15e2eb859355
Gitweb:
https://git.kernel.org/tip/0eb1a8af01d6264cf948d67c8bff15e2eb859355
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:21 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Make local variables 64-bit
Change the type of local variables/fields that store mem_vector
addresses to u64 to make it less likely that 32-bit overflow will cause
issues on 32-bit.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-21-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 13 +++--
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index db8589c..80cdd20 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -461,7 +461,7 @@ static bool mem_avoid_overlap(struct mem_vector *img,
{
int i;
struct setup_data *ptr;
- unsigned long earliest = img->start + img->size;
+ u64 earliest = img->start + img->size;
bool is_overlapping = false;
for (i = 0; i < MEM_AVOID_MAX; i++) {
@@ -506,7 +506,7 @@ static bool mem_avoid_overlap(struct mem_vector *img,
}
struct slot_area {
- unsigned long addr;
+ u64 addr;
unsigned long num;
};
@@ -537,7 +537,8 @@ static void store_slot_info(struct mem_vector *region,
unsigned long image_size)
static void
process_gb_huge_pages(struct mem_vector *region, unsigned long image_size)
{
- unsigned long pud_start, pud_end, gb_huge_pages;
+ u64 pud_start, pud_end;
+ unsigned long gb_huge_pages;
struct mem_vector tmp;
if (!IS_ENABLED(CONFIG_X86_64) || !max_gb_huge_pages) {
@@ -579,7 +580,7 @@ process_gb_huge_pages(struct mem_vector *region, unsigned
long image_size)
}
}
-static unsigned long slots_fetch_random(void)
+static u64 slots_fetch_random(void)
{
unsigned long slot;
unsigned int i;
@@ -595,7 +596,7 @@ static unsigned long slots_fetch_random(void)
slot -= slot_areas[i].num;
continue;
}
- return slot_areas[i].addr + slot * CONFIG_PHYSICAL_ALIGN;
+ return slot_areas[i].addr + ((u64)slot * CONFIG_PHYSICAL_ALIGN);
}
if (i == slot_area_index)
@@ -608,7 +609,7 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long image_size)
{
struct mem_vector region, overlap;
- unsigned long region_end;
+ u64 region_end;
/* Enforce minimum and memory limit. */
region.start = max_t(u64, entry->start, minimum);
[tip: x86/kaslr] x86/kaslr: Replace 'unsigned long long' with 'u64'
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 3a066990a35eb289d54036637d2793d4743b8f07
Gitweb:
https://git.kernel.org/tip/3a066990a35eb289d54036637d2793d4743b8f07
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:20 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Replace 'unsigned long long' with 'u64'
No functional change.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-20-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 13 ++---
arch/x86/boot/compressed/misc.h | 4 ++--
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 3244f5b..db8589c 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -98,7 +98,7 @@ static bool memmap_too_large;
* Store memory limit: MAXMEM on 64-bit and KERNEL_IMAGE_SIZE on 32-bit.
* It may be reduced by "mem=nn[KMG]" or "memmap=nn[KMG]" command line options.
*/
-static unsigned long long mem_limit;
+static u64 mem_limit;
/* Number of immovable memory regions */
static int num_immovable_mem;
@@ -141,8 +141,7 @@ enum parse_mode {
};
static int
-parse_memmap(char *p, unsigned long long *start, unsigned long long *size,
- enum parse_mode mode)
+parse_memmap(char *p, u64 *start, u64 *size, enum parse_mode mode)
{
char *oldp;
@@ -172,7 +171,7 @@ parse_memmap(char *p, unsigned long long *start, unsigned
long long *size,
*/
*size = 0;
} else {
- unsigned long long flags;
+ u64 flags;
/*
* efi_fake_mem=nn@ss:attr the attr specifies
@@ -211,7 +210,7 @@ static void mem_avoid_memmap(enum parse_mode mode, char
*str)
while (str && (i < MAX_MEMMAP_REGIONS)) {
int rc;
- unsigned long long start, size;
+ u64 start, size;
char *k = strchr(str, ',');
if (k)
@@ -612,7 +611,7 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long region_end;
/* Enforce minimum and memory limit. */
- region.start = max_t(unsigned long long, entry->start, minimum);
+ region.start = max_t(u64, entry->start, minimum);
region_end = min(entry->start + entry->size, mem_limit);
/* Give up if slot area array is full. */
@@ -673,7 +672,7 @@ static bool process_mem_region(struct mem_vector *region,
* immovable memory and @region.
*/
for (i = 0; i < num_immovable_mem; i++) {
- unsigned long long start, end, entry_end, region_end;
+ u64 start, end, entry_end, region_end;
struct mem_vector entry;
if (!mem_overlaps(region, _mem[i]))
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index 726e264..3efce27 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -70,8 +70,8 @@ int cmdline_find_option(const char *option, char *buffer, int
bufsize);
int cmdline_find_option_bool(const char *option);
struct mem_vector {
- unsigned long long start;
- unsigned long long size;
+ u64 start;
+ u64 size;
};
#if CONFIG_RANDOMIZE_BASE
[tip: x86/kaslr] x86/kaslr: Make minimum/image_size 'unsigned long'
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: e4cb955bf173474a61f56200610004aacc7a62ff
Gitweb:
https://git.kernel.org/tip/e4cb955bf173474a61f56200610004aacc7a62ff
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:19 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Make minimum/image_size 'unsigned long'
Change type of minimum/image_size arguments in process_mem_region to
'unsigned long'. These actually can never be above 4G (even on x86_64),
and they're 'unsigned long' in every other function except this one.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-19-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index ecdf33d..3244f5b 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -649,8 +649,8 @@ static void __process_mem_region(struct mem_vector *entry,
}
static bool process_mem_region(struct mem_vector *region,
- unsigned long long minimum,
- unsigned long long image_size)
+ unsigned long minimum,
+ unsigned long image_size)
{
int i;
/*
[tip: x86/kaslr] x86/kaslr: Small cleanup of find_random_phys_addr()
The following commit has been merged into the x86/kaslr branch of tip: Commit-ID: 4268b4da572f8bde8bc2f3243927ff5795687a6f Gitweb: https://git.kernel.org/tip/4268b4da572f8bde8bc2f3243927ff5795687a6f Author:Arvind Sankar AuthorDate:Tue, 28 Jul 2020 18:57:18 -04:00 Committer: Ingo Molnar CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00 x86/kaslr: Small cleanup of find_random_phys_addr() Just a trivial rearrangement to do all the processing together, and only have one call to slots_fetch_random() in the source. Signed-off-by: Arvind Sankar Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20200728225722.67457-18-nived...@alum.mit.edu --- arch/x86/boot/compressed/kaslr.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index ce34a05..ecdf33d 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -813,10 +813,9 @@ static unsigned long find_random_phys_addr(unsigned long minimum, return 0; } - if (process_efi_entries(minimum, image_size)) - return slots_fetch_random(); + if (!process_efi_entries(minimum, image_size)) + process_e820_entries(minimum, image_size); - process_e820_entries(minimum, image_size); return slots_fetch_random(); }
[tip: x86/kaslr] x86/kaslr: Eliminate 'start_orig' local variable from __process_mem_region()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: ee435ee6490d147c1b9963cc8b331665e4cea634
Gitweb:
https://git.kernel.org/tip/ee435ee6490d147c1b9963cc8b331665e4cea634
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:08 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Eliminate 'start_orig' local variable from __process_mem_region()
Set the region.size within the loop, which removes the need for
start_orig.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-8-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 8 ++--
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index f2454ee..e978c35 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -623,7 +623,7 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long image_size)
{
struct mem_vector region, overlap;
- unsigned long start_orig, end;
+ unsigned long end;
/* Ignore entries entirely below our minimum. */
if (entry->start + entry->size < minimum)
@@ -635,12 +635,9 @@ static void __process_mem_region(struct mem_vector *entry,
return;
region.start = entry->start;
- region.size = end - entry->start;
/* Give up if slot area array is full. */
while (slot_area_index < MAX_SLOT_AREA) {
- start_orig = region.start;
-
/* Potentially raise address to minimum location. */
if (region.start < minimum)
region.start = minimum;
@@ -653,7 +650,7 @@ static void __process_mem_region(struct mem_vector *entry,
return;
/* Reduce size by any delta from the original address. */
- region.size -= region.start - start_orig;
+ region.size = end - region.start;
/* Return if region can't contain decompressed kernel */
if (region.size < image_size)
@@ -679,7 +676,6 @@ static void __process_mem_region(struct mem_vector *entry,
return;
/* Clip off the overlapping region and start over. */
- region.size -= overlap.start - region.start + overlap.size;
region.start = overlap.start + overlap.size;
}
}
[tip: x86/kaslr] x86/kaslr: Simplify process_gb_huge_pages()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: be9e8d9541a95bdfac1c13d112cc032ea7fc745f
Gitweb:
https://git.kernel.org/tip/be9e8d9541a95bdfac1c13d112cc032ea7fc745f
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:13 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Simplify process_gb_huge_pages()
Replace the loop to determine the number of 1Gb pages with arithmetic.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-13-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 47 +--
1 file changed, 21 insertions(+), 26 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 3727e97..00ef84b 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -547,49 +547,44 @@ static void store_slot_info(struct mem_vector *region,
unsigned long image_size)
static void
process_gb_huge_pages(struct mem_vector *region, unsigned long image_size)
{
- unsigned long addr, size = 0;
+ unsigned long pud_start, pud_end, gb_huge_pages;
struct mem_vector tmp;
- int i = 0;
if (!IS_ENABLED(CONFIG_X86_64) || !max_gb_huge_pages) {
store_slot_info(region, image_size);
return;
}
- addr = ALIGN(region->start, PUD_SIZE);
- /* Did we raise the address above the passed in memory entry? */
- if (addr < region->start + region->size)
- size = region->size - (addr - region->start);
-
- /* Check how many 1GB huge pages can be filtered out: */
- while (size >= PUD_SIZE && max_gb_huge_pages) {
- size -= PUD_SIZE;
- max_gb_huge_pages--;
- i++;
- }
+ /* Are there any 1GB pages in the region? */
+ pud_start = ALIGN(region->start, PUD_SIZE);
+ pud_end = ALIGN_DOWN(region->start + region->size, PUD_SIZE);
/* No good 1GB huge pages found: */
- if (!i) {
+ if (pud_start >= pud_end) {
store_slot_info(region, image_size);
return;
}
- /*
-* Skip those 'i'*1GB good huge pages, and continue checking and
-* processing the remaining head or tail part of the passed region
-* if available.
-*/
-
- if (addr >= region->start + image_size) {
+ /* Check if the head part of the region is usable. */
+ if (pud_start >= region->start + image_size) {
tmp.start = region->start;
- tmp.size = addr - region->start;
+ tmp.size = pud_start - region->start;
store_slot_info(, image_size);
}
- size = region->size - (addr - region->start) - i * PUD_SIZE;
- if (size >= image_size) {
- tmp.start = addr + i * PUD_SIZE;
- tmp.size = size;
+ /* Skip the good 1GB pages. */
+ gb_huge_pages = (pud_end - pud_start) >> PUD_SHIFT;
+ if (gb_huge_pages > max_gb_huge_pages) {
+ pud_end = pud_start + (max_gb_huge_pages << PUD_SHIFT);
+ max_gb_huge_pages = 0;
+ } else {
+ max_gb_huge_pages -= gb_huge_pages;
+ }
+
+ /* Check if the tail part of the region is usable. */
+ if (region->start + region->size >= pud_end + image_size) {
+ tmp.start = pud_end;
+ tmp.size = region->start + region->size - pud_end;
store_slot_info(, image_size);
}
}
[tip: x86/kaslr] x86/kaslr: Drop redundant check in store_slot_info()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 46a5b29a4a63a3ba987cbb5467774a4b5787618e
Gitweb:
https://git.kernel.org/tip/46a5b29a4a63a3ba987cbb5467774a4b5787618e
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:16 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Drop redundant check in store_slot_info()
Drop unnecessary check that number of slots is not zero in
store_slot_info, it's guaranteed to be at least 1 by the calculation.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-16-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 5c7457c..0c64026 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -525,13 +525,10 @@ static void store_slot_info(struct mem_vector *region,
unsigned long image_size)
return;
slot_area.addr = region->start;
- slot_area.num = (region->size - image_size) /
- CONFIG_PHYSICAL_ALIGN + 1;
+ slot_area.num = 1 + (region->size - image_size) / CONFIG_PHYSICAL_ALIGN;
- if (slot_area.num > 0) {
- slot_areas[slot_area_index++] = slot_area;
- slot_max += slot_area.num;
- }
+ slot_areas[slot_area_index++] = slot_area;
+ slot_max += slot_area.num;
}
/*
[tip: x86/kaslr] x86/kaslr: Drop unnecessary alignment in find_random_virt_addr()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: eb38be6db516fb72ccf7282628b545a185b3bc7a
Gitweb:
https://git.kernel.org/tip/eb38be6db516fb72ccf7282628b545a185b3bc7a
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:17 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Drop unnecessary alignment in find_random_virt_addr()
Drop unnecessary alignment of image_size to CONFIG_PHYSICAL_ALIGN in
find_random_virt_addr, it cannot change the result: the largest valid
slot is the largest n that satisfies
minimum + n * CONFIG_PHYSICAL_ALIGN + image_size <= KERNEL_IMAGE_SIZE
(since minimum is already aligned) and so n is equal to
(KERNEL_IMAGE_SIZE - minimum - image_size) / CONFIG_PHYSICAL_ALIGN
even if image_size is not aligned to CONFIG_PHYSICAL_ALIGN.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-17-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 6 +-
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 0c64026..ce34a05 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -825,16 +825,12 @@ static unsigned long find_random_virt_addr(unsigned long
minimum,
{
unsigned long slots, random_addr;
- /* Align image_size for easy slot calculations. */
- image_size = ALIGN(image_size, CONFIG_PHYSICAL_ALIGN);
-
/*
* There are how many CONFIG_PHYSICAL_ALIGN-sized slots
* that can hold image_size within the range of minimum to
* KERNEL_IMAGE_SIZE?
*/
- slots = (KERNEL_IMAGE_SIZE - minimum - image_size) /
-CONFIG_PHYSICAL_ALIGN + 1;
+ slots = 1 + (KERNEL_IMAGE_SIZE - minimum - image_size) /
CONFIG_PHYSICAL_ALIGN;
random_addr = kaslr_get_random_long("Virtual") % slots;
[tip: x86/kaslr] x86/kaslr: Fix process_efi_entries comment
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 08705365560a352d3f5b4f1f52270b4d4ff7911e
Gitweb:
https://git.kernel.org/tip/08705365560a352d3f5b4f1f52270b4d4ff7911e
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:56 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:12 +02:00
x86/kaslr: Fix process_efi_entries comment
Since commit:
0982adc74673 ("x86/boot/KASLR: Work around firmware bugs by excluding
EFI_BOOT_SERVICES_* and EFI_LOADER_* from KASLR's choice")
process_efi_entries() will return true if we have an EFI memmap, not just
if it contained EFI_MEMORY_MORE_RELIABLE regions.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Reviewed-by: Kees Cook
Link: https://lore.kernel.org/r/20200727230801.3468620-4-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index c31f3a5..1ab67a8 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -742,8 +742,8 @@ static bool process_mem_region(struct mem_vector *region,
#ifdef CONFIG_EFI
/*
- * Returns true if mirror region found (and must have been processed
- * for slots adding)
+ * Returns true if we processed the EFI memmap, which we prefer over the E820
+ * table if it is available.
*/
static bool
process_efi_entries(unsigned long minimum, unsigned long image_size)
[tip: x86/kaslr] x86/kaslr: Drop redundant variable in __process_mem_region()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: ef7b07d59e2f18042622cecde0c7a89b60f33a89
Gitweb:
https://git.kernel.org/tip/ef7b07d59e2f18042622cecde0c7a89b60f33a89
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:09 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Drop redundant variable in __process_mem_region()
region.size can be trimmed to store the portion of the region before the
overlap, instead of a separate mem_vector variable.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-9-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index e978c35..8cc47fa 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -664,11 +664,8 @@ static void __process_mem_region(struct mem_vector *entry,
/* Store beginning of region if holds at least image_size. */
if (overlap.start >= region.start + image_size) {
- struct mem_vector beginning;
-
- beginning.start = region.start;
- beginning.size = overlap.start - region.start;
- process_gb_huge_pages(, image_size);
+ region.size = overlap.start - region.start;
+ process_gb_huge_pages(, image_size);
}
/* Return if overlap extends to or past end of region. */
[tip: x86/kaslr] x86/kaslr: Drop redundant cur_entry from __process_mem_region()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 3f9412c73053a5be311607e42560c1303a873be7
Gitweb:
https://git.kernel.org/tip/3f9412c73053a5be311607e42560c1303a873be7
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:07 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Drop redundant cur_entry from __process_mem_region()
cur_entry is only used as cur_entry.start + cur_entry.size, which is
always equal to end.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-7-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 848346f..f2454ee 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -624,7 +624,6 @@ static void __process_mem_region(struct mem_vector *entry,
{
struct mem_vector region, overlap;
unsigned long start_orig, end;
- struct mem_vector cur_entry;
/* Ignore entries entirely below our minimum. */
if (entry->start + entry->size < minimum)
@@ -634,11 +633,9 @@ static void __process_mem_region(struct mem_vector *entry,
end = min(entry->size + entry->start, mem_limit);
if (entry->start >= end)
return;
- cur_entry.start = entry->start;
- cur_entry.size = end - entry->start;
- region.start = cur_entry.start;
- region.size = cur_entry.size;
+ region.start = entry->start;
+ region.size = end - entry->start;
/* Give up if slot area array is full. */
while (slot_area_index < MAX_SLOT_AREA) {
@@ -652,7 +649,7 @@ static void __process_mem_region(struct mem_vector *entry,
region.start = ALIGN(region.start, CONFIG_PHYSICAL_ALIGN);
/* Did we raise the address above the passed in memory entry? */
- if (region.start > cur_entry.start + cur_entry.size)
+ if (region.start > end)
return;
/* Reduce size by any delta from the original address. */
[tip: x86/kaslr] x86/kaslr: Fix off-by-one error in __process_mem_region()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 8d1cf8595860f4807f4ff1f8f1fc53e7576e0d71
Gitweb:
https://git.kernel.org/tip/8d1cf8595860f4807f4ff1f8f1fc53e7576e0d71
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:06 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Fix off-by-one error in __process_mem_region()
In case of an overlap, the beginning of the region should be used even
if it is exactly image_size, not just strictly larger.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-6-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index da45e66..848346f 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -669,7 +669,7 @@ static void __process_mem_region(struct mem_vector *entry,
}
/* Store beginning of region if holds at least image_size. */
- if (overlap.start > region.start + image_size) {
+ if (overlap.start >= region.start + image_size) {
struct mem_vector beginning;
beginning.start = region.start;
[tip: x86/kaslr] x86/kaslr: Make command line handling safer
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 709709ac6410f4a14ded158a4b23b979e33e10fb
Gitweb:
https://git.kernel.org/tip/709709ac6410f4a14ded158a4b23b979e33e10fb
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:54 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:07:51 +02:00
x86/kaslr: Make command line handling safer
Handle the possibility that the command line is NULL.
Replace open-coded strlen with a function call.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Reviewed-by: Kees Cook
Link: https://lore.kernel.org/r/20200727230801.3468620-2-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 26 ++
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index d7408af..e0f69f3 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -268,15 +268,19 @@ static void parse_gb_huge_pages(char *param, char *val)
static void handle_mem_options(void)
{
char *args = (char *)get_cmd_line_ptr();
- size_t len = strlen((char *)args);
+ size_t len;
char *tmp_cmdline;
char *param, *val;
u64 mem_size;
+ if (!args)
+ return;
+
if (!strstr(args, "memmap=") && !strstr(args, "mem=") &&
!strstr(args, "hugepages"))
return;
+ len = strlen(args);
tmp_cmdline = malloc(len + 1);
if (!tmp_cmdline)
error("Failed to allocate space for tmp_cmdline");
@@ -399,8 +403,7 @@ static void mem_avoid_init(unsigned long input, unsigned
long input_size,
{
unsigned long init_size = boot_params->hdr.init_size;
u64 initrd_start, initrd_size;
- u64 cmd_line, cmd_line_size;
- char *ptr;
+ unsigned long cmd_line, cmd_line_size;
/*
* Avoid the region that is unsafe to overlap during
@@ -421,16 +424,15 @@ static void mem_avoid_init(unsigned long input, unsigned
long input_size,
/* No need to set mapping for initrd, it will be handled in VO. */
/* Avoid kernel command line. */
- cmd_line = (u64)boot_params->ext_cmd_line_ptr << 32;
- cmd_line |= boot_params->hdr.cmd_line_ptr;
+ cmd_line = get_cmd_line_ptr();
/* Calculate size of cmd_line. */
- ptr = (char *)(unsigned long)cmd_line;
- for (cmd_line_size = 0; ptr[cmd_line_size++];)
- ;
- mem_avoid[MEM_AVOID_CMDLINE].start = cmd_line;
- mem_avoid[MEM_AVOID_CMDLINE].size = cmd_line_size;
- add_identity_map(mem_avoid[MEM_AVOID_CMDLINE].start,
-mem_avoid[MEM_AVOID_CMDLINE].size);
+ if (cmd_line) {
+ cmd_line_size = strlen((char *)cmd_line) + 1;
+ mem_avoid[MEM_AVOID_CMDLINE].start = cmd_line;
+ mem_avoid[MEM_AVOID_CMDLINE].size = cmd_line_size;
+ add_identity_map(mem_avoid[MEM_AVOID_CMDLINE].start,
+mem_avoid[MEM_AVOID_CMDLINE].size);
+ }
/* Avoid boot parameters. */
mem_avoid[MEM_AVOID_BOOTPARAMS].start = (unsigned long)boot_params;
[tip: x86/kaslr] x86/kaslr: Remove bogus warning and unnecessary goto
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: e2ee6173162b28053cb76b25887a0be9331c9e21
Gitweb:
https://git.kernel.org/tip/e2ee6173162b28053cb76b25887a0be9331c9e21
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:55 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:07 +02:00
x86/kaslr: Remove bogus warning and unnecessary goto
Drop the warning on seeing "--" in handle_mem_options(). This will trigger
whenever one of the memory options is present in the command line
together with "--", but there's no problem if that is the case.
Replace goto with break.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Reviewed-by: Kees Cook
Link: https://lore.kernel.org/r/20200727230801.3468620-3-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index e0f69f3..c31f3a5 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -295,10 +295,8 @@ static void handle_mem_options(void)
while (*args) {
args = next_arg(args, , );
/* Stop at -- */
- if (!val && strcmp(param, "--") == 0) {
- warn("Only '--' specified in cmdline");
- goto out;
- }
+ if (!val && strcmp(param, "--") == 0)
+ break;
if (!strcmp(param, "memmap")) {
mem_avoid_memmap(PARSE_MEMMAP, val);
@@ -311,7 +309,7 @@ static void handle_mem_options(void)
continue;
mem_size = memparse(p, );
if (mem_size == 0)
- goto out;
+ break;
mem_limit = mem_size;
} else if (!strcmp(param, "efi_fake_mem")) {
@@ -319,7 +317,6 @@ static void handle_mem_options(void)
}
}
-out:
free(tmp_cmdline);
return;
}
[tip: x86/kaslr] x86/kaslr: Short-circuit gb_huge_pages on x86-32
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 50def2693a900dfb1d91872056dc8164245820fc
Gitweb:
https://git.kernel.org/tip/50def2693a900dfb1d91872056dc8164245820fc
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:12 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Short-circuit gb_huge_pages on x86-32
32-bit does not have GB pages, so don't bother checking for them. Using
the IS_ENABLED() macro allows the compiler to completely remove the
gb_huge_pages code.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-12-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 0df513e..3727e97 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -303,7 +303,7 @@ static void handle_mem_options(void)
if (!strcmp(param, "memmap")) {
mem_avoid_memmap(PARSE_MEMMAP, val);
- } else if (strstr(param, "hugepages")) {
+ } else if (IS_ENABLED(CONFIG_X86_64) && strstr(param,
"hugepages")) {
parse_gb_huge_pages(param, val);
} else if (!strcmp(param, "mem")) {
char *p = val;
@@ -551,7 +551,7 @@ process_gb_huge_pages(struct mem_vector *region, unsigned
long image_size)
struct mem_vector tmp;
int i = 0;
- if (!max_gb_huge_pages) {
+ if (!IS_ENABLED(CONFIG_X86_64) || !max_gb_huge_pages) {
store_slot_info(region, image_size);
return;
}
[tip: x86/kaslr] x86/kaslr: Replace strlen() with strnlen()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 76167e5c5457aee8fba3edc5b8554183696fc94d
Gitweb:
https://git.kernel.org/tip/76167e5c5457aee8fba3edc5b8554183696fc94d
Author:Arvind Sankar
AuthorDate:Sun, 02 Aug 2020 21:15:34 -04:00
Committer: Ingo Molnar
CommitterDate: Thu, 06 Aug 2020 17:03:19 +02:00
x86/kaslr: Replace strlen() with strnlen()
strnlen is safer in case the command line is not NUL-terminated.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200803011534.730645-2-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 735fcb2..6d39743 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -43,6 +43,10 @@
#define STATIC
#include
+#define _SETUP
+#include /* For COMMAND_LINE_SIZE */
+#undef _SETUP
+
#ifdef CONFIG_X86_5LEVEL
unsigned int __pgtable_l5_enabled;
unsigned int pgdir_shift __ro_after_init = 39;
@@ -278,7 +282,7 @@ static void handle_mem_options(void)
if (!args)
return;
- len = strlen(args);
+ len = strnlen(args, COMMAND_LINE_SIZE-1);
tmp_cmdline = malloc(len + 1);
if (!tmp_cmdline)
error("Failed to allocate space for tmp_cmdline");
@@ -425,7 +429,7 @@ static void mem_avoid_init(unsigned long input, unsigned
long input_size,
cmd_line = get_cmd_line_ptr();
/* Calculate size of cmd_line. */
if (cmd_line) {
- cmd_line_size = strlen((char *)cmd_line) + 1;
+ cmd_line_size = strnlen((char *)cmd_line, COMMAND_LINE_SIZE-1)
+ 1;
mem_avoid[MEM_AVOID_CMDLINE].start = cmd_line;
mem_avoid[MEM_AVOID_CMDLINE].size = cmd_line_size;
add_identity_map(mem_avoid[MEM_AVOID_CMDLINE].start,
[tip: x86/kaslr] x86/kaslr: Drop some redundant checks from __process_mem_region()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: bf457be1548eee6d106daf9604e029b36fed2b11
Gitweb:
https://git.kernel.org/tip/bf457be1548eee6d106daf9604e029b36fed2b11
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:10 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Drop some redundant checks from __process_mem_region()
Clip the start and end of the region to minimum and mem_limit prior to
the loop. region.start can only increase during the loop, so raising it
to minimum before the loop is enough.
A region that becomes empty due to this will get checked in
the first iteration of the loop.
Drop the check for overlap extending beyond the end of the region. This
will get checked in the next loop iteration anyway.
Rename end to region_end for symmetry with region.start.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-10-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 27 ++-
1 file changed, 6 insertions(+), 21 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 8cc47fa..d074986 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -623,34 +623,23 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long image_size)
{
struct mem_vector region, overlap;
- unsigned long end;
+ unsigned long region_end;
- /* Ignore entries entirely below our minimum. */
- if (entry->start + entry->size < minimum)
- return;
-
- /* Ignore entries above memory limit */
- end = min(entry->size + entry->start, mem_limit);
- if (entry->start >= end)
- return;
-
- region.start = entry->start;
+ /* Enforce minimum and memory limit. */
+ region.start = max_t(unsigned long long, entry->start, minimum);
+ region_end = min(entry->start + entry->size, mem_limit);
/* Give up if slot area array is full. */
while (slot_area_index < MAX_SLOT_AREA) {
- /* Potentially raise address to minimum location. */
- if (region.start < minimum)
- region.start = minimum;
-
/* Potentially raise address to meet alignment needs. */
region.start = ALIGN(region.start, CONFIG_PHYSICAL_ALIGN);
/* Did we raise the address above the passed in memory entry? */
- if (region.start > end)
+ if (region.start > region_end)
return;
/* Reduce size by any delta from the original address. */
- region.size = end - region.start;
+ region.size = region_end - region.start;
/* Return if region can't contain decompressed kernel */
if (region.size < image_size)
@@ -668,10 +657,6 @@ static void __process_mem_region(struct mem_vector *entry,
process_gb_huge_pages(, image_size);
}
- /* Return if overlap extends to or past end of region. */
- if (overlap.start + overlap.size >= region.start + region.size)
- return;
-
/* Clip off the overlapping region and start over. */
region.start = overlap.start + overlap.size;
}
[tip: x86/kaslr] x86/kaslr: Fix off-by-one error in process_gb_huge_pages()
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 79c2fd2afe55944098047721c33e06fd48654e57
Gitweb:
https://git.kernel.org/tip/79c2fd2afe55944098047721c33e06fd48654e57
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:11 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Fix off-by-one error in process_gb_huge_pages()
If the remaining size of the region is exactly 1Gb, there is still one
hugepage that can be reserved.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-11-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index d074986..0df513e 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -562,7 +562,7 @@ process_gb_huge_pages(struct mem_vector *region, unsigned
long image_size)
size = region->size - (addr - region->start);
/* Check how many 1GB huge pages can be filtered out: */
- while (size > PUD_SIZE && max_gb_huge_pages) {
+ while (size >= PUD_SIZE && max_gb_huge_pages) {
size -= PUD_SIZE;
max_gb_huge_pages--;
i++;
[tip: x86/kaslr] x86/kaslr: Drop test for command-line parameters before parsing
The following commit has been merged into the x86/kaslr branch of tip: Commit-ID: 3870d971791f13df88a7a656e3fd6e2df8686097 Gitweb: https://git.kernel.org/tip/3870d971791f13df88a7a656e3fd6e2df8686097 Author:Arvind Sankar AuthorDate:Tue, 28 Jul 2020 18:57:14 -04:00 Committer: Ingo Molnar CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00 x86/kaslr: Drop test for command-line parameters before parsing This check doesn't save anything. In the case when none of the parameters are present, each strstr will scan args twice (once to find the length and then for searching), six scans in total. Just going ahead and parsing the arguments only requires three scans: strlen, memcpy, and parsing. This will be the first malloc, so free will actually free up the memory, so the check doesn't save heap space either. Signed-off-by: Arvind Sankar Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20200728225722.67457-14-nived...@alum.mit.edu --- arch/x86/boot/compressed/kaslr.c | 4 1 file changed, 4 deletions(-) diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index 00ef84b..bd13dc5 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -279,10 +279,6 @@ static void handle_mem_options(void) if (!args) return; - if (!strstr(args, "memmap=") && !strstr(args, "mem=") && - !strstr(args, "hugepages")) - return; - len = strlen(args); tmp_cmdline = malloc(len + 1); if (!tmp_cmdline)
[tip: x86/kaslr] x86/kaslr: Initialize mem_limit to the real maximum address
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 451286940d95778e83fa7f97006316d995b4c4a8
Gitweb:
https://git.kernel.org/tip/451286940d95778e83fa7f97006316d995b4c4a8
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:57 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Initialize mem_limit to the real maximum address
On 64-bit, the kernel must be placed below MAXMEM (64TiB with 4-level
paging or 4PiB with 5-level paging). This is currently not enforced by
KASLR, which thus implicitly relies on physical memory being limited to
less than 64TiB.
On 32-bit, the limit is KERNEL_IMAGE_SIZE (512MiB). This is enforced by
special checks in __process_mem_region().
Initialize mem_limit to the maximum (depending on architecture), instead
of ULLONG_MAX, and make sure the command-line arguments can only
decrease it. This makes the enforcement explicit on 64-bit, and
eliminates the 32-bit specific checks to keep the kernel below 512M.
Check upfront to make sure the minimum address is below the limit before
doing any work.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Acked-by: Kees Cook
Link: https://lore.kernel.org/r/20200727230801.3468620-5-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 41 ---
1 file changed, 22 insertions(+), 19 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 1ab67a8..da45e66 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -94,8 +94,11 @@ static unsigned long get_boot_seed(void)
static bool memmap_too_large;
-/* Store memory limit specified by "mem=nn[KMG]" or "memmap=nn[KMG]" */
-static unsigned long long mem_limit = ULLONG_MAX;
+/*
+ * Store memory limit: MAXMEM on 64-bit and KERNEL_IMAGE_SIZE on 32-bit.
+ * It may be reduced by "mem=nn[KMG]" or "memmap=nn[KMG]" command line options.
+ */
+static unsigned long long mem_limit;
/* Number of immovable memory regions */
static int num_immovable_mem;
@@ -221,7 +224,7 @@ static void mem_avoid_memmap(enum parse_mode mode, char
*str)
if (start == 0) {
/* Store the specified memory limit if size > 0 */
- if (size > 0)
+ if (size > 0 && size < mem_limit)
mem_limit = size;
continue;
@@ -311,7 +314,8 @@ static void handle_mem_options(void)
if (mem_size == 0)
break;
- mem_limit = mem_size;
+ if (mem_size < mem_limit)
+ mem_limit = mem_size;
} else if (!strcmp(param, "efi_fake_mem")) {
mem_avoid_memmap(PARSE_EFI, val);
}
@@ -322,7 +326,9 @@ static void handle_mem_options(void)
}
/*
- * In theory, KASLR can put the kernel anywhere in the range of [16M, 64T).
+ * In theory, KASLR can put the kernel anywhere in the range of [16M, MAXMEM)
+ * on 64-bit, and [16M, KERNEL_IMAGE_SIZE) on 32-bit.
+ *
* The mem_avoid array is used to store the ranges that need to be avoided
* when KASLR searches for an appropriate random address. We must avoid any
* regions that are unsafe to overlap with during decompression, and other
@@ -620,10 +626,6 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long start_orig, end;
struct mem_vector cur_entry;
- /* On 32-bit, ignore entries entirely above our maximum. */
- if (IS_ENABLED(CONFIG_X86_32) && entry->start >= KERNEL_IMAGE_SIZE)
- return;
-
/* Ignore entries entirely below our minimum. */
if (entry->start + entry->size < minimum)
return;
@@ -656,11 +658,6 @@ static void __process_mem_region(struct mem_vector *entry,
/* Reduce size by any delta from the original address. */
region.size -= region.start - start_orig;
- /* On 32-bit, reduce region size to fit within max size. */
- if (IS_ENABLED(CONFIG_X86_32) &&
- region.start + region.size > KERNEL_IMAGE_SIZE)
- region.size = KERNEL_IMAGE_SIZE - region.start;
-
/* Return if region can't contain decompressed kernel */
if (region.size < image_size)
return;
@@ -845,15 +842,16 @@ static void process_e820_entries(unsigned long minimum,
static unsigned long find_random_phys_addr(unsigned long minimum,
unsigned long image_size)
{
+ /* Bail out early if it's impossible to succeed. */
+ if (minimum + image_size > mem_limit)
+ return 0;
+
/* Check if we had too many memmaps. */
if (memmap_too_large) {
debug_putstr("Aborted memory entries
[tip: x86/kaslr] x86/kaslr: Make the type of number of slots/slot areas consistent
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: d6d0f36c735367ed7cf42b5ba454ba5858e17816
Gitweb:
https://git.kernel.org/tip/d6d0f36c735367ed7cf42b5ba454ba5858e17816
Author:Arvind Sankar
AuthorDate:Tue, 28 Jul 2020 18:57:15 -04:00
Committer: Ingo Molnar
CommitterDate: Fri, 31 Jul 2020 11:08:17 +02:00
x86/kaslr: Make the type of number of slots/slot areas consistent
The number of slots can be 'unsigned int', since on 64-bit, the maximum
amount of memory is 2^52, the minimum alignment is 2^21, so the slot
number cannot be greater than 2^31. But in case future processors have
more than 52 physical address bits, make it 'unsigned long'.
The slot areas are limited by MAX_SLOT_AREA, currently 100. It is
indexed by an int, but the number of areas is stored as 'unsigned long'.
Change both to 'unsigned int' for consistency.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200728225722.67457-15-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index bd13dc5..5c7457c 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -508,17 +508,15 @@ static bool mem_avoid_overlap(struct mem_vector *img,
struct slot_area {
unsigned long addr;
- int num;
+ unsigned long num;
};
#define MAX_SLOT_AREA 100
static struct slot_area slot_areas[MAX_SLOT_AREA];
-
+static unsigned int slot_area_index;
static unsigned long slot_max;
-static unsigned long slot_area_index;
-
static void store_slot_info(struct mem_vector *region, unsigned long
image_size)
{
struct slot_area slot_area;
@@ -588,7 +586,7 @@ process_gb_huge_pages(struct mem_vector *region, unsigned
long image_size)
static unsigned long slots_fetch_random(void)
{
unsigned long slot;
- int i;
+ unsigned int i;
/* Handle case of no slots stored. */
if (slot_max == 0)
[tip: x86/kaslr] x86/kaslr: Remove bogus warning and unnecessary goto
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 237069512d18f0f20b230680f067a2c070a7903a
Gitweb:
https://git.kernel.org/tip/237069512d18f0f20b230680f067a2c070a7903a
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:55 -04:00
Committer: Ingo Molnar
CommitterDate: Tue, 28 Jul 2020 12:54:42 +02:00
x86/kaslr: Remove bogus warning and unnecessary goto
Drop the warning on seeing "--" in handle_mem_options(). This will trigger
whenever one of the memory options is present in the command line
together with "--", but there's no problem if that is the case.
Replace goto with break.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200727230801.3468620-3-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index a4af896..21cd9e0 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -295,10 +295,8 @@ static void handle_mem_options(void)
while (*args) {
args = next_arg(args, , );
/* Stop at -- */
- if (!val && strcmp(param, "--") == 0) {
- warn("Only '--' specified in cmdline");
- goto out;
- }
+ if (!val && strcmp(param, "--") == 0)
+ break;
if (!strcmp(param, "memmap")) {
mem_avoid_memmap(PARSE_MEMMAP, val);
@@ -311,7 +309,7 @@ static void handle_mem_options(void)
continue;
mem_size = memparse(p, );
if (mem_size == 0)
- goto out;
+ break;
mem_limit = mem_size;
} else if (!strcmp(param, "efi_fake_mem")) {
@@ -319,7 +317,6 @@ static void handle_mem_options(void)
}
}
-out:
free(tmp_cmdline);
return;
}
[tip: x86/kaslr] x86/kaslr: Fix process_efi_entries comment
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: a68bcea591a040cce5e08615c829a316beb3e4b4
Gitweb:
https://git.kernel.org/tip/a68bcea591a040cce5e08615c829a316beb3e4b4
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:56 -04:00
Committer: Ingo Molnar
CommitterDate: Tue, 28 Jul 2020 12:54:43 +02:00
x86/kaslr: Fix process_efi_entries comment
Since commit:
0982adc74673 ("x86/boot/KASLR: Work around firmware bugs by excluding
EFI_BOOT_SERVICES_* and EFI_LOADER_* from KASLR's choice")
process_efi_entries() will return true if we have an EFI memmap, not just
if it contained EFI_MEMORY_MORE_RELIABLE regions.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200727230801.3468620-4-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 21cd9e0..207fcb7 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -741,8 +741,8 @@ static bool process_mem_region(struct mem_vector *region,
#ifdef CONFIG_EFI
/*
- * Returns true if mirror region found (and must have been processed
- * for slots adding)
+ * Returns true if we processed the EFI memmap, which we prefer over the E820
+ * table if it is available.
*/
static bool
process_efi_entries(unsigned long minimum, unsigned long image_size)
[tip: x86/kaslr] x86/kaslr: Make command line handling safer
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: c8465b03acf9d4bb43b9714b6cfb99442defe664
Gitweb:
https://git.kernel.org/tip/c8465b03acf9d4bb43b9714b6cfb99442defe664
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:54 -04:00
Committer: Ingo Molnar
CommitterDate: Tue, 28 Jul 2020 12:54:42 +02:00
x86/kaslr: Make command line handling safer
Handle the possibility that the command line is NULL.
Replace open-coded strlen with a function call.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200727230801.3468620-2-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index d7408af..a4af896 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -268,15 +268,19 @@ static void parse_gb_huge_pages(char *param, char *val)
static void handle_mem_options(void)
{
char *args = (char *)get_cmd_line_ptr();
- size_t len = strlen((char *)args);
+ size_t len;
char *tmp_cmdline;
char *param, *val;
u64 mem_size;
+ if (!args)
+ return;
+
if (!strstr(args, "memmap=") && !strstr(args, "mem=") &&
!strstr(args, "hugepages"))
return;
+ len = strlen(args);
tmp_cmdline = malloc(len + 1);
if (!tmp_cmdline)
error("Failed to allocate space for tmp_cmdline");
@@ -399,8 +403,7 @@ static void mem_avoid_init(unsigned long input, unsigned
long input_size,
{
unsigned long init_size = boot_params->hdr.init_size;
u64 initrd_start, initrd_size;
- u64 cmd_line, cmd_line_size;
- char *ptr;
+ unsigned long cmd_line, cmd_line_size = 0;
/*
* Avoid the region that is unsafe to overlap during
@@ -421,12 +424,10 @@ static void mem_avoid_init(unsigned long input, unsigned
long input_size,
/* No need to set mapping for initrd, it will be handled in VO. */
/* Avoid kernel command line. */
- cmd_line = (u64)boot_params->ext_cmd_line_ptr << 32;
- cmd_line |= boot_params->hdr.cmd_line_ptr;
+ cmd_line = get_cmd_line_ptr();
/* Calculate size of cmd_line. */
- ptr = (char *)(unsigned long)cmd_line;
- for (cmd_line_size = 0; ptr[cmd_line_size++];)
- ;
+ if (cmd_line)
+ cmd_line_size = strlen((char *)cmd_line);
mem_avoid[MEM_AVOID_CMDLINE].start = cmd_line;
mem_avoid[MEM_AVOID_CMDLINE].size = cmd_line_size;
add_identity_map(mem_avoid[MEM_AVOID_CMDLINE].start,
[tip: x86/kaslr] x86/kaslr: Initialize mem_limit to the real maximum address
The following commit has been merged into the x86/kaslr branch of tip:
Commit-ID: 7f104ea54e69ec45cbc4948cdb7d7f74d46def6d
Gitweb:
https://git.kernel.org/tip/7f104ea54e69ec45cbc4948cdb7d7f74d46def6d
Author:Arvind Sankar
AuthorDate:Mon, 27 Jul 2020 19:07:57 -04:00
Committer: Ingo Molnar
CommitterDate: Tue, 28 Jul 2020 12:54:43 +02:00
x86/kaslr: Initialize mem_limit to the real maximum address
On 64-bit, the kernel must be placed below MAXMEM (64TiB with 4-level
paging or 4PiB with 5-level paging). This is currently not enforced by
KASLR, which thus implicitly relies on physical memory being limited to
less than 64TiB.
On 32-bit, the limit is KERNEL_IMAGE_SIZE (512MiB). This is enforced by
special checks in __process_mem_region().
Initialize mem_limit to the maximum (depending on architecture), instead
of ULLONG_MAX, and make sure the command-line arguments can only
decrease it. This makes the enforcement explicit on 64-bit, and
eliminates the 32-bit specific checks to keep the kernel below 512M.
Check upfront to make sure the minimum address is below the limit before
doing any work.
Signed-off-by: Arvind Sankar
Signed-off-by: Ingo Molnar
Link: https://lore.kernel.org/r/20200727230801.3468620-5-nived...@alum.mit.edu
---
arch/x86/boot/compressed/kaslr.c | 41 ---
1 file changed, 22 insertions(+), 19 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 207fcb7..758d784 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -94,8 +94,11 @@ static unsigned long get_boot_seed(void)
static bool memmap_too_large;
-/* Store memory limit specified by "mem=nn[KMG]" or "memmap=nn[KMG]" */
-static unsigned long long mem_limit = ULLONG_MAX;
+/*
+ * Store memory limit: MAXMEM on 64-bit and KERNEL_IMAGE_SIZE on 32-bit.
+ * It may be reduced by "mem=nn[KMG]" or "memmap=nn[KMG]" command line options.
+ */
+static unsigned long long mem_limit;
/* Number of immovable memory regions */
static int num_immovable_mem;
@@ -221,7 +224,7 @@ static void mem_avoid_memmap(enum parse_mode mode, char
*str)
if (start == 0) {
/* Store the specified memory limit if size > 0 */
- if (size > 0)
+ if (size > 0 && size < mem_limit)
mem_limit = size;
continue;
@@ -311,7 +314,8 @@ static void handle_mem_options(void)
if (mem_size == 0)
break;
- mem_limit = mem_size;
+ if (mem_size < mem_limit)
+ mem_limit = mem_size;
} else if (!strcmp(param, "efi_fake_mem")) {
mem_avoid_memmap(PARSE_EFI, val);
}
@@ -322,7 +326,9 @@ static void handle_mem_options(void)
}
/*
- * In theory, KASLR can put the kernel anywhere in the range of [16M, 64T).
+ * In theory, KASLR can put the kernel anywhere in the range of [16M, MAXMEM)
+ * on 64-bit, and [16M, KERNEL_IMAGE_SIZE) on 32-bit.
+ *
* The mem_avoid array is used to store the ranges that need to be avoided
* when KASLR searches for an appropriate random address. We must avoid any
* regions that are unsafe to overlap with during decompression, and other
@@ -619,10 +625,6 @@ static void __process_mem_region(struct mem_vector *entry,
unsigned long start_orig, end;
struct mem_vector cur_entry;
- /* On 32-bit, ignore entries entirely above our maximum. */
- if (IS_ENABLED(CONFIG_X86_32) && entry->start >= KERNEL_IMAGE_SIZE)
- return;
-
/* Ignore entries entirely below our minimum. */
if (entry->start + entry->size < minimum)
return;
@@ -655,11 +657,6 @@ static void __process_mem_region(struct mem_vector *entry,
/* Reduce size by any delta from the original address. */
region.size -= region.start - start_orig;
- /* On 32-bit, reduce region size to fit within max size. */
- if (IS_ENABLED(CONFIG_X86_32) &&
- region.start + region.size > KERNEL_IMAGE_SIZE)
- region.size = KERNEL_IMAGE_SIZE - region.start;
-
/* Return if region can't contain decompressed kernel */
if (region.size < image_size)
return;
@@ -844,15 +841,16 @@ static void process_e820_entries(unsigned long minimum,
static unsigned long find_random_phys_addr(unsigned long minimum,
unsigned long image_size)
{
+ /* Bail out early if it's impossible to succeed. */
+ if (minimum + image_size > mem_limit)
+ return 0;
+
/* Check if we had too many memmaps. */
if (memmap_too_large) {
debug_putstr("Aborted memory entries scan (more than 4
[tip: x86/build] x86/build: Move max-page-size option to LDFLAGS_vmlinux
The following commit has been merged into the x86/build branch of tip: Commit-ID: 587af649bcc04eb016822f209a975005c0092151 Gitweb: https://git.kernel.org/tip/587af649bcc04eb016822f209a975005c0092151 Author:Arvind Sankar AuthorDate:Wed, 22 Jul 2020 14:43:34 -04:00 Committer: Ingo Molnar CommitterDate: Fri, 24 Jul 2020 16:39:27 +02:00 x86/build: Move max-page-size option to LDFLAGS_vmlinux This option is only required for vmlinux on 64-bit, to enforce 2MiB alignment, so set it in LDFLAGS_vmlinux instead of KBUILD_LDFLAGS. Also drop the ld-option check: this option was added in binutils-2.18 and all the other places that use it already don't have the check. This reduces the size of the intermediate ELF files arch/x86/boot/setup.elf and arch/x86/realmode/rm/realmode.elf by about 2MiB each. The binary versions are unchanged. Move the LDFLAGS settings to all be together and just after CFLAGS settings are done. Signed-off-by: Arvind Sankar Signed-off-by: Ingo Molnar Tested-by: Sedat Dilek Cc: Masahiro Yamada Cc: Michal Marek Link: https://lore.kernel.org/r/20200722184334.3785418-1-nived...@alum.mit.edu --- arch/x86/Makefile | 32 +--- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 00e378d..1e634d7 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -47,10 +47,6 @@ export REALMODE_CFLAGS # e.g.: obj-y += foo_$(BITS).o export BITS -ifdef CONFIG_X86_NEED_RELOCS -LDFLAGS_vmlinux := --emit-relocs --discard-none -endif - # # Prevent GCC from generating any FP code by mistake. # @@ -177,17 +173,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE) - -# -# The 64-bit kernel must be aligned to 2MB. Pass -z max-page-size=0x20 to -# the linker to force 2MB page size regardless of the default page size used -# by the linker. -# -ifdef CONFIG_X86_64 -KBUILD_LDFLAGS += $(call ld-option, -z max-page-size=0x20) -endif - # Workaround for a gcc prelease that unfortunately was shipped in a suse release KBUILD_CFLAGS += -Wno-sign-compare # @@ -207,6 +192,23 @@ ifdef CONFIG_RETPOLINE endif endif +KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE) + +ifdef CONFIG_X86_NEED_RELOCS +LDFLAGS_vmlinux := --emit-relocs --discard-none +else +LDFLAGS_vmlinux := +endif + +# +# The 64-bit kernel must be aligned to 2MB. Pass -z max-page-size=0x20 to +# the linker to force 2MB page size regardless of the default page size used +# by the linker. +# +ifdef CONFIG_X86_64 +LDFLAGS_vmlinux += -z max-page-size=0x20 +endif + archscripts: scripts_basic $(Q)$(MAKE) $(build)=arch/x86/tools relocs
[tip: x86/cleanups] x86/mm: Drop unused MAX_PHYSADDR_BITS
The following commit has been merged into the x86/cleanups branch of tip:
Commit-ID: 0a787b28b7a375ad9d5c77bc3922ae1a8305239e
Gitweb:
https://git.kernel.org/tip/0a787b28b7a375ad9d5c77bc3922ae1a8305239e
Author:Arvind Sankar
AuthorDate:Thu, 23 Jul 2020 19:15:42 -04:00
Committer: Thomas Gleixner
CommitterDate: Fri, 24 Jul 2020 09:53:06 +02:00
x86/mm: Drop unused MAX_PHYSADDR_BITS
The macro is not used anywhere, and has an incorrect value (going by the
comment) on x86_64 since commit c898faf91b3e ("x86: 46 bit physical address
support on 64 bits")
To avoid confusion, just remove the definition.
Signed-off-by: Arvind Sankar
Signed-off-by: Thomas Gleixner
Link: https://lkml.kernel.org/r/20200723231544.17274-2-nived...@alum.mit.edu
---
arch/x86/include/asm/sparsemem.h | 6 +-
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/sparsemem.h b/arch/x86/include/asm/sparsemem.h
index 1992187..6bfc878 100644
--- a/arch/x86/include/asm/sparsemem.h
+++ b/arch/x86/include/asm/sparsemem.h
@@ -10,24 +10,20 @@
*field of the struct page
*
* SECTION_SIZE_BITS 2^n: size of each section
- * MAX_PHYSADDR_BITS 2^n: max size of physical address space
- * MAX_PHYSMEM_BITS2^n: how much memory we can have in that space
+ * MAX_PHYSMEM_BITS2^n: max size of physical address space
*
*/
#ifdef CONFIG_X86_32
# ifdef CONFIG_X86_PAE
# define SECTION_SIZE_BITS29
-# define MAX_PHYSADDR_BITS36
# define MAX_PHYSMEM_BITS 36
# else
# define SECTION_SIZE_BITS26
-# define MAX_PHYSADDR_BITS32
# define MAX_PHYSMEM_BITS 32
# endif
#else /* CONFIG_X86_32 */
# define SECTION_SIZE_BITS 27 /* matt - 128 is convenient right now */
-# define MAX_PHYSADDR_BITS (pgtable_l5_enabled() ? 52 : 44)
# define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 52 : 46)
#endif
[tip: efi/urgent] efi/x86: Only copy upto the end of setup_header
The following commit has been merged into the efi/urgent branch of tip: Commit-ID: 59476f80d8781a84e25f0cbcf378ccab1ad7abf8 Gitweb: https://git.kernel.org/tip/59476f80d8781a84e25f0cbcf378ccab1ad7abf8 Author:Arvind Sankar AuthorDate:Thu, 18 Jun 2020 16:43:15 -04:00 Committer: Ard Biesheuvel CommitterDate: Thu, 25 Jun 2020 18:09:48 +02:00 efi/x86: Only copy upto the end of setup_header When copying the setup_header into the boot_params buffer, only the data that is actually part of the setup_header should be copied. efi_pe_entry() currently copies the entire second sector, which initializes some of the fields in boot_params beyond the setup_header with garbage (i.e. part of the real-mode boot code gets copied into those fields). This does not cause any issues currently because the fields that are overwritten are padding, BIOS EDD information that won't get used, and the E820 table which will get properly filled in later. Fix this to only copy data that is actually part of the setup_header structure. Signed-off-by: Arvind Sankar Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/x86-stub.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 37e82bf..3672539 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -8,6 +8,7 @@ #include #include +#include #include #include @@ -388,8 +389,9 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, hdr = _params->hdr; - /* Copy the second sector to boot_params */ - memcpy(>jump, image_base + 512, 512); + /* Copy the setup header from the second sector to boot_params */ + memcpy(>jump, image_base + 512, + sizeof(struct setup_header) - offsetof(struct setup_header, jump)); /* * Fill out some of the header fields ourselves because the
[tip: efi/urgent] efi/x86: Remove unused variables
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: 0bda49f30ca48998102eb0a0b53970c3a3558be0
Gitweb:
https://git.kernel.org/tip/0bda49f30ca48998102eb0a0b53970c3a3558be0
Author:Arvind Sankar
AuthorDate:Thu, 18 Jun 2020 15:10:59 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 25 Jun 2020 18:06:10 +02:00
efi/x86: Remove unused variables
Commit
987053a30016 ("efi/x86: Move command-line initrd loading to efi_main")
made the ramdisk_addr/ramdisk_size variables in efi_pe_entry unused, but
neglected to delete them.
Delete these unused variables.
Signed-off-by: Arvind Sankar
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/x86-stub.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/firmware/efi/libstub/x86-stub.c
b/drivers/firmware/efi/libstub/x86-stub.c
index 5a48d99..37e82bf 100644
--- a/drivers/firmware/efi/libstub/x86-stub.c
+++ b/drivers/firmware/efi/libstub/x86-stub.c
@@ -361,8 +361,6 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
int options_size = 0;
efi_status_t status;
char *cmdline_ptr;
- unsigned long ramdisk_addr;
- unsigned long ramdisk_size;
efi_system_table = sys_table_arg;
[tip: x86/urgent] x86/boot: Don't add the EFI stub to targets
The following commit has been merged into the x86/urgent branch of tip: Commit-ID: da05b143a308bd6a7a01f9732678ae63fc70 Gitweb: https://git.kernel.org/tip/da05b143a308bd6a7a01f9732678ae63fc70 Author:Arvind Sankar AuthorDate:Tue, 14 Jul 2020 23:26:31 -04:00 Committer: Thomas Gleixner CommitterDate: Sun, 19 Jul 2020 13:07:11 +02:00 x86/boot: Don't add the EFI stub to targets vmlinux-objs-y is added to targets, which currently means that the EFI stub gets added to the targets as well. It shouldn't be added since it is built elsewhere. This confuses Makefile.build which interprets the EFI stub as a target $(obj)/$(objtree)/drivers/firmware/efi/libstub/lib.a and will create drivers/firmware/efi/libstub/ underneath arch/x86/boot/compressed, to hold this supposed target, if building out-of-tree. [0] Fix this by pulling the stub out of vmlinux-objs-y into efi-obj-y. [0] See scripts/Makefile.build near the end: # Create directories for object files if they do not exist Signed-off-by: Arvind Sankar Signed-off-by: Thomas Gleixner Reviewed-by: Masahiro Yamada Acked-by: Ard Biesheuvel Link: https://lkml.kernel.org/r/20200715032631.1562882-1-nived...@alum.mit.edu --- arch/x86/boot/compressed/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 7619742..5a828fd 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -90,8 +90,8 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o -vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o +efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a # The compressed kernel is built with -fPIC/-fPIE so that a boot loader # can place it anywhere in memory and it will still run. However, since @@ -115,7 +115,7 @@ endef quiet_cmd_check-and-link-vmlinux = LD $@ cmd_check-and-link-vmlinux = $(cmd_check_data_rel); $(cmd_ld) -$(obj)/vmlinux: $(vmlinux-objs-y) FORCE +$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE $(call if_changed,check-and-link-vmlinux) OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
[tip: efi/urgent] efi/x86: Setup stack correctly for efi_pe_entry
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: 41d90b0c1108d1e46c48cf79964636c553844f4c
Gitweb:
https://git.kernel.org/tip/41d90b0c1108d1e46c48cf79964636c553844f4c
Author:Arvind Sankar
AuthorDate:Wed, 17 Jun 2020 09:19:57 -04:00
Committer: Ard Biesheuvel
CommitterDate: Wed, 17 Jun 2020 15:28:58 +02:00
efi/x86: Setup stack correctly for efi_pe_entry
Commit
17054f492dfd ("efi/x86: Implement mixed mode boot without the handover
protocol")
introduced a new entry point for the EFI stub to be booted in mixed mode
on 32-bit firmware.
When entered via efi32_pe_entry, control is first transferred to
startup_32 to setup for the switch to long mode, and then the EFI stub
proper is entered via efi_pe_entry. efi_pe_entry is an MS ABI function,
and the ABI requires 32 bytes of shadow stack space to be allocated by
the caller, as well as the stack being aligned to 8 mod 16 on entry.
Allocate 40 bytes on the stack before switching to 64-bit mode when
calling efi_pe_entry to account for this.
For robustness, explicitly align boot_stack_end to 16 bytes. It is
currently implicitly aligned since .bss is cacheline-size aligned,
head_64.o is the first object file with a .bss section, and the heap and
boot sizes are aligned.
Fixes: 17054f492dfd ("efi/x86: Implement mixed mode boot without the handover
protocol")
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200617131957.2507632-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
arch/x86/boot/compressed/head_64.S | 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S
b/arch/x86/boot/compressed/head_64.S
index e821a7d..97d37f0 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -213,7 +213,6 @@ SYM_FUNC_START(startup_32)
* We place all of the values on our mini stack so lret can
* used to perform that far jump.
*/
- pushl $__KERNEL_CS
lealstartup_64(%ebp), %eax
#ifdef CONFIG_EFI_MIXED
movlefi32_boot_args(%ebp), %edi
@@ -224,11 +223,20 @@ SYM_FUNC_START(startup_32)
movlefi32_boot_args+8(%ebp), %edx // saved bootparams pointer
cmpl$0, %edx
jnz 1f
+ /*
+* efi_pe_entry uses MS calling convention, which requires 32 bytes of
+* shadow space on the stack even if all arguments are passed in
+* registers. We also need an additional 8 bytes for the space that
+* would be occupied by the return address, and this also results in
+* the correct stack alignment for entry.
+*/
+ subl$40, %esp
lealefi_pe_entry(%ebp), %eax
movl%edi, %ecx // MS calling convention
movl%esi, %edx
1:
#endif
+ pushl $__KERNEL_CS
pushl %eax
/* Enter paged protected Mode, activating Long Mode */
@@ -784,6 +792,7 @@ SYM_DATA_LOCAL(boot_heap, .fill BOOT_HEAP_SIZE, 1, 0)
SYM_DATA_START_LOCAL(boot_stack)
.fill BOOT_STACK_SIZE, 1, 0
+ .balign 16
SYM_DATA_END_LABEL(boot_stack, SYM_L_LOCAL, boot_stack_end)
/*
[tip: efi/urgent] efi/x86: Fix build with gcc 4
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: 5435f73d5c4a1b7504356876e69ba52de83f4975
Gitweb:
https://git.kernel.org/tip/5435f73d5c4a1b7504356876e69ba52de83f4975
Author:Arvind Sankar
AuthorDate:Fri, 05 Jun 2020 11:06:38 -04:00
Committer: Ard Biesheuvel
CommitterDate: Mon, 15 Jun 2020 11:41:14 +02:00
efi/x86: Fix build with gcc 4
Commit
bbf8e8b0fe04 ("efi/libstub: Optimize for size instead of speed")
changed the optimization level for the EFI stub to -Os from -O2.
Andrey Ignatov reports that this breaks the build with gcc 4.8.5.
Testing on godbolt.org, the combination of -Os,
-fno-asynchronous-unwind-tables, and ms_abi functions doesn't work,
failing with the error:
sorry, unimplemented: ms_abi attribute requires
-maccumulate-outgoing-args or subtarget optimization implying it
This does appear to work with gcc 4.9 onwards.
Add -maccumulate-outgoing-args explicitly to unbreak the build with
pre-4.9 versions of gcc.
Reported-by: Andrey Ignatov
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200605150638.1011637-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
drivers/firmware/efi/libstub/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/firmware/efi/libstub/Makefile
b/drivers/firmware/efi/libstub/Makefile
index 75daaf2..4cce372 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -6,7 +6,8 @@
# enabled, even if doing so doesn't break the build.
#
cflags-$(CONFIG_X86_32):= -march=i386
-cflags-$(CONFIG_X86_64):= -mcmodel=small
+cflags-$(CONFIG_X86_64):= -mcmodel=small \
+ $(call cc-option,-maccumulate-outgoing-args)
cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ \
-fPIC -fno-strict-aliasing -mno-red-zone \
-mno-mmx -mno-sse -fshort-wchar \
[tip: x86/cleanups] x86/mm: Stop printing BRK addresses
The following commit has been merged into the x86/cleanups branch of tip:
Commit-ID: 67d631b7c05eff955ccff4139327f0f92a5117e5
Gitweb:
https://git.kernel.org/tip/67d631b7c05eff955ccff4139327f0f92a5117e5
Author:Arvind Sankar
AuthorDate:Sat, 29 Feb 2020 18:11:20 -05:00
Committer: Borislav Petkov
CommitterDate: Sat, 23 May 2020 09:34:18 +02:00
x86/mm: Stop printing BRK addresses
This currently leaks kernel physical addresses into userspace.
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Acked-by: Kees Cook
Acked-by: Dave Hansen
Link: https://lkml.kernel.org/r/20200229231120.1147527-1-nived...@alum.mit.edu
---
arch/x86/mm/init.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 1bba16c..a573a3e 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -121,8 +121,6 @@ __ref void *alloc_low_pages(unsigned int num)
} else {
pfn = pgt_buf_end;
pgt_buf_end += num;
- printk(KERN_DEBUG "BRK [%#010lx, %#010lx] PGTABLE\n",
- pfn << PAGE_SHIFT, (pgt_buf_end << PAGE_SHIFT) - 1);
}
for (i = 0; i < num; i++) {
[tip: efi/urgent] x86/boot: Mark global variables as static
The following commit has been merged into the efi/urgent branch of tip:
Commit-ID: e78d334a5470ead861590ec83158f3b17bd6c807
Gitweb:
https://git.kernel.org/tip/e78d334a5470ead861590ec83158f3b17bd6c807
Author:Arvind Sankar
AuthorDate:Mon, 11 May 2020 18:58:49 -04:00
Committer: Ard Biesheuvel
CommitterDate: Thu, 14 May 2020 11:11:20 +02:00
x86/boot: Mark global variables as static
Mike Lothian reports that after commit
964124a97b97 ("efi/x86: Remove extra headroom for setup block")
gcc 10.1.0 fails with
HOSTCC arch/x86/boot/tools/build
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld:
error: linker defined: multiple definition of '_end'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld:
/tmp/ccEkW0jM.o: previous definition here
collect2: error: ld returned 1 exit status
make[1]: *** [scripts/Makefile.host:103: arch/x86/boot/tools/build] Error 1
make: *** [arch/x86/Makefile:303: bzImage] Error 2
The issue is with the _end variable that was added, to hold the end of
the compressed kernel from zoffsets.h (ZO__end). The name clashes with
the linker-defined _end symbol that indicates the end of the build
program itself.
Even when there is no compile-time error, this causes build to use
memory past the end of its .bss section.
To solve this, mark _end as static, and for symmetry, mark the rest of
the variables that keep track of symbols from the compressed kernel as
static as well.
Fixes: 964124a97b97 ("efi/x86: Remove extra headroom for setup block")
Reported-by: Mike Lothian
Tested-by: Mike Lothian
Signed-off-by: Arvind Sankar
Link: https://lore.kernel.org/r/20200511225849.1311869-1-nived...@alum.mit.edu
Signed-off-by: Ard Biesheuvel
---
arch/x86/boot/tools/build.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/arch/x86/boot/tools/build.c b/arch/x86/boot/tools/build.c
index 8f8c8e3..c8b8c1a 100644
--- a/arch/x86/boot/tools/build.c
+++ b/arch/x86/boot/tools/build.c
@@ -59,14 +59,14 @@ u8 buf[SETUP_SECT_MAX*512];
#define PECOFF_COMPAT_RESERVE 0x0
#endif
-unsigned long efi32_stub_entry;
-unsigned long efi64_stub_entry;
-unsigned long efi_pe_entry;
-unsigned long efi32_pe_entry;
-unsigned long kernel_info;
-unsigned long startup_64;
-unsigned long _ehead;
-unsigned long _end;
+static unsigned long efi32_stub_entry;
+static unsigned long efi64_stub_entry;
+static unsigned long efi_pe_entry;
+static unsigned long efi32_pe_entry;
+static unsigned long kernel_info;
+static unsigned long startup_64;
+static unsigned long _ehead;
+static unsigned long _end;
/*--*/
[tip: x86/boot] x86/boot: Correct relocation destination on old linkers
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 5214028dd89e49ba27007c3ee475279e584261f0
Gitweb:
https://git.kernel.org/tip/5214028dd89e49ba27007c3ee475279e584261f0
Author:Arvind Sankar
AuthorDate:Fri, 07 Feb 2020 16:49:26 -05:00
Committer: Borislav Petkov
CommitterDate: Tue, 19 May 2020 14:11:22 +02:00
x86/boot: Correct relocation destination on old linkers
For the 32-bit kernel, as described in
6d92bc9d483a ("x86/build: Build compressed x86 kernels as PIE"),
pre-2.26 binutils generates R_386_32 relocations in PIE mode. Since the
startup code does not perform relocation, any reloc entry with R_386_32
will remain as 0 in the executing code.
Commit
974f221c84b0 ("x86/boot: Move compressed kernel to the end of the
decompression buffer")
added a new symbol _end but did not mark it hidden, which doesn't give
the correct offset on older linkers. This causes the compressed kernel
to be copied beyond the end of the decompression buffer, rather than
flush against it. This region of memory may be reserved or already
allocated for other purposes by the bootloader.
Mark _end as hidden to fix. This changes the relocation from R_386_32 to
R_386_RELATIVE even on the pre-2.26 binutils.
For 64-bit, this is not strictly necessary, as the 64-bit kernel is only
built as PIE if the linker supports -z noreloc-overflow, which implies
binutils-2.27+, but for consistency, mark _end as hidden here too.
The below illustrates the before/after impact of the patch using
binutils-2.25 and gcc-4.6.4 (locally compiled from source) and QEMU.
Disassembly before patch:
48: 8b 86 60 02 00 00 mov0x260(%esi),%eax
4e: 2d 00 00 00 00 sub$0x0,%eax
4f: R_386_32_end
Disassembly after patch:
48: 8b 86 60 02 00 00 mov0x260(%esi),%eax
4e: 2d 00 f0 76 00 sub$0x76f000,%eax
4f: R_386_RELATIVE *ABS*
Dump from extract_kernel before patch:
early console in extract_kernel
input_data: 0x0207c098 <--- this is at output + init_size
input_len: 0x0074fef1
output: 0x0100
output_len: 0x00fa63d0
kernel_total_size: 0x0107c000
needed_size: 0x0107c000
Dump from extract_kernel after patch:
early console in extract_kernel
input_data: 0x0190d098 <--- this is at output + init_size - _end
input_len: 0x0074fef1
output: 0x0100
output_len: 0x00fa63d0
kernel_total_size: 0x0107c000
needed_size: 0x0107c000
Fixes: 974f221c84b0 ("x86/boot: Move compressed kernel to the end of the
decompression buffer")
Signed-off-by: Arvind Sankar
Signed-off-by: Borislav Petkov
Link: https://lkml.kernel.org/r/20200207214926.3564079-1-nived...@alum.mit.edu
---
arch/x86/boot/compressed/head_32.S | 5 +++--
arch/x86/boot/compressed/head_64.S | 1 +
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/head_32.S
b/arch/x86/boot/compressed/head_32.S
index ab33070..03557f2 100644
--- a/arch/x86/boot/compressed/head_32.S
+++ b/arch/x86/boot/compressed/head_32.S
@@ -49,16 +49,17 @@
* Position Independent Executable (PIE) so that linker won't optimize
* R_386_GOT32X relocation to its fixed symbol address. Older
* linkers generate R_386_32 relocations against locally defined symbols,
- * _bss, _ebss, _got and _egot, in PIE. It isn't wrong, just less
+ * _bss, _ebss, _got, _egot and _end, in PIE. It isn't wrong, just less
* optimal than R_386_RELATIVE. But the x86 kernel fails to properly handle
* R_386_32 relocations when relocating the kernel. To generate
- * R_386_RELATIVE relocations, we mark _bss, _ebss, _got and _egot as
+ * R_386_RELATIVE relocations, we mark _bss, _ebss, _got, _egot and _end as
* hidden:
*/
.hidden _bss
.hidden _ebss
.hidden _got
.hidden _egot
+ .hidden _end
__HEAD
SYM_FUNC_START(startup_32)
diff --git a/arch/x86/boot/compressed/head_64.S
b/arch/x86/boot/compressed/head_64.S
index 6b11060..e821a7d 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -42,6 +42,7 @@
.hidden _ebss
.hidden _got
.hidden _egot
+ .hidden _end
__HEAD
.code32
[tip: x86/urgent] lib/string: Make memzero_explicit() inline instead of external
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: bec500777089b3c96c53681fc0aa6fee59711d4a
Gitweb:
https://git.kernel.org/tip/bec500777089b3c96c53681fc0aa6fee59711d4a
Author:Arvind Sankar
AuthorDate:Mon, 07 Oct 2019 18:00:02 -04:00
Committer: Ingo Molnar
CommitterDate: Tue, 08 Oct 2019 13:27:05 +02:00
lib/string: Make memzero_explicit() inline instead of external
With the use of the barrier implied by barrier_data(), there is no need
for memzero_explicit() to be extern. Making it inline saves the overhead
of a function call, and allows the code to be reused in arch/*/purgatory
without having to duplicate the implementation.
Tested-by: Hans de Goede
Signed-off-by: Arvind Sankar
Reviewed-by: Hans de Goede
Cc: Ard Biesheuvel
Cc: Borislav Petkov
Cc: H . Peter Anvin
Cc: Herbert Xu
Cc: Linus Torvalds
Cc: Peter Zijlstra
Cc: Stephan Mueller
Cc: Thomas Gleixner
Cc: linux-cry...@vger.kernel.org
Cc: linux-s...@vger.kernel.org
Fixes: 906a4bb97f5d ("crypto: sha256 - Use get/put_unaligned_be32 to get input,
memzero_explicit")
Link: https://lkml.kernel.org/r/2019100722.ga408...@rani.riverdale.lan
Signed-off-by: Ingo Molnar
---
include/linux/string.h | 21 -
lib/string.c | 21 -
2 files changed, 20 insertions(+), 22 deletions(-)
diff --git a/include/linux/string.h b/include/linux/string.h
index b2f9df7..b6ccdc2 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -227,7 +227,26 @@ static inline bool strstarts(const char *str, const char
*prefix)
}
size_t memweight(const void *ptr, size_t bytes);
-void memzero_explicit(void *s, size_t count);
+
+/**
+ * memzero_explicit - Fill a region of memory (e.g. sensitive
+ * keying data) with 0s.
+ * @s: Pointer to the start of the area.
+ * @count: The size of the area.
+ *
+ * Note: usually using memset() is just fine (!), but in cases
+ * where clearing out _local_ data at the end of a scope is
+ * necessary, memzero_explicit() should be used instead in
+ * order to prevent the compiler from optimising away zeroing.
+ *
+ * memzero_explicit() doesn't need an arch-specific version as
+ * it just invokes the one of memset() implicitly.
+ */
+static inline void memzero_explicit(void *s, size_t count)
+{
+ memset(s, 0, count);
+ barrier_data(s);
+}
/**
* kbasename - return the last part of a pathname.
diff --git a/lib/string.c b/lib/string.c
index cd7a10c..08ec58c 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -748,27 +748,6 @@ void *memset(void *s, int c, size_t count)
EXPORT_SYMBOL(memset);
#endif
-/**
- * memzero_explicit - Fill a region of memory (e.g. sensitive
- * keying data) with 0s.
- * @s: Pointer to the start of the area.
- * @count: The size of the area.
- *
- * Note: usually using memset() is just fine (!), but in cases
- * where clearing out _local_ data at the end of a scope is
- * necessary, memzero_explicit() should be used instead in
- * order to prevent the compiler from optimising away zeroing.
- *
- * memzero_explicit() doesn't need an arch-specific version as
- * it just invokes the one of memset() implicitly.
- */
-void memzero_explicit(void *s, size_t count)
-{
- memset(s, 0, count);
- barrier_data(s);
-}
-EXPORT_SYMBOL(memzero_explicit);
-
#ifndef __HAVE_ARCH_MEMSET16
/**
* memset16() - Fill a memory area with a uint16_t
[tip: x86/urgent] x86/purgatory: Disable the stackleak GCC plugin for the purgatory
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: ca14c996afe7228ff9b480cf225211cc17212688
Gitweb:
https://git.kernel.org/tip/ca14c996afe7228ff9b480cf225211cc17212688
Author:Arvind Sankar
AuthorDate:Mon, 23 Sep 2019 13:17:54 -04:00
Committer: Ingo Molnar
CommitterDate: Mon, 23 Sep 2019 19:48:02 +02:00
x86/purgatory: Disable the stackleak GCC plugin for the purgatory
Since commit:
b059f801a937 ("x86/purgatory: Use CFLAGS_REMOVE rather than reset
KBUILD_CFLAGS")
kexec breaks if GCC_PLUGIN_STACKLEAK=y is enabled, as the purgatory
contains undefined references to stackleak_track_stack.
Attempting to load a kexec kernel results in this failure:
kexec: Undefined symbol: stackleak_track_stack
kexec-bzImage64: Loading purgatory failed
Fix this by disabling the stackleak plugin for the purgatory.
Signed-off-by: Arvind Sankar
Reviewed-by: Nick Desaulniers
Cc: Borislav Petkov
Cc: H. Peter Anvin
Cc: Linus Torvalds
Cc: Peter Zijlstra
Cc: Thomas Gleixner
Fixes: b059f801a937 ("x86/purgatory: Use CFLAGS_REMOVE rather than reset
KBUILD_CFLAGS")
Link: https://lkml.kernel.org/r/20190923171753.ga2252...@rani.riverdale.lan
Signed-off-by: Ingo Molnar
---
arch/x86/purgatory/Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
index 10fb42d..b81b517 100644
--- a/arch/x86/purgatory/Makefile
+++ b/arch/x86/purgatory/Makefile
@@ -23,6 +23,7 @@ KCOV_INSTRUMENT := n
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss
+PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN)
# Default KBUILD_CFLAGS can have -pg option set when FTRACE is enabled. That
# in turn leaves some undefined symbols like __fentry__ in purgatory and not
[tip: x86/urgent] x86/purgatory: Disable the stackleak GCC plugin for the purgatory
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: ca14c996afe7228ff9b480cf225211cc17212688
Gitweb:
https://git.kernel.org/tip/ca14c996afe7228ff9b480cf225211cc17212688
Author:Arvind Sankar
AuthorDate:Mon, 23 Sep 2019 13:17:54 -04:00
Committer: Ingo Molnar
CommitterDate: Mon, 23 Sep 2019 19:48:02 +02:00
x86/purgatory: Disable the stackleak GCC plugin for the purgatory
Since commit:
b059f801a937 ("x86/purgatory: Use CFLAGS_REMOVE rather than reset
KBUILD_CFLAGS")
kexec breaks if GCC_PLUGIN_STACKLEAK=y is enabled, as the purgatory
contains undefined references to stackleak_track_stack.
Attempting to load a kexec kernel results in this failure:
kexec: Undefined symbol: stackleak_track_stack
kexec-bzImage64: Loading purgatory failed
Fix this by disabling the stackleak plugin for the purgatory.
Signed-off-by: Arvind Sankar
Reviewed-by: Nick Desaulniers
Cc: Borislav Petkov
Cc: H. Peter Anvin
Cc: Linus Torvalds
Cc: Peter Zijlstra
Cc: Thomas Gleixner
Fixes: b059f801a937 ("x86/purgatory: Use CFLAGS_REMOVE rather than reset
KBUILD_CFLAGS")
Link: https://lkml.kernel.org/r/20190923171753.ga2252...@rani.riverdale.lan
Signed-off-by: Ingo Molnar
---
arch/x86/purgatory/Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile
index 10fb42d..b81b517 100644
--- a/arch/x86/purgatory/Makefile
+++ b/arch/x86/purgatory/Makefile
@@ -23,6 +23,7 @@ KCOV_INSTRUMENT := n
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel
PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss
+PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN)
# Default KBUILD_CFLAGS can have -pg option set when FTRACE is enabled. That
# in turn leaves some undefined symbols like __fentry__ in purgatory and not
