Re: [PATCH v3 3/4] x86, boot: Implement ASLR for kernel memory sections (x86_64)

[Thomas Garnier]

On Tue, May 10, 2016 at 11:53 AM, Kees Cook  wrote:
> On Tue, May 3, 2016 at 12:31 PM, Thomas Garnier  wrote:
>> Randomizes the virtual address space of kernel memory sections (physical
>> memory mapping, vmalloc & vmemmap) for x86_64. This security feature
>> mitigates exploits relying on predictable kernel addresses. These
>> addresses can be used to disclose the kernel modules base addresses or
>> corrupt specific structures to elevate privileges bypassing the current
>> implementation of KASLR. This feature can be enabled with the
>> CONFIG_RANDOMIZE_MEMORY option.
>
> I'm struggling to come up with a more accurate name for this, since
> it's a base randomization of the kernel memory sections. Everything
> else seems needlessly long (CONFIG_RANDOMIZE_BASE_MEMORY). I wonder if
> things should be renamed generally to CONFIG_KASLR_BASE,
> CONFIG_KASLR_MEMORY, etc, but that doesn't need to be part of this
> series. Let's leave this as-is, and just make sure it's clear in the
> Kconfig.
>

I agree, leaving it like this for now.

>> The physical memory mapping holds most allocations from boot and heap
>> allocators. Knowning the base address and physical memory size, an
>> attacker can deduce the PDE virtual address for the vDSO memory page.
>> This attack was demonstrated at CanSecWest 2016, in the "Getting
>  Physical Extreme Abuse of Intel Based Paged Systems"
>> https://goo.gl/ANpWdV (see second part of the presentation). The
>> exploits used against Linux worked successfuly against 4.6+ but fail
>> with KASLR memory enabled (https://goo.gl/iTtXMJ). Similar research
>> was done at Google leading to this patch proposal. Variants exists to
>> overwrite /proc or /sys objects ACLs leading to elevation of privileges.
>> These variants were testeda against 4.6+.
>
> Typo "tested".
>

Corrected for next iteration.

>>
>> The vmalloc memory section contains the allocation made through the
>> vmalloc api. The allocations are done sequentially to prevent
>> fragmentation and each allocation address can easily be deduced
>> especially from boot.
>>
>> The vmemmap section holds a representation of the physical
>> memory (through a struct page array). An attacker could use this section
>> to disclose the kernel memory layout (walking the page linked list).
>>
>> The order of each memory section is not changed. The feature looks at
>> the available space for the sections based on different configuration
>> options and randomizes the base and space between each. The size of the
>> physical memory mapping is the available physical memory. No performance
>> impact was detected while testing the feature.
>>
>> Entropy is generated using the KASLR early boot functions now shared in
>> the lib directory (originally written by Kees Cook). Randomization is
>> done on PGD & PUD page table levels to increase possible addresses. The
>> physical memory mapping code was adapted to support PUD level virtual
>> addresses. An additional low memory page is used to ensure each CPU can
>> start with a PGD aligned virtual address (for realmode).
>>
>> x86/dump_pagetable was updated to correctly display each section.
>>
>> Updated documentation on x86_64 memory layout accordingly.
>>
>> Performance data:
>>
>> Kernbench shows almost no difference (-+ less than 1%):
>>
>> Before:
>>
>> Average Optimal load -j 12 Run (std deviation):
>> Elapsed Time 102.63 (1.2695)
>> User Time 1034.89 (1.18115)
>> System Time 87.056 (0.456416)
>> Percent CPU 1092.9 (13.892)
>> Context Switches 199805 (3455.33)
>> Sleeps 97907.8 (900.636)
>>
>> After:
>>
>> Average Optimal load -j 12 Run (std deviation):
>> Elapsed Time 102.489 (1.10636)
>> User Time 1034.86 (1.36053)
>> System Time 87.764 (0.49345)
>> Percent CPU 1095 (12.7715)
>> Context Switches 199036 (4298.1)
>> Sleeps 97681.6 (1031.11)
>>
>> Hackbench shows 0% difference on average (hackbench 90
>> repeated 10 times):
>>
>> attemp,before,after
>> 1,0.076,0.069
>> 2,0.072,0.069
>> 3,0.066,0.066
>> 4,0.066,0.068
>> 5,0.066,0.067
>> 6,0.066,0.069
>> 7,0.067,0.066
>> 8,0.063,0.067
>> 9,0.067,0.065
>> 10,0.068,0.071
>> average,0.0677,0.0677
>>
>> Signed-off-by: Thomas Garnier 
>> ---
>> Based on next-20160502
>> ---
>>  Documentation/x86/x86_64/mm.txt |   4 +
>>  arch/x86/Kconfig|  15 
>>  arch/x86/include/asm/kaslr.h|  12 +++
>>  arch/x86/include/asm/page_64_types.h|  11 ++-
>>  arch/x86/include/asm/pgtable_64.h   |   1 +
>>  arch/x86/include/asm/pgtable_64_types.h |  15 +++-
>>  arch/x86/kernel/head_64.S   |   2 +-
>>  arch/x86/kernel/setup.c |   3 +
>>  arch/x86/mm/Makefile|   1 +
>>  arch/x86/mm/dump_pagetables.c   |  11 ++-
>>  arch/x86/mm/init.c  |   4 +
>>  arch/x86/mm/kaslr.c | 136 
>> 
>>  arch/x86/realmode/init.c|   4 +
>>  13 files 

Re: [PATCH v3 3/4] x86, boot: Implement ASLR for kernel memory sections (x86_64)

[Thomas Garnier]

On Tue, May 10, 2016 at 11:53 AM, Kees Cook  wrote:
> On Tue, May 3, 2016 at 12:31 PM, Thomas Garnier  wrote:
>> Randomizes the virtual address space of kernel memory sections (physical
>> memory mapping, vmalloc & vmemmap) for x86_64. This security feature
>> mitigates exploits relying on predictable kernel addresses. These
>> addresses can be used to disclose the kernel modules base addresses or
>> corrupt specific structures to elevate privileges bypassing the current
>> implementation of KASLR. This feature can be enabled with the
>> CONFIG_RANDOMIZE_MEMORY option.
>
> I'm struggling to come up with a more accurate name for this, since
> it's a base randomization of the kernel memory sections. Everything
> else seems needlessly long (CONFIG_RANDOMIZE_BASE_MEMORY). I wonder if
> things should be renamed generally to CONFIG_KASLR_BASE,
> CONFIG_KASLR_MEMORY, etc, but that doesn't need to be part of this
> series. Let's leave this as-is, and just make sure it's clear in the
> Kconfig.
>

I agree, leaving it like this for now.

>> The physical memory mapping holds most allocations from boot and heap
>> allocators. Knowning the base address and physical memory size, an
>> attacker can deduce the PDE virtual address for the vDSO memory page.
>> This attack was demonstrated at CanSecWest 2016, in the "Getting
>  Physical Extreme Abuse of Intel Based Paged Systems"
>> https://goo.gl/ANpWdV (see second part of the presentation). The
>> exploits used against Linux worked successfuly against 4.6+ but fail
>> with KASLR memory enabled (https://goo.gl/iTtXMJ). Similar research
>> was done at Google leading to this patch proposal. Variants exists to
>> overwrite /proc or /sys objects ACLs leading to elevation of privileges.
>> These variants were testeda against 4.6+.
>
> Typo "tested".
>

Corrected for next iteration.

>>
>> The vmalloc memory section contains the allocation made through the
>> vmalloc api. The allocations are done sequentially to prevent
>> fragmentation and each allocation address can easily be deduced
>> especially from boot.
>>
>> The vmemmap section holds a representation of the physical
>> memory (through a struct page array). An attacker could use this section
>> to disclose the kernel memory layout (walking the page linked list).
>>
>> The order of each memory section is not changed. The feature looks at
>> the available space for the sections based on different configuration
>> options and randomizes the base and space between each. The size of the
>> physical memory mapping is the available physical memory. No performance
>> impact was detected while testing the feature.
>>
>> Entropy is generated using the KASLR early boot functions now shared in
>> the lib directory (originally written by Kees Cook). Randomization is
>> done on PGD & PUD page table levels to increase possible addresses. The
>> physical memory mapping code was adapted to support PUD level virtual
>> addresses. An additional low memory page is used to ensure each CPU can
>> start with a PGD aligned virtual address (for realmode).
>>
>> x86/dump_pagetable was updated to correctly display each section.
>>
>> Updated documentation on x86_64 memory layout accordingly.
>>
>> Performance data:
>>
>> Kernbench shows almost no difference (-+ less than 1%):
>>
>> Before:
>>
>> Average Optimal load -j 12 Run (std deviation):
>> Elapsed Time 102.63 (1.2695)
>> User Time 1034.89 (1.18115)
>> System Time 87.056 (0.456416)
>> Percent CPU 1092.9 (13.892)
>> Context Switches 199805 (3455.33)
>> Sleeps 97907.8 (900.636)
>>
>> After:
>>
>> Average Optimal load -j 12 Run (std deviation):
>> Elapsed Time 102.489 (1.10636)
>> User Time 1034.86 (1.36053)
>> System Time 87.764 (0.49345)
>> Percent CPU 1095 (12.7715)
>> Context Switches 199036 (4298.1)
>> Sleeps 97681.6 (1031.11)
>>
>> Hackbench shows 0% difference on average (hackbench 90
>> repeated 10 times):
>>
>> attemp,before,after
>> 1,0.076,0.069
>> 2,0.072,0.069
>> 3,0.066,0.066
>> 4,0.066,0.068
>> 5,0.066,0.067
>> 6,0.066,0.069
>> 7,0.067,0.066
>> 8,0.063,0.067
>> 9,0.067,0.065
>> 10,0.068,0.071
>> average,0.0677,0.0677
>>
>> Signed-off-by: Thomas Garnier 
>> ---
>> Based on next-20160502
>> ---
>>  Documentation/x86/x86_64/mm.txt |   4 +
>>  arch/x86/Kconfig|  15 
>>  arch/x86/include/asm/kaslr.h|  12 +++
>>  arch/x86/include/asm/page_64_types.h|  11 ++-
>>  arch/x86/include/asm/pgtable_64.h   |   1 +
>>  arch/x86/include/asm/pgtable_64_types.h |  15 +++-
>>  arch/x86/kernel/head_64.S   |   2 +-
>>  arch/x86/kernel/setup.c |   3 +
>>  arch/x86/mm/Makefile|   1 +
>>  arch/x86/mm/dump_pagetables.c   |  11 ++-
>>  arch/x86/mm/init.c  |   4 +
>>  arch/x86/mm/kaslr.c | 136 
>> 
>>  arch/x86/realmode/init.c|   4 +
>>  13 files changed, 211 insertions(+), 8 deletions(-)
>>  create mode 100644 

Re: [PATCH v3 3/4] x86, boot: Implement ASLR for kernel memory sections (x86_64)

[Kees Cook]

On Tue, May 3, 2016 at 12:31 PM, Thomas Garnier  wrote:
> Randomizes the virtual address space of kernel memory sections (physical
> memory mapping, vmalloc & vmemmap) for x86_64. This security feature
> mitigates exploits relying on predictable kernel addresses. These
> addresses can be used to disclose the kernel modules base addresses or
> corrupt specific structures to elevate privileges bypassing the current
> implementation of KASLR. This feature can be enabled with the
> CONFIG_RANDOMIZE_MEMORY option.

I'm struggling to come up with a more accurate name for this, since
it's a base randomization of the kernel memory sections. Everything
else seems needlessly long (CONFIG_RANDOMIZE_BASE_MEMORY). I wonder if
things should be renamed generally to CONFIG_KASLR_BASE,
CONFIG_KASLR_MEMORY, etc, but that doesn't need to be part of this
series. Let's leave this as-is, and just make sure it's clear in the
Kconfig.

> The physical memory mapping holds most allocations from boot and heap
> allocators. Knowning the base address and physical memory size, an
> attacker can deduce the PDE virtual address for the vDSO memory page.
> This attack was demonstrated at CanSecWest 2016, in the "Getting
 Physical Extreme Abuse of Intel Based Paged Systems"
> https://goo.gl/ANpWdV (see second part of the presentation). The
> exploits used against Linux worked successfuly against 4.6+ but fail
> with KASLR memory enabled (https://goo.gl/iTtXMJ). Similar research
> was done at Google leading to this patch proposal. Variants exists to
> overwrite /proc or /sys objects ACLs leading to elevation of privileges.
> These variants were testeda against 4.6+.

Typo "tested".

>
> The vmalloc memory section contains the allocation made through the
> vmalloc api. The allocations are done sequentially to prevent
> fragmentation and each allocation address can easily be deduced
> especially from boot.
>
> The vmemmap section holds a representation of the physical
> memory (through a struct page array). An attacker could use this section
> to disclose the kernel memory layout (walking the page linked list).
>
> The order of each memory section is not changed. The feature looks at
> the available space for the sections based on different configuration
> options and randomizes the base and space between each. The size of the
> physical memory mapping is the available physical memory. No performance
> impact was detected while testing the feature.
>
> Entropy is generated using the KASLR early boot functions now shared in
> the lib directory (originally written by Kees Cook). Randomization is
> done on PGD & PUD page table levels to increase possible addresses. The
> physical memory mapping code was adapted to support PUD level virtual
> addresses. An additional low memory page is used to ensure each CPU can
> start with a PGD aligned virtual address (for realmode).
>
> x86/dump_pagetable was updated to correctly display each section.
>
> Updated documentation on x86_64 memory layout accordingly.
>
> Performance data:
>
> Kernbench shows almost no difference (-+ less than 1%):
>
> Before:
>
> Average Optimal load -j 12 Run (std deviation):
> Elapsed Time 102.63 (1.2695)
> User Time 1034.89 (1.18115)
> System Time 87.056 (0.456416)
> Percent CPU 1092.9 (13.892)
> Context Switches 199805 (3455.33)
> Sleeps 97907.8 (900.636)
>
> After:
>
> Average Optimal load -j 12 Run (std deviation):
> Elapsed Time 102.489 (1.10636)
> User Time 1034.86 (1.36053)
> System Time 87.764 (0.49345)
> Percent CPU 1095 (12.7715)
> Context Switches 199036 (4298.1)
> Sleeps 97681.6 (1031.11)
>
> Hackbench shows 0% difference on average (hackbench 90
> repeated 10 times):
>
> attemp,before,after
> 1,0.076,0.069
> 2,0.072,0.069
> 3,0.066,0.066
> 4,0.066,0.068
> 5,0.066,0.067
> 6,0.066,0.069
> 7,0.067,0.066
> 8,0.063,0.067
> 9,0.067,0.065
> 10,0.068,0.071
> average,0.0677,0.0677
>
> Signed-off-by: Thomas Garnier 
> ---
> Based on next-20160502
> ---
>  Documentation/x86/x86_64/mm.txt |   4 +
>  arch/x86/Kconfig|  15 
>  arch/x86/include/asm/kaslr.h|  12 +++
>  arch/x86/include/asm/page_64_types.h|  11 ++-
>  arch/x86/include/asm/pgtable_64.h   |   1 +
>  arch/x86/include/asm/pgtable_64_types.h |  15 +++-
>  arch/x86/kernel/head_64.S   |   2 +-
>  arch/x86/kernel/setup.c |   3 +
>  arch/x86/mm/Makefile|   1 +
>  arch/x86/mm/dump_pagetables.c   |  11 ++-
>  arch/x86/mm/init.c  |   4 +
>  arch/x86/mm/kaslr.c | 136 
> 
>  arch/x86/realmode/init.c|   4 +
>  13 files changed, 211 insertions(+), 8 deletions(-)
>  create mode 100644 arch/x86/mm/kaslr.c
>
> diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt
> index 5aa7383..602a52d 100644
> --- a/Documentation/x86/x86_64/mm.txt
> +++ 

Re: [PATCH v3 3/4] x86, boot: Implement ASLR for kernel memory sections (x86_64)

[Kees Cook]

On Tue, May 3, 2016 at 12:31 PM, Thomas Garnier  wrote:
> Randomizes the virtual address space of kernel memory sections (physical
> memory mapping, vmalloc & vmemmap) for x86_64. This security feature
> mitigates exploits relying on predictable kernel addresses. These
> addresses can be used to disclose the kernel modules base addresses or
> corrupt specific structures to elevate privileges bypassing the current
> implementation of KASLR. This feature can be enabled with the
> CONFIG_RANDOMIZE_MEMORY option.

I'm struggling to come up with a more accurate name for this, since
it's a base randomization of the kernel memory sections. Everything
else seems needlessly long (CONFIG_RANDOMIZE_BASE_MEMORY). I wonder if
things should be renamed generally to CONFIG_KASLR_BASE,
CONFIG_KASLR_MEMORY, etc, but that doesn't need to be part of this
series. Let's leave this as-is, and just make sure it's clear in the
Kconfig.

> The physical memory mapping holds most allocations from boot and heap
> allocators. Knowning the base address and physical memory size, an
> attacker can deduce the PDE virtual address for the vDSO memory page.
> This attack was demonstrated at CanSecWest 2016, in the "Getting
 Physical Extreme Abuse of Intel Based Paged Systems"
> https://goo.gl/ANpWdV (see second part of the presentation). The
> exploits used against Linux worked successfuly against 4.6+ but fail
> with KASLR memory enabled (https://goo.gl/iTtXMJ). Similar research
> was done at Google leading to this patch proposal. Variants exists to
> overwrite /proc or /sys objects ACLs leading to elevation of privileges.
> These variants were testeda against 4.6+.

Typo "tested".

>
> The vmalloc memory section contains the allocation made through the
> vmalloc api. The allocations are done sequentially to prevent
> fragmentation and each allocation address can easily be deduced
> especially from boot.
>
> The vmemmap section holds a representation of the physical
> memory (through a struct page array). An attacker could use this section
> to disclose the kernel memory layout (walking the page linked list).
>
> The order of each memory section is not changed. The feature looks at
> the available space for the sections based on different configuration
> options and randomizes the base and space between each. The size of the
> physical memory mapping is the available physical memory. No performance
> impact was detected while testing the feature.
>
> Entropy is generated using the KASLR early boot functions now shared in
> the lib directory (originally written by Kees Cook). Randomization is
> done on PGD & PUD page table levels to increase possible addresses. The
> physical memory mapping code was adapted to support PUD level virtual
> addresses. An additional low memory page is used to ensure each CPU can
> start with a PGD aligned virtual address (for realmode).
>
> x86/dump_pagetable was updated to correctly display each section.
>
> Updated documentation on x86_64 memory layout accordingly.
>
> Performance data:
>
> Kernbench shows almost no difference (-+ less than 1%):
>
> Before:
>
> Average Optimal load -j 12 Run (std deviation):
> Elapsed Time 102.63 (1.2695)
> User Time 1034.89 (1.18115)
> System Time 87.056 (0.456416)
> Percent CPU 1092.9 (13.892)
> Context Switches 199805 (3455.33)
> Sleeps 97907.8 (900.636)
>
> After:
>
> Average Optimal load -j 12 Run (std deviation):
> Elapsed Time 102.489 (1.10636)
> User Time 1034.86 (1.36053)
> System Time 87.764 (0.49345)
> Percent CPU 1095 (12.7715)
> Context Switches 199036 (4298.1)
> Sleeps 97681.6 (1031.11)
>
> Hackbench shows 0% difference on average (hackbench 90
> repeated 10 times):
>
> attemp,before,after
> 1,0.076,0.069
> 2,0.072,0.069
> 3,0.066,0.066
> 4,0.066,0.068
> 5,0.066,0.067
> 6,0.066,0.069
> 7,0.067,0.066
> 8,0.063,0.067
> 9,0.067,0.065
> 10,0.068,0.071
> average,0.0677,0.0677
>
> Signed-off-by: Thomas Garnier 
> ---
> Based on next-20160502
> ---
>  Documentation/x86/x86_64/mm.txt |   4 +
>  arch/x86/Kconfig|  15 
>  arch/x86/include/asm/kaslr.h|  12 +++
>  arch/x86/include/asm/page_64_types.h|  11 ++-
>  arch/x86/include/asm/pgtable_64.h   |   1 +
>  arch/x86/include/asm/pgtable_64_types.h |  15 +++-
>  arch/x86/kernel/head_64.S   |   2 +-
>  arch/x86/kernel/setup.c |   3 +
>  arch/x86/mm/Makefile|   1 +
>  arch/x86/mm/dump_pagetables.c   |  11 ++-
>  arch/x86/mm/init.c  |   4 +
>  arch/x86/mm/kaslr.c | 136 
> 
>  arch/x86/realmode/init.c|   4 +
>  13 files changed, 211 insertions(+), 8 deletions(-)
>  create mode 100644 arch/x86/mm/kaslr.c
>
> diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt
> index 5aa7383..602a52d 100644
> --- a/Documentation/x86/x86_64/mm.txt
> +++ b/Documentation/x86/x86_64/mm.txt
> @@ -39,4 +39,8 @@ memory window (this