Module Name: src Committed By: maxv Date: Thu Nov 14 16:23:53 UTC 2019
Modified Files: src/sys/arch/amd64/amd64: amd64_trap.S busfunc.S cpu_in_cksum.S cpufunc.S lock_stubs.S locore.S machdep.c mptramp.S spl.S src/sys/arch/amd64/conf: GENERIC Makefile.amd64 src/sys/arch/amd64/include: cpu.h frameasm.h param.h pmap.h types.h src/sys/arch/x86/include: bus_defs.h pmap.h src/sys/arch/x86/x86: bus_dma.c pmap.c src/sys/conf: files src/sys/kern: files.kern kern_lwp.c kern_malloc.c subr_kmem.c subr_pool.c src/sys/lib/libkern: libkern.h src/sys/net: if.c src/sys/sys: atomic.h bus_proto.h cdefs.h lwp.h systm.h src/sys/uvm: uvm_km.c Added Files: src/sys/arch/amd64/include: msan.h src/sys/kern: subr_msan.c src/sys/sys: msan.h Log Message: Add support for Kernel Memory Sanitizer (kMSan). It detects uninitialized memory used by the kernel at run time, and just like kASan and kCSan, it is an excellent feature. It has already detected 38 uninitialized variables in the kernel during my testing, which I have since discreetly fixed. We use two shadows: - "shad", to track uninitialized memory with a bit granularity (1:1). Each bit set to 1 in the shad corresponds to one uninitialized bit of real kernel memory. - "orig", to track the origin of the memory with a 4-byte granularity (1:1). Each uint32_t cell in the orig indicates the origin of the associated uint32_t of real kernel memory. The memory consumption of these shadows is consequent, so at least 4GB of RAM is recommended to run kMSan. The compiler inserts calls to specific __msan_* functions on each memory access, to manage both the shad and the orig and detect uninitialized memory accesses that change the execution flow (like an "if" on an uninitialized variable). We mark as uninit several types of memory buffers (stack, pools, kmem, malloc, uvm_km), and check each buffer passed to copyout, copyoutstr, bwrite, if_transmit_lock and DMA operations, to detect uninitialized memory that leaves the system. This allows us to detect kernel info leaks in a way that is more efficient and also more user-friendly than KLEAK. Contrary to kASan, kMSan requires comprehensive coverage, ie we cannot tolerate having one non-instrumented function, because this could cause false positives. kMSan cannot instrument ASM functions, so I converted most of them to __asm__ inlines, which kMSan is able to instrument. Those that remain receive special treatment. Contrary to kASan again, kMSan uses a TLS, so we must context-switch this TLS during interrupts. We use different contexts depending on the interrupt level. The orig tracks precisely the origin of a buffer. We use a special encoding for the orig values, and pack together in each uint32_t cell of the orig: - a code designating the type of memory (Stack, Pool, etc), and - a compressed pointer, which points either (1) to a string containing the name of the variable associated with the cell, or (2) to an area in the kernel .text section which we resolve to a symbol name + offset. This encoding allows us not to consume extra memory for associating information with each cell, and produces a precise output, that can tell for example the name of an uninitialized variable on the stack, the function in which it was pushed on the stack, and the function where we accessed this uninitialized variable. kMSan is available with LLVM, but not with GCC. The code is organized in a way that is similar to kASan and kCSan, so it means that other architectures than amd64 can be supported. To generate a diff of this commit: cvs rdiff -u -r1.49 -r1.50 src/sys/arch/amd64/amd64/amd64_trap.S cvs rdiff -u -r1.11 -r1.12 src/sys/arch/amd64/amd64/busfunc.S cvs rdiff -u -r1.3 -r1.4 src/sys/arch/amd64/amd64/cpu_in_cksum.S cvs rdiff -u -r1.46 -r1.47 src/sys/arch/amd64/amd64/cpufunc.S cvs rdiff -u -r1.32 -r1.33 src/sys/arch/amd64/amd64/lock_stubs.S cvs rdiff -u -r1.189 -r1.190 src/sys/arch/amd64/amd64/locore.S cvs rdiff -u -r1.338 -r1.339 src/sys/arch/amd64/amd64/machdep.c cvs rdiff -u -r1.27 -r1.28 src/sys/arch/amd64/amd64/mptramp.S cvs rdiff -u -r1.41 -r1.42 src/sys/arch/amd64/amd64/spl.S cvs rdiff -u -r1.545 -r1.546 src/sys/arch/amd64/conf/GENERIC cvs rdiff -u -r1.79 -r1.80 src/sys/arch/amd64/conf/Makefile.amd64 cvs rdiff -u -r1.64 -r1.65 src/sys/arch/amd64/include/cpu.h cvs rdiff -u -r1.45 -r1.46 src/sys/arch/amd64/include/frameasm.h cvs rdiff -u -r0 -r1.1 src/sys/arch/amd64/include/msan.h cvs rdiff -u -r1.32 -r1.33 src/sys/arch/amd64/include/param.h cvs rdiff -u -r1.63 -r1.64 src/sys/arch/amd64/include/pmap.h \ src/sys/arch/amd64/include/types.h cvs rdiff -u -r1.4 -r1.5 src/sys/arch/x86/include/bus_defs.h cvs rdiff -u -r1.104 -r1.105 src/sys/arch/x86/include/pmap.h cvs rdiff -u -r1.80 -r1.81 src/sys/arch/x86/x86/bus_dma.c cvs rdiff -u -r1.338 -r1.339 src/sys/arch/x86/x86/pmap.c cvs rdiff -u -r1.1243 -r1.1244 src/sys/conf/files cvs rdiff -u -r1.36 -r1.37 src/sys/kern/files.kern cvs rdiff -u -r1.207 -r1.208 src/sys/kern/kern_lwp.c cvs rdiff -u -r1.157 -r1.158 src/sys/kern/kern_malloc.c cvs rdiff -u -r1.76 -r1.77 src/sys/kern/subr_kmem.c cvs rdiff -u -r0 -r1.1 src/sys/kern/subr_msan.c cvs rdiff -u -r1.261 -r1.262 src/sys/kern/subr_pool.c cvs rdiff -u -r1.133 -r1.134 src/sys/lib/libkern/libkern.h cvs rdiff -u -r1.464 -r1.465 src/sys/net/if.c cvs rdiff -u -r1.16 -r1.17 src/sys/sys/atomic.h cvs rdiff -u -r1.10 -r1.11 src/sys/sys/bus_proto.h cvs rdiff -u -r1.148 -r1.149 src/sys/sys/cdefs.h cvs rdiff -u -r1.187 -r1.188 src/sys/sys/lwp.h cvs rdiff -u -r0 -r1.1 src/sys/sys/msan.h cvs rdiff -u -r1.288 -r1.289 src/sys/sys/systm.h cvs rdiff -u -r1.146 -r1.147 src/sys/uvm/uvm_km.c Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.