kimage_load_cma_segment() and kimage_map_segment() translate a CMA segment to a kernel virtual address with page_address(cma), ignoring segment->mem. But arm64's image_load() adds text_offset to segment->mem, so the kernel payload is copied to the wrong offset while image->start points past it, and kexec jumps into the middle of the kernel. kimage_map_segment() has the same problem for any CMA segment whose mem was moved.
Add kimage_cma_vaddr() to translate a boot physical address inside a CMA segment to its virtual address, and use it in both places. Cc: Andrew Morton <[email protected]> Cc: Baoquan He <[email protected]> Cc: Mike Rapoport <[email protected]> Cc: Pasha Tatashin <[email protected]> Cc: Pratyush Yadav <[email protected]> Cc: Pingfan Liu <[email protected]> Cc: Justinien Bouron <[email protected]> Cc: Sourabh Jain <[email protected]> Cc: [email protected] Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation") Signed-off-by: Jinjie Ruan <[email protected]> --- kernel/kexec_core.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index dc770b9a6d05..7f7cb77f0caa 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -738,11 +738,23 @@ static struct page *kimage_alloc_page(struct kimage *image, return page; } +/* + * Translate a boot physical address inside a CMA segment to a kernel + * virtual address. Architecture loaders may move segment->mem away from + * the CMA base (arm64 adds text_offset), so the offset must be preserved. + */ +static void *kimage_cma_vaddr(struct page *cma, unsigned long mem) +{ + unsigned long cma_base = page_to_boot_pfn(cma) << PAGE_SHIFT; + + return page_address(cma) + (mem - cma_base); +} + static int kimage_load_cma_segment(struct kimage *image, int idx) { struct kexec_segment *segment = &image->segment[idx]; struct page *cma = image->segment_cma[idx]; - char *ptr = page_address(cma); + char *ptr = kimage_cma_vaddr(cma, segment->mem); size_t ubytes, mbytes; int result = 0; unsigned char __user *buf = NULL; @@ -965,7 +977,7 @@ void *kimage_map_segment(struct kimage *image, int idx) cma = image->segment_cma[idx]; if (cma) - return page_address(cma); + return kimage_cma_vaddr(cma, image->segment[idx].mem); addr = image->segment[idx].mem; size = image->segment[idx].memsz; -- 2.34.1
