[PATCH v9 6/7] crash: memory and cpu hotplug sysfs attributes
This introduces the crash_hotplug attribute for memory and CPUs
for use by userspace. This change directly facilitates the udev
rule for managing userspace re-loading of the crash kernel upon
hot un/plug changes.
For memory, this changeset introduces the crash_hotplug attribute
to the /sys/devices/system/memory directory. For example:
# udevadm info --attribute-walk /sys/devices/system/memory/memory81
looking at device '/devices/system/memory/memory81':
KERNEL=="memory81"
SUBSYSTEM=="memory"
DRIVER==""
ATTR{online}=="1"
ATTR{phys_device}=="0"
ATTR{phys_index}=="0051"
ATTR{removable}=="1"
ATTR{state}=="online"
ATTR{valid_zones}=="Movable"
looking at parent device '/devices/system/memory':
KERNELS=="memory"
SUBSYSTEMS==""
DRIVERS==""
ATTRS{auto_online_blocks}=="offline"
ATTRS{block_size_bytes}=="800"
ATTRS{crash_hotplug}=="1"
For CPUs, this changeset introduces the crash_hotplug attribute
to the /sys/devices/system/cpu directory. For example:
# udevadm info --attribute-walk /sys/devices/system/cpu/cpu0
looking at device '/devices/system/cpu/cpu0':
KERNEL=="cpu0"
SUBSYSTEM=="cpu"
DRIVER=="processor"
ATTR{crash_notes}=="277c38600"
ATTR{crash_notes_size}=="368"
ATTR{online}=="1"
looking at parent device '/devices/system/cpu':
KERNELS=="cpu"
SUBSYSTEMS==""
DRIVERS==""
ATTRS{crash_hotplug}=="1"
ATTRS{isolated}==""
ATTRS{kernel_max}=="8191"
ATTRS{nohz_full}==" (null)"
ATTRS{offline}=="4-7"
ATTRS{online}=="0-3"
ATTRS{possible}=="0-7"
ATTRS{present}=="0-3"
With these sysfs attributes in place, it is possible to efficiently
instruct the udev rule to skip crash kernel reloading.
For example, the following is the proposed udev rule change for RHEL
system 98-kexec.rules (as the first lines of the rule file):
# The kernel handles updates to crash elfcorehdr for cpu and memory changes
SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
When examined in the context of 98-kexec.rules, the above change
tests if crash_hotplug is set, and if so, it skips the userspace
initiated unload-then-reload of the crash kernel.
Cpu and memory checks are separated in accordance with
CONFIG_HOTPLUG_CPU and CONFIG_MEMORY_HOTPLUG kernel config options.
If an architecture supports, for example, memory hotplug but not
CPU hotplug, then the /sys/devices/system/memory/crash_hotplug
attribute file is present, but the /sys/devices/system/cpu/crash_hotplug
attribute file will NOT be present. Thus the udev rule will skip
userspace processing of memory hot un/plug events, but the udev
rule will fail for CPU events, thus allowing userspace to process
cpu hot un/plug events (ie the unload-then-reload of the kdump
capture kernel).
Signed-off-by: Eric DeVolder
---
.../admin-guide/mm/memory-hotplug.rst | 8
Documentation/core-api/cpu_hotplug.rst | 18 ++
drivers/base/cpu.c | 13 +
drivers/base/memory.c | 13 +
include/linux/kexec.h | 2 ++
kernel/crash_core.c| 2 ++
6 files changed, 56 insertions(+)
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst
b/Documentation/admin-guide/mm/memory-hotplug.rst
index 0f56ecd8ac05..494d7a63c543 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -293,6 +293,14 @@ The following files are currently defined:
Availability depends on the CONFIG_ARCH_MEMORY_PROBE
kernel configuration option.
``uevent``read-write: generic udev file for device subsystems.
+``crash_hotplug`` read-only: when changes to the system memory map
+ occur due to hot un/plug of memory, this file contains
+ '1' if the kernel updates the kdump capture kernel memory
+ map itself (via elfcorehdr), or '0' if userspace must
update
+ the kdump capture kernel memory map.
+
+ Availability depends on the CONFIG_MEMORY_HOTPLUG kernel
+ configuration option.
==
=
.. note::
diff --git a/Documentation/core-api/cpu_hotplug.rst
b/Documentation/core-api/cpu_hotplug.rst
index c6f4ba2fb32d..13e33d098645 100644
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -750,6 +750,24 @@ will receive all events. A script like::
can process the event further.
+When changes to the CPUs in the system occur, the sysfs file
+/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
+updates the kdump capture kernel list of CPUs itself (via elfcorehdr),
+or '0' if users
[PATCH v9 3/7] crash: add generic infrastructure for crash hotplug support
CPU and memory change notifications are received in order to
regenerate the elfcorehdr.
To support cpu hotplug, a callback is registered to capture the
CPUHP_AP_ONLINE_DYN online and offline events via
cpuhp_setup_state_nocalls().
To support memory hotplug, a notifier is registered to capture the
MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().
The cpu callback and memory notifiers call handle_hotplug_event()
to handle the hot plug/unplug event. Then handle_hotplug_event()
dispatches the event to the architecture specific
arch_crash_handle_hotplug_event(). During the process, the
kexec_mutex is held.
Signed-off-by: Eric DeVolder
---
include/linux/crash_core.h | 12
include/linux/kexec.h | 7 +++
kernel/crash_core.c| 124 +
3 files changed, 143 insertions(+)
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..564534820db9 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -84,4 +84,16 @@ int parse_crashkernel_high(char *cmdline, unsigned long long
system_ram,
int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
unsigned long long *crash_size, unsigned long long *crash_base);
+#define KEXEC_CRASH_HP_REMOVE_CPU 0
+#define KEXEC_CRASH_HP_ADD_CPU 1
+#define KEXEC_CRASH_HP_REMOVE_MEMORY 2
+#define KEXEC_CRASH_HP_ADD_MEMORY 3
+#define KEXEC_CRASH_HP_INVALID_CPU -1U
+
+struct kimage;
+void *arch_map_crash_pages(unsigned long paddr, unsigned long size);
+void arch_unmap_crash_pages(void **ptr);
+void arch_crash_handle_hotplug_event(struct kimage *image, unsigned int
hp_action,
+
unsigned int cpu);
+
#endif /* LINUX_CRASH_CORE_H */
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index f93f2591fc1e..f9b13882debf 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -308,6 +308,13 @@ struct kimage {
struct purgatory_info purgatory_info;
#endif
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+ bool hotplug_event;
+ unsigned int offlinecpu;
+ bool elfcorehdr_index_valid;
+ int elfcorehdr_index;
+#endif
+
#ifdef CONFIG_IMA_KEXEC
/* Virtual address of IMA measurement buffer for kexec syscall */
void *ima_buffer;
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 59ad87a7e40d..84eb10a4241f 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -10,12 +10,16 @@
#include
#include
#include
+#include
+#include
#include
#include
#include
+#include "kexec_internal.h"
+
/* vmcoreinfo stuff */
unsigned char *vmcoreinfo_data;
size_t vmcoreinfo_size;
@@ -590,3 +594,123 @@ static int __init crash_save_vmcoreinfo_init(void)
}
subsys_initcall(crash_save_vmcoreinfo_init);
+
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+void __weak arch_crash_handle_hotplug_event(struct kimage *image,
+ unsigned int hp_action,
unsigned int cpu)
+{
+ WARN_ONCE(1, "crash hotplug handler not implemented");
+}
+
+static void handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
+{
+ /* Obtain lock while changing crash information */
+ mutex_lock(&kexec_mutex);
+
+ /* Check kdump is loaded */
+ if (kexec_crash_image) {
+ struct kimage *image = kexec_crash_image;
+
+ pr_debug("crash hp: hp_action %u, cpu %u", hp_action, cpu);
+
+ /*
+* When the struct kimage is alloced, it is wiped to zero, so
+* the elfcorehdr_index_valid defaults to false. Find the
+* segment containing the elfcorehdr, if not already found.
+* This works for both the kexec_load and kexec_file_load paths.
+*/
+ if (!image->elfcorehdr_index_valid) {
+ unsigned char *ptr;
+ unsigned long mem, memsz;
+ unsigned int n;
+
+ for (n = 0; n < image->nr_segments; n++) {
+ mem = image->segment[n].mem;
+ memsz = image->segment[n].memsz;
+ ptr = arch_map_crash_pages(mem, memsz);
+ if (ptr) {
+ /* The segment containing elfcorehdr */
+ if (memcmp(ptr, ELFMAG, SELFMAG) == 0) {
+ image->elfcorehdr_index =
(int)n;
+ image->elfcorehdr_index_valid =
true;
+ }
+ }
+ arch_unmap_crash_pages((void **)&ptr);
+ }
+ }
+
+
[PATCH v9 7/7] x86/crash: Add x86 crash hotplug support
For x86_64, when CPU or memory is hot un/plugged, the crash
elfcorehdr, which describes the CPUs and memory in the system,
must also be updated.
When loading the crash kernel via kexec_load or kexec_file_load,
the elfcorehdr is identified at run time in
crash_core:handle_hotplug_event().
To update the elfcorehdr for x86_64, a new elfcorehdr must be
generated from the available CPUs and memory. The new elfcorehdr
is prepared into a buffer, and then installed over the top of
the existing elfcorehdr.
In the patch 'kexec: exclude elfcorehdr from the segment digest'
the need to update purgatory due to the change in elfcorehdr was
eliminated. As a result, no changes to purgatory or boot_params
(as the elfcorehdr= kernel command line parameter pointer
remains unchanged and correct) are needed, just elfcorehdr.
To accommodate a growing number of resources via hotplug, the
elfcorehdr segment must be sufficiently large enough to accommodate
changes, see the CRASH_MAX_MEMORY_RANGES configure item.
With this change, crash hotplug for kexec_file_load syscall
is supported. The kexec_load is also supported, but also
requires a corresponding change to userspace kexec-tools.
Signed-off-by: Eric DeVolder
---
arch/x86/Kconfig| 11
arch/x86/kernel/crash.c | 116
2 files changed, 127 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 762a0b6ab8b6..e9eecff3b97d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2082,6 +2082,17 @@ config CRASH_DUMP
(CONFIG_RELOCATABLE=y).
For more details see Documentation/admin-guide/kdump/kdump.rst
+config CRASH_MAX_MEMORY_RANGES
+ depends on CRASH_DUMP && KEXEC_FILE && (HOTPLUG_CPU || MEMORY_HOTPLUG)
+ int
+ default 32768
+ help
+ For the kexec_file_load path, specify the maximum number of
+ memory regions, eg. as represented by the 'System RAM' entries
+ in /proc/iomem, that the elfcorehdr buffer/segment can accommodate.
+ This value is combined with NR_CPUS and multiplied by Elf64_Phdr
+ size to determine the final buffer size.
+
config KEXEC_JUMP
bool "kexec jump"
depends on KEXEC && HIBERNATION
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 9db41cce8d97..b9cdf7a0d868 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -25,6 +25,7 @@
#include
#include
#include
+#include
#include
#include
@@ -398,7 +399,17 @@ int crash_load_segments(struct kimage *image)
image->elf_headers = kbuf.buffer;
image->elf_headers_sz = kbuf.bufsz;
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+ /* Ensure elfcorehdr segment large enough for hotplug changes */
+ kbuf.memsz = (CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES)
* sizeof(Elf64_Phdr);
+ /* For marking as usable to crash kernel */
+ image->elf_headers_sz = kbuf.memsz;
+ /* Record the index of the elfcorehdr segment */
+ image->elfcorehdr_index = image->nr_segments;
+ image->elfcorehdr_index_valid = true;
+#else
kbuf.memsz = kbuf.bufsz;
+#endif
kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
ret = kexec_add_buffer(&kbuf);
@@ -413,3 +424,108 @@ int crash_load_segments(struct kimage *image)
return ret;
}
#endif /* CONFIG_KEXEC_FILE */
+
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+int crash_hotplug_support(void) { return 1; }
+void *arch_map_crash_pages(unsigned long paddr, unsigned long size)
+{
+ /*
+* NOTE: The addresses and sizes passed to this routine have
+* already been fully aligned on page boundaries. There is no
+* need for massaging the address or size.
+*/
+ void *ptr = NULL;
+
+ /* NOTE: requires arch_kexec_[un]protect_crashkres() for write access */
+ if (size > 0) {
+ struct page *page = pfn_to_page(paddr >> PAGE_SHIFT);
+
+ ptr = kmap(page);
+ }
+
+ return ptr;
+}
+
+void arch_unmap_crash_pages(void **ptr)
+{
+ if (ptr) {
+ if (*ptr)
+ kunmap(*ptr);
+ *ptr = NULL;
+ }
+}
+
+/**
+ * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
+ * @image: the active struct kimage
+ * @hp_action: the hot un/plug action being handled
+ * @cpu: when KEXEC_CRASH_HP_ADD/REMOVE_CPU, the cpu affected
+ *
+ * To accurately reflect hot un/plug changes, the elfcorehdr (which
+ * is passed to the crash kernel via the elfcorehdr= parameter)
+ * must be updated with the new list of CPUs and memories. The new
+ * elfcorehdr is prepared in a kernel buffer, and then it is
+ * written on top of the existing/old elfcorehdr.
+ *
+ * For hotplug changes to elfcorehdr to work, two conditions are
+ * needed:
+ * First, the segment containing the elfcorehdr must be large enough
+ * to pe
[PATCH v9 2/7] crash: prototype change for crash_prepare_elf64_headers
>From within crash_prepare_elf64_headers() there is a need to
reference the struct kimage hotplug members. As such, this
change passes the struct kimage as a parameter to the
crash_prepare_elf64_headers(). The hotplug members are added
in "crash: add generic infrastructure for crash hotplug support".
This is preparation for later patch, no functionality change.
Signed-off-by: Eric DeVolder
Acked-by: Baoquan He
Acked-by: David Hildenbrand
---
arch/arm64/kernel/machine_kexec_file.c | 6 +++---
arch/powerpc/kexec/file_load_64.c | 2 +-
arch/x86/kernel/crash.c| 3 ++-
include/linux/kexec.h | 5 +++--
kernel/crash_core.c| 4 ++--
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/kernel/machine_kexec_file.c
b/arch/arm64/kernel/machine_kexec_file.c
index 889951291cc0..421d42c338c1 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -39,7 +39,7 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image)
return kexec_image_post_load_cleanup_default(image);
}
-static int prepare_elf_headers(void **addr, unsigned long *sz)
+static int prepare_elf_headers(struct kimage *image, void **addr, unsigned
long *sz)
{
struct crash_mem *cmem;
unsigned int nr_ranges;
@@ -74,7 +74,7 @@ static int prepare_elf_headers(void **addr, unsigned long *sz)
goto out;
}
- ret = crash_prepare_elf64_headers(cmem, true, addr, sz);
+ ret = crash_prepare_elf64_headers(image, cmem, true, addr, sz);
out:
kfree(cmem);
@@ -104,7 +104,7 @@ int load_other_segments(struct kimage *image,
/* load elf core header */
if (image->type == KEXEC_TYPE_CRASH) {
- ret = prepare_elf_headers(&headers, &headers_sz);
+ ret = prepare_elf_headers(image, &headers, &headers_sz);
if (ret) {
pr_err("Preparing elf core header failed\n");
goto out_err;
diff --git a/arch/powerpc/kexec/file_load_64.c
b/arch/powerpc/kexec/file_load_64.c
index b4981b651d9a..07da6bf1cf24 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -797,7 +797,7 @@ static int load_elfcorehdr_segment(struct kimage *image,
struct kexec_buf *kbuf)
goto out;
/* Setup elfcorehdr segment */
- ret = crash_prepare_elf64_headers(cmem, false, &headers, &headers_sz);
+ ret = crash_prepare_elf64_headers(image, cmem, false, &headers,
&headers_sz);
if (ret) {
pr_err("Failed to prepare elf headers for the core\n");
goto out;
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 9730c88530fc..9db41cce8d97 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -265,7 +265,8 @@ static int prepare_elf_headers(struct kimage *image, void
**addr,
goto out;
/* By default prepare 64bit headers */
- ret = crash_prepare_elf64_headers(cmem, IS_ENABLED(CONFIG_X86_64),
addr, sz);
+ ret = crash_prepare_elf64_headers(image, cmem,
+ IS_ENABLED(CONFIG_X86_64), addr, sz);
out:
vfree(cmem);
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 58d1b58a971e..f93f2591fc1e 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -227,8 +227,9 @@ struct crash_mem {
extern int crash_exclude_mem_range(struct crash_mem *mem,
unsigned long long mstart,
unsigned long long mend);
-extern int crash_prepare_elf64_headers(struct crash_mem *mem, int kernel_map,
- void **addr, unsigned long *sz);
+extern int crash_prepare_elf64_headers(struct kimage *image,
+ struct crash_mem *mem, int kernel_map,
+ void **addr, unsigned long *sz);
#endif /* CONFIG_KEXEC_FILE */
#ifdef CONFIG_KEXEC_ELF
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 85058fdce308..59ad87a7e40d 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -306,8 +306,8 @@ static int __init parse_crashkernel_dummy(char *arg)
}
early_param("crashkernel", parse_crashkernel_dummy);
-int crash_prepare_elf64_headers(struct crash_mem *mem, int kernel_map,
- void **addr, unsigned long *sz)
+int crash_prepare_elf64_headers(struct kimage *image, struct crash_mem *mem,
+ int kernel_map, void **addr, unsigned long *sz)
{
Elf64_Ehdr *ehdr;
Elf64_Phdr *phdr;
--
2.31.1
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
[PATCH v9 5/7] kexec: exclude hot remove cpu from elfcorehdr notes
Due to use of CPUHP_AP_ONLINE_DYN, upon CPU unplug, the CPU is
still in the for_each_present_cpu() list when within the
handle_hotplug_event(). Thus the CPU must be explicitly excluded
when building the new list of CPUs.
This change identifies in handle_hotplug_event() the CPU to be
excluded, and the check for excluding the CPU in
crash_prepare_elf64_headers().
Signed-off-by: Eric DeVolder
Acked-by: Baoquan He
---
kernel/crash_core.c | 15 +++
1 file changed, 15 insertions(+)
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 84eb10a4241f..47f667df1c0b 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -358,6 +358,11 @@ int crash_prepare_elf64_headers(struct kimage *image,
struct crash_mem *mem,
/* Prepare one phdr of type PT_NOTE for each present CPU */
for_each_present_cpu(cpu) {
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+ /* Skip the soon-to-be offlined cpu */
+ if (image->hotplug_event && (cpu == image->offlinecpu))
+ continue;
+#endif
phdr->p_type = PT_NOTE;
notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
phdr->p_offset = phdr->p_paddr = notes_addr;
@@ -650,6 +655,16 @@ static void handle_hotplug_event(unsigned int hp_action,
unsigned int cpu)
/* Flag to differentiate between normal load and hotplug */
image->hotplug_event = true;
+ /*
+* Due to use of CPUHP_AP_ONLINE_DYN, upon unplug and during
+* this callback, the CPU is still in the for_each_present_cpu()
+* list. Must explicitly look to exclude this CPU when building
+* new list.
+*/
+ image->offlinecpu =
+ (hp_action == KEXEC_CRASH_HP_REMOVE_CPU) ?
+ cpu : KEXEC_CRASH_HP_INVALID_CPU;
+
/* Now invoke arch-specific update handler */
arch_crash_handle_hotplug_event(image, hp_action, cpu);
--
2.31.1
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
[PATCH v9 0/7] crash: Kernel handling of CPU and memory hot un/plug
When the kdump service is loaded, if a CPU or memory is hot
un/plugged, the crash elfcorehdr (for x86), which describes the CPUs
and memory in the system, must also be updated, else the resulting
vmcore is inaccurate (eg. missing either CPU context or memory
regions).
The current solution utilizes udev to initiate an unload-then-reload
of the kdump image (e. kernel, initrd, boot_params, puratory and
elfcorehdr) by the userspace kexec utility. In previous posts I have
outlined the significant performance problems related to offloading
this activity to userspace.
This patchset introduces a generic crash hot un/plug handler that
registers with the CPU and memory notifiers. Upon CPU or memory
changes, this generic handler is invoked and performs important
housekeeping, for example obtaining the appropriate lock, and then
invokes an architecture specific handler to do the appropriate
updates.
In the case of x86_64, the arch specific handler generates a new
elfcorehdr, and overwrites the old one in memory. No involvement
with userspace needed.
To realize the benefits/test this patchset, one must make a couple
of minor changes to userspace:
- Prevent udev from updating kdump crash kernel on hot un/plug changes.
Add the following as the first lines to the udev rule file
/usr/lib/udev/rules.d/98-kexec.rules:
# The kernel handles updates to crash elfcorehdr for cpu and memory changes
SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
These lines will cause cpu and memory hot un/plug events to be
skipped within this rule file, if the kernel has these changes
enabled.
- Change to the kexec_file_load for loading the kdump kernel:
Eg. on RHEL: in /usr/bin/kdumpctl, change to:
standard_kexec_args="-p -d -s"
which adds the -s to select kexec_file_load syscall.
This patchset supports kexec_load with a modified kexec userspace
utility, and a working changeset to the kexec userspace utility
is provided here (and to use, the above change to standard_kexec_args
would be, for example, to append --hotplug instead of -s).
diff --git a/kexec/arch/i386/crashdump-x86.c b/kexec/arch/i386/crashdump-x86.c
index 9826f6d..4ed395a 100644
--- a/kexec/arch/i386/crashdump-x86.c
+++ b/kexec/arch/i386/crashdump-x86.c
@@ -48,6 +48,7 @@
#include
extern struct arch_options_t arch_options;
+extern int do_hotplug;
static int get_kernel_page_offset(struct kexec_info *UNUSED(info),
struct crash_elf_info *elf_info)
@@ -975,6 +976,14 @@ int load_crashdump_segments(struct kexec_info *info,
char* mod_cmdline,
} else {
memsz = bufsz;
}
+
+ /* If hotplug support enabled, use larger size to accomodate changes */
+ if (do_hotplug) {
+ long int nr_cpus = get_nr_cpus();
+ memsz = (nr_cpus + CRASH_MAX_MEMORY_RANGES) *
sizeof(Elf64_Phdr);
+ }
+
+info->elfcorehdr =
elfcorehdr = add_buffer(info, tmp, bufsz, memsz, align, min_base,
max_addr, -1);
dbgprintf("Created elf header segment at 0x%lx\n", elfcorehdr);
diff --git a/kexec/crashdump-elf.c b/kexec/crashdump-elf.c
index b8bb686..5e29f7a 100644
--- a/kexec/crashdump-elf.c
+++ b/kexec/crashdump-elf.c
@@ -43,11 +43,7 @@ int FUNC(struct kexec_info *info,
int (*get_note_info)(int cpu, uint64_t *addr, uint64_t *len);
long int count_cpu;
- if (xen_present())
- nr_cpus = xen_get_nr_phys_cpus();
- else
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
+ nr_cpus = get_nr_cpus();
if (nr_cpus < 0) {
return -1;
}
diff --git a/kexec/crashdump.h b/kexec/crashdump.h
index 18bd691..28d3278 100644
--- a/kexec/crashdump.h
+++ b/kexec/crashdump.h
@@ -57,7 +57,6 @@ unsigned long phys_to_virt(struct crash_elf_info *elf_info,
unsigned long long paddr);
unsigned long xen_architecture(struct crash_elf_info *elf_info);
-int xen_get_nr_phys_cpus(void);
int xen_get_note(int cpu, uint64_t *addr, uint64_t *len);
int xen_get_crashkernel_region(uint64_t *start, uint64_t *end);
diff --git a/kexec/kexec-xen.h b/kexec/kexec-xen.h
index 70fb576..f54a2dd 100644
--- a/kexec/kexec-xen.h
+++ b/kexec/kexec-xen.h
@@ -83,5 +83,6 @@ extern int __xc_interface_close(xc_interface *xch);
#endif
int xen_get_kexec_range(int range, uint64_t *start, uint64_t *end);
+int xen_get_nr_phys_cpus(void);
#endif /* KEXEC_XEN_H */
diff --git a/kexec/kexec.c b/kexec/kexec.c
index 829a6ea..3668b73 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
@@ -58,6 +58,7 @@
unsigned long long mem_min = 0;
unsigned long long mem_max = ULONG_MAX;
+int do_hotplug = 0;
static unsigned long kexec_flags = 0;
/* Flags for
[PATCH v9 4/7] kexec: exclude elfcorehdr from the segment digest
When a crash kernel is loaded via the kexec_file_load syscall, the
kernel places the various segments (ie crash kernel, crash initrd,
boot_params, elfcorehdr, purgatory, etc) in memory. For those
architectures that utilize purgatory, a hash digest of the segments
is calculated for integrity checking. This digest is embedded into
the purgatory image prior to placing purgatory in memory.
Since hotplug events cause changes to the elfcorehdr, purgatory
integrity checking fails (at crash time, and no kdump created).
As a result, this change explicitly excludes the elfcorehdr segment
from the list of segments used to create the digest. By doing so,
this permits changes to the elfcorehdr in response to hotplug events,
without having to also reload purgatory due to the change to the
digest.
Signed-off-by: Eric DeVolder
Acked-by: Baoquan He
---
kernel/kexec_file.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 7adf883c091c..ae1d3295dfdd 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -765,6 +765,12 @@ static int kexec_calculate_store_digests(struct kimage
*image)
for (j = i = 0; i < image->nr_segments; i++) {
struct kexec_segment *ksegment;
+#if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_MEMORY_HOTPLUG)
+ /* This segment excluded to allow future changes via hotplug */
+ if (image->elfcorehdr_index_valid && (j ==
image->elfcorehdr_index))
+ continue;
+#endif
+
ksegment = &image->segment[i];
/*
* Skip purgatory as it will be modified once we put digest
--
2.31.1
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
[PATCH v9 1/7] crash: move crash_prepare_elf64_headers
At the outcome of this patch set, the crash_prepare_elf64_headers()
is utilized on both the kexec_file_load and kexec_load paths. As
such, need to move this function out of kexec_file.c and into a
common location crash_core.c.
No functionality change.
Signed-off-by: Eric DeVolder
---
kernel/crash_core.c | 100
kernel/kexec_file.c | 99 ---
2 files changed, 100 insertions(+), 99 deletions(-)
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 4d57c03714f4..85058fdce308 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -9,6 +9,7 @@
#include
#include
#include
+#include
#include
#include
@@ -305,6 +306,105 @@ static int __init parse_crashkernel_dummy(char *arg)
}
early_param("crashkernel", parse_crashkernel_dummy);
+int crash_prepare_elf64_headers(struct crash_mem *mem, int kernel_map,
+ void **addr, unsigned long *sz)
+{
+ Elf64_Ehdr *ehdr;
+ Elf64_Phdr *phdr;
+ unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
+ unsigned char *buf;
+ unsigned int cpu, i;
+ unsigned long long notes_addr;
+ unsigned long mstart, mend;
+
+ /* extra phdr for vmcoreinfo ELF note */
+ nr_phdr = nr_cpus + 1;
+ nr_phdr += mem->nr_ranges;
+
+ /*
+* kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
+* area (for example, 8000 - a000 on x86_64).
+* I think this is required by tools like gdb. So same physical
+* memory will be mapped in two ELF headers. One will contain kernel
+* text virtual addresses and other will have __va(physical) addresses.
+*/
+
+ nr_phdr++;
+ elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
+ elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
+
+ buf = vzalloc(elf_sz);
+ if (!buf)
+ return -ENOMEM;
+
+ ehdr = (Elf64_Ehdr *)buf;
+ phdr = (Elf64_Phdr *)(ehdr + 1);
+ memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
+ ehdr->e_ident[EI_CLASS] = ELFCLASS64;
+ ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
+ ehdr->e_ident[EI_VERSION] = EV_CURRENT;
+ ehdr->e_ident[EI_OSABI] = ELF_OSABI;
+ memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
+ ehdr->e_type = ET_CORE;
+ ehdr->e_machine = ELF_ARCH;
+ ehdr->e_version = EV_CURRENT;
+ ehdr->e_phoff = sizeof(Elf64_Ehdr);
+ ehdr->e_ehsize = sizeof(Elf64_Ehdr);
+ ehdr->e_phentsize = sizeof(Elf64_Phdr);
+
+ /* Prepare one phdr of type PT_NOTE for each present CPU */
+ for_each_present_cpu(cpu) {
+ phdr->p_type = PT_NOTE;
+ notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
+ phdr->p_offset = phdr->p_paddr = notes_addr;
+ phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
+ (ehdr->e_phnum)++;
+ phdr++;
+ }
+
+ /* Prepare one PT_NOTE header for vmcoreinfo */
+ phdr->p_type = PT_NOTE;
+ phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
+ phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
+ (ehdr->e_phnum)++;
+ phdr++;
+
+ /* Prepare PT_LOAD type program header for kernel text region */
+ if (kernel_map) {
+ phdr->p_type = PT_LOAD;
+ phdr->p_flags = PF_R|PF_W|PF_X;
+ phdr->p_vaddr = (unsigned long) _text;
+ phdr->p_filesz = phdr->p_memsz = _end - _text;
+ phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
+ ehdr->e_phnum++;
+ phdr++;
+ }
+
+ /* Go through all the ranges in mem->ranges[] and prepare phdr */
+ for (i = 0; i < mem->nr_ranges; i++) {
+ mstart = mem->ranges[i].start;
+ mend = mem->ranges[i].end;
+
+ phdr->p_type = PT_LOAD;
+ phdr->p_flags = PF_R|PF_W|PF_X;
+ phdr->p_offset = mstart;
+
+ phdr->p_paddr = mstart;
+ phdr->p_vaddr = (unsigned long) __va(mstart);
+ phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
+ phdr->p_align = 0;
+ ehdr->e_phnum++;
+ pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx,
paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
+ phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
+ ehdr->e_phnum, phdr->p_offset);
+ phdr++;
+ }
+
+ *addr = buf;
+ *sz = elf_sz;
+ return 0;
+}
+
Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
void *data, size_t data_len)
{
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 8347fc158d2b..7adf883c091c 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -1259,102 +1259,3 @@ int crash_e
Re: [PATCH 0/2] Expose kallsyms data in vmcoreinfo note
Andrew Morton writes: > On Mon, 13 Jun 2022 14:59:44 -0700 Stephen Brennan > wrote: >> >> Related discussion around the BTF side of this: >> >> https://lore.kernel.org/bpf/586a6288-704a-f7a7-b256-e18a67592...@oracle.com/T/#u >> >> >> >> Some work-in-progress branches using this feature: >> >> https://github.com/brenns10/dwarves/tree/remove_percpu_restriction_1 >> >> https://github.com/brenns10/drgn/tree/kallsyms_plus_btf >> > >> > What's the story on using gdb with this? >> >> There is no story with GDB as of yet. I was already familiar with the >> code of drgn when I started down this path, so that's what I used. Drgn >> happens to have a very extensible type system which made it quite simple >> to do. I'd love to see support for doing this with GDB, and might look >> into the feasibility of it, but it's not on my roadmap right now. > > Naive question - could some standalone tool take this kallsyms-based > info, combine it with a core image and create a minimally-dwarfified > file which any debugger can munch on? I'm not too familiar with the guts of DWARF, so honestly my guess may not be any better than yours. One thing that strikes me is that DWARF is typically included in an ELF section, whereas x86_64 kernels are typically a bzImage, so I'm not entirely sure how you'd get back an ELF file suitable to stick the DWARF into. And from there, I really have no guess about the DWARF. I will say that Compact Type Format (CTF) [1] is rather similar to BTF in scope, and it already has support in the GNU Binutils. I'd imagine that there's much more of a fighting chance of converting BTF to CTF and getting GDB to use that instead. [1] https://lwn.net/Articles/795384/ Stephen ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2] Expose kallsyms data in vmcoreinfo note
On Mon, 13 Jun 2022 14:59:44 -0700 Stephen Brennan wrote: > >> Related discussion around the BTF side of this: > >> https://lore.kernel.org/bpf/586a6288-704a-f7a7-b256-e18a67592...@oracle.com/T/#u > >> > >> Some work-in-progress branches using this feature: > >> https://github.com/brenns10/dwarves/tree/remove_percpu_restriction_1 > >> https://github.com/brenns10/drgn/tree/kallsyms_plus_btf > > > > What's the story on using gdb with this? > > There is no story with GDB as of yet. I was already familiar with the > code of drgn when I started down this path, so that's what I used. Drgn > happens to have a very extensible type system which made it quite simple > to do. I'd love to see support for doing this with GDB, and might look > into the feasibility of it, but it's not on my roadmap right now. Naive question - could some standalone tool take this kallsyms-based info, combine it with a core image and create a minimally-dwarfified file which any debugger can munch on? ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2] Expose kallsyms data in vmcoreinfo note
Andrew Morton writes: > On Mon, 16 May 2022 17:05:06 -0700 Stephen Brennan > wrote: > >> The kernel can be configured to contain a lot of introspection or >> debugging information built-in, such as ORC for unwinding stack traces, >> BTF for type information, and of course kallsyms. Debuggers could use >> this information to navigate a core dump or live system, but they need >> to be able to find it. >> >> This patch series adds the necessary symbols into vmcoreinfo, which >> would allow a debugger to find and interpret the kallsyms table. Using >> the kallsyms data, the debugger can then lookup any symbol, allowing it >> to find ORC, BTF, or any other useful data. >> >> This would allow a live kernel, or core dump, to be debugged without >> any DWARF debuginfo. This is useful for many cases: the debuginfo may >> not have been generated, or you may not want to deploy the large files >> everywhere you need them. > > Am trying to understand the value of all of this. Can you explain > further why carrying the dwarf info is problematic? How problematic > are these large files? The primary value I see in this is that it unlocks data that's already present in core dumps. Most distribution kernels already have CONFIG_KALLSYMS_ALL, but you can't reliably find the symbol table in a core dump without a change like this. Most distribution kernels already have a fair amount of type information via BTF already built-in (though as I mention below, there's a bit more that would be useful). So this change can unlock that data, and make core dumps self-describing: you can open one up at any time without extra debuginfo, and start reading symbols and data values out of memory. As far as the DWARF, I do find it unwieldy, we're talking about file sizes ranging from 500 MiB to a 1 GiB or maybe a bit more. This isn't huge for on-disk storage. But for things like OS images, it's pretty hefty just for the off-chance that you need debugging. A bigger concern than the size of the files is that, we frequently deal with customers who, due to some sort of requirements, have a policy against installing debuginfo packages on production machine. When they inevitably ask folks like me to help debug the kernel on their "no debuginfo" machines, I'd like to be able to just get my job done using the info already there, rather than battling them over debuginfo. There's also some corner cases. I've definitely taken a core dump on my personal machine, only to find that the debuginfo had been deleted due to a kernel package upgrade, so the core dump was useless. And some distributions build kernels with no debuginfo packages. This can suit all of those possibilities without undue overhead. >> I've demonstrated a proof of concept for this at LSF/MM+BPF during a >> lighting talk. Using a work-in-progress branch of the drgn debugger, and >> an extended set of BTF generated by a patched version of dwarves, I've >> been able to open a core dump without any DWARF info and do basic tasks >> such as enumerating slab caches, block devices, tasks, and doing >> backtraces. I hope this series can be a first step toward a new >> possibility of "DWARFless debugging". >> >> Related discussion around the BTF side of this: >> https://lore.kernel.org/bpf/586a6288-704a-f7a7-b256-e18a67592...@oracle.com/T/#u >> >> Some work-in-progress branches using this feature: >> https://github.com/brenns10/dwarves/tree/remove_percpu_restriction_1 >> https://github.com/brenns10/drgn/tree/kallsyms_plus_btf > > What's the story on using gdb with this? There is no story with GDB as of yet. I was already familiar with the code of drgn when I started down this path, so that's what I used. Drgn happens to have a very extensible type system which made it quite simple to do. I'd love to see support for doing this with GDB, and might look into the feasibility of it, but it's not on my roadmap right now. That said, the drgn work is moving ahead nicely, much of this code has moved from a "work-in-progress branch" to pull requests which are in review. I have every intention of seeing those changes through. Thanks for taking a look at this. I'm happy to answer more questions or clarify anytihng you want. Stephen ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2] Expose kallsyms data in vmcoreinfo note
On Mon, 16 May 2022 17:05:06 -0700 Stephen Brennan wrote: > The kernel can be configured to contain a lot of introspection or > debugging information built-in, such as ORC for unwinding stack traces, > BTF for type information, and of course kallsyms. Debuggers could use > this information to navigate a core dump or live system, but they need > to be able to find it. > > This patch series adds the necessary symbols into vmcoreinfo, which > would allow a debugger to find and interpret the kallsyms table. Using > the kallsyms data, the debugger can then lookup any symbol, allowing it > to find ORC, BTF, or any other useful data. > > This would allow a live kernel, or core dump, to be debugged without > any DWARF debuginfo. This is useful for many cases: the debuginfo may > not have been generated, or you may not want to deploy the large files > everywhere you need them. Am trying to understand the value of all of this. Can you explain further why carrying the dwarf info is problematic? How problematic are these large files? > I've demonstrated a proof of concept for this at LSF/MM+BPF during a > lighting talk. Using a work-in-progress branch of the drgn debugger, and > an extended set of BTF generated by a patched version of dwarves, I've > been able to open a core dump without any DWARF info and do basic tasks > such as enumerating slab caches, block devices, tasks, and doing > backtraces. I hope this series can be a first step toward a new > possibility of "DWARFless debugging". > > Related discussion around the BTF side of this: > https://lore.kernel.org/bpf/586a6288-704a-f7a7-b256-e18a67592...@oracle.com/T/#u > > Some work-in-progress branches using this feature: > https://github.com/brenns10/dwarves/tree/remove_percpu_restriction_1 > https://github.com/brenns10/drgn/tree/kallsyms_plus_btf What's the story on using gdb with this? ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v5] x86/kexec: Carry forward IMA measurement log on kexec
On Mon, 2022-06-13 at 10:30 +, Jonathan McDowell wrote:
> On kexec file load Integrity Measurement Architecture (IMA) subsystem
> may verify the IMA signature of the kernel and initramfs, and measure
> it. The command line parameters passed to the kernel in the kexec call
> may also be measured by IMA. A remote attestation service can verify
> a TPM quote based on the TPM event log, the IMA measurement list, and
> the TPM PCR data. This can be achieved only if the IMA measurement log
> is carried over from the current kernel to the next kernel across
> the kexec call.
>
> powerpc and ARM64 both achieve this using device tree with a
> "linux,ima-kexec-buffer" node. x86 platforms generally don't make use of
> device tree, so use the setup_data mechanism to pass the IMA buffer to
> the new kernel.
>
> (Mimi, Baoquan, I haven't included your reviewed-bys because this has
> changed the compile guards around the ima_(free|get)_kexec_buffer
> functions in order to fix the warning the kernel test robot found. I
> think this is the right thing to do and avoids us compiling them on
> platforms where they won't be used. The alternative would be to drop
> the guards in ima.h that Mimi requested for
> v4.)
>
> Signed-off-by: Jonathan McDowell
> ---
> v5:
> - Guard ima_(free|get)_kexec_buffer functions with
>CONFIG_HAVE_IMA_KEXEC (kernel test robot)
> - Use setup_data_offset in setup_boot_parameters and update rather than
>calculating in call to setup_ima_state.
> v4:
> - Guard ima.h function prototypes with CONFIG_HAVE_IMA_KEXEC
> diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
> index 8d374cc552be..42a6c5721a43 100644
> --- a/drivers/of/kexec.c
> +++ b/drivers/of/kexec.c
> @@ -9,6 +9,7 @@
> * Copyright (C) 2016 IBM Corporation
> */
>
> +#include
> #include
> #include
> #include
> @@ -115,6 +116,7 @@ static int do_get_kexec_buffer(const void *prop, int len,
> unsigned long *addr,
> return 0;
> }
>
> +#ifdef CONFIG_HAVE_IMA_KEXEC
> /**
> * ima_get_kexec_buffer - get IMA buffer from the previous kernel
> * @addr:On successful return, set to point to the buffer contents.
> @@ -173,6 +175,7 @@ int ima_free_kexec_buffer(void)
>
> return memblock_phys_free(addr, size);
> }
> +#endif
Inside ima_{get,free}_kexec_buffer(), there's no need now to test
whether CONFIG_HAVE_IMA_KEXEC is enabled.
if (!IS_ENABLED(CONFIG_HAVE_IMA_KEXEC))
return -ENOTSUPP;
Otherwise,
Reviewed-by: Mimi Zohar # IMA function
definitions
>
> /**
> * remove_ima_buffer - remove the IMA buffer property and reservation from
> @fdt
> diff --git a/include/linux/ima.h b/include/linux/ima.h
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2] Expose kallsyms data in vmcoreinfo note
Hi Andrew, Baoquan He writes: > On 05/23/22 at 11:00am, Stephen Brennan wrote: >> Do you know which tree this patch will go through? > > I would like to ask Andrew to help check and pick this if no concern. Just wanted to check if you would be willing to take a look at this series and possibly queue it up. I can resend the series directly to you if you need. Thanks, Stephen ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
