Re: [PATCH 01/13] x86/acpi: Extract ACPI MADT wakeup code into a separate file
Hi Kirill, On 10/5/2023 6:13 AM, Kirill A. Shutemov wrote: > In order to prepare for the expansion of support for the ACPI MADT > wakeup method, the relevant code has been moved into a separate file. > A new configuration option has been introduced to clearly indicate > dependencies without the use of ifdefs. > > There have been no functional changes. > > Signed-off-by: Kirill A. Shutemov > --- > arch/x86/Kconfig | 7 +++ > arch/x86/include/asm/acpi.h| 5 ++ > arch/x86/kernel/acpi/Makefile | 11 ++-- > arch/x86/kernel/acpi/boot.c| 86 +- > arch/x86/kernel/acpi/madt_wakeup.c | 80 +++ > 5 files changed, 99 insertions(+), 90 deletions(-) > create mode 100644 arch/x86/kernel/acpi/madt_wakeup.c > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index 3154dbc49cf5..7368d254d01f 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -1108,6 +1108,13 @@ config X86_LOCAL_APIC > depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || > PCI_MSI > select IRQ_DOMAIN_HIERARCHY > > +config X86_ACPI_MADT_WAKEUP > + def_bool y > + depends on X86_64 > + depends on ACPI > + depends on SMP > + depends on X86_LOCAL_APIC > + > config X86_IO_APIC > def_bool y > depends on X86_LOCAL_APIC || X86_UP_IOAPIC > diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h > index c8a7fc23f63c..b536b5a6a57b 100644 > --- a/arch/x86/include/asm/acpi.h > +++ b/arch/x86/include/asm/acpi.h > @@ -73,6 +73,11 @@ static inline bool acpi_skip_set_wakeup_address(void) > > #define acpi_skip_set_wakeup_address acpi_skip_set_wakeup_address > > +union acpi_subtable_headers; > + > +int __init acpi_parse_mp_wake(union acpi_subtable_headers *header, > + const unsigned long end); > + IMO, you don't need to declare acpi_parse_mp_wake() in asm/acpi.h. Since the only user of this function is in arch/x86/kernel/acpi, you can either create a header file there or re-use sleep.h. If you want to leave it here, do you want to protect it with CONFIG_X86_ACPI_MADT_WAKEUP? -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 02/13] kernel/cpu: Add support for declaring CPU hotplug not supported
On 10/5/2023 6:13 AM, Kirill A. Shutemov wrote:
> The function cpu_hotplug_not_supported() can be called to indicate that
> CPU hotplug should be disabled. It does not prevent the initial bring up
> of the CPU, but it stops subsequent offlining.
>
> This function is intended to replace CC_ATTR_HOTPLUG_DISABLED.
>
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> Signed-off-by: Kirill A. Shutemov
> ---
> include/linux/cpu.h | 2 ++
> kernel/cpu.c| 17 -
> 2 files changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index f19f56501809..aab3887cadbc 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -132,6 +132,7 @@ extern void cpus_read_lock(void);
> extern void cpus_read_unlock(void);
> extern int cpus_read_trylock(void);
> extern void lockdep_assert_cpus_held(void);
> +extern void cpu_hotplug_not_supported(void);
> extern void cpu_hotplug_disable(void);
> extern void cpu_hotplug_enable(void);
> void clear_tasks_mm_cpumask(int cpu);
> @@ -147,6 +148,7 @@ static inline void cpus_read_lock(void) { }
> static inline void cpus_read_unlock(void) { }
> static inline int cpus_read_trylock(void) { return true; }
> static inline void lockdep_assert_cpus_held(void) { }
> +static inline void cpu_hotplug_not_supported(void) { }
> static inline void cpu_hotplug_disable(void) { }
> static inline void cpu_hotplug_enable(void) { }
> static inline int remove_cpu(unsigned int cpu) { return -EPERM; }
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 6de7c6bb74ee..cf536fe1a88a 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -484,6 +484,9 @@ static int cpu_hotplug_disabled;
>
> DEFINE_STATIC_PERCPU_RWSEM(cpu_hotplug_lock);
>
> +/* Cleared if platform declares CPU hotplug not supported */
> +static bool cpu_hotplug_supported = true;
> +
> void cpus_read_lock(void)
> {
> percpu_down_read(&cpu_hotplug_lock);
> @@ -543,6 +546,18 @@ static void lockdep_release_cpus_lock(void)
> rwsem_release(&cpu_hotplug_lock.dep_map, _THIS_IP_);
> }
>
> +/*
> + * Declare CPU hotplug not supported.
> + *
> + * It doesn't prevent initial bring up of the CPU, but stops offlining.
> + */
> +void cpu_hotplug_not_supported(void)
> +{
> + cpu_maps_update_begin();
> + cpu_hotplug_supported = false;
> + cpu_maps_update_done();
> +}
Since this function is not used in this patch, do you need to add
__maybe_unused to
avoid warnings?
> +
> /*
> * Wait for currently running CPU hotplug operations to complete (if any) and
> * disable future CPU hotplug (from sysfs). The 'cpu_add_remove_lock'
> protects
> @@ -1507,7 +1522,7 @@ static int cpu_down_maps_locked(unsigned int cpu, enum
> cpuhp_state target)
>* If the platform does not support hotplug, report it explicitly to
>* differentiate it from a transient offlining failure.
>*/
> - if (cc_platform_has(CC_ATTR_HOTPLUG_DISABLED))
> + if (cc_platform_has(CC_ATTR_HOTPLUG_DISABLED) || !cpu_hotplug_supported)
> return -EOPNOTSUPP;
> if (cpu_hotplug_disabled)
> return -EBUSY;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 03/13] cpu/hotplug, x86/acpi: Disable CPU hotplug for ACPI MADT wakeup
On 10/5/2023 6:13 AM, Kirill A. Shutemov wrote:
> ACPI MADT doesn't allow to offline CPU after it got woke up.
>
I think you can use the term "CPU hotplug" instead of just offline.
> Currently hotplug prevented based on the confidential computing
> attribute which is set for Intel TDX. But TDX is not the only possible
> user of the wake up method.
>
> Mark CPU hotplug as "not supported" on ACPI MADT wakeup enumeration.
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
>
> Signed-off-by: Kirill A. Shutemov
> ---
> arch/x86/coco/core.c | 1 -
> arch/x86/kernel/acpi/madt_wakeup.c | 4
> include/linux/cc_platform.h| 10 --
> kernel/cpu.c | 2 +-
> 4 files changed, 5 insertions(+), 12 deletions(-)
>
> diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c
> index eeec9986570e..f07c3bb7deab 100644
> --- a/arch/x86/coco/core.c
> +++ b/arch/x86/coco/core.c
> @@ -20,7 +20,6 @@ static bool noinstr intel_cc_platform_has(enum cc_attr attr)
> {
> switch (attr) {
> case CC_ATTR_GUEST_UNROLL_STRING_IO:
> - case CC_ATTR_HOTPLUG_DISABLED:
> case CC_ATTR_GUEST_MEM_ENCRYPT:
> case CC_ATTR_MEM_ENCRYPT:
> return true;
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c
> b/arch/x86/kernel/acpi/madt_wakeup.c
> index 1b9747bfd5b9..15bdf10b1393 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -1,4 +1,5 @@
> #include
> +#include
> #include
>
> /* Physical address of the Multiprocessor Wakeup Structure mailbox */
> @@ -74,6 +75,9 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers
> *header,
>
> acpi_mp_wake_mailbox_paddr = mp_wake->base_address;
>
> + /* Disable CPU onlining/offlining */
> + cpu_hotplug_not_supported();
> +
> apic_update_callback(wakeup_secondary_cpu_64, acpi_wakeup_cpu);
>
> return 0;
> diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
> index cb0d6cd1c12f..d08dd65b5c43 100644
> --- a/include/linux/cc_platform.h
> +++ b/include/linux/cc_platform.h
> @@ -80,16 +80,6 @@ enum cc_attr {
>* using AMD SEV-SNP features.
>*/
> CC_ATTR_GUEST_SEV_SNP,
> -
> - /**
> - * @CC_ATTR_HOTPLUG_DISABLED: Hotplug is not supported or disabled.
> - *
> - * The platform/OS is running as a guest/virtual machine does not
> - * support CPU hotplug feature.
> - *
> - * Examples include TDX Guest.
> - */
> - CC_ATTR_HOTPLUG_DISABLED,
> };
>
> #ifdef CONFIG_ARCH_HAS_CC_PLATFORM
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index cf536fe1a88a..9d4279476b40 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1522,7 +1522,7 @@ static int cpu_down_maps_locked(unsigned int cpu, enum
> cpuhp_state target)
>* If the platform does not support hotplug, report it explicitly to
>* differentiate it from a transient offlining failure.
>*/
> - if (cc_platform_has(CC_ATTR_HOTPLUG_DISABLED) || !cpu_hotplug_supported)
> + if (!cpu_hotplug_supported)
> return -EOPNOTSUPP;
> if (cpu_hotplug_disabled)
> return -EBUSY;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 04/13] x86/kvm: Do not try to disable kvmclock if it was not enabled
On 10/5/2023 6:13 AM, Kirill A. Shutemov wrote:
> kvm_guest_cpu_offline() tries to disable kvmclock regardless if it is
> present in the VM. It leads to write to a MSR that doesn't exist on some
> configurations, namely in TDX guest:
>
> unchecked MSR access error: WRMSR to 0x12 (tried to write
> 0x)
> at rIP: 0x8110687c (kvmclock_disable+0x1c/0x30)
>
> kvmclock enabling is gated by CLOCKSOURCE and CLOCKSOURCE2 KVM paravirt
> features.
>
> Do not disable kvmclock if it was not enumerated or disabled by user
> from kernel command line.
For the above warning, check for CLOCKSOURCE and CLOCKSOURCE2
feature is sufficient, right? Do we need to include user/command-line
disable check here?
>
> Signed-off-by: Kirill A. Shutemov
> Fixes: c02027b5742b ("x86/kvm: Disable kvmclock on all CPUs on shutdown")
> ---
> arch/x86/kernel/kvmclock.c | 9 +++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
> index fb8f52149be9..cba2e732e53f 100644
> --- a/arch/x86/kernel/kvmclock.c
> +++ b/arch/x86/kernel/kvmclock.c
> @@ -22,7 +22,7 @@
> #include
> #include
>
> -static int kvmclock __initdata = 1;
> +static int kvmclock __ro_after_init = 1;
> static int kvmclock_vsyscall __initdata = 1;
> static int msr_kvm_system_time __ro_after_init = MSR_KVM_SYSTEM_TIME;
> static int msr_kvm_wall_clock __ro_after_init = MSR_KVM_WALL_CLOCK;
> @@ -195,7 +195,12 @@ static void kvm_setup_secondary_clock(void)
>
> void kvmclock_disable(void)
> {
> - native_write_msr(msr_kvm_system_time, 0, 0);
> + if (!kvm_para_available() || !kvmclock)
> + return;
> +
> + if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE) ||
> + kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE2))
> + native_write_msr(msr_kvm_system_time, 0, 0);
> }
>
> static void __init kvmclock_init_mem(void)
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv2 01/13] x86/acpi: Extract ACPI MADT wakeup code into a separate file
On 10/20/2023 8:12 AM, Kirill A. Shutemov wrote:
> In order to prepare for the expansion of support for the ACPI MADT
> wakeup method, move the relevant code into a separate file.
>
> Introduce a new configuration option to clearly indicate dependencies
> without the use of ifdefs.
>
> There have been no functional changes.
>
> Signed-off-by: Kirill A. Shutemov
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/Kconfig | 7 +++
> arch/x86/include/asm/acpi.h| 5 ++
> arch/x86/kernel/acpi/Makefile | 11 ++--
> arch/x86/kernel/acpi/boot.c| 86 +-
> arch/x86/kernel/acpi/madt_wakeup.c | 81
> 5 files changed, 100 insertions(+), 90 deletions(-)
> create mode 100644 arch/x86/kernel/acpi/madt_wakeup.c
>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 799102f4d909..9957a73bb386 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -1108,6 +1108,13 @@ config X86_LOCAL_APIC
> depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC ||
> PCI_MSI
> select IRQ_DOMAIN_HIERARCHY
>
> +config X86_ACPI_MADT_WAKEUP
> + def_bool y
> + depends on X86_64
> + depends on ACPI
> + depends on SMP
> + depends on X86_LOCAL_APIC
> +
> config X86_IO_APIC
> def_bool y
> depends on X86_LOCAL_APIC || X86_UP_IOAPIC
> diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
> index c8a7fc23f63c..b536b5a6a57b 100644
> --- a/arch/x86/include/asm/acpi.h
> +++ b/arch/x86/include/asm/acpi.h
> @@ -73,6 +73,11 @@ static inline bool acpi_skip_set_wakeup_address(void)
>
> #define acpi_skip_set_wakeup_address acpi_skip_set_wakeup_address
>
> +union acpi_subtable_headers;
> +
> +int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,
> + const unsigned long end);
> +
> /*
> * Check if the CPU can handle C2 and deeper
> */
> diff --git a/arch/x86/kernel/acpi/Makefile b/arch/x86/kernel/acpi/Makefile
> index fc17b3f136fe..8c7329c88a75 100644
> --- a/arch/x86/kernel/acpi/Makefile
> +++ b/arch/x86/kernel/acpi/Makefile
> @@ -1,11 +1,12 @@
> # SPDX-License-Identifier: GPL-2.0
>
> -obj-$(CONFIG_ACPI) += boot.o
> -obj-$(CONFIG_ACPI_SLEEP) += sleep.o wakeup_$(BITS).o
> -obj-$(CONFIG_ACPI_APEI) += apei.o
> -obj-$(CONFIG_ACPI_CPPC_LIB) += cppc.o
> +obj-$(CONFIG_ACPI) += boot.o
> +obj-$(CONFIG_ACPI_SLEEP) += sleep.o wakeup_$(BITS).o
> +obj-$(CONFIG_ACPI_APEI) += apei.o
> +obj-$(CONFIG_ACPI_CPPC_LIB) += cppc.o
> +obj-$(CONFIG_X86_ACPI_MADT_WAKEUP) += madt_wakeup.o
>
> ifneq ($(CONFIG_ACPI_PROCESSOR),)
> -obj-y+= cstate.o
> +obj-y+= cstate.o
> endif
>
> diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
> index 2a0ea38955df..111bd226ad99 100644
> --- a/arch/x86/kernel/acpi/boot.c
> +++ b/arch/x86/kernel/acpi/boot.c
> @@ -66,13 +66,6 @@ static u64 acpi_lapic_addr __initdata =
> APIC_DEFAULT_PHYS_BASE;
> static bool acpi_support_online_capable;
> #endif
>
> -#ifdef CONFIG_X86_64
> -/* Physical address of the Multiprocessor Wakeup Structure mailbox */
> -static u64 acpi_mp_wake_mailbox_paddr;
> -/* Virtual address of the Multiprocessor Wakeup Structure mailbox */
> -static struct acpi_madt_multiproc_wakeup_mailbox *acpi_mp_wake_mailbox;
> -#endif
> -
> #ifdef CONFIG_X86_IO_APIC
> /*
> * Locks related to IOAPIC hotplug
> @@ -357,60 +350,6 @@ acpi_parse_lapic_nmi(union acpi_subtable_headers *
> header, const unsigned long e
>
> return 0;
> }
> -
> -#ifdef CONFIG_X86_64
> -static int acpi_wakeup_cpu(int apicid, unsigned long start_ip)
> -{
> - /*
> - * Remap mailbox memory only for the first call to acpi_wakeup_cpu().
> - *
> - * Wakeup of secondary CPUs is fully serialized in the core code.
> - * No need to protect acpi_mp_wake_mailbox from concurrent accesses.
> - */
> - if (!acpi_mp_wake_mailbox) {
> - acpi_mp_wake_mailbox = memremap(acpi_mp_wake_mailbox_paddr,
> - sizeof(*acpi_mp_wake_mailbox),
> - MEMREMAP_WB);
> - }
> -
> - /*
> - * Mailbox memory is shared between the firmware and OS. Firmware will
> - * listen on mailbox command address, and once it receives the wakeup
> - * command, the CPU associated with the given apicid will be booted.
> - *
> - * The
Re: [PATCHv2 12/13] x86/acpi: Rename fields in acpi_madt_multiproc_wakeup structure
On 10/20/2023 8:12 AM, Kirill A. Shutemov wrote:
> To prepare for the addition of support for MADT wakeup structure version
> 1, it is necessary to provide more appropriate names for the fields in
> the structure.
>
> The field 'mailbox_version' renamed as 'version'. This field signifies
> the version of the structure and the related protocols, rather than the
> version of the mailbox. This field has not been utilized in the code
> thus far.
>
> The field 'base_address' renamed as 'mailbox_address' to clarify the
> kind of address it represents. In version 1, the structure includes the
> reset vector address. Clear and distinct naming helps to prevent any
> confusion.
>
> Signed-off-by: Kirill A. Shutemov
> ---
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/kernel/acpi/madt_wakeup.c | 4 ++--
> include/acpi/actbl2.h | 4 ++--
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c
> b/arch/x86/kernel/acpi/madt_wakeup.c
> index 9bbe829737e7..ad170def2367 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -79,7 +79,7 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers
> *header,
>
> acpi_table_print_madt_entry(&header->common);
>
> - acpi_mp_wake_mailbox_paddr = mp_wake->base_address;
> + acpi_mp_wake_mailbox_paddr = mp_wake->mailbox_address;
>
> cpu_hotplug_disable_offlining();
>
> @@ -98,7 +98,7 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers
> *header,
>*
>* This is Linux-specific protocol and not reflected in ACPI spec.
>*/
> - mp_wake->base_address = 0;
> + mp_wake->mailbox_address = 0;
>
> apic_update_callback(wakeup_secondary_cpu_64, acpi_wakeup_cpu);
>
> diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h
> index 3751ae69432f..23b4cfb640fc 100644
> --- a/include/acpi/actbl2.h
> +++ b/include/acpi/actbl2.h
> @@ -1109,9 +1109,9 @@ struct acpi_madt_generic_translator {
>
> struct acpi_madt_multiproc_wakeup {
> struct acpi_subtable_header header;
> - u16 mailbox_version;
> + u16 version;
> u32 reserved; /* reserved - must be zero */
> - u64 base_address;
> + u64 mailbox_address;
> };
>
> #define ACPI_MULTIPROC_WAKEUP_MB_OS_SIZE2032
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv2 11/13] x86/acpi: Do not attempt to bring up secondary CPUs in kexec case
On 10/20/2023 8:12 AM, Kirill A. Shutemov wrote:
> ACPI MADT doesn't allow to offline CPU after it got woke up. It limits
> kexec: the second kernel won't be able to use more than one CPU.
>
> Now acpi_mp_wake_mailbox_paddr already has the mailbox address.
> The acpi_wakeup_cpu() will use it to bring up secondary cpus.
>
> Zero out mailbox address in the ACPI MADT wakeup structure to indicate
> that the mailbox is not usable. This prevents the kexec()-ed kernel
> from reading a vaild mailbox, which in turn makes the kexec()-ed kernel
> only be able to use the boot CPU.
>
> This is Linux-specific protocol and not reflected in ACPI spec.
>
> Booting the second kernel with signle CPU is enough to cover the most
> common case for kexec -- kdump.
>
> Signed-off-by: Kirill A. Shutemov
> ---
> arch/x86/kernel/acpi/madt_wakeup.c | 22 ++
> 1 file changed, 22 insertions(+)
>
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c
> b/arch/x86/kernel/acpi/madt_wakeup.c
> index 4bc1d5106afd..9bbe829737e7 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -13,6 +13,11 @@ static struct acpi_madt_multiproc_wakeup_mailbox
> *acpi_mp_wake_mailbox;
>
> static int acpi_wakeup_cpu(int apicid, unsigned long start_ip)
> {
> + if (!acpi_mp_wake_mailbox_paddr) {
> + pr_warn_once("No MADT mailbox: cannot bringup secondary CPUs.
> Booting with kexec?\n");
> + return -EOPNOTSUPP;
> + }
> +
> /*
>* Remap mailbox memory only for the first call to acpi_wakeup_cpu().
>*
> @@ -78,6 +83,23 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers
> *header,
>
> cpu_hotplug_disable_offlining();
>
> + /*
> + * ACPI MADT doesn't allow to offline CPU after it got woke up.
> + * It limits kexec: the second kernel won't be able to use more than
> + * one CPU.
> + *
> + * Now acpi_mp_wake_mailbox_paddr already has the mailbox address.
> + * The acpi_wakeup_cpu() will use it to bring up secondary cpus.
> + *
> + * Zero out mailbox address in the ACPI MADT wakeup structure to
> + * indicate that the mailbox is not usable. This prevents the
> + * kexec()-ed kernel from reading a vaild mailbox, which in turn
> + * makes the kexec()-ed kernel only be able to use the boot CPU.
> + *
> + * This is Linux-specific protocol and not reflected in ACPI spec.
> + */
> + mp_wake->base_address = 0;
Is there any way to skip secondary CPU bring-up for kexec case instead of
returning error in ->wakeup_secondary_cpu_64()?
> +
> apic_update_callback(wakeup_secondary_cpu_64, acpi_wakeup_cpu);
>
> return 0;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv3 13/14] x86/acpi: Do not attempt to bring up secondary CPUs in kexec case
On 11/15/2023 4:00 AM, Kirill A. Shutemov wrote:
> ACPI MADT doesn't allow to offline CPU after it got woke up. It limits
> kexec: the second kernel won't be able to use more than one CPU.
>
> Now acpi_mp_wake_mailbox_paddr already has the mailbox address.
> The acpi_wakeup_cpu() will use it to bring up secondary cpus.
>
> Zero out mailbox address in the ACPI MADT wakeup structure to indicate
> that the mailbox is not usable. This prevents the kexec()-ed kernel
> from reading a vaild mailbox, which in turn makes the kexec()-ed kernel
> only be able to use the boot CPU.
>
> This is Linux-specific protocol and not reflected in ACPI spec.
>
> Booting the second kernel with signle CPU is enough to cover the most
> common case for kexec -- kdump.
>
> Signed-off-by: Kirill A. Shutemov
> Reviewed-by: Kai Huang
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/kernel/acpi/madt_wakeup.c | 22 ++
> 1 file changed, 22 insertions(+)
>
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c
> b/arch/x86/kernel/acpi/madt_wakeup.c
> index 386adbb03094..5d92d12f1042 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -13,6 +13,11 @@ static struct acpi_madt_multiproc_wakeup_mailbox
> *acpi_mp_wake_mailbox __ro_afte
>
> static int acpi_wakeup_cpu(u32 apicid, unsigned long start_ip)
> {
> + if (!acpi_mp_wake_mailbox_paddr) {
> + pr_warn_once("No MADT mailbox: cannot bringup secondary CPUs.
> Booting with kexec?\n");
> + return -EOPNOTSUPP;
> + }
> +
> /*
>* Remap mailbox memory only for the first call to acpi_wakeup_cpu().
>*
> @@ -78,6 +83,23 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers
> *header,
>
> cpu_hotplug_disable_offlining();
>
> + /*
> + * ACPI MADT doesn't allow to offline CPU after it got woke up.
> + * It limits kexec: the second kernel won't be able to use more than
> + * one CPU.
> + *
> + * Now acpi_mp_wake_mailbox_paddr already has the mailbox address.
> + * The acpi_wakeup_cpu() will use it to bring up secondary cpus.
> + *
> + * Zero out mailbox address in the ACPI MADT wakeup structure to
> + * indicate that the mailbox is not usable. This prevents the
> + * kexec()-ed kernel from reading a vaild mailbox, which in turn
> + * makes the kexec()-ed kernel only be able to use the boot CPU.
> + *
> + * This is Linux-specific protocol and not reflected in ACPI spec.
> + */
> + mp_wake->mailbox_address = 0;
> +
> apic_update_callback(wakeup_secondary_cpu_64, acpi_wakeup_cpu);
>
> return 0;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 00/11] Implement generic prot_guest_has() helper function
Hi Tom,
On 7/27/21 3:26 PM, Tom Lendacky wrote:
This patch series provides a generic helper function, prot_guest_has(),
to replace the sme_active(), sev_active(), sev_es_active() and
mem_encrypt_active() functions.
It is expected that as new protected virtualization technologies are
added to the kernel, they can all be covered by a single function call
instead of a collection of specific function calls all called from the
same locations.
The powerpc and s390 patches have been compile tested only. Can the
folks copied on this series verify that nothing breaks for them.
With this patch set, select ARCH_HAS_PROTECTED_GUEST and set
CONFIG_AMD_MEM_ENCRYPT=n, creates following error.
ld: arch/x86/mm/ioremap.o: in function `early_memremap_is_setup_data':
arch/x86/mm/ioremap.c:672: undefined reference to `early_memremap_decrypted'
It looks like early_memremap_is_setup_data() is not protected with
appropriate config.
Cc: Andi Kleen
Cc: Andy Lutomirski
Cc: Ard Biesheuvel
Cc: Baoquan He
Cc: Benjamin Herrenschmidt
Cc: Borislav Petkov
Cc: Christian Borntraeger
Cc: Daniel Vetter
Cc: Dave Hansen
Cc: Dave Young
Cc: David Airlie
Cc: Heiko Carstens
Cc: Ingo Molnar
Cc: Joerg Roedel
Cc: Maarten Lankhorst
Cc: Maxime Ripard
Cc: Michael Ellerman
Cc: Paul Mackerras
Cc: Peter Zijlstra
Cc: Thomas Gleixner
Cc: Thomas Zimmermann
Cc: Vasily Gorbik
Cc: VMware Graphics
Cc: Will Deacon
---
Patches based on:
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git master
commit 79e920060fa7 ("Merge branch 'WIP/fixes'")
Tom Lendacky (11):
mm: Introduce a function to check for virtualization protection
features
x86/sev: Add an x86 version of prot_guest_has()
powerpc/pseries/svm: Add a powerpc version of prot_guest_has()
x86/sme: Replace occurrences of sme_active() with prot_guest_has()
x86/sev: Replace occurrences of sev_active() with prot_guest_has()
x86/sev: Replace occurrences of sev_es_active() with prot_guest_has()
treewide: Replace the use of mem_encrypt_active() with
prot_guest_has()
mm: Remove the now unused mem_encrypt_active() function
x86/sev: Remove the now unused mem_encrypt_active() function
powerpc/pseries/svm: Remove the now unused mem_encrypt_active()
function
s390/mm: Remove the now unused mem_encrypt_active() function
arch/Kconfig | 3 ++
arch/powerpc/include/asm/mem_encrypt.h | 5 --
arch/powerpc/include/asm/protected_guest.h | 30 +++
arch/powerpc/platforms/pseries/Kconfig | 1 +
arch/s390/include/asm/mem_encrypt.h| 2 -
arch/x86/Kconfig | 1 +
arch/x86/include/asm/kexec.h | 2 +-
arch/x86/include/asm/mem_encrypt.h | 13 +
arch/x86/include/asm/protected_guest.h | 27 ++
arch/x86/kernel/crash_dump_64.c| 4 +-
arch/x86/kernel/head64.c | 4 +-
arch/x86/kernel/kvm.c | 3 +-
arch/x86/kernel/kvmclock.c | 4 +-
arch/x86/kernel/machine_kexec_64.c | 19 +++
arch/x86/kernel/pci-swiotlb.c | 9 ++--
arch/x86/kernel/relocate_kernel_64.S | 2 +-
arch/x86/kernel/sev.c | 6 +--
arch/x86/kvm/svm/svm.c | 3 +-
arch/x86/mm/ioremap.c | 16 +++---
arch/x86/mm/mem_encrypt.c | 60 +++---
arch/x86/mm/mem_encrypt_identity.c | 3 +-
arch/x86/mm/pat/set_memory.c | 3 +-
arch/x86/platform/efi/efi_64.c | 9 ++--
arch/x86/realmode/init.c | 8 +--
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c| 4 +-
drivers/gpu/drm/drm_cache.c| 4 +-
drivers/gpu/drm/vmwgfx/vmwgfx_drv.c| 4 +-
drivers/gpu/drm/vmwgfx/vmwgfx_msg.c| 6 +--
drivers/iommu/amd/init.c | 7 +--
drivers/iommu/amd/iommu.c | 3 +-
drivers/iommu/amd/iommu_v2.c | 3 +-
drivers/iommu/iommu.c | 3 +-
fs/proc/vmcore.c | 6 +--
include/linux/mem_encrypt.h| 4 --
include/linux/protected_guest.h| 37 +
kernel/dma/swiotlb.c | 4 +-
36 files changed, 218 insertions(+), 104 deletions(-)
create mode 100644 arch/powerpc/include/asm/protected_guest.h
create mode 100644 arch/x86/include/asm/protected_guest.h
create mode 100644 include/linux/protected_guest.h
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 06/11] x86/sev: Replace occurrences of sev_es_active() with prot_guest_has()
On 8/9/21 2:59 PM, Tom Lendacky wrote: Not sure how TDX will handle AP booting, are you sure it needs this special setup as well? Otherwise a check for SEV-ES would be better instead of the generic PATTR_GUEST_PROT_STATE. Yes, I'm not sure either. I figure that change can be made, if needed, as part of the TDX support. We don't plan to set PROT_STATE. So it does not affect TDX. For SMP, we use MADT ACPI table for AP booting. -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 07/11] treewide: Replace the use of mem_encrypt_active() with prot_guest_has()
On 7/27/21 3:26 PM, Tom Lendacky wrote:
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index de01903c3735..cafed6456d45 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -19,7 +19,7 @@
#include
#include
#include
-#include
+#include
#include
#include
@@ -285,7 +285,7 @@ unsigned long __head __startup_64(unsigned long physaddr,
* there is no need to zero it after changing the memory encryption
* attribute.
*/
- if (mem_encrypt_active()) {
+ if (prot_guest_has(PATTR_MEM_ENCRYPT)) {
vaddr = (unsigned long)__start_bss_decrypted;
vaddr_end = (unsigned long)__end_bss_decrypted;
Since this change is specific to AMD, can you replace PATTR_MEM_ENCRYPT with
prot_guest_has(PATTR_SME) || prot_guest_has(PATTR_SEV). It is not used in
TDX.
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 07/11] treewide: Replace the use of mem_encrypt_active() with prot_guest_has()
On 8/10/21 12:48 PM, Tom Lendacky wrote:
On 8/10/21 1:45 PM, Kuppuswamy, Sathyanarayanan wrote:
On 7/27/21 3:26 PM, Tom Lendacky wrote:
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index de01903c3735..cafed6456d45 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -19,7 +19,7 @@
#include
#include
#include
-#include
+#include
#include
#include
@@ -285,7 +285,7 @@ unsigned long __head __startup_64(unsigned long
physaddr,
* there is no need to zero it after changing the memory encryption
* attribute.
*/
- if (mem_encrypt_active()) {
+ if (prot_guest_has(PATTR_MEM_ENCRYPT)) {
vaddr = (unsigned long)__start_bss_decrypted;
vaddr_end = (unsigned long)__end_bss_decrypted;
Since this change is specific to AMD, can you replace PATTR_MEM_ENCRYPT with
prot_guest_has(PATTR_SME) || prot_guest_has(PATTR_SEV). It is not used in
TDX.
This is a direct replacement for now. I think the change you're requesting
should be done as part of the TDX support patches so it's clear why it is
being changed.
Ok. I will include it part of TDX changes.
But, wouldn't TDX still need to do something with this shared/unencrypted
area, though? Or since it is shared, there's actually nothing you need to
do (the bss decrpyted section exists even if CONFIG_AMD_MEM_ENCRYPT is not
configured)?
Kirill had a requirement to turn on CONFIG_AMD_MEM_ENCRYPT for adding lazy
accept support in TDX guest kernel. Kirill, can you add details here?
Thanks,
Tom
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 01/11] mm: Introduce a function to check for virtualization protection features
On 7/27/21 3:26 PM, Tom Lendacky wrote:
diff --git a/include/linux/protected_guest.h b/include/linux/protected_guest.h
new file mode 100644
index ..f8ed7b72967b
--- /dev/null
+++ b/include/linux/protected_guest.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Protected Guest (and Host) Capability checks
+ *
+ * Copyright (C) 2021 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky
+ */
+
+#ifndef _PROTECTED_GUEST_H
+#define _PROTECTED_GUEST_H
+
+#ifndef __ASSEMBLY__
Can you include headers for bool type and false definition?
--- a/include/linux/protected_guest.h
+++ b/include/linux/protected_guest.h
@@ -12,6 +12,9 @@
#ifndef __ASSEMBLY__
+#include
+#include
Otherwise, I see following errors in multi-config auto testing.
include/linux/protected_guest.h:40:15: error: unknown type name 'bool'
include/linux/protected_guest.h:40:63: error: 'false' undeclared (first use in
this functi
+
+#define PATTR_MEM_ENCRYPT 0 /* Encrypted memory */
+#define PATTR_HOST_MEM_ENCRYPT 1 /* Host encrypted memory */
+#define PATTR_GUEST_MEM_ENCRYPT2 /* Guest encrypted
memory */
+#define PATTR_GUEST_PROT_STATE 3 /* Guest encrypted state */
+
+#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST
+
+#include
+
+#else /* !CONFIG_ARCH_HAS_PROTECTED_GUEST */
+
+static inline bool prot_guest_has(unsigned int attr) { return false; }
+
+#endif /* CONFIG_ARCH_HAS_PROTECTED_GUEST */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _PROTECTED_GUEST_H */
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 02/12] mm: Introduce a function to check for virtualization protection features
On 8/13/21 9:59 AM, Tom Lendacky wrote: In prep for other protected virtualization technologies, introduce a generic helper function, prot_guest_has(), that can be used to check for specific protection attributes, like memory encryption. This is intended to eliminate having to add multiple technology-specific checks to the code (e.g. if (sev_active() || tdx_active())). Reviewed-by: Joerg Roedel Co-developed-by: Andi Kleen Signed-off-by: Andi Kleen Co-developed-by: Kuppuswamy Sathyanarayanan Signed-off-by: Kuppuswamy Sathyanarayanan Signed-off-by: Tom Lendacky --- arch/Kconfig| 3 +++ include/linux/protected_guest.h | 35 + 2 files changed, 38 insertions(+) create mode 100644 include/linux/protected_guest.h Reviewed-by: Kuppuswamy Sathyanarayanan -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 03/12] x86/sev: Add an x86 version of prot_guest_has()
On 8/19/21 11:33 AM, Tom Lendacky wrote: There was some talk about this on the mailing list where TDX and SEV may need to be differentiated, so we wanted to reserve a range of values per technology. I guess I can remove them until they are actually needed. In TDX also we have similar requirements and we need some flags for TDX specific checks. So I think it is fine to leave some space for vendor flags. -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v3 0/8] Implement generic cc_platform_has() helper function
On 9/16/21 8:02 AM, Borislav Petkov wrote: On Wed, Sep 15, 2021 at 10:26:06AM -0700, Kuppuswamy, Sathyanarayanan wrote: I have a Intel variant patch (please check following patch). But it includes TDX changes as well. Shall I move TDX changes to different patch and just create a separate patch for adding intel_cc_platform_has()? Yes, please, so that I can expedite that stuff separately and so that it can go in early in order for future work to be based ontop. Sent it part of TDX patch series. Please check and cherry pick it. https://lore.kernel.org/lkml/20210916183550.15349-2-sathyanarayanan.kuppusw...@linux.intel.com/ Thx. -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v3 0/8] Implement generic cc_platform_has() helper function
On 9/15/21 9:46 AM, Borislav Petkov wrote:
Sathya,
if you want to prepare the Intel variant intel_cc_platform_has() ontop
of those and send it to me, that would be good because then I can
integrate it all in one branch which can be used to base future work
ontop.
I have a Intel variant patch (please check following patch). But it includes
TDX changes as well. Shall I move TDX changes to different patch and just
create a separate patch for adding intel_cc_platform_has()?
commit fc5f98a0ed94629d903827c5b44ee9295f835831
Author: Kuppuswamy Sathyanarayanan
Date: Wed May 12 11:35:13 2021 -0700
x86/tdx: Add confidential guest support for TDX guest
TDX architecture provides a way for VM guests to be highly secure and
isolated (from untrusted VMM). To achieve this requirement, any data
coming from VMM cannot be completely trusted. TDX guest fixes this
issue by hardening the IO drivers against the attack from the VMM.
So, when adding hardening fixes to the generic drivers, to protect
custom fixes use cc_platform_has() API.
Also add TDX guest support to cc_platform_has() API to protect the
TDX specific fixes.
Signed-off-by: Kuppuswamy Sathyanarayanan
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index a5b14de03458..2e78358923a1 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -871,6 +871,7 @@ config INTEL_TDX_GUEST
depends on SECURITY
select X86_X2APIC
select SECURITY_LOCKDOWN_LSM
+ select ARCH_HAS_CC_PLATFORM
help
Provide support for running in a trusted domain on Intel processors
equipped with Trusted Domain eXtensions. TDX is a new Intel
diff --git a/arch/x86/include/asm/intel_cc_platform.h
b/arch/x86/include/asm/intel_cc_platform.h
new file mode 100644
index ..472c3174beac
--- /dev/null
+++ b/arch/x86/include/asm/intel_cc_platform.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 Intel Corporation */
+#ifndef _ASM_X86_INTEL_CC_PLATFORM_H
+#define _ASM_X86_INTEL_CC_PLATFORM_H
+
+#if defined(CONFIG_CPU_SUP_INTEL) && defined(CONFIG_ARCH_HAS_CC_PLATFORM)
+bool intel_cc_platform_has(unsigned int flag);
+#else
+static inline bool intel_cc_platform_has(unsigned int flag) { return false; }
+#endif
+
+#endif /* _ASM_X86_INTEL_CC_PLATFORM_H */
+
diff --git a/arch/x86/kernel/cc_platform.c b/arch/x86/kernel/cc_platform.c
index 3c9bacd3c3f3..e83bc2f48efe 100644
--- a/arch/x86/kernel/cc_platform.c
+++ b/arch/x86/kernel/cc_platform.c
@@ -10,11 +10,16 @@
#include
#include
#include
+#include
+
+#include
bool cc_platform_has(enum cc_attr attr)
{
if (sme_me_mask)
return amd_cc_platform_has(attr);
+ else if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL)
+ return intel_cc_platform_has(attr);
return false;
}
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8321c43554a1..ab486a3b1eb0 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -60,6 +61,21 @@ static u64 msr_test_ctrl_cache __ro_after_init;
*/
static bool cpu_model_supports_sld __ro_after_init;
+#ifdef CONFIG_ARCH_HAS_CC_PLATFORM
+bool intel_cc_platform_has(enum cc_attr attr)
+{
+ switch (attr) {
+ case CC_ATTR_GUEST_TDX:
+ return cpu_feature_enabled(X86_FEATURE_TDX_GUEST);
+ default:
+ return false;
+ }
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(intel_cc_platform_has);
+#endif
+
/*
* Processors which have self-snooping capability can handle conflicting
* memory type across CPUs by snooping its own cache. However, there exists
diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
index 253f3ea66cd8..e38430e6e396 100644
--- a/include/linux/cc_platform.h
+++ b/include/linux/cc_platform.h
@@ -61,6 +61,15 @@ enum cc_attr {
* Examples include SEV-ES.
*/
CC_ATTR_GUEST_STATE_ENCRYPT,
+
+ /**
+* @CC_ATTR_GUEST_TDX: Trusted Domain Extension Support
+*
+* The platform/OS is running as a TDX guest/virtual machine.
+*
+* Examples include SEV-ES.
+*/
+ CC_ATTR_GUEST_TDX,
};
#ifdef CONFIG_ARCH_HAS_CC_PLATFORM
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v4 0/8] Implement generic cc_platform_has() helper function
On 9/28/21 12:10 PM, Borislav Petkov wrote: From: Borislav Petkov Hi all, here's v4 of the cc_platform_has() patchset with feedback incorporated. I'm going to route this through tip if there are no objections. Intel CC support patch is not included in this series. You want me to address the issue raised by Joerg before merging it? Thx. Tom Lendacky (8): x86/ioremap: Selectively build arch override encryption functions arch/cc: Introduce a function to check for confidential computing features x86/sev: Add an x86 version of cc_platform_has() powerpc/pseries/svm: Add a powerpc version of cc_platform_has() x86/sme: Replace occurrences of sme_active() with cc_platform_has() x86/sev: Replace occurrences of sev_active() with cc_platform_has() x86/sev: Replace occurrences of sev_es_active() with cc_platform_has() treewide: Replace the use of mem_encrypt_active() with cc_platform_has() arch/Kconfig | 3 + arch/powerpc/include/asm/mem_encrypt.h | 5 -- arch/powerpc/platforms/pseries/Kconfig | 1 + arch/powerpc/platforms/pseries/Makefile | 2 + arch/powerpc/platforms/pseries/cc_platform.c | 26 ++ arch/powerpc/platforms/pseries/svm.c | 5 +- arch/s390/include/asm/mem_encrypt.h | 2 - arch/x86/Kconfig | 1 + arch/x86/include/asm/io.h| 8 ++ arch/x86/include/asm/kexec.h | 2 +- arch/x86/include/asm/mem_encrypt.h | 12 +-- arch/x86/kernel/Makefile | 6 ++ arch/x86/kernel/cc_platform.c| 69 +++ arch/x86/kernel/crash_dump_64.c | 4 +- arch/x86/kernel/head64.c | 9 +- arch/x86/kernel/kvm.c| 3 +- arch/x86/kernel/kvmclock.c | 4 +- arch/x86/kernel/machine_kexec_64.c | 19 +++-- arch/x86/kernel/pci-swiotlb.c| 9 +- arch/x86/kernel/relocate_kernel_64.S | 2 +- arch/x86/kernel/sev.c| 6 +- arch/x86/kvm/svm/svm.c | 3 +- arch/x86/mm/ioremap.c| 18 ++-- arch/x86/mm/mem_encrypt.c| 55 arch/x86/mm/mem_encrypt_identity.c | 9 +- arch/x86/mm/pat/set_memory.c | 3 +- arch/x86/platform/efi/efi_64.c | 9 +- arch/x86/realmode/init.c | 8 +- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 +- drivers/gpu/drm/drm_cache.c | 4 +- drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 4 +- drivers/gpu/drm/vmwgfx/vmwgfx_msg.c | 6 +- drivers/iommu/amd/init.c | 7 +- drivers/iommu/amd/iommu.c| 3 +- drivers/iommu/amd/iommu_v2.c | 3 +- drivers/iommu/iommu.c| 3 +- fs/proc/vmcore.c | 6 +- include/linux/cc_platform.h | 88 include/linux/mem_encrypt.h | 4 - kernel/dma/swiotlb.c | 4 +- 40 files changed, 310 insertions(+), 129 deletions(-) create mode 100644 arch/powerpc/platforms/pseries/cc_platform.c create mode 100644 arch/x86/kernel/cc_platform.c create mode 100644 include/linux/cc_platform.h -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v4 0/8] Implement generic cc_platform_has() helper function
On 9/28/21 1:31 PM, Borislav Petkov wrote: On Tue, Sep 28, 2021 at 12:19:49PM -0700, Kuppuswamy, Sathyanarayanan wrote: Intel CC support patch is not included in this series. You want me to address the issue raised by Joerg before merging it? Did you not see my email to you today: https://lkml.kernel.org/r/yvl4zughfsh1q...@zn.tnic Just read it. If you want to use cpuid_has_tdx_guest() directly in cc_platform_has(), then you want to rename intel_cc_platform_has() to tdx_cc_platform_has()? ? -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v4 0/8] Implement generic cc_platform_has() helper function
On 9/28/21 1:58 PM, Borislav Petkov wrote: On Tue, Sep 28, 2021 at 01:48:46PM -0700, Kuppuswamy, Sathyanarayanan wrote: Just read it. If you want to use cpuid_has_tdx_guest() directly in cc_platform_has(), then you want to rename intel_cc_platform_has() to tdx_cc_platform_has()? Why? You simply do: if (cpuid_has_tdx_guest()) intel_cc_platform_has(...); and lemme paste from that mail: " ...you should use cpuid_has_tdx_guest() instead but cache its result so that you don't call CPUID each time the kernel executes cc_platform_has()." Makes sense? Yes. But, since the check is related to TDX, I just want to confirm whether you are fine with naming the function as intel_*(). Since this patch is going to have dependency on TDX code, I will include this patch in TDX patch set. -- Sathyanarayanan Kuppuswamy Linux Kernel Developer ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv6 16/16] x86/acpi: Add support for CPU offlining for ACPI MADT wakeup method
On 1/24/24 4:55 AM, Kirill A. Shutemov wrote:
> MADT Multiprocessor Wakeup structure version 1 brings support of CPU
> offlining: BIOS provides a reset vector where the CPU has to jump to
> for offlining itself. The new TEST mailbox command can be used to test
> whether the CPU offlined itself which means the BIOS has control over
> the CPU and can online it again via the ACPI MADT wakeup method.
>
> Add CPU offling support for the ACPI MADT wakeup method by implementing
> custom cpu_die(), play_dead() and stop_this_cpu() SMP operations.
>
> CPU offlining makes is possible to hand over secondary CPUs over kexec,
> not limiting the second kernel to a single CPU.
>
> The change conforms to the approved ACPI spec change proposal. See the
> Link.
>
> Signed-off-by: Kirill A. Shutemov
> Link: https://lore.kernel.org/all/13356251.uLZWGnKmhe@kreacher
> ---
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/include/asm/acpi.h | 2 +
> arch/x86/kernel/acpi/Makefile| 2 +-
> arch/x86/kernel/acpi/madt_playdead.S | 28
> arch/x86/kernel/acpi/madt_wakeup.c | 184 ++-
> include/acpi/actbl2.h| 15 ++-
> 5 files changed, 227 insertions(+), 4 deletions(-)
> create mode 100644 arch/x86/kernel/acpi/madt_playdead.S
>
> diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
> index 2625b915ae7f..021cafa214c2 100644
> --- a/arch/x86/include/asm/acpi.h
> +++ b/arch/x86/include/asm/acpi.h
> @@ -81,6 +81,8 @@ union acpi_subtable_headers;
> int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,
> const unsigned long end);
>
> +void asm_acpi_mp_play_dead(u64 reset_vector, u64 pgd_pa);
> +
> /*
> * Check if the CPU can handle C2 and deeper
> */
> diff --git a/arch/x86/kernel/acpi/Makefile b/arch/x86/kernel/acpi/Makefile
> index 8c7329c88a75..37b1f28846de 100644
> --- a/arch/x86/kernel/acpi/Makefile
> +++ b/arch/x86/kernel/acpi/Makefile
> @@ -4,7 +4,7 @@ obj-$(CONFIG_ACPI)+= boot.o
> obj-$(CONFIG_ACPI_SLEEP) += sleep.o wakeup_$(BITS).o
> obj-$(CONFIG_ACPI_APEI) += apei.o
> obj-$(CONFIG_ACPI_CPPC_LIB) += cppc.o
> -obj-$(CONFIG_X86_ACPI_MADT_WAKEUP) += madt_wakeup.o
> +obj-$(CONFIG_X86_ACPI_MADT_WAKEUP) += madt_wakeup.o madt_playdead.o
>
> ifneq ($(CONFIG_ACPI_PROCESSOR),)
> obj-y+= cstate.o
> diff --git a/arch/x86/kernel/acpi/madt_playdead.S
> b/arch/x86/kernel/acpi/madt_playdead.S
> new file mode 100644
> index ..4e498d28cdc8
> --- /dev/null
> +++ b/arch/x86/kernel/acpi/madt_playdead.S
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include
> +#include
> +#include
> +#include
> +
> + .text
> + .align PAGE_SIZE
> +
> +/*
> + * asm_acpi_mp_play_dead() - Hand over control of the CPU to the BIOS
> + *
> + * rdi: Address of the ACPI MADT MPWK ResetVector
> + * rsi: PGD of the identity mapping
> + */
> +SYM_FUNC_START(asm_acpi_mp_play_dead)
> + /* Turn off global entries. Following CR3 write will flush them. */
> + movq%cr4, %rdx
> + andq$~(X86_CR4_PGE), %rdx
> + movq%rdx, %cr4
> +
> + /* Switch to identity mapping */
> + movq%rsi, %cr3
> +
> + /* Jump to reset vector */
> + ANNOTATE_RETPOLINE_SAFE
> + jmp *%rdi
> +SYM_FUNC_END(asm_acpi_mp_play_dead)
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c
> b/arch/x86/kernel/acpi/madt_wakeup.c
> index 30820f9de5af..9e984e2191ba 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -1,10 +1,19 @@
> // SPDX-License-Identifier: GPL-2.0-or-later
> #include
> #include
> +#include
> #include
> +#include
> +#include
> +#include
> +#include
> #include
> #include
> +#include
> +#include
> +#include
> #include
> +#include
>
> /* Physical address of the Multiprocessor Wakeup Structure mailbox */
> static u64 acpi_mp_wake_mailbox_paddr __ro_after_init;
> @@ -12,6 +21,154 @@ static u64 acpi_mp_wake_mailbox_paddr __ro_after_init;
> /* Virtual address of the Multiprocessor Wakeup Structure mailbox */
> static struct acpi_madt_multiproc_wakeup_mailbox *acpi_mp_wake_mailbox
> __ro_after_init;
>
> +static u64 acpi_mp_pgd __ro_after_init;
> +static u64 acpi_mp_reset_vector_paddr __ro_after_init;
> +
> +static void acpi_mp_stop_this_cpu(void)
> +{
> + asm_acpi_mp_play_dead(acpi_mp_reset_vector_paddr, acpi_mp_pgd);
> +}
> +
> +static void acpi_mp_play_dead(void)
> +{
> + play_dead_common();
> + asm_a
Re: [PATCHv8 17/17] ACPI: tables: Print MULTIPROC_WAKEUP when MADT is parsed
On 2/27/24 1:24 PM, Kirill A. Shutemov wrote:
> When MADT is parsed, print MULTIPROC_WAKEUP information:
>
> ACPI: MP Wakeup (version[1], mailbox[0x7fffd000], reset[0x7fffe068])
>
> This debug information will be very helpful during bring up.
>
> Signed-off-by: Kirill A. Shutemov
> Reviewed-by: Baoquan He
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> drivers/acpi/tables.c | 14 ++
> 1 file changed, 14 insertions(+)
>
> diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c
> index b07f7d091d13..c59a3617bca7 100644
> --- a/drivers/acpi/tables.c
> +++ b/drivers/acpi/tables.c
> @@ -198,6 +198,20 @@ void acpi_table_print_madt_entry(struct
> acpi_subtable_header *header)
> }
> break;
>
> + case ACPI_MADT_TYPE_MULTIPROC_WAKEUP:
> + {
> + struct acpi_madt_multiproc_wakeup *p =
> + (struct acpi_madt_multiproc_wakeup *)header;
> + u64 reset_vector = 0;
> +
> + if (p->version >= ACPI_MADT_MP_WAKEUP_VERSION_V1)
> + reset_vector = p->reset_vector;
> +
> + pr_debug("MP Wakeup (version[%d], mailbox[%#llx],
> reset[%#llx])\n",
> + p->version, p->mailbox_address, reset_vector);
> + }
> + break;
> +
> case ACPI_MADT_TYPE_CORE_PIC:
> {
> struct acpi_madt_core_pic *p = (struct
> acpi_madt_core_pic *)header;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv9 08/17] x86/tdx: Account shared memory
On 3/25/24 3:39 AM, Kirill A. Shutemov wrote:
> The kernel will convert all shared memory back to private during kexec.
> The direct mapping page tables will provide information on which memory
> is shared.
>
> It is extremely important to convert all shared memory. If a page is
> missed, it will cause the second kernel to crash when it accesses it.
>
> Keep track of the number of shared pages. This will allow for
> cross-checking against the shared information in the direct mapping and
> reporting if the shared bit is lost.
>
> Signed-off-by: Kirill A. Shutemov
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/coco/tdx/tdx.c | 7 +++
> 1 file changed, 7 insertions(+)
>
> diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c
> index 26fa47db5782..979891e97d83 100644
> --- a/arch/x86/coco/tdx/tdx.c
> +++ b/arch/x86/coco/tdx/tdx.c
> @@ -38,6 +38,8 @@
>
> #define TDREPORT_SUBTYPE_0 0
>
> +static atomic_long_t nr_shared;
> +
> /* Called from __tdx_hypercall() for unrecoverable failure */
> noinstr void __noreturn __tdx_hypercall_failed(void)
> {
> @@ -821,6 +823,11 @@ static int tdx_enc_status_change_finish(unsigned long
> vaddr, int numpages,
> if (!enc && !tdx_enc_status_changed(vaddr, numpages, enc))
> return -EIO;
>
> + if (enc)
> + atomic_long_sub(numpages, &nr_shared);
> + else
> + atomic_long_add(numpages, &nr_shared);
> +
> return 0;
> }
>
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCHv9 17/17] ACPI: tables: Print MULTIPROC_WAKEUP when MADT is parsed
On 3/25/24 3:39 AM, Kirill A. Shutemov wrote:
> When MADT is parsed, print MULTIPROC_WAKEUP information:
>
> ACPI: MP Wakeup (version[1], mailbox[0x7fffd000], reset[0x7fffe068])
>
> This debug information will be very helpful during bring up.
>
> Signed-off-by: Kirill A. Shutemov
> Reviewed-by: Baoquan He
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> drivers/acpi/tables.c | 14 ++
> 1 file changed, 14 insertions(+)
>
> diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c
> index b976e5fc3fbc..9e1b01c35070 100644
> --- a/drivers/acpi/tables.c
> +++ b/drivers/acpi/tables.c
> @@ -198,6 +198,20 @@ void acpi_table_print_madt_entry(struct
> acpi_subtable_header *header)
> }
> break;
>
> + case ACPI_MADT_TYPE_MULTIPROC_WAKEUP:
> + {
> + struct acpi_madt_multiproc_wakeup *p =
> + (struct acpi_madt_multiproc_wakeup *)header;
> + u64 reset_vector = 0;
> +
> + if (p->version >= ACPI_MADT_MP_WAKEUP_VERSION_V1)
> + reset_vector = p->reset_vector;
> +
> + pr_debug("MP Wakeup (version[%d], mailbox[%#llx],
> reset[%#llx])\n",
> + p->version, p->mailbox_address, reset_vector);
> + }
> + break;
> +
> case ACPI_MADT_TYPE_CORE_PIC:
> {
> struct acpi_madt_core_pic *p = (struct
> acpi_madt_core_pic *)header;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v3 1/4] efi/x86: skip efi_arch_mem_reserve() in case of kexec.
On 4/4/24 4:11 PM, Ashish Kalra wrote:
> From: Ashish Kalra
>
> For kexec use case, need to use and stick to the EFI memmap passed
> from the first kernel via boot-params/setup data, hence,
> skip efi_arch_mem_reserve() during kexec.
>
> Additionally during SNP guest kexec testing discovered that EFI memmap
> is corrupted during chained kexec. kexec_enter_virtual_mode() during
> late init will remap the efi_memmap physical pages allocated in
> efi_arch_mem_reserve() via memblock & then subsequently cause random
> EFI memmap corruption once memblock is freed/teared-down.
>
> Suggested-by: Dave Young
> [Dave Young: checking the md attribute instead of checking the efi_setup]
> Signed-off-by: Ashish Kalra
> ---
> arch/x86/platform/efi/quirks.c | 23 ---
> 1 file changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c
> index f0cc00032751..2b65b3863912 100644
> --- a/arch/x86/platform/efi/quirks.c
> +++ b/arch/x86/platform/efi/quirks.c
> @@ -255,15 +255,32 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64
> size)
> struct efi_memory_map_data data = { 0 };
> struct efi_mem_range mr;
> efi_memory_desc_t md;
> - int num_entries;
> + int num_entries, ret;
> void *new;
>
> - if (efi_mem_desc_lookup(addr, &md) ||
> - md.type != EFI_BOOT_SERVICES_DATA) {
> + /*
> + * For kexec use case, we need to use the EFI memmap passed from the
> first
> + * kernel via setup data, so we need to skip this.
> + * Additionally kexec_enter_virtual_mode() during late init will remap
> + * the efi_memmap physical pages allocated here via memboot & then
> + * subsequently cause random EFI memmap corruption once memblock is
> freed.
> + */
> +
> + ret = efi_mem_desc_lookup(addr, &md);
Since you are not using ret, why not directly use if (efi_mem_desc_lookup(..))?
> + if (ret) {
> pr_err("Failed to lookup EFI memory descriptor for %pa\n",
> &addr);
> return;
> }
>
> + if (md.type != EFI_BOOT_SERVICES_DATA) {
> + pr_err("Skip reserving non EFI Boot Service Data memory for
> %pa\n", &addr);
> + return;
> + }
> +
> + /* Kexec copied the efi memmap from the first kernel, thus skip the
> case */
> + if (md.attribute & EFI_MEMORY_RUNTIME)
> + return;
> +
> if (addr + size > md.phys_addr + (md.num_pages << EFI_PAGE_SHIFT)) {
> pr_err("Region spans EFI memory descriptors, %pa\n", &addr);
> return;
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v3 2/4] x86/sev: add sev_es_enabled() function.
On 4/4/24 4:11 PM, Ashish Kalra wrote:
> From: Ashish Kalra
>
> Add sev_es_enabled() function to detect if SEV-ES
> support is enabled.
>
> Signed-off-by: Ashish Kalra
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/boot/compressed/sev.c | 5 +
> arch/x86/boot/compressed/sev.h | 2 ++
> 2 files changed, 7 insertions(+)
>
> diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
> index ec71846d28c9..4ae4cc51e6b8 100644
> --- a/arch/x86/boot/compressed/sev.c
> +++ b/arch/x86/boot/compressed/sev.c
> @@ -134,6 +134,11 @@ bool sev_snp_enabled(void)
> return sev_status & MSR_AMD64_SEV_SNP_ENABLED;
> }
>
> +bool sev_es_enabled(void)
> +{
> + return sev_status & MSR_AMD64_SEV_ES_ENABLED;
> +}
> +
> static void __page_state_change(unsigned long paddr, enum psc_op op)
> {
> u64 val;
> diff --git a/arch/x86/boot/compressed/sev.h b/arch/x86/boot/compressed/sev.h
> index fc725a981b09..5008c80e66e6 100644
> --- a/arch/x86/boot/compressed/sev.h
> +++ b/arch/x86/boot/compressed/sev.h
> @@ -11,11 +11,13 @@
> #ifdef CONFIG_AMD_MEM_ENCRYPT
>
> bool sev_snp_enabled(void);
> +bool sev_es_enabled(void);
> void snp_accept_memory(phys_addr_t start, phys_addr_t end);
>
> #else
>
> static inline bool sev_snp_enabled(void) { return false; }
> +static inline bool sev_es_enabled(void) { return false; }
> static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { }
>
> #endif
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v3 3/4] x86/boot/compressed: Skip Video Memory access in Decompressor for SEV-ES/SNP.
On 4/4/24 4:11 PM, Ashish Kalra wrote:
> From: Ashish Kalra
>
> Accessing guest video memory/RAM during kernel decompressor
> causes guest termination as boot stage2 #VC handler for
> SEV-ES/SNP systems does not support MMIO handling.
>
> This issue is observed with SEV-ES/SNP guest kexec as
> kexec -c adds screen_info to the boot parameters
> passed to the kexec kernel, which causes console output to
> be dumped to both video and serial.
>
> As the decompressor output gets cleared really fast, it is
> preferable to get the console output only on serial, hence,
> skip accessing video RAM during decompressor stage to
> prevent guest termination.
>
> Serial console output during decompressor stage works as
> boot stage2 #VC handler already supports handling port I/O.
>
> Suggested-by: Thomas Lendacy
> Signed-off-by: Ashish Kalra
> ---
Looks good to me.
Reviewed-by: Kuppuswamy Sathyanarayanan
> arch/x86/boot/compressed/misc.c | 6 --
> arch/x86/boot/compressed/misc.h | 1 +
> 2 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
> index b70e4a21c15f..47b4db200e1f 100644
> --- a/arch/x86/boot/compressed/misc.c
> +++ b/arch/x86/boot/compressed/misc.c
> @@ -427,8 +427,10 @@ asmlinkage __visible void *extract_kernel(void *rmode,
> unsigned char *output)
> vidport = 0x3d4;
> }
>
> - lines = boot_params_ptr->screen_info.orig_video_lines;
> - cols = boot_params_ptr->screen_info.orig_video_cols;
> + if (!sev_es_enabled()) {
> + lines = boot_params_ptr->screen_info.orig_video_lines;
> + cols = boot_params_ptr->screen_info.orig_video_cols;
> + }
>
> init_default_io_ops();
>
> diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
> index b353a7be380c..3c12ca987554 100644
> --- a/arch/x86/boot/compressed/misc.h
> +++ b/arch/x86/boot/compressed/misc.h
> @@ -37,6 +37,7 @@
> #include
>
> #include "tdx.h"
> +#include "sev.h"
>
> #define BOOT_CTYPE_H
> #include
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
