Re: [PATCH v1 1/2] Implemented AMD SEV discovery and enabling.
Thank you Jan for your feedback.
From what I understood, Andrew is planning some "ground" changes
on CPUID leaf discovery, so it probably will move here. I'll wait for his
changes to apply your and Andrew's remarks and to re-post the patches.
Andrei.
On 4/18/24 10:13, Jan Beulich wrote:
On 10.04.2024 17:36, Andrei Semenov wrote:
Signed-off-by: Andrei Semenov
A couple more nits on top of what Andrew said. First: Please no patches
(which aren't blindingly trivial) without description.
@@ -1030,6 +1031,54 @@ static void amd_check_erratum_1485(void)
wrmsrl(MSR_AMD64_BP_CFG, val | chickenbit);
}
+#ifdef CONFIG_HVM
+static void amd_enable_mem_encrypt(const struct cpuinfo_x86 *c)
+{
+ unsigned int eax, ebx, ecx, edx;
+ uint64_t syscfg;
+
+ if (!smp_processor_id()) {
+
+ cpuid_count(0x8000,0,, , , );
No blank line above here please.
+ if (eax < 0x801f)
+ return;
+
+ cpuid_count(0x801f,0,, , , );
+
+ if (eax & 0x1)
+ setup_force_cpu_cap(X86_FEATURE_SME);
+
+ if (eax & 0x2) {
+ setup_force_cpu_cap(X86_FEATURE_SEV);
I guess this goes along with what Andrew said: Using synthetic features here
looks suspicious. These want to be recorded as an ordinary leaf.
+ max_sev_asid = ecx;
+ min_sev_asid = edx;
+ }
+
+ if (eax & 0x3)
+ pte_c_bit_mask = 1UL << (ebx & 0x3f);
+ }
+
+ if (!(cpu_has_sme || cpu_has_sev))
+ return;
+
+ if (!smp_processor_id()) {
+ if (cpu_has_sev)
Two if()-s like these want folding, unless it is made clear that very
soon (see above as to the missing description) further content is going
to appear inside the outer one.
+ printk(XENLOG_INFO "SEV: ASID range [0x%x - 0x%x]\n",
%#x is preferred over 0x%x.
+ min_sev_asid, max_sev_asid);
+ }
+
+ rdmsrl(MSR_K8_SYSCFG, syscfg);
+
+ if (syscfg & SYSCFG_MEM_ENCRYPT) {
+ return;
+ }
No need for braces in cases like this one.
Jan
Re: [PATCH v1 2/2] Implemented Amd Secure Processor device driver
On 4/11/24 20:42, Andrew Cooper wrote: On 10/04/2024 4:36 pm, Andrei Semenov wrote: Signed-off-by: Andrei Semenov --- xen/arch/x86/include/asm/psp-sev.h | 655 +++ xen/drivers/Kconfig| 2 + xen/drivers/Makefile | 1 + xen/drivers/crypto/Kconfig | 10 + xen/drivers/crypto/Makefile| 1 + xen/drivers/crypto/asp.c | 808 + xen/include/xen/types.h| 2 +- 7 files changed, 1478 insertions(+), 1 deletion(-) create mode 100644 xen/arch/x86/include/asm/psp-sev.h create mode 100644 xen/drivers/crypto/Kconfig create mode 100644 xen/drivers/crypto/Makefile create mode 100644 xen/drivers/crypto/asp.c I'm not going to dive into all of this, but give some high level feedback to start with. CCP is driver/crypto in Linux for historical reasons, but is it really right here? We can pick whatever we think is suitable. Yes, I only picked SEV interface (for instance) and I called the the directory "crypto", for some reasons as this is the name in Linux, so to "compliant" and SEV is related to crypto too. Later we potentially will need to export some of interfaces to guest so will need to extend the driver. As Marek remarked some AMD GPU may need this to load GPU firmware ... That's beeing said, I have not very strong opinion on "crypto" name. psp-sev.h looks like it's only the MMIO protocol to the ASP, and that it shouldn't need including anywhere else? If so, we're trying to move those header files to be local to the asp.c dir. SVM operations (at least) will need this. Very probably toolstack also will need a part of this, so potentially this part will move to "public" interface. For instance hard to say what parts (are moving), so all this in the same psp-sev.h file. Can you discuss this comment: CET shadow stack: adapt #CP handler??? some more. What's going on? Yep. Actually CET Shadow Stack raised #21 exception (near ret) on older versions of Xen (when I said older I talk about 4.19 unstable). This is no more the case on staging branch. So it was fixed somehow. Sorry didn't check - will fix. diff --git a/xen/include/xen/types.h b/xen/include/xen/types.h index 449947b353..f7599845fd 100644 --- a/xen/include/xen/types.h +++ b/xen/include/xen/types.h @@ -6,7 +6,7 @@ /* Linux inherited types which are being phased out */ typedef int8_t s8; -typedef uint8_t u8; +typedef uint8_t u8, __u8; typedef int16_t s16; typedef uint16_t u16, __u16; typedef int32_t s32; The comment is here for a reason, so reviewers don't accept hunks like this. psp-sev.h should be written using normal C99 integer types please. Got it. Will fix ~Andrew
Re: [PATCH v1 1/2] Implemented AMD SEV discovery and enabling.
On 4/11/24 20:32, Andrew Cooper wrote:
On 10/04/2024 4:36 pm, Andrei Semenov wrote:
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index ab92333673..a5903613f0 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -1030,6 +1031,54 @@ static void amd_check_erratum_1485(void)
wrmsrl(MSR_AMD64_BP_CFG, val | chickenbit);
}
+#ifdef CONFIG_HVM
+static void amd_enable_mem_encrypt(const struct cpuinfo_x86 *c)
+{
+ unsigned int eax, ebx, ecx, edx;
+ uint64_t syscfg;
+
+ if (!smp_processor_id()) {
c == _cpu_info.
Agree, will fix.
+
+ cpuid_count(0x8000,0,, , , );
+
+ if (eax < 0x801f)
+ return;
Max leaf is already collected. c->extended_cpuid_level
Agree, will fix.
+
+ cpuid_count(0x801f,0,, , , );
+
+ if (eax & 0x1)
+ setup_force_cpu_cap(X86_FEATURE_SME);
+
+ if (eax & 0x2) {
+ setup_force_cpu_cap(X86_FEATURE_SEV);
+ max_sev_asid = ecx;
+ min_sev_asid = edx;
+ }
+
+ if (eax & 0x3)
+ pte_c_bit_mask = 1UL << (ebx & 0x3f);
This is decoding the main SEV feature leaf, but outside of normal
mechanisms.
I've got half a mind to brute-force through the remaining work to
un-screw our boot sequence order, and express this in a cpu-policy
straight away. This is wanted for the SVM leaf info too.
Leave it with me for a bit.
OK. I wait for your insights on this so.
+ }
+
+ if (!(cpu_has_sme || cpu_has_sev))
+ return;
+
+ if (!smp_processor_id()) {
+ if (cpu_has_sev)
+ printk(XENLOG_INFO "SEV: ASID range [0x%x - 0x%x]\n",
+ min_sev_asid, max_sev_asid);
Why do we have a min as well as a max? Isn't min always 1?
Well, "normally it is". But this is the part of CPUID leaf specs. Do
they plan to potentially change it?
No idea.
+ }
+
+ rdmsrl(MSR_K8_SYSCFG, syscfg);
+
+ if (syscfg & SYSCFG_MEM_ENCRYPT) {
+ return;
+ }
+
+ syscfg |= SYSCFG_MEM_ENCRYPT;
+ wrmsrl(MSR_K8_SYSCFG, syscfg);
+}
+#endif
+
static void cf_check init_amd(struct cpuinfo_x86 *c)
{
u32 l, h;
@@ -1305,6 +1354,10 @@ static void cf_check init_amd(struct cpuinfo_x86 *c)
check_syscfg_dram_mod_en();
amd_log_freq(c);
+
+#ifdef CONFIG_HVM
+ amd_enable_mem_encrypt(c);
+#endif
I think we want to drop the CONFIG_HVM here.
Memory encryption is an all-or-nothing thing. If it's active, it
affects all pagetables that Xen controls, even dom0's. And we likely do
want get to the point of Xen running on encrypted mappings even if dom0
can't operate it very nicely.
Thoughts?
Basically I put CONFIG_HVM here because I also wanted to put related
variables
(max/min_asid) in sev.c. And sev.c is in "HVM" part of the code as SEV
is only related to HVM guests. Now, basically I agree that
- Xen would like potentially use encrypted memory for itself
- in SME case, some encryption could be offered for non-HVM guests, so they
can protect their memory (even though the key is shared and the
hypervisor can
read it).
OK, so I will drop CONFIG_HVM and put these variables elsewhere. amd.h
is probably
a good candidate?
}
const struct cpu_dev __initconst_cf_clobber amd_cpu_dev = {
diff --git a/xen/arch/x86/hvm/svm/Makefile b/xen/arch/x86/hvm/svm/Makefile
index 760d2954da..9773d539ef 100644
--- a/xen/arch/x86/hvm/svm/Makefile
+++ b/xen/arch/x86/hvm/svm/Makefile
@@ -6,3 +6,4 @@ obj-y += nestedsvm.o
obj-y += svm.o
obj-y += svmdebug.o
obj-y += vmcb.o
+obj-y += sev.o
Please keep this sorted by object file name.
Got it. Will do.
diff --git a/xen/arch/x86/hvm/svm/sev.c b/xen/arch/x86/hvm/svm/sev.c
new file mode 100644
index 00..336fad25f5
--- /dev/null
+++ b/xen/arch/x86/hvm/svm/sev.c
@@ -0,0 +1,4 @@
+#include
+uint64_t __read_mostly pte_c_bit_mask;
+unsigned int __read_mostly min_sev_asid;
+unsigned int __read_mostly max_sev_asid;
Several things. All new files should come with an SPDX tag. Unless you
have other constraints, GPL-2.0-only is preferred. There also wants to
be at least a oneline summary of what's going on here.
Will do.
All these variables look like they should be __ro_after_init. However,
it's rather hard to judge, given no users yet.
Yes, this is not supposed to dynamically change. Will fix.
pte_c_bit_mask may want to be an intpte_t rather than uint64_t.
Agree. Will fix
~Andrew
Andrei.
Re: [PATCH v1 0/2] Starting AMD SEV work
Yes, actually a PSP device and all its interfaces come as unique PCI device (one BDF). So, yes we will need to export other interfaces as a paravirtualized operations for dom0. Actual PSP design feet very well for KVM, but for "type 1" hypervisors it oblige to export all other potential PSP interfaces for priveleged guests. Best regards, Andrei Semenov Le jeudi 11/04/2024 02:50, Marek Marczykowski-Górecki a écrit : On Wed, Apr 10, 2024 at 05:36:34PM +0200, Andrei Semenov wrote: This patch series initiate work on AMD SEV technology implementation in Xen. SEV stands for "Secure Encrypted Virtualization" and allows the memory contents of a VM to be encrypted with a key unique to this VM. In this way the neither other VMs nor hypervisor can't read the memory content of this "encrypted" VM. In order to create and to run such a VM different layers of software must interact (bascally Xen hypevisor, Xen toolstack in dom0 and the encrypted VM itself). In this work we start with discovering and enabling SEV feature on the platform. The second patch ports AMD Secure Processor driver on Xen. This AMD Secure Processor device (a.k.a PSP) is the way the different software layers interact with AMD firmware/hardware to manage and run the encrypted VM. How will that interact with the PSP driver in dom0? AFAIK amdgpu driver uses PSP for loading the GPU firmware. Does it mean one need to choose either GPU in dom0 or encrypted VMs, or is it going to work somehow together? -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
Re: [PATCH v1 0/2] AMD SEV initial work
Also sent emacs ~ file. To ignore, sorry Le mercredi 10/04/2024 17:39, Andrei Semenov a écrit : ///HERE YOU GO Andrei Semenov (2): Implemented AMD SEV discovery and enabling. Implemented Amd Secure Processor device driver xen/arch/x86/cpu/amd.c | 53 ++ xen/arch/x86/hvm/svm/Makefile | 1 + xen/arch/x86/hvm/svm/sev.c | 4 + xen/arch/x86/include/asm/cpufeature.h | 3 + xen/arch/x86/include/asm/cpufeatures.h | 2 + xen/arch/x86/include/asm/msr-index.h | 1 + xen/arch/x86/include/asm/psp-sev.h | 655 xen/arch/x86/include/asm/sev.h | 11 + xen/drivers/Kconfig| 2 + xen/drivers/Makefile | 1 + xen/drivers/crypto/Kconfig | 10 + xen/drivers/crypto/Makefile| 1 + xen/drivers/crypto/asp.c | 808 + xen/include/xen/types.h| 2 +- 14 files changed, 1553 insertions(+), 1 deletion(-) create mode 100644 xen/arch/x86/hvm/svm/sev.c create mode 100644 xen/arch/x86/include/asm/psp-sev.h create mode 100644 xen/arch/x86/include/asm/sev.h create mode 100644 xen/drivers/crypto/Kconfig create mode 100644 xen/drivers/crypto/Makefile create mode 100644 xen/drivers/crypto/asp.c -- 2.35.3
[PATCH v1 1/2] Implemented AMD SEV discovery and enabling.
Signed-off-by: Andrei Semenov
---
xen/arch/x86/cpu/amd.c | 53 ++
xen/arch/x86/hvm/svm/Makefile | 1 +
xen/arch/x86/hvm/svm/sev.c | 4 ++
xen/arch/x86/include/asm/cpufeature.h | 3 ++
xen/arch/x86/include/asm/cpufeatures.h | 2 +
xen/arch/x86/include/asm/msr-index.h | 1 +
xen/arch/x86/include/asm/sev.h | 11 ++
7 files changed, 75 insertions(+)
create mode 100644 xen/arch/x86/hvm/svm/sev.c
create mode 100644 xen/arch/x86/include/asm/sev.h
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index ab92333673..a5903613f0 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -16,6 +16,7 @@
#include
#include
#include
+#include
#include "cpu.h"
@@ -1030,6 +1031,54 @@ static void amd_check_erratum_1485(void)
wrmsrl(MSR_AMD64_BP_CFG, val | chickenbit);
}
+#ifdef CONFIG_HVM
+static void amd_enable_mem_encrypt(const struct cpuinfo_x86 *c)
+{
+ unsigned int eax, ebx, ecx, edx;
+ uint64_t syscfg;
+
+ if (!smp_processor_id()) {
+
+ cpuid_count(0x8000,0,, , , );
+
+ if (eax < 0x801f)
+ return;
+
+ cpuid_count(0x801f,0,, , , );
+
+ if (eax & 0x1)
+ setup_force_cpu_cap(X86_FEATURE_SME);
+
+ if (eax & 0x2) {
+ setup_force_cpu_cap(X86_FEATURE_SEV);
+ max_sev_asid = ecx;
+ min_sev_asid = edx;
+ }
+
+ if (eax & 0x3)
+ pte_c_bit_mask = 1UL << (ebx & 0x3f);
+ }
+
+ if (!(cpu_has_sme || cpu_has_sev))
+ return;
+
+ if (!smp_processor_id()) {
+ if (cpu_has_sev)
+ printk(XENLOG_INFO "SEV: ASID range [0x%x - 0x%x]\n",
+ min_sev_asid, max_sev_asid);
+ }
+
+ rdmsrl(MSR_K8_SYSCFG, syscfg);
+
+ if (syscfg & SYSCFG_MEM_ENCRYPT) {
+ return;
+ }
+
+ syscfg |= SYSCFG_MEM_ENCRYPT;
+ wrmsrl(MSR_K8_SYSCFG, syscfg);
+}
+#endif
+
static void cf_check init_amd(struct cpuinfo_x86 *c)
{
u32 l, h;
@@ -1305,6 +1354,10 @@ static void cf_check init_amd(struct cpuinfo_x86 *c)
check_syscfg_dram_mod_en();
amd_log_freq(c);
+
+#ifdef CONFIG_HVM
+ amd_enable_mem_encrypt(c);
+#endif
}
const struct cpu_dev __initconst_cf_clobber amd_cpu_dev = {
diff --git a/xen/arch/x86/hvm/svm/Makefile b/xen/arch/x86/hvm/svm/Makefile
index 760d2954da..9773d539ef 100644
--- a/xen/arch/x86/hvm/svm/Makefile
+++ b/xen/arch/x86/hvm/svm/Makefile
@@ -6,3 +6,4 @@ obj-y += nestedsvm.o
obj-y += svm.o
obj-y += svmdebug.o
obj-y += vmcb.o
+obj-y += sev.o
diff --git a/xen/arch/x86/hvm/svm/sev.c b/xen/arch/x86/hvm/svm/sev.c
new file mode 100644
index 00..336fad25f5
--- /dev/null
+++ b/xen/arch/x86/hvm/svm/sev.c
@@ -0,0 +1,4 @@
+#include
+uint64_t __read_mostly pte_c_bit_mask;
+unsigned int __read_mostly min_sev_asid;
+unsigned int __read_mostly max_sev_asid;
diff --git a/xen/arch/x86/include/asm/cpufeature.h
b/xen/arch/x86/include/asm/cpufeature.h
index 743f11f989..a41374d0b7 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -231,6 +231,9 @@ static inline bool boot_cpu_has(unsigned int feat)
#define cpu_has_msr_tsc_aux (cpu_has_rdtscp || cpu_has_rdpid)
+#define cpu_has_sme boot_cpu_has(X86_FEATURE_SME)
+#define cpu_has_sev boot_cpu_has(X86_FEATURE_SEV)
+
/* Bugs. */
#define cpu_bug_fpu_ptrsboot_cpu_has(X86_BUG_FPU_PTRS)
#define cpu_bug_null_segboot_cpu_has(X86_BUG_NULL_SEG)
diff --git a/xen/arch/x86/include/asm/cpufeatures.h
b/xen/arch/x86/include/asm/cpufeatures.h
index ba3df174b7..9b67ea2427 100644
--- a/xen/arch/x86/include/asm/cpufeatures.h
+++ b/xen/arch/x86/include/asm/cpufeatures.h
@@ -42,6 +42,8 @@ XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses
CET Shadow Stacks *
XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect
Branch Tracking */
XEN_CPUFEATURE(IBPB_ENTRY_PV, X86_SYNTH(28)) /* MSR_PRED_CMD used by Xen
for PV */
XEN_CPUFEATURE(IBPB_ENTRY_HVM,X86_SYNTH(29)) /* MSR_PRED_CMD used by Xen
for HVM */
+XEN_CPUFEATURE(SME, X86_SYNTH(30)) /* AMD Secure Memory
Encrypion */
+XEN_CPUFEATURE(SEV, X86_SYNTH(31)) /* AMD Secure Encryption
Virtualization */
/* Bug words follow the synthetic words. */
#define X86_NR_BUG 1
diff --git a/xen/arch/x86/include/asm/msr-index.h
b/xen/arch/x86/include/asm/msr-index.h
index 92dd9fa496..318e8ca0c0 100644
--- a/xen/arch/x86/include/asm/msr-index.h
+++ b/xen/arch/x86/include/asm/msr-index.h
@@ -221,6 +221,7 @@
#define SYSCFG_MTRR_VAR_DRAM_EN(_AC(1, ULL) << 20)
#define SYSCFG_MTRR_TOM2_EN(_AC(1, ULL) << 21)
#
[PATCH v1 2/2] Implemented Amd Secure Processor device driver
Signed-off-by: Andrei Semenov
---
xen/arch/x86/include/asm/psp-sev.h | 655 +++
xen/drivers/Kconfig| 2 +
xen/drivers/Makefile | 1 +
xen/drivers/crypto/Kconfig | 10 +
xen/drivers/crypto/Makefile| 1 +
xen/drivers/crypto/asp.c | 808 +
xen/include/xen/types.h| 2 +-
7 files changed, 1478 insertions(+), 1 deletion(-)
create mode 100644 xen/arch/x86/include/asm/psp-sev.h
create mode 100644 xen/drivers/crypto/Kconfig
create mode 100644 xen/drivers/crypto/Makefile
create mode 100644 xen/drivers/crypto/asp.c
diff --git a/xen/arch/x86/include/asm/psp-sev.h
b/xen/arch/x86/include/asm/psp-sev.h
new file mode 100644
index 00..3413de41a9
--- /dev/null
+++ b/xen/arch/x86/include/asm/psp-sev.h
@@ -0,0 +1,655 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * AMD Secure Encrypted Virtualization (SEV) driver interface
+ *
+ * Copyright (C) 2016-2017 Advanced Micro Devices, Inc.
+ *
+ * Author: Brijesh Singh
+ *
+ * SEV API spec is available at https://developer.amd.com/sev
+ */
+
+#ifndef __PSP_SEV_H__
+#define __PSP_SEV_H__
+
+#include
+
+/**
+ * SEV platform and guest management commands
+ */
+enum sev_cmd {
+ /* platform commands */
+ SEV_CMD_INIT= 0x001,
+ SEV_CMD_SHUTDOWN= 0x002,
+ SEV_CMD_FACTORY_RESET = 0x003,
+ SEV_CMD_PLATFORM_STATUS = 0x004,
+ SEV_CMD_PEK_GEN = 0x005,
+ SEV_CMD_PEK_CSR = 0x006,
+ SEV_CMD_PEK_CERT_IMPORT = 0x007,
+ SEV_CMD_PDH_CERT_EXPORT = 0x008,
+ SEV_CMD_PDH_GEN = 0x009,
+ SEV_CMD_DF_FLUSH= 0x00A,
+ SEV_CMD_DOWNLOAD_FIRMWARE = 0x00B,
+ SEV_CMD_GET_ID = 0x00C,
+ SEV_CMD_INIT_EX = 0x00D,
+
+ /* Guest commands */
+ SEV_CMD_DECOMMISSION= 0x020,
+ SEV_CMD_ACTIVATE= 0x021,
+ SEV_CMD_DEACTIVATE = 0x022,
+ SEV_CMD_GUEST_STATUS= 0x023,
+
+ /* Guest launch commands */
+ SEV_CMD_LAUNCH_START= 0x030,
+ SEV_CMD_LAUNCH_UPDATE_DATA = 0x031,
+ SEV_CMD_LAUNCH_UPDATE_VMSA = 0x032,
+ SEV_CMD_LAUNCH_MEASURE = 0x033,
+ SEV_CMD_LAUNCH_UPDATE_SECRET= 0x034,
+ SEV_CMD_LAUNCH_FINISH = 0x035,
+ SEV_CMD_ATTESTATION_REPORT = 0x036,
+
+ /* Guest migration commands (outgoing) */
+ SEV_CMD_SEND_START = 0x040,
+ SEV_CMD_SEND_UPDATE_DATA= 0x041,
+ SEV_CMD_SEND_UPDATE_VMSA= 0x042,
+ SEV_CMD_SEND_FINISH = 0x043,
+ SEV_CMD_SEND_CANCEL = 0x044,
+
+ /* Guest migration commands (incoming) */
+ SEV_CMD_RECEIVE_START = 0x050,
+ SEV_CMD_RECEIVE_UPDATE_DATA = 0x051,
+ SEV_CMD_RECEIVE_UPDATE_VMSA = 0x052,
+ SEV_CMD_RECEIVE_FINISH = 0x053,
+
+ /* Guest debug commands */
+ SEV_CMD_DBG_DECRYPT = 0x060,
+ SEV_CMD_DBG_ENCRYPT = 0x061,
+
+ SEV_CMD_MAX,
+};
+
+/**
+ * struct sev_data_init - INIT command parameters
+ *
+ * @flags: processing flags
+ * @tmr_address: system physical address used for SEV-ES
+ * @tmr_len: len of tmr_address
+ */
+struct sev_data_init {
+ u32 flags; /* In */
+ u32 reserved; /* In */
+ u64 tmr_address;/* In */
+ u32 tmr_len;/* In */
+} __packed;
+
+/**
+ * struct sev_data_init_ex - INIT_EX command parameters
+ *
+ * @length: len of the command buffer read by the PSP
+ * @flags: processing flags
+ * @tmr_address: system physical address used for SEV-ES
+ * @tmr_len: len of tmr_address
+ * @nv_address: system physical address used for PSP NV storage
+ * @nv_len: len of nv_address
+ */
+struct sev_data_init_ex {
+ u32 length; /* In */
+ u32 flags; /* In */
+ u64 tmr_address;/* In */
+ u32 tmr_len;/* In */
+ u32 reserved; /* In */
+ u64 nv_address; /* In/Out */
+ u32 nv_len; /* In */
+} __packed;
+
+#define SEV_INIT_FLAGS_SEV_ES 0x01
+
+/**
+ * struct sev_data_pek_csr - PEK_CSR command parameters
+ *
+ * @address: PEK certificate chain
+ * @len: len of certificate
+ */
+struct sev_data_pek_csr {
+ u64 address;/* In */
+ u32 len;/* In/Out */
+} __packed;
+
+/**
+ * struct sev_data_cert_import - PEK_CERT_IMPORT command parameters
+ *
+ * @pek_address: PEK certificate chain
+ * @pek_len: len of PEK certificate
+ * @oca_address: OCA certificate chain
+ * @oca_len: len of OCA certificate
+ */
+struct
[PATCH v1 0/2] AMD SEV initial work
///HERE YOU GO Andrei Semenov (2): Implemented AMD SEV discovery and enabling. Implemented Amd Secure Processor device driver xen/arch/x86/cpu/amd.c | 53 ++ xen/arch/x86/hvm/svm/Makefile | 1 + xen/arch/x86/hvm/svm/sev.c | 4 + xen/arch/x86/include/asm/cpufeature.h | 3 + xen/arch/x86/include/asm/cpufeatures.h | 2 + xen/arch/x86/include/asm/msr-index.h | 1 + xen/arch/x86/include/asm/psp-sev.h | 655 xen/arch/x86/include/asm/sev.h | 11 + xen/drivers/Kconfig| 2 + xen/drivers/Makefile | 1 + xen/drivers/crypto/Kconfig | 10 + xen/drivers/crypto/Makefile| 1 + xen/drivers/crypto/asp.c | 808 + xen/include/xen/types.h| 2 +- 14 files changed, 1553 insertions(+), 1 deletion(-) create mode 100644 xen/arch/x86/hvm/svm/sev.c create mode 100644 xen/arch/x86/include/asm/psp-sev.h create mode 100644 xen/arch/x86/include/asm/sev.h create mode 100644 xen/drivers/crypto/Kconfig create mode 100644 xen/drivers/crypto/Makefile create mode 100644 xen/drivers/crypto/asp.c -- 2.35.3
[PATCH v2 1/2] live migration: do not use deffered bitmap when inappropriate
Use deffered bitmap only in PV guests context as it not used for HVM guests.
This allow to reduce memory pressure on domain0 while migrating very large
(memory wise) HVM guests.
Signed-off-by: Andrei Semenov
---
tools/libs/guest/xg_sr_common.h | 26 --
tools/libs/guest/xg_sr_save.c | 23 +++-
tools/libs/guest/xg_sr_save_x86_hvm.c | 21 +++
tools/libs/guest/xg_sr_save_x86_pv.c | 39 +++
4 files changed, 93 insertions(+), 16 deletions(-)
diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h
index 36d45ef56f..941e24d7b7 100644
--- a/tools/libs/guest/xg_sr_common.h
+++ b/tools/libs/guest/xg_sr_common.h
@@ -96,6 +96,24 @@ struct xc_sr_save_ops
*/
int (*check_vm_state)(struct xc_sr_context *ctx);
+/**
+ * For some reasons the page can't be sent for the moment. Postpone this
+ * send to the later stage when domain is suspended.
+ */
+int (*defer_page)(struct xc_sr_context *ctx, xen_pfn_t pfn);
+
+/**
+ * Merge all deferred pages with the dirty pages bitmap (in order to be
+ * sent).
+ */
+int (*merge_deferred)(const struct xc_sr_context *ctx,
+ unsigned long *bitmap, unsigned long *count);
+
+/**
+ * Deferred pages was successfully sent. Reset all associated information.
+ */
+int (*reset_deferred)(struct xc_sr_context *ctx);
+
/**
* Clean up the local environment. Will be called exactly once, either
* after a successful save, or upon encountering an error.
@@ -243,8 +261,6 @@ struct xc_sr_context
xen_pfn_t *batch_pfns;
unsigned int nr_batch_pfns;
-unsigned long *deferred_pages;
-unsigned long nr_deferred_pages;
xc_hypercall_buffer_t dirty_bitmap_hbuf;
} save;
@@ -349,6 +365,12 @@ struct xc_sr_context
union
{
+struct
+{
+unsigned long *deferred_pages;
+unsigned long nr_deferred_pages;
+} save;
+
struct
{
/* State machine for the order of received records. */
diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c
index 9853d8d846..602b18488d 100644
--- a/tools/libs/guest/xg_sr_save.c
+++ b/tools/libs/guest/xg_sr_save.c
@@ -132,8 +132,7 @@ static int write_batch(struct xc_sr_context *ctx)
/* Likely a ballooned page. */
if ( mfns[i] == INVALID_MFN )
{
-set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages);
-++ctx->save.nr_deferred_pages;
+ctx->save.ops.defer_page(ctx, ctx->save.batch_pfns[i]);
}
}
@@ -192,8 +191,7 @@ static int write_batch(struct xc_sr_context *ctx)
{
if ( rc == -1 && errno == EAGAIN )
{
-set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages);
-++ctx->save.nr_deferred_pages;
+ctx->save.ops.defer_page(ctx, ctx->save.batch_pfns[i]);
types[i] = XEN_DOMCTL_PFINFO_XTAB;
--nr_pages;
}
@@ -641,6 +639,7 @@ static int suspend_and_send_dirty(struct xc_sr_context *ctx)
xc_interface *xch = ctx->xch;
xc_shadow_op_stats_t stats = { 0, ctx->save.p2m_size };
char *progress_str = NULL;
+unsigned long merged;
int rc;
DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap,
>save.dirty_bitmap_hbuf);
@@ -669,7 +668,7 @@ static int suspend_and_send_dirty(struct xc_sr_context *ctx)
else
xc_set_progress_prefix(xch, "Checkpointed save");
-bitmap_or(dirty_bitmap, ctx->save.deferred_pages, ctx->save.p2m_size);
+ctx->save.ops.merge_deferred(ctx, dirty_bitmap, );
if ( !ctx->save.live && ctx->stream_type == XC_STREAM_COLO )
{
@@ -681,12 +680,11 @@ static int suspend_and_send_dirty(struct xc_sr_context
*ctx)
}
}
-rc = send_dirty_pages(ctx, stats.dirty_count +
ctx->save.nr_deferred_pages);
+rc = send_dirty_pages(ctx, stats.dirty_count + merged);
if ( rc )
goto out;
-bitmap_clear(ctx->save.deferred_pages, ctx->save.p2m_size);
-ctx->save.nr_deferred_pages = 0;
+ctx->save.ops.reset_deferred(ctx);
out:
xc_set_progress_prefix(xch, NULL);
@@ -805,18 +803,16 @@ static int setup(struct xc_sr_context *ctx)
xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size)));
ctx->save.batch_pfns = malloc(MAX_BATCH_SIZE *
sizeof(*ctx->save.batch_pfns));
-ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size);
-if ( !ctx->
[PATCH v2 2/2] live migration: use superpages for physmap population on restore when possible
Implement an heuristic for X86 HVM guests which tries to use superpages while
populating guest physmap on live migration. This should impove memory accesses
performances for these guests.
Signed-off-by: Andrei Semenov
---
tools/include/xen-tools/libs.h | 4 ++
tools/libs/guest/xg_private.h| 3 +
tools/libs/guest/xg_sr_common.h | 18 -
tools/libs/guest/xg_sr_restore.c | 60 +++-
tools/libs/guest/xg_sr_restore_x86_hvm.c | 88 +++-
tools/libs/guest/xg_sr_restore_x86_pv.c | 22 +-
6 files changed, 154 insertions(+), 41 deletions(-)
diff --git a/tools/include/xen-tools/libs.h b/tools/include/xen-tools/libs.h
index a16e0c3807..bdd903eb7b 100644
--- a/tools/include/xen-tools/libs.h
+++ b/tools/include/xen-tools/libs.h
@@ -63,4 +63,8 @@
#define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
#endif
+#ifndef ROUNDDOWN
+#define ROUNDDOWN(_x,_w) ((unsigned long)(_x) & (-1UL << (_w)))
+#endif
+
#endif /* __XEN_TOOLS_LIBS__ */
diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
index 09e24f1227..dcf63b5188 100644
--- a/tools/libs/guest/xg_private.h
+++ b/tools/libs/guest/xg_private.h
@@ -134,6 +134,9 @@ typedef uint64_t x86_pgentry_t;
#define PAGE_SIZE_X86 (1UL << PAGE_SHIFT_X86)
#define PAGE_MASK_X86 (~(PAGE_SIZE_X86-1))
+#define S_PAGE_1GB_ORDER18
+#define S_PAGE_2MB_ORDER 9
+
#define NRPAGES(x) (ROUNDUP(x, PAGE_SHIFT) >> PAGE_SHIFT)
static inline xen_pfn_t xc_pfn_to_mfn(xen_pfn_t pfn, xen_pfn_t *p2m,
diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h
index 941e24d7b7..96365e05a8 100644
--- a/tools/libs/guest/xg_sr_common.h
+++ b/tools/libs/guest/xg_sr_common.h
@@ -137,7 +137,8 @@ struct xc_sr_restore_ops
bool (*pfn_is_valid)(const struct xc_sr_context *ctx, xen_pfn_t pfn);
/* Set the GFN of a PFN. */
-void (*set_gfn)(struct xc_sr_context *ctx, xen_pfn_t pfn, xen_pfn_t gfn);
+void (*set_gfn)(struct xc_sr_context *ctx, xen_pfn_t pfn, xen_pfn_t gfn,
+unsigned int order);
/* Set the type of a PFN. */
void (*set_page_type)(struct xc_sr_context *ctx, xen_pfn_t pfn,
@@ -175,6 +176,17 @@ struct xc_sr_restore_ops
#define BROKEN_CHANNEL 2
int (*process_record)(struct xc_sr_context *ctx, struct xc_sr_record *rec);
+/**
+ * Guest physmap population order is based on heuristic which is family
+ * dependant. X86 HVM heuristic is interested in observing the whole
+ * record (the first) in order to guess how the physmap should be
populated.
+ */
+void (*guess_physmap)(struct xc_sr_context *ctx, unsigned int count,
+ const xen_pfn_t *pfns, const uint32_t *types);
+
+/* Get the physmap population order for given PFN */
+int (*get_physmap_order)(const struct xc_sr_context *ctx, xen_pfn_t pfn);
+
/**
* Perform any actions required after the static data has arrived. Called
* when the STATIC_DATA_COMPLETE record has been recieved/inferred.
@@ -404,6 +416,10 @@ struct xc_sr_context
{
/* HVM context blob. */
struct xc_sr_blob context;
+
+/* Set guest type (based on the first record) */
+bool set_guest_type;
+bool pvh_guest;
} restore;
};
} hvm;
diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c
index 074b56d263..af864bd5ea 100644
--- a/tools/libs/guest/xg_sr_restore.c
+++ b/tools/libs/guest/xg_sr_restore.c
@@ -86,18 +86,21 @@ static bool pfn_is_populated(const struct xc_sr_context
*ctx, xen_pfn_t pfn)
* avoid realloc()ing too excessively, the size increased to the nearest power
* of two large enough to contain the required pfn.
*/
-static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn)
+static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn,
+ unsigned int order)
{
xc_interface *xch = ctx->xch;
+xen_pfn_t start_pfn = ROUNDDOWN(pfn, order),
+end_pfn = (ROUNDUP(pfn + 1, order) - 1);
-if ( pfn > ctx->restore.max_populated_pfn )
+if ( end_pfn > ctx->restore.max_populated_pfn )
{
xen_pfn_t new_max;
size_t old_sz, new_sz;
unsigned long *p;
/* Round up to the nearest power of two larger than pfn, less 1. */
-new_max = pfn;
+new_max = end_pfn;
new_max |= new_max >> 1;
new_max |= new_max >> 2;
new_max |= new_max >> 4;
@@ -123,8 +126,11 @@ static int pfn_set_populated(struct xc_sr_context *ctx,
xen_pfn_t pfn)
ctx->restore.max_populated_pfn = new_max;
}
-assert(!test_bit(pfn, ctx->restore.populated_pfns));
-
[PATCH v2 0/2] live migration: optimisations
This 2 patches implement some optimisations for guests live migraiton. Andrei Semenov (2): live migration: do not use deffered bitmap when inappropriate live migration: use superpages for physmap population on restore when possible tools/include/xen-tools/libs.h | 4 ++ tools/libs/guest/xg_private.h| 3 + tools/libs/guest/xg_sr_common.h | 44 +++- tools/libs/guest/xg_sr_restore.c | 60 +++- tools/libs/guest/xg_sr_restore_x86_hvm.c | 88 +++- tools/libs/guest/xg_sr_restore_x86_pv.c | 22 +- tools/libs/guest/xg_sr_save.c| 23 +++ tools/libs/guest/xg_sr_save_x86_hvm.c| 21 ++ tools/libs/guest/xg_sr_save_x86_pv.c | 39 +++ 9 files changed, 247 insertions(+), 57 deletions(-) -- 2.34.1 Andrei Semenov | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions w: vates.fr | xcp-ng.org | xen-orchestra.com
[Xen-devel] out of memory issue on dom0_hvm start
Hi all,
I have some troubles to start xen when using dom0 in HVM mode on X86 Intel
nuc7i5bnh
board. After some investigation it seems that "domheap" memory allocator is
out of memory while
constructing the p2m mapping (pvh_setup_p2m).
This misbehavior, from my understanding, is the result of wrong computation of
number of pages
to "give" to dom0 (dom0_compute_nr_pages function). In fact the pages needed to
paging
(dom0_paging_pages function) are not reserved in this function if IOMMU
mappings are shareable
with HAP(EPT) mappings. Moreover this memory (for paging needs) is allocated
just after that in
"pvh_setup_p2m" in subroutine "paging_set_allocation".
Generally speaking from what I understood it's not the IOMMU driver that offers
its mappings to HAP,
but it's rather the HAP that shares its mappings with IOMMU driver. So the
obvious patch IMHO would
be:
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -294,8 +294,7 @@ unsigned long __init dom0_compute_nr_pages(
avail -= max_pdx >> s;
}
- need_paging = is_hvm_domain(d) &&
- (!iommu_hap_pt_share || !paging_mode_hap(d));
+ need_paging = is_hvm_domain(d);
for ( ; ; need_paging = false )
{
nr_pages = dom0_nrpages;
Did anyone encounter the same problem or has some thoughts on this issue?
Andrei.
1
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
