Re: [PATCH v1 08/40] i386/tdx: Adjust the supported CPUID based on TDX restrictions
On 8/3/2022 3:33 PM, Chenyi Qiang wrote: On 8/2/2022 3:47 PM, Xiaoyao Li wrote: According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types: 1 | As configured | configurable by VMM, independent of native value; 2 | As configured | configurable by VMM if the bit is supported natively (if native) | Otherwise it equals as native(0). 3 | Fixed | fixed to 0/1 4 | Native | reflect the native value 5 | Calculated | calculated by TDX module. 6 | Inducing #VE | get #VE exception Note: 1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3. 2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec. When they are queried, TDX module injects #VE to TDs. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM. Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported CPUID bits for TDS. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions. Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord. Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields: - tdx_fixed0/1: The bits that are fixed as 0/1; - vmm_fixup: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when they are configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them. - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM. - supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs. By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps: - get the base of VMM supported feature set; - if the leaf is not a FeatureWord just return VMM's value without modification; - if the leaf is an inducing_ve type, applying supported_on_ve mask and return; - include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.) - apply fixed0/1 to it (it covers #3, and rectifies the previous step); - add configurable bits (it covers the other part of type #1); - fix the ones in vmm_fixup; - filter the one has valid .supported field; What does .supported field filter mean here? Sorry I missed this comment before. Above statement is the leftover during internal development. It needs to be removed actually.
Re: [PATCH v1 08/40] i386/tdx: Adjust the supported CPUID based on TDX restrictions
On 8/25/2022 7:26 PM, Gerd Hoffmann wrote: Hi, between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions. Automatic adjustment depending on hardware capabilities isn't going to fly long-term, you'll run into compatibility problems sooner or later, for example when different hardware with diverging capabilities (first vs. second TDX generation) leads to different CPUID capsets in a otherwise identical configuration. Verification should happen of course, but I think qemu should just throw an error in case the tdx can't support a given cpu configuration. I think you misunderstand this patch. It's to adjust the supported feature set of the platform, not the feature set of the given VM/TD. I.e, the adjusted supported feature set will be used to *verify* the VM's setting that specified by user. Of course, if user requires unsupported feature, QEMU will throw an error. (see also Daniels reply to the cover letter). take care, Gerd
Re: [PATCH v1 08/40] i386/tdx: Adjust the supported CPUID based on TDX restrictions
Hi, > between VMM and TDs. Adjust supported CPUID for TDs based on TDX > restrictions. Automatic adjustment depending on hardware capabilities isn't going to fly long-term, you'll run into compatibility problems sooner or later, for example when different hardware with diverging capabilities (first vs. second TDX generation) leads to different CPUID capsets in a otherwise identical configuration. Verification should happen of course, but I think qemu should just throw an error in case the tdx can't support a given cpu configuration. (see also Daniels reply to the cover letter). take care, Gerd
Re: [PATCH v1 08/40] i386/tdx: Adjust the supported CPUID based on TDX restrictions
On 8/3/2022 3:33 PM, Chenyi Qiang wrote: On 8/2/2022 3:47 PM, Xiaoyao Li wrote: According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types: 1 | As configured | configurable by VMM, independent of native value; 2 | As configured | configurable by VMM if the bit is supported natively (if native) | Otherwise it equals as native(0). 3 | Fixed | fixed to 0/1 4 | Native | reflect the native value 5 | Calculated | calculated by TDX module. 6 | Inducing #VE | get #VE exception Note: 1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3. 2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec. When they are queried, TDX module injects #VE to TDs. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM. Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported CPUID bits for TDS. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions. Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord. Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields: - tdx_fixed0/1: The bits that are fixed as 0/1; - vmm_fixup: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when they are configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them. - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM. - supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs. By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps: - get the base of VMM supported feature set; - if the leaf is not a FeatureWord just return VMM's value without modification; - if the leaf is an inducing_ve type, applying supported_on_ve mask and return; - include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.) - apply fixed0/1 to it (it covers #3, and rectifies the previous step); - add configurable bits (it covers the other part of type #1); - fix the ones in vmm_fixup; - filter the one has valid .supported field; What does .supported field filter mean here? (Calculated type is ignored since it's determined at runtime). Co-developed-by: Chenyi Qiang Signed-off-by: Chenyi Qiang Signed-off-by: Xiaoyao Li --- target/i386/cpu.h | 16 +++ target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 255 ++ target/i386/kvm/tdx.h | 2 + 4 files changed, 277 insertions(+) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 82004b65b944..cc9da9fc4318 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -771,6 +771,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1) /* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2) /* 1st Group of Advanced Bit Manipulation Extensions */ @@ -789,8 +791,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EBX_INVPCID (1U << 10) /* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12) /* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15) /* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16) /*
Re: [PATCH v1 08/40] i386/tdx: Adjust the supported CPUID based on TDX restrictions
On 8/2/2022 3:47 PM, Xiaoyao Li wrote: According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types: 1 | As configured | configurable by VMM, independent of native value; 2 | As configured | configurable by VMM if the bit is supported natively (if native) | Otherwise it equals as native(0). 3 | Fixed | fixed to 0/1 4 | Native| reflect the native value 5 | Calculated| calculated by TDX module. 6 | Inducing #VE | get #VE exception Note: 1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3. 2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec. When they are queried, TDX module injects #VE to TDs. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM. Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported CPUID bits for TDS. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions. Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord. Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields: - tdx_fixed0/1: The bits that are fixed as 0/1; - vmm_fixup: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when they are configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them. - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM. - supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs. By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps: - get the base of VMM supported feature set; - if the leaf is not a FeatureWord just return VMM's value without modification; - if the leaf is an inducing_ve type, applying supported_on_ve mask and return; - include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.) - apply fixed0/1 to it (it covers #3, and rectifies the previous step); - add configurable bits (it covers the other part of type #1); - fix the ones in vmm_fixup; - filter the one has valid .supported field; What does .supported field filter mean here? (Calculated type is ignored since it's determined at runtime). Co-developed-by: Chenyi Qiang Signed-off-by: Chenyi Qiang Signed-off-by: Xiaoyao Li --- target/i386/cpu.h | 16 +++ target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 255 ++ target/i386/kvm/tdx.h | 2 + 4 files changed, 277 insertions(+) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 82004b65b944..cc9da9fc4318 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -771,6 +771,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST(1U << 1) /* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2) /* 1st Group of Advanced Bit Manipulation Extensions */ @@ -789,8 +791,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EBX_INVPCID (1U << 10) /* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12) /* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15) /* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16) /* AVX-512
