On 9/24/19 8:42 AM, Laszlo Ersek wrote: > On 09/19/19 21:52, Lendacky, Thomas wrote: >> From: Tom Lendacky <thomas.lenda...@amd.com> >> >> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2198 >> >> During BSP startup, the reset vector code will issue a CPUID instruction >> while in 32-bit mode. When running as an SEV-ES guest, this will trigger >> a #VC exception. > > (1) In the assembly source code, please annotate both CPUID instructions > under CheckSevFeature, such as > > ; raises #VC when in an SEV-ES guest
Will do. > >> >> Add exception handling support to the early reset vector code to catch >> these exceptions. Also, since the guest is in 32-bit mode at this point, >> writes to the GHCB will be encrypted and thus not able to be read by the >> hypervisor, so use the GHCB CPUID request/response protocol to obtain the >> requested CPUID function values and provide these to the guest. >> >> The exception handling support is active during the SEV check and uses the >> OVMF temporary RAM space for a stack. After the SEV check is complete, the >> exception handling support is removed and the stack pointer cleared. >> >> Cc: Jordan Justen <jordan.l.jus...@intel.com> >> Cc: Laszlo Ersek <ler...@redhat.com> >> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org> >> Signed-off-by: Tom Lendacky <thomas.lenda...@amd.com> >> --- >> OvmfPkg/ResetVector/ResetVector.inf | 2 + >> OvmfPkg/ResetVector/Ia32/PageTables64.asm | 177 +++++++++++++++++++++- >> OvmfPkg/ResetVector/ResetVector.nasmb | 1 + >> 3 files changed, 179 insertions(+), 1 deletion(-) >> >> diff --git a/OvmfPkg/ResetVector/ResetVector.inf >> b/OvmfPkg/ResetVector/ResetVector.inf >> index b0ddfa5832a2..960b47cd0797 100644 >> --- a/OvmfPkg/ResetVector/ResetVector.inf >> +++ b/OvmfPkg/ResetVector/ResetVector.inf >> @@ -35,3 +35,5 @@ [BuildOptions] >> [Pcd] >> gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesBase >> gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesSize >> + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase >> + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize >> diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm >> b/OvmfPkg/ResetVector/Ia32/PageTables64.asm >> index abad009f20f5..40f7814c1134 100644 >> --- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm >> +++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm >> @@ -33,10 +33,21 @@ BITS 32 >> >> ; Check if Secure Encrypted Virtualization (SEV) feature is enabled >> ; >> -; If SEV is enabled then EAX will be at least 32 >> +; Modified: EAX, EBX, ECX, EDX, ESP >> +; >> +; If SEV is enabled then EAX will be at least 32. >> ; If SEV is disabled then EAX will be zero. >> ; >> CheckSevFeature: >> + ; >> + ; Set up exception handlers to check for SEV-ES >> + ; Load temporary RAM stack based on PCDs >> + ; Establish exception handlers >> + ; >> + mov esp, SEV_TOP_OF_STACK > > (2) Can we %define SEV_TOP_OF_STACK in this file, or does it have to be > in "ResetVector.nasmb"? It looks like it has to be in ResetVector.nasmb for some reason. If I move it into this file, it fails: Ia32/PageTables64.asm:76: error: expecting `)' > > (3) Do we have an estimate how much stack we need? This would be a > constraint on PcdOvmfSecPeiTempRamSize. The limit would be nice to > document (perhaps in a comment somewhere). It's a fairly small amount of stack space, about 44 bytes (12 bytes for the IRET frame and 32 bytes for the #VC "local" variables). The temporary RAM size is 64K, so we're fine. I'll document the stack usage at the start of the SevEsIdtVmmComm exception handler. > >> + mov eax, ADDR_OF(Idtr) >> + lidt [cs:eax] >> + >> ; Check if we have a valid (0x8000_001F) CPUID leaf >> mov eax, 0x80000000 >> cpuid >> @@ -73,6 +84,15 @@ NoSev: >> xor eax, eax >> >> SevExit: >> + ; >> + ; Clear exception handlers and stack >> + ; >> + push eax >> + mov eax, ADDR_OF(IdtrClear) >> + lidt [cs:eax] >> + pop eax >> + mov esp, 0 >> + > > I'm not sure the resultant IDT and ESP contents are the same as before > (pre-patch), but I guess these values should be OK too. At this stage of the boot, both the IDT and ESP values are zero. They haven't been set up this early in the boot. > >> OneTimeCallRet CheckSevFeature >> >> ; >> @@ -146,3 +166,158 @@ pageTableEntriesLoop: >> mov cr3, eax >> >> OneTimeCallRet SetCr3ForPageTables64 >> + >> +SevEsIdtCommon: >> + hlt >> + jmp SevEsIdtCommon >> + iret >> + >> +SevEsIdtVmmComm: >> + ; >> + ; If we're here, then we are an SEV-ES guest and this >> + ; was triggered by a CPUID instruction >> + ; >> + pop ecx ; Error code >> + cmp ecx, 0x72 ; Be sure it was CPUID >> + jne SevEsIdtCommon > > (4) Can you steal the DebugLog / PrintStringSi code from > "OvmfPkg/QemuVideoDxe/VbeShim.asm", and print a simple message to the > QEMU debug port, whenever you jump to SevEsIdtCommon? > > This is basically an ASSERT(). It should have a message, if possible. > (I'm not sure if the OUT instruction will work with SEV-ES at this > stage, i.e. in 32-bit mode.) Right, there isn't a way to share the OUT instruction information with the hypervisor at this point because we are running in 32-bit mode (this might be nice to add to the GHCB protocol, similar to the CPUID support, I'll have to think about that). Since we can't output a message, I can do something along the line of using the GHCB protocol to request the hypervisor to terminate the guest. Would that be better? > >> + >> + ; >> + ; Set up local variable room on the stack >> + ; CPUID function : + 28 >> + ; CPUID register : + 24 >> + ; GHCB MSR (EAX) : + 20 >> + ; GHCB MSR (EDX) : + 16 >> + ; CPUID result (EDX) : + 12 >> + ; CPUID result (ECX) : + 8 >> + ; CPUID result (EBX) : + 4 >> + ; CPUID result (EAX) : + 0 >> + sub esp, 32 > > (5) Can we %define macros for these offsets? (Including the size > constant 32.) Will do. > >> + >> + ; Save CPUID function and initial register request >> + mov [esp + 28], eax >> + xor eax, eax >> + mov [esp + 24], eax > > (6) The comment "CPUID register" (at offset 24), and the other comment > "initial register", are pretty confusing. Can you please document: > > - the mapping: 0->EAX, ... 3->EDX, > - and the fact that the dword at [esp + 24] is the loop variable? Yup, can do. > >> + >> + ; Save current GHCB MSR value >> + mov ecx, 0xc0010130 >> + rdmsr >> + mov [esp + 20], eax >> + mov [esp + 16], edx >> + >> +NextReg: >> + ; >> + ; Setup GHCB MSR >> + ; GHCB_MSR[63:32] = CPUID function >> + ; GHCB_MSR[31:30] = CPUID register >> + ; GHCB_MSR[11:0] = CPUID request protocol >> + ; >> + mov eax, [esp + 24] >> + cmp eax, 4 >> + jge VmmDone >> + >> + shl eax, 30 >> + or eax, 0x004 > > (7) Please %define GHCBInfoCpuIdRequest or something similar for the > value 4. Ok. I'll remove all of the hardcoded values and %define them. > >> + mov edx, [esp + 28] >> + mov ecx, 0xc0010130 >> + wrmsr >> + >> + ; Issue VMGEXIT (rep; vmmcall) >> + db 0xf3 >> + db 0x0f >> + db 0x01 >> + db 0xd9 > > (8) Can you please file an RFE at <https://bugzilla.nasm.us/>, for > supporting this instruction, and add the link here, as a comment? I've > been fighting an uphill battle against DB-encoded instructions in edk2 > assembly code. Yes, let me look into that. > >> + >> + ; >> + ; Read GHCB MSR >> + ; GHCB_MSR[63:32] = CPUID register value >> + ; GHCB_MSR[31:30] = CPUID register >> + ; GHCB_MSR[11:0] = CPUID response protocol >> + ; >> + mov ecx, 0xc0010130 >> + rdmsr >> + mov ecx, eax >> + and ecx, 0xfff >> + cmp ecx, 0x005 > > (9) Please %define GHCBInfoCpuIdResponse for value 5. > >> + jne SevEsIdtCommon > > (10) Please see (4). The message could be, "no GHCBInfoCpuIdResponse > received", or similar. Sure, if I find a way to issue messages at this point. > >> + >> + ; Save returned value >> + shr eax, 30 >> + and eax, 0x3 > > (11) Do we need the AND after the SHR? I think the new high order bits > from the SHR should be zero. No, we don't actually need the AND, just a habit to show the size of the area. But since we're at the top of the register range (bits 30 and 31) it's not really necessary. I'll remove it. > >> + shl eax, 2 >> + mov ecx, esp >> + add ecx, eax >> + mov [ecx], edx > > (12) The beauty of the lean and mean x86 instruction set: > > mov [esp + eax * 4], edx > > (I tested just this one instruction with nasm + ndisasm; the encoding is > 0x89, 0x14, 0x84.) Ah, yes. Will do. > >> + >> + ; Next register >> + inc word [esp + 24] >> + >> + jmp NextReg >> + >> +VmmDone: >> + ; >> + ; At this point we have all CPUID register values. Restore the GHCB MSR, >> + ; set the return register values and return. >> + ; >> + mov eax, [esp + 20] >> + mov edx, [esp + 16] >> + mov ecx, 0xc0010130 >> + wrmsr >> + >> + mov eax, [esp + 0] >> + mov ebx, [esp + 4] >> + mov ecx, [esp + 8] >> + mov edx, [esp + 12] >> + >> + add esp, 32 > > (13) Please see (5). > >> + add word [esp], 2 ; Skip over the CPUID instruction > > OK, this seems safe. CPUID has only one encoding (0x0F, 0xA2), and we > checked SW_EXITCODE = 0x72 at the top. In this case, yes. Since we're covering a small piece of code and we generated the instructions I think we can rely on that. Later on, that's where the instruction parsing comes in. > >> + iret >> + >> +ALIGN 2 >> + >> +Idtr: >> + dw IDT_END - IDT_BASE - 1 ; Limit >> + dd ADDR_OF(IDT_BASE) ; Base >> + >> +IdtrClear: >> + dw 0 ; Limit >> + dd 0 ; Base >> + >> +ALIGN 16 >> + >> +; >> +; The Interrupt Descriptor Table (IDT) >> +; This will be used to determine if SEV-ES is enabled. Upon execution >> +; of the CPUID instruction, a VMM Communication Exception will occur. >> +; This will tell us if SEV-ES is enabled. We can use the current value >> +; of the GHCB MSR to determine the SEV attributes. >> +; >> +IDT_BASE: >> +; >> +; Vectors 0 - 28 >> +; >> +%rep 29 >> + dw (ADDR_OF(SevEsIdtCommon) & 0xffff) ; Offset low bits 15..0 >> + dw 0x10 ; Selector >> + db 0 ; Reserved >> + db 0x8E ; Gate Type >> (IA32_IDT_GATE_TYPE_INTERRUPT_32) >> + dw (ADDR_OF(SevEsIdtCommon) >> 16) ; Offset high bits 31..16 >> +%endrep >> +; >> +; Vector 29 (VMM Communication Exception) >> +; >> + dw (ADDR_OF(SevEsIdtVmmComm) & 0xffff) ; Offset low bits 15..0 >> + dw 0x10 ; Selector >> + db 0 ; Reserved >> + db 0x8E ; Gate Type >> (IA32_IDT_GATE_TYPE_INTERRUPT_32) >> + dw (ADDR_OF(SevEsIdtVmmComm) >> 16) ; Offset high bits 31..16 >> +; >> +; Vectors 30 - 31 >> +; >> +%rep 2 >> + dw (ADDR_OF(SevEsIdtCommon) & 0xffff) ; Offset low bits 15..0 >> + dw 0x10 ; Selector >> + db 0 ; Reserved >> + db 0x8E ; Gate Type >> (IA32_IDT_GATE_TYPE_INTERRUPT_32) >> + dw (ADDR_OF(SevEsIdtCommon) >> 16) ; Offset high bits 31..16 >> +%endrep >> +IDT_END: > > (14) Above, we have two explicit jumps to SevEsIdtCommon, and I've asked > for meaningful assertion messages there. > > For the uninteresting exception vectors 0-28 and 30-31, can we use a > handler that is not directly SevEsIdtCommon, but logs a message, and > then jumps to SevEsIdtCommon? (It could even be implemented as a > "prefix" of SevEsIdtCommon, and then no jump would be required.) If I come up with a way to get a message out, we can look at that. I'll need to see if it is too late to update the GHCB protocol specification to add an OUT protocol to support messages when in 32-bit. I think I can actually set just the #VC exception entry so that any other exception just causes the guest to die, similar to what would happen if an exception were to occur at this stage today. Thanks for the thorough review! Tom > >> diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb >> b/OvmfPkg/ResetVector/ResetVector.nasmb >> index 75cfe16654b1..3b213cd05ab2 100644 >> --- a/OvmfPkg/ResetVector/ResetVector.nasmb >> +++ b/OvmfPkg/ResetVector/ResetVector.nasmb >> @@ -55,6 +55,7 @@ >> >> %define PT_ADDR(Offset) (FixedPcdGet32 (PcdOvmfSecPageTablesBase) + >> (Offset)) >> %include "Ia32/Flat32ToFlat64.asm" >> + %define SEV_TOP_OF_STACK (FixedPcdGet32 (PcdOvmfSecPeiTempRamBase) + >> FixedPcdGet32 (PcdOvmfSecPeiTempRamSize)) >> %include "Ia32/PageTables64.asm" >> %endif >> >> > > (I've commented on this under (2).) > > Thanks! > Laszlo > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#47990): https://edk2.groups.io/g/devel/message/47990 Mute This Topic: https://groups.io/mt/34203539/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
