Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
On 8/19/21 3:32 AM, Marc Orr wrote: > On Wed, Aug 18, 2021 at 1:38 AM Varad Gautam wrote: >> >> Hi Marc, Zixuan, >> >> On 8/18/21 3:52 AM, Marc Orr wrote: >>> On Tue, Aug 17, 2021 at 3:49 AM Joerg Roedel wrote: Hi Marc, On Fri, Aug 13, 2021 at 11:44:39AM -0700, Marc Orr wrote: > To date, we have _most_ x86 test cases (39/44) working under UEFI and > we've also got some of the test cases to boot under SEV-ES, using the > UEFI #VC handler. While the EFI APP approach simplifies the implementation a lot, I don't think it is the best path to SEV and TDX testing for a couple of reasons: 1) It leaves the details of #VC/#VE handling and the SEV-ES specific communication channels (GHCB) under control of the firmware. So we can't reliably test those interfaces from an EFI APP. 2) Same for the memory validation/acceptance interface needed for SEV-SNP and TDX. Using an EFI APP leaves those under firmware control and we are not able to reliably test them. 3) The IDT also stays under control of the firmware in an EFI APP, otherwise the firmware couldn't provide a #VC handler. This makes it unreliable to test anything IDT or IRQ related. 4) Relying on the firmware #VC hanlder limits the tests to its abilities. Implementing a separate #VC handler routine for kvm-unit-tests is more work, but it makes test development much more flexible. So it comes down to the fact that and EFI APP leaves control over SEV/TDX specific hypervisor interfaces in the firmware, making it hard and unreliable to test these interfaces from kvm-unit-tests. The stub approach on the other side gives the tests full control over the VM, allowing to test all aspects of the guest-host interface. >>> >>> I think we might be using terminology differently. (Maybe I mis-used >>> the term “EFI app”?) With our approach, it is true that all >>> pre-existing x86_64 test cases work out of the box with the UEFI #VC >>> handler. However, because kvm-unit-tests calls `ExitBootServices` to >>> take full control of the system it executes as a “UEFI-stubbed >>> kernel”. Thus, it should be trivial for test cases to update the IDT >>> to set up a custom #VC handler for the duration of a test. (Some of >>> the x86_64 test cases already do something similar where they install >>> a temporary exception handler and then restore the “default” >>> kvm-unit-tests exception handler.) >>> >>> In general, our approach is to set up the test cases to run with the >>> kvm-unit-tests configuration (e.g., IDT, GDT). The one exception is >>> the #VC handler. However, all of this state can be overridden within a >>> test as needed. >>> >>> Zixuan just posted the patches. So hopefully they make things more clear. >>> >> >> Nomenclature aside, I believe Zixuan's patchset [1] takes the same approach >> as I posted here. In the end, we need to: >> - build the testcases as ELF shared objs and link them to look like a PE >> - switch away from UEFI GDT/IDT/pagetable states on early boot to what >> kvm-unit-tests needs >> - modify the testcases that contain non-PIC asm stubs to allow building >> them as shared objs >> >> I went with avoiding to bring in gnu-efi objects into kvm-unit-tests >> for EFI helpers, and disabling the non-PIC testcases for the RFC's sake. >> >> I'll try out "x86 UEFI: Convert x86 test cases to PIC" [2] from Zixuan's >> patchset with my series and see what breaks. I think we can combine >> the two patchsets. >> >> [1] https://lore.kernel.org/r/20210818000905.226-1-zixuanw...@google.com/ >> [2] >> https://lore.kernel.org/r/20210818000905.226-10-zixuanw...@google.com/ > > This sounds great to us. We will also experiment with combining the > two patchsets and report back when we have some experience with this. > Though, please do also report back if you have an update on this > before we do. > I sent out a v2 [1] with Zixuan's "x86 UEFI: Convert x86 test cases to PIC" [2] pulled in, PTAL. [1] https://lore.kernel.org/r/20210819113400.26516-1-varad.gau...@suse.com/ [2] https://lore.kernel.org/r/20210818000905.226-10-zixuanw...@google.com/ Thanks, Varad > Thanks, > Marc > -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany HRB 36809, AG Nürnberg Geschäftsführer: Felix Imendörffer ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
> On Aug 18, 2021, at 6:32 PM, Marc Orr wrote: > > This sounds great to us. We will also experiment with combining the > two patchsets and report back when we have some experience with this. > Though, please do also report back if you have an update on this > before we do. Just wondering (aka, my standard question): does it work on bare-metal (putting aside tests that rely on test-devices)? ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
Hi Marc, Zixuan, On 8/18/21 3:52 AM, Marc Orr wrote: > On Tue, Aug 17, 2021 at 3:49 AM Joerg Roedel wrote: >> >> Hi Marc, >> >> On Fri, Aug 13, 2021 at 11:44:39AM -0700, Marc Orr wrote: >>> To date, we have _most_ x86 test cases (39/44) working under UEFI and >>> we've also got some of the test cases to boot under SEV-ES, using the >>> UEFI #VC handler. >> >> While the EFI APP approach simplifies the implementation a lot, I don't >> think it is the best path to SEV and TDX testing for a couple of >> reasons: >> >> 1) It leaves the details of #VC/#VE handling and the SEV-ES >>specific communication channels (GHCB) under control of the >>firmware. So we can't reliably test those interfaces from an >>EFI APP. >> >> 2) Same for the memory validation/acceptance interface needed >>for SEV-SNP and TDX. Using an EFI APP leaves those under >>firmware control and we are not able to reliably test them. >> >> 3) The IDT also stays under control of the firmware in an EFI >>APP, otherwise the firmware couldn't provide a #VC handler. >>This makes it unreliable to test anything IDT or IRQ related. >> >> 4) Relying on the firmware #VC hanlder limits the tests to its >>abilities. Implementing a separate #VC handler routine for >>kvm-unit-tests is more work, but it makes test development >>much more flexible. >> >> So it comes down to the fact that and EFI APP leaves control over >> SEV/TDX specific hypervisor interfaces in the firmware, making it hard >> and unreliable to test these interfaces from kvm-unit-tests. The stub >> approach on the other side gives the tests full control over the VM, >> allowing to test all aspects of the guest-host interface. > > I think we might be using terminology differently. (Maybe I mis-used > the term “EFI app”?) With our approach, it is true that all > pre-existing x86_64 test cases work out of the box with the UEFI #VC > handler. However, because kvm-unit-tests calls `ExitBootServices` to > take full control of the system it executes as a “UEFI-stubbed > kernel”. Thus, it should be trivial for test cases to update the IDT > to set up a custom #VC handler for the duration of a test. (Some of > the x86_64 test cases already do something similar where they install > a temporary exception handler and then restore the “default” > kvm-unit-tests exception handler.) > > In general, our approach is to set up the test cases to run with the > kvm-unit-tests configuration (e.g., IDT, GDT). The one exception is > the #VC handler. However, all of this state can be overridden within a > test as needed. > > Zixuan just posted the patches. So hopefully they make things more clear. > Nomenclature aside, I believe Zixuan's patchset [1] takes the same approach as I posted here. In the end, we need to: - build the testcases as ELF shared objs and link them to look like a PE - switch away from UEFI GDT/IDT/pagetable states on early boot to what kvm-unit-tests needs - modify the testcases that contain non-PIC asm stubs to allow building them as shared objs I went with avoiding to bring in gnu-efi objects into kvm-unit-tests for EFI helpers, and disabling the non-PIC testcases for the RFC's sake. I'll try out "x86 UEFI: Convert x86 test cases to PIC" [2] from Zixuan's patchset with my series and see what breaks. I think we can combine the two patchsets. [1] https://lore.kernel.org/r/20210818000905.226-1-zixuanw...@google.com/ [2] https://lore.kernel.org/r/20210818000905.226-10-zixuanw...@google.com/ Thanks, Varad > Thanks, > Marc > -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany HRB 36809, AG Nürnberg Geschäftsführer: Felix Imendörffer ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
Hi Marc, On Fri, Aug 13, 2021 at 11:44:39AM -0700, Marc Orr wrote: > To date, we have _most_ x86 test cases (39/44) working under UEFI and > we've also got some of the test cases to boot under SEV-ES, using the > UEFI #VC handler. While the EFI APP approach simplifies the implementation a lot, I don't think it is the best path to SEV and TDX testing for a couple of reasons: 1) It leaves the details of #VC/#VE handling and the SEV-ES specific communication channels (GHCB) under control of the firmware. So we can't reliably test those interfaces from an EFI APP. 2) Same for the memory validation/acceptance interface needed for SEV-SNP and TDX. Using an EFI APP leaves those under firmware control and we are not able to reliably test them. 3) The IDT also stays under control of the firmware in an EFI APP, otherwise the firmware couldn't provide a #VC handler. This makes it unreliable to test anything IDT or IRQ related. 4) Relying on the firmware #VC hanlder limits the tests to its abilities. Implementing a separate #VC handler routine for kvm-unit-tests is more work, but it makes test development much more flexible. So it comes down to the fact that and EFI APP leaves control over SEV/TDX specific hypervisor interfaces in the firmware, making it hard and unreliable to test these interfaces from kvm-unit-tests. The stub approach on the other side gives the tests full control over the VM, allowing to test all aspects of the guest-host interface. Regards, Joerg ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
On Fri, Aug 13, 2021 at 11:44:39AM -0700, Marc Orr wrote: > On Fri, Jul 2, 2021 at 4:48 AM Varad Gautam wrote: > > > > This series brings EFI support to a reduced subset of kvm-unit-tests > > on x86_64. I'm sending it out for early review since it covers enough > > ground to allow adding KVM testcases for EFI-only environments. > > > > EFI support works by changing the test entrypoint to a stub entry > > point for the EFI loader to jump to in long mode, where the test binary > > exits EFI boot services, performs the remaining CPU bootstrapping, > > and then calls the testcase main(). > > > > Since the EFI loader only understands PE objects, the first commit > > introduces a `configure --efi` mode which builds each test as a shared > > lib. This shared lib is repackaged into a PE via objdump. > > > > Commit 2-4 take a trip from the asm entrypoint to C to exit EFI and > > relocate ELF .dynamic contents. > > > > Commit 5 adds post-EFI long mode x86_64 setup and calls the testcase. > > > > Commit 6 patches out some broken tests for EFI. Testcases that refuse > > to build as shared libs are also left disabled, these need some massaging. > > > > git tree: https://github.com/varadgautam/kvm-unit-tests/commits/efi-stub > > Thanks for this patchset. My colleague, Zixuan Wang > , has also been working to extend > kvm-unit-tests to run under UEFI. Our goal is to enable running > kvm-unit-tests under SEV-ES. > > Our approach is a bit different. Rather than pull in bits of the EFI > library and Linux EFI ABI, we've elected to build the entire > kvm-unit-tests binaries as an EFI app (similar to the ARM approach). > > To date, we have _most_ x86 test cases (39/44) working under UEFI and > we've also got some of the test cases to boot under SEV-ES, using the > UEFI #VC handler. > > We will post our patchset as soon as possible (hopefully by Monday) so > that the community can see our approach. We are very eager to see > kvm-unit-tests running under SEV-ES (and SNP) and are happy to work > with you all on either approach, depending on what the community > thinks is the best approach. > > Thanks in advance, > Marc > Hi Marc, I'm definitely eager to see your approach. I was actually working on a second version of EFI support for ARM using the stub approach like this series before getting perpetually sidetracked. I've been wanted to experiment with Varad's code to continue that, but haven't been able to find the time. I'm curious if you considered the stub approach as well, but then opted for the app approach in the end. I was leaning towards the stub approach to avoid the gnu-efi dependency. Thanks, drew ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
On Fri, Jul 02, 2021 at 01:48:14PM +0200, Varad Gautam wrote: > This series brings EFI support to a reduced subset of kvm-unit-tests > on x86_64. I'm sending it out for early review since it covers enough > ground to allow adding KVM testcases for EFI-only environments. > > EFI support works by changing the test entrypoint to a stub entry > point for the EFI loader to jump to in long mode, where the test binary > exits EFI boot services, performs the remaining CPU bootstrapping, > and then calls the testcase main(). > > Since the EFI loader only understands PE objects, the first commit > introduces a `configure --efi` mode which builds each test as a shared > lib. This shared lib is repackaged into a PE via objdump. > > Commit 2-4 take a trip from the asm entrypoint to C to exit EFI and > relocate ELF .dynamic contents. > > Commit 5 adds post-EFI long mode x86_64 setup and calls the testcase. > > Commit 6 patches out some broken tests for EFI. Testcases that refuse > to build as shared libs are also left disabled, these need some massaging. > > git tree: https://github.com/varadgautam/kvm-unit-tests/commits/efi-stub Hi Varad, Thanks for this. I haven't reviewed it in detail yet (I just got back from vacation), but this looks like the right approach. In fact, I had started going down the efi stub approach for AArch64 a while back as well, but the effort got preempted by other work [again]. I'll try to allocate time to play with this for x86 and to build on it for AArch64 in the coming weeks. drew > > Varad Gautam (6): > x86: Build tests as PE objects for the EFI loader > x86: Call efi_main from _efi_pe_entry > x86: efi_main: Get EFI memory map and exit boot services > x86: efi_main: Self-relocate ELF .dynamic addresses > cstart64.S: x86_64 bootstrapping after exiting EFI > x86: Disable some breaking tests for EFI and modify vmexit test > > .gitignore | 2 + > Makefile| 16 ++- > configure | 11 ++ > lib/linux/uefi.h| 337 > x86/Makefile.common | 45 -- > x86/Makefile.x86_64 | 43 +++--- > x86/cstart64.S | 78 ++ > x86/efi.lds | 67 + > x86/efi_main.c | 167 ++ > x86/vmexit.c| 7 + > 10 files changed, 741 insertions(+), 32 deletions(-) > create mode 100644 lib/linux/uefi.h > create mode 100644 x86/efi.lds > create mode 100644 x86/efi_main.c > > -- > 2.30.2 > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[kvm-unit-tests PATCH 0/6] Initial x86_64 UEFI support
This series brings EFI support to a reduced subset of kvm-unit-tests on x86_64. I'm sending it out for early review since it covers enough ground to allow adding KVM testcases for EFI-only environments. EFI support works by changing the test entrypoint to a stub entry point for the EFI loader to jump to in long mode, where the test binary exits EFI boot services, performs the remaining CPU bootstrapping, and then calls the testcase main(). Since the EFI loader only understands PE objects, the first commit introduces a `configure --efi` mode which builds each test as a shared lib. This shared lib is repackaged into a PE via objdump. Commit 2-4 take a trip from the asm entrypoint to C to exit EFI and relocate ELF .dynamic contents. Commit 5 adds post-EFI long mode x86_64 setup and calls the testcase. Commit 6 patches out some broken tests for EFI. Testcases that refuse to build as shared libs are also left disabled, these need some massaging. git tree: https://github.com/varadgautam/kvm-unit-tests/commits/efi-stub Varad Gautam (6): x86: Build tests as PE objects for the EFI loader x86: Call efi_main from _efi_pe_entry x86: efi_main: Get EFI memory map and exit boot services x86: efi_main: Self-relocate ELF .dynamic addresses cstart64.S: x86_64 bootstrapping after exiting EFI x86: Disable some breaking tests for EFI and modify vmexit test .gitignore | 2 + Makefile| 16 ++- configure | 11 ++ lib/linux/uefi.h| 337 x86/Makefile.common | 45 -- x86/Makefile.x86_64 | 43 +++--- x86/cstart64.S | 78 ++ x86/efi.lds | 67 + x86/efi_main.c | 167 ++ x86/vmexit.c| 7 + 10 files changed, 741 insertions(+), 32 deletions(-) create mode 100644 lib/linux/uefi.h create mode 100644 x86/efi.lds create mode 100644 x86/efi_main.c -- 2.30.2 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
