Sergio Lopez <s...@redhat.com> writes: > Paolo Bonzini <pbonz...@redhat.com> writes: > >> On 02/07/19 12:52, Sergio Lopez wrote: >>> As I said, I'm also in favor of microvm supporting booting from >>> firmware in the future, as long we keep the simple mode too. >> >> The simple mode adds code to QEMU's x86 target that only exists to >> support microvm. It should be motivated by a clear win in boot times. > > OK. When I'm back from my PTO, I'll work on adding the firmware > support to microvm. I'll run and share some numbers to see whether the > simple mode makes sense or we can just rely on qboot for lower boot > times plus SeaBIOS for compatibility.
I've just added support for starting the machine from SeaBIOS (Stefan Hajnoczi pointed in another thread that it can be as fast as qboot, and given that the latter doesn't support mptables, I just tested this one). I tried to keep it as minimalistic as possible, but it still required an RTC (mc146818), which dragged in an ISA BUS, and this one a KVM PIT. I ran some numbers using Stefano Garzarella's qemu-boot-time scripts [1] on a server with 2xIntel Xeon Silver 4114 2.20GHz, using the upstream QEMU (474f3938d79ab36b9231c9ad3b5a9314c2aeacde) built with minimal features [2]. The VM boots a minimal kernel [3] without initrd, using a kata container image as root via virtio-blk (though this isn't really relevant, as we're just taking measurements until the kernel is about to exec init). --------------------- | QEMU with SeaBIOS | --------------------- Command line: ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm,legacy -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet virtio_mmio.device=512@0xd0000600:15 virtio_mmio.device=512@0xd0000400:14" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/usr/share/kata-containers/kata-containers.img,format=raw,if=none -device virtio-blk-device,drive=test -monitor stdio Average boot times after 10 consecutive runs: qemu_init_end: 65.958714 linux_start_kernel: 77.735803 (+11.777089) linux_start_user: 127.360739 (+49.624936) Exposed I/O ports and MMIOs: address-space: memory 0000000000000000-ffffffffffffffff (prio 0, i/o): system 0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff 00000000000e0000-00000000000fffff (prio 1, i/o): alias isa-bios @pc.bios 0000000000000000-000000000001ffff 00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio 00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio 00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio 00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi 00000000fffe0000-00000000ffffffff (prio 0, ram): pc.bios address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic 0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit 0000000000000070-0000000000000071 (prio 0, i/o): rtc 0000000000000070-0000000000000070 (prio 0, i/o): rtc-index 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic 00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic 00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr 00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr 0000000000000510-0000000000000511 (prio 0, i/o): fwcfg 0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma ------------------- | QEMU direct PVH | ------------------- Command line: ./x86_64-softmmu/qemu-system-x86_64 -m 512m -enable-kvm -M microvm -kernel /root/src/images/vmlinux-5.2 -append "console=hvc0 reboot=k panic=1 root=/dev/vda quiet" -smp 1 -nodefaults -no-user-config -chardev pty,id=virtiocon0,server -device virtio-serial-device -device virtconsole,chardev=virtiocon0 -drive id=test,file=/usr/share/kata-containers/kata-containers.img,format=raw,if=none -device virtio-blk-device,drive=test -monitor stdio Average boot times after 10 consecutive runs: qemu_init_end: 64.043264 linux_start_kernel: 65.481782 (+1.438518) linux_start_user: 114.938353 (+49.456571) Exposed I/O ports and MMIOs: address-space: memory 0000000000000000-ffffffffffffffff (prio 0, i/o): system 0000000000000000-000000001fffffff (prio 0, i/o): alias ram-below-4g @microvm.ram 0000000000000000-000000001fffffff 00000000d0000000-00000000d00001ff (prio 0, i/o): virtio-mmio 00000000d0000200-00000000d00003ff (prio 0, i/o): virtio-mmio 00000000d0000400-00000000d00005ff (prio 0, i/o): virtio-mmio 00000000d0000600-00000000d00007ff (prio 0, i/o): virtio-mmio 00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic 00000000fee00000-00000000feefffff (prio 4096, i/o): kvm-apic-msi address-space: I/O 0000000000000000-000000000000ffff (prio 0, i/o): io 000000000000007e-000000000000007f (prio 0, i/o): kvmvapic -------------- | Comparison | -------------- Average boot time: * Relying on SeaBIOS, when compared with direct PVH boot, as a total average overhead of ~12ms. The cost of initializing QEMU increases in ~2ms (probably due to need to instantiate more devices), while the other ~10ms is the SeaBIOS overhead. Exposed I/O ports and MMIOs: * The following 8 I/O ports are only present in the version relying on SeaBIOS: 0000000000000020-0000000000000021 (prio 0, i/o): kvm-pic 0000000000000040-0000000000000043 (prio 0, i/o): kvm-pit 0000000000000070-0000000000000071 (prio 0, i/o): rtc 0000000000000070-0000000000000070 (prio 0, i/o): rtc-index 00000000000000a0-00000000000000a1 (prio 0, i/o): kvm-pic 00000000000004d0-00000000000004d0 (prio 0, i/o): kvm-elcr 00000000000004d1-00000000000004d1 (prio 0, i/o): kvm-elcr 0000000000000510-0000000000000511 (prio 0, i/o): fwcfg 0000000000000514-000000000000051b (prio 0, i/o): fwcfg.dma * The following MMIO region is only present in the direct boot version: 00000000fec00000-00000000fec00fff (prio 0, i/o): kvm-ioapic --------------- | Conclusions | --------------- Objectively, the version that boots directly the kernel using PVH is 10% faster and has a slightly larger exposed surface. Whether this is enough to justify its existence is quite subjective. In my opinion, not only I think it makes sense to have it, but I also think there's little reason to have the firmware reliant version, given the nature and purpose of microvm. Sergio. [1] https://github.com/stefano-garzarella/qemu-boot-time [2] https://paste.fedoraproject.org/paste/YZ9Ok-dJtQrc0xxctFm-nw [3] https://paste.fedoraproject.org/paste/sck0jfioAJdMq51HH6wkmA
signature.asc
Description: PGP signature