On Tue, Nov 26, 2019 at 3:35 PM Waldek Kozaczuk <jwkozac...@gmail.com> wrote:
> The review and discussion around proper poweroff/reboot sequence on > microvm/firecracker with Nadav (see other email) inspired me to check > another thing - remove '-no-reboot` from QEMU parameter list. This should > then trigger reboot rather than poweroff, right? And then constantly > re-execute the same OSv image over and over sequentially. And this is what > I did and the results are astonishing. > > cat test_micro.sh > > time > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > \ > -m 64M \ > -smp 1 \ > --nographic \ > -kernel > /home/wkozaczuk/projects/osv/scripts/../build/last/loader-stripped.elf \ > -append "--nopci /hello" \ > -M microvm,x-option-roms=off,pit=off,pic=off,rtc=off \ > -nodefaults \ > -no-user-config \ > -global virtio-mmio.force-legacy=off \ > -device virtio-blk-device,id=blk0,drive=hd0,scsi=off \ > -drive > file=/home/wkozaczuk/projects/osv/build/last/usr.img,if=none,id=hd0,cache=writeback,aio=threads > \ > -enable-kvm \ > -cpu host,+x2apic \ > -serial stdio > > /test_micro.sh > /tmp/log > > qemu-system-x86_64: terminating on signal 2 > > real 0m10.467s > user 0m6.366s > sys 0m3.328s > > grep Hello /tmp/log | wc -l > 1772 > > As you can see OSv re-executed 1772 times in 10.467 seconds. Which gives > you: > Too bad the Guiness Book of World Records doesn't keep records of which OS can reboot the quickest :-) > > - *6ms - average execution time including boo time* > - *170 - number of executions per second * > > So in this scenario, we pay for QEMU overhead once. > By the way, with such quick executions, the serial port code writing all the output may prove to be a noticable slowdown. I wonder is we can shave another millisecond by spewing less output :-) > Waldek > > On Monday, November 25, 2019 at 5:20:29 PM UTC-5, Waldek Kozaczuk wrote: >> >> First please note that this and other old (5ms) firecracker boot times >> were recorded when running on my old 2013 MacBook Pro with 4-way 2.3GHz i7 >> cpu with Ubuntu 19.04 on it. I think it still a decent laptop and the cpus >> have not gotten that much faster (Moore's law is dead now, right?), but it >> would be interesting to measure boot times on something more modern and >> powerful. >> >> On Sunday, November 24, 2019 at 6:46:30 AM UTC-5, Nadav Har'El wrote: >>> >>> >>> On Fri, Nov 22, 2019 at 3:34 PM Waldek Kozaczuk <jwkoz...@gmail.com> >>> wrote: >>> >>>> As you may have heard QEMU developers added new machine type: microvm - >>>> https://github.com/qemu/qemu/blob/master/docs/microvm.rst. It has been >>>> merged into master but not part of any official QEMU release. So if you >>>> want to try you need to build QEMU from the source. >>>> >>>> Recently I have played with it and eventually managed to boot OSv >>>> successfully on it after some adjustments (see some of my recent virtio >>>> code tweaks). And the same OSv ROFS image that would boot in ~5 ms on >>>> firecracker, can boot in only *~3ms* on QEMU microvm. >>>> >>> >>> Very nice! >>> >>> >>>> OSv v0.54.0-34-gccb9cd12 >>>> Booted up in 2.91 ms >>>> Cmdline: /hello >>>> Hello from C code >>>> >>>> real 0m0.057s >>>> user 0m0.037s >>>> sys 0m0.020s >>>> >>>> >>> Can you please remind me what this 2.91ms includes? When do we start >>> measuring and when do we stop measuring? >>> >> The 'Booted up' message is what I added couple of months ago and simply >> prints the total from bootchart. The boot time with full bootchart will >> explain what is included: >> >> OSv v0.54.0-40-g08466354 >> disk read (real mode): 0.00ms, (+0.00ms) >> uncompress lzloader.elf: 0.00ms, (+0.00ms) >> TLS initialization: 0.51ms, (+0.51ms) >> .init functions: 1.22ms, (+0.71ms) >> SMP launched: 1.55ms, (+0.32ms) >> VFS initialized: 1.76ms, (+0.21ms) >> Network initialized: 2.01ms, (+0.25ms) >> pvpanic done: 2.08ms, (+0.07ms) >> drivers probe: 2.12ms, (+0.04ms) >> drivers loaded: 2.70ms, (+0.59ms) >> ROFS mounted: 2.93ms, (+0.22ms) >> Total time: 2.93ms, (+0.00ms) >> Cmdline: /hello >> Hello from C code >> >> Please note that OSv boots in PVH mode (direct kernel mode) so the >> real-mode boot is bypassed and that is why disk read and uncompress are 0 >> (just like on firecracker). >> >> >>> The 57ms at the end, that is 20 slower - what does *that* include that >>> the 2.91ms figure didn't? >>> >>> Some of that will the execution of the application and poweroff. But >> most of it will be the overhead of the QEMU itself. Even though OSv boots >> very fast, QEMU process with microvm is still relatively slow to start. >> >> Here is the gist with the same thing with timed strace around it - >> https://gist.github.com/wkozaczuk/0806809f4003f1c3761ce4ba8c2f634f. >> Please note that running with strace slows everything down twice. >> >> My sense is that most of the overall 50-60ms are spent in QEMU startup. >> >> Now if you compare it with firecracker, firecracker starts much faster >> (based on the printouts I added to firecracker) and only adds 5 ms when it >> gives control to the guest. But then wastes 30-40 ms shutting down - see >> this from my firecracker issue comments - >> https://github.com/firecracker-microvm/firecracker/issues/1164 and >> https://github.com/firecracker-microvm/firecracker/issues/923#issuecomment-497336403. >> Somehow from the moment exit_group() is called until the process officially >> dies it takes 30-45 ms. Weird. >> >> -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to osv-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/osv-dev/287b75a3-b556-4ac4-a598-9f04323875fd%40googlegroups.com > <https://groups.google.com/d/msgid/osv-dev/287b75a3-b556-4ac4-a598-9f04323875fd%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CANEVyjuh6KRCQ1SZAU2TkKN9zeO7VpbTMe15AyeSgOJxU%2BGhpA%40mail.gmail.com.
