Wow, awesome results! I will try to reproduce shortly from here! If you want to increase the problem size, add -n 16777216 (which is 64x bigger than the current problem size of 262144, something will take take a couple of seconds to run).
I am taking note of your commands to run the build and will be doing the same on my machine to see what I produce. I will post screenshots :) On Tuesday, February 25, 2020 at 1:09:08 PM UTC-5, Waldek Kozaczuk wrote: > > So I did try to build and run the radix test (please note my Ubuntu laptop > has only 4 cores and hyper-threading disabled). BTW it seems that > particular benchmark does not need read-write FS so I used ROFS): > > ./scripts/manifest_from_host.sh -w ../splash2-posix/kernels/radix/radix && > ./scripts/*build* fs=rofs --append-manifest -j4 > > Linux host 1 cpu: > > ./radix -p 1 -r4096 > > > Integer Radix Sort > > 262144 Keys > > 1 Processors > > Radix = 4096 > > Max key = 524288 > > > > PROCESS STATISTICS > > Total Rank Sort > > Proc Time Time Time > > 0 7335 2568 4765 > > > TIMING INFORMATION > > Start time : 1582652832386234 > > Initialization finish time : 1582652832444092 > > Overall finish time : 1582652832451427 > > Total time with initialization : 65193 > > Total time without initialization : 7335 > > > Linux host 2 cpus: > ./radix -p 2 -r4096 > > Integer Radix Sort > 262144 Keys > 2 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 4325 1571 2704 > > TIMING INFORMATION > Start time : 1582652821496771 > Initialization finish time : 1582652821531279 > Overall finish time : 1582652821535604 > Total time with initialization : 38833 > Total time without initialization : 4325 > > host 4 cpus: > ./radix -p 4 -r4096 > > Integer Radix Sort > 262144 Keys > 4 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 2599 1077 1470 > > TIMING INFORMATION > Start time : 1582653906150199 > Initialization finish time : 1582653906171932 > Overall finish time : 1582653906174531 > Total time with initialization : 24332 > Total time without initialization : 2599 > > > OSv 1 CPU > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 1 --block-device-cache writeback,aio=threads -e > '/radix -p 1 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 3.75 ms > Cmdline: /radix -p 1 -r4096 > > Integer Radix Sort > 262144 Keys > 1 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 6060 2002 4049 > > TIMING INFORMATION > Start time : 1582652845450708 > Initialization finish time : 1582652845500348 > Overall finish time : 1582652845506408 > Total time with initialization : 55700 > Total time without initialization : 6060 > > OSv 2 CPUs: > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 2 --block-device-cache writeback,aio=threads -e > '/radix -p 2 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 4.81 ms > Cmdline: /radix -p 2 -r4096 > > Integer Radix Sort > 262144 Keys > 2 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 5797 1702 4089 > > TIMING INFORMATION > Start time : 1582653305076852 > Initialization finish time : 1582653305129792 > Overall finish time : 1582653305135589 > Total time with initialization : 58737 > Total time without initialization : 5797 > > OSv 4 cpus > ./scripts/run.py -p qemu_microvm --qemu-path > /home/wkozaczuk/projects/qemu/bin/release/native/x86_64-softmmu/qemu-system-x86_64 > > --nics 0 --nogdb -m 64M -c 4 --block-device-cache writeback,aio=threads -e > '/radix -p 4 -r4096' > OSv v0.54.0-119-g4ee4b788 > Booted up in 5.26 ms > Cmdline: /radix -p 4 -r4096 > > Integer Radix Sort > 262144 Keys > 4 Processors > Radix = 4096 > Max key = 524288 > > > PROCESS STATISTICS > Total Rank Sort > Proc Time Time Time > 0 6498 2393 4099 > > TIMING INFORMATION > Start time : 1582653946823458 > Initialization finish time : 1582653946875522 > Overall finish time : 1582653946882020 > Total time with initialization : 58562 > Total time without initialization : 6498 > > > As you can see with single CPU the benchmark seems to be 10-15 % faster. > But with two and four CPUs OSv barely sees any improvements, whereas on > host the app runs 40% faster. So OSv does not seem to scale at all > (somebody mentioned it used to) so it would be nice to understand why. OSv > has many sophisticated tracing tools that can help here - > https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py > > Waldek > > BTW1. I tried to bump size of the matrix to something higher but with > -r8192 the app crashes on both Linux and OSv. > BTW2. It would be interestingly to compare OSv with Linux guest (vs host). > > On Tuesday, February 25, 2020 at 10:05:08 AM UTC-5, twee...@comcast.net > wrote: >> >> Thanks for the response! I will get this information to you after work >> with the few modifications you recommended! The application is essentially >> just testing CPU performance using multiprocessing. Nothing too fancy about >> it! The code I am using can be found at: >> >> https://www.github.com/ProfessorWest/splash2-posix >> >> In side of the kernels folder located at radix.c and I change the problem >> size to 16,777,206. >> >> If you happen to examine the code, do ignore the lacking cleanness of the >> code...we just smashed everything into one file for simplicity on our end. >> (Running the same code across all platforms being benchmarked). >> >> On Tuesday, February 25, 2020 at 8:52:48 AM UTC-5, Waldek Kozaczuk wrote: >>> >>> Hi, >>> >>> I am quite late to the party :-) Could you run OSv on single CPU with >>> verbose on (add -V to run.py) and send us the output so we can see a little >>> more what is happening. To disable networking you need to add '--nics=0' >>> (for all 50 options run.py supports run it with '--help'). I am not >>> familiar with that benchmark but I wonder if it needs read-write FS (ZFS in >>> OSv case), if not that you can build OSv images with read-only FS >>> (./scripts/build fs=rofs). Lastly, you can improve boot time by running OSv >>> on firecracker ( >>> https://github.com/cloudius-systems/osv/wiki/Running-OSv-on-Firecracker) >>> or on QEMU microvm (-p qemu_imcrovm - requires QEMU >= 4.1), with read-only >>> FS on both OSv should boot within 5ms, ZFS within 40ms). Last thing - >>> writing to console on OSv can be quite slow, I wonder how much this >>> benchmark does it. >>> >>> While I definitely agree with my colleague Nadav, where he essentially >>> says do not use OSv if the raw performance matters (database for example) >>> and Linux will beat it no matter what, OSv may have advantages in use cases >>> where pure performance does not matter (it still needs to be reasonable). I >>> think the best use cases for OSv are serverless or stateless apps >>> (microservices or web assembly) running on single CPU where all state >>> management is delegated to a remote persistent store (most custom-built >>> business apps are like that) and where high isolation matters. >>> >>> Relatedly, I think it might be more useful to think of OSv (and other >>> unikernels) as highly isolated processes. To that end, we still need to >>> optimize memory overhead (stacks for example) and improve virtio-fs support >>> (in this case to run the app on OSv you do not need full image, just kernel >>> to run a Linux app). >>> >>> Also, I think the lack of good tooling in unikernel space affects their >>> adoption. Compare it with docker - build, push, pull, run. OSv has its >>> equivalent - capstan - but at this point, we do not really have a registry >>> where one can pull the latest OSv kernel or push, pull images. Trying to >>> run an app on OSv is still quite painful to a business app developer - it >>> probably takes at least 30 minutes or so. >>> >>> Lastly, I think one of the main reasons for Docker adoption, was >>> repeatability (besides its fantastic ease of use) where one can create an >>> image and expect it to run almost the same way in production. Imagine you >>> can achieve that with OSv. >>> >>> Waldek >>> >>> On Tuesday, February 25, 2020 at 7:00:16 AM UTC-5, twee...@comcast.net >>> wrote: >>>> >>>> Very well explained. Thank you for that. That does make perfect sense >>>> as well. >>> >>> -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/86749b7a-c3da-45ad-ba3a-f3e4628e9197%40googlegroups.com.
