Hi, On Tuesday, August 6, 2019 at 12:43:18 PM UTC-4, Nadav Har'El wrote: > > On Tue, Aug 6, 2019 at 7:21 PM Joe Duarte <songof...@gmail.com > <javascript:>> wrote: > >> Hi all, >> >> Has anyone tested both OSv and IncludeOS <https://www.includeos.org/>, >> or evaluated them as alternatives? I'd be curious to know the results of >> your evaluation. >> > > I have no experience with IncludeOS, but I can comment on your OSv > questions: > > >> They have some things in common: clean-sheet unikernels written in modern >> C++, amenable to running C and C++ applications (though OSv seemed to >> emphasize Java applications early on). And I think both use the musl C >> library, though maybe in subtly different ways. >> >> The IncludeOS docs make no mention of Xen as far as I can tell, just KVM, >> VirtualBox, and VMWare. I'm not clear if it can run on bare metal, meaning >> without a hypervisor or any kind of virtualization – they talk about having >> their own bootloader and drivers. But true bare metal might be pointless >> for a unikernel, since the modal use case would be the cloud, and you'd >> want a bunch of unikernels running on each server, not just one – the only >> way to do that would be virtualization, with at least with a hypervisor. >> > > I don't know about IncludeOS, but the reason why OSv doesn't run on "bare > metal" is the drivers: When you run on bare metal, you suddenly need to be > able to drive all these dozens of different types of disks, network cards, > etc. This is how Linux ended up with a gazillion drivers, which took a huge > amount of effort to write. On a VM, there is a much smaller set of drivers > the kernel needs to support. > I would also 3 more things:
1) OSv by design has always been Linux compatible and implement the subset of Linux ABI (no fork, etc), I am not sure about IncludeOS; it used to be that OSv could only run Linux shared libraries so the apps had to be compiled from source as a shared library which was painful, with fairly recent changes OSv can run both PIEs and position dependant Linux binaries "as is" taken from Linux host as long as they do not use fork() or other unsupported symbols (most notably completely unmodified Java, Node.JS, Python); see this https://github.com/cloudius-systems/osv-apps#linux-applications-from-host 2) OSv supports more hypervisors - most notably Firecracker (see http://blog.osv.io/blog/2019/04/19/making-OSv-run-on-firecraker/) and recently hyperkit for Mac (if you have Mac and Docker for Mac than you have hyperkit as well) (see https://github.com/cloudius-systems/osv/commit/1e460f5985c54496795f5aebea413881bc43ebd3). The support of hyperkit is also about to be added to capstan (on a branch). 3) OSv used to run on ARM (aarch64); it does not boot anymore (somebody tried recently and posted on the group) but at least it compiles (that is also one of the reasons why Makefile is so big); see other recent email on the mailing list about resurrecting ARM support; I believe there is ongoing work to have IncludeOS support ARM > > >> >> I don't see Xen mentioned in the OSv build instructions either, which >> confuses me. I wonder why it's all KVM lately – I'd rather eliminate Linux >> from the stack completely. >> > > OSv does run on Xen, both local installations and Amazon (when Amazon > still used Xen). But you're right, KVM is more popular these days, and I > use it exclusively. > By the way, does Xen really "eliminate Linux from the stack"? Doesn't it > have a Linux dom0 as well? > There is this "Containers vs Unikernels performance comparison" study on Xen when OSv is faster than containers in some cases - https://biblio.ugent.be/publication/8582433/file/8582438.pdf > > >> >> Actually, it would be interesting to see some testing of OSv on KVM vs. >> Xen – maybe I'll rig something up. In any case, does anyone have any >> thoughts on OSv vs. IncludeOS? >> > I also did some causal measurements - https://groups.google.com/d/msg/osv-dev/TwMwLS6TIFo/EFIfrpkmAgAJ. > >> Relatedly, how fast does OSv boot? IncludeOS boots in tens of >> milliseconds apparently. That's better than Intel's Clear Linux containers, >> but I don't know what's typical for a unikernel OS. >> > > OSv can also boot in "tens of milliseconds" - Waldek (CCed) can provide > some recent boot times he measured. But it very much depends on the variant > of the hypervisor, some classic qemu setups take a long time (still below 1 > second, but relatively long) to start even if you boot a tiny kernel. > Yeah, I was able to boot ROFS image of OSv in 5-6ms on my 5-years old laptop using firecracker which I think is the faster hypervisor OSv can boot on> Firecracker itself takes 5 ms to start. See the blog above for details. > > >> >> Cheers, >> >> Joe >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "OSv Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to osv...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/osv-dev/9b7ae4e3-87d5-4e22-986b-3bf4f92d5192%40googlegroups.com >> >> <https://groups.google.com/d/msgid/osv-dev/9b7ae4e3-87d5-4e22-986b-3bf4f92d5192%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/37445c35-b354-4194-8366-b7c275f4b97e%40googlegroups.com.
