Hi, On Fri, Nov 10, 2017 at 12:26 AM, Waldek Kozaczuk <jwkozac...@gmail.com> wrote:
> I found this article on OSv blog - http://blog.osv.io/blog/ > 2017/06/12/serverless-computing-with-OSv/ - very interesting and > inspiring. > Thanks :-) > Besides the claim that OSv is a perfect platform for "serverless" it made > me think that "stateless" does not apply only to "serverless" but also to > many long-lived apps running on regular VMs. In my experience I have > developed many apps reading/writing from/to external storage (MongoDB in > Mongo Labs, DynamoDB, S3, RDS on AWS, etc) and communicating over HTTP, > SNS, SQS with outside world that would not need to store anything locally > except for logs (even that can be set up to be pushed to a remote logging > service like Loggly (https://www.loggly.com/docs/java-logback/)). > I agree. When we set out to develop OSv we talked a lot about "stateless", but then we went ahead and chose the most complex filesystem out there, ZFS, for keeping, hmm, state. :-) > But more to the point: if one wanted to use OSv for serverless or > stateless service I think that ramfs instead of zfs would be the best file > system to use. It would allow for fastest boot time as well as disk I/O (am > I right here?). As I understand the existing ramfs implementation in OSv > has some limitations like size of the image related to how kernel get > decompressed, etc. Also comes with unnecessary linked-in zfs code and > possibly dependant libraries. Also it probably unnecessarily probes storage > devices which means the storage drivers code could also be removed. > > Could someone please elaborate/summarize the status of current ramfs > implementation? What features does it have? Please see some questions below: > > - Is it read-only? > - Is it faster than zfs? > - How does it utilize memory relative to the application? > - What limits does it have > - Does size of the image affect boot time? > > Also what would be the list of things to be enhanced/fixed in OSv to make > ramfs images more "production-ready" and applicable to bigger apps (running > on JVM) in priority order? > I see you already found the two issues relevant to your questions are https://github.com/cloudius-systems/osv/issues/195 ("Allow images without ZFS") and https://github.com/cloudius-systems/osv/pull/646 (patches for "read only filesystem"). The current ramfs "ram disk" works like this: 1. It is not read-only - you can create new files or modify existing files, and the files are saved in RAM (currently each file is one contiguous malloc()). 2. During boot, we unpack an ad-hoc bunch of files tacked on to the end of the kernel and write it to the ram disk. See fs/vfs/main.cc, unpack_bootfs(). Some significant drawbacks of this approach are: 1. Boot is slowed down as we uncompress and unpack *ALL* the files from the disk image into the ram disk. In large projects, e.g., Java, it makes sense to load the files on demand, not on boot like this. Moreover, the loading from disk during boot is done in a slower method than disk reads after boot. 2. More memory is used. Not only are all the files loaded into memory (even if we never want to read all of them), if I remember correctly they are actually loaded twice - we have the entire file pack as part of the kernel in memory, and then we make an additional copy when we write() the content into the ramfs filesystem in unpack_boofs(). We may even have a third copy (!) in memory when files are mmap'ed (we do this for shared objects, and I think Java uses mmap to read class files?). A better solution could be to not have a read-write ramdisk at all, but rather a simple read-only filesystem on the disk image where we only read file blocks as they are needed, and also cache them somehow for the next reads. This is what the proposed patches were about https://github.com/cloudius-systems/osv/pull/646 (which you already found). I reviewed those patches in this thread: https://groups.google.com/forum/#!searchin/osv-dev/readonly/osv-dev/DYSH4_NqyMg/YFJxM9NKBgAJ James Root came in that thread back with a new patch, but sadly it appears that nobody ever reviewed it, and James never complained about forgetting this code. At this point I don't really remember if there was anything wrong in Jame's last patch - it might be in good working order and you can try it (and patch the build script to use it instead of ramfs). If you're interested in reviving it, it might be very close to being complete! > > I experimented with ramfs a little and I was able to build and run ramfs > image with httpserver module. For some reason I failed to create and run an > image with java app possibly due to the image size limit - I think the > build process at some point got stuck. > I used openjdk8 compact 3 with java beans which I think is under 50MB > uncompressed. I thought I found an example of someone successfully building > and running spring boot on ramfs. > > I once tried "scripts/build ... fs=ramfs" with fairly large images and it did work once you change lzkernel_base (the Makefile now print instructions how to do that). Can you show me the exact command you used? Also, when you use fs=ramfs, how can the build process itself get stuck? In that case, we don't need to *run* OSv (we only run OSv during build to populate the ZFS filesystem), so what gets stuck? > Also I found more information here: > > - https://groups.google.com/forum/#!searchin/osv-dev/zfs$ > 20less%7Csort:date/osv-dev/dK3jDR-TYyk/069QUCJeCQAJ > > <https://groups.google.com/forum/#!searchin/osv-dev/zfs$20less%7Csort:date/osv-dev/dK3jDR-TYyk/069QUCJeCQAJ> > - https://github.com/cloudius-systems/osv/pull/646 > - https://github.com/cloudius-systems/osv/commit/ > 714cfd97e48ba5fecba224430762d2c8aac7c0eb > > <https://github.com/cloudius-systems/osv/commit/714cfd97e48ba5fecba224430762d2c8aac7c0eb> > - https://github.com/cloudius-systems/osv/issues/195) > > Waldek > > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
