Hello everyone. It's already almost July! Which means I've been working on replacing the build daemon for about a month now. So it seems like a good time for an update.
So far, I've replaced the register-path procedure in store.scm, which means we now have some support for using sqlite (via guile-sqlite3 and some utility macros) to access and modify the state database. We can also now perform deduplication on individual store items. That's pretty much all that's been produced so far code-wise (that you can see on the guile-daemon branch at savannah, anyway), but that's okay, because I've also gained a lot of knowledge. Recently I've been trying to understand the details of how nix/libstore/build.cc operates, and this has taught me the value of trying to keep procedures no longer than one screen's height. It's also taught me the value of immediately writing any insights I have down, since the details, even of a single procedure I'm reading, are well past the point where I can keep them all in my head. Right now I'm working on implementing the building of derivations. There are two main difficulties. The first is that the details of this process (which files are included in the chroot by default, permissions, ownership, environment variables, etc) are all in C++. I have discovered that the picture I had of C++ going into this was, unfortunately, pretty accurate. Here's a great example: reading through globals.cc, try to figure out what "settings" refers to at each occurrence (I had a long rant prepared, but that about sums it up). The second, less obnoxious difficulty is that of code duplication. A lot of the functionality of guix duplicates functionality from the daemon, which was one of the motivations for the rewrite in the first place, but the duplicated functionality is still built on top of remote procedure calls, especially for accessing the database. For example, topologically-sorted in store.scm is exactly what I wanted in order to implement support for exportReferencesGraph, but it uses an RPC to get the list of references of a store item. At the same time, there are still many procedures that are what I need (derivations.scm is lovely), so it's still worth checking if it's already implemented, but often a miss. My difficulties with Scheme are pretty minor, and the Guile reference has been very useful. It took me awhile to realize that dynamic scoping was very much not a thing in Scheme, as opposed to the sort-of-a-thing state it's in in Common Lisp (I had a hack set up to shadow a hashing function so that I could get the size of the thing being hashed as a byproduct, but discovered it didn't work that way and ended up just making a wrapper port that counted what went through it). I do get the feeling when writing Scheme that set! is an unholy word, which makes for a fun challenge, especially where iteration is involved. I want to finally figure out how monads are supposed to be used, though. Perhaps I should just read through the implementation in guix. I think I have the build environment just about figured out (I've been writing a detailed description of the process, which I plan to post once it's finished), and can get to implementing derivation-building soon. I'll start with assuming that a chroot is used, then extend it to work with non-chroot builds later. So yeah, that's the state of the project so far. - reepca