Re: [Nix-dev] How to get rid of systemd
Ertugrul Söylemez ert...@gmx.de skribis:
I would prefer and do propose an algebraic solution (view services as a
toolbox from which you can pick the services you want and compose them
together). A services configuration then might look like this:
services =
bitlbee { port = 1; stateDir = /var/lib/bitlbee1; }
bitlbee { port = 10001; stateDir = /var/lib/bitlbee2; }
nginx { httpConfig = ...; }
postfix { postMasterAlias = blah; }
The big difference is that now services are first class values and form
a monoid under composition, which brings a number of advantages:
FWIW, Guix uses an approach along these lines: the ‘bitlbee-service’
function, for instance, returns a ‘service’ object as a monadic value
(see http://www.gnu.org/software/guix/manual/guix.html#Defining-Services
for details.)
Unlike in NixOS, the service implementation doesn’t have access to the
rest of the system configuration, which I think is a good thing, as you
note.
What’s unsatisfying (and thus subject to change) is that
‘operating-system’ objects (which are pure declarations) end up with
monadic values in their ‘services’ field. That makes it inconvenient
to, say, filter items from that list, or to tweak their configuration,
because one first needs to bind them. That said, it’s probably not a
problem for Nix, because every Nix function is really a function in what
we call the “store monad”.
In your example, what would ‘bitlbee’, ‘nginx’ etc. return exactly? An
attribute set describing the service?
Anyway, I’m interested in following this discussion.
Thanks,
Ludo’.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] How to get rid of systemd (was: Modifying the init system (introducing S6 supervision suite))
Hi Alexander, We can use a monoid system to construct configurations, but the socket activation standard for example, is centered on optimizing the activation script itself. What are your thoughts on the activation script? I have no concrete thoughts at this point, just throwing around ideas. Socket activation, for those daemons that support it, should be something sufficiently transparent from the point of view of the configuration author. I can easily see that using systemd might be overkill and way too complex for a container-based system, so I think there is something to research here. The trouble with systemd for this particular purpose is that it wants everything to have an identity, so basically you need to map a compositional system to a system of global variables. In many ways you go through the same trouble when compiling a functional language to an imperative architecture. I also think upgrading services doesn't really seem to work in systemd or in the current setup. Similarly to how, when using a distributed docker setup, we have a load-balancer that atomically switches instances, we should not need to take down the old instance service before the new one is created. Rather, the upgraded service should be started in isolation (using containers), and after ensuring that it has started, is working etc, then should the switch happen using namespaces, routing entries etc. This should be the preferred way to deal with non-transactional services (i.e. non-database stuff). That's pretty much the way I want upgrading to work. The idea that the old service must be stopped before the new one is started is based on what I think is a conflation of concerns, namely treating the whole service state as global state. Yes, that's exactly what I want to get rid of. Ironically that's something systemd does very well. Systemd gets the stuff right that we don't want in the first place. =) Instead a lot of services can be treated as a sequence of isolated containers, and a small set of load-balanced, mutable service entry points. Namespace magic can make a lot of previously global state, local, such as ports, pid-files, log files etc. For these kinds of services, you don't need graceful shutdown, which is a great simplification, similarly to what docker typically provides. I feel that a great design in this area should blend well with distributing services across machines, failover, etc. which are concerns that systemd doesn't start to cover. That's a very good summary of why I want to implement my proposal. Let me see if I can get a working prototype. Greets, Ertugrul ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] How to get rid of systemd
Hi Ludovic, FWIW, Guix uses an approach along these lines: the ‘bitlbee-service’ function, for instance, returns a ‘service’ object as a monadic value (see http://www.gnu.org/software/guix/manual/guix.html#Defining-Services for details.) The way I understand the section on the store monad is that it's really a state monad with a view on the current store (they should really show the underlying types to make this easier to understand). That's not related to the way they define services, but rather just how the store itself is used. Nix has the store built into the language. Side effects happen to build store paths and, once built, they appear like regular immutable values within the language. It's convenient, but makes the assumption that build outputs depend solely on their inputs, which unfortunately, as we know, is not always true at this point. However, the way they define services seems indeed a lot better than what we currently have. In part they already have what I think we should have (functional services), but with no algebraic structure behind it other than the store monad. Unlike in NixOS, the service implementation doesn’t have access to the rest of the system configuration, which I think is a good thing, as you note. Absolutely. Everything they depend on should be an argument. What’s unsatisfying (and thus subject to change) is that ‘operating-system’ objects (which are pure declarations) end up with monadic values in their ‘services’ field. That makes it inconvenient to, say, filter items from that list, or to tweak their configuration, because one first needs to bind them. That said, it’s probably not a problem for Nix, because every Nix function is really a function in what we call the “store monad”. You could think of it that way, if you wanted. I prefer to think of what Nix does as the same magic that functional languages do with other resources like memory (alloc and GC). In your example, what would ‘bitlbee’, ‘nginx’ etc. return exactly? An attribute set describing the service? As a first approximation there would be a Service monoid, which would combine certain typical attributes about a service, including startup scripts, required bind-mounts, required system resources and if really necessary shutdown scripts (most programs should shutdown properly when the container goes down). Service : Mon Its identity idS would be the empty service, which you can think of as an empty group of processes with no mounts, no resources, etc. The Bitlbee service would be constructed through a regular function that takes the typical bitlbee options. Nothing special here: bitlbee : BitlbeeConf - Service The nginx function is more interesting. It involves a second monoid, the monoid of web services (identity idW): WebService : Mon Then nginx is a monoid morphism, nginx : WebService - Service that is a function with additional structure, in particular: nginx x ◇ nginx y ≡ nginx (x ◇ y) nginx idW ≡ idS The most important part is to get the equivalence `≡` relation right, i.e. when do we regard two services as equivalent? This requires a lot of careful thought, because services in the real world are messy and effectful. Example: 1. One nginx instance serves two websites x and y. 2. Load balancer scenario: Three nginx instances. One serves x, one serves y, the other redirects to the correct instance depending on the request. 3. Mapping identities: A hundred load-balanced nginx instances, one of them serving x, a second one serving y and all others serving nothing at all (returning 404 unconditionally). These three examples really describe the same service. One may be a more efficient implementation, but they should be equivalent with respect to `≡`, because they are indistinguishable from the outside other than in exceptional cases (case 3 is more likely to run out of memory than case 1). That enables us to use equational reasoning for correctness. It also enables us to apply automatic semantics-preserving optimisations. Example: The host may be serving two sites. At the same time a system monitoring daemon might generate health reports periodically that are also served via HTTP. Such a configuration might look like: nginxStatic site1.host /var/www/site1 nginxStatic site2.host /var/www/site2 systemHealth health.host A semantics-preserving optimistation (which is really not an optimistation, but the way web services are combined) would collapse all of that to a single instance of nginx and one entry in the crontab for the health reporter. I hope this gives a first impression of how everything would fit together. Greets, Ertugrul ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] FOSDEM planning Hangout
Is anyone planning on giving a talk? If so, who? On Wed, Dec 17, 2014 at 1:47 PM, Wout Mertens wout.mert...@gmail.com wrote: Hi all, We got a table at FOSDEM so we'll need volunteers to man the table and we need to think about what we'll show/do. Swag sales would be nice too. If you'd like to help with the planning, please fill out the doodle : http://doodle.com/br47eeidzgrangqm Cheers, Wout. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Haskell NG
I'm trying to think of a way of selling my oop library [...] What OOP library are you talking about? Is it available for me to view online? -Charles On Thu, Dec 18, 2014 at 12:38 PM, rocon...@theorem.ca wrote: On Thu, 18 Dec 2014, Peter Simons wrote: Hi Russel, 2) Haskell packages support 'deepOverride'. How would you feel about using my overrideScope functionality from my haskellPackagesFixpoint branch instead? the first version of my re-factored Haskell code defined deep overrides exactly the way you did it. Check out the definePackage function in [1]. You'll find that I studied your work thoroughly. My impressions are: 1. overrideScope and deepOverride achieve the same goal. I discovered no case where one function worked but the other one didn't. 2. deepOverride is a standard function every callPackage derivation supports. overrideScope, on the other hand, is not. 3. deepOverride is implemented in a simple generic algorithm (for appropriate definitions of the term simple) that requires no special support from the haskellPackages record to function. overrideScope, on the other hand, needs the magic nixClass attribute (__unfix__ in my code) to build a stack of extension functions. Ultimately, I felt that I should stick to the standard deepOverride mechanism mostly due to Occam's razor'ish reasoning: I'd rather not introduce a new sophisticated tying-the-knot mechanism to haskellPackages just to do the same thing that deepOverride can do already. Does that make sense? That is fair reasoning. I will add that I expect overrideScope to run exponentially faster than deepOverride in worst case. A similar change to replace-dependency took the evaluation time for replacing bash in a NixOS system from 65 minutes to instantaneous https://githum.com/NixOS/nixpkgs/pull/4313. Replacing bash in a NixOS system is probably *the* worst case scenario because bash so deep in the dependency graph. Until deepOverride starts taking appreciable time, the efficencies of overrideScope is probably not yet a pursuasive argument. So please keep an eye out for long evaluation times when using deepOverride as you continue your work. :) Haskell applications have deeper and more fine grained dependencies than regular applications so you might be more likely to run into evaluation time problems than others. What I really need to do is convince Eelco Dolstra and Michael Raskin that semantics of overriding packages and semantics of object oriented programmed coincide and they should adopt my oop library for implementing the overriding mechanism. I'm trying to think of a way of selling my oop library to them that is a little more light weight that flying to europe to give them a presenation. Maybe I should use the Ocaml package set as a testbed to demonstrate overrideScope. -- Russell O'Connor http://r6.ca/ ``All talk about `theft,''' the general counsel of the American Graphophone Company wrote, ``is the merest claptrap, for there exists no property in ideas musical, literary or artistic, except as defined by statute.'' ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] How to get rid of systemd (was: Modifying the init system (introducing S6 supervision suite))
On Sun, Dec 28, 2014 at 11:13 AM, Ertugrul Söylemez ert...@gmx.de wrote: Instead a lot of services can be treated as a sequence of isolated containers, and a small set of load-balanced, mutable service entry points. Namespace magic can make a lot of previously global state, local, such as ports, pid-files, log files etc. For these kinds of services, you don't need graceful shutdown, which is a great simplification, similarly to what docker typically provides. I feel that a great design in this area should blend well with distributing services across machines, failover, etc. which are concerns that systemd doesn't start to cover. That's a very good summary of why I want to implement my proposal. Let me see if I can get a working prototype. This sounds a lot like Disnix! How would it be similar? How would it be different? -Colin ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] extension error?
Was trying this example at http://wiki.ocharles.org.uk/Nix#how-do-i-use-cabal2nix-for-local-projects and I when I enter nix-shell I an error $ nix-shell error: anonymous function at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/top-level/haskell-defaults.nix:230:5 called with unexpected argument `extension', at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/customisation.nix:58 Anybody seen this error or have any insight to share? Thanks, Tim ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] extension error?
Hi Tim, It looks like you're using the latest from the stabe 14.04 release, which doesn't support the `extension` argument: https://github.com/NixOS/nixpkgs/blob/release-14.04/pkgs/top-level/haskell-defaults.nix#L230-237 Compare that to master: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/haskell-defaults.nix#L148-155 You can see your current channel running: sudo nix-channel --list I would suggest updating your nixos channel to nixos-14.12 or nixos-unstable: sudo nix-channel --remove nixos sudo nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixos # ...or... sudo nix-channel --add https://nixos.org/channels/nixpkgs-14.12 nixos (The above commands are assuming that you're using NixOS, as opposed to using Nix standalone.) Let me know if that helps! -Charles On Mon, Dec 29, 2014 at 12:48 AM, Tim Sears t...@timsears.com wrote: Was trying this example at http://wiki.ocharles.org.uk/Nix#how-do-i-use-cabal2nix-for-local-projects and I when I enter nix-shell I an error $ nix-shell error: anonymous function at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/top-level/haskell-defaults.nix:230:5 called with unexpected argument `extension', at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/customisation.nix:58 Anybody seen this error or have any insight to share? Thanks, Tim ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] extension error?
Oops, minor correction (s/nixpkgs/nixos): sudo nix-channel --remove nixos sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos # ...or... sudo nix-channel --add https://nixos.org/channels/nixos-14.12 nixos -Charles On Mon, Dec 29, 2014 at 1:22 AM, Charles Strahan charles.c.stra...@gmail.com wrote: Hi Tim, It looks like you're using the latest from the stabe 14.04 release, which doesn't support the `extension` argument: https://github.com/NixOS/nixpkgs/blob/release-14.04/pkgs/top-level/haskell-defaults.nix#L230-237 Compare that to master: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/haskell-defaults.nix#L148-155 You can see your current channel running: sudo nix-channel --list I would suggest updating your nixos channel to nixos-14.12 or nixos-unstable: sudo nix-channel --remove nixos sudo nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixos # ...or... sudo nix-channel --add https://nixos.org/channels/nixpkgs-14.12 nixos (The above commands are assuming that you're using NixOS, as opposed to using Nix standalone.) Let me know if that helps! -Charles On Mon, Dec 29, 2014 at 12:48 AM, Tim Sears t...@timsears.com wrote: Was trying this example at http://wiki.ocharles.org.uk/Nix#how-do-i-use-cabal2nix-for-local-projects and I when I enter nix-shell I an error $ nix-shell error: anonymous function at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/top-level/haskell-defaults.nix:230:5 called with unexpected argument `extension', at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/lib/customisation.nix:58 Anybody seen this error or have any insight to share? Thanks, Tim ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] How to get rid of systemd (was: Modifying the init system (introducing S6 supervision suite))
Hi Colin, This sounds a lot like Disnix! How would it be similar? How would it be different? Disnix and NixOps are machine-oriented. You have a network of machines with configurations. The goal is a service-oriented architecture. Every service itself is a machine, and those machines can be combined to larger machines. Greets, Ertugrul ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
