Re: [Nix-dev] Crontab and mail / ssmtp troubles
Hi, SH=/run/current-system/sw/bin/sh MAILTO=d...@gtf.org PATH=/run/current-system/sw/sbin 10 02 * * * echo test that $PATH doesn't look right. Shouldn't that directory list allow the cron daemon to resolve 'sendmail' to '/var/setuid-wrappers/sendmail'? Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Proposal: Standard installation procedure
Hi there fellow Nixers, I'd like to propose an installation standard that would make it a lot easier and more reliable to deploy NixOS to virtual and container-based systems, including qemu, LXC and Docker. As a nice side effect it would greatly simplify most NixOS tools, including nixos-install and nixos-rebuild. First of all: Yes, I'm aware of Nixops, and I don't want to use it for various reasons. Currently I'm using my own variant of make-system-tarball.nix (really just to avoid the expensive and in my case unnecessary compression step) together with a bunch of Makefiles. It's a mess and the sooner I can dispense with it, the better. Proposal The basic idea is this: To build a system, you use a standard function from Nixpkgs and pass it a configuration module. The result is a derivation that represents a complete self-contained system. The initialisation program ($out/init) is not systemd directly, but a script that first checks whether the filesystem has already been initialised. If not, it sets up all necessary directories (like /etc) and then starts systemd as normal. In other words a container-based NixOS system is really just a Nix store, nothing more. It doesn't need to contain any additional files or directories and building it does not require us to understand the complexity of nixos-install, because it will all be moved to the system initialisation phase. Ideally we should factor out the initialisation into a separate script $out/pre-init that accepts a prefix argument. Then $out/init would simply look like this: #! /bin/sh /nix/store/abcxyz-nixos/pre-init --prefix=/ exec /nix/store/abc123-systemd/bin/systemd If we can also move pre-boot initialisation (like installing the boot loader) into the system itself (say, $out/pre-boot), then we could build entire virtual machines completely without relying on a hypervisor and the complexity of having to communicate with the installation process. Just set up the disk image, install the system and use the resulting $out/pre-boot script with appropriate override arguments: out=`nix-build my-machine.nix` # ... copy the closure ... $out/pre-boot --override-grub-device=disk.img This would greatly simplify the implementation of nixos-install (just build the system and invoke the pre-boot script without overrides), but more importantly it would provide a very simple and standardised installation procedure for virtual machines and containers that we can rely on. How to use it = Create a file that defines the system derivation (build-system.nix contains a currently hypothetical function that we would implement as part of this proposal): # my-machine.nix let cfg = {config, pkgs, ...}: ...; in import nixos/build-system.nix cfg To build the system, we simply use nix-build: % nix-build my-machine.nix /nix/store/abcxyz-nixos It is now very easy to create the filesystem for the container. Just copy the full closure of the derivation and point the container driver to the proper initialisation script (or link /sbin/init to it). Liberated system environments = Another advantage of this proposal is that we can easily put those builds in a local user environment and copy only the closure to the containers (I use *non-remote* containers a lot). We can even update them off-line. That prevents a lot of duplication, because the containers can be garbage-collected after every change and will contain only the bare minimum required for operation. The actual history is saved on a separate management machine and uses sharing where appropriate. In fact we could switch multiple VMs on multiple hosts almost fully transactionally. Build and upload everything, then tell every machine to switch. Finally update the environment and collect garbage on the target machines. Updating the local system would follow the same scheme, so even nixos-rebuild would become very simple. Final remarks = The main point of this is that it becomes really easy to reliably implement something like a custom nixos-install, a remote nixos-rebuild, Nixops or even make-system-tarball.nix. No deep knowledge is required and the instructions and relevant information can be put on a fairly simple wiki page. What do you think? I would love to provide a prototype, but I really need help with this one. Greets, Ertugrul -- Ertugrul Söylemez ert...@gmx.de Key-ID: F9B5212A Fingerprint: 8D89 7AC9 21CF F868 F074 9058 30CB D56F F9B5 212A ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Emacs daemon as a user systemd service in NixOS
On Sun, Oct 12, 2014 at 8:17 PM, Eelco Dolstra eelco.dols...@logicblox.com wrote: You can define user services using the option systemd.user.services, where you should be able to say: systemd.user.services.emacs = { ... environment.GTK_DATA_PREFIX = config.system.path; }; thank you very much. Here is my service description now: systemd.user.services.emacs = { description = Emacs: the extensible, self-documenting text editor; environment.GTK_DATA_PREFIX = config.system.path; environment.SSH_AUTH_SOCK = %t/keyring/ssh; environment.GTK_PATH= ${config.system.path}/lib/gtk-3.0:${config.system.path}/lib/gtk-2.0; serviceConfig = { Type = forking; ExecStart = ${emacs-24}/bin/emacs --daemon; ExecStop = ${emacs-24}/bin/emacsclient --eval (kill-emacs); Restart = always; }; wantedBy = [ default.target ]; }; My next problem is that DBUS_SESSION_BUS_ADDRESS is not set in this emacs service. Do you know how I could set it? Bonus question: would nixos be interested in this file? Should I push it in nixpkgs/nixos/modules (or somewhere else)? Thanks -- Damien Cassou http://damiencassou.seasidehosting.st Success is the ability to go from one failure to another without losing enthusiasm. Winston Churchill ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Emacs daemon as a user systemd service in NixOS
On 16/10/2014 13:23, Damien Cassou wrote: thank you very much. Here is my service description now: systemd.user.services.emacs = { description = Emacs: the extensible, self-documenting text editor; environment.GTK_DATA_PREFIX = config.system.path; environment.SSH_AUTH_SOCK = %t/keyring/ssh; environment.GTK_PATH= ${config.system.path}/lib/gtk-3.0:${config.system.path}/lib/gtk-2.0; serviceConfig = { Type = forking; ExecStart = ${emacs-24}/bin/emacs --daemon; ExecStop = ${emacs-24}/bin/emacsclient --eval (kill-emacs); Restart = always; }; wantedBy = [ default.target ]; }; My next problem is that DBUS_SESSION_BUS_ADDRESSis not set in this emacs service. Do you know how I could set it? Try https://wiki.archlinux.org/index.php/Systemd/User . There's an example on how to set it. Bonus question: would nixos be interested in this file? Should I push it in nixpkgs/nixos/modules (or somewhere else)? It's a nixos service like any other, IMO. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Our policy for upgrading haskellPackages
On 10/15/2014 08:27 PM, John Wiegley wrote: This message is a follow-on to a discuss Peter and I were having on GitHub, since I believe it is of more general interest: Peter Simons writes: Generally speaking, the two goals 1. have recent versions of all major Haskell packages and 2. all Haskell packages should compile are contradictory. The 2.6.x version of network has been out there since Tue Sep 2 18:14:36 UTC 2014, i.e. for more than 1,5 months. Since 2.5.x and 2.6.x have incompatible APIs, many package authors don't bother supporting the old version: they update their packages to compile against 2.6.x and never look back. Now, in that situation, we must switch to 2.6.x eventually, because network 2.5.x cannot compile many available updates. At the same time, the switch to 2.6.x breaks the packages of all those people who didn't update their libraries. So what are we supposed to do? Forgo the available updates to keep a stable system or update at the cost of breaking packages that are sort-of unmaintained? I try to keep as many packages building as possible, and getting those ~200 updates into master was a major effort for me, i.e. I worked on those commits several hours per day for the better part of a week. Even with all that effort spent, however, I cannot remedy the fundamental conflict of interest between a system that's up-to-date and a system that's stable. At some point, I just push whatever I have come up with and I rely on other people, like yourself, to help finding the best balance between those two contradictory goals. Hi Peter, First, let me state how much I appreciate the contribution you're making to nixpkgs. Its support of Haskell is superb, and that is in large part due to your time and effort. My hope is to support you as best I can, and not to criticize your efforts in any way. You are exactly right that we have a tension between those two goals. I can think of two things that might be done to remedy this, and perhaps make updates to master more smooth: 1. We keep a dedicated branch, haskell-updates, to which only your Hackage updates get pushed, or fixes to those updates. I will personally pull and rebuild this branch every day on my machine, just as I presently rebuild master nearly every day -- compiling more than 2,000 packages that I keep locally updated through --leq. I (and hopefully others) will help to discover which packages can be fixed by inserting references to older packages, which requires patches, and which must truly be marked as broken until the maintainer of that package can be contacted. Further, I'll help you to maintain a list of outstanding broken packages, and see what can be done to make sure this list decreases over time. This already exists, in part: the haskell-updates job at [1]. I say in part because it: 1. Only builds for x86_64 (so the recent vector failure was not caught) 2. It doesn't make clear what packages were marked as broken. 3. Once it builds, it goes into master (usually) without any further review: partially a good thing because we get updates fast. So I think a better plan would be instead to have a separate branch (not haskell-updates) which contains only Haskell updates that result in some packages getting broken. Everything else just goes into master (through haskell-updates or whatever) as it is doing today. Say, keep the changes in this pending-broken branch for two days behind master: any interested party has a chance to jump in and say ‘oh, I know how to make this update not break anything’ or ‘I'll take care of specifying proper versions for these packages’. Obviously what would go onto this branch is at updaters discretion: no point delaying update of really popular package because one really unpopular package breaks. This approach of only putting up breaking updates for public review is, I feel, a better solution than requiring every Haskell update to wait in a branch for a day or two for no particular reason. 2. The second option is to create a new haskellPackages set, called 'stackage'. The Stackage maintainers already do a lot of the work implied by #1, ensuring that every package within the Stackage set can build together. Further, they only upgrade a package once they've either created a patch, or worked with upstream to update the package. Of course, the downside to this is: - less frequent updates of packages - a smaller available package set - life-draining maintenance of a mostly parallel package set The upside being that all patching/curating work is done for us, likely for as long as FP Complete keeps funding people to maintain Stackage. This should be pretty easy to maintain on nix side as we should be able to hackage4nix/cabal2nix all of hackage and have it Just Work™. Personally I'd like to see it go
Re: [Nix-dev] Crontab and mail / ssmtp troubles
On Thu, Oct 16, 2014, at 04:39 AM, Peter Simons wrote: Hi, SH=/run/current-system/sw/bin/sh MAILTO=d...@gtf.org PATH=/run/current-system/sw/sbin 10 02 * * * echo test that $PATH doesn't look right. Shouldn't that directory list allow the cron daemon to resolve 'sendmail' to '/var/setuid-wrappers/sendmail'? Well, I'm a bit naive I'm afraid -- 'which sendmail' pointed at /run/current-system/sw/sbin, and that worked from the command-line, so I put it in there. I didn't know about /var/setuid-wrappers. Maybe that's the thing to use? I see it mentioned in the NixOS manual. I'll try that. Thanks -- I'm learning quite a bit from this! --Michael Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] how to write nix expressions for packages that have plugins
Hi, I'm trying to write a nix expression for a plugin for a package. I'm running into a problem where the package doesn't know where to look for the new plugin. The original package stores all of it's plugins in /nix/store/xxx-package/lib/packagename/{plugin1,plugin2,mycoolplugin}.so. It expects all of it's plugins to be there. There doesn't appear to be any way of telling the program that it needs to look for plugins in other directories. How is this type of problem usually solved in other nix packages? The package/plugin combination in question is fcitx and fcitx-anthy. It's an input method editor commonly used to input Chinese, Japanese, and Korean (among other languages). I raised a question about this package/plugin combination on github, but I am also interested in how it's solved in general. https://github.com/NixOS/nixpkgs/issues/4550 I have also raised a question on the fcitx issue tracker for anyone curious: https://github.com/fcitx/fcitx/issues/179 Finally, in case anyone wants to look at the the original nix-expressions, here they are: fcitx: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/default.nix fcitx-anthy: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/fcitx-anthy.nix Thanks, Dennis ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] how to write nix expressions for packages that have plugins
There are two methods I've seen: 1. patch the system to support inspecting multiple directories for plugins. Either specified using an environment variable or command line option. 2. create a merge nix package: A package which takes the application plus all plugins requested and builds the directory layout the application expects with all the plugins linked into the right folder. This package is then added to the environment of the user. Cheers, Corey -Corey O'Connor coreyocon...@gmail.com http://corebotllc.com/ On Thu, Oct 16, 2014 at 8:17 AM, cdep.illab...@gmail.com cdep.illab...@gmail.com wrote: Hi, I'm trying to write a nix expression for a plugin for a package. I'm running into a problem where the package doesn't know where to look for the new plugin. The original package stores all of it's plugins in /nix/store/xxx-package/lib/packagename/{plugin1,plugin2,mycoolplugin}.so. It expects all of it's plugins to be there. There doesn't appear to be any way of telling the program that it needs to look for plugins in other directories. How is this type of problem usually solved in other nix packages? The package/plugin combination in question is fcitx and fcitx-anthy. It's an input method editor commonly used to input Chinese, Japanese, and Korean (among other languages). I raised a question about this package/plugin combination on github, but I am also interested in how it's solved in general. https://github.com/NixOS/nixpkgs/issues/4550 I have also raised a question on the fcitx issue tracker for anyone curious: https://github.com/fcitx/fcitx/issues/179 Finally, in case anyone wants to look at the the original nix-expressions, here they are: fcitx: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/default.nix fcitx-anthy: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/fcitx-anthy.nix Thanks, Dennis ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Broken ghc for OS X on cache.nixos.org
Hello all, In an effort to get our team to start using nix, I've setup a project of ours to be built with nix. One obstacle I've run into is that most of the team (pretty much everyone except me) is using OS X, so I've been trying to overcome the OS X ghc being broken in the nix cache [1]. I've gone ahead and built a new ghc binary on OS X and created a cache containing it and other Haskell packages. I thought everything was going great until I tried to actually use it by running $ nix-shell shell.nix --option binary-caches http://mycache --command 'cabal configure --enable-tests' This still downloads the ghc package from http://cache.nixos.org, which is what I'm trying to avoid. I used wireshark to see what is going on, and it appears that my cache is never checked, nix just goes and starts downloading from cache.nixos.org. My guess is that this is because of the MANIFEST from the nixpkgs-unstable channel. Is that correct? If so, how can I override that? Thanks, Rich PS It would be great if this weren't a problem. I was told this issue should be getting resolved soon since there is a new OS X build box and that I just needed to wait a few days. Any idea when a fixed ghc for OS X package will appear on cache.nixos.org? [1] https://github.com/NixOS/nixpkgs/issues/2689 ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Reengineered npm2nix: call for feedback
On Tue, Sep 30, 2014 at 3:13 AM, Sander van der Burg svanderb...@gmail.com wrote: I've used the reengineered npm2nix on a private project for the company I work for, as well as some other utilities and it seems to work fine for me. Hopefully, you can also try npm2nix on your projects to find out whether there any additional issues. :) Furthermore, I'm still working a blog post to rationalize all the stuff, which should give you better insights in the problem, details and the choices I made. Let me know what you think! Hi Sander, I've just tried out this version and I'm finding it pleasant improvement over the old way. The generated code is clean and simple, and I like the minimal dependency it has on nixpkgs. However, there's one thing I haven't been able to get working: local path dependencies. I have several node apps that rely on a collection of node modules. These modules may, in turn rely on other modules. In addition, both the apps and the modules depend on modules from npm. My current setup involves generating nix expressions using npm2nix (Sheay's version) to get the NPM dependencies, then hand-editing the default.nix files for each app and module, to pull in the additional dependencies that don't come from npm. As of NPM 2.0, dependencies can be specified as paths on the local file system, and I'd like to use that feature to generate nix expressions for both local and npm dependencies using npm2nix. It almost works, but not quite. AFAICT, the nix expressions are generated just fine, but then when the derivation is being created, the relative paths are resolved against the current directory of the of the process, rather than against the directory that contains the package.json file. That causes the derivation to fail with ENOENT. All in all, looks like big improvement! Colin ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Broken ghc for OS X on cache.nixos.org
On 10/16/2014 09:53 PM, Richard Wallace wrote: Hello all, In an effort to get our team to start using nix, I've setup a project of ours to be built with nix. One obstacle I've run into is that most of the team (pretty much everyone except me) is using OS X, so I've been trying to overcome the OS X ghc being broken in the nix cache [1]. I've gone ahead and built a new ghc binary on OS X and created a cache containing it and other Haskell packages. I thought everything was going great until I tried to actually use it by running $ nix-shell shell.nix --option binary-caches http://mycache --command 'cabal configure --enable-tests' I myself often wonder how to get nix to DWIM when it comes to binary caches. Does nix-build --option binary-caches … do what you want? Maybe nix-shell just ignores it… This still downloads the ghc package from http://cache.nixos.org, which is what I'm trying to avoid. I used wireshark to see what is going on, and it appears that my cache is never checked, nix just goes and starts downloading from cache.nixos.org. My guess is that this is because of the MANIFEST from the nixpkgs-unstable channel. Is that correct? If so, how can I override that? You could try doing a manual nix-pull of your manifest. I do it to a local machine like so: nix-pull http://yuuki:3000/jobset/nixpkgs/trunk/channel/latest/MANIFEST Thanks, Rich PS It would be great if this weren't a problem. I was told this issue should be getting resolved soon since there is a new OS X build box and that I just needed to wait a few days. Any idea when a fixed ghc for OS X package will appear on cache.nixos.org? The update announcement was on the 9th and the unstable is currently on the commit from the 14th so I would have thought if it was fixed then it would be in the cache already. I think however than Haskell stuff on darwin was switched off so perhaps it just needs to be turned on again. You should ask around on #2689 about what's going on in that area. [1] https://github.com/NixOS/nixpkgs/issues/2689 ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev -- Mateusz K. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Broken ghc for OS X on cache.nixos.org
On Thu, Oct 16, 2014 at 3:28 PM, Richard Wallace rwall...@thewallacepack.net wrote: On Thu, Oct 16, 2014 at 2:55 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 10/16/2014 09:53 PM, Richard Wallace wrote: Hello all, In an effort to get our team to start using nix, I've setup a project of ours to be built with nix. One obstacle I've run into is that most of the team (pretty much everyone except me) is using OS X, so I've been trying to overcome the OS X ghc being broken in the nix cache [1]. I've gone ahead and built a new ghc binary on OS X and created a cache containing it and other Haskell packages. I thought everything was going great until I tried to actually use it by running $ nix-shell shell.nix --option binary-caches http://mycache --command 'cabal configure --enable-tests' I myself often wonder how to get nix to DWIM when it comes to binary caches. Does nix-build --option binary-caches … do what you want? Maybe nix-shell just ignores it… No, same thing happens with nix-build or nix-env. This still downloads the ghc package from http://cache.nixos.org, which is what I'm trying to avoid. I used wireshark to see what is going on, and it appears that my cache is never checked, nix just goes and starts downloading from cache.nixos.org. My guess is that this is because of the MANIFEST from the nixpkgs-unstable channel. Is that correct? If so, how can I override that? You could try doing a manual nix-pull of your manifest. I do it to a local machine like so: nix-pull http://yuuki:3000/jobset/nixpkgs/trunk/channel/latest/MANIFEST I will give that a try. I'm not entirely sure it will work. I guess the question is, when doing a nix-pull, will the entry for ghc be replaced or not? It seems reasonable to me that it wouldn't be because the two caches have the same checksum and everything, so they appear to be the same. So, from the perspective of nix, which one it pulls from should be arbitrary. That does seem to work. Not ideal because I'll have to make sure it is run consistently, but it should get me by for now. Thanks again! Thanks, Rich PS It would be great if this weren't a problem. I was told this issue should be getting resolved soon since there is a new OS X build box and that I just needed to wait a few days. Any idea when a fixed ghc for OS X package will appear on cache.nixos.org? The update announcement was on the 9th and the unstable is currently on the commit from the 14th so I would have thought if it was fixed then it would be in the cache already. I think however than Haskell stuff on darwin was switched off so perhaps it just needs to be turned on again. You should ask around on #2689 about what's going on in that area. Done. Hopefully someone in the know will update that issue with the latest status soon. Thanks. [1] https://github.com/NixOS/nixpkgs/issues/2689 ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev -- Mateusz K. ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] how to write nix expressions for packages that have plugins
There are two methods I've seen: (...) 2. create a merge nix package: A package which takes the application plus all plugins requested and builds the directory layout the application expects with all the plugins linked into the right folder. This package is then added to the environment of the user. Can you give an example of a package in nix that is using this approach? I'm interested in how to go about writing a nix-expression that does this. On Thu, Oct 16, 2014 at 8:17 AM, cdep.illab...@gmail.com cdep.illab...@gmail.com wrote: Hi, I'm trying to write a nix expression for a plugin for a package. I'm running into a problem where the package doesn't know where to look for the new plugin. The original package stores all of it's plugins in /nix/store/xxx-package/lib/packagename/{plugin1,plugin2,mycoolplugin}.so. It expects all of it's plugins to be there. There doesn't appear to be any way of telling the program that it needs to look for plugins in other directories. How is this type of problem usually solved in other nix packages? The package/plugin combination in question is fcitx and fcitx-anthy. It's an input method editor commonly used to input Chinese, Japanese, and Korean (among other languages). I raised a question about this package/plugin combination on github, but I am also interested in how it's solved in general. https://github.com/NixOS/nixpkgs/issues/4550 I have also raised a question on the fcitx issue tracker for anyone curious: https://github.com/fcitx/fcitx/issues/179 Finally, in case anyone wants to look at the the original nix-expressions, here they are: fcitx: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/default.nix fcitx-anthy: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/inputmethods/fcitx/fcitx-anthy.nix Thanks, Dennis ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev