Re: [Nix-dev] RTEMS cross compiler
Benjamin Franksen wrote: > I want to package the tool chain for RTEMS 4.9. The sources are at > ftp://www.rtems.org/pub/rtems/SOURCES/4.9. The gcc version is 4.3.2 plus a > few RTEMS specific patches. I successfully packaged the corresponding > binutils. I can also build and install the cross gcc. However, when using > the resulting gcc as a "driver" e.g. to compile a single .c file to an > executable, I get errors that suggest that it tries to call the 'as' > (assembler, part of binutils) for the host system, not the cross version. > Specifically, > > franksen@tiber: ~/tmp > powerpc-rtems4.9-gcc -mcpu=603e -Dmpc603e > -Dppc603e -O2 -g -mmultiple -mstring -mstrict-align conftest.c > as: unrecognized option '-mppc' > > I have been banging my head against this problem with no success, so now I > am asking for help here, hoping that you guys have more experience with > building gcc with nix and can give me a hint how to get this to work. For the record: I found a solution. It seems that gcc insists on finding the binutils under its own prefix. This can be overridden on the command line with the -B option (I checked that this works), but there is no way to configure gcc to use a different path by default. So what I did was to link the directory under binutils that contains the tools (with their canonic names e.g. as, ld, ...) to where the running instance of gcc expects them: preConfigure = '' #...other stuff... mkdir -p $out/$target ln -s $targetArchBinutils/$target/bin $out/$target ''; (where target = "${targetArch}-${targetOs}") I don't know if this is the 'idiomatic' way to solve such problems in Nix, but it seems to work. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] builder is root, not nixbld [was: cannot build coreutils]
Ben Franksen wrote: > tl;dr I can verify with a very simple nix expression that the builder > indeed is root, and not one of the nixbld users. I think this is not as it > should be. I finally solved this after looking into the source tree. It turned out that after I re-compiled and installed Nix from source, the configuration directory (sysconfdir) is /usr/local/etc (a somewhat unconventional choice, if you ask me). It was /etc when I had installed Nix from the ubuntu package, but I recently upgraded my system to ubuntu trusty and there is no binary package for trusty yet, so I re-installed from source. Thus my existing entry for build-users-group in /etc/nix.conf was ignored. BTW, I see that there are rpm specs in the sources. How can I create package for my system from the sources? Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] builder is root, not nixbld [was: cannot build coreutils]
Hello tl;dr I can verify with a very simple nix expression that the builder indeed is root, and not one of the nixbld users. I think this is not as it should be. Any idea what I am doing wrong? Details follow: Here is the test expression: {stdenv, coreutils}: stdenv.mkDerivation { name = "trivial-0.0"; builder = ./builder.sh; inherit coreutils; } and the builder is just $coreutils/bin/id > nix-build '' -A trivial these derivations will be built: /nix/store/11h7h69wfw56d4j4xgmpgwk2k2hccq6r-trivial-0.0.drv building path(s) `/nix/store/jvqphwydc3p3ggsn5bmdwks0n5lppqwk-trivial-0.0' uid=0(root) gid=0(root) groups=0(root) [...] When I give the -K option I can see that everything under /tmp/nix-... is owned by root. I re-installed nix-1.7 from source and re-checked my configuration. These are the build users: > grep nixbld /etc/{passwd,group} /etc/passwd:nixbld0:x:4:4:Nix build user 0,,,:/var/run/empty:/bin/false /etc/passwd:nixbld1:x:40001:4:Nix build user 1,,,:/var/run/empty:/bin/false /etc/passwd:nixbld2:x:40002:4:Nix build user 2,,,:/var/run/empty:/bin/false /etc/passwd:nixbld3:x:40003:4:Nix build user 3,,,:/var/run/empty:/bin/false /etc/passwd:nixbld4:x:40004:4:Nix build user 4,,,:/var/run/empty:/bin/false /etc/passwd:nixbld5:x:40005:4:Nix build user 5,,,:/var/run/empty:/bin/false /etc/passwd:nixbld6:x:40006:4:Nix build user 6,,,:/var/run/empty:/bin/false /etc/passwd:nixbld7:x:40007:4:Nix build user 7,,,:/var/run/empty:/bin/false /etc/passwd:nixbld8:x:40008:4:Nix build user 8,,,:/var/run/empty:/bin/false /etc/passwd:nixbld9:x:40009:4:Nix build user 9,,,:/var/run/empty:/bin/false /etc/group:nixbld:x:4:nixbld0,nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8,nixbld9 That looks ok to me. My /etc/nix/nix.conf contains one line build-users-group = nixbld (I disabled chroot builds for the moment) Environment: > env|grep NIX NIX_REMOTE=daemon NIX_PATH=nixpkgs=/home/ben/.nix-defexpr/channels/nixos-13.10/nixpkgs For the multi-user setup I largely followed the instructions here: http://sandervanderburg.blogspot.de/2013/06/setting-up-multi-user-nix-installation.html except that I use this very simple upstart job for the nix-daemon: # Nix build daemon. description "Nix build daemon" start on runlevel [2345] stop on runlevel [!2345] exec /usr/local/bin/nix-daemon Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] cannot build coreutils
Hi All I am trying to use an older version of gcc (4.4) to compile a new cross-gcc for RTEMS-4.9, since building with a current gcc fails. I am using the nixpkgs from the stable nixos-13.10 branch. I have this binutils_powerpc_rtems49 = callPackage rtems/4.9/binutils { stdenv = stdenvAdapters.overrideGCC stdenv gcc44; targetArch = "powerpc"; }; in packageOverrides inside config.nix. When I try to build, I get an error in the configure phase for coreutils: > nix-env -i binutils-powerpc-rtems4.9 replacing old `binutils-powerpc-rtems4.9' installing `binutils-powerpc-rtems4.9' these derivations will be built: /nix/store/6c9ddidbjifccdqs6yxy74iisy9zzq70-gcc-wrapper-4.4.6.drv /nix/store/7ap9h7j2j4vi92syb6s9c1gaps0l7rz0-glibc-2.17.drv /nix/store/gkv50ilpimla17bjrn5fbkzby6ycg1yl-binutils-powerpc-rtems4.9.drv /nix/store/lafw93rpmf3y0gdjd726ny6zraygasd6-coreutils-8.21.drv /nix/store/m4mvgikl0kkmk2z84xlx830xgx5hiwzi-binutils-2.23.1.drv /nix/store/vb7x59gb9g8zfy9y066n2acgj7yfgadx-linux-headers-3.7.1.drv /nix/store/ws89nxbis7dv7604s0dm1442gq944vic-bash-4.2-p42.drv building path(s) `/nix/store/vhdkzr2za2qly65fy39lxz7k1i6jmg92- coreutils-8.21' building /nix/store/vhdkzr2za2qly65fy39lxz7k1i6jmg92-coreutils-8.21 unpacking sources unpacking source archive /nix/store/zv8flybdrdr5sprbjlrbna64iprll80k- coreutils-8.21.tar.xz [...etc...] checking whether mknod can create fifo without root privileges... configure: error: in `/tmp/nix-build-coreutils-8.21.drv-0/coreutils-8.21': configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check) See `config.log' for more details note: keeping build directory `/tmp/nix-build-coreutils-8.21.drv-0' builder for `/nix/store/lafw93rpmf3y0gdjd726ny6zraygasd6-coreutils-8.21.drv' failed with exit code 1 cannot build derivation `/nix/store/6c9ddidbjifccdqs6yxy74iisy9zzq70-gcc- wrapper-4.4.6.drv': 1 dependencies couldn't be built cannot build derivation `/nix/store/q81c6v7xhhysgaqnjrdhihjj79bcg4kl- binutils-powerpc-rtems4.9.drv': 1 dependencies couldn't be built error: build of `/nix/store/q81c6v7xhhysgaqnjrdhihjj79bcg4kl-binutils- powerpc-rtems4.9.drv' failed I think something is wrong here. Why does configure think it is running as root? I am using the standard multi-user setup with nixbld0..nixbld9 as build users. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] impureEnvVars, the second
Ben Franksen wrote: > Shea Levy wrote: >> On Thu, Apr 17, 2014 at 06:13:35PM +0200, Ben Franksen wrote: >>> Eelco Dolstra wrote: >>> > On 17/04/14 17:04, Ben Franksen wrote: >>> > >>> >> sorry to bother you again regarding impureEnvVars. I still can't get >>> >> my fetchdarcs over ssh to work, even though I am now using >>> >> constant-output derivations and have >>> >> >>> >> impureEnvVars = [ "SSH_AGENT_PID" "SSH_AUTH_SOCK" ]; >>> >> >>> >> in my fetchdarcs/default.nix. >>> >> >>> >> The environment variables are now defined in the builder. However, >>> >> they are empty :( >>> >> >>> >> Could this be because I am using the Nix multi-user setup where >>> >> building is delegated to a number of nixbld users? >>> > >>> > Right, environment variables from the client are not passed to the >>> > builder. Even if they were, the builder probably would not have file >>> > system access to the socket identifier by $SSH_AUTH_SOCK. >>> >>> The latter could, I guess, be worked-around (using build-chroot-dirs) >> >> Actually, fixed-output derivations are done outside of the chroot, so >> you just need to ensure the socket is accessible to the build users >> group. In fact, *not* having it in the chroot is better so that >> non-fixed-output builds don't have access. >> >>> but >>> the former seems... hopeless :( >> >> Why? Just start the daemon with the right environment settings. > > Well, those would have to be static, as the daemon is not started > separately by each user. But the environment variables are created > dynamically when the ssh agent is started. > > But: what you said here gave me an idea for a much simpler solution: I > don't have to rely on the developer's credentials at all. Instead I'll > give the Nix build users their own ssh identity (key pair) w/o passphrase. > Then add their public key to the @'s > .ssh/authorized_keys. No impureEnvVars needed, since the build users don't > need to connect to an ssh agent. > > I hope that Nix build users not having a home directory is only a > recommendation, not a requirement for the multi-user setup to work. Just wanted to report that this approach finally worked! Yay! (BTW, this is possible only because fixed-output derivations are built outside the chroot, else ssh could not access the identity file in the build user's home.) Cheers & thanks again Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] impureEnvVars, the second
Shea Levy wrote: > On Thu, Apr 17, 2014 at 06:13:35PM +0200, Ben Franksen wrote: >> Eelco Dolstra wrote: >> > On 17/04/14 17:04, Ben Franksen wrote: >> > >> >> sorry to bother you again regarding impureEnvVars. I still can't get >> >> my fetchdarcs over ssh to work, even though I am now using >> >> constant-output derivations and have >> >> >> >> impureEnvVars = [ "SSH_AGENT_PID" "SSH_AUTH_SOCK" ]; >> >> >> >> in my fetchdarcs/default.nix. >> >> >> >> The environment variables are now defined in the builder. However, >> >> they are empty :( >> >> >> >> Could this be because I am using the Nix multi-user setup where >> >> building is delegated to a number of nixbld users? >> > >> > Right, environment variables from the client are not passed to the >> > builder. Even if they were, the builder probably would not have file >> > system access to the socket identifier by $SSH_AUTH_SOCK. >> >> The latter could, I guess, be worked-around (using build-chroot-dirs) > > Actually, fixed-output derivations are done outside of the chroot, so > you just need to ensure the socket is accessible to the build users > group. In fact, *not* having it in the chroot is better so that > non-fixed-output builds don't have access. > >> but >> the former seems... hopeless :( > > Why? Just start the daemon with the right environment settings. Well, those would have to be static, as the daemon is not started separately by each user. But the environment variables are created dynamically when the ssh agent is started. But: what you said here gave me an idea for a much simpler solution: I don't have to rely on the developer's credentials at all. Instead I'll give the Nix build users their own ssh identity (key pair) w/o passphrase. Then add their public key to the @'s .ssh/authorized_keys. No impureEnvVars needed, since the build users don't need to connect to an ssh agent. I hope that Nix build users not having a home directory is only a recommendation, not a requirement for the multi-user setup to work. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] impureEnvVars, the second
Eelco Dolstra wrote: > On 17/04/14 17:04, Ben Franksen wrote: > >> sorry to bother you again regarding impureEnvVars. I still can't get my >> fetchdarcs over ssh to work, even though I am now using constant-output >> derivations and have >> >> impureEnvVars = [ "SSH_AGENT_PID" "SSH_AUTH_SOCK" ]; >> >> in my fetchdarcs/default.nix. >> >> The environment variables are now defined in the builder. However, they >> are empty :( >> >> Could this be because I am using the Nix multi-user setup where building >> is delegated to a number of nixbld users? > > Right, environment variables from the client are not passed to the > builder. Even if they were, the builder probably would not have file > system access to the socket identifier by $SSH_AUTH_SOCK. The latter could, I guess, be worked-around (using build-chroot-dirs) but the former seems... hopeless :( I wonder what other options I have to achieve my goal. ATM I can see two: (1) Make the repositories available via http. That means I'll have to convince the sysadmin to set up a web server on the repo server and let it offer the directory that contains our repos. Or I can setup the web werver on another machine that can NFS mount the repo directory; yes, that sounds more practical ;-) (2) Use my self-written nix-prefetch-darcs. That works because it does not delegate the downloading to a nixbld user. The problem is that this breaks automatic dependency resolution during nix-build, if the source derivation is not yet in the Nix store. Developers will have to use nix-prefetch-darcs at least once for every new dependency an application acquires. I could modify the source derivations to display a message to users like Builder cannot proceed: please call nix-prefetch-darcs @: from the command line and try again. where the placeholders , , and are automatically filled in by the builder, so the user can cut and paste this line into their shell. The main question with regard to (2) is: how do I prevent the source derivations from being garbage collected (for instance in case the final build did not succeed and no result link gets created) so that users at least have to do this only once for each dependency? Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] impureEnvVars, the second
Hi All sorry to bother you again regarding impureEnvVars. I still can't get my fetchdarcs over ssh to work, even though I am now using constant-output derivations and have impureEnvVars = [ "SSH_AGENT_PID" "SSH_AUTH_SOCK" ]; in my fetchdarcs/default.nix. The environment variables are now defined in the builder. However, they are empty :( Could this be because I am using the Nix multi-user setup where building is delegated to a number of nixbld users? Is impureEnvVars supposed to work in this setting? If not, what can I do to make it work? If yes, then I probably made a mistake when setting up the nix build users. (I am using Nix-1.7.) Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Eelco Dolstra wrote: > On 16/04/14 00:40, Ben Franksen wrote: >> So everything in my sources is patched to use stuff from /nix/store (I >> checked that the path exists and is executable) and still it complains >> about some /bin/sh missing? > > There are many programs that have a (implicit) dependency on /bin/sh. > Notably, the system() function in Glibc calls /bin/sh. This is why chroots > on NixOS do include /bin/sh (as well as the dynamic library dependencies > of /bin/sh). Note that for this project I am using Nix stand-alone (on Ubuntu), not NixOS. I found that /bin/sh is not available by default, so I experimented with "build-chroot-dirs = /bin=/nix-bin" (in /etc/nix/nix.conf, copying bash from the nix store to /nix-bin/sh). However, that made no difference at all. It turned out the culprit in this case was make. As soon as I figured this out, I could solve the problem by passing "SHELL=sh" to make on the command line, so that make uses the sh in the path instead of /bin/sh. Cheers and many thanks for all the help Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Ben Franksen wrote: > building /nix/store/y57pn9yk7r9j06nqfkyd4b1h5p8gfdzr-epics- > base-3.14.12.2.1/configure/CONFIG_ENV > make[2]: /bin/sh: Command not found Hmmm, this sounds strangely familiar. Am I interpreting this wrong and it means the shell itself (which is not /bin/sh but actually /nix/store/... but maybe it doesn't know that) cannot find some other command? Some hidden impurity? -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Ben Franksen wrote: > I will now read up on how to enable chroot builds in multi-user configured > Nix... Ok, I now have build-use-chroot = true in my /etc/nix/nix.conf. Now, my build fails with this strange error: building /nix/store/y57pn9yk7r9j06nqfkyd4b1h5p8gfdzr-epics- base-3.14.12.2.1/configure/CONFIG_ENV make[2]: /bin/sh: Command not found I _did_ run the patchShebangs function on my sources before building. I verified that this actually worked by using the -K option to nix-build, then looked at the build directory: ben@sarun[1]: .../apps/HoBiCaT-Controls > cd /tmp/nix-build-epics- base-3.14.12.2.1.drv-8 ben@sarun[1]: /tmp/nix-build-epics-base-3.14.12.2.1.drv-8 > rgrep '/bin/sh' . ./fetchdarcs/config/tools/installEpics:#!/nix/store/7mjplwq1r0agngqj2058hl5z8kb3w15g- bash-4.2-p42/bin/sh ./fetchdarcs/startup/EpicsHostArch:#!/nix/store/7mjplwq1r0agngqj2058hl5z8kb3w15g- bash-4.2-p42/bin/sh ./fetchdarcs/startup/Site.profile:#!/nix/store/7mjplwq1r0agngqj2058hl5z8kb3w15g- bash-4.2-p42/bin/sh ./fetchdarcs/src/util/rc2.logServer:#!/nix/store/7mjplwq1r0agngqj2058hl5z8kb3w15g- bash-4.2-p42/bin/sh ./fetchdarcs/src/util/rc2.caRepeater:#!/nix/store/7mjplwq1r0agngqj2058hl5z8kb3w15g- bash-4.2-p42/bin/sh So everything in my sources is patched to use stuff from /nix/store (I checked that the path exists and is executable) and still it complains about some /bin/sh missing? Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Nixos.org down
Ben Franksen wrote: > Eelco Dolstra wrote: >> On 15/04/14 18:38, Eelco Dolstra wrote: >> >>> On 15/04/14 18:27, Colin Putney wrote: >>> >>>> Hope this right place to post this. I don't speak German, but it looks >>>> like the domain is being blocked, for some reason. >>> >>> Yeah, problem with the registrar. Should be fixed soon... >> >> Where "soon" might be tomorrow, unfortunately. In the meantime, if you >> need Nix stuff right now, you can use the following /etc/hosts entries: >> >> 54.217.220.47 nixos.org tarball.nixos.org releases.nixos.org >> 131.180.119.77 hydra.nixos.org > > Many thanks for providing a work-around, but that doesn't work here (with > firefox). Do I have to restart/reload something so this has an effect? (I > even tried to enter the IP address directly in firefox, no use). > > Sorry for my ignorance... Never mind, this seems to be a quirk of (my (ubuntu) version of) firefox, probably to protect lusers from... whatever. It works with e.g. konqueror. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Nixos.org down
Eelco Dolstra wrote: > On 15/04/14 18:38, Eelco Dolstra wrote: > >> On 15/04/14 18:27, Colin Putney wrote: >> >>> Hope this right place to post this. I don't speak German, but it looks >>> like the domain is being blocked, for some reason. >> >> Yeah, problem with the registrar. Should be fixed soon... > > Where "soon" might be tomorrow, unfortunately. In the meantime, if you > need Nix stuff right now, you can use the following /etc/hosts entries: > > 54.217.220.47 nixos.org tarball.nixos.org releases.nixos.org > 131.180.119.77 hydra.nixos.org Many thanks for providing a work-around, but that doesn't work here (with firefox). Do I have to restart/reload something so this has an effect? (I even tried to enter the IP address directly in firefox, no use). Sorry for my ignorance... Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Hi Shea thanks for taking the time to explain this to me. I see now that what I wanted is not possible in a clean way. I think what confused me was that Nix does not enforce purity of builds by default. Out of the box, i.e. without chroot builds, getting sources for instance via darcs or plain file path seems to "just work" so I wondered why not via ssh. I got the false impression that it is okay to develop Nix packages in this setting, while in fact the default is probably only okay for installing (existing) packages that have already been purity tested in a chroot environment. I will now read up on how to enable chroot builds in multi-user configured Nix... Cheers Ben Shea Levy wrote: > On Mon, Apr 14, 2014 at 04:12:00AM +0200, Ben Franksen wrote: >> Shea Levy wrote: >> > Fetching source over the network is the main reason fixed output >> > derivations even exist. When chroot builds are enabled, networking is >> > not allowed for non-fixed output derivations. >> >> Interesting, I did not know this. >> >> I agree that this policy makes sense for stuff that gets downloaded from >> the internet, and especially if you base a complete linux distribution on >> it (security considerations: you want to make sure that the source has >> not been tampered with). >> > > It's not just security considerations, it's also that the invariants nix > maintains require that a derivation contains *all* of the information > about the output, and if you have general access to global resources > (even the filesystem, let alone the network) the only way nix can > guarantee that is if the derivation specifies exactly what the output > will look like (thus, the hash in fixed derivation outputs). > >> >> > Why is this case special? >> >> The repo server is in-house, and we talk about ssh access, limited to the >> trusted maintainers of the code base. The level of security here is >> arguably better than using an NFS share (which would appear as a local >> path to Nix and thus allow unrestricted (non-fixed output) derivations >> even with chroot builds if I understood you correctly... or maybe not, >> depending on whether the share gets bind-mounted into the builder's >> environment or not). >> > > When adding files to the store from the local filesystem, they are > hashed as they are added and thus nix's invariants are maintained. The > build should not (and in chroot mode does not) have access to any paths > outside of the nix store, so if you're keeping sources on the local FS > they are copied to the store (with a path that corresponds to their > contents) first. > >> >> We have many modules in many versions. I was hoping I could avoid to list >> each and every one of them in our "local" nix expression tree, and >> instead (mostly) rely on functions that get the version number as a >> parameter. >> > > I think the best you can do here without hacks (and not just hacks of > implementation but hacks around nix's core model) is to have your > functions take the version and the hash. > >> >> But I must admit: I am half convinced that relying on module name and >> version alone to correctly identify a source may not be strong enough to >> actually *guarantee* consistent and repeatable builds, even in our >> limited setting. >> > > Yeah, content hashing is a much sounder general basis for repeatable > builds IMO. > > ~Shea > >> >> Cheers >> Ben >> >> >> On Apr 13, 2014, at 16:03, Ben Franksen >> >> wrote: >> >> >> >> Hello >> >> >> >> Is there *any* way (besides hacking the Nix source code) to circumvent >> >> the limitation on impureEnvVars, i.e. that one has to provide the hash >> >> up front, making the derivation fixed-output? What I am asking for is >> >> something akin to unsafePerformIO in Haskell (which exists for similar >> >> situations, and motivated the name unsafeImpureEnvVars in the >> >> subject). >> >> >> >> Effectively I want to be able to say to Nix: "I swear these variables >> >> are used in a pure way, just believe me." >> >> >> >> Background: >> >> >> >> I want to use fetchdarcs for remote repositories via ssh; we use darcs >> >> to version control our software internally (even stuff we get from the >> >> outside, to track patches for customisation and bug fixes). We use >> >> tags to identify versions and have lots of darcs repos, all on
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Peter Simons wrote: > >> Fetching source over the network is the main reason fixed output > >> derivations even exist. When chroot builds are enabled, networking is > >> not allowed for non-fixed output derivations. > > > > Interesting, I did not know this. > > > > I agree that this policy makes sense for stuff that gets downloaded > > from the internet, and especially if you base a complete linux > > distribution on it (security considerations: you want to make sure that > > the source has not been tampered with). > > another feature of fixed output derivations is that the $out hash > does not depend on the commands that were used to generate $out. If a > command like > >${curl}/bin/curl http://example.org/foobar-1.1.tar.gz > > would go into the $out hash -- like build scripts normally do ---, then > source tarballs would have to be re-downloaded every time their URL > changes. If curl ever changed, then we'd have to re-download everything! > Fixed output derivations avoid this overhead (while also improving > security somewhat, because we track hashes for our sources). Hi Peter exactly that thought occurred to me, too, lately; thanks for confirming it. I guess I'll have to see the Nix sources as some kind of database that associates module and version with a hash of the output to be on the safe (and efficient) side. I have some idea now how to do that, it's probably not as much work as I thought it might be. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] is there something like unsafeImpureEnvVars?
Shea Levy wrote: > Fetching source over the network is the main reason fixed output > derivations even exist. When chroot builds are enabled, networking is not > allowed for non-fixed output derivations. Interesting, I did not know this. I agree that this policy makes sense for stuff that gets downloaded from the internet, and especially if you base a complete linux distribution on it (security considerations: you want to make sure that the source has not been tampered with). > Why is this case special? The repo server is in-house, and we talk about ssh access, limited to the trusted maintainers of the code base. The level of security here is arguably better than using an NFS share (which would appear as a local path to Nix and thus allow unrestricted (non-fixed output) derivations even with chroot builds if I understood you correctly... or maybe not, depending on whether the share gets bind-mounted into the builder's environment or not). We have many modules in many versions. I was hoping I could avoid to list each and every one of them in our "local" nix expression tree, and instead (mostly) rely on functions that get the version number as a parameter. But I must admit: I am half convinced that relying on module name and version alone to correctly identify a source may not be strong enough to actually *guarantee* consistent and repeatable builds, even in our limited setting. Cheers Ben >> On Apr 13, 2014, at 16:03, Ben Franksen wrote: >> >> Hello >> >> Is there *any* way (besides hacking the Nix source code) to circumvent >> the limitation on impureEnvVars, i.e. that one has to provide the hash up >> front, making the derivation fixed-output? What I am asking for is >> something akin to unsafePerformIO in Haskell (which exists for similar >> situations, and motivated the name unsafeImpureEnvVars in the subject). >> >> Effectively I want to be able to say to Nix: "I swear these variables are >> used in a pure way, just believe me." >> >> Background: >> >> I want to use fetchdarcs for remote repositories via ssh; we use darcs to >> version control our software internally (even stuff we get from the >> outside, to track patches for customisation and bug fixes). We use tags >> to identify versions and have lots of darcs repos, all on a single >> server, accessible to developers via ssh. >> >> Unfortunately, fetchdarcs fails in this case because, to avoid being >> asked for a passphrase, ssh (in this case called via darcs) needs to know >> how to access a runnig ssh agent, and this knowledge is encoded in two >> environment variables: SSH_AGENT_PID and SSH_AUTH_SOCK. But of course >> these are not defined in the builder, its environment gets cleared before >> it starts. >> >> Now, there is the attribute impureEnvVars. But that works only if you >> pass the expected hash to the builder (so the derivation becomes >> fixed-output). That means either I hard-code the expected hash into the >> derivation, or pass it as an argument. In both cases, I need to find out >> the hash before-hand. This makes it very inconvenient for users to add >> new version of packages. >> >> Note that this is *only* because of the two environment variables. I can >> use fetchdarcs just fine with any local path or remote http url without >> providing any hash. >> >> Note also that use of these variables does not imply any impurity, they >> are only used for authentication. >> >> Cheers >> Ben >> -- >> "Make it so they have to reboot after every typo." -- Scott Adams >> >> ___ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] is there something like unsafeImpureEnvVars?
Hello Is there *any* way (besides hacking the Nix source code) to circumvent the limitation on impureEnvVars, i.e. that one has to provide the hash up front, making the derivation fixed-output? What I am asking for is something akin to unsafePerformIO in Haskell (which exists for similar situations, and motivated the name unsafeImpureEnvVars in the subject). Effectively I want to be able to say to Nix: "I swear these variables are used in a pure way, just believe me." Background: I want to use fetchdarcs for remote repositories via ssh; we use darcs to version control our software internally (even stuff we get from the outside, to track patches for customisation and bug fixes). We use tags to identify versions and have lots of darcs repos, all on a single server, accessible to developers via ssh. Unfortunately, fetchdarcs fails in this case because, to avoid being asked for a passphrase, ssh (in this case called via darcs) needs to know how to access a runnig ssh agent, and this knowledge is encoded in two environment variables: SSH_AGENT_PID and SSH_AUTH_SOCK. But of course these are not defined in the builder, its environment gets cleared before it starts. Now, there is the attribute impureEnvVars. But that works only if you pass the expected hash to the builder (so the derivation becomes fixed-output). That means either I hard-code the expected hash into the derivation, or pass it as an argument. In both cases, I need to find out the hash before-hand. This makes it very inconvenient for users to add new version of packages. Note that this is *only* because of the two environment variables. I can use fetchdarcs just fine with any local path or remote http url without providing any hash. Note also that use of these variables does not imply any impurity, they are only used for authentication. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] cinnamon-settings-daemon will not compile
Vladimír Čunát wrote: > On 04/08/2014 02:29 PM, Roelof Wobben wrote: >> but when I do nix-build -A cinnamon-settings-daeemon in the directory >> which contains the nix file I get a message that default.nix cannot be >> found. > > Normally, you should call "nix-build" in the root of the nixpkgs tree, > or supply the root by -f parameter. Also, you need "-A > cinnamon.cinnamon-settings-daemon" to specify the attribute path. I must say I find this part of the CLI a bit unfriendly. In everything I did with Nix until now I always used the same top-level Nix expression, currently this is ~/.nix-defexpr/channels/nixos-13.10/nixpkgs (which gets automatically created by nix-channel). I should probably make myself a shell alias alias nix-build='nix-build ~/.nix-defexpr/channels/nixos-13.10/nixpkgs' The reason I always use the same top-level expression is that I do not (yet) want to make changes in the nixpkgs themselves. I do all my work under .nixpkgs (which I made a symlink to my real working dir where I keep my stuff under version control) (ab-)using .nixpkgs/config.nix to merge in my own packages: ben@sarun[1]: ~ > ls .nixpkgs/ config.nix epics ben@sarun[1]: ~ > cat .nixpkgs/config.nix { packageOverrides = pkgs: with pkgs; rec { epics_base = callPackage epics/base/full.nix { version = "3.14.12.4"; }; ... } (If there is a simpler or more idiomatic way to achieve this, please tell me.) Cheers -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] reboot hangs most of the time
Peter Simons wrote: > > When I want to reboot my box I enter 'reboot' in the prompt. And most > > of the time I see unmounting /nix/store and nothings happens after > > that. > > I've run into this issue, too, but only sporadically. I have no idea > what might be the cause of this phenomenon. It feels like "reboot" is > more likely to fail on machines with a long uptime (like servers), > whereas my desktop machine and laptop -- which reboot frequently --, > hardly ever hang this way. FWIW, I experienced the same (also when rebooting from the kde session, but I guess that just calls out to the CL tools). Mine is an experimental installation on a desktop machine at home and has never run for long (a day at most). Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Hash Collisions
Eelco Dolstra wrote: > On 08/04/14 09:54, Marc Weber wrote: >> Also "waste of space/bandwidth" always happens because: >> >> - nixos always ships with header files for all libraries >> (this could be fixed because nix supports multiple outputs) > > Yeah, multiple outputs will fix this eventually (reducing system closure > size is > my main goal for 14.10). See > https://github.com/NixOS/nixpkgs/commit/dbe432e6f4502fc7b4e290d1b3356083bd93b099 > for an example of the savings you can get. Ah, I was wondering about multiple outputs (if we actually mean the same thing here: one source tree, multiple packages, e.g. binary, dev, doc) as this seems to be standard practice in packaging nowadays. I am looking forward to 14.10, then... :) BTW, it is currently a common limitation (e.g. in debian) that although one can have multiple versions of a binary package installed, there can be only one version of the development package. I think NixOS should be able to easily avoid such a restriction, and this would be a very strong selling point! Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] customizing the profile and environment
Vladimír Čunát wrote: > On 04/06/2014 06:24 PM, Ben Franksen wrote: >>How do I customize what gets put in my profile and environment? > > Currently there are no options. All paths are linked into user > profiles/environments (but for a few hard-coded exceptions). Ok, thanks. > I plan some work on allowing to exclude some files from linking into > user/system envs, but I can't say when or if ever that will be pushed. I guess the perfect solution would be to make the list of directories to merge into the user profile configurable from inside the nix expression, and similarly with environment variables. When (and if) you actually do this, please contact me, I might be able to help. > As for search paths... for the purpose of being in build inputs of other > packages, one can define hooks that modify the environment of the > builder (e.g. extend PATH, add library search paths). No such > (automatic) option exists for user/system profiles (yet). I found out about setup hooks, and I am currently employing them to amend default search paths. For the moment I can live with that: it turned out that I can avoid overlaps between installed (EPICS) modules by judiciously removing some of the files that get installed by default (and which aren't really needed). Also, there do exist debian packages for parts of what I'm interested in; I learned a lot from looking at how they did that ;-) Another thing I am considering is to provide users with a tool (including a piece of shell script they can source) to get an environment in where they have the stuff they need. It will be parameterized by a set of modules and their versions, and I also plan to let them cache this input somehow in the source tree, so they can easily re-generate the environment they need. I just need a bit more proficiency with hacking in the nix language to get there... Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] customizing the profile and environment
Hi my goal is to write nix expressions / packages for EPICS [1]. EPICS is quite non-standard w.r.t. building and configuration. The build system was designed 20 years ago to make cross-compilation to lots of embedded target architectures easy for end users. That was at a time when cross-builds where very poorly, if at all, supported by standard tool-chains. EPICS has no separate configure or install step. Configuration is done by editing files like e.g. configure/os/CONFIG_SITE... I have managed to create a minimal Nix-conformant configuration with a few lines in the builder script. The EPICS install step is integrally mixed with building, one says make INSTALL_LOCATION=$out and then it installs things like bin/linux-x86_64 bin/linux-x86_64/ex bin/RTEMS-mvme5500 bin/RTEMS-mvme5500/ex.obj bin/RTEMS-mvme2100 bin/RTEMS-mvme2100/ex.obj into $out (and similar for lib and some other EPICS specific directories). So far this is exactly what I want to happen. But when I look at my user- environment after installing, I see ben@sarun[1]: .../epics/base > ls ~/.nix-profile bin configure db dbd html include lib manifest.nix templates ben@sarun[1]: .../epics/base > ls ~/.nix-profile/bin linux-x86_64 ben@sarun[1]: .../epics/base > ls ~/.nix-profile/lib linux-x86_64 perl and e.g. bin/linux-x86_64 is not in my PATH, nor lib/linux-x86_64 in the library search path. Nor should the db, dbd, etc directories be there. Which gets me to the main QUESTION: How do I customize what gets put in my profile and environment? For reference, I have attached my default.nix and builder.sh. [1] http://www.aps.anl.gov/epics/ Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams builder.sh Description: application/shellscript { stdenv, binutils, fetchurl, perl, readline, version ? "3.14.12.4" }: let md5s = { "3.14.12.4" = "4e71c07053c4458eece4d492eb85fca8"; "3.14.12.3" = "67f0021d270d8fa142c9d978b4599b32"; }; in stdenv.mkDerivation { inherit version; name = "epics-base-" + version; builder = ./builder.sh; src = fetchurl { url = http://www.aps.anl.gov/epics/download/base/baseR + version + ".tar.gz"; md5 = builtins.getAttr version md5s; }; inherit perl readline binutils; inherit (stdenv) gcc; propagatedBuildInputs = [ perl readline ]; } ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] strange nix error
James Cook wrote: > /nix/var/nix/profiles has the same permissions/owner/group on my > system. If the daemon is running as root and you have the env > variable set, then I'm not sure what's happening. Never mind, I googled a bit and found this on the net: http://sandervanderburg.blogspot.de/2013/06/setting-up-multi-user-nix-installation.html It was very helpful and allowed me to successfully use the multi-user setup (note this is about a standalone Nix installation, not NixOS). Cheers & thanks Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] strange nix error
Ben Franksen wrote: > James Cook wrote: >> On 5 April 2014 16:47, Ben Franksen wrote: >>> I have installed nix under ubuntu, using the ubuntu deb package. Then I >>> followed the instructions in the nix manual for multi-user installs, >>> i.e. created build users & groups and added a nix.conf: >>> [...] >>> Then I do nix-env -i and get >>> >>> error: setting synchronous mode: unable to open database file >> >> I had a similar problem, and fixed it by setting the environment >> variable NIX_REMOTE to "daemon" as suggested in the user's guide. > > That was it. Thanks! > > I had set the variable but only in my .zshenv and forgot to restart the > shell. Stupid me. I have successfully created a new package and the build runs through just fine, yay! Unfortunately the installation into the user environment fails: ... building path(s) `/nix/store/8jw7qly1mq2nzdjlnnhiqwgsbhwc3ns9-user- environment' created 8 symlinks in user environment error: opening lock file `/nix/var/nix/profiles/default.lock': Permission denied ben@sarun[1]: ~ > ll -d /nix/var/nix/profiles drwxr-xr-x 3 root root 4096 Apr 5 21:53 /nix/var/nix/profiles Does /nix/var/nix/profiles have to be writeable by the nixbld users or the nixbld group? I searched the nix manual but found no hint. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] strange nix error
James Cook wrote: > On 5 April 2014 16:47, Ben Franksen wrote: >> I have installed nix under ubuntu, using the ubuntu deb package. Then I >> followed the instructions in the nix manual for multi-user installs, i.e. >> created build users & groups and added a nix.conf: >> [...] >> Then I do nix-env -i and get >> >> error: setting synchronous mode: unable to open database file > > I had a similar problem, and fixed it by setting the environment > variable NIX_REMOTE to "daemon" as suggested in the user's guide. That was it. Thanks! I had set the variable but only in my .zshenv and forgot to restart the shell. Stupid me. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] strange nix error
Hi I have installed nix under ubuntu, using the ubuntu deb package. Then I followed the instructions in the nix manual for multi-user installs, i.e. created build users & groups and added a nix.conf: ben@sarun[1]: ~ > sudo dpkg -i nix_1.6.1-1_amd64.deb bben@sarun[1]: ~ > cat nix-setup-build-users.sh #!/bin/sh addgroup --quiet --gid 4 nixbld for i in 1 2 3 4 5 6 7 8 9; do adduser --quiet --system --disabled-password --no-create-home -- uid="4000$i" --gecos "Nix build user $i" --ingroup nixbld "nixbld$i" adduser --quiet "nixbld$i" nixbld done chgrp nixbld /nix/store chmod 1775 /nix/store ben@sarun[1]: ~ > sudo sh ./nix-setup-build-users.sh ben@sarun[1]: ~ > grep nix /etc/{group,passwd} /etc/group:nixbld:x:4:nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8,nixbld9 /etc/passwd:nixbld1:x:40001:4:Nix build user 1,,,:/home/nixbld1:/bin/false /etc/passwd:nixbld2:x:40002:4:Nix build user 2,,,:/home/nixbld2:/bin/false /etc/passwd:nixbld3:x:40003:4:Nix build user 3,,,:/home/nixbld3:/bin/false /etc/passwd:nixbld4:x:40004:4:Nix build user 4,,,:/home/nixbld4:/bin/false /etc/passwd:nixbld5:x:40005:4:Nix build user 5,,,:/home/nixbld5:/bin/false /etc/passwd:nixbld6:x:40006:4:Nix build user 6,,,:/home/nixbld6:/bin/false /etc/passwd:nixbld7:x:40007:4:Nix build user 7,,,:/home/nixbld7:/bin/false /etc/passwd:nixbld8:x:40008:4:Nix build user 8,,,:/home/nixbld8:/bin/false /etc/passwd:nixbld9:x:40009:4:Nix build user 9,,,:/home/nixbld9:/bin/false ben@sarun[1]: ~ > cat /etc/nix/nix.conf build-users-group = nixbld build-max-jobs = 2 auto-optimise-store = true Then I do nix-env -i and get error: setting synchronous mode: unable to open database file Any idea what went wrong? Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Some NixOS beginner questions...
Eelco Dolstra wrote: > On 30/03/14 23:01, Ben Franksen wrote: > >> (2) It seems impossible to install NixOS w/o installing the boot loader. > > You can set: > > boot.loader.grub.device = "nodev"; > > to prevent the installation of GRUB in the MBR while still generating a > GRUB configuration in /boot that might be used for chainloading. This is what I do now and it indeed works fine. (I was confused by the other problem I had (installing from USB stick didn't work) and first thought it had to do with not installing grub.) > It should be possible to install on a partition as well, but that might > not work > with GRUB 2 due to the increased size (never tried). I fixed that by creating a btrfs on some partition which is compatible with grub2. Thanks Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Some NixOS beginner questions...
Pablo Costa wrote: >> (3) I love the declarative approach to system configuration. However, for > a >> newcomer it is hard to find out how to configure his favorite packages. > What >> options are there and how can I set them? Is looking at the 'sources' >> (for the package's nix expressions) the only way? > > On NixOS command line: > man configuration.nix > > On the browser, chapter 6 of the manual: > http://nixos.org/nixos/manual/#ch-options Ah, yes, I was aware of that (I did read the nixos manual). I was talking of configuration options for single nix packages, of which (IIRC) the nix- manual itself says that unforntunately there is currently no way to query them. Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Some NixOS beginner questions...
Kirill Elagin wrote: > On Mon, Mar 31, 2014 at 1:35 AM, Kirill Elagin > wrote: >> >> You'll have to look it up in `pkgs/top-level/all-packages.nix`. >> Either clone the nixpkgs repository or you can find a copy at >> `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`. >> > > Oh, and there is also `nix-env -qa` thing, that lists all the available > packages. > It has `-P` flag that tells it to print attribute paths. Combined with > grep this lets you find what you need: > > % nix-env -qaP | grep kdepim > nixos.pkgs.kde4_prev.kdepim > kdepim-4.10.5 > nixos.pkgs.kde4.kdepim > kdepim-4.11.5 > nixos.pkgs.kde4_next.kdepim > kdepim-4.12.2 > ... Thanks, this is very useful! Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Some NixOS beginner questions...
Kirill Elagin wrote: > On Mon, Mar 31, 2014 at 1:01 AM, Ben Franksen > wrote: > >> (2) It seems impossible to install NixOS w/o installing the boot loader. > > Hm, > > boot.loader.grub.enable = false; Nope, that's what I tried first, of course, nixos-install complains that you must enable it. >> (3) I love the declarative approach to system configuration. However, for >> a newcomer it is hard to find out how to configure his favorite packages. >> What >> options are there and how can I set them? Is looking at the 'sources' >> (for the package's nix expressions) the only way? > > Yes, I guess, looking at the expressions is the only option right now. > You set them in `~/.nixpkgs/config.nix`. That's the file where your > configuration options for > packages and all your package overrides go. OK, I'll bite this bullet then. >> Related question: I tried to >> put pkgs.kdepim into environment.systemPackages, which failed for no >> apparent reason. Then I stumbled (really, by accident) over a nix snippet >> where it said pkgs.kde4. and indeed adding the .kde4 >> worked. How can I find out how packages are organised in the pkgs nix >> expression? > > You'll have to look it up in `pkgs/top-level/all-packages.nix`. > Either clone the nixpkgs repository or you can find a copy at > `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`. Ok, thanks. >> (4) How do I fix something if it is broken? Note I don't want to upgrade >> my whole NixOS configuration to the unstable branch (as all the >> documentation suggests), just make one or two fixes in certain packages >> (e.g. the nedit package is currently broken, when starting the binary it >> crashes with some X11 bad opcode error). I expected the nixpkgs >> repository to have a 'stable' branch so I could check that out and work >> from there but such a branch seems >> not to exist. How else can I replace/fix/change the nix expression for a >> single package, leaving the rest of the system as is? >> > > Actually, right now, using stable instead of unstable doesn't make any > sense. Why? I gave NixOS a try in the past, and that (i.e. when I upgraded the nixpkgs to the current head of the repository) was always the point at which everything broke. > That's how I modify packages: > I have a clone of the nixpkgs repository and I modify expressions there. Ok. But I /can/ use the stable branch, and make my changes there, right? I guess it makes sense to have the master branch lying around somewhere, too, for reference at least, so I can check if some fix has already been made there. > Then I > do `nix-env -i` (or whatever I need) with `-f > `. That tells Nix to use _my_ expressions. In > case of `nixos-rebuild` there is `-I` flag. As I said, I really like the declarative system configuration and therefore want to avoid ad-hoc installation using nix-env if possible. However, I guess I can adapt that approach to changing my configuration.nix and then doing a nixos-rebuild. Cheers (& many thanks for answering my questions) Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Some NixOS beginner questions...
Ben Franksen wrote: > (4) How do I fix something if it is broken? Note I don't want to upgrade > my whole NixOS configuration to the unstable branch (as all the > documentation suggests), just make one or two fixes in certain packages > (e.g. the nedit package is currently broken, when starting the binary it > crashes with some X11 bad opcode error). I expected the nixpkgs repository > to have a 'stable' branch so I could check that out and work from there > but such a branch seems not to exist. How else can I replace/fix/change > the nix expression for a single package, leaving the rest of the system as > is? Part of the mystery is solved. I don't use git on a regular basis, so I missed out on the '-a' switch for git branch. I guess 'release-13.10' is the branch I am interested in (please tell me if I am wrong). Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Some NixOS beginner questions...
Hi Everyone I am new to NixOS (but not to Linux), just installed the stable (13.10) version and have a number of questions/comments. (1) Installation with USB stick failed during the first reboot with "waiting for /dev/sdb1 to appear.". (Also, the 'boot form harddisk' entry in the bootloader menu did not exist.) I could install fine after I burned the image on a CD. It would be nice if installation from a USB stick could be made to work. (2) It seems impossible to install NixOS w/o installing the boot loader. I did not want to overwrite what's in my MBR because I have another system (kubuntu) on the machine and did not want to loose my boot menu. I tried installation to some partition but that also failed for various reasons. I know, this is a general problem with grub2, not partcular to NixOS. Anyway, after many failed attempts my solution was to create a btrfs on an unused partition, adding a chainloader entry to my kubuntu-maintained boot menu. A hint how to do that (in the NixOS manual or on the wiki) would be nice, I guess experimental installation of NixOS side-by-side with an existing system is common, considering the experimental status of NixOS. (3) I love the declarative approach to system configuration. However, for a newcomer it is hard to find out how to configure his favorite packages. What options are there and how can I set them? Is looking at the 'sources' (for the package's nix expressions) the only way? Related question: I tried to put pkgs.kdepim into environment.systemPackages, which failed for no apparent reason. Then I stumbled (really, by accident) over a nix snippet where it said pkgs.kde4. and indeed adding the .kde4 worked. How can I find out how packages are organised in the pkgs nix expression? (4) How do I fix something if it is broken? Note I don't want to upgrade my whole NixOS configuration to the unstable branch (as all the documentation suggests), just make one or two fixes in certain packages (e.g. the nedit package is currently broken, when starting the binary it crashes with some X11 bad opcode error). I expected the nixpkgs repository to have a 'stable' branch so I could check that out and work from there but such a branch seems not to exist. How else can I replace/fix/change the nix expression for a single package, leaving the rest of the system as is? Cheers Ben -- "Make it so they have to reboot after every typo." -- Scott Adams ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] nix configuration language
Ben Franksen wrote: > Checking out the nix sources via git, I can't even build it: > > ben@sarun[1]: .../tmp/nix > git status > # On branch master > nothing to commit (working directory clean) > ben@sarun[1]: .../tmp/nix > ./bootstrap.sh > autoreconf: Entering directory `.' > autoreconf: configure.ac: not using Gettext > autoreconf: running: aclocal --force > autoreconf: configure.ac: tracing > autoreconf: configure.ac: not using Libtool > autoreconf: running: /usr/bin/autoconf --force > configure.ac:59: error: possibly undefined macro: AC_DISABLE_SHARED > If this token and others are legitimate, please use > m4_pattern_allow. See the Autoconf documentation. > configure.ac:60: error: possibly undefined macro: AC_ENABLE_STATIC > configure.ac:85: error: possibly undefined macro: AC_DISABLE_STATIC > configure.ac:86: error: possibly undefined macro: AC_ENABLE_SHARED > configure.ac:87: error: possibly undefined macro: AC_PROG_LIBTOOL > autoreconf: /usr/bin/autoconf failed with exit status: 1 > > I do have autoconf, automake etc installed at version 2.68. Found the solution: I did not have libtool installed. After installing libtool it works. Cheers Ben ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] nix configuration language
Marc Weber wrote: > Excerpts from Ben Franksen's message of Mon Apr 23 12:42:14 +0200 2012: >> I don't know how I can execute a 'nix-language script'. Is there a tool >> in the nix package that I could use for that? > the nix suite ships with test cases. If you look at those you'll find > that its nix-instantiate --eval-only [--strict] foo.nix which evaluates > nix code and eventually writes .drv files or such. Thank you, this is exactly what I needed to know. Now I can start to play with the idea... > nix-build basically does a nix-instantiate and the a nix-store --realise > .drv file (maybe its hardcoded in C). > > If you use Vim you may want to look at vim-addon-nix which provides > default mappings for running nix via vim-addon-actions. > It also syntax checks the nix files on buf write. Cool. Will look at this. > Now it depends on which output you need and how to do what would be > called "IO" Haskell, or which is nixos-rebuild for nixos - thus how to > do something with your description (which might be called evaluated > result). Yes. > Eg the "nix" way would be create a store paths such as > > $out/bin/realise > $out/machine1/target-ssh-data > $out/machine1/configuration.nix > > where the realise script tries to connect to the target machines, > copying over the configuration then running nixos-rebuild. We do have a self-written tool for distributing the configuration in a quasi-immutable way, similar to nix (but a lot less elegant). The problem with nix is that it is a bit too radical for our purposes, i.e. we do not want it to manage binaries, only the configuration data. > However in the real world you have to cope with state - eg do HTTP > forwarding until domain servers caches time out if you move web sites, > and drop that after a couple of days etc. What we do is very different from web/cloud stuff. It /is/ a distributed system, but rather one for which you would want to use Erlang (though we do not, unfortunately, else we'd use that for configuration, too). > I'm not entirely sure - maybe what you're looking for is what disnix is > about: http://nixos.org/disnix/ I looked at disnix, and it is certainly interesting, but as it builds on nix it is most probably overkill, even more so than nix itself is. >> Do you think it would be very difficult to extract the language from the >> nix source tree? > define "very difficult". Checking out the nix sources via git, I can't even build it: ben@sarun[1]: .../tmp/nix > git status # On branch master nothing to commit (working directory clean) ben@sarun[1]: .../tmp/nix > ./bootstrap.sh autoreconf: Entering directory `.' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --force autoreconf: configure.ac: tracing autoreconf: configure.ac: not using Libtool autoreconf: running: /usr/bin/autoconf --force configure.ac:59: error: possibly undefined macro: AC_DISABLE_SHARED If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.ac:60: error: possibly undefined macro: AC_ENABLE_STATIC configure.ac:85: error: possibly undefined macro: AC_DISABLE_STATIC configure.ac:86: error: possibly undefined macro: AC_ENABLE_SHARED configure.ac:87: error: possibly undefined macro: AC_PROG_LIBTOOL autoreconf: /usr/bin/autoconf failed with exit status: 1 I do have autoconf, automake etc installed at version 2.68. > Think about how to use the result. Wait for > more replies. Think about which tool will turn your "configuration" done > with any DSL into something "real". There will probably have to be 'builders' of some sort or the other, depending on the format of the output. I will play around a bit and see... > Another nice keyword might be "chef": > https://github.com/opscode/chef Looking at the web-site I don't think this fits the bill at all. The output from our configuration spec is typically a number of simple text files in various formats not widely known outside the application area. I want a small versatile tool, not a huge 'framework'. And open source, please ;-) > to find related work before spending huge amounts of time this. No fear, I really cannot invest a lot of time to do such a split/re- factoring. But what you wrote above about nix-instantiate --eval-only sounds promising. > You're welcome in any case :) Thanks! Cheers Ben ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] nix configuration language
Marc Weber wrote: > Excerpts from Ben Franksen's message of Mon Apr 23 10:53:26 +0200 2012: >> Would it be sensible or even possible to factor out the configuration >> language from the nix package and distribute it separately? > Such things are always possible. I think you should talk about your use > case first. Large control systems: many devices (hundreds, thousands), most of them quite similar to each other, but now and then the occasional exception to the rule. A few devices very complex, needing hierarchies of configuration data. > Eventually using another lazy language such as Haskell get's > your job done as well. I love programming in Haskell, but it is too large and complex for use as a configuration language. It also too hard to understand for the typical control system engineer. > That's why you should be aware about why the nix > language is the tool of choice for you Well, I see nix (the language) as something like yaml or json (only smaller and cleaner) with the additional feature of user-defined functions (and some built-in ones). Nix has a number of features that make it especially suitable for configuration (no wonder: it's what it has been designed for), for instance the inherit feature which makes it easy to define exceptions to a general rule. > Writing a DSL for configurations can be done easily using many scripting > languages - thus you should be aware why you like nix that much. Mostly it's the purity and laziness that make it IMO superior to the standard scripting languages. We have been using perl, python, tcl, but none of them are as suited to the task as I think the nix language would be. > For prototyping you can use nix and make it create arbitrary > configuration files. I don't know how I can execute a 'nix-language script'. Is there a tool in the nix package that I could use for that? I would not want to fork the configuration language from nix, but rather factor the language out, so users who only need/want the language can still profit from improvements made by the nix development. Do you think it would be very difficult to extract the language from the nix source tree? Cheers Ben ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] nix configuration language
Hi Everyone I hope this is the right forum for my question. I like the nix configuration language and I think it would be a useful tool in applications other than nix and nixos. Would it be sensible or even possible to factor out the configuration language from the nix package and distribute it separately? Cheers Ben ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev