>I discovered Nix and NixOS a few weeks ago and love what I see. I love >the execution of immutable and standalone environments. IMO Nix is >packaging done right. There is so much potential for Nix and NixOS on so >many fronts.
A disclaimer from my side: I am not speaking in the name of the entire project, and I never had a right to do so. But I am an experienced contributor. >I'm curious about what people in the Nix community feel about the >applicability of Nix to the deterministic and verifiable builds problem >space. There are some things that are hard to check, and there are some tradeoffs that stop us from using the most deterministic system we have manpower to achieve. I think the biggest of our problems wouldn't impact you as much (and maybe, in a sense, your using Nix would _solve_ them by providing a large buildfarm exporting the most deterministic variety of core tools up to NixPkgs Firefox). > From my limited experience with Nix, I *think* the determinism part is >on solid ground: if you assemble the same environment on different >machines and a package is capable of building deterministically, you're >in a good position to achieve a bit-identical result. (We really only >care that Firefox's output is deterministic - it doesn't matter too much >if the toolchain and dependent packages aren't deterministic as long as >Firefox builds the same. Although, having bit-identical output >everywhere would be awesome.) Deterministic output means that something has to be done about profile- guided optimistion. It is a subject of an old discussion in our community as some people would like bit-perfect builds and others think that PGO improves performance significantly and should be used. In our case the main discussion is about letting GCC to PGO itself during bootstrap. >I *think* you can facilitate determinism over time by providing your own >channel or version control repository of Nix expressions. Then, you tell >people "checkout version X and install the 'firefox-build-env' package." >You attain trust via regular auditing of the Nix expressions. Is it >really this simple? How do you achieve full isolation of a Nix >expressions "database" from other installed channels/sources? Is having >a version controlled repository of Nix expressions that can be used to >derive the same (hopefully identical) packages over time something that >people do? e.g. if I check out a copy of the Nix expressions from a year >ago and realize the "firefox-build-env" package, I should compile the >environment as it was a year ago, right? Yes, that's right. Let me summarize my impression of what Nix reliably provides. 1) If you check out an old version of the package instruction repository, exact same build instructions will be generated. The same tarballs will be fetched from the same lists of mirrors and verified against the same checksums, then unpacked and built with the same patches applied and the same configure flag order etc. Tarball availability is an issue here; I guess Mozilla can solve this part of the problem for all the tarballs needed for Firefox bootstrap build. Also there is a problem of «trusting trust» and bootstrap build; our basic build environment is built from a binary set of bootstrap tools, which is the checksumed build environment you want to avoid. To solve this, there is a procedure of building these bootstrap tools and if you disable all non-determinism it should converge to the same tarball when you take two acceptable bootstrap toolsets, build the entire working environments from these, and use them to generate two new sets of bootstrap tools. As for irrelevant-for-the-job tools: Binary caches say «if you perform build X you get output Y anyway, you can save the time and trust me». You will probably want to only use binary caches under your complete control. Hope fully some of these would also serve the toolchain and library chain to the world… Channels give you some snapshots of expressions and also some binary caches. The expression snapshots do not have any effect on your results if you take care to evaluate only the expressions from the VCS checkout during the release process. You don't really need to use channels in this specific task. You can provide some, of course. 2) Even with the same commands there is an issue of build determinism. We have quite reliable build isolation using chroot environments (and hard-to-guess paths). Whatever optional dependencies of a package you install, if they are not listed in the package descriptions, it is unlikely that the package will be able to notice them (unless user helps it in some way in runtime). There is the issue of «what /bin/sh gets used in chroot», but in your case you can use the simple configuration options there are to fix the /bin/sh once for every Firefox version. We have a few approaches for mitigating time information leak… They probably need some testing. One of the problems is when some debug information gets leaked into the final package and it contains build directory paths. We want them to correcspond to external paths for easier debugging, and so they may leak information. With minimal care they turn out to be completely deterministic, though. Many issues are discussed in https://github.com/NixOS/nixpkgs/pull/2281 which was already mentioned. With Nix it is relatively easy to check determinism step-by-step (rebuilding a single package at a time at different machines and multiple times) when you have some buildfarm capacity to spare; so you can do some test runs and create a summary. And post it at AreWeDeterministicYet.net… When there are no problems that actually change the output, it is unlikely they will reappear until upstream changes the build system. 3) Deterministically producing a Firefox intended for non-NixPkgs installation (i.e. not referring to /nix/ for libraries) is a separate question. I would say that it is quite easy: we have a few chroot generators, so Firefox can simply be built — or just get its ELF header edited — inside a chroot mimicking the layout you want by bind-mounting /nix/ and then simply symlinking the libraries into /usr/lib. Here you won't get any new determinism problems (or any non-trivial problems at all). Michael Raskin _______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev