The topic of today's posting is: Fixing Build Failures!
I know all about Cabal builds. How can I override a Nix build environment? -------------------------------------------------------------------------- Every Haskell expression expects an argument called "mkDerivation" -- the function that builds the derivation from the build description. You can modify the environment of a build "foo" by replacing mkDerivation with a version that applies some function "f" to the expression first: | foo.override (args: args // { | mkDerivation = expr: args.mkDerivation (expr // f expr); | }) All Haskell configuration modules import the 'lib' library <https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/lib.nix> that defines a bunch of neat little helper functions on top of this basic mechanism. Just check out out the configuration-XYZ.nix files to see some examples of how these functions are used. My build fails with "the following dependencies are missing"! -------------------------------------------------------------- The most common build error is that the configure phase complains about missing dependencies. <http://hydra.cryp.to/build/354149/nixlog/1/raw>, for example, reporting these packages as missing: | Configuring AbortT-transformers-1.0.1... | Setup: At least the following dependencies are missing: | QuickCheck >=2.4 && <2.6, | test-framework ==0.6.*, | test-framework-hunit ==0.2.*, | test-framework-quickcheck2 ==0.2.* "Missing" means that these libraries are available in the build environment, but the build isn't happy about their versions. Usually, the Cabal files specifies upper bounds that our versions exceed, i.e. our packages are *too new*. Now, the big question is whether those upper bounds are justified or whether they exist solely because upstream hasn't updated the Cabal file recently? An easy to test this is to add a "jailbreak" and to run the build again: <https://github.com/peti/nixpkgs/commit/0dd413458ef1a5c05e64bee2462de2edfe0fe620>, If it succeeds now, then this fact should be communicated upstream: <https://github.com/gcross/AbortT-transformers/issues/1>. If you commit the override afterwards, then please include a reference to the upstream ticket in a comment, or at least add a brief comment explaining briefly why that jailbreak was added! A trickier case was reported on the nix-dev mailing list in <http://permalink.gmane.org/gmane.linux.distributions.nixos/15526>: | Configuring cabal-test-quickcheck-0.1.2... | Setup: At least the following dependencies are missing: | Cabal ==1.20.* First of all, let's check which Cabal version we have during the build: | $ nix-shell '<nixpkgs>' -A haskellngPackages.cabal-test-quickcheck.env --command "ghc-pkg list Cabal" | /nix/store/a3dj9sjg5lh9wxl90lj4shp9s3brlscd-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d | Cabal-1.18.1.5 'Cabal' is available, indeed, but our version doesn't meet the "==1.20.*" constraint specified by 'cabal-test-quickcheck'. Curiously enough, the package's Cabal file imposes the following constraints on the library: | library | [...] | build-depends: | Cabal >= 1.16.0 && < 1.21, This is strange, right? Our Cabal library fits into that range! It turns out that there is a test suite -- which we enable by default --, and that says: | test-suite example | [...] | build-depends: | Cabal >= 1.20 && < 1.21, The test suite imposes a narrower constraint than the library itself. An easy way out of this situation is to just disable the testing phase: constraint: <https://github.com/NixOS/nixpkgs/commit/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5>. Similarly, libraries may imposes additional restrictions when certain optional features are enabled during the build, i.e. features controlled by a Cabal flag. The "aeson" library uses this mechanism to choose between "time" versions before and after 1.5: | if flag(old-locale) | build-depends: time < 1.5, old-locale | else | build-depends: time >= 1.5 We have '-fold-locale' hard-coded for that build in hackage-packages.nix (for reasons that will be addressed in a later installment), so aeson wouldn't build with ghc-7.9.x, because that compiler ships time 1.5.0.1. The solution in this case is to disable the "old-locale" for the the 7.9.x branch: <https://github.com/NixOS/nixpkgs/blob/2ff8d1940f0986b572760edf1923539687f41ac8/pkgs/development/haskell-modules/configuration-ghc-7.9.x.nix#L52>. I tried "jailbreak", but that broke the build even more than before! -------------------------------------------------------------------- Removing dependency restrictions from a Cabal file sensibly is hard, and jailbreak-cabal sometimes doesn't succeed doing that. In these cases, it may be worth trying to patch the offending constraints out of Cabal file manually. For example, the build of "darcs" cannot be fixed by jailbreak alone, so instead we added a custom "patchPhase" that uses 'sed' to show the build who's the boss: <https://github.com/NixOS/nixpkgs/blob/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5/pkgs/development/haskell-modules/configuration-common.nix#L91>. If a "jailbroken" build succeeds, does this mean the resulting binary is okay? ------------------------------------------------------------------------------ This is almost always the case. Haskell is a strongly typed language, right? So a successful compiler run actualy means something. In some cases, however, package authors consciously exclude certain versions of the dependencies for other reasons, i.e. because these versions have a bug that their software triggers. In this case, a jailbroken build would succeed, but the binaries that come out of it might be broken in subtle and non-obvious way. Generally speaking, jailbreaking is fine if, and only if, you report the fact that you had to do that upstream to give them a chance to comment. I tried jailbreaking, and the package doesn't compile with the newer dependency. -------------------------------------------------------------------------------- Report this issue upstream. Ask the package authors to, please, release a new version of the library that supports the latest versions of its respective dependencies. Refer them to <http://packdeps.haskellers.com/> if they don't believe you. If they refuse, tell them that they are a bunch of mindless jerks who will be the first against the wall when the revolution comes. If that doesn't help, then it's also possible to build the package with those old versions that they want, apparently, but doing that isn't easy. The magic keyword is "deep override". This subject will be covered in a separate installment. My build failed while "Preprocessing test suite XYZ"? ----------------------------------------------------- This is almost certainly a bug in the package: the test suite tries to import a Haskell module that's missing from the release tarball, i.e. because the Cabal file doesn't mention that file as a required source code. This happens when developers build their code in their Git checkout (which contains the file) but don't whether the build still succeeds in solely from the release tarball that's generated by "cabal sdist". A real-life example of such an issue is <http://hydra.cryp.to/build/349188/nixlog/2/raw>. The proper way to fix this bug is to report it upstream. I've done that at <https://github.com/techtangents/ablist/issues/1>. If the author fixes the problem and releases an update, then the new version will find its way into our package database automatically with 1-2 days. Now, if you can't wait, then it's possible to disable the test suite for this particular build in pkgs/development/haskell-modules/configuration-common.nix: <https://github.com/peti/nixpkgs/commit/1d223754dee2dab61d4d143c1cbdf17289613a10>. If you add an override, then please *add a comment* that refers to an upstream ticket, or at least describe what the problem was that this override is supposed to solve. This is important, because manually added overrides tend to become unnecessary after a while, and then it's very hard to figure out why they were added in the first place and whether it's safe to remove them or not. This bit of extra information can be very helpful! My build fails because of a "missing dependency on a foreign library"! ---------------------------------------------------------------------- Consider the build failure <http://hydra.cryp.to/build/350912/nixlog/2/raw> of the package ALUT: | Setup: Missing dependency on a foreign library: | * Missing C library: alut The corresponding build expression in "hackage-packages.nix" was: | "ALUT" = callPackage | ({ mkDerivation, alut, base, OpenAL, OpenGL }: | mkDerivation { | pname = "ALUT"; | version = "2.3.0.2"; | sha256 = "02kfyb4g7sfjfzqlddxqbjffrj4a0gfrzkisdpbj2lw67j1gq5dp"; | buildDepends = [ base OpenAL OpenGL ]; | extraLibraries = [ alut ]; | configureFlags = [ "-fusenativewindowslibraries" ]; | homepage = "https://github.com/haskell-openal/ALUT"; | description = "A binding for the OpenAL Utility Toolkit"; | license = stdenv.lib.licenses.bsd3; | }) { alut = null; }; The package specifies "alut" in its extraLibraries section, but unfortunately that argument is later hard-coded to "null"! The reason for that "alut = null" override is that the hackage2nix utility -- which generates this expression -- couldn't find any package called "alut" in Nixpkgs. Instead, we have "freealut". This issue is best be fixed in the hackage2nix source code: <https://github.com/NixOS/cabal2nix/commit/dd0e5c08cb8c5bd471db1843e387467eb4e2abd2>. With that change applied to the code, hackage2nix generates the change <https://github.com/peti/nixpkgs/commit/b362dc8fdf038db64c48218a88f207231937bfab>, which fixes the build. Hackage2nix will be discussed in-depth in a later installment. If you run into an issue like this one, then feel free to open a ticket on Github at <https://github.com/NixOS/cabal2nix/issues/new>. Another case was "sfml-audio", which failed with the following error: | Setup: Missing dependency on a foreign library: | * Missing (or bad) header file: al.h The library that provides this header -- openal -- was part of the build environment, but Cabal still wouldn't find the header because it lives in a sub-directory that the package author clearly didn't expect. This problem could be solved by adding the appropriate directory to the header search path: <https://github.com/peti/nixpkgs/blob/0dd413458ef1a5c05e64bee2462de2edfe0fe620/pkgs/development/haskell-modules/configuration-common.nix#L40>. That's it for today! Have fun, Peter _______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev