Dear Haskellers `intersect` Nixers, this is the first installment of a series of postings to describe technical aspects of the re-factored Haskell infrastructure. When this is all done, I intend to use these articles to create some kind of *gasp* user documentation for the Nixpkgs manual. So I encourage everyone to point out of omissions and technical errors, or to ask for clarification if something is vague or misleading. I've chosen an FAQ-style approach because I felt that this is easiest for me to write. Hopefully this translates into the document being easy to read, too! Here we go ...
Why should I care about this "new infrastructure"? -------------------------------------------------- The new code will break evaluation of any Haskell-related configuration you may have in ~/.nixpkgs/config.nix or /etc/nixos/configuration.nix. Privately generated cabal2nix expressions will cease to compile. Installations that rely on ghc-wrapper finding GHC libraries automatically in your ~/.nix-profile are obsolete. If you use this approach, you won't be able to update your profile anymore. Duh, I don't want that! ----------------------- You can stick to the established 'haskellPackages' set, if you prefer. The new code exists separately in 'haskellngPackages', and it's invisible to "nix-env -i <package-name>" and "nix-env -u". If you don't make a conscious effort to switch, you won't see any of it. So I'll never have to update if I don't want to? ------------------------------------------------ My guess is that 'haskellPackages' and 'haskellngPackages' will co-exist for a while. Personally, I switched to Haskell NG, though, and I won't maintain any packages in the old hierarchy anymore. I suppose other contributors will do the same thing. Once you've converted your setup to 'haskellngPackages', there's no reason to look back, really. In other words, you don't have to convert your setup *right now*, but at some point in the future you will probably have to. Is it difficult to migrate to Haskell NG? ----------------------------------------- Users of 'ghcWithPackages' can easily switch back and forth between both package sets. Your ~/.nixpkgs/config.nix file probably defines an attribute like this one: | { | packageOverrides = super: let self = super.pkgs; in | { | haskellEnv = self.haskellPackages.ghcWithPackages (p: with p; [ | async attoparsec caseInsensitive fgl GLUT GLURaw haskellSrc | ]); | }; | } Change this definition as follows to switch to Haskell NG: | { | provideOldHaskellAttributeNames = true; | | packageOverrides = super: let self = super.pkgs; in | { | haskellEnv = self.haskellngPackages.ghcWithPackages (p: with p; [ | async attoparsec caseInsensitive fgl GLUT GLURaw haskellSrc | ]); | }; | } There are only two differences: the "ng" bit in the name of the package set, and the new 'provideOldHaskellAttributeNames' config attribute. (Note that this lives at the top level of ~/.nixpkgs/config.nix -- not inside of 'packageOverrides'.) Users of the ghc-wrapper have to make a greater effort, I'm afraid, because ghc-wrapper no longer exists in the NG code. You'll have to switch to 'ghcWithPackages' first. You may find this article helpful for the process: <http://permalink.gmane.org/gmane.linux.distributions.nixos/15161>. What does 'provideOldHaskellAttributeNames' do? ----------------------------------------------- The package 'cabal-install' has the attribute 'cabalInstall' in the old package set. In the new one, however, the attribute is now called 'cabal-install' -- just like the package. provideOldHaskellAttributeNames enables a compatibility layer that defins the old camel-case style names on top of the new ones. This allows you to use the same configuration for both the new and old package sets. Once you feel ready to migrate permanently to NG code, you can switch the compatibility layer off, to ensure that all your definition use the new names consistently. Why is the ghc-wrapper gone? ---------------------------- ghc-wrapper's underlying promise is that you can install a random Haskell package into your Nix profile, and GHC will pick it up from there automatically. Now, this worked reasonably well for many packages, but for may other packages it didn't work, and those cases caused us a lot trouble, bug reports, and heated discussions in the past, and we ended up adding more and more layers of plumbing around xmonad, ghc-mod, and whatnot in an attempt to rescue the idea that GHC would somehow find packages automatically without being told about them. In the NG infrastructure, this whole notion has been abandoned. The proper way to install ghc is trough the 'ghcWithPackages' function. It knows about exactly those packages that you tell it to. It's slightly more effort to define your Haskell environment this way, but what you get in return for this extra effort is a totally deterministic compiler that doesn't depend on any wrapper scripts, it produces predictable results independent of what else you may or may not have installed, and it interacts just fine with cabal-install -- something the ghc-wrapper never managed to get right. When will the new code be available? ------------------------------------ It already is. 'haskellngPackages' was introduced to 'master' by commit 54baa53d on 2015-01-07, and it has now been picked up by the "unstable" channel. Why would I want to migrate? What are the advantages? ----------------------------------------------------- The NG package set contains the latest version of every package available from Hackage, plus some older versions that we need to make legacy stuff compile. In total, we deliver in excess of 7,500 packages. Just look at https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/development/haskell-modules/hackage-packages.nix if you don't believe it. The 'hackage-packages.nix' file is generated automatically by the hackage2nix utility -- which I'll describe in detail in a later installment --, and it allows us to easily track Hackage on a daily basis if we want to. Haskell NG supports GHC 7.10.x. Haskell NG allows you to configure the hell out of your installation. You can add new packages easily, you can build your own customized package sets, you can change Cabal flags, compiler arguments, or any other aspect of how a package is built by means of the 'overrideCabal' function without editing the source of Nixpgs. The file https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-common.nix illustrates this point nicely. It's possible to specify deep overrides with 'overrideScope', which I'll describe in detail in a later installment. To get a first impression, check out https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-ghc-7.8.x.nix#L61 for a concrete example of a deep override that replaces the version of mtl 2.1.x with 2.2.x in the context of the 'amazonka' package family. Is there a binary cache available for the NG packages? ------------------------------------------------------ Haskelll NG is built by http://hydra.nixos.org/jobset/nixpkgs/haskell-updates and http://hydra.cryp.to/jobset/nixpkgs/haskell-ng. You can get binaries from these servers by adding the command line flags --option extra-binary-caches http://hydra.nixos.org --option extra-binary-caches http://hydra.cryp.to to your invocations of nix-build, nix-env, etc. The normal binary cache that backs the "unstable" channel does not yet have binaries for NG. Does Haskell NG support compilers other than GHC 7.8.4? ------------------------------------------------------- Yes, there is a family of package sets 'haskell-ng.packages.ghcXYZ' that define package sets for GHC versions 6.10.4, 6.12.3, 7.0.4, 7.2.2, 7.4.2, 7.6.3, 7.8.4, and HEAD a.k.a. 7.10.1-rc1. For example, to install a version of GHC 7.6.3, define ghc763Env = self.haskell-ng.packages.ghc763.ghcWithPackages (p: with p; [ mtl network parallel parsec QuickCheck random regex-base regex-compat regex-posix stm syb text zlib alex cabal-install cpphs happy ghc-paths monad-par ]); in ~/.nixpkgs/config.nix. Note that the compiler-specific package sets do *not* have the compatibility layer. Inside of 'haskell-ng.packages.ghcXYZ', only the new attribute names exist! Furthermore, the package sets for GHC 7.6.3, 7.8.4, and 7.10.1-r1 are the only ones that can be expected to work. I have never tested any pre-7.6.3 compiler. Getting those package sets to work well is almost certainly going to be a bit of an effort. What can I do to help develop Haskell NG? ----------------------------------------- Use it! Switch your configuration to the 'haskellngPackages' package set and try whether you can update your profile to the new version. If you cannot, then please report the issue in this e-mail thread so that we can try to figure out what's going wrong. As of today, we can compile approximately 50% of Hackage successfully. See http://hydra.nixos.org/jobset/nixpkgs/haskell-updates for the current state of affairs. Obviously, we would like to get that percentage way up! Patches, bug reports, and feature requests will help us accomplish that. Have fun, Peter _______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev