I wrote an issue for Cabal that goes into more detail at what the Cabal end of things might look like: https://github.com/haskell/cabal/issues/3882
On Sun, Sep 18, 2016 at 11:48 PM, John Ericson < john_eric...@alumni.brown.edu> wrote: > After much pondering, I've decided the best way to work with > language-specific build managers is for them to come up with the end-to-end > build plan, and us to build each package in that build plan. > > Our current method with Haskell of trying to replicate each package and > all the possible ways it can be configured just doesn't scale very well > with respect to configuaration options. Things like `if impl(ghcjs):` have > long remained unimplemented, and implementing them bakes more and more > build-plan resolution machinery into machine until we approach > re-implementing cabal-install. [Peti has discussed this in depth in > https://github.com/NixOS/cabal2nix/blob/master/doc/03- > map-cabal-files-to-nix-without-information-loss.md.] > > At the same time, or current method isn't great for developing multiple > packages simultaneously. The default thing to do is make a package set with > each one, but if one doesn't handle the source very carefully, a change to > any of them will cause them all to be rebuilt. Furthermore, one needs > switch shells per each package---which is a pain in itself---and that > prevents the use of neat things like `stack repl` which interprets multiple > packages at once for seamless concurrent development. [I discussed part of > this in https://github.com/NixOS/nixpkgs/issues/10902 a bit.] > > `stack --nix` at first glance seems to solve both of these areas, but in > fact comes with it's own set of downsides. Yes, since Stack manages the > build plan less knowledge is needed on the nix side, but stack also > *executes* the build plan. This means that both Nix's superior caching is > not utilized, and per-package patches (like foreign dependencies) are > unavailable. [See https://github.com/NixOS/nixpkgs/issues/15995.] > > What I'm proposing should give us the best of both worlds in that language > package manager makes most of the decisions, but we still have an > opportunity to cache and patch, and development and deployment workflows > overlap as much as possible. This methodology could in principle work with > any language and its package manager, but I single out Cabal+Haskell > because the new Nix-style workflow (http://cabal.readthedocs.io/ > en/latest/nix-local-build.html) makes it the most ready. > > For both deployment and development, the idea is we first run > cabal-install to compute a build plan. During this stage it in principle > only needs access to .cabal files / the Hackage index and a project > config---certainly not any already built binaries like today. Next we > instruct cabal-install to decompose the plan into "sub plans" for each > dependency/would-be system package---this is the biggest change needed for > cabal-install, but one that any system package manager could leverage, not > just us. After that, using a reworked cabal2nix, we convert each of those > plans to a Nix derivation---this may include turning hackage deps into > local deps with their source fetched by nix. Finally, each of those > derivations can use `cabal new-install` to install their root package, and > the caches/package DBs can be passed downstream and combined just like > today. > > There are two important concepts to note here: First, naively, the use of > entire sub plans in each Nix derivation would cause cabal-install to > rebuild each dependency for package in the plan that depends that needs it, > but since the Nix derivations will also depend on each other, `cabal > new-build`'s own caching should kick in so each Nix derivation does in fact > correspond to each Haskell package being built. Second, while cabal-install > can be instructed to build any package in a plan, the decomposition of > build plans is necessary so build plans that differ in ways that don't > affect common dependencies still end up sharing nix-executed builds. In > both cases, we rely on cabal-install's imitation of our methodology. > > For development, we leverage the new nix-style build's preexisting notion > of a local vs external package. For convenience, it's probably best to let > cabal-install devise the build plan impurely (this is analogous to > non-restricted mode for Nix expr evaluation), and not copy any local > sources to the Nix store. Then, only external packages (and only those > which are dependencies of the current goal) should be built by Nix in > accordance with the plan above. In a shell with those built, cabal-install > can be run again building local packages normally and locally using its > more fine-grained per-file hashing and enabling tricks analogous to > `stack-repl`. While the default shell derivation will need tweaking if any > local packages have special requirements like foreign dependencies, this is > better than with `stack --nix` where the shell needs tweaking if *any* > dependency has such special requirements. > like today. > > For deployment we don't need to worry about local vs external packages, > and on the cabal-install side `cabal new-install` should treat them all the > same IIUC. However, we do need to make sure that the build plan resolution > can be done in a pure environment: this means making sure all local > packages locations are in the Nix store and the Hackage index is > pre-downloaded by Nix. After that however no further complications arise. > > There's a lot of work with nixpkgs and cabal2nix required for all this, > but comparatively little upstream for cabal-install. And the end result I > believe is hugely worthwhile, solving just about every pain point with our > existing infrastructure. > > John > > N.B. not quite sure on the division of labor between Cabal and > cabal-install with the new system, and erring on the side of cabal-install. >
_______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
