Hello Nixpkgs users, One of the issues of Nixpkgs is the lack of modularity. By modularity, I mean that someone else can provide a complementary source of packages which works side-by-side with Nixpkgs, and that as a user, you can combine multiple of these sources.
# Extending Nixpkgs Today Today, we have a lot of way to extend Nixpkgs: 1. We can import nixpkgs, and add packages on top of it. ```nix let pkgs = import <nixpkgs> {}; in pkgs // rec { foo = pkgs.foo.override { enableBar = true; }; bar = import ./pkgs/bar { inherit (pkgs) stdenv fetchurl foo; }; } ``` 2. We have `packageOverrides` from ~/.nixpkgs/config.nix. ```nix { packageOverrides = pkgs: rec { foo = pkgs.foo.override { enableBar = true; }; bar = import ./pkgs/bar { inherit (pkgs) stdenv fetchurl foo; }; }; } ``` 3. We have `pkgs.overridePackages`. ```nix let pkgs = import <nixpkgs> {}; in pkgs.overridePackages (self: super: { foo = super.foo.override { enableBar = true; }; bar = import ./pkgs/bar { inherit (self) stdenv fetchurl foo; }; }) ``` But none of these approaches can easily be composed with other sources doing similar things. # Extending Nixpkgs with Overlays Fortunately, the recent improvements of Nixpkgs inner structure [2,3,4,5], highlights that we have a list of layers which are stacked one on top of the others. These layers are literally what composes Nixpkgs, we could technically rewrite the entire Nixpkgs collection into thousands of these layers, where each one handles a single package at a time, but this would be inefficient. On the other side, we could expose this layering mechanism to external sources of Nixpkgs, and gain the modularity that Nixpkgs never had. This is what I did with the `overlays` pull-request [1]. An overlay is literally an additional layer added on top of Nixpkgs, within the fix-point of Nixpkgs. The same principles was already used by `pkgs.overridePackages` without adding the composition ability of overlays. An overlay would look something like [6]: ```nix self: super: { foo = super.foo.override { enableBar = true; }; bar = import ./pkgs/bar { inherit (self) stdenv fetchurl foo; }; } ``` Which is essentially the minimal part of all the previous examples, and more over the exact same way we represents the different layers within Nixpkgs. The modularity aspect of overlays comes from the fact that overlays are listed, either in the directory, or as argument of nixpkgs. By default the ~.nixpkgs/overlays/ directory is considered unless NIXPKGS_OVERLAYS specify otherwise. This directory should only contain Nix expression files (or directory with a default.nix) which are overlays, with the same layout as the example above. One can also use a symbolic link to where each of these overlays are installed on your file system. The directory is converted in a list of elements, which are ordered based on the alphabetical order of the elements within the directory. [1] https://github.com/NixOS/nixpkgs/pull/21243 [2] https://github.com/NixOS/nixpkgs/pull/9400 [3] https://github.com/NixOS/nixpkgs/pull/14000 [4] https://github.com/NixOS/nixpkgs/pull/15043 [5] With the help of Peter Simons' extend function. [6] https://github.com/nbp/nixpkgs-mozilla/blob/nixpkgs-overlay/moz-overlay.nix # Planning for the future. As part of this modification, I intend to remove the `pkgs.overridePackages` function, as this one can literally be replaced by the following Nix expression: ```nix let pkgs = import <nixpkgs> {}; in import pkgs.path { overlay = [ (self: super: { foo = super.foo.override { enableBar = true; }; bar = import ./pkgs/bar { inherit (self) stdenv fetchurl foo; }; }) ]; } ``` One additional reason to remove `pkgs.overridePackages` function, is that the creation of this function also stands as the function which adds Nixpkgs fix-point. The `pkgs.overridePackages` function does not play nicely with the `security-updates` branch, as for the applying patches to packages, we have peels the fix-point out, as well as this function. Thus, there would be no way to benefit from the security updates while using `pkgs.overridePackages`, without making it much more complex. On the other hand, the overlays system play nicely with the `security-updates` branch, as it add layers within the existing fix-point, and thus peeling the fix-point does not change the overlays. # If you are not yet convinced A while ago, I suggested to add modularity to NixOS [7], and I personally consider it as a great success for one reason. This reason is allow any user to extend NixOS without changing syntax, and without changing NixOS sources. Adding overlays would add the same benefits to Nixpkgs, such as giving the opportunities to multiple companies to distribute software [6 (above)] without providing a full clone of Nixpkgs them-self. [7] http://lists.science.uu.nl/pipermail/nix-dev/2008-November/001467.html -- Nicolas Pierron http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/ _______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev