Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Eelco Dolstra
On 07/05/15 14:29, Daniel Peebles wrote:

 I thought that at some point there was an effort to autogenerate
 (dynamically, so not nix codegen) the list from readDir. 

I made an abortive attempt at reducing the size of all-packages.nix a while 
back:

https://github.com/NixOS/nixpkgs/commit/ece61b7cc803d374e81b1094bd9c1f6d5a9ca5d0

It would allow getting rid of all packages that don't override any default
arguments (i.e. pass { } to callPackage).

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Daniel Peebles
I thought that at some point there was an effort to autogenerate (dynamically, 
so not nix codegen) the list from readDir. If not, it seems like the majority 
of packages could work that way, along with room to override where necessary.



 On May 7, 2015, at 08:10, Wout Mertens wout.mert...@gmail.com wrote:
 
 all-packages.nix is gigantic, and because it's so large, github search 
 doesn't index it.
 
 Would it be a good idea to move the attribute definitions into files per 
 package section?
 
 So everything would still be in the pkgs attrset, only the definitions would 
 be split across multiple files.
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


[Nix-dev] modularize all-packages?

2015-05-07 Thread Wout Mertens
all-packages.nix is gigantic, and because it's so large, github search
doesn't index it.

Would it be a good idea to move the attribute definitions into files per
package section?

So everything would still be in the pkgs attrset, only the definitions
would be split across multiple files.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Wout Mertens
the problem with readDir is slowness due to reading thousands of files...
nix codegen would fix that :)


On Thu, May 7, 2015 at 2:29 PM Daniel Peebles pumpkin...@gmail.com wrote:

 I thought that at some point there was an effort to autogenerate
 (dynamically, so not nix codegen) the list from readDir. If not, it seems
 like the majority of packages could work that way, along with room to
 override where necessary.



  On May 7, 2015, at 08:10, Wout Mertens wout.mert...@gmail.com wrote:
 
  all-packages.nix is gigantic, and because it's so large, github search
 doesn't index it.
 
  Would it be a good idea to move the attribute definitions into files per
 package section?
 
  So everything would still be in the pkgs attrset, only the definitions
 would be split across multiple files.
  ___
  nix-dev mailing list
  nix-dev@lists.science.uu.nl
  http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Daniel Peebles
No need to read all the files, right? You import them lazily based on what 
readDir returns.



 On May 7, 2015, at 08:35, Wout Mertens wout.mert...@gmail.com wrote:
 
 the problem with readDir is slowness due to reading thousands of files... nix 
 codegen would fix that :)
 
 
 On Thu, May 7, 2015 at 2:29 PM Daniel Peebles pumpkin...@gmail.com wrote:
 I thought that at some point there was an effort to autogenerate 
 (dynamically, so not nix codegen) the list from readDir. If not, it seems 
 like the majority of packages could work that way, along with room to 
 override where necessary.
 
 
 
  On May 7, 2015, at 08:10, Wout Mertens wout.mert...@gmail.com wrote:
 
  all-packages.nix is gigantic, and because it's so large, github search 
  doesn't index it.
 
  Would it be a good idea to move the attribute definitions into files per 
  package section?
 
  So everything would still be in the pkgs attrset, only the definitions 
  would be split across multiple files.
  ___
  nix-dev mailing list
  nix-dev@lists.science.uu.nl
  http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Wout Mertens
yes indeed that looks nice, it would eliminate some 3855 lines of 9351
non-empty lines... With more probably a small refactor away.

On Thu, May 7, 2015 at 3:08 PM Daniel Peebles pumpkin...@gmail.com wrote:

 What went wrong? I think that's what I was thinking of



  On May 7, 2015, at 08:50, Eelco Dolstra eelco.dols...@logicblox.com
 wrote:
 
  On 07/05/15 14:29, Daniel Peebles wrote:
 
  I thought that at some point there was an effort to autogenerate
  (dynamically, so not nix codegen) the list from readDir.
 
  I made an abortive attempt at reducing the size of all-packages.nix a
 while back:
 
 
 https://github.com/NixOS/nixpkgs/commit/ece61b7cc803d374e81b1094bd9c1f6d5a9ca5d0
 
  It would allow getting rid of all packages that don't override any
 default
  arguments (i.e. pass { } to callPackage).
 
  --
  Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
  ___
  nix-dev mailing list
  nix-dev@lists.science.uu.nl
  http://lists.science.uu.nl/mailman/listinfo/nix-dev
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Eelco Dolstra
Hi,

On 07/05/15 15:02, Daniel Peebles wrote:

 No need to read all the files, right? You import them lazily based on what
 readDir returns.

Yes, but you still need to traverse the directory tree, which itself takes a
long time on non-SSD disks.

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Wout Mertens
well, that traversal could be automated at commit time? It's just an
optimization...


On Thu, May 7, 2015 at 3:24 PM Eelco Dolstra eelco.dols...@logicblox.com
wrote:

 Hi,

 On 07/05/15 15:02, Daniel Peebles wrote:

  No need to read all the files, right? You import them lazily based on
 what
  readDir returns.

 Yes, but you still need to traverse the directory tree, which itself takes
 a
 long time on non-SSD disks.

 --
 Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


Re: [Nix-dev] modularize all-packages?

2015-05-07 Thread Daniel Peebles
What went wrong? I think that's what I was thinking of



 On May 7, 2015, at 08:50, Eelco Dolstra eelco.dols...@logicblox.com wrote:
 
 On 07/05/15 14:29, Daniel Peebles wrote:
 
 I thought that at some point there was an effort to autogenerate
 (dynamically, so not nix codegen) the list from readDir.
 
 I made an abortive attempt at reducing the size of all-packages.nix a while 
 back:
 
 https://github.com/NixOS/nixpkgs/commit/ece61b7cc803d374e81b1094bd9c1f6d5a9ca5d0
 
 It would allow getting rid of all packages that don't override any default
 arguments (i.e. pass { } to callPackage).
 
 -- 
 Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev