Hi, Currently we use PYTHONPATH <https://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH> a lot in Nixpkgs to let applications and the interpreter find Python modules. This typically works fine, but there are problems with this method and so I would like to get rid of it and use only `python.withPackages` which uses `python.buildEnv`.
The two main issues with the use of PYTHONPATH: 1. Before we used `--prefix $PYTHONPATH`, but this would leak PYTHONPATH into subprocesses, which is especially problematic when both parent and child depend on Python but of different versions. `--set $PYTHONPATH` would solve that issue, but that makes it impossible to add other (impure) paths, which is an important feature. This also breaks alternative shells like `ipython`. This issue is currently solved by extending `sys.path` in the Python applications. 2. Limits the use of multiple outputs. When moving a module of a package into a separate output it becomes problematic to load this again, since just adding the module to PYTHONPATH typically doesn't work because the order matters. While Python modules are typically small, and build fast, rebuilding can take a lot of time in cases like `matplotlib` which supports multiple backends and is depended on by quite some packages. A method that is more reliable is building an environment with symbolic links to all the modules, and wrapping the applications with a wrapper that sets PYTHONHOME <https://docs.python.org/2/using/cmdline.html#envvar-PYTHONHOME>. This is exactly what `python.buildEnv` does, and it solves both 1) and 2). `PYTHONPATH` is mainly constructed with the Python interpreter setupHook. It is used in `buildPythonPackage` for building the package, and after installing it is extended so the tests can run. Furthermore, `PYTHONPATH` is set by the `setupHook` when using `nix-shell` like `nix-shell -p pythonPackages.numpy`. I think we can get rid of the setupHook. For the building we can create an env. This would be able to support multiple outputs as inputs, but will be more expensive than setting PYTHONPATH. For the tests we do add the newly constructed package to PYTHONPATH; there's no way around it and it doesn't cause any problems either. The biggest impact will be on how `nix-shell` is used. It won't be possible anymore to use it as shown before, instead `nix-shell -p 'python.withPackages(ps:[ps.numpy])'` would have to be used. While it is possible to keep the setupHook (or use it as a shellHook) it will be unreliable in the case of multiple outputs. What do you think about this change? Do you see any problems with it? Any other suggestions? Freddy
_______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev