Hello again,
since I haven't got any replies yet I'm trying to make myself a bit more
precise now. I consider the behaviour described in my original posting a
bug. I posted to this list because the setuptools docs say Please use
the distutils-sig mailing list [3] for questions and discussion about
setuptools, and the setuptools bug tracker [4] ONLY for issues you have
confirmed via the list.
I have two questions, which I hope some expert here can answer:
1) How do I properly handle PEP420 namespace packages with setuptools
and pip (Python = 3.3) assuming the scenario described below?
2) Is the behaviour of setuptools/pip as I encountered it intended
(including the solution I found) or is it a bug that should be filed?
So here is, with some more detail, what I am trying to do: I am using
Python 3.4. There are two projects with modules in the namaspace
coolpkg. These implement the sub-packages coolpkg.bar and and
coolpkg.foo respectively. Both projects have (as allowed by PEP420) no
coolpkg/__init__.py. Both projects have a setup.py using setuptools
(15.0) to create a source distribution and will be installed using pip
(6.1.1).
In addition to that, there is another submodule coolpkg.baz, which will
not be packaged/installed using setuptools/pip. Instead the folder
containing it, will we added to the PYTHONPATH.
Here is the complete layout of the projects and the additional module.
project-bar/
project-bar/setup.py
project-bar/src
project-bar/src/coolpkg
project-bar/src/coolpkg/bar
project-bar/src/coolpkg/bar/__init__.py
project-bar/src/coolpkg/bar/barmod.py
project-foo/
project-foo/setup.py
project-foo/src
project-foo/src/coolpkg
project-foo/src/coolpkg/foo
project-foo/src/coolpkg/foo/foomod.py
project-foo/src/coolpkg/foo/__init__.py
shady-folder/
shady-folder/coolpkg
shady-folder/coolpkg/baz
shady-folder/coolpkg/baz/__init__.py
shady-folder/coolpkg/baz/bazmod.py
My goal is to have a runtime configuration such that modules
coolpkg.foo.foomod, coolpkg.bar.barmod, coolpkg.baz.bazmod can all be
imported.
Test 1) (just a basic test to verify general setup)
Add project-bar/src, project-foo/src and shady-folder manually to
sys.path
Result: works (obviously)
For further tests:
Create source distributions with setup.py sdist for project-bar and
project-foo, install them with pip and put shady-folder on PYTHONPATH.
Declare packages=['coolpkg', 'coopkg.foo'] and packages=['coolpkg',
'coopkg.bar'] in respectice setup.py, since find_packages does not play
well with PEP420.
Test 2)
Delcare namespace_packages=['coolpkg'] in setup.py of each project
Result: coolpkg.foo.foomod and coolpkg.bar.barmod can be imported, but
importing coolpkg.baz.bazmod fails
Suspected reason of failure: pip creates *-nspkg.path files which
prevent coolpkg.baz from being found (breaking PEP420)
Test 3)
DO NOT delcare namespace_packages=['coolpkg'] in setup.py of each
project
Result: all modules can be imported
To put it bluntly:
The setup of test 2) is the one I would expect to work when writing a
setup.py as described by the documentation, but pip unneccessarily
creates *-nspkg.path files which undo the whole point of PEP420.
The setup of test 3) seems like an ulgy hack as we fool pip into not
creating *-nspkg.path files by hiding the existance of the namepsace
package from setuptools.
To summarize my questions stated above:
Which is a bug, which is intended and is there a way to handle
PEP420-compliant packages properly with the current versions of
setuptools and pip.
Kind regards,
Christoph Schmitt
Attachment: Example with scripts, modiy setup.py files according to test
2) and 3)
Am 17.04.2015 um 09:31 schrieb Christoph Schmitt:
I am using the newest versions of setuptools (15.0) and pip (6.1.1)
with Python 3.4. I wanted to satisfy the following two requirements at
the same time, but had some trouble:
A) creating and installing two source distribution tarballs which have
the same top level namespace package (no __init__.py for namespace
package) with setuptools and pip
B) having other packages within the same namespace package outside of
python/site-packages in the PYTHONPATH (not managed by
pip/setuptools)
Since Python = 3.3 supports namespace packages out of the box (PEP420)
this seemed to me like a straightforward task. However, it turned out
to be not. Either, setuptools would not find my modules, pip would not
install them properly or requirement B) was broken. The solution that
worked for me, was to omit the
namespace_packages=['whatever_namespace'] declaration in setup.py (to
prevent the creation of *-nspkg.pth files by pip) and not to use
find_packages, which does not comply with PEP420 (no packages found).
This solution is somewhat counter-intuitive and I am not sure, whether
it is an intended/valid configuration of a setup.py file. Definitely,
it is not clear from the documentation (which has a section about
namespace packages btw.). Since I read
