How does the import machinery handle relative imports?
I'm fussing over some details of relative imports while trying to mimic Python
module loading in my personal project. This is getting more into corner cases,
but I can spare time to talk about it while working on more normal stuff.
I first found this place:
https://manikos.github.io/how-pythons-import-machinery-works
And eventually just started looking at PEP 451. Neither is really explaining
relative imports. I decided to try this garbage:
from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader
spec = spec_from_loader("..import_star",
SourceFileLoader("package_test.import_star",
r"C:\coding\random_python_projects\package_test\import_star.py"))
print(spec)
mod = module_from_spec(spec)
print(mod)
spec.loader.exec_module(mod)
...exec_module ultimately fails to do the job. Note the syntax so that I can
actually perform a relative import hahaha:
C:\Python36\python.exe -m package_test.second_level.import_upwards
ModuleSpec(name='..import_star',
loader=<_frozen_importlib_external.SourceFileLoader object at
0x0226E914B080>, origin='')
'>
Traceback (most recent call last):
File "C:\Python36\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Python36\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File
"C:\coding\random_python_projects\package_test\second_level\import_upwards.py",
line 15, in
spec.loader.exec_module(mod)
File "", line 674, in exec_module
File "", line 750, in get_code
File "", line 398, in
_check_name_wrapper
ImportError: loader for package_test.import_star cannot handle ..import_star
Yeah I don't think I'm doing this right! At this point I'm just trying to
figure out where I feed in the relative path. Is that all deduced in advance of
creating finding the spec? Can I even give the finders a relative path like
that?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python programs and relative imports
On Fri, Apr 8, 2016 at 11:50 AM, Rob Gaddi wrote: > Sort of. If I've got a directory full of files (in a package) > that I'm working on, the relative import semantics change based on > whether I'm one directory up and importing the package or in the same > directory and importing the files locally. That is to say if I've got: > > pkg/ > __init__.py > a.py > usedbya.py > > then there is no single syntax I can use in a.py that allows me to both > sit in the pkg directory at the shell and poke at things and import pkg > from the higher level. > > If the 'from . import usedbya' syntax were always available, then it > would work the same in either context. Not necessarily. Inside the package, 'from . import usedbya' is effectively equivalent to 'import pkg.usedbya as usedbya'. Without the package, all of these modules are at the top level, and 'from . import usedbya' would conceptually be equivalent to 'import usedbya'. But there's no guarantee that the 'usedbya' module at the top level of the module tree is the same 'usedbya.py' file in the current directory; it could be shadowed by some other module. Whereas with the package, the packaging ensures that you'll get the module expect. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python programs and relative imports
On Sat, Apr 9, 2016 at 3:50 AM, Rob Gaddi wrote: > Sort of. If I've got a directory full of files (in a package) > that I'm working on, the relative import semantics change based on > whether I'm one directory up and importing the package or in the same > directory and importing the files locally. That is to say if I've got: > > pkg/ > __init__.py > a.py > usedbya.py > > then there is no single syntax I can use in a.py that allows me to both > sit in the pkg directory at the shell and poke at things and import pkg > from the higher level. > > If the 'from . import usedbya' syntax were always available, then it > would work the same in either context. And if as I refactored things, > as they moved in and out of packages, it would all still "just work" for > files that haven't moved relative to one another. Ah, I see what you mean. You're working inside an actual package here. So you can "cd ..; python3 -m pkg.a", or you can "python3 a.py", but not both. The simplest fix for that would be to allow "python3 -m .a" to mean "current directory is a package". I don't think there's currently a way to spell that, but it ought to be completely backward compatible. You could raise this on python-ideas and see what people say. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python programs and relative imports
Chris Angelico wrote: > On Sat, Apr 9, 2016 at 2:59 AM, Rob Gaddi > wrote: >> Rob Gaddi wrote: >> >>> Does anyone know the history of why relative imports are only available >>> for packages and not for "programs"? It certainly complicates life. >>> >> >> Really, no one? It seems like a fairly obvious thing to have included; >> all of the reasons that you want to be explicit in saying: >> >> from . import mypkg >> >> in a package apply just as well in an executable script. But instead, >> they've got different semantics such that you expressly _cannot_ use >> relative imports in a script. This feels like such a glaring oversight >> that there must have been some rationale behind it. > > You can use the simple "import mypkg" syntax to load these up. I'm not > sure what you're looking for - do you want to prevent that syntax from > working, to prevent accidental shadowing? > > ChrisA Sort of. If I've got a directory full of files (in a package) that I'm working on, the relative import semantics change based on whether I'm one directory up and importing the package or in the same directory and importing the files locally. That is to say if I've got: pkg/ __init__.py a.py usedbya.py then there is no single syntax I can use in a.py that allows me to both sit in the pkg directory at the shell and poke at things and import pkg from the higher level. If the 'from . import usedbya' syntax were always available, then it would work the same in either context. And if as I refactored things, as they moved in and out of packages, it would all still "just work" for files that haven't moved relative to one another. But it would also address the accidental shadowing issue. If you could use from __future__ import force_relative_imports in an exectable, then the import semantics would ALWAYS be that "import xxx" looks in sys.path and "from . import xxx" looks locally. This is akin to what the C preprocessor has done for decades by differentiating #include #include "localdefs.h" As is, it's a bit of a hodge podge. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python programs and relative imports
On Sat, Apr 9, 2016 at 2:59 AM, Rob Gaddi wrote: > Rob Gaddi wrote: > >> Does anyone know the history of why relative imports are only available >> for packages and not for "programs"? It certainly complicates life. >> > > Really, no one? It seems like a fairly obvious thing to have included; > all of the reasons that you want to be explicit in saying: > > from . import mypkg > > in a package apply just as well in an executable script. But instead, > they've got different semantics such that you expressly _cannot_ use > relative imports in a script. This feels like such a glaring oversight > that there must have been some rationale behind it. You can use the simple "import mypkg" syntax to load these up. I'm not sure what you're looking for - do you want to prevent that syntax from working, to prevent accidental shadowing? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python programs and relative imports
Rob Gaddi wrote: > Does anyone know the history of why relative imports are only available > for packages and not for "programs"? It certainly complicates life. > Really, no one? It seems like a fairly obvious thing to have included; all of the reasons that you want to be explicit in saying: from . import mypkg in a package apply just as well in an executable script. But instead, they've got different semantics such that you expressly _cannot_ use relative imports in a script. This feels like such a glaring oversight that there must have been some rationale behind it. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Python programs and relative imports
Does anyone know the history of why relative imports are only available for packages and not for "programs"? It certainly complicates life. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
On Sunday, February 8, 2015 at 3:52:19 AM UTC+5:30, Gregory Ewing wrote: > Rustom Mody wrote: > > Wanted to try out sympy. > > apt-install promised ź GB download, ž GB space usage > > > > Just getting a src-tarball was: 6M download, 30M after opening the tar. > > Have you actually tried compiling and using that > tarball, though? > > Sympy hooks into a lot of other libraries that > are themselves quite large. Apt-get was probably > planning to download and install all of them, > hence the large download size. > > If you tried to install from source, you would > likely have to install all those dependencies > yourself, or configure it to skip the ones you > don't want if it allows that. > > -- > Greg Yeah... Sympy output as on display http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-5-Sympy.ipynb looks quite beautiful and probably needs all those Tex etc fonts As it happens I did not need that I just wanted to demo some differences between object-level and meta-level . I guess the best choice for that would be lisp. I could not figure racket out – terminologies seem to have changed in the last couple of decades – so I took sympy's distinction of python-variable and math-variable as a good enough exemplar of meta-level and object-level. [Python's namespaces are not really firstclass like (some) scheme's are -- eg locals() makes a copy] Anyway… My point was not that particular use-case but the general point that one may want to play around with alternate packaging systems -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
Rustom Mody wrote: Wanted to try out sympy. apt-install promised ¼ GB download, ¾ GB space usage Just getting a src-tarball was: 6M download, 30M after opening the tar. Have you actually tried compiling and using that tarball, though? Sympy hooks into a lot of other libraries that are themselves quite large. Apt-get was probably planning to download and install all of them, hence the large download size. If you tried to install from source, you would likely have to install all those dependencies yourself, or configure it to skip the ones you don't want if it allows that. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
On Saturday, February 7, 2015 at 8:43:44 AM UTC+5:30, Rustom Mody wrote: > There is on the one hand python modules/packages mechanism with all the > hell of dozens of incompatible versions of setuptools/distribute/distutils > etc. > > On the other there is the OS-specific practices/policy such as > https://www.debian.org/doc/packaging-manuals/python-policy/ > > And the dark unexplored territory in between. > [Its a research project to figure out clashes between pip and apt] > Small example of this which I just tried. Wanted to try out sympy. apt-install promised ¼ GB download, ¾ GB space usage Just getting a src-tarball was: 6M download, 30M after opening the tar. Sure the full stuff with tex fonts and all may be pretty. But does not seem necessary to push it down everyone's throat unless needed/desired -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
On Saturday, February 7, 2015 at 7:35:12 AM UTC+5:30, Ben Finney wrote: > Ethan Furman writes: > > > On 02/06/2015 04:44 PM, Ben Finney wrote: > > > A program will often have enough complexity that its implementation > > > occupies several sub-modules. There's no need to explose those in a > > > site package, they normally only need to be local to the > > > application. > > > > If they are not in the Python module path, how are they imported at > > all? > > Only absolute imports use the module search path. The whole point of > relative imports is to import a module within the same (or a sub-) > package, without modifying the search path. > > https://www.python.org/dev/peps/pep-0328/> > > > > Python deliberately divorces the top-level module from its package, > > > so it can't access its own relative modules! > > > > My understanding is that imports is for libraries [1], and libraries > > must be in sys.path. > > That's the arbitrary limitation I'm pointing out, yes. The program's > private implementation modules should not need to be in the global > module search path, merely to allow the top-level program to import > them. > > It's contradictory to present relative imports with the justification > they are for intra-package imports, and then cripple a top-level module > so it can't access its own implementation modules via relative imports. Thanks Ben for these points. We techies need to always slide the rheostat along: 1. I am dumb 2. I am ignorant - need to read up some 3. Stuff is complex - more to read up 4. Stuff is complicated - after all the reading up, some general ideas may get clear but there remain all sorts of dark nooks and crannies 5. Stuff is a complete bloody mess - anyone who claims to understand, doesn't understand and doesn't understand that he doesn't understand. [Think Stroustrup saying something to the effect it takes a lifetime to learn C++ !] Your points and the earlier thread have helped me to slide the rheostat a couple of notches down! My analysis of the problem: Python docs are a model of clarity... mostly. They fail when the rubber hits the road - OS-specific stuff. In the attempt at keeping the docs generic, when things like how the 'insides' of python map to the 'outsides' - file-system-mapping - I find the docs are sorely inadequate. [You are of course free to slide my rheostat up :-) ] There is on the one hand python modules/packages mechanism with all the hell of dozens of incompatible versions of setuptools/distribute/distutils etc. On the other there is the OS-specific practices/policy such as https://www.debian.org/doc/packaging-manuals/python-policy/ And the dark unexplored territory in between. [Its a research project to figure out clashes between pip and apt] Hopefully python-devs will give up trying to be generic in this area and start documenting these things in an OS-specific way. Yeah I know it can be significantly more of a babel than just: Windows - This-a-way Linux - That-a-way eg: - Mismatch between say Fedora and Debian - Different 32-64 bit path-disambiguation on different windows systems - etc Still... we need to bite the bullet Since this has been somewhat of a rant I should say: Wherever comparable to python we may look, its just like this or worse. Perl/Ruby are probably slightly more mature Haskell is much worse - search for "cabal hell" Latex is probably about the same: eg a debian texlive bugreport of mine: https://lists.debian.org/debian-tex-maint/2014/06/msg00029.html followed by unhelpful/improper response of the debian-dev -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
Ethan Furman writes: > On 02/06/2015 04:44 PM, Ben Finney wrote: > > A program will often have enough complexity that its implementation > > occupies several sub-modules. There's no need to explose those in a > > site package, they normally only need to be local to the > > application. > > If they are not in the Python module path, how are they imported at > all? Only absolute imports use the module search path. The whole point of relative imports is to import a module within the same (or a sub-) package, without modifying the search path. https://www.python.org/dev/peps/pep-0328/> > > Python deliberately divorces the top-level module from its package, > > so it can't access its own relative modules! > > My understanding is that imports is for libraries [1], and libraries > must be in sys.path. That's the arbitrary limitation I'm pointing out, yes. The program's private implementation modules should not need to be in the global module search path, merely to allow the top-level program to import them. It's contradictory to present relative imports with the justification they are for intra-package imports, and then cripple a top-level module so it can't access its own implementation modules via relative imports. -- \ “Capitalism is the astounding belief that the most wickedest of | `\men will do the most wickedest of things for the greatest good | _o__) of everyone.” —John Maynard Keynes | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Incompatible idioms: relative imports, top-level program file
On 02/06/2015 04:44 PM, Ben Finney wrote: > Ethan Furman writes: > >> On 02/06/2015 02:56 PM, Ben Finney wrote: >>> It is a deliberate design decision that direct import of a module >>> makes that module blind to its location in the package hierarchy. >>> >>> That's a design decision I deplore, because it makes something that >>> should be easy (write a command directly into a file and invoke that >>> file as an executable program) tortuously difficult when the program >>> comprises several modules. >> >> Can you explain that a bit more? > > A program will often have enough complexity that its implementation > occupies several sub-modules. There's no need to explose those in a site > package, they normally only need to be local to the application. If they are not in the Python module path, how are they imported at all? > Python deliberately divorces the top-level module from its package, so > it can't access its own relative modules! The relative import fails with > an ImportError “Attempted relative import in non-package”. My understanding is that imports is for libraries [1], and libraries must be in sys.path. Are you saying that after placing your top-level file's path in sys.path, that relative imports do not work? -- ~Ethan~ [1] which can be a single module signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Incompatible idioms: relative imports, top-level program file (was: Setuptools, __init__ and __main__)
Ethan Furman writes: > On 02/06/2015 02:56 PM, Ben Finney wrote: > > It is a deliberate design decision that direct import of a module > > makes that module blind to its location in the package hierarchy. > > > > That's a design decision I deplore, because it makes something that > > should be easy (write a command directly into a file and invoke that > > file as an executable program) tortuously difficult when the program > > comprises several modules. > > Can you explain that a bit more? A program will often have enough complexity that its implementation occupies several sub-modules. There's no need to explose those in a site package, they normally only need to be local to the application. So the correct idiom is ‘from __future__ import absolute_import’ and keep the application's own implementation imported via relative imports in the same package: ‘from . import implementation_detail’. But relative import is not possible when the top-level module is executed by the normal ‘fooprog’ usage. The top-level file is named ‘fooprog’ and is marked executable, so that it can be run directly as soon as it's written. Python deliberately divorces the top-level module from its package, so it can't access its own relative modules! The relative import fails with an ImportError “Attempted relative import in non-package”. https://mail.python.org/pipermail/python-list/2014-November/694385.html> So the two correct idioms – relative import for non-public modules, and run a file directly as a command – are explicitly thwarted by Python's import behaviour when you try to use them together. > Surely if the writing a command into a file was going to be easy, then > so would be writing it to __main__.py instead? With Bash shell, or Perl, or Ruby, or countless other languages, I put the top-level code in the *exact same file* that I'm going to execute, named ‘fooprog’. No additional infrastructure files unless my program architecture indicates they'll help. Python's refusal to allow this leads to roundabout hacks like “make a sub-directory ‘fooprog/’, write the code into ‘__main__.py’”, or “run the program by typing ‘python -m fooprog’”, or “write a ‘./setup.py’ configuration and dicker around with Distutils's rules and install it before you can run your program”. A system needing baroque hacks like this doesn't really deserve to be characterised as a “scripting language”, IMO. Fortunately, I don't like that term anyway (we already have the term “program” for what Python calls scripts) so avoid characterising Python that way for other reasons too :-) -- \“Sane people have an appropriate perspective on the relative | `\ importance of foodstuffs and human beings. Crazy people can't | _o__) tell the difference.” —Paul Z. Myers, 2010-04-18 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
nosetests vs. relative imports?
Environment: Ubuntu Linux 12.04 (precise) Python 2.7.3 I have a package (i.e. a directory with a __init__.py file). In that directory, I have files math.py and test_math.py; test_math.py contains the single line: from .math import EMA If I run on the command line, "nosetests test_math.py" everything is fine (it reports "Ran 0 tests in 0.001s", which is what I expect, since I haven't written any tests yet). But, if I leave off the .py, I get: $ nosetests test_math E == ERROR: Failure: ValueError (Attempted relative import in non-package) -- Traceback (most recent call last): File "/home/roy/deploy/current/python/local/lib/python2.7/site-packages/nose/l oader.py", line 413, in loadTestsFromName addr.filename, addr.module) File "/home/roy/deploy/current/python/local/lib/python2.7/site-packages/nose/i mporter.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/home/roy/deploy/current/python/local/lib/python2.7/site-packages/nose/i mporter.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/roy/deploy/current/code/songza/util/test_math.py", line 1, in from .math import EMA ValueError: Attempted relative import in non-package -- Ran 1 test in 0.001s What is going on that nose can't deal with the relative import when run that way? Nose has always felt like a magic ring. I'm really glad I discovered it because it makes my life so much better. But I'm never quite sure I really understand everything it's doing. -- https://mail.python.org/mailman/listinfo/python-list
python 2.x and python 3 relative imports and
Hi, let say I have a legacy code with the following structure: pkg1/__init__.py pkg1/pkg2/__init__.py pkg1/pkg2/bar.py pkg1/pkg2/pkg3/__init__.py pkg1/pkg2/pkg3/foo.py In pkg1/pkg2/bar.py I have: # pkg1/pkg2/bar.py import pkg3.foo class Bar(pkg3.foo): pass in pkg1/pkg2/pkg3/foo.py: # pkg1/pkg2/pkg3/foo.py class Foo: pass Now I want to adapt bar.py such that it works in Python 3, but without modifying the definition of Bar class (I wan't restrict modification to import directives). My first thought was that I to just modify the import directive, such that the 'pkg3.foo' would be still available in bar.py. Unfortunately I can't find any way to do it. Obviously, this relative import is not going to work: from . import pkg3.foo because it's a syntax error (pkg3.foo is not an identifier). This is accepted by python: from .pkg3 import foo but it binds 'foo' instead of 'pkg3' to the local (bar) module, and I still have no access to 'pkg3.foo'. The only way I found do have 'pkg3.foo' in 'bar' is this two-line trick: from . import pkg3 from .pkg3 import foo or from . import pkg3 import pkg1.pk2.pk3.foo but this clutters local (bar) namespace with symbols 'foo' (first case) or 'pkg1' (second approach). Do you know any other way for relative imports to achieve exactly same effect as with old import semantics? -- https://mail.python.org/mailman/listinfo/python-list
Re: Understanding relative imports in package - and running pytest with relative imports?
On Sun, Nov 24, 2013 at 5:30 PM, Victor Hooi wrote: > The advice seems to be either to run it from the parent directory of > furniture with: > > python -m furniture.chair.build_chair Yes. More pedantically, run it from somewhere such that the furniture package is importable. For example, if you install furniture, then it's accessible everywhere. > However, I don't see having a separate single main.py outside my package > would work with keeping my code tidy/organised, and or how it'd work with the > other files (config.yaml, or create_table.sql) which are associated with each > script? The other files are contained within the package. As long as build_table looks in the right places, they'll still be accessible. (e.g. http://docs.python.org/2/library/pkgutil#pkgutil.get_data ). > A third way I thought of way just to create a setup.py and install the > package into site-packages - and then everything will work? However, I don't > think that solves my problem of understanding how things work, or getting my > directory structure right. No, installing furniture won't change anything about how you need to run it. Either you run a separate script that imports it and runs the right main function, or you use python -m furniture.foo.bar > Although apparently running a script inside a package is an anti-pattern? > (https://mail.python.org/pipermail/python-3000/2007-April/006793.html) It's not an antipattern, it's just plain broken. Some things stop working as you'd expect, like exception handling. > How would you guys organise the code above? If I really cared about the command line interface, I would probably reorganize something like this: furniture/ - __init__.py - __main__.py - common.py * chair/ - __init__.py - build.py - create.sql - config.yaml * furniture/ - __init__.py - build.py - create.sql - config.yaml And then create chairs by running `python -m furniture build chair`. furniture/__main__.py would dynamically import furniture.chair.build and run furniture.chair.build.main(). There would also be an executable I might install separately, furniture, the contents of which are "from furniture import __main__; __main__.main()" > Also, if I have tests (say with pyttest), inside > furniture/table/tests/test_table.py, how would I run these as well? If I run > py.test from there, I get the same: > > $ py.test > > from ..table.build_table import Table > E ValueError: Attempted relative import in non-package > > > (Above is just an extract). > > Assuming I use pytest, where should my tests be in the directory structure, > and how should I be running them? As far as I understand py.test and nose, they are designed such that you don't put your tests as subpackages. If you want to put your tests as subpackages, use a different unit testing framework, like unittest. Then you can just do `python -m unittest discover .` If you do want to use py.test (or nose), then either there is some flag you need to pass in that I'm not aware of, or your directory structure should be: my-project/ - README - LICENSE - setup.py * furniture/ ... * tests/ - test_chair.py - test_common.py - test_table.py Note that tests is not a package either. py.test and nose work by loading files, NOT modules. So they won't work with loading test submodules in a package. For this and other reasons, I personally stay away from nose and py.test. -- Devin -- https://mail.python.org/mailman/listinfo/python-list
Understanding relative imports in package - and running pytest with relative imports?
Hi, Ok, this is a topic that I've never really understood properly, so I'd like to find out what's the "proper" way of doing things. Say I have a directory structure like this: furniture/ __init__.py chair/ __init__.py config.yaml build_chair.py common/ __init__.py shared.py table/ __init__.py config.yaml create_table.sql build_table.py The package is called furniture, and we have modules chair, common and table underneath that. build_chair.py and build_table.py are supposed to import from common/shared.py using relative imports. e.g.: from ..common.shared import supplies However, if you then try to run the scripts build_chair.py, or build_table.py, they'll complain about: ValueError: Attempted relative import in non-package After some Googling: http://stackoverflow.com/questions/11536764/attempted-relative-import-in-non-package-even-with-init-py http://stackoverflow.com/questions/72852/how-to-do-relative-imports-in-python http://stackoverflow.com/questions/1198/getting-attempted-relative-import-in-non-package-error-in-spite-of-having-init http://stackoverflow.com/questions/14664313/attempted-relative-import-in-non-package-although-packaes-with-init-py-in http://melitamihaljevic.blogspot.com.au/2013/04/python-relative-imports-hard-way.html The advice seems to be either to run it from the parent directory of furniture with: python -m furniture.chair.build_chair Or to have a main.py outside of the package directory and run that, and have it import things. However, I don't see having a separate single main.py outside my package would work with keeping my code tidy/organised, and or how it'd work with the other files (config.yaml, or create_table.sql) which are associated with each script? A third way I thought of way just to create a setup.py and install the package into site-packages - and then everything will work? However, I don't think that solves my problem of understanding how things work, or getting my directory structure right. Although apparently running a script inside a package is an anti-pattern? (https://mail.python.org/pipermail/python-3000/2007-April/006793.html) How would you guys organise the code above? Also, if I have tests (say with pyttest), inside furniture/table/tests/test_table.py, how would I run these as well? If I run py.test from there, I get the same: $ py.test from ..table.build_table import Table E ValueError: Attempted relative import in non-package (Above is just an extract). Assuming I use pytest, where should my tests be in the directory structure, and how should I be running them? Cheers, Victor -- https://mail.python.org/mailman/listinfo/python-list
Relative imports in packages
Hi list,
I've these two minor problems which bothered me for quite some time,
maybe you can help me. I'm using Python 3.2.
For some project I have a component in its own package. Let's say the
structure looks like this:
pkg/__init__.py
pkg/Foo.py
pkg/Bar.py
Foo.py and Bar.py contain their classes "Foo" and "Bar", __init__.py
looks like this:
from .Foo import Foo
from .Bar import Bar
This allows me to "naturally" access classes, i.e. from my main program:
import pkg
pkg.Foo("initme")
or
from pkg import Foo
Foo("initme")
So far, so good. Now let's say Bar uses Foo, i.e. in Bar's header is
something like:
from .Foo import Foo
If this all weren't a package the declaration would just read "from Foo
import Foo" and I could easily append a small unit-test to Bar:
if __name__ == "__main__":
# test
pass
However, when using a package this fails: Obviously, when I directly go
into the pkg/ subdirectory and try to execute Bar.py, the import of Foo
fails and it doesn't work. Is there a nice solution to this or am I
doing it all wrong?
Then another minor question: Let's say my __init__.py contains a constant:
VERSION = "0.01"
>From my main program I can easily import that:
from pkg import VERSION
print(VERSION)
However, from Foo.py, I cannot seem to get the syntax right:
from . import VERSION
File "Foo.py", line 10, in
from . import VERSION
ImportError: cannot import name VERSION
How do I do this right?
Thanks for your advice,
Best regards,
Johannes
--
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
- Karl Kaos über Rüdiger Thomas in dsa
--
http://mail.python.org/mailman/listinfo/python-list
Dynamic imports + relative imports in Python 3
This is a copy-paste of a StackOverflow question. Nobody answered
there, but I figured I might have better luck here.
I have a Python 3 project where I'm dynamically importing modules from
disk, using `imp.load_module`. But, I've run into an problem where
relative imports fail, when the relative import occurs within a
dynamically imported module.
From what I've read, I came to the conclusion that only `__file__`,
`__path__`, `__package__`, and `__name__` were used by the default
importer when determining the path of an import. Yet, I've verified
these in the code below, and it still fails when dynamically imported.
(It works when imported in the interpreter with an updated `sys.path`)
# File structure:
# [root]
# ├─ __init__.py
# ├─ board.py
# └─ test.py
# Contents of 'board.py':
import os, sys
import root # Already imported... just need a reference
ROOT_DIR = os.path.dirname(root.__file__)
assert root is sys.modules['root']
assert root.__package__ is None
assert root.__name__ == 'root'
assert root.__file__ == os.path.join(ROOT_DIR, '__init__.py')
assert not hasattr(root, '__path__')
xx = object()
assert xx is sys.modules['root.board'].xx
assert __package__ is None
assert __name__ == 'root.board'
assert __file__ == os.path.join(ROOT_DIR, 'board.py')
assert not hasattr(sys.modules['root.board'], '__path__')
assert os.path.isfile(os.path.join(ROOT_DIR, 'test.py'))
from . import test # ImportError('cannot import name test',)
But if I hack `sys.path` and reimport the current package just before
the failed import, it works fine:
oldroot = root
del sys.modules['root']
try:
sys.path.append(os.path.dirname(ROOT_DIR))
import root
finally:
sys.path.pop(-1)
from . import test # No error here
And further, the four golden attributes mentioned above are the same
in both the new and old packages:
assert oldroot.__package__ == root.__package__
assert oldroot.__name__ == root.__name__
assert oldroot.__file__ == root.__file__
assert not hasattr(root, '__path__')
Which means that `__package__`, `__name__`, `__file__`, and `__path__`
can't be the full story. Are there any other attributes that Python
uses to locate imports? What am I overlooking that would cause the
import to fail?
--
http://mail.python.org/mailman/listinfo/python-list
Re: relative imports and sub-module execution
King writes: > Hi, > > After reading couple of docs and articles, I have implemented a simple > test package with nested modules. > When running "main.py", everything is working fine. Some of my sub- > modules has some small test routines for debug purpose. > It's because I am using relative package imports at the top, I am not > able to run these sub modules individually. > > For example, this works when running from main.py > > Here is example structure > > main.py > __init__.py > core/ > __init__.py >folder1 > __init__.py >f1a.py >f1b.py > folder2 > __init__.py >f2a.py >f2b.py > > in folder2/f2b.py, I am importing > > from core.folder1 import f1a > print f1a.myvar > > Now I can't execute 'f2b.py' individually. It's giving an error: > > ImportError: No module named core.folder2 > > Is this mean when you have created a package where modules are using > relative imports, they can't execute individually? The problem is your python-path. Python will put the path of the script that you execute into the sys.path - but not attempt to guess that you actually are deep within a package hierarchy and want the path two up from there in sys.path as well. Which is the reason why you have to do that yourself. My solution to this is to always use setuptools, even for the tiniest of projects, and simply create a setup.py with some minimal information in it, and then do python setup.py develop This will put a EGG-link into the site-packages directory. Usually, for not clobbering my system's python, I use a virtualenv also. With this setup, I can safely import from "core" all the time. Diez -- http://mail.python.org/mailman/listinfo/python-list
relative imports and sub-module execution
Hi,
After reading couple of docs and articles, I have implemented a simple
test package with nested modules.
When running "main.py", everything is working fine. Some of my sub-
modules has some small test routines for debug purpose.
It's because I am using relative package imports at the top, I am not
able to run these sub modules individually.
For example, this works when running from main.py
Here is example structure
main.py
__init__.py
core/
__init__.py
folder1
__init__.py
f1a.py
f1b.py
folder2
__init__.py
f2a.py
f2b.py
in folder2/f2b.py, I am importing
from core.folder1 import f1a
print f1a.myvar
Now I can't execute 'f2b.py' individually. It's giving an error:
ImportError: No module named core.folder2
Is this mean when you have created a package where modules are using
relative imports, they can't execute individually?
Another solution that I have discovered is using sys.path. Check this:
import sys
import os
sys.path.append(os.path.abspath("../../"))
from core.folder1 import f1a
print f1a.myvar
This works fine and easier too. I can execute modules individually as
well as package is also working. Are there any disadvantages when
using above technique? Couple of articles suggests that this is a bad
practice and should not be used.
Need some clarifications.
Thanks
Prashant
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with relative imports and pep 366
Gabriele Lanaro wrote: I've yet asked this question on SO, I'll copy the contents: I have a "canonical file structure" like that (I'm giving sensible names to ease the reading): mainpack/ __main__.py __init__.py - helpers/ __init__.py path.py - network/ __init__.py clientlib.py server.py - gui/ __init__.py mainwindow.py controllers.py In this structure, for example modules contained in each package may want to access the helpers utilities through relative imports in something like: # network/clientlib.py from ..helpers.path import create_dir The program is runned "as a script" using the __main__.py file in this way: python mainpack/ Trying to follow the PEP 366 I've put in __main__.py these lines: ___package___ = "mainpack" from .network.clientlib import helloclient But when running: $ python mainpack Traceback (most recent call last): File "/usr/lib/python2.6/runpy.py", line 122, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.6/runpy.py", line 34, in _run_code exec code in run_globals File "path/mainpack/__main__.py", line 2, in from .network.clientlib import helloclient SystemError: Parent module 'mainpack' not loaded, cannot perform relative import What's wrong? What is the correct way to handle and effectively use relative imports? I've tried also to add the current directory to the PYTHONPATH, nothing changes. link: http://stackoverflow.com/questions/2943847/nightmare-with-relative-imports-how-does-pep-366-work I'm not using relative imports at all, so I can't help you on that point, however, did you consider using absolute imports ? The world becomes so simple with these :) . BTW, ___package___ = "mainpack" looks weird to me, are you sure it would not be better with __package__ = "mainpack" # only 2 leading/traling underscore JM -- http://mail.python.org/mailman/listinfo/python-list
Problems with relative imports and pep 366
I've yet asked this question on SO, I'll copy the contents: I have a "canonical file structure" like that (I'm giving sensible names to ease the reading): mainpack/ __main__.py __init__.py - helpers/ __init__.py path.py - network/ __init__.py clientlib.py server.py - gui/ __init__.py mainwindow.py controllers.py In this structure, for example modules contained in each package may want to access the helpers utilities through relative imports in something like: # network/clientlib.py from ..helpers.path import create_dir The program is runned "as a script" using the __main__.py file in this way: python mainpack/ Trying to follow the PEP 366 I've put in __main__.py these lines: ___package___ = "mainpack" from .network.clientlib import helloclient But when running: $ python mainpack Traceback (most recent call last): File "/usr/lib/python2.6/runpy.py", line 122, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.6/runpy.py", line 34, in _run_code exec code in run_globals File "path/mainpack/__main__.py", line 2, in from .network.clientlib import helloclient SystemError: Parent module 'mainpack' not loaded, cannot perform relative import What's wrong? What is the correct way to handle and effectively use relative imports? I've tried also to add the current directory to the PYTHONPATH, nothing changes. link: http://stackoverflow.com/questions/2943847/nightmare-with-relative-imports-how-does-pep-366-work -- http://mail.python.org/mailman/listinfo/python-list
Re: relative imports with the __import__ function
Chris Colbert wrote: > It seems the relative import level is dependent on the location of the > main entry module. I thought the whole idea of relative imports was to > make the import independent of the entry point? You don't have to specify it explicitly, so you can move a module containing from .foo import bar into another package without changing its source code (provided there is a foo submodule with a bar attribute). This includes the trivial case of renaming the parent package. Of course Python has to know the importing module's location, just like you cannot meet me one block north and three blocks west unless you know where I currently am. See also http://www.python.org/dev/peps/pep-0328/#rationale-for-relative-imports Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: relative imports with the __import__ function
On Tue, Dec 8, 2009 at 5:48 PM, Peter Otten <__pete...@web.de> wrote:
> Chris Colbert wrote:
>
>> I have package tree that looks like this:
>>
>> main.py
>> package
>> __init__.py
>> configuration.ini
>> server
>> __init__.py
>> xmlrpc_server.py
>> controller.py
>> reco
>>
>> segmentation
>> __init__.py
>> red_objects.py
>>
>>
>>
>> main.py launches an instance of xmlrpc_server.py which, in turn,
>> imports controller.py.
>> controller.py reads configuration.ini to determine which
>> module/function to import from the segmentation directory and
>> subsequently use.
>>
>> that config file specifies the module as 'red_objects' and the
>> function as 'segment_red'.
>>
>> I am trying to dynamically import that module and func using the
>> __import__ statement but keep getting empty module errors.
>
>
>> I'm assuming i'm missing something fundamental on the import resolution...
>
> After some experimentation it turns out you have to provide some context for
> __import__() to determine the absolute location of the requested module. The
> required bit of information is the current module's __name__ attribute which
> you can provide via the globals parameter:
>
> def import_segmentation(name):
> return getattr(__import__("segmentation." + name, level=2,
> globals=globals()), name)
>
> Peter
> --
> http://mail.python.org/mailman/listinfo/python-list
>
Many thanks Peter!
Almost like a charm!
It seems the relative import level is dependent on the location of the
main entry module. I thought the whole idea of relative imports was to
make the import independent of the entry point?
here is the import function i'm using
def import_segmentation(self):
# get the segmentation function defined in configuration.ini
parent_dir = os.path.split(os.path.dirname(__file__))[0]
prsr = ConfigParser.ConfigParser()
prsr.read(os.path.join(parent_dir, 'configuration.ini'))
seg_mod = prsr.get('segmentation', 'module')
seg_func = prsr.get('segmentation', 'function')
print __name__
smod = __import__('segmentation.%s' % seg_mod, globals=globals(),
fromlist=[seg_func], level=2)
sfunc = getattr(smod, seg_func)
return sfunc
for that import level of 2 to work the tree must look like this:
main.py
package
__init__.py
configuration.ini
server
__init__.py
xmlrpc_server.py
controller.py
reco
segmentation
__init__.py
red_objects.py
but if I rearrange the package structure like this (just moving the
location of main.py):
package
main.py
__init__.py
configuration.ini
server
__init__.py
xmlrpc_server.py
controller.py
reco
segmentation
__init__.py
red_objects.py
I have to change the import to level=1 or I get this error:
ValueError: Attempted relative import beyond toplevel package
I don't understand why the location of my main.py should have ANY
bearing on relative import resolution.
But again, i'm probably just being dense and need someone to explain
it to me in newb speak ;)
Cheers,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: relative imports with the __import__ function
Chris Colbert wrote:
> I have package tree that looks like this:
>
> main.py
> package
> __init__.py
> configuration.ini
> server
> __init__.py
> xmlrpc_server.py
> controller.py
> reco
>
> segmentation
> __init__.py
> red_objects.py
>
>
>
> main.py launches an instance of xmlrpc_server.py which, in turn,
> imports controller.py.
> controller.py reads configuration.ini to determine which
> module/function to import from the segmentation directory and
> subsequently use.
>
> that config file specifies the module as 'red_objects' and the
> function as 'segment_red'.
>
> I am trying to dynamically import that module and func using the
> __import__ statement but keep getting empty module errors.
> I'm assuming i'm missing something fundamental on the import resolution...
After some experimentation it turns out you have to provide some context for
__import__() to determine the absolute location of the requested module. The
required bit of information is the current module's __name__ attribute which
you can provide via the globals parameter:
def import_segmentation(name):
return getattr(__import__("segmentation." + name, level=2,
globals=globals()), name)
Peter
--
http://mail.python.org/mailman/listinfo/python-list
relative imports with the __import__ function
I have package tree that looks like this:
main.py
package
__init__.py
configuration.ini
server
__init__.py
xmlrpc_server.py
controller.py
reco
segmentation
__init__.py
red_objects.py
main.py launches an instance of xmlrpc_server.py which, in turn,
imports controller.py.
controller.py reads configuration.ini to determine which
module/function to import from the segmentation directory and
subsequently use.
that config file specifies the module as 'red_objects' and the
function as 'segment_red'.
I am trying to dynamically import that module and func using the
__import__ statement but keep getting empty module errors.
In the following code segment, the failing code is uncommented, but
the commented code works fine:
seg_mod = 'red_objects'
smod = __import__('..segmentation.%s' % seg_mod, fromlist=[seg_func], level=-1)
#from ..segmentation import red_objects
#smod = red_objects
I have tried all sorts of values for the 'level' kwarg as well as
everywhich variation of the dotted relative notation.
I'm assuming i'm missing something fundamental on the import resolution...
As an aside, I would like to move main.py inside of the package
directory, but I dont know if that is possible with what i'm trying to
do here.
Thanks for any help.
Cheers!
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 31, 6:39 pm, Kay Schluehr wrote: > On 1 Apr., 00:38, Carl Banks wrote: > > > On Mar 31, 12:08 pm, Kay Schluehr wrote: > > > > > And your proposal is? > > > > I have still more questions than answers. > > > That's obvious. > > > Perhaps you should also refrain from making sweeping negative > > judgments about a system you have more questions than answers about. > > > (Feel free to make sweeping negative judgments once you have the > > answers, though.) > > > Carl Banks > > Diagnosing a problem means having a detailed cure? Wow, that's > critical thinking I guess. A diagnosis commesurate with the understanding you've displayed in this thread so far would be something like "Python's import system is too complicated", not "Python's import system is fundamentally broken". The latter is not something someone who has more questions than answers has any grounds to claim. But, as I said, once you have learned the answers to your questions and you claim that, it's all good. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 31, 12:08 pm, Kay Schluehr wrote: > > And your proposal is? > > I have still more questions than answers. That's obvious. Perhaps you should also refrain from making sweeping negative judgments about a system you have more questions than answers about. (Feel free to make sweeping negative judgments once you have the answers, though.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On 1 Apr., 00:38, Carl Banks wrote: > On Mar 31, 12:08 pm, Kay Schluehr wrote: > > > > And your proposal is? > > > I have still more questions than answers. > > That's obvious. > > Perhaps you should also refrain from making sweeping negative > judgments about a system you have more questions than answers about. > > (Feel free to make sweeping negative judgments once you have the > answers, though.) > > Carl Banks Diagnosing a problem means having a detailed cure? Wow, that's critical thinking I guess. O.K. I have got some ideas for a new import system and I'm going to blog about them within the next days. If I have some results I'll leave a notice in this thread. -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Kay Schluehr wrote: On 31 Mrz., 20:50, Terry Reedy wrote: Although the ceremony has been performed basically correct the interpreter god is not pacified and doesn't respond. But the import 'ceremony' has not been performed. There is no import ceremony. Imports are just stated in the source. There is a package ceremony for whatever reasons. Sorry, I have no idea what 'package ceremony' means. But why not? Because it looks up for *living* imported packages in the module cache ( in sys.modules ). I don't think there is any particular design idea behind it. The module cache is just a simple flat dictionary; a no-brainer to implement and efficient for look ups. This all dates to the time before packages and imports from zip files and such. > But it counteracts a domain model. What is that? Object oriented programming. Domain model == oop? New one for me. All you are left with is those Finders, Loaders and Importers in Brett Cannons importlib. Everything remains deeply mysterious and I don't wonder that it took long for him to work this out. And your proposal is? I have still more questions than answers. OK. I will just note that import statements are syntactic sugar for __import__ calls and name bindings. One could try out other import schemes, just without the sugar. tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On 31 Mrz., 20:50, Terry Reedy wrote: > Nothing is added to sys.modules, except the __main__ module, unless > imported (which so are on startup). Yes. The startup process is opaque but at least user defined modules are not accidentally imported. > > > Although the ceremony has been performed > > basically correct the interpreter god is not pacified and doesn't > > respond. > > But the import 'ceremony' has not been performed. There is no import ceremony. Imports are just stated in the source. There is a package ceremony for whatever reasons. > > But why not? Because it looks up for *living* imported > > packages in the module cache ( in sys.modules ). > > > I don't think there is any particular design idea behind it. The > > module cache is just a simple flat dictionary; a no-brainer to > > implement and efficient for look ups. > > This all dates to the time before packages and imports from zip files > and such. > > > But it counteracts a domain model. > > What is that? Object oriented programming. > > > All you are left with is those Finders, Loaders and Importers > > in Brett Cannons importlib. Everything remains deeply mysterious and I > > don't wonder that it took long for him to work this out. > > And your proposal is? I have still more questions than answers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Kay Schluehr wrote: On 31 Mrz., 18:48, s4g wrote: This and similar solutions ( see Istvan Alberts ) point me to a fundamental problem of the current import architecture. Suppose you really want to run a module as a script without a prior import from a module path: ...A\B\C> python my_module.py then the current working directory C is added to sys.path which means that the module finder searches in C but C isn't a known package. There is no C package in sys.modules even if the C directory is "declared" as a package by placing an __init__.py file in it. Same goes of course with B and A. Nothing is added to sys.modules, except the __main__ module, unless imported (which so are on startup). Although the ceremony has been performed basically correct the interpreter god is not pacified and doesn't respond. But the import 'ceremony' has not been performed. But why not? Because it looks up for *living* imported packages in the module cache ( in sys.modules ). I don't think there is any particular design idea behind it. The module cache is just a simple flat dictionary; a no-brainer to implement and efficient for look ups. This all dates to the time before packages and imports from zip files and such. > But it counteracts a domain model. What is that? All you are left with is those Finders, Loaders and Importers in Brett Cannons importlib. Everything remains deeply mysterious and I don't wonder that it took long for him to work this out. And your proposal is? tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On 31 Mrz., 18:48, s4g wrote: > Hi, > > I was looking for a nice idiom for interpackage imports as I found > this thread. > Here come a couple of solutions I came up with. Any discussion is > welcome. > > I assume the same file structure > > \ App > | main.py > +--\subpack1 > | | __init__.py > | | module1.py > | > +--\subpack2 > | | __init__.py > | | module2.py > > When you run main.py all imports relative to \App work fine, so the > only problem is running a module from within a subpackage as a script. > I therefore assume we want to run module1.py as a script, which wants > to import module2. > > I hope the following solutions are self-evident > > = solution 1 > --> in module1.py > import sys, os > if __name__ == '__main__': > sys.path.append(os.path.normpath(__file__+'/../..')) > > import subpack2.module2 > > = solution 2 > --> in subpack1/__init__.py > import sys, os > > _top_package_level = 1 # with current package being level 0 > > _top_package = os.path.normpath(__file__ + '/..'*(_top_package_level > +1)) > if _top_package not in sys.path: > sys.path.append(_top_package) > > --> in module1 or any module in the package, which requires import > relative to the package top > import __init__ > import subpack2.module2 > > = solution 3 > --> in package_import.py, somewhere on the PYTHONPATH ( perhaps in > standard lib ;) > > def set_top_package(module, level): > _top_package = os.path.normpath(module + '/..'*(level+1)) > if _top_package not in sys.path: > sys.path.append(_top_package) > > class absolute_import(object): > def __init__(self, module, level): > self.level = level > self.module = module > > def __enter__(self): > sys.path.insert( 0, > os.path.normpath(self.module + '/..'*(self.level+1)) > ) > > def __exit__(self, exc_type, exc_val, exc_tb): > del sys.path[0] > > --> in module1 > import package_import > package_import.set_top_package(__file__, 1) > import subpack2.module2 > > --> or in module1 > import package_import > with package_import.absolute_import(__file__, 1): > import subpack2.module2 > ... This and similar solutions ( see Istvan Alberts ) point me to a fundamental problem of the current import architecture. Suppose you really want to run a module as a script without a prior import from a module path: ...A\B\C> python my_module.py then the current working directory C is added to sys.path which means that the module finder searches in C but C isn't a known package. There is no C package in sys.modules even if the C directory is "declared" as a package by placing an __init__.py file in it. Same goes of course with B and A. Although the ceremony has been performed basically correct the interpreter god is not pacified and doesn't respond. But why not? Because it looks up for *living* imported packages in the module cache ( in sys.modules ). I don't think there is any particular design idea behind it. The module cache is just a simple flat dictionary; a no-brainer to implement and efficient for look ups. But it counteracts a domain model. All you are left with is those Finders, Loaders and Importers in Brett Cannons importlib. Everything remains deeply mysterious and I don't wonder that it took long for him to work this out. -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Hi, I was looking for a nice idiom for interpackage imports as I found this thread. Here come a couple of solutions I came up with. Any discussion is welcome. I assume the same file structure \ App | main.py +--\subpack1 | | __init__.py | | module1.py | +--\subpack2 | | __init__.py | | module2.py When you run main.py all imports relative to \App work fine, so the only problem is running a module from within a subpackage as a script. I therefore assume we want to run module1.py as a script, which wants to import module2. I hope the following solutions are self-evident = solution 1 --> in module1.py import sys, os if __name__ == '__main__': sys.path.append(os.path.normpath(__file__+'/../..')) import subpack2.module2 = solution 2 --> in subpack1/__init__.py import sys, os _top_package_level = 1 # with current package being level 0 _top_package = os.path.normpath(__file__ + '/..'*(_top_package_level +1)) if _top_package not in sys.path: sys.path.append(_top_package) --> in module1 or any module in the package, which requires import relative to the package top import __init__ import subpack2.module2 = solution 3 --> in package_import.py, somewhere on the PYTHONPATH ( perhaps in standard lib ;) def set_top_package(module, level): _top_package = os.path.normpath(module + '/..'*(level+1)) if _top_package not in sys.path: sys.path.append(_top_package) class absolute_import(object): def __init__(self, module, level): self.level = level self.module = module def __enter__(self): sys.path.insert( 0, os.path.normpath(self.module + '/..'*(self.level+1)) ) def __exit__(self, exc_type, exc_val, exc_tb): del sys.path[0] --> in module1 import package_import package_import.set_top_package(__file__, 1) import subpack2.module2 --> or in module1 import package_import with package_import.absolute_import(__file__, 1): import subpack2.module2 ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On 31 Mrz., 04:55, "Gabriel Genellina" wrote:
> En Mon, 30 Mar 2009 21:15:59 -0300, Aahz escribió:
>
> > In article ,
> > Gabriel Genellina wrote:
>
> >> I'd recommend the oposite - use relative (intra-package) imports when
> >> possible. Explicit is better than implicit - and starting with 2.7 -when
> >> "absolute" import semantics will be enabled by default- you'll *have* to
> >> use relative imports inside a package, or fail.
>
> > Really? I thought you would still be able to use absolute imports; you
> > just won't be able to use implied relative imports instead of explicit
> > relative imports.
>
> You're right, I put it wrongly. To make things clear, inside a package
> "foo" accessible thru sys.path, containing a.py and b.py:
>
> site-packages/
>foo/
> a.py
> b.py
> __init__.py
>
> Currently, the "a" module can import "b" this way:
>
> from foo import b
> import foo.b
> from . import b
> import b
>
> When implicit relative imports are disabled ("from __future__ import
> absolute_import", or after 2.7 supposedly) the last one won't find b.py
> anymore.
> (I hope I put it right this time).
>
> --
> Gabriel Genellina
So it even breaks more code which is great ;)
Do you know of any near or far past attempts to re-design the import
system from the ground up? I do not mean a rather faithful and
accessible reconstruction such as Brett Cannons work but a radical re-
design which starts with a domain model and does not end with Loaders,
Importers and Finders which are actually services that pretend to be
objects.
Kay
--
http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Mon, 30 Mar 2009 21:15:59 -0300, Aahz escribió:
In article ,
Gabriel Genellina wrote:
I'd recommend the oposite - use relative (intra-package) imports when
possible. Explicit is better than implicit - and starting with 2.7 -when
"absolute" import semantics will be enabled by default- you'll *have* to
use relative imports inside a package, or fail.
Really? I thought you would still be able to use absolute imports; you
just won't be able to use implied relative imports instead of explicit
relative imports.
You're right, I put it wrongly. To make things clear, inside a package
"foo" accessible thru sys.path, containing a.py and b.py:
site-packages/
foo/
a.py
b.py
__init__.py
Currently, the "a" module can import "b" this way:
from foo import b
import foo.b
from . import b
import b
When implicit relative imports are disabled ("from __future__ import
absolute_import", or after 2.7 supposedly) the last one won't find b.py
anymore.
(I hope I put it right this time).
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
In article , Gabriel Genellina wrote: > >I'd recommend the oposite - use relative (intra-package) imports when >possible. Explicit is better than implicit - and starting with 2.7 -when >"absolute" import semantics will be enabled by default- you'll *have* to >use relative imports inside a package, or fail. Really? I thought you would still be able to use absolute imports; you just won't be able to use implied relative imports instead of explicit relative imports. -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 25, 1:07 am, Kay Schluehr wrote: > On 25 Mrz., 05:56, Carl Banks wrote: > > > > > > > On Mar 24, 8:32 pm, Istvan Albert wrote: > > > > On Mar 24, 9:35 pm, Maxim Khitrov wrote: > > > > > Works perfectly fine with relative imports. > > > > This only demonstrates that you are not aware of what the problem > > > actually is. > > > > Try using relative imports so that it works when you import the module > > > itself. Now run the module as a program. The same module that worked > > > fine when you imported it will raise the exception: > > > PEP 366 addresses this issue. > > > Not the best solution, one that still involves boilerplate, but it is > > much less of a hack than your version, and at least it has the > > blessing of the language designers so it won't unceremoniously break > > at some point. > > A workaround that is hardly acceptable when we are working with / > debugging 3rd party packages. Python was simpler without relative > imports and occasional ambiguity resolutions by means of absolute > imports. Unfortunately Brett Cannons reconstruction of import > semantics comes a little late for Python 3 and I suppose we have to > live with the current mess Out of curiosity, is there anything--aside from explicit relative imports--worked before but doesn't now? The issue Istvan is talking about exists even with the implicit imports. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Tue, 24 Mar 2009 21:57:12 -0300, Istvan Albert escribió: On Mar 24, 3:16 pm, "Gabriel Genellina" wrote: Did you know, once a module is imported by the first time yeah yeah, could we not get sidetracked with details that are not relevant? what it obviously means is to import it in all of your modules that need to access to relative paths Uh? You posted a module containing a function and some code that modifies sys.path. I imagine you are interested in those side effects in sys.path -- which will only happen the first time it is imported. Isn't it relevant? I don't understand, how is this supposed to help relative imports? That's only because you have not had to deal with the problem that it solves. But I had to deal with the problem that it *creates*, even before relative imports existed, and it's a lot worse. Bindly inserting directories into sys.path can easily confuse the import systemn confuse the import system? what the heck does that mean? You either have a path in the sys.path or not. FWIW it is far cleaner than doing a relative import that does not work correctly. Others have posted the way to execute modules inside a package. For the problem of adding random directories to sys.path, see: http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed47d8e31ca3d411/e22039a2cf166f42?#e22039a2cf166f42 -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On 25 Mrz., 05:56, Carl Banks wrote: > On Mar 24, 8:32 pm, Istvan Albert wrote: > > > On Mar 24, 9:35 pm, Maxim Khitrov wrote: > > > > Works perfectly fine with relative imports. > > > This only demonstrates that you are not aware of what the problem > > actually is. > > > Try using relative imports so that it works when you import the module > > itself. Now run the module as a program. The same module that worked > > fine when you imported it will raise the exception: > > PEP 366 addresses this issue. > > Not the best solution, one that still involves boilerplate, but it is > much less of a hack than your version, and at least it has the > blessing of the language designers so it won't unceremoniously break > at some point. > > Carl Banks A workaround that is hardly acceptable when we are working with / debugging 3rd party packages. Python was simpler without relative imports and occasional ambiguity resolutions by means of absolute imports. Unfortunately Brett Cannons reconstruction of import semantics comes a little late for Python 3 and I suppose we have to live with the current mess. -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 24, 8:32 pm, Istvan Albert wrote: > On Mar 24, 9:35 pm, Maxim Khitrov wrote: > > > Works perfectly fine with relative imports. > > This only demonstrates that you are not aware of what the problem > actually is. > > Try using relative imports so that it works when you import the module > itself. Now run the module as a program. The same module that worked > fine when you imported it will raise the exception: PEP 366 addresses this issue. Not the best solution, one that still involves boilerplate, but it is much less of a hack than your version, and at least it has the blessing of the language designers so it won't unceremoniously break at some point. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 24, 9:35 pm, Maxim Khitrov wrote: > Works perfectly fine with relative imports. This only demonstrates that you are not aware of what the problem actually is. Try using relative imports so that it works when you import the module itself. Now run the module as a program. The same module that worked fine when you imported it will raise the exception: ValueError: Attempted relative import in non-package when running it on its own. Istvan -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
CinnamonDonkey wrote: > Top responses guys! This has all helped increadibly. > > Bearophile, > > My applogies if I have offended you, but: > > 1. "I can't know much about you from the start" - Is kind of my point. > Perhaps it would be better to avoid jumping to conclusions and pre- > judging someones abilities simply because they are lacking knowledge > in a particular area. > I agree in your case it meant you were misjudged, but experience has proved that it's important to establish the correct level of discourse before proceeding to discuss solutions. > Would it have been better if I had opened my thread with a copy of my > CV? I've got a Degree in Digital Systems Engineering (yes I am also an > Engineer)... I went on to do a Phd in AI and Robotics where I also > developed embedded systems. I bummed out on that after 3 years and > went on to work for a games company where I worked on 6 game titles > across 5 different platforms. I was a Lead Software Engineer for the > last 5 years. Before now moving on again. > No, none of that would have helped ;-) > 2. I am also very much a follower of the K.I.S.S. approach, 9 times > out of 10 the simplest solution is often the best. > > As an Engineer though I would also suggest that the best way to learn > is to try solving a problem being sure to constantly assess your > approach and then your final solution. By dismissing a possible avenue > you are dismissing a whole new path of knowledge. Is it not better to > try and fail than never try at all? Surely this is how we gain the > valuable experience we need. > > By simply suggesting a "simple default" solution, I may never have > consider the alternatives nor understand why they are or are not > suitable. > That's true, but we seem to be converging to rational discussion. > 3. I get your point about Students, sometimes there is such a thing as > too much knowledge or information overload. Whilst doing a PhD I had > to take labs and teach students... The approach I tried to take was > one off, "You may not need packages, why do you think you need > packages?" or "This is how packages would be used and why they would > be used... do you still think you need packages" or better still, for > a capable student, "This is how packages work, try solving your > problem and then tell me if you think it was a good solution." > > > Going with R. David Murray, perhaps I also jumped too my own > conclusion too quickly and for that I appologise. > People round here generally have broad shoulders and are slow to take offense. No need to worry - and welcome to c.l.py! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Want to know? Come to PyCon - soon! http://us.pycon.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Tue, Mar 24, 2009 at 8:57 PM, Istvan Albert wrote: > Does it not bother you that a module that uses relative imports cannot > be run on its own anymore? $ python --help -m mod : run library module as a script (terminates option list) $ python -m some.module.name Works perfectly fine with relative imports. - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 24, 3:16 pm, "Gabriel Genellina" wrote: > Did you know, once a module is imported by the first time yeah yeah, could we not get sidetracked with details that are not relevant? what it obviously means is to import it in all of your modules that need to access to relative paths > I don't understand, how is this supposed to help relative imports? That's only because you have not had to deal with the problem that it solves. If you need to have a module that can do both: 1. access relative paths (even other packages) 2. be executable on its own (for example a modules may execute its own doctests when running them directly) this is the only way to achieve it. > I'd recommend the oposite - use relative (intra-package) imports when > possible. Does it not bother you that a module that uses relative imports cannot be run on its own anymore? To me that is irritating because it forces me to change a habit (running the doctests when the module is executed) that I consider a very good practice. It is extremely handy to be writing a module, press a key and the module is executed and I can see the tests results. The relative import takes away this from me. Like I said, it is irritating. > Bindly inserting directories into sys.path can easily confuse the import > systemn confuse the import system? what the heck does that mean? You either have a path in the sys.path or not. FWIW it is far cleaner than doing a relative import that does not work correctly. Istvan -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Tue, 24 Mar 2009 09:01:01 -0300, R. David Murray escribió: CinnamonDonkey wrote: On 23 Mar, 18:57, bearophileh...@lycos.com wrote: > CinnamonDonkey: > > >what makes something a package? > > If you don't know what a package is, then maybe you don't need > packages. Thanx for taking the time to post a response but I am afraid I feel the need to point out that it is exactly this kind of response that I find un-helpful. It is neither constructive nor educational. I think bearophile could have left out the first sentence, but otherwise his question is perfectly sensible. If you have a bunch of independent modules, then you don't need to put them in packages. Your example only showed one module file in each package...I understand now that was just for simplicity of the example, but we had no way of knowing that. Especially, since people coming from Java: - think that they *have* to use a package - think that they *have* to put a single class per file Neither is true in Python. And from the example alone, one could infer the OP was making such mistakes. Glad to see it was not the case. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Tue, 24 Mar 2009 12:49:08 -0300, Istvan Albert escribió: On Mar 23, 10:16 am, CinnamonDonkey wrote: I'm fairly new to Python so I still have a lot to learn. But I'd like to know how to correectly use relative imports. Relative imports are *fundamentally* broken in python. You will soon see that code using relative import will break if you attempt to run the module on its own. Yes, it is mindboggling! How import works in general is hard to grasp -- the semantics are rather complicated, partly because the import system has grown patch over patch over the years, and because it's mostly underdocumented. But I would not say that relative imports are "fundamentally broken" -- they're one of the best improvements to the import system! Why is it so you ask? It is one of those issue that would be trivial to implement correctly (it is relative to the current file location, duh!!!), would be tremendously useful yet for some reason it is controversial with those who would need to implement it. A module doesn't *have* to reside in the filesystem - so in the general case, there is no such thing as "current file location". Packages provide such hierarchical disposition - and relative imports work with packages only. It looks like they think that the expected mode of operation would greatly confuse the 'rest' of us. So that is the reason you end up with a broken implementation that is worse than not having it at all. All I can do is recommend that you avoid relative imports. I'd recommend the oposite - use relative (intra-package) imports when possible. Explicit is better than implicit - and starting with 2.7 -when "absolute" import semantics will be enabled by default- you'll *have* to use relative imports inside a package, or fail. The easiest way around the issue is to create a module named pathfix.py like the one below and import it in all of your modules. This is the only way to fix this issue in a way that does not come back and bite you, it is ugly, you are repeating yourself in multiple places, but it is still better than the alternative. "import it in all of your modules"? Did you know, once a module is imported by the first time, subsequent imports simply return the same module instance? The initialization code is not executed again. import os, sys def path_join(*args): return os.path.abspath(os.path.join(*args)) # adds base directory to import path to allow relative imports __currdir = os.path.dirname( __file__ ) __basedir = path_join(__currdir, '..' ) if __basedir not in sys.path: sys.path.append( __basedir ) I don't understand, how is this supposed to help relative imports? Bindly inserting directories into sys.path can easily confuse the import system (and you!) -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 23, 10:16 am, CinnamonDonkey wrote: > I'm fairly new to Python so I still have a lot to learn. But I'd like > to know how to correectly use relative imports. Relative imports are *fundamentally* broken in python. You will soon see that code using relative import will break if you attempt to run the module on its own. Yes, it is mindboggling! Why is it so you ask? It is one of those issue that would be trivial to implement correctly (it is relative to the current file location, duh!!!), would be tremendously useful yet for some reason it is controversial with those who would need to implement it. It looks like they think that the expected mode of operation would greatly confuse the 'rest' of us. So that is the reason you end up with a broken implementation that is worse than not having it at all. All I can do is recommend that you avoid relative imports. The easiest way around the issue is to create a module named pathfix.py like the one below and import it in all of your modules. This is the only way to fix this issue in a way that does not come back and bite you, it is ugly, you are repeating yourself in multiple places, but it is still better than the alternative. --- import os, sys def path_join(*args): return os.path.abspath(os.path.join(*args)) # adds base directory to import path to allow relative imports __currdir = os.path.dirname( __file__ ) __basedir = path_join(__currdir, '..' ) if __basedir not in sys.path: sys.path.append( __basedir ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Top responses guys! This has all helped increadibly. Bearophile, My applogies if I have offended you, but: 1. "I can't know much about you from the start" - Is kind of my point. Perhaps it would be better to avoid jumping to conclusions and pre- judging someones abilities simply because they are lacking knowledge in a particular area. Would it have been better if I had opened my thread with a copy of my CV? I've got a Degree in Digital Systems Engineering (yes I am also an Engineer)... I went on to do a Phd in AI and Robotics where I also developed embedded systems. I bummed out on that after 3 years and went on to work for a games company where I worked on 6 game titles across 5 different platforms. I was a Lead Software Engineer for the last 5 years. Before now moving on again. 2. I am also very much a follower of the K.I.S.S. approach, 9 times out of 10 the simplest solution is often the best. As an Engineer though I would also suggest that the best way to learn is to try solving a problem being sure to constantly assess your approach and then your final solution. By dismissing a possible avenue you are dismissing a whole new path of knowledge. Is it not better to try and fail than never try at all? Surely this is how we gain the valuable experience we need. By simply suggesting a "simple default" solution, I may never have consider the alternatives nor understand why they are or are not suitable. 3. I get your point about Students, sometimes there is such a thing as too much knowledge or information overload. Whilst doing a PhD I had to take labs and teach students... The approach I tried to take was one off, "You may not need packages, why do you think you need packages?" or "This is how packages would be used and why they would be used... do you still think you need packages" or better still, for a capable student, "This is how packages work, try solving your problem and then tell me if you think it was a good solution." Going with R. David Murray, perhaps I also jumped too my own conclusion too quickly and for that I appologise. Cheers, Shaun On 24 Mar, 12:37, bearophileh...@lycos.com wrote: > CinnamonDonkey: > > > It is neither constructive nor educational. > > > It's a bit like saying "If you don't know what a function is, then > > maybe you don't need it. ... have you tried having a single block of > > code?" > > > The point of people coming to these forums is to LEARN and share > > knowledge. Perhaps it's not the best solution for me right now but > > without trying it I won't know when or how to apply it as a solution. > > > By the way, my project has about 50 files (modules) in it with a lot > > of shared code that could be used across other projects... seems as > > good a reason as any to try packages out ;-) > > I don't agree. My answer can be wrong for your situation, but I can't > know much about you from the start, and for most people my answer was > the right one. > > When a student asks me how to improve his/her usage of metaclasses I > usually answer that metaclasses aren't required to solve that small > problem. > > Generally in engineering the simplest solution is the one you have to > try first (and often second and third), and only later, if practical > experience shows the simple solution doesn't work, you try a more > complex solution. > > So I have suggested you a simple solution by "default". If later you > see that you have many modules and you really need packages, then it's > easy to ignore my first answer. > > Bye, > bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
CinnamonDonkey: > It is neither constructive nor educational. > > It's a bit like saying "If you don't know what a function is, then > maybe you don't need it. ... have you tried having a single block of > code?" > > The point of people coming to these forums is to LEARN and share > knowledge. Perhaps it's not the best solution for me right now but > without trying it I won't know when or how to apply it as a solution. > > By the way, my project has about 50 files (modules) in it with a lot > of shared code that could be used across other projects... seems as > good a reason as any to try packages out ;-) I don't agree. My answer can be wrong for your situation, but I can't know much about you from the start, and for most people my answer was the right one. When a student asks me how to improve his/her usage of metaclasses I usually answer that metaclasses aren't required to solve that small problem. Generally in engineering the simplest solution is the one you have to try first (and often second and third), and only later, if practical experience shows the simple solution doesn't work, you try a more complex solution. So I have suggested you a simple solution by "default". If later you see that you have many modules and you really need packages, then it's easy to ignore my first answer. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Top posting corrected for clarity. CinnamonDonkey wrote: > On 23 Mar, 18:57, bearophileh...@lycos.com wrote: > > CinnamonDonkey: > > > > >what makes something a package? > > > > If you don't know what a package is, then maybe you don't need > > packages. > > > > In your project is it possible to avoid using packages and just use > > modules in the same directory? > > > > Bye, > > bearophile > > Hi Bearophile, > > Thanx for taking the time to post a response but I am afraid I feel > the need to point out that it is exactly this kind of response that I > find un-helpful. It is neither constructive nor educational. > > It's a bit like saying "If you don't know what a function is, then > maybe you don't need it. ... have you tried having a single block of > code?" > > The point of people coming to these forums is to LEARN and share > knowledge. Perhaps it's not the best solution for me right now but > without trying it I won't know when or how to apply it as a solution. > > By the way, my project has about 50 files (modules) in it with a lot > of shared code that could be used across other projects... seems as > good a reason as any to try packages out ;-) > > Thanx anyway :) I think bearophile could have left out the first sentence, but otherwise his question is perfectly sensible. If you have a bunch of independent modules, then you don't need to put them in packages. Your example only showed one module file in each package...I understand now that was just for simplicity of the example, but we had no way of knowing that. We've had newbies come in and think they _need_ to put a module file into a subpackage even when they'd only have one module file per subdirectory and they don't really know what a package is...thus bearophile's (perhaps poorly phrased) question. Now that you know what packages are and what the restrictions on relative imports are, and you've told us that you have '50 modules' with a lot of shared code that 'could be used across other projects', perhaps you see why relative imports are generally discouraged. If you use only relative imports, the code _can't_ be shared across multiple projects, because all that project code would have to be in one monster package, and not be separate projects at all. So now you'll know better where it makes Pythonic (as opposed to C++) sense to use it and where not. -- R. David Murray http://www.bitdance.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Tue, Mar 24, 2009 at 5:05 AM, CinnamonDonkey
wrote:
> Thanx Max - your explanation sorted it :-), and a big thank you to
> everyone else also!
>
> >From the various posts, Python considers any directory containing the
> __init__.py file to be a package. The top level package is the highest
> directory (closest to root) with a __init__.py file.
>
> Inter-package communication is not allowed unless the packages
> themselves are contained by a parent package.
>
> How does this relate to the site-packages folder? Is it a top level
> package for all installed packages?
>
> Let's say I have installed the "Trac" system which uses "Genshi", they
> are both packages. They are also both installed at the same level and
> I know "Trac" uses "Genshi" to work. \Python25\Lib\site-packages does
> not contain a __init__.py file so it is not a package (i.e. not a
> parent package to "Trac" and "Genshi") :0.
Trac does not use relative imports to access genshi. When relative
imports are not used, python goes through sys.path list to find
modules (with a small exception made when absolute_imports are not
enabled, but that should be default in 2.7). The site-packages
directory is added to sys.path, so when trac executes something like
"from genshi import some_module", python will look in site-packages,
among other directories, for a directory called "genshi" that contains
an __init__.py file.
When you execute a script, the directory of that script is
automatically added to sys.path, so with your example you could have
used absolute imports between subpack1 and subpack2, with the \App
directory performing the same function as site-packages (Gabriel's
suggestion). This is for your original version of the code when
main.py was under App.
Once you moved main.py outside of \App, running "import sybpack2"
would no longer work. You can, however, append directories to
sys.path, so by doing the following in main.py you could again allow
non-relative imports between subpack1 and subpack2:
import os
import sys
sys.path.append(os.path.realpath('App'))
- Max
--
http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Thanx Max - your explanation sorted it :-), and a big thank you to everyone else also! >From the various posts, Python considers any directory containing the __init__.py file to be a package. The top level package is the highest directory (closest to root) with a __init__.py file. Inter-package communication is not allowed unless the packages themselves are contained by a parent package. How does this relate to the site-packages folder? Is it a top level package for all installed packages? Let's say I have installed the "Trac" system which uses "Genshi", they are both packages. They are also both installed at the same level and I know "Trac" uses "Genshi" to work. \Python25\Lib\site-packages does not contain a __init__.py file so it is not a package (i.e. not a parent package to "Trac" and "Genshi") :0. -=- bearophile -=- Hi Bearophile, Thanx for taking the time to post a response but I am afraid I feel the need to point out that it is exactly this kind of response that I find un-helpful. It is neither constructive nor educational. It's a bit like saying "If you don't know what a function is, then maybe you don't need it. ... have you tried having a single block of code?" The point of people coming to these forums is to LEARN and share knowledge. Perhaps it's not the best solution for me right now but without trying it I won't know when or how to apply it as a solution. By the way, my project has about 50 files (modules) in it with a lot of shared code that could be used across other projects... seems as good a reason as any to try packages out ;-) Thanx anyway :) On 23 Mar, 18:57, bearophileh...@lycos.com wrote: > CinnamonDonkey: > > >what makes something a package? > > If you don't know what a package is, then maybe you don't need > packages. > > In your project is it possible to avoid using packages and just use > modules in the same directory? > > Bye, > bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
CinnamonDonkey: >what makes something a package? If you don't know what a package is, then maybe you don't need packages. In your project is it possible to avoid using packages and just use modules in the same directory? Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mar 23, 11:22 am, CinnamonDonkey wrote: > Hi Guys, > > Thanx for the quick responses, it is very much appreciated! > > Skip, that's a good point about "C++ != Python" and I assure you I am > very much aware of that ;-). > > Looking athttp://www.python.org/dev/peps/pep-0328/#guido-s-decision > would suggest, unless I am completely miss-understanding the example, > that '.' refers to the current level and '..' pops up a level. First > three uses: > > # Access moduleY in same level as ModuleX > from .moduleY import spam > from .moduleY import spam as ham > from . import moduleY > > Just to be sure though I tried both ;): > > from ..subpack2 import module1 #ValueError: Attempted relative import > beyond toplevel package > from .subpack2 import module1 #ImportError: No module named subpack2 > from . import subpack2 #ImportError: cannot import name subpack2 > from .. import subpack2 #ValueError: Attempted relative import beyond > toplevel package > > Max, thank you for the response... I tried adding "from __future__ > import absolute_import" which made no difference. I still get exactly > the same error messages. Perhaps I should have mentioned that I am > using Python 2.5, which I understand alread supports relative imports > out of the box. I'll keep this line in for now anyway though :-) > Cheers! I wrote http://code.google.com/p/pyimport-relative so I could have a simple and more reliable way to specify a *file* relative import in Python 2.5 (and it works with 2.6 as well). I'm not sure it's for everyone, or that it is flawless. Actually, it's not. But I use it for in pydbgr (http://code.google.com/p/pydbgr) so see that for examples of how to use. It allows me to run the development code out of the source tree while having possibly a different version installed as an egg. > > #subpack1.module1 > from __future__ import absolute_import > > from .. import subpack2 > > def subpack1_module1_foo(): > print "subpack1_module1_foo()" > call_subpack2_module1() > > def call_subpack2_module1(): > subpack2.module2.subpack2_module2_foo() > > #subpack2.module2 > def subpack2_module2_foo(): > print "subpack2_module2_foo()" > > #main.py > import subpack1.module1 > > if __name__ == "__main__": > subpack1.module1.subpack1_module1_foo() > > I am starting the sandbox app with the command line: python main.py > with the current working directory in \app where main.py is located. > > Shaun >8) > > On 23 Mar, 14:44, s...@pobox.com wrote: > > > >> Please, please... please! don't go off on rants about why you think > > >> relative imports should not be used. I've got 15+ years in C++ and > > >> relative inclusion of other sections of code has never been a > > >> problem. As far as I am concerned what I am trying to do is > > >> perfectly reasonable and valid. > > > However, C++ != Python. Regardless whether or not you can "make it work", > > translating idioms from one language to another is often suboptimal. That > > may not be the case here, but it bears keeping in mind. > > > >> Example: > > > >> \ App > > >> | main.py > > >> +--\subpack1 > > >> | | __init__.py > > >> | | module1.py > > >> | > > >> +--\subpack2 > > >> | | __init__.py > > >> | | module2.py > > > >> Module1 needs to access functionality in Module2. > > > >> #module1.py > > >> from ..subpack2 import module2 > > > >> Seems reasonable to me... but it just does not work and I was so > > >> liking Python. :( > > > You might try removing a dot from your import statement. Python is also not > > Unix. Popping up one level in the package hierarchy is done with ".", not > > "..". > > > -- > > Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/ > > -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Mon, 23 Mar 2009 13:19:51 -0300, CinnamonDonkey escribió: My applogies if this is a silly question... but what makes something a package? A package is a directory with an __init__.py file [that Python is aware of]. and does that mean that what I am trying to do is not possible ? You can do an "absolute" import of subpack1 and subpack2. But you cannot import them "relatively" - not using your current configuration, but see Maxim Khitrov post for an alternate disposition that works. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 12:19 PM, CinnamonDonkey wrote: > My applogies if this is a silly question... but what makes something a > package? and does that mean that what I am trying to do is not > possible ? A package is a directory that has an __init__.py file. That file can be empty, or contain some initialization code. In your original example, subpack1 and subpack2 are packages. By adding an empty __init__.py file under \App, I made App into a package, which allowed me to execute "import App.subpack1.module1" in main.py. See the following url for additional info: http://docs.python.org/tutorial/modules.html - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
My applogies if this is a silly question... but what makes something a package? and does that mean that what I am trying to do is not possible ? :( On 23 Mar, 15:53, "Gabriel Genellina" wrote: > En Mon, 23 Mar 2009 12:22:21 -0300, CinnamonDonkey > escribió: > > > > >> >> \ App > >> >> | main.py > >> >> +--\subpack1 > >> >> | | __init__.py > >> >> | | module1.py > >> >> | > >> >> +--\subpack2 > >> >> | | __init__.py > >> >> | | module2.py > > >> >> Module1 needs to access functionality in Module2. > > >> >> #module1.py > >> >> from ..subpack2 import module2 > > >> >> Seems reasonable to me... but it just does not work and I was so > >> >> liking Python. :( > > Another name for relative imports is "intra-package imports". They work > *inside* a package, and you cannot go out of the package. > If App is not a package, then subpack1 and subpack2 are separate packages > and you cannot use relative imports between them. So module1 must refer to > module2 absolutely: > > from subpack2 import module2 > > > from ..subpack2 import module1 #ValueError: Attempted relative import > > beyond toplevel package > > See the exception message. > > > Max, thank you for the response... I tried adding "from __future__ > > import absolute_import" which made no difference. I still get exactly > > the same error messages. Perhaps I should have mentioned that I am > > using Python 2.5, which I understand alread supports relative imports > > out of the box. I'll keep this line in for now anyway though :-) > > That __future__ line is not to enable relative imports (since they have > incompatible syntax, don't require anything special) but to ensure Python > interprets "normal" imports (that is, without leading dots) always as > absolute. The default behavior in 2.5 is to try *both* ways before failing. > > -- > Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
En Mon, 23 Mar 2009 12:22:21 -0300, CinnamonDonkey escribió: >> \ App >> | main.py >> +--\subpack1 >> | | __init__.py >> | | module1.py >> | >> +--\subpack2 >> | | __init__.py >> | | module2.py >> Module1 needs to access functionality in Module2. >> #module1.py >> from ..subpack2 import module2 >> Seems reasonable to me... but it just does not work and I was so >> liking Python. :( Another name for relative imports is "intra-package imports". They work *inside* a package, and you cannot go out of the package. If App is not a package, then subpack1 and subpack2 are separate packages and you cannot use relative imports between them. So module1 must refer to module2 absolutely: from subpack2 import module2 from ..subpack2 import module1 #ValueError: Attempted relative import beyond toplevel package See the exception message. Max, thank you for the response... I tried adding "from __future__ import absolute_import" which made no difference. I still get exactly the same error messages. Perhaps I should have mentioned that I am using Python 2.5, which I understand alread supports relative imports out of the box. I'll keep this line in for now anyway though :-) That __future__ line is not to enable relative imports (since they have incompatible syntax, don't require anything special) but to ensure Python interprets "normal" imports (that is, without leading dots) always as absolute. The default behavior in 2.5 is to try *both* ways before failing. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 11:22 AM, CinnamonDonkey wrote: > Looking at http://www.python.org/dev/peps/pep-0328/#guido-s-decision > would suggest, unless I am completely miss-understanding the example, > that '.' refers to the current level and '..' pops up a level. That is correct, but you cannot jump beyond the parent package, which is why your code isn't working. > Max, thank you for the response... I tried adding "from __future__ > import absolute_import" which made no difference. I still get exactly > the same error messages. Perhaps I should have mentioned that I am > using Python 2.5, which I understand alread supports relative imports > out of the box. I'll keep this line in for now anyway though :-) > Cheers! Sorry, I use that line to avoid conflicts with standard modules, and forgot that relative imports are already enabled. Basically, the reason your code doesn't work is because you're trying to use relative imports between two separate packages. As far as I know, this isn't possible. What I did to get your code working was move main.py one directory up and create an empty __init__.py under \App. The following code should then work: # main.py import App.subpack1.module1 if __name__ == "__main__": App.subpack1.module1.subpack1_module1_foo() # App.subpack1.module1 from ..subpack2 import module2 def subpack1_module1_foo(): print "subpack1_module1_foo()" call_subpack2_module1() def call_subpack2_module1(): module2.subpack2_module2_foo() # App.subpack2.module2 def subpack2_module2_foo(): print "subpack2_module2_foo()" - Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
Hi Guys, Thanx for the quick responses, it is very much appreciated! Skip, that's a good point about "C++ != Python" and I assure you I am very much aware of that ;-). Looking at http://www.python.org/dev/peps/pep-0328/#guido-s-decision would suggest, unless I am completely miss-understanding the example, that '.' refers to the current level and '..' pops up a level. First three uses: # Access moduleY in same level as ModuleX from .moduleY import spam from .moduleY import spam as ham from . import moduleY Just to be sure though I tried both ;): from ..subpack2 import module1 #ValueError: Attempted relative import beyond toplevel package from .subpack2 import module1 #ImportError: No module named subpack2 from . import subpack2 #ImportError: cannot import name subpack2 from .. import subpack2 #ValueError: Attempted relative import beyond toplevel package Max, thank you for the response... I tried adding "from __future__ import absolute_import" which made no difference. I still get exactly the same error messages. Perhaps I should have mentioned that I am using Python 2.5, which I understand alread supports relative imports out of the box. I'll keep this line in for now anyway though :-) Cheers! #subpack1.module1 from __future__ import absolute_import from .. import subpack2 def subpack1_module1_foo(): print "subpack1_module1_foo()" call_subpack2_module1() def call_subpack2_module1(): subpack2.module2.subpack2_module2_foo() #subpack2.module2 def subpack2_module2_foo(): print "subpack2_module2_foo()" #main.py import subpack1.module1 if __name__ == "__main__": subpack1.module1.subpack1_module1_foo() I am starting the sandbox app with the command line: python main.py with the current working directory in \app where main.py is located. Shaun >8) On 23 Mar, 14:44, s...@pobox.com wrote: > >> Please, please... please! don't go off on rants about why you think > >> relative imports should not be used. I've got 15+ years in C++ and > >> relative inclusion of other sections of code has never been a > >> problem. As far as I am concerned what I am trying to do is > >> perfectly reasonable and valid. > > However, C++ != Python. Regardless whether or not you can "make it work", > translating idioms from one language to another is often suboptimal. That > may not be the case here, but it bears keeping in mind. > > >> Example: > > >> \ App > >> | main.py > >> +--\subpack1 > >> | | __init__.py > >> | | module1.py > >> | > >> +--\subpack2 > >> | | __init__.py > >> | | module2.py > > >> Module1 needs to access functionality in Module2. > > >> #module1.py > >> from ..subpack2 import module2 > > >> Seems reasonable to me... but it just does not work and I was so > >> liking Python. :( > > You might try removing a dot from your import statement. Python is also not > Unix. Popping up one level in the package hierarchy is done with ".", not > "..". > > -- > Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
>> Please, please... please! don't go off on rants about why you think >> relative imports should not be used. I've got 15+ years in C++ and >> relative inclusion of other sections of code has never been a >> problem. As far as I am concerned what I am trying to do is >> perfectly reasonable and valid. However, C++ != Python. Regardless whether or not you can "make it work", translating idioms from one language to another is often suboptimal. That may not be the case here, but it bears keeping in mind. >> Example: >> \ App >> | main.py >> +--\subpack1 >> | | __init__.py >> | | module1.py >> | >> +--\subpack2 >> | | __init__.py >> | | module2.py >> Module1 needs to access functionality in Module2. >> #module1.py >> from ..subpack2 import module2 >> Seems reasonable to me... but it just does not work and I was so >> liking Python. :( You might try removing a dot from your import statement. Python is also not Unix. Popping up one level in the package hierarchy is done with ".", not "..". -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports, why the hell is it so hard?
On Mon, Mar 23, 2009 at 10:16 AM, CinnamonDonkey wrote: > Hi All, > > I'm fairly new to Python so I still have a lot to learn. But I'd like > to know how to correectly use relative imports. > > Please, please... please! don't go off on rants about why you think > relative imports should not be used. I've got 15+ years in C++ and > relative inclusion of other sections of code has never been a problem. > As far as I am concerned what I am trying to do is perfectly > reasonable and valid. > > Thank you in advance to everyone who helps solve this, because I just > don't get it. > > Example: > > \ App > | main.py > +--\subpack1 > | | __init__.py > | | module1.py > | > +--\subpack2 > | | __init__.py > | | module2.py > > > Module1 needs to access functionality in Module2. > > #module1.py > from ..subpack2 import module2 > > Seems reasonable to me... but it just does not work and I was so > liking Python. :( Relative imports are perfectly fine, in my opinion. Do you have "from __future__ import absolute_import" at the top of module1.py? Should work fine once you add that line. - Max -- http://mail.python.org/mailman/listinfo/python-list
Relative Imports, why the hell is it so hard?
Hi All, I'm fairly new to Python so I still have a lot to learn. But I'd like to know how to correectly use relative imports. Please, please... please! don't go off on rants about why you think relative imports should not be used. I've got 15+ years in C++ and relative inclusion of other sections of code has never been a problem. As far as I am concerned what I am trying to do is perfectly reasonable and valid. Thank you in advance to everyone who helps solve this, because I just don't get it. Example: \ App | main.py +--\subpack1 | | __init__.py | | module1.py | +--\subpack2 | | __init__.py | | module2.py Module1 needs to access functionality in Module2. #module1.py from ..subpack2 import module2 Seems reasonable to me... but it just does not work and I was so liking Python. :( -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports in Python 3.0
On 17 Dez., 11:01, Nicholas wrote: > I am sure I am not the first to run into this issue, but what is the > solution? When you use 2to3 just uncomment or delete the file fix_import.py in lib2to3/fixes/ . -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports in Python 3.0
On Dec 17, 4:01 am, Nicholas wrote: > Imagine a module that looks like > > ModuleDir > __init__.py > a.py > b.py > > In python 2.x I used to have tests at the end of each of my modules, > so that module b.py might look something like > > import a > .. > .. > > if __name__ == '__main__': > runtests() > > But under Python 3.0 this seems impossible. For usual use import a.py > has to become the line: > > from . import a > > But if I use that form it is no longer possible to run b.py as a > standalone script without raising an error about using relative > imports. > > I am sure I am not the first to run into this issue, but what is the > solution? Use absolute imports: from ModuleDir import a > > Best wishes, > > Nicholas -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports in Python 3.0
nicholas.c...@gmail.com wrote:
Imagine a module that looks like
ModuleDir
__init__.py
a.py
b.py
In python 2.x I used to have tests at the end of each of my modules,
so that module b.py might look something like
import a
..
..
if __name__ == '__main__':
runtests()
But under Python 3.0 this seems impossible. For usual use import a.py
has to become the line:
from . import a
But if I use that form it is no longer possible to run b.py as a
standalone script without raising an error about using relative
imports.
I am sure I am not the first to run into this issue, but what is the
solution?
Best wishes,
Nicholas
--
http://mail.python.org/mailman/listinfo/python-list
Sorry for the duplicate, sent to wrong email.
Python 3 (and I think 2.6) now use absolute import when using a 'import
blah' statement.
if ('.' in __name__) or hasattr(globals, '__path__'):
from . import a
else:
import a
If '__name__' has a'.' then it is either a package or a module in a
package, in which case relative imports can be used. If it does not
have a '.' it may still be a package but the '__init__.py' file, in
which case the module has a '__path__' attribute, so relative imports
can be used. Otherwise it is not a package or in a package so absolute
imports must used. Also, since it is not in a package it is assumed
that it is top module (__main__) or possible module imported from the
top that is not in a package, such as a.py doing an 'import b', b would
be a module but not a package so still probably need absolute imports,
my guess anyway.
But I also think that 'from . import a' would be nice if it would work
from non-packages as well, meaning just 'import a' if it is a non-package.
Brian A. Vanderburg II
--
http://mail.python.org/mailman/listinfo/python-list
Relative imports in Python 3.0
Imagine a module that looks like ModuleDir __init__.py a.py b.py In python 2.x I used to have tests at the end of each of my modules, so that module b.py might look something like import a .. .. if __name__ == '__main__': runtests() But under Python 3.0 this seems impossible. For usual use import a.py has to become the line: from . import a But if I use that form it is no longer possible to run b.py as a standalone script without raising an error about using relative imports. I am sure I am not the first to run into this issue, but what is the solution? Best wishes, Nicholas -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports and "import X as Y"
Gabriel Genellina wrote: > En Sun, 31 Aug 2008 07:27:12 -0300, Wojtek Walczak > <[EMAIL PROTECTED]> escribió: > >> On Sun, 31 Aug 2008 06:40:35 GMT, OKB (not okblacke) wrote: >> Download the latest beta for your system and give it a try. >>> >>> Thanks for the advice, but I'd really rather not deal >>> with >>> installing the entire thing alongside my existing version, >>> possibly causing conflicts in who knows what ways. >> >> Then you can download tar.gz package, compile it, and try it >> without installing :-) > > Or use a virtual machine, or a live CD. Gosh, thanks, everyone, for being so helpful! -- --OKB (not okblacke) Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports and "import X as Y"
En Sun, 31 Aug 2008 07:27:12 -0300, Wojtek Walczak <[EMAIL PROTECTED]> escribió: > On Sun, 31 Aug 2008 06:40:35 GMT, OKB (not okblacke) wrote: > >>> Download the latest beta for your system and give it a try. >> >> Thanks for the advice, but I'd really rather not deal with >> installing the entire thing alongside my existing version, possibly >> causing conflicts in who knows what ways. > > Then you can download tar.gz package, compile it, and try it > without installing :-) Or use a virtual machine, or a live CD. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports and "import X as Y"
On Sun, 31 Aug 2008 06:40:35 GMT, OKB (not okblacke) wrote: >> Download the latest beta for your system and give it a try. > > Thanks for the advice, but I'd really rather not deal with > installing the entire thing alongside my existing version, possibly > causing conflicts in who knows what ways. Then you can download tar.gz package, compile it, and try it without installing :-) -- Regards, Wojtek Walczak, http://tosh.pl/gminick/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports and "import X as Y"
Terry Reedy wrote: > >> So, will relative imports in Python 3.0 allow things like >> "import >> ..relative.importing.path as prettyname"? If not, why not? > > Download the latest beta for your system and give it a try. Thanks for the advice, but I'd really rather not deal with installing the entire thing alongside my existing version, possibly causing conflicts in who knows what ways. -- --OKB (not okblacke) Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports and "import X as Y"
So, will relative imports in Python 3.0 allow things like "import ..relative.importing.path as prettyname"? If not, why not? Download the latest beta for your system and give it a try. -- http://mail.python.org/mailman/listinfo/python-list
Relative imports and "import X as Y"
I was looking at PEP 328. It says that relative imports with the dot syntax will only be allowed with the "from" import variety (e.g., "from ..somemodule import somename"). The reason given for this in the PEP is that after import xxx.yyy, "xxx.yyy" is usable in an expression, but the leading period syntax is inconsistent with this (you can't use ".somepackage.somemodule" in an expression). However, this reasoning doesn't apply to "import X as Y", as long as Y is an ordinary name. As a few recent posts here have touched on, there are actual differences between the "import X (as Y)" and "from X import name" varieties, most notably that the second form binds name in the importing namespace whereas the first does not. So, will relative imports in Python 3.0 allow things like "import ..relative.importing.path as prettyname"? If not, why not? -- --OKB (not okblacke) Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown -- http://mail.python.org/mailman/listinfo/python-list
Re: relative imports improve program organization... suggestions?
Carl: Your solution is kind of what I was leaning towards after a bit of thinking. Since I have to have the modules each have their own detect() method, then it wouldn't be too hard to have their own test() method to put them through their paces. Catrironpi: I will look into this as it might help just get around the problem, but in my eyes it's still not 'clean'. Simply because people (including me) would expect the regular 'import' statement over the method I would think. Thanks to all who responded. -- http://mail.python.org/mailman/listinfo/python-list
Re: relative imports improve program organization... suggestions?
On Aug 8, 12:37 pm, DG <[EMAIL PROTECTED]> wrote: > Alright, I have searched and searched and read many conversations on > the topic of relative and absolute imports and am still not getting > the whole thing through my skull. > > Highlights of what I've > read:http://mail.python.org/pipermail/python-list/2007-January/422973.htmlhttp://groups.google.com/group/comp.lang.python/browse_thread/thread/...http://www.python.org/dev/peps/pep-0328/http://docs.python.org/whatsnew/pep-328.html > > So my problem and argument: > I want to create a package organized as the following: > pckg/ > __init__.py > main.py > moduleA/ > __init__.py > A_base.py > A1/ > __init__.py > A_inherit1.py > other_A1_files... > A2/ > __init__.py > A_inherit2.py > other_A2_files... > moduleB/ > ... > Explanation: > The main program is located in main.py and it implements the different > modules (A, B). Within the modules the basic organization is; the > base class for all different types of A is directly within the moduleA > directory. All of the different inherited classes of A are within > their own subdirectory with a mess of their own files. This is done > so that a new subclass of A can be added/removed by just adding/ > removing the subdirectory and each of these subclasses may have their > own maintainer, but they should all inherit from A_base.py > > If I am developing the A1 directory, I want to be able to test > A_inherit1.py by using 'if __name__ == "__main__"' within the > A_inherit1.py file and by typing 'python A_inherit1.py' on the command > line. I prefer this simply to keep all unit tests within the same > directory and same file as the inherited class. > > My Problem: > A_inherit1.py has the line: > 'from ..A_base import A_Base_Class' > so that I can later declare the inherited class as such: > 'A1_Inherited_Class(A_Base_Class):' > > *BUT* I get the 'attempted relative import in non-package' error even > when I try the > 'from __future__ import absolute_import' command. > I would prefer to be able to test the file without adding anything to > the PYTHONPATH, like I said by using the name == main trick. > > So could someone explain to me what the rationale behind not allowing > parent directory relative imports is? And possibly what I can do to > get around it? (I really don't like messing with the sys.path for > something like this) > > Thanks, > Danny G Didn't read the whole thing, but would imp.load_source( name, relative_path ) help you at all? -- http://mail.python.org/mailman/listinfo/python-list
relative imports improve program organization... suggestions?
Alright, I have searched and searched and read many conversations on the topic of relative and absolute imports and am still not getting the whole thing through my skull. Highlights of what I've read: http://mail.python.org/pipermail/python-list/2007-January/422973.html http://groups.google.com/group/comp.lang.python/browse_thread/thread/b1e8dc93471a7079/8751c82cfe1ca3f2?lnk=gst&q=absolute+import#8751c82cfe1ca3f2 http://www.python.org/dev/peps/pep-0328/ http://docs.python.org/whatsnew/pep-328.html So my problem and argument: I want to create a package organized as the following: pckg/ __init__.py main.py moduleA/ __init__.py A_base.py A1/ __init__.py A_inherit1.py other_A1_files... A2/ __init__.py A_inherit2.py other_A2_files... moduleB/ ... Explanation: The main program is located in main.py and it implements the different modules (A, B). Within the modules the basic organization is; the base class for all different types of A is directly within the moduleA directory. All of the different inherited classes of A are within their own subdirectory with a mess of their own files. This is done so that a new subclass of A can be added/removed by just adding/ removing the subdirectory and each of these subclasses may have their own maintainer, but they should all inherit from A_base.py If I am developing the A1 directory, I want to be able to test A_inherit1.py by using 'if __name__ == "__main__"' within the A_inherit1.py file and by typing 'python A_inherit1.py' on the command line. I prefer this simply to keep all unit tests within the same directory and same file as the inherited class. My Problem: A_inherit1.py has the line: 'from ..A_base import A_Base_Class' so that I can later declare the inherited class as such: 'A1_Inherited_Class(A_Base_Class):' *BUT* I get the 'attempted relative import in non-package' error even when I try the 'from __future__ import absolute_import' command. I would prefer to be able to test the file without adding anything to the PYTHONPATH, like I said by using the name == main trick. So could someone explain to me what the rationale behind not allowing parent directory relative imports is? And possibly what I can do to get around it? (I really don't like messing with the sys.path for something like this) Thanks, Danny G -- http://mail.python.org/mailman/listinfo/python-list
Importing modules from packages with relative imports
With the old import system, and a package that looks like this: foo/ __init__.py main.py bar/ __init__.py baz.py If I wanted to delay importing baz until it was actually used, I could leave the __init__.py files empty and simply "import bar.baz". However, with the new relative imports syntax, "from . import bar.baz" causes a syntax error. I could use something like "from .bar import baz as bar_baz", but that's long, annoying to write, and requires changing all the uses of "bar.baz.spam" to "bar_baz.spam" through the file. Is there any way to achieve the "bar.baz" name with relative imports? -- http://mail.python.org/mailman/listinfo/python-list
Fun with relative imports and py3k
Since their introduction in Python 2.5 I only reviewed the new "relative import" notation briefly by reading the "What's new in Python 2.5" article. Now I wanted checkout if I get comfortable with them. Opening the tutorial I found following notice ( 6.4.2 ): "Note that both explicit and implicit relative imports are based on the name of the current module. Since the name of the main module is always "__main__", modules intended for use as the main module of a Python application should always use absolute imports." The distinction between modules and scripts was new to me. One of the primary features I always appreciated in Python was that modules had a dual purpose: they were components and they were programs. One could use them in isolation or as a part of a package something I always missed when I programmed in languages with the tyranny of a single main. However 'implicit relative imports' work as usual and Py3K just changes the priority of search - one might simply ignore the 'explicit' variant and care for unambiguous names. Now things are not so easy in life and when you are confronted with other peoples software you have to know at least the rules of the game. Digging a little deeper I found the following section in "What's new in Python 2.6" 'Python’s -m switch allows running a module as a script. When you ran a module that was located inside a package, relative imports didn’t work correctly.' Hmm... they worked as specified or at least as documented. I admit I'm confused but I suspect I'm not the only one. Now everything is fine. You just run each and every module with the -m option "as a script" because there is a chance that somewhere is a relative import ( e.g. local to a function ) and suddenly the module fails to import stuff due to a modality in the import behavior. Let's see how it works in practice. This here my very first attempt to start Pythons 2to3 refactoring tool written by the master himself. I looked at it and he used relative imports. C:\Python30\Lib\lib2to3>C:\Python30\python -m refactor.py Traceback (most recent call last): File "C:\Python30\lib\runpy.py", line 103, in _run_module_as_main loader, code, fname = _get_module_details(mod_name) File "C:\Python30\lib\runpy.py", line 78, in _get_module_details loader = get_loader(mod_name) File "C:\Python30\lib\pkgutil.py", line 456, in get_loader return find_loader(fullname) File "C:\Python30\lib\pkgutil.py", line 466, in find_loader for importer in iter_importers(fullname): File "C:\Python30\lib\pkgutil.py", line 422, in iter_importers __import__(pkg) File "C:\Python30\lib\lib2to3\refactor.py", line 23, in from .pgen2 import driver ValueError: Attempted relative import in non-package Not working. I'm perfectly sure I don't understand the error message but what's worse is that runpy.py fails which was just created ( in Python 2.6 ) to fix the issue with the apparently broken relative imports that is o.k. according to the introductory material that is dedicated to newbies. I'm trying to fix this naively by turning the explicit into implicit relative (or absolute?) imports and see what happens. refactor.py ... # Local imports from pgen2 import driver from pgen2 import tokenize import pytree import patcomp import fixes import pygram C:\Python30\Lib\lib2to3>C:\Python30\python -m refactor.py Traceback (most recent call last): File "C:\Python30\lib\runpy.py", line 103, in _run_module_as_main loader, code, fname = _get_module_details(mod_name) File "C:\Python30\lib\runpy.py", line 78, in _get_module_details loader = get_loader(mod_name) File "C:\Python30\lib\pkgutil.py", line 456, in get_loader return find_loader(fullname) File "C:\Python30\lib\pkgutil.py", line 466, in find_loader for importer in iter_importers(fullname): File "C:\Python30\lib\pkgutil.py", line 422, in iter_importers __import__(pkg) File "refactor.py", line 27, in import patcomp File "C:\Python30\lib\lib2to3\patcomp.py", line 17, in from .pgen2 import driver ValueError: Attempted relative import in non-package This time patcomp fails with its relative imports despite the fact that it is not imported as a "script" but as a module inside the lib2to3 package and it is not __main__ but refactor.py is. I give up. I always thought migration from Python 2.X to Python 3.0 was an issue not the conversion tool ;) -- http://mail.python.org/mailman/listinfo/python-list
Cyclic relative imports don't work
Hallöchen! Assume the follwing package structure: main.py package/ __init__.py [empty] moduleX.py moduleY.py main.py says: from package import moduleX moduleX.py says: from . import moduleY and moduleY.py says: from . import moduleX However, this doesn't work: [EMAIL PROTECTED]:~/temp/packages-test$ python main.py Traceback (most recent call last): File "main.py", line 1, in from package import moduleX File "/home/bronger/temp/packages-test/package/moduleX.py", line 1, in from . import moduleY File "/home/bronger/temp/packages-test/package/moduleY.py", line 1, in from . import moduleX ImportError: cannot import name moduleX If I turn the relative imports to absolute ones, it works. But I'd prefer the relative notation for intra-package imports. Why is this restriction? Tschö, Torsten. -- Torsten Bronger, aquisgrana, europa vetus Jabber ID: [EMAIL PROTECTED] (See http://ime.webhop.org for further contact info.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative Imports
Pat O'Hara wrote: > Hey guys, I know this is a really stupid question, but I've tried > googling and nothing came up. I also tried IRC, but it was too crowded > and I didn't get much useful information. > > I'm using Python 2.5 on WinXP, and I'm trying to do a relative import. > Here's the package structure > > A/ > __init__.py > aneededmodule.py > [some more modules] > B/ > __init__.py > anothermodule.py > > anothermodule.py needs to use aneededmodule.py; package A's __init__.py > looks like this: > > from aneededmodule import somestuff > > My problem is that when anothermodule tries to import ..aneededmodule or > ..somestuff (because somestuff was imported into __init__), I get a > ValueError: Attempted relative import in non-package. > > What's my problem? This seems like something very trivial, but I've > never had to use python for a project of this size before, so I've never > dealt with this. > > Thanks for your help, > -Pat My guess (without seeing your code or error messages; shame on you) is that you are running A/B/anothermodule.py; not -m A.B.anothermodule -- --Scott David Daniels [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Relative Imports
Hey guys, I know this is a really stupid question, but I've tried googling and nothing came up. I also tried IRC, but it was too crowded and I didn't get much useful information. I'm using Python 2.5 on WinXP, and I'm trying to do a relative import. Here's the package structure A/ __init__.py aneededmodule.py [some more modules] B/ __init__.py anothermodule.py anothermodule.py needs to use aneededmodule.py; package A's __init__.py looks like this: from aneededmodule import somestuff My problem is that when anothermodule tries to import ..aneededmodule or ..somestuff (because somestuff was imported into __init__), I get a ValueError: Attempted relative import in non-package. What's my problem? This seems like something very trivial, but I've never had to use python for a project of this size before, so I've never dealt with this. Thanks for your help, -Pat -- http://mail.python.org/mailman/listinfo/python-list
Re: Having trouble with relative imports
On 4/9/07, Echo <[EMAIL PROTECTED]> wrote: > Here is my setup: > rpg > -objects > --__init__.py > --gameobject.py > --material.py > -__init__.py > -run_tests.py > -stats.py > > the contents of run_test.py is: > import objects as o > > the contents of objects/__init__.py is: > from material import * > > in objects/material.py I have: > from .gameobject import GameObject > from ..stats import stats > > When I try to run run_tests.py, I get this traceback: > (1:30:59 PM) OshEcho: [EMAIL PROTECTED] ~/projects/rpg $ python run_tests.py > Traceback (most recent call last): > File "run_tests.py", line 4, in > import objects as o > File "/home/echo/projects/rpg/objects/__init__.py", line 3, in > from material import * > File "/home/echo/projects/rpg/objects/material.py", line 4, in > from ..stats import stats > ValueError: Attempted relative import beyond toplevel package > > > Could someone point out to me what I am doing wrong? > I'm running Python 2.5 on Gentoo > > -- > -Echo > Well, since I've gotten no answer and since that Python 2.5 doesn't play well with some programs on Gentoo, I will be switch back to 2.4 for now. -- -Echo -- http://mail.python.org/mailman/listinfo/python-list
Having trouble with relative imports
Here is my setup: rpg -objects --__init__.py --gameobject.py --material.py -__init__.py -run_tests.py -stats.py the contents of run_test.py is: import objects as o the contents of objects/__init__.py is: from material import * in objects/material.py I have: from .gameobject import GameObject from ..stats import stats When I try to run run_tests.py, I get this traceback: (1:30:59 PM) OshEcho: [EMAIL PROTECTED] ~/projects/rpg $ python run_tests.py Traceback (most recent call last): File "run_tests.py", line 4, in import objects as o File "/home/echo/projects/rpg/objects/__init__.py", line 3, in from material import * File "/home/echo/projects/rpg/objects/material.py", line 4, in from ..stats import stats ValueError: Attempted relative import beyond toplevel package Could someone point out to me what I am doing wrong? I'm running Python 2.5 on Gentoo -- -Echo -- http://mail.python.org/mailman/listinfo/python-list
Re: What is bad with "Relative imports"
In article <[EMAIL PROTECTED]>, Alexander Eisenhuth <[EMAIL PROTECTED]> wrote: > >PyLint says that "Relative imports" ... are worth to be warned . > >And I ask myself why? http://www.python.org/dev/peps/pep-0328 -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM -- http://mail.python.org/mailman/listinfo/python-list
What is bad with "Relative imports"
Hi, PyLint says that "Relative imports" ... are worth to be warned . And I ask myself why? - Example directory structure - Sound/ Top-level package __init__.py Initialize the sound package Utils/ Subpackage __init__.py iobuffer.py errors.py misc.py ... Formats/ Let's say in misc.py exist the class Player(). What could be bad to expose it in Utils.__init__() like: -- __init__.py - import misc # provoke PyLint warning Player = misc.Player ... with Sound.Utils.Player() ?? Thaks for your experience and comments Regards Alexander -- http://mail.python.org/mailman/listinfo/python-list
Re: help with relative imports
Robert Kern wrote: > Remember that this is code in the A.B.C module. Oh! That clears it all up! I wasn't realizing that the import statements were being executed from within the C module! Thanks! :) -- http://mail.python.org/mailman/listinfo/python-list
Re: help with relative imports
John Salerno wrote: > Robert Kern wrote: > >> What is ambiguous about A.B.D, A.E, and A.F.G? But if you like: > > I guess maybe I was looking at it backwards. From the way it was worded, > I thought the only information we had to use was the structure A.B.C, > and then given a statement like: > > from . import D > > we just had to figure out for ourselves that this results in A.B.D, > instead of, for example, A.C.D, or any other possibility. > > But I'm still a little confused about the use of the single or double > period. In this case: > > from . import D # Imports A.B.D > from .. import E# Imports A.E > > why do you need a single period in the first example, and a double in > the second, if they both are importing from A? If E is directly under A, > couldn't you just use a single period? And since D is nested twice > beneath A (i.e., in A, then in B), wouldn't you need two periods there > instead? Remember that this is code in the A.B.C module. The first form looks for modules in the A.B package, that is, next to A.B.C . The second looks for modules in the A package, next to A.B . -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: help with relative imports
John Salerno wrote: > Robert Kern wrote: > > > What is ambiguous about A.B.D, A.E, and A.F.G? But if you like: > > I guess maybe I was looking at it backwards. From the way it was worded, > I thought the only information we had to use was the structure A.B.C, > and then given a statement like: > > from . import D > > we just had to figure out for ourselves that this results in A.B.D, > instead of, for example, A.C.D, or any other possibility. > > But I'm still a little confused about the use of the single or double > period. In this case: > > from . import D # Imports A.B.D > from .. import E# Imports A.E > > why do you need a single period in the first example, and a double in > the second, if they both are importing from A? If E is directly under A, > couldn't you just use a single period? And since D is nested twice > beneath A (i.e., in A, then in B), wouldn't you need two periods there > instead? I was staring at this too for a bit. You just have to remember basic directory relative paths: . -> Current Directory .. -> One level up from current. At least I'm assuming this is right based on the test and the example above. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with relative imports
Robert Kern wrote: > What is ambiguous about A.B.D, A.E, and A.F.G? But if you like: I guess maybe I was looking at it backwards. From the way it was worded, I thought the only information we had to use was the structure A.B.C, and then given a statement like: from . import D we just had to figure out for ourselves that this results in A.B.D, instead of, for example, A.C.D, or any other possibility. But I'm still a little confused about the use of the single or double period. In this case: from . import D # Imports A.B.D from .. import E# Imports A.E why do you need a single period in the first example, and a double in the second, if they both are importing from A? If E is directly under A, couldn't you just use a single period? And since D is nested twice beneath A (i.e., in A, then in B), wouldn't you need two periods there instead? -- http://mail.python.org/mailman/listinfo/python-list
Re: help with relative imports
John Salerno wrote: > I'm reading the "What's New" section of the 2.5 docs, and I'm a little > confused by the last section of "Absolute and Relative Imports": > > --- > For example, code in the A.B.C module can do: > > from . import D # Imports A.B.D > from .. import E# Imports A.E > from ..F import G # Imports A.F.G > --- > > Can someone explain this? It seems like information is missing. How do > you know where D, E, F and G are to figure this out? If all you are > given is A.B.C, and then someone gives you the above three examples, > what steps do you take to figure out what gets imported from where? What is ambiguous about A.B.D, A.E, and A.F.G? But if you like: A/ B/ C.py D.py E.py F/ G.py -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
help with relative imports
I'm reading the "What's New" section of the 2.5 docs, and I'm a little confused by the last section of "Absolute and Relative Imports": --- For example, code in the A.B.C module can do: from . import D # Imports A.B.D from .. import E# Imports A.E from ..F import G # Imports A.F.G --- Can someone explain this? It seems like information is missing. How do you know where D, E, F and G are to figure this out? If all you are given is A.B.C, and then someone gives you the above three examples, what steps do you take to figure out what gets imported from where? Thanks, John -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports
Chris wrote: After reading that link I tried to change my imports like this: " from .myPythonFileInTheSameFolder import MyClass" This style of import is not yet implemented. I'm getting more and more confused... How can I correctly do a relative import ? I think your choices are - keep doing what you have been doing and ignore the warnings from PyLint - keep doing what you have been doing and turn off the warnings from PyLint - rewrite your imports to be absolute imports Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports
After reading that link I tried to change my imports like this: " from .myPythonFileInTheSameFolder import MyClass" Well, this caused an error in PyLint: Encountered "." at line 1, column 6. Was expecting one of: "or" ... "and" ... "not" ... "is" ... "in" ... "lambda" ... "if" ... "else" ... "elif" ... "while" ... "for" ... "try" ... "except" ... "def" ... "class" ... "finally" ... "print" ... "pass" ... "break" ... "continue" ... "return" ... "yield" ... "import" ... "from" ... "del" ... "raise" ... "global" ... "exec" ... "assert" ... "as" ... ... ID:E0001 invalid syntax I'm getting more and more confused... How can I correctly do a relative import ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports
Michael Hoffman wrote: Chris wrote: Why do relative imports cause warnings in PyLint? http://www.python.org/peps/pep-0328.html#rationale-for-absolute-imports I notice that this section says that from __future__ import absolute_import will be a feature of Python 2.4. Apparently it didn't make the cut. I've posted a bug report. Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Relative imports
Chris wrote: Why do relative imports cause warnings in PyLint? http://www.python.org/peps/pep-0328.html#rationale-for-absolute-imports -- Michael Hoffman -- http://mail.python.org/mailman/listinfo/python-list
Relative imports
Why do relative imports cause warnings in PyLint? A warning like this: ID:W0403 Relative import 'myPythonFileInTheSameFolder' When the import is like: from myPythonFileInTheSameFolder import MyClass -- http://mail.python.org/mailman/listinfo/python-list
Writing apps without using relative imports
I've noticed the push by Guido and others to use absolute imports instead of relative imports. I've always enjoyed the ease of relative imports, but am starting to understand that "explicit is better than implicitly" as the Python philosophy goes. I'm trying to develop a strategy for writing packages using only absolute imports and would really like to know what others have learned. Randall -- http://mail.python.org/mailman/listinfo/python-list
