Re: Q on naming nested packages/modules
On Tue, Sep 1, 2009 at 11:58 AM, kjno.em...@please.post wrote: I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below. The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs? Suppose I have a module (I'm not even sure this is the right word) called spam.ham, so I start out with the following file structure: spam/ |-- ham.py `-- __init__.py With this arrangement, the line import spam.ham ...in client code works as expected. But now suppose that I want to factor out some code in spam/ham.py to a helper module. (The reason behind factoring out this new module is to declutter spam/ham.py, and improve its readibility.) My instinct (from my Perl past) is to put this factored-out code in a file spam/ham/eggs.py, i.e. to create the nested module spam.ham.eggs, which requires expanding the tree as follows spam/ |-- ham/ | |-- eggs.py | `-- __init__.py |-- ham.py `-- __init__.py ...and adding the following spam/ham.py to # spam/ham.py from . import eggs This doesn't work so well, because now spam/ham.py is not read. It seems that adding the spam/ham directory, or maybe adding the file spam/ham/__init__.py, causes spam/ham.py to be overlooked. Clearly, I'm not playing this game right... What is considered best practice for the use case sketched above? Should I, e.g. rename the directory spam/ham something like spam/ham_ and refer to the helper module as spam.ham_.eggs? Or is some other convention preferred? I consulted PEP 8, but besides recommending short, all-lowercase names for modules, it gives little guidance on the situation described above. Take everything in ham.py and stick it in ham/__init__.py instead. This will give you the behavior you're looking for (don't import spam.ham.__init__. Everything in __init__.py is loaded into spam.ham) TIA! kynn -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
In h7jga8$ij...@reader1.panix.com kj no.em...@please.post writes: I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below. The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs? Following up my own post... From inspecting the directory structure of some of the standard Python modules I infer the following rules: 1. the source for leaf modules lives in files named after them (e.g. if x.y.z is a leaf module, its source code is in x/y/z.py) 2. the source for non-leaf modules lives in files named __init__.py (e.g. if x.y is a non-leaf module, its source code lives in the file x/y/__init__.py) In the examples above, the module x.y is a non-leaf module because there is a module x.y.z. I.e. the leaf-ness of a module depends solely on whether other modules deeper in the hierarchy are present. An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module Is the above correct? (BTW, to my Perl-pickled brain, step 2 above is the one that causes most distress... But I think I can cope.) kynn -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
kj wrote: In h7jga8$ij...@reader1.panix.com kj no.em...@please.post writes: I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below. The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs? Following up my own post... From inspecting the directory structure of some of the standard Python modules I infer the following rules: 1. the source for leaf modules lives in files named after them (e.g. if x.y.z is a leaf module, its source code is in x/y/z.py) 2. the source for non-leaf modules lives in files named __init__.py (e.g. if x.y is a non-leaf module, its source code lives in the file x/y/__init__.py) In the examples above, the module x.y is a non-leaf module because there is a module x.y.z. I.e. the leaf-ness of a module depends solely on whether other modules deeper in the hierarchy are present. An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module Is the above correct? (BTW, to my Perl-pickled brain, step 2 above is the one that causes most distress... But I think I can cope.) kynn Looking at the layout of the (most excellent!-) xlrd package, the bulk of the code is in the __init__.py file, and other supporting code is the same directory, accessed in __init__ with normal imports. I also am unclear on when it's best to have supporting files in the same directory versus a subdirectory... perhaps it is that flat is better than nested? ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
On Sep 1, 8:58 am, kj no.em...@please.post wrote: I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below. The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs? Suppose I have a module (I'm not even sure this is the right word) called spam.ham, so I start out with the following file structure: spam/ |-- ham.py `-- __init__.py With this arrangement, the line import spam.ham ...in client code works as expected. But now suppose that I want to factor out some code in spam/ham.py to a helper module. (The reason behind factoring out this new module is to declutter spam/ham.py, and improve its readibility.) My instinct (from my Perl past) is to put this factored-out code in a file spam/ham/eggs.py, i.e. to create the nested module spam.ham.eggs, which requires expanding the tree as follows spam/ |-- ham/ | |-- eggs.py | `-- __init__.py |-- ham.py `-- __init__.py ...and adding the following spam/ham.py to # spam/ham.py from . import eggs This doesn't work so well, because now spam/ham.py is not read. It seems that adding the spam/ham directory, or maybe adding the file spam/ham/__init__.py, causes spam/ham.py to be overlooked. There's a hint here. You can move the contents of ham.py into ham/__init__.py, and it'll work the way you want, maybe with only a minor change or two. Clearly, I'm not playing this game right... What is considered best practice for the use case sketched above? Should I, e.g. rename the directory spam/ham something like spam/ham_ and refer to the helper module as spam.ham_.eggs? Or is some other convention preferred? The way you were intending is often the approach people use. I doubt there are any detailed consensus recommendations about this in the Python community. I consulted PEP 8, but besides recommending short, all-lowercase names for modules, it gives little guidance on the situation described above. Of course it doesn't, PEP 8 is a style guide, not a project organization guide. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
kj wrote: But now suppose that I want to factor out some code in spam/ham.py to a helper module. (The reason behind factoring out this new module is to declutter spam/ham.py, and improve its readibility.) My instinct (from my Perl past) is to put this factored-out code in a file spam/ham/eggs.py, i.e. to create the nested module spam.ham.eggs, which requires expanding the tree as follows spam/ |-- ham/ | |-- eggs.py | `-- __init__.py |-- ham.py `-- __init__.py ...and adding the following spam/ham.py to # spam/ham.py from . import eggs This doesn't work so well, because now spam/ham.py is not read. It seems that adding the spam/ham directory, or maybe adding the file spam/ham/__init__.py, causes spam/ham.py to be overlooked. Clearly, I'm not playing this game right... One way is the __init__.py answer you have already. Another is to put eggs.py (or _eggs.py to indicate that it is private) in spam and skip the ham directory. spam/ __init__.py ham.py _eggs.py tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module With regard to point 2 -- would it be possible to just move z.py into x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same effect? Or is that not a good idea? On Tue, 01 Sep 2009 09:53:08 -0700, kj no.em...@please.post wrote: In h7jga8$ij...@reader1.panix.com kj no.em...@please.post writes: I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below. The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs? Following up my own post... From inspecting the directory structure of some of the standard Python modules I infer the following rules: 1. the source for leaf modules lives in files named after them (e.g. if x.y.z is a leaf module, its source code is in x/y/z.py) 2. the source for non-leaf modules lives in files named __init__.py (e.g. if x.y is a non-leaf module, its source code lives in the file x/y/__init__.py) In the examples above, the module x.y is a non-leaf module because there is a module x.y.z. I.e. the leaf-ness of a module depends solely on whether other modules deeper in the hierarchy are present. An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module Is the above correct? (BTW, to my Perl-pickled brain, step 2 above is the one that causes most distress... But I think I can cope.) kynn -- Rami Chowdhury Never attribute to malice that which can be attributed to stupidity -- Hanlon's Razor 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD) -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
In mailman.809.1251832870.2854.python-l...@python.org Rami Chowdhury rami.chowdh...@gmail.com writes: An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module With regard to point 2 -- would it be possible to just move z.py into x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same effect? Or is that not a good idea? I think in this case what you would need is something like 'from ..z import *', but this does not work either; one gets the error SyntaxError: 'import *' not allowed with 'from .'. A third solution would be to change step 2 above to this: 2. create the *symlink* x/y/z/__init__.py -- ../z.py This certainly would make things clearer-looking to me. Having all these files called the same thing, __init__.py, but having completely different meaning, makes my head hurt... In contrast, the __init__.py *symlinks* would be there just to keep python happy; the real content is where I'd expect it. I ran a simple test of this idea and it works, but there may be some subtle reasons for not doing it... Or maybe it's not a good idea simply because it is not the Python way, and I'd agree that this would be reason enough to avoid it. kynn -- http://mail.python.org/mailman/listinfo/python-list
Re: Q on naming nested packages/modules
An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously leaf-module x.y.z would become non-leaf. In other words, I'd have to: 1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module With regard to point 2 -- would it be possible to just move z.py into x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same effect? Or is that not a good idea? I think in this case what you would need is something like 'from ..z import *', but this does not work either; one gets the error SyntaxError: 'import *' not allowed with 'from .'. If you have a spam.ham module and you want to refactor it into a package, there's basically two common things to do as you've discovered. The first is to turn ham into a package: that entails creating a directory named ham, and moving ham.py to ham/__init__.py That works fine and is used by a number of people: I personally hate doing it, though. I don't like to put significant chunks of code into __init__.py, but instead just code which largely handles... namespace-issues or bootstrapping of a package. Partly for the reason you specified: it makes life a little annoying to have multiple files with completely different content have the same filename even if they aren't ambiguous due to directories. I personally MUCH prefer turning ham into a package, and moving the original ham.py to spam/ham/_ham.py. In spam/ham/__init__.py, I'd then do: from spam.ham._ham import * From a usage perspective, everything that used to be in spam.ham will still be there with all the contents you'd expect (the stuff in spam/ham/_ham.py)... and people can import spam.ham.eggs if they want or need to access those details. (I get around the above traceback you mention by just using an absolute import -- but I never use relative imports, anyways. They make my soul cry from a style and management and sanity perspective.) A third solution would be to change step 2 above to this: 2. create the *symlink* x/y/z/__init__.py -- ../z.py This is IMHO a very, very, very bad idea :) Yes, it'll work. But involving symlinks will make everything from distribution to revision control a bit more complicated (even if you're never intending any Windows users to use it or use a system like svn which handles symlinks adequately). It'll just create, in essence, two separate files on your system with the same content/code. Sure they're not really two separate files, but as far as Python knows they are. It just seems like it's asking for trouble :) --S -- http://mail.python.org/mailman/listinfo/python-list
