Re: How to install your personal module/package on site.
Op 21/08/20 om 01:49 schreef Cameron Simpson: On 16Aug2020 08:32, Marco Sulla wrote: Sorry, didn't read well, Apart the other suggestion, you (or your sysop) can create a private Pypi: https://pypi.org/project/private-pypi/ Even simpler, you can put a code repo path into your requirements.txt (or directly with pip via its "-e" option). Example from a project requirements.txt file: -e git+https://github.com/SpectraLogic/ds3_python3_sdk.git@v5.0.3#egg=ds3-sdk So if you've an work internal service for your revision control you can pull directly from that with pip. Note that the above example pulls a particular tagged release via the "@5.0.3" suffix. That is a nice feature. I will have a closer look at this. -- Antoon Pardon. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
On 16Aug2020 08:32, Marco Sulla wrote: >Sorry, didn't read well, Apart the other suggestion, you (or your >sysop) can create a private Pypi: >https://pypi.org/project/private-pypi/ Even simpler, you can put a code repo path into your requirements.txt (or directly with pip via its "-e" option). Example from a project requirements.txt file: -e git+https://github.com/SpectraLogic/ds3_python3_sdk.git@v5.0.3#egg=ds3-sdk So if you've an work internal service for your revision control you can pull directly from that with pip. Note that the above example pulls a particular tagged release via the "@5.0.3" suffix. Cheers, Cameron Simpson (formerly c...@zip.com.au) -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
Sorry, didn't read well, Apart the other suggestion, you (or your sysop) can create a private Pypi: https://pypi.org/project/private-pypi/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
Does the word "public" mean world-wide, or perhaps only amongst your work-colleagues? Only among work-colleagues. We only want that anyone writing and running python scripts on particular hosts, can easily import these modules/packages. Of possible interest:- Private Python package management https://stackoverflow.com/questions/63320653/private-python-package-management -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
Op 15/08/20 om 07:33 schreef dn via Python-list: > On 14/08/2020 22:32, Antoon Pardon wrote: >> Well the question is in the subject. >> >> I have a number of modules/packages which were until recently >> personal use only. However python is getting more popular >> at work and some of my work was considered useful enough to >> install in a public available spot. >> >> How should I approach this? > > > Does the word "public" mean world-wide, or perhaps only amongst your > work-colleagues? Only among work-colleagues. We only want that anyone writing and running python scripts on particular hosts, can easily import these modules/packages. -- Antoon -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
On 14/08/2020 22:32, Antoon Pardon wrote: Well the question is in the subject. I have a number of modules/packages which were until recently personal use only. However python is getting more popular at work and some of my work was considered useful enough to install in a public available spot. How should I approach this? Does the word "public" mean world-wide, or perhaps only amongst your work-colleagues? -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install your personal module/package on site.
https://www.google.com/search?channel=fs&client=ubuntu&q=publish+python+code First result. -- https://mail.python.org/mailman/listinfo/python-list
How to install your personal module/package on site.
Well the question is in the subject. I have a number of modules/packages which were until recently personal use only. However python is getting more popular at work and some of my work was considered useful enough to install in a public available spot. How should I approach this? -- Antoon. -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
On Tuesday, 8 May 2018 13:05:58 UTC+5:30, Steven D'Aprano wrote: > On Mon, 07 May 2018 09:53:45 -0700, Sharan Basappa wrote: > > > I am a bit confused between module and package in Python. Does a module > > contain package or vice versa? When we import something in Python, do we > > import a module or a package? > > The term "module" in Python has multiple meanings: > > - a particular kind of object, types.ModuleType > > - a single importable .py, .pyc etc file > > A package is a logical collection of importable .py etc files, usually > collected inside a single directory. When you import a module of a > package, that gives you a module object. > > Normally we would say that packages contain modules. For example, if you > have this file structure: > > > library/ > +-- __init__.py # special file which defines a package > +-- widgets.py > +-- stuff/ > +-- __init__.py > +-- things.py > > > then we have a package "library", which in turn contains a submodule > "library.widgets", and a subpackage "library.stuff", which in turn > contains a submodule "library.stuff.things". > > Each of these lines imports a module object: > > import library > import library.stuff > import library.stuff.things > import library.widgets > > from library import widgets > from library.stuff import things > > > Effectively, "packages" relates to how you arrange the files on disk; > "modules" relates to what happens when you import them. > > > -- > Steve Wow! Thanks a lot. -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
On Mon, 07 May 2018 09:53:45 -0700, Sharan Basappa wrote: > I am a bit confused between module and package in Python. Does a module > contain package or vice versa? When we import something in Python, do we > import a module or a package? The term "module" in Python has multiple meanings: - a particular kind of object, types.ModuleType - a single importable .py, .pyc etc file A package is a logical collection of importable .py etc files, usually collected inside a single directory. When you import a module of a package, that gives you a module object. Normally we would say that packages contain modules. For example, if you have this file structure: library/ +-- __init__.py # special file which defines a package +-- widgets.py +-- stuff/ +-- __init__.py +-- things.py then we have a package "library", which in turn contains a submodule "library.widgets", and a subpackage "library.stuff", which in turn contains a submodule "library.stuff.things". Each of these lines imports a module object: import library import library.stuff import library.stuff.things import library.widgets from library import widgets from library.stuff import things Effectively, "packages" relates to how you arrange the files on disk; "modules" relates to what happens when you import them. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
Sharan Basappa writes: > One question. So, we can import the entire package or just a module in > a given package. Is this correct? Each time you ‘import foo’, you are getting a module. > For example, > import nltk That results in a module object, and you can use the name ‘nltk’ to reference that module. > import nltk.stem That results in a different module object, and you can use the name ‘nltk.stem’ (which is the name ‘stem’ in the namespace ‘nltk’) to reference that module. See the Python documentation for a good description of the import system https://docs.python.org/3/reference/import.html>. -- \ “If we could change ourselves, the tendencies in the world | `\ would also change.” —Mohandas K. Gandhi, _Collected Works_, 1913 | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
MRAB, ChirisA, One question. So, we can import the entire package or just a module in a given package. Is this correct? For example, import nltk import nltk.stem -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
On Monday, 7 May 2018 23:09:41 UTC+5:30, Chris Angelico wrote: > On Tue, May 8, 2018 at 2:53 AM, Sharan Basappa > wrote: > > I am a bit confused between module and package in Python. > > Does a module contain package or vice versa? > > When we import something in Python, do we import a module or a package? > > You import a module. > > A package is one particular form of module, which is built out of > other modules (usually a directory full of .py files). There are a > number of different sorts of modules; regardless, you 'import X' and > get module X. > > ChrisA Thank you very much. Much appreiated -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
On Tue, May 8, 2018 at 2:53 AM, Sharan Basappa wrote: > I am a bit confused between module and package in Python. > Does a module contain package or vice versa? > When we import something in Python, do we import a module or a package? You import a module. A package is one particular form of module, which is built out of other modules (usually a directory full of .py files). There are a number of different sorts of modules; regardless, you 'import X' and get module X. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Module, Package
On 2018-05-07 17:53, Sharan Basappa wrote: I am a bit confused between module and package in Python. Does a module contain package or vice versa? When we import something in Python, do we import a module or a package? A module is a file. A package is a collection of one or more modules. -- https://mail.python.org/mailman/listinfo/python-list
Module, Package
I am a bit confused between module and package in Python. Does a module contain package or vice versa? When we import something in Python, do we import a module or a package? Thanks -- https://mail.python.org/mailman/listinfo/python-list
Tools/libraries to determine the call graph(call flow) of an python program (module/package)
Hi Folks, I am trying to do the following. I have a moderately complex python module/application X, whose source code i have access to. I run X with the following command python x.py ... Internally, x.py callls y.py, which in turn calls z.py, etc etc x.py ---> y.py ---> z.py ---> u.py ---> v.py Is there a python library/tool/module , to which i give input the start point of X, x.py and the input arguments, arg1, arg2, ..., argn and which can come up with the call graph of X I have tried looking at pycallgraph[0], but havent had much luck with it. 0. https://pypi.python.org/pypi/pycallgraph Any suggestions,advice, pointers welcome. Thanks a ton, ashish "Talk is cheap. Show me the code." - Linus Torvalds [ https://lkml.org/lkml/2000/8/25/132 ] -- https://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
Big Stu writes: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates [good adherence to style and > practice conventions] I find the code base for Bazaar to be particularly good quality https://code.launchpad.net/~bzr-pqm/bzr/bzr.dev>: $ sudo aptitude install bzr $ bzr branch lp:bzr $ cd bzr/ $ # browse browse browse -- \ “Probably the earliest flyswatters were nothing more than some | `\sort of striking surface attached to the end of a long stick.” | _o__) —Jack Handey | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
Josh Holland writes: > On 2010-01-28, exar...@twistedmatrix.com wrote: > > Have you actually looked at any of the standard library? > Not recently or in depth, no. I would have thought that it would be of > high quality. I must have been mistaken. Style conventions were introduced relatively late in the history of Python (PEP 7, PEP 8, and PEP 257 were created in 2001). The existing standard library code works as-is, so no particular effort has gone into cleaning it up to retroactively conform. Also, existing APIs in the standard library tend to be preserved as non-conformant (I'm looking at you, ‘logging’ and ‘unittest’) rather than breaking existing code by changing the API. The end result is that there are huge swaths of the standard library that do not adhere to the style conventions. It could even be argued that part of the reason for introducing the official conventions was to limit the scale of the damage already done. -- \ “Facts do not cease to exist because they are ignored.” —Aldous | `\Huxley | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On 07:49 pm, stu.dohe...@gmail.com wrote: Have you actually looked at any of the standard library? Jean-Paul I'm looking at urllib2 right now and it is covering a bunch of the bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder for import statements on various things I'm interested in is bringing up some good examples to check out as well. Given that I'm still fairly novice to this I'm not yet in the position to make a good judgment on what is and isn't a good python practice so I was hoping someone on here might be able to point at a module or 2 that has really done a good job of following best practices. Seems like a reasonable question with an answer that others in a similar position to me might find useful. You're right. I was actually wondering if Josh had looked before suggesting it. :) The stdlib varies wildly in quality, with much of it not serving as a particular good example of most of the points you mentioned. urllib2 is probably better than a lot, but, for example, even it only manages about 75% line coverage by its test suite. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
> Have you actually looked at any of the standard library? > > Jean-Paul I'm looking at urllib2 right now and it is covering a bunch of the bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder for import statements on various things I'm interested in is bringing up some good examples to check out as well. Given that I'm still fairly novice to this I'm not yet in the position to make a good judgment on what is and isn't a good python practice so I was hoping someone on here might be able to point at a module or 2 that has really done a good job of following best practices. Seems like a reasonable question with an answer that others in a similar position to me might find useful. -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On 2010-01-28, exar...@twistedmatrix.com wrote: > Have you actually looked at any of the standard library? Not recently or in depth, no. I would have thought that it would be of high quality. I must have been mistaken. -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On 28 ene, 19:17, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates: > > -Strict adherence to PEP 8 > -thorough use of Docstrings > -Conventional directory structure/package layout > -Appropriate use of the latest accepted coding guidelines in the > python community (e.g., new classes versus old classes, Python 3000 > compatibility, newer language features, etc. etc.) > -Some amount of object oriented design > > Bonus: > -Unit tests > -Logging mechanism > > I can't imagine a package that's been around longer than a few years > will hit upon all these things well unless the maintainer went back > and did some serious refactoring and re-tooling. > > Is this question possible to answer? Look here: http://bitbucket.org/ares/scripy/src/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On Jan 28, 2:28 pm, Josh Holland wrote: > On 2010-01-28, Big Stu wrote: > > > I'm hoping someone on here can point me to an example of a python > > package that is a great example of how to put it all together. I'm > > hoping for example code that demonstrates: > > Surely most of the Standard Library should satisfy all your > requirements? > > -- > Josh "dutchie" Holland > http://www.joshh.co.uk/http://twitter.com/jshhollandhttp://identi.ca/jshholland That's definitely a place I've started to poke around, but the standard library stuff always comes to me by way of my standard python installation. I was hoping to have a template of a 3rd party package to follow. Complete with conventions to follow for easily packaging and distributing via the usual python channels (pypi, easy_install, egg, etc.). -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On 07:28 pm, j...@joshh.co.uk wrote: On 2010-01-28, Big Stu wrote: I'm hoping someone on here can point me to an example of a python package that is a great example of how to put it all together. I'm hoping for example code that demonstrates: Surely most of the Standard Library should satisfy all your requirements? Have you actually looked at any of the standard library? Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Great example of a python module/package following up to date conventions.
On 2010-01-28, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates: Surely most of the Standard Library should satisfy all your requirements? -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland -- http://mail.python.org/mailman/listinfo/python-list
Great example of a python module/package following up to date conventions.
I'm hoping someone on here can point me to an example of a python package that is a great example of how to put it all together. I'm hoping for example code that demonstrates: -Strict adherence to PEP 8 -thorough use of Docstrings -Conventional directory structure/package layout -Appropriate use of the latest accepted coding guidelines in the python community (e.g., new classes versus old classes, Python 3000 compatibility, newer language features, etc. etc.) -Some amount of object oriented design Bonus: -Unit tests -Logging mechanism I can't imagine a package that's been around longer than a few years will hit upon all these things well unless the maintainer went back and did some serious refactoring and re-tooling. Is this question possible to answer? -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Jan 31, 12:27 am, Gabriel Genellina <[EMAIL PROTECTED]> wrote: > Python stores filename and line number information in code objects > (only). If you have a reference to any code object (a method, a > function, a traceback...) inspect can use it to retrieve that > information. Aside from general concerns about increasing the size of class objects (and most programs don't contain enough of those to make a big difference) I don't immediately see anything that would prevent the interpreter being modified to add file and line number information to class objects as well. While the information wouldn't always be present (types implemented in C, types created by calling the metaclass constructor directly), it would help address the inspect module bugs Steven illustrated. I would agree with Carl that modifying __module__ in the way he suggests is legitimate - if it breaks the inspect module, then it is the inspect module that needs fixing (and/or better support from the interpreter to help find the real source code). Cheers, Nick. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
Peter Schuller <[EMAIL PROTECTED]> writes: > I *DON'T* want anything to depend on the physical location on disk. Importing the code in the first place will — unavoidably, it seems to me — depend on the file location from which to load the module. After that, nothing depends on the physical location on disk, unless it's buggy. Imported modules are available from 'sys.modules', and that's where subsequent 'import' statements will find them, with no reference to file locations. > That was exactly what I was after from the beginning; a total > separation of location on disk from the location in the module > hiearachy. As you say, the location of the source should be an > implementation detail. That is exactly what I am after. It *is* an implementation detail, once the module is loaded from disk. -- \ "Philosophy is questions that may never be answered. Religion | `\ is answers that may never be questioned." —anonymous | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
> It what sense will it not be? Why do you care so much about where the > source code for Monkey is defined? If you actually want to read the > source, you might need to follow the chain from "animal", see that Monkey > is imported from "monkey", and go look at that. But the rest of the time, > why would you care? > > There is a very good reason to care *in practice*: if there is code out > there that assumes that the source code from Monkey is in the file it was > found in. In practice, you might be stuck needing to work around that. > But that's not a good reason to care *in principle*. In principle, the > actual location of the source code should be an implementation detail of > which we care nothing. It's possible that the source for Monkey doesn't Exactly. I *DON'T* want anything to depend on the physical location on disk. That was exactly what I was after from the beginning; a total separation of location on disk from the location in the module hiearachy. As you say, the location of the source should be an implementation detail. That is exactly what I am after. I'll have a closer look at the suggested practice of modifying __module__. For this particular use case we probably won't end up doing that, but it may come to be useful in the future. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
> Well, all I will say is that many people on this list, myself > included, do know Python internals, and we use the method we've been > suggesting here, without problems. Ok. That is useful to know (that it is being done in practice without problems). Thanks! -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Jan 30, 4:31 am, Peter Schuller <[EMAIL PROTECTED]> wrote: > I don't know Python internals enough to state of believe with any > authority wither, let's say, stomping __module__ and hacking > sys.modules would be enough to *truly* do it correctly in a proper way > such that it is entirely transparent. This is why I care about whether > it truly changes the real identity of the class; it's not about > satisfying my particular list of examples (because they *were* just > examples). Well, all I will say is that many people on this list, myself included, do know Python internals, and we use the method we've been suggesting here, without problems. I think you're slipping to a level of paranoia that's more harmful that helpful now. The ironic thing is, breaking the one-to-one module-to-file relationship is more likely to have "unintended consequences", by a very large margin. Python has always been one-to-one module-to-file (excepting modules built into the interpretter), and many codes and tools have come to depend on it. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On 30 ene, 12:00, Steven D'Aprano <[EMAIL PROTECTED] cybersource.com.au> wrote: > I call that a bug in the inspect module. In fact, looking at the source > for the findsource() function, I can see no fewer than two bugs, just in > the way it handles classes: > > (1) it assumes that the only way to create a class is with a class > statement, which is wrong; and > > (2) it assumes that the first occurrence of "class " must be the > correct definition, which is also wrong. Yes, it's broken. But I'm afraid that's the only available thing to do. Python stores filename and line number information in code objects (only). If you have a reference to any code object (a method, a function, a traceback...) inspect can use it to retrieve that information. Once a class is defined, there is no code object attached to it. (The class statement is executed when the module is loaded and initialized, but that code object is discarded afterwards because it's not required anymore). If you *know* that a certain method is defined in a class, you can use it to find the real module. But in general, there is nothing to start with. I'm eagerly waiting for someone to come and say I'm wrong... -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Tue, 29 Jan 2008 06:48:59 -0600, Peter Schuller wrote: >> You can also put, in animal/__init__.py: >> from monkey import Monkey >> and now you can refer to it as org.lib.animal.Monkey, but keep the >> implementation of Monkey class and all related stuff into >> .../animal/monkey.py > > The problem is that we are now back to the identity problem. The class > won't actually *BE* org.lib.animal.Monkey. It what sense will it not be? Why do you care so much about where the source code for Monkey is defined? If you actually want to read the source, you might need to follow the chain from "animal", see that Monkey is imported from "monkey", and go look at that. But the rest of the time, why would you care? There is a very good reason to care *in practice*: if there is code out there that assumes that the source code from Monkey is in the file it was found in. In practice, you might be stuck needing to work around that. But that's not a good reason to care *in principle*. In principle, the actual location of the source code should be an implementation detail of which we care nothing. It's possible that the source for Monkey doesn't exist *anywhere*. It is important to deal with buggy tools. But the correct way to do so is to fix the bugs, not to throw away perfectly good abstractions. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Tue, 29 Jan 2008 13:44:33 -0600, Robert Kern wrote: > Carl Banks wrote: >> On Jan 29, 7:48 am, Peter Schuller <[EMAIL PROTECTED]> wrote: You can also put, in animal/__init__.py: from monkey import Monkey and now you can refer to it as org.lib.animal.Monkey, but keep the implementation of Monkey class and all related stuff into .../animal/monkey.py >>> The problem is that we are now back to the identity problem. The class >>> won't actually *BE* org.lib.animal.Monkey. >> >> The usage is the same; it works in all cases once you redefine >> __module__. Who cares what it really is? > > The inspect module. [snip example] I call that a bug in the inspect module. In fact, looking at the source for the findsource() function, I can see no fewer than two bugs, just in the way it handles classes: (1) it assumes that the only way to create a class is with a class statement, which is wrong; and (2) it assumes that the first occurrence of "class " must be the correct definition, which is also wrong. It isn't hard to break the inspect module. Here's an example: >>> import broken >>> import inspect >>> lines, lineno = inspect.findsource(broken.Parrot) >>> lines[lineno] 'class Parrot which will be defined later.\n' >>> >>> lines, lineno = inspect.findsource(broken.Wensleydale) >>> lines[lineno] 'class Wensleydale: # THIS IS GONE\n' Here's the source of broken.py: $ cat broken.py """Here is a doc string, where I happen to discuss the class Parrot which will be defined later. """ class Parrot: pass class Wensleydale: # THIS IS GONE pass del Wensleydale class Wensleydale(object): # but this exists pass It isn't often that I would come right out and say that part of the Python standard library is buggy, but this is one of those cases. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
>> The problem is that we are now back to the identity problem. The class >> won't actually *BE* org.lib.animal.Monkey. > > The usage is the same; it works in all cases once you redefine > __module__. Who cares what it really is? The cases I listed were just examples. My point was that I wanted it to *be* the right class, to avoid unintended consequences. If I knew what all those possible consequences were, there would not be a problem to begin with. The other follow-up to your E-Mail points out a possible problem for example. I would not have come up with that, but that does not mean the effect does not exist. And committing to a solution that "seems to work", only to break massively for some particular use case in the future, is exactly why I don't want a "hack" for a solution. I don't know Python internals enough to state of believe with any authority wither, let's say, stomping __module__ and hacking sys.modules would be enough to *truly* do it correctly in a proper way such that it is entirely transparent. This is why I care about whether it truly changes the real identity of the class; it's not about satisfying my particular list of examples (because they *were* just examples). > Whatever. ISTM you came here looking for a particular means and not a > particular end. My particular preferred end is to be able to separate file hierarchy from module hierarchy without causing unforseen consequences. This was the stated goal all along. > Python already has the power to meet your stated > needs, but you won't use that solution because it's "hacky". > Apparently all you really wanted was the loosened file structure in > the first place. Yes, or failing that an alternative that mitigates the problem. And it *is* hacky, in my opinion, if things break as a result of it (such as the other poster's inspect example). -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
Carl Banks wrote:
> On Jan 29, 7:48 am, Peter Schuller <[EMAIL PROTECTED]>
> wrote:
>>> You can also put, in animal/__init__.py:
>>> from monkey import Monkey
>>> and now you can refer to it as org.lib.animal.Monkey, but keep the
>>> implementation of Monkey class and all related stuff into
>>> .../animal/monkey.py
>> The problem is that we are now back to the identity problem. The class
>> won't actually *BE* org.lib.animal.Monkey.
>
> The usage is the same; it works in all cases once you redefine
> __module__. Who cares what it really is?
The inspect module.
[animals]$ ls
animals
[animals]$ rm animals/*.pyc
[animals]$ ls
animals
[animals]$ ls animals
__init__.py monkey.py
[animals]$ cat animals/monkey.py
class Monkey(object):
pass
[animals]$ cat animals/__init__.py
from animals.monkey import Monkey
Monkey.__module__ = 'animals'
[animals]$ python
Python 2.5.1 (r251:54869, Apr 18 2007, 22:08:04)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from animals import Monkey
>>> import inspect
>>> inspect.getsource(Monkey)
Traceback (most recent call last):
File "", line 1, in
File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/inspect.py",
line 629, in getsource
lines, lnum = getsourcelines(object)
File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/inspect.py",
line 618, in getsourcelines
lines, lnum = findsource(object)
File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/inspect.py",
line 494, in findsource
raise IOError('could not find class definition')
IOError: could not find class definition
>>>
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Jan 29, 7:48 am, Peter Schuller <[EMAIL PROTECTED]> wrote: > > You can also put, in animal/__init__.py: > > from monkey import Monkey > > and now you can refer to it as org.lib.animal.Monkey, but keep the > > implementation of Monkey class and all related stuff into > > .../animal/monkey.py > > The problem is that we are now back to the identity problem. The class > won't actually *BE* org.lib.animal.Monkey. The usage is the same; it works in all cases once you redefine __module__. Who cares what it really is? > Perhaps manipulating > __module__ is enough; perhaps not (for example, what about > sys.modules?). It's enough. It satisfies the criteria you listed. sys.modules has nothing to do with it. Monkey is a class, not a module. If you set __module__, the only remaining discernable difference is that the global variables accessed from the Monkey class will be in org.lib.animal.monkey instead of org.lib.animal. This has no ill effects when unpickling or instantiating the class from org.lib.animal. > Looks like I'll just live with putting more than I > would like in the same file. Whatever. ISTM you came here looking for a particular means and not a particular end. Python already has the power to meet your stated needs, but you won't use that solution because it's "hacky". Apparently all you really wanted was the loosened file structure in the first place. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
> You can reassign the class's module: > > from org.lib.animal.monkey import Monkey > Monkey.__module__ = 'org.lib.animal' > > > (Which, I must admit, is not a bad idea in some cases.) Is there a sense whether this is truly a supported way of doing this, in terms of not running into various unintended side-effects? One example would be sys.modules that I mentioned in the previous post. Another, possibly related, might be interaction with the import keyword and its implementation. I will probably have to read up more on the semantics of __import__ and related machinery. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
> You can also put, in animal/__init__.py: > from monkey import Monkey > and now you can refer to it as org.lib.animal.Monkey, but keep the > implementation of Monkey class and all related stuff into > .../animal/monkey.py The problem is that we are now back to the identity problem. The class won't actually *BE* org.lib.animal.Monkey. Perhaps manipulating __module__ is enough; perhaps not (for example, what about sys.modules?). Looks like I'll just live with putting more than I would like in the same file. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
Carl Banks <[EMAIL PROTECTED]> writes: > On Jan 25, 6:45 pm, Ben Finney <[EMAIL PROTECTED]> > wrote: > > "Gabriel Genellina" <[EMAIL PROTECTED]> writes: > > > You can also put, in animal/__init__.py: > > > from monkey import Monkey > > > and now you can refer to it as org.lib.animal.Monkey, but keep the > > > implementation of Monkey class and all related stuff into > > > .../animal/monkey.py > > > > This (as far as I can understand) is exactly the solution the > > original poster desired to "shoot down", for reasons I still don't > > understand. > > The solution is to modify the class's __module__ attribute as well as > importing it, as I've already pointed out: > > from org.lib.animal.monkey import Monkey > Monkey.__module__ = 'org.lib.animal' Thanks, that makes it clear. > This should be enough to satisfy the OP's requirements, at least for > classes, without softening the one-to-one module-to-file > relationship, or using "hacks". > > In fact, I'd say this is good practice. I've not seen that before, but it seems an elegant way to address what the OP is asking for. -- \"Madness is rare in individuals, but in groups, parties, | `\ nations and ages it is the rule." -- Friedrich Nietzsche | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Jan 25, 6:45 pm, Ben Finney <[EMAIL PROTECTED]> wrote: > "Gabriel Genellina" <[EMAIL PROTECTED]> writes: > > You can also put, in animal/__init__.py: > > from monkey import Monkey > > and now you can refer to it as org.lib.animal.Monkey, but keep the > > implementation of Monkey class and all related stuff into > > .../animal/monkey.py > > This (as far as I can understand) is exactly the solution the original > poster desired to "shoot down", for reasons I still don't understand. Come on, the OP explained it quite clearly in his original post. Did you guys even read it? The module where org.lib.animal.Monkey is actually defined should be an implementation detail of the library, but simply importing Monkey into org.lib.animal doesn't quite make it one. If a user pickles a Monkey class, and then the OP decides to refactor the Monkey class into a new module (say org.lib.animal.primate.monkey), then the user would not be able to unpickle it. Because, you see, pickles record the module a class is defined in. So, now the user has to worry about where Monkey is actually defined. It is not an implementation detail. The solution is to modify the class's __module__ attribute as well as importing it, as I've already pointed out: from org.lib.animal.monkey import Monkey Monkey.__module__ = 'org.lib.animal' This should be enough to satisfy the OP's requirements, at least for classes, without softening the one-to-one module-to-file relationship, or using "hacks". In fact, I'd say this is good practice. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
"Gabriel Genellina" <[EMAIL PROTECTED]> writes: > You can also put, in animal/__init__.py: > from monkey import Monkey > and now you can refer to it as org.lib.animal.Monkey, but keep the > implementation of Monkey class and all related stuff into > .../animal/monkey.py This (as far as I can understand) is exactly the solution the original poster desired to "shoot down", for reasons I still don't understand. -- \ "Reichel's Law: A body on vacation tends to remain on vacation | `\ unless acted upon by an outside force." -- Carol Reichel | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
En Thu, 24 Jan 2008 11:57:49 -0200, Peter Schuller <[EMAIL PROTECTED]> escribió: > In this case my > problem is more related to the "file == module" and "directory == > module" semantics, since I want to break contents in a single module > out into several files. You already can do that, just import the public interfase of those several files onto the desired container module. See below for an example. >> Isn't org.lib.animal a package, reflected as a directory on disk? That's >> the same both for Java and Python. Monkey.py and Tiger.py would be >> modules >> inside that directory, just like Monkey.java and Tiger.java. Aren't the >> same thing? > > No, because in Java Monkey.java is a class. So we have class Monkey in > package org.lib.animal. In Python we would have class Monkey in module > org.lib.animal.monkey, which is redundant and does not reflect the > intended hierarchy. I have to either live with this, or put Monkey in > .../animal/__init__.py. Neither option is what I would want, ideally. You can also put, in animal/__init__.py: from monkey import Monkey and now you can refer to it as org.lib.animal.Monkey, but keep the implementation of Monkey class and all related stuff into .../animal/monkey.py -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
>> Not necessarily. In part it is the name, in that __name__ will be >> different. But to the extent that calling code can potentially import >> them under differents names, it's identity. Because importing the same >> module under two names results in two distinct modules (two distinct >> module objects) that have no realation with each other. So for >> example, if a module has a single global protected by a mutex, there >> are suddenly two copies of that. In short: identity matters. > > That's not true. It doesn't matter if you Import a module several times > at different places and with different names, it's always the same module > object. Sorry, this is all my stupidity. I was being daft. When I said importing under different names, I meant exactly that. As in, applying hacks to import a module under a different name by doing it relative to a different root directory. This is however not what anyone is suggesting in this discussion. I got my wires crossed. I fully understand that "import x.y.z" or "import x.y.z as B", and so one do not affect the identity of the module. > Ok, there is one exception: the main script is loaded as __main__, but if > you import it using its own file name, you get a duplicate module. > You could confuse Python adding a package root to sys.path and doing > imports from inside that package and from the outside with different > names, but... just don't do that! Right :) > I don't really understand what your problem is exactly, but I think you > don't require any __import__ magic or arcane hacks. Perhaps the __path__ > package attribute may be useful to you. You can add arbitrary directories > to this list, which are searched for submodules of the package. This way > you can (partially) decouple the file structure from the logical package > structure. But I don't think it's a good thing... That sounds useful if I want to essentially put the contents of a directory somewhere else, without using a symlink. In this case my problem is more related to the "file == module" and "directory == module" semantics, since I want to break contents in a single module out into several files. > Isn't org.lib.animal a package, reflected as a directory on disk? That's > the same both for Java and Python. Monkey.py and Tiger.py would be modules > inside that directory, just like Monkey.java and Tiger.java. Aren't the > same thing? No, because in Java Monkey.java is a class. So we have class Monkey in package org.lib.animal. In Python we would have class Monkey in module org.lib.animal.monkey, which is redundant and does not reflect the intended hierarchy. I have to either live with this, or put Monkey in .../animal/__init__.py. Neither option is what I would want, ideally. Java does still suffer from the same problem since it forces "class == file" (well, "public class == file"). However it is less of a problem since you tend to want to keep a single class in a single file, while I have a lot more incentive to split up a module into different files (because you may have a lot of code hiding behind the public interface of a module). So essentially, Java and Python have the same problem, but certain aspects of Java happens to mitigate the effects of it. Languages like Ruby do not have the problem at all, because the relationship between files and modules is non-existent. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
En Thu, 24 Jan 2008 05:16:51 -0200, Peter Schuller <[EMAIL PROTECTED]> escribió: >>> I do *not* want to simply break out X into org.lib.animal.x, and >>> have org.lib.animal import org.lib.animal.x.X as X. >> >>> While this naively solves the problem of being able to refer to X as >>> org.lib.animal.X, the solution is anything but consistent because >>> the *identity* of X is still org.lib.animal.x.X. >> >> The term "identity" in Python means something separate from this >> concept; you seem to mean "the name of X". > > Not necessarily. In part it is the name, in that __name__ will be > different. But to the extent that calling code can potentially import > them under differents names, it's identity. Because importing the same > module under two names results in two distinct modules (two distinct > module objects) that have no realation with each other. So for > example, if a module has a single global protected by a mutex, there > are suddenly two copies of that. In short: identity matters. That's not true. It doesn't matter if you Import a module several times at different places and with different names, it's always the same module object. py> from xml.etree import ElementTree py> import xml.etree.ElementTree as ET2 py> import xml.etree py> ET3 = getattr(xml.etree, 'ElementTree') py> ElementTree is ET2 True py> ET2 is ET3 True Ok, there is one exception: the main script is loaded as __main__, but if you import it using its own file name, you get a duplicate module. You could confuse Python adding a package root to sys.path and doing imports from inside that package and from the outside with different names, but... just don't do that! > I realize that technically Python does not have this either. Like I > said in the original post, I do realize that I can override __import__ > with any arbitrary function, and/or do magic in __init__. But I also > did not want to resort to hacks, and would prefer that there be some > kind of well-established solution to the problem. I don't really understand what your problem is exactly, but I think you don't require any __import__ magic or arcane hacks. Perhaps the __path__ package attribute may be useful to you. You can add arbitrary directories to this list, which are searched for submodules of the package. This way you can (partially) decouple the file structure from the logical package structure. But I don't think it's a good thing... > in Java, you would have the class > >org.lib.animal.Monkey > > in > >/org/lib/animal/Monkey.java > > and > >org.lib.animal.Tiger > > in > >/org/lib/animal/Tiger.java > > In other words, introducing a separate file does not introduce a new > package. This works well as long as you are fine with having > everything related to a class in the same file. > > The problem is that with Python, everything is not a classes, and a > file translates to a module, not a class. So you cannot have your > source in different files without introducing as many packages as you > introduce files. Isn't org.lib.animal a package, reflected as a directory on disk? That's the same both for Java and Python. Monkey.py and Tiger.py would be modules inside that directory, just like Monkey.java and Tiger.java. Aren't the same thing? -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Jan 23, 4:49 am, Peter Schuller <[EMAIL PROTECTED]> wrote: > I do *not* want to simply break out X into org.lib.animal.x, and have > org.lib.animal import org.lib.animal.x.X as X. While this naively > solves the problem of being able to refer to X as org.lib.animal.X, > the solution is anything but consistent because the *identity* of X is > still org.lib.animal.x.X. Examples of way this breaks things: > > * X().__class__.__name__ gives unexpected results. > * Automatically generated documentation will document using the "real" > package name. > * Moving the *actual* classes around by way of this aliasing would > break things like pickled data structure as a result of the change > of actual identity, unless one *always* pre-emptively maintains > this shadow hierarchy (which is a problem in and of itself). You can reassign the class's module: from org.lib.animal.monkey import Monkey Monkey.__module__ = 'org.lib.animal' (Which, I must admit, is not a bad idea in some cases.) Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
>> I do *not* want to simply break out X into org.lib.animal.x, and >> have org.lib.animal import org.lib.animal.x.X as X. > > Nevertheless, that seems the best (indeed, the Pythonic) solution to > your problem as stated. Rather than just shooting it down, we'll have > to know more about ehat actual problem you're trying to solve to > understand why this solution doesn't fit. That is exactly what my original post was trying very hard to explain. The problem is the discrepancy that I described between the organization desired in terms of file system structure, and the organization required in terms of module hierarchy. The reason it is a problem is that, by default, there is an (in my opinion) too strong connection between file system structure and module hierarchy in Python. >> While this naively solves the problem of being able to refer to X as >> org.lib.animal.X, the solution is anything but consistent because >> the *identity* of X is still org.lib.animal.x.X. > > The term "identity" in Python means something separate from this > concept; you seem to mean "the name of X". Not necessarily. In part it is the name, in that __name__ will be different. But to the extent that calling code can potentially import them under differents names, it's identity. Because importing the same module under two names results in two distinct modules (two distinct module objects) that have no realation with each other. So for example, if a module has a single global protected by a mutex, there are suddenly two copies of that. In short: identity matters. >> Examples of way this breaks things: >> >> * X().__class__.__name__ gives unexpected results. > > Who is expecting them otherwise, and why is that a problem? Depends on situation. One example is that if your policy is that instances log using a logger named by the fully qualified name of the class, than someone importing and using x.y.z.Class will expect to be able to grep for x.y.z.Class in the output of the log file. >> * Automatically generated documentation will document using the >> "real" package name. > > Here I lose all track of what problem you're trying to solve. You want > the documentation to say exactly where the class "is" (by name), but > you don't want the class to actually be defined at that location? I > can't make sense of that, so probably I don't understand the > requirement. You are baffled that what I seem to want is that the definition of the class (file on disk) be different from the location inferred by the module name. Well, this is *exactly* what I want because, like I said, I do not want the strong connection beteween file system structure and module hierarchy. The fact that this connection exists, is what is causing my problems. Please note that this is not any kind of crazy-brained idea; lots of languages have absolutely zero relationship between file location and modules/namespaces. I realize that technically Python does not have this either. Like I said in the original post, I do realize that I can override __import__ with any arbitrary function, and/or do magic in __init__. But I also did not want to resort to hacks, and would prefer that there be some kind of well-established solution to the problem. Although I was originally hesitant to use an actual example for fear of giving the sense that I was trying to start a language war, your answer above prompts me to do so anyway, to show in concrete terms what I mean, for those that wonder why/how it would work. So for example, in Ruby, there is no problem having: File monkey.rb: module Org module Lib module Animal class Monkey ... .. end end end end File tiger.rb: module Org module Lib module Animal class Tiger ... .. end end end end This is possible because the act of addressing code to be loaded into the interpreter is not connected to the namespace/module system, but rather to the file system. Some languages avoid (but does not eliminate) the problem I am having without having this disconnect. For example, Java does have a strong connection between file system structure and class names. However the critical difference is that in Java, everything is modeled around classes, and class names map directly to the file system structure. So in Java, you would have the class org.lib.animal.Monkey in /org/lib/animal/Monkey.java and org.lib.animal.Tiger in /org/lib/animal/Tiger.java In other words, introducing a separate file does not introduce a new package. This works well as long as you are fine with having everything related to a class in the same file. The problem is that with Python, everything is not a classes, and a file translates to a module, not a class. So you cannot have your source in different files without introducing as many packages as you introduce files. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [E
Re: Module/package hierarchy and its separation from file structure
Peter Schuller <[EMAIL PROTECTED]> writes: > Let me just shoot down one possible suggestion right away, to show > you what I am trying to accomplish: > > I do *not* want to simply break out X into org.lib.animal.x, and > have org.lib.animal import org.lib.animal.x.X as X. Nevertheless, that seems the best (indeed, the Pythonic) solution to your problem as stated. Rather than just shooting it down, we'll have to know more about ehat actual problem you're trying to solve to understand why this solution doesn't fit. > While this naively solves the problem of being able to refer to X as > org.lib.animal.X, the solution is anything but consistent because > the *identity* of X is still org.lib.animal.x.X. The term "identity" in Python means something separate from this concept; you seem to mean "the name of X". > Examples of way this breaks things: > > * X().__class__.__name__ gives unexpected results. Who is expecting them otherwise, and why is that a problem? > * Automatically generated documentation will document using the > "real" package name. Here I lose all track of what problem you're trying to solve. You want the documentation to say exactly where the class "is" (by name), but you don't want the class to actually be defined at that location? I can't make sense of that, so probably I don't understand the requirement. -- \"If it ain't bust don't fix it is a very sound principle and | `\ remains so despite the fact that I have slavishly ignored it | _o__) all my life." —Douglas Adams | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Module/package hierarchy and its separation from file structure
On Wed, 23 Jan 2008 03:49:56 -0600, Peter Schuller wrote: > Let me just shoot down one possible suggestion right away, to show you > what I am trying to accomplish: > > I do *not* want to simply break out X into org.lib.animal.x, and have > org.lib.animal import org.lib.animal.x.X as X. Then you shoot down the idiomatic answer I guess. That's what most people do. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Module/package hierarchy and its separation from file structure
Hello, In writing some non-trivial amount of Python code I keep running into an organizational issue. I will try to state the problem fairly generally, and follow up with a (contrived) example. The root cause of my difficulties is that by default, the relationship between a module hierarchy and the structure of files on disk is too strong for my taste. I want to separate the two as much as possible, but I do not want to resort to non-conventional "hacks" to do it. I am posting this in an attempt to present what I perceive to be a practical problem, and to get suggestions for solutions, or opinions on the most practical policy for how to deal with it. Like I said, I would like a weaker relationship between file system structure and module hierarchy. In particular there are two things I would like: * Least importantly, I don't like jamming code into __init__.py, as a personal preference. * Most importantly, I do not like to jam large amounts of code into a single source file, just for the purpose of keeping the public interface in the same package. An contrived but hopefully illustrative example: We have an organization "Org", which has a library, and as part of that library is code that relates to doing something with animals. As a result, the interesting top-level package for this example is: org.lib.animal Suppose now that I want an initial implementation of the most important animal. I want to create the class (but see [1]): org.lib.animal.Monkey The public interface consists of that class only (and possibly a small handful of functions). The implementation is quite significant however - it is 500 lines of code long. At this point, we had to jam those 500 lines of code into __init__.py. Let's ignore my personal preference of not liking to put code in __init__.py; the fact remains that we have 500 lines of code in a single source file. Now, we want to continue working on this library, adding ten additional animals. At this point, we have these choices (it seems to me): (1) Simply add these to __init__.py, resulting in __init__.py being 5000 lines long[2]. (2) Put each animal into its own file, resulting in org.lib.animal.Monkey now becoming org.lib.animal.monkey.Monkey, and animal X becoming org.lib.animal.x.X. The problem I have is that both of these solutions are, in my opinion, very ugly: * (1) is ugly from a source code management perspective, because jamming 5000 lines of code for ten different animals into a single file is bad for obvious reasons. * (2) is ugly because we introduce org.lib.animal.x.X for animal X, which: (a) is redundant in terms of naming (b) redundant in function since we have a single package for each animal containing nothing but a single class of the same name Clearly, (1) is bad due to file/source structure reasons, and (2) is bad for module organizational reasons. So we are back to my original wish - I want to separate the two, so that I can solve (1) indepeendently of (2). Now, I realize that __init__.py can contain arbitrary code, and that one can override __import__. However, I do not want to resort to "hacks" just to solve this problem; I would prefer some established convention in the community, or at least something that is elegant. Does are people's thoughts on this problem? Let me just shoot down one possible suggestion right away, to show you what I am trying to accomplish: I do *not* want to simply break out X into org.lib.animal.x, and have org.lib.animal import org.lib.animal.x.X as X. While this naively solves the problem of being able to refer to X as org.lib.animal.X, the solution is anything but consistent because the *identity* of X is still org.lib.animal.x.X. Examples of way this breaks things: * X().__class__.__name__ gives unexpected results. * Automatically generated documentation will document using the "real" package name. * Moving the *actual* classes around by way of this aliasing would break things like pickled data structure as a result of the change of actual identity, unless one *always* pre-emptively maintains this shadow hierarchy (which is a problem in and of itself). Thus, it's not clean. It breaks the module abstraction and as a result has unintended consequences. I am looking for some kind of clean solution. What do people do about this in practice? [1] Optionally, we might introduce an "animals" package such that it would become org.lib.animal.animals.Monkey, if we thought we were going to have a lot of public API outside of the animals themselves. This does not affect this dicussion however, as the exact same thing would apply to org.lib.animal.animals as applies to org.lib.animal in the above example. [2] Ignoring for now that it may not be realistic that every animal implementation would be that long; in many cases a lot of code would be in common. But feel free to substitude for something else (a Zoo say). -
Re: file / module / package - import problem
aspineux wrote: > import os.path > > file=open(os.path.join(os.path.dirname(__file__), 'hauteur.yaml')) Thanks that worked ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: file / module / package - import problem
The filename and its path is in global variable __file__ (that is
different in any source file)
try
import os.path
file=open(os.path.join(os.path.dirname(__file__), 'hauteur.yaml'))
On 30 mai, 22:22, EuGeNe Van den Bulke <[EMAIL PROTECTED]>
wrote:
> Hi there,
>
> I have a "problem" which could be a bad design on my behalf but I am not
> sure so ...
>
> I have a package WMI which contains a module hauteur.py which, when
> imported, load data from a file located in WMI/data/. In hauteur.py I
> call open('data/hauteur.yaml').
>
> test.py
> WMI/
> hauteur.py
> data/
> hauteur.yaml
> lot.py
>
> It works well when hauteur is imported in lot.py but if I try import
> WMI.hauteur in test.py it doesn't work because it looks for the
> hauteur.yaml file in the "wrong" place.
>
> Is there a way to tell a module in a package to look for a file in a
> specific place i.e. a within package location?
>
> Thanks,
>
> EuGeNe --http://www.3kwa.com
--
http://mail.python.org/mailman/listinfo/python-list
file / module / package - import problem
Hi there,
I have a "problem" which could be a bad design on my behalf but I am not
sure so ...
I have a package WMI which contains a module hauteur.py which, when
imported, load data from a file located in WMI/data/. In hauteur.py I
call open('data/hauteur.yaml').
test.py
WMI/
hauteur.py
data/
hauteur.yaml
lot.py
It works well when hauteur is imported in lot.py but if I try import
WMI.hauteur in test.py it doesn't work because it looks for the
hauteur.yaml file in the "wrong" place.
Is there a way to tell a module in a package to look for a file in a
specific place i.e. a within package location?
Thanks,
EuGeNe -- http://www.3kwa.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Global utility module/package
Christoph Haas wrote: > Evening, > > I'm currently working on a larger Python project that consists of multiple > programs and packages. As I need a few utility functions time and again I > moved them all into a Utility package and created a class there. ... > As I know that importing packages from multiple modules always keeps it a > singleton I thought of something like this: > > Util.py: > > debugFlag = False > vibranceLevel = 'good' > > def function1(): >global debugFlag >print debugFlag > > main.py: > > import Util > Util.debugFlag = True > Util.function1(whatever) > > def doThis(): >Util.function1(42) > > Here I don't use classes any longer. Good. But to access the "package > variables" I probably need to use "global" again which just moved the > ugliness to another position. This is fine. You don't need 'global' statements to read global variables, function1() can be simply def function1(): print debugFlag Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Global utility module/package
Christoph Haas wrote: > As I know that importing packages from multiple modules always keeps it a > singleton I thought of something like this: > > Util.py: > > debugFlag = False > vibranceLevel = 'good' > > def function1(): >global debugFlag >print debugFlag The global line is not needed: global declarations are only required if you need to _write_ on globals w/in the function. --Scott David Daniels [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Global utility module/package
Evening, I'm currently working on a larger Python project that consists of multiple programs and packages. As I need a few utility functions time and again I moved them all into a Utility package and created a class there. Like this: Util.py: class Util: def __init__(self, debugFlag=False, vibranceLevel='good'): self.debugFlag = debugFlag self.vibranceLevel = vibranceLevel def function1(self): do this do that main.py: import Util util = Util.Util() util.function1(whatever) def some_function(): global util util.function1(dingdong) However this feels like I'm abusing classes because I don't really need several instances of an object. I just need one instance. Even worse is that I'm starting to use "global util" to get access to the "alibi instance" I created. (I'm not sure whether I could even omit it due to scoping rules.) As I know that importing packages from multiple modules always keeps it a singleton I thought of something like this: Util.py: debugFlag = False vibranceLevel = 'good' def function1(): global debugFlag print debugFlag main.py: import Util Util.debugFlag = True Util.function1(whatever) def doThis(): Util.function1(42) Here I don't use classes any longer. Good. But to access the "package variables" I probably need to use "global" again which just moved the ugliness to another position. What would be a good practice here? All I want is a utility package that I can import from everywhere (as a singleton) and that I can use even in subroutines (def) without needing to write "global" here. (The "global" could probably even be omitted because unless I define a variable in a "def" scope the global variable should be visible.) Please enlighten me. :) Kindly Christoph P.S.: Code parts untested. More a schema than something that would actually run. -- Please reply to the list - not to me personally. -- http://mail.python.org/mailman/listinfo/python-list
Re: Setting a module package to use new-style classes
Ben Finney <[EMAIL PROTECTED]> wrote: > "Panos Laganakos" <[EMAIL PROTECTED]> writes: > > > Is there a way to have a whole module package use the new-style > > classes, without having to specify it per module-file or even worse, > > per class definition? > > TTBOMK, you do that with a single statement per module, before any > class definitions: > > __metaclass__ = object Almost: the assignment you need is __metaclass__ = type but yes, it has to be per-module. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: Setting a module package to use new-style classes
"Panos Laganakos" <[EMAIL PROTECTED]> writes: > Is there a way to have a whole module package use the new-style > classes, without having to specify it per module-file or even worse, > per class definition? TTBOMK, you do that with a single statement per module, before any class definitions: __metaclass__ = object In general, code in one module doesn't magically affect other modules. > Maybe by declaring the __metaclass__ in the module's __init__.py? Presumably you mean "the package's __init__.py", which is itself a separate module; so no. -- \"I hate it when my foot falls asleep during the day, because | `\ that means it's gonna be up all night." -- Steven Wright | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Setting a module package to use new-style classes
Is there a way to have a whole module package use the new-style classes, without having to specify it per module-file or even worse, per class definition? Maybe by declaring the __metaclass__ in the module's __init__.py? -- http://mail.python.org/mailman/listinfo/python-list
