Re: Module Structure/Import Design Problem
Rafe a écrit :
On Nov 20, 2:06 pm, Arnaud Delobelle [EMAIL PROTECTED] wrote:
(snip)
Or (better IMHO) you can make types register themselves with the factory
function (in which case it would have some state so it would make more
sense to make it a factory object).
Can you elaborate on what you mean by 'register' with the factory
function?
I think Arnaud means something like:
# baselib.py
class Base(object):
_registry = {}
@classmethod
def register(cls, subclass):
cls._registry[subclass.__name__] = subclass
@classmethod
def get_class(cls, classname):
return cls._registry[classname]
# typelib.py
from baselib import Base
class Sub(Base):
pass
Base.register(Sub)
Also...holy [EMAIL PROTECTED], I got a clean import working! I swear I tried
that
before with unhappy results. I'll carefully try this in my real code.
Is this the right way to impliment the imports?
baselib.py
[1] class BaseClass(object):
[2] def factory(self):
[3] import typelib # -- import inside function
[4] return typelib.TypeA()
This is one possible way to solve circular imports, but it has some
drawbacks - mostly, the performance hit of the import statement on each
call to BaseClass.factory
typelib.py
[1] import baselib # -- module level import
[2]
[3] class TypeA(baselib.BaseClass):
[4] def __init__(self):
[5] print TypeA : __init__()
import typelib
type = typelib.TypeA()
ot
This shadows the builtin type 'type'. Better to use another name here...
/ot
TypeA : __init__()
another_type = type.factory()
TypeA : __init__()
another_type
typelib.TypeA object at 0x00B45F10
I am curious (not directed at Arnaud), why not have an 'include'-like
import for special cases in python (or do I not understand includes
either?)
In C, #include somefile.h is a pre-processor statement that does a
textual inclusion before compilation. Python just doesn't work that way.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Bruno Desthuilliers [EMAIL PROTECTED] writes:
Rafe a écrit :
On Nov 20, 2:06 pm, Arnaud Delobelle [EMAIL PROTECTED] wrote:
(snip)
Or (better IMHO) you can make types register themselves with the factory
function (in which case it would have some state so it would make more
sense to make it a factory object).
Can you elaborate on what you mean by 'register' with the factory
function?
I think Arnaud means something like:
# baselib.py
class Base(object):
_registry = {}
@classmethod
def register(cls, subclass):
cls._registry[subclass.__name__] = subclass
@classmethod
def get_class(cls, classname):
return cls._registry[classname]
# typelib.py
from baselib import Base
class Sub(Base):
pass
Base.register(Sub)
That's what I meant. Sorry Rafe I missed your follow-up.
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
On Nov 22, 2:39 am, Stef Mientki [EMAIL PROTECTED] wrote: Gabriel Genellina wrote: En Thu, 20 Nov 2008 17:36:11 -0200, Stef Mientki [EMAIL PROTECTED] escribió: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-) http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html May I reiterate my criticism to your solution posted last week? You're welcome, if there are better (even more important simpler) solutions, I'm in. I don't think extending sys.path to include every directory under your project is a good idea. Basically, you're flattening the directory layout, removing any structure, like it was before Python 1.5 added package support. It is like dumping all your modules in a single directory, with no hierarchy and lots of name conflicts. Well I started at 2.4 and now arrived at 2.5, so I don't know what it was at 1.5. And I'm probably going to stay at 2.5 for a long time, upgrading is one of the few (maybe the only) disadvantages of Python ;-) For Python, I guess you're right, all files are in one flat directory, ... ... but it's just 1 program, so what's the objection ? For the program itself, which is highly dynamic, it can find sets of python-files in certain subpaths, so that's quit ordered I think ? Worse: because your proposal makes the same file reachable under many different names, the *same* source module will generate *different* module objects stored as *different* entries in sys.modules. Globals don't work anymore, subclasses aren't subclasses... lots of problems. Could you explain this a little more ... ... I'm just adding every subpath to the Pythonpath, so in my ignorant believe, every module is imported the same way, but I might be wrong. Globals not working , a good program doesn't have any globals ( said a non-programmer ;-) Subclasses aren't subclasses ? ( I always derive subclasses in the module itself, or in the same directory as the parentclass) Relative imports (PEP328 [1]) were introduced -among other things- as an attempt to avoid such problems. You're going in the opposite direction. Please stop doing that - or at least keep us informed of where you work! [1]http://www.python.org/dev/peps/pep-0328 Sorry I don't understand all that pep-talk (I'm not a programmer ;-) I read it starts with For the second problem, it is proposed that all import statements be absolute by default (searching sys.path only) with special syntax (leading dots) for accessing package-relative imports. I think that's exactly what I'm doing: absolute imports. Please explain in simple language what I'm all doing wrong, or how I get a better solution. thanks, Stef Hi Stef, I'm not a programmer either (and still in my first year of using python), so I understand a bit of where you are coming from. There is some good info in this thread, but put simply... - There is very rarely a reason to add things to the python path since only the module, or package top folder, needs to be in the pthon path for everything to work (which really means the parent folder is in the path as python looks in the folder on the path for modules and packages). If python can find the top folder in a package (folder structure with __init__.py files in each folder), then you can access everything easily. - Use the simplest form of import that works for what you need. From my reading on python standards, this is the simplest to use and read: import foo.bar as bar Only use 'as' if the path is really long and you don't want to type it every time (I often use modules if they are only two levels deep without 'as'. It makes the code easier to read in many cases). It's like creating a shortcut or alias. The next most common is probably the from ... import ... form. I'd say the general rule-of-thumb is to avoid situations where you don't know where a name comes from. This is why from foo import * is generally evil. I have only found a single case to use this and that is when I have a globals module that is designed for use with 'import *'. 99% of the time I use the standard 'import module path' Examples: If I have: package __init__.py # Can be empty subpackage __init__.py # Can be empty module.py# has ClassA in it I will almost always do this: import package.subpackage.module as module module.ClassA() This makes it easy to understand where it is coming from (or at the very least offers a clue to look for import statements). Worse is: from package.subpackage import module module.ClassA() I'm not sure if there is any difference to the above, but again, I try to use the simplest form possible, and this is just a more complicated version of the same thing (IMO). It isn't
Re: Module Structure/Import Design Problem
En Fri, 21 Nov 2008 17:39:14 -0200, Stef Mientki [EMAIL PROTECTED] escribió: Gabriel Genellina wrote: En Thu, 20 Nov 2008 17:36:11 -0200, Stef Mientki [EMAIL PROTECTED] escribió: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html May I reiterate my criticism to your solution posted last week? You're welcome, if there are better (even more important simpler) solutions, I'm in. I don't think extending sys.path to include every directory under your project is a good idea. Basically, you're flattening the directory layout, removing any structure, like it was before Python 1.5 added package support. It is like dumping all your modules in a single directory, with no hierarchy and lots of name conflicts. Well I started at 2.4 and now arrived at 2.5, so I don't know what it was at 1.5. And I'm probably going to stay at 2.5 for a long time, upgrading is one of the few (maybe the only) disadvantages of Python ;-) For Python, I guess you're right, all files are in one flat directory, ... ... but it's just 1 program, so what's the objection ? (note that it's *you* who wrote a single program.) For the program itself, which is highly dynamic, it can find sets of python-files in certain subpaths, so that's quit ordered I think ? (again, you're talking of *your* program.) On the contrary, you're destroying the directory hierarchy on disk, making modules live in a flat namespace, like a big bag without structure. Worse: because your proposal makes the same file reachable under many different names, the *same* source module will generate *different* module objects stored as *different* entries in sys.modules. Globals don't work anymore, subclasses aren't subclasses... lots of problems. Could you explain this a little more ... ... I'm just adding every subpath to the Pythonpath, so in my ignorant believe, every module is imported the same way, but I might be wrong. The import mechanism in Python is not perfect. It uses a mapping module name - module object (sys.modules). If a module is not found by name in sys.modules, it is loaded from disk. If you have two different ways to find the *same* module, Python won't notice, and will create two different copies of it. So it is important to have only *one* way to locate a module; that is, entries in sys.path should not overlap (uhm, well, not exactly, but it's hard to explain right) Consider this structure: main/ +--- PkgA/ +--- PkgB/ foo.py Without using your recipe, only main is is sys.path. The normal way to reach module foo from main, is to import it as PkgA.PkgB.foo After using your recipe, sys.path will contain .../main/PkgA and /main/PkgA/PkgB. You may import foo using: PkgA.PkgB.foo, PkbB.foo, and even foo directly. All of these different ways to reach the same file create different module objects and different entries in sys.modules. If you initialize something when the module is loaded, by example, you'll run the initialization three times. There is a well known case: the main module. Even if the file is named app.py, when the program runs, it is known as __main__. If another module executes import app it will get a different module object. (I think this case is listed in the FAQ.) Under your proposal, this aliasing could happen with virtually any module. Globals not working , a good program doesn't have any globals ( said a non-programmer ;-) Uh? Classes are usually globals. Modules too. Global state is saved in global variables... A good programmer knows when to use global state - like the registry of classes used by the factory method that someone suggested previously in this thread. In the 'foo' example above, there will be three copies of the module's global variables - each with different values and evolving separately. Subclasses aren't subclasses ? ( I always derive subclasses in the module itself, or in the same directory as the parentclass) (again, *you* do it in that simple way.) If you define a class X inside foo, you'll have three different classes, all named X. A subclass Y of X is *not* a subclass of the other X's, depending on how you import the foo module. Relative imports (PEP328 [1]) were introduced -among other things- as an attempt to avoid such problems. You're going in the opposite direction. Please stop doing that - or at least keep us informed of where you work! [1] http://www.python.org/dev/peps/pep-0328 Sorry I don't understand all that pep-talk (I'm not a programmer ;-) I read it starts with For the second problem, it is proposed that all import statements be absolute by default (searching sys.path only) with special syntax (leading dots) for accessing package-relative imports. I think that's exactly what I'm doing: absolute imports. What you're
Re: Module Structure/Import Design Problem
Steve Holden wrote: Stef Mientki wrote: Gabriel Genellina wrote: [...] Sorry I don't understand all that pep-talk (I'm not a programmer ;-) And I'm not a plumber. The difference between us is that I don't write blogs telling people how to lay out and connect their pipework. regards Steve PS: Q: What's the difference between God and a doctor? A: God doesn't think she's a doctor Probably this a a local disease after all, as only me and a couple of other non-programmers in my surrounding seems to have the same view on imports as me. Let me try to explain my point of view. Undoubtable the import statement gives enormous functionality and flexibility to programmers. But for non-programmers the import statement is of no value whatsoever, and it's just a necessary evil. Above that, the import statement is (for non programmers) one of the most difficult to understand parts of Python. On the other hand, why do need Lutz and Ascher (which might not be the best book) over 50 pages to explain imports and packages ? Delphi, known as an old language, has already automated imports for more than 15 years. I would love to see a cookbook recipe (of at most 2 pages) , written by experts, of how to use imports in the right way. Or even better an automated import mechanism. But as it looks that only a few people have the same problem as me, it has no value to pay any further attention to this issue. cheers, Stef -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Stef Mientki wrote: Steve Holden wrote: Stef Mientki wrote: Gabriel Genellina wrote: [...] Sorry I don't understand all that pep-talk (I'm not a programmer ;-) And I'm not a plumber. The difference between us is that I don't write blogs telling people how to lay out and connect their pipework. regards Steve PS: Q: What's the difference between God and a doctor? A: God doesn't think she's a doctor Probably this a a local disease after all, as only me and a couple of other non-programmers in my surrounding seems to have the same view on imports as me. Let me try to explain my point of view. Undoubtable the import statement gives enormous functionality and flexibility to programmers. But for non-programmers the import statement is of no value whatsoever, and it's just a necessary evil. Above that, the import statement is (for non programmers) one of the most difficult to understand parts of Python. On the other hand, why do need Lutz and Ascher (which might not be the best book) over 50 pages to explain imports and packages ? Delphi, known as an old language, has already automated imports for more than 15 years. I would love to see a cookbook recipe (of at most 2 pages) , written by experts, of how to use imports in the right way. Or even better an automated import mechanism. But as it looks that only a few people have the same problem as me, it has no value to pay any further attention to this issue. I don't think the existing (2.6) documentation on packages is at all satisfactory, consisting as it does of a single paragraph: Hierarchical module names: when the module names contains one or more dots, the module search path is carried out differently. The sequence of identifiers up to the last dot is used to find a “package”; the final identifier is then searched inside the package. A package is generally a subdirectory of a directory on sys.path that has a file __init__.py. Brett Cannon, a fairly active core developer, gives interesting talks about the way the import mechanism is broken, and has plans to eventually include a replacement implemented entirely in Python. When that happens, we might expect the documentation to improve, but I am afraid that in open source it has to be an itch that someone needs to scratch, and it seems like so far nobody is itching badly enough. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Rafe wrote: On Nov 21, 2:36 am, Stef Mientki [EMAIL PROTECTED] wrote: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-)http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html cheers, Stef I really don't understand what you are trying to accomplish in your article. I strongly disagree with this statement... A second demand is that every module should be able to act as a main file by running it's main section. ...I am finding the best programs have only one entry point or interface (though some libraries can be useful from outside the package.) At first the program already basically consists of 3 different user interfaces, MatLab-like, LabView-like, Punthoofd-like. Then it maybe a matter of taste, but for the moment I've planned to have 3 to 5 different IDEs (besides the main programs), but that might change when they are all finished. It might be a matter of taste, but I think you'll be better of by an optimal dedicated IDE, instead of something like Word ( I estimate that I, as many others, use less than 1% of the items, so you can imagine how much time we spent on searching the right item) Being able to run any file in a package seems like it creates a confusing user/developer experience. What kind of problem makes this solution applicable? Testing ... ... by others on other operating systems ... in combination with other Python versions ... in combination with other Library versions (the program interacts heavily with wxPython, Numpy, Scipy, MatPlotLib, VPython, PyGame, PIL, Rpyc, winpdb, wmi , LXML., ConfigObj, Nucular, ...) ... automatically testing after modifications Next, you say... ...recursive searches for all subdirectories and adds them to the Python Path. ...it seems like you add every module in your packages directly to the sys.path. Doesn't this destroy the package name-spaces? For example, I have a module called 'types' in my package if I add that to the python path, 'import types' still returns the built-in 'types' module. Wouldn't this collision be confusing? Regardless, isn't putting the package in the right place enough? Please explain. Well I've to confess, I'm not a programmer, And if I can program, well, I leave that up to you, and is not very relevant, because I know I can make useful programs ;-) But I've to admit, I don't understand packages. I've never experienced any collision, but then I probably always use much longer, self explaining names than the build-ins. This my directory structure ( no module names) , which only contains the program and its libraries. A few paths are 1 or 2 levels deeper. The lang contains internationalization strings in normal python files (another 300 files per language ;-) Now in my opinion, each module in each path should be able to reach any other module in this directory structure. So other / better / simpler solutions are welcome. cheers, Stef main_path |__ lang |__ sub_path1 |__lang |__sub_sub_path_1a |__lang |__sub_sub_path_1b |__lang |__sub_sub_path_1c |__lang |__ sub_path2 |__lang |__sub_sub_path_2a |__lang |__sub_sub_path_2b |__lang ... |__ sub_path8 Cheers, - Rafe -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Gabriel Genellina wrote: En Thu, 20 Nov 2008 17:36:11 -0200, Stef Mientki [EMAIL PROTECTED] escribió: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-) http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html May I reiterate my criticism to your solution posted last week? You're welcome, if there are better (even more important simpler) solutions, I'm in. I don't think extending sys.path to include every directory under your project is a good idea. Basically, you're flattening the directory layout, removing any structure, like it was before Python 1.5 added package support. It is like dumping all your modules in a single directory, with no hierarchy and lots of name conflicts. Well I started at 2.4 and now arrived at 2.5, so I don't know what it was at 1.5. And I'm probably going to stay at 2.5 for a long time, upgrading is one of the few (maybe the only) disadvantages of Python ;-) For Python, I guess you're right, all files are in one flat directory, ... ... but it's just 1 program, so what's the objection ? For the program itself, which is highly dynamic, it can find sets of python-files in certain subpaths, so that's quit ordered I think ? Worse: because your proposal makes the same file reachable under many different names, the *same* source module will generate *different* module objects stored as *different* entries in sys.modules. Globals don't work anymore, subclasses aren't subclasses... lots of problems. Could you explain this a little more ... ... I'm just adding every subpath to the Pythonpath, so in my ignorant believe, every module is imported the same way, but I might be wrong. Globals not working , a good program doesn't have any globals ( said a non-programmer ;-) Subclasses aren't subclasses ? ( I always derive subclasses in the module itself, or in the same directory as the parentclass) Relative imports (PEP328 [1]) were introduced -among other things- as an attempt to avoid such problems. You're going in the opposite direction. Please stop doing that - or at least keep us informed of where you work! [1] http://www.python.org/dev/peps/pep-0328 Sorry I don't understand all that pep-talk (I'm not a programmer ;-) I read it starts with For the second problem, it is proposed that all import statements be absolute by default (searching sys.path only) with special syntax (leading dots) for accessing package-relative imports. I think that's exactly what I'm doing: absolute imports. Please explain in simple language what I'm all doing wrong, or how I get a better solution. thanks, Stef -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Stef Mientki wrote: Gabriel Genellina wrote: [...] Sorry I don't understand all that pep-talk (I'm not a programmer ;-) And I'm not a plumber. The difference between us is that I don't write blogs telling people how to lay out and connect their pipework. regards Steve PS: Q: What's the difference between God and a doctor? A: God doesn't think she's a doctor -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
On Nov 20, 2:06 pm, Arnaud Delobelle [EMAIL PROTECTED] wrote: Rafe [EMAIL PROTECTED] writes: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? It's not very clear what your problem is. I guess your factory functions are defined in baselib.py whereas types.py and special.py import baselib, therefore you don't know how to make the factory function aware of the types defined in special.py and types.py. You can use cyclic import in many cases. Or (better IMHO) you can make types register themselves with the factory function (in which case it would have some state so it would make more sense to make it a factory object). -- Arnaud hi Arnaud, You got my problem right, sorry it wasn't more clear. Can you elaborate on what you mean by 'register' with the factory function? Also...holy [EMAIL PROTECTED], I got a clean import working! I swear I tried that before with unhappy results. I'll carefully try this in my real code. Is this the right way to impliment the imports? baselib.py [1] class BaseClass(object): [2] def factory(self): [3] import typelib # -- import inside function [4] return typelib.TypeA() typelib.py [1] import baselib # -- module level import [2] [3] class TypeA(baselib.BaseClass): [4] def __init__(self): [5] print TypeA : __init__() import typelib type = typelib.TypeA() TypeA : __init__() another_type = type.factory() TypeA : __init__() another_type typelib.TypeA object at 0x00B45F10 I am curious (not directed at Arnaud), why not have an 'include'-like import for special cases in python (or do I not understand includes either?) Thanks! - Rafe -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Rafe wrote: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. cheers, Stef Cheers - Rafe -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Rafe wrote: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? Well a simple way to do this is to observe that even when a base class's method is inherited by an instance of a subclass, when the method is called the type of self is the subclass. And you can call the subclass's type to create an instance. Perhaps the following code would make it more obvious: $ cat baseclass.py class Base(object): def factory(self, arg): return type(self)(arg) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ cat subclass.py from baseclass import Base class sub(Base): def __init__(self, arg): print Creating a sub with arg, arg s = sub(Manual) thing = s.factory(Auto) print type(thing) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ python subclass.py Creating a sub with arg Manual Creating a sub with arg Auto class '__main__.sub' Hope this helps. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Stef Mientki wrote: Rafe wrote: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? regards steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-) http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html cheers, Stef -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
On Nov 21, 1:39 am, Steve Holden [EMAIL PROTECTED] wrote: Rafe wrote: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? Well a simple way to do this is to observe that even when a base class's method is inherited by an instance of a subclass, when the method is called the type of self is the subclass. And you can call the subclass's type to create an instance. Perhaps the following code would make it more obvious: $ cat baseclass.py class Base(object): def factory(self, arg): return type(self)(arg) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ cat subclass.py from baseclass import Base class sub(Base): def __init__(self, arg): print Creating a sub with arg, arg s = sub(Manual) thing = s.factory(Auto) print type(thing) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ python subclass.py Creating a sub with arg Manual Creating a sub with arg Auto class '__main__.sub' Hope this helps. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Hi Steve, Correct me if I have this wrong, but the problem with your solution is that it only creates a new instance of the same class, type(self), while I need to return any number of different possibilities. I thought about getting the module from self... class base(object): def factory(self, type): module = sys.modules[self.__class__.__module__] return getattr(module, type) ...but my baseclass is used from several modules so this would be inaccurate for me (the factory method only uses my 'types' module, so a hard import works) I'm still wondering what Arnaud meant by make types register themselves with the factory function - Rafe -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
On Nov 21, 2:36 am, Stef Mientki [EMAIL PROTECTED] wrote: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-)http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html cheers, Stef I really don't understand what you are trying to accomplish in your article. I strongly disagree with this statement... A second demand is that every module should be able to act as a main file by running it's main section. ...I am finding the best programs have only one entry point or interface (though some libraries can be useful from outside the package.) Being able to run any file in a package seems like it creates a confusing user/developer experience. What kind of problem makes this solution applicable? Next, you say... ...recursive searches for all subdirectories and adds them to the Python Path. ...it seems like you add every module in your packages directly to the sys.path. Doesn't this destroy the package name-spaces? For example, I have a module called 'types' in my package if I add that to the python path, 'import types' still returns the built-in 'types' module. Wouldn't this collision be confusing? Regardless, isn't putting the package in the right place enough? Please explain. Cheers, - Rafe -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Rafe wrote: On Nov 21, 1:39 am, Steve Holden [EMAIL PROTECTED] wrote: Rafe wrote: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? Well a simple way to do this is to observe that even when a base class's method is inherited by an instance of a subclass, when the method is called the type of self is the subclass. And you can call the subclass's type to create an instance. Perhaps the following code would make it more obvious: $ cat baseclass.py class Base(object): def factory(self, arg): return type(self)(arg) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ cat subclass.py from baseclass import Base class sub(Base): def __init__(self, arg): print Creating a sub with arg, arg s = sub(Manual) thing = s.factory(Auto) print type(thing) [EMAIL PROTECTED] /c/Users/sholden/Projects/Python $ python subclass.py Creating a sub with arg Manual Creating a sub with arg Auto class '__main__.sub' Hope this helps. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Hi Steve, Correct me if I have this wrong, but the problem with your solution is that it only creates a new instance of the same class, type(self), while I need to return any number of different possibilities. In that case you need to pass the type of the new instance you require as an argument to the factory method, I guess, or something similar. My example has each subclass returning instances of the subclass that was used to call the factory method, yes. If A instances need to be able to get B instances and B instances need to be able to create B instances then you do indeed have a tricky problem when it comes to separating your classes into different modules,. But it's not insoluble! I thought about getting the module from self... class base(object): def factory(self, type): module = sys.modules[self.__class__.__module__] return getattr(module, type) ...but my baseclass is used from several modules so this would be inaccurate for me (the factory method only uses my 'types' module, so a hard import works) I am not sure yet if I understand the requirement properly. It seems from the above code that you always want to create instances of a named type defined in the same module as the subclass? I'm still wondering what Arnaud meant by make types register themselves with the factory function You would have to ask him, but I suspect he means having each subtype call a method of the base type to add itself to some dictionary so you can call the factory method with a key that will tell it which subtype to produce. This is a fairly sensible way to separate definitions form references. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
En Thu, 20 Nov 2008 17:36:11 -0200, Stef Mientki [EMAIL PROTECTED] escribió: I'm not an expert, I even don't fully understand your problem, but having struggled with imports in the past, I've a solution now, which seems to work quit well. That's not very helpful, is it? Were you planning to keep the solution secret? sorry slip of the keyboard ;-) http://mientki.ruhosting.nl/data_www/pylab_works/pw_importing.html May I reiterate my criticism to your solution posted last week? I don't think extending sys.path to include every directory under your project is a good idea. Basically, you're flattening the directory layout, removing any structure, like it was before Python 1.5 added package support. It is like dumping all your modules in a single directory, with no hierarchy and lots of name conflicts. Worse: because your proposal makes the same file reachable under many different names, the *same* source module will generate *different* module objects stored as *different* entries in sys.modules. Globals don't work anymore, subclasses aren't subclasses... lots of problems. Relative imports (PEP328 [1]) were introduced -among other things- as an attempt to avoid such problems. You're going in the opposite direction. Please stop doing that - or at least keep us informed of where you work! [1] http://www.python.org/dev/peps/pep-0328 -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Module Structure/Import Design Problem
Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? Cheers - Rafe -- http://mail.python.org/mailman/listinfo/python-list
Re: Module Structure/Import Design Problem
Rafe [EMAIL PROTECTED] writes: Hi, I am in a situation where I feel I am being forced to abandon a clean module structure in favor of a large single module. If anyone can save my sanity here I would be forever grateful. My problem is that classes in several modules share a common base class which needs to implement a factory method to return instances of these same classes. An example to help illustrate what I mean: Lets say I have the following modules with the listed classes: - baselib.py with BaseClass - types.py with TypeA, ... - special.py with SpecialTypeA, ... Which would be used a bit like this: type_a = any_type_instance.get_type(TypeA) special_type = type_a.get_type(SpecialTypeA) Again, I can get around this by dumping everything in to one module, but it muddies the organization of the package a bit. This seems like a problem that would come up a lot. Are there any design paradigms I can apply here? It's not very clear what your problem is. I guess your factory functions are defined in baselib.py whereas types.py and special.py import baselib, therefore you don't know how to make the factory function aware of the types defined in special.py and types.py. You can use cyclic import in many cases. Or (better IMHO) you can make types register themselves with the factory function (in which case it would have some state so it would make more sense to make it a factory object). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
