Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
It can be fun when talking several subjects in the same post . In this case, it was just a matter of thinking about what the main question was about unit testing code that also required other production packages, but with the caveats that he didn't want to duplicate packages just to test code or hack around during testing, then have to change the test code when it was time to add to production code (which may introduce more errors that the testing would not find). CVS just found it's way from one of the suggestions as a side note in reply to a minor point about how to keep track of test code without messing with/duplicating production code. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Thanks everyone for their assistance. I have managed to achieve all that I set out to do: - separation between src and test folders - now successfully sharing code between projects - running unittest s and suites - avoiding namespace collisions The solution involved the following (if anyone else is interested) - ammending all my __init__.py packages so that they included the following: from pkgutil import extend_path __path__ = extend_path(__path__, __name__) as per Peter Otten's suggestion - refactoring all the code so that the imports and froms conformed to the new package structure (a.b.c.common, a.b.c.app1 etc) and physically moving all the required files - ammending the PYTHONPATH so that the src and test directories for each project is included at the time of running ie PYTHONPATH=COMMON/src;COMMON/test;APP1/src;APP1/test;APP2/src;APP2/test Of course the /test entries are only required for testing and not runtime but you get the idea. I understand that flatter package structures are apparently the python way (http://dirtsimple.org/2004/12/python-is-not-java.html) but I like a nice little small little related package of functionality when I am l carving up a complex problem so I'm happy to incur any extra performance penalty in dict lookups. thanks again Todd -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
COMMONsrca.b.c.commontesta.b.c.common Ok, first thing... it won't work becaus as soon as python finds the first a.b.c.common it will bail on the other... That's the default approach for java, becase it will give you access to module-level things declared as 'default', that otherwise you wouldn't have access (in python all are grown-ups, so that's not necessary) and it won't bail out in the first match. So, in python, you'd probably do something as: test/test.a.b.c.common or create a tests dir as src/a.b.c.common.tests (I guess there are other solutions, but those 2 seem more appealing to me -- coming from a java background). I believe you'd be able to keep that structure and play with the pythonpath if you wanted, but it could be too brittle in the end, and lead to hard-to-discover problems.Cheers,Fabio -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
alisonken1 wrote:
> As to the question "fail to see how version control relates to
> code/test separation", the original poster asked several questions, one
> of which was production/testing code separation.
>
> Along with the separation (so while you're testing new functionality,
> you don't break production files), a properly setup CVS allows you to
> do this by importing files from a production branch into your testing
> branch
Ok, get it - you're talking about branching, when the OP talked about
unit testing, which is something totally different.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
[EMAIL PROTECTED] wrote: > ie does the first instance of the package a.b.c effectively mask out > the others so that if a.b.c.common occurs first in the PYTHONPATH then > a.b.c.project1 will never be found as the "a.b.c" package is only > loaded from the a.b.c.common folder? By default, yes. But there's a way around this limitation: http://docs.python.org/lib/module-pkgutil.html Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Thanks for all the prompt helpful responses :- ) I have read and reread section 6.4 -> of the python docs and it doesn't really help me. Just a couple of clarifications. I am already using a VCS (Version Control System) - subversion actually. When I mentioned a clear separation of production and test code I was more referring to this sort of arrangement: src a.b.c.common a.b.c.common.test which I believe is bad versus the clearer physical separation test a.b.c.commom.test The first option has the testing code within the same folder hierarchy which makes it harder to separate the two physically at "release" time without writing some complicated scripts to selectively exclude items. In the Java world the second option is the standard, and my preference if it's possible. I suspect that some of my import / from problems stem from the fact that I want to use a standard base package for all my projects to avoid namespace issues. ie all my projects use packages of the form a.b.c.projectName where a.b.c is my namespace (dns address actually) a.b.c.common a.b.c.project1 a.b.c.project2 Is this not the way python projects should be packaged? Is there a problem with having a common base package name like this? ie does the first instance of the package a.b.c effectively mask out the others so that if a.b.c.common occurs first in the PYTHONPATH then a.b.c.project1 will never be found as the "a.b.c" package is only loaded from the a.b.c.common folder? Does anyone else have any issues with running a master test suite and indvidual tests. It seems that if I set up the imports/froms for the master testsuite then I can't run the tests individuallly and vice versa ... unless I remove all packages and have a single flat directory. There has got to be big projects written in python that need to separate out their testing code according to some logical grouping. thanks Todd -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
As to the question "fail to see how version control relates to code/test separation", the original poster asked several questions, one of which was production/testing code separation. Along with the separation (so while you're testing new functionality, you don't break production files), a properly setup CVS allows you to do this by importing files from a production branch into your testing branch so you can re-use vetted code (production) in your trial code (testing) without affecting what's already out there (inadvertently breaking currently shipping code to customers). -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Not sure about "vcs" - but "cvs" can be more fun if the hard disk dies
{g}.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Paddy a écrit : > Thanks alisonken1, I did indeed mean that when you have production code > and test code then its time to bring in the version control systems. Hopefully all this is under version control right from the start... And I still fail to see how version control relates to code/test separation (sorry, my only neuron left just refuses to work this late). (snip hacks) > But we should all be using a version control system right :-) But we *are* all using a vcs - aren't we ?-) -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Thanks alisonken1, I did indeed mean that when you have production code and test code then its time to bring in the version control systems. But having said that ;-) i have been naughty in the past and used the -s or --symbolic-link option of the gnu cp (sometimes available as gcp), command which copies a whole directory tree of the production release to a test area, copying directories but putting links in to the original files. I make sure that the production code is all read-only thenwork in the test area. If I need to modify a file I: * list the link * remove the link, replacing it with a copy of the file linked to * get write priviliges to the new copy * edit the copy. But we should all be using a version control system right :-) - paddy. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
I believe that Paddy was referencing his second point about keeping "production code" and "test code" clearly delimited, so was recommending that a version control system be used rather than the local disk structure required by python for building module packages. Although, I have found that symlinks work fine for delimiting file structure - ex.: $BASEDIR/ $TESTDIR/ $TESTDIR/ Using an example program that I wrote, the following directory structure snippet works for me (where 'bin' is a directory to shared modules of production code): -rw-r--r-- 1 ken users 509 2004-11-30 13:35 label.ini lrwxrwxrwx 1 ken users 9 2004-12-29 01:54 bin -> ../../bin/ drwxr-xr-x 3 ken users 72 2004-12-10 16:01 data/ drwxr-xr-x 2 ken users 80 2004-12-10 16:46 disp/ drwxr-xr-x 2 ken users 48 2004-12-10 16:01 etc/ drwxr-xr-x 2 ken users 80 2004-12-10 16:27 prn/ drwxr-xr-x 2 ken users 208 2004-12-10 16:14 test/ drwxr-xr-x 2 ken users 1208 2004-12-10 17:15 label.py -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Paddy a écrit : > Your second point 'clear separation of "production" code and "test" > code' is best handled by using a revision control system. I'm afraid I don't see the point here ??? -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
Your second point 'clear separation of "production" code and "test" code' is best handled by using a revision control system. I know Clearcase, because I use it at work, but I have heard good things about Subversion. -- http://mail.python.org/mailman/listinfo/python-list
Re: How should multiple (related) projects be arranged (structured) and configured so that they can share code, have a related package structure and enable proper unittesting, and ensuring no namespac
[EMAIL PROTECTED] wrote:
> Summary:
(snip)
>
> I'm working on a few projects concurrently so I have tried to
> arranged my projects like this:
>
> COMMON
> src
> a.b.c.common
> test
> a.b.c.common
>
> APP1
> src
> a.b.c.app1
> test
> a.b.c.app1
>
> APP2
> src
> a.b.c.app2
> test
> a.b.c.app2
>
>
> But it has not worked due to import/from issues. It appears that using
> a common base package hierarchy (as is standard practice in the java
> world) caused issues. Ie not able to use "a.b.c" as the base
> package for all my projects, in order to avoid namespace collisions.
What you need to understand here is that Python's packages/modules
system is directly tied to the filesystem. A package is a directory with
a __init__.py file and possibly modules and sub-packages in it, and the
package name is the directory name (plus parents packages for
sub-packages). You *cannot* add arbitrary prefix like 'a.b.c' to the
package name.
Regarding import problems, I suggest that you re-read:
http://www.python.org/doc/2.4.2/tut/node8.html#SECTION00840
with particular attention to 6.4.2.
Note that you can dynamically modify sys.path - this may solve some
problems with imports !-)
Hoping some guru will be more helpfull...
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
