Robin Haswell wrote: > Can anyone give me advice on making this all a bit more transparent? I > guess I really would like a method to bring all these files in to the same > scope to make everything seem to be all one application, even though > everything is broken up in to different files.
This is very much a deliberate design decision in Python. I haven't used PHP, but in e.g. C, the #include directive means that you pollute your namespace with all sorts of strange names from all the third party libraries you are using, and this doesn't scale well. As your application grows, you'll get mysterious bugs due to strange name clashes, removing some module you no-longer need means that your app won't build since the include file you no longer include in turn included another file that you should have included but didn't etc. In Python, explicit is better than implicit (type "import this" at the Python prompt) and while this causes some extra typing it helps with code maintenance. You can always see where a name in your current namespace comes from (unless you use "from xxx import *"). No magic! Concerning your database operations, it seems they are distributed over a lot of different modules, and that might also cause problems, whatever programming language we use. In typical database applications, you need to keep track of transactions properly. For each opened connection, you can perform a number of transactions after each other. A transaction starts with the first database operation after a connect, commit or rollback. A cursor should only live within a transaction. In other words, you should close all cursors before you perform a commit or rollback. I find it very difficult to manage transactions properly if the commits are spread out in the code. Usually I want one module to contain some kind of transaction management logic, where I determine the transaction boundries. This logic will hand out cursor object to various pieces of code, and determine when to close the cursors and commit the transaction. I haven't really written multithreaded applications, so I don't have any experiences in the problems that might cause. I know that it's a fairly common pattern to have all database transactions in one thread though, and to use Queue.Queue instances to pass data to and from the thread that handles DB. Anyway, you can only have one transaction going on at a time for a connection, so if you share connections between threads (or use a separate DB thread and queues) a rollback or commit in one thread will affect the other threads as well... Each DB-API 2.0 compliant library should be able to declare how it can be used in a threaded application. See the DB-API 2.0 spec: http://python.org/peps/pep-0249.html Look for "threadsafety". -- http://mail.python.org/mailman/listinfo/python-list