On 1/19/06, Kent Johnson <[EMAIL PROTECTED]> wrote: > Hi Bernard, > > I'm glad you got it working but kind of surprised at what you had to do. > You shouldn't have to have a single thread to access the database. In > your original desing were you sharing a connection between threads? That > could cause trouble. But if each connection has its own thread and you > are using transactions and isolation levels appropriately, they > shouldn't stomp on each other.
I'm not sure what you mean by transaction and isolation levels. I get some sort of an idea, but could you elaborate on the idea? I have not considered having one connection per thread, and have not tested the idea. The way it works right now is that there is only one connection, as a class instance attribute. All transaction go through that connection. > > There are some good recipes for queuing up tasks and getting callbacks > that might be simpler than your dictionary solution. For example see > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/435883 > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/203871 > > You could also use a second queue to return results to the main thread. Thanks I'll check these out. Bernard > > Kent > > Bernard Lebel wrote: > > Hello, > > > > For the record, in case anyone would like to know what happened with > > this issue... > > > > > > It seems that the problem was related to the way I managed MySQL > > connections. There was the main thread that would query every 5 > > seconds the database to see if a someone wanted to abort the job > > running on the render node. > > > > However, the actual rendering was managed by a child thread, who would > > perform many MySQL transactions, depending on the output produced by > > the rendering software. > > > > So there were two separate threads doing MySQL queries. I noticed by > > reading my debug logs that some of these queries would become > > interlaced, that is, one query would try to update while another would > > try to select-fetch. In turned out that this would severe the > > connection with the database, and soon after Python would crash > > altogether. > > > > So the solution was to start some sort of queue server in a separate > > thread. This queue would consist of a list, and each time the program > > would want to perform a MySQL operation, it would add it to the queue. > > Then, the caller would enter a while loop that would wait for the > > server to carry the operation. > > > > The server would run a while loop, that check the queue once every > > iteration to see if there is something to do. When it finds something, > > it would carry the entire transaction, then put the result in a > > dictionary, using an ID number assigned when the transaction was > > submitted to the queue. > > > > The "submitter" would then pick up the result, and return it to the > > original caller. > > The result is that no two queries would take place at the same time. > > > > Once I did that, my program stopped crashing, and actually has been > > running very smooth since then. In fact my progam is now very very > > close to completion. > > > > > > > > Kent: I took your suggestion and changed the program design and > > splitted the code into separate modules. It was tricky to "re-wire" > > everything together (it took me at least 4 days), but past that, I'm > > glad I did it. > > > > > > Thanks everyone for the help, it has been a great learning experience! > > > > > > Bernard > > _______________________________________________ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor