Hello, For the record, in case anyone would like to know what happened with this issue...
It seems that the problem was related to the way I managed MySQL connections. There was the main thread that would query every 5 seconds the database to see if a someone wanted to abort the job running on the render node. However, the actual rendering was managed by a child thread, who would perform many MySQL transactions, depending on the output produced by the rendering software. So there were two separate threads doing MySQL queries. I noticed by reading my debug logs that some of these queries would become interlaced, that is, one query would try to update while another would try to select-fetch. In turned out that this would severe the connection with the database, and soon after Python would crash altogether. So the solution was to start some sort of queue server in a separate thread. This queue would consist of a list, and each time the program would want to perform a MySQL operation, it would add it to the queue. Then, the caller would enter a while loop that would wait for the server to carry the operation. The server would run a while loop, that check the queue once every iteration to see if there is something to do. When it finds something, it would carry the entire transaction, then put the result in a dictionary, using an ID number assigned when the transaction was submitted to the queue. The "submitter" would then pick up the result, and return it to the original caller. The result is that no two queries would take place at the same time. Once I did that, my program stopped crashing, and actually has been running very smooth since then. In fact my progam is now very very close to completion. Kent: I took your suggestion and changed the program design and splitted the code into separate modules. It was tricky to "re-wire" everything together (it took me at least 4 days), but past that, I'm glad I did it. Thanks everyone for the help, it has been a great learning experience! Bernard On 1/4/06, Bernard Lebel <[EMAIL PROTECTED]> wrote: > Hi Danny, > See [Bernard] below... > > On 1/4/06, Danny Yoo <[EMAIL PROTECTED]> wrote: > > > > > rn200.bbfxa.com Wed Jan 4 16:23:36 2006 [jobid: 9]: Get status of all > > > local jobs for this job... > > > > > > *** glibc detected *** double free or corruption: 0x09484d58 *** > > > Aborted > > > > > > Hi Bernard, > > > > Ugh. I hate memory allocation bugs. > > > > Is it reproducable? Can you run this through a debugger like GDB to find > > out where we are when this happens? A stack trace will be invaluable > > because we need to know who to blame. > > [Bernard] I am quite unfamiliar with Linux in general and a lot more > with these tools. Where can I find information about GDB? > > > > *grin* We need to know if it's > > Python that's doing this, or if it's MySQLdb, or if it's something else > > entirely... > > > > Just checking up on this: what version of MySQLdb do you have on those > > systems? I do remember some ugly memory-leaking bugs that haunted ancient > > versions of MySQLdb, so I want to make sure we're not hitting those. > > [Bernard] I am using 1.2.0, with Python 2.3.4. > > > > > > Analyzing this might goes out of Tutor's scope, so you may want to bring > > this up on a more general forum like comp.lang.python instead. > > [Bernard] Okay thanks a lot. I'll check it out, but if you have any idea... > :-) > > > Bernard > _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
