Re: help with threads
In article 5a744bd6-6b9a-4f46-97c1-bb7fd65b8...@l5g2000pra.googlegroups.com, Michael Mossey michaelmos...@gmail.com wrote: I have a simple application that needs one thread to manage networking in addition to the main thread that does the main job. It's not working right. I know hardly anything about threads, so I was hoping someone could point me in the right direction to research this. You might also consider the multiprocessing module. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ ...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons. --Aahz -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
On Aug 7, 5:03 pm, Piet van Oostrum p...@cs.uu.nl wrote: Michael Mossey michaelmos...@gmail.com (MM) wrote: MM Ah yes, that explains it. Some of these long computations are done in MM pure C, so I'm sure the GIL is not being released. Is that C code under your own control? Or at least the glue from Python to C? In that case, and if the C code is not manipulating Python objects or something in the Python interpreter, it could be changed to release the GIL during the computation. That's also how Numpy does it, IIRC. -- Piet van Oostrum p...@cs.uu.nl URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org I don't have any control over this code, and it's easier to solve my problem in other ways. I just put a sleep() call between calls to the C library, and that gives the network thread enough responsiveness for my particular task. I am grateful, anyway, to understand why this kind of thing happens. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
Ah yes, that explains it. Some of these long computations are done in pure C, so I'm sure the GIL is not being released. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
On Friday 07 August 2009 05:02:10 Michael Mossey wrote: Hello, My problem is that in some cases, the network thread appears to stop, while the main thread is doing a long computation. Is this computation done in pure python or are you calling some underlying thing in C? I would be surprised if a pure python computation thread were to hog the cpu - have you been able to figure out what the computation thread is actually doing when the hogging occurs? I'm hoping someone can give me a general idea what to read about. For example, under what conditions does a thread stop running? Can other threads take priority? Are there certain operations that block other threads (such as disk access)? AFAIK python threads are all at the same level of priority, and the running thread is interrupted every N python bytecode instructions, so what you are experiencing should not happen. Try to figure out, if you can, where the computation thread is spending its time. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
Michael Mossey wrote: Hello, I have a simple application that needs one thread to manage networking in addition to the main thread that does the main job. It's not working right. I know hardly anything about threads, so I was hoping someone could point me in the right direction to research this. Basically, I have a program that does some computational work, and also conveys its status to a monitor program elsewhere on the network via sockets. I wanted to use a thread to manage the networking so that the main program can run without regard to networking (i.e. they would be asynchronous). So the network thread loops and calls select. My problem is that in some cases, the network thread appears to stop, while the main thread is doing a long computation. I'm hoping someone can give me a general idea what to read about. For example, under what conditions does a thread stop running? Can other threads take priority? Are there certain operations that block other threads (such as disk access)? Thanks, Mike I may be a little out of subject, anyway *if* you are writing also the server side program, you should really take a look at http://docs.python.org/library/xmlrpclib.html or http://pyro.sourceforge.net/ ... and forget about network coding :o) To help you for your initial question, exceptions in thread are not propagated to the main thread, so when an unhanded exception occurs in your thread, it will just stop. Maybe this is what is happening. A quick workaround is to embed all you threaded code in a try except clause and log the exception before re-raising it. You should only do that for debugging purpose. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
Michael Mossey michaelmos...@gmail.com (MM) wrote: MM Ah yes, that explains it. Some of these long computations are done in MM pure C, so I'm sure the GIL is not being released. Is that C code under your own control? Or at least the glue from Python to C? In that case, and if the C code is not manipulating Python objects or something in the Python interpreter, it could be changed to release the GIL during the computation. That's also how Numpy does it, IIRC. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: help with threads
Hallo Michael, I have a simple application that needs one thread to manage networking in addition to the main thread that does the main job. It's not working right. I know hardly anything about threads, so I was hoping someone could point me in the right direction to research this. Basically, I have a program that does some computational work, and also conveys its status to a monitor program elsewhere on the network via sockets. I wanted to use a thread to manage the networking so that the main program can run without regard to networking (i.e. they would be asynchronous). So the network thread loops and calls select. My problem is that in some cases, the network thread appears to stop, while the main thread is doing a long computation. I'm hoping someone can give me a general idea what to read about. For example, under what conditions does a thread stop running? Can other threads take priority? Are there certain operations that block other threads (such as disk access)? May I suggest: http://www.dabeaz.com/python/GIL.pdf HTH, Loïc -- My Blog: http://www.domaigne.com/blog There is only one problem with common sense; it’s not very common. -- Milt Bryce -- http://mail.python.org/mailman/listinfo/python-list
