Re: os.wait() losing child?

2007-07-13 Thread Jason Zheng
Hrvoje Niksic wrote: > Jason Zheng <[EMAIL PROTECTED]> writes: >> I'm concerned about portability of my code. It will be run on >> multiple machines with mixed Python 2.4 and 2.5 environments. > > I don't think there is a really clean way to handle this. I think the following might just work, alb

Re: os.wait() losing child?

2007-07-13 Thread Hrvoje Niksic
Jason Zheng <[EMAIL PROTECTED]> writes: >>> Nope it still doesn't work. I'm running python 2.4.4, tho. >> That explains it, then, and also why greg's code didn't work. You >> still have the option to try to run 2.5's subprocess.py under 2.4. > Is it more convenient to just inherit the Popen class

Re: os.wait() losing child?

2007-07-13 Thread Jason Zheng
Hrvoje Niksic wrote: > Jason Zheng <[EMAIL PROTECTED]> writes: > >> Hrvoje Niksic wrote: greg wrote: >>> Actually, it's not that bad. _cleanup only polls the instances that >>> are no longer referenced by user code, but still running. If you hang >>> on to Popen instances, they won't be add

Re: os.wait() losing child?

2007-07-13 Thread Hrvoje Niksic
Jason Zheng <[EMAIL PROTECTED]> writes: > Hrvoje Niksic wrote: >>> greg wrote: >> Actually, it's not that bad. _cleanup only polls the instances that >> are no longer referenced by user code, but still running. If you hang >> on to Popen instances, they won't be added to _active, and __init__ >>

Re: os.wait() losing child?

2007-07-12 Thread Hrvoje Niksic
Nick Craig-Wood <[EMAIL PROTECTED]> writes: >> This can still be a problem for applications that call wait in a >> dedicated thread, but the program can always ignore the processes >> it doesn't know anything about. > > Ignoring them isn't good enough because it means that the bit of > code whi

Re: os.wait() losing child?

2007-07-12 Thread Jason Zheng
Nick Craig-Wood wrote: > Sure! > > You could get rid of this by sleeping until a SIGCHLD arrived maybe. Yah, I could also just dump Popen class and use fork(). But then what's the point of having an abstraction layer any more? >> This can still be a problem for applications that call wait in a

Re: os.wait() losing child?

2007-07-12 Thread Matthew Woodcraft
Jason Zheng <[EMAIL PROTECTED]> wrote: >Hrvoje Niksic wrote: >> Actually, it's not that bad. _cleanup only polls the instances that >> are no longer referenced by user code, but still running. If you hang >> on to Popen instances, they won't be added to _active, and __init__ >> won't reap them (

Re: os.wait() losing child?

2007-07-12 Thread Nick Craig-Wood
Hrvoje Niksic <[EMAIL PROTECTED]> wrote: > Nick Craig-Wood <[EMAIL PROTECTED]> writes: > > >> I think your polling way works; it seems there no other way around this > >> problem other than polling or extending Popen class. > > > > I think polling is probably the right way of doing it... > >

Re: os.wait() losing child?

2007-07-12 Thread Jason Zheng
Hrvoje Niksic wrote: >> greg wrote: > > Actually, it's not that bad. _cleanup only polls the instances that > are no longer referenced by user code, but still running. If you hang > on to Popen instances, they won't be added to _active, and __init__ > won't reap them (_active is only populated f

Re: os.wait() losing child?

2007-07-12 Thread Hrvoje Niksic
Jason Zheng <[EMAIL PROTECTED]> writes: > greg wrote: >> Jason Zheng wrote: >>> Hate to reply to my own thread, but this is the working program >>> that can demonstrate what I posted earlier: >> I've figured out what's going on. The Popen class has a >> __del__ method which does a non-blocking wai

Re: os.wait() losing child?

2007-07-12 Thread Hrvoje Niksic
Nick Craig-Wood <[EMAIL PROTECTED]> writes: >> I think your polling way works; it seems there no other way around this >> problem other than polling or extending Popen class. > > I think polling is probably the right way of doing it... It requires the program to wake up every 0.1s to poll for

Re: os.wait() losing child?

2007-07-12 Thread Nick Craig-Wood
Jason Zheng <[EMAIL PROTECTED]> wrote: > Nick Craig-Wood wrote: > > The problem you are having is you are letting Popen do half the job > > and doing the other half yourself. > > Except that I never wanted Popen to do any thread management for me to > begin with. Popen class has advertised its

Re: os.wait() losing child?

2007-07-11 Thread Jason Zheng
Nick Craig-Wood wrote: > The problem you are having is you are letting Popen do half the job > and doing the other half yourself. Except that I never wanted Popen to do any thread management for me to begin with. Popen class has advertised itself as a replacement for os.popen, popen2, popen4, an

Re: os.wait() losing child?

2007-07-11 Thread Nick Craig-Wood
Jason Zheng <[EMAIL PROTECTED]> wrote: > greg wrote: > > Jason Zheng wrote: > >> Hate to reply to my own thread, but this is the working program that > >> can demonstrate what I posted earlier: > > > > I've figured out what's going on. The Popen class has a > > __del__ method which does a non-bl

Re: os.wait() losing child?

2007-07-11 Thread Jason Zheng
Matthew Woodcraft wrote: > greg <[EMAIL PROTECTED]> wrote: >> I've figured out what's going on. The Popen class has a >> __del__ method which does a non-blocking wait of its own. >> So you need to keep the Popen instance for each subprocess >> alive until your wait call has cleaned it up. > > I d

Re: os.wait() losing child?

2007-07-11 Thread Jason Zheng
greg wrote: > Jason Zheng wrote: >> Hate to reply to my own thread, but this is the working program that >> can demonstrate what I posted earlier: > > I've figured out what's going on. The Popen class has a > __del__ method which does a non-blocking wait of its own. > So you need to keep the Pope

Re: os.wait() losing child?

2007-07-11 Thread Matthew Woodcraft
greg <[EMAIL PROTECTED]> wrote: > I've figured out what's going on. The Popen class has a > __del__ method which does a non-blocking wait of its own. > So you need to keep the Popen instance for each subprocess > alive until your wait call has cleaned it up. I don't think this will be enough for

Re: os.wait() losing child?

2007-07-11 Thread Jason Zheng
Greg, That explains it! Thanks a lot for your help. I guess this is something they do to prevent zombie threads? ~Jason greg wrote: > Jason Zheng wrote: >> Hate to reply to my own thread, but this is the working program that >> can demonstrate what I posted earlier: > > I've figured out what'

Re: os.wait() losing child?

2007-07-11 Thread greg
Jason Zheng wrote: > Hate to reply to my own thread, but this is the working program that can > demonstrate what I posted earlier: I've figured out what's going on. The Popen class has a __del__ method which does a non-blocking wait of its own. So you need to keep the Popen instance for each subp

Re: os.wait() losing child?

2007-07-10 Thread Jason Zheng
greg wrote: > Jason Zheng wrote: >> while (True): >> pid = os.wait() >> ... >> if (someCondition): >> break > > ... > > Are you sure that someCondition() always becomes true > when the list of pids is empty? If not, you may end > up making more wait() calls than there are children. >

Re: os.wait() losing child?

2007-07-10 Thread Jason Zheng
Hate to reply to my own thread, but this is the working program that can demonstrate what I posted earlier: import os from subprocess import Popen pids = {} counts = [0,0,0] for i in xrange(3): p = Popen('sleep 1', shell=True, cwd='/home', stdout=file(os.devnull,'w')) pids[p.pid] = i

Re: os.wait() losing child?

2007-07-10 Thread greg
Jason Zheng wrote: > while (True): > pid = os.wait() > ... > if (someCondition): > break > ... Are you sure that someCondition() always becomes true when the list of pids is empty? If not, you may end up making more wait() calls than there are children. It might be safer to do wh

os.wait() losing child?

2007-07-10 Thread Jason Zheng
This may be a silly question but is possible for os.wait() to lose track of child processes? I'm running Python 2.4.4 on Linux kernel 2.6.20 (i686), gcc4.1.1, and glibc-2.5. Here's what happened in my situation. I first created a few child processes with Popen, then in a while(True) loop wait o