On 26-04-2005 22:24, "Adam R. B. Jack" <[EMAIL PROTECTED]> wrote:
> This looks 'interesting' :
Ooh, then I'd better take a look ;)
> Kill process group (anything launched by PID 21107)
> Kill process group 18226 (anything launched by PID 21107) [from 21109]
>
> for:
>
> pgrpID=os.getpgid(pid)
> log.warn('Kill process group %s (anything launched by PID %s) [from
> %s]' \
> % (pgrpID, pid, os.getpid()))
>
> It is unlikely that 21109 is the 'parent' of 21107. (Even Brutus isn't that
> busy :-) Hmm, maybe it is that the timer is running in a sub-process (some
> Python impl detail?),
That could be the case. The threading module builds on the thread module
which is in native code.
> and the main process was 18226. That might be
> possible, seeing as the lock file contains 18231.
>
> Mind you, this leads to ...
>
> Looking at the code, I see no 'setpgid', which is disturbing (to say the
> least).
Well, there is a setpgrp. That's setpgid(0,0), or something.
> The logic of this kill means the forked child needs to place itself
> into it's own process group.
That can never work properly. A killpg() might lead to the forked child
being killed early on, meaning that the rest of the killpg() could be
aborted and none of the children is ever killed. I.e. The code that does the
actual killing cannot be executed from a process that is in the process
group that is being killed; it needs to remain active for however long it
takes the os to do all the killing.
> Without that, the parent (main Gump) is likely
> a equal target for termination.
You'll just need to create a process group in another way, making sure it
doesn't contain the child. In the Gump3 code I just create a long-running
process whose sole purpose is to keep a process group around.
An alternative is to put all direct children in a process group belonging to
the parent, and all their children in a process group assocated with the
process. This is what tools like ie bash do, but its way more bookkeeping
than we need to do.
> I (now) suspect that is the problem. I
> certainly makes sense w/ the output we see.
>
> [BTW: Somehow when I ported from the test script, python/misc/pgrp.py I lost
> this 'minor' detail. Hopefully with Gump3 we all have better luck w/
> repeatably unit testing "integration" code.]
The code in Gump3 responsible for all this is much much simpler, because:
1) it uses the subprocess module which simplifies things like error code
handling
2) it uses a single global process group for all of gump instead of one
for each command
3) it doesn't use timeouts or any kind of multithreading but only kills
processes before system exit
4) it doesn't bother writing "exec" files
5) it does one thing only (we hope it does it well :-D)
It shouldn't be too hard to adapt that code into gump2. Just replace
executeIntoResult:
from gump.util.executor import Popen
def execute(cmd,tmp=dir.tmp):
res = command.CmdResult(cmd)
return executeIntoResult(cmd,res,tmp)
def executeIntoResult(cmd,result,tmp=dir.tmp):
execString = cmd.formatCommandLine()
p = Popen(command,stdout=PIPE,stderr=STDOUT)
output = p.communicate()[0]
result.exit_code = p.exitcode
outputFile = \
os.path.abspath(os.path.join(tmp,gumpSafeName(cmd.name)+'.txt'))
if os.path.exists(outputFile): os.remove(outputFile)
o = open(outputFile,'w')
o.write(output)
o.close()
And additionally reap children before system exit:
# rigorously clean up our child processes
try:
timeout = 300
try:
log.debug("Cleaning up child processes. This may take up to
a little over %s seconds." % (timeout+100))
except:
pass
from gump.util.executor import clean_up_processes
clean_up_processes(timeout)
except:
try:
log.exception("Error cleaning up child processes!")
except:
pass
And you can get rid of everything else in launcher.py, saving a few hundred
lines of complex stuff! I think. It's not tested. I don't fully understand
the code. I'm in the dark here. I'm no expert. It doesn't kill stuff on
windows. Try at your own risk. Don't blame me if it blows up. ***insert your
favorite disclaimer here***
G'night!
- LSD
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
