On 26-04-2005 22:24, "Adam R. B. Jack" <[EMAIL PROTECTED]> wrote: > This looks 'interesting' :
Ooh, then I'd better take a look ;) > Kill process group (anything launched by PID 21107) > Kill process group 18226 (anything launched by PID 21107) [from 21109] > > for: > > pgrpID=os.getpgid(pid) > log.warn('Kill process group %s (anything launched by PID %s) [from > %s]' \ > % (pgrpID, pid, os.getpid())) > > It is unlikely that 21109 is the 'parent' of 21107. (Even Brutus isn't that > busy :-) Hmm, maybe it is that the timer is running in a sub-process (some > Python impl detail?), That could be the case. The threading module builds on the thread module which is in native code. > and the main process was 18226. That might be > possible, seeing as the lock file contains 18231. > > Mind you, this leads to ... > > Looking at the code, I see no 'setpgid', which is disturbing (to say the > least). Well, there is a setpgrp. That's setpgid(0,0), or something. > The logic of this kill means the forked child needs to place itself > into it's own process group. That can never work properly. A killpg() might lead to the forked child being killed early on, meaning that the rest of the killpg() could be aborted and none of the children is ever killed. I.e. The code that does the actual killing cannot be executed from a process that is in the process group that is being killed; it needs to remain active for however long it takes the os to do all the killing. > Without that, the parent (main Gump) is likely > a equal target for termination. You'll just need to create a process group in another way, making sure it doesn't contain the child. In the Gump3 code I just create a long-running process whose sole purpose is to keep a process group around. An alternative is to put all direct children in a process group belonging to the parent, and all their children in a process group assocated with the process. This is what tools like ie bash do, but its way more bookkeeping than we need to do. > I (now) suspect that is the problem. I > certainly makes sense w/ the output we see. > > [BTW: Somehow when I ported from the test script, python/misc/pgrp.py I lost > this 'minor' detail. Hopefully with Gump3 we all have better luck w/ > repeatably unit testing "integration" code.] The code in Gump3 responsible for all this is much much simpler, because: 1) it uses the subprocess module which simplifies things like error code handling 2) it uses a single global process group for all of gump instead of one for each command 3) it doesn't use timeouts or any kind of multithreading but only kills processes before system exit 4) it doesn't bother writing "exec" files 5) it does one thing only (we hope it does it well :-D) It shouldn't be too hard to adapt that code into gump2. Just replace executeIntoResult: from gump.util.executor import Popen def execute(cmd,tmp=dir.tmp): res = command.CmdResult(cmd) return executeIntoResult(cmd,res,tmp) def executeIntoResult(cmd,result,tmp=dir.tmp): execString = cmd.formatCommandLine() p = Popen(command,stdout=PIPE,stderr=STDOUT) output = p.communicate()[0] result.exit_code = p.exitcode outputFile = \ os.path.abspath(os.path.join(tmp,gumpSafeName(cmd.name)+'.txt')) if os.path.exists(outputFile): os.remove(outputFile) o = open(outputFile,'w') o.write(output) o.close() And additionally reap children before system exit: # rigorously clean up our child processes try: timeout = 300 try: log.debug("Cleaning up child processes. This may take up to a little over %s seconds." % (timeout+100)) except: pass from gump.util.executor import clean_up_processes clean_up_processes(timeout) except: try: log.exception("Error cleaning up child processes!") except: pass And you can get rid of everything else in launcher.py, saving a few hundred lines of complex stuff! I think. It's not tested. I don't fully understand the code. I'm in the dark here. I'm no expert. It doesn't kill stuff on windows. Try at your own risk. Don't blame me if it blows up. ***insert your favorite disclaimer here*** G'night! - LSD --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]