Bugs item #886488, was opened at 2004-01-28 15:28 Message generated for change (Comment added) made by rupole You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=886488&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Frank Terhaar-Yonkers (frankty) Assigned to: Nobody/Anonymous (nobody) Summary: WinPython 2.3.3 crashes using popen2 to spawn lots of child Initial Comment: WinPython 2.3.3 crashes when using popen2 to spawn *lots* of children > 100. Running the test case on either W2k or Wxp causes Python 2.3.3 to crash. Looking at the drwatson file, it appears to be a race condition where the child is in the process of crashing, while the popen2 code in posixmodule.c is attempting to PyFile_SetBufSize() on one of the pipes. Evil juju in ntdll.dll .. I've no idea what system resource is being exhausted (heap memory?) causing the crash. If you look at the hacks in posixmodule.c, the Python crash can be eliminated by simply commenting out the calls to PyFile_SetBufSize. There is still the problem of resource exhaustion. To gain more child processes I included a hack such that with the setting of an env-var (NOPYCMD) popen2 will NOT use cmd.exe /c to run a child, but simply run the child command as passed (yeah, I know that is has to be an absolute path or in the CWD then ..). I'm sure there is a better way to eliminate the cmd.exe /c - like maybe a new method: popen2.popenXabspath .. In any case, it appears we have an NT kernel bug combined with an inefficient way of doing popen2 that is making a mess. I started doing some exploration of heap_chunk_size to attack to resource issue but haven't gotten very far. As for *why* I'm doing this, it's because we're using Pyton as a system simulator tool to emulate a large pile of PCs. The actual work *must* be done by external .exe code that cannot be changed. The attached zip contains a test case, the posixmodule hack, drwatson file and a sample program written in VisC++ which does basically the same as the Python test case. cheers, - Frank ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-06-05 21:40 Message: Logged In: YES user_id=771074 Originator: NO The code for the popen* functions doesn't check the return value of _fdopen, which will return null when the limit of open stdio streams is reached. A similar issue exists in the win32pipe module, which uses almost exactly the same code. See Pywin32 bug 1731778 ( http://sourceforge.net/tracker/?func=detail&atid=551954&aid=1731778&group_id=78018 ) for more details. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=886488&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
