[issue10482] subprocess and deadlock avoidance

Change by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue10482] subprocess and deadlock avoidance

Martin Panter added the comment: Related: Issue 1260171, essentially proposing streaming readers and writers for communicate() instead of fixed buffers, but without using OS threads. -- ___ Python tracker rep...@bugs.python.org

[issue10482] subprocess and deadlock avoidance

Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___ ___ Python-bugs-list

[issue10482] subprocess and deadlock avoidance

Changes by Akira Li 4kir4...@gmail.com: -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___ ___ Python-bugs-list mailing list

[issue10482] subprocess and deadlock avoidance

Mark Lawrence added the comment: @Glenn can you provide a formal patch so we can take this forward? -- nosy: +BreamoreBoy versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482

[issue10482] subprocess and deadlock avoidance

Changes by Christian Heimes li...@cheimes.de: -- stage: - needs patch versions: +Python 3.4 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___

[issue10482] subprocess and deadlock avoidance

Changes by Ross Lagerwall rosslagerw...@gmail.com: -- nosy: +rosslagerwall ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___ ___

[issue10482] subprocess and deadlock avoidance

Changes by Jon Brandvein jon.brandv...@gmail.com: -- nosy: +brandj ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___ ___ Python-bugs-list

[issue10482] subprocess and deadlock avoidance

Glenn Linderman v+pyt...@g.nevcal.com added the comment: Looking at the code the way I've used it in my modified server.py: stderr = [] stderr_thread = threading.Thread(target=self._readerthread, args=(p.stderr, stderr))

[issue10482] subprocess and deadlock avoidance

Nick Coghlan ncogh...@gmail.com added the comment: Or various incarnations of functools.partial applied to subprocess.Popen. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___

[issue10482] subprocess and deadlock avoidance

Nick Coghlan ncogh...@gmail.com added the comment: The general idea is sound. My work colleagues have certainly had to implement their own reader/writer thread equivalents to keep subprocess from blocking. It makes sense to provide more robust public support for such techniques in process

[issue10482] subprocess and deadlock avoidance

Glenn Linderman v+pyt...@g.nevcal.com added the comment: Here's an updated _writerthread idea that handles more cases: def _writerthread(self, fhr, fhw, length=None): if length is None: flag = True while flag: buf = fhr.read( 512 )

[issue10482] subprocess and deadlock avoidance

Glenn Linderman v+pyt...@g.nevcal.com added the comment: Sorry, left some extraneous code in the last message, here is the right code: def _writerthread(self, fhr, fhw, length=None): if length is None: flag = True while flag: buf = fhr.read(

[issue10482] subprocess and deadlock avoidance

Glenn Linderman v+pyt...@g.nevcal.com added the comment: So I've experimented a bit, and it looks like simply exposing ._readerthread as an external API would handle the buffered case for stdout or stderr. For http.server CGI scripts, I think it is fine to buffer stderr, as it should not be

[issue10482] subprocess and deadlock avoidance

New submission from Glenn Linderman v+pyt...@g.nevcal.com: .communicate is a nice API for programs that produce little output, and can be buffered. While that may cover a wide range of uses, it doesn't cover launching CGI programs, such as is done in http.server. Now there are nice warnings