[issue9929] subprocess.Popen unbuffered not work
Ross Lagerwall rosslagerw...@gmail.com added the comment: Unbuffered subprocess was fixed in 1dc52ecb8949 Closing this as a duplicate of #11459. -- nosy: +rosslagerwall resolution: - duplicate status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: I implemented msg117279 with v2 patch. Can I commit it? # If your are already working on this issue, please ignore # my patch. -- Added file: http://bugs.python.org/file19280/py3k_fix_unbuffered_in_subprocess_v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: Umm, v2 patch broke test_subprocess. I'll repost v3 patch after removing bufsize=0 universal_newlines check. == ERROR: test_universal_newlines (__main__.ProcessTestCase) -- Traceback (most recent call last): File e:\python-dev\py3k\lib\test\test_subprocess.py, line 422, in test_unive rsal_newlines universal_newlines=1) File e:\python-dev\py3k\lib\subprocess.py, line 621, in __init__ raise ValueError(cannot use bufsize=0 with universal newlines) ValueError: cannot use bufsize=0 with universal newlines == ERROR: test_universal_newlines_communicate (__main__.ProcessTestCase) -- Traceback (most recent call last): File e:\python-dev\py3k\lib\test\test_subprocess.py, line 442, in test_unive rsal_newlines_communicate universal_newlines=1) File e:\python-dev\py3k\lib\subprocess.py, line 621, in __init__ raise ValueError(cannot use bufsize=0 with universal newlines) ValueError: cannot use bufsize=0 with universal newlines -- Added file: http://bugs.python.org/file19281/py3k_fix_unbuffered_in_subprocess_v3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
R. David Murray rdmur...@bitdance.com added the comment: But if line buffering doesn't work, disabling buffering on stdout/stderr does have a functional consequence: it allows process output to appear as generated instead of coming in chunks when the buffer is full. Of course, I could be completely misunderstanding what you are talking about :) -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Antoine Pitrou pit...@free.fr added the comment: But if line buffering doesn't work, disabling buffering on stdout/stderr does have a functional consequence: it allows process output to appear as generated instead of coming in chunks when the buffer is full Yes, sorry, I had it backwards. It's buffering on stdin which doesn't make any functional difference (whether it's buffered or not, you always get data as soon as it arrives). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Antoine Pitrou pit...@free.fr added the comment: But if line buffering doesn't work, disabling buffering on stdout/stderr does have a functional consequence: it allows process output to appear as generated instead of coming in chunks when the buffer is full Yes, sorry, I had it backwards. It's buffering on stdin which doesn't make any functional difference (whether it's buffered or not, you always get data as soon as it arrives). Actually, I had it right (you confused me :-)). In the context of subprocess, you write to the child's stdin pipe, and you read from the child's stdout and stderr pipes. So, whether or not you buffer the reads from stdout and stderr pipes doesn't make a difference (except in performance): as soon as the child outputs a single byte, it becomes available for the parent. But if you buffer the writes to stdin, the child process will see data arrive only when the buffer is flushed. Here is the relevant code in subprocess.py: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if self.universal_newlines: self.stdin = io.TextIOWrapper(self.stdin) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if universal_newlines: self.stdout = io.TextIOWrapper(self.stdout) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if universal_newlines: self.stderr = io.TextIOWrapper(self.stderr) Only stdin is opened in write mode. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: Antoine, can you create the patch? I'm not familiar with IO. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Antoine Pitrou pit...@free.fr added the comment: You are right that it doesn't work anymore (under Linux too). Your patch is almost ok. It should raise a ValueError if bufsize=0 and universal_newlines is true (the two can't be satisfied together, since universal newlines implies buffering). It would also be better to add a test in test_subprocess. As a sidenote, bufsize=1 (line buffering) doesn't work anymore either, and this can't be fixed without breaking compatibility. -- components: +Library (Lib) -Windows nosy: +gregory.p.smith, pitrou title: subprocess.Popen unbuffered not work (windows) - subprocess.Popen unbuffered not work type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work
Antoine Pitrou pit...@free.fr added the comment: As another sidenote, `bufsize` should only be relevant for stdin. For stdout and stderr, disabling buffering will only reduce performance, not add functionality. So I would suggest only setting buffering on stding. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work (windows)
New submission from Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp: Following script hangs on Python3.x. from subprocess import * import sys p = Popen([sys.executable, -c, import sys; print(sys.stdin.read(1))], stdin=PIPE) p.stdin.write(b'x') p.wait() This is because unbuffered functionality of subprocess.Popen is disabled. Is this still needed? I confirmed test_subprocess passes with the attached patch. -- components: Windows files: py3k_fix_unbuffered_in_subprocess.patch keywords: patch messages: 117213 nosy: ocean-city priority: normal severity: normal status: open title: subprocess.Popen unbuffered not work (windows) versions: Python 3.1, Python 3.2 Added file: http://bugs.python.org/file18980/py3k_fix_unbuffered_in_subprocess.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9929] subprocess.Popen unbuffered not work (windows)
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: Interestingly, following code also hangs. Maybe I'm misunderstanding the meaning of *bufsize* from subprocess import * import sys p = Popen([sys.executable, -c, import sys; print(sys.stdin.read(1))], stdin=PIPE) p.stdin.write(b'xx') # size 2 p.wait() -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9929 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
