New submission from Gregory P. Smith: The subprocess module in Python 3 uses io.open(file_descriptor, mode, bufsize) to create its Popen stdout, stderr and stdin file objects.
In Python 2, it used the old os.fdopen which created an old-style python 2 file object that simply wraps libc's FILE* interface. This results in a behavior difference between Python 2 and Python 3 subprocesses as the bufsize=0 on io.open()ed files results in a RawIOBase file object whos read() or write() methods map directly to a single underlying system call. ie: In Python 3 if you Popen.read(30000) and there are only 12345 bytes in the pipe from the child, it will return 12345 bytes rather than blocking while it makes further read() syscalls until it gets enough data or EOF as it would with the libc backed file objects in Python 2. This tripped up the imaplib module in Issue17443. (since fixed by explicitly enabling buffered I/O). This behavior difference will make porting code to work on both Python 2 and 3 a bit more painful as bufsize=non-zero must be specified by the user for consistent behavior. I'd like to fix this by changing the default bufsize=0 to bufsize=io.DEFAULT_BUFFER_SIZE, but only if I can do that in 3.2 and 3.3 . If I can't, it _does_ seem like an API change, I'll just make a documentation update to mention the issue and the best practice for 2.x and 3.x compatibility. [note: this issue does not apply to my subprocess32 backport running on python 2.x because that uses python2's the old style file os.fdopen] marking release blocker to ask for comments from the 3.2 and 3.3 release managers on if i can consider changing the default subprocess.Popen bufsize parameter value or not. ---------- assignee: gregory.p.smith messages: 184718 nosy: georg.brandl, gregory.p.smith, larry priority: release blocker severity: normal status: open title: subprocess.Popen bufsize=0 parameter behaves differently in Python 3 than in 2 type: behavior versions: Python 3.2, Python 3.3, Python 3.4 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue17488> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
