[issue12623] "universal newlines" subprocess support broken with select- and poll-based communicate()

Changes by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___

[issue12623] "universal newlines" subprocess support broken with select- and poll-based communicate()

Martin Panter added the comment: . Summary: There was originally a bug, but it has been fixed. At best, we leave this open to work on including Andrew’s patch. Andrew’s patch adds a modified copy of test_universal_newlines_communicate_stdin(). But it does not look correct, and would fail on

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Mark Lawrence added the comment: Can we have an update on this please, I'm assuming that the priority should now be marked as normal? -- nosy: +BreamoreBoy versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3 ___ Python tracker

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: I like to leave fixes to 3.4. Any change can produce side-effects, which can be nightmare for upcoming release candidate. Sure, Georg will share my opinion. Though absence '\n' - '\r\n' for input if OS is Windows and universal_newlines=True is not good.

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Georg Brandl added the comment: The main question: can be fix applied to 3.3 or it can wait for 3.4? 3.2 has the same problem BTW. I don't see any fix attached :) If it is a bug, it can be fixed before 3.3rc1, no need for release blocker status. -- priority: release blocker -

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: Antoine, can you explain why subprocess support for universal_newlines is broken? As I can see tests for universal_newlines passed and these looks correct. In general I like your idea to get rid of os.write, but maybe that patch should be landed in 3.4? If

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Chris Jerdonek added the comment: Andrew, I'm not sure if this is the issue, but it seems like the only tests in which input is passed to communicate() with universal newlines is when stdin is the only PIPE, i.e.: def test_universal_newlines_communicate_stdin(self): # universal

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Antoine Pitrou added the comment: As Chris said. Look at the POSIX version of _communicate(), nowhere is input given the newlines/encoding treatment. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: I've pushed test for piping stdin, stdout and and stderr: 4af2c0687970 What other test we should to add? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: Оор. I see. Pushing to communicate input with \r (see attached patch) produces the error. Will work on fixing. Thanks. -- keywords: +patch Added file: http://bugs.python.org/file26837/issue12623_failing_test.diff

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: I like to set status for the issue to Release blocker if Georg Brandl agree with that. -- nosy: +georg.brandl priority: normal - release blocker ___ Python tracker rep...@bugs.python.org

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Antoine Pitrou added the comment: Оор. I see. Pushing to communicate input with \r (see attached patch) produces the error. Hmm, no, it's the reverse. Pushing input with \n should produce b\r\n on the other side, under Windows. -- ___ Python

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Antoine Pitrou added the comment: Well, it's not a regression and it may be slightly delicate, so I don't think it should be a release blocker. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: The main question: can be fix applied to 3.3 or it can wait for 3.4? 3.2 has the same problem BTW. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Chris Jerdonek added the comment: Pushing to communicate input with \r (see attached patch) produces the error. Is this a supported use case? In universal newlines, stdin line endings are supposed to be \n. From the subprocess documentation: For stdin, line ending characters '\n' in the

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Andrew Svetlov added the comment: The real test should to put '\n' and subprocess should get '\r\n', right? Looking the code this test will fail on Windows. I cannot check it just now but looks like this is the problem. -- ___ Python tracker

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Chris Jerdonek added the comment: Looking the code this test will fail on Windows. I cannot check it just now but looks like this is the problem. Would it be possible to do something like the following to check this on a non-Windows machine (since the Python implementation of io respects

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623 ___ ___

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

Changes by Chris Jerdonek chris.jerdo...@gmail.com: -- nosy: +cjerdonek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12623 ___ ___

[issue12623] universal newlines subprocess support broken with select- and poll-based communicate()

New submission from Antoine Pitrou pit...@free.fr: The select() and poll() loop implementations of Popen.communicate() call os.write() instead of the write() method on the stdin pipe, meaning any newline translation *and* unicode-to-bytes encoding step is skipped. To use the write() method on