[issue10963] "subprocess" can raise OSError (EPIPE) when communicating with short-lived processes

[Martin Panter]

Martin Panter added the comment:

FTR Issue 26372 has been opened about Serhiy’s bug with stdin.close() raising 
EPIPE.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

self.stdout.close() also can fail with EPIPE or EINVAL if the stream is 
buffered (text streams are buffered). I.e. communicate() in text mode can loss 
the data. See also issue21619.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Roundup Robot]

Roundup Robot devnull@devnull added the comment:

New changeset c10d55c51d81 by Ross Lagerwall in branch '2.7':
Issue #10963: Ensure that subprocess.communicate() never raises EPIPE.
http://hg.python.org/cpython/rev/c10d55c51d81

New changeset 158495d49f58 by Ross Lagerwall in branch '3.1':
Issue #10963: Ensure that subprocess.communicate() never raises EPIPE.
http://hg.python.org/cpython/rev/158495d49f58

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Ross Lagerwall]

Ross Lagerwall rosslagerw...@gmail.com added the comment:

Committed, thanks.

--
assignee:  - rosslagerwall
resolution:  - fixed
stage: patch review - committed/rejected
status: open - closed
type: feature request - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 I think it's best to remove all this inconsistency and fix it so that
 EPIPE is never generated and then backport it to 2.7, 3.1, 3.2.
 
 Attached is a patch which fixes it for poll, select, windows and adds
 two tests.

The patch looks good to me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Ross Lagerwall]

Ross Lagerwall rosslagerw...@gmail.com added the comment:

Marked #6457 as a duplicate. See #6457 for more discussion.

--
nosy: +Yaniv.Aknin, amaury.forgeotdarc, dwalczak, mcrute

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Ross Lagerwall]

Ross Lagerwall rosslagerw...@gmail.com added the comment:

I'd argue that this is not a feature request but a bug.

I did some testing of this issue and the problem is that EPIPE is only 
generated sometimes depending on the time the process takes to finish, the size 
of the data sent, the underlying mechanism used (select vs poll) and the 
whether anything happens between the starting of the process and the 
communicate() call.

Here are some results (on my PC, I think some of these will vary on others):
With poll:
 [sys.executable, 'c', 'pass']- no error
 ['dd', 'option=bad'] - varies between EPIPE and no error
 ['dd', 'option=bad'], sleep(1) - EPIPE

With select:
 [sys.executable, 'c', 'pass']- EPIPE
 ['dd', 'option=bad'] - EPIPE
 ['dd', 'option=bad'], sleep(1) - EPIPE

Only stdin (neither select or poll):
 [sys.executable, 'c', 'pass']- no error (error in 2.7)
 ['dd', 'option=bad'] - no error (error in 2.7)
 ['dd', 'option=bad'], sleep(1) - EPIPE

(all of my tests appear to fail on Windows, they also generate EINVAL sometimes)

I think it's best to remove all this inconsistency and fix it so that EPIPE is 
never generated and then backport it to 2.7, 3.1, 3.2.

Attached is a patch which fixes it for poll, select, windows and adds two tests.

--
nosy: +rosslagerwall
Added file: http://bugs.python.org/file21491/10963.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Federico Simoncelli]

Federico Simoncelli fsimo...@redhat.com added the comment:

I agree they are orthogonal. The problem is that Popen.communicate doesn't 
support a way to ignore the exception and keep reading from stdout and sterr 
until the process dies.
When the child closes the stdin its (error/debug/info) messages on stout/sterr 
will be lost.
I am not just concerned about checking both the return-code and this exception, 
I am also worried about losing important information.

--
nosy: +simon3z

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

You are right, this is suboptimal. It would also be a behaviour change from 
currently, but it seems beneficial.

--
nosy: +gregory.p.smith
type:  - feature request

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy: +giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Dave Malcolm]

New submission from Dave Malcolm dmalc...@redhat.com:

If we start a short-lived process which finishes before we begin communicating 
with it (e.g. by crashing), we can receive a SIGPIPE due to the receiving 
process no longer existing.  This becomes an EPIPE, which becomes an:
  OSError: [Errno 32] Broken pipe

Arguably this is a bug; if the subprocess could crash, the user currently has 
to check for it by both monitoring the returncode _and_ catching OSError (then 
examining for this specific errno), which seems ugly to me.

I'm attaching a patch for subprocess which handles this case, masking the 
OSError within the Popen implementation, so that you have to test for it within 
the returncode, in one place, rather than wrap every call with a try/except.

It could be argued that this is incorrect, as it masks under-reads of stdin by 
the subprocess.  However I believe a sanely-written subprocess ought to 
indicate success/failure back with its return code.

This was originally filed downstream within the Red Hat bugzilla instance as:
  https://bugzilla.redhat.com/show_bug.cgi?id=667431

The handler part of the patch is based on a patch attached there by Federico 
Simoncelli; I don't know if he has signed a PSF contributor agreement, however 
the size of the patch is sufficiently small that I suspect it's not 
copyrightable, and I've somewhat rewritten it since; I wrote the unit test.

--
components: Library (Lib)
files: py3k-handle-EPIPE-for-short-lived-subprocesses-2011-01-20-001.patch
keywords: patch
messages: 126643
nosy: dmalcolm
priority: normal
severity: normal
stage: patch review
status: open
title: subprocess can raise OSError (EPIPE) when communicating with 
short-lived processes
versions: Python 3.3
Added file: 
http://bugs.python.org/file20469/py3k-handle-EPIPE-for-short-lived-subprocesses-2011-01-20-001.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10963] subprocess can raise OSError (EPIPE) when communicating with short-lived processes

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 It could be argued that this is incorrect, as it masks under-reads of
 stdin by the subprocess.  However I believe a sanely-written subprocess 
 ought to indicate success/failure back with its return code.

It seems quite orthogonal. The subprocess could have terminated successfully 
while not all of the stdin bytes were consumed; EPIPE informs you of the latter.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10963
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com