[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by Gregory P. Smith g...@krypto.org: -- resolution: fixed - stage: committed/rejected - commit review status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Roundup Robot added the comment: New changeset efcdf2a70f2a by Gregory P. Smith in branch '3.3': Undo supposed fix for Issue #15798 until I understand why this is http://hg.python.org/cpython/rev/efcdf2a70f2a New changeset ddbf9632795b by Gregory P. Smith in branch 'default': Undo supposed fix for Issue #15798 until I understand why this is http://hg.python.org/cpython/rev/ddbf9632795b -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Roundup Robot added the comment: New changeset 2df5e1f537b0 by Gregory P. Smith in branch 'default': Fixes issue #15798: subprocess.Popen() no longer fails if file http://hg.python.org/cpython/rev/2df5e1f537b0 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Roundup Robot added the comment: New changeset 07425df887b5 by Gregory P. Smith in branch '3.3': Fixes issue #15798: subprocess.Popen() no longer fails if file http://hg.python.org/cpython/rev/07425df887b5 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment: i went with the less invasive in terms of behavior change approach of making sure that the errpipe_write fd is always = 3. In Python 3.4 the code change was different and much simpler and on the Python only side as all fd's are opened O_CLOEXEC by default. I'm not entirely sure why the test_multiprocessing_forkserver and test_multiprocessing_spawn failures happened on 3.4 with the earlier change that attempted to always list 0,1,2 in the fds_to_keep (derived from pass_fds) list so there _may_ be a bug lurking elsewhere there but I suspect the bug is actually that we don't always want to blindly list them in fds_to_keep as some programs may have reused some of them for other non-stdio purposes but still want them closed. The change has been backported to the python-subprocess32 repo. -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Roundup Robot added the comment: New changeset c4cd891cf167 by Gregory P. Smith in branch '3.3': Fixes Issue #15798 - subprocess.Popen() no longer fails if file http://hg.python.org/cpython/rev/c4cd891cf167 New changeset 0387054b2038 by Gregory P. Smith in branch 'default': Fixes Issue #15798 - subprocess.Popen() no longer fails if file http://hg.python.org/cpython/rev/0387054b2038 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by Gregory P. Smith g...@krypto.org: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed versions: +Python 3.4 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment:
adding {0,1,2} to fds_to_keep (populated from pass_fds) is indeed an alternate
approach. A variant of an alternate patch doing that attached.
This actually simplifies code. Is there anything this would hurt that i'm not
seeing?
I suppose it adds minor overhead to the fork_exec() call by passing more in and
lookups into the fds_to_keep list now that it will always contain at least 4
values instead of the previous 1. I doubt that is matters (I haven't measured
anything).
--
Added file:
http://bugs.python.org/file32830/issue15798_alternate-pass_fds-gps01.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15798
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Ross Lagerwall added the comment: I sent a review through on rietveld; I'm attaching a patch with the changes so that it compiles and passes the tests. -- Added file: http://bugs.python.org/file27053/issue15798_v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Antoine Pitrou added the comment: I haven't tested Ross's latest patch, but it looks ok to me. -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Richard Oudkerk added the comment: Would it simplify matters to stop treating 0,1,2 specially and just add them to pass_fds instead? -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Ross Lagerwall added the comment:
It's caused by the following check in _posixsubprocess.c:
if (close_fds errpipe_write 3) { /* precondition */
PyErr_SetString(PyExc_ValueError, errpipe_write must be = 3);
return NULL;
}
which was written by Gregory P. Smith in 2010 (adding to nosy list).
I'm not entirely sure why this check is here, presumably its due to the way
close_fds=True is handled.
The close fds logic is also hardcoded to close fds from 3 upwards,.
--
nosy: +gregory.p.smith
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15798
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Changes by Chris Rebert pyb...@rebertia.com: -- nosy: +cvrebert ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment: easy enough to reproduce... $ ./python.exe -c 'import os, subprocess as s; os.close(0); os.close(1); s.Popen([/bin/true])' Traceback (most recent call last): File string, line 1, in module File /Users/gps/python/hg/default/Lib/subprocess.py, line 818, in __init__ restore_signals, start_new_session) File /Users/gps/python/hg/default/Lib/subprocess.py, line 1363, in _execute_child restore_signals, start_new_session, preexec_fn) ValueError: errpipe_write must be = 3 Examining the code, it looks like that restriction is to prevent the dup2's for any passed in stdin, stdout or stderr pipes from overwriting errpipe_write in Modules/_posixsubprocess.c's child_exec() function. First guess at a fix: child_exec() needs to detect this situation and dup(errpipe_write) until it gets a fd not in the 0..2 range before the dup2(...) calls that could otherwise blindly clobber it. This could possibly be done by the parent process's _create_pipe() in Lib/subprocess.py when allocating the errpipe_read and errpipe_write fds. Suggested Workaround: for now for any code running into this (Python daemons launching subprocesses?) - Call os.pipe() twice at the start of your program to burn 4 fds. That'll guarantee 0, 1 and 2 will not be used for this pipe. -- assignee: - gregory.p.smith versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Ross Lagerwall added the comment: The attached patch + test seems to fix the issue. It's not very elegant. -- keywords: +patch Added file: http://bugs.python.org/file27042/issue15798.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment: Yes, something along the lines of that patch is what I was thinking. BTW, this is only necessary for the errpipe_write fd. errpipe_read is for the parent process. I'm going to do it within _create_pipe so that the optimal _posixsubprocess.cloexec_pipe pipe2() based implementation can be used when possible rather than needing to call _set_cloexec() on the dup'ed fd. There are some recent Linux specific possibilities such as fcntl with F_DUPFD, or better F_DUPFD_CLOEXEC, that would make this a single call. Using that may be overkill for this situation but it looks easy enough while I'm in there. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Aleksey Filippov added the comment: It may be implemented simplier. fcntl(fd, F_DUPFD_CLOEXEC, min_fd_number) will allocate new fd at least min_fd_number. So, it is not necessary to do a lot of dup() calls. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment: F_DUPFD_CLOEXEC appears exclusive to modern Linux kernels. Any idea how wide spread support for plain F_DUPFD is? If that is everywhere the code I've just whipped up could lose a lot of loops... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Gregory P. Smith added the comment: Here's my initial fix. If fcntl(errpipe_write, F_DUPFD, 3) is widely available this could be shrunk a bit to avoid the for loop potentially calling dup a few times and tracking the wasted fds to close later. Otherwise if it isn't I'd rather not bother with F_DUPFD as this code takes the optimal path when F_DUPFD_CLOEXEC is supported (true on all modern linux systems) and should cause no harm beyond a couple extra dup and close syscalls otherwise. Note: Linux pipe2() support appears in kernels a few revisions after F_DUPFD_CLOEXEC support so the good behavior when pipe2 exists will be kept in this situation as that also means F_DUPFD_CLOEXEC support should exist. The code will work regardless of that (incase someone has a frankenkernel, or other OSes choose to implement one but not the other). -- Added file: http://bugs.python.org/file27048/issue15798-fix-gps01.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
New submission from Aleksey Filippov: System info: kernel: 3.4.8-1-ARCH dist: Arch linux python: 3.2.3 subprocess.Popen() fails if python interpreter is started with closed 0, 1 or 2 descriptor. Traceback (most recent call last): File string, line 14, in module File /usr/lib/python3.2/subprocess.py, line 745, in __init__ restore_signals, start_new_session) File /usr/lib/python3.2/subprocess.py, line 1197, in _execute_child restore_signals, start_new_session, preexec_fn) ValueError: errpipe_write must be = 3 -- messages: 169276 nosy: sarum9in priority: normal severity: normal status: open title: subprocess.Popen() fails if 0, 1 or 2 descriptor is closed versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15798] subprocess.Popen() fails if 0, 1 or 2 descriptor is closed
Ezio Melotti added the comment: #10806 seems related. -- components: +Library (Lib) nosy: +ezio.melotti, pitrou, rosslagerwall type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15798 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
