Alexey Izbyshev <izbys...@ispras.ru> added the comment:
I've checked subprocess.Popen() error reporting in QEMU user-mode and WSL and
confirm that it works both with my patch (vfork/exec) and the traditional
fork/exec, but doesn't work with glibc's posix_spawn.
The first command below uses posix_spawn() internally because close_fds=False.
Under QEMU posix_spawn() from glibc doesn't report errors because it relies on
address-space sharing of clone(CLONE_VM|CLONE_VFORK), which is not emulated by
QEMU. The second command uses vfork()/exec() in _posixsubprocess, but lack of
address-space sharing doesn't matter because the error data is transferred via
a pipe.
$ qemu-x86_64 --version
qemu-x86_64 version 2.11.1
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
$ ldd --version
ldd (GNU libc) 2.27
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
$ qemu-x86_64 ./python3 -c 'import subprocess; subprocess.call("/xxx",
close_fds=False)'
$ qemu-x86_64 ./python3 -c 'import subprocess; subprocess.call("/xxx")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/izbyshev/install/python-3.8a/lib/python3.8/subprocess.py", line
324, in call
with Popen(*popenargs, **kwargs) as p:
File "/home/izbyshev/install/python-3.8a/lib/python3.8/subprocess.py", line
830, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/izbyshev/install/python-3.8a/lib/python3.8/subprocess.py", line
1648, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'xxx'
For WSL, I've tested Ubuntu 18.04 (glibc 2.27) on Windows 10 1709 (Fall
Creators Update) and 1803.
In 1709, the result is the same as for QEMU (i.e. the first command silently
succeeds). In 1803, both commands raise the exception because address-space
sharing was fixed for clone(CLONE_VM|CLONE_VFORK) and vfork() in WSL:
https://github.com/Microsoft/WSL/issues/1878
I've also run all subprocess tests under QEMU user-mode (by making
sys.executable to point to a wrapper that runs python under QEMU) for the
following code bases:
* current master (fork/exec or posix_spawn can be used)
* classic (fork/exec always)
* my patch + master (vfork/exec or posix_spawn)
* vfork/exec always
All tests pass in all four flavors, which indicates that we don't have a test
for error reporting with close_fds=False :)
Out of curiosity I also did the same on WSL in 1803. Only 3 groups of tests
fail, all because of WSL bugs:
* It's not possible to send signals to zombies, e.g.:
======================================================================
ERROR: test_terminate_dead (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/test/cpython/Lib/test/test_subprocess.py", line 1972, in
test_terminate_dead
self._kill_dead_process('terminate')
File "/home/test/cpython/Lib/test/test_subprocess.py", line 1941, in
_kill_dead_process
getattr(p, method)(*args)
File "/home/test/cpython/Lib/subprocess.py", line 1877, in terminate
self.send_signal(signal.SIGTERM)
File "/home/test/cpython/Lib/subprocess.py", line 1872, in send_signal
os.kill(self.pid, sig)
ProcessLookupError: [Errno 3] No such process
----------------------------------------------------------------------
* getdents64 syscall doesn't work properly (doesn't return the rest of entries
on the second call, so close_fds=True doesn't fully work with large number of
open descriptors).
======================================================================
FAIL: test_close_fds_when_max_fd_is_lowered
(test.test_subprocess.POSIXProcessTestCase)
Confirm that issue21618 is fixed (may fail under valgrind).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/test/cpython/Lib/test/test_subprocess.py", line 2467, in
test_close_fds_when_max_fd_is_lowered
self.assertFalse(remaining_fds & opened_fds,
AssertionError: {34, 35, 36, 37, 38, 39, 40, 41, 42} is not false : Some fds
were left open.
----------------------------------------------------------------------
* Signal masks in /proc/self/status are not correct (always zero)
======================================================================
FAIL: test_restore_signals (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/test/cpython/Lib/test/test_subprocess.py", line 1643, in
test_restore_signals
self.assertNotEqual(default_sig_ign_mask, restored_sig_ign_mask,
AssertionError: b'SigIgn:\t0000000000000000' == b'SigIgn:\t0000000000000000' :
restore_signals=True should've unblocked SIGPIPE and friends.
----------------------------------------------------------------------
None of the above is related to fork/vfork/posix_spawn -- the set of failing
tests is the same in all four flavors.
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue35823>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
