Cristian Rodriguez added the comment:
My 2 CLP.
On linux/glibc you do not have to do this, there is a
posix_spawn_file_actions_addclosefrom_np action which is implemented using
close_range(2) which will give you the fastest way to this job without hassle.
for the not posix_spawn case,
Marc-Andre Lemburg added the comment:
Gregory P. Smith wrote:
> A higher level "best practices for launching child processes module" with
> APIs reflecting explicit intents (performance vs security vs simplicity)
> rather than requiring users to understand subprocess platform specific
>
Alexey Izbyshev added the comment:
> As a concrete example, we have a (non-Python) build system and task runner
> that orchestrates many tasks to run in parallel. Some of those tasks end up
> invoking Python scripts that use subprocess.run() to run other programs. Our
> task runner
Richard Xia added the comment:
I'd like to provide another, non-performance-related use case for changing the
default value of Popen's close_fds parameters back to False.
In some scenarios, a (non-Python) parent process may want its descendant
processes to inherit a particular file
Eryk Sun added the comment:
> Python 3.7 added the support for the PROC_THREAD_ATTRIBUTE_HANDLE_LIST
> in subprocess.STARTUPINFO: lpAttributeList['handle_list'] parameter.
The motivating reason to add support for the WinAPI handle list was to allow
changing the default to close_fds=True
Gregory P. Smith added the comment:
Note that vfork() support has been merged for 3.10 via bpo-35823, so
posix_spawn() is less of a performance carrot than it used to be on Linux.
vfork() exists macOS, that code could likely be enabled there after some
investigation+testing.
Regardless,
Alexey Izbyshev added the comment:
> Using close_fds=False, subprocess can use posix_spawn() which is safer and
> faster than fork+exec. For example, on Linux, the glibc implements it as a
> function using vfork which is faster than fork if the parent allocated a lot
> of memory. On macOS,
New submission from STINNER Victor :
To make subprocess faster, I propose to no longer close all file descriptors by
default in subprocess: change Popen close_fds parameter default to False
(close_fds=False).
Using close_fds=False, subprocess can use posix_spawn() which is safer and
faster
