New submission from Evan Andrews:
The method used for spawning subprocesses on Windows is not thread-safe under
certain circumstances. The following example demonstrates how this manifests:
>>> import threading
>>> import subprocess
>>> for i in range(100):
... threading.Thread(
... target=subprocess.Popen,
... args=('ping localhost',),
... kwargs={'stdout': subprocess.DEVNULL},
... ).start()
...
Exception in thread Thread-1202:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\threading.py", line 916, in
_bootstrap_inner
self.run()
File "C:\Program Files\Python36\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python36\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "C:\Program Files\Python36\lib\subprocess.py", line 990, in
_execute_child
startupinfo)
ValueError: embedded null character
Exception in thread Thread-1206:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\threading.py", line 916, in
_bootstrap_inner
self.run()
File "C:\Program Files\Python36\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python36\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "C:\Program Files\Python36\lib\subprocess.py", line 990, in
_execute_child
startupinfo)
ValueError: embedded null character
>>>
subprocess.Popen calls down to _winapi.CreateProcess, which calls
CreateProcessW. When args is passed as a fixed string, the result of the
argument conversion is attached to the object and shared by future calls into C
code. However, the documentation for CreateProcess states:
"The Unicode version of this function, CreateProcessW, can modify the contents
of this string. Therefore, this parameter cannot be a pointer to read-only
memory (such as a const variable or a literal string). If this parameter is a
constant string, the function may cause an access violation." (Source:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx)
It appears CreateProcessW is briefly inserting null characters into the buffer,
causing errors if that buffer is used elsewhere before it is changed back.
The call to CreateProcessW using the shared buffer can be seen here:
https://github.com/python/cpython/blob/b8f4163da30e16c7cd58fe04f4b17e38d53cd57e/Modules/_winapi.c#L879
Note that this error does not occur when passing args as a list, as
subprocess.list2cmdline creates a new (though identical) string for each
invocation.
One potential solution is to allocate a copy of command_line (the shared
buffer) instead of using the original.
----------
components: Library (Lib)
messages: 302045
nosy: evan_
priority: normal
severity: normal
status: open
title: _winapi.CreateProcess (used by subprocess) is not thread-safe
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue31446>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
