New submission from Christian Ullrich <[email protected]>:
Using concurrent.futures.ProcessPoolExecutor on Windows fails immediately with
a lot of exceptions of the "access denied", "file not found", and "invalid
handle" varieties. Running the script that creates the ProcessPoolExecutor from
the main system-wide installation works correctly.
Due to Windows' infamous lack of fork(), ProcessPoolExecutor launches its
worker processes by setting up an inheritable handle to a pipe and passing the
handle on the command line.
In a venv situation, it appears that the venv's python.exe internally launches
the parent environment's python.exe and passes on its command line, but not its
handle table. This sub-subprocess therefore does not have the original handle,
and may have a different handle at the same index.
Output of the ProcessPoolExecutor example program from the docs when run with
the main installation:
C:\Daten>py cft.py
112272535095293 is prime: True
112582705942171 is prime: True
112272535095293 is prime: True
115280095190773 is prime: True
115797848077099 is prime: True
1099726899285419 is prime: False
Output when run using a venv:
C:\Daten>pyv\v37\Scripts\python.exe cft.py
Process SpawnProcess-4:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in
_bootstrap
self.run()
File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in
run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226,
in _process_worker
call_item = call_queue.get(block=True)
File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in
get
with self._rlock:
File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95,
in __enter__
return self._semlock.__enter__()
PermissionError: [WinError 5] Access is denied
Process SpawnProcess-5:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in
_bootstrap
self.run()
File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in
run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226,
in _process_worker
call_item = call_queue.get(block=True)
File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in
get
with self._rlock:
File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95,
in __enter__
return self._semlock.__enter__()
PermissionError: [WinError 5] Access is denied
Traceback (most recent call last):
File "cft.py", line 28, in <module>
main()
File "cft.py", line 24, in main
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 476,
in _chain_from_iterable_of_lists
for element in iterable:
File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 586,
in result_iterator
yield fs.pop().result()
File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 432,
in result
return self.__get_result()
File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 384,
in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was
terminated abruptly while the future was running or pending.
----------
assignee: docs@python
components: Documentation, Library (Lib), Windows
messages: 334142
nosy: chrullrich, docs@python, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: concurrent.futures.ProcessPoolExecutor does not work in venv on Windows
type: behavior
versions: Python 3.7
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue35797>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
