New submission from Christian Ullrich <ch...@chrullrich.net>:

Using concurrent.futures.ProcessPoolExecutor on Windows fails immediately with 
a lot of exceptions of the "access denied", "file not found", and "invalid 
handle" varieties. Running the script that creates the ProcessPoolExecutor from 
the main system-wide installation works correctly.

Due to Windows' infamous lack of fork(), ProcessPoolExecutor launches its 
worker processes by setting up an inheritable handle to a pipe and passing the 
handle on the command line.

In a venv situation, it appears that the venv's python.exe internally launches 
the parent environment's python.exe and passes on its command line, but not its 
handle table. This sub-subprocess therefore does not have the original handle, 
and may have a different handle at the same index.


Output of the ProcessPoolExecutor example program from the docs when run with 
the main installation:

C:\Daten>py cft.py
112272535095293 is prime: True
112582705942171 is prime: True
112272535095293 is prime: True
115280095190773 is prime: True
115797848077099 is prime: True
1099726899285419 is prime: False


Output when run using a venv:

C:\Daten>pyv\v37\Scripts\python.exe cft.py
Process SpawnProcess-4:
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in 
_bootstrap
    self.run()
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in 
run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226, 
in _process_worker
    call_item = call_queue.get(block=True)
  File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in 
get
    with self._rlock:
  File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95, 
in __enter__
    return self._semlock.__enter__()
PermissionError: [WinError 5] Access is denied
Process SpawnProcess-5:
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in 
_bootstrap
    self.run()
  File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in 
run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226, 
in _process_worker
    call_item = call_queue.get(block=True)
  File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in 
get
    with self._rlock:
  File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95, 
in __enter__
    return self._semlock.__enter__()
PermissionError: [WinError 5] Access is denied
Traceback (most recent call last):
  File "cft.py", line 28, in <module>
    main()
  File "cft.py", line 24, in main
    for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
  File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 476, 
in _chain_from_iterable_of_lists
    for element in iterable:
  File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 586, 
in result_iterator
    yield fs.pop().result()
  File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 432, 
in result
    return self.__get_result()
  File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 384, 
in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was 
terminated abruptly while the future was running or pending.

----------
assignee: docs@python
components: Documentation, Library (Lib), Windows
messages: 334142
nosy: chrullrich, docs@python, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: concurrent.futures.ProcessPoolExecutor does not work in venv on Windows
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue35797>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to