STINNER Victor <vstin...@python.org> added the comment:
> And setup.py does use use multiple processes via a thread pool, where each > thread calls fork(), if it detects that "make" was invoked with the parallel > (-j) option. Oh ok, now I get it :-) setup.py uses distutils to build extensions and distutils.command.build_ext uses concurrent.futures for parallelism: if self.parallel: self._build_extensions_parallel() else: self._build_extensions_serial() with: def _build_extensions_parallel(self): workers = self.parallel if self.parallel is True: workers = os.cpu_count() # may return None try: from concurrent.futures import ThreadPoolExecutor except ImportError: workers = None if workers is None: self._build_extensions_serial() return with ThreadPoolExecutor(max_workers=workers) as executor: futures = [executor.submit(self.build_extension, ext) for ext in self.extensions] for ext, fut in zip(self.extensions, futures): with self._filter_build_errors(ext): fut.result() The problem is not concurrent.futures but the job submitted to the executor: build_extension() uses distutils.spawn() which is unsafe. distutils.spawn() is the root issue if I understood correctly. It must be rewritten with subprocess. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue39763> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com