New submission from JustAnotherArchivist <justanotherarchiv...@riseup.net>:
The extension building does not respect the --jobs option passed to make. Specifically, in that step, `python setup.py build` always spawns as many gcc processes as there are CPU cores available regardless of that option. This caused problems for me because I have a VM that sees all host machine CPU cores but only has a limited amount of RAM. Despite running `make -j 4`, many more gcc processes are spawned, and this immediately causes memory starvation and a system freeze after a few seconds. The reason for this is that setup.py blindly enables parallelism in the extension compilation if '-j' appears in the MAKEFLAGS at 3.9.2/setup.py:355. Later on, distutils uses os.cpu_count to set the worker count, i.e. the '-j' *value* is ignored. This behaviour was first introduced with #5309 as far as I can see, though I haven't tested anything other than version 3.9.2. Hacky workaround: patching the above setup.py line to `self.parallel = 4`. Cf. https://github.com/pyenv/pyenv/issues/1857 ---------- components: Build messages: 389558 nosy: JustAnotherArchivist priority: normal severity: normal status: open title: Extensions build does not respect --jobs setting type: resource usage versions: Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue43634> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
