[issue26692] cgroups support in multiprocessing

2018-01-16 Thread David Chin
David Chin added the comment: I would like to state strong support if is.get_usable_cpu_count() I administer a typical HPC cluster which may have multiple jobs scheduled on the same physical server. The fact that multiprocessing ignores cgroups leads to bad

[issue26692] cgroups support in multiprocessing

2017-11-07 Thread Mihai Capotă
Change by Mihai Capotă : -- nosy: +mihaic ___ Python tracker ___ ___ Python-bugs-list

[issue26692] cgroups support in multiprocessing

2017-09-04 Thread Giampaolo Rodola'
Changes by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___

[issue26692] cgroups support in multiprocessing

2017-07-23 Thread Antoine Pitrou
Antoine Pitrou added the comment: Agreed that it is not possible for multiprocessing to choose an optimal default in all settings. However, making the default adequate for more use cases sounds like a reasonable goal. Currently, we are using `os.cpu_count()`. Ideally, we would have a second

[issue26692] cgroups support in multiprocessing

2017-07-23 Thread Charles-François Natali
Charles-François Natali added the comment: I'm not convinced. The reason is that using the number of CPU cores is just a heuristic for a *default value*: the API allows the user to specify the number of workers to use, so it's not really a limitation. The problem is that if you try to think

[issue26692] cgroups support in multiprocessing

2017-07-22 Thread Antoine Pitrou
Changes by Antoine Pitrou : -- nosy: +neologix, pitrou ___ Python tracker ___ ___

[issue26692] cgroups support in multiprocessing

2017-07-22 Thread Antoine Pitrou
Changes by Antoine Pitrou : -- stage: -> needs patch type: behavior -> enhancement versions: +Python 3.7 -Python 3.6 ___ Python tracker

[issue26692] cgroups support in multiprocessing

2016-04-05 Thread SilentGhost
Changes by SilentGhost : -- nosy: +jnoller, sbt versions: +Python 3.6 ___ Python tracker ___

[issue26692] cgroups support in multiprocessing

2016-04-04 Thread Satrajit Ghosh
New submission from Satrajit Ghosh: multiprocessing cpucount returns the number of cpus on the system as returned by /proc/cpuinfo. this is true even on machines where linux kernel cgroups is being used to restrict cpu usage for a given process. this results in significant thread swithcing