subject:"PySpark\: Overusing allocated cores \/ too many processes"

Re: PySpark: Overusing allocated cores / too many processes

2017-09-27 Thread Fabian Böhnlein

It ended up being unintended multi-threading of numpy , solved by export MKL_NUM_THREADS=1 On Tue, 26 Sep 2017 at 09:05 Fabian Böhnlein wrote: > Hi all, > > above topic has

PySpark: Overusing allocated cores / too many processes

2017-09-26 Thread Fabian Böhnlein

Hi all, above topic has been mentioned before in this list between March - June 2016 , again mentioned