Re: Question about Spark process and thread

2015-06-29 Thread Reynold Xin
Most of those threads are not for task execution. They are for RPC, scheduling, ... On Sun, Jun 28, 2015 at 8:32 AM, Dogtail Ray spark.ru...@gmail.com wrote: Hi, I was looking at Spark source code, and I found that when launching a Executor, actually Spark is launching a threadpool; each

Question about Spark process and thread

2015-06-28 Thread Dogtail Ray
Hi, I was looking at Spark source code, and I found that when launching a Executor, actually Spark is launching a threadpool; each time the scheduler launches a task, the executor will launch a thread within the threadpool. However, I also found that the Spark process always has approximately 40