Which version are you using? Could you post these thread names here? Best Regards, Shixiong Zhu
2015-12-07 14:30 GMT-08:00 Richard Marscher <rmarsc...@localytics.com>: > Hi, > > I've been running benchmarks against Spark in local mode in a long running > process. I'm seeing threads leaking each time it runs a job. It doesn't > matter if I recycle SparkContext constantly or have 1 context stay alive > for the entire application lifetime. > > I see a huge accumulation ongoing of "pool-xxxx-thread-1" threads with the > creating thread "Executor task launch worker-xx" where x's are numbers. The > number of leaks per launch worker varies but usually 1 to a few. > > Searching the Spark code the pool is created in the Executor class. It is > `.shutdown()` in the stop for the executor. I've wired up logging and also > tried shutdownNow() and awaitForTermination on the pools. Every seems okay > there for every Executor that is called with `stop()` but I'm still not > sure yet if every Executor is called as such, which I am looking into now. > > What I'm curious to know is if anyone has seen a similar issue? > > -- > *Richard Marscher* > Software Engineer > Localytics > Localytics.com <http://localytics.com/> | Our Blog > <http://localytics.com/blog> | Twitter <http://twitter.com/localytics> | > Facebook <http://facebook.com/localytics> | LinkedIn > <http://www.linkedin.com/company/1148792?trk=tyah> >
