Spark 1.3.1 - Does SparkConext in multi-threaded env requires SparkEnv.set(env) anymore

2015-12-10 Thread Nirav Patel
As subject says, do we still need to use static env in every thread that access sparkContext? I read some ref here. http://qnalist.com/questions/4956211/is-spark-context-in-local-mode-thread-safe -- [image: What's New with Xactly]

Re: Spark 1.3.1 - Does SparkConext in multi-threaded env requires SparkEnv.set(env) anymore

2015-12-10 Thread Josh Rosen
Nope, you shouldn't have to do that anymore. As of https://github.com/apache/spark/pull/2624, which is in Spark 1.2.0+, SparkEnv's thread-local stuff was removed and replaced by a simple global variable (since it was used in an *effectively* global way before (see my comments on that PR)). As a