Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2624#issuecomment-57840767 Okay, this is a pretty significant change to remove the threadlocal object completely. There are two things we can do - Either we can do something minimal to just clear the reference, so that repeated sparkContext creation works from pySpark. - Or we do this scary refactoring and eliminate the threadlocal completely. I know that there are probably no usecases where the threadlocal object on any thread would be different from that of the global lastSparkEnv, but are we entirely sure? I think we need more input from others @pwendell @mateiz @rxin
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org