Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21185#discussion_r184993529
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -299,6 +316,9 @@ private[spark] class Executor(
Thread.currentThread.setContextClassLoader(replClassLoader)
val ser = env.closureSerializer.newInstance()
logInfo(s"Running $taskName (TID $taskId)")
+ // When running in local mode, we might end up with the active
session from the driver set on
+ // this thread, though we never should, so we defensively clear it.
See SPARK-23894.
+ clearActiveSparkSessionMethod.foreach(_.invoke(null))
--- End diff --
ThreadFactories only create threads, they don't start them, so you can't do
it directly there, as I need this to run *inside* the thread.
I thought about trying to do something fancier, like having the thread pool
also start the thread and submit one runnable to do this, or having threads get
promoted to the correct thread pool after this ran, but I felt like it was too
error-prone for minimal gain.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
