Kapil Singh created SPARK-44458: ----------------------------------- Summary: ThreadLocal not being copied to child thread when child thread is reused from pool Key: SPARK-44458 URL: https://issues.apache.org/jira/browse/SPARK-44458 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.4.1 Reporter: Kapil Singh
Currently setting the Spark configuration using the statement `spark.conf.set("spark.sql.caseSensitive", "true")` and subsequently executing a Spark action in a separate thread. An intermittent observation reveals that occasionally the configuration set in the main thread fails to propagate to the child thread. !image-2023-07-17-16-54-26-372.png! !image-2023-07-17-16-54-36-860.png! This is limitation due to ThreadLocal not being copied to child thread when child thread is reused from pool and only done instead in thread creation or when idle thread has expired KeepAlive in Executor Refer: [https://users.scala-lang.org/t/future-executioncontext-and-threadlocal/7675/2] [https://www.stevenskelton.ca/threadlocal-variables-scala-futures/] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org