[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kapil Singh updated SPARK-44458: -------------------------------- Attachment: image-2023-07-17-16-56-59-388.png > ThreadLocal not being copied to child thread when child thread is reused from > pool > ----------------------------------------------------------------------------------- > > Key: SPARK-44458 > URL: https://issues.apache.org/jira/browse/SPARK-44458 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.4.1 > Reporter: Kapil Singh > Priority: Major > Attachments: image-2023-07-17-16-56-59-388.png, > image-2023-07-17-16-57-09-669.png > > > Currently setting the Spark configuration using the statement > `spark.conf.set("spark.sql.caseSensitive", "true")` and subsequently > executing a Spark action in a separate thread. An intermittent observation > reveals that occasionally the configuration set in the main thread fails to > propagate to the child thread. > !image-2023-07-17-16-54-26-372.png! > !image-2023-07-17-16-54-36-860.png! > > This is limitation due to ThreadLocal not being copied to child thread when > child thread is reused from pool and only done instead in thread creation or > when idle thread has expired KeepAlive in Executor > Refer: > [https://users.scala-lang.org/t/future-executioncontext-and-threadlocal/7675/2] > > [https://www.stevenskelton.ca/threadlocal-variables-scala-futures/] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org