Jose Torres created SPARK-27253: ----------------------------------- Summary: SparkSession clone discards SQLConf overrides in favor of SparkConf defaults Key: SPARK-27253 URL: https://issues.apache.org/jira/browse/SPARK-27253 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 2.4.0 Reporter: Jose Torres
SparkSession.cloneSession() is normally supposed to create a child session which inherits all the SQLConf values of its parent session. But when a SQL conf is given a global default through the SparkConf, this does not happen; the child session will receive the SparkConf default rather than its parent's SQLConf override. This is particularly impactful in structured streaming, as the microbatches run in a cloned child session. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org