Shixiong Zhu created SPARK-41040:
------------------------------------
Summary: Self-union streaming query may fail when using
readStream.table
Key: SPARK-41040
URL: https://issues.apache.org/jira/browse/SPARK-41040
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.4.0
Reporter: Shixiong Zhu
In a self-union query, the batch plan created by the source will be shared by
multiple nodes in the plan. When we transform the plan, the batch plan will be
visited multiple times. Hence, the first visit will set the CatalogTable and
the second visit will try to set it again and fail the query at [this
line|https://github.com/apache/spark/blob/406d0e243cfec9b29df946e1a0e20ed5fe25e152/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala#L625].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
