Burak Yavuz created SPARK-29568:
-----------------------------------
Summary: Add flag to stop existing stream when new copy starts
Key: SPARK-29568
URL: https://issues.apache.org/jira/browse/SPARK-29568
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.0.0
Reporter: Burak Yavuz
In multi-tenant environments where you have multiple SparkSessions, you can
accidentally start multiple copies of the same stream (i.e. streams using the
same checkpoint location). This will cause all new instantiations of the new
stream to fail. However, sometimes you may want to turn off the old stream, as
the old stream may have turned into a zombie (you no longer have access to the
query handle or SparkSession).
It would be nice to have a SQL flag that allows the stopping of the old stream
for such zombie cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
