Eric Marnadi created SPARK-54909:
------------------------------------
Summary: Enabling addition, removal and reordering of streaming
sources
Key: SPARK-54909
URL: https://issues.apache.org/jira/browse/SPARK-54909
Project: Spark
Issue Type: Epic
Components: Structured Streaming
Affects Versions: 4.2.0
Reporter: Eric Marnadi
Currently, sources are identified by their position in the logical plan. This
is brittle, and doesn't allow for addition, removal, or reordering of sources
in a streaming query. The user either has to manually edit their checkpoint, or
start with a new checkpoint location, both of which can be cumbersome and/or
error-prone.
By allowing the user to name their streaming sources, we have a stable
identifier which can natively enable such operations on their streaming query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
