[
https://issues.apache.org/jira/browse/SPARK-54909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-54909.
---------------------------------
Fix Version/s: 4.2.0
Assignee: Eric Marnadi
Resolution: Fixed
> Enabling addition, removal and reordering of streaming sources
> --------------------------------------------------------------
>
> Key: SPARK-54909
> URL: https://issues.apache.org/jira/browse/SPARK-54909
> Project: Spark
> Issue Type: Epic
> Components: Structured Streaming
> Affects Versions: 4.2.0
> Reporter: Eric Marnadi
> Assignee: Eric Marnadi
> Priority: Major
> Fix For: 4.2.0
>
>
> Currently, sources are identified by their position in the logical plan. This
> is brittle, and doesn't allow for addition, removal, or reordering of sources
> in a streaming query. The user either has to manually edit their checkpoint,
> or start with a new checkpoint location, both of which can be cumbersome
> and/or error-prone.
> By allowing the user to name their streaming sources, we have a stable
> identifier which can natively enable such operations on their streaming query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
