[
https://issues.apache.org/jira/browse/FLINK-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460972#comment-16460972
]
Fabian Hueske commented on FLINK-9278:
--------------------------------------
Hi [~ahains],
This sounds like a very valuable features. So far, we kept the generated IDs
because we wanted to ensure that we only restore valid queries from a savepoint.
However, we did not consider the multiple independent queries per job use case
that you described.
It shouldn't be a big issue to generate deterministic operator IDs per query.
Best, Fabian
> Allow restore savepoint with some SQL queries added/removed
> -----------------------------------------------------------
>
> Key: FLINK-9278
> URL: https://issues.apache.org/jira/browse/FLINK-9278
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Affects Versions: 1.4.2
> Reporter: Adrian Hains
> Priority: Major
>
> We are running a Flink job that contains multiple SQL queries. This is
> configured by calling sqlQuery(String) one time for each SQL query, on a
> single instance of StreamTableEnvironment. The queries are simple
> aggregations with a tumble window.
> Currently I can configure my environment with queries Q1, Q2, and Q3, create
> a savepoint, and restart the job from that savepoint if the same set of SQL
> queries are used.
> If I remove some queries and add some others, Q2, Q4, and Q3, I am unable to
> restart the job from the same savepoint. This behavior is expected, as the
> documentation clearly describes that the operator IDs are generated if they
> are not explicitly defined, and they cannot be explicitly defined when using
> flink SQL.
> I would like to be able to specify a scoping operator id prefix when
> registering a SQL query to a StreamTableEnvironment. This can then be used to
> programmatically generate unique IDs for each of the operators created to
> execute the SQL queries. For example, if I specify a prefix of "ID:Q2:" for
> my Q2 query, and I restart the job with an identical SQL query for this
> prefix, then I would be able to restore the state for this query even in the
> presence of other queries being added or removed to the job graph.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
