Adrian Hains created FLINK-9278:
-----------------------------------
Summary: Allow restore savepoint with some SQL queries
added/removed
Key: FLINK-9278
URL: https://issues.apache.org/jira/browse/FLINK-9278
Project: Flink
Issue Type: Improvement
Components: State Backends, Checkpointing
Affects Versions: 1.4.2
Reporter: Adrian Hains
We are running a Flink job that contains multiple SQL queries. This is
configured by calling sqlQuery(String) one time for each SQL query, on a single
instance of StreamTableEnvironment. The queries are simple aggregations with a
tumble window.
Currently I can configure my environment with queries Q1, Q2, and Q3, create a
savepoint, and restart the job from that savepoint if the same set of SQL
queries are used.
If I remove some queries and add some others, Q2, Q4, and Q3, I am unable to
restart the job from the same savepoint. This behavior is expected, as the
documentation clearly describes that the operator IDs are generated if they are
not explicitly defined, and they cannot be explicitly defined when using flink
SQL.
I would like to be able to specify a scoping operator id prefix when
registering a SQL query to a StreamTableEnvironment. This can then be used to
programmatically generate unique IDs for each of the operators created to
execute the SQL queries. For example, if I specify a prefix of "ID:Q2:" for my
Q2 query, and I restart the job with an identical SQL query for this prefix,
then I would be able to restore the state for this query even in the presence
of other queries being added or removed to the job graph.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
