Hi,
Has there been any progress on
https://issues.apache.org/jira/browse/FLINK-2491?
For the Flink Runner in Apache Beam we keep operators alive to avoid
checkpointing to stop working [1]. Users of Flink's native API have to
take care of this themselves.
To fix FLINK-2491 we have to:
1) Remove shutdown operators from the list of to-be-checkpointed
operators.
2) Persist the shutdown operators in checkpoints to be able to
restore
the job correctly afterwards.
It would be great to fix this long-standing issue. Apart from removing
the need for workarounds it would also simplify some of the test setup
which relies on checkpointing to continue working when operators shut down.
Do you think we can make progress on this matter?
Cheers,
Max
[1]
https://github.com/apache/beam/blob/6e89b6c7a8191429fc422228c5b8d5c75c9caa05/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkPipelineOptions.java#L198
