[
https://issues.apache.org/jira/browse/FLINK-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282564#comment-17282564
]
Kezhu Wang commented on FLINK-21133:
------------------------------------
{quote}I think what stop-with-savepoint should really do is tell the mailbox to
take a savepoint and then exit (as if input was empty).
{quote}
Hi [~sewen], I agree and second this point.
Currently, stop-with-savepoint is implemented through propagation of
{{EndOfPartitionEvent}}, it indeed need bookkeeping to avoid {{endInput}} in
downstream not only source.
I actually pushed a [preview
working|https://github.com/kezhuw/flink/commit/d6c6837cd724913716007265f03a50098bee985e]
in discussion of FLINK-21132. It works for all stream tasks but more like an
rework but not bug fix. The key change is that it stops tasks independently.
The consequence is that {{StreamOperator.close}} will not be called as
flushing/sending operation probably will fail.
If {{StreamOperator.close}} is required in api side as guarantee to
stop-with-savepoint, I think we finally will resort to another shape of data
flow event, say {{StopOfPartitionEvent}}, to avoid bookkeeping.
> FLIP-27 Source does not work with synchronous savepoint
> -------------------------------------------------------
>
> Key: FLINK-21133
> URL: https://issues.apache.org/jira/browse/FLINK-21133
> Project: Flink
> Issue Type: Bug
> Components: API / Core, API / DataStream, Runtime / Checkpointing
> Affects Versions: 1.11.3, 1.12.1
> Reporter: Kezhu Wang
> Priority: Critical
> Fix For: 1.11.4, 1.12.2, 1.13.0
>
>
> I have pushed branch
> [synchronous-savepoint-conflict-with-bounded-end-input-case|https://github.com/kezhuw/flink/commits/synchronous-savepoint-conflict-with-bounded-end-input-case]
> in my repository. {{SavepointITCase.testStopSavepointWithFlip27Source}}
> failed due to timeout.
> See also FLINK-21132 and
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033]..
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
