[
https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17301066#comment-17301066
]
Kezhu Wang commented on FLINK-21467:
------------------------------------
Hi [~pnowojski], I guess it depends on various subtleties:
# "MAX_WATERMARK" could come from last unaligned checkpoint.
# Last unaligned checkpoint considered as completed but fail at
"notifyCheckpointComplete" phase".
# Recovered subtask gets splits assigned from either source enumerator or
redistributed operator list state.
The key unknown questions are:
# Will "MAX_WATERMARK" be persisted in unaligned checkpoint ?
# When an operator is considered finished ?
# A recovered finishing subtask could get new splits assigned ?
> Document possible recommended usage of Bounded{One/Multi}Input.endInput and
> emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-21467
> URL: https://issues.apache.org/jira/browse/FLINK-21467
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}},
> to commit final result before FLIP-147 delivered. And this will cause
> re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have
> [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620]
> this out in
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please
> correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
