[
https://issues.apache.org/jira/browse/FLINK-30238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17689588#comment-17689588
]
Yun Gao commented on FLINK-30238:
---------------------------------
Hi [~pnowojski] , for this specific issue, based on my understanding,
# It looks to me the first issue might refer to the situation that on
endOfInput(), the Writer will emit the summary message tag with \{checkpoint id
= MAX_VALUE} to the Committer, followed by the remaining Committables. But with
the current process, endOfInput() should not be called on stop-with-savepoint
without draining, thus sorry I'm also not fully understand this part.
# For the second issue, it looks to me it refers to how the last piece of the
committables should be passed to the post-committer topology, as posted in the
above. Although here it refers to stop-with-savepoint, but it looks to me it
should be stop-with-savepoint –-drain, since for stop-with-savepoint, the last
piece of Committables should be still recorded in the last savepoint, and they
will be re-committed after restarted.
Perhaps [~fpaul] could give some more details?
> Unified Sink committer does not clean up state on final savepoint
> -----------------------------------------------------------------
>
> Key: FLINK-30238
> URL: https://issues.apache.org/jira/browse/FLINK-30238
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Affects Versions: 1.17.0, 1.15.3, 1.16.1
> Reporter: Fabian Paul
> Priority: Critical
>
> During stop-with-savepoint the committer only commits the pending
> committables on notifyCheckpointComplete.
> This has several downsides.
> * Last committableSummary has checkpoint id LONG.MAX and is never cleared
> from the state leading to that stop-with-savepoint does not work when the
> pipeline recovers from a savepoint
> * While the committables are committed during stop-with-savepoint they are
> not forwarded to post-commit topology, potentially losing data and preventing
> to close open transactions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
