[
https://issues.apache.org/jira/browse/FLINK-31846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714562#comment-17714562
]
Piotr Nowojski edited comment on FLINK-31846 at 4/20/23 11:47 AM:
------------------------------------------------------------------
Hey, [~hongfanxo], your feature request would be very risky, as lack of final
checkpoint could cause data losses. Records are committed do downstream systems
only on checkpoints, so without final checkpoint, final records wouldn't be
committed. Usually:
If your jobs are short (so you don't need to have checkpoints in case of
failure recovery and you can simply start over), and you don't care about
committing results, you can simply disable checkpointing.
If your jobs are bounded, but long lived, one extra checkpoint doesn't matter.
was (Author: pnowojski):
Hey, [~hongfanxo], your feature request would be very risky, as lack of final
checkpoint could cause data losses. Records are committed do downstream systems
only on checkpoints, so without final checkpoint, final records wouldn't be
committed. Usually
If your jobs are short (so you don't need to have checkpoints in case of
failure recovery and you can simply start over), and you don't care about
committing results, you can simply disable checkpointing.
If your jobs are bounded, but long lived, one extra checkpoint doesn't matter.
> Support cancel final checkpoint when all tasks are finished
> -----------------------------------------------------------
>
> Key: FLINK-31846
> URL: https://issues.apache.org/jira/browse/FLINK-31846
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Checkpointing
> Affects Versions: 1.15.2
> Reporter: Fan Hong
> Priority: Major
>
> As stated in [1], all tasks will wait for the final checkpoint before
> exiting. It also mentioned this mechanism will prolong the execution time.
> So, can we provide configurations to make tasks NOT wait for the final
> checkpoint?
>
> [1]:
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpointing/#waiting-for-the-final-checkpoint-before-task-exit
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
