[ https://issues.apache.org/jira/browse/FLINK-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488679#comment-16488679 ]
ASF GitHub Bot commented on FLINK-9428: --------------------------------------- Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/6066#discussion_r190510791 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamOperator.java --- @@ -93,6 +93,20 @@ // state snapshots // ------------------------------------------------------------------------ + /** + * This method is called when the operator should do a snapshot, before it emits its + * own checkpoint barrier. This method is intended not for any actual state persistence, + * but only for emitting some data before emitting the checkpoint barrier. + * + * <p><b>Important:</b> This method should not be used for any actual state snapshot logic, because + * it will inherently be within the synchronous part of the operator's checkpoint. If heavy work is done + * withing this method, it will affect latency and downstream checkpoint alignments. --- End diff -- typo: withing -> within > Allow operators to flush data on checkpoint pre-barrier > ------------------------------------------------------- > > Key: FLINK-9428 > URL: https://issues.apache.org/jira/browse/FLINK-9428 > Project: Flink > Issue Type: New Feature > Components: State Backends, Checkpointing > Reporter: Stephan Ewen > Assignee: Stephan Ewen > Priority: Major > Fix For: 1.6.0 > > > Some operators maintain some small transient state that may be inefficient to > checkpoint, especially when it would need to be checkpointed also in a > re-scalable way. > An example are opportunistic pre-aggregation operators, which have small the > pre-aggregation state that is frequently flushed downstream. > Rather that persisting that state in a checkpoint, it can make sense to flush > the data downstream upon a checkpoint, to let it be part of the downstream > operator's state. > This feature is sensitive, because flushing state has a clean implication on > the downstream operator's checkpoint alignment. However, used with care, and > with the new back-pressure-based checkpoint alignment, this feature can be > very useful. > Because it is sensitive, I suggest to make this only an internal feature > (accessible to operators) and NOT expose it in the public API at this point. -- This message was sent by Atlassian JIRA (v7.6.3#76005)