[
https://issues.apache.org/jira/browse/FLINK-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648226#comment-15648226
]
ASF GitHub Bot commented on FLINK-4975:
---------------------------------------
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/2754#discussion_r87043464
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/BarrierBuffer.java
---
@@ -135,19 +138,20 @@ else if (next.getEvent().getClass() ==
CheckpointBarrier.class) {
processBarrier((CheckpointBarrier) next.getEvent(), next.getChannelIndex());
}
}
+ else if (next.getEvent().getClass() ==
CancelCheckpointMarker.class) {
+
processCancellationBarrier((CancelCheckpointMarker) next.getEvent());
+ }
else {
if (next.getEvent().getClass() ==
EndOfPartitionEvent.class) {
- numClosedChannels++;
- // no chance to complete this
checkpoint
- releaseBlocks();
+
processEndOfPartition(next.getChannelIndex());
}
return next;
}
}
else if (!endOfStream) {
// end of input stream. stream continues with
the buffered data
endOfStream = true;
- releaseBlocks();
+ releaseBlocksAndResetBarriers();
return getNextNonBlocked();
--- End diff --
Sure. The recursion should also have a non-measurable impact, I assume.
> Add a limit for how much data may be buffered during checkpoint alignment
> -------------------------------------------------------------------------
>
> Key: FLINK-4975
> URL: https://issues.apache.org/jira/browse/FLINK-4975
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Affects Versions: 1.1.3
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Fix For: 1.2.0, 1.1.4
>
>
> During checkpoint alignment, data may be buffered/spilled.
> We should introduce an upper limit for the spilled data volume. After
> exceeding that limit, the checkpoint alignment should abort and the
> checkpoint be canceled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
