rkhachatryan commented on a change in pull request #11507: [FLINK-16587] Add
basic CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r408013290
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java
##########
@@ -206,6 +214,25 @@ void retriggerSubpartitionRequest(int subpartitionIndex)
throws IOException {
return Optional.of(new BufferAndAvailability(next,
moreAvailable, getSenderBacklog()));
}
+ @Override
+ public List<Buffer> requestInflightBuffers(long checkpointId) throws
IOException {
+ synchronized (receivedBuffers) {
+ checkState(checkpointId > lastRequestedCheckpointId,
"Need to request the next checkpointId");
+
+ final List<Buffer> inflightBuffers = new
ArrayList<>(receivedBuffers.size());
+ for (Buffer buffer : receivedBuffers) {
+ if (!buffer.isBuffer() &&
parseCheckpointBarrierOrNull(buffer) != null) {
+ break;
+ }
+ inflightBuffers.add(buffer.retainBuffer());
Review comment:
We shouldn't persist anything except buffers (`isBuffer == true`).
With the check above we break if it's checkpoint barrier, but it can also be
e.g. `CancelCheckpointMarker`.
I couldn't find any checks for this in other places.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
