AHeise commented on a change in pull request #11507: [FLINK-16587] Add basic
CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r406328788
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java
##########
@@ -83,24 +86,45 @@
/** The total number of bytes (both data and event buffers). */
private long totalNumberOfBytes;
+ /** The collection of buffers which are spanned over by checkpoint
barrier and needs to be persisted for snapshot. */
+ private final List<Buffer> inflightBufferSnapshot = new ArrayList<>();
+
//
------------------------------------------------------------------------
PipelinedSubpartition(int index, ResultPartition parent) {
super(index, parent);
}
@Override
- public boolean add(BufferConsumer bufferConsumer) {
- return add(bufferConsumer, false);
+ public boolean add(BufferConsumer bufferConsumer) throws IOException {
+ return add(bufferConsumer, false, false);
}
@Override
public void finish() throws IOException {
-
add(EventSerializer.toBufferConsumer(EndOfPartitionEvent.INSTANCE), true);
+
add(EventSerializer.toBufferConsumer(EndOfPartitionEvent.INSTANCE), true,
false);
LOG.debug("{}: Finished {}.", parent.getOwningTaskName(), this);
}
- private boolean add(BufferConsumer bufferConsumer, boolean finish) {
+ @Override
+ public EnqueueAction priorityEvent(AbstractEvent event, BufferConsumer
eventBufferConsumer) throws IOException {
+ if (readView != null) {
+ switch (readView.priorityEvent(event,
eventBufferConsumer)) {
Review comment:
Merging `PriorityEventListener` into `BufferAvailabilityListener` also
greatly helped to reduce commit size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
