pnowojski commented on a change in pull request #11507: [FLINK-16587] Add basic
CheckpointBarrierHandler for unaligned checkpoint
URL: https://github.com/apache/flink/pull/11507#discussion_r405582156
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java
##########
@@ -83,24 +86,45 @@
/** The total number of bytes (both data and event buffers). */
private long totalNumberOfBytes;
+ /** The collection of buffers which are spanned over by checkpoint
barrier and needs to be persisted for snapshot. */
+ private final List<Buffer> inflightBufferSnapshot = new ArrayList<>();
+
//
------------------------------------------------------------------------
PipelinedSubpartition(int index, ResultPartition parent) {
super(index, parent);
}
@Override
- public boolean add(BufferConsumer bufferConsumer) {
- return add(bufferConsumer, false);
+ public boolean add(BufferConsumer bufferConsumer) throws IOException {
+ return add(bufferConsumer, false, false);
}
@Override
public void finish() throws IOException {
-
add(EventSerializer.toBufferConsumer(EndOfPartitionEvent.INSTANCE), true);
+
add(EventSerializer.toBufferConsumer(EndOfPartitionEvent.INSTANCE), true,
false);
LOG.debug("{}: Finished {}.", parent.getOwningTaskName(), this);
}
- private boolean add(BufferConsumer bufferConsumer, boolean finish) {
+ @Override
+ public EnqueueAction priorityEvent(AbstractEvent event, BufferConsumer
eventBufferConsumer) throws IOException {
+ if (readView != null) {
+ switch (readView.priorityEvent(event,
eventBufferConsumer)) {
Review comment:
I think for the `PipelinedSubpartition` and `BoundedBlockingSubpartition` we
do not need to return those values. We could just define `void
addPriorityEvent(...)` method in the `ResultSubpartition` method instead of
inheriting from `PriorityEventListener`. The same applies for
`PipelinedSubpartitionView`, as those classes are not being used as listeners,
they are just forwarding the call. This extra abundant inheritance makes the
code a little bit less readable and messes up with IDE support of usages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
