curcur commented on a change in pull request #13501:
URL: https://github.com/apache/flink/pull/13501#discussion_r498883527
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedApproximateSubpartition.java
##########
@@ -0,0 +1,138 @@
+package org.apache.flink.runtime.io.network.partition;
+
+import org.apache.flink.runtime.io.network.buffer.Buffer;
+import org.apache.flink.runtime.io.network.buffer.BufferConsumer;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+
+import static org.apache.flink.util.Preconditions.checkState;
+
+public class PipelinedApproximateSubpartition extends PipelinedSubpartition {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(PipelinedApproximateSubpartition.class);
+
+ private boolean isPartialBuffer = false;
+
+ PipelinedApproximateSubpartition(int index, ResultPartition parent) {
+ super(index, parent);
+ }
+
+ public void releaseView() {
+ readView = null;
+ isPartialBuffer = true;
+ isBlockedByCheckpoint = false;
+ sequenceNumber = 0;
+ }
+
+ @Override
+ public PipelinedSubpartitionView
createReadView(BufferAvailabilityListener availabilityListener) {
+ synchronized (buffers) {
+ checkState(!isReleased);
+
+ // if the view is not released yet
+ if (readView != null) {
+ releaseView();
+ }
+
+ LOG.debug("{}: Creating read view for subpartition {}
of partition {}.",
+ parent.getOwningTaskName(),
getSubPartitionIndex(), parent.getPartitionId());
+
+ readView = new
PipelinedApproximateSubpartitionView(this, availabilityListener);
+ }
+
+ return readView;
+ }
+
+ @Nullable
+ @Override
+ BufferAndBacklog pollBuffer() {
Review comment:
the difference is :
```
buffer = bufferConsumer.build();
```
==>
```
if (isPartialBuffer) {
BufferConsumer.PartialRecordCleanupResult result =
bufferConsumer.skipPartialRecord();
buffer = result.getBuffer();
isPartialBuffer = result.getCleaned();
} else {
buffer = bufferConsumer.build();
}
```
That's why this method is very clean, it limits the change to very few
classes/methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
