azagrebin commented on a change in pull request #8566: [FLINK-12673][network]
Introduce NetworkEnvironment.getUnreleasedPartitions instead of using
getResultPartitionManager
URL: https://github.com/apache/flink/pull/8566#discussion_r288943729
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/NetworkEnvironment.java
##########
@@ -201,6 +203,18 @@ public void
releasePartitions(Collection<ResultPartitionID> partitionIds) {
}
}
+ /**
+ * Query unreleased partitions.
+ *
+ * @return collection of partitions which still occupy some resources
locally on this task executor
+ * and have not been released yet. The partition can be released either
with {@link ResultPartitionWriter#fail(Throwable)}
+ * or {@link ResultPartitionWriter#finish()} and then with {@link
ResultPartitionWriter#close()} after fail or finish.
Review comment:
In this comment I was actually thinking more about how to describe which
user calls of shuffle API will lead to the release.
`ResultPartitionWriter#finish()` triggers end of production and will result
eventually internally in `ResultPartitionManager#onConsumedPartition`. But I
think you are right we should mention that the actual release can happen later
e.g. when the consumption is done. I will rewrite the comment a bit.
In general, `Close` could also potentially release some resources associated
with partition, like memory now, we just do not associate it with release state
of partition now because of current implementation details.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
