Jiayi-Liao commented on a change in pull request #11567: [FLINK-16645] Limit
the maximum backlogs in subpartitions
URL: https://github.com/apache/flink/pull/11567#discussion_r401043310
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartition.java
##########
@@ -375,4 +398,33 @@ void onConsumedSubpartition(int subpartitionIndex) {
private void checkInProduceState() throws IllegalStateException {
checkState(!isFinished, "Partition already finished.");
}
+
+ /**
+ * Check whether all subpartitions' backlogs are less than the
limitation of max backlogs, and make this partition
+ * available again if yes.
+ */
+ @GuardedBy("buffers")
Review comment:
Another solution comes to my mind is BitSet structure and each subpartition
take care of its own single bit. We can easily construct such a structure by
using `long[]` (new long[1024] can be used by 64 * 1024 subpartitions). And
instead of using a counter, we can use bit operations to check whether all long
values in `long[]` are 0.
Update: The solution looks still not thread safe...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
