dawidwys commented on code in PR #27066:
URL: https://github.com/apache/flink/pull/27066#discussion_r2390424481
##########
flink-core/src/main/java/org/apache/flink/api/common/eventtime/CombinedWatermarkStatus.java:
##########
@@ -57,34 +55,51 @@ public void add(PartialWatermark element) {
}
/**
- * Checks whether we need to update the combined watermark.
+ * Checks whether we need to update the combined watermark. It can update
{@link #isIdle()}
+ * status.
*
- * <p><b>NOTE:</b>It can update {@link #isIdle()} status.
+ * <p><b>NOTE:</b>The logic here should be kept in sync with {@code
StatusWatermarkValve}.
*
* @return true, if the combined watermark changed
*/
public boolean updateCombinedWatermark() {
- long minimumOverAllOutputs = Long.MAX_VALUE;
-
- // if we don't have any outputs minimumOverAllOutputs is not valid,
it's still
- // at its initial Long.MAX_VALUE state and we must not emit that
+ // if we don't have any outputs, we should not emit
if (partialWatermarks.isEmpty()) {
return false;
}
+ long maximumOverAllOutputs = Long.MIN_VALUE;
+ long minimumOverAllActiveOutputs = Long.MAX_VALUE;
+
boolean allIdle = true;
for (PartialWatermark partialWatermark : partialWatermarks) {
+ final long watermark = partialWatermark.getWatermark();
+ maximumOverAllOutputs = Math.max(maximumOverAllOutputs, watermark);
if (!partialWatermark.isIdle()) {
- minimumOverAllOutputs =
- Math.min(minimumOverAllOutputs,
partialWatermark.getWatermark());
+ minimumOverAllActiveOutputs =
Math.min(minimumOverAllActiveOutputs, watermark);
allIdle = false;
}
}
this.idle = allIdle;
- if (!allIdle && minimumOverAllOutputs > combinedWatermark) {
- combinedWatermark = minimumOverAllOutputs;
+ final long combinedWatermark;
+ if (allIdle) {
+ // If all splits are idle, we should flush all watermarks, which
effectively
+ // means emitting the maximum watermark.
Review Comment:
nit: is the comment correct? You emit a maximumOverAllOutputs, which is
rarely the maximum watermark (Long.MAX_WATERMARK) or did you mean maximum
watermark over all inputs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
