Admaing commented on PR #29239:
URL: https://github.com/apache/flink/pull/29239#issuecomment-5805962567
@rkhachatryan you're right on all counts, and thanks for digging into it.
Confirmed and fixed in 627b6ce:
* `PartialWatermark` is now tri-state (`UNKNOWN`/`ACTIVE`/`IDLE`) and starts
`UNKNOWN`, and
`updateCombinedWatermark()` guards on
`CombinedWatermarkStatus#hasKnownActivity()` instead of
`hasOutputs()` — which also lets `PartialWatermark#isIdle()` go back to
`private`, as you noted.
* `SourceOperatorAlignmentTest` passes again locally (21/21).
* Added
`WatermarkOutputMultiplexerTest#whenRegisteredOutputReportsNothingNothingIsReported`;
it is
red against the previous commit (40e286b) and green now.
One thing your proposal did not cover, which I hit while running the tests:
your
`subtaskMustNotGoIdleWhileRecordsFlowThroughItsSplits` uses a generator that
never emits, so the
split never reaches `setWatermark()` and stays `UNKNOWN` — with the
tri-state alone the subtask still
went idle while records flowed. So `WatermarksWithIdleness.onEvent()` now
reports activity on the
first record (and again after each idle period), not only when undoing its
own `markIdle()`:
if (isIdleNow || !isActivityReported) { output.markActive(); ... }
That keeps "a generator that emits nothing" (explicitly listed in the
ticket) covered without
reporting on every record.
Two consequences worth knowing:
* `testMainAndPerSplitWatermarkIdleness` is back to its original
expectation: registering a split no
longer emits ACTIVE by itself. FLINK-22926 still works when at least one
other split has reported
its state (the common case), but a subtask with no splits at all and an
idle main output now
re-activates on the first record of the new split rather than at
registration. I can add the
immediacy back if you think it matters.
* Regression over all watermark-related classes in flink-core/flink-runtime:
22 classes, 164 tests,
green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
