pnowojski opened a new pull request, #28752:
URL: https://github.com/apache/flink/pull/28752
With unaligned checkpoints + interruptible timers, an operator's output
watermark could stall for hours (surviving restarts) because it only advances
once an entire due-timer backlog drains in one uninterrupted pass — a large
backlog (e.g. after a rescale) can outlast every single attempt.
InternalTimerServiceImpl/InternalTimeServiceManagerImpl now track the
highest watermark known to be fully fired even when interrupted partway, and
MailboxWatermarkProcessor emits that as an intermediate watermark instead of
withholding all progress. This progress lives in a new field, not
currentWatermark, since currentWatermark's eager semantics are relied on
elsewhere (WindowOperator cleanup timers, user ProcessFunctions). Emission is
paced by a configurable interval (default 5s, 0 disables) via an internal no-op
processing-time nudge, avoiding per-timer clock checks.
## Verifying this change
Added new unit tests and is covered by various existing unit/IT tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no**
/ don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't
know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (**yes** / no)
- If yes, how is the feature documented? (not applicable / **docs** /
**JavaDocs** / not documented)
---
##### Was generative AI tooling used to co-author this PR?
Yes, claude 5.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
