fhueske opened a new pull request, #28229:
URL: https://github.com/apache/flink/pull/28229
## What is the purpose of the change
TemporalRowTimeJoinOperator cleans up build-side state when the watermark
advances. Hence, late arriving probe-side records might not see the correct
build-side state and join with no or incorrect data. Especially for probe-side
inputs with retractions, this is causing problems for downstream operators that
cannot match retraction because insertions and retractions differ. This change
drops late arriving probe-side records and tracks the number of dropped records
in a metric. The new behavior is aligned with other operators that operate in
event-time and clean up state on watermark progress such as windowed or OVER
aggregates and interval joins.
## Brief change log
* drop late arriving probe-side records
* track dropped record count as a metric
* adjust existing tests
* add new unit tests to ensure correct dropping behavior
* add warning to docs about late data handling
## Verifying this change
* existing and new unit tests
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): **yes**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? yes
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code 2.1.148 (Opus 4.7)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
