Grzegorz Kołakowski created FLINK-37153:
-------------------------------------------
Summary: Monitor late event count in temporal join
Key: FLINK-37153
URL: https://issues.apache.org/jira/browse/FLINK-37153
Project: Flink
Issue Type: Improvement
Components: Table SQL / API
Reporter: Grzegorz Kołakowski
In temporal join, events on the right side (build side, versioned table) are
not discarded even if they arrive late (event timestamp < current watermark),
which is generally good.
As far as I understand, late events can sometimes lead to incorrect results.
This happens when the latest right-side event is delayed for some reason, and
as the watermark progresses, a left-side event gets joined with the most
recently known right-side event (stored in Flink's state), which might not be
the true latest one.
I believe it would be valuable to have a metric tracking late event count on
the right side, just to be aware if such situation might have occurred.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
