Piotr Nowojski created FLINK-40353:
--------------------------------------
Summary: Leverage RocksDB ordered MapState
TemporalRowTimeJoinOperator
Key: FLINK-40353
URL: https://issues.apache.org/jira/browse/FLINK-40353
Project: Flink
Issue Type: Improvement
Components: Table SQL / Runtime
Reporter: Piotr Nowojski
Some state backends (e.g. RocksDB) always iterate a map state's keys in a
well-defined order (their serialized/binary representation); others (e.g. the
heap backend) don't. For example TemporalRowTimeJoinOperator could benefit from
ordered iteration have currently no way today to detect this at runtime.
TemporalRowTimeJoinOperator (row-time FOR SYSTEM_TIME AS OF temporal joins)
does a full scan of its left/probe-side state plus a fresh sort of its
right/build-side state on every watermark-triggered timer firing, regardless of
how many rows actually need processing on that firing. For a join with
substantial buffered state, this is a recurring, cost that can be avoided once
we expose the information that the underlying state is binary ordered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
